From 4ebb8f42ed344ccad1890b71d9c29cb50d477c11 Mon Sep 17 00:00:00 2001 From: Florian Klink Date: Tue, 15 Oct 2024 18:00:37 +0300 Subject: [PATCH] docs(tvix/TODO): extend NAR rendering prefetching idea With the seekable NAR renderer, figuring out the next few blobs to render became possible. Change-Id: I1214302f88e6f9aba74227f84df0f964d587baf2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12652 Tested-by: BuildkiteCI Reviewed-by: edef Autosubmit: flokli --- tvix/docs/src/TODO.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tvix/docs/src/TODO.md b/tvix/docs/src/TODO.md index 8788d18d8..b0fdc68a8 100644 --- a/tvix/docs/src/TODO.md +++ b/tvix/docs/src/TODO.md @@ -75,8 +75,13 @@ correctness: This is somewhat the "spiritual counterpart" to our sequential ingestion code (`ConcurrentBlobUploader`, used by `ingest_nar`), which keeps "some amount of outgoing bytes" in memory. - This is somewhat blocked until the {Chunk/Blob}Service split is done, as then - prefetching would only be a matter of adding it into the one `BlobReader`. + Our seekable NAR AsyncRead implementation already removes most complexity in + rendering everything between blobs. + It should be possible to extend / write a wrapped version of it that + prefetches a configurable sliding window of blobs. + Per-blob prefetching itself is somewhat blocked until the {Chunk/Blob}Service + split is done, as then prefetching there would only be a matter of adding it + into the one `BlobReader`. ### Error cleanup - Currently, all services use tvix_castore::Error, which only has two kinds