From 1446e3be9991291367a2b0cd7daa1d3c0f86e8b7 Mon Sep 17 00:00:00 2001 From: Florian Klink Date: Sun, 16 Jun 2024 23:28:22 +0300 Subject: [PATCH] docs(tvix/docs): document nar renderer optimization potential Change-Id: I747ab5c8ba3424828b1c88e0353170ad0efab3a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11849 Tested-by: BuildkiteCI Reviewed-by: Simon Hauser --- tvix/docs/src/TODO.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/tvix/docs/src/TODO.md b/tvix/docs/src/TODO.md index 10fb08cab..a999e1370 100644 --- a/tvix/docs/src/TODO.md +++ b/tvix/docs/src/TODO.md @@ -59,6 +59,18 @@ which are supposed to be skipped. indices into that. We might need to have different representations for small amount of context elements or larger ones, and need tooling to reason about the amount of contexts we have. + - To calculate NAR size and digest (used for output path calculation of FODs), + our current `SimpleRenderer` `NarCalculationService` sequentially asks for + one blob after another (and internally these might consists out of multiple + chunks too). + That's a lot of roundtrips, adding up to a lot of useless waiting. + While we cannot avoid having to feed all bytes sequentially through sha256, + we already know what blobs to fetch and in which order. + There should be a way to buffer some "amount of upcoming bytes" in memory, + and not requesting these seqentially. + This is somewhat the "spiritual counterpart" to our sequential ingestion + code (`ConcurrentBlobUploader`, used by `ingest_nar`), which keeps + "some amount of outgoing bytes" in memory. ### Error cleanup - Currently, all services use tvix_castore::Error, which only has two kinds