tvl-depot/tvix/store/src
Connor Brewster e7be342256 feat(tvix/store): concurrently upload small blobs during nar ingestion
Currently all blobs are uploaded serially when ingesting NARs. If a NAR
contains many, small blobs, ingestion may become slow if there is a lot
of round-trip latency to the blob service.

This makes the NAR ingester use the ConcurrentBlobUploader which allows
for buffering small blobs in memory so they can be uploaded concurrently
to the blob service without blocking further deserialization.

Change-Id: I093a73770232df12d9a11e5d901b99c08505c3cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11694
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-05-20 15:21:51 +00:00
..
bin refactor(tvix/store): drop calculate_nar from PathInfoService 2024-05-11 13:33:59 +00:00
nar feat(tvix/store): concurrently upload small blobs during nar ingestion 2024-05-20 15:21:51 +00:00
pathinfoservice refactor(tvix/store/pathinfo): test with PathInfoService directly 2024-05-20 15:03:10 +00:00
proto refactor(tvix/store): drop calculate_nar from PathInfoService 2024-05-11 13:33:59 +00:00
tests chore(tvix/store): migrate import.rs and tests/pathinfo.rs to rstest 2024-04-19 19:22:10 +00:00
import.rs refactor(tvix/store): drop calculate_nar from PathInfoService 2024-05-11 13:33:59 +00:00
lib.rs feat(tvix/store): add rstest-based PathInfoService tests 2024-03-28 07:02:18 +00:00
utils.rs refactor(tvix/store): drop calculate_nar from PathInfoService 2024-05-11 13:33:59 +00:00