Commit graph

62 commits

Author SHA1 Message Date
Florian Klink
4e9e4b19ef refactor(tvix/castore/blobservice): rm AsyncBufRead from BlobReader
There's no need to already require this to be buffered here.

Change-Id: Ib9a11b194e0754d87ab8d2ef0b8cb0f4edc01229
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10074
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-19 12:54:19 +00:00
Ryan Lahfa
df63b719ac feat(tvix/castore): fix tracing instrument in MemoryBlobService
Change-Id: Iedba57e8b3e1a44f14f5baa1e981275d4b02eb56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10070
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-11-18 19:23:55 +00:00
Florian Klink
30e0c32066 refactor(tvix/castore/tonic): make async, support wait-connect=?
This moves the sync `channel::from_url` to a async
`tonic::channel_from_url`. It now allows connecting non-lazily if `wait-
connect=1` is set in the URL params.

Also, make the pingpong tests for blobsvc and directorysvc use the wait-
connect=1 codepath.

Change-Id: Ibeea33117c8121814627e7f6aba0e943ae2e92ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10030
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-15 06:43:31 +00:00
Florian Klink
840bee1e97 refactor(tvix/castore/blobsvc): remove BlobService::from_url
Make blobservice::from_addr use the more specific constructors.

Change-Id: Id9637e279d6910ce6d92ff0086a984be5c65a8c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10028
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-15 06:43:31 +00:00
edef
a8e7f4eadb fix(tvix/castore): B3Digest::{to_vec -> as_slice}
Not a single call site actually makes use of the Vec.

Change-Id: I6cf31073c9f443d1702a21937a0c3938c2c643b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9988
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-11-07 12:35:41 +00:00
Florian Klink
67999f0dcf feat(tvix/castore): extend blobstore protos for verified streaming
This pdates the proto docstrings a bit, especially w.r.t. verified
streaming.
It also adds send_chunks, send_bao fields to StatBlobRequest (renamed
from BlobMeta)

Change-Id: I590cc8646d86b73bca9f38a9b6d9ea15e4df5cb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9951
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-05 15:13:52 +00:00
Florian Klink
9757bf6377 refactor(tvix/*store): helper for channel creation from url
This moves the repetitive code to parse a URL and create a channel
connected to it into `tvix_castore::channel::from_url`.

Part of b/308

Change-Id: Idd342cd71cad5e78a9b258b38c1b227993e75310
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9707
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-14 12:26:40 +00:00
Florian Klink
3f011d2762 refactor(tvix/*store): reorganize from_url
Move the channel creation depending on the string-based URL into its
own block.

Change-Id: I546b769acd2296b548eb966b62c495f910266df5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9706
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-14 12:26:40 +00:00
Florian Klink
199e5e0339 refactor(tvix/*store): remove some grpc_client let bindings
We had to have these all while the traits where sync, and there was a
lot of spawning and moving.

Most of this can now be removed in favor of some inline `.clone()`.

Change-Id: Id5466c32a403100bc3347866b3172e06a792e311
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9705
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-14 12:26:40 +00:00
Florian Klink
b6bf3a87f1 test(tvix/castore): use tokio-retry for exp backoff
Rather than using this loop, use exponential backoff while waiting for
the socket path to be created.

Change-Id: I18706a64ce06f8916a07892dfbcd409ac5b3bff1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9568
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-08 22:50:42 +00:00
Florian Klink
b196cbbc67 refactor(tvix/castore): simplify test_valid_unix_path_ping_pong
We don't need to spawn two tokio runtimes anymore, and can do the URL
parsing at once, too.

Change-Id: I38ab96978cb7f8c31ded2726262e0b1366655094
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9566
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
2023-10-08 22:47:34 +00:00
Florian Klink
32f41458c0 refactor(tvix): move castore into tvix-castore crate
This splits the pure content-addressed layers from tvix-store into a
`castore` crate, and only leaves PathInfo related things, as well as the
CLI entrypoint in the tvix-store crate.

Notable changes:
 - `fixtures` and `utils` had to be moved out of the `test` cfg, so they
   can be imported from tvix-store.
 - Some ad-hoc fixtures in the test were moved to proper fixtures in the
   same step.
 - The protos are now created by a (more static) recipe in the protos/
   directory.

The (now two) golang targets are commented out, as it's not possible to
update them properly in the same CL. This will be done by a followup CL
once this is merged (and whitby deployed)

Bug: https://b.tvl.fyi/issues/301

Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-22 12:51:21 +00:00