Commit graph

9 commits

Author SHA1 Message Date
Florian Klink
9748543f1c chore(tvix): drop sled compression support
It's been a while since the last sled release, and that one binds to a
pretty old version of zstd, requiring workarounds like cl/10090.

Upstream sled main branch currently has zstd halfway patched out (it's
a no-op, but the feature flag and options are still there), and it's in
that state for a year.

Rather than maintaining our own fork of sled, let's just stop using the
compression feature in sled, dropping the version pin to zstd that way,
removing the need for cl/10090.

This doesn't mean we won't reintroduce per-blob compression - but we
probably just won't let sled take care of the compression, but do it
ourselves - which is necessary for more chunked blob storage anyways.

Even though we do drop the feature flag, we still need to explicitly use
use_compression(false).

Change-Id: I0e4892d29e41c76653272dc1a3625180da6fee12
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10257
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-12-11 22:46:50 +00:00
edef
6a0a75c8e1 refactor(tvix): condense long bytestrings
Change-Id: I3bea0827ec2c8db835334ce378a7bf3a39e9b1a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9849
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-27 07:37:26 +00:00
edef
930edb69d7 fix(tvix/castore): use bstr for formatting names in errors
Much friendlier than either Base64 or raw byte slices.

Change-Id: I9b4cdd57c83ddc76c0be8103da4320207657a72b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9622
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-10 22:40:55 +00:00
Florian Klink
b6bf3a87f1 test(tvix/castore): use tokio-retry for exp backoff
Rather than using this loop, use exponential backoff while waiting for
the socket path to be created.

Change-Id: I18706a64ce06f8916a07892dfbcd409ac5b3bff1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9568
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-08 22:50:42 +00:00
Florian Klink
c63c55f052 chore(tvix): bump tokio[-util], tonic, prost
This bumps all these dependencies to their more recent versions.

We have to enable the `macros` tonic feature, and looks like we should
also enable `rt`, not only `rt-multi-thread`.

Change-Id: Icc3600848fca0bacce24f4889d088e75711594ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8984
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-10-08 21:53:20 +00:00
Florian Klink
c847cc32d9 refactor(tvix/castore): move tests to grpc client, rm tonic-mock
Similar to gen_directorysvc_grpc_client, introduce a
gen_blobsvc_grpc_client function that provides a gRPC client connected
to a blobservice.

The test is update to use that client to test against, rather than the
server trait, removing the last usage of tonic_mock, so it's removed
as well.

Fixes b/243.

Change-Id: If746e8600588da247eb53a63b70fe72f139e9e77
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9564
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
2023-10-08 21:53:20 +00:00
Florian Klink
6b02800e48 chore(tvix/[ca]store): allow building without tonic-reflection
- rename the feature to `tonic-reflection` in both tvix-store and tvix-
   castore.
 - set it to disabled in tvix-castore by default
 - enable it in tvix-store, and pull in tvix-castore with the feature
   enabled.

Change-Id: Ie22833d85569502cae55812f6eeb17a9c15b9e2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9472
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-09-26 10:07:40 +00:00
Florian Klink
80a02f96ce chore(tvix/castore): remove dependency on anyhow
We don't use this here, except in a test, where we don't really use
it either.

Change-Id: Ia6c45fccf663fe328942b1e1a2cd1c3a1b7f9ae5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9376
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-09-22 12:51:21 +00:00
Florian Klink
32f41458c0 refactor(tvix): move castore into tvix-castore crate
This splits the pure content-addressed layers from tvix-store into a
`castore` crate, and only leaves PathInfo related things, as well as the
CLI entrypoint in the tvix-store crate.

Notable changes:
 - `fixtures` and `utils` had to be moved out of the `test` cfg, so they
   can be imported from tvix-store.
 - Some ad-hoc fixtures in the test were moved to proper fixtures in the
   same step.
 - The protos are now created by a (more static) recipe in the protos/
   directory.

The (now two) golang targets are commented out, as it's not possible to
update them properly in the same CL. This will be done by a followup CL
once this is merged (and whitby deployed)

Bug: https://b.tvl.fyi/issues/301

Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-22 12:51:21 +00:00