Commit graph

1692 commits

Author SHA1 Message Date
edef
3c0e42167f feat(tvix/nix-compat): use AsRef<[u8]> for base32 decode input
Change-Id: If2275dda62b852aedb64b4f9915f7dc4173cabd2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9851
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-27 12:18:23 +00:00
edef
6a0a75c8e1 refactor(tvix): condense long bytestrings
Change-Id: I3bea0827ec2c8db835334ce378a7bf3a39e9b1a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9849
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-27 07:37:26 +00:00
edef
55c37a2871 feat(tvix/nix-compat): decode base32 with a lookup table
This also takes input validation out of the loop, leaving the loop
backedge as the sole branch in the hot path.

Change-Id: Id08e6fb9cf5b074780efa09a7ad389352a601bcc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9847
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-27 07:31:46 +00:00
edef
67b08469db refactor(tvix/nix-compat): clean up base32 decoding
Change-Id: I8591a3e1075f2281b7dc49f7fa9e0027062a78a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9846
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-27 07:31:46 +00:00
edef
4d35a56798 refactor(tvix/nix-compat): simplify encode_len
Change-Id: Idfefd3b5b0d1ffbd4e9208e634a2d9989965a291
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9845
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-27 07:31:46 +00:00
Florian Klink
a89695c4c5 chore(tvix): move buildkite.yml from views/tvix to tvix/
This should remove the ambiguity on where buildkite.yml should show up
at least. We might still want to move workspace.josh into tvix/ too at
some point, but that's blocked on
https://github.com/josh-project/josh/issues/1287

This commit doesn't cause any change in views/tvix, with a previous
version of this commit added to my HEAD, I ran josh-filter to verify it
still points to ce0df88561fee8a4d6052ca9f2a1792f899a202a.

Change-Id: Ic03e92260dc7725bc2fcf01b7c72a097d92c028e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9828
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-10-25 08:22:57 +00:00
Florian Klink
6a04ba6bc5 fix(tvix/cli/derivation): fix populate_output_configuration
The `if let` wasn't matching `outputHashAlgo` being unset, and didn't
populate it in that case.

Port the remaining commented-out testcases over to nix-lang based tests.

Change-Id: I140b5643b9ed9d29f9522ec65d98d0b12262d728
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9825
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-10-23 14:57:32 +00:00
Florian Klink
2d99bfc7b7 refactor(tvix/cli): rename a to algo_str
Change-Id: I65d307126647841b50db008c21c155441a6b6cb5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9818
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-10-23 14:57:32 +00:00
Florian Klink
6800f1cdb9 test(tvix/cli/derivation): add test cases for output path calc
Get some test coverage with some calls to builtins.derivation.

The expected output paths were calculated with Nix.

Change-Id: I2ce205ea5244e8ef939d9cacb033283fc6f15d17
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9817
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-23 14:57:32 +00:00
Florian Klink
077bf06c1d refactor(tvix/cli): move evaluator instantiation to helper
Have a private `eval` function in the test module that returns an
EvaluationResult, and migrate the existing tests over to use it, rather
than repeating itself.

Change-Id: I879987700c8507248c644ef03b62a8cb8e308139
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9816
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-23 14:57:32 +00:00
Florian Klink
6fe7684832 fix(nix-compat/src/derivation/validate): remove break
If the output was fixed, we broke out of the for loop too early, before
actually validating individual outputs.

Change-Id: I2259697dfa2a157764358f6d326a1f7f6610647c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9815
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-23 14:57:32 +00:00
Florian Klink
34fc4637eb refactor(tvix/nix-compat): rename NixHashWithMode -> CAHash
This specific struct is only used to represent content-addressed paths
(in case a Derivation has a fixed-output hash, for example).
Rename `Output`'s `hash_with_mode` to `ca_hash`.

We now also include `CAHash::Text`, and update the `validate` function
of the `Output` struct to reject text hashes there.

This allows cleaning up the various output path calculation functions
inside nix-compat/src/store_path/utils.rs, as they can now match on
the type.

`make_type` is renamed to `make_references_string`,
`build_regular_ca_path` is renamed to `build_ca_path`, and
`build_text_path` has a disclaimer added, because you might not actually
want to use it.

Change-Id: I674d065f2ed5c804012ddfed56e161ac49d23931
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9814
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-23 14:57:32 +00:00
Florian Klink
833957b374 feat(tvix/store/protos): add CA field
This adds support to represent the `CA` field found in some .narinfo
files. As `deriver`, it's also a subfield of the `narinfo` field.

Extending nix-compat with a more accessible data structure that can
take care of formatting, as well as writing validation functions in Rust
+ Golang, and integrating it into nar-bridge is something for a followup
CL.

Change-Id: I71e9c30957bcd03051a491aa54d7baac25b6dd2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9795
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-22 17:04:37 +00:00
Florian Klink
9118dc8a50 refactor(tvix/store): move deriver field into narinfo submessage
This information is present in the .narinfo files, it should have gone
there.

Change-Id: Ib43d0cf30c2795bf1fe77c46646174353ade0458
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9794
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-22 17:04:37 +00:00
Florian Klink
42d3c1a748 test(tvix/nix-compat/nar/writer/sync): add tests
Change-Id: I04c556e7b57f20dd33f71b0379e6104e1702a8ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9785
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-18 12:31:57 +00:00
Florian Klink
05bca22121 feat(tvix/nix-compat/nar/reader): add tests
Change-Id: If0abaf74a3b7d201af8972e54e3e34b20b8c0bc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9764
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-18 12:31:57 +00:00
edef
e64acba454 fix(tvix/nix-compat/nar/wire): correct Node::Some doc comment
It is *eventually* followed by a Node, but there is some stuff in
between.

Change-Id: Ie7c7b462828bd3e066f4a7e774895f30b82763ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9768
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-18 12:22:47 +00:00
edef
2cae6a02b9 refactor(nix-compat/nar/writer): use clone_into
Equivalent to the existing code, but a little less cryptic.

Change-Id: Ib9b2f9aedddc84d0e79840bba4cce01f92d9bc56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9766
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-18 11:42:28 +00:00
edef
6638f4d4ea feat(tvix/nix-compat): NAR reader
Change-Id: I50d51baf62c0419eaf17f0dc262f728aaff9794d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9688
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-18 11:40:06 +00:00
edef
08b98b7503 docs(tvix/nix-compat/nar): document the wire format
Change-Id: I6c8e23bad27fa6ada1b8973482b4d99190cf050d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9767
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-18 11:40:06 +00:00
Florian Klink
57b69baf7c refactor(tvix/nix-compat): use hash_with_mode's nix_hash_string
HashWithMode already provides a to_nix_hash_string() method, giving us
exactly the string we need here.

Change-Id: Id2635bf3ea6c2514faf3c26b297866d774f4ff4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9799
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-18 10:32:43 +00:00
Florian Klink
1bb7e84241 docs(tvix/castore/directorysvc): document drop without put and close
As seen in cl/9761, we only use put() and close() in some cases.

Make sure this is documented, so we don't end up having implementations
not able to deal with that.

Change-Id: I406ad23f2f4c9604eca731d898ae41a4663fb846
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9786
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-17 21:12:43 +00:00
Florian Klink
718af22dbb chore(tvix/nar-bridge): update to latest go mod renames
- code.tvl.fyi/tvix/castore/protos -> code.tvl.fyi/tvix/castore-go
 - code.tvl.fyi/tvix/store/protos -> code.tvl.fyi/tvix/store-go

 See cl/9791, cl/9792 for context.

Change-Id: I44614c6ed40b9f52d9dcdea8e61fe2c3c830ce78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9793
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-10-17 20:46:38 +00:00
Florian Klink
2d51da7f40 chore(tvix/store-go): rename go module
`code.tvl.fyi/tvix/store/protos` now points to a directory that only
contains the `.proto` files, while all golang tooling and .pb.go files
live in tvix/store-go.

As discussed in
https://cl.tvl.fyi/c/depot/+/9787/comment/fc5d155c_1bd38e3a/, the amount
of people currently using this is still small, so rename the go.mod now,
while it doesn't yet hurt.

Also, use code.tvl.fyi/tvix/castore-go instead of code.tvl.fyi/tvix/
castore/protos, to make use of cl/9791.

Change-Id: I9ea89957d7c29dfae4c893b9aae8ac8a0bad2d8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9792
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-17 20:36:36 +00:00
Florian Klink
6e8fbc830a chore(tvix/castore-go): rename go module
`code.tvl.fyi/tvix/castore/protos` now points to a directory that only
contains the `.proto` files, while all golang tooling and .pb.go files
live in tvix/castore-go.

As discussed in
https://cl.tvl.fyi/c/depot/+/9787/comment/fc5d155c_1bd38e3a/, the amount
of people currently using this is still small, so rename the go.mod now,
while it doesn't yet hurt.

Change-Id: Ib3c6a2dac2923b3806ebb05be00af66d0da9f698
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9791
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-10-17 20:25:45 +00:00
Florian Klink
45d8a78a8b chore(tvix): move regenerate scripts to passthru
This keeps things isolated a bit more.

Change-Id: I437f2f63ee1567c0cbc02298514ad4d89f1dce85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9790
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-17 20:25:45 +00:00
Florian Klink
1b26bf21e3 chore(tvix): move store golang bindings to tvix/store-go
Similar to the castore-go CL before, this also updates the store-go
bindings to the new layout.

Change-Id: Id73d7ad43f7d70171ab021728e303300c5db71f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9788
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-17 19:51:24 +00:00
Florian Klink
e38733a955 chore(tvix): move castore golang bindings to tvix/castore-go
Have `tvix/castore/protos` only contain the protos, no go noise.

Make the `.pb.go` file generation a pure Nix build
at `//tvix/castore/protos:go-bindings`, and have a script at
`//tvix:castore-go-generate` (TBD) that copies the results to
`tvix/castore-go`.

`//tvix:castore-go`, with sources in `tvix/castore-go` now contains the
tooling around the generated bindings, and the generated bindings
themselves (So go mod replace workflows still work).

An additional CI step is added from there to ensure idempotenty of
the .pb.go files.

The code.tvl.fyi webserver config is updated to the new source code
path. I'm still unsure if we want to also update the go.mod name. While
being a backwards-incompatible change, it'll probbaly make it easier
where to find these files, and the amount of external consumers is still
low enough.

Part of b/323.

Change-Id: I2edadd118c22ec08e57c693f6cc2ef3261c62489
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9787
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-17 19:51:24 +00:00
Connor Brewster
0325ae3ba3 fix(tvix/castore): Fix race when ingesting into castore
After finishing the ingestion, the directory putter was not being
closed. This caused a race where the root directory node was accessed
before the directory node had been flushed to the server.

This patch makes it so we close the putter before returning the root
node which should ensure that the root node exists on the directory
service server before the `ingest_path` function returns.

Fixes b/326

Change-Id: Id16cf46bc48962121dde76d3c9c23a845d87d0f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9761
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-17 13:01:29 +00:00
Florian Klink
73a19c249e refactor(tvix/nix-compat/derivation): remove some clones
Change-Id: Ie464467edfafd2050aa9e5fd0e07ff745cb15df8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9753
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-16 16:17:41 +00:00
Florian Klink
7df720855d feat(tvix): add go to shell
That's needed to build nar-bridge, as well as things in tvix/*store/
protos.

Change-Id: If5f71b6c528a27689faa3c5ab3489cc528f9e6e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9759
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-16 16:12:18 +00:00
Florian Klink
e0e3dc08a9 fix(tvix/cli): don't use protobuf from pkgs, but pkgs.buildPackages
We need to invoke protoc at build time (only).

Change-Id: Ib4c101d2ccdbdbb078725fc98c30d6c00086667d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9754
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-16 16:08:04 +00:00
Florian Klink
0f44451919 fix(tvix): fix binary building on Darwin
On Darwin, some crates producing binaries need to be able to link
against security.

Change-Id: I5bdd69247c12729b9efd5c4f18527d361ef99e87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9758
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-16 15:14:21 +00:00
Florian Klink
5ce1a03a3a refactor(tvix): push rarely-used darwin-specific hacks down
Also, make some of them buildInputs, not nativeBuildInputs.

We likely need to link against libiconv, so it shouldn't be a native
build input.

Change-Id: I757bc7f5d8c864ecaa3db26988d00b496c085c8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9757
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
2023-10-16 15:09:28 +00:00
Florian Klink
6aa8a1a8ea fix(tvix/shell): fix cargo build on MacOS
It looks like we need Security Framework in the shell to be able to
imperatively run `cargo build`.

Change-Id: Ia5df9052bafca3610d4ed235b180b812d33d0d15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9756
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
2023-10-16 15:09:27 +00:00
Florian Klink
652afd21b3 feat(nix-compat): add bench for aterm parsing
Change-Id: Ie9416f0046b6ee0a02096a7712cdba9cc84db4f6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9733
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Reviewed-by: Gabriel Doriath Döhler <gabriel.doriath.dohler@ens.fr>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-10-16 12:26:36 +00:00
Florian Klink
c3fad811f4 refactor(nix-compat/derivation): drop non_unicode test
The previous CLs did already absorb all the logic into the common tests,
no need to write this here again.

Change-Id: I7ba84ba86d5445ed247e5d11d5e59b7fa815670e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9732
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-16 12:26:36 +00:00
Florian Klink
398cf8e7ce refactor(nix-compat/derivation): use ATerm for output_paths test
Due to the lack of a ATerm parser, we were previously loading the JSON
fixtures to construct our Derivation structs to run the output path
calculations with.

However, as we now have a ATerm parser, we can load the ATerm
representation directly.
This also means we can test the output path calculation for non-UTF8
Derivations.

Change-Id: I0e53f41a23566b5ad5f0fed12724e02a10b02707
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9731
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-16 12:26:36 +00:00
Florian Klink
8934b34489 fix(nix-compat/derivation): handle dups
This now properly checks for duplicate output names in input derivation
output names, and duplicate input sources.

Change-Id: I0053854bfbf504f4f511fb3fe1a77a82b3aa61dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9738
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-16 12:23:00 +00:00
Florian Klink
2410f2292f feat(nix-compat/{aterm,derivation}): init parser
This provides a nom-based parser for Nix derivations in ATerm format,
which can be reached via `Derivation::from_aterm_bytes`.

Some of the lower-level ATerm primitives are moved into a (new) aterm
module, and some more higher-level ones that construct derivation-
specific types.

Also, move the escape_bytes function into there, this is a generic ATerm
thing.

Change-Id: I2b03b8a1461c7ea2fcb8640c2fc3d1fa3ea719fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9730
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-10-16 12:23:00 +00:00
Florian Klink
c739df3d78 refactor(nix-compat/derivation): fix function name
typo: write_enviroment -> write_environment.
Change-Id: I1a2dc826c0bf71f6ba97fe41c4d44619b5e6beff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9739
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-10-16 11:32:13 +00:00
Florian Klink
c50967c1f1 refactor(tvix/cli): clippy
Change-Id: I35a28a320c890f62e6a76641dfdf99e33e839c02
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9749
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
2023-10-16 11:13:33 +00:00
Florian Klink
72872785b4 refactor(tvix/store): clippy
Change-Id: I31ba268bda21c1f07c63520a269dd62f7b309e99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9748
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-16 11:13:33 +00:00
Florian Klink
1a3c610081 refactor(tvix/nix-compat): clippy
Change-Id: I56282d9754b42e0e6cdbb08e9ae71f5700d28dd1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9747
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-10-16 11:13:33 +00:00
Florian Klink
1d97b4230f refactor(tvix/castore): clippy
Change-Id: I8a792d6b238190a78390c5d8f638669a2681f70f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9746
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-16 11:13:33 +00:00
Florian Klink
869be82e09 refactor(tvix/store/fs): clippy
Change-Id: I11247e24475f0e49dc22f2d10bf3e1e276737899
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9709
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-10-16 10:45:37 +00:00
Florian Klink
9aafbe8d95 refactor(nix-compat/derivation): make write_environment more generic
We don't actually care if it's a BTreeMap of strings, bstrings or any of
that sort.

The only thing we want to be able to do is get a reference to the bytes
from the keys and values.

Change-Id: I21b85811a9ea47fe06afa3108836ef9295e5d89b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9737
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-10-15 15:10:19 +00:00
Florian Klink
f337601f69 refactor(nix-compat/derivation): move test fixtures to subdir
We use test_resources and globbing for some of the test cases, so adding
additional files in there will also create new test cases, which we
don't always want.

Move it down one level to make some more space.

Change-Id: I619867dc80a4ced59d45096d0703678663b559cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9729
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-15 15:10:12 +00:00
Florian Klink
8a0fa69c6c feat(tvix/nix-compat/nixhash/NixHashWithMode): from_algo_mode_hash
Change-Id: Ieae628fab1926a498ae5c3eb27df1b722e6151d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9728
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-15 15:10:07 +00:00
Florian Klink
c3446da1c7 refactor(nix-compat/nixhash): add Result type alias
Change-Id: Id0248047e9642d38afc106629957a2e7608f8c78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9727
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-10-15 02:21:34 +00:00
Florian Klink
cb1a14334a refactor(nix-compat/nixhash): box sha512
Limit the amount of memory consumed on the stack for NixHash. Sha512
isn't used that often, so it's fine if we heap-allocate it.

Change-Id: I4a9eecd20c6184610124dc130c41bfa5d0dc04c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9726
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-15 02:21:34 +00:00
Connor Brewster
e3d72cc4cb refactor(tvix/store): Upgrade tokio-listener to get tonic support
Tonic support was added to tokio-listener upstream which removes the
need for use to have tonic compatibility wrapper types around it.

See: https://github.com/vi/tokio-listener/pull/2

Fixes b/311

Change-Id: I04a2dbb3bc3c8bfe9339583c0b46070c7ec97811
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9721
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-14 18:56:51 +00:00
Florian Klink
cc040a5ad3 refactor(tvix/nix-compat/derivation): fod_digest return [u8; 32]
This is of known length.

Change-Id: Iba48ccc486d5bf9e38ec1a2da6e7b80997d2c6ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9723
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-14 17:55:08 +00:00
Florian Klink
4ae0f428bd refactor(tvix/nix-compat): make NixHash an enum with fixed-len bytes
Less Vec<u8> passed around.

Change-Id: Ie153a6bfaa084d7490ffa38634efdf5f3c31a768
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9722
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-14 17:55:08 +00:00
Florian Klink
786b0324a9 feat(tvix/store/protos): use Validate() function on root node
This updates the code to make use of the new Validate() function defined
on a Node.

Change-Id: I9b6ed694661f41e700f19cc78d53d2224b61852d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9718
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-10-14 14:21:32 +00:00
Florian Klink
2d2c4322d9 feat(tvix/store/protos): add Deriver field to PathInfo
This uses the newly introduced StorePath message type to add a Deriver
field to the PathInfo message.

Support for validation is added to both the golang and rust
implementation. This includes extending unit tests.

Change-Id: Ifc3eb3263fa25b9eec260db354cd74234c40af7e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9647
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-14 13:23:07 +00:00
Florian Klink
5f8eb4eeaa feat(tvix/store/protos): add StorePath message
This encodes a store path a bit more concise, which is used in the
Deriver field.

Change-Id: Ibfb54d3b206917e51970d1d5fe94fcedb901704b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9646
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-14 13:23:07 +00:00
Florian Klink
173641ed37 refactor(tvix/store/fs): make fetch_directory_inode_data async
To make this easier, move it outside of TvixStoreFs, and accept the
DirectoryService as a function argument, so we don't need to worry about
the lifetime of self.

This also aligns with how we spawn async tasks inside the rest of
TvixStoreFs.

Change-Id: I3b95072209d32039f05aed122240f2d6db7ad172
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9713
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-14 12:34:36 +00:00
Florian Klink
9757bf6377 refactor(tvix/*store): helper for channel creation from url
This moves the repetitive code to parse a URL and create a channel
connected to it into `tvix_castore::channel::from_url`.

Part of b/308

Change-Id: Idd342cd71cad5e78a9b258b38c1b227993e75310
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9707
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-14 12:26:40 +00:00
Florian Klink
3f011d2762 refactor(tvix/*store): reorganize from_url
Move the channel creation depending on the string-based URL into its
own block.

Change-Id: I546b769acd2296b548eb966b62c495f910266df5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9706
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-14 12:26:40 +00:00
Florian Klink
199e5e0339 refactor(tvix/*store): remove some grpc_client let bindings
We had to have these all while the traits where sync, and there was a
lot of spawning and moving.

Most of this can now be removed in favor of some inline `.clone()`.

Change-Id: Id5466c32a403100bc3347866b3172e06a792e311
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9705
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-14 12:26:40 +00:00
Florian Klink
0b18be3b57 feat(tvix/castore/protos): add more granular validation methods
Similar to cl/9715, this makes the validation checks more granular,
introducing a Validate on all *Node.

A check for symlink targets is added too.

Once merged, it can also be used from tvix/store/protos.

Change-Id: I0909a89fadcd74b74ef0c9a8a1f22658fccc83b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9716
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2023-10-14 12:21:18 +00:00
Florian Klink
532f414da6 refactor(tvix/castore): simplify node validation checks
We can just check the digest length to be correct, and move the symlink
target checks to a single line.

Change-Id: I41d2e3a50e7990ef6c04f02acd754b1e17b43e77
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9717
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-14 12:09:10 +00:00
Linus Heckemann
b1ab8075cd docs(tvix/castore): point out use of contents_first
Change-Id: I7620d2abe01675ea7028a478d4f8447e36d5768b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9605
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-13 17:14:48 +00:00
Florian Klink
64beaab163 refactor(tvix/store/protos): remove unused mustBlobDigest function
Change-Id: I79a73de0cb06abcf16a6579ae298876004ab0ba8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9719
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-13 14:43:26 +00:00
Florian Klink
9b7629826f refactor(tvix/castore): factor out node checks
Implement `validate()` on `node::Node`, and call it from PathInfo's
validate() too. Node-related errors are moved to a ValidateNodeError
error type.

This additionally adds some more validations for symlink targets (they
must not be empty, and not contain null bytes).

Change-Id: Ib9b89f1c9c795e868a1533281239bc8a36d97c5d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9715
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-10-12 21:05:48 +00:00
Florian Klink
c9e90b4dd7 refactor(tvix/store/fs): use while let statement
Suggested by clippy.

Change-Id: I686b1abcb40fab42017972a3f814fae29fa70c25
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9712
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-10-12 20:46:24 +00:00
Florian Klink
4da906bf34 refactor(tvix/store/proto): merge two match statements into one
Change-Id: I3daca008dff5527169f5916f4845234e8f3263cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9711
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-10-12 20:46:24 +00:00
Florian Klink
3dd5ba042b refactor(tvix/store): clippy lint
Change-Id: I03aa5c6cb0167ff6cb7312d81d1e43116fcb6d70
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9710
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-10-12 20:46:24 +00:00
Florian Klink
04c1f151f3 refactor(tvix/castore/utils): clippy
Change-Id: Ib0fa1e1fb90d7f315f943a3f28266c0fb3714a31
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9708
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-10-12 20:46:24 +00:00
Florian Klink
a9e5a7cec9 feat(tvix/store): enable tls[-roots] feature in tonic
This teaches tonic to speak https, so `grpc+https` becomes a working
connection string.

Change-Id: I9c550ea665b829725b545a164cf6f78587bd5213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9714
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
2023-10-12 19:18:59 +00:00
edef
be32ab1eb2 fix(tvix/store/fs): fix typo
Change-Id: I132987dda1073dcc0b30e50e26682d6c8724d988
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9689
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-12 09:51:30 +00:00
Florian Klink
1c5bf366be refactor(tvix/nar-bridge): panic on inval PathInfo.NarInfo.NarSha256
Bump code.tvl.fyi/tvix/store/protos past cl/9649, where Validate()
already ensures the NarSha256 has the correct size.

Change-Id: I774668822f4d9dbd4dea47dde6e4745dc95e8e7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9665
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-10-12 06:24:07 +00:00
Florian Klink
e64e97ee23 feat(tvix/store): validate nar_sha256
Change-Id: I4c4dcdb75ea7748f2ab01a0bab218596b90b7b58
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9648
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-10-11 15:29:08 +00:00
Florian Klink
8ced43f3c7 feat(tvix/store/protos): validate NarSha256
Change-Id: I016d1b020b04850f0dca68c2ea96643230dada10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9649
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
2023-10-11 15:29:08 +00:00
Florian Klink
6dab608198 docs(tvix/nar-bridge): remove redundant comments
These really only describe what the code is doing, remove.

Pointed out in
https://cl.tvl.fyi/c/depot/+/9652/comment/987b3b7f_c7b5e47b/

Change-Id: I512a274295244f96744a489871861fed9523bb37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9654
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
2023-10-11 15:22:22 +00:00
Florian Klink
155ab77cff refactor(tvix/nar-bridge): use mh.SHA2_256
As correctly mentioned in
https://cl.tvl.fyi/c/depot/+/9652/comment/03b9b96e_bbb337fd/,
we shouldn't be using these magic constants, but pull them from where
they're defined.

This already is a dependency of go-nix, and pkg/pathinfosvc/server.go,
so no changes in go.mod.

Change-Id: I0cc41ce040fcbddf4b6171417bc9b0de55af4991
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9653
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-11 15:22:22 +00:00
Florian Klink
fb04645b0d docs(tvix/store/protos): explain a bit more what we're doing
Explain a bit better what we do in the codeblock below,
and add a newline to make it clear we don't just document the first
branch.

Change-Id: Ifb142d9984e6d2cbca648525c10298f2fcbdfedf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9645
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Autosubmit: flokli <flokli@flokli.de>
2023-10-11 11:43:29 +00:00
Florian Klink
673f5febbb feat(tvix/nar-bridge): stop parsing nixbase32 manually, validate
We have nixhash.FromHashTypeAndDigest now.

Also, run Validate() on the PathInfo received from the remote
PathInfoService.

Change-Id: I14db0d9356c539c084afc9dd712314b56da2587e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9652
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-11 11:41:50 +00:00
Florian Klink
98c17147c6 refactor(tvix/nar-bridge): have Export return root node
… and nar size / sha256 digest.

Instead of producing sparse PathInfo messages when NARs are sent to
nar-bridge, the nar-bridge http server now keeps a lookup table
(narsha256) -> (rootNode, narSize)

This removes a whole bunch of noise, because we don't need to keep
sparse fields around.

A convenience function
`GenPathInfo(rootNode *castorev1pb.Node, narInfo *narinfo.NarInfo)` is
added, which is used to produce PathInfo messages, either when receiving
a NAR file over http and uploading it to a remote PathInfoService, or to
synthesize the PathInfoMessage to return to the client, if nar-bridge is
acting as a PathInfoService for a remove Nix HTTP Binary cache.

Change-Id: Ibba1ab6238a050816c4fab29cb21ae88877d8613
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9651
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-11 11:41:50 +00:00
Florian Klink
ceb1674e9f refactor(tvix/nar-bridge): pass root node to storev1pb.Export
Bumps the go module past cl/9604 and update the consumer side.

Change-Id: Id44245017f1dc2f8aac28051cdbb45b83bdc5be3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9650
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
2023-10-11 11:41:50 +00:00
edef
930edb69d7 fix(tvix/castore): use bstr for formatting names in errors
Much friendlier than either Base64 or raw byte slices.

Change-Id: I9b4cdd57c83ddc76c0be8103da4320207657a72b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9622
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-10 22:40:55 +00:00
edef
baae5ce473 fix(tvix/castore): handle Directory::size overflow explicitly
We use checked arithmetic for computing the total size, and verify
that size is in-bounds in Directory::validate.

If an out-of-bounds size makes it to the "unchecked" size method,
we either panic (in debug mode), or silently saturate to u32::MAX.

No new panic sites are added, since overflows in debug mode already
panic at the language level.

Change-Id: I95b8c066a42614fa447f08b4f8fe74e16fbe8bf9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9616
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-10 20:33:52 +00:00
edef
e2dba089c4 test(tvix/castore): demonstrate that Directory::size may panic
Change-Id: Iebfa12df51c8c9d39b4d9eb2fccc6ffa312639cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9620
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-10 20:33:52 +00:00
edef
d38e6502ad fix(third_party/crate2nix): run tests in debug mode
Behave like `cargo test` rather than `cargo test --release`.

Change-Id: Ie013d04ac68d7dec2a3b870fa9f0060a70a9635d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9621
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-10 20:33:52 +00:00
Florian Klink
d94749ac22 refactor(tvix/store/protos): have Export accept root node
We don't need the full PathInfo message, only the root node.

Change-Id: I667045ed766875dfbf8ac126a50b02baa2df67a4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9604
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2023-10-10 20:22:47 +00:00
Florian Klink
fe963ae0a3 chore(tvix/nar-bridge): move to Export from storev1pb
This removes the Export method in nar-bridge, and updates all users to
the version now in storev1pb.

It moves the roundtrip test to the importer crate, and some of the
utility functions into a separate util_test.go file.

Change-Id: I81d9e0b35dfd78ef1042bed307281eecd2aaa2a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9603
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
2023-10-10 19:08:29 +00:00
Florian Klink
e6ba84ea50 feat(tvix/store/protos): add Export
Export will traverse a given PathInfo structure, and write the contents
in NAR format to the passed Writer.
It uses directoryLookupFn and blobLookupFn to resolve references.

This is being moved over from nar-bridge. We need to keep the code there
around until we can bump go.mod to storev1 with this merged, but the
tests can already be moved entirely.

Change-Id: Ie0de3077b09344cafa00ff1e2ddb8b52e9e631bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9602
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Autosubmit: flokli <flokli@flokli.de>
2023-10-10 18:55:49 +00:00
Florian Klink
6fe34b7ba0 refactor(tvix/nar-bridge): use castorev1pb.RenamedNode
We can use the helper to rename the node.

Change-Id: Id8defea7e5ebbd43d7b7a9b2992c62084e1828ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9601
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
2023-10-10 18:02:38 +00:00
Florian Klink
2699b61052 chore(tvix/nar-bridge): bump google.golang.org/protobuf
… to 1.28.1 as well.

Change-Id: Ieaaf9363a8dd63c2437c34d78a0ab369e7492f19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9600
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-10 18:02:38 +00:00
edef
d23fe6ee20 feat(tvix/store): use tvix_compat::nar::writer::async
Change-Id: Iad36872244df6f2225a2884f6b20cacd8f918b31
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9619
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: edef <edef@edef.eu>
2023-10-10 17:33:33 +00:00
edef
8b35d97b4b feat(tvix/nix-compat): add nar::writer::async
Change-Id: Ib962f889836659fd278beec461ee329b00e64ea5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9615
Autosubmit: edef <edef@edef.eu>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-10 17:13:27 +00:00
edef
f38290cfb0 refactor(tvix/nix-compat): move nar::writer to nar::writer::sync
This is preparation for adding an async port.

Change-Id: Id638ec1f6f46e2f3935448184eed51e2233263fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9618
Tested-by: BuildkiteCI
Autosubmit: edef <edef@edef.eu>
Reviewed-by: flokli <flokli@flokli.de>
2023-10-10 17:13:27 +00:00
edef
bb54e04c1b fix(tvix/nix-compat): clean up the debug assertions a bit
Consistent error messages, and slightly nicer code layout. We avoid
printing the input data, since we primarily want to point out the
specific violated invariant. In the one place where we do want to,
we use BStr's Debug implementation, since byte slices don't print
nicely.

Change-Id: I3a9a0c37be270ea5f16cf124922c254608fb849e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9617
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: edef <edef@edef.eu>
2023-10-10 16:58:17 +00:00
edef
17d6f4320f fix(tvix/nix-compat): drop unnecessary reference taking
Found by Clippy, which we should probably run in CI.

Change-Id: Id79c30b63f681021ab79358e02d29454d43c0aa6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9614
Autosubmit: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-10-10 08:57:04 +00:00
Florian Klink
d1bccfe82f chore(tvix/*store): use google.golang.org/protobuf
The other one is deprecated, according to
https://pkg.go.dev/github.com/golang/protobuf/proto.

Change-Id: I502d5bfa7431df0dea3d564f1e0ecfea14c90e10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9597
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-09 22:05:07 +00:00
Florian Klink
abf9c05090 feat(tvix/castore): add RenamedNode helper
This takes a castorev1pb.Node and updates its name, making sure to copy
it.

Change-Id: I57bf2b2b4a74496a1a697bcbce72bf67a12e69fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9594
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-09 22:05:05 +00:00
Florian Klink
78abeaa920 refactor(tvix/nar-bridge): use storepath for store paths
Remove the handwritten parsing and formatting of store paths.

Change-Id: Ia4ba486b4363c33b98937bcbf6f5f7bcda289b82
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9588
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-09 22:05:02 +00:00
Florian Klink
28cd4b1a2f feat(tvix/nar-bridge): add ToNixNarInfo()
Convenience function, moves all code converting from a PathInfo struct
to to go-nix's NarInfo.

Change-Id: Idf0dcc38675674563f2dfd3286a4a55fa2a24a82
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9593
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-09 22:05:02 +00:00
Florian Klink
92481825b3 test(tvix/store/protos): add go tests for Validate()
We already have validation tests for Rust, let's add the missing ones
for golang too.

Change-Id: Iaf3a3e1ee72d5647da3f2aa977d6e0d0379b2ce5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9595
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-10-09 22:01:59 +00:00