This also takes input validation out of the loop, leaving the loop
backedge as the sole branch in the hot path.
Change-Id: Id08e6fb9cf5b074780efa09a7ad389352a601bcc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9847
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This should remove the ambiguity on where buildkite.yml should show up
at least. We might still want to move workspace.josh into tvix/ too at
some point, but that's blocked on
https://github.com/josh-project/josh/issues/1287
This commit doesn't cause any change in views/tvix, with a previous
version of this commit added to my HEAD, I ran josh-filter to verify it
still points to ce0df88561fee8a4d6052ca9f2a1792f899a202a.
Change-Id: Ic03e92260dc7725bc2fcf01b7c72a097d92c028e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9828
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
The `if let` wasn't matching `outputHashAlgo` being unset, and didn't
populate it in that case.
Port the remaining commented-out testcases over to nix-lang based tests.
Change-Id: I140b5643b9ed9d29f9522ec65d98d0b12262d728
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9825
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Get some test coverage with some calls to builtins.derivation.
The expected output paths were calculated with Nix.
Change-Id: I2ce205ea5244e8ef939d9cacb033283fc6f15d17
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9817
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Have a private `eval` function in the test module that returns an
EvaluationResult, and migrate the existing tests over to use it, rather
than repeating itself.
Change-Id: I879987700c8507248c644ef03b62a8cb8e308139
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9816
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
If the output was fixed, we broke out of the for loop too early, before
actually validating individual outputs.
Change-Id: I2259697dfa2a157764358f6d326a1f7f6610647c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9815
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
This specific struct is only used to represent content-addressed paths
(in case a Derivation has a fixed-output hash, for example).
Rename `Output`'s `hash_with_mode` to `ca_hash`.
We now also include `CAHash::Text`, and update the `validate` function
of the `Output` struct to reject text hashes there.
This allows cleaning up the various output path calculation functions
inside nix-compat/src/store_path/utils.rs, as they can now match on
the type.
`make_type` is renamed to `make_references_string`,
`build_regular_ca_path` is renamed to `build_ca_path`, and
`build_text_path` has a disclaimer added, because you might not actually
want to use it.
Change-Id: I674d065f2ed5c804012ddfed56e161ac49d23931
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9814
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
This adds support to represent the `CA` field found in some .narinfo
files. As `deriver`, it's also a subfield of the `narinfo` field.
Extending nix-compat with a more accessible data structure that can
take care of formatting, as well as writing validation functions in Rust
+ Golang, and integrating it into nar-bridge is something for a followup
CL.
Change-Id: I71e9c30957bcd03051a491aa54d7baac25b6dd2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9795
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This information is present in the .narinfo files, it should have gone
there.
Change-Id: Ib43d0cf30c2795bf1fe77c46646174353ade0458
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9794
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
It is *eventually* followed by a Node, but there is some stuff in
between.
Change-Id: Ie7c7b462828bd3e066f4a7e774895f30b82763ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9768
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Equivalent to the existing code, but a little less cryptic.
Change-Id: Ib9b2f9aedddc84d0e79840bba4cce01f92d9bc56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9766
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
As seen in cl/9761, we only use put() and close() in some cases.
Make sure this is documented, so we don't end up having implementations
not able to deal with that.
Change-Id: I406ad23f2f4c9604eca731d898ae41a4663fb846
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9786
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
`code.tvl.fyi/tvix/store/protos` now points to a directory that only
contains the `.proto` files, while all golang tooling and .pb.go files
live in tvix/store-go.
As discussed in
https://cl.tvl.fyi/c/depot/+/9787/comment/fc5d155c_1bd38e3a/, the amount
of people currently using this is still small, so rename the go.mod now,
while it doesn't yet hurt.
Also, use code.tvl.fyi/tvix/castore-go instead of code.tvl.fyi/tvix/
castore/protos, to make use of cl/9791.
Change-Id: I9ea89957d7c29dfae4c893b9aae8ac8a0bad2d8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9792
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
`code.tvl.fyi/tvix/castore/protos` now points to a directory that only
contains the `.proto` files, while all golang tooling and .pb.go files
live in tvix/castore-go.
As discussed in
https://cl.tvl.fyi/c/depot/+/9787/comment/fc5d155c_1bd38e3a/, the amount
of people currently using this is still small, so rename the go.mod now,
while it doesn't yet hurt.
Change-Id: Ib3c6a2dac2923b3806ebb05be00af66d0da9f698
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9791
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This keeps things isolated a bit more.
Change-Id: I437f2f63ee1567c0cbc02298514ad4d89f1dce85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9790
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Similar to the castore-go CL before, this also updates the store-go
bindings to the new layout.
Change-Id: Id73d7ad43f7d70171ab021728e303300c5db71f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9788
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Have `tvix/castore/protos` only contain the protos, no go noise.
Make the `.pb.go` file generation a pure Nix build
at `//tvix/castore/protos:go-bindings`, and have a script at
`//tvix:castore-go-generate` (TBD) that copies the results to
`tvix/castore-go`.
`//tvix:castore-go`, with sources in `tvix/castore-go` now contains the
tooling around the generated bindings, and the generated bindings
themselves (So go mod replace workflows still work).
An additional CI step is added from there to ensure idempotenty of
the .pb.go files.
The code.tvl.fyi webserver config is updated to the new source code
path. I'm still unsure if we want to also update the go.mod name. While
being a backwards-incompatible change, it'll probbaly make it easier
where to find these files, and the amount of external consumers is still
low enough.
Part of b/323.
Change-Id: I2edadd118c22ec08e57c693f6cc2ef3261c62489
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9787
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
After finishing the ingestion, the directory putter was not being
closed. This caused a race where the root directory node was accessed
before the directory node had been flushed to the server.
This patch makes it so we close the putter before returning the root
node which should ensure that the root node exists on the directory
service server before the `ingest_path` function returns.
Fixes b/326
Change-Id: Id16cf46bc48962121dde76d3c9c23a845d87d0f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9761
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
That's needed to build nar-bridge, as well as things in tvix/*store/
protos.
Change-Id: If5f71b6c528a27689faa3c5ab3489cc528f9e6e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9759
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
We need to invoke protoc at build time (only).
Change-Id: Ib4c101d2ccdbdbb078725fc98c30d6c00086667d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9754
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
On Darwin, some crates producing binaries need to be able to link
against security.
Change-Id: I5bdd69247c12729b9efd5c4f18527d361ef99e87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9758
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Also, make some of them buildInputs, not nativeBuildInputs.
We likely need to link against libiconv, so it shouldn't be a native
build input.
Change-Id: I757bc7f5d8c864ecaa3db26988d00b496c085c8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9757
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
It looks like we need Security Framework in the shell to be able to
imperatively run `cargo build`.
Change-Id: Ia5df9052bafca3610d4ed235b180b812d33d0d15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9756
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
The previous CLs did already absorb all the logic into the common tests,
no need to write this here again.
Change-Id: I7ba84ba86d5445ed247e5d11d5e59b7fa815670e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9732
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Due to the lack of a ATerm parser, we were previously loading the JSON
fixtures to construct our Derivation structs to run the output path
calculations with.
However, as we now have a ATerm parser, we can load the ATerm
representation directly.
This also means we can test the output path calculation for non-UTF8
Derivations.
Change-Id: I0e53f41a23566b5ad5f0fed12724e02a10b02707
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9731
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This provides a nom-based parser for Nix derivations in ATerm format,
which can be reached via `Derivation::from_aterm_bytes`.
Some of the lower-level ATerm primitives are moved into a (new) aterm
module, and some more higher-level ones that construct derivation-
specific types.
Also, move the escape_bytes function into there, this is a generic ATerm
thing.
Change-Id: I2b03b8a1461c7ea2fcb8640c2fc3d1fa3ea719fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9730
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
