Commit graph

20637 commits

Author SHA1 Message Date
Profpatsch
1f65a7b0d0 fix(users/Profpatsch/whatcd-resolver): better show ApplicationError
Change-Id: I7a1087afc4000299529a7518f273bfee8d651c72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12054
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2024-07-30 12:41:26 +00:00
Aspen Smith
8aa038a90e test(tvix/glue): Add a benchmark for firefox outPath
This is nice to test too - it's similar to hello, but runs for a lot
longer (like 7.5 seconds on my laptop) which means we get even better
stats for stuff.

Change-Id: I7935818f10a6d846d446e685b9263a72d7e2aabd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12061
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
2024-07-30 08:58:49 +00:00
Ilan Joselevich
3f6cd7aebc fix(tvix/store): Immediately return an error when using sled on /
We already do this for redb and for sled in SledDirectoryService.

Change-Id: I34c7178257a6a04e9f12ed4037a4ef585d7b0d54
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12060
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2024-07-29 16:43:04 +00:00
Ilan Joselevich
47a8baf178 docs(tvix/docs): Remove redb path info service from TODO
It is now implemented and tested since https://cl.tvl.fyi/c/depot/+/11995

Change-Id: Ie08f511edf10bba4f46efc2c13f8db17eb7182a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12059
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-29 15:50:11 +00:00
Ilan Joselevich
9cbebfea27 fix(tvix/store) RedbPathInfoService: improve logs and errors
Add more logging and remove context from errors because that's already
provided by the logs (Errors also need to be refactored anyway, there's
also confusion about StorageError vs InvalidRequest, there's no
consistency)

Change-Id: Ia43c0d237d9075152490c635b05fb3fb343abcc8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12058
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-29 15:50:11 +00:00
Brian Olsen
a794790203 fix(tvix/nix-compat): Fix panic in nixbase32 decoding
The decode function didn't check that the input had a valid length and
so would panic when given input with invalid length.

Change-Id: Ie27d006b8fe20f005b4a47a1763821a61e9a95c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12051
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: Brian Olsen <me@griff.name>
2024-07-28 20:55:51 +00:00
Ilan Joselevich
a982f73477 fix(tvix/store): adjust from_addr redb test to do what it says
Change-Id: If15f161d5c7aba05ac1d8e2a4f6fac5a7053943a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12040
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-28 17:47:41 +00:00
Ilan Joselevich
d17f0592da docs(tvix/store): Document redb in from_addr
Change-Id: Id2ef4ee1b22c20e5b79156f40821578979105ddc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12039
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-28 16:40:40 +00:00
Matthew Tromp
038d1dd551 feat(tvix/eval): ConstantIdx expansion for more ops
Previously, OpConstant would display some detail about its
ConstantIdx: whether it's a thunk or closure, and what its address
is. This has been expanded to also show when the ConstantIdx is a
blueprint, along with the blueprint's address, and to the other
opcodes that use a ConstantIdx.

Currently, it seems like blueprint addresses don't correspond to the
address of the thunk listed in the bytecode output, but it's still
useful to see that the constant being grabbed is a blueprint, and
maybe this pointer can be made to make more sense in the future.

Change-Id: Ia212b0d52b004c87051542c093274e7106ee08e4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12044
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: chickadee <matthewktromp@gmail.com>
2024-07-27 22:37:59 +00:00
edef
f3c80c10b9 feat(tvix/tools/narinfo2parquet): preserve the deriver
Change-Id: Idbada585b87eef81cad5d40cb7592a7890704695
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12037
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-27 21:31:18 +00:00
Matthew Tromp
0b987a509f fix(tvix/eval): don't bubble up io errors from path_exists
path_exists was returning an error when certain common IO errors were
encountered. e.g. in the path "/dev/null/.", path_exists would throw
an error because the underlying call to Path::try_exists threw an
error because null isn't a directory. But if null isn't a directory,
then the path is invalid, so this should really be returning
false. That's what nix's behavior is and that's what makes sense.

The trait function isn't being changed because some other
implementers (e.g. tvix_store_io) have actual errors they can throw.

Fixes: b/411
Change-Id: I9e810e7a198bffe61365697c6d3d7e71f264c20b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12042
Tested-by: BuildkiteCI
Autosubmit: chickadee <matthewktromp@gmail.com>
Reviewed-by: aspen <root@gws.fyi>
2024-07-27 21:29:46 +00:00
Aspen Smith
4769d047b2 fix(tvix/cli): Make :q actually quit
whoops

Change-Id: I5f24163c276992c7858b038ae2bb636da75f3f91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12043
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2024-07-27 21:19:43 +00:00
Florian Klink
c8f92c6e9f refactor(tvix/nix-compat): introduce CaHash::algo_str
This is the `{fixed,fixed:r,text}:{sha*,md5}` prefix used in various
string representations.

Factor that code out, and use it in the two places it can be used in.

Change-Id: Ic9555fa9e1884198d435e55c7f630b8d3ba2a032
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12041
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
2024-07-27 10:53:56 +00:00
Yureka
5114504b67 test(tvix/castore/dirsvc): check for a pitfall with deduplicated dirs
Change-Id: I3cff2c2e8b2c8a2fd8d8074647d0d99a1db8e693
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12034
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-25 09:52:29 +00:00
Yureka
6a988a1598 fix(tvix/castore/GRPCDirectorySvc): fix a bug in the get_recursive fn
When retrieving a closure with get_recursive, the following could happen in the GRPC client:
- The first reference to the deduplicated directory is added to expected_directory_digests
- The deduplicated directory is obtained removed from expected_directory_digests
- The second reference to the deduplicated directory is added to expected_directory_digests
- The deduplicated directory has already been sent, but is still in the
  expected_directory_digests. It looks to the GRPC client like the
  closure is incomplete and the stream ended prematurely.

Change-Id: Ic62bca12e7f8fb85af5fa4dacd199f0f3b8eea8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12033
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-25 09:52:29 +00:00
Yureka
e97202e54a refactor(directoryservice): get rid of NaiveSeeker helper
Use the ChunkedReader in CombinedBlobService instead which also supports seeking.

Change-Id: I681331a80763172c27e55362b7044fe81aaa323b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12031
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-23 20:53:08 +00:00
Florian Klink
9385b3bcfb fix(tvix/boot/tests): check nar-bridge being fully up
Ensure nar-bridge is healthy before connecting to it, don't just check
for the unix socket to be present.

We don't have a proper /health endpoint yet, but nix-cache-info works
fine for now.

Change-Id: I22df2c3b7bffcf52dbd3d00f3ba5382dc06ab03d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12030
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
2024-07-23 14:47:03 +00:00
Florian Klink
636647ce03 fix(tvix/boot/tests): use grpc health check
Ensure the service is healthy before connecting to it, don't just check
for the unix socket to be present.

Change-Id: If6501828677c247910d91f35b860960802084691
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12029
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-07-23 14:47:03 +00:00
Florian Klink
ca5134c3da feat(3p/overlays/tvl): init grpc-health-check
Change-Id: Icc2429070b0daea9fd230f1f4bb97b9c2eaf24d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12027
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
2024-07-23 14:34:59 +00:00
Yureka
94a0e21e68 fix(tvix/directoryservice): clarify get_recursive not found
Change-Id: I47f21b4b7db9304eadb04094d41bf04d443fcc3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12025
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: yuka <yuka@yuka.dev>
2024-07-23 14:04:23 +00:00
Florian Klink
b9aa6456e2 docs(tvix/docs): add TODO for protobuf
Change-Id: Iacf4ef517885bdd6d6366446124863d9a138b12b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12024
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-07-23 13:24:42 +00:00
Ilan Joselevich
9f10a71ec5 feat(tvix/store): add redb PathInfoService
This provides a PathInfoService implementation using redb
(https://github.com/cberner/redb) as the underlying storage engine.

Both an in-memory variant, as well as a filesystem one is provided,
similar how it's done with the sled implementation.

Supersedes: https://cl.tvl.fyi/c/depot/+/11692
Change-Id: I744619c51bf2efd0fb63659b12a27cbe0b2fd6fc
Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11995
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-22 18:49:04 +00:00
Yureka
dbe698042d test(tvix/store): add xp-store-composition to feature powerset
Change-Id: Ibbd9a7585ec2f70c8f0f4d25ad858aa5ebc031dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12023
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-22 16:30:08 +00:00
Yureka
2547ff2cf6 feat(tvix/store): add grpc healthcheck service to daemon
Change-Id: Ib95fc9352a45d54f9a16c8841c7e8f7cbeeaee8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12019
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: yuka <yuka@yuka.dev>
2024-07-22 15:57:30 +00:00
Yureka
39c8b6dece fix(store): add toml dependency for feature xp-store-composition
Fixes 'use of undeclared crate or module ' with --features xp-store-composition

Change-Id: I44dce86e656094d180b91a00f385f685d21f3fbd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12021
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: yuka <yuka@yuka.dev>
2024-07-22 15:56:59 +00:00
Vincent Ambo
849e639c71 fix(tazjin/emacs): fix Go indentation offset in tree-sitter mode
Change-Id: Id72ddc6345ee3eb70b7a1d594fe6bcd60d8d0868
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12020
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-07-22 14:39:44 +00:00
Yureka
14a4b4cbc3 feat(tvix): add experimental-store-composition option
Change-Id: I61661fbb0e77ce3c00c2a467dfabdf3fc77d8575
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12011
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-22 13:36:08 +00:00
Yureka
67335c41b7 refactor(tvix): move service addrs into shared clap struct
Change-Id: I7cab29ecfa1823c2103b4c47b7d784bc31459d55
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12008
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: yuka <yuka@yuka.dev>
2024-07-22 13:36:08 +00:00
Yureka
6774d9c59c chore(tvix/tracing): switch tracing-opentelemetry from git to release
Change-Id: Ib830c8b642496a6cdf3d4b9093f0343f5fb80622
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12018
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-22 13:36:08 +00:00
Yureka
8342f70c60 feat(tvix/composition): improve error message for unknown tag
Change-Id: I61a31488de17725ae4311f5f5bf8e02edb752cf9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12017
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-22 13:36:08 +00:00
sinavir
5d82e1e62f fix(tvix/store): Fix narinfo compression selection
Parsing of the narinfo file sets the compression field to None instead
of Some("none"). The mapping selecting the decompression reader expected
the former in //tvix/store/src/pathinfoservice/nix_http.rs.

Change-Id: I254a825b88a4016aab087446bdc0c7b6286de40c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12007
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-21 22:11:20 +00:00
Florian Klink
a3194e9280 feat(tvix/nix-compat): add SigningKey, NARInfo::add_signature
This adds a generic `SigningKey` struct that can be used to sign
NARInfos with signers.

It also includes tooling to parse keypairs from bytes generated by Nix,
returning a specialized ed25519_dalek variant.

Change-Id: Ic9780c370939af54e7177c93cde3321adf189fc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12014
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 21:41:54 +00:00
Florian Klink
05b4e805ee refactor(tvix/nix-compat): rename PubKey to VerifyingKey
Align these with the way it's called in the ed25519 crates.

Change-Id: Ia52d3bb9bf831dc6b5f7d5356f5ac62135672883
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12013
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
2024-07-21 21:41:54 +00:00
Florian Klink
62184ee35a docs(tvix): document the builder API
This documents some thoughts and goals of the Tvix Build protocol, and
how it is possible to express Nix builds with it.

Additionally, it explains a proposed design for reference scanning.

Change-Id: I4b1f3feb2278e3c7ce06de831eb8eb1715cba1c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12012
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
2024-07-21 21:35:22 +00:00
Yureka
e408783bac test(tvix/composition): add recursion and concurrent tests
Change-Id: Ic10773b08d940b45c3067bd514c3080cadac9606
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12003
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 21:18:17 +00:00
Yureka
eea9f3d9ee docs(tvix/composition): add comment about stack
Change-Id: I3888b5034c53728e2f9cfe24213f8854eb38bfe1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12005
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 18:37:50 +00:00
Yureka
76394daca3 feat(tvix/composition): downcast boxed error if possible
We still have the unique store name to identify which instantiation caused the error. For recursion errors, the full chain is still retained inside the CompositionError.

Change-Id: Iaddcece445a5df331e578d7c69d710db3d5f8dcd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12002
Tested-by: BuildkiteCI
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
2024-07-21 17:03:27 +00:00
sinavir
1f89cf9027 feat(ops/users): add sinavir to users
Change-Id: I54c47f8119d38f7403e27cbc23efd919dcf8e8d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12006
Reviewed-by: yuka <yuka@yuka.dev>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 14:02:44 +00:00
Yureka
a805a80b48 fix(tvix/composition): include typeid in recursion check
Change-Id: Icc279d41a4980d4b57acbb4243bbd509039b753f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12000
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 13:20:00 +00:00
Florian Klink
ba38883a4f fix(tvix/nar-bridge): remove leftover debug statement
Change-Id: Ie6fb822e1a4b76f2f6aef15bbe846e0a89c644fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11999
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
2024-07-21 11:26:13 +00:00
Florian Klink
98f59cb828 feat(tvix/nar-bridge): change default port
Having `tvix-store daemon` and `nar-bridge` listen on the same port by
default is silly.
Use the same port that nar-bridge-go was using previously.

Change-Id: Id374a12e52ba9dac4318e43d0ca0853866eadabc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11998
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-07-21 11:26:13 +00:00
Yureka
ca8e2b9fbf chore(tvix): upgrade to tonic 0.12 / hyper 1.0
Change-Id: Idd8ce48869ddd869d51a10959b920f1290a8a9b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11991
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 11:01:38 +00:00
Simon Hauser
1515a970be feat(tvix/tracing): http propagation for axum
It introduces a new accept_trace function for axum0.7 which can be used
to accept a header trace from a received request. This function can be
used for tonic 0.12 once that version is released, and the specific
`accept_trace` function within `tvix_tracing::propagate::tonic` can then
be removed.

This also integrates http propagation into the nar_bridge crate.

Change-Id: I46dcc797d494bb3977c2633753e7060d88d29129
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11925
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
Reviewed-by: flokli <flokli@flokli.de>
2024-07-21 05:45:19 +00:00
Florian Klink
fdc0cf0c94 refactor(tvix/boot/tests): use nar-bridge-rs
This switches the boot tests from the golang implementation to the rust
one.

Change-Id: Ide2a47aebe40c172077147c05bb6dacd74ba6b1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11997
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
2024-07-21 00:06:23 +00:00
Florian Klink
d136f2c881 fix(tvix/nar-bridge): fix root node decoding and validation
This got broken while moving things around. We need to parse the
b64-decoded bytes.

Since we're now validating the root node, we also need to rename the
root node to get past the node name validation.

There probably should be some tests for this.

Co-Authored-By: sinavir@sinavir.fr
Change-Id: I8f24a4a0ac107b1ea5b94c0e0ed872a34eb7b587
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11996
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
2024-07-21 00:06:23 +00:00
Florian Klink
fca8462530 fix(tvix): fix outputHashes calculation for crates with versions
In case two crate versions are present, the key in the Cargo.nix file
includes the version number too.

To be able to set a specific hash for "tracing-opentelemetry 0.25.0" for
example, this needs to account for keys with the version included.

Access `crateName`, `version` and `src.outputHash` individually.

Change-Id: Ib9800691a445ac403ff646cb32e85a7a4cbef9d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11994
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
2024-07-20 22:17:15 +00:00
Yureka
8b77c7fcd7 refactor(tvix/store): use composition in tvix_store crate
Change-Id: Ie6290b296baba2b987f1a61c9bb4c78549ac11f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11983
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
2024-07-20 19:37:27 +00:00
Florian Klink
6180a7cecf feat(tvix/nar-bridge): implement PUT $outhash.narinfo
This adds support to upload NARInfo files. We lookup the root node from
the LRU cache, rename it appropriately and then put it into the
PathInfoService.

Change-Id: I5479032b51cd855363bc016dee63cf84b3304a36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11988
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
2024-07-20 17:38:58 +00:00
Florian Klink
5bd48de418 feat(tvix/nar-bridge): add 404 handler for GET/HEAD /nar/…
We currently send 405, and that confuses `nix copy`.
Send a 404 for now, and add a futurework, as we can actually at least do
something more meaningful in case we still have that nar hash in our
LRU, which would avoid some unnecessary uploads in some cases.

Change-Id: If625e9bd0fd6506cb73b88962d889aa08315fcea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11987
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
2024-07-20 17:38:58 +00:00
Florian Klink
5d906054da feat(tvix/nar-bridge): support uploading NAR files
This ingests NAR files into the {Blob,Directory}Service, which are
already part of the AppState.

As we then need to correlate the root node to the uploaded PathInfo, we
need to keep a (short-lived) lookup table from NARHash to root node
around. We insert it into a `LruCache` after the NAR is uploaded, and
use `peek()` to do the lookup, which doesn't update the LRU list.

Change-Id: I48a4c6246bacf76559c5a4ccad2a0bc25c1b7900
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11986
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
2024-07-20 17:23:18 +00:00