Commit graph

14 commits

Author SHA1 Message Date
Vincent Ambo
cb032b250e chore(tvix/tools): move narinfo2parquet to //users/edef
This is not a core Tvix tool, it's a tool that uses a Tvix component.

Change-Id: I81d2b2374da23489df0097dcabb8295c82652fc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12606
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2024-10-12 23:17:29 +00:00
Brian Olsen
9af6920478 feat(nix-compat): Add NixDeserialize and NixRead traits
Add a trait for deserializing a type from a daemon worker connection.
This adds the NixDeserialize trait which is kind of like the serde
Deserialize trait in that individual types are meant to implement it
and it can potentially be derived in the future.

The NixDeserialize trait takes something that implements NixRead as
input so that you can among other things mock the reader.

Change-Id: Ibb59e3562dfc822652f7d18039f00a1c0d422997
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11990
Autosubmit: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-08-25 15:05:50 +00:00
Ilan Joselevich
424b1ab5ad feat(tvix/tools/narinfo2parquet): Add crate2nix-check
Change-Id: I22bc6e7711e5e2c0a76d4eea4dd6edaf724e4cef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12301
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 19:55:10 +00:00
Florian Klink
e03ea11bad feat(nix-compat/nix_http): init parse_nar[info]_str
This moves the URL component parsing code we had in nar-bridge to
nix-compat.

We change the function signature to return an Option, not a
Result<_, StatusCode>.

This allows returning more appropriate error codes, as we can
ok_or(…) at the callsite, which we now do: on an upload to an
invalid path, we now return "unauthorized", while on a GET/HEAD, we
return "not found".

This also adds support to parse compression suffixes. While not
supported in nar-bridge, other users of nix-compat might very well want
to parse these paths.

Also fix the error message when parsing NAR urls, it mentioned 32, not
52, which is a copypasta error from the narinfo URL parsing code.

Change-Id: Id1be9a8044814b54ce68b125c52dfe933c9c4f74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12260
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-08-21 09:40:32 +00:00
Florian Klink
a259613c76 feat(nix-compat/narinfo/signature): generalize name field
Requiring `name` to be a `&str` means it'll get annoying to pass around
`Signature`, but being able to pass them around in an owned fashion is
kinda a requirement for a stronger typed `PathInfo` struct, where we
want to have full ownership.

Rework the `Signature` struct to become generic over the type of the
`name` field. This means, it becomes possible to have owned versions
of it.

We don't want to impose `String` or `SmolStr` for example, but want to
leave it up to the nix-compat user to decide.

Provide a type alias for the existing `&str` variant (`SignatureRef`),
and use it where we previously used the non-generic `Signature` one.

Add some tests to ensure it's possible to *use* `Signature` with both
`String` and `SmolStr` (but only pull in `smol_str` as dev dependency
for the tests).

Also, add some more docstrings, these were a bit sparse.

Change-Id: I3f75691498c6bda9cd072d2d9dac83c4f6c57287
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12253
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-08-19 19:46:34 +00:00
Florian Klink
fa05f12d5d chore(tvix/tools/narinfo2parquet): bump curve25519-dalek
Fix for RUSTSEC-2024-0344.

Change-Id: Iad02bcf312af8b6bf450ab3c4c6de31d9c02ab87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12212
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-08-16 06:52:15 +00:00
Florian Klink
ad92083dea chore(tvix/tools/narinfo2parquet): update Cargo.{lock,nix}
These were out of date.

Change-Id: I89df37f088ad6c53b676d965f0dc1023f40481f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12208
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-15 21:10:44 +00:00
edef
f3c80c10b9 feat(tvix/tools/narinfo2parquet): preserve the deriver
Change-Id: Idbada585b87eef81cad5d40cb7592a7890704695
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12037
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-07-27 21:31:18 +00:00
Florian Klink
c8f92c6e9f refactor(tvix/nix-compat): introduce CaHash::algo_str
This is the `{fixed,fixed:r,text}:{sha*,md5}` prefix used in various
string representations.

Factor that code out, and use it in the two places it can be used in.

Change-Id: Ic9555fa9e1884198d435e55c7f630b8d3ba2a032
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12041
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
2024-07-27 10:53:56 +00:00
Ilan Joselevich
f52b142035 chore(tvix/tools/narinfo2parquet): Regenerate Cargo.nix and filter src
The Cargo.lock also got changed because of crate2nix using --freezed.

Change-Id: Id5225e73ec5894bc324f363660472a7efdd803cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11967
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-09 16:47:56 +00:00
Florian Klink
bdf4df1ac7 chore(tvix/narinfo2parquet): bump dependencies
Get rid of the RUSTSEC warnings.

Also upgrade tokio to drop the Sized requirement, similar to cl/11608.

Change-Id: Idc6fece23c79eb30fd1dfc5fe64fa2e4c08cd412
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11624
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2024-05-11 14:18:30 +00:00
edef
f4963cdfd8 chore(tvix/tools/narinfo2parquet): polars 0.34.2 -> 0.36.2
Change-Id: Ib17a7bbe560000c3fef3ba6182819f710d707909
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10967
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-02-19 16:03:14 +00:00
Florian Klink
bb18556bf3 feat(tvix/nix-compat): add narinfo::Signature::verify
This adds support for verifying signatures found in NARInfo files.

This still needs to be hooked together with the nix+http[s] backend.

Change-Id: Ic1c8ddbdecfb05cefca2492808388b0f7f3f2637
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10081
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-11-19 22:06:07 +00:00
edef
8d02928b14 feat(tvix/tools/narinfo2parquet): init
Convert turbofetch output to queryable Parquet.

Change-Id: I076f5a431f8aab8cfe7d973bdc9fe019cebde111
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9989
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-11-10 19:35:04 +00:00