This now exists in tvix-store directly, as NixHTTPPathInfoService, and
contrary to this version, also validates signatures.
Change-Id: Ib6ca161e40d627b7d9741839fc849f2392f422da
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10155
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This allows setitng the trusted-public-keys URL parameter to a
(whitespace-separated) list of public keys.
NARInfo files retrieved need to contain a valid signature.
Change-Id: Ifd6580b723cbae3182e9cadfa54f1ca2b41d6599
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10153
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Introduce an Option<Vec<narinfo::PubKey>>, configurable with a
`set_public_keys` method.
If set, this configures NixHTTPPathInfoService to validate signatures.
Change-Id: I157c5e13c41fc9bfd40b0655381fb4cf33900868
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10152
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This makes it easy for each PubKey to check if a given Signature is
correct for a given fingerprint.
Change-Id: I56e6211d133f74f390fd1ae3ae799eef12221904
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10151
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This represents a ed25519 public key and "name".
These are normally passed in the `trusted-public-keys` Nix config option,
and consist of a name and base64-encoded ed25519 pubkey, separated by a `:`.
Change-Id: I9ab4b3e0e5821805ea6faf2499626630fc5a3f0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10150
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Just call this Error, we can infer from the package what error this is.
Change-Id: I5df25d2873ec739c49c08804f35562c84c222e06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10149
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Ensure the initially communicated NarHash/NarSize from the NarInfo
matches what we read, and don't return a PathInfo message if there's
a mismatch.
Also move the buffering layer around a bit.
Change-Id: I68c60ecfaf0f9cd5edacea648437ecb0c9729251
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10148
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
An error in the PathInfoService request can appear in case the
underlying request returns an error.
We shouldn't panic and bork the fuse mount, but instead return an IO
error.
Change-Id: I2daeae629e1627d06adcd7b82ddb76c50c602212
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10154
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
* picked avrdude from stable channel
* removed override for texlive, as the upstream fix is merged
* picked awscli2 from stable channel
* bump tdlib to 1.8.21 (new minimum for telega.el)
* tvix/turbofetch: switch to nixpkgs-native mechanism for
CARGO_MANIFEST_LINKS (whatever that is)
Change-Id: Ic695721b5ca750b89d21cab7a257e1db682b23c0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10083
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
We should restrict this to alphanumeric mostly, and we definitely don't
want newlines.
Not entirely sure about the exact additionally allowed characters
outside of alphanumeric, but this can always be extended further.
Change-Id: I1357e79e553f2df2fa97792889f63f0f35d50ed5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10147
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
BASE64.decode_mut panics if we're passing data that has the wrong size.
Do the size check first and error out there.
Also update the error, and talk about b64-encoded sizes.
Change-Id: I290f80a37d48526a30bf1df9d1d9fe34865008eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10146
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
No need to hardcode magic numbers here, we have a constant for that.
Change-Id: I67b671c0c4bb7c3bfb001e9c36499f31873ee717
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10145
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This fixes a future clippy lint.
Change-Id: Ic830e94ef23595580c1037f10878c76bbb546dd9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10110
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
This commit adds Thunk::unwrap_or_clone(), which uses
Rc::try_unwrap() to avoid cloning the Value out of a an Rc which has
only one strong reference.
Change-Id: Icacefe0c823dcddf046d90c0c5cd5ed59fe976d4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10037
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Change-Id: Ibddaa111a5b7a86c42dbe153ae8e53f9a5601a54
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10112
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
This allows seeing a PathInfo as a nix_compat::narinfo::NarInfo<'_>.
It doesn't allocate any new data, but the NarInfo<'_> view allows us to
access things like signature verification, or rendering out
(alternations of this) as strings.
Change-Id: Id0d8d7feeb626ee02c3d8a4932f24ace77022619
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10108
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is useful when creating a new Signature struct where the individual
elements are already parsed.
Change-Id: Ie33c66287641951e7a030aaa1e7ff0a86b2628ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10111
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We can simply use .to_owned() on that thing afterwards if we want to
construct an owned StorePath.
Change-Id: I0f3e2e4434b99ee522f2a7dbfa391e13a987479c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10105
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
We always know this needs to end with a .drv, and fail parsing if it
doesn't, so there's no need to hang onto these 4 bytes.
This will make it much easier to synthesize a NarInfo<'_> later on from
a PathInfo proto, because we don't have to make this ".drv" appear out
of thin air.
Change-Id: Id95e7fd937d7c9a420a39b5a4bab73985640ca3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10084
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
These were used to format to and parse from strings.
Move this to the CAHash and NixHash structs directly, and be explicit in
the name about which encoding for digests is used.
For output path calculation, nix encodes the nixpaths in hex, but for
writing out NARInfos, it's using nixbase32.
Change-Id: Ia585a76a3811b2609e7ce259fda66a29403b7e07
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10079
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This adds support to compute the fingerprint string, which is what's
ed25519-signed in binary caches.
Change-Id: I8947239c609896acfd7261f110450014bedf465a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10080
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This adds support for verifying signatures found in NARInfo files.
This still needs to be hooked together with the nix+http[s] backend.
Change-Id: Ic1c8ddbdecfb05cefca2492808388b0f7f3f2637
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10081
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
NixHTTPPathInfoService acts as a bridge in between the Nix HTTP Binary
cache protocol provided by Nix binary caches such as cache.nixos.org,
and the Tvix Store Model.
It implements the [PathInfoService] trait in an interesting way: Every
[PathInfoService::get] fetches the .narinfo and referred NAR file,
inserting components into a [BlobService] and [DirectoryService], then
returning a [PathInfo] struct with the root.
Due to this being quite a costly operation, clients are expected to
layer this service with store composition, so they're only ingested
once.
The client is expected to be (indirectly) using the same [BlobService]
and [DirectoryService], so able to fetch referred Directories and Blobs.
[PathInfoService::put] and [PathInfoService::nar] are not implemented
and return an error if called.
This behaves very similar to the nar-bridge-pathinfo code in nar-bridge,
except it's now in Rust.
Change-Id: Ia03d4fed9d0657965d100299af97cd917a03f2f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10069
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
There's no need to already require this to be buffered here.
Change-Id: Ib9a11b194e0754d87ab8d2ef0b8cb0f4edc01229
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10074
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Allow taking advantage of the buffer of the underlying reader to avoid
unnecessary copies of file data.
We can't easily implement the methods of BufRead directly, since we
have some extra I/O to perform in the final consume() invocation.
That could be resolved at the cost of additional bookkeeping, but this
will suffice for now.
Change-Id: I8100cf0abd79e7469670b8596bd989be5db44a91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10089
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We rely on being able to make small reads cheaply, so this was already
an implicit practical requirement. Requiring it explicitly removes a
performance footgun, and makes further optimisations possible.
Change-Id: I7f65880a41b1d6b5e6bf2e52dfe47d4c49b34bcd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10088
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We don't need to validate UTF-8 separately, since valid names are
a strict subset of ASCII, and therefore a strict subset of UTF-8.
Change-Id: I3261bf0efe3480b5b315074efafcf5e47a6c5a65
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10087
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
We can just use take(size) to restrict reading to that as a max.
Change-Id: I0fbda74e4fb98ffeababae86a325233416029acf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10072
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This allows converting from the NarInfo falling out of the NarInfo
parser (which is a bit annoying to handle due to lifetimes) to the
PathInfo proto struct.
The narinfo field, containing most of the data from the original
NARInfo file, as well as the references (bytes) are populated.
The node field is not populated, because it requires ingesting the NAR
itself to describe the root node.
Change-Id: I9c04dd6ad4cae556b455188a4255e34b4f6443c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10067
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This suggests it's cheap to convert around, but name actually does
allocate.
Move to a `to_owned(&self) -> StorePath`, to better signal that this
does allocate.
Change-Id: Ifaf7c21599e2a467d06e2b4ae1364228370275db
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10066
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Make it less annoying to convert from io::Error to this. We already have
one direction, doesn't hurt to have the other too.
Change-Id: I9fe2c6da608c9d54910ee8c397572aadb1d90d99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10068
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Having random if blocks and returning from them is error-prone.
Also, turns out we only need the unprefixed scheme in the fallback case,
so move it down to there.
Change-Id: Ifcb09279c963f8a39e0dbabe145990263f3d7cf9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10041
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
This has been renamed to descend_to in cl/9373.
Change-Id: Ia6201fb81c7d4fa953d311451cfff95373549a50
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10045
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
This wasn't used at all, let's remove it.
Change-Id: I426e3d93c32ebe65247ae5cf8d05b5bf686be2d6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10044
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
This moves the sync `channel::from_url` to a async
`tonic::channel_from_url`. It now allows connecting non-lazily if `wait-
connect=1` is set in the URL params.
Also, make the pingpong tests for blobsvc and directorysvc use the wait-
connect=1 codepath.
Change-Id: Ibeea33117c8121814627e7f6aba0e943ae2e92ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10030
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Make directoryservice::from_addr use the more specific constructors.
Change-Id: I9fee2afed77692505988d631d9fe246d9843d25a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10029
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Make blobservice::from_addr use the more specific constructors.
Change-Id: Id9637e279d6910ce6d92ff0086a984be5c65a8c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10028
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
All we do is constructing some strings, and checking if from_addr
succeeds or not.
This can be written in a much more concise way using test_case.
Use lazy_static to provide temporary directories.
Also add some more grpc-related test cases.
Change-Id: Ia310dd01f617f7628f1e7e21304ac70da2ab3534
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10027
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
