This adds a function which can be used across the monorepo to create a
an extra CI step that checks whether the Cargo.nix file is up-to-date.
Change-Id: Idb8298b29ddc2ca5dff1facb1b9ed86a236ee66d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12227
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This adds almost a gigabyte of closure size for formatting files that almost
never change.
I'll look into building just the formatter somehow, but it's not a very high
priority task.
Change-Id: Ib0f841e1a98133381c5ae154e2a57df8af52dc1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12293
Tested-by: BuildkiteCI
Reviewed-by: emery <emery@dmz.rs>
Autosubmit: tazjin <tazjin@tvl.su>
This prevents browsers from treating NARInfo and nix-cache-info paths as
a separate "Download", but just show it in plaintext.
Change-Id: If99abe20ef1d24e4fa86c055160861ca47aa81ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12267
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This adds the MIME types Nix uses when uploading files to a S3 bucket /
HTTP endpoint. It ignores them when retrieving.
Change-Id: I75fa96d5e53349f0470bbe8cbba93f961fbd5f05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12266
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
We're moving away from sourcegraph to livegrep and so on, as Sourcegraph has
gone fully proprietary.
This removes support for redirecting to Sourcegraph.
Relates to b/290
Change-Id: I04ccf8dfef72113cd49d444151cb0c3eb834845d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12268
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Makes it possible to construct persistent German Strings from data with 'static
lifetime. This doesn't leak any additional data.
Change-Id: Ifb3a36ff72b106724883302e4c2195ac54acd352
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12263
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Adds a StorageClassPtr type that is set up to be able to steal bits from an
aligned pointer to encode the storage class of a German String.
Change-Id: I64591174eac1ebcb73e624a59bd107ba1e02c69d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12262
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Fix a discrepancy in the error message for DirectoryError::SizeOverflow.
The message indicates that the SizeOverflow error occurs when total size
exceeds u32::MAX, but that's not true. All size fields within the
castore's internal Directory ADT are u64, and the SizeOverflow error is
only returned after a call to the checked_add implementation on u64.
See tvix/castore/nodes/directory.rs +111
and tvix/castore/nodes/directory.rs +88
as of this commit.
Change-Id: I74d161ea8927362e1cb601ba163489aa96fb91b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12259
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This moves the URL component parsing code we had in nar-bridge to
nix-compat.
We change the function signature to return an Option, not a
Result<_, StatusCode>.
This allows returning more appropriate error codes, as we can
ok_or(…) at the callsite, which we now do: on an upload to an
invalid path, we now return "unauthorized", while on a GET/HEAD, we
return "not found".
This also adds support to parse compression suffixes. While not
supported in nar-bridge, other users of nix-compat might very well want
to parse these paths.
Also fix the error message when parsing NAR urls, it mentioned 32, not
52, which is a copypasta error from the narinfo URL parsing code.
Change-Id: Id1be9a8044814b54ce68b125c52dfe933c9c4f74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12260
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Similar to how cl/12253 already did this for `Signature`, we apply the
same logic to `StorePath`.
`StorePathRef<'a>'` is now a `StorePath<&'a str>`, and there's less
redundant code for the two different implementation.
`.as_ref()` returns a `StorePathRef<'_>`, `.to_owned()` gives a
`StorePath<String>` (for now).
I briefly thought about only publicly exporting `StorePath<String>`
as `StorePath`, but the diff is not too large and this will make it
easier to gradually introduce more flexibility in which store paths to
accept.
Also, remove some silliness in `StorePath::from_absolute_path_full`,
which now doesn't allocate anymore.
Change-Id: Ife8843857a1a0a3a99177ca997649fd45b8198e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12258
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Don't call functions bstr or str when they return BString or String,
it's confusing.
Rename them to `string` and `bytes`. We might be able to generalize over
this being BString or Vec<u8> later.
Change-Id: I8198551ed3ba1cfc479bf7e3cbbc13a426faf4c0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12257
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This allows importing just //tools/depotfmt.nix when using a josh
workspace.
Change-Id: Idf7eb438be1cc1e60445aace982fdca1f4367595
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12254
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
This implements a simple DFS locator in listing structures.
It is interoperable with the Rust standard library paths, we also build our
own errors to restrict path values to reasonable secure defaults, e.g.
relative paths with no `..` component.
Tests are added for this new feature for a positive and a negative
check.
In addition, a path validation test was added. The Windows-style prefix
is gated on the Windows platform as UNIX does not parse `C:\\` as a
`Component::Prefix(_)` but as a `Component::Normal(_)`.
Change-Id: Iae2a80bebd8138e41af94aa7d09f2842c3c5a786
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12255
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Requiring `name` to be a `&str` means it'll get annoying to pass around
`Signature`, but being able to pass them around in an owned fashion is
kinda a requirement for a stronger typed `PathInfo` struct, where we
want to have full ownership.
Rework the `Signature` struct to become generic over the type of the
`name` field. This means, it becomes possible to have owned versions
of it.
We don't want to impose `String` or `SmolStr` for example, but want to
leave it up to the nix-compat user to decide.
Provide a type alias for the existing `&str` variant (`SignatureRef`),
and use it where we previously used the non-generic `Signature` one.
Add some tests to ensure it's possible to *use* `Signature` with both
`String` and `SmolStr` (but only pull in `smol_str` as dev dependency
for the tests).
Also, add some more docstrings, these were a bit sparse.
Change-Id: I3f75691498c6bda9cd072d2d9dac83c4f6c57287
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12253
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
.ls files are useful to seek in a NAR without parsing it entirely.
The responsibility of validating the files is on the caller.
Change-Id: I5d1da28b5479c38f20ca5babe60e362a2217c9ea
Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12196
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We can avoid depending on things outside //tvix by just using a similar
util from nixpkgs.
Change-Id: I9ea3e1f0a8a059ea10caaec173569ba9f316aec6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12247
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Using pkgs.nixos directly allows us to create a smaller nixos closure
for the tests and also not depend on things in depot.ops which can be
beneficial for extending the tvix josh workspace.
Change-Id: Ic6ad2122733418114b43aa692d6e42ac1e308eeb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12251
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
It's a `[u8; SIGNATURE_LENGTH]` type alias, and conveys what we're
accepting or returning a bit nicer.
Change-Id: I974cd97d56d383e51417eb0f26e1431a05711922
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12252
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Makes it possible to construct a GermanString from an owned byte vector, without
having to clone the data.
This is done by "disowning" the vector using ManuallyDrop to access its internal
pointer. For transient strings, this memory is then owned (and freed) by the
GermanString instance.
Small strings are copied out of the heap and stored inline as before, to avoid
any dereferencing operations.
Change-Id: I754736099f71d646d430aed73e558a5a7626c394
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12249
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This can short-circuit two large string comparisons.
Change-Id: If45e7cf33921fe571482dc710c27ef8cc7c70885
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12245
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is where one of the advantages of this string representation starts to
shine: For small strings there's no derefencing any heap memory at all, and for
the long representation we can compare the prefix before moving on.
Change-Id: Iac333a52e8d7c9dd09e33dbcf51754e321c880e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12238
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
//tvix depending less on other parts of depot is prefered as it will
help with extending the josh workspace of tvix.
Change-Id: Ifcac3af1782dfd82e7543cb4c3ae57fbd186edff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12250
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
It's easier to implement readTree/depot polyfills for gitignoreSource
when it's imported from third_party.sources, rather than in a file at
//third_party.gitignoreSource.
Change-Id: I1323f932bd0feeb2c50ccc76397a80e035842992
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12248
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
All of these strings are currently transient (the storage class is not yet
represented anywhere), and the ones that are heap allocated need to be
deallocated when the transient string dies.
Change-Id: Iba0ca926df5db7594f304c5d5318db9d69c6f42c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12235
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This adds an initial implementation of the so-called "German Strings" in Rust.
https://cedardb.com/blog/german_strings/
This implementation is *far from* complete, the only thing that can be done
right now is construct a string, and even that I'm not fully happy with.
Change-Id: I2697932a0ef373be76ffd14d59677493a5783b58
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12234
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Removes imbl::OrdMap in favour of an Rc over the standard library's BTreeMap,
which allows us to drop the imbl dependency completely.
In my local tests this is actually slightly faster for `hello` and `firefox`.
Change-Id: Ic9597ead4e98bf9530f290c6a94a3c5c3efd0acc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12201
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
