We're using this in the NAR reader now.
Change-Id: I28f17b1ccedd62ffcaf2fa32b517f16bcd036d94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11603
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We specifically structured the code this way to allow using
`this.filled` as-is, so we should use it.
Change-Id: I7e11bddceb4d5f37b1dd4c453a9d53b85fc1f6c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11602
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This is a first cut at the async NAR reader, with some rough edges.
Poisoning is left unimplemented for now, pending future work.
Change-Id: Ifaafe0581a5e0e165a13357b909fb441f7bd8bab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11524
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This allows using BytesReader with a custom tag, eg the closing parens
for the NAR reader.
No public constructor is provided for custom-tagged readers, since this
feature isn't public API.
Change-Id: I82e73d064edc4b6783ead1d6fe46a5b35f45c844
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11543
Reviewed-by: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This documents which Nix version uses which daemon protocol version so that
any implementor of the protocol can make informed choices about how far back to
support legacy.
Change-Id: If2fc11702f7f731816d51928efc9ea9d939e1c00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11579
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
There's no need for us to come up with our own names for this.
Also update the comments/docstrings a bit, and inline the intermediate
hash_derivation_modulo calculation.
Change-Id: I09dab8ffe1ebfb6601841e98119eee4ff25d8f39
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11578
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Nix 2.4 accidentally permitted this behaviour, but the revert came
too late to beat Hyrum's law. It is now considered permissible.
Link: https://github.com/NixOS/nix/pull/9867
Change-Id: Ie97777af6765fe1c12c8aa593afe1c9b69125775
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11553
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
u64 is an inappropriate type for something memory-sized, and most
callers end up with off-by-ones when using `..` rather than `..=`,
including the tests for the module itself.
Change-Id: If3b7bea27eb0a6c01e0a5d7e64966acbbb664268
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11550
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
These may as well be inlined, and hardly need tests, since they just
alias AsyncReadExt::read_u64_le / AsyncWriteExt::write_u64_le.
Boolean reading is worth making explicit, since callers may differ on
how they want to handle values other than 0 and 1.
Boolean writing simplifies to `.write_u64_le(x as u64)`, which is also
fine to inline.
Change-Id: Ief9722fe886688693feb924ff0306b5bc68dd7a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11549
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
with_size only existed to allow separating the phases of reading size
and reading data, and similarly the Size state only existed to allow
folding size reading into poll_read.
Neither of these are necessary if we make the constructor async,
and handle the size reading there.
As a bonus, this makes BytesReader::len sensibly typed, and permits
implementing is_empty, as Clippy demands of us.
Change-Id: I72173ec06d60b7998d16a3ecfc5e6ac5424bbed3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11542
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We don't actually hold a Tag, we just want to bind the type.
Change-Id: Ida67c026f852ed54c3f18df914cf5c31e6227fd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11541
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The (min, max) pair is already a RangeInclusive in essence, so we might
as well represent it that way.
Change-Id: I2f67f3c47dc36b87e866ff5dc2e0cd28f01fbb04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11540
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The TrailerReader has no purpose separate from BytesReader, and the
code gets a fair bit simpler this way.
EOF handling is simplified, since we just rely on the implicit
behaviour of the existing case.
Change-Id: Id9b9f022c7c89fbc47968a96032fc43553af8290
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11539
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The API is a bit odd here, because we don't have a distinct type for a
known-length reader.
Change-Id: I4a1dd07fbed0a400004dbe4aa2095c51898ad3bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11538
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
The poisoning API is now always available, whether debug_assertions is
enabled or not. When debug assertions are not enabled, it is equivalent
to a unit struct, and is always considered ready and unpoisoned.
Change-Id: I950237f4816d480330d9acab32661ed4f1663049
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11525
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We separate ingesting the trailer block into a Future of its own,
parametrised on the specifics of the trailer pattern.
This is intended to be used for future work on an async NAR reader,
which needs to read a terminating parenthesis as well as the regular
padding.
Thanks to @griff for suggesting separating the ingestion into its own
Future.
Change-Id: I36c2503baa67937046a63e9bf0cfc38201394025
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11522
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Pin<&mut T> is DerefMut when T: Unpin, so we don't actually need to
explicitly call get_mut.
Change-Id: Iaa312ec49c87100010e09c94f319e57e31da0cd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11520
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We don't use it in the reader anymore.
Change-Id: I98fe204a747711464e9e7ca17df06fa9854eb344
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11519
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We now *never* return the final bytes until we've read the padding
in full, so read_exact is safe to use.
This is implemented by TrailerReader, which splits the phases of
reading (and validating) the final 8-byte block, and providing
the contained payload bytes to the caller.
Change-Id: I0d05946a8af9c260a18d71d2b763ba7a68b3c27f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11518
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We already require R: Unpin in the constructor, so there's not much use
to pin projection.
Change-Id: Ia7bf734dc3aa86ffa6d1d5de778939baa9676bb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11516
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This is semantically a RangeInclusive, since we can only have
0..=u64::MAX at most, and monomorphising on the bounds doesn't
buy us anything.
Change-Id: Ib601d7fd77d703d6c8c5ec27ac9e67bb122ce1c0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11515
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Adds debug assertions to ensure that the reader's variants are upheld.
If any of the following happens, then the currently in use reader must
be abandoned:
* A directory or file reader encounters an error
* A directory or file reader is dropped before being fully read from
Additionally, a directory reader must not be read from again after it
has returned None.
These checks are only used when debug_assertions are on, so vanish in
release mode.
Resolves two TODO items added by edef
Change-Id: I27bd9643a632798db5351957506c166b9bd5ca4e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11508
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Autosubmit: Aria Shrimpton <me@aria.rip>
Tested-by: BuildkiteCI
This is a ReadBuf, not a BufRead.
Change-Id: Ie80e894f4b24b77cdd60409ddfaa66dae0ffeec9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11511
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
This is especially nice, as it allows us getting rid of the other
versions of proc-macro2 and quote.
Change-Id: I9fdd012ee6c0ded3e18ec30504b83ac2032d1390
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11474
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
There's no point in not enabling these, and rust-analyzer is quite
useless too.
nix-compat users outside of here can still explicitly disable default
features if they don't want to pull in these features.
Change-Id: I8f2f24e7734ad152d1733bc3421098fc91af290a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11455
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
While the compiler might realize it's a good idea to inline these,
especially considering this is in the same crate [1], it doesn't hurt to
add that annotation too.
Suggested in cl/11385.
[1]: https://nnethercote.github.io/perf-book/inlining.html
Change-Id: If679fce0e1aab9ec681ba60f689d386d6fd92d61
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11403
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
There's little reason to keep the nar writer using Async{Read,Write}
traits from futures, while everything else async in tvix (and
nix-compat) uses tokio.
Change-Id: I8cd1efcd0dd5bb76471de997603c7b701a5095de
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11391
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Brian Olsen <me@griff.name>
This doesn't have much to do with the plain "wire" format, it's merely
one user of it.
Also, use the more "public" `wire::` API to read/write bytes, strings,
bools and u64s.
Change-Id: I98dddcc3004dfde7a0c009958fe84a840f77b188
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11390
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Brian Olsen <me@griff.name>
Move everything bytes-related into its own module, and re-export
both bytes and primitive in a flat space from wire/mod.rs.
Expose this if a `wire` feature flag is set. We only have `async` stuff
in here.
Change-Id: Ia4ce4791f13a5759901cc9d6ce6bd6bbcca587c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11389
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
We don't have it in all Cargo.toml files either.
Change-Id: Ifcb333a39603a7e402ab0068e0f0f1b5af769e13
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11388
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Autosubmit: flokli <flokli@flokli.de>
This is another representation about information Nix has on a Store
Path. It's encountered in the exportReferencesGraph feature.
Change-Id: Ia0c08c369f4af9e50ebe3a5053bd98fadaa0003d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11396
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
We use the (slightly more tolerant) from_str to deserialize, and
serialize out as SRI.
Change-Id: If76b0ed2d4e243904f02df34f6c90b976c0bab8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11393
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Move the code implementing it from StorePath to StorePathRef, and have
the StorePath impls use that too.
Drop the debug_assert in every comparison - we have tests for this to
ensure it keeps working, and built up some confidence by piping a lot of
other store paths through it in the meantime.
Change-Id: I288bad3dfa597f68d63c4bcda7791f722b7a8ced
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11392
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
