u64 is an inappropriate type for something memory-sized, and most
callers end up with off-by-ones when using `..` rather than `..=`,
including the tests for the module itself.
Change-Id: If3b7bea27eb0a6c01e0a5d7e64966acbbb664268
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11550
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
These may as well be inlined, and hardly need tests, since they just
alias AsyncReadExt::read_u64_le / AsyncWriteExt::write_u64_le.
Boolean reading is worth making explicit, since callers may differ on
how they want to handle values other than 0 and 1.
Boolean writing simplifies to `.write_u64_le(x as u64)`, which is also
fine to inline.
Change-Id: Ief9722fe886688693feb924ff0306b5bc68dd7a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11549
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
with_size only existed to allow separating the phases of reading size
and reading data, and similarly the Size state only existed to allow
folding size reading into poll_read.
Neither of these are necessary if we make the constructor async,
and handle the size reading there.
As a bonus, this makes BytesReader::len sensibly typed, and permits
implementing is_empty, as Clippy demands of us.
Change-Id: I72173ec06d60b7998d16a3ecfc5e6ac5424bbed3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11542
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We don't actually hold a Tag, we just want to bind the type.
Change-Id: Ida67c026f852ed54c3f18df914cf5c31e6227fd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11541
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The (min, max) pair is already a RangeInclusive in essence, so we might
as well represent it that way.
Change-Id: I2f67f3c47dc36b87e866ff5dc2e0cd28f01fbb04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11540
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The TrailerReader has no purpose separate from BytesReader, and the
code gets a fair bit simpler this way.
EOF handling is simplified, since we just rely on the implicit
behaviour of the existing case.
Change-Id: Id9b9f022c7c89fbc47968a96032fc43553af8290
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11539
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The API is a bit odd here, because we don't have a distinct type for a
known-length reader.
Change-Id: I4a1dd07fbed0a400004dbe4aa2095c51898ad3bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11538
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
The poisoning API is now always available, whether debug_assertions is
enabled or not. When debug assertions are not enabled, it is equivalent
to a unit struct, and is always considered ready and unpoisoned.
Change-Id: I950237f4816d480330d9acab32661ed4f1663049
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11525
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We separate ingesting the trailer block into a Future of its own,
parametrised on the specifics of the trailer pattern.
This is intended to be used for future work on an async NAR reader,
which needs to read a terminating parenthesis as well as the regular
padding.
Thanks to @griff for suggesting separating the ingestion into its own
Future.
Change-Id: I36c2503baa67937046a63e9bf0cfc38201394025
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11522
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Pin<&mut T> is DerefMut when T: Unpin, so we don't actually need to
explicitly call get_mut.
Change-Id: Iaa312ec49c87100010e09c94f319e57e31da0cd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11520
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We don't use it in the reader anymore.
Change-Id: I98fe204a747711464e9e7ca17df06fa9854eb344
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11519
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We now *never* return the final bytes until we've read the padding
in full, so read_exact is safe to use.
This is implemented by TrailerReader, which splits the phases of
reading (and validating) the final 8-byte block, and providing
the contained payload bytes to the caller.
Change-Id: I0d05946a8af9c260a18d71d2b763ba7a68b3c27f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11518
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We already require R: Unpin in the constructor, so there's not much use
to pin projection.
Change-Id: Ia7bf734dc3aa86ffa6d1d5de778939baa9676bb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11516
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This is semantically a RangeInclusive, since we can only have
0..=u64::MAX at most, and monomorphising on the bounds doesn't
buy us anything.
Change-Id: Ib601d7fd77d703d6c8c5ec27ac9e67bb122ce1c0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11515
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Adds debug assertions to ensure that the reader's variants are upheld.
If any of the following happens, then the currently in use reader must
be abandoned:
* A directory or file reader encounters an error
* A directory or file reader is dropped before being fully read from
Additionally, a directory reader must not be read from again after it
has returned None.
These checks are only used when debug_assertions are on, so vanish in
release mode.
Resolves two TODO items added by edef
Change-Id: I27bd9643a632798db5351957506c166b9bd5ca4e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11508
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Autosubmit: Aria Shrimpton <me@aria.rip>
Tested-by: BuildkiteCI
This is a ReadBuf, not a BufRead.
Change-Id: Ie80e894f4b24b77cdd60409ddfaa66dae0ffeec9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11511
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
This is especially nice, as it allows us getting rid of the other
versions of proc-macro2 and quote.
Change-Id: I9fdd012ee6c0ded3e18ec30504b83ac2032d1390
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11474
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
There's no point in not enabling these, and rust-analyzer is quite
useless too.
nix-compat users outside of here can still explicitly disable default
features if they don't want to pull in these features.
Change-Id: I8f2f24e7734ad152d1733bc3421098fc91af290a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11455
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
While the compiler might realize it's a good idea to inline these,
especially considering this is in the same crate [1], it doesn't hurt to
add that annotation too.
Suggested in cl/11385.
[1]: https://nnethercote.github.io/perf-book/inlining.html
Change-Id: If679fce0e1aab9ec681ba60f689d386d6fd92d61
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11403
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
There's little reason to keep the nar writer using Async{Read,Write}
traits from futures, while everything else async in tvix (and
nix-compat) uses tokio.
Change-Id: I8cd1efcd0dd5bb76471de997603c7b701a5095de
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11391
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Brian Olsen <me@griff.name>
This doesn't have much to do with the plain "wire" format, it's merely
one user of it.
Also, use the more "public" `wire::` API to read/write bytes, strings,
bools and u64s.
Change-Id: I98dddcc3004dfde7a0c009958fe84a840f77b188
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11390
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Brian Olsen <me@griff.name>
Move everything bytes-related into its own module, and re-export
both bytes and primitive in a flat space from wire/mod.rs.
Expose this if a `wire` feature flag is set. We only have `async` stuff
in here.
Change-Id: Ia4ce4791f13a5759901cc9d6ce6bd6bbcca587c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11389
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
We don't have it in all Cargo.toml files either.
Change-Id: Ifcb333a39603a7e402ab0068e0f0f1b5af769e13
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11388
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Autosubmit: flokli <flokli@flokli.de>
This is another representation about information Nix has on a Store
Path. It's encountered in the exportReferencesGraph feature.
Change-Id: Ia0c08c369f4af9e50ebe3a5053bd98fadaa0003d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11396
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
We use the (slightly more tolerant) from_str to deserialize, and
serialize out as SRI.
Change-Id: If76b0ed2d4e243904f02df34f6c90b976c0bab8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11393
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Move the code implementing it from StorePath to StorePathRef, and have
the StorePath impls use that too.
Drop the debug_assert in every comparison - we have tests for this to
ensure it keeps working, and built up some confidence by piping a lot of
other store paths through it in the meantime.
Change-Id: I288bad3dfa597f68d63c4bcda7791f722b7a8ced
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11392
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
We forgot to only write the part of the padding that has not been
written already.
Unfortunately it seems the tokio mocking thing does buffer small writes,
so the tests succeeds even with the bug :-/
Change-Id: I5a6cf04212d559728639427c57207094d507ec75
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11384
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Brian Olsen <me@griff.name>
Autosubmit: flokli <flokli@flokli.de>
Make it a bit more clear that this is the total padding length, not the
padding length we still need to write.
Change-Id: I9ff4aa16f256fda367b4b9295abf82ed01b1f989
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11383
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
This is perfectly fine to track the position inside a reader too, so
rename it to reflect that.
Also make the docstring a bit less write-specific.
Change-Id: I831b0a8fe44a2477d4af96fefc692b9aabc378f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11382
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
We actually did construct other payload sizes than the test function was
intended to, because I'm too stupid to count. Fix that, and also update
a leftover _debug in a test name.
Change-Id: I1bff8551f2078afe82f789646b851f2fcff3e82c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11381
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Moving out the handshake route out of tvix-daemon to nix-compat.
We're bumping the protocol to version 37. It does not change anything
for us. Version 37 introduced a change in the build result
serialization. We do not implement that for now.
Tested tvix-daemon against Nix via the integration test to make sure
we did not break anything in the process.
Taking advantage of this refactor to remove the unused anyhow
dependency in tvix-daemon.
Change-Id: I8a2aad0b6dbb0aaa997446a612a365fd7d70336c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11379
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This allows using read_u64, write_u64, which is a bit easier to juggle
with.
Also, update names to align with the nix codebase, which makes it easier
to spot both the constant name as well as the value.
Leave the ASCII interpretation as a comment afterwards.
Change-Id: I0b9ab187acd22807e2785b0722aa4300dab37c51
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11378
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
`primitive.rs` implements reading and writing primitive (fixed-length)
types in the wire format, used in the the nix daemon protocol and NAR
format.
Move worker-protocol specific magic bytes to worker_protocol.rs (and
possibly further split there once needed)
Change-Id: If681c01e9460294619f1d000229b81f0ac745810
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11377
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
