We actually did construct other payload sizes than the test function was
intended to, because I'm too stupid to count. Fix that, and also update
a leftover _debug in a test name.
Change-Id: I1bff8551f2078afe82f789646b851f2fcff3e82c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11381
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Moving out the handshake route out of tvix-daemon to nix-compat.
We're bumping the protocol to version 37. It does not change anything
for us. Version 37 introduced a change in the build result
serialization. We do not implement that for now.
Tested tvix-daemon against Nix via the integration test to make sure
we did not break anything in the process.
Taking advantage of this refactor to remove the unused anyhow
dependency in tvix-daemon.
Change-Id: I8a2aad0b6dbb0aaa997446a612a365fd7d70336c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11379
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This allows using read_u64, write_u64, which is a bit easier to juggle
with.
Also, update names to align with the nix codebase, which makes it easier
to spot both the constant name as well as the value.
Leave the ASCII interpretation as a comment afterwards.
Change-Id: I0b9ab187acd22807e2785b0722aa4300dab37c51
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11378
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
`primitive.rs` implements reading and writing primitive (fixed-length)
types in the wire format, used in the the nix daemon protocol and NAR
format.
Move worker-protocol specific magic bytes to worker_protocol.rs (and
possibly further split there once needed)
Change-Id: If681c01e9460294619f1d000229b81f0ac745810
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11377
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
This includes String, &str etc. An example testcase with &str is
provided.
Change-Id: I900186d6ceb52f52bd41ef4596524c1f5b52470b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11376
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Use the same EMPTY_BYTES trick from BytesWriter to write out the
padding, rather than allocating a Vec.
Change-Id: Ifb4ba1b45b7388adbc135fc8e46fd3d3cedd30aa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11375
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Only describe the format once (in `read_bytes`, and simplify the other
docstrings a bit.
Change-Id: Iff898f3c4173d506a357bc14bdffbf69c4c6e0e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11374
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
This isn't used anywhere so far, and if someone really wants to, they
can simply make the range open-ended.
Change-Id: Iae9bcaa1f7ea032dd3ee76c8c142a38b6b72894d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11373
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Tested-by: BuildkiteCI
Add the primitives necessary to read the client settings from the Nix
daemon wire protocol.
Introducing the read_string primitive. This trivial primitive parses a
read_bytes call, check the bytes are valid utf-8 bytes and wraps the
result in a String.
Change-Id: Ie1253523a6bd4e31e7924e9898a0898109da2fa0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11358
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This deals with writing byte packets of larger sizes to an underlying
AsyncWrite.
Its constructor receives the expected size. It also deals with writing
padding if flush/shutdown is called after writing all the payload.
Change-Id: I8acbf992467f3862ffb8c7d669e8c0c8eced14c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11355
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Let's make this usable for the entire crate.
Change-Id: I754408908a00296ee80dd52680f84b8a7cb22317
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11354
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Actually these are all u64 LE encoded on the wire.
Change-Id: I5ca22c7639607ac47117cd946e036a444271885a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11348
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Using all the primitives recently implemented to nix-compat to reach
the point where the Nix client start to send us operation requests.
Using a small integration test script (or the VM test, but let's face
it, it's too slow to be useful), we manage to reach the point where
we're able to read a store operation:
2024-03-21T18:53:27.624876Z INFO tvix_daemon: Incoming connection addr=unix
2024-03-21T18:53:27.625312Z INFO worker:perform_init_handshake: tvix_daemon: Trust sent conn=Connection(unix) conn=Connection(unix)
2024-03-21T18:53:27.625406Z INFO worker: tvix_daemon: Client hanshake succeeded conn=Connection(unix)
2024-03-21T18:53:27.625488Z INFO worker: tvix_daemon: Operation received op=SetOptions conn=Connection(unix)
We had to take some shortcuts wrt. stderr/log management. The CPP Nix
codebase is a bit confusing in that area. I'll need to spend more time
reading this to fully understand what's happening there. For now,
sending the STDERR_LAST command to the client does the trick.
Change-Id: I9b0e20a52d885e64fe29188496aac5334de61edd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11233
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This is used by the nix client to determine whether or not the daemon
trust it. The trust conditions check are daemon-specific, hence not
part of nix-compat.
Change-Id: Icbcba2f7f1fd58f67e7da72d22a264f5a3f3619d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11231
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Write counterpart of read_bytes. Despite its name, we mostly use it to
write strings (as in ascii strings) to the wire.
We also extract the padding calculation in its own function.
Change-Id: I8d936e989961107261b3089e4275acbd2c093a7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11230
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Setting the wire module as public to re-use it from tvix-daemon.
Change-Id: I570cffc480c7b784d813663f77572bbe9d4e8259
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11197
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This introduces a version reading sized byte packets. Both read_bytes,
accepting a range of allowed sizes, as well as read_bytes_unchecked,
which doesn't care, are added, including tests.
Co-Authored-By: picnoir <picnoir@alternativebit.fr>
Change-Id: I9fc1c61eb561105e649eecca832af28badfdaaa8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11150
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Tested-by: BuildkiteCI
These are not streams, but AsyncRead and AsyncWrite.
Change-Id: I7d988fa0490800b72862f4f0fcac3dceac70ec26
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11149
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
This brings some initial Nix wire format parsing code, used in the nix
daemon protocol, remote store/builder protocol, as well as the NAR
format itself (note we already have more specialized code for the last
one).
Thanks to embr, this code already exists, in
https://codeberg.org/gorgon/gorgon/src/branch/main/nix-daemon/src/wire.rs,
and we can vendor it into here, as EUPL is compatible with GPL (in that
direction).
The code uses the tokio::io Reader and Writer traits, not the ones from
the `futures` crate, as they provide some more convenient `read_u64_le`
functions.
More application-specific parsing code, as well as code to read strings,
or bytes are left out for now, as we want to be be more restrictive
w.r.t allowed max sizes, and need to parse bytes, not strings.
The code slightly diverges, as we have clippy looped into CI.
`Ok(…?)` can be turned into just the inner expression, and
some .and_then can be expressed in a simpler fashion.
Change-Id: Ie3adcb485e9d66786673b1962a08d4e5df3781d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11148
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
