This doesn't have much to do with the plain "wire" format, it's merely
one user of it.
Also, use the more "public" `wire::` API to read/write bytes, strings,
bools and u64s.
Change-Id: I98dddcc3004dfde7a0c009958fe84a840f77b188
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11390
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Brian Olsen <me@griff.name>
Move everything bytes-related into its own module, and re-export
both bytes and primitive in a flat space from wire/mod.rs.
Expose this if a `wire` feature flag is set. We only have `async` stuff
in here.
Change-Id: Ia4ce4791f13a5759901cc9d6ce6bd6bbcca587c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11389
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Moving out the handshake route out of tvix-daemon to nix-compat.
We're bumping the protocol to version 37. It does not change anything
for us. Version 37 introduced a change in the build result
serialization. We do not implement that for now.
Tested tvix-daemon against Nix via the integration test to make sure
we did not break anything in the process.
Taking advantage of this refactor to remove the unused anyhow
dependency in tvix-daemon.
Change-Id: I8a2aad0b6dbb0aaa997446a612a365fd7d70336c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11379
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This allows using read_u64, write_u64, which is a bit easier to juggle
with.
Also, update names to align with the nix codebase, which makes it easier
to spot both the constant name as well as the value.
Leave the ASCII interpretation as a comment afterwards.
Change-Id: I0b9ab187acd22807e2785b0722aa4300dab37c51
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11378
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
`primitive.rs` implements reading and writing primitive (fixed-length)
types in the wire format, used in the the nix daemon protocol and NAR
format.
Move worker-protocol specific magic bytes to worker_protocol.rs (and
possibly further split there once needed)
Change-Id: If681c01e9460294619f1d000229b81f0ac745810
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11377
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
This includes String, &str etc. An example testcase with &str is
provided.
Change-Id: I900186d6ceb52f52bd41ef4596524c1f5b52470b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11376
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
The protocol is more stateful than I initially thought. We need to
keep track to a bunch of things, including but not limited to: the
client settings, the client version. I moved things around a bit to
keep this state along with the client socket.
Change-Id: Ibd34fbe7821c20a460934ea1af0719f5de46e491
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11359
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This deals with writing byte packets of larger sizes to an underlying
AsyncWrite.
Its constructor receives the expected size. It also deals with writing
padding if flush/shutdown is called after writing all the payload.
Change-Id: I8acbf992467f3862ffb8c7d669e8c0c8eced14c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11355
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Remove potential partial reads/writes and instead read/write the full
buffer size: we want those to be 64 bits.
Change-Id: I1f767baf23fa80c2babb8113f61d1a9e72a8d8dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11350
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Adding a verbosity flag available through the CLI/ENV variable.
Change-Id: If04cc2e6e26e7cb3c2df7821fce222da2b85a95a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11349
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Actually these are all u64 LE encoded on the wire.
Change-Id: I5ca22c7639607ac47117cd946e036a444271885a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11348
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Using all the primitives recently implemented to nix-compat to reach
the point where the Nix client start to send us operation requests.
Using a small integration test script (or the VM test, but let's face
it, it's too slow to be useful), we manage to reach the point where
we're able to read a store operation:
2024-03-21T18:53:27.624876Z INFO tvix_daemon: Incoming connection addr=unix
2024-03-21T18:53:27.625312Z INFO worker:perform_init_handshake: tvix_daemon: Trust sent conn=Connection(unix) conn=Connection(unix)
2024-03-21T18:53:27.625406Z INFO worker: tvix_daemon: Client hanshake succeeded conn=Connection(unix)
2024-03-21T18:53:27.625488Z INFO worker: tvix_daemon: Operation received op=SetOptions conn=Connection(unix)
We had to take some shortcuts wrt. stderr/log management. The CPP Nix
codebase is a bit confusing in that area. I'll need to spend more time
reading this to fully understand what's happening there. For now,
sending the STDERR_LAST command to the client does the trick.
Change-Id: I9b0e20a52d885e64fe29188496aac5334de61edd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11233
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Write counterpart of read_bytes. Despite its name, we mostly use it to
write strings (as in ascii strings) to the wire.
We also extract the padding calculation in its own function.
Change-Id: I8d936e989961107261b3089e4275acbd2c093a7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11230
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is currently unused, and even if it would be used, it'd probably
be with `default-features = false`, explicitly enabling what's needed.
Change-Id: I3175a19adf8d65f0b805e5234267e460cd5e68d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11287
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Tested-by: BuildkiteCI
Implementing the full connection handshake.
The integration test is a bit naive, but there's not much to test yet.
Tested this against cpp nix. We reach the stage where cppnix sends the
opcode.
Change-Id: I98322832848ee5b048f22105731b0adeb44b2ce0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11227
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Adding a VM integration test setup for tvix-daemon. This test acts as
our first milestone: implementing enough daemon operations to be able
to add a new store path to tvix-store.
The test is expected to fail for now. We don't want to run it on CI
yet.
Change-Id: I2bd8eb9a07c5de2ef91099e10fcac23c087b880b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11199
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This daemon is a re-implementation of the Nix daemon except it uses
tvix-store as a remote store.
For now, it's very barebones, this is just a quick and dirty setup to
get started with the project. We bind to the unix socket provided by
systemd, wait for cpp Nix to send the magic hello bytes, respond with
the magic hello bytes and call it a day.
Storing this under my username for now, the project is mostly
irrelevant as it is. We'll move it to Tvix if it gets complete and
relevant at some point.
Change-Id: Ifc5dce2df37413504f9de1942c5b7d425eddf759
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11198
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
