The boot tests are sometimes flaky, and we don't want them to
periodically fail other build. Have Buildkite auto-retry them 2 times
on failure.
Logs for individual failures are still available, so it won't hinder
flakiness debuggability.
See https://buildkite.com/docs/pipelines/command-step#retry-attributes
for a documentation of a parameter, and cl/8983 for the introduction of
that feature to //nix/buildkite.
Change-Id: I1c0d25fa1d0ca940b3bdcd145ede87154b0c28eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12228
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Use redb instead of sled for the default filesystem implementation of
PathInfoService and DirectoryService. In the future we'll also drop sled
support completely.
Change-Id: I513ff0c2ff953d59714aa50b9aa1301b02f53d40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12085
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This provides a DirectoryService implementation which uses
redb (https://github.com/cberner/redb) as the database. It provides both
in-memory and persistent on-filesystem implementations.
Change-Id: Id8f7c812e2cf401cccd1c382b19907b17a6887bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12038
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
The environment variables caused both nar-bridge and tvix-store daemon
to try to connect to the same store, which fails due to locking issues.
Pass the config to `tvix-store daemon` directly. Also, add the
`--otlp=false` to the instructions to remove spam, most users don't have
a OTLP collector running.
Explain when to use gRPC in `tvix-store virtiofs` and when not.
Also, point out to be able to boot a NixOS closure, it needs to be
copied into the store first.
Change-Id: If4eda07bba28ad0bbe70e468cb727441a21b0588
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12067
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
It was previously pointing to a sled implementation which no longer
exists. It was also using nar-bridge-go instead of the new nar-bridge
(rust).
Closes: https://b.tvl.fyi/issues/413
Change-Id: Id0df61d4728198c3ae95a8e27ba7303434892966
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12063
Reviewed-by: aspen <root@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Ensure nar-bridge is healthy before connecting to it, don't just check
for the unix socket to be present.
We don't have a proper /health endpoint yet, but nix-cache-info works
fine for now.
Change-Id: I22df2c3b7bffcf52dbd3d00f3ba5382dc06ab03d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12030
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Ensure the service is healthy before connecting to it, don't just check
for the unix socket to be present.
Change-Id: If6501828677c247910d91f35b860960802084691
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12029
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
This provides a PathInfoService implementation using redb
(https://github.com/cberner/redb) as the underlying storage engine.
Both an in-memory variant, as well as a filesystem one is provided,
similar how it's done with the sled implementation.
Supersedes: https://cl.tvl.fyi/c/depot/+/11692
Change-Id: I744619c51bf2efd0fb63659b12a27cbe0b2fd6fc
Signed-off-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11995
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This switches the boot tests from the golang implementation to the rust
one.
Change-Id: Ide2a47aebe40c172077147c05bb6dacd74ba6b1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11997
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
reqwest wants to be able to read a file of trust roots when constructed,
but as it doesn't actually do any HTTPS connections inside the nix
build, an empty list of trust roots is totally sufficient.
Thankfully /dev/null provides such a file.
Change-Id: I9bd1619b2c9f8ff2a6640d2ac410d4de5b20c2ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11961
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
The aarch64-linux version of this is too big for cloud-hypervisor to
accept.
The correct place for this is in
`${kernel}/${stdenv.hostPlatform.linux-kernel.target}`, the other output
is only useful for debugging purposes.
This gets cloud-hypervisor to start booting up the kernel. It still
doesn't boot successfully, but it's getting further.
Part of b/407.
Change-Id: I3fbe52597fb1a6f5728a7f375dd35b50e00b3b1a
Co-Authored-By: Alyssa Ross <hi@alyssa.is>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11905
Tested-by: BuildkiteCI
Reviewed-by: Alyssa Ross <hi@alyssa.is>
Autosubmit: flokli <flokli@flokli.de>
This uses the `preStart` mechanism to boot up a minio instance, which
we point blob/directoryservice to.
Change-Id: I9e67b19006cb08ce82fe63c885cb0178e97389e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11896
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This uses the `preStart` mechanism to boot up a bigtable emulator, which
we point directory/pathinfoservice to.
Change-Id: Icb383403cb2d1f7e19c71b6298e82b391f9c8d22
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11895
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
This allows running some code before tvix-daemon is started up, which is
helpful to spin up some mock services.
Change-Id: Ie7b6eaaf76c40def493f50879ee5255fd1ebd2fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11893
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
This is still useful to test the ingestion part of nar-bridge.
Reintroduce the logic removed in cl/11398, but put it behind a
`useNarBridge` parameter, defaulting to false.
Add an additional target to tvix.boot.tests,
`closure-nixos-nar-bridge`, which does the same as `closure-nixos`,
except seeding the contents through nar-bridge.
Change-Id: Iee87d5d9c7b7d02df11ee47fbc3f1fbe951935e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11891
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
These fields don't need to be set if isClosure is false.
Change-Id: Id29ed034e0757c61cef09885962dfc5507ca106c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11890
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Make some space for the rust implementation.
Change-Id: I924dc1657be10abe5a11951c3b9de50bae06db19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11662
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
This never did any chunking, and sled (rightfully) performs really bad
if values get too large.
We switched the default to using the objectstore backend with the local
filesystem a while ago, no need to keep this footgun around anymore.
Change-Id: I2c12672f2ea6a22e40d0cbf9161560baddd73d4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11616
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This makes use of the vmCmdline and assertVMOutput to boot a NixOS VM
off of `tvix-store virtiofs` have it write a message on successful
bootup, then have it shut down again.
We check we see the message in the output.
Seeding the store with the closure it's what's currently taking most
of the time (as it's all linear still), so log a message before booting
the VM.
Change-Id: I6887d935f589eb0ad6507033c01337e1ec500a0b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11191
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Instead of using an additional Nix build producing a directory structure
containing xz-compressed NAR files and NARInfos, and uploading these via
`curl` to NAR-bridge, which then decomposes them and uploads them to
tvix-store, have tvix-store interpret the exported reference graph
directly, and deal with the uploading on its own.
Change-Id: I44ee0b132944c53d11abb688d2d017f0cc9c3d97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11398
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Bump to the latest u-root release. This seems to be compatible with go
1.22, too, which previously segfaulted the compiler in cl/11289.
Change-Id: Ie8a0d3cb51a435d6a0b194b38a9c83ee260618df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11291
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This can be used in the future to boot proper NixOS VMs, which want a
init= in their cmdline.
Change-Id: Iad4a25d5081f3a6af1c7f62f15853f3afaae4a0c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11190
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We don't have a OTLP collector inside the Nix build, this removes some
noise from the log about not being able to reach it.
Change-Id: I058839c8f214821b536306c8c30e1a8e7192153b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11189
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This updates the parameters mkBootTest can be called with. It now
accepts a `path`, and then either importPathName needs to be set, or
isClosure needs to be set to true.
The former activates the existing functionality, tvix-store import is
used to import contents as a NAR-addressed store path.
The latter uploads the path as a closure (so including its references,
and keeping the store paths intact) to tvix-store.
We use nar-bridge, and the HTTP interface it provides to do this.
As `nix copy` can't be used inside a Nix build, we use
`pkgs.mkBinaryCache` to come up with the .narinfo and .nar files that
would be in a binary cache, and then use a bit of GNU Parallel and bash
to upload store paths ourselves.
Change-Id: Icfa5c0af0c22ab5418686947aa2c060f5987b873
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11188
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
We currently use a pretty big kernel to boot the tvix VMs, with a lot
of drivers.
It can probably be slimmed down significantly, at least on the hardware
driver and filesystem front.
Change-Id: I4c7f73fb2dafbf2bcdec8057d2b14a9b0e9b3275
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11187
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Fork off a long-living tvix-store daemon in the background, and have
both the import, as well as the virtiofs invocation connect to that one,
rather than use the underlying storage directly.
This also opens up the potential to have other things seed the store
(like nar-bridge).
Change-Id: Ie13aedaf7ab31e5ebe78a7be42f144af3920fa9c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11186
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This makes BLOB_SERVICE_ADDR configurable, and creates a flavor setting
it to objectstore+file://$PWD/blobs.
Change-Id: I68c21367f83f68b4dee701c5678f438c7d8fbe43
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11137
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Rename the nar-bridge CLI to nar-bridge-http, because it's the one
spinning up an http server.
Change-Id: I0fb75c50e4299272a128dd5ecaa4be8f06fa3dbe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9538
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This used to be in tvix/test but was moved to tvix/boot, but not all the
command were updated so they no longer work as documented.
This updates the commands to use the new tvix/boot directory.
Change-Id: I8658d928a152140df182c159bf98ea9d9ecd1302
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9482
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This is mostly boot tooling, the integration test is just one instance
making use of it.
Expose initrd, kernel and runVM as a separate target to CI, and move the
tests to a subdirectory.
Change-Id: I1d22cd68bf5af095bc11dd9d7117b62956c7f7f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9465
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
