mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
tvl-depot/tvix/boot/README.md

137 lines
4.5 KiB
Markdown
Raw Normal View History

chore(tvix/tests): rename to //tvix/boot This is mostly boot tooling, the integration test is just one instance making use of it. Expose initrd, kernel and runVM as a separate target to CI, and move the tests to a subdirectory. Change-Id: I1d22cd68bf5af095bc11dd9d7117b62956c7f7f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9465 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-25 12:59:25 +02:00
# tvix/boot
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
chore(tvix/tests): rename to //tvix/boot This is mostly boot tooling, the integration test is just one instance making use of it. Expose initrd, kernel and runVM as a separate target to CI, and move the tests to a subdirectory. Change-Id: I1d22cd68bf5af095bc11dd9d7117b62956c7f7f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9465 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-25 12:59:25 +02:00
This directory provides tooling to boot VMs with /nix/store provided by
virtiofs.
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
chore(tvix/tests): rename to //tvix/boot This is mostly boot tooling, the integration test is just one instance making use of it. Expose initrd, kernel and runVM as a separate target to CI, and move the tests to a subdirectory. Change-Id: I1d22cd68bf5af095bc11dd9d7117b62956c7f7f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9465 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-25 12:59:25 +02:00
In the `tests/` subdirectory, there's some integration tests.
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
docs(tvix/boot): Update command to reflect new tvix/boot directory This used to be in tvix/test but was moved to tvix/boot, but not all the command were updated so they no longer work as documented. This updates the commands to use the new tvix/boot directory. Change-Id: I8658d928a152140df182c159bf98ea9d9ecd1302 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9482 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-28 01:07:54 +02:00
## //tvix/boot:runVM
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
A script spinning up a `tvix-store virtiofs` daemon, then starting a cloud-
hypervisor VM.
The cloud-hypervisor VM is using a (semi-)minimal kernel image with virtiofs
support, and a custom initrd (using u-root). It supports various command line
options, to be able to do VM tests, act as an interactive shell or exec a binary
from a closure.
It supports the following env vars:
- `CH_NUM_CPUS=1` controls the number of CPUs available to the VM
- `CH_MEM_SIZE=512M` controls the memory availabe to the VM
- `CH_CMDLINE=` controls the kernel cmdline (which can be used to control the
boot)
### Usage
First, ensure you have `tvix-store` in `$PATH`, as that's what `run-tvix-vm`
expects:
Assuming you ran `cargo build --profile=release-with-debug` before, and are in
the `tvix` directory:
```
export PATH=$PATH:$PWD/target/release-with-debug
```
Secondly, configure tvix to use the local backend:
```
export BLOB_SERVICE_ADDR=sled://$PWD/blobs.sled
export DIRECTORY_SERVICE_ADDR=sled://$PWD/directories.sled
export PATH_INFO_SERVICE_ADDR=sled://$PWD/pathinfo.sled
```
Potentially copy some data into tvix-store (via nar-bridge):
```
mg run //tvix:store -- daemon &
chore(tvix/nar-bridge): move to nar-bridge-go Make some space for the rust implementation. Change-Id: I924dc1657be10abe5a11951c3b9de50bae06db19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11662 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: yuka <yuka@yuka.dev>
2024-05-14 12:35:55 +02:00
$(mg build //tvix:nar-bridge-go)/bin/nar-bridge-http &
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
rm -Rf ~/.cache/nix; nix copy --to http://localhost:9000\?compression\=none $(mg build //third_party/nixpkgs:hello)
refactor(tvix/nar-bridge): mv nar-bridge{,-http} Rename the nar-bridge CLI to nar-bridge-http, because it's the one spinning up an http server. Change-Id: I0fb75c50e4299272a128dd5ecaa4be8f06fa3dbe Reviewed-on: https://cl.tvl.fyi/c/depot/+/9538 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-10-03 14:23:36 +02:00
pkill nar-bridge-http; pkill tvix-store
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
```
#### Interactive shell
Run the VM like this:
```
docs(tvix/boot): Update command to reflect new tvix/boot directory This used to be in tvix/test but was moved to tvix/boot, but not all the command were updated so they no longer work as documented. This updates the commands to use the new tvix/boot directory. Change-Id: I8658d928a152140df182c159bf98ea9d9ecd1302 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9482 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-28 01:07:54 +02:00
CH_CMDLINE=tvix.shell mg run //tvix/boot:runVM --
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
```
fix(tvix/tests/tvix-init): run as pid1, support init= We were previously not running as PID1, but were executing things via uinitcmd (by u-root init). This had some problems - a real system pid1 really wants to be pid1. u-root init itself seems to be too complicated to configure as a chainloader, so instead, we just provide init/pid1 ourselves and do the little bit of mount setup / stage 1 that's necessary. `tvix.exec` is renamed to `tvix.run`, to further distinguish it from the `exec` bash function. Our init will now also power off the machine if the shell, or the custom tvix.run command finished. In the case of switch_root (`init=` being passed), we terminate before, so it's up to the real system anyways. Change-Id: If00c0c11ca9393968e76ae0d131594b5c135fa48 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9454 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-24 22:01:16 +02:00
You'll get dropped into an interactive shell, from which you can do things with
the store:
```
______ _ ____ _ __
/_ __/ __(_) __ / _/___ (_) /_
/ / | | / / / |/_/ / // __ \/ / __/
/ / | |/ / /> < _/ // / / / / /_
/_/ |___/_/_/|_| /___/_/ /_/_/\__/
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
/# ls -la /nix/store/
dr-xr-xr-x root 0 0 Jan 1 00:00 .
dr-xr-xr-x root 0 989 Jan 1 00:00 aw2fw9ag10wr9pf0qk4nk5sxi0q0bn56-glibc-2.37-8
dr-xr-xr-x root 0 3 Jan 1 00:00 jbwb8d8l28lg9z0xzl784wyb9vlbwss6-xgcc-12.3.0-libgcc
dr-xr-xr-x root 0 82 Jan 1 00:00 k8ivghpggjrq1n49xp8sj116i4sh8lia-libidn2-2.3.4
dr-xr-xr-x root 0 141 Jan 1 00:00 mdi7lvrn2mx7rfzv3fdq3v5yw8swiks6-hello-2.12.1
dr-xr-xr-x root 0 5 Jan 1 00:00 s2gi8pfjszy6rq3ydx0z1vwbbskw994i-libunistring-1.1
```
fix(tvix/tests/tvix-init): run as pid1, support init= We were previously not running as PID1, but were executing things via uinitcmd (by u-root init). This had some problems - a real system pid1 really wants to be pid1. u-root init itself seems to be too complicated to configure as a chainloader, so instead, we just provide init/pid1 ourselves and do the little bit of mount setup / stage 1 that's necessary. `tvix.exec` is renamed to `tvix.run`, to further distinguish it from the `exec` bash function. Our init will now also power off the machine if the shell, or the custom tvix.run command finished. In the case of switch_root (`init=` being passed), we terminate before, so it's up to the real system anyways. Change-Id: If00c0c11ca9393968e76ae0d131594b5c135fa48 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9454 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-24 22:01:16 +02:00
Once you exit the shell, the VM will power off itself.
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
#### Execute a specific binary
Run the VM like this:
```
hello_cmd=$(mg build //third_party/nixpkgs:hello)/bin/hello
docs(tvix/boot): Update command to reflect new tvix/boot directory This used to be in tvix/test but was moved to tvix/boot, but not all the command were updated so they no longer work as documented. This updates the commands to use the new tvix/boot directory. Change-Id: I8658d928a152140df182c159bf98ea9d9ecd1302 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9482 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-28 01:07:54 +02:00
CH_CMDLINE=tvix.run=$hello_cmd mg run //tvix/boot:runVM --
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
```
Observe it executing the file (and closure) from the tvix-store:
```
fix(tvix/tests/tvix-init): run as pid1, support init= We were previously not running as PID1, but were executing things via uinitcmd (by u-root init). This had some problems - a real system pid1 really wants to be pid1. u-root init itself seems to be too complicated to configure as a chainloader, so instead, we just provide init/pid1 ourselves and do the little bit of mount setup / stage 1 that's necessary. `tvix.exec` is renamed to `tvix.run`, to further distinguish it from the `exec` bash function. Our init will now also power off the machine if the shell, or the custom tvix.run command finished. In the case of switch_root (`init=` being passed), we terminate before, so it's up to the real system anyways. Change-Id: If00c0c11ca9393968e76ae0d131594b5c135fa48 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9454 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-24 22:01:16 +02:00
[ 0.277486] Run /init as init process
______ _ ____ _ __
/_ __/ __(_) __ / _/___ (_) /_
/ / | | / / / |/_/ / // __ \/ / __/
/ / | |/ / /> < _/ // / / / / /_
/_/ |___/_/_/|_| /___/_/ /_/_/\__/
feat(tvix/tests): add basic plumbing for integration tests This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-23 16:34:01 +02:00
Hello, world!
fix(tvix/tests/tvix-init): run as pid1, support init= We were previously not running as PID1, but were executing things via uinitcmd (by u-root init). This had some problems - a real system pid1 really wants to be pid1. u-root init itself seems to be too complicated to configure as a chainloader, so instead, we just provide init/pid1 ourselves and do the little bit of mount setup / stage 1 that's necessary. `tvix.exec` is renamed to `tvix.run`, to further distinguish it from the `exec` bash function. Our init will now also power off the machine if the shell, or the custom tvix.run command finished. In the case of switch_root (`init=` being passed), we terminate before, so it's up to the real system anyways. Change-Id: If00c0c11ca9393968e76ae0d131594b5c135fa48 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9454 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-24 22:01:16 +02:00
2023/09/24 21:10:19 Nothing left to be done, powering off.
[ 0.299122] ACPI: PM: Preparing to enter system sleep state S5
[ 0.299422] reboot: Power down
```
#### Execute a NixOS system closure
It's also possible to invoke a system closure. To do this, tvix-init honors the
init= cmdline option, and will switch_root to it.
```
docs(tvix/boot): Update command to reflect new tvix/boot directory This used to be in tvix/test but was moved to tvix/boot, but not all the command were updated so they no longer work as documented. This updates the commands to use the new tvix/boot directory. Change-Id: I8658d928a152140df182c159bf98ea9d9ecd1302 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9482 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-28 01:07:54 +02:00
CH_CMDLINE=init=/nix/store/…-nixos-system-…/init mg run //tvix/boot:runVM --
fix(tvix/tests/tvix-init): run as pid1, support init= We were previously not running as PID1, but were executing things via uinitcmd (by u-root init). This had some problems - a real system pid1 really wants to be pid1. u-root init itself seems to be too complicated to configure as a chainloader, so instead, we just provide init/pid1 ourselves and do the little bit of mount setup / stage 1 that's necessary. `tvix.exec` is renamed to `tvix.run`, to further distinguish it from the `exec` bash function. Our init will now also power off the machine if the shell, or the custom tvix.run command finished. In the case of switch_root (`init=` being passed), we terminate before, so it's up to the real system anyways. Change-Id: If00c0c11ca9393968e76ae0d131594b5c135fa48 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9454 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-24 22:01:16 +02:00
```
```
______ _ ____ _ __
/_ __/ __(_) __ / _/___ (_) /_
/ / | | / / / |/_/ / // __ \/ / __/
/ / | |/ / /> < _/ // / / / / /_
/_/ |___/_/_/|_| /___/_/ /_/_/\__/
2023/09/24 21:16:43 switch_root: moving mounts
2023/09/24 21:16:43 switch_root: Skipping "/run" as the dir does not exist
2023/09/24 21:16:43 switch_root: Changing directory
2023/09/24 21:16:43 switch_root: Moving /
2023/09/24 21:16:43 switch_root: Changing root!
2023/09/24 21:16:43 switch_root: Deleting old /
2023/09/24 21:16:43 switch_root: executing init
<<< NixOS Stage 2 >>>
[ 0.322096] booting system configuration /nix/store/g657sdxinpqfcdv0162zmb8vv9b5c4c5-nixos-system-client-23.11.git.82102fc37da
running activation script...
setting up /etc...
starting systemd...
[ 0.980740] systemd[1]: systemd 253.6 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -XKBCOMMON +UTMP -SYSVINIT default-hierarchy=unified)
```
docs(tvix/boot): Update command to reflect new tvix/boot directory This used to be in tvix/test but was moved to tvix/boot, but not all the command were updated so they no longer work as documented. This updates the commands to use the new tvix/boot directory. Change-Id: I8658d928a152140df182c159bf98ea9d9ecd1302 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9482 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-28 01:07:54 +02:00
This effectively replaces the NixOS Stage 1 entirely.
Reference in a new issue Copy permalink