runc is not available on MacOS, we only want to include it in the shell if on
Linux.
Point TVIX_BUILD_SANDBOX_SHELL to /bin/sh if not on Linux.
While we cannot use the OCI/runc executor on MacOS, other implementations might
make use of this env var, so it's good to keep setting it.
Also update the other occurence of it, in tvix/utils.nix.
Change-Id: If7b47e1bb7c41bbde84c93016713754a252c4355
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12691
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
This is just patchset 10 of CL10855, before the color_eyre changes,
rebased to the tvix_castore api.
Change-Id: If4b42412ff8568058908cda971ad7d6f2d9f9b7b
---
This provides a build service invoking runc. It can be used by using the
`oci://$path_to_some_tempdir` builder URL for now.
For now, it can be tested as such:
```
BUILD_SERVICE_ADDR=oci://$PWD/bundles target/debug/tvix
let pkgs = (import <nixpkgs> {}); in builtins.readDir pkgs.perl
```
readDir is to actually trigger IO into the store path (which triggers
the builds).
For now it fails due to missing reference scanning (see followup CLs).
Change-Id: I09b40e410114ce69966a41a0e3c33281b859e443
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12526
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This provides a plugin for callouts.
It needs to have additional CSS file added to `book.toml`,
which can be (re-)generated by `mdbook-admonish install`.
See https://github.com/tommilligan/mdbook-admonish/issues/171#issuecomment-2009166079
for more context.
Use it by adding a warning one to the architecture document.
Change-Id: I75c9a33d00acb603c6da10d3f9ce3485731c1672
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11805
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: yuka <yuka@yuka.dev>
The emulator and bigtable client are quite big. Remove them from the
default //tvix:shell.
Put the tests behind a `integration` feature flag, and add a variant
with that enabled to CI, and drop the bigtable tools from //tvix:shell.
Change-Id: Ie042097a0d6fc26542faa96c139b77298ccb160a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11582
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This tells josh to include the patch from
third_party/overlays/patches/cbtemulator-uds.patch at
nixpkgs/cbtemulator-uds.patch.
We then use the `additionalOverlays` feature introduced earlier, and
pass it an overlay adding the patch to `cbtemulator`.
This only gets applied if tvix/shell.nix is not called with a pkgs
argument, which is the case in the tvix view.
Change-Id: I7bc5543460ddd30325ab0998e891c15fcaf8bcfe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11408
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
This adds a Directory service using
https://cloud.google.com/bigtable/docs/ as a K/V store.
Directory (closures) are put in individual keys.
We don't do any bucketed upload of directory closures (yet), as castore/
fs does query individually, does not request recursively (and buffers).
This will be addressed by store composition at some point.
Change-Id: I7fada45bf386a78b7ec93be38c5f03879a2a6e22
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11212
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
I lost a lot of hope and had to read the source code of `quote!`, `cargo expand` was invaluable
in this adventure. We should keep it IMHO.
Change-Id: Icfb4c80d413602f2bdc6deab0d595183825d88ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10635
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is a bit more useful than evans when sending over the same message
over and over again.
Change-Id: I600b6b9f591c0c963c5d270005aa1cc84d2a0770
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10453
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
This adds a criterion.rs-based testbench into tvix-glue.
It can be invoked by running `cargo bench` from inside the `tvix-glue`
crate.
`target/criterion/report/index.html` contains nice graphs.
It's able to diff against the previous run, so you can invoke `cargo
bench` before and after a certain change to reason about the impact in
evaluation performance.
Currently, we need to create a bunch of Evaluator resources inside the
benchmark loop itself, which is a bit annoying, as it leaks into the
things we benchmark.
This should become better with b/262.
Fixes b/322.
Change-Id: I91656a308887baa1d459ed54d58baae919a4aaf2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10245
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Add some hyperfine benchmarks to Tvix's windtunnel benchmark script for
evaluating the outPath of hello and a cross-compiled hello.
Change-Id: I9d76e5ce0a3fd7d9c125c36c5fced675b660a8a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10248
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
As described in https://b.tvl.fyi/issues/313, the `nix_oracle` tests
currently fail if run without Nix 2.3 in `$PATH`.
This is not a problem for people running Nix 2.3 on their systems, and
neither is it for the tests running inside Nix derivations itself (as we
hardcode Nix 2.3 there), but for interactive `cargo test` runs, either
using `mg shell //tvix:shell`, or `nix-shell` (from the tvix workspace)
it currently is broken.
Adding Nix 2.3 to the shell fixes that.
Change-Id: Idbe01aa92d63eb1a3dab5797b9be5eba1e8d7f7a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9907
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
That's needed to build nar-bridge, as well as things in tvix/*store/
protos.
Change-Id: If5f71b6c528a27689faa3c5ab3489cc528f9e6e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9759
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
It looks like we need Security Framework in the shell to be able to
imperatively run `cargo build`.
Change-Id: Ia5df9052bafca3610d4ed235b180b812d33d0d15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9756
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
So far, we provided a custom `default.nix` in the root of the tvix josh
workspace, which re-defined the shell attribute from `tvix/default.nix`.
Some of the recent fixes, e.g. the MacOS-specific additions to the list
of dependencies however didn't get ported over to this file, and in
general, it's quite annoying to have two different places for these
things.
Initially I explored the idea of moving this default.nix file to a
default-depot.nix file in the josh worktree only, and then "polyfill"
some of the dependencies, or set up readTree in the josh workspace too,
but it turned out to pull in too many dependencies to be worth
the effort (nix.sparseTree, tools.depotfmt, crate2nix overlay,
third_party.gitignoreSource).
I now took a different approach - moving the definition of the `shell`
attribute from `tvix/default.nix` to its own `shell.nix` file, which is
imported from `tvix/default.nix` in regular depot usecases.
Josh workspace consumers only see the `shell.nix`, which can be used
in a self-contained fashion, the other `default.nix` is gone entirely,
and we update the workspace file to also not show `tvix/default.nix` at
the root either, so running `nix-shell` and then `cargo build` should
still work.
Change-Id: I6cb54d45d150c597612530ba44bc578f9d7f9120
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9556
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Manually maintained shell target based on the previous "loose"
`shell.nix` file.
We might want to have a function that combines the dependencies of all
the targets automatically, but at a quick glance that was actually
non-trivial so I'm leaving it as an exercise for someone else.
Change-Id: I74754940088f1b58e3b6754fb782470c80ea4292
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7540
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This uses prost-build to build the proto files.
Change-Id: Ia99fcfa4a19c741683cf28a45202e1d9211f9131
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7286
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Both //tvix/eval and //tvix/nix_cli have need to for rust tooling available
in $PATH.
Move this one level up, so it's accessible in all subdirectories.
Change-Id: I0763bbe9cefdc962f3a8f86c51e8f67cde8b4b04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7248
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
For some reason a top-level Rust project ended up in this location,
which is incompatible with the actual project structure that's being
prepared for merge right now.
Change-Id: I9d919ad72fc7e4e4d8cbb9899e7f8d90fa7ca87a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6060
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
First steps for baba
Change-Id: Id6a68c5630cb85f280f4dcc7b2acf10c02454fd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4732
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
