C++ nix uses C-style zero-terminated char pointers to represent strings
internally - however, up to this point, tvix has used Rust `String` and
`str` for string values. Since those are required to be valid utf-8, we
haven't been able to properly represent all the string values that Nix
supports.
To fix that, this change converts the internal representation of the
NixString struct from `Box<str>` to `BString`, from the `bstr` crate -
this is a wrapper around a `Vec<u8>` with extra functions for treating
that byte vector as a "morally string-like" value, which is basically
exactly what we need.
Since this changes a pretty fundamental assumption about a pretty core
type, there are a *lot* of changes in a lot of places to make this work,
but I've tried to keep the general philosophy and intent of most of the
code in most places intact. Most notably, there's nothing that's been
done to make the derivation stuff in //tvix/glue work with non-utf8
strings everywhere, instead opting to just convert to String/str when
passing things into that - there *might* be something to be done there,
but I don't know what the rules should be and I don't want to figure
them out in this change.
To deal with OS-native paths in a way that also works in WASM for
tvixbolt, this also adds a dependency on the "os_str_bytes" crate.
Fixes: b/189
Fixes: b/337
Change-Id: I5e6eb29c62f47dd91af954f5e12bfc3d186f5526
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10200
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Have a Evaluation::new() function that's used to set up the Evaluation
struct initially - which is also used by both new_pure and new_impure
internally.
It's generic over the exact type of IO, making it easier to instantiate
Evaluation with non-tvix-eval EvalIO implementations, that might not be
in a Box.
Change-Id: Ibf728da24aca59639c5b6df58d00ae98c99a63f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10640
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Previously, we were assembling very naively an attribute set composed of context we saw.
But it was forgetting that `"${drv}${drv.drvPath}"` would contain 2 contexts with the same key, but
with different values, one with `outputs = [ "out" ];` and `allOutputs = true;`.
Following this reasoning and comparing with what Nix does, we ought to merge underlying values systematically.
Hence, I bring `itertools` to perform a group by on the key and merge everything on the fly, it's not
beautiful but it's the best I could find, notice that I don't use
`group_by` but I talk about group by, that is, because `group_by` is a
`group_by_consecutive`, see
https://github.com/rust-itertools/itertools/issues/374.
Initially, I tried to do it without a `into_grouping_map_by`, it was akin to assemble the final `NixAttrs` directly,
it was less readable and harder to pull out because we don't have a lot of in-place mutable functions on
our data structures.
Change-Id: I9933c9bd88ffe04de50dda14f21879b60d8b8cd4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10620
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Instead, it's passed in the evaluate/compile_only functions, which feels
more naturally. It lets us set up the Evaluation struct long before
we actually feed it with data to evaluate.
Now that Evaluation::new() would be accepting an empty list of
arguments, we can simply implement Default, making things a bit more
idiomatic.
Change-Id: I4369658634909a0c504fdffa18242a130daa0239
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10475
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
* all: update wasm-bindgen to 0.2.89 in WASM projects
* users/grfn: explicitly set pinentry for gpg-agent
* 3p/crate2nix: drop patches that were merged upstream
* 3p/rust-crates: fix one more package name that was broken by crates.io
* 3p/overlays: bump telega backend to new required version
The update for agenix has been dropped. It caused strange build errors
with messages like these:
patching script interpreter paths in /nix/store/0g0wpa3vxfb4w461s6ny3s1wr08faj73-agenix-0.15.0
/nix/store/0g0wpa3vxfb4w461s6ny3s1wr08faj73-agenix-0.15.0/bin/agenix: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/q8qq40xg2grfh9ry1d9x4g7lq4ra7n81-bash-5.2-p21/bin/bash"
stripping (with command strip and flags -S -p) in /nix/store/0g0wpa3vxfb4w461s6ny3s1wr08faj73-agenix-0.15.0/bin
Running phase: installCheckPhase
no Makefile or custom installCheckPhase, doing nothing
agenix version: 0.15.0
error: creating directory '/nix/var': Permission denied
There is no rule for secret1.age in ./secrets.nix.
/nix/store/d4jf1cbbk494zwgbqz31pxgigpsbh6w2-stdenv-linux/setup: line 138: test: =: unary operator expected
/nix/store/d4jf1cbbk494zwgbqz31pxgigpsbh6w2-stdenv-linux/setup: line 131: pop_var_context: head of shell_variables not a function context
builder for '/nix/store/0ivvf44hxy0zv4gg8nvchdkp895xw5ri-agenix-0.15.0.drv' failed with exit code 2
I can't be bothered to deal with that right now.
Change-Id: Ia052af0d97dbe9ef0c0d4f3e2214ac00ca8645a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10458
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
* update wasm-bindgen in all Rust-wasm projects
* remove stable overlays that work again in unstable
* add texlive to stable overlays (see linked nixpkgs PR)
* bump tdlib to 1.8.18, new minimum for telega.el
Change-Id: Ib8e202de7dfbc35115fda31d0a98b6314b2adf17
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10010
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Gotta use a shared crate for this ..
Change-Id: I57669bdcace9676ff1311f3171c015702c934f56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8734
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This UUID stays the same even if a user submits the form multiple
times (unless they edit it manually in local storage, of course).
Change-Id: I4190fbfeb1027ce8a8d87bc283099539e8722b39
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8733
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Note that there doesn't seem to be a TF resource type for the IAM
binding between the bucket and the service account itself (other than
applying to all buckets in the folder, which I don't want).
For this reason I've added the `storage.uploader` IAM binding to the
`rih-backend` service account *on the bucket* manually.
Change-Id: I9fb06c7857e61dc642d9ea0d89159a0e343dc984
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8728
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The completion detection is very naive for now, but baby steps etc.
Change-Id: Iec6eae77fca13f3fc0ee1225268eef04277b504a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8702
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This turned out a lot nicer than I expected it to be.
Change-Id: I427670644eba789ea2037423fa9af8e632b19b34
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8695
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This makes more sense to me.
Change-Id: I013bf9457f20a31a9762768607f4094358e1b7cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8693
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Mounts the privacy policy at `/privacy-policy`. Using yew_router
"properly" is difficult in components that don't make use of macros
and context magic, so I've opted to use the gloo history handling
directly to parse the location here.
Change-Id: Icde11485f9947bc860a7b2c43772bb0f4cdf2ea1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8653
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Doesn't actually have bucket serving or access configuration yet, one
step at a time!
Change-Id: I0ce9b3b077252395bd807fad44cbdca40cdeac49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8649
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This makes it possible to embed long texts from Markdown files instead
of dealing with writing the weird HTML-tags inside the yew macros,
which will be much easier for content editors to deal with.
Change-Id: Idc4e67404fcfe2b8d5083cf556df1c701ba17660
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8648
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This doesn't actually submit anything to the (not-yet-existing)
backend, but will help the designers figure out what we're actually
looking for here.
Change-Id: I680d88151fb0706953f18eb6256da6f205da7ffb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8489
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Sets up a virtual machine image that is bootable on Yandex Cloud.
There are some slightly wonky behaviours still, like cloud-init
apparently putting all keys into root's authorized_keys no matter what
is specified in the metadata, but it does work now.
Change-Id: I57dcb7fcfa6872a28855dc1347f73a6db3c56828
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8496
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This was a bit trickier than I anticipated, because there's no good
ways to avoid passing the credentials around manually.
What's basically happening now is that the credentials for the state
bucket are checked in (encrypted), and sourcing `creds.fish` uses the
cloud HSM to decrypt and load them into the environment.
Change-Id: I3f5ce1c9bd9d5efbf1013414f94771a09ea3a488
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8494
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Doesn't actually contain any configuration yet, just setting up TF
with the right providers and so on.
Change-Id: Ia7128dd977b4ff69eebaa36c6cad6ac104cafcdb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8492
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This module contains the request/response types for generators
requesting actions from the VM.
For most of these, an async helper function is added that will be used
inside of generator functions to make use of these requests/responses
instead of constructing them directly.
Change-Id: I1e085f88adaf784a34867957a0e82532d3a83d7c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8148
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
