When we start unrecursivifying (sp?) things, Rust's borrow checker
is going to be a headache; its magic only works when you use the CPU
stack as your call stack.
Fixing the borrow checker issues usually involves adding lots of
`clone()`s. Right now `NixList` is the only variant of `Value` that
isn't cheap to clone() -- all the others are either a wrapper around
Rc or else are of bounded size.
Note that this requires dropping the `DerefMut for NixList` instance
and using `Vec<Value>` instead in those situations.
Change-Id: I5a47df66855342aa2064f8f3cb7934ff422d26bd
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7359
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
When comparing Nix values for equality, an issue can occur where
recursive values contain thunks to themselves which causes borrow
errors when forcing them for comparison later down the line.
To work around this we clone the values for now. There might be some
optimisations possible like checking for thunk equality directly and
short-circuiting on that (we have to check what Nix does).
Change-Id: I7e75c992ea68f100058f52b4b46168da7d671994
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7314
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
When passing multiple arguments, every intermediate callable needs to
be forced as this is expected by the VM's call_value function.
Also adds a debug assertion for this which makes it easier to spot
exactly what went wrong.
Change-Id: I3aa519cb6cdaab713bd18282bef901c4cd77c535
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7312
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This function covers builtins.genericClosure, seemingly including
weird behaviour around the order in which the work set is processed.
For some reason, in C++ Nix the test expectation is written in XML
which we do not yet support, so I have created a new expectation file
using `nix-instantiate --eval --strict` on the file (yes, using C++
Nix).
Change-Id: Id90e7117d120dc66d963a51083c4d8e8f2d9f181
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7311
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This implementation closely follows the original implementation in
Nix, including the use of an equality-based "set" structure to track
keys that have already been processed.
Note that this test does not yet enable the `notyetpassing` test for
builtins.genericClosure because (for as of yet unknown reasons) this
test compares against XML output (however, evaluating the test case
actually does work).
This takes us one step closer to nixpkgs eval.
This commit was written somewhere in the North Sea.
Co-Authored-By: Griffin Smith <root@gws.fyi>
Change-Id: I450a866e6f2888b27c2fe7c7f77ce0f79bfe3e6c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7310
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
**TL;DR:**
- Delete half-baked packaging attempts (`job.nix`, `token.nix`).
- Ensure golang code compiles.
- Some "packages" were being treated like "programs" presumably for
debugging/testing purposes back when I was working on this. Make those
behave like libraries.
- Remove stale imports.
- Fix syntax errors.
- Miscellaneous other chores.
- Drop `shell.nix` and `use_nix` directive.
Change-Id: I63c275680bac55a3cad3b9cb48d51cdc431fbe48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7318
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Something I missed last time reading through the process documentation
is that you can use a combination of `process` and `process-wait` to
determine the exit status of a child process *and* read from its
standard output. With `process*` we could even capture stderr, but we
probably want it mounted to the parent process' stderr anyways.
Change-Id: I9840f607df465caa80d28109e344e5fc1402949d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7259
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
**TL;DR:**
Most of these changes predicate behavior on the platform:
- At buildtime this is `localSystem == $something`. (`localSystem` is from
`readTree`)
- At runtime this is `(memq window-system '(ns))`.
- Add `coreutils` so `dired` listing with `--group-directories-first` works
because that flag depends on the GNU version of `ls`.
**Background:**
I need to support a bunch of OSX users at $WORK. As such, I'm planning
on using my MBP for the next few weeks to build empathy for our
userbase and polish some currently rough edges.
If I'm going to get an serious work done, I need my Emacs setup. Step
one is making sure it can build and run.
Change-Id: I918efccfa5f149e218aeea476c2c7df1c7b64ae8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7309
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Drop into a new shell environment with the same variables defined in a systemd
unit file (for debugging purposes).
Change-Id: Iaf513809b524f3f3e845b512450da71694bb7c7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7308
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
It's impossible to log into many public wifi networks otherwise when
the login depends on DNS hijacking.
Change-Id: I09f8b504810eebeb788997d2100a6db4777a8725
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7307
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Proof-reading much easier when Markdown is rendered in the browser.
Change-Id: Ia173dea817866d214547546bc3428ac6fe6782bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7305
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
By re-using the nix_2_3 expression from nixpkgs we are no longer stuck
with the 2021 expression from Nix's release.nix and the resulting
derivation has a shape that other expressions in nixpkgs expect (e.g.
nix-serve), so we can actually overlay our fork into the nix_2_3
attribute. This should reduce duplication on e.g. whitby.
Since there is no nixUnstable expression in nixpkgs anymore, it expects
a Nix release tarball which we produle using the appropriate release.nix
job from the repository. Sadly there is some trickery involved in
getting it to respect localSystem, since the tarball job is not based on
the passed in systems list.
Change-Id: Ib49f298334d166327f91559a06b0a37b2488bc63
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7262
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Since cl/7260 has eliminated all uses of builtins.currentSystem
in the main evaluation path of depot, we can ensure that we use
localSystem consistently in the future by making it impossible
to access the value of builtins.currentSystem in readTree nodes.
Change-Id: I6ded54021c42fcf31a80268149179f95f22bad88
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7261
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
cl/5832 added a global system parameter to depot which allowed specifying
what `system` should be used for nixpkgs and all depot derivations
(assuming a native compilation case) which was implemented in cl/5846.
This allows instantiating derivations for a different system than
whatever builtins.currentSystem happens to be. This is useful for
debugging, allows you to schedule builds on build servers for
other platforms or build for architectures that are a subset of
the one you are running (e.g. i686-linux).
This change eliminates all remaining uses of builtins.currentSystem
which could lead to an inconsistent combination of `system` values
when passing `localSystem`.
Change-Id: I0f824f4f0afa88ef1ddd9a8cecb24bf94bacde7a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7260
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Both //tvix/eval and //tvix/nix_cli have need to for rust tooling available
in $PATH.
Move this one level up, so it's accessible in all subdirectories.
Change-Id: I0763bbe9cefdc962f3a8f86c51e8f67cde8b4b04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7248
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
This detects if the second argument of a division is a zero (either as integer
or as float). If so, an error message is displayed.
This fixes b/219.
Change-Id: I50203d14a71482bc757832a2c8dee08eb7d35c49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7258
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This reverts commit 6813598c17.
Reason for revert: This function is a re-implementation of
builtins.unsafeDiscardOutputDependency which I missed at the time.
Change-Id: I5bb52bfd5e8d51defaf90ee795b0fe99be84f6db
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7265
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
As per the discussion in
https://cl.tvl.fyi/c/depot/+/7128/2..5/tvix/proto/castore.proto#b39, ref
sounds more like an external, stateful ID. Call this `digest`, to make
clear it's precisely this.
Change-Id: I81dd3769e2ce017de470ae92f72a38fb72015f10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7134
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
`buf` doesn't like protos with different package names in the same
directory.
Change-Id: I30806b46b88f103779faa40466461091a4a01e06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7130
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Make it clear these are symlinks, not hardlinks.
The term "link" is too heavily correlated to other meanings in IPFS/IPLD
world, and calling this symlink removes this confusion.
Change-Id: Id3f1eaa32098510b05f3e1a1348344503bcb4d5a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7129
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Add a new `documentation: Option<&'static str>` field to Builtin, and
populate it in the `#[builtins]` macro with the docstring of the builtin
function, if any.
Change-Id: Ic68fdf9b314d15a780731974234e2ae43f6a44b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7205
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Refactor the arguments of a Builtin to be a vec of a new BuiltinArgument
struct, which contains the old strictness boolean and also a static
`name` str - this is automatically determined via the ident for the
corresponding function argument in the proc-macro case, and passed in in
the cases where we're still manually calling Builtin::new.
Currently this name is unused, but in the future this can be used as
part of a documentation system for builtins.
Change-Id: Ib9dadb15b69bf8c9ea1983a4f4f197294a2394a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7204
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Some new top-level re-exports (specifically VM, Builtin, and ErrorKind)
were added to lib.rs in tvix/eval to allow the builtin-macros tests to
work - we should be clear which of these are part of the public
interface (I think it's reasonable for ErrorKind to be) and which
aren't (specifically I'm not sure VM and Builtin necessarily should be,
at least yet).
Change-Id: I3bbeaa63cdda9227224cd3bc298a9bb8da4deb7c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7203
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Similar to what we did with pure builtins, define the impure builtins
within a module at the top-level using the new #[builtins] attribute
macro
Change-Id: Ie5d5135d00bb65e651531df6eadba642cd4eb08e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7202
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Break out all pure builtin functions to top-level functions defined
within the `pure_builtins` module in `builtins/mod.rs`.
Change-Id: I9a10660446d557b1a86da4c45a463e9a1a9b4f2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7201
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Mostly as a proof-of-concept of the new proc-macros for defining
builtins, define a single builtin (the first in the list, `abort`) at
the top-level of a child module within builtins/mod.rs, and add it to
the list of builtins returned from `pure_builtins`.
If this works nicely, we can start breaking out the rest of the builtins
into the top-level too, in addition to introducing additional sets of
builtins (to differentiate between pure and impure builtins).
Change-Id: I5bdd57c57fecf8d63c9fed4fc6b1460f533b20f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7199
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Add a single new proc macro to a new proc-macro crate,
`tvix-eval-proc-macros` for defining an inline module containing nix
builtins, and automatically generating a function within that module
which returns a list of those builtins as `tvix_eval::value::Builtin`.
Change-Id: Ie4afae438914d2af93d15637151a49b4c68aa352
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7198
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
This commit adds a markdown document which explains how the
thread-local VM infrastructure works, in case it is useful in the
future.
Change-Id: Id10e32a9e3c5fa38a15d4bec9800f7234c59234a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7193
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
This is a follow up to cl/7191 which neglected to adjust the
status.tvl.su.nix module and re-enable it.
Change-Id: Icc1917004cd50e5eab61a29bc68b393ba9bd6325
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7226
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Profpatsch originally implemented an advisory checker from scratch in
Rust. We now ended up just using cargo-audit for the global checks
exposed via CI and the custom implementation is unused. To clean up
//tools/rust-crates-advisory a bit, we can move the unused parts to his
user directory.
Change-Id: Iacbd27c163edd07c804220fd1b3569c23aebd3e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7171
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
We need a user in the context when we ask the groups backend to look up
groups by name, so for now if we don't have a _real_ user in the context
(such as during change indexing), then populate the context with the
anonymous user just for the duration of the groups backend calls.
Change-Id: If961d84fe57443cb95deb59628802658585ed1cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7172
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is more generally useful than just inside the VM, until it is
stabilised in Rust itself.
Change-Id: Id9aa3d5b533ff38e3d2c6b85ad484394fdd05dcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7186
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
