Commit graph

766 commits

Author SHA1 Message Date
Adam Joseph
a98cb9d56f fix(tvix/eval): parseDrvName should not coerce, and xfail test
builtins.parseDrvName should not coerce its argument to a string.
This commit fixes that oversight in my previous commit, and adds an
xfail test to cover this condition.

Thanks to @sterni for noticing this.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I76bc78f1a82e1e08fe5c787c563a221d55de2639
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6991
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-10-13 17:42:22 +00:00
Adam Joseph
01bc04b1d2 feat(tvix/eval): enable the XFAIL tests
This commit adds support for running the "expected failure" tests in
both the nix and tvix test suites.

I have disabled the eval-fail-blackhole.nix test because it gets
stuck running forever.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: Iba75ce6c8f2becab3c834fcfdd9f4fdc5a4bdb9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6990
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-13 17:42:04 +00:00
Adam Joseph
706186eb5d test(tvix/eval): update for cppnix PR #7149
cppnix has merged #7149, so let's update our copy of their tests:

  https://github.com/NixOS/nix/pull/7149

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I0be3bf9da937abd24102e1997daa2087ecfafd98
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6992
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-13 17:40:34 +00:00
Vincent Ambo
0f24120a6c fix(tvix/eval): fix Compiler::new on wasm
This path normalisation business causes runtime panics on WebAssembly
because those operations are unsupported.

Maybe this shouldn't be happening in the compiler anyways, not sure,
but for now this commit adds a workaround based on the target to
disable the normalisation if we're compiling for wasm.

Change-Id: I908a84fbdffc3401f8d443e2c73ec673e9f397ff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7004
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-13 16:29:49 +00:00
Adam Joseph
78d3d9150b fix(tvix/eval): src/compiler: ensure root_dir is absolute
Cppnix immediately absolutizes pathnames at parse time; if you write
`./foo`, it is immediately converted to `$(pwd)/foo` and manipulated
as an absolute path at all times.

To avoid having to introduce filesystem access operations in the
implementation of otherwise-pure builtins, let's guarantee that the
`root_dir` of the VM is always an absolute path.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I7cbbae2cba4b2716ff3f5ff7c9ce0ad529358c8a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6995
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-13 09:48:32 +00:00
Adam Joseph
25336fc47b refactor(tvix/eval): factor out all calls to canon_path
Right now we're pretending that the Rust library path_clean does the
same thing that cppnix's canonPath() does.  This is not true.  It's
close enough for the test suite, but may come back to bite us.

Let's create our own canon_path() function and call that in all the
places where we intend to match the behavior of cppnix's
canonPath().  That way when we fix this we can fix it once, in one
place.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: Ia6f9577f62f49ef352ff9cfa5efdf37c32d31b11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6993
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-13 09:07:47 +00:00
Adam Joseph
534a2f95f8 fix(tvix/eval): nix_search_path.rs: use /etc instead of /bin
On Debian machines (and most FHS distros) /bin is now a symlink to
/usr/bin, which causes this test to fail.  Let's use /etc as a
root-relative test case, since it is less likely to be a symlink.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I1c94de0d2a242394182442fe1c895dc17eb04a4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6994
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-13 09:07:30 +00:00
Adam Joseph
e83609a061 feat(tvix/eval): allow to disable warnings
The nix_tests test suite produces lots of warnings.  We can't fix
these, since they are kept in sync with upstream, so there's little
point in cluttering up the console with them every time the tests
are run.

Let's add a clap flag "warnings" and TVIX_WARNINGS environment
variable.  The default is "true".  The test runner overrides this
default and mutes the warnings.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I4b065f96fe15838afcca6970491a54e248ae4df7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6985
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-13 02:49:59 +00:00
Adam Joseph
224ec708f7 test(tvix/eval): add eval-okay-parsedrvname.{exp,nix}
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I59254a0ac513b3d2a6b2184554358bb6eba2ba99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6949
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 09:50:21 +00:00
Adam Joseph
8111f2de1e feat(tvix/eval): builtins: implement parseDrvName
This commit passes nix_eval_okay_src_tests_nix_tests_eval_okay_versions_nix.

See also: https://github.com/NixOS/nix/pull/7149

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I24605c2a0cd0da434f37f6c518f20693bfa1b799
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6913
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 09:50:10 +00:00
Adam Joseph
32ac7d6c6d refactor(tvix/eval) s/NixPath/NixSearchPath/
Since NixString is the Rust type for nix strings, people might
mistake NixPath for the Rust type of nix paths, which it is not.
Let's call it NixSearchPath instead.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: Ib2ea155c4b27cb90d6180a04ea7b951d86607373
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6927
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 08:09:49 +00:00
Adam Joseph
f2bf65a40b fix(tvix/eval): remove overlapping instances
I broke the build.

I didn't understand that before hitting "submit" you need to re-test
your changes on latest HEAD (and that CI doesn't do that for you); I
failed to re-test cl/6912 following the merge of cl/6907.

This commit fixes the build by removing the overlapping instances.

Change-Id: I2a720d2c60cc7103b350f78102a8998f93bac828
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6965
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 07:25:51 +00:00
Adam Joseph
4e94a2696b feat(tvix/eval): From<Into<NixString>> for Value
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I27615baf4104ab96600708b12a780599a3714bae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6912
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 04:23:07 +00:00
Adam Joseph
2e0a4aaa84 feat(tvix/eval): From<Utf8Error> for ErrorKind
Change-Id: I074d9e862fbdd4e78e443cbaf0e77c7ffe825a10
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6911
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 04:22:44 +00:00
sterni
fcd5e53703 fix(tvix/eval/builtins): force acc not list element in foldl'
When investigating discrepancies between foldl' in tvix and C++ Nix,
I discovered that C++ Nix's foldl' doesn't seem to be strict at all.
Since this seemed wrong, I looked into Haskell's foldl' implementation
which doesn't force the list elements (`val` in our code), but the
accumulation value (`res` in our code). You can look at the code here:
https://hackage.haskell.org/package/base-4.17.0.0/docs/src/GHC.List.html#foldl%27

This actually makes a lot of sense: If `res` is not forced after each
application of `op`, we'll end up thunks nested as deeply as the list is
long, potentially taking up a lot of space. This can be limited by
forcing the `res` thunk before applying `op` again (and creating a new
thunk).

I've also PR-ed an equivalent change for C++ Nix at
https://github.com/NixOS/nix/pull/7158. Since this is not merged nor
backported to our Nix 2.3 fork, I've not copied the eval fail test yet,
since it wouldn't when checking our tests against C++ Nix in depot.

Change-Id: I34edf6fc3031fc1485c3e714f2280b4fba8f004b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6947
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-11 15:53:29 +00:00
Griffin Smith
1181bd78fc feat(tvix/eval): Allow directly evaluating an expr via main
This *maybe* should do something to check that we don't pass both a file
and an expr, but for now this is useful enough to cut corners (plus
we're probably due for a CLI revamp eventually anyway).

Change-Id: Id44357074150b336b6215ba596cc52d01d037dbd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6941
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-11 00:33:49 +00:00
Griffin Smith
4283f0139a feat(tvix/eval): Implement builtins.removeAttrs
Change-Id: I28910991a0108436a42ac7bf3458f9180a44154e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6928
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-11 00:33:39 +00:00
Griffin Smith
562c50fadd feat(tvix/eval): Implement builtins.trace
This is currently implemented with a simple println inline, but in the
future we could hook into this via something pluggable on the VM.

Change-Id: Idd9cc3b34aa13d6ebc64c02aade81ecdf439656a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6938
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-11 00:16:40 +00:00
Griffin Smith
4a058a9c00 fix(tvix/eval): Pop frames even if running op fails
Previously, the VM assumed that if an error was returned from `run()`,
the evaluation was "finished" and the state of the VM didn't matter.
This used to be a reasonable assumption, but now that we've got
`tryEval` around we need to actually make sure that we clean up after
ourselves if we're about to return an error. Specifically, if the *last*
instruction in an evaluation frame returns an error, we previously
wouldn't pop that evaluation frame, which could cause all sorts of
bizarre errors based on what happened to be in the stack at the time.

This commit splits out a `run_op` method from `VM::run`, and uses that
to check the evaluation frame return condition even if the op we're
running is about to return an error, and pop the evaluation frame if
we're at the last instruction.

Change-Id: Ib40649d8915ee1571153cb71e3d76492542fc3d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6940
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-11 00:08:05 +00:00
Vincent Ambo
76e0c37b9e feat(tvix/eval): observe stack after exiting call frames/builtins
Change-Id: I1937d37551503a0b6bb0ac899d067302e4791e5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6939
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-11 00:07:39 +00:00
Vincent Ambo
8e122574c2 fix(tvix/eval): implement functor calling for non-tail-calls
Change-Id: I1113921c46010021b61c412d74d60193c156e0f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6937
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-11 00:07:38 +00:00
Griffin Smith
06ec4bebe7 fix(tvix/eval): Actually trace spans for thunks
Currently, the span on *all* thunk force errors is the span at which the
thunk is forced, which for recursive thunk forcing ends up just being
the same span over and over again. This changes the span on thunk force
errors to be the span at which point the thunk is *created*, which is a
bit more helpful (though the printing atm is a little... crowded). To
make this work, we have to thread through the span at which a thunk is
created into a field on the thunk itself.

Change-Id: I81474810a763046e2eb3a8f07acf7d8ec708824a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6932
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 23:51:09 +00:00
Griffin Smith
90ec632fd1 feat(tvix/eval): Implement builtins.getEnv
Change-Id: I09f512a60989a37184e73e521d4a3aa23f33a1a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6922
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2022-10-10 23:38:25 +00:00
Griffin Smith
9669fa8ff1 feat(tvix/eval): Implement builtins.readFile
Change-Id: If3fd0b087009a2bfbad8bb7aca0aa20de906eb12
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6921
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-10 23:38:25 +00:00
Vincent Ambo
1f84d90811 refactor(tvix/eval): after calling, the caller has to pop
Previously the various call functions either returned `EvalResult<()>`
or `EvalResult<Value>`, which was confusing.

Now only vm::call_with returns a Value directly, and other parts of
the API just leave the stack top in the post-call state.

This makes it easier to reason about what's going on in non-tail-call
cases (which are making a comeback).

Change-Id: I264ffc683a11aca72dd06e2220a5ff6e7c5fc2b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6936
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-10 23:36:49 +00:00
Vincent Ambo
019ea51e5c fix(tvix/eval): format nested compiler errors in fancy output
Instead of just printing the number of errors (useless!) actually emit
separate diagnostics for each nested error.

Change-Id: I97b53c3276c906af5def89077b5b6ba6ec108b37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6933
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 22:07:22 +00:00
Griffin Smith
75f637c1f5 fix(tvix/eval): path resolution errors are catchable
Despite this not being documented, `tryEval` is empirically able to
catch errors caused by a <...> path not resolving (and nixpkgs depends
on this).

Change-Id: Ia3b78a2d9d2d0c603aba829518b351102dc55396
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6926
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-10 20:58:14 +00:00
Griffin Smith
2592113435 fix(tvix/eval): Thunk if expr
Since the body of an `if` expr can refer to deferred upvalues, it needs
to be thunked so when we actually compile those deferred upvalues we
have something for the finalize op to point at. Without this all sorts
of weird things can happen due to the finalize op being run in the wrong
lambda context, up to and including a panic.

Change-Id: I040d5e1a7232fd841cfa4953539898fa49cbbb83
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6929
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-10 20:35:11 +00:00
Griffin Smith
4fd18cbc9a feat(tvix/eval): Implement builtins.tryEval
With asserts compiled using conditional jumps, this ends up being quite
straightforward - the only real tricky bit is that we have to know
whether an error can or can't be handled.

Change-Id: I75617da73b7a9c5cdd888c0e26ae81d2c5c0d714
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6924
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-10 20:31:10 +00:00
Griffin Smith
0b8a967aca feat(tvix/eval): Implement builtins.pathExists
Change-Id: Ife8a690e9036868964771893ab29a9ae3a2d2365
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6919
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 20:23:41 +00:00
Griffin Smith
273ba73754 feat(tvix/eval): Initial resolution of <...> paths
This commit implements (lazy) resolution of `<...>` paths via either the
NIX_PATH environment variable, or the -I command-line flag - both
handled via EvalOptions. As a result, EvalOptions can no longer derive
Copy, meaning we have to clone it at each line of the repl - this is
probably not a huge deal as repl performance is not exactly an inner
loop and we're not cloning very much.

Internally, this works by creating a thunk which pushes a constant
containing the string inside the brackets to the stack, then a new
opcode to resolve that path via the `NixPath`. To get that opcode to
work, we now have to pass in the NixPath when constructing the VM.

This (intentionally) leaves out proper implementation of path resolution
via `findFile` (cppnix just calls whatever identifier called findFile is
in scope!!!) as that's widely considered a bit of a misfeature, but if
we do decide to implement that down the road it likely wouldn't be more
than a few extra ops within the thunk introduced here.

Change-Id: Ibc979b7e425b65cbe88599940520239a4a10cee2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6918
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 20:23:41 +00:00
Griffin Smith
5e2b44b416 feat(tvix/eval): Add a struct implementing NIX_PATH
Add a simple struct implementing both the string parsing and path
resolution rules of Nix's `NIX_PATH` environment variable, for use in
resolving `<...>`-style paths

Change-Id: Ife75f39aa5c12928278d81fe428fbadc98bac5cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6917
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2022-10-10 20:23:41 +00:00
Griffin Smith
0e9f5d6890 feat(tvix/eval): Allow adding strings to paths
Implement adding paths and strings via OpAdd. Since the nix rules are
quite obscure, I'm electing to test this one with an oracle test to
avoid the danger of getting the actual asserted result wrong.

Change-Id: Icdcca3690ca2e8459e386c1f29cc48eaaa39e9a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6914
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 20:23:41 +00:00
Griffin Smith
66a35de3b6 feat(tvix/eval): Implement builtins.concatStringsSep
Change-Id: I6e46bcdbf3b5258b60edb017709fee577eb8ec74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6907
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 20:23:41 +00:00
William Carroll
41ddc37725 feat(tvix/eval): Support builtins.readDir
Co-authored-by: Griffin Smith <root@gws.fyi>
Change-Id: I5ff19efbe87d8f571f22ab0480500505afa624c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6552
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 19:36:49 +00:00
Griffin Smith
899fbdbddb refactor(tvix/eval): Compile OpAssert using conditional jumps
In order to behave nicely with tryEval, asserts need to leave the
instruction pointer in a reasonable place even if they fail - whereas
with the previous implementation catching a failed assert would still
end up running the op for the *body* of the assert. With this change, we
compile asserts much more like an `if` expression with conditional jumps
rather than having an OpAssert op.

Change-Id: I1b266c3be90185c84000da6b1995ac3e6fd5471b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6925
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 17:51:22 +00:00
Griffin Smith
8190046190 feat(tvix/eval): Implement builtins.elem
Change-Id: Id99c1d33f87ad9866990d3483d3531e9e48f861f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6916
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 17:42:48 +00:00
Griffin Smith
0b06d94606 refactor(tvix/eval): Abstract away calling functions
The process of calling a function from a builtin, especially if it's got
more than 1 arrgument, is reasonably involved and easy to get wrong due
to having to interact directly with the stack - instead of having that
done entirely manually in builtins, this wraps it up in a new
`call_with` function which handles pushing arguments onto the stack and
recursively calling the (partially applied) function.

Change-Id: I14700c639a0deca53b9a060f6d70dbc7762e9007
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6910
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 15:34:31 +00:00
Griffin Smith
ba8ec1910b feat(tvix/eval): Implement builtins.foldl'
Change-Id: Ibc97db4343cb3a1a1677f69fb6c3518c61978aad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6906
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 15:26:29 +00:00
Griffin Smith
0e97555644 feat(tvix/eval): Implement builtins.genList
Change-Id: Iabe28656229f508226b244d81382e517961eb3cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6901
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-10 13:54:16 +00:00
Griffin Smith
b6089fb1e5 feat(tvix/eval): Implement builtins.concatMap
Change-Id: I08bfd040a242aa43b64760c19f48a28303f206ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6900
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-10 13:54:16 +00:00
Griffin Smith
afdf1e0ed0 feat(tvix/eval): Implement builtins.listToAttrs
Implement the listToAttrs builtin, which constructs an attribute set
from a list of attribute sets with keys name and value.

This is tested using an adaptation of the nix `eval-ok-listtoattrs.nix`,
with the utilities from `lib.nix` inlined.

Change-Id: Ib5bf743466dda9722c2c1e00797df4b58448cf0f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6894
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-09 22:18:53 +00:00
Griffin Smith
d0f571dcc0 refactor(tvix/eval): Use Display impl for Error message
This is generally more idiomatic (over just delegating to Debug), and
also allows us to avoid intermediate allocations if we ever end up
using error messages as part of larger strings (because we don't have to
allocate a full String for the return value).

Change-Id: I67e48b44570c72761ed0fcaded9ae4bf3fcbaacf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6896
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-09 17:12:41 +00:00
Griffin Smith
1e85209809 feat(tvix/eval): Handle invoking binary with a directory
Similar to what we do for import, push on a `default.nix` to the path
that the top-level is invoked with (if any) if it's a directory.

Change-Id: I281bd44e3c8803b6765c886ae5fd08f549e2e563
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6895
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-08 19:16:10 +00:00
Vincent Ambo
207f3dd47e fix(tvix/eval): end scope after compiling legacy let bindings
When contrasting the compilation of the desugared version to the
"sugared" version, this was the noticeable difference.

This fixes b/203.

Change-Id: Iae02ffc56e06de1de091b84cdc59d8fe83a17d69
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6898
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 19:10:09 +00:00
Griffin Smith
b4d978cd6a feat(tvix/tests): Import default.nix inside directory
This requires actually passing the source directory into `interpret` in
the eval tests, but otherwise this is fairly straightforward - if we're
trying to import a directory, just push `default.nix` onto it and import
that instead.

Change-Id: I0b7d4234f81977e78d14dfa651bf0cf9721017e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6893
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-08 19:02:25 +00:00
Griffin Smith
278bccc1ea refactor(tvix/eval): Encapsulate Value::Attrs construction
Factor out the construction of Value::Attrs (including the Rc) into a
new `attrs` constructor function, to abstract away the presence of the
Rc itself.

Change-Id: I42fd4c3841e1db368db999ddd651277ff995f025
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6892
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 19:02:25 +00:00
Griffin Smith
f6bcd11cad fix(tvix/eval): Force thunks when comparing against ground vals
Thunks correctly force when comparing for equality against other thunks,
but weren't being forced correctly when comparing against non-thunk
values, in either direction.

Change-Id: Ia03702895ec4d70aed3445c1b0a9a7a641d1a300
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6897
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 19:02:25 +00:00
Vincent Ambo
1677186144 feat(tvix/eval): add some slightly more descriptive span labels
Change-Id: I530c491f60a33fdb97e1553b193de51e7ee57d9a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6873
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 17:27:57 +00:00
Vincent Ambo
9b1a266197 feat(tvix/eval): chain error spans for thunk errors
Adds secondary spans for errors that occur deeply nested within a
thunk.

This is pretty raw right now, there's technically nothing stopping one
of these error chains from being a hundred thunks deep into code,
producing unmanageable error output. We should trim these down
according to some heuristics (e.g. when crossing file boundaries, o r
just - for starters - beginning and end).

Change-Id: Ia73892512737850b6fa3e07cabc37fa9c534c4d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6872
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 17:27:57 +00:00
Vincent Ambo
1e2d323a7c feat(tvix/eval): fancy-format parse errors returned by rnix
This change is quite verbose, so a little bit of explaining:

1. To correctly format parse errors, errors must be able to return
   more than one annotated span (the parser returns a list of errors
   for each span).

   To accomplish this, the structure of how the `Diagnostic` struct
   which formats an error is constructed has changed to delegate the
   creation of the `SpanLabel` vector to the kind of error.

2. The rnix structures don't have human-readable output formats by
   default, so some verbose methods for formatting them in
   human-readable ways have been added in the errors module. We might
   want to move these out into a submodule.

3. In many cases, the errors returned by rnix are a bit strange - so
   while we format them with all information that is easily available
   they may look weird or not necessarily help users. Consider this CL
   only a first step in the right direction.

Change-Id: Ie7dd74751af9e7ecb35d751f8b087aae5ae6e2e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6871
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 17:27:57 +00:00
Vincent Ambo
70113407d2 refactor(tvix/eval): implement ToSpan directly for rnix::TextRange
This logic was duplicated between the two rnix types before, but more
crucially - it is also needed for correctly displaying the text ranges
contained in syntax errors.

Change-Id: Ifc6a521de1594d6ced9cba6274f1931df99b6634
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6870
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-08 10:58:42 +00:00
Vincent Ambo
50baf0bcfc refactor(tvix/eval): move spans module to crate root
This is also useful for error-handling related logic, outside of just
the compiler module.

Change-Id: I5c386e2b4c31cda0a0209b31136ca07f00e39e45
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6869
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-08 10:58:42 +00:00
Vincent Ambo
2478a6c5ba feat(tvix/eval): coerce values to paths when importing
This enables the use of string paths (and, in the future,
derivations), as long as their string values represent an absolute
path.

Change-Id: I4b198efeb70415ed52f58bd1da6fa79a24dad14c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6866
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-07 14:24:36 +00:00
Vincent Ambo
07e03498f2 feat(tvix/eval): add method for emitting runtime warnings
This lets the VM emit warnings when it encounters situations that
should only be warned about at runtime.

For starters, this is used to pass through compilation warnings that
come up when `import` is used.

Change-Id: I0c4bc8c534d699999887c430d93629fadfa662c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6868
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-07 14:24:36 +00:00
Vincent Ambo
4b9178fa2a feat(tvix/eval): insert import into the builtins itself
Adding `import` to builtins causes causes a bootstrap cycle because
the `import` builtin needs to be initialised with the set of globals
before being inserted into the globals, which also must contain
itself.

To break out of the cycle this hack wraps the builtins passed to the
compiler in an `Rc` (probably sensible anyways, as they will end up
getting cloned a bunch), containing a RefCell which gives us mutable
access to the builtins.

This opens up a potentially dangerous footgun in which we could mutate
the builtins at runtime leading to different compiler invocations
seeing different builtins, so it'd be nice to have some kind of
"finalised" status for them or some such, but I'm not sure how to
represent that atm.

Change-Id: I25f8d4d2a7e8472d401c8ba2f4bbf9d86ab2abcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6867
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-07 14:24:36 +00:00
Vincent Ambo
b530e496a5 feat(tvix/eval): initial implementation of builtins.import
This adds an initial working version of builtins.import which
encapsulates the entire functionality of `import` within the builtin
itself, without requiring any changes in the compiler or VM.

The key insight that enables this is that we can simply return a Thunk
from `import` that is constructed from the output of running the
compiler and - ta-da! - no other component needs to know about it.

A couple of notes:

* builtins.import needs to capture variables like the SourceCode
  structure. This means it can not currently be constructed the same
  way as other builtins and has special handling, which leaks out to
  `eval.rs`. I have postponed dealing with that until we have this
  working a bit more.

* the `globals` are not yet passed through

* the error representation for the new variants is absolutely not done
  yet, we probably want to switch to something that supports
  cause-chaining now (like miette)

* there is no mechanism for emitting warnings at runtime; we need to
  add that

Change-Id: I3117a7ae3ff2432bf44f5ff05ad35f47faca31d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6857
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-06 15:22:32 +00:00
Vincent Ambo
44acffc688 refactor(tvix/eval): builtins now contain closures
For some upcoming builtins (notably, import) we need to capture
arguments in the builtin's implementation.

To allow this, we can no longer use function pointers for builtins,
but must use a reference-counted closure object instead.

Unfortunately this adds an extra pointer operation to every builtin
call. We should benchmark this later against having a split builtin
representation.

Change-Id: I109d98d0e25998870542f47573eb1ec2e546f2a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6856
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-06 15:22:32 +00:00
Vincent Ambo
7fe1331570 feat(tvix/eval): implement tvix's user-agent, err, nixVersion
nixpkgs has hardcoded references to Nix versions, we need to provide
it with something that looks like a Nix version while actually being a
Tvix version.

For now, we do this by stealing a trick out of the browser book and
constructing a version that looks like a Nix version to Nix, but like
a Tvix version to people who know what they are looking for.

Nevermind that we don't actually have any kind of versioning for
Tvix (yet?), other than depot revisions.

Change-Id: I7ce8079dd8164a2079891d38e707f09a45f0bbc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6858
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-05 13:07:55 +00:00
Vincent Ambo
3530404a4a refactor(tvix/eval): introduce source::SourceCode type
This type hides away the lower-level handling of most codemap data
structures, especially to library consumers (see corresponding changes
in tvixbolt).

This will help with implement `import` by giving us central control
over how the codemap works.

Change-Id: Ifcea36776879725871b30c518aeb96ab5fda035a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6855
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-10-05 10:29:47 +00:00
Vincent Ambo
2ff764ceb7 refactor(tvix/eval): remove unnecessary clones in compiler
There's basically nothing that needs *ownership* of an AST
node (which is just a little box full of references to other things
anyways), so we can thread this through as references all the way.

Change-Id: I35a1348a50c0e8e07d51dfc18847829379166fbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6853
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-04 21:27:30 +00:00
Vincent Ambo
b69b50feb1 refactor(tvix/eval): split observer traits in two
There are actually two different types of observers, the ones that
observe the compiler (and emitted chunks from different kinds of
expressions), and the ones that trace runtime execution.

Use of the NoOpObserver is unchanged, it simply implements both
traits.

Change-Id: I4277b82674c259ec55238a0de3bb1cdf5e21a258
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6852
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-04 21:27:30 +00:00
Vincent Ambo
d6643f66b1 feat(tvix/eval): use fancy error formatting in REPL
Change-Id: I8d5652d797b012bc3d5248ca2b2d0c87572dbf2e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6851
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-10-04 21:04:03 +00:00
Vincent Ambo
ee0657c292 fix(tvix/eval): forward thunk error codes from inner errors
Until we can display a chained representatino of errors in thunks, it
is most useful to forward the error code from the innermost error to
the user.

Change-Id: I8d67254d52313be40387f080e57966c001e0d51c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6854
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-04 13:13:20 +00:00
Vincent Ambo
89dc26cece feat(tvix/eval): implement builtins.currentTime
Returns time since epoch in seconds.

This has a slight behaviour difference from Nix, in that we don't pin
the time between REPL entries (Nix pins it for the program lifetime),
but this is probably inconsequential as long as it is pinned during an
evaluation.

Change-Id: I010c02e93097a209d8ad69e278397c7e30e54c86
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6846
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-10-04 08:03:50 +00:00
Vincent Ambo
f0179c92d3 refactor(tvix/eval): allow impure Value builtins
Allows impure builtins that have a different shape than a Rust
function pointer; specifically this is required for
builtins.currentTime which does not work in WASM.

Change-Id: I1362d8eeafe770ce4d1c5ebe4d119aeb0abb5c9b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6849
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-10-04 08:03:50 +00:00
Vincent Ambo
0537d88078 feat(tvix/eval): implement builtins.any
Change-Id: I640ee20e7c0a68c4e024a577e429fed9b3a49ece
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6845
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-10-03 17:39:09 +00:00
Vincent Ambo
844d288949 feat(tvix/eval): implement builtins.all
Change-Id: I19ec2b2194681efd73041f4aa1e5f2c893e839c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6844
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-10-03 17:39:09 +00:00
Vincent Ambo
939b2194fe feat(tvix/eval): implement builtins.concatLists
Concatenates (but not flattens) a list of lists.

Change-Id: I692e0b3e7b5a5ff93d5768d3a27849b432ec5747
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6843
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-03 07:53:21 +00:00
Vincent Ambo
3c9520a4e5 refactor(tvix/eval): implement IntoIterator for NixList
This is the same code as before, just moved into a trait impl to gain
access to stuff that needs IntoIterator

Change-Id: Iff9375cd05593dd2681fa85ccc7f4554bf944a02
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6842
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-03 07:53:21 +00:00
Vincent Ambo
8d95e35ced fix(tvix/eval): do not fail when finalising non-capturing values
This can actually legitimately be emitted by the compiler currently
when compiling formals with default values. See the scope6 test from
the Nix test suite for an example.

We should restructure this slightly to be able to reintroduce a
runtime error here in case something was compiled incorrectly.

Change-Id: Ib81f0f58ae0e850db9fbc459458b7bd0d3ac6f23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6841
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-03 07:50:41 +00:00
Vincent Ambo
8336577c41 feat(tvix/eval): implement tail-calling of __functor attributes
This implements __functor calling in situations where `OpTailCall` is
used, but not yet for `OpCall`.

For some reason I have not yet figured out, this same implementation
does not work in call_value, which means that it also doesn't yet work
in builtins that apply functions.

Change-Id: I378f9065ac53d4c05166a7d0151acb1f55c91579
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6826
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-03 07:50:41 +00:00
Griffin Smith
c1d8cee215 feat(tvix/eval): Add passthru build for benchmark binaries
Add a new derivation target to the passthru of tvix.eval that builds the
benchmark binaries, *and* copies them to the outupts of the derivation
via the (somewhat arcane) `copyBinsFilter` jq script arg to naersk. This
is a bit annoying because (as far as I can tell) the derivations
returned by naersk aren't directly overridable, so we have to explicitly
fixpoint the attrs we're passing.

Also, since this is now a separate target to build the benchmarks, we
can remove `--all-targets` from the build of `tvix-eval` itself since
that was only added to build benchmarks in CI, and make
regular (non-benchmark) builds a bit faster.

Change-Id: I136b8526790545e93b1ae666abaefb51cbbee390
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6847
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-10-02 22:43:34 +00:00
sterni
1e25ba1b09 docs(tvix/eval): start doc about problematic/weird lang behavior
The idea is that we can keep track of the more unexpected behavior,
behavior that maybe should not be a thing at all and behavior we are not
sure about yet.

Change-Id: I70933f00af1230a7ab9d30e917b61199fe571caf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6803
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-01 12:30:17 +00:00
Vincent Ambo
68a4d4a759 fix(tvix/eval): fix thunk borrowing error in force_for_output
This function previously kept a borrow in the form of the
`Thunk::value` result alive while performing arbitrary actions in the
VM, which caused a borrowing error in the test case attached.

The `Ref` value must never be used in cases where control flow is
passed to other parts of the VM.

Change-Id: I41d10aa1882a2166614b670e8ba77aab0e67deca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6825
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-30 12:31:14 +00:00
Vincent Ambo
746f1ee702 test(tvix/eval): add simple nested key tests in let & rec attrs
Change-Id: Icdc41ae383cd7cd1531c65c97d955bc59a56489a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6809
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-30 12:31:14 +00:00
Vincent Ambo
6c5e83f8bc chore(tvix/eval): remove unused field in TrackedBindings
Change-Id: I65e31e9173e4f5bba19cc4e3d45eb4f8bf91b424
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6808
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-30 12:31:14 +00:00
Vincent Ambo
f0f5435e25 test(tvix/eval): reactivate nested key tests in plain attribute sets
Change-Id: Id25cde5a6562e1598dcbbf7945ecdfacac5faed7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6807
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-30 12:31:14 +00:00
Vincent Ambo
da1e3e9ac5 feat(tvix/eval): implement nested keys
This finishes up the implementation of nested keys after the key
insight that the nesting level does not need to be tracked, and
instead the attribute iterator can simply be retained inside the
structures as is (in an advanced state).

With this implementation, when encountering a nested key, the Tvix
compiler will first analyse whether there is already a matching
binding that can be merged (i.e. a binding that is a literal attribute
set), and perform the merge, or otherwise create a new recursive set
of bindings in which the entry is inserted with the path iterator
advanced beyond the first name component.

With this, all the logic simply applies recursively until there are no
more nested bindings (i.e. until all iterators are "empty").

Note that this has one (potentially insignificant) deviation from Nix
currently: If a non-mergable value is supplied (e.g. `a.b = 1; a =
2;`), Tvix will emit a *runtime* error (whereas it is *parse* time in
Nix) as the branch which could statically analyse this is currently
unreachable. There's a TODO for this, so we can fix it up later.

Change-Id: I53df70e09614ff4281a70b80eac7da3beca12da9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6806
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-30 12:31:14 +00:00
Vincent Ambo
9cd5f03835 refactor(tvix/eval): split out AttributeSet::from_ast helper
Change-Id: Id43dbd06aef14cf01b4901d9b3668d790cd2b5ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6805
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-30 07:07:17 +00:00
Vincent Ambo
9e9dde0106 chore(tvix/eval): remove nesting_level tracking
This is actually quite useless, as we can just pass
`AstChildren<ast::Attr>` around after partially consuming it.

Change-Id: If0aefa2b53fc801fced1ae0709bff93966bf19f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6804
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-30 07:07:17 +00:00
Vincent Ambo
ccf9dd651b refactor(tvix/eval): clean up representation flip in bindings
When encountering a nested binding for the first time, cleanly flip
the representation to `Binding::Set` in `Binding::merge` before
proceeding with the actual merge.

This reduces the number of points where we have to deal with the (soon
to be slightly more complex) construction of the nested binding
representation.

Change-Id: Ifd43aac7b59ebd15a72c3ec512386a5bcf26ec13
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6802
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 17:46:02 +00:00
Vincent Ambo
18fcf0d79d feat(tvix/eval): (partially) track nesting level of attrsets
This adds the scaffolding required for tracking the nesting level (and
appropriately skipping the correct amount of attrpath entries when
inserting nested sets).

In order for all of this to work correctly, we can no longer track
`AttrpathValue` directly in the entries vector as rnix does not allow
us to construct values of that type - so instead we have to track its
inner components.

Change-Id: Icb18e105586bf6c247c2e66c302cde5609ad9789
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6801
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 17:46:02 +00:00
Vincent Ambo
cece9eae4a feat(tvix/eval): merge attribute sets in bindings
This is a significant step towards correctly implemented nested
bindings. All attribute sets defined within the same binding scope
will now be merged as in Nix, if they use the same key.

Change-Id: I13e056693d5e73192280043c6dd93b47d1306ed6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6800
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 17:46:02 +00:00
Vincent Ambo
0cee44838c feat(tvix/eval): add error kind for unmergeable nested attributes
Change-Id: Ic5e6d1bf2625c33938360affb0d1a7c922af11bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6799
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 17:46:02 +00:00
Vincent Ambo
3f34af205f feat(tvix/eval): add scaffolding for merging nested attribute sets
This sets up the required logic for finding and merging attribute sets
into nested bindings if they exist. This is absolutely not complete
yet and can, at this commit, probably cause undefined runtime
behaviour if nested attributes are specified.

The basic idea is that a new helper function on the `TrackedBindings`
struct is called with each encountered attribute and determines
whether the new entry can be merged into an existing attribute or not.

Right now the only effect this has in practice is that a new error
becomes available if somebody attempts to cause a merge into an
inherited key.

Change-Id: Id010df3605055eb1ad7fa65241055889dd21bab0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6798
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 17:46:02 +00:00
Vincent Ambo
09a57e7857 refactor(tvix/eval): emit OpAttrs inside of compile_bindings
This needs to move here so that we can reuse compile_bindings for the
nested attribute sets we're about to start constructing.

Change-Id: Ie83f52f7e1d128886e96a1da47792211fa826f21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6796
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00
Vincent Ambo
82df0b432a refactor(tvix/eval): introduce TrackedBindings struct
This struct will be the key to correctly compiling nested bindings, by
having insertions flow through some logic that will attempt to bind
attribute-set-like things when encountering them.

Change-Id: I8b5b20798de60688f3b6dc4526a460ebb2079f6e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6795
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00
Vincent Ambo
e96f94ac88 refactor(tvix/eval): compile_recursive_scope -> compile_bindings
Change-Id: Iff18d0f84ba2b7a4194797e6c52c55b1c37e419c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6794
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00
Vincent Ambo
e31f8f735f chore(tvix/eval): fix all current clippy lints
Change-Id: I28d6af8cb408f8427a75d30b9120aaa809a1ea40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6784
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00
Vincent Ambo
d54aeb1f20 chore(tvix/eval): remove existing nested key implementation
This implementation, which only ever worked for non-recursive
attribute sets, is no longer needed and thus removed here.

We have a new implementation of these nested keys coming up instead.

Change-Id: I0c2875154026a4f5f6e0aa038e465f54444bf721
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6783
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00
Vincent Ambo
3f21606278 refactor(tvix/eval): merge all bindings creation logic
As of this commit, all three types of bindings scopes are compiled the
same way (i.e. compilation of non-recursive attribute sets has been
switched over to the new code paths).

This sets us up for doing the final implementation of nested attribute
sets.

HOWEVER, this breaks the existing implementation of nested attributes
in non-recursive attribute sets. That implementation is flawed and
unworkable in practice, so we need to do this dance to be able to
implement it correctly.

Change-Id: Iba2545c0d1d6b51f5e1a31a5d005b8d01da546d3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6782
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00
Vincent Ambo
949897651e feat(tvix/eval): implement dynamic keys in recursive attrs
This wires up the new bindings setup logic to be able to thread
through & compile dynamic attributes in recursive attrs.

It seems like we don't actually need to retain the phasing of Nix
exactly, as we can use the phantom mechanism to declare all locals
without making the dynamic ones accessible.

Change-Id: Ic2d43dd8fd97d7ccd56d8c6adf2ff97274cd837a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6781
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00
Vincent Ambo
3d9eb14e7a feat(tvix/eval): add KeySlot::Dynamic variant for dynamic keys
Another slice of the salami, but no functionality changes yet (other
than opening a code path that can reach a `todo!()`, but this will be
removed soon).

Change-Id: I56b4ed323f70754ed1ab27964ee3c99cf3bf3292
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6780
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 00:48:33 +00:00
Vincent Ambo
d0636f1e24 refactor(tvix/eval): clean up unused attrpath normalisation logic
The previous way of sanitising dynamic keys is going away as we're
slowly introducing the new nested key logic.

While touching this stuff, I've also changed all the related string
types to SmolStr as that is more sensible for identifiers.

Change-Id: If30c74151508719d646d0e68e7d6f62c36f4d23f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6779
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 00:48:33 +00:00
Vincent Ambo
e253e5239d chore(tvix/eval): reflow comments in compiler::bindings
Change-Id: I6d74f71ecd671feaec96ee4ff39f218907c517fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6777
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 00:48:33 +00:00
Vincent Ambo
001e0520dc refactor(tvix/eval): merge inherits logic between all binding kinds
Removes the `compile_inherit_attrs` logic which was only used for
BindingsKind::Attrs (i.e. non-recursive attrs).

This brings us a step closer to fully merging all the binding logic
into one block that can dispatch based on the kind of bindings (and
thus giving us a good point to introduce the final logic for nested
bindings).

Change-Id: If48d7a9497fc084a5cc03a130c2a7da5e2b8ef0c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6776
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 00:48:33 +00:00
Vincent Ambo
1a4486c92d refactor(tvix/eval): Factor out declare_bindings helper
This helper is responsible for declaring the bindings in the
compiler's scope tracking structures.

It is almost equivalent to the previous logic, but also accounts for
`BindingsKind::Attrs` - though those code paths do not end up here
yet.

Change-Id: I44f617b99b10f2a7b9675f7b23e2c803a4a93d29
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6775
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 00:48:33 +00:00
sterni
0c178a0ef6 chore(3p/sources): Bump channels & overlays
Upstream nixpkgs removed a lot of aliases this time, so we needed to do
the following transformations. It's a real shame that aliases only
really become discoverable easily when they are removed.

* runCommandNoCC -> runCommand
* gmailieer -> lieer
  We also need to work around the fact that home-manager hasn't catched
  on to this rename.
* mysql -> mariadb
* pkgconfig -> pkg-config
  This also affects our Nix fork which needs to be bumped.
* prometheus_client -> prometheus-client
* rxvt_unicode -> rxvt-unicode-unwrapped
* nix-review -> nixpkgs-review
* oauth2_proxy -> oauth2-proxy

Additionally, some Go-related builders decided to drop support for
passing the sha256 hash in directly, so we need to use the generic hash
arguments.

Change-Id: I84aaa225ef18962937f8616a9ff064822f0d5dc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6792
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-09-28 08:02:31 +00:00
Vincent Ambo
f40283e098 refactor(tvix/eval): Factor out bind_values helper
This is responsible for actually setting up `TrackedBinding`s on the
stack, i.e. in some sense "actually compiling" values in bindings.

There is no functionality change to before, i.e. this is a salami
slice.

Change-Id: Idb0312038e004470a7d130c020ae0fe87c55c218
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6774
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-28 00:09:49 +00:00
Vincent Ambo
7e9169bcf7 refactor(tvix/eval): split compile_inherits into two
Splits the large `compile_inherits` function which
previously *compiled* plain inherits and *declared* namespaced
inherits into `compile_plain_inherits` and
`declare_namespaced_inherits`.

This is supposed to make more sense than before, but is still not
consistently used (notably, non-recursive attribute sets still
duplicate most of this logic).

Another salami slice.

Change-Id: Id97fac1cbd5ee97b24d047e7728655e6b7734153
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6773
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-28 00:09:49 +00:00
Vincent Ambo
d01786d888 refactor(tvix/eval): add non-recursive logic to compile_inherit
... but do not use it yet.

This refactoring is pretty complicated, so I'm applying salami-slicing
tactics here.

Change-Id: I66e04ee10548f68bf67dc842f3f14cc279426c22
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6772
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-28 00:09:49 +00:00
Vincent Ambo
14147e9b5f refactor(tvix/eval): introduce type to track kind of bindings
As part of the unification of binding logic between different carriers
of bindings, we need to track which kind of bindings we are dealing
with (attribute set? recursive scope? ...) to correctly emit keys and
declare identifiers in the locals stack.

Right now this changes no functionality as `BindingsKind::Attrs` is
not yet used (only RecAttrs and LetIn, which was previously
represented by the `rec_attrs` boolean).

Change-Id: Id2ac27894079ab584521cb568d75c124f7bf2403
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6771
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-28 00:09:49 +00:00
Vincent Ambo
6d60c22b2a refactor(tvix/eval): move recursive inherit logic into helper
This helper will gain the ability to compile both kinds of inherits,
but it is kind of tricky to get right so I am doing it in smaller
steps. Right now there is no change in functionality.

Change-Id: Ie990b88dd90a5e0f9fd79961ee09a6c83f2c872d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6770
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-28 00:09:49 +00:00
Vincent Ambo
dd2d45e1cd refactor(tvix/eval): rename BindingKind -> Binding
This just describes a binding, and we do need a good name for the kind
of binding*s*, which is going to be introduced soon.

Change-Id: I53900ee52da8a07dae8b918fa6a4cb308e627efb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6768
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-28 00:00:05 +00:00
Vincent Ambo
846215ae2b refactor(tvix/eval): generalise error variant for dynamic keys
Change-Id: I08f40b4b53652a519e76d6e8344c7c3fe10a0689
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6767
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-28 00:00:05 +00:00
Vincent Ambo
71a8db108d refactor(tvix/eval): bye compiler::attrs, hello compiler::bindings
Changes the module structure of the compiler to have a module
dedicated to the logic of setting up bindings. This logic is in the
process of being merged between attribute sets and `let`-expressions,
and the structure of the modules makes more sense when ecapsulating
that specifically.

(Other bits of code related to e.g. attribute sets are pretty
straightforward and can just live in the main compiler module).

Change-Id: I9469b73a7034e5b5f3bb211694d97260c4c9ef54
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6766
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-28 00:00:05 +00:00
Vincent Ambo
e7d24fd2b8 chore(tvix/eval): mark coerce_value_to_path as intentionally unused
This was added in preparation for some builtins that don't exist yet
and is producing some noise during compilation.

Change-Id: I51fb0d14c3edf0bd6d9a288d50e44dacf35166c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6769
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-23 17:37:52 +00:00
Vincent Ambo
f816813d41 feat(tvix/eval): implement 'builtins.filter'
This is a little ugly because the plain Iterator::filter method can
not be used (it does not support fallible primitives), so we need to
resort to an `Iterator::filter_map` and deal with the wrapping in
Options everywhere.

This prevents use of `?` which introduces the need for some matching,
but it's not *too* bad.

Change-Id: Ie2c3c0c9756c4c627176f64fb4e0054e717c26d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6765
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-23 09:55:14 +00:00
Vincent Ambo
1015f2f8e7 fix(tvix/eval): manually count entries in recursive scopes
The previous version had a bug where we assumed that the number of
entries in an attribute set AST node would be equivalent to the number
of entries in the runtime attribute set, but due to inherit nodes
containing a variable number of entries, this did not work out.

Fixes b/199

Change-Id: I6f7f7729f3512b297cf29a2e046302ca28477854
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6749
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 23:23:47 +00:00
sterni
bd9cda2af7 feat(tvix/eval): add builtins.stringLength
Fairly straightforward, only thing of note is that we coerce (weakly) to
string here as well.

Change-Id: I03b427e657e402f1f9eb0f795b689bbf5092aba1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6745
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-22 23:12:29 +00:00
sterni
64d3efcc2c fix(tvix/eval): handle thunks in arithmetic builtins
The simplest solution seems to be to pass references to arithmetic_op!()
which avoids the moving annoyance we had to deal with in the
builtins (no more popping!). We then use .force() to force the values
and dereference any Thunks (which arithmetic_op! doesn't do for us).

Change-Id: I0eb8ad60e80a0b3ba9d9f411e973ef8bcf136989
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6724
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-22 23:12:29 +00:00
Vincent Ambo
55459f02fc test(tvix/eval): add a test for accessing "strange" identifiers
Because `inherit` can take string identifiers, we can access
arbitrarily weird identifiers in scopes using it.

Change-Id: Ic868233221befa160538dd2ffaff1cc7f566585a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6748
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 23:07:14 +00:00
Vincent Ambo
c46025d520 fix(tvix/eval): support string identifiers in inherits
This updates rnix-parser to a version where inherits provide an
iterator over `ast::Attr` instead of `ast::Ident`, which mirrors the
behaviour of Nix (inherits can have (statically known) strings as
their identifiers).

This actually required some fairly significant code reshuffling in the
compiler, as there was an implicit assumption in many places that we
would have an `ast::Ident` node available when dealing with variable
access (which is then explicitly only not true in this case).

Change-Id: I12f1e786c0030c85107b1aa409bd49adb5465546
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6747
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 23:07:14 +00:00
sterni
ee0b89c402 test(tvix/eval): test word behavior in builtins.compareVersions
We delegate to Rust std's Ord trait, so we should at least do a sanity
check here.

Change-Id: I9596068f8ab3e51b79602de0bac79af6d558086e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6723
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-22 23:06:45 +00:00
William Carroll
890bbf9b1f feat(tvix/eval): Support builtins.lessThan
Extend and export the `cmp_op`, and this becomes trivial.

Change-Id: I9c93fa4db0f5a1fc8b56928ea144676f79247de1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6557
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 17:42:01 +00:00
Griffin Smith
9e16d70809 feat(tvix/eval): Sketch out impure builtins
Sketch out a new set of "impure" builtins, which supplement the existing
set of "pure" builtins but are gated behind a feature flag, which allows
them to be omitted by crates depending on tvix-eval that only want pure
evaluation, such as tvixbolt.

Change-Id: I2736017b5c9b4776bbba8758e108ec84887abd66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6655
Reviewed-by: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-22 01:36:44 +00:00
sterni
7bf1b4deea refactor(tvix/eval): drop empty Word version cmp rule
This was derived from

    else if (c1 == "" && n2) return true; // true implies c1 < n2

However, this has no effect since Word always looses out against Number
anyways and the `pre` rules are also unaffected by this change – since
this only affects comparison of an empty Word part with a Number.

Change-Id: Ia04e42ac726352b688c87674b0fdb355f06edbcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6722
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
7ea249969a test(tvix/eval): add more compareVersions test cases involving pre
This asserts the not-quite lexicographical property of the comparison.

Change-Id: Iad68081e4b3a7106513f479643de87065dc47739
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6721
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
367a6a13d4 test(tvix/eval): add more compareVersions cases from C++ Nix
Change-Id: I25e7e7a2c547d0874e1e949bf96e6e066b1075ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6705
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
083fc1dbe5 fix(tvix/eval): compare versions with an extra empty component
This is necessary because builtins.compareVersions compares versions in
a subtly not-quite-but-still-lexicographical way: `pre` for example can
have an effect if it is post-fixed: `2.3 < 2.3pre`. This is a violation
of the rule that in a lexicographical ordering, the longer string is
considered greater if they are otherwise equal. builtins.compareVersion
is comparing lexicographically though, if you do the following
transformation beforehand:

  2.3 --split--> [ "2" "3" ] --append--> [ "2" "3" "" ]
  2.3pre --split--> [ "2" "3" "pre" ] --append--> [ "2" "3" "pre" "" ]

Comparing the transformed version is then done lexicographically:

  2.3 < 2.3.0pre since [ "2" "3" "" ] < [ "2" "3" "0" "pre" ]

Here, the `pre` rule never comes into effect because no comparison on it
happens, instead we use the longer string rule of a lexicographical
comparison.

In the C++ codebase, the reason for this behavior is that the
iterator-esque construct they use always yields the empty string before
it exposes it has been fully consumed. This is probably intentional to
support the postfixed `pre` which is, for example, used by NixOS
versions (e.g. unstable post 22.05 is 22.11-pre). We replicate this
behavior using the `Chain` iterator in `VersionPartsIter::new_for_cmp`.

Change-Id: I021c69aa27b0b7deb949dffe50ed18b6de3a7b1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6720
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
9a8a6a33f9 fix(tvix/eval): implement C++ Nix version part comparison algorithm
This is based on the [relevant code] in C++ Nix. Our version has more
branches because the C++ one only checks if it is less than or not, so
can save handling a few cases. We on the other hand, can avoid calling
the algorithm twice. It'd be nice to implement proptests for this in the
future and to make sure that this weird little algorithm doesn't violate
the Ord laws.

[relevant code]: cd35bbbeef/src/libstore/names.cc (L81-L94)

Change-Id: I46642e6da5eac7c0883cdce860622cdba04cd12b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6719
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
e9e06b8bae test(tvix/eval): add test cases for builtins.splitVersion
Change-Id: I60648f6c5cea34d19afb5ced3e98051201190a78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6711
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-21 13:08:07 +00:00
sterni
834fe833e5 fix(tvix/eval/versions): preserve the Number string exactly
This is relevant for builtins.splitVersion:

    nix-repl> builtins.splitVersion "unstable-2022-02-21"
    [ "unstable" "2022" "02" "21" ]

Change-Id: I0a0add178d95d5a82e112b41ed5f3ca5a19608f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6710
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-21 13:08:07 +00:00
sterni
6e6edcce6a fix(tvix/eval/versions): dash is a divider, not underscore
https: //github.com/NixOS/nix/blob/cd35bbbeef72375873e396b9ffed14a4638693a8/src/libstore/names.cc#L63
Change-Id: I1d5aba6060d11778f3b79089f4b27ef8849e4d37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6709
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-21 13:08:07 +00:00
sterni
0096939bf6 feat(tvix/eval): implement builtins.splitVersion
This was fairly easy, thanks to the work already done by Thomas Frank.
However, the implementation is suboptimal because we parse number parts
only to convert them back to strings afterwards. I've chosen to tackle
this problem in the future, since having an (inefficient) implementation
of splitVersion will be helpful for debugging the slight discrepancies
between C++ Nix and Tvix in the case of compareVersions.

Change-Id: Id6ed8eeb77663ff650c8c53ea952875b1fb7ea84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6688
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-21 13:08:07 +00:00
Vincent Ambo
489395448f feat(tvix/eval): track other type in NotCallable error kind
This makes for slightly nicer error messages if something isn't, well,
callable.

Change-Id: I821c8d7447b93aea9ccaaa52ed329de0cca4b18e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6718
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 23:48:57 +00:00
Vincent Ambo
8f2004d360 refactor(tvix/eval): add VM::call_value helper method
This makes it possible to call a callable value (builtin or
closure/lambda) directly, without unwrapping it first. This is needed
for pretty much all higher-order functions to work correctly.

This is mostly equivalent to the previous code in coerce_to_string for
calling `__toString`, except it expects the argument(s) to already be
placed on the stack.

Note that the span for the `NotCallable` error is not currently
guaranteed to make any sense, will experiment with this.

Change-Id: I821224368d438a28900858b343defc1817e46a0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6717
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 23:48:57 +00:00
Griffin Smith
f600aa5322 test(tvix/eval): Make proptests a bit smaller
Generate smaller recursive values for generated Values, and run fewer
cases for the attrs proptests which are particularly egregious.

Change-Id: Ia35c7c120270feaf045be1deb440c87ebb185c27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6716
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-20 23:41:58 +00:00
Vincent Ambo
876c477256 feat(tvix/eval): implement builtins.map
As we already have a VM passed to the builtins, we can simply execute
the provided closure/lambda in it for each value.

The primary annoyance with this is that we have to clone the upvalues
for each element, but we can try making this cheaper in the
future (it's also a general problem in the VM itself).

Change-Id: I5bcf56d58c509c0eb081e7cf52f6093216451ce4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6714
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 23:37:53 +00:00
Vincent Ambo
8207977648 chore(tvix/eval): clean up this one weird trick ...
... Rust developers don't want you to know about!

Change-Id: Ic43b670ac0982c726bfb8cd27cb57b17934e4b70
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6715
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2022-09-20 23:18:53 +00:00
Griffin Smith
ce9dfad6cb refactor(tvix/eval): Define a Compiler::new function
Change-Id: I6b9283d16447c83dd3978371d9a6ac1beb985926
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6657
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-20 21:19:41 +00:00
sterni
be18df1dab test(tvix/eval): assert flipped compareVersions behaves correctly
Change-Id: I7c239340de5c83a3124dc6a8ba0c70290466966d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6698
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-20 10:59:38 +00:00
sterni
7506983ce2 fix(tvix/eval): force condition of an assert
Change-Id: I3ad2234e8a8e4280e498c6d7af8ea0733ed4c7ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6699
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-20 10:59:38 +00:00
sterni
e04ccc9354 fix(tvix/eval): make sure to deref thunk in type predicate builtins
Previously we only matched the outer constructor after forcing which
would mean that we would always return `false` if the inspected value
was a thunk, regardless what value would be present inside.

Change-Id: I361ea6e855e23ef8e5b59098a50b9cd59253803f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6692
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-20 10:59:38 +00:00
sterni
cd280e0796 fix(tvix/eval): make sure to force before selecting in catAttrs
Previously, this would almost always crash because list items are
thunked more often nowadays and selecting from a thunk would fail. Also
we no longer pop from args, accessing it by index should avoid an
unnecessary clone here.

Change-Id: I4410c4c2e28cc255a2c7cf2a5322db3d2c556a0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6693
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-20 09:59:57 +00:00
sterni
151f4437bc refactor(tvix/eval/builtins): avoid unnecessary popping
Change-Id: I631b442e19e5c05455d705291c11037eae9ed9e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6694
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-20 09:44:48 +00:00
sterni
a01524823a refactor(tvix/eval): don't unnecessarily force in toString
Change-Id: I2bcc720ce7b6aa67ea5f145b1f2381a3ae833ac5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6691
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-20 09:42:57 +00:00
William Carroll
61cce49a8a feat(tvix/eval): Support builtins.getAttr
Support looking-up values from attrsets by their keys.

Change-Id: Ib37a472a511dab145f99ebc849879b3494e8e89f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6554
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-19 16:29:12 +00:00
William Carroll
059f4b964f feat(tvix/eval): Support builtins.hasAttr
See unit tests for examples :)

Change-Id: Ieec51d780a7762cc455ca03a9dc1648a0711924a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6553
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-19 16:18:10 +00:00
Luke Granger-Brown
f190712b7f chore(gerrit): migrate OWNERS files to code-owners style
Change-Id: Iacc521dfdd4b4a2d5cef3920cf8189bcce35a488
2022-09-19 11:13:28 +00:00
sterni
f358be4a1d fix(tvix/eval): move compareVersions test so cargo test finds them
And insert the missing newline the C++ Nix test script needs.

Change-Id: I04ddd7268f9caa1414fd23314c281bb7c1e854cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6689
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-19 09:58:14 +00:00
sterni
4bf38df09c fix(tvix/eval): fix flipped sign in builtins.compareVersions
Change-Id: I42fc0dc633fbd525eaa35689a41ec3717f4352a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6690
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 09:58:14 +00:00
sterni
b79e248959 refactor(tvix/eval): handle forcing in Builtin::apply
Instead of arity, we pass a array reference to Builtin::new that
describes how many arguments there are and which of them need to be
forced, eliminating the need to force manually.

Note that this change doesn't fix some of the instances where the the
Builtin doesn't consider that the value could be a Thunk.

Change-Id: Iadb58bb79886c30dc6b09dcf0ffad8abf28036a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6662
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 09:58:14 +00:00
William Carroll
fefa8c55c4 feat(tvix/eval): Support builtins.tail
TL;DR:
- support `builtins.tail`
- define `ErrorKind::TailEmptyList` and canonical error code
- support basic unit tests

Unsure whether or not the error should be a dedicated `ErrorKind`...

Change-Id: Iae90fda1bb21ce7bdb1aaa2aeb2b8c1e6dcb0f05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6545
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 00:57:11 +00:00
William Carroll
2fe18e4486 feat(tvix/eval): Support builtins.substring
Nix's `builtins.substring`:
- doesn't accept negative indexes for `beg` or `end`. Compare the error messages
  for:
  - `builtins.substring -3  5 "testing"`
  - `builtins.substring  3 -5 "testing"`
  - `builtins.substring -3 -5 "testing"`
- returns an empty string when `beg >= end`
- allows `end` to exceed the length of the input string

Change-Id: I83af7a099d81b6d537ebe5029d946c7031cb7113
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6555
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 00:51:10 +00:00
sterni
0c75ce2d3d chore(tvix/eval): add rustfmt to the shell
Change-Id: Ia049bf08feeb1a75f30df8fed7f88fcdf650521b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6663
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-18 23:19:54 +00:00
Griffin Smith
69cbcc1eda refactor(tvix/eval): Simplify forcing in builtins
Refactor the `force!` macro to a method on `Value` which returns a
smart-pointer-esque type, which simplifies the callsite and eliminates
rightward drift, especially for high-arity builtins.

Change-Id: I97a7837580accfb4bbd03b24f2acdbd38645efa5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6656
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-18 22:33:19 +00:00
Vincent Ambo
bcbe1603c8 fix(tvix/eval): ensure all thunks are forced in nested selects
Previously only the first one was guaranteed to be forced, but we need
to do this for all of them.

Fixes b/190

Change-Id: I76b5667dbfb2f3fde3587e7b91d268cbf32aca00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6645
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2022-09-18 22:16:04 +00:00
Griffin Smith
d42d8f3089 docs(tvix/eval): Document OpAttrs
Document the OpAttrs op, paying special attention to the (perhaps
confusing) behavior of taking the number of *pairs*, not the number
of *values*, which will be popped off the stack into the resulting attr
set.

Change-Id: I64df0290308ecae7a5c7e14ead37091d32701507
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6654
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:13:16 +00:00
Griffin Smith
e720545e5b refactor(tvix/eval): use Clap for arg+env parsing
Refactor the environment variable and argument parsing for the tvix repl
to use Clap instead of doing things ad-hoc, and thread through options
obtained from environment variables via explicit arguments rather than
obtaining them from the environment as they're needed. This makes adding
more flags more sustainable, and also makes the binary fully
self-documenting, including supported env vars, via `--help`.

Change-Id: Ib1f6a0cd20056e8c9196760ff755fa5729667760
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6653
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:08:43 +00:00
Griffin Smith
6f70f32513 fix(tvix/eval): Force thunks during equality comparison
Thunks might be encountered deep in equality comparison (eg nested
inside a list or attr-set), at which point we need to force them in
order to compare them for equality (or else we panic when trying to get
at their value).

Fixes: b/192
Change-Id: I912151085f8298f30d5214c7965251c9266443f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6652
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:08:43 +00:00
Griffin Smith
0b76ed5615 chore(tvix/eval): Pass in VM to nix_eq
Pass in, but ignore, a mutable reference to the VM to the `nix_eq`
functions, in preparation for using that VM to force thunks during
comparison.

Change-Id: I565435d8dfb33768f930fdb5a6b0fb1365d7e161
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6651
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:03:42 +00:00
Griffin Smith
915ff5ac2a refactor(tvix/eval): Don't (ab)use PartialEq for Nix equality
Using rust's PartialEq trait to implement Nix equality semantics is
reasonably fraught with peril, both because the actual laws are
different than what nix expects, and (more importantly) because certain
things actually require extra context to compare for equality (for
example, thunks need to be forced). This converts the manual PartialEq
impl for Value (and all its descendants) to a *derived* PartialEq
impl (which requires a lot of extra PartialEq derives on miscellanious
other types within the codebase), and converts the previous
nix-semantics equality comparison into a new `nix_eq` method. This
returns an EvalResult, even though it can't currently return an error,
to allow it to fail when eg forcing thunks (which it will do soon).

Since the PartialEq impls for Value and NixAttrs are now quite boring,
this converts the generated proptests for those into handwritten ones
that cover `nix_eq` instead

Change-Id: If3da7171f88c22eda5b7a60030d8b00c3b76f672
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6650
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:03:41 +00:00
Vincent Ambo
0e5baae7ad refactor(tvix/eval): clone the Arc<codemap::File> for the compiler
This disconnects ownership of the `File` reference in a compiler from
the calling scope, which is required for when we implement `import`.

`import` will need to carry an `Rc<RefCell<CodeMap>>` (or maybe, in
the future, Arc) to give us the ability to add new detected code
files at runtime.

Note that the choice of `Arc` over `Rc` here is not ours - it's the
codemap crate's.

Change-Id: I3aeca4ffc167acbd1701846a332d93550b56ba7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6630
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-18 21:12:50 +00:00
Griffin Smith
5dd5c7e254 test(tvix/eval): Add Eq-laws tests for NixAttrs
As before, this limits the cases to a relatively small number because
otherwise things get quite large.

Change-Id: I5371dc56418fca52e1dd1d905b20868f647091ba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6649
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:57:09 +00:00
Griffin Smith
c7faba0c8e test(tvix/eval): Add tests for the Eq laws of Value
Only running 20 cases for now, since Value can get quite big if you let
it run for a while.

Change-Id: I09ef19da22c789c4869793836c98937c44595340
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6648
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:57:09 +00:00
Griffin Smith
96d869923a fix(tvix/value): Properly match on Path for PartialEq
this was found by proptests!

Change-Id: I16d6a6ece3b20cdddd6f78c94cc87befb1b651e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6647
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:55:08 +00:00
Griffin Smith
78d19ff3e9 test(tvix/eval): Allow passing ProptestConfig to generated props
Add an optional config argument to the `<trait>_laws` macros, to allow
configuring the generated tests with a ProptestConfig struct (to limit
the number of cases run)

Change-Id: I2143ddb72c6a870e8be4a9058135b6f9a703039e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6646
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:55:07 +00:00
Griffin Smith
f8b3806720 test(tvix/eval): impl Arbitrary for Value
Impl Arbitrary for Value (and NixAttrs and NixList) in the same way we
did for NixString. Value currently only generates non-"internal"
values (no thunks, AttrNotFound, etc.) and can't generate
functions (builtins or closures), because those'd require full
generation of tvix bytecode, which is a bit more work than I'd like to
do now - there's a `todo!` left in the code for a place where we could
allow opting-in to internal values and functions later.

Change-Id: I07a59e2b1d89cfaa912d4ecebd642caf4ddb040a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6627
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:55:07 +00:00
Griffin Smith
221d3b9485 test(tvix/eval): Add proof-of-concept test for Chunk
This is pretty boring at the moment, but mostly serves as a foot in the
door in the direction of writing more tests

Change-Id: Id88eb4ec7e53ebb2d5b5c254c8f45ff750238811
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6637
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:55:06 +00:00
Griffin Smith
d431aa7743 fix(tvix/eval): Emit errors for invalid integers
Invalid integers (eg integers that're too long) end up as error returns
on the `.value()` returned from the literal in the AST - previously we'd
unwrap this error, causing it to panic the compiler, but now we've got a
nice error variant for it (which just unwraps the underlying
std::num::ParseIntError).

Change-Id: I50c3c5ba89407d86659e20d8991b9658415f39a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6635
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-18 16:40:14 +00:00
Vincent Ambo
b3cc4c8c2c chore(tvix/eval): bump rnix-parser to latest HEAD
This bumps rnix-parser to a commit that should be unaffected by the
Nix >= 2.4 bug that prevents it from cloning repositories with filters.

Change-Id: Ie01da95245ec6740fa889eb710819e512202f665
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6634
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-18 16:24:40 +00:00
sterni
a798efc77b docs(tvix/eval): note C++ implementation details for C++ Nix
Just want to note those down somewhere before I forget them again,
we can delete them later if we have it all figured out.

Change-Id: Icafa2d8fc7ca39e38e9637b7eca6f2bbf487c2b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6632
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 15:01:34 +00:00
Griffin Smith
3935c34401 test(tvix/eval): Add proptests covering trait impls for String
Add a suite of proptests covering the laws of the handwritten stdlib
trait impls (Eq, Ord, and Hash) for String, generated from a new set of
macros for generating those tests which can be applied to other types.

Change-Id: Ib3276c9e96fca497aece094e5612707d3dc77ccd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6626
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-17 19:51:16 +00:00
Griffin Smith
51f1924f19 test(tvix/eval): Test StringRepr::Smol as well
The From<String> impl for NixString only generates StringRepr::Heap
strings, but we want to make sure we're testing StringRepr::Smol too

Change-Id: I6d04b9cf12ef8462fe2788e0c6414b165f40311d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6629
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-17 19:51:15 +00:00
Griffin Smith
e6fe7b1687 test(tvix/eval): impl Arbitrary for NixString
Change-Id: I3fe2d410c789429493a1278d571ca8fe74c2a69d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6625
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-17 19:04:19 +00:00
Griffin Smith
84def7ef19 chore(tvix/eval): Add deps on proptest+test_strategy
It's proptest time!

Add deps on the proptest and test_strategy crates, gated behind a
new (default-enabled) "arbitrary" feature flag so that they don't affect
dependencies of things like tvixbolt that depend on tvix.

These are going in dependencies, not dev-dependencies, so that we can
impl Arbitrary for stuff outside of test modules (which will be
important for integration suites which want to run proptests)

Change-Id: I1613bd3ea9a835e22986ad4e59700e8736007963
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6624
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-17 19:02:59 +00:00
Vincent Ambo
da9e885e06 refactor(tvix/eval): rename OpAttrsIsSet -> OpHasAttr
This matches the name of the AST node from which it was compiled.
Suggested by sterni in cl/6231

Change-Id: Ia51525158d2f47467c01fce2282005b1a8417a47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6623
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-17 18:58:42 +00:00
Vincent Ambo
337d626f00 refactor(tvix/eval): clean up implementation of compile_literal
Suggested by sterni in cl/6231

Change-Id: I58bbc8a922d360ea79a4dacb76cf8aa1fad93757
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6622
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 18:58:42 +00:00
Vincent Ambo
2f91543a42 test(tvix/eval): add a test for lazy evaluation of with namespace
Suggested by grfn in cl/6222

Change-Id: I8413af6404786bd43e5cb462a307cc236c4fb9ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6621
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 18:58:42 +00:00
sterni
65dea58ef8 chore(tvix/eval): add simple .envrc / shell.nix
Change-Id: I10a44b92391a8ac4d7ec5e4b878953c215f59908
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6620
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-17 17:31:08 +00:00
Griffin Smith
67e796b2e1 test(tvix/eval): Add the start of a nix oracle test suite
Add the start of a test suite that compares tvix eval results against
nix, using the string repr of the value as the comparison. This shells
out to a nix-instantiate binary, which is configurable as an environment
variable, to eval - there's some extra machinery there to setup a new
nix store as a tempdir to allow running this test inside the nix build
for tvix-eval itself.

Currently this has a macro that'll allow writing lots and lots of
hardcoded tests, but going forward I'm also going to be looking into
adding proptest-based generation of expressions to compare.

Change-Id: I9f4895fab1e668ed2b7dfd6f92f8c80de1bbb16b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6307
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-17 17:23:55 +00:00
ThoFrank
ef80d00b06 feat(tvix_eval): Support builtins.compareVersions
Added an Iterator over &str wich yields the VersionParts.

Change-Id: I8043d423127446a173d01d290aab10de0c24a6fc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6619
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-17 14:48:58 +00:00
Vincent Ambo
1ee0f670b9 refactor(tvix/eval): use new ToSpan trait wherever possible
... it would be nice if we could thread it through to the `Scope`
stuff (declaring phantoms & locals).

Change-Id: Id3b84e79032b8fbb12138b719e657565355fbc79
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6616
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 14:11:19 +00:00
Vincent Ambo
337d670d0c feat(tvix/eval): introduce ToSpan trait in compiler module
This trait can be used to convert most structures from rnix-parser
into a codemap::Span. It uses a macro to implement the trait for the
various expression types in the rnix AST, as Rust's silly semantic
versioning restriction stops us from doing a blanket implementation.

This will be used in the next commit to clean up the span handling in
the compiler a bit.

Change-Id: I0a437034e5fa203b5a49c6f25c45932a9f3b2bca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6615
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 14:11:19 +00:00
William Carroll
3a67a14009 feat(tvix/eval): Support builtins.elemAt
(Attempt to) index into a list.

Change-Id: I3592e60a79e64d265e34100d4062041b0b410e00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6551
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-17 12:54:14 +00:00
sterni
53fbc75df9 refactor(tvix/eval): fix current clippy lints
Change-Id: I88482453a62955515a0dcc0b243351b2bbac5236
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6618
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-16 22:58:24 +00:00
Vincent Ambo
0532cb6172 test(tvix/eval): add tests for legacy let syntax
Change-Id: I42d6240056c93cf90986ef3fec88b42f804b06ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6613
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:13:26 +00:00
Vincent Ambo
15c8e8c459 test(tvix/eval): add some tests for recursive attribute sets
Change-Id: I6a4fbe6d5aa6d2ce223bf77330d6c0f512d4300d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6612
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:13:26 +00:00
Vincent Ambo
e86acd3212 feat(tvix/eval): implement legacy let syntax
... and emit a warning if anyone decides to use.

Change-Id: Iaa6fe9fa932340e6d0fa9f357155e78823702576
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6611
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-16 18:13:26 +00:00
Vincent Ambo
20230e1e2d feat(tvix/eval): implement recursive attribute sets
Yep.

This is kind of ugly right now. The idea is that the recursive_scope
compilation function is used for recursive sets as well by emitting
the keys. The rest of the logic is pretty much identical.

There is quite a lot of code here that can be cleaned up (duplication
between attrs and let, duplication inside of the recursive scope
compilation function etc.), but I'd like to get it working first and
then make it nice.

Note that nested keys are *not* supported yet.

Change-Id: I45c7cdd5f0e1d35fd94797093904740af3a97134
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6610
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:12:13 +00:00
Vincent Ambo
a7e280ec00 refactor(tvix/eval): introduce a type for tracking bindings
This type is used in the list temporarily populated by the *second*
pass over all identifiers in a recursive scope. This first pass only
serves to make all bindings known to the compiler, without populating
their values yet.

Having a type here is going to be useful once we implement `rec`,
which needs to thread through slightly more information.

Change-Id: Ie33e0f096c5fcb6c864c991255466748b6f0d1eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6609
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:12:13 +00:00
Vincent Ambo
1cf07051cb refactor(tvix/eval): extract recursive scope logic into a helper
This needs to be reused between let & `rec` attrs.

Change-Id: I4a3bb90af4be32771b0f9e405c19370e105c0fef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6608
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:02:33 +00:00
Vincent Ambo
01a239c955 refactor(tvix/eval): move compile_inherit_attrs to compiler::attrs
Plain move, no other changes.

Change-Id: Ic4f89709f5c2cbc03182a848af080c820e39a0fd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6607
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:02:33 +00:00
Vincent Ambo
6cc9d298aa refactor(tvix/eval): explicitly construct attrs in phases
This makes the phases of attribute set construction that Nix has very
explicit (inherits, static keys, dynamic keys).

This change focuses on the split between dynamic/static keys by
collecting all dynamic ones while compiling the static ones, and then
phasing them in afterwards. It's possible we also need to do some
additional splitting inside of the inherits.

Change-Id: Icae782e2a5c106e3ce0831dda47ed81c923c0a42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6530
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:02:33 +00:00
Florian Klink
d8f667e01e docs(tvix): Update components.md
This extends various ideas around the store and its slightly different
internal model.

Thanks to store composition, lazy substitution and more efficient
transfer, we can also simplify some complicated logic while building.

Change-Id: Ib3380af650fe06e114f54e8dc2df231f18af876b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6585
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-16 11:55:57 +00:00
William Carroll
f859751c6a feat(tvix/eval): Support builtins.bitXor
Bitwise "exclusive-or" on integers.

Change-Id: I90a0a15afb3a58662d70e82ea14e48b877476e04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6550
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:53:10 +00:00
William Carroll
031ca653a3 feat(tvix/eval): Support builtins.bitOr
Bitwise `or` on integers.

Change-Id: I45b0897331d1a9b6840f9d0feedcf10acc67fcec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6549
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:47:09 +00:00
William Carroll
f5f6bd319d feat(tvix/eval): Support builtins.bitAnd
Bitwise `and` on integers.

Change-Id: I9f2a9182a057af26906683acd97a40dfabbdded8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6548
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:41:08 +00:00
William Carroll
5a6d9ee1a0 feat(tvix/eval): Support builtins.attrValues
:)

Change-Id: Idf170b506ed6fab035b1e6f61055fee02e5c9be8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6547
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-15 21:13:03 +00:00
sterni
067f2b16f6 feat(tvix/eval): implement Value::coerce_to_path()
This function is necessary for all builtins that expect some form of
path as an argument. It is merely a wrapper around coerce_to_string that
can shortcut if we already have a path. The absolute path check is done
in the same way as in C++ Nix for compatibility, although it should
probably be revised in the long term (think about Windows, for example).

Since coercing to a path is not an operation possible in the language
directly, this function can live in the builtins module as the only
place it is required.

Change-Id: I69ed5455c00d193fea88b8fa83e28907a761cab5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6574
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 20:58:00 +00:00
William Carroll
e834a2cbc4 feat(tvix/eval): Support builtins.attrNames
Define `.len()` method on `NixAttrs` to preallocate the capacity of the result
vector.

Also anchor an errant comment to its context (I think).

Change-Id: I268f15025d453d7b3ae1146558c80e51433dd2a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6546
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-15 20:37:57 +00:00
Vincent Ambo
04503cf063 chore(tvix): add grfn & sterni to OWNERS
Change-Id: Ieb7647a571332f49f896b5c4a86784d8ea31a5d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6605
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 18:44:41 +00:00
William Carroll
85b3f17007 feat(tvix/eval): Support builtins.head
TL;DR:
- support `builtins.head`
- define `ErrorKind::IndexOutOfBounds` and canonical error code
- support basic unit tests

Change-Id: I859107ffb4e220cba1be8c2ac41d1913dcca37ff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6544
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 16:48:24 +00:00
sterni
0595870341 refactor(tvix/eval): don't move parts Vec in compile_str_parts
This allows us to get rid of the count local variable which was a bit
confusing. Calling parts.len() multiple times is fine, since the length
doesn't need to be computed.

Change-Id: I4f626729ad1bf23a93cb701385c3f4b50c57456d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6584
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 15:52:53 +00:00
sterni
4eb33e82ff fix(tvix/eval): coerce string interpolation parts to string
With this puzzle piece of string compilation in place, `compile_str`
becomes less redundant, as every part now needs to be compiled the same.
The thunking logic becomes a bit trickier, since we need to thunk even
in the case of `count == 1` if the single part is interpolating.
Splitting the inner (shared) code in a separate function turned out to
be easier for making rustc content.

Change-Id: I6a554ca599926ae5907d7acffce349c9616f568f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6582
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 15:52:53 +00:00
sterni
bcd7e520f0 fix(tvix/eval): thunk string interpolation
If we have multiple string parts, we need to thunk assembling the
string. If we have a single literal, it is strict (like all literals),
but a single interpolation part may compile to a thunk, depending on how
the expression inside is compiled – we can avoid forcing to early here
compared to the previous behavior.

Note that this CL retains the bug that `"${x}"` is erroneously
translated to `x`, implying e.g. `"${12}" == 12`.

The use of `parts.len()` is unproblematic, since normalized_parts()
builds a `Vec` instead of returning an iterator.

Change-Id: I3aecbfefef65cc627b1b8a65be27cbaeada3582b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6580
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 15:52:53 +00:00
sterni
da1d71a4e8 feat(tvix/eval): implement correct toString behavior
Implement C++ Nix's `EvalState::coerceToString` minus some of the Path
/ store handling. This is currently only used for `toString` which does
all possible coercions, but we've already prepared the weaker coercion
variant which is e.g. used for builtins that expect string arguments.

`EvalState::coerceToPath` is still missing for builtins that need a
path, but it'll be easy to build on top of this.

Change-Id: I78d15576b18921791d04b6b1e964b951fdef22c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6571
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 11:47:30 +00:00
sterni
162e21f2bb fix(tvix/eval): force left argument of ? before checking for attrs
OpAttrsIsSet and OpAttrsTrySelect will fail silently if the attribute
set value on the stack is actually a thunk, so we need to make sure to
force at every step of the way.

Emitting the force instructions in the compiler because it is easier to
add, but maybe the VM should do this when handling the relevant opcodes?
Comments welcome.

Change-Id: I65c5ef348d59b2d07c9bb06abb24f9f3e6a0fdb2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6540
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-13 22:06:29 +00:00