Commit graph

139 commits

Author SHA1 Message Date
sterni
64bb501de1 fix(tvix): distinguish search- and relative path resolution errors
Failures to resolve a nix search path lookup in angle brackets can be
caught using tryEval (if it reaches the runtime). Resolving relative
paths (either to the current directory or the current user's home) can
never be caught, even if they happen inside a thunk at runtime (which is
currently the case for home-relative paths).

Change-Id: I7f73221df66d82a381dd4063358906257826995a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7025
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-21 00:11:29 +00:00
Adam Joseph
d978b556e6 feat(tvix/eval): deduplicate overlap between Closure and Thunk
This commit deduplicates the Thunk-like functionality from Closure
and unifies it with Thunk.

Specifically, we now have one and only one way of breaking reference
cycles in the Value-graph: Thunk.  No other variant contains a
RefCell.  This should make it easier to reason about the behavior of
the VM.  InnerClosure and UpvaluesCarrier are no longer necessary.

This refactoring allowed an improvement in code generation:
`Rc<RefCell<>>`s are now created only for closures which do not have
self-references or deferred upvalues, instead of for all closures.
OpClosure has been split into two separate opcodes:

- OpClosure creates non-recursive closures with no deferred
  upvalues.  The VM will not create an `Rc<RefCell<>>` when executing
  this instruction.

- OpThunkClosure is used for closures with self-references or
  deferred upvalues.  The VM will create a Thunk when executing this
  opcode, but the Thunk will start out already in the
  `ThunkRepr::Evaluated` state, rather than in the
  `ThunkRepr::Suspeneded` state.

To avoid confusion, OpThunk has been renamed OpThunkSuspended.

Thanks to @sterni for suggesting that all this could be done without
adding an additional variant to ThunkRepr.  This does however mean
that there will be mutating accesses to `ThunkRepr::Evaluated`,
which was not previously the case.  The field `is_finalised:bool`
has been added to `Closure` to ensure that these mutating accesses
are performed only on finalised Closures.  Both the check and the
field are present only if `#[cfg(debug_assertions)]`.

Change-Id: I04131501029772f30e28da8281d864427685097f
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7019
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-19 10:38:54 +00:00
Griffin Smith
2a3d498104 feat(tvix/eval): Validate closed formals
Validate "closed formals" (formal parameters without an ellipsis) via a
new ValidateClosedFormals op, which checks the arguments (in an attr set
at the top of the stack) against the formal parameters on the Lambda in
the current frame, and returns a new UnexpectedArgument error (including
the span of the formals themselves!!) if any arguments aren't allowed

Change-Id: Idcc47a59167a83be1832a6229f137d84e426c56c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7002
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-17 11:29:49 +00:00
sterni
4dcb8f38c2 fix(tvix/eval): resolve home relative paths at runtime
Home relative paths depend on the environment to be resolved. We have
elected to do everything that depends on the environment, e.g. resolving
SPATH expressions using NIX_PATH, at runtime, so tvix evaluation would
continue to behave correctly even if we separated the compilation and
execution phases more, e.g. via serializing bytecode. Then the value of
HOME, NIX_PATH etc. could reasonably change in the time until execution,
yielding wrong results if the resolution results were cached in the
bytecode.

We also take the opportunity to fix the broken path concatenation
previously found in the compiler, fixing b/205.

Another thing we could consider is emitting a warning for home relative
path literals, as they are by nature relatively fragile.

One sideeffect of this change is that home path resolution errors
become catchable which is not the case in C++ Nix. This will need to be
fixed up in a subsequent change.

Change-Id: I30bd69b575667c49170a9fdea23a020565d0f9ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7024
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2022-10-16 19:11:23 +00:00
sterni
0624d78af0 refactor(tvix/eval): make OpFindFile use internal UnresolvedPath
To assert that OpFindFile is only emitted for specially compiled SPATH
expressions, as well as make sure it doesn't accidentally operate on
“ordinary values”, introduce an UnresolvedPath internal value. If
OpFindFile sees a non-UnresolvedPath value, it'll crash.

Note that this change is not done purely for OpFindFile: We may want to
compile SPATH expressions as function calls to __findFile (like C++ Nix
does) in the future, so the UnresolvedPath value would definitely need
to be an ordinary string again then. Rather, this change is done in
preparation for resolving home dir relative paths at runtime (since they
depend on the environment) for which we'll need a similar mechanism to
OpFindFile.

Change-Id: I6acf287f35197cd9e13377079f972b9d36e5b22e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7023
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-16 19:11:23 +00:00
Adam Joseph
96dcc51297 refactor(tvix/eval) remove Value::DynamicUpvalueMissing
I believe this variant is left over from a previous implementation.
If not, please let me know.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I02a3bf2f63794d09e96a5a92a034c0ad3d1ff221
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7027
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-16 09:17:02 +00:00
Adam Joseph
d6088005ef docs(tvix/eval) vm: explain VM::{frames,stack,with_stack}
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I94ba31ae25c1ff744f929a722c76a0c33cc361ff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7016
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-14 09:35:28 +00:00
Adam Joseph
f868e730be feat(tvix/eval): eliminate the only unsafe in the codebase
Maybe I misunderstood this part of the code, but the use of `unsafe`
appears unnecessary here?  In any event it is the one and only
`unsafe` in the codebase.

Hopefully getting to "no `unsafe` anywhere" is worth the extra
never-taken branch caused by unwrap() instead of unwrap_unchecked()?

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I33fbd5aad9d8307ea82c24b6220412783e1973c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7011
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-14 09:18:45 +00:00
Adam Joseph
91ad5b825b refactor(tvix/eval): remove OpResolveWithOrUpvalue
The case branch in vm.rs for OpResolveWithOrUpvalue is
unreachable/deadcode.

I believe this opcode is unnecessary, since it should always be
statically detectable (at parse-time) whether a reference is to an
upvalue (i.e. enclosing binding); otherwise, and only then, is
with-resolution applicable.

Perhaps I've misunderstood how with-resolution works.  If so, please
explain it to me and -1/-2 this CL.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I4a90b9eb6cb3396df92a6a943d42ecc301871ba0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7009
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-14 09:18:38 +00:00
Adam Joseph
25336fc47b refactor(tvix/eval): factor out all calls to canon_path
Right now we're pretending that the Rust library path_clean does the
same thing that cppnix's canonPath() does.  This is not true.  It's
close enough for the test suite, but may come back to bite us.

Let's create our own canon_path() function and call that in all the
places where we intend to match the behavior of cppnix's
canonPath().  That way when we fix this we can fix it once, in one
place.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: Ia6f9577f62f49ef352ff9cfa5efdf37c32d31b11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6993
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-13 09:07:47 +00:00
Adam Joseph
32ac7d6c6d refactor(tvix/eval) s/NixPath/NixSearchPath/
Since NixString is the Rust type for nix strings, people might
mistake NixPath for the Rust type of nix paths, which it is not.
Let's call it NixSearchPath instead.

Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: Ib2ea155c4b27cb90d6180a04ea7b951d86607373
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6927
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 08:09:49 +00:00
Griffin Smith
4a058a9c00 fix(tvix/eval): Pop frames even if running op fails
Previously, the VM assumed that if an error was returned from `run()`,
the evaluation was "finished" and the state of the VM didn't matter.
This used to be a reasonable assumption, but now that we've got
`tryEval` around we need to actually make sure that we clean up after
ourselves if we're about to return an error. Specifically, if the *last*
instruction in an evaluation frame returns an error, we previously
wouldn't pop that evaluation frame, which could cause all sorts of
bizarre errors based on what happened to be in the stack at the time.

This commit splits out a `run_op` method from `VM::run`, and uses that
to check the evaluation frame return condition even if the op we're
running is about to return an error, and pop the evaluation frame if
we're at the last instruction.

Change-Id: Ib40649d8915ee1571153cb71e3d76492542fc3d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6940
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-11 00:08:05 +00:00
Vincent Ambo
76e0c37b9e feat(tvix/eval): observe stack after exiting call frames/builtins
Change-Id: I1937d37551503a0b6bb0ac899d067302e4791e5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6939
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-11 00:07:39 +00:00
Vincent Ambo
8e122574c2 fix(tvix/eval): implement functor calling for non-tail-calls
Change-Id: I1113921c46010021b61c412d74d60193c156e0f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6937
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-11 00:07:38 +00:00
Griffin Smith
06ec4bebe7 fix(tvix/eval): Actually trace spans for thunks
Currently, the span on *all* thunk force errors is the span at which the
thunk is forced, which for recursive thunk forcing ends up just being
the same span over and over again. This changes the span on thunk force
errors to be the span at which point the thunk is *created*, which is a
bit more helpful (though the printing atm is a little... crowded). To
make this work, we have to thread through the span at which a thunk is
created into a field on the thunk itself.

Change-Id: I81474810a763046e2eb3a8f07acf7d8ec708824a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6932
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 23:51:09 +00:00
Vincent Ambo
1f84d90811 refactor(tvix/eval): after calling, the caller has to pop
Previously the various call functions either returned `EvalResult<()>`
or `EvalResult<Value>`, which was confusing.

Now only vm::call_with returns a Value directly, and other parts of
the API just leave the stack top in the post-call state.

This makes it easier to reason about what's going on in non-tail-call
cases (which are making a comeback).

Change-Id: I264ffc683a11aca72dd06e2220a5ff6e7c5fc2b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6936
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-10 23:36:49 +00:00
Griffin Smith
273ba73754 feat(tvix/eval): Initial resolution of <...> paths
This commit implements (lazy) resolution of `<...>` paths via either the
NIX_PATH environment variable, or the -I command-line flag - both
handled via EvalOptions. As a result, EvalOptions can no longer derive
Copy, meaning we have to clone it at each line of the repl - this is
probably not a huge deal as repl performance is not exactly an inner
loop and we're not cloning very much.

Internally, this works by creating a thunk which pushes a constant
containing the string inside the brackets to the stack, then a new
opcode to resolve that path via the `NixPath`. To get that opcode to
work, we now have to pass in the NixPath when constructing the VM.

This (intentionally) leaves out proper implementation of path resolution
via `findFile` (cppnix just calls whatever identifier called findFile is
in scope!!!) as that's widely considered a bit of a misfeature, but if
we do decide to implement that down the road it likely wouldn't be more
than a few extra ops within the thunk introduced here.

Change-Id: Ibc979b7e425b65cbe88599940520239a4a10cee2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6918
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 20:23:41 +00:00
Griffin Smith
0e9f5d6890 feat(tvix/eval): Allow adding strings to paths
Implement adding paths and strings via OpAdd. Since the nix rules are
quite obscure, I'm electing to test this one with an oracle test to
avoid the danger of getting the actual asserted result wrong.

Change-Id: Icdcca3690ca2e8459e386c1f29cc48eaaa39e9a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6914
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 20:23:41 +00:00
Griffin Smith
899fbdbddb refactor(tvix/eval): Compile OpAssert using conditional jumps
In order to behave nicely with tryEval, asserts need to leave the
instruction pointer in a reasonable place even if they fail - whereas
with the previous implementation catching a failed assert would still
end up running the op for the *body* of the assert. With this change, we
compile asserts much more like an `if` expression with conditional jumps
rather than having an OpAssert op.

Change-Id: I1b266c3be90185c84000da6b1995ac3e6fd5471b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6925
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 17:51:22 +00:00
Griffin Smith
0b06d94606 refactor(tvix/eval): Abstract away calling functions
The process of calling a function from a builtin, especially if it's got
more than 1 arrgument, is reasonably involved and easy to get wrong due
to having to interact directly with the stack - instead of having that
done entirely manually in builtins, this wraps it up in a new
`call_with` function which handles pushing arguments onto the stack and
recursively calling the (partially applied) function.

Change-Id: I14700c639a0deca53b9a060f6d70dbc7762e9007
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6910
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 15:34:31 +00:00
Griffin Smith
278bccc1ea refactor(tvix/eval): Encapsulate Value::Attrs construction
Factor out the construction of Value::Attrs (including the Rc) into a
new `attrs` constructor function, to abstract away the presence of the
Rc itself.

Change-Id: I42fd4c3841e1db368db999ddd651277ff995f025
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6892
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 19:02:25 +00:00
Vincent Ambo
07e03498f2 feat(tvix/eval): add method for emitting runtime warnings
This lets the VM emit warnings when it encounters situations that
should only be warned about at runtime.

For starters, this is used to pass through compilation warnings that
come up when `import` is used.

Change-Id: I0c4bc8c534d699999887c430d93629fadfa662c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6868
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-07 14:24:36 +00:00
Vincent Ambo
b69b50feb1 refactor(tvix/eval): split observer traits in two
There are actually two different types of observers, the ones that
observe the compiler (and emitted chunks from different kinds of
expressions), and the ones that trace runtime execution.

Use of the NoOpObserver is unchanged, it simply implements both
traits.

Change-Id: I4277b82674c259ec55238a0de3bb1cdf5e21a258
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6852
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-04 21:27:30 +00:00
Vincent Ambo
8d95e35ced fix(tvix/eval): do not fail when finalising non-capturing values
This can actually legitimately be emitted by the compiler currently
when compiling formals with default values. See the scope6 test from
the Nix test suite for an example.

We should restructure this slightly to be able to reintroduce a
runtime error here in case something was compiled incorrectly.

Change-Id: Ib81f0f58ae0e850db9fbc459458b7bd0d3ac6f23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6841
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-03 07:50:41 +00:00
Vincent Ambo
8336577c41 feat(tvix/eval): implement tail-calling of __functor attributes
This implements __functor calling in situations where `OpTailCall` is
used, but not yet for `OpCall`.

For some reason I have not yet figured out, this same implementation
does not work in call_value, which means that it also doesn't yet work
in builtins that apply functions.

Change-Id: I378f9065ac53d4c05166a7d0151acb1f55c91579
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6826
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-03 07:50:41 +00:00
Vincent Ambo
68a4d4a759 fix(tvix/eval): fix thunk borrowing error in force_for_output
This function previously kept a borrow in the form of the
`Thunk::value` result alive while performing arbitrary actions in the
VM, which caused a borrowing error in the test case attached.

The `Ref` value must never be used in cases where control flow is
passed to other parts of the VM.

Change-Id: I41d10aa1882a2166614b670e8ba77aab0e67deca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6825
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-30 12:31:14 +00:00
Vincent Ambo
d54aeb1f20 chore(tvix/eval): remove existing nested key implementation
This implementation, which only ever worked for non-recursive
attribute sets, is no longer needed and thus removed here.

We have a new implementation of these nested keys coming up instead.

Change-Id: I0c2875154026a4f5f6e0aa038e465f54444bf721
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6783
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00
Vincent Ambo
f816813d41 feat(tvix/eval): implement 'builtins.filter'
This is a little ugly because the plain Iterator::filter method can
not be used (it does not support fallible primitives), so we need to
resort to an `Iterator::filter_map` and deal with the wrapping in
Options everywhere.

This prevents use of `?` which introduces the need for some matching,
but it's not *too* bad.

Change-Id: Ie2c3c0c9756c4c627176f64fb4e0054e717c26d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6765
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-23 09:55:14 +00:00
sterni
64d3efcc2c fix(tvix/eval): handle thunks in arithmetic builtins
The simplest solution seems to be to pass references to arithmetic_op!()
which avoids the moving annoyance we had to deal with in the
builtins (no more popping!). We then use .force() to force the values
and dereference any Thunks (which arithmetic_op! doesn't do for us).

Change-Id: I0eb8ad60e80a0b3ba9d9f411e973ef8bcf136989
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6724
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-22 23:12:29 +00:00
William Carroll
890bbf9b1f feat(tvix/eval): Support builtins.lessThan
Extend and export the `cmp_op`, and this becomes trivial.

Change-Id: I9c93fa4db0f5a1fc8b56928ea144676f79247de1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6557
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 17:42:01 +00:00
Vincent Ambo
489395448f feat(tvix/eval): track other type in NotCallable error kind
This makes for slightly nicer error messages if something isn't, well,
callable.

Change-Id: I821c8d7447b93aea9ccaaa52ed329de0cca4b18e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6718
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 23:48:57 +00:00
Vincent Ambo
8f2004d360 refactor(tvix/eval): add VM::call_value helper method
This makes it possible to call a callable value (builtin or
closure/lambda) directly, without unwrapping it first. This is needed
for pretty much all higher-order functions to work correctly.

This is mostly equivalent to the previous code in coerce_to_string for
calling `__toString`, except it expects the argument(s) to already be
placed on the stack.

Note that the span for the `NotCallable` error is not currently
guaranteed to make any sense, will experiment with this.

Change-Id: I821224368d438a28900858b343defc1817e46a0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6717
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 23:48:57 +00:00
Griffin Smith
0b76ed5615 chore(tvix/eval): Pass in VM to nix_eq
Pass in, but ignore, a mutable reference to the VM to the `nix_eq`
functions, in preparation for using that VM to force thunks during
comparison.

Change-Id: I565435d8dfb33768f930fdb5a6b0fb1365d7e161
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6651
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:03:42 +00:00
Griffin Smith
915ff5ac2a refactor(tvix/eval): Don't (ab)use PartialEq for Nix equality
Using rust's PartialEq trait to implement Nix equality semantics is
reasonably fraught with peril, both because the actual laws are
different than what nix expects, and (more importantly) because certain
things actually require extra context to compare for equality (for
example, thunks need to be forced). This converts the manual PartialEq
impl for Value (and all its descendants) to a *derived* PartialEq
impl (which requires a lot of extra PartialEq derives on miscellanious
other types within the codebase), and converts the previous
nix-semantics equality comparison into a new `nix_eq` method. This
returns an EvalResult, even though it can't currently return an error,
to allow it to fail when eg forcing thunks (which it will do soon).

Since the PartialEq impls for Value and NixAttrs are now quite boring,
this converts the generated proptests for those into handwritten ones
that cover `nix_eq` instead

Change-Id: If3da7171f88c22eda5b7a60030d8b00c3b76f672
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6650
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:03:41 +00:00
Vincent Ambo
da9e885e06 refactor(tvix/eval): rename OpAttrsIsSet -> OpHasAttr
This matches the name of the AST node from which it was compiled.
Suggested by sterni in cl/6231

Change-Id: Ia51525158d2f47467c01fce2282005b1a8417a47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6623
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-17 18:58:42 +00:00
sterni
4eb33e82ff fix(tvix/eval): coerce string interpolation parts to string
With this puzzle piece of string compilation in place, `compile_str`
becomes less redundant, as every part now needs to be compiled the same.
The thunking logic becomes a bit trickier, since we need to thunk even
in the case of `count == 1` if the single part is interpolating.
Splitting the inner (shared) code in a separate function turned out to
be easier for making rustc content.

Change-Id: I6a554ca599926ae5907d7acffce349c9616f568f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6582
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 15:52:53 +00:00
sterni
da1d71a4e8 feat(tvix/eval): implement correct toString behavior
Implement C++ Nix's `EvalState::coerceToString` minus some of the Path
/ store handling. This is currently only used for `toString` which does
all possible coercions, but we've already prepared the weaker coercion
variant which is e.g. used for builtins that expect string arguments.

`EvalState::coerceToPath` is still missing for builtins that need a
path, but it'll be easy to build on top of this.

Change-Id: I78d15576b18921791d04b6b1e964b951fdef22c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6571
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 11:47:30 +00:00
Vincent Ambo
f69e83ae7b refactor(tvix/eval): cover all Value variants in force_for_output
Avoids accidentally dropping one on the floor if we add more, pointed
out by sterni in cl/6372

Change-Id: Ib7bb0ce9c8331c8337003d20c4d5240dfae1c32a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6570
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
d5ee893fb1 refactor(tvix/eval): use CodeIdx wrapper for instruction pointer
As suggested by sterni in cl/6453.

Change-Id: I3cf80d97c11fd7d085ab510f6be4b5f937c791ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6562
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
07ea30370e refactor(tvix/eval): capture entire with_stack in upvalues
This completely rewrites the handling of "dynamic upvalues" to,
instead of resolving them at thunk/closure instantiation time (which
forces some values too early), capture the entire with stack of parent
contexts if it exists.

There are a couple of things in here that could be written more
efficiently, but I'm first working through this to get to a bug
related to with + recursion and the code complexity of some of the
optimisations is distracting.

Change-Id: Ia538e06c9146e3bf8decb9adf02dd726d2c651cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6486
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:26:23 +00:00
Vincent Ambo
d75b207a63 refactor(tvix/eval): introduce Upvalues struct in closures & thunks
This struct will be responsible for tracking upvalues (and is a
convenient place to introduce optimisations for reducing value clones)
instead of a plain value vector.

The main motivation for this is that the upvalues will have to capture
the `with`-stack fully and I want to avoid duplicating the logic for
this between the two capturing types.

Change-Id: I6654f8739fc2e04ca046e6667d4a015f51724e99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6485
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:16:46 +00:00
Vincent Ambo
27e69503a7 fix(tvix/eval): avoid forcing with-target until absolutely necessary
Change-Id: I00efbbb8b9d3d22f32becf0919c6adf1be8b4b69
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6465
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:04:27 +00:00
Vincent Ambo
06909f1821 fix(tvix/eval): fix doc comment syntax where applicable
As pointed out by grfn on cl/6091

Change-Id: I28308577b7cf99dffb4a4fd3cc8783eb9ab4d0d6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6460
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-10 21:57:13 +00:00
Vincent Ambo
2e018a50a7 feat(tvix/eval): implement OpTailCall
If the last operation within a chunk is a function call, the call can
be executed in the same call frame without increasing the depth of the
call stack.

To enable this, a new OpTailCall instruction (similar to OpCall) is
introduced, but not yet emitted by the compiler.

Change-Id: I9ffbd7da6d2d6a8ec7a724646435dc6ee89712f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6457
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-10 21:57:13 +00:00
Vincent Ambo
6deaa0d6ce fix(tvix/eval): force value passed to builtins.toString
This introduces a macro to do the forcing, but this solution isn't
very nice and also does not work in all cases yet.

Change-Id: Icd18862ec47edb82c0efc3af5835a6cb6126f629
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6456
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-09 21:14:56 +00:00
Vincent Ambo
14ff889d60 feat(tvix/eval): implement runtime tracing methods for Observer
These methods make it possible to trace the runtime execution of the
VM through an observer.

Change-Id: I90e26853ba2fe44748613e7f761ed5c1c5fc9ff7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6452
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-09 21:10:10 +00:00
Vincent Ambo
1fe6cfe5a2 refactor(tvix/eval): index into Chunk with ConstantIdx/CodeIdx
This is a step towards hiding the internal fields of thunk, and making
the interface of the type more predictable.

Part of the preparation for implementing observers.

Change-Id: I1a88a96419c72eb9e2332b56a2dd94afa47e6f88
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6447
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-09 21:10:10 +00:00
Vincent Ambo
8fdb67847c chore(tvix/eval): debug_assert that all jumps are patched
Suggestion from sterni in cl/6282

Change-Id: I1adbdda9ff74f55a2f72892ffa524808b305f403
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6445
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-09 10:48:10 +00:00
Vincent Ambo
48d5f4fd57 feat(tvix/eval): print lambda memory adresses in disassembler
This makes it easier to track exactly which lambda is which when
inspecting e.g. the concrete representation of a thunk.

At runtime all lambdas live in an Rc. To make this print the right
address, the construction of these Rcs had to be moved up right to the
point where the lambda is first emitted (and disassembled).

Change-Id: I6070e6c8ac55f0bd697966c4e7c5565c20d19106
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6435
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
09eaa0d4ae fix(tvix/eval): address current clippy & grfn lints
Change-Id: I65c6feb9f817b5b367d37204a1f57acfe4100d97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6430
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00