Commit graph

278 commits

Author SHA1 Message Date
William Carroll
059f4b964f feat(tvix/eval): Support builtins.hasAttr
See unit tests for examples :)

Change-Id: Ieec51d780a7762cc455ca03a9dc1648a0711924a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6553
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-19 16:18:10 +00:00
sterni
f358be4a1d fix(tvix/eval): move compareVersions test so cargo test finds them
And insert the missing newline the C++ Nix test script needs.

Change-Id: I04ddd7268f9caa1414fd23314c281bb7c1e854cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6689
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-19 09:58:14 +00:00
William Carroll
fefa8c55c4 feat(tvix/eval): Support builtins.tail
TL;DR:
- support `builtins.tail`
- define `ErrorKind::TailEmptyList` and canonical error code
- support basic unit tests

Unsure whether or not the error should be a dedicated `ErrorKind`...

Change-Id: Iae90fda1bb21ce7bdb1aaa2aeb2b8c1e6dcb0f05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6545
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 00:57:11 +00:00
William Carroll
2fe18e4486 feat(tvix/eval): Support builtins.substring
Nix's `builtins.substring`:
- doesn't accept negative indexes for `beg` or `end`. Compare the error messages
  for:
  - `builtins.substring -3  5 "testing"`
  - `builtins.substring  3 -5 "testing"`
  - `builtins.substring -3 -5 "testing"`
- returns an empty string when `beg >= end`
- allows `end` to exceed the length of the input string

Change-Id: I83af7a099d81b6d537ebe5029d946c7031cb7113
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6555
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 00:51:10 +00:00
Vincent Ambo
bcbe1603c8 fix(tvix/eval): ensure all thunks are forced in nested selects
Previously only the first one was guaranteed to be forced, but we need
to do this for all of them.

Fixes b/190

Change-Id: I76b5667dbfb2f3fde3587e7b91d268cbf32aca00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6645
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2022-09-18 22:16:04 +00:00
Griffin Smith
e720545e5b refactor(tvix/eval): use Clap for arg+env parsing
Refactor the environment variable and argument parsing for the tvix repl
to use Clap instead of doing things ad-hoc, and thread through options
obtained from environment variables via explicit arguments rather than
obtaining them from the environment as they're needed. This makes adding
more flags more sustainable, and also makes the binary fully
self-documenting, including supported env vars, via `--help`.

Change-Id: Ib1f6a0cd20056e8c9196760ff755fa5729667760
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6653
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:08:43 +00:00
Griffin Smith
6f70f32513 fix(tvix/eval): Force thunks during equality comparison
Thunks might be encountered deep in equality comparison (eg nested
inside a list or attr-set), at which point we need to force them in
order to compare them for equality (or else we panic when trying to get
at their value).

Fixes: b/192
Change-Id: I912151085f8298f30d5214c7965251c9266443f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6652
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:08:43 +00:00
Vincent Ambo
2f91543a42 test(tvix/eval): add a test for lazy evaluation of with namespace
Suggested by grfn in cl/6222

Change-Id: I8413af6404786bd43e5cb462a307cc236c4fb9ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6621
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 18:58:42 +00:00
William Carroll
3a67a14009 feat(tvix/eval): Support builtins.elemAt
(Attempt to) index into a list.

Change-Id: I3592e60a79e64d265e34100d4062041b0b410e00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6551
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-17 12:54:14 +00:00
Vincent Ambo
0532cb6172 test(tvix/eval): add tests for legacy let syntax
Change-Id: I42d6240056c93cf90986ef3fec88b42f804b06ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6613
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:13:26 +00:00
Vincent Ambo
15c8e8c459 test(tvix/eval): add some tests for recursive attribute sets
Change-Id: I6a4fbe6d5aa6d2ce223bf77330d6c0f512d4300d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6612
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:13:26 +00:00
William Carroll
f859751c6a feat(tvix/eval): Support builtins.bitXor
Bitwise "exclusive-or" on integers.

Change-Id: I90a0a15afb3a58662d70e82ea14e48b877476e04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6550
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:53:10 +00:00
William Carroll
031ca653a3 feat(tvix/eval): Support builtins.bitOr
Bitwise `or` on integers.

Change-Id: I45b0897331d1a9b6840f9d0feedcf10acc67fcec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6549
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:47:09 +00:00
William Carroll
f5f6bd319d feat(tvix/eval): Support builtins.bitAnd
Bitwise `and` on integers.

Change-Id: I9f2a9182a057af26906683acd97a40dfabbdded8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6548
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:41:08 +00:00
William Carroll
5a6d9ee1a0 feat(tvix/eval): Support builtins.attrValues
:)

Change-Id: Idf170b506ed6fab035b1e6f61055fee02e5c9be8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6547
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-15 21:13:03 +00:00
William Carroll
e834a2cbc4 feat(tvix/eval): Support builtins.attrNames
Define `.len()` method on `NixAttrs` to preallocate the capacity of the result
vector.

Also anchor an errant comment to its context (I think).

Change-Id: I268f15025d453d7b3ae1146558c80e51433dd2a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6546
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-15 20:37:57 +00:00
William Carroll
85b3f17007 feat(tvix/eval): Support builtins.head
TL;DR:
- support `builtins.head`
- define `ErrorKind::IndexOutOfBounds` and canonical error code
- support basic unit tests

Change-Id: I859107ffb4e220cba1be8c2ac41d1913dcca37ff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6544
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 16:48:24 +00:00
sterni
bcd7e520f0 fix(tvix/eval): thunk string interpolation
If we have multiple string parts, we need to thunk assembling the
string. If we have a single literal, it is strict (like all literals),
but a single interpolation part may compile to a thunk, depending on how
the expression inside is compiled – we can avoid forcing to early here
compared to the previous behavior.

Note that this CL retains the bug that `"${x}"` is erroneously
translated to `x`, implying e.g. `"${12}" == 12`.

The use of `parts.len()` is unproblematic, since normalized_parts()
builds a `Vec` instead of returning an iterator.

Change-Id: I3aecbfefef65cc627b1b8a65be27cbaeada3582b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6580
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 15:52:53 +00:00
sterni
da1d71a4e8 feat(tvix/eval): implement correct toString behavior
Implement C++ Nix's `EvalState::coerceToString` minus some of the Path
/ store handling. This is currently only used for `toString` which does
all possible coercions, but we've already prepared the weaker coercion
variant which is e.g. used for builtins that expect string arguments.

`EvalState::coerceToPath` is still missing for builtins that need a
path, but it'll be easy to build on top of this.

Change-Id: I78d15576b18921791d04b6b1e964b951fdef22c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6571
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 11:47:30 +00:00
sterni
162e21f2bb fix(tvix/eval): force left argument of ? before checking for attrs
OpAttrsIsSet and OpAttrsTrySelect will fail silently if the attribute
set value on the stack is actually a thunk, so we need to make sure to
force at every step of the way.

Emitting the force instructions in the compiler because it is easier to
add, but maybe the VM should do this when handling the relevant opcodes?
Comments welcome.

Change-Id: I65c5ef348d59b2d07c9bb06abb24f9f3e6a0fdb2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6540
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-13 22:06:29 +00:00
sterni
7e625afc59 fix(tvix/eval): force value in builtins.typeOf
This prevents Nix programs to observe the "internal" type of thunks.
Possibly .type_of() is also an area of the runtime where we should panic
if "internal" would ever be returned.

Change-Id: I9f358044c48ad64896fb6a1b1a42f00a29efac00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6539
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-13 22:06:29 +00:00
sterni
7046604cfe fix(tvix/eval): place plain inherits in correct stack slots
We need to make sure that we compile all plain inherits in a let
expression before declaring any other locals. Plain inherits are special
in the sense that they can never be recursive, instead resolving to a
higher scope. Thus we need to compile their value, before declaring
them. If we don't do that, before any other local can be declared,
we cause a situation where the plain inherits' values are placed into
other locals' stack slots.

Note that we can't integrate the plain inherit compilation into the
regular 2-3 phase model where we defer the compilation of the value or
we'd compile `let inherit x; in …` as `let x = x; in …`.

Change-Id: I951d5df3c9661a054e12401546875f4685b5bf08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6496
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:26:23 +00:00
sterni
bb1adbb05b test(tvix/eval): add test for mutually recursive let bindings
This test shows that let bindings' dependencies can form a cyclical
graph, so we need to use thunking to break this cycle.

Change-Id: I2a4de71fd7024f3d3d1166154784139a82f39411
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6495
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:26:23 +00:00
sterni
240d90aa8a fix(tvix/eval): wrap asserts in a thunk
As the new test case demonstrates, asserts need to be evaluated lazily.

Change-Id: I808046722c5a504e9497855ca5026d255c7a4c34
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6494
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:26:23 +00:00
sterni
f95b23d743 test(tvix/eval): test “useful” plain inherits
Change-Id: Ic4700f0618a393e45a2ee7c70045edff97e30c49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6493
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:26:23 +00:00
sterni
4d8f35353b fix(tvix/eval): declare let inherit (from) locals before compiling
The recent change that split declaration of let based locals and the
compilation of their values did not touch locals bound by inherit in
let. These were previously declared and compiled immediately before
starting to work on the other locals introduced in a let.

In the case of plain inherits, this behavior is kept in this change,
because there's nothing wrong with it: The value of a plain inherit will
always resolve to a higher scope, either statically or dynamically.

Since inherit (from) expression might refer to other locals bound in the
same let, we need to handle them in the same three steps as ordinary let
based locals:

1. We need to declare the (uninitialised) locals.

2. We need to compile the expression that obtains their value. For this,
   we create a new thunk, since the from expression may very well return
   a thunk which we need to force before selecting the value we are
   interested in.

3. Thunks need to be finalised.

For 1., we create an extra pass over the inherits that already declares
and initialises plain inherits and notes inherit (from) expressions in
the entries vector after declaring them. 2. only needs a bit of adapting
to create the thunks for selecting if appropriate, the rest of the
existing code can be reused.

Change-Id: Ie4ac1c0f9ffcbf7c07c452036aa8e577443af773
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6490
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-11 12:26:23 +00:00
Vincent Ambo
627dfabef1 fix(tvix/eval): thunk all uses of with
With this all other "weird scope" logic starts working for `with` as
well.

Change-Id: I0ea1d8c5fbd9cec5084bd574224f77b71ff2b487
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6487
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:26:23 +00:00
Vincent Ambo
fd14eefed6 fix(tvix/eval): correctly account for slots during list construction
Similarly to attribute sets, list elements can be arbitrary
expressions and their (temporary) stack slots during construction must
be accounted for by the compiler.

Change-Id: I3b6f7927860627fd867c64d0cab9104fd636d4f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6470
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:16:46 +00:00
Vincent Ambo
f68c76d07d fix(tvix/eval): account for attrset temporaries during construction
The temporaries left on the stack as operands to `OpAttrs` must be
accounted for in the locals array in order for operations within them
to receive correct slots.

Some test cases that were previously broken have been added.

Change-Id: Ib52b629bbdf7931f63fd45a45af1073022da923c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6468
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 12:04:27 +00:00
Vincent Ambo
abdfa7459e feat(tvix/eval): thunk binary operations and select expressions
With this, most cases of `fix` in attribute sets will work correctly.
A simple test exercising both has been added.

Change-Id: I70fd431177bb6e48ecb33a87518b050c4c3d1c09
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6437
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
678533bfc4 test(tvix/eval): add test for stack slot accounting edge-case
This was fixed by some of the previous commits around scopes. It's
somewhat similar to a few other tests, but I had this one failing
earlier and everything else succeeding, so it is useful to keep it
around for sure.

Change-Id: Ie6cf372b5c805daf992cd87aeb3dfe91542c381c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6431
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
fe047885d7 fix(tvix/eval): consider local depth when deciding to defer
Deferred local upvalues can *only* occur at the same depth as the
thing that is closing over them, but there are various situations with
scope nesting where the actual stack indexes of the local and the
closer look like a deferred value is being accessed.

To fix this, simply compare the depth as well.

Change-Id: Ice77424cc87ab0a2c4f01379e68d4399a917b12b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6429
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
360c805efc chore(tvix/eval): remove accidentally duplicated test
This is the same as `eval-okay-attrs-simple-inherit`.

Change-Id: I23878accc6cd62c16ec96601239838a385d31306
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6428
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 19:54:02 +00:00
Vincent Ambo
5ee89bcf5c fix(tvix/eval): inherit scope poisoning data in nested contexts
Scope poisoning must be inherited across lambda context boundaries,
e.g. if an outer scope has a poisoned `null`, any lambdas defined on
the same level must reference that poisoned identifier correctly.

Change-Id: I1aac64e1c048a6f3bacadb6d78ed295fa439e8b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6410
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 07:59:15 +00:00
William Carroll
197fe37dae feat(tvix/eval): Support builtins.length
Get the length of a list

Change-Id: I41d91e96d833269541a1b3c23b7cc879f96d1e5a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6407
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 20:56:02 +00:00
William Carroll
15e2bc54f1 feat(tvix/eval): Support builtins.{mul,div}
Multiplication and division :)

Change-Id: I79e76f3b55b7e9b8d1eb09b3d9741140b4d75742
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6406
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 20:56:02 +00:00
William Carroll
b3097de75d feat(tvix/eval): Support builtins.{add,sub}
First pass at supporting `builtins` for tvix. The following tests appear to be
WAI:

```shell
$ cd tvix/eval
$ cargo build
$ cargo test
```

Change-Id: I27cce23d503b17a886d1109e285e8b4be4264977
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6405
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 20:56:02 +00:00
Vincent Ambo
afcfa78364 feat(tvix/eval): semi-strictly evaluate output values of the VM
This essentially makes the VM behave like `nix-instantiate --eval
--strict`, i.e. data structures are traversed strictly and thunks are
forced. Thunks embedded in closures are not forced.

This allows us to re-enable tests that were disabled because they
needed to output nested thunk contents, but is overall a behaviour
that must be configurable later on, as it is not cmopatible with e.g.
an evaluation of nixpkgs.

Change-Id: I5303a5c8e4322feab1384fdb7712fecb950afca5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6372
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
23a5caabec feat(tvix/eval): construct attribute sets lazily
This thunks the construction of attribute sets. Because Tvix does not
currently have a "strict output" mode, a test had to be disabled that
now displays a thunk representation.

The test will be re-enabled once that is available.

Change-Id: I360332be64cd5c154f9caea21828f6f1b37a265c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6363
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
372bf4a0b7 test(tvix/eval): test very simple late-bound identifier access
This is the simplest kind of thunk that can be created (and so far the
only one the compiler knows how to create), in which an identifier
inside a `let` encounters a value that is bound *after* it is
initialised.

Change-Id: I6ea4408a3baef1e7d5137365d70804283f2dbf8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6354
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
786e12a738 feat(tvix/eval): always emit OpForce as the last instruction
Change-Id: Id70c987f654dc5d9b47db74e395281309762b468
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6353
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
70b27a6080 test(tvix/eval): add a test case for nested, deferred upvalues
This test case was previously broken by the bug introduced by
confusing local and stack indexes.

Change-Id: Ibef299dad266c6105deac1da5dde112fe9f640b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6341
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
68aad6d4cf feat(tvix/eval): emit OpFinalise when local scopes are complete
With this change, the runtime can correctly capture deferred upvalues.

Change-Id: I1e43b7b1ac2553b1812424adfc8bd08ef77bf1ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6339
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
776120de05 fix(tvix/eval): instantiate *new* closures from blueprints each time
The previous closure refactoring introduced a bug in which the same
closure object would get mutated constantly for each instance of a
closure, which is incorrect behaviour.

This commit instead introduces an explicit new Value variant for the
internal "blueprint" that the compiler generates (essentially just the
lambda) and uses this variant to construct the closure at runtime.

If the blueprint ever leaks out to a user somehow that is a critical
bug and tvix-eval will panic.

As a ~treat~ test for this, the fibonacci function is being used as it
is a self-recursive closure (i.e. different instantiations of the same
"blueprint") getting called with different values and it's good to
have it around.

Change-Id: I485de675e9bb0c599ed7d5dc0f001eb34ab4c15f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6323
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
ccfb971dc5 fix(tvix/eval): correctly thread through dynamic upvalues
This puts together the puzzle pieces for threading dynamic
upvalues (that is, upvalues resolved from the `with`-stack) all the
way through.

Reading the test case enclosed in this commit and walking through it
is recommended to understand what problem is being tackled here.

In short, because the compiler can not statically know *which*
with-scope a dynamic argument is resolved from it needs to lay the
groundwork for resolving from *all* possible scopes.

There are multiple different approaches to doing this. The approach
chosen in this commit is that if a dynamic upvalue is detected, the
compiler will emit instructions to close over this dynamic value
in *all* enclosing lambda contexts.

It uses a new instruction for this that will leave around a sentinel
value in case an identifier could not be resolved, and wire the
location of this found value (or sentinel) up through the upvalues to
the next level of nesting.

In this tradeoff, tvix potentially closes over more upvalues than are
needed (but in practice, how often do people create *really* deep
`with`-stacks? and in *this* kind of code situation? maybe we should
even warn for this!) but avoids keeping the entire attribute sets
themselves around.

Looking at the test case, each surrounding closure will close
over *all* dynamic identifiers that are referenced later on visible to
it, but only the last one for each identifier will actually end up
being used.

This also covers our bases for an additional edge-case this creates,
in which an identifier potentially resolves to a dynamic upvalue *and*
to a dynamic value within the function's own scope (again, would
anyone really do this?) by introducing a resolution instruction for
that particular case.

There is likely some potential for cleaning up this code which is
quite ugly in some parts, but as this implementation is now carefully
calibrated to work I decided it is time to commit it and clean it up
in subsequent commits.

Change-Id: Ib701e3e6da39bd2c95938d1384036ff4f9fb3749
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6322
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
a5e22c532b fix(tvix/eval): correctly resolve dynamic upvalues one scope up
This does not yet correctly resolve them if they are more than one
scope up, however.

Change-Id: I6687073c60aee0282f2b6ffc98b34c1e96a60f20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6319
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
ae531a2245 feat(tvix/eval): implement capture of self-recursive upvalues
With this change, it becomes possible for functions to call themselves
as they are being defined in local bindings.

Change-Id: Ib46a39ba17b1452b5673d96fa729d633d237241a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6314
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
33cde1422e feat(tvix/eval): implement upvalue resolution in with scopes
These need to be handled specially by the runtime if the compiler
determines that a given local must be resolved via `with`.

Note that this implementation has a bug: It currently allows `with`
inside of nested lambdas to shadow statically known identifiers. This
will be cleaned up in the next commit.

Change-Id: If196b99cbd1a0f2dbb4a40a0e88cdb09a009c6b9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6299
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
26acc2e636 test(tvix/eval): add tests for very simple closures
Change-Id: Ib8287ade4d5df6d29e1812fb2d349cee5d92ca6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6296
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
d761d934ed test(tvix/eval): add a test for float representation
Change-Id: I4893a37719b9bf08b35963d48e6851a194a08aa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6274
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
39b01c3029 fix(tvix/eval): correctly escape ${ in strings
Without this escape, it is possible for Nix to produce escaped
representations which are not literal Nix values again.

This was fixed in upstream Nix in
https://github.com/NixOS/nix/pull/4012 (though only for eval, not in
the REPL) and the updated test is picked from upstream after that commit.

Because we run the C++ Nix tests against our test suite as well, this
also bumps our custom Nix 2.3 to a commit that includes the
cherry-picked fix from the PR above.

Change-Id: I478547ade65f655c606ec46f7143932064192283
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6271
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
2023b8e33f fix(tvix/eval): consider let ... inherit ... in dynamic scopes
In conditions where no dynamic identifiers exist in a scope,
inheriting is usually a no-op - *unless* the identifier is not
statically known and the scope has a non-empty `with`-stack.

Change-Id: Iff4138d9cd4c56e844bc574203708dacc11c3f73
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6264
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-02 14:13:00 +00:00
Vincent Ambo
b8ac62d5fc test(tvix/eval): add a simple test for builtins resolution
Change-Id: I91f54778b8a17f3448664c21308de656b4b04b3e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6255
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
64746388e2 feat(tvix/eval): compile function applications
Change-Id: I1b9230601895a1f09ef1a8037201147020b85f36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6250
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 12:31:04 +00:00
Vincent Ambo
16e9703f38 refactor(tvix/eval): use pretty_assertions for tests
This makes for much more readable output especially when long strings
are involved.

Change-Id: I43dd73a0480535d7181a760788c42883a9b083f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6229
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-01 12:50:17 +00:00
Vincent Ambo
0f739cd944 feat(tvix/eval): implement scope poisoning for true/false/null
These tokens are optionally parsed as identifiers by Nix, which means
that within any scopes that resolve them the compiler needs to track
whether they have been overridden to know whether to emit the literal
instructions or resolve a variable.

This is implemented by a new concept of "scope poisoning", where the
compiler's scope structure tracks whether or not any builtin
identifiers have been overridden.

Change-Id: I3ab711146e229f843f6e1f0343385382ee0aecb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6227
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
0b51d63081 refactor(tvix/eval): simplify let ... in ... before recursion
While full recursion through thunking is not available, there are
actually incorrect behaviours introduced by declaring before
binding (example in the newly introduced test).

This commit simplifies the implementation to avoid this issue, and
also because I intend to explore a bit more how far we can get in non
left-to-right bindings *without* introducing thunks immediately.

Change-Id: I21fd3007ac3946570639772d7d624d70bd209958
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6226
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
75ba7c0120 test(tvix/eval): add basic tests for with expressions
Change-Id: I94664090e7a2b060dfbe21c1eeb859fb31e417b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6225
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
f173161f4c feat(tvix/eval): implement inherit in let expressions
Note that at this point recursive bindings do not yet work in either
attrsets or let, so inheriting from the same scope is generally not
possible yet.

Change-Id: I6ca820d04b8ded5c22fb7ea18e2ec203bcaa8e9c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6215
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
8c1c9aee3c feat(tvix/eval): implement inherit in attribute set literals
Straightforward implementation, evaluating the elements of an inherit
and preparing the stack so that `OpAttrs` sees all relevant values
when constructing the attribute set itself.

The emitted instructions for inheriting a lot of values from the same
attribute set are inefficient, but it's too early to say whether this
actually matters.

Change-Id: Icb55a20936d4ef77173f34433811c5fa5d2c9ecc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6214
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
e07556493f feat(tvix/eval): Implement inherit from outer scope in attrs
Change-Id: I97fa45059b7f2cbe96eb60bd1821e3e25832df66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6212
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
3a67f91228 fix(tvix/eval): ? operator should use OpAttrsOrNotFound
Using `OpAttrSelect`, the ? operator will fail when encountering a
nested value that is not an attribute set.

This however breaks valid code, such as:

  { bs = 42; } ? bs.a.b

The fix is simply to use the same operator used in the `or` statement,
which leaves a sentinal on the stack if a field is not found or the
value is not an attribute set.

Change-Id: Ib28fc8a96e6d592b4cdbc3e65ba129ad8faecd66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6211
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
cfe37d36f7 fix(tvix/eval): or should handle non-attrset values, too
If a nested attrpath encounters a non-set value, the sentinel value
denoting a lack of next values should be emitted. This mirrors the
behaviour of Nix.

Change-Id: Ia80443d5a11243cc6d98dcab1249a3f5fdf77e27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6210
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
76846fe220 fix(tvix/eval): allow use of ? operator on non-set types
Nix allows this, but always returns false. Tvix needs to do the same.

Change-Id: Ic9eec90834a0d0969eea5316d5c25032d3691d94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6209
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
43658a5b90 fix(tvix/eval): emit correct count in OpAttrPath
Not sure how exactly this snuck in, but it caused some subtle
breakages in deeply nested attribute sets.

Change-Id: I8049ce912405d3750031f79cc8d86ff1c3c02c2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6208
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
36748fe4f6 test(tvix/eval): add basic tests for let expressions
Change-Id: I788504f0c1848a1422c99006bf659738df614716
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6191
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-29 20:13:26 +00:00
Vincent Ambo
6fe5e2d752 feat(tvix/eval): resolve relative path literals
Resolves relative paths (e.g. `./foo`) either relative to the location
of the Nix file, or relative to the working directory if none is
supplied.

Change-Id: I70ec574657b221b458015117a004b6e4a9c25a30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6185
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-28 11:02:15 +00:00
sterni
4523703a98 test(tvix/eval): make sure all language tests also pass with C++ Nix
Change-Id: I8428a82db9bf7af5c0273178acbf3e4d7355f5d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6153
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-08-26 20:46:03 +00:00
Vincent Ambo
15c2986597 test(tvix/eval): add tests for the attrset or operator
Change-Id: I81f3ab94d8b987f5a4c87a27b2bcbb49cccee1ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6168
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 16:05:09 +00:00
Vincent Ambo
75a22321ce feat(tvix/eval): implement list concatenation
Change-Id: Icdf715d116371a9f139bdf95266410bf967bef25
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6144
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 12:07:30 +00:00
Vincent Ambo
4eafaae9e6 feat(tvix/eval): implement binary comparison operators
This is accomplished by simply delegating to the Rust implementations
of (Partial)Ord and (Partial)Eq, which are implemented for Value and
underlying wrapper types to behave like they do in Nix.

To ease the implementation overhead, a new comparison operator macro
has been added to the VM module.

Incomparable types will raise a new error variant when a comparison is
attempted, containing both supplied types. This mimics the information
carried in the error thrown by C++ Nix.

Change-Id: Ia19634d69119d40722f3ca672387bc3a80096998
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6143
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 12:07:30 +00:00
Vincent Ambo
7d55b776de feat(tvix): implement not-equals (!=) operator
Change-Id: Ic259d6d0cf30553e944682a0d1d2c610df7fe5c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6141
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 11:34:00 +00:00
Vincent Ambo
058e77bab2 feat(tvix/eval): implement attrset update (//) operator
The underlying implementation does a few tricks based on which pair of
attrset representations is encountered.

Particularly the effect of short-circuiting the empty cases might be
relevant in nixpkgs/NixOS, due to the use of lib.optionalAttrs.

Change-Id: I22b978b1c69af12926489a71087c6a6219c012f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6140
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 11:34:00 +00:00
Vincent Ambo
fb3d024d75 feat(tvix/eval): implement string concatenation
Change-Id: If61066e59232b2bad42b5cb5f0f2d9b9c416be8b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6137
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:11:27 +00:00
Vincent Ambo
8150803e77 test(tvix/eval): add some eval-okay-* tests for trivial types
Change-Id: I85ccc07e08c67abf4fcd3752c58e1702943239ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6135
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:07:16 +00:00
Vincent Ambo
96e9f750c5 test(tvix/eval): add identity tests for literal evaluation
Change-Id: Id3f37fbe341e15e9428ef1d579d61a514232c0e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6132
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 23:11:12 +00:00
Vincent Ambo
7d5691f0e5 chore(tvix/tests): gate Nix test suite behind nix_tests feature
Once we have full coverage they should be enabled by default.

Change-Id: Iace9e1ae9a9f901a0979ad336434004b8028fe8a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6129
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 23:11:12 +00:00
Vincent Ambo
92c53fe982 feat(tvix/tests): check in Nix' language test suite
This adds scaffolding code for running the Nix language test suite.

The majority of eval-okay-* tests should eventually be runnable as-is
by Tvix, however the eval-fail-* tests might not as we intend to have
more useful error messages than upstream Nix.

Change-Id: I4f3227f0889c55e4274b804a3072850fb78dd1bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6126
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 21:25:41 +00:00