Commit graph

1958 commits

Author SHA1 Message Date
Vincent Ambo
3cf5c40209 chore(tvix/eval): export some symbols from the crate
These are required for tvixbolt to work. This interface is definitely
not stable yet, though.

Change-Id: I4076498e8f42311de74ee4f33c93a3ee0c5f8d3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6450
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-09 21:10:10 +00:00
Vincent Ambo
8ee4d6d5db feat(tvix/eval): implement DisassemblingObserver for compiler
This type implements an observer that is called whenever the compiler
emits a chunk (after the toplevel, thunks, or lambdas) and prints the
output of the disassembler to its internal writer.

This replaces half of the uses of the `disassembler` feature, which
has been removed from the Cargo configuration.

Note that at this commit runtime tracing is not yet implemented as an
observer.

Change-Id: I7894ca1ba445761aba4ad51d98e4a7b6445f1aea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6449
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-09 21:10:10 +00:00
Vincent Ambo
7ae45342df feat(tvix/eval): implement (compilation) observer trait
This trait will enable library users of tvix-eval to observe internal
happenings of the compilation and runtime processes.

The initial methods of the observer will be called whenever the
compiler emits a chunk.

Change-Id: I668f6c2cfe3d6f4c1a1612c0f293831011768437
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6448
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-09 21:10:10 +00:00
Vincent Ambo
1fe6cfe5a2 refactor(tvix/eval): index into Chunk with ConstantIdx/CodeIdx
This is a step towards hiding the internal fields of thunk, and making
the interface of the type more predictable.

Part of the preparation for implementing observers.

Change-Id: I1a88a96419c72eb9e2332b56a2dd94afa47e6f88
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6447
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-09 21:10:10 +00:00
Vincent Ambo
8c64ebe074 chore(tvix/eval): gate REPL-only dependencies behind repl feature
With this change, it becomes possible to compile tvix-eval to
webassembly if the `repl` feature is disabled.

Change-Id: Icc0a059964cd0bea2054110c682d50fc5c87ec01
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6446
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-09 10:48:10 +00:00
Vincent Ambo
8fdb67847c chore(tvix/eval): debug_assert that all jumps are patched
Suggestion from sterni in cl/6282

Change-Id: I1adbdda9ff74f55a2f72892ffa524808b305f403
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6445
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-09 10:48:10 +00:00
Vincent Ambo
abdfa7459e feat(tvix/eval): thunk binary operations and select expressions
With this, most cases of `fix` in attribute sets will work correctly.
A simple test exercising both has been added.

Change-Id: I70fd431177bb6e48ecb33a87518b050c4c3d1c09
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6437
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
9f379ef6df fix(tvix/eval): hold thunk borrow as shortly as possible
At the point where control flow exits Thunk::force (which may be due
to recursing), it is vital that there is no longer a borrow to the
inner thunk representation, otherwise this can cause accidental
infinite recursion (which will be detected, but cause failures on
valid code).

Change-Id: I2846f3142830ae3110a4f5d2299e9d7928634504
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6436
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
48d5f4fd57 feat(tvix/eval): print lambda memory adresses in disassembler
This makes it easier to track exactly which lambda is which when
inspecting e.g. the concrete representation of a thunk.

At runtime all lambdas live in an Rc. To make this print the right
address, the construction of these Rcs had to be moved up right to the
point where the lambda is first emitted (and disassembled).

Change-Id: I6070e6c8ac55f0bd697966c4e7c5565c20d19106
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6435
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
d3421c1cb9 fix(tvix/eval): ensure disassembler prints continous lines correctly
There can be different spans on the same line, so the previous
implementation would duplicate line numbers unnecessarily.

Change-Id: I8d8db77177aee0d834a6ec3584641e1bd5f31c3e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6434
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
fc1c50498e feat(tvix/eval): thunk function applications
Change-Id: I18065ed234ec104ac74d0e1c2d0937c2d78ca7db
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6433
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
e129ce15d7 feat(tvix/eval): thunk creation of lists
Change-Id: I84b68c5d002ec613d278315bbf49e9839f0fe8e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6432
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
678533bfc4 test(tvix/eval): add test for stack slot accounting edge-case
This was fixed by some of the previous commits around scopes. It's
somewhat similar to a few other tests, but I had this one failing
earlier and everything else succeeding, so it is useful to keep it
around for sure.

Change-Id: Ie6cf372b5c805daf992cd87aeb3dfe91542c381c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6431
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
09eaa0d4ae fix(tvix/eval): address current clippy & grfn lints
Change-Id: I65c6feb9f817b5b367d37204a1f57acfe4100d97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6430
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
fe047885d7 fix(tvix/eval): consider local depth when deciding to defer
Deferred local upvalues can *only* occur at the same depth as the
thing that is closing over them, but there are various situations with
scope nesting where the actual stack indexes of the local and the
closer look like a deferred value is being accessed.

To fix this, simply compare the depth as well.

Change-Id: Ice77424cc87ab0a2c4f01379e68d4399a917b12b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6429
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 20:17:26 +00:00
Vincent Ambo
360c805efc chore(tvix/eval): remove accidentally duplicated test
This is the same as `eval-okay-attrs-simple-inherit`.

Change-Id: I23878accc6cd62c16ec96601239838a385d31306
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6428
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 19:54:02 +00:00
Vincent Ambo
0af1df4be2 refactor(tvix/eval): clean up logic in Compiler::end_scope
The condition here was extremely hard to read prior to this change.

As the locals vector is now guaranteed to never be empty (there is
always at least a phantom for the current chunk's root expression),
the logic here can be simplified to just dropping tailing locals
entries while their depth matches that of the scope being closed.

Change-Id: I24973e23bc2ad25e62ece64ab4d8624e6e274c16
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6427
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 19:54:02 +00:00
Vincent Ambo
23f248b530 fix(tvix/eval): set up root stack slot in closures & thunks
Similar to setting up a phantom slot when compiling the root value of
a file, closures and thunks need to have a phantom stack slot for the
root of the expression yielded by their thunk to make all accounting
work correctly.

The tricky thing here is that closures & thunks *escape* their inner
lambda context (that's the point!), so the functions emitting them
need to know both the *inner* slot (to resolve everything correctly
while compiling the slot) and the *outer* slot (to correctly emit
instructions for closing over upvalues).

Change-Id: I62ac58e2f639c4b9e09cc702bdbfd2373e985d7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6426
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 19:54:02 +00:00
Vincent Ambo
4e24bd56b4 fix(tvix/eval): only pop initialised locals when closing scopes
This avoids emitting OpPop instructions for locals that only existed
virtually (as uninitialised phantoms).

Change-Id: I8105afcca80c3f7b7ef93ce5e2f0d08a93f4ad27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6425
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 19:54:02 +00:00
Vincent Ambo
ecbd7c6ca1 fix(tvix/eval): ensure that root stack slot actually exists
Instead of using a sentinel LocalIdx which potentially points to a
value in the locals stack that does not actually exist, set up an
initial uninitialised phantom value representing the result of the
root expression.

Change-Id: I82ea774daab83168020a3850bed57d35ab25c7df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6424
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 19:54:02 +00:00
Vincent Ambo
f8575fcd74 fix(tvix/eval): compare *stack* slots when deciding whether to defer
When deciding whether an upvalue needs to have a deferred resolution
step, the *stack* indexes should be compared - not the locals indexes.

The results are almost always the same, but there are tricky
situations where this can cause errors.

It's difficult to reproduce these errors in isolation, as they depend
on other scope behaviour, so this is one in a series of commits to
address the combination of issues which will gain some tests at the
end.

Change-Id: Iaa400b8d9500af58f493ab10e4f95022f3b5dd21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6423
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 19:54:02 +00:00
Vincent Ambo
e9b6ecb0ee fix(tvix/eval): propagate scope depth when nesting scopes
Change-Id: Id441646db550f6195c2e247a0afbb5c9d91da8a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6422
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 13:36:26 +00:00
Vincent Ambo
9973ddfcba refactor(tvix/eval): refactor locals to use an enum for phantoms
Instead of using sentinel values and an additional bool, this tracks
the identifier of a local as an enum that is either a statically known
name, or a phantom.

To make this work correctly some more locals related logic has been
encapsulated in the `scope` module, which is a good thing (that's the
goal).

Phantom values are now not initialised by default, but the only
current call site of phantoms (`with` expression compilation) performs
the initialisation right away.

This commit changes no actual functionality right now, but paves the
way for fixing an issue related to `let` bodies.

Change-Id: I679f93a59a4daeacfe40f4012263cfb7bc05034e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6421
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 13:36:26 +00:00
Vincent Ambo
7bc6e5984d refactor(tvix/eval): always pass slot to compiler methods
The slot is now always known (at the root of the file it is simply
stack slot 0 once the scope drops back down to 0), so it does not need
to be wrapped in an `Option` and accessed in cumbersome ways anymore.

Change-Id: I46bf67a4cf5cb96e4874dffd0e3fb07c551d44f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6420
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 13:36:26 +00:00
Vincent Ambo
a303ea3ff5 refactor(tvix/eval): implement much clearer disassembler output
With this change the runtime trace contains much more exact
information about the context of the computation (entering/exiting
calls etc.)

This is in large part due to moving the tracer to be a field on the VM
itself, which enables consistent ordering of traces across the
execution, and tracing an execution with its *input* instead
of *output* stack.

Change-Id: Ibe525e6e7d869756501e52bef1a441619ce7332c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6419
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 13:36:26 +00:00
Vincent Ambo
60ff8d046c chore(tvix/eval): print value when panicking on internals
This makes it much easier to figure out what happened while debugging
this sort of thing.

Change-Id: I2e0e8096709adc647d63c04f213c547c415e5f44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6418
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 13:36:26 +00:00
Vincent Ambo
b83723a1ed fix(tvix/eval): force argument of builtins.length
Change-Id: Iaf94dfc7cb058f5c1c311066d92e01512e09cf71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6417
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 12:53:20 +00:00
Vincent Ambo
0c0ae50f02 fix(tvix/eval): don't panic when printing a black hole
This could occur when the disassembler is enabled and tracing the
runtime while a thunk is being evaluated, as it would not be possible
for the *tracer* to borrow the thunk at this exact moment.

However, we know that if the borrowing fails here we are dealing with
a not-fully evaluated thunk (blackhole), which should just print the
internal representation.

Change-Id: I4bdb4f17818d55795368e3d28842048f488f0a91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6416
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 12:53:20 +00:00
Vincent Ambo
2246a31e72 refactor(tvix/eval): return call frame result from VM::call
Previously, "calling" (setting up the VM run loop for executing a call
frame) and "running" (running this loop to completion) were separate
operations.

This was basically an attempt to avoid nesting `VM::run` invocations.
However, doing things this way introduced some tricky bugs for exiting
out of the call frames of thunks vs. builtins & closures.

For now, we unify the two operations and always return the value to
the caller directly. For now this makes calls a little less effective,
but it gives us a chance to nail down some other strange behaviours
and then re-optimise this afterwards.

To make sure we tackle this again further down I've added it to the
list of known possible optimisations.

Change-Id: I96828ab6a628136e0bac1bf03555faa4e6b74ece
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6415
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 12:53:20 +00:00
Vincent Ambo
cc526a2c87 feat(tvix/eval): thread codemap through to disassembler
If the disassembler feature is enabled, make sure that an Rc of the
codemap is available through the chunk.

Change-Id: I700f27ab665a704f73457b19bd2d7efc93828a16
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6414
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 12:53:08 +00:00
Vincent Ambo
a3b19ad8be docs(tvix/eval): add notes for builtins access optimisation
Change-Id: Iadbfbe2864ae42fe5492ef3ede0925baee4872b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6413
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 08:45:15 +00:00
Vincent Ambo
6b3c3c9826 refactor(tvix/eval): add macros for generating Value casters
The casting methods of `Value` are pretty verbose, and actually
incorrect before this commit as they did not account for inner thunk
values.

To address this, we first attempt to make them correct by introducing
a standard macro to generate them and traverse the inner thunk(s) if
necessary.

This is likely to be a performance hit as it will now involve more
cloning of values. We can do multiple things to alleviate this, but
should do some measurements first.

Change-Id: If315d6e2afe7b69db727df535bc6cbfb89a691aa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6412
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 08:45:15 +00:00
Vincent Ambo
0d7ad5e6d1 refactor(tvix/eval): pass a VM reference to builtins
This makes it possible for builtins to force values on their own,
without the VM having to apply a strictness mask to the arguments
first.

Change-Id: Ib49a94e56ca2a8d515c39647381ab55a727766e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6411
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 07:59:15 +00:00
Vincent Ambo
5ee89bcf5c fix(tvix/eval): inherit scope poisoning data in nested contexts
Scope poisoning must be inherited across lambda context boundaries,
e.g. if an outer scope has a poisoned `null`, any lambdas defined on
the same level must reference that poisoned identifier correctly.

Change-Id: I1aac64e1c048a6f3bacadb6d78ed295fa439e8b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6410
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 07:59:15 +00:00
Vincent Ambo
0a13d267f0 fix(tvix/eval): thread thunk forcing errors through correctly
With this, if an error occurs while forcing a thunk (which is very
likely) it is threaded through to the top by wrapping it in the
ErrorKind::ThunkForce variant.

We could use this to generate "stacktrace-like" error output if we
wanted, or simply jump through and discard everything except the
innermost error.

Change-Id: I3c1c8708c2f73ae062815adf490ce935b1979da8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6409
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 07:59:15 +00:00
Vincent Ambo
377ba19d75 feat(tvix/eval): ensure all errors always carry a span
Previously error spans were optional because the information about
code spans was not available at runtime. Now that this information has
been added, the error type will always carry a span.

This change is very invasive all throughout the codebase. This is due
to the fact that many functions that are called *by* the VM expected
to return `EvalResult`, but this no longer works as the span
information is not available to those functions - only to the VM
itself.

To work around this the majority of these functions have been changed
to return `Result<T, ErrorKind>` instead and an accompanying macro in
the VM constructs the "real" error.

Note that this implementatino currently has a bug where errors
occuring within thunks will yield the location at which the thunk was
forced, not the location at which the error occured within the code.
This will be fixed soon, but the commit is large enough as is.

Change-Id: Ib1ecb81a4d09d464a95ea7ea9e589f3bd08d5202
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6408
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-08 07:59:15 +00:00
William Carroll
197fe37dae feat(tvix/eval): Support builtins.length
Get the length of a list

Change-Id: I41d91e96d833269541a1b3c23b7cc879f96d1e5a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6407
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 20:56:02 +00:00
William Carroll
15e2bc54f1 feat(tvix/eval): Support builtins.{mul,div}
Multiplication and division :)

Change-Id: I79e76f3b55b7e9b8d1eb09b3d9741140b4d75742
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6406
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 20:56:02 +00:00
William Carroll
b3097de75d feat(tvix/eval): Support builtins.{add,sub}
First pass at supporting `builtins` for tvix. The following tests appear to be
WAI:

```shell
$ cd tvix/eval
$ cargo build
$ cargo test
```

Change-Id: I27cce23d503b17a886d1109e285e8b4be4264977
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6405
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 20:56:02 +00:00
Vincent Ambo
55d21a1389 refactor(tvix/eval): store spans instead of nodes in Warning/Error
Another step towards being able to report accurate errors. The codemap
spans contain strictly more accessible information, as they now retain
information about which input file something came from.

This required some shuffling around in the compiler to thread all the
right information to the right places.

Change-Id: I18ccfb20f07b0c33e1c4f51ca00cd09f7b2d19c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6404
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:26 +00:00
Vincent Ambo
8033300900 feat(tvix/eval): track source spans for builtin access
As of this commit, the source spans of all emitted bytecode are fully
tracked.

Change-Id: I4c83deee0fc3f5e6fd6acad5a39047aec693b388
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6403
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
6cbd580ba5 feat(tvix/eval): track source spans for OpForce instructions
These source spans will always point to the *value* that is being
forced, not the instruction that caused the force to be emitted. This
makes sense so that errors during forcing point at the value and not
the surrounding expression.

Change-Id: I4694414a3281a0de878f71634105b92148ec61f6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6402
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
1ea88fcb65 feat(tvix/eval): track source spans for scopes
Change-Id: Iaedb0742940e4c2b9f4210579ed28e6deb32d5c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6401
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
c269b44f41 feat(tvix/eval): track source spans for upvalues
With this change, the upvalue data instructions used by finalisers for
thunks and closures track the source span of the first identifier that
created the upvalue (if the same value is closed over multiple times
the upvalue will be reused, hence only the first one).

To do this the upvalue struct used by the compiler's scope now carries
an identifier node, which had to be threaded through quite a few
places.

Change-Id: I15a5fcb4c8abbd48544a2325f297a5ad14ec06ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6400
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
3efed26940 refactor(tvix/eval): split out Upvalue struct & UpvalueKind enum
This separation makes it possible to annotate the upvalue itself with
the span that created it, which (due to upvalue reuse) is only the
first one for an instance of the given UpvalueKind.

Change-Id: I9a991da6a3e8d71a92f981314bed900bcf434d44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6399
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
b8874f8d35 feat(tvix/eval): track source spans for thunks
Change-Id: I73f13c914d84e052a43b4eddb09db1acd8a7076a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6398
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
ec8ff79aee feat(tvix/eval): track source spans for function calls
Change-Id: If1bb463026414597acd561dc2032549fdb70f986
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6397
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
e590b585e7 feat(tvix/eval): track source spans for lambdas
Change-Id: I2f420fc915b6bfc5b197e9d0c128b539c4bc7467
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6396
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
20996601ea feat(tvix/eval): track source spans for with expressions
Change-Id: I1d58ce548b5b47e967928e85bb64acf5ed69ecc8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6395
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 20:04:17 +00:00
Vincent Ambo
9f5f85a1c1 feat(tvix/eval): track source spans for identifier access
Change-Id: I8e6ec0a84430d6e417fc7fd3e722a588913e4d18
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6394
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:23:22 +00:00
Vincent Ambo
f37e4a0b9c feat(tvix/eval): track source spans for let bindings
Change-Id: I9457917277a7fdd8bdbe227a567b28969312d06e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6393
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:23:22 +00:00
Vincent Ambo
7f687f0e99 feat(tvix/eval): track source spans for if expressions
These are again a bit tricky in terms of emitted errors. The main
error is that the condition is not a boolean, which means that the
jump inspecting the condition must derive from the condition itself to
return an error at the correct position.

For other parts of the expression, it is simply the node itself.

Change-Id: I72411630e5d57dfc199f4c3c48afe443fe966322
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6392
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
baa012d5a3 feat(tvix/eval): track source spans for assert
Change-Id: I5415f63ddde388847a261da4ce9a8d8235657535
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6391
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
805c1870ee feat(tvix/eval): track source spans for or operator
This one is tricky, specifically the span used for the final jump. I
decided that it makes sense to use the attrpath node, as the final
jump is the one that jumps *over* the default value, so the effect of
this is more closely related to the selector than the default.

It might be more correct to pass through the `or` token itself and
point to this for the jumps, but it depends a bit on what shape of
errors we could end up producing from this.

Change-Id: I29fbc97ba6b9e14e1a0e5f3a7759ddc299dd9c0c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6390
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
2b1468dde3 feat(tvix/eval): track source spans for attribute selects
Change-Id: Ifa8b0e7905f9d2746f83d6503ef0e8d42ce20f9c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6389
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
d1e0c6f680 feat(tvix/eval): track source spans for attrsets
Change-Id: I0fcb07146b5a38c67bc46ed3f67533b056858390
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6388
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
eb169c51a7 feat(tvix/eval): track source spans for lists
Change-Id: Icfd057ee29180cac17264bb3299c67eae0051aee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6387
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
d8f494b50b feat(tvix/eval): track source spans for literal identifiers
Change-Id: I7ca08c6c0124f653e55fcc86413a847f0a4c50c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6386
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
2ad89a00fb feat(tvix/eval): track source spans for ? operator
Change-Id: Idb842fb20fab14150455a81ea6c947e75b314d8a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6385
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
c417a084cc feat(tvix/eval): track source spans for binary operators
Change-Id: I43e433ff3094232b244d38335d31e56a79ca70c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6384
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
176e729556 feat(tvix/eval): track source spans for unary operators
Change-Id: I79034a4aa04aea66ec598e33e6eab35e1e19c0d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6383
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
f14b368514 feat(tvix/eval): track source spans for strings
Change-Id: I29e6c7f9e25d1b2e6bafa602c463eb9ccee2131b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6382
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
3c4eed9b76 feat(tvix/eval): track source spans for paths
Change-Id: I42fbd0bb6c2a8feb520e262a25f59ff27dcd035c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6381
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
eaf7af18be feat(tvix/eval): track source spans for literals
Change-Id: Icfe77f85c4f65b6bf28b8752c2795419e8e396ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6380
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
72adcdf965 feat(tvix/eval): add methods for emitting code with tracked spans
These are not actually used yet; this is in preparation for a
multi-commit chain for emitting all the right spans in the right
locations.

Change-Id: Ie99d6add2696c1cc0acb9ab928917a10237159de
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6379
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:23:22 +00:00
Vincent Ambo
c5a8b93eaf chore(tvix/eval): thread a codemap::File reference to the compiler
This instantiates a codemap outside of the compiler and passes a
reference to the file currently under compilation to it. Note that the
"file" might just be a REPL line.

Change-Id: I131ae1ddb6d718e1374750da9ba0b99608c6058d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6378
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:10:40 +00:00
Vincent Ambo
b9566da5c9 refactor(tvix/eval): add and use Compiler::push_op method
This is currently just a wrapper around Chunk::push_op, but will gain
the span resolution logic in a moment.

Change-Id: I862bf9ecff0932f8da6708401ea044b9442c5d5b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6377
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:08:56 +00:00
Vincent Ambo
6f6bbc12f8 feat(tvix/eval): add data structures for tracking spans in chunks
This adds a new vector to the chunk data structure which tracks spans
into a codemap. The compiler will emit this information to the chunk
when adding instructions.

The internal representation of the spans is slightly optimised to
avoid storing duplicate spans, as there are cases where many
instructions might be derived from the same span.

Change-Id: I336f8c912e7eb50ea02ed71e6164f651ca3ca790
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6376
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:08:56 +00:00
Vincent Ambo
18f8fecba4 chore(tvix/eval): bump all dependencies
Updated the rnix hash manually, and ran `cargo update` for the rest.

Change-Id: I457262625d648e25d745efa4d33ae44cb8f21326
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6375
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:08:56 +00:00
Vincent Ambo
39e960daa6 chore(tvix/eval): add codemap dependency
This will be used to track source spans when emitting bytecode.

The codemap is a data structure which tracks *all* the source files
visited by an evaluation, and makes it possible to represent locations
across all of the files using a simple span (i.e. pair of offsets).

When reporting errors, this even contains enough information to
reconstruct the rnix AST to create fancier reporting in certain cases
if desired.

Change-Id: I4ae98620b9b150fb5a389bd7f1e12670e3192c62
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6374
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 19:08:56 +00:00
Vincent Ambo
f8919dbcd6 fix(tvix/eval): address current clippy lints
Note that I've allowed `needless_lifetimes` for the attribute set
iterator, as I find the type easier to understand with these
annotations present.

Change-Id: I33abb17837ee4813076cdb9a87f54bac4a37044e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6373
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
afcfa78364 feat(tvix/eval): semi-strictly evaluate output values of the VM
This essentially makes the VM behave like `nix-instantiate --eval
--strict`, i.e. data structures are traversed strictly and thunks are
forced. Thunks embedded in closures are not forced.

This allows us to re-enable tests that were disabled because they
needed to output nested thunk contents, but is overall a behaviour
that must be configurable later on, as it is not cmopatible with e.g.
an evaluation of nixpkgs.

Change-Id: I5303a5c8e4322feab1384fdb7712fecb950afca5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6372
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
6c6f9ae661 feat(tvix/eval): implement NixList::iter
This does not require a custom iterator type (for now?)

Change-Id: I5beb194bd8629571bd4040c69c977c27149807fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6371
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
fea7f90a7f fix(tvix/eval): thread Display & PartialEq through to thunk values
If a thunk is already evaluated, there are cases where due to the
memoisation implementation something might observe a value wrapped in
a thunk.

In these cases, the implementation of `Display` and `PartialEq` must
delegate to the underlying value.

Note that there are a handful of other cases like these which we need
to cover.

It is a little tricky to write integration tests for these directly,
especially as some of the open-upvalue optimisations coming down the
pipe will reduce the number of observable thunks.

One test that covers a part of this behaviour is currently
disabled (needs some more machinery), but it's being brought back in
the next commits.

Change-Id: Iaa8cd338c12236af844bbc99d8cec2205f0d0095
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6370
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
38d3db5fb8 feat(tvix/eval): implement NixAttrs::iter()
Implementing iteration over NixAttrs requires a custom iterator type
in order to encapsulate the different representations. The BTreeMap
for example has its own iterator type which needs to be encapsulated.

This is mostly boilerplate code, but for a change some simple unit
tests have been added in.

Change-Id: Ie13b063241d461b810876f95f53878388e918ef2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6367
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
7d5dca7ba3 chore(tvix/eval): provide 'static references to "name"/"value"
These static strings show up a bunch when dealing with the internals
of attribute sets, and having them available as static references is
required.

Due to the way const expressions are evaluated, taking a reference to
the existing NixString::NAME / NixString::VALUE items does not work
and the references themselves need to be const-evaluated.

Change-Id: If6e75847af978118a3b266fe6a3242321722434d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6366
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
80713f207e refactor(tvix/eval): encapsulate all thunk-forcing logic in module
The VM previously took care of repeatedly forcing a thunk until it
reached an evaluated state. This logic is now encapsulated inside of
the `Thunk::force` implementation.

In addition, force no longer returns a reference to the value by
default, leaving it up to callers to decide whether they want to
borrow the value or not (a helper is provided for this).

Change-Id: I2aa7da922058ad1c57fbf8bfc7785aab7971c02b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6365
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
23a5caabec feat(tvix/eval): construct attribute sets lazily
This thunks the construction of attribute sets. Because Tvix does not
currently have a "strict output" mode, a test had to be disabled that
now displays a thunk representation.

The test will be re-enabled once that is available.

Change-Id: I360332be64cd5c154f9caea21828f6f1b37a265c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6363
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
727845645d chore(tvix/eval): add release-with-debug cargo profile
This lets us create a release build with debug info, for use with e.g.
perf + hotspot

Change-Id: I03897de36c872d318abf1332ca0c1aeabe344ec6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6362
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
79a2ba5175 fix(tvix/eval): avoid recomputing width in disassemble_op constantly
As noticed by sterni in cl/6195

Change-Id: Ie9c1e80e2e709284fa8412334af9188d999f64dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6361
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
a8b2ba07df fix(tvix/eval): correctly resolve identifiers in inherit
This should not have grown a second implementation of the identifier
resolution logic, but it somehow did.

This implementation ended up being incorrect because it did not
account for upvalues inside of thunks.

Change-Id: Ieb1364d8fe43c96aaf4b125fd4b8a522aedff167
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6360
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
17dfb92a9f chore(tvix/eval): implement Debug for compiler::scope::Scope
Change-Id: I112b0119bd0511f26bb72f7e73d867d1b7144a36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6359
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
9420a3b53d feat(tvix/eval): insert strictness points for attribute set keys
All attribute set *key* related operations strictly evaluate all key
fragments, including during construction of an attribute set.

Change-Id: I3519e5e9b0886c2cdc8615ea7dcb5f7be0c59b3f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6358
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
5b4f3811e6 feat(tvix/eval): insert strictness points for unary/binary operators
The arguments of all unary/binary operators that are built in to Nix
are forced when encountered. This emits the necessary OpForce operations.

Change-Id: I691fcdbebfe7586cfe217c68d44b10b1192f82d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6357
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
6a42d9bddf chore(tvix/eval): thread slot value through all compiler methods
With this change any compilation of an expression is aware of its own
stack slot if it is leaving identifiers on the stack.

Change-Id: I0c9f148ae06b078a46b25180c4961686d5f2e166
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6356
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
50c33989dc chore(tvix/eval): thread slot value through Compiler:compile_attr
This threads the current local slot through the `compile_attr`
function and all of its callers. At the moment this does not improve
any user-facing behaviour, just internally changes the way in which
some correct expressions would fail to run.

Eventually this slot will need to reach everywhere ...

Change-Id: Iba73123dd1ced421093d8fc18ebeeffc16efacf8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6355
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
372bf4a0b7 test(tvix/eval): test very simple late-bound identifier access
This is the simplest kind of thunk that can be created (and so far the
only one the compiler knows how to create), in which an identifier
inside a `let` encounters a value that is bound *after* it is
initialised.

Change-Id: I6ea4408a3baef1e7d5137365d70804283f2dbf8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6354
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
786e12a738 feat(tvix/eval): always emit OpForce as the last instruction
Change-Id: Id70c987f654dc5d9b47db74e395281309762b468
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6353
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
9a783e50a4 feat(tvix/eval): implement OpForce in VM
This operation forces the evaluation of a thunk.

There is some potential here for making an implementation that avoids
some copies, but the thunk machinery is tricky to get right so the
first priority is to make sure it is correct by keeping the
implementation simple.

Change-Id: Ib381455b02f42ded717faff63f55afed4c8fb7e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6352
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
a8c13a5ce1 fix(tvix/eval): allocate Thunk::upvalues with known capacity
The capacity (i.e. number of builtins) is known from the lambda, so we
can size it correctly right away.

Change-Id: Iab0b5a3f47d450fa9866c091ebbbed935b934907
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6351
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:54:42 +00:00
Vincent Ambo
6c895d4b28 feat(tvix/eval): implement OpThunk for runtime thunk construction
Implements an operation very similar to `OpClosure` which populates a
thunk's upvalues and leaves it on the stack.

Change-Id: I753b4dfeeaae6919316c7028ec361aaa13d87646
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6350
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:54:42 +00:00
Vincent Ambo
6c4b9bdce9 refactor(tvix/eval): extract VM::populate_upvalues function
This function is reusable between thunks & closures.

Change-Id: I44d5f9897b087a385c8e75027d2ff39c48a096f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6349
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:36:46 +00:00
Vincent Ambo
ce498052e6 refactor(tvix/eval): Carry lambda & upvalues directly in CallFrame
CallFrame has to work for both thunks & closures (as a thunk is
basically a "weird 0-argument closure").

We opt to store the common, relevant fields directly in the frame to
avoid having to dereference through the nested structures
constantly (which would be especially annoying in the case of thunks).

Change-Id: I47781597b84ec5cd55502dba1713e92cf2592af3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6348
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
25c62dd0ef refactor(tvix/eval): introduce UpvalueCarrier trait
This trait abstracts over the commonalities of upvalue handling
between closures and thunks.

It allows the VM to simplify the code used for setting up upvalues,
without duplicating between the two different types.

Note that this does not yet refactor the VM code to optimally make use
of this.

Change-Id: If8de5181f26ae1fa00d554f1ae6ea473ee4b6070
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6347
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
8033a7abae refactor(tvix/eval): simplify thunk representations
For now, do not distinguish between closing and non-closing thunks, it
will make the initial implementation easier. See Knuth etc.

Change-Id: I0bd51e0f89f2c77e90bac63b507e5027b649e3d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6346
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
28a9847c65 feat(tvix/eval): emit thunks for recursive local scope resolution
When resolving a value on the same level that is known but not yet
defined, emit a thunk.

Consider for example:

    let
      #       v--- requires a thunk
      a = 1 * b;
      b = 10;
    in a

Change-Id: I922cb50973ebe05e335a7bc7cb851960cf34733b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6345
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
d1798444be feat(tvix/eval): Add Compiler::thunk method for emitting thunks
The logic in this method is *very* similar to `compile_lambda`. It is
intended to be called around any expression that should be
thunked (such as function applications, attribute set values, etc.).

Change-Id: Idfbb2daa9f4b735095378fb9c39a2fd07c8cff91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6344
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
1f37275cfb feat(tvix/eval): introduce Value::Thunk variant
Introduces the representation of runtime thunks, that is lazily
evaluated values. Their representation is very similar to closures.

Change-Id: I24d1ab7947c070ae72ca6260a7bbe6198bc8c7c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6343
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
49296cebe3 refactor(tvix/eval): extract compiler's upvalue logic into helper
This exact same logic is reused for thunk creation.

Change-Id: I731db9cc659a1f2ca87db55d58d6ff632f417812
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6342
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
70b27a6080 test(tvix/eval): add a test case for nested, deferred upvalues
This test case was previously broken by the bug introduced by
confusing local and stack indexes.

Change-Id: Ibef299dad266c6105deac1da5dde112fe9f640b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6341
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
24811d7655 fix(tvix/eval): distinguish statically between StackIdx and LocalIdx
Previously the functions in the scope module returned usize values,
which - sometimes from the same function - were either indexes into
the runtime stack *or* indexes into the compiler's local stack.

This is extremely confusing because it requires the caller to be aware
of the difference, and it actually caused subtle bugs.

To avoid this, there is now a new LocalIdx wrapper type which is used
by the scope module to return indexes into the compiler's stack, as
well as helpers for accounting for the differences between these
indexes and the runtime indexes.

Change-Id: I58f0b50ad94b28a304e3372fd9731b6590b3fdb8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6340
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
68aad6d4cf feat(tvix/eval): emit OpFinalise when local scopes are complete
With this change, the runtime can correctly capture deferred upvalues.

Change-Id: I1e43b7b1ac2553b1812424adfc8bd08ef77bf1ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6339
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
5c4e102ac8 feat(tvix/eval): track whether locals needs to be finalised
When encountering a deferred local upvalue, the compiler will now mark
the corresponding local as needing a finaliser which makes it possible
to emit the OpFinalise instruction for this stack slot a little bit
down the line.

Change-Id: I3962066f10fc6c6e1472722b8bdb415a811e0740
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6338
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
025a9a4a0a feat(tvix/eval): implement OpFinalise instruction
This instruction finalises the initialisation of deferred upvalues in
closures (and soon, thunks).

The compiler does not yet emit this instruction, some more accounting
is needed for that.

Change-Id: Ic4181b26e19779e206f51e17388559400da5f93a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6337
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
1bfe32f412 feat(tvix/eval): set up deferred upvalues at runtime
This will leave a sentinel value in the upvalue slot in which the
actual value is to be captured after resolution once a scope is fully
set up.

Change-Id: I12b37b0dc8d32603b03e675c3bd039468e70b354
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6336
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
6c1948a71a feat(tvix/eval): detect deferred upvalue capturing
Uses the threaded through slot offset to determine whether
initialisation of a captured local upvalue must be defered to a later
point where all values of a scope are available.

This adds a new data representation to the opcode for this situation,
but the equivalent runtime handling is not yet implemented. This is in
part because there is more compiler machinery needed to find the
resolution point.

Change-Id: Ifd0c393f76abfe6e2d91483faf0f58947ab1dedc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6329
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-06 14:58:52 +00:00
Vincent Ambo
203d9f2e3e chore(tvix/eval): thread current slot index through compiler
While compiling local bindings, we now know the stack slot of the
value currently being compiled.

This will let us determine whether an upvalue can be captured directly
or whether it needs to wait for a synchronisation point at which the
upvalue can be instantiated.

This machinery lets us avoid unnecessary work at runtime when
instantiating closures that actually do not need complicated recursive
resolution.

This change itself introduces no new functionality, but since the
threading is noisy it is split out as a separate change.

Change-Id: I847c677ee8f6725fda1d2efd689b6a58bdccb779
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6328
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-06 14:58:52 +00:00
Vincent Ambo
9dd1fa875d refactor(tvix/eval): optimise initialisation of locals
Instead of looking up the local to be initialised by its name again,
we can simply track the index at which it was declared from the point
where the declaration was made.

This reduces some string cloning and removes unnecessary logic. It
also theoretically makes the *current* index available during locals
compilation, which can be used to optimise some recursion cases.

Change-Id: I06f403603d4f86c3d319debfe74b5a52eec00990
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6327
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
900a92935d refactor(tvix/eval): declare all locals before compiling them
This actually makes things full-circle, as this tree already had this
implementation once before all the other required components were in
place.

With this commit, the compiler can resolve recursive upvalues within
the same scope (though they will not yet work at runtime).

Change-Id: I6267e477d08f367257c3a6dde054b880d7b47211
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6326
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
47b3562867 refactor(tvix/eval): decouple local depth & initialisation tracking
In order to resolve recursive references correctly, these two can not
be initialised the same way as a potentially large number of (nested!)
locals can be declared without initialising their depth.

This would lead to issues with detecting things like shadowed
variables, so making both bits explicit is preferable.

Change-Id: I100cdf1724faa4a2b5a0748429841cf8ef206252
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6325
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
705c1b9d53 fix(tvix/eval): correct runtime error for missing dynamic upvalue
Change-Id: I75c351619780fdc5186a54f3df9b244ada984069
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6324
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
776120de05 fix(tvix/eval): instantiate *new* closures from blueprints each time
The previous closure refactoring introduced a bug in which the same
closure object would get mutated constantly for each instance of a
closure, which is incorrect behaviour.

This commit instead introduces an explicit new Value variant for the
internal "blueprint" that the compiler generates (essentially just the
lambda) and uses this variant to construct the closure at runtime.

If the blueprint ever leaks out to a user somehow that is a critical
bug and tvix-eval will panic.

As a ~treat~ test for this, the fibonacci function is being used as it
is a self-recursive closure (i.e. different instantiations of the same
"blueprint") getting called with different values and it's good to
have it around.

Change-Id: I485de675e9bb0c599ed7d5dc0f001eb34ab4c15f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6323
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
ccfb971dc5 fix(tvix/eval): correctly thread through dynamic upvalues
This puts together the puzzle pieces for threading dynamic
upvalues (that is, upvalues resolved from the `with`-stack) all the
way through.

Reading the test case enclosed in this commit and walking through it
is recommended to understand what problem is being tackled here.

In short, because the compiler can not statically know *which*
with-scope a dynamic argument is resolved from it needs to lay the
groundwork for resolving from *all* possible scopes.

There are multiple different approaches to doing this. The approach
chosen in this commit is that if a dynamic upvalue is detected, the
compiler will emit instructions to close over this dynamic value
in *all* enclosing lambda contexts.

It uses a new instruction for this that will leave around a sentinel
value in case an identifier could not be resolved, and wire the
location of this found value (or sentinel) up through the upvalues to
the next level of nesting.

In this tradeoff, tvix potentially closes over more upvalues than are
needed (but in practice, how often do people create *really* deep
`with`-stacks? and in *this* kind of code situation? maybe we should
even warn for this!) but avoids keeping the entire attribute sets
themselves around.

Looking at the test case, each surrounding closure will close
over *all* dynamic identifiers that are referenced later on visible to
it, but only the last one for each identifier will actually end up
being used.

This also covers our bases for an additional edge-case this creates,
in which an identifier potentially resolves to a dynamic upvalue *and*
to a dynamic value within the function's own scope (again, would
anyone really do this?) by introducing a resolution instruction for
that particular case.

There is likely some potential for cleaning up this code which is
quite ugly in some parts, but as this implementation is now carefully
calibrated to work I decided it is time to commit it and clean it up
in subsequent commits.

Change-Id: Ib701e3e6da39bd2c95938d1384036ff4f9fb3749
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6322
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
8982e16e26 refactor(tvix/eval): thread dynamic upvalues through all contexts
With this change, encountering a dynamic upvalue will thread through
all contexts starting from the lowest context that has a non-empty
`with`-stack.

The additional upvalues are not actually used yet, so the effective
behaviour remains mostly the same. This is done in preparation for an
upcoming change, which will implement proper dynamic resolution for
complex cases of nested dynamic upvalues.

Yes, this whole upvalue + dynamic values thing is a little bit
mind-bending, but we would like to not give up being able to resolve a
large chunk of the scoping behaviour statically.

Change-Id: Ia58cdd47d79212390a6503ef13cef46b6b3e19a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6321
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
f6de4434c3 feat(tvix/eval): allow ignoring locals by prefixing with _
This is a common idiom in both Nix and other languages when a local is
declared without actually being used.

Since Tvix warns for unused locals, having this available is useful
and can be included in the final error message as a suggestion if an
unused variable is intentional.

Change-Id: Ia85f704ba183499a3bae657c58166e2e29f9bde5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6320
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
a5e22c532b fix(tvix/eval): correctly resolve dynamic upvalues one scope up
This does not yet correctly resolve them if they are more than one
scope up, however.

Change-Id: I6687073c60aee0282f2b6ffc98b34c1e96a60f20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6319
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
e46a2ce3ef fix(tvix/eval): account for stack offset when pushing a with scope
Change-Id: I4b98eaea3ed5059c29938a117a9d59499a0bb95d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6318
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
ae531a2245 feat(tvix/eval): implement capture of self-recursive upvalues
With this change, it becomes possible for functions to call themselves
as they are being defined in local bindings.

Change-Id: Ib46a39ba17b1452b5673d96fa729d633d237241a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6314
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
06b07f5c47 docs(tvix/eval): start a document on known optimisation potential
Change-Id: I9bc41e57e1cfdf536d7b9048bac2e7aff1ee2ffa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6313
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
3089e46eb1 refactor(tvix/eval): encapsulate internal mutability within Closure
This is required to efficiently construct the upvalue array at
runtime, as there are situations where during Closure construction
multiple things already have a reference to the closure (e.g. a
self-reference).

Change-Id: I35263b845fdc695dc873de489f5168d39b370f6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6312
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
sterni
18efbac5c5 docs(tvix/eval): pad pure column so it can fit "false"
Change-Id: Iaedfc281db82de1e8eb2400db1118c8431d2579f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6333
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-04 21:05:57 +00:00
sterni
e25972877a docs(tvix/eval): add a list of builtins added in Nix >= 2.4
`builtins.getFlake` doesn't interest us, of course, but some others may
be worth (or easy) to implement. They are pretty low priority, though,
since nixpkgs has compatiblity wrappers for the ones it uses.

The new debugging-related builtins (break and traceVerbose) are
interesting to note, but may not make sense to implement at all.

Change-Id: Icae547aa3bd9d6ee6b87897ba8210eb9b9b044c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6332
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-04 21:02:51 +00:00
sterni
b0d39aa19f fix(tvix/eval): don't rebuild drv upon changes in cargo target dir
This avoids unnecessary rebuilds of //tvix/eval when working on it
locally using impure cargo.

Change-Id: I028033a6345a9655e2877a534448706b8f85a1a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6317
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-04 21:02:50 +00:00
Vincent Ambo
932a70d0c2 refactor(tvix/eval): track with stack size as a simple integer
The `With` struct no longer contained any internals after the cleanup
logic for the stack had been moved into Compiler::compile_with,
leaving the `Vec<With>` to essentially act as a counter for the number
of things on the with stack.

That's inefficient of course, so with this commit it actually becomes
an integer (with an encapsulated API within scope::Scope).

Change-Id: I67a00987fc8b46b30d369a96d41e83c8af5b1998
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6311
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:53:08 +00:00
Vincent Ambo
2cd08d136e refactor(tvix/eval): move compiler's scope logic to separate module
The compiler module is getting quite long and this will help keep some
order.

Right now the scope internals are not very well encapsulated; this
paves a way to reducing the API surface of the `scope` type to the
things that are actually used by the compiler instead of giving access
to its internals.

Change-Id: I8c16c26d263f018baa263f395c9cd80715199241
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6310
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:53:08 +00:00
Vincent Ambo
83aa5917b4 fix(tvix/eval): declare locals before marking them initialised
This has no effect yet, other than changing the way in which some
upvalue captures break (that are already not working correctly).

However, after this change the compiler correctly detects
self-recursion and can start emitting the instructions to deal with
this at runtime.

Change-Id: Id3b0ac206c0204739597a4325bcc66f9c806c242
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6309
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:53:08 +00:00
Vincent Ambo
4bf096ee6e fix(tvix/eval): Account for uninitialised variables in with_idx
Calculating the with_idx (i.e. the stack offset of the "phantom"
variable from which a `with` dynamically reads at runtime) needs to
account for unitialised variables the same way as the resolution of
normal locals does.

Change-Id: I9ffe404535bf1c3cb5dfe8d9e005798c857fff94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6308
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:53:08 +00:00
Vincent Ambo
27bc8cb3d4 fix(tvix/eval): open/close additional scope around with
This is required to correctly clean up the `with` values.

At the moment, the attrset from which identifiers are being taken is
always pushed on the stack. This means that it must also be removed
again, otherwise in an expression like

  with { a = 15; }; a

The final stack is `[ { a = 15; } 15 ]` *after the last operation*,
which means that the attrset is still on there as garbage.

This has little practical impact right now because it is always
shadowed by the fact that the actual expression value is at the right
location, but becomes relevant when accounting for upvalue captures.

Change-Id: I69e9745bfaa4d6bbcb60ee71f4dc3f8d8695d16a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6303
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
abd8f12f5d refactor(tvix/eval): extend resolve_local logic for self-recursion
This extends the logic of `Scope::resolve_local` to detect cases where
self-recursion is occuring (i.e. an identifier is being accessed in
its own identifier).

These cases are not yet handled specially, and the logic of when
things are marked initialised (which was previously always at the same
spot as their declaration) has not changed, making this commit a
runtime no-op for now.

Change-Id: I3179642a7c55869ad4465fdd2678b0cd51a20f15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6302
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
a52db14820 feat(tvix/eval): detect illegally shadowed variables
Nix does not allow things like `let a = 1; a = 2; in a`, but doing it
across depths is allowed.

Change-Id: I6a259f8b01a254b433b58c736e245c9c764641b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6301
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
00aeb6dfaf refactor(tvix/eval): introduce Depth enum to track variable status
This does not yet change anything semantically, but will be useful for
resolving simple cases of self-recursion etc.

Change-Id: I139ecb7e4a8a81193774392a96e73e0ea6b9f85d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6300
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
33cde1422e feat(tvix/eval): implement upvalue resolution in with scopes
These need to be handled specially by the runtime if the compiler
determines that a given local must be resolved via `with`.

Note that this implementation has a bug: It currently allows `with`
inside of nested lambdas to shadow statically known identifiers. This
will be cleaned up in the next commit.

Change-Id: If196b99cbd1a0f2dbb4a40a0e88cdb09a009c6b9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6299
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
10b0879c00 fix(tvix/eval): ensure OpResolveWith can be traced
The previous implementation of OpResolveWith manually controlled the
loop iteration, which skipped over the disassembler's tracing
instruction.

Instead, the resolution of dynamic variables has been delegated to a
new helper function. This has the additional benefit that the loop
labels are no longer required, making things a bit cleaner.

Change-Id: If22b74c3d49c74bf3a1ec4497cb761a9ee6cf2a4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6298
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
7c2fdefcd8 fix(tvix/eval): pop with stack immediately after processing body
Instead of tying the popping of the with stack to scope depth, clean
up the stack immediately after processing a with body.

The previous behaviour was actually incorrect, as it would leave
things on the with-stack longer than they were supposed to be there.
This could lead to false positive resolutions in some situations
involving closures.

Change-Id: I7b0638557503f1f71eb602e3d5ff193cdfcb67cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6297
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
26acc2e636 test(tvix/eval): add tests for very simple closures
Change-Id: Ib8287ade4d5df6d29e1812fb2d349cee5d92ca6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6296
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
f88d248f48 feat(tvix/eval): implement runtime closure construction (OpClosure)
Implements the final bit of logic remaining for wiring up closures,
which is the runtime construction of closure objects.

When encountering an OpClosure, the VM walks through the bytecode
collecting all the upvalue location operands (see commit introducing
the OpCode::Data* variants for details) and stores the runtime values
in the new closures upvalue vector.

After that, the handling of the closure itself becomes functionally
identical to that of lambdas.

With this initial implementation of closures there are several large
optimisation potentials available, the two most notable ones are:

- Distinguish the runtime representation of lambdas and closures
  explicitly.

- Detect and handle multiple-arity functions directly in the compiler.

However, for both of these we should wait until we have appropriate
benchmarking infrastructure in place. This is because our test
implementations have shown that the complexity of either of these
changes is quite significant, and we do not yet know if they really
pay off.

Change-Id: I077e977810fd5cb2b1ecd7f1a119e728025dd786
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6295
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
0f06d0ca33 feat(tvix/eval): implement OpGetUpvalue in the VM
This resolves an upvalue at runtime by pushing it on the stack from
the closure's upvalue vector.

Change-Id: Ic3e7a7ecd9f7032f679114a1995e5bbf83062fcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6294
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
3d8888a13e refactor(tvix/eval): store Closure inside of the VM's call frames
In preparation for implementing calling of closures, store a closure
directly in the VMs call frame.

Change-Id: Iad24cd8c49fee4ebd4d0c84ffaa4c2505ee3dfd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6293
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
5ecf573dff feat(tvix/eval): add Value::to_closure
... same as the others

Change-Id: I9c8868388c10b0b6484c5bdd3799d801296c6979
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6292
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
69d8f17a26 feat(tvix/eval): compile creation of closure objects
Fully implements the instructions for compiling closure
objects (without runtime handling yet).

Closure (and thunk) objects are created at runtime by capturing all
known upvalues. To represent this, the instructions for creating them
need to have a variable number of arguments. Due to that, this commit
introduces new variants in OpCode that are not actually operations,
but data.

If the VM is implemented correctly, the instruction pointer should
never point at these. Due to this, the VM will panic if it sees a data
operand during an execution run.

Change-Id: Ic56b49b3a42736dc437751e76df0e89c8d0619c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6291
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
e4fadfaaf8 chore(tvix/eval): print node representation for compiler errors
Better for development flow.

Change-Id: I038efb39caca804f28a44fd4c83457e90abbcee4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6290
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
1163ef3e41 feat(tvix/eval): implement compilation of upvalue access
This adds a new upvalue tracking structure in the compiler to resolve
upvalues and track their positions within a function when compiling a
closure.

The compiler will emit runtime upvalue access instructions after this
commit, but the creation of the runtime closure object etc. is not yet
wired up.

Change-Id: Ib0c2c25f686bfd45f797c528753068858e3a770d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6289
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
2f93ed297e refactor(tvix/eval): add opcode::Count type for less ambiguity
Change-Id: Ibde0b2baa1128a74c1364ee9a6330b62db3da699
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6288
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:27:40 +00:00
Vincent Ambo
c73e84d957 refactor(tvix/eval): add opcode::StackIdx type for less ambiguity
Change-Id: I9b9de1f681972c205d4d20bc5731d2ce79858edb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6287
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:22:42 +00:00
Vincent Ambo
2cdc6192b4 docs(tvix/eval): add docstrings for usize wrappers in opcode
Change-Id: I11b9324233c0aa48bd2fbac15a484962f925e72e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6283
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:22:42 +00:00
Vincent Ambo
4f00f75e31 refactor(tvix/eval): add opcode::JumpOffset type for less ambiguity
This adds a transparent wrapper around `usize` used for jump offsets
in the opcodes. This is a step towards getting rid of ambiguous plain
`usize` usage in the opcode.

Change-Id: I21e35e67d94b32d68251908b96c7f62b6f56a8bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6282
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:22:42 +00:00
Vincent Ambo
3b64b7eb2e refactor(tvix/eval): rename CompilationResult -> CompilationOutput
grfn pointed out in cl/6174 that `Result` might cause developers to
believe that this behaves like std::Result, which it does not.

Change-Id: Ia30ab0dcb7e8da7bf842777ee3fe17bcf35cb0c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6281
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:19:48 +00:00
Vincent Ambo
7d34e6e880 refactor(tvix/eval): explicitly discard uninteresting results
Change-Id: I95a2ad61d9512b91017c801f325d0193b4da9c7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6280
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:19:48 +00:00
Vincent Ambo
3a2fcc8bc2 refactor(tvix/eval): avoid cloning in NixAttrs::update if possible
Refactors the update function to take the attribute sets by value
instead.

To facilitate this, we use an equivalent of the currently unstable
`Rc::clone_or_unwrap` in the VM when encountering attribute sets, so
that in cases where the only references to the attrs being updated are
the ones on the stack those clones are avoided completely.

This does make update() a little bit more tricky internally, as some
optimised branches can directly return the moved value, and others
need to destructure with ownership. For this reason there are now two
different match statements handling the different ownership cases.

Change-Id: Ia77d3ba5c86afb75b9f1f51758bda61729ba5aab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6279
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:19:48 +00:00
Griffin Smith
3270817b90 test(tvix/eval): Add attr merge benchmarks
Add a quick couple of benchmarks for merging attribute sets, large and
small.

Change-Id: I26940a9cf4e0d30e3d9eb07a7b8c366ca4072ca3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6286
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: grfn <grfn@gws.fyi>
2022-09-03 06:56:02 +00:00
Vincent Ambo
48b0fac76b refactor(tvix/eval): slightly more readable AttrsRep::select
Suggestion from grfn in cl/6158.

Change-Id: I16dcf2296a5ec5d299d5a080ca099b8eda6c254e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6278
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
7da5076191 refactor(tvix/eval): rename Value::NotFound & OpAttrOrNotFound
grfn suggested clearer naming for these in cl/6166.

Change-Id: I83164bf1d1902ebd42272e9d5d63819a0f6a72c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6277
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
903b57be04 docs(tvix/eval): add doc comment on compiler::patch_jump
Change-Id: Ifdd7b99223d239d955ac7eeeae95db97eb742bf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6276
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
bc9351f811 refactor(tvix/eval): get rid of Value::Blackhole variant
This is no longer needed for anything and the extra clone here is not
really more costly than constructing a blackhole value in a different
place.

Change-Id: I5c63085b1b4418b629ea58a42e3bfe9a9b586d76
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6275
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
d761d934ed test(tvix/eval): add a test for float representation
Change-Id: I4893a37719b9bf08b35963d48e6851a194a08aa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6274
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
8d7ccc78d0 docs(tvix/eval): add a note on how to run Nix tests
Change-Id: I9cd61ac79ed11b4c6580f31c5af5ebbfd45054b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6273
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
edee8eecdf fix(tvix/eval): address all current clippy lints
Change-Id: I758fc4f3b9078de7ca6228a75a4351c3e085c4cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6272
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
39b01c3029 fix(tvix/eval): correctly escape ${ in strings
Without this escape, it is possible for Nix to produce escaped
representations which are not literal Nix values again.

This was fixed in upstream Nix in
https://github.com/NixOS/nix/pull/4012 (though only for eval, not in
the REPL) and the updated test is picked from upstream after that commit.

Because we run the C++ Nix tests against our test suite as well, this
also bumps our custom Nix 2.3 to a commit that includes the
cherry-picked fix from the PR above.

Change-Id: I478547ade65f655c606ec46f7143932064192283
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6271
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
265393301e refactor(tvix/eval): move resolve_local to Scope struct
This is a more sensible place for this function to live and makes
upvalue resolution easier down the line.

Change-Id: I48ee39bdcdb4f96a16a327f7015aff60db5b15fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6270
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
6ce2c666c3 refactor(tvix/eval): introduce Closure struct in Value type
This struct will carry the upvalue machinery in addition to the lambda
itself. For now, all lambdas are wrapped in closures (though
technically analysis of the environment can later remove innermost
Closure wrapper, but this optimisation may not be worth it).

Change-Id: If2b68549ec1ea4ab838fdc47a2181c694ac937f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6269
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Griffin Smith
af9dca3663 chore(tvix/eval): Build with --all-targets
Primarily to make sure we build benchmark targets, and avoid breaking
them

Change-Id: I0c43f4cf99ddfd38e7545ef2d8276ef6b240a1e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6285
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-02 19:10:45 +00:00
Griffin Smith
29f8f06319 fix(tvix/eval): Fix build of benchmarks
Interpret was updated to take an optional path arg in
6fe5e2d75 (feat(tvix/eval): resolve relative path literals, 2022-08-12),
but since benchmarks aren't building in CI the resulting breakage of
benchmarks was missed.

Change-Id: I8a93f1b25ae62e2d032fafc153d91977c6466712
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6284
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-02 19:10:34 +00:00
Vincent Ambo
b29ab1776b chore(tvix/eval): move compiler module to a new folder
Change-Id: I76157f9cf1369cd17506de1b1ded1a4fd06f004a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6268
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 15:45:43 +00:00
Vincent Ambo
1416f1ab8a refactor(tvix/eval): avoid a use of Value::Blackhole
The blackhole allocation is not going to be cheaper than cloning this.

Change-Id: Id3ad44812decb4392830be06645e67bb0a982b96
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6267
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 15:45:43 +00:00
Vincent Ambo
babc249316 refactor(tvix/eval): separate out let inherit ... logic
Compilation of `let`-expressions is going to become a lot more
complicated due to attempts to avoid thunking when encountering
internal references, so this is just being moved out of the way.

Change-Id: Iecfa4b13d14532e21c2540e6561b4235ce29736a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6266
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 15:45:43 +00:00
Vincent Ambo
b8f36ba097 chore(tvix/eval): print slightly more information about warnings
This is just for dev comfort, it's not going to be useful for the
final version.

Change-Id: I05fdd590097a61085ed641810655d9ddaf8f3511
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6265
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-02 14:13:00 +00:00
Vincent Ambo
2023b8e33f fix(tvix/eval): consider let ... inherit ... in dynamic scopes
In conditions where no dynamic identifiers exist in a scope,
inheriting is usually a no-op - *unless* the identifier is not
statically known and the scope has a non-empty `with`-stack.

Change-Id: Iff4138d9cd4c56e844bc574203708dacc11c3f73
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6264
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-02 14:13:00 +00:00
Vincent Ambo
c3b13416b0 refactor(tvix/eval): add NixAttrs::contains function
This avoids copying around the value more than needed.

Change-Id: I35949d16dad7fb8f76e0f641eaccf48322144777
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6263
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 13:36:48 +00:00
Vincent Ambo
86b21f9c33 feat(tvix/eval): implement builtins.catAttrs
Change-Id: Idf92ac82438fbfcf7b2f6e058830e4744637d8c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6262
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
aeea772b1c feat(tvix/eval): implement builtins.typeOf
Change-Id: Ibc5039b444fadf6f9e5cd9132fcd825a871cee06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6261
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
5b98ece96b feat(tvix/eval): implement type-checking builtins
Change-Id: I70d7d837beaaed7e10cdc7577d96130f9e1b6d39
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6260
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
bd0fc69f07 feat(tvix/eval): implement 'throw' and 'abort' builtins
These do essentially the same, but return different error variants as
upstream Nix considers `throw` to be (sometimes) catchable.

Change-Id: I1a9ea84567d46fb37287dbf3f3f67052f9382cca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6259
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
3d0280ec03 refactor(tvix/eval): implement clearer mechanism for globals
The set of things that can leak out of `builtins` into the global
scope is statically known (it is what Nix 2.3 leaks there,
essentially).

This is a mild change over the previous mechanism, where instead at
the point where the `builtins` set is constructed we "lift" the
globals out of there (if they exist).

This way users will still eventually be able to add additional
builtins, HOWEVER they will not be able to leak them into the global
scope.

Note that upstream Nix technically leaks _all_ builtins into the
global scope using the `__*` prefix, but we are trying to avoid this
in Tvix if it is not required in nixpkgs.

Change-Id: Ie9dec2ce33740134f3b2464eba3749f421dd5953
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6258
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
07fcaf034b feat(tvix/eval): emit warnings when globals are being shadowed
Change-Id: I7dae6978c2a4548382d7fa059b20ccdf35d2cf7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6257
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
b41960c55e feat(tvix/eval): add builtins.isNull
Change-Id: Iae251d41b4ac6b77df56078a954ec3e33b7f9ccf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6256
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
b8ac62d5fc test(tvix/eval): add a simple test for builtins resolution
Change-Id: I91f54778b8a17f3448664c21308de656b4b04b3e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6255
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
e1147b57c7 feat(tvix/eval): introduce mechanism for defining builtins
Adds a new builtins module in which builtins can be constructed. The
functions in this module should return a correctly structured value to
be passed to the compiler's `globals`.

This is wired up all the way to the compiler with an example
`toString` builtin, available as a global. Note that this does not yet
actually behave like the real toString, which has some differences
from `Display`.

Change-Id: Ibb5f6fbe6207782fdf2434435567fc1bd80039a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6254
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
ca90c0f45a refactor(tvix/eval): handle scope poisoning & globals dynamically
Previously, the tokens that could poison a scope (`true`, `false`,
`null`) had individual fields in the scope to track whether or not
they were poisoned.

This commit sets up new machinery that instead tracks scope poisoning
dynamically using a HashMap, and which makes it possible to introduce
additional tokens to the top-level ("global") scope that are directly
resolved by the compiler by passing a map of runtime values to be
used.

With this solution, the compiler now contains all machinery required
for wiring up builtins resolution.

The set of builtins to be exposed at runtime must, however, be
constructed *outside* of the compiler and passed in. Everything is
prepared for this, but it is not yet wired up (so the only existing
builtins are the ones we already had before).

Note that this technically opens up an optimisation potential when
compiling selection operations, where the attribute set being selected
from is `builtins`. The compiler could directly resolve the builtins
and place the right values on the stack.

Change-Id: Ia7dad3c2a98703e7ea0c6ace1a722d57cc70a65c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6253
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
c48aa058a7 docs(tvix/eval): add an overview of all builtins in Nix
Change-Id: Ie187f3317046c6c9e59852d4a128f25ceed99309
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6252
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
e0f1356ae3 feat(tvix/eval): add initial representation of builtins
Builtins are represented as a Rust function pointer that accepts a
vector of arguments, which represents variable arity builtins.

Change-Id: Ibab7e662a646caf1172695d876d2f55e187c03dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6251
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-02 12:59:23 +00:00
Vincent Ambo
64746388e2 feat(tvix/eval): compile function applications
Change-Id: I1b9230601895a1f09ef1a8037201147020b85f36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6250
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 12:31:04 +00:00
Vincent Ambo
1239a85e23 feat(tvix/eval): implement opcode for function calls in VM
Nix functions always have a single argument and we do not yet make
efforts to optimise this in Tvix for known multi-argument functions
being directly applied.

For this reason, the call instruction is fairly simple and just calls
out to construct a new call frame.

Note that the logic for terminating the run loop has moved to the top
of the dispatch; this is because the loop run needs to be skipped if
the call frame for the current lambda has just been dropped.

Change-Id: I259bc07e19c1e55cd0a65207fa8105b23052b967
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6249
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 12:31:04 +00:00
Vincent Ambo
fc892b7a9d refactor(tvix/eval): add VM::call helper to set up call frames
Change-Id: Ia7ff572af90ae379b23bbd0f5215cd13a4dc0ab5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6248
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-02 12:31:04 +00:00
Vincent Ambo
269788086e feat(tvix/eval): compile lambda definitions
Compiles lambda definitions of the simple form (i.e. without formals
arguments) and emits them as constants like any other value.

This does not yet implement actually invoking these functions in the VM.

Change-Id: Ie1e0a13220b68c1728be229b875f0992e685c5ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6247
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
fc865eb157 refactor(tvix/eval): introduce LambdaCtx structure to compiler
This structure carries context about the lambda currently being
compiled (which may well be the top-level lambda of an input AST).

Using the indirection helpers in the compiler, things like the scope,
code and constants of the function being compiled are now taken from
the current lambda context instead.

Change-Id: If5f864d826c2e72855cee4b728ea1830e9b5ac06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6246
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
9d1451773b refactor(tvix/eval): add compiler accessor for current scope
Change-Id: I7488087d95c1b3fb7f70fc29af0d5b0d0a25a428
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6245
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
a060c8f467 refactor(tvix/eval): use call frame for top-level lambda
This wires up most of the machinery for executing different call
frames inside of the VM and stuffs the top-level lambda which the
compiler outputs in there, as well.

Change-Id: Ib6201b3e3be1af96a4d195f6eb147f452860ffc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6242
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
0b37b3f2a9 feat(tvix/eval): add call frame struct to VM
This is going to carry the data for a function invocation inside of
the VM.

Change-Id: I86664563a7e35697a64294acd37ffde037fbd32d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6241
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
6f31c895ff refactor(tvix/eval): return a lambda from the compiler
Changes the internal compiler plumbing to not just return a chunk of
code, but the same chunk wrapped inside of a lambda value.

This is one more step towards compiling runtime lambdas.

Change-Id: If0035f8e65a2970c5ae123fc068a2396e1d8fd72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6240
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
4715f9a3a0 refactor(tvix/eval): add accessor indirection helpers to compiler
With these indirections in place it becomes easier to change internals
of the compiler when introducing functions, which need the compiler to
be able to target different code chunks.

Change-Id: I4eb11572a93c140b1d059ba0a5af905756745d65
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6239
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
d57366a6c2 feat(tvix/eval): introduce initial Lambda type
Change-Id: Ifa9766f5ffeff99e926936bafd697e885e733b78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6238
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:41:12 +00:00
Vincent Ambo
3ed40b4eea feat(tvix/eval): emit warnings for unused local bindings
Change-Id: I6e876a8f4d062297abae812b14ed8ec17a502f2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6237
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:40:50 +00:00
Vincent Ambo
f7305eed47 refactor(tvix/eval): collect vector of errors in compiler
Instead of exiting the compiler at the first sight of an error,
skip any erroneous nodes and continue compiling, collecting more
errors along the way.

This paves the way for nicer error reporting in which multiple errors
can be reported at once, avoiding situations in which users are
hunting a fault error-by-error and possibly getting distracted by
less useful output.

Change-Id: I80c9a87272e33a31297167ae2eb2706a46adf15a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6236
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:40:50 +00:00
Vincent Ambo
2662376941 feat(tvix/eval): carry optional SyntaxNode in error type
This starts paving the way for nicer, source-code based error
reporting.

Right now the code paths in the VM do not emit annotated errors, as we
do not yet preserve that structure from the compiler. However, error
emitting code paths in the compiler have been amended to include known
nodes.

Change-Id: I1b74410ffd891c40cd913361bd73c4336ec8aa5b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6235
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:40:50 +00:00
Vincent Ambo
51be6542c9 refactor(tvix/eval): add helper for emitting compiler warnings
Change-Id: I2d98dbb7274d07985f64e7cc8944e316bf42e1bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6234
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
3bf487b98b chore(tvix/eval): bump rnix-parser to latest master
In this commit, the string interpolation parsing is identical to
nixpkgs which makes some of the upstream Nix tests for
interpolation-related weirdness pass.

Change-Id: I3a295cfdc404c32228a54846e6efd3c0dcee5842
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6233
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
5522ddfbf5 fix(tvix/eval): fix several string escapings
These were missing an additional level of escaping, silly oversight
caught by an upstream test.

Change-Id: I0312084475e4b88c83945614e9aa5b34c6bc3ec2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6232
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
813fb98470 refactor(tvix/eval): Upgrade to latest rnix-parser
Since the latest published version of rnix-parser on crates.io, the
crate has undergone major changes which are only available in the git
repository at the moment. This commit updates the compiler to this
newer version of rnix.

Most notably, the entire AST provided by rnix is now wrapped in the
AST type system. As a result of this traversal is much nicer in many
places, especially for things like nested attribute selection.

There are a handful of smaller features missing for full feature
parity with the previous version, especially handling of path
literals, but PRs for these already exist in rnix-parser.

Change-Id: Icde6d393067976549492b7d89c4cc49e5e575fc7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6231
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
a00e4730a5 feat(tvix/eval): implement assert operator
This implements `assert`, which evaluates an expression and aborts
evaluation if the value is not `true`.

At this point we should introduce eval-failed-* tests; probably
asserting against some representation of the error enum?

Change-Id: If54c8f616d89b829c1860a4835dde60a2cd70d7a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6230
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
16e9703f38 refactor(tvix/eval): use pretty_assertions for tests
This makes for much more readable output especially when long strings
are involved.

Change-Id: I43dd73a0480535d7181a760788c42883a9b083f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6229
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-01 12:50:17 +00:00
Vincent Ambo
f153a163a6 refactor(tvix/eval): improve naming for locals manipulator methods
`push_local`/`push_phantom` were worse names because they sound like
the value itself is being pushed, where in actuality it is just being
declared to the compiler.

Change-Id: Ibfda5c4c8e47d5d3262bfe005b0f1f84908a117e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6228
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-01 12:50:17 +00:00
Vincent Ambo
0f739cd944 feat(tvix/eval): implement scope poisoning for true/false/null
These tokens are optionally parsed as identifiers by Nix, which means
that within any scopes that resolve them the compiler needs to track
whether they have been overridden to know whether to emit the literal
instructions or resolve a variable.

This is implemented by a new concept of "scope poisoning", where the
compiler's scope structure tracks whether or not any builtin
identifiers have been overridden.

Change-Id: I3ab711146e229f843f6e1f0343385382ee0aecb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6227
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
0b51d63081 refactor(tvix/eval): simplify let ... in ... before recursion
While full recursion through thunking is not available, there are
actually incorrect behaviours introduced by declaring before
binding (example in the newly introduced test).

This commit simplifies the implementation to avoid this issue, and
also because I intend to explore a bit more how far we can get in non
left-to-right bindings *without* introducing thunks immediately.

Change-Id: I21fd3007ac3946570639772d7d624d70bd209958
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6226
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
75ba7c0120 test(tvix/eval): add basic tests for with expressions
Change-Id: I94664090e7a2b060dfbe21c1eeb859fb31e417b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6225
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
0257f89917 chore(tvix/eval): return parse errors out of eval::interpret
Change-Id: I14f25b9c85260c68be38abf07ed80121ead60c7b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6224
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
89f566ef57 feat(tvix/eval): emit instructions for dynamic var resolution
If an unknown variable is encountered and the with stack is not empty,
emit instructions for resolving the variable at runtime.

Change-Id: I752f4bd0025335744e4747364abd1bd34130374e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6223
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
59f50dc81a feat(tvix/eval): Implement OpResolveWith instruction
Change-Id: I4d2a69f28a6b6199b3ff48ef81135e7da9fe1c3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6222
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
911fb96eca feat(tvix/eval): add Value::as_attrs method
Change-Id: I2f39122ac85b67837335aab308d845907160e132
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6221
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
83e5f3b0d3 feat(tvix/eval): emit instructions to close with at scope end
Change-Id: I340b7a0964a4d4c5be58f46d00d2376ec5682517
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6220
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
19878c5fef feat(tvix/eval): implement OpPopWith
Change-Id: I752d9428a73f893741746e9d5b79e8d8d570bb81
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6219
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
5a497cdf1c feat(tvix/eval): implement with_stack in VM
Change-Id: I805c24c9a751ded15725ba9be651aa0869e013e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6218
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
7cfdedfdfb feat(tvix/eval): compile with expression
Adds an additional structure to the compiler's scope to track the
runtime "with stack", i.e. the stack of values through which
identifiers should be dynamically resolved within a with-scope.

When encountering a `with` expression, the value from which the
bindings should be resolved is pushed onto the stack and tracked by
the compiler in the "with stack", as well as with a "phantom value"
which indicates that the stack contains an additional slot which is
not available to users via identifiers.

Runtime handling of this is not yet implemented.

Change-Id: I5e96fb55b6378e8e2a59c20c8518caa6df83da1c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6217
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-31 22:26:11 +00:00
Vincent Ambo
ec7db0235f refactor(tvix/eval): rename Locals to Scope
This name is much more sensible actually; its more than just a
collection of locals as it tracks additional scope information in the
case of Nix.

Change-Id: Ia2739bbd39aab222b1c4355e9248828973b0db43
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6216
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
f173161f4c feat(tvix/eval): implement inherit in let expressions
Note that at this point recursive bindings do not yet work in either
attrsets or let, so inheriting from the same scope is generally not
possible yet.

Change-Id: I6ca820d04b8ded5c22fb7ea18e2ec203bcaa8e9c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6215
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
8c1c9aee3c feat(tvix/eval): implement inherit in attribute set literals
Straightforward implementation, evaluating the elements of an inherit
and preparing the stack so that `OpAttrs` sees all relevant values
when constructing the attribute set itself.

The emitted instructions for inheriting a lot of values from the same
attribute set are inefficient, but it's too early to say whether this
actually matters.

Change-Id: Icb55a20936d4ef77173f34433811c5fa5d2c9ecc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6214
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
7db4f8d774 fix(tvix/eval): gently attempt to create state dir
If the directory in which REPL history is stored does not exist,
gently try to create it, but do not raise an error if it doesn't work.

We may want to warn about it, but in general this sort of
non-essential feature should not cause a hard failure.

Change-Id: If4fe8db0c7893c39627efe72c9cd9ebf7ed63f04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6213
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
e07556493f feat(tvix/eval): Implement inherit from outer scope in attrs
Change-Id: I97fa45059b7f2cbe96eb60bd1821e3e25832df66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6212
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
3a67f91228 fix(tvix/eval): ? operator should use OpAttrsOrNotFound
Using `OpAttrSelect`, the ? operator will fail when encountering a
nested value that is not an attribute set.

This however breaks valid code, such as:

  { bs = 42; } ? bs.a.b

The fix is simply to use the same operator used in the `or` statement,
which leaves a sentinal on the stack if a field is not found or the
value is not an attribute set.

Change-Id: Ib28fc8a96e6d592b4cdbc3e65ba129ad8faecd66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6211
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
cfe37d36f7 fix(tvix/eval): or should handle non-attrset values, too
If a nested attrpath encounters a non-set value, the sentinel value
denoting a lack of next values should be emitted. This mirrors the
behaviour of Nix.

Change-Id: Ia80443d5a11243cc6d98dcab1249a3f5fdf77e27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6210
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
76846fe220 fix(tvix/eval): allow use of ? operator on non-set types
Nix allows this, but always returns false. Tvix needs to do the same.

Change-Id: Ic9eec90834a0d0969eea5316d5c25032d3691d94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6209
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
43658a5b90 fix(tvix/eval): emit correct count in OpAttrPath
Not sure how exactly this snuck in, but it caused some subtle
breakages in deeply nested attribute sets.

Change-Id: I8049ce912405d3750031f79cc8d86ff1c3c02c2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6208
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
8d45fbadea docs(tvix/eval): Use correct syntax for module doc comments
Change-Id: I35741856f34b86a538f226a8eaf8806edede60ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6207
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
11ea7b82d8 chore(tvix/eval): minor readability improvement in attrs
Change-Id: If9d9eaf60934e96ec4b41c57818afe0c2a99c862
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6206
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
de26894814 refactor(tvix/eval): remove Error::InvalidKeyType
We're confident that we're handling all branches that can reasonably
occur from valid AST, any other cases should be considered a critical
evaluator bug and panic rather than surfacing something that looks
like user error.

Change-Id: If96966eb32b8ff12fcaeb9ea3b0c8fc51b6abd11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6205
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
ab9407bded fix(tvix/eval): address various clippy lints
Change-Id: I3ea0f51475e80948adfeb5d1620c1f2665cc39bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6201
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-30 16:53:40 +00:00
Vincent Ambo
c4f73eecdc feat(tvix/eval): implement attribute set equality
Change-Id: Ia25f02610f2575e5e7fca81643e05b40f4a07820
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6200
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-30 16:53:40 +00:00
Vincent Ambo
2ea71aa4c3 fix(tvix/eval): null in dynamic attribute keys skips the element
This is actually *tested* behaviour in C++ Nix, so we need to
implement it here, too.

Change-Id: Ic4a4659a2f04cdd928cbe78a85dae90401515371
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6199
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-30 16:53:40 +00:00
Vincent Ambo
e041851581 feat(tvix/eval): implement chunk disassembler output
This makes for a much nicer view of an execution if `--feature
disassembler` is set, for example:

  tvix-repl> let value = [ 1 2 { a = 1; } ]; in value ++ [ 1 ]
  === compiled bytecode (11 operations) ===
  02   OpConstant(1)
  02   OpConstant(2)
  02   OpConstant("a")
  02   OpConstant(1)
  02   OpAttrs(1)
  02   OpList(3)
  02   OpGetLocal(0)
  02   OpConstant(1)
  02   OpList(1)
  02   OpConcat
  02   OpCloseScope(1)
  === runtime trace ===
  0001 OpConstant(ConstantIdx(0))  [ 1 ]
  0002 OpConstant(ConstantIdx(1))  [ 1 2 ]
  0003 OpConstant(ConstantIdx(2))  [ 1 2 "a" ]
  0004 OpConstant(ConstantIdx(3))  [ 1 2 "a" 1 ]
  0005 OpAttrs(1)                  [ 1 2 { a = 1; } ]
  0006 OpList(3)                   [ [ 1 2 { a = 1; } ] ]
  0007 OpGetLocal(0)               [ [ 1 2 { a = 1; } ] [ 1 2 { a = 1; } ] ]
  0008 OpConstant(ConstantIdx(4))  [ [ 1 2 { a = 1; } ] [ 1 2 { a = 1; } ] 1 ]
  0009 OpList(1)                   [ [ 1 2 { a = 1; } ] [ 1 2 { a = 1; } ] [ 1 ] ]
  0010 OpConcat                    [ [ 1 2 { a = 1; } ] [ 1 2 { a = 1; } 1 ] ]
  0011 OpCloseScope(1)             [ [ 1 2 { a = 1; } 1 ] ]
  => [ 1 2 { a = 1; } 1 ] :: list

Change-Id: If79c7fd1f0f18255ddb3763c1ba585fda8041b1b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6195
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-30 16:53:40 +00:00
Vincent Ambo
57d0dbb1c6 feat(tvix/eval): implement optional runtime tracing
This adds a `disassembler` feature to the crate configuration that
traces the operations executed and the state of the stack at runtime.

This can be enabled by compiling with `--feature disassembler`.

This will also gain a more sensible layout of code slices eventually.

Change-Id: I34c15e1cd346ecc4362b5afba6bf82dd49359d20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6193
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-30 16:53:40 +00:00
Florian Klink
6c46ecd1c9 chore(tvix/nix_cli): add default.nix
This exposes tvix.nix_cli as a proper attribute to readTree, so it's
actually built by CI.

Change-Id: I3cef085bd872b61c5944270c8926727bf1fa705d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6083
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-08-30 11:03:48 +00:00
Vincent Ambo
1cb1931406 feat(tvix/eval): emit warnings on useless inherit
Change-Id: Ifb9993cf8b85393eb43e1b204c7ab2f889b7113b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6194
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-29 20:13:26 +00:00
Vincent Ambo
ab12b0abff fix(tvix/eval): skip inherit with no explicit parent in let
Using `inherit` in a let-binding can not possibly have an effect, as
the given identifier is already bound exactly the same way in the
current scope.

This introduces a subtle bug that is fixed later on, as there
actually *is* a (single) condition where these inherits are
meaningful.

Change-Id: I8b24f0edcfe80db8153bb7e86cf478d36957d6f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6192
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-29 20:13:26 +00:00
Vincent Ambo
36748fe4f6 test(tvix/eval): add basic tests for let expressions
Change-Id: I788504f0c1848a1422c99006bf659738df614716
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6191
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-29 20:13:26 +00:00
Vincent Ambo
342b233a0a feat(tvix/eval): add local identifier access
This makes basic `let ... in ...` statements work correctly. It does
not yet account for the call frames pushed into the VM during function
application.

Change-Id: I67155171daf1a43011b96716dd9d1ab04b27db33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6190
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-28 17:50:34 +00:00
Vincent Ambo
2d401a32e5 feat(tvix/eval): detect dynamic identifier names in let
Nix does not allow dynamic identifiers in let expressions (only in
attribute sets), but there are several different kinds of things it
considers static identifiers.

The functions introduced here put the path components of a let
expression into normalised (string) form, and surface an error about
dynamic keys if one is encountered.

Change-Id: Ia3ebd95c6f3ed3cd33b94e156930d2e9c39b6cbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6189
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-28 17:50:34 +00:00
Vincent Ambo
691a596aac feat(tvix/eval): compile simple let ... in ... expressions
These expressions now leave the binding values on the stack, and clean
up the scope after the body of the expression.

While variable access is not yet implemented (as the identifier node
remains unhandled), this already gives us the correct stack behaviour.

Change-Id: I138c20ace9c64502c94b2c0f99a6077cd912c00d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6188
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-28 17:50:34 +00:00
Vincent Ambo
bbad338017 feat(tvix/eval): prepare structures for simulating locals stack
These are going to be used during compilation of `let`-expressions to
determine stack offsets for local variables.

Change-Id: Ibb79f3f1ae86650303f88eacf623ae456458de87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6187
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-28 17:50:34 +00:00
Vincent Ambo
de21d201ba fix(tvix/eval): use path_clean instead of fs::canonicalize for paths
Canonicalisation performs much more functionality than what C++ Nix
implements for paths, and causes some undesirable behaviour (e.g.
handling non-existant files becomes difficult, but should be possible
in literals).

Instead, the path_clean crate provides a pure normalisation method.

There is an intention to add this to Rust itself:
https://github.com/rust-lang/rfcs/issues/2208

Change-Id: I775d238136db0a52cf6b12a68985833c8fb32882
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6186
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-28 11:02:15 +00:00
Vincent Ambo
6fe5e2d752 feat(tvix/eval): resolve relative path literals
Resolves relative paths (e.g. `./foo`) either relative to the location
of the Nix file, or relative to the working directory if none is
supplied.

Change-Id: I70ec574657b221b458015117a004b6e4a9c25a30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6185
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-28 11:02:15 +00:00
Vincent Ambo
1f8aad0ab4 refactor(tvix/eval): implement error variant for path resolution
There are multiple things that can theoretically fail while resolving
a path, as some of it includes I/O. A new error variant has been added
for this and appropriate errors have been introduced.

Change-Id: Ie222245425207dabbf203166eb5ed1eec0114483
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6184
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-28 11:02:15 +00:00
Vincent Ambo
e68b6d33b0 feat(tvix/eval): support absolute and home-anchored paths
These two paths are the easiest to handle, as they are simply built up
from the components supplied in the text node and then normalised.

Note that the normalisation of fs::canonicalize includes symlink
resolution, which Nix does not actually do. We will need to fix that
at some point.

Change-Id: I54158f0684f197dd2a2583f7d0982d54c7619993
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6183
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-28 11:02:15 +00:00
Vincent Ambo
bf10c35201 chore(tvix/eval): add variant_size_differences warning to Value
Change-Id: I2f4719a81512a9a970edf22390afed490125bda7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6182
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-27 09:27:13 +00:00
Vincent Ambo
e33eaa2cb5 feat(tvix/eval): add Path representation to Value enum
Change-Id: I4827384201912131ea8fc2362188dcd862b94852
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6181
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-27 09:27:13 +00:00
Vincent Ambo
1d86c4537e refactor(tvix/eval): use write! macro instead of f.write_fmt
grfn pointed out in cl/6082 that this is actually the desugaring of
the write! macro, so it doesn't make sense to  write it out.

Change-Id: If7c055b042ad22b034722aec1eaadba92736d684
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6180
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-27 09:27:13 +00:00
Vincent Ambo
0a09356f82 refactor(tvix/eval): rename Chunk::add_* functions to ::push_*
grfn pointed out in cl/6069 that naming them like this makes it clear
that things are being added to the end of the state.

Change-Id: I6a23215c4fef713869a3c85b0dde1ebbda7637e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6179
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-27 09:27:13 +00:00
Vincent Ambo
4c9d3fa2a6 chore(tvix/eval): explicitly set #[repr(transparent)] on wrappers
For representation wrappers that are used to control the visibility of
type internals, this ensures that the wrapper does not increase the
size of the type.

In practice, the optimiser likely does this anyways but it is good to
guarantee it.

Change-Id: Ic6df7d668fe6006dfbd5b6cfcfc2088afa95b810
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6178
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-27 09:27:13 +00:00
sterni
4523703a98 test(tvix/eval): make sure all language tests also pass with C++ Nix
Change-Id: I8428a82db9bf7af5c0273178acbf3e4d7355f5d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6153
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-08-26 20:46:03 +00:00
Vincent Ambo
d952b29fca chore(tvix/eval): add variant_size_differences warning for OpCode
This makes it possible to quickly detect code errors that might blow
up the size of the OpCode type.

Change-Id: I7662dd0aa30c4762c0f9e4fa346418c9ca8b9994
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6177
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 20:32:12 +00:00
Vincent Ambo
598c197422 feat(tvix/eval): add warning for deprecated URL literals
Change-Id: I8a9cfcb5d99680344ee0e3461a4a52526b846c92
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6175
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 17:19:51 +00:00
Vincent Ambo
7e77972d71 feat(tvix/eval): add mechanism for emitting warnings from compiler
These can be used predominantly to emit warnings about things that the
compiler can infer, such as deprecated language features.

Change-Id: I3649c625459d7f3f95cdf42d5c651d23d66569ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6174
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-26 17:19:51 +00:00
Vincent Ambo
5512108ca7 fix(tvix/compiler): handle literal URL values
The parser creates this node type from literal URL values. Technically
these are deprecated and have been removed from nixpkgs.

Change-Id: I4d05034dd9b4d8348e4ed8a2bbb37c1b6ccef8bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6173
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-26 17:19:51 +00:00
Vincent Ambo
7c803a7e72 feat(tvix/eval): use rustyline crate for REPL
This is a substantially nicer experience, immediately granting us
history, proper exiting and so on.

Change-Id: Iba4cb1713b9ac53d0799722bdbe2cd0e94a2f527
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6171
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-26 17:19:51 +00:00
Vincent Ambo
d14db8dcaa chore(tvix/cargo): add dirs dependency
This crate provides sensible default locations for directories on
different platforms.

Change-Id: I0b61cc7f626dc6c8df903ba0f873be24e07d69b5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6170
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-26 17:19:51 +00:00
Vincent Ambo
49f416c176 chore(tvix/cargo): add rustyline dependency
There's a few options for this, but this one seems fine in terms of
features and doesn't have an insane amount of dependencies.

Change-Id: Ief99e66bfee0ba0ba1cfdd09568b002121b7325b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6169
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-26 16:36:03 +00:00
Vincent Ambo
6c5d0344d9 style(tvix/eval): minor rephrasing in a comment
From cl/6070

Change-Id: I8ce8c1b40032d85cd2c01b04e225ae4a2842c7b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6172
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 16:32:55 +00:00
Vincent Ambo
15c2986597 test(tvix/eval): add tests for the attrset or operator
Change-Id: I81f3ab94d8b987f5a4c87a27b2bcbb49cccee1ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6168
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 16:05:09 +00:00
Vincent Ambo
941d718a8a feat(tvix/compiler): implement or operator for attribute sets
This operator allows for accessing attribute sets (including nested
access) while also providing a default value.

This is one of the more complex operations to compile, as it needs to
keep track of a fairly large number of jumps that all need to be
patched correctly.

To make this easier to understand there's a small diagram included in
the comments.

Change-Id: Ia53bb20d8f779859bfd1692fa3f6d72af74c3a1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6167
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 15:06:52 +00:00
Vincent Ambo
b8cec6d61e feat(tvix/vm): add opcodes for new Value::NotFound sentinel
This sentinel value is going to be used for certain nested accesses
into attribute sets.

There is a new instruction similar to `OpAttrsSelect` which leaves the
sentinel on the stack if a key is not found, instead of raising an
error.

Additionally, a new jump instruction makes its jump operation
conditional on finding such a sentinel value.

Change-Id: I2642f0a0bcc85bbe0ead68ea09a7dd794dbedeac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6166
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 15:06:52 +00:00
Vincent Ambo
999b9c7a13 refactor(tvix/value): replace static representation with SmolStr
The only uses of the static variant were for `"name"` and `"value"`,
which are both small enough to fit into a SmolStr. The size of
NixString accomodates `String` anyways, so we may as well inline them.

Additionally smol_str is already in the dependency graph because rnix
uses it, and using it for representations of identifiers is sensible.

Change-Id: I9969312256d1657d69128e54c47dc7294a18ce58
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6165
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 15:06:52 +00:00
Vincent Ambo
f331874aeb chore(tvix/eval): implement improved Display for internal values
Having these visible more explicitly is useful while debugging.

Change-Id: I86b497883063d32792b635eb4514b7aeae484af4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6164
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 15:06:52 +00:00
Vincent Ambo
dd8f5f1dc8 feat(tvix/eval): implement nested '?' operator
Implements the nested presence check operator for attribuet sets by
traversing the chain of lookups through instructions that push/pop
sequentially deeper attribute sets onto the stack.

Note that this commit introduces a bug in case of nested attributes
not being found, which is fixed in a later commit.

Change-Id: Ic8b4c8648736f6cb048e3aa52592e4d075bf0544
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6163
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 15:06:52 +00:00
Vincent Ambo
cf3e3b784b feat(tvix/eval): implement ? operator (single-level only)
This makes it possible to check things like `{} ? a` with a single
level of nesting.

Change-Id: I567c36fcfd2f9e2f60071acd3ebfe56dea59b26f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6161
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 09:02:38 +00:00
Vincent Ambo
e8c4e26b41 feat(tvix/compiler): handle dynamic nodes for attribute access
Dynamic nodes (essentially unquoted interpolation) simply need to be
unwrapped (and, hopefully, evaluate to a string).

Example: `let key = "a"; in { a = 1; }.${key}`
Change-Id: Idafd376e19d0e237151b7b6c26d97713beae5041
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6160
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 09:02:38 +00:00
Vincent Ambo
5eb523e882 fix(tvix/compiler): support identifier literals in select expression
With this change, attribute set access is working as intended.

Change-Id: Ic5dbbd68aa59156106069289e7375a696909f78b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6159
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 09:02:38 +00:00
Vincent Ambo
20f5ccefeb feat(tvix/eval): implement attribute set access operator
Fairly straightforward, handling the optimised representations
manually and otherwise delegating to BTreeMap.

Note that parsing of raw identifiers is not yet implemented.

Encountering an identifier node usually means that there is locals
access going on, so we need a special case for compiling a node in
such a way that an identifier's literal value ends up on the stack.

Change-Id: I13fbab7ac657b17ef3f4c5859fe737c321890c8a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6158
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 09:02:38 +00:00
Vincent Ambo
671915837a fix(tvix/eval): add operation to assert boolean type
This operation is required because both sides of the logical operators
are strictly evaluated by Nix, even if the resulting value is not used
further.

For example, in our implementation of `&&`, if the left-hand side is
`true`, then the result of the expression is simply the right-hand
side value. This value must be asserted to be a boolean for the
semantics of the language to work correctly.

Change-Id: I34f5364f2a444753fa1d8b0a1a2b2d9cdf7c6700
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6157
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 09:02:38 +00:00
Griffin Smith
0c4c307926 feat(tvix/eval): Add initial benchmarking infrastructure
Bootstrap some (initially very simple, mostly proof-of-concept)
benchmarking infrastructure using Criterion, using the newly-exposed lib
from tvix-eval.

Change-Id: I4bb93c142ba8d018d7e67e58ac8907a0429398a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6156
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
2022-08-25 16:03:55 +00:00
Griffin Smith
f648cec78e feat(tvix/eval): Expose interpret + related types from lib
Add a new `lib.rs` to tvix/eval, which `pub use`s the `interpret`
function, and all types mentioned in its return type, and then uses
*this* instead of direct `mod` statements in the `main.rs` to implement
the entrypoints to the interpreter. This is in preparation for calling
these functions from integrated benchmarking infrastructure using
Criterion, though other things (like integration tests) might want to do
that as well.

Change-Id: I7b585134a96b1c56a2ac64d2036b0e51d321bd27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6155
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
2022-08-25 16:01:46 +00:00
Vincent Ambo
63a0cc83d1 feat(tvix/compiler): implement -> (implication) operator
Similar to `||`, but inverting the left-hand side.

In other words, `a -> b` is essentially rewritten as `!a || b`.

Change-Id: I8a62da65ff070b389e46048d047a54279060a97b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6152
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 16:01:14 +00:00
Vincent Ambo
c7c7ab9bd4 feat(tvix/compiler): implement || operator
Same dance as `&&` but logically inverted.

Change-Id: I213e200e3836527e9abe510f354ee7cd1f70d041
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6151
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 16:00:55 +00:00
Vincent Ambo
4b920912b8 feat(tvix/compiler): implement && operator
This logical operator is implemented in terms of jumping operations
and thus requires slightly different treatment than other binary
operators.

Change-Id: Ib3d768b70dd7e16014c9b47d770aa74eec60ae92
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6150
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 16:00:55 +00:00
Vincent Ambo
f3f8262637 docs(tvix/compiler): add a note on use of unwrap/expect for rnix
Change-Id: I62ca28285685b69d1883afcf18c6068fc2defb5d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6149
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 16:00:55 +00:00
Vincent Ambo
d9d94eb27f feat(tvix/eval): implement if/else expressions
These expressions use simple jumps to skip the correct expression
conditionally in the bytecode by advancing the instruction pointer.

Note that these expressions are already covered by a test behind the
`nix_tests` feature flag, but adding more is probably sensible.

Change-Id: Ibe0eba95d216321c883d3b6b5816e2ab6fe7eef1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6148
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 16:00:55 +00:00
Vincent Ambo
2422f2f224 refactor(tvix/value): hide internal string representation
Wraps the string representation in an additional newtype struct with a
private field in order to hide the representation from other modules.

This is done in order to avoid accidental leakage of the internals
outside of value::string.

In fact, this caught a mistake in the compiler module which was
directly constructing an internal variant.

Change-Id: If4b627d3cff7ab9cd50ca1a3ac73245d4dcf7aef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6147
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 12:07:30 +00:00
Vincent Ambo
a0cbc78a83 refactor(tvix/value): ensure internal attrs representation is hidden
Wraps the attrs representation in an additional newtype struct with a
private field in order to hide the representation from other modules.

This is done in order to avoid accidental leakage of the internals
outside of value::attrs.

Change-Id: I68d1d02514aa0443df4c39801001a3f1f6cc5d5c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6146
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 12:07:30 +00:00
Vincent Ambo
9407af5684 refactor(tvix/eval): encapsulate list construction in value::list
Ensuring that the implementation is not leaking out of the module lets
us keep things open for optimisations (e.g. empty list or pairs
through tuples).

Change-Id: I18fd9b7740f28c55736471e16c6b4095a05dd6d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6145
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 12:07:30 +00:00
Vincent Ambo
75a22321ce feat(tvix/eval): implement list concatenation
Change-Id: Icdf715d116371a9f139bdf95266410bf967bef25
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6144
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 12:07:30 +00:00
Vincent Ambo
4eafaae9e6 feat(tvix/eval): implement binary comparison operators
This is accomplished by simply delegating to the Rust implementations
of (Partial)Ord and (Partial)Eq, which are implemented for Value and
underlying wrapper types to behave like they do in Nix.

To ease the implementation overhead, a new comparison operator macro
has been added to the VM module.

Incomparable types will raise a new error variant when a comparison is
attempted, containing both supplied types. This mimics the information
carried in the error thrown by C++ Nix.

Change-Id: Ia19634d69119d40722f3ca672387bc3a80096998
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6143
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 12:07:30 +00:00
Vincent Ambo
5a15ee3f5d refactor(tvix/vm): simplify implementation of OpEqual
This can now fully delegate to PartialEq of Value

Change-Id: Iaa9f4ec9b8830d516d72f83a93ab2df9a6e5697c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6142
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:34:00 +00:00
Vincent Ambo
7d55b776de feat(tvix): implement not-equals (!=) operator
Change-Id: Ic259d6d0cf30553e944682a0d1d2c610df7fe5c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6141
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 11:34:00 +00:00
Vincent Ambo
058e77bab2 feat(tvix/eval): implement attrset update (//) operator
The underlying implementation does a few tricks based on which pair of
attrset representations is encountered.

Particularly the effect of short-circuiting the empty cases might be
relevant in nixpkgs/NixOS, due to the use of lib.optionalAttrs.

Change-Id: I22b978b1c69af12926489a71087c6a6219c012f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6140
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 11:34:00 +00:00
Vincent Ambo
fa2d250d1a fix(tvix/value): explicitly delegate Ord to &str representation
I assumed that `Ord` is a marker trait like `Eq`, but it actually has
a member. Without this ordering was incoherent.

Change-Id: Id37cbdf333daf748d29b85243046c7e061b1ce29
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6139
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:11:27 +00:00
Vincent Ambo
58db7e3a65 fix(tvix/value): ensure only string values of NixString are hashed
Change-Id: I1b97cf5a9e4e235fac72a507de49a8df508bcaa1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6138
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:11:27 +00:00
Vincent Ambo
fb3d024d75 feat(tvix/eval): implement string concatenation
Change-Id: If61066e59232b2bad42b5cb5f0f2d9b9c416be8b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6137
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:11:27 +00:00
Vincent Ambo
322ce36cea refactor(tvix/vm): use a macro to handle binary arithmetic operators
Instead of constructing another runtime value representing the pair on
which to perform arithmetic, implement the same logic in the shape of
a macro.

This is designed to be compatible with operators like `+` that work
both as an arithmetic operator AND as an operator on another pair of
types.

Change-Id: I1c83649ead6117f811f1fb45482d0cadf811125e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6136
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:07:16 +00:00
Vincent Ambo
8150803e77 test(tvix/eval): add some eval-okay-* tests for trivial types
Change-Id: I85ccc07e08c67abf4fcd3752c58e1702943239ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6135
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:07:16 +00:00
Vincent Ambo
522d93745c fix(tvix/value): fix display representation of floats
Nix displays a maximum of 5 digits for floating points.

Change-Id: Ifa3c0d96fa0b24e3be8f94dfebc99e602a258355
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6133
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 10:41:39 +00:00
Vincent Ambo
ad1c11b606 chore(tvix/eval): enable test runs in naersk
Change-Id: I08c3a66a3f5c98d914d17a2ad6de98b1c02e50b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6134
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 23:18:07 +00:00
Vincent Ambo
96e9f750c5 test(tvix/eval): add identity tests for literal evaluation
Change-Id: Id3f37fbe341e15e9428ef1d579d61a514232c0e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6132
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 23:11:12 +00:00
Vincent Ambo
3671056640 fix(tvix/value): align Display representation with Nix
Displaying the optimised representation is not useful anymore.

Change-Id: Icb962ff8865ec4207c144fbcb1aae87483b0fb7c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6131
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 23:11:12 +00:00
Vincent Ambo
c75fa365cd docs(tvix/eval): add README note on running the Nix test suite
Change-Id: I792154d5fea05414f2b14367ea4087dc2a88758e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6130
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 23:11:12 +00:00
Vincent Ambo
7d5691f0e5 chore(tvix/tests): gate Nix test suite behind nix_tests feature
Once we have full coverage they should be enabled by default.

Change-Id: Iace9e1ae9a9f901a0979ad336434004b8028fe8a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6129
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 23:11:12 +00:00
Vincent Ambo
8235384d2d chore(tvix/eval): only print AST if requested
Change-Id: If32c7fd03458a27aecb8c473f2c48b12f8cb5175
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6128
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-08-24 21:25:41 +00:00
Vincent Ambo
5685f7c594 fix(tvix/value): add ident_str representation of strings
When printing strings as identifiers (in attribute sets), the string
should only be quoted and escaped if it contains escape characters.

Change-Id: If2bcfa1e93dc8f00be4d7a57ec1d82fc679103c3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6127
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-08-24 21:25:41 +00:00
Vincent Ambo
92c53fe982 feat(tvix/tests): check in Nix' language test suite
This adds scaffolding code for running the Nix language test suite.

The majority of eval-okay-* tests should eventually be runnable as-is
by Tvix, however the eval-fail-* tests might not as we intend to have
more useful error messages than upstream Nix.

Change-Id: I4f3227f0889c55e4274b804a3072850fb78dd1bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6126
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 21:25:41 +00:00
Vincent Ambo
b28da8ad56 chore(tvix/eval): add 'test-generator' as dev-dependency
This crate makes it easy to generate test cases from files on disk,
which is used for running the Nix test suite against Tvix.

Change-Id: I60ef26484d05e723982679dd42958adf52735916
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6125
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 21:23:12 +00:00
Vincent Ambo
4c9aad17ad fix(tvix/value): add escaping logic for Nix strings
Nix strings displayed to users must be escaped the same way as they
are in C++ Nix. This adds the scaffolding for escapes, but is most
likely not yet complete.

Change-Id: Icfdcb2ac98d292c567ba894a92b6529a53e0cc17
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6124
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 21:23:12 +00:00
Vincent Ambo
a2b4b4a485 fix(tvix/value): implement PartialOrd/PartialEq for strings
Instead of comparing the enum variants (which does not yield useful
behaviour), compare &str representations of the string instead.

Change-Id: I5e94b5f6c91b4561e1bc7c36d586f3d23c243764
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6112
Tested-by: BuildkiteCI
Reviewed-by: eta <tvl@eta.st>
2022-08-24 18:19:52 +00:00
Vincent Ambo
6dc9ca5723 feat(tvix/value): introduce string representation with &'static str
For cases where the strings are statically known (such as the
oft-occuring name/value), this can be a useful optimisation.

It's also much more convenient in tests.

Change-Id: Ie462b684805bd4986ea5e85ca4bff663bc2d3c3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6111
Tested-by: BuildkiteCI
Reviewed-by: eta <tvl@eta.st>
2022-08-24 18:19:52 +00:00
Vincent Ambo
c7ba2dec04 test(tvix/value): add simple attrset construction tests
These do not yet test nested attribute sets; we need to add some more
inspection primitives first.

Change-Id: Icfc99bf17c73ebefc0d882a84f0ca73ec688a54d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6110
Reviewed-by: eta <tvl@eta.st>
Tested-by: BuildkiteCI
2022-08-24 18:19:52 +00:00
Vincent Ambo
08b4d65fbd feat(tvix/value): implement nested attribute set literals
With this change, nested attribute sets can now be created from
literals.

This required some logic for dealing with cases where at a deeper
nesting point a literal attribute set was constructed from an
optimised representation.

For example, this is valid Nix code:

```nix
{
  a = {};   # creates optimised empty representation
  a.b = 1;  # wants to add a `b = 1` to it

  b = { name = "foo"; value = "bar"; }; # creates optimised K/V repr
  b.foo = 42; # wants to add an additional `foo = 42`
}
```

In these cases, the attribute set must be coerced to a map
representation first which is achieved by the new internal
NixAttr::map_mut helper.

Change-Id: Ia61d3d9d14c4e0f5e207c00f6a2f4daa3265afb2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6109
Reviewed-by: eta <tvl@eta.st>
Tested-by: BuildkiteCI
2022-08-24 18:19:52 +00:00
Vincent Ambo
293fb0ef53 refactor(tvix/value): encapsulate attrset logic within value::attrs
The internal optimisations of the set representation were previously
leaking into the VM, which is highly undesirable.

Keeping it encapsulated allows us to do additional optimisations
within value::attrs without being concerned about its use in the VM.

Change-Id: I7e7020bb0983b9d355d3db747b049b2faa60131f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6108
Reviewed-by: eta <tvl@eta.st>
Tested-by: BuildkiteCI
2022-08-24 18:19:52 +00:00
Vincent Ambo
52736a1dbc refactor(tvix/rm): introduce helper for AttrSet Entry API
There are multiple points where an insertion needs to be done into an
attribute set, but copying the key or checking for presence before
insertion should be avoided

As that is a little bit noisy, it's been factored out into a helper
function in this commit.

Change-Id: Ibcb054ebeb25a1236c06c812f47c8e74180d4fc9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6107
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-14 00:24:33 +00:00
Vincent Ambo
295d6e1d59 feat(tvix/vm): implement first nested attribute set construction
This can construct non-overlapping nested attribute sets (i.e. `{ a.b
= 1; b.c = 2; }`, but not `{ a.b = 1; a.c = 2; }`).

In order to do the latter, it's necessary to gain the ability to
manipulate the in-progress attribute set construction. There's
multiple different options for this ...

Change-Id: If1a762a720b175e8eb4216cbf96a7434d22640fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6106
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-14 00:23:32 +00:00
Vincent Ambo
8c2bc683cd refactor(tvix/value): explicitly implement PartialEq for value
There are some notions of equality (due to e.g. different backing
variants for types, or Nix particularities) that don't work correctly
when deriving PartialEq.

Change-Id: Ide83ae67d051cc0b3ca89cefb283f17d0207acce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6105
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 20:24:12 +00:00
Vincent Ambo
e15bd9aa63 fix(tvix/eval): Fail on duplicate attribute set keys
Change-Id: I57373ca76d0e25a5d08a8dfce9d5949099326fc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6104
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 20:24:12 +00:00
Vincent Ambo
175eb97505 feat(tvix/eval): construct internal attribute path representation
This is required for constructing nested attribute sets at runtime.

There'll be quite a lot of optimisation potential with this solution
eventually, if it should turn out to be a bottleneck.

This introduces a conceptual change, in that the `Value` enum is now
an enum representing "all runtime values" instead of "all Nix language
types". This makes sense in general, as this type will also contain
Chunk representations etc. which are not exposed to users.

Change-Id: Ic5f72b2a0965b146c6a451efad34c6a81ca1aad8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6103
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-13 20:24:12 +00:00
Vincent Ambo
ec1770f95a feat(tvix/vm): implement construction of optimised KV attrsets
For name/value pairs (which occur extremely often in Nix and make up a
significant chunk of the runtime cost of evaluating nixpkgs) we
substitute an optimised representation.

For now this will only be used if the name/value pair keys were
specified as literal identifiers or strings (i.e. if chunks are
encountered as keys they are not forced and a normal attribute set
backed by a map will be constructed).

Change-Id: Ic79746c323e627528bd58b1a6024ee8d0aff7858
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6102
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-13 20:24:12 +00:00
Vincent Ambo
e876c3a41c fix(tvix/value): KV struct needs to carry name as Value, too
Users may construct a pair that falls into the name/value optimisation
but where `name` is not actually a string, as from the language
perspective there is nothing special about this attribute set.

We also can not conditionally apply this by forcing the key at this
point, as this would change the language semantics.

Therefore, the name in the optimised representation is also carried as
`Value`.

Change-Id: I5be8a4c98ba19ebdfb7203a929f714a04492512e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6101
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-13 20:24:12 +00:00
Vincent Ambo
2dcbbb8c4a feat(tvix/value): implement Display properly for lists
Change-Id: I991d235cf52fbd42eb839b384f9c55ee64fa86c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6100
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 18:28:36 +00:00
Vincent Ambo
56caaf70ca fix(tvix/errors): display a useful intermediate error representation
Change-Id: If979a2aa21444320427f54e6530a55cab873856b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6099
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 18:28:36 +00:00
Vincent Ambo
a93933b487 feat(tvix): implement string interpolation
This adds a new instruction which assembles an interpolated string
from a specified number of fragments, which are already going to be
located on the stack in the right position.

This will raise a type error if any of the fragments do not evaluate
to a string.

Change-Id: I5756248fa3e9fcc3d063c14db40b332f7e20a588
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6098
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 18:28:36 +00:00
Vincent Ambo
3577841bde feat(tvix/compiler): compile non-interpolated string literals
This sets up the scaffolding for compiling interpolation, but those
instructions do not yet exist.

Change-Id: Ife41bbbf432d9661abe566c92437409dd0da44e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6097
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 18:28:36 +00:00
Vincent Ambo
72863d81fc style(tvix/eval): display Display representation of runtime values
Change-Id: Ifbe05c2da9493c9e132a2d2e94a451d6091542a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6096
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 18:28:36 +00:00
Vincent Ambo
910336c68c feat(tvix/compiler): compile list literals
Change-Id: I2c6fedc3dbb7d449d700f3972c3fbd4a7d147f6c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6095
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-08-13 15:39:26 +00:00
Vincent Ambo
5763b62172 feat(tvix/vm): implement list construction
Change-Id: Iec2b4910800ab29daae6d71b58a8acd14ccb1cc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6094
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-08-13 15:35:24 +00:00
Vincent Ambo
c67747cbe1 feat(tvix/value): add runtime representation of simple lists
There might be more logic in the future to encapsulate different
backing implementations of lists as well.

Change-Id: Ib7064fab48bf88b0c8913b0ecfa2108177c7c9fd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6093
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-08-13 15:35:24 +00:00
Vincent Ambo
6f13c16f28 docs(tvix/eval): add design documentation for attrset opcodes
Change-Id: I303b57e035543f4597c6247983d1d533e4014638
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6092
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 15:31:50 +00:00
Vincent Ambo
57a723aaa9 feat(tvix/eval): implement trivial attribute set literals
Implements attribute set literals without nesting. Technically this
already supports dynamic key fragments (evaluating to strings), though
the only way to create these (interpolation) is not yet implemented.

However, creating simple attribute sets like `{ }`, or `{ a = 15; }`
or `{ a = 10 * 2; }` works.

Recursive attribute sets are not yet implemented as we do not have any
kind of scope access yet anyways.

This is implemented using a new instruction that creates an attribute
set with a given number of elements by popping key/value pairs off the
stack.

Change-Id: I0f9aac7a131a112d3f66b131297686b38aaeddf2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6091
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 15:31:50 +00:00
Vincent Ambo
e24248df3b feat(tvix/value): add some necessary helpers for strings
Deriving Ord/Eq is required for the ordered BTreeMaps. Once interning
is implemented this will require some extra magic for the sort order,
but that's fine.

Change-Id: I0c654648eb3609a4a01d84868c25f43a4d35bc2e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6089
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 11:50:37 +00:00
Vincent Ambo
865717a8db fix(tvix/eval): print code even if runtime fails
Change-Id: I357c9adf939cb6001afa73ad02282d94ee22d0ba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6088
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 11:50:37 +00:00
Vincent Ambo
2ed38a7cdb feat(tvix/eval): add Value variants for strings & attrsets
Change-Id: Idebf663ab7fde3955aae50f635320f7eb6c353e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6087
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 11:50:37 +00:00
Vincent Ambo
ba03226e51 feat(tvix/eval): add module for attribute set implementations
Change-Id: I6002bd5e5596b93325dea6c862370ba5235c0f08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6086
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 11:50:37 +00:00
Vincent Ambo
d26c097d1b feat(tvix/eval): add module for string type implementation
Change-Id: I5e4465acc4a676c10d7374b14f7a09240202b466
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6085
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 11:50:37 +00:00
Vincent Ambo
31443d21ee chore(tvix): move nix-store CLI scaffolding to subfolder
For some reason a top-level Rust project ended up in this location,
which is incompatible with the actual project structure that's being
prepared for merge right now.

Change-Id: I9d919ad72fc7e4e4d8cbb9899e7f8d90fa7ca87a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6060
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
2022-08-12 15:58:59 +00:00
Vincent Ambo
3ed45caad1 feat(tvix/eval): implement Display trait for Value enum
This representation should match what the Nix REPL shows for result
values.

Change-Id: If3143d969fcdc123a6029e2aeb7bbd6ae51aeb71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6082
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:12:07 +00:00
Vincent Ambo
0d2896519c refactor(tvix/eval): move NumberPair struct definition to vm module
This isn't relevant to the value type itself.

Change-Id: I678bc92a8a530b1081ed498bf3ff7925217bcc01
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6081
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:09:27 +00:00
Vincent Ambo
9c5a249b30 feat(tvix/compiler): incompletely handle true/false/null literals
These are a bit tricky to implement because Nix technically treats
them as identifiers, and only if the identifier is not explicitly
overridden within the scope does it yield the expected literal values.

Note that weirdness even occurs with scopedImport.

Change-Id: Ie55723405ccfcc25da37c5a08fa3332f37cf9ae5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6080
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:09:27 +00:00
Vincent Ambo
d431f43f5f feat(tvix/eval): implement boolean inversion operator
Change-Id: Icb1d449fdee4d67b5f1eefdbc01baa1584ea0a67
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6079
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:09:27 +00:00
Vincent Ambo
ded0fb9e21 feat(tvix/eval): implement equality operator
Change-Id: I9dd54aed72cd7e67593dc76f5a046ebbda40c26f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6078
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:09:27 +00:00
Vincent Ambo
28f57abac1 refactor(tvix/compiler): use rnix's typed AST for literal values
Change-Id: Ic56ab64ad82343c7cdf8168ef41ee0a97f7e1dd9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6077
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:24:10 +00:00
Vincent Ambo
18fe188c3e feat(tvix/compiler): implement parens precedence
Change-Id: I8944354b3690d7504e4fe4254f14be5b849b9bcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6076
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:24:10 +00:00
Vincent Ambo
72be759e1e feat(tvix/eval): implement unary negation operator
Change-Id: I5d012cc073e55d79d7b34b88283aab3164864293
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6075
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:24:10 +00:00
Vincent Ambo
d35ecc0caf feat(tvix/eval): implement simple arithmetic binary operations
Implements simple arithmetic operations (+, -, *, /).

There is some scaffolding included to pop and coerce pairs of numbers,
as the Nix language will let arithmetic operators apply to arbitrary
pairs of number types (always resulting in floats if the types are
mixed).

Change-Id: I5f62c363bdea8baa6ef812cc64c5406759d257cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6074
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:24:10 +00:00
Vincent Ambo
e96a2934ad feat(tvix/eval): add error variant for runtime type errors
Change-Id: I74155cf01766b7a991a69522945bff67fbca5a16
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6073
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:24:10 +00:00
Vincent Ambo
e8253c7044 chore(tvix/eval): wire things up for development flow
This creates a REPL which outputs compiled bytecode, constants, and VM
results for code snippets.

Change-Id: If63f79a961456afd6a4cdf59b994107ff7ab8b47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6072
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-08-12 13:05:28 +00:00
Vincent Ambo
d59968649e feat(tvix/eval): add initial stack-based VM
This can't do anything other than compute a single literal, for now

Change-Id: Ia28f9da51c906b590a198e77a4ca5d45a871106b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6071
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:05:16 +00:00
Vincent Ambo
34df2c8473 feat(tvix/eval): add initial barebones compiler
This compiler can only take care of very trivial literals so far.

Change-Id: I9dfac75a801b7235f868061a979ae24159fe1425
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6070
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:05:16 +00:00
Vincent Ambo
3aaefc3000 feat(tvix/eval): add initial chunk representation
Change-Id: I53202e93938bede421c8f1c98901e4c67544e257
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6069
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:05:16 +00:00
Vincent Ambo
48a7449834 feat(tvix/eval): add initial opcode representation
Change-Id: Ibc7685a6b0b92e08f0b6c82cf7d9b04fbb593a4e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6068
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:05:16 +00:00
Vincent Ambo
8b7d2fd59e feat(tvix/eval): add initial value representation
Change-Id: I427a04e89994662df2750dffe21991bad48aab15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6066
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 12:39:19 +00:00
Vincent Ambo
bef8d67b6c chore(tvix/eval): add rnix-parser dependency
Change-Id: I81bd8416b3837a728ecd7911fe1ca06e89b9e90e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6065
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 12:39:19 +00:00
Vincent Ambo
8921688334 chore(tvix/eval): bootstrap some evaluator boilerplate
Change-Id: I7770a20948d18a8506c2418dea21202aa21a6ddc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6064
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 12:39:19 +00:00
Vincent Ambo
a9b2157fba chore(tvix/eval): check in naersk-based default.nix
Change-Id: I9ff0c4a196747361a3e421863beaff07a4972b0f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6067
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 12:39:19 +00:00
Vincent Ambo
7be4581876 docs(tvix/eval): Add initial evaluator README
Change-Id: I85df002dc13c91a184d064586244f6a7440320fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6063
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 12:34:29 +00:00
Vincent Ambo
d1bf3d3577 feat(tvix/eval): check in generated project skeleton
Change-Id: Iecc8283abb289de71f22076fd88892f6ded99cb3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6062
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 12:34:29 +00:00
Vincent Ambo
3d8ee62087 style(rust): Format all Rust code with rustfmt
Change-Id: Iab7e00cc26a4f9727d3ab98691ef379921a33052
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5240
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-02-08 12:06:39 +00:00
Vincent Ambo
aa122cbae7 style: format entire depot with nixpkgs-fmt
This CL can be used to compare the style of nixpkgs-fmt against other
formatters (nixpkgs, alejandra).

Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: cynthia <cynthia@tvl.fyi>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: eta <tvl@eta.st>
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-31 16:11:53 +00:00
Profpatsch
80defc2101 feat(tvix): set up a simple command line parser for nix-store
We are going to have a 1:1 drop-in replacement for the old-style nix
tools, and this starts implementing the cli parser part.

The first step is to have a simple integration test suite that can
verify that we match the nix CLI.

clap is a super complicated parsing library, but looking through the
rest they are either too opinioated to be of use for us, or depend on
clap as implementation.

Change-Id: I4cf6051f3a4f782c3242fd0d2b9eab3fbe33d8ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4756
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Profpatsch <mail@profpatsch.de>
2022-01-06 19:35:11 +00:00
Profpatsch
7c92b07efd feat(tvix): set up cargo rust project
First steps for baba

Change-Id: Id6a68c5630cb85f280f4dcc7b2acf10c02454fd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4732
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-01-06 15:38:52 +00:00
Vincent Ambo
8c2084d0a1 feat(tvix/proto): Add evaluator communication proto
This adds the proto messages and service definition used for
communicating with our evaluator.

Please refer to the `Tvix - Component interaction` document for a
detailed description of the interactions that this service is used
for.

Change-Id: If44cdbff66c6cfe88c22a748fe29c0bbd8510b0d
Co-Authored-By: Florian Klink <flokli@flokli.de>
Co-Authored-By: Adam Höse <adisbladis@gmail.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2685
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: grfn <grfn@gws.fyi>
2021-05-08 22:50:34 +00:00
Vincent Ambo
9c8b904b56 docs(tvix): Begin specification of the Nix language
This is only the introductory text, without any of the components
specified so far.

Change-Id: I6cd8eff6ec2daf14cc2420ee0a8e8983bfaa57d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3052
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-24 20:07:18 +00:00
adisbladis
c693e8bf51 chore(tvix): Make docs multi output
We also want to access the produced SVGs separately.

Change-Id: I5bdfd95b8a56323ed8bf5f4585b703f6e728c677
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2742
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-03-31 22:41:07 +00:00
Florian Klink
3c4e401c9e chore(tvix/docs): move from doc
Most other docs folders in the repo are called `doc` too, let's make
this consistent.

Change-Id: Icd712429b51763076548c977321e370f2a77877e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2741
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-03-31 22:34:59 +00:00
Vincent Ambo
1fccc23f3c docs(tvix/components): Editing pass
* general editing
* reflowed text to fit on my screen :p

Change-Id: I1ae41ef5524d3174d063c66a7bcb990808424bdb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2716
Tested-by: BuildkiteCI
Reviewed-by: adisbladis <adisbladis@gmail.com>
2021-03-31 22:29:07 +00:00
adisbladis
58cca2faaa chore(tvix): Add doc build infrastructure
Change-Id: I2cf67df085d0c008b5ff5efff2235a670207024a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2711
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <mail@tazj.in>
2021-03-31 21:51:59 +00:00
adisbladis
5fa2d39dde docs(tvix): Add components doc
Change-Id: I53856bf98c5c2057f95e9c9af5f07c16ced55e0f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2712
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2021-03-31 21:51:59 +00:00
Vincent Ambo
959b8f3766 chore(tvix): Bootstrap Tvix folder
Change-Id: Ib4d4736c48a31510660b2861aabb9b7ef1320209
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2684
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2021-03-27 00:09:49 +00:00