Commit graph

138 commits

Author SHA1 Message Date
Vincent Ambo
7cfdedfdfb feat(tvix/eval): compile with expression
Adds an additional structure to the compiler's scope to track the
runtime "with stack", i.e. the stack of values through which
identifiers should be dynamically resolved within a with-scope.

When encountering a `with` expression, the value from which the
bindings should be resolved is pushed onto the stack and tracked by
the compiler in the "with stack", as well as with a "phantom value"
which indicates that the stack contains an additional slot which is
not available to users via identifiers.

Runtime handling of this is not yet implemented.

Change-Id: I5e96fb55b6378e8e2a59c20c8518caa6df83da1c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6217
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-31 22:26:11 +00:00
Vincent Ambo
cfe37d36f7 fix(tvix/eval): or should handle non-attrset values, too
If a nested attrpath encounters a non-set value, the sentinel value
denoting a lack of next values should be emitted. This mirrors the
behaviour of Nix.

Change-Id: Ia80443d5a11243cc6d98dcab1249a3f5fdf77e27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6210
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
76846fe220 fix(tvix/eval): allow use of ? operator on non-set types
Nix allows this, but always returns false. Tvix needs to do the same.

Change-Id: Ic9eec90834a0d0969eea5316d5c25032d3691d94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6209
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
ab9407bded fix(tvix/eval): address various clippy lints
Change-Id: I3ea0f51475e80948adfeb5d1620c1f2665cc39bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6201
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-30 16:53:40 +00:00
Vincent Ambo
57d0dbb1c6 feat(tvix/eval): implement optional runtime tracing
This adds a `disassembler` feature to the crate configuration that
traces the operations executed and the state of the stack at runtime.

This can be enabled by compiling with `--feature disassembler`.

This will also gain a more sensible layout of code slices eventually.

Change-Id: I34c15e1cd346ecc4362b5afba6bf82dd49359d20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6193
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-30 16:53:40 +00:00
Vincent Ambo
342b233a0a feat(tvix/eval): add local identifier access
This makes basic `let ... in ...` statements work correctly. It does
not yet account for the call frames pushed into the VM during function
application.

Change-Id: I67155171daf1a43011b96716dd9d1ab04b27db33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6190
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-28 17:50:34 +00:00
Vincent Ambo
691a596aac feat(tvix/eval): compile simple let ... in ... expressions
These expressions now leave the binding values on the stack, and clean
up the scope after the body of the expression.

While variable access is not yet implemented (as the identifier node
remains unhandled), this already gives us the correct stack behaviour.

Change-Id: I138c20ace9c64502c94b2c0f99a6077cd912c00d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6188
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-28 17:50:34 +00:00
Vincent Ambo
b8cec6d61e feat(tvix/vm): add opcodes for new Value::NotFound sentinel
This sentinel value is going to be used for certain nested accesses
into attribute sets.

There is a new instruction similar to `OpAttrsSelect` which leaves the
sentinel on the stack if a key is not found, instead of raising an
error.

Additionally, a new jump instruction makes its jump operation
conditional on finding such a sentinel value.

Change-Id: I2642f0a0bcc85bbe0ead68ea09a7dd794dbedeac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6166
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 15:06:52 +00:00
Vincent Ambo
cf3e3b784b feat(tvix/eval): implement ? operator (single-level only)
This makes it possible to check things like `{} ? a` with a single
level of nesting.

Change-Id: I567c36fcfd2f9e2f60071acd3ebfe56dea59b26f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6161
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-26 09:02:38 +00:00
Vincent Ambo
20f5ccefeb feat(tvix/eval): implement attribute set access operator
Fairly straightforward, handling the optimised representations
manually and otherwise delegating to BTreeMap.

Note that parsing of raw identifiers is not yet implemented.

Encountering an identifier node usually means that there is locals
access going on, so we need a special case for compiling a node in
such a way that an identifier's literal value ends up on the stack.

Change-Id: I13fbab7ac657b17ef3f4c5859fe737c321890c8a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6158
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 09:02:38 +00:00
Vincent Ambo
671915837a fix(tvix/eval): add operation to assert boolean type
This operation is required because both sides of the logical operators
are strictly evaluated by Nix, even if the resulting value is not used
further.

For example, in our implementation of `&&`, if the left-hand side is
`true`, then the result of the expression is simply the right-hand
side value. This value must be asserted to be a boolean for the
semantics of the language to work correctly.

Change-Id: I34f5364f2a444753fa1d8b0a1a2b2d9cdf7c6700
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6157
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 09:02:38 +00:00
Vincent Ambo
c7c7ab9bd4 feat(tvix/compiler): implement || operator
Same dance as `&&` but logically inverted.

Change-Id: I213e200e3836527e9abe510f354ee7cd1f70d041
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6151
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 16:00:55 +00:00
Vincent Ambo
d9d94eb27f feat(tvix/eval): implement if/else expressions
These expressions use simple jumps to skip the correct expression
conditionally in the bytecode by advancing the instruction pointer.

Note that these expressions are already covered by a test behind the
`nix_tests` feature flag, but adding more is probably sensible.

Change-Id: Ibe0eba95d216321c883d3b6b5816e2ab6fe7eef1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6148
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 16:00:55 +00:00
Vincent Ambo
9407af5684 refactor(tvix/eval): encapsulate list construction in value::list
Ensuring that the implementation is not leaking out of the module lets
us keep things open for optimisations (e.g. empty list or pairs
through tuples).

Change-Id: I18fd9b7740f28c55736471e16c6b4095a05dd6d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6145
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 12:07:30 +00:00
Vincent Ambo
75a22321ce feat(tvix/eval): implement list concatenation
Change-Id: Icdf715d116371a9f139bdf95266410bf967bef25
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6144
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 12:07:30 +00:00
Vincent Ambo
4eafaae9e6 feat(tvix/eval): implement binary comparison operators
This is accomplished by simply delegating to the Rust implementations
of (Partial)Ord and (Partial)Eq, which are implemented for Value and
underlying wrapper types to behave like they do in Nix.

To ease the implementation overhead, a new comparison operator macro
has been added to the VM module.

Incomparable types will raise a new error variant when a comparison is
attempted, containing both supplied types. This mimics the information
carried in the error thrown by C++ Nix.

Change-Id: Ia19634d69119d40722f3ca672387bc3a80096998
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6143
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 12:07:30 +00:00
Vincent Ambo
5a15ee3f5d refactor(tvix/vm): simplify implementation of OpEqual
This can now fully delegate to PartialEq of Value

Change-Id: Iaa9f4ec9b8830d516d72f83a93ab2df9a6e5697c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6142
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:34:00 +00:00
Vincent Ambo
058e77bab2 feat(tvix/eval): implement attrset update (//) operator
The underlying implementation does a few tricks based on which pair of
attrset representations is encountered.

Particularly the effect of short-circuiting the empty cases might be
relevant in nixpkgs/NixOS, due to the use of lib.optionalAttrs.

Change-Id: I22b978b1c69af12926489a71087c6a6219c012f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6140
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-25 11:34:00 +00:00
Vincent Ambo
fb3d024d75 feat(tvix/eval): implement string concatenation
Change-Id: If61066e59232b2bad42b5cb5f0f2d9b9c416be8b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6137
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:11:27 +00:00
Vincent Ambo
322ce36cea refactor(tvix/vm): use a macro to handle binary arithmetic operators
Instead of constructing another runtime value representing the pair on
which to perform arithmetic, implement the same logic in the shape of
a macro.

This is designed to be compatible with operators like `+` that work
both as an arithmetic operator AND as an operator on another pair of
types.

Change-Id: I1c83649ead6117f811f1fb45482d0cadf811125e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6136
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 11:07:16 +00:00
Vincent Ambo
6dc9ca5723 feat(tvix/value): introduce string representation with &'static str
For cases where the strings are statically known (such as the
oft-occuring name/value), this can be a useful optimisation.

It's also much more convenient in tests.

Change-Id: Ie462b684805bd4986ea5e85ca4bff663bc2d3c3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6111
Tested-by: BuildkiteCI
Reviewed-by: eta <tvl@eta.st>
2022-08-24 18:19:52 +00:00
Vincent Ambo
293fb0ef53 refactor(tvix/value): encapsulate attrset logic within value::attrs
The internal optimisations of the set representation were previously
leaking into the VM, which is highly undesirable.

Keeping it encapsulated allows us to do additional optimisations
within value::attrs without being concerned about its use in the VM.

Change-Id: I7e7020bb0983b9d355d3db747b049b2faa60131f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6108
Reviewed-by: eta <tvl@eta.st>
Tested-by: BuildkiteCI
2022-08-24 18:19:52 +00:00
Vincent Ambo
52736a1dbc refactor(tvix/rm): introduce helper for AttrSet Entry API
There are multiple points where an insertion needs to be done into an
attribute set, but copying the key or checking for presence before
insertion should be avoided

As that is a little bit noisy, it's been factored out into a helper
function in this commit.

Change-Id: Ibcb054ebeb25a1236c06c812f47c8e74180d4fc9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6107
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-14 00:24:33 +00:00
Vincent Ambo
295d6e1d59 feat(tvix/vm): implement first nested attribute set construction
This can construct non-overlapping nested attribute sets (i.e. `{ a.b
= 1; b.c = 2; }`, but not `{ a.b = 1; a.c = 2; }`).

In order to do the latter, it's necessary to gain the ability to
manipulate the in-progress attribute set construction. There's
multiple different options for this ...

Change-Id: If1a762a720b175e8eb4216cbf96a7434d22640fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6106
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-14 00:23:32 +00:00
Vincent Ambo
e15bd9aa63 fix(tvix/eval): Fail on duplicate attribute set keys
Change-Id: I57373ca76d0e25a5d08a8dfce9d5949099326fc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6104
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 20:24:12 +00:00
Vincent Ambo
175eb97505 feat(tvix/eval): construct internal attribute path representation
This is required for constructing nested attribute sets at runtime.

There'll be quite a lot of optimisation potential with this solution
eventually, if it should turn out to be a bottleneck.

This introduces a conceptual change, in that the `Value` enum is now
an enum representing "all runtime values" instead of "all Nix language
types". This makes sense in general, as this type will also contain
Chunk representations etc. which are not exposed to users.

Change-Id: Ic5f72b2a0965b146c6a451efad34c6a81ca1aad8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6103
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-13 20:24:12 +00:00
Vincent Ambo
ec1770f95a feat(tvix/vm): implement construction of optimised KV attrsets
For name/value pairs (which occur extremely often in Nix and make up a
significant chunk of the runtime cost of evaluating nixpkgs) we
substitute an optimised representation.

For now this will only be used if the name/value pair keys were
specified as literal identifiers or strings (i.e. if chunks are
encountered as keys they are not forced and a normal attribute set
backed by a map will be constructed).

Change-Id: Ic79746c323e627528bd58b1a6024ee8d0aff7858
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6102
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-13 20:24:12 +00:00
Vincent Ambo
a93933b487 feat(tvix): implement string interpolation
This adds a new instruction which assembles an interpolated string
from a specified number of fragments, which are already going to be
located on the stack in the right position.

This will raise a type error if any of the fragments do not evaluate
to a string.

Change-Id: I5756248fa3e9fcc3d063c14db40b332f7e20a588
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6098
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 18:28:36 +00:00
Vincent Ambo
5763b62172 feat(tvix/vm): implement list construction
Change-Id: Iec2b4910800ab29daae6d71b58a8acd14ccb1cc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6094
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-08-13 15:35:24 +00:00
Vincent Ambo
57a723aaa9 feat(tvix/eval): implement trivial attribute set literals
Implements attribute set literals without nesting. Technically this
already supports dynamic key fragments (evaluating to strings), though
the only way to create these (interpolation) is not yet implemented.

However, creating simple attribute sets like `{ }`, or `{ a = 15; }`
or `{ a = 10 * 2; }` works.

Recursive attribute sets are not yet implemented as we do not have any
kind of scope access yet anyways.

This is implemented using a new instruction that creates an attribute
set with a given number of elements by popping key/value pairs off the
stack.

Change-Id: I0f9aac7a131a112d3f66b131297686b38aaeddf2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6091
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 15:31:50 +00:00
Vincent Ambo
2ed38a7cdb feat(tvix/eval): add Value variants for strings & attrsets
Change-Id: Idebf663ab7fde3955aae50f635320f7eb6c353e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6087
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 11:50:37 +00:00
Vincent Ambo
0d2896519c refactor(tvix/eval): move NumberPair struct definition to vm module
This isn't relevant to the value type itself.

Change-Id: I678bc92a8a530b1081ed498bf3ff7925217bcc01
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6081
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:09:27 +00:00
Vincent Ambo
9c5a249b30 feat(tvix/compiler): incompletely handle true/false/null literals
These are a bit tricky to implement because Nix technically treats
them as identifiers, and only if the identifier is not explicitly
overridden within the scope does it yield the expected literal values.

Note that weirdness even occurs with scopedImport.

Change-Id: Ie55723405ccfcc25da37c5a08fa3332f37cf9ae5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6080
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:09:27 +00:00
Vincent Ambo
d431f43f5f feat(tvix/eval): implement boolean inversion operator
Change-Id: Icb1d449fdee4d67b5f1eefdbc01baa1584ea0a67
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6079
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:09:27 +00:00
Vincent Ambo
ded0fb9e21 feat(tvix/eval): implement equality operator
Change-Id: I9dd54aed72cd7e67593dc76f5a046ebbda40c26f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6078
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 14:09:27 +00:00
Vincent Ambo
72be759e1e feat(tvix/eval): implement unary negation operator
Change-Id: I5d012cc073e55d79d7b34b88283aab3164864293
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6075
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:24:10 +00:00
Vincent Ambo
d35ecc0caf feat(tvix/eval): implement simple arithmetic binary operations
Implements simple arithmetic operations (+, -, *, /).

There is some scaffolding included to pop and coerce pairs of numbers,
as the Nix language will let arithmetic operators apply to arbitrary
pairs of number types (always resulting in floats if the types are
mixed).

Change-Id: I5f62c363bdea8baa6ef812cc64c5406759d257cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6074
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:24:10 +00:00
Vincent Ambo
d59968649e feat(tvix/eval): add initial stack-based VM
This can't do anything other than compute a single literal, for now

Change-Id: Ia28f9da51c906b590a198e77a4ca5d45a871106b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6071
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-12 13:05:16 +00:00