Commit graph

383 commits

Author SHA1 Message Date
Vincent Ambo
7d5dca7ba3 chore(tvix/eval): provide 'static references to "name"/"value"
These static strings show up a bunch when dealing with the internals
of attribute sets, and having them available as static references is
required.

Due to the way const expressions are evaluated, taking a reference to
the existing NixString::NAME / NixString::VALUE items does not work
and the references themselves need to be const-evaluated.

Change-Id: If6e75847af978118a3b266fe6a3242321722434d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6366
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
80713f207e refactor(tvix/eval): encapsulate all thunk-forcing logic in module
The VM previously took care of repeatedly forcing a thunk until it
reached an evaluated state. This logic is now encapsulated inside of
the `Thunk::force` implementation.

In addition, force no longer returns a reference to the value by
default, leaving it up to callers to decide whether they want to
borrow the value or not (a helper is provided for this).

Change-Id: I2aa7da922058ad1c57fbf8bfc7785aab7971c02b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6365
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
23a5caabec feat(tvix/eval): construct attribute sets lazily
This thunks the construction of attribute sets. Because Tvix does not
currently have a "strict output" mode, a test had to be disabled that
now displays a thunk representation.

The test will be re-enabled once that is available.

Change-Id: I360332be64cd5c154f9caea21828f6f1b37a265c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6363
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
727845645d chore(tvix/eval): add release-with-debug cargo profile
This lets us create a release build with debug info, for use with e.g.
perf + hotspot

Change-Id: I03897de36c872d318abf1332ca0c1aeabe344ec6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6362
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
79a2ba5175 fix(tvix/eval): avoid recomputing width in disassemble_op constantly
As noticed by sterni in cl/6195

Change-Id: Ie9c1e80e2e709284fa8412334af9188d999f64dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6361
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
a8b2ba07df fix(tvix/eval): correctly resolve identifiers in inherit
This should not have grown a second implementation of the identifier
resolution logic, but it somehow did.

This implementation ended up being incorrect because it did not
account for upvalues inside of thunks.

Change-Id: Ieb1364d8fe43c96aaf4b125fd4b8a522aedff167
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6360
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
17dfb92a9f chore(tvix/eval): implement Debug for compiler::scope::Scope
Change-Id: I112b0119bd0511f26bb72f7e73d867d1b7144a36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6359
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
9420a3b53d feat(tvix/eval): insert strictness points for attribute set keys
All attribute set *key* related operations strictly evaluate all key
fragments, including during construction of an attribute set.

Change-Id: I3519e5e9b0886c2cdc8615ea7dcb5f7be0c59b3f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6358
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
5b4f3811e6 feat(tvix/eval): insert strictness points for unary/binary operators
The arguments of all unary/binary operators that are built in to Nix
are forced when encountered. This emits the necessary OpForce operations.

Change-Id: I691fcdbebfe7586cfe217c68d44b10b1192f82d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6357
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
6a42d9bddf chore(tvix/eval): thread slot value through all compiler methods
With this change any compilation of an expression is aware of its own
stack slot if it is leaving identifiers on the stack.

Change-Id: I0c9f148ae06b078a46b25180c4961686d5f2e166
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6356
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
50c33989dc chore(tvix/eval): thread slot value through Compiler:compile_attr
This threads the current local slot through the `compile_attr`
function and all of its callers. At the moment this does not improve
any user-facing behaviour, just internally changes the way in which
some correct expressions would fail to run.

Eventually this slot will need to reach everywhere ...

Change-Id: Iba73123dd1ced421093d8fc18ebeeffc16efacf8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6355
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
372bf4a0b7 test(tvix/eval): test very simple late-bound identifier access
This is the simplest kind of thunk that can be created (and so far the
only one the compiler knows how to create), in which an identifier
inside a `let` encounters a value that is bound *after* it is
initialised.

Change-Id: I6ea4408a3baef1e7d5137365d70804283f2dbf8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6354
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
786e12a738 feat(tvix/eval): always emit OpForce as the last instruction
Change-Id: Id70c987f654dc5d9b47db74e395281309762b468
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6353
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
9a783e50a4 feat(tvix/eval): implement OpForce in VM
This operation forces the evaluation of a thunk.

There is some potential here for making an implementation that avoids
some copies, but the thunk machinery is tricky to get right so the
first priority is to make sure it is correct by keeping the
implementation simple.

Change-Id: Ib381455b02f42ded717faff63f55afed4c8fb7e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6352
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
a8c13a5ce1 fix(tvix/eval): allocate Thunk::upvalues with known capacity
The capacity (i.e. number of builtins) is known from the lambda, so we
can size it correctly right away.

Change-Id: Iab0b5a3f47d450fa9866c091ebbbed935b934907
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6351
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:54:42 +00:00
Vincent Ambo
6c895d4b28 feat(tvix/eval): implement OpThunk for runtime thunk construction
Implements an operation very similar to `OpClosure` which populates a
thunk's upvalues and leaves it on the stack.

Change-Id: I753b4dfeeaae6919316c7028ec361aaa13d87646
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6350
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:54:42 +00:00
Vincent Ambo
6c4b9bdce9 refactor(tvix/eval): extract VM::populate_upvalues function
This function is reusable between thunks & closures.

Change-Id: I44d5f9897b087a385c8e75027d2ff39c48a096f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6349
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:36:46 +00:00
Vincent Ambo
ce498052e6 refactor(tvix/eval): Carry lambda & upvalues directly in CallFrame
CallFrame has to work for both thunks & closures (as a thunk is
basically a "weird 0-argument closure").

We opt to store the common, relevant fields directly in the frame to
avoid having to dereference through the nested structures
constantly (which would be especially annoying in the case of thunks).

Change-Id: I47781597b84ec5cd55502dba1713e92cf2592af3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6348
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
25c62dd0ef refactor(tvix/eval): introduce UpvalueCarrier trait
This trait abstracts over the commonalities of upvalue handling
between closures and thunks.

It allows the VM to simplify the code used for setting up upvalues,
without duplicating between the two different types.

Note that this does not yet refactor the VM code to optimally make use
of this.

Change-Id: If8de5181f26ae1fa00d554f1ae6ea473ee4b6070
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6347
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
8033a7abae refactor(tvix/eval): simplify thunk representations
For now, do not distinguish between closing and non-closing thunks, it
will make the initial implementation easier. See Knuth etc.

Change-Id: I0bd51e0f89f2c77e90bac63b507e5027b649e3d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6346
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
28a9847c65 feat(tvix/eval): emit thunks for recursive local scope resolution
When resolving a value on the same level that is known but not yet
defined, emit a thunk.

Consider for example:

    let
      #       v--- requires a thunk
      a = 1 * b;
      b = 10;
    in a

Change-Id: I922cb50973ebe05e335a7bc7cb851960cf34733b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6345
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
d1798444be feat(tvix/eval): Add Compiler::thunk method for emitting thunks
The logic in this method is *very* similar to `compile_lambda`. It is
intended to be called around any expression that should be
thunked (such as function applications, attribute set values, etc.).

Change-Id: Idfbb2daa9f4b735095378fb9c39a2fd07c8cff91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6344
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
1f37275cfb feat(tvix/eval): introduce Value::Thunk variant
Introduces the representation of runtime thunks, that is lazily
evaluated values. Their representation is very similar to closures.

Change-Id: I24d1ab7947c070ae72ca6260a7bbe6198bc8c7c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6343
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
49296cebe3 refactor(tvix/eval): extract compiler's upvalue logic into helper
This exact same logic is reused for thunk creation.

Change-Id: I731db9cc659a1f2ca87db55d58d6ff632f417812
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6342
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
70b27a6080 test(tvix/eval): add a test case for nested, deferred upvalues
This test case was previously broken by the bug introduced by
confusing local and stack indexes.

Change-Id: Ibef299dad266c6105deac1da5dde112fe9f640b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6341
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
24811d7655 fix(tvix/eval): distinguish statically between StackIdx and LocalIdx
Previously the functions in the scope module returned usize values,
which - sometimes from the same function - were either indexes into
the runtime stack *or* indexes into the compiler's local stack.

This is extremely confusing because it requires the caller to be aware
of the difference, and it actually caused subtle bugs.

To avoid this, there is now a new LocalIdx wrapper type which is used
by the scope module to return indexes into the compiler's stack, as
well as helpers for accounting for the differences between these
indexes and the runtime indexes.

Change-Id: I58f0b50ad94b28a304e3372fd9731b6590b3fdb8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6340
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
68aad6d4cf feat(tvix/eval): emit OpFinalise when local scopes are complete
With this change, the runtime can correctly capture deferred upvalues.

Change-Id: I1e43b7b1ac2553b1812424adfc8bd08ef77bf1ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6339
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
5c4e102ac8 feat(tvix/eval): track whether locals needs to be finalised
When encountering a deferred local upvalue, the compiler will now mark
the corresponding local as needing a finaliser which makes it possible
to emit the OpFinalise instruction for this stack slot a little bit
down the line.

Change-Id: I3962066f10fc6c6e1472722b8bdb415a811e0740
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6338
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
025a9a4a0a feat(tvix/eval): implement OpFinalise instruction
This instruction finalises the initialisation of deferred upvalues in
closures (and soon, thunks).

The compiler does not yet emit this instruction, some more accounting
is needed for that.

Change-Id: Ic4181b26e19779e206f51e17388559400da5f93a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6337
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
1bfe32f412 feat(tvix/eval): set up deferred upvalues at runtime
This will leave a sentinel value in the upvalue slot in which the
actual value is to be captured after resolution once a scope is fully
set up.

Change-Id: I12b37b0dc8d32603b03e675c3bd039468e70b354
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6336
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
6c1948a71a feat(tvix/eval): detect deferred upvalue capturing
Uses the threaded through slot offset to determine whether
initialisation of a captured local upvalue must be defered to a later
point where all values of a scope are available.

This adds a new data representation to the opcode for this situation,
but the equivalent runtime handling is not yet implemented. This is in
part because there is more compiler machinery needed to find the
resolution point.

Change-Id: Ifd0c393f76abfe6e2d91483faf0f58947ab1dedc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6329
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-06 14:58:52 +00:00
Vincent Ambo
203d9f2e3e chore(tvix/eval): thread current slot index through compiler
While compiling local bindings, we now know the stack slot of the
value currently being compiled.

This will let us determine whether an upvalue can be captured directly
or whether it needs to wait for a synchronisation point at which the
upvalue can be instantiated.

This machinery lets us avoid unnecessary work at runtime when
instantiating closures that actually do not need complicated recursive
resolution.

This change itself introduces no new functionality, but since the
threading is noisy it is split out as a separate change.

Change-Id: I847c677ee8f6725fda1d2efd689b6a58bdccb779
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6328
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-06 14:58:52 +00:00
Vincent Ambo
9dd1fa875d refactor(tvix/eval): optimise initialisation of locals
Instead of looking up the local to be initialised by its name again,
we can simply track the index at which it was declared from the point
where the declaration was made.

This reduces some string cloning and removes unnecessary logic. It
also theoretically makes the *current* index available during locals
compilation, which can be used to optimise some recursion cases.

Change-Id: I06f403603d4f86c3d319debfe74b5a52eec00990
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6327
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
900a92935d refactor(tvix/eval): declare all locals before compiling them
This actually makes things full-circle, as this tree already had this
implementation once before all the other required components were in
place.

With this commit, the compiler can resolve recursive upvalues within
the same scope (though they will not yet work at runtime).

Change-Id: I6267e477d08f367257c3a6dde054b880d7b47211
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6326
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
47b3562867 refactor(tvix/eval): decouple local depth & initialisation tracking
In order to resolve recursive references correctly, these two can not
be initialised the same way as a potentially large number of (nested!)
locals can be declared without initialising their depth.

This would lead to issues with detecting things like shadowed
variables, so making both bits explicit is preferable.

Change-Id: I100cdf1724faa4a2b5a0748429841cf8ef206252
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6325
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
705c1b9d53 fix(tvix/eval): correct runtime error for missing dynamic upvalue
Change-Id: I75c351619780fdc5186a54f3df9b244ada984069
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6324
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
776120de05 fix(tvix/eval): instantiate *new* closures from blueprints each time
The previous closure refactoring introduced a bug in which the same
closure object would get mutated constantly for each instance of a
closure, which is incorrect behaviour.

This commit instead introduces an explicit new Value variant for the
internal "blueprint" that the compiler generates (essentially just the
lambda) and uses this variant to construct the closure at runtime.

If the blueprint ever leaks out to a user somehow that is a critical
bug and tvix-eval will panic.

As a ~treat~ test for this, the fibonacci function is being used as it
is a self-recursive closure (i.e. different instantiations of the same
"blueprint") getting called with different values and it's good to
have it around.

Change-Id: I485de675e9bb0c599ed7d5dc0f001eb34ab4c15f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6323
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
ccfb971dc5 fix(tvix/eval): correctly thread through dynamic upvalues
This puts together the puzzle pieces for threading dynamic
upvalues (that is, upvalues resolved from the `with`-stack) all the
way through.

Reading the test case enclosed in this commit and walking through it
is recommended to understand what problem is being tackled here.

In short, because the compiler can not statically know *which*
with-scope a dynamic argument is resolved from it needs to lay the
groundwork for resolving from *all* possible scopes.

There are multiple different approaches to doing this. The approach
chosen in this commit is that if a dynamic upvalue is detected, the
compiler will emit instructions to close over this dynamic value
in *all* enclosing lambda contexts.

It uses a new instruction for this that will leave around a sentinel
value in case an identifier could not be resolved, and wire the
location of this found value (or sentinel) up through the upvalues to
the next level of nesting.

In this tradeoff, tvix potentially closes over more upvalues than are
needed (but in practice, how often do people create *really* deep
`with`-stacks? and in *this* kind of code situation? maybe we should
even warn for this!) but avoids keeping the entire attribute sets
themselves around.

Looking at the test case, each surrounding closure will close
over *all* dynamic identifiers that are referenced later on visible to
it, but only the last one for each identifier will actually end up
being used.

This also covers our bases for an additional edge-case this creates,
in which an identifier potentially resolves to a dynamic upvalue *and*
to a dynamic value within the function's own scope (again, would
anyone really do this?) by introducing a resolution instruction for
that particular case.

There is likely some potential for cleaning up this code which is
quite ugly in some parts, but as this implementation is now carefully
calibrated to work I decided it is time to commit it and clean it up
in subsequent commits.

Change-Id: Ib701e3e6da39bd2c95938d1384036ff4f9fb3749
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6322
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
8982e16e26 refactor(tvix/eval): thread dynamic upvalues through all contexts
With this change, encountering a dynamic upvalue will thread through
all contexts starting from the lowest context that has a non-empty
`with`-stack.

The additional upvalues are not actually used yet, so the effective
behaviour remains mostly the same. This is done in preparation for an
upcoming change, which will implement proper dynamic resolution for
complex cases of nested dynamic upvalues.

Yes, this whole upvalue + dynamic values thing is a little bit
mind-bending, but we would like to not give up being able to resolve a
large chunk of the scoping behaviour statically.

Change-Id: Ia58cdd47d79212390a6503ef13cef46b6b3e19a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6321
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
f6de4434c3 feat(tvix/eval): allow ignoring locals by prefixing with _
This is a common idiom in both Nix and other languages when a local is
declared without actually being used.

Since Tvix warns for unused locals, having this available is useful
and can be included in the final error message as a suggestion if an
unused variable is intentional.

Change-Id: Ia85f704ba183499a3bae657c58166e2e29f9bde5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6320
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
a5e22c532b fix(tvix/eval): correctly resolve dynamic upvalues one scope up
This does not yet correctly resolve them if they are more than one
scope up, however.

Change-Id: I6687073c60aee0282f2b6ffc98b34c1e96a60f20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6319
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
e46a2ce3ef fix(tvix/eval): account for stack offset when pushing a with scope
Change-Id: I4b98eaea3ed5059c29938a117a9d59499a0bb95d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6318
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
ae531a2245 feat(tvix/eval): implement capture of self-recursive upvalues
With this change, it becomes possible for functions to call themselves
as they are being defined in local bindings.

Change-Id: Ib46a39ba17b1452b5673d96fa729d633d237241a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6314
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
06b07f5c47 docs(tvix/eval): start a document on known optimisation potential
Change-Id: I9bc41e57e1cfdf536d7b9048bac2e7aff1ee2ffa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6313
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
Vincent Ambo
3089e46eb1 refactor(tvix/eval): encapsulate internal mutability within Closure
This is required to efficiently construct the upvalue array at
runtime, as there are situations where during Closure construction
multiple things already have a reference to the closure (e.g. a
self-reference).

Change-Id: I35263b845fdc695dc873de489f5168d39b370f6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6312
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:29:25 +00:00
sterni
18efbac5c5 docs(tvix/eval): pad pure column so it can fit "false"
Change-Id: Iaedfc281db82de1e8eb2400db1118c8431d2579f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6333
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-04 21:05:57 +00:00
sterni
e25972877a docs(tvix/eval): add a list of builtins added in Nix >= 2.4
`builtins.getFlake` doesn't interest us, of course, but some others may
be worth (or easy) to implement. They are pretty low priority, though,
since nixpkgs has compatiblity wrappers for the ones it uses.

The new debugging-related builtins (break and traceVerbose) are
interesting to note, but may not make sense to implement at all.

Change-Id: Icae547aa3bd9d6ee6b87897ba8210eb9b9b044c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6332
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-04 21:02:51 +00:00
sterni
b0d39aa19f fix(tvix/eval): don't rebuild drv upon changes in cargo target dir
This avoids unnecessary rebuilds of //tvix/eval when working on it
locally using impure cargo.

Change-Id: I028033a6345a9655e2877a534448706b8f85a1a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6317
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-04 21:02:50 +00:00
Vincent Ambo
932a70d0c2 refactor(tvix/eval): track with stack size as a simple integer
The `With` struct no longer contained any internals after the cleanup
logic for the stack had been moved into Compiler::compile_with,
leaving the `Vec<With>` to essentially act as a counter for the number
of things on the with stack.

That's inefficient of course, so with this commit it actually becomes
an integer (with an encapsulated API within scope::Scope).

Change-Id: I67a00987fc8b46b30d369a96d41e83c8af5b1998
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6311
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:53:08 +00:00
Vincent Ambo
2cd08d136e refactor(tvix/eval): move compiler's scope logic to separate module
The compiler module is getting quite long and this will help keep some
order.

Right now the scope internals are not very well encapsulated; this
paves a way to reducing the API surface of the `scope` type to the
things that are actually used by the compiler instead of giving access
to its internals.

Change-Id: I8c16c26d263f018baa263f395c9cd80715199241
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6310
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:53:08 +00:00
Vincent Ambo
83aa5917b4 fix(tvix/eval): declare locals before marking them initialised
This has no effect yet, other than changing the way in which some
upvalue captures break (that are already not working correctly).

However, after this change the compiler correctly detects
self-recursion and can start emitting the instructions to deal with
this at runtime.

Change-Id: Id3b0ac206c0204739597a4325bcc66f9c806c242
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6309
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:53:08 +00:00
Vincent Ambo
4bf096ee6e fix(tvix/eval): Account for uninitialised variables in with_idx
Calculating the with_idx (i.e. the stack offset of the "phantom"
variable from which a `with` dynamically reads at runtime) needs to
account for unitialised variables the same way as the resolution of
normal locals does.

Change-Id: I9ffe404535bf1c3cb5dfe8d9e005798c857fff94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6308
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:53:08 +00:00
Vincent Ambo
27bc8cb3d4 fix(tvix/eval): open/close additional scope around with
This is required to correctly clean up the `with` values.

At the moment, the attrset from which identifiers are being taken is
always pushed on the stack. This means that it must also be removed
again, otherwise in an expression like

  with { a = 15; }; a

The final stack is `[ { a = 15; } 15 ]` *after the last operation*,
which means that the attrset is still on there as garbage.

This has little practical impact right now because it is always
shadowed by the fact that the actual expression value is at the right
location, but becomes relevant when accounting for upvalue captures.

Change-Id: I69e9745bfaa4d6bbcb60ee71f4dc3f8d8695d16a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6303
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
abd8f12f5d refactor(tvix/eval): extend resolve_local logic for self-recursion
This extends the logic of `Scope::resolve_local` to detect cases where
self-recursion is occuring (i.e. an identifier is being accessed in
its own identifier).

These cases are not yet handled specially, and the logic of when
things are marked initialised (which was previously always at the same
spot as their declaration) has not changed, making this commit a
runtime no-op for now.

Change-Id: I3179642a7c55869ad4465fdd2678b0cd51a20f15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6302
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
a52db14820 feat(tvix/eval): detect illegally shadowed variables
Nix does not allow things like `let a = 1; a = 2; in a`, but doing it
across depths is allowed.

Change-Id: I6a259f8b01a254b433b58c736e245c9c764641b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6301
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
00aeb6dfaf refactor(tvix/eval): introduce Depth enum to track variable status
This does not yet change anything semantically, but will be useful for
resolving simple cases of self-recursion etc.

Change-Id: I139ecb7e4a8a81193774392a96e73e0ea6b9f85d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6300
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
33cde1422e feat(tvix/eval): implement upvalue resolution in with scopes
These need to be handled specially by the runtime if the compiler
determines that a given local must be resolved via `with`.

Note that this implementation has a bug: It currently allows `with`
inside of nested lambdas to shadow statically known identifiers. This
will be cleaned up in the next commit.

Change-Id: If196b99cbd1a0f2dbb4a40a0e88cdb09a009c6b9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6299
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
10b0879c00 fix(tvix/eval): ensure OpResolveWith can be traced
The previous implementation of OpResolveWith manually controlled the
loop iteration, which skipped over the disassembler's tracing
instruction.

Instead, the resolution of dynamic variables has been delegated to a
new helper function. This has the additional benefit that the loop
labels are no longer required, making things a bit cleaner.

Change-Id: If22b74c3d49c74bf3a1ec4497cb761a9ee6cf2a4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6298
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
7c2fdefcd8 fix(tvix/eval): pop with stack immediately after processing body
Instead of tying the popping of the with stack to scope depth, clean
up the stack immediately after processing a with body.

The previous behaviour was actually incorrect, as it would leave
things on the with-stack longer than they were supposed to be there.
This could lead to false positive resolutions in some situations
involving closures.

Change-Id: I7b0638557503f1f71eb602e3d5ff193cdfcb67cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6297
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
26acc2e636 test(tvix/eval): add tests for very simple closures
Change-Id: Ib8287ade4d5df6d29e1812fb2d349cee5d92ca6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6296
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 17:40:10 +00:00
Vincent Ambo
f88d248f48 feat(tvix/eval): implement runtime closure construction (OpClosure)
Implements the final bit of logic remaining for wiring up closures,
which is the runtime construction of closure objects.

When encountering an OpClosure, the VM walks through the bytecode
collecting all the upvalue location operands (see commit introducing
the OpCode::Data* variants for details) and stores the runtime values
in the new closures upvalue vector.

After that, the handling of the closure itself becomes functionally
identical to that of lambdas.

With this initial implementation of closures there are several large
optimisation potentials available, the two most notable ones are:

- Distinguish the runtime representation of lambdas and closures
  explicitly.

- Detect and handle multiple-arity functions directly in the compiler.

However, for both of these we should wait until we have appropriate
benchmarking infrastructure in place. This is because our test
implementations have shown that the complexity of either of these
changes is quite significant, and we do not yet know if they really
pay off.

Change-Id: I077e977810fd5cb2b1ecd7f1a119e728025dd786
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6295
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
0f06d0ca33 feat(tvix/eval): implement OpGetUpvalue in the VM
This resolves an upvalue at runtime by pushing it on the stack from
the closure's upvalue vector.

Change-Id: Ic3e7a7ecd9f7032f679114a1995e5bbf83062fcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6294
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
3d8888a13e refactor(tvix/eval): store Closure inside of the VM's call frames
In preparation for implementing calling of closures, store a closure
directly in the VMs call frame.

Change-Id: Iad24cd8c49fee4ebd4d0c84ffaa4c2505ee3dfd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6293
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
5ecf573dff feat(tvix/eval): add Value::to_closure
... same as the others

Change-Id: I9c8868388c10b0b6484c5bdd3799d801296c6979
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6292
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
69d8f17a26 feat(tvix/eval): compile creation of closure objects
Fully implements the instructions for compiling closure
objects (without runtime handling yet).

Closure (and thunk) objects are created at runtime by capturing all
known upvalues. To represent this, the instructions for creating them
need to have a variable number of arguments. Due to that, this commit
introduces new variants in OpCode that are not actually operations,
but data.

If the VM is implemented correctly, the instruction pointer should
never point at these. Due to this, the VM will panic if it sees a data
operand during an execution run.

Change-Id: Ic56b49b3a42736dc437751e76df0e89c8d0619c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6291
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
e4fadfaaf8 chore(tvix/eval): print node representation for compiler errors
Better for development flow.

Change-Id: I038efb39caca804f28a44fd4c83457e90abbcee4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6290
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
1163ef3e41 feat(tvix/eval): implement compilation of upvalue access
This adds a new upvalue tracking structure in the compiler to resolve
upvalues and track their positions within a function when compiling a
closure.

The compiler will emit runtime upvalue access instructions after this
commit, but the creation of the runtime closure object etc. is not yet
wired up.

Change-Id: Ib0c2c25f686bfd45f797c528753068858e3a770d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6289
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-03 21:55:04 +00:00
Vincent Ambo
2f93ed297e refactor(tvix/eval): add opcode::Count type for less ambiguity
Change-Id: Ibde0b2baa1128a74c1364ee9a6330b62db3da699
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6288
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:27:40 +00:00
Vincent Ambo
c73e84d957 refactor(tvix/eval): add opcode::StackIdx type for less ambiguity
Change-Id: I9b9de1f681972c205d4d20bc5731d2ce79858edb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6287
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:22:42 +00:00
Vincent Ambo
2cdc6192b4 docs(tvix/eval): add docstrings for usize wrappers in opcode
Change-Id: I11b9324233c0aa48bd2fbac15a484962f925e72e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6283
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:22:42 +00:00
Vincent Ambo
4f00f75e31 refactor(tvix/eval): add opcode::JumpOffset type for less ambiguity
This adds a transparent wrapper around `usize` used for jump offsets
in the opcodes. This is a step towards getting rid of ambiguous plain
`usize` usage in the opcode.

Change-Id: I21e35e67d94b32d68251908b96c7f62b6f56a8bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6282
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:22:42 +00:00
Vincent Ambo
3b64b7eb2e refactor(tvix/eval): rename CompilationResult -> CompilationOutput
grfn pointed out in cl/6174 that `Result` might cause developers to
believe that this behaves like std::Result, which it does not.

Change-Id: Ia30ab0dcb7e8da7bf842777ee3fe17bcf35cb0c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6281
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:19:48 +00:00
Vincent Ambo
7d34e6e880 refactor(tvix/eval): explicitly discard uninteresting results
Change-Id: I95a2ad61d9512b91017c801f325d0193b4da9c7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6280
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:19:48 +00:00
Vincent Ambo
3a2fcc8bc2 refactor(tvix/eval): avoid cloning in NixAttrs::update if possible
Refactors the update function to take the attribute sets by value
instead.

To facilitate this, we use an equivalent of the currently unstable
`Rc::clone_or_unwrap` in the VM when encountering attribute sets, so
that in cases where the only references to the attrs being updated are
the ones on the stack those clones are avoided completely.

This does make update() a little bit more tricky internally, as some
optimised branches can directly return the moved value, and others
need to destructure with ownership. For this reason there are now two
different match statements handling the different ownership cases.

Change-Id: Ia77d3ba5c86afb75b9f1f51758bda61729ba5aab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6279
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 13:19:48 +00:00
Griffin Smith
3270817b90 test(tvix/eval): Add attr merge benchmarks
Add a quick couple of benchmarks for merging attribute sets, large and
small.

Change-Id: I26940a9cf4e0d30e3d9eb07a7b8c366ca4072ca3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6286
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: grfn <grfn@gws.fyi>
2022-09-03 06:56:02 +00:00
Vincent Ambo
48b0fac76b refactor(tvix/eval): slightly more readable AttrsRep::select
Suggestion from grfn in cl/6158.

Change-Id: I16dcf2296a5ec5d299d5a080ca099b8eda6c254e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6278
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
7da5076191 refactor(tvix/eval): rename Value::NotFound & OpAttrOrNotFound
grfn suggested clearer naming for these in cl/6166.

Change-Id: I83164bf1d1902ebd42272e9d5d63819a0f6a72c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6277
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
903b57be04 docs(tvix/eval): add doc comment on compiler::patch_jump
Change-Id: Ifdd7b99223d239d955ac7eeeae95db97eb742bf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6276
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
bc9351f811 refactor(tvix/eval): get rid of Value::Blackhole variant
This is no longer needed for anything and the extra clone here is not
really more costly than constructing a blackhole value in a different
place.

Change-Id: I5c63085b1b4418b629ea58a42e3bfe9a9b586d76
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6275
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
d761d934ed test(tvix/eval): add a test for float representation
Change-Id: I4893a37719b9bf08b35963d48e6851a194a08aa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6274
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:49:45 +00:00
Vincent Ambo
8d7ccc78d0 docs(tvix/eval): add a note on how to run Nix tests
Change-Id: I9cd61ac79ed11b4c6580f31c5af5ebbfd45054b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6273
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
edee8eecdf fix(tvix/eval): address all current clippy lints
Change-Id: I758fc4f3b9078de7ca6228a75a4351c3e085c4cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6272
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
39b01c3029 fix(tvix/eval): correctly escape ${ in strings
Without this escape, it is possible for Nix to produce escaped
representations which are not literal Nix values again.

This was fixed in upstream Nix in
https://github.com/NixOS/nix/pull/4012 (though only for eval, not in
the REPL) and the updated test is picked from upstream after that commit.

Because we run the C++ Nix tests against our test suite as well, this
also bumps our custom Nix 2.3 to a commit that includes the
cherry-picked fix from the PR above.

Change-Id: I478547ade65f655c606ec46f7143932064192283
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6271
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
265393301e refactor(tvix/eval): move resolve_local to Scope struct
This is a more sensible place for this function to live and makes
upvalue resolution easier down the line.

Change-Id: I48ee39bdcdb4f96a16a327f7015aff60db5b15fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6270
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Vincent Ambo
6ce2c666c3 refactor(tvix/eval): introduce Closure struct in Value type
This struct will carry the upvalue machinery in addition to the lambda
itself. For now, all lambdas are wrapped in closures (though
technically analysis of the environment can later remove innermost
Closure wrapper, but this optimisation may not be worth it).

Change-Id: If2b68549ec1ea4ab838fdc47a2181c694ac937f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6269
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-03 00:47:58 +00:00
Griffin Smith
af9dca3663 chore(tvix/eval): Build with --all-targets
Primarily to make sure we build benchmark targets, and avoid breaking
them

Change-Id: I0c43f4cf99ddfd38e7545ef2d8276ef6b240a1e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6285
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-02 19:10:45 +00:00
Griffin Smith
29f8f06319 fix(tvix/eval): Fix build of benchmarks
Interpret was updated to take an optional path arg in
6fe5e2d75 (feat(tvix/eval): resolve relative path literals, 2022-08-12),
but since benchmarks aren't building in CI the resulting breakage of
benchmarks was missed.

Change-Id: I8a93f1b25ae62e2d032fafc153d91977c6466712
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6284
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-02 19:10:34 +00:00
Vincent Ambo
b29ab1776b chore(tvix/eval): move compiler module to a new folder
Change-Id: I76157f9cf1369cd17506de1b1ded1a4fd06f004a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6268
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 15:45:43 +00:00
Vincent Ambo
1416f1ab8a refactor(tvix/eval): avoid a use of Value::Blackhole
The blackhole allocation is not going to be cheaper than cloning this.

Change-Id: Id3ad44812decb4392830be06645e67bb0a982b96
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6267
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 15:45:43 +00:00
Vincent Ambo
babc249316 refactor(tvix/eval): separate out let inherit ... logic
Compilation of `let`-expressions is going to become a lot more
complicated due to attempts to avoid thunking when encountering
internal references, so this is just being moved out of the way.

Change-Id: Iecfa4b13d14532e21c2540e6561b4235ce29736a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6266
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 15:45:43 +00:00
Vincent Ambo
b8f36ba097 chore(tvix/eval): print slightly more information about warnings
This is just for dev comfort, it's not going to be useful for the
final version.

Change-Id: I05fdd590097a61085ed641810655d9ddaf8f3511
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6265
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-02 14:13:00 +00:00
Vincent Ambo
2023b8e33f fix(tvix/eval): consider let ... inherit ... in dynamic scopes
In conditions where no dynamic identifiers exist in a scope,
inheriting is usually a no-op - *unless* the identifier is not
statically known and the scope has a non-empty `with`-stack.

Change-Id: Iff4138d9cd4c56e844bc574203708dacc11c3f73
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6264
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-02 14:13:00 +00:00
Vincent Ambo
c3b13416b0 refactor(tvix/eval): add NixAttrs::contains function
This avoids copying around the value more than needed.

Change-Id: I35949d16dad7fb8f76e0f641eaccf48322144777
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6263
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 13:36:48 +00:00
Vincent Ambo
86b21f9c33 feat(tvix/eval): implement builtins.catAttrs
Change-Id: Idf92ac82438fbfcf7b2f6e058830e4744637d8c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6262
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
aeea772b1c feat(tvix/eval): implement builtins.typeOf
Change-Id: Ibc5039b444fadf6f9e5cd9132fcd825a871cee06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6261
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
5b98ece96b feat(tvix/eval): implement type-checking builtins
Change-Id: I70d7d837beaaed7e10cdc7577d96130f9e1b6d39
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6260
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
bd0fc69f07 feat(tvix/eval): implement 'throw' and 'abort' builtins
These do essentially the same, but return different error variants as
upstream Nix considers `throw` to be (sometimes) catchable.

Change-Id: I1a9ea84567d46fb37287dbf3f3f67052f9382cca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6259
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
3d0280ec03 refactor(tvix/eval): implement clearer mechanism for globals
The set of things that can leak out of `builtins` into the global
scope is statically known (it is what Nix 2.3 leaks there,
essentially).

This is a mild change over the previous mechanism, where instead at
the point where the `builtins` set is constructed we "lift" the
globals out of there (if they exist).

This way users will still eventually be able to add additional
builtins, HOWEVER they will not be able to leak them into the global
scope.

Note that upstream Nix technically leaks _all_ builtins into the
global scope using the `__*` prefix, but we are trying to avoid this
in Tvix if it is not required in nixpkgs.

Change-Id: Ie9dec2ce33740134f3b2464eba3749f421dd5953
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6258
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
07fcaf034b feat(tvix/eval): emit warnings when globals are being shadowed
Change-Id: I7dae6978c2a4548382d7fa059b20ccdf35d2cf7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6257
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
b41960c55e feat(tvix/eval): add builtins.isNull
Change-Id: Iae251d41b4ac6b77df56078a954ec3e33b7f9ccf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6256
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
b8ac62d5fc test(tvix/eval): add a simple test for builtins resolution
Change-Id: I91f54778b8a17f3448664c21308de656b4b04b3e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6255
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
e1147b57c7 feat(tvix/eval): introduce mechanism for defining builtins
Adds a new builtins module in which builtins can be constructed. The
functions in this module should return a correctly structured value to
be passed to the compiler's `globals`.

This is wired up all the way to the compiler with an example
`toString` builtin, available as a global. Note that this does not yet
actually behave like the real toString, which has some differences
from `Display`.

Change-Id: Ibb5f6fbe6207782fdf2434435567fc1bd80039a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6254
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
ca90c0f45a refactor(tvix/eval): handle scope poisoning & globals dynamically
Previously, the tokens that could poison a scope (`true`, `false`,
`null`) had individual fields in the scope to track whether or not
they were poisoned.

This commit sets up new machinery that instead tracks scope poisoning
dynamically using a HashMap, and which makes it possible to introduce
additional tokens to the top-level ("global") scope that are directly
resolved by the compiler by passing a map of runtime values to be
used.

With this solution, the compiler now contains all machinery required
for wiring up builtins resolution.

The set of builtins to be exposed at runtime must, however, be
constructed *outside* of the compiler and passed in. Everything is
prepared for this, but it is not yet wired up (so the only existing
builtins are the ones we already had before).

Note that this technically opens up an optimisation potential when
compiling selection operations, where the attribute set being selected
from is `builtins`. The compiler could directly resolve the builtins
and place the right values on the stack.

Change-Id: Ia7dad3c2a98703e7ea0c6ace1a722d57cc70a65c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6253
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
c48aa058a7 docs(tvix/eval): add an overview of all builtins in Nix
Change-Id: Ie187f3317046c6c9e59852d4a128f25ceed99309
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6252
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-02 12:59:23 +00:00
Vincent Ambo
e0f1356ae3 feat(tvix/eval): add initial representation of builtins
Builtins are represented as a Rust function pointer that accepts a
vector of arguments, which represents variable arity builtins.

Change-Id: Ibab7e662a646caf1172695d876d2f55e187c03dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6251
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-02 12:59:23 +00:00
Vincent Ambo
64746388e2 feat(tvix/eval): compile function applications
Change-Id: I1b9230601895a1f09ef1a8037201147020b85f36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6250
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 12:31:04 +00:00
Vincent Ambo
1239a85e23 feat(tvix/eval): implement opcode for function calls in VM
Nix functions always have a single argument and we do not yet make
efforts to optimise this in Tvix for known multi-argument functions
being directly applied.

For this reason, the call instruction is fairly simple and just calls
out to construct a new call frame.

Note that the logic for terminating the run loop has moved to the top
of the dispatch; this is because the loop run needs to be skipped if
the call frame for the current lambda has just been dropped.

Change-Id: I259bc07e19c1e55cd0a65207fa8105b23052b967
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6249
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-02 12:31:04 +00:00
Vincent Ambo
fc892b7a9d refactor(tvix/eval): add VM::call helper to set up call frames
Change-Id: Ia7ff572af90ae379b23bbd0f5215cd13a4dc0ab5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6248
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-02 12:31:04 +00:00
Vincent Ambo
269788086e feat(tvix/eval): compile lambda definitions
Compiles lambda definitions of the simple form (i.e. without formals
arguments) and emits them as constants like any other value.

This does not yet implement actually invoking these functions in the VM.

Change-Id: Ie1e0a13220b68c1728be229b875f0992e685c5ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6247
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
fc865eb157 refactor(tvix/eval): introduce LambdaCtx structure to compiler
This structure carries context about the lambda currently being
compiled (which may well be the top-level lambda of an input AST).

Using the indirection helpers in the compiler, things like the scope,
code and constants of the function being compiled are now taken from
the current lambda context instead.

Change-Id: If5f864d826c2e72855cee4b728ea1830e9b5ac06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6246
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
9d1451773b refactor(tvix/eval): add compiler accessor for current scope
Change-Id: I7488087d95c1b3fb7f70fc29af0d5b0d0a25a428
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6245
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
a060c8f467 refactor(tvix/eval): use call frame for top-level lambda
This wires up most of the machinery for executing different call
frames inside of the VM and stuffs the top-level lambda which the
compiler outputs in there, as well.

Change-Id: Ib6201b3e3be1af96a4d195f6eb147f452860ffc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6242
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
0b37b3f2a9 feat(tvix/eval): add call frame struct to VM
This is going to carry the data for a function invocation inside of
the VM.

Change-Id: I86664563a7e35697a64294acd37ffde037fbd32d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6241
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
6f31c895ff refactor(tvix/eval): return a lambda from the compiler
Changes the internal compiler plumbing to not just return a chunk of
code, but the same chunk wrapped inside of a lambda value.

This is one more step towards compiling runtime lambdas.

Change-Id: If0035f8e65a2970c5ae123fc068a2396e1d8fd72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6240
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
4715f9a3a0 refactor(tvix/eval): add accessor indirection helpers to compiler
With these indirections in place it becomes easier to change internals
of the compiler when introducing functions, which need the compiler to
be able to target different code chunks.

Change-Id: I4eb11572a93c140b1d059ba0a5af905756745d65
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6239
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:56:17 +00:00
Vincent Ambo
d57366a6c2 feat(tvix/eval): introduce initial Lambda type
Change-Id: Ifa9766f5ffeff99e926936bafd697e885e733b78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6238
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:41:12 +00:00
Vincent Ambo
3ed40b4eea feat(tvix/eval): emit warnings for unused local bindings
Change-Id: I6e876a8f4d062297abae812b14ed8ec17a502f2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6237
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:40:50 +00:00
Vincent Ambo
f7305eed47 refactor(tvix/eval): collect vector of errors in compiler
Instead of exiting the compiler at the first sight of an error,
skip any erroneous nodes and continue compiling, collecting more
errors along the way.

This paves the way for nicer error reporting in which multiple errors
can be reported at once, avoiding situations in which users are
hunting a fault error-by-error and possibly getting distracted by
less useful output.

Change-Id: I80c9a87272e33a31297167ae2eb2706a46adf15a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6236
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:40:50 +00:00
Vincent Ambo
2662376941 feat(tvix/eval): carry optional SyntaxNode in error type
This starts paving the way for nicer, source-code based error
reporting.

Right now the code paths in the VM do not emit annotated errors, as we
do not yet preserve that structure from the compiler. However, error
emitting code paths in the compiler have been amended to include known
nodes.

Change-Id: I1b74410ffd891c40cd913361bd73c4336ec8aa5b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6235
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-01 21:40:50 +00:00
Vincent Ambo
51be6542c9 refactor(tvix/eval): add helper for emitting compiler warnings
Change-Id: I2d98dbb7274d07985f64e7cc8944e316bf42e1bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6234
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
3bf487b98b chore(tvix/eval): bump rnix-parser to latest master
In this commit, the string interpolation parsing is identical to
nixpkgs which makes some of the upstream Nix tests for
interpolation-related weirdness pass.

Change-Id: I3a295cfdc404c32228a54846e6efd3c0dcee5842
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6233
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
5522ddfbf5 fix(tvix/eval): fix several string escapings
These were missing an additional level of escaping, silly oversight
caught by an upstream test.

Change-Id: I0312084475e4b88c83945614e9aa5b34c6bc3ec2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6232
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
813fb98470 refactor(tvix/eval): Upgrade to latest rnix-parser
Since the latest published version of rnix-parser on crates.io, the
crate has undergone major changes which are only available in the git
repository at the moment. This commit updates the compiler to this
newer version of rnix.

Most notably, the entire AST provided by rnix is now wrapped in the
AST type system. As a result of this traversal is much nicer in many
places, especially for things like nested attribute selection.

There are a handful of smaller features missing for full feature
parity with the previous version, especially handling of path
literals, but PRs for these already exist in rnix-parser.

Change-Id: Icde6d393067976549492b7d89c4cc49e5e575fc7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6231
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
a00e4730a5 feat(tvix/eval): implement assert operator
This implements `assert`, which evaluates an expression and aborts
evaluation if the value is not `true`.

At this point we should introduce eval-failed-* tests; probably
asserting against some representation of the error enum?

Change-Id: If54c8f616d89b829c1860a4835dde60a2cd70d7a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6230
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-01 17:41:22 +00:00
Vincent Ambo
16e9703f38 refactor(tvix/eval): use pretty_assertions for tests
This makes for much more readable output especially when long strings
are involved.

Change-Id: I43dd73a0480535d7181a760788c42883a9b083f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6229
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-01 12:50:17 +00:00
Vincent Ambo
f153a163a6 refactor(tvix/eval): improve naming for locals manipulator methods
`push_local`/`push_phantom` were worse names because they sound like
the value itself is being pushed, where in actuality it is just being
declared to the compiler.

Change-Id: Ibfda5c4c8e47d5d3262bfe005b0f1f84908a117e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6228
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-01 12:50:17 +00:00
Vincent Ambo
0f739cd944 feat(tvix/eval): implement scope poisoning for true/false/null
These tokens are optionally parsed as identifiers by Nix, which means
that within any scopes that resolve them the compiler needs to track
whether they have been overridden to know whether to emit the literal
instructions or resolve a variable.

This is implemented by a new concept of "scope poisoning", where the
compiler's scope structure tracks whether or not any builtin
identifiers have been overridden.

Change-Id: I3ab711146e229f843f6e1f0343385382ee0aecb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6227
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
0b51d63081 refactor(tvix/eval): simplify let ... in ... before recursion
While full recursion through thunking is not available, there are
actually incorrect behaviours introduced by declaring before
binding (example in the newly introduced test).

This commit simplifies the implementation to avoid this issue, and
also because I intend to explore a bit more how far we can get in non
left-to-right bindings *without* introducing thunks immediately.

Change-Id: I21fd3007ac3946570639772d7d624d70bd209958
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6226
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
75ba7c0120 test(tvix/eval): add basic tests for with expressions
Change-Id: I94664090e7a2b060dfbe21c1eeb859fb31e417b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6225
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
0257f89917 chore(tvix/eval): return parse errors out of eval::interpret
Change-Id: I14f25b9c85260c68be38abf07ed80121ead60c7b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6224
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:42:48 +00:00
Vincent Ambo
89f566ef57 feat(tvix/eval): emit instructions for dynamic var resolution
If an unknown variable is encountered and the with stack is not empty,
emit instructions for resolving the variable at runtime.

Change-Id: I752f4bd0025335744e4747364abd1bd34130374e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6223
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
59f50dc81a feat(tvix/eval): Implement OpResolveWith instruction
Change-Id: I4d2a69f28a6b6199b3ff48ef81135e7da9fe1c3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6222
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
911fb96eca feat(tvix/eval): add Value::as_attrs method
Change-Id: I2f39122ac85b67837335aab308d845907160e132
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6221
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
83e5f3b0d3 feat(tvix/eval): emit instructions to close with at scope end
Change-Id: I340b7a0964a4d4c5be58f46d00d2376ec5682517
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6220
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
19878c5fef feat(tvix/eval): implement OpPopWith
Change-Id: I752d9428a73f893741746e9d5b79e8d8d570bb81
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6219
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
5a497cdf1c feat(tvix/eval): implement with_stack in VM
Change-Id: I805c24c9a751ded15725ba9be651aa0869e013e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6218
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-31 22:26:11 +00:00
Vincent Ambo
7cfdedfdfb feat(tvix/eval): compile with expression
Adds an additional structure to the compiler's scope to track the
runtime "with stack", i.e. the stack of values through which
identifiers should be dynamically resolved within a with-scope.

When encountering a `with` expression, the value from which the
bindings should be resolved is pushed onto the stack and tracked by
the compiler in the "with stack", as well as with a "phantom value"
which indicates that the stack contains an additional slot which is
not available to users via identifiers.

Runtime handling of this is not yet implemented.

Change-Id: I5e96fb55b6378e8e2a59c20c8518caa6df83da1c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6217
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-31 22:26:11 +00:00
Vincent Ambo
ec7db0235f refactor(tvix/eval): rename Locals to Scope
This name is much more sensible actually; its more than just a
collection of locals as it tracks additional scope information in the
case of Nix.

Change-Id: Ia2739bbd39aab222b1c4355e9248828973b0db43
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6216
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
f173161f4c feat(tvix/eval): implement inherit in let expressions
Note that at this point recursive bindings do not yet work in either
attrsets or let, so inheriting from the same scope is generally not
possible yet.

Change-Id: I6ca820d04b8ded5c22fb7ea18e2ec203bcaa8e9c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6215
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
8c1c9aee3c feat(tvix/eval): implement inherit in attribute set literals
Straightforward implementation, evaluating the elements of an inherit
and preparing the stack so that `OpAttrs` sees all relevant values
when constructing the attribute set itself.

The emitted instructions for inheriting a lot of values from the same
attribute set are inefficient, but it's too early to say whether this
actually matters.

Change-Id: Icb55a20936d4ef77173f34433811c5fa5d2c9ecc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6214
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
7db4f8d774 fix(tvix/eval): gently attempt to create state dir
If the directory in which REPL history is stored does not exist,
gently try to create it, but do not raise an error if it doesn't work.

We may want to warn about it, but in general this sort of
non-essential feature should not cause a hard failure.

Change-Id: If4fe8db0c7893c39627efe72c9cd9ebf7ed63f04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6213
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-31 22:10:40 +00:00
Vincent Ambo
e07556493f feat(tvix/eval): Implement inherit from outer scope in attrs
Change-Id: I97fa45059b7f2cbe96eb60bd1821e3e25832df66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6212
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
3a67f91228 fix(tvix/eval): ? operator should use OpAttrsOrNotFound
Using `OpAttrSelect`, the ? operator will fail when encountering a
nested value that is not an attribute set.

This however breaks valid code, such as:

  { bs = 42; } ? bs.a.b

The fix is simply to use the same operator used in the `or` statement,
which leaves a sentinal on the stack if a field is not found or the
value is not an attribute set.

Change-Id: Ib28fc8a96e6d592b4cdbc3e65ba129ad8faecd66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6211
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
cfe37d36f7 fix(tvix/eval): or should handle non-attrset values, too
If a nested attrpath encounters a non-set value, the sentinel value
denoting a lack of next values should be emitted. This mirrors the
behaviour of Nix.

Change-Id: Ia80443d5a11243cc6d98dcab1249a3f5fdf77e27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6210
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
76846fe220 fix(tvix/eval): allow use of ? operator on non-set types
Nix allows this, but always returns false. Tvix needs to do the same.

Change-Id: Ic9eec90834a0d0969eea5316d5c25032d3691d94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6209
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
43658a5b90 fix(tvix/eval): emit correct count in OpAttrPath
Not sure how exactly this snuck in, but it caused some subtle
breakages in deeply nested attribute sets.

Change-Id: I8049ce912405d3750031f79cc8d86ff1c3c02c2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6208
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
8d45fbadea docs(tvix/eval): Use correct syntax for module doc comments
Change-Id: I35741856f34b86a538f226a8eaf8806edede60ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6207
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
11ea7b82d8 chore(tvix/eval): minor readability improvement in attrs
Change-Id: If9d9eaf60934e96ec4b41c57818afe0c2a99c862
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6206
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
de26894814 refactor(tvix/eval): remove Error::InvalidKeyType
We're confident that we're handling all branches that can reasonably
occur from valid AST, any other cases should be considered a critical
evaluator bug and panic rather than surfacing something that looks
like user error.

Change-Id: If96966eb32b8ff12fcaeb9ea3b0c8fc51b6abd11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6205
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-08-30 17:13:27 +00:00
Vincent Ambo
ab9407bded fix(tvix/eval): address various clippy lints
Change-Id: I3ea0f51475e80948adfeb5d1620c1f2665cc39bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6201
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-30 16:53:40 +00:00