Commit graph

17275 commits

Author SHA1 Message Date
Vincent Ambo
f8919dbcd6 fix(tvix/eval): address current clippy lints
Note that I've allowed `needless_lifetimes` for the attribute set
iterator, as I find the type easier to understand with these
annotations present.

Change-Id: I33abb17837ee4813076cdb9a87f54bac4a37044e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6373
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
afcfa78364 feat(tvix/eval): semi-strictly evaluate output values of the VM
This essentially makes the VM behave like `nix-instantiate --eval
--strict`, i.e. data structures are traversed strictly and thunks are
forced. Thunks embedded in closures are not forced.

This allows us to re-enable tests that were disabled because they
needed to output nested thunk contents, but is overall a behaviour
that must be configurable later on, as it is not cmopatible with e.g.
an evaluation of nixpkgs.

Change-Id: I5303a5c8e4322feab1384fdb7712fecb950afca5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6372
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
6c6f9ae661 feat(tvix/eval): implement NixList::iter
This does not require a custom iterator type (for now?)

Change-Id: I5beb194bd8629571bd4040c69c977c27149807fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6371
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
fea7f90a7f fix(tvix/eval): thread Display & PartialEq through to thunk values
If a thunk is already evaluated, there are cases where due to the
memoisation implementation something might observe a value wrapped in
a thunk.

In these cases, the implementation of `Display` and `PartialEq` must
delegate to the underlying value.

Note that there are a handful of other cases like these which we need
to cover.

It is a little tricky to write integration tests for these directly,
especially as some of the open-upvalue optimisations coming down the
pipe will reduce the number of observable thunks.

One test that covers a part of this behaviour is currently
disabled (needs some more machinery), but it's being brought back in
the next commits.

Change-Id: Iaa8cd338c12236af844bbc99d8cec2205f0d0095
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6370
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-07 19:08:56 +00:00
Vincent Ambo
38d3db5fb8 feat(tvix/eval): implement NixAttrs::iter()
Implementing iteration over NixAttrs requires a custom iterator type
in order to encapsulate the different representations. The BTreeMap
for example has its own iterator type which needs to be encapsulated.

This is mostly boilerplate code, but for a change some simple unit
tests have been added in.

Change-Id: Ie13b063241d461b810876f95f53878388e918ef2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6367
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
7d5dca7ba3 chore(tvix/eval): provide 'static references to "name"/"value"
These static strings show up a bunch when dealing with the internals
of attribute sets, and having them available as static references is
required.

Due to the way const expressions are evaluated, taking a reference to
the existing NixString::NAME / NixString::VALUE items does not work
and the references themselves need to be const-evaluated.

Change-Id: If6e75847af978118a3b266fe6a3242321722434d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6366
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
80713f207e refactor(tvix/eval): encapsulate all thunk-forcing logic in module
The VM previously took care of repeatedly forcing a thunk until it
reached an evaluated state. This logic is now encapsulated inside of
the `Thunk::force` implementation.

In addition, force no longer returns a reference to the value by
default, leaving it up to callers to decide whether they want to
borrow the value or not (a helper is provided for this).

Change-Id: I2aa7da922058ad1c57fbf8bfc7785aab7971c02b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6365
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
23a5caabec feat(tvix/eval): construct attribute sets lazily
This thunks the construction of attribute sets. Because Tvix does not
currently have a "strict output" mode, a test had to be disabled that
now displays a thunk representation.

The test will be re-enabled once that is available.

Change-Id: I360332be64cd5c154f9caea21828f6f1b37a265c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6363
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
727845645d chore(tvix/eval): add release-with-debug cargo profile
This lets us create a release build with debug info, for use with e.g.
perf + hotspot

Change-Id: I03897de36c872d318abf1332ca0c1aeabe344ec6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6362
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
79a2ba5175 fix(tvix/eval): avoid recomputing width in disassemble_op constantly
As noticed by sterni in cl/6195

Change-Id: Ie9c1e80e2e709284fa8412334af9188d999f64dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6361
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
a8b2ba07df fix(tvix/eval): correctly resolve identifiers in inherit
This should not have grown a second implementation of the identifier
resolution logic, but it somehow did.

This implementation ended up being incorrect because it did not
account for upvalues inside of thunks.

Change-Id: Ieb1364d8fe43c96aaf4b125fd4b8a522aedff167
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6360
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
17dfb92a9f chore(tvix/eval): implement Debug for compiler::scope::Scope
Change-Id: I112b0119bd0511f26bb72f7e73d867d1b7144a36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6359
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
9420a3b53d feat(tvix/eval): insert strictness points for attribute set keys
All attribute set *key* related operations strictly evaluate all key
fragments, including during construction of an attribute set.

Change-Id: I3519e5e9b0886c2cdc8615ea7dcb5f7be0c59b3f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6358
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
5b4f3811e6 feat(tvix/eval): insert strictness points for unary/binary operators
The arguments of all unary/binary operators that are built in to Nix
are forced when encountered. This emits the necessary OpForce operations.

Change-Id: I691fcdbebfe7586cfe217c68d44b10b1192f82d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6357
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
6a42d9bddf chore(tvix/eval): thread slot value through all compiler methods
With this change any compilation of an expression is aware of its own
stack slot if it is leaving identifiers on the stack.

Change-Id: I0c9f148ae06b078a46b25180c4961686d5f2e166
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6356
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
50c33989dc chore(tvix/eval): thread slot value through Compiler:compile_attr
This threads the current local slot through the `compile_attr`
function and all of its callers. At the moment this does not improve
any user-facing behaviour, just internally changes the way in which
some correct expressions would fail to run.

Eventually this slot will need to reach everywhere ...

Change-Id: Iba73123dd1ced421093d8fc18ebeeffc16efacf8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6355
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
372bf4a0b7 test(tvix/eval): test very simple late-bound identifier access
This is the simplest kind of thunk that can be created (and so far the
only one the compiler knows how to create), in which an identifier
inside a `let` encounters a value that is bound *after* it is
initialised.

Change-Id: I6ea4408a3baef1e7d5137365d70804283f2dbf8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6354
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
786e12a738 feat(tvix/eval): always emit OpForce as the last instruction
Change-Id: Id70c987f654dc5d9b47db74e395281309762b468
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6353
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
Vincent Ambo
9a783e50a4 feat(tvix/eval): implement OpForce in VM
This operation forces the evaluation of a thunk.

There is some potential here for making an implementation that avoids
some copies, but the thunk machinery is tricky to get right so the
first priority is to make sure it is correct by keeping the
implementation simple.

Change-Id: Ib381455b02f42ded717faff63f55afed4c8fb7e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6352
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 15:25:59 +00:00
William Carroll
db9cb70d5d feat(wpcarro/rust): Define Rc<T> example
My foray into "smart pointer" land.

Change-Id: I4ca775c72168dd34d90bf88fa41149867cd7fdae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6244
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
2022-09-06 18:09:30 +00:00
William Carroll
e23a556873 feat(wpcarro/emacs): Drop racer.el support
AFAIU `racer` is deprecated in favor of LSP.

Change-Id: I8bee3bb32d208b4f21cd7699ae93fc76a69ec964
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6051
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
2022-09-06 17:48:23 +00:00
William Carroll
7b8e30bdf3 chore(overlays/tvl): Remove stale overlays
As the comments explain:

> Workaround for srcOnly with separateDebugInfo until
> https://github.com/NixOS/nixpkgs/pull/179170 is merged.

> upgrade home-manager until the service-generation fix has landed upstream
> https://github.com/nix-community/home-manager/issues/2846

> Outdated test-data, see https://github.com/pganssle/zoneinfo/pull/115
> Can be dropped when https://github.com/NixOS/nixpkgs/pull/170450 lands.

All of the linked issues appear merged (or closed with a link to a merge PR),
so -- barring any failing Buildkite builds -- these may be safe to remove.

Change-Id: I85e9f3ba2ca970a11754a85cb9574705144bfdfa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6162
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-06 17:43:29 +00:00
William Carroll
fd68fe043c fix(users/wpcarro): Fix Hadrian path
This looks like a copy-paste error.

Change-Id: I21594412075d361d824074b7612c8dbafb4f9f30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6369
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
2022-09-06 17:32:21 +00:00
William Carroll
bc60a7e454 feat(wpcarro/emacs): Set initial-buffer-choice per-host
Work machines should prefer `/hadrian` and personal machines should prefer
`/depot`.

Change-Id: I03b605cc5ce233767146d83f14f368863897cf8b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6368
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
2022-09-06 17:29:20 +00:00
Vincent Ambo
a8c13a5ce1 fix(tvix/eval): allocate Thunk::upvalues with known capacity
The capacity (i.e. number of builtins) is known from the lambda, so we
can size it correctly right away.

Change-Id: Iab0b5a3f47d450fa9866c091ebbbed935b934907
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6351
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:54:42 +00:00
Vincent Ambo
6c895d4b28 feat(tvix/eval): implement OpThunk for runtime thunk construction
Implements an operation very similar to `OpClosure` which populates a
thunk's upvalues and leaves it on the stack.

Change-Id: I753b4dfeeaae6919316c7028ec361aaa13d87646
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6350
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:54:42 +00:00
sterni
ffbb05e738 chore(3p/sources): Bump channels & overlays
Change-Id: I3a5cca3ac327d6437c8b47fcd496ad4d969b958c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6364
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-06 16:38:14 +00:00
Vincent Ambo
6c4b9bdce9 refactor(tvix/eval): extract VM::populate_upvalues function
This function is reusable between thunks & closures.

Change-Id: I44d5f9897b087a385c8e75027d2ff39c48a096f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6349
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 16:36:46 +00:00
Vincent Ambo
ce498052e6 refactor(tvix/eval): Carry lambda & upvalues directly in CallFrame
CallFrame has to work for both thunks & closures (as a thunk is
basically a "weird 0-argument closure").

We opt to store the common, relevant fields directly in the frame to
avoid having to dereference through the nested structures
constantly (which would be especially annoying in the case of thunks).

Change-Id: I47781597b84ec5cd55502dba1713e92cf2592af3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6348
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
25c62dd0ef refactor(tvix/eval): introduce UpvalueCarrier trait
This trait abstracts over the commonalities of upvalue handling
between closures and thunks.

It allows the VM to simplify the code used for setting up upvalues,
without duplicating between the two different types.

Note that this does not yet refactor the VM code to optimally make use
of this.

Change-Id: If8de5181f26ae1fa00d554f1ae6ea473ee4b6070
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6347
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
8033a7abae refactor(tvix/eval): simplify thunk representations
For now, do not distinguish between closing and non-closing thunks, it
will make the initial implementation easier. See Knuth etc.

Change-Id: I0bd51e0f89f2c77e90bac63b507e5027b649e3d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6346
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
28a9847c65 feat(tvix/eval): emit thunks for recursive local scope resolution
When resolving a value on the same level that is known but not yet
defined, emit a thunk.

Consider for example:

    let
      #       v--- requires a thunk
      a = 1 * b;
      b = 10;
    in a

Change-Id: I922cb50973ebe05e335a7bc7cb851960cf34733b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6345
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
d1798444be feat(tvix/eval): Add Compiler::thunk method for emitting thunks
The logic in this method is *very* similar to `compile_lambda`. It is
intended to be called around any expression that should be
thunked (such as function applications, attribute set values, etc.).

Change-Id: Idfbb2daa9f4b735095378fb9c39a2fd07c8cff91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6344
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
1f37275cfb feat(tvix/eval): introduce Value::Thunk variant
Introduces the representation of runtime thunks, that is lazily
evaluated values. Their representation is very similar to closures.

Change-Id: I24d1ab7947c070ae72ca6260a7bbe6198bc8c7c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6343
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
49296cebe3 refactor(tvix/eval): extract compiler's upvalue logic into helper
This exact same logic is reused for thunk creation.

Change-Id: I731db9cc659a1f2ca87db55d58d6ff632f417812
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6342
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
70b27a6080 test(tvix/eval): add a test case for nested, deferred upvalues
This test case was previously broken by the bug introduced by
confusing local and stack indexes.

Change-Id: Ibef299dad266c6105deac1da5dde112fe9f640b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6341
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
24811d7655 fix(tvix/eval): distinguish statically between StackIdx and LocalIdx
Previously the functions in the scope module returned usize values,
which - sometimes from the same function - were either indexes into
the runtime stack *or* indexes into the compiler's local stack.

This is extremely confusing because it requires the caller to be aware
of the difference, and it actually caused subtle bugs.

To avoid this, there is now a new LocalIdx wrapper type which is used
by the scope module to return indexes into the compiler's stack, as
well as helpers for accounting for the differences between these
indexes and the runtime indexes.

Change-Id: I58f0b50ad94b28a304e3372fd9731b6590b3fdb8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6340
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
68aad6d4cf feat(tvix/eval): emit OpFinalise when local scopes are complete
With this change, the runtime can correctly capture deferred upvalues.

Change-Id: I1e43b7b1ac2553b1812424adfc8bd08ef77bf1ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6339
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
5c4e102ac8 feat(tvix/eval): track whether locals needs to be finalised
When encountering a deferred local upvalue, the compiler will now mark
the corresponding local as needing a finaliser which makes it possible
to emit the OpFinalise instruction for this stack slot a little bit
down the line.

Change-Id: I3962066f10fc6c6e1472722b8bdb415a811e0740
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6338
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
025a9a4a0a feat(tvix/eval): implement OpFinalise instruction
This instruction finalises the initialisation of deferred upvalues in
closures (and soon, thunks).

The compiler does not yet emit this instruction, some more accounting
is needed for that.

Change-Id: Ic4181b26e19779e206f51e17388559400da5f93a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6337
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
1bfe32f412 feat(tvix/eval): set up deferred upvalues at runtime
This will leave a sentinel value in the upvalue slot in which the
actual value is to be captured after resolution once a scope is fully
set up.

Change-Id: I12b37b0dc8d32603b03e675c3bd039468e70b354
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6336
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 14:58:52 +00:00
Vincent Ambo
6c1948a71a feat(tvix/eval): detect deferred upvalue capturing
Uses the threaded through slot offset to determine whether
initialisation of a captured local upvalue must be defered to a later
point where all values of a scope are available.

This adds a new data representation to the opcode for this situation,
but the equivalent runtime handling is not yet implemented. This is in
part because there is more compiler machinery needed to find the
resolution point.

Change-Id: Ifd0c393f76abfe6e2d91483faf0f58947ab1dedc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6329
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-06 14:58:52 +00:00
Vincent Ambo
203d9f2e3e chore(tvix/eval): thread current slot index through compiler
While compiling local bindings, we now know the stack slot of the
value currently being compiled.

This will let us determine whether an upvalue can be captured directly
or whether it needs to wait for a synchronisation point at which the
upvalue can be instantiated.

This machinery lets us avoid unnecessary work at runtime when
instantiating closures that actually do not need complicated recursive
resolution.

This change itself introduces no new functionality, but since the
threading is noisy it is split out as a separate change.

Change-Id: I847c677ee8f6725fda1d2efd689b6a58bdccb779
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6328
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-06 14:58:52 +00:00
Vincent Ambo
9dd1fa875d refactor(tvix/eval): optimise initialisation of locals
Instead of looking up the local to be initialised by its name again,
we can simply track the index at which it was declared from the point
where the declaration was made.

This reduces some string cloning and removes unnecessary logic. It
also theoretically makes the *current* index available during locals
compilation, which can be used to optimise some recursion cases.

Change-Id: I06f403603d4f86c3d319debfe74b5a52eec00990
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6327
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
900a92935d refactor(tvix/eval): declare all locals before compiling them
This actually makes things full-circle, as this tree already had this
implementation once before all the other required components were in
place.

With this commit, the compiler can resolve recursive upvalues within
the same scope (though they will not yet work at runtime).

Change-Id: I6267e477d08f367257c3a6dde054b880d7b47211
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6326
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
47b3562867 refactor(tvix/eval): decouple local depth & initialisation tracking
In order to resolve recursive references correctly, these two can not
be initialised the same way as a potentially large number of (nested!)
locals can be declared without initialising their depth.

This would lead to issues with detecting things like shadowed
variables, so making both bits explicit is preferable.

Change-Id: I100cdf1724faa4a2b5a0748429841cf8ef206252
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6325
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
705c1b9d53 fix(tvix/eval): correct runtime error for missing dynamic upvalue
Change-Id: I75c351619780fdc5186a54f3df9b244ada984069
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6324
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
776120de05 fix(tvix/eval): instantiate *new* closures from blueprints each time
The previous closure refactoring introduced a bug in which the same
closure object would get mutated constantly for each instance of a
closure, which is incorrect behaviour.

This commit instead introduces an explicit new Value variant for the
internal "blueprint" that the compiler generates (essentially just the
lambda) and uses this variant to construct the closure at runtime.

If the blueprint ever leaks out to a user somehow that is a critical
bug and tvix-eval will panic.

As a ~treat~ test for this, the fibonacci function is being used as it
is a self-recursive closure (i.e. different instantiations of the same
"blueprint") getting called with different values and it's good to
have it around.

Change-Id: I485de675e9bb0c599ed7d5dc0f001eb34ab4c15f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6323
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
ccfb971dc5 fix(tvix/eval): correctly thread through dynamic upvalues
This puts together the puzzle pieces for threading dynamic
upvalues (that is, upvalues resolved from the `with`-stack) all the
way through.

Reading the test case enclosed in this commit and walking through it
is recommended to understand what problem is being tackled here.

In short, because the compiler can not statically know *which*
with-scope a dynamic argument is resolved from it needs to lay the
groundwork for resolving from *all* possible scopes.

There are multiple different approaches to doing this. The approach
chosen in this commit is that if a dynamic upvalue is detected, the
compiler will emit instructions to close over this dynamic value
in *all* enclosing lambda contexts.

It uses a new instruction for this that will leave around a sentinel
value in case an identifier could not be resolved, and wire the
location of this found value (or sentinel) up through the upvalues to
the next level of nesting.

In this tradeoff, tvix potentially closes over more upvalues than are
needed (but in practice, how often do people create *really* deep
`with`-stacks? and in *this* kind of code situation? maybe we should
even warn for this!) but avoids keeping the entire attribute sets
themselves around.

Looking at the test case, each surrounding closure will close
over *all* dynamic identifiers that are referenced later on visible to
it, but only the last one for each identifier will actually end up
being used.

This also covers our bases for an additional edge-case this creates,
in which an identifier potentially resolves to a dynamic upvalue *and*
to a dynamic value within the function's own scope (again, would
anyone really do this?) by introducing a resolution instruction for
that particular case.

There is likely some potential for cleaning up this code which is
quite ugly in some parts, but as this implementation is now carefully
calibrated to work I decided it is time to commit it and clean it up
in subsequent commits.

Change-Id: Ib701e3e6da39bd2c95938d1384036ff4f9fb3749
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6322
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00
Vincent Ambo
8982e16e26 refactor(tvix/eval): thread dynamic upvalues through all contexts
With this change, encountering a dynamic upvalue will thread through
all contexts starting from the lowest context that has a non-empty
`with`-stack.

The additional upvalues are not actually used yet, so the effective
behaviour remains mostly the same. This is done in preparation for an
upcoming change, which will implement proper dynamic resolution for
complex cases of nested dynamic upvalues.

Yes, this whole upvalue + dynamic values thing is a little bit
mind-bending, but we would like to not give up being able to resolve a
large chunk of the scoping behaviour statically.

Change-Id: Ia58cdd47d79212390a6503ef13cef46b6b3e19a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6321
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 07:45:43 +00:00