Commit graph

534 commits

Author SHA1 Message Date
Vincent Ambo
c46025d520 fix(tvix/eval): support string identifiers in inherits
This updates rnix-parser to a version where inherits provide an
iterator over `ast::Attr` instead of `ast::Ident`, which mirrors the
behaviour of Nix (inherits can have (statically known) strings as
their identifiers).

This actually required some fairly significant code reshuffling in the
compiler, as there was an implicit assumption in many places that we
would have an `ast::Ident` node available when dealing with variable
access (which is then explicitly only not true in this case).

Change-Id: I12f1e786c0030c85107b1aa409bd49adb5465546
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6747
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 23:07:14 +00:00
sterni
ee0b89c402 test(tvix/eval): test word behavior in builtins.compareVersions
We delegate to Rust std's Ord trait, so we should at least do a sanity
check here.

Change-Id: I9596068f8ab3e51b79602de0bac79af6d558086e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6723
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-22 23:06:45 +00:00
William Carroll
890bbf9b1f feat(tvix/eval): Support builtins.lessThan
Extend and export the `cmp_op`, and this becomes trivial.

Change-Id: I9c93fa4db0f5a1fc8b56928ea144676f79247de1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6557
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 17:42:01 +00:00
Griffin Smith
9e16d70809 feat(tvix/eval): Sketch out impure builtins
Sketch out a new set of "impure" builtins, which supplement the existing
set of "pure" builtins but are gated behind a feature flag, which allows
them to be omitted by crates depending on tvix-eval that only want pure
evaluation, such as tvixbolt.

Change-Id: I2736017b5c9b4776bbba8758e108ec84887abd66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6655
Reviewed-by: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-22 01:36:44 +00:00
sterni
7bf1b4deea refactor(tvix/eval): drop empty Word version cmp rule
This was derived from

    else if (c1 == "" && n2) return true; // true implies c1 < n2

However, this has no effect since Word always looses out against Number
anyways and the `pre` rules are also unaffected by this change – since
this only affects comparison of an empty Word part with a Number.

Change-Id: Ia04e42ac726352b688c87674b0fdb355f06edbcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6722
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
7ea249969a test(tvix/eval): add more compareVersions test cases involving pre
This asserts the not-quite lexicographical property of the comparison.

Change-Id: Iad68081e4b3a7106513f479643de87065dc47739
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6721
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
367a6a13d4 test(tvix/eval): add more compareVersions cases from C++ Nix
Change-Id: I25e7e7a2c547d0874e1e949bf96e6e066b1075ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6705
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
083fc1dbe5 fix(tvix/eval): compare versions with an extra empty component
This is necessary because builtins.compareVersions compares versions in
a subtly not-quite-but-still-lexicographical way: `pre` for example can
have an effect if it is post-fixed: `2.3 < 2.3pre`. This is a violation
of the rule that in a lexicographical ordering, the longer string is
considered greater if they are otherwise equal. builtins.compareVersion
is comparing lexicographically though, if you do the following
transformation beforehand:

  2.3 --split--> [ "2" "3" ] --append--> [ "2" "3" "" ]
  2.3pre --split--> [ "2" "3" "pre" ] --append--> [ "2" "3" "pre" "" ]

Comparing the transformed version is then done lexicographically:

  2.3 < 2.3.0pre since [ "2" "3" "" ] < [ "2" "3" "0" "pre" ]

Here, the `pre` rule never comes into effect because no comparison on it
happens, instead we use the longer string rule of a lexicographical
comparison.

In the C++ codebase, the reason for this behavior is that the
iterator-esque construct they use always yields the empty string before
it exposes it has been fully consumed. This is probably intentional to
support the postfixed `pre` which is, for example, used by NixOS
versions (e.g. unstable post 22.05 is 22.11-pre). We replicate this
behavior using the `Chain` iterator in `VersionPartsIter::new_for_cmp`.

Change-Id: I021c69aa27b0b7deb949dffe50ed18b6de3a7b1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6720
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
9a8a6a33f9 fix(tvix/eval): implement C++ Nix version part comparison algorithm
This is based on the [relevant code] in C++ Nix. Our version has more
branches because the C++ one only checks if it is less than or not, so
can save handling a few cases. We on the other hand, can avoid calling
the algorithm twice. It'd be nice to implement proptests for this in the
future and to make sure that this weird little algorithm doesn't violate
the Ord laws.

[relevant code]: cd35bbbeef/src/libstore/names.cc (L81-L94)

Change-Id: I46642e6da5eac7c0883cdce860622cdba04cd12b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6719
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-21 14:23:46 +00:00
sterni
e9e06b8bae test(tvix/eval): add test cases for builtins.splitVersion
Change-Id: I60648f6c5cea34d19afb5ced3e98051201190a78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6711
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-21 13:08:07 +00:00
sterni
834fe833e5 fix(tvix/eval/versions): preserve the Number string exactly
This is relevant for builtins.splitVersion:

    nix-repl> builtins.splitVersion "unstable-2022-02-21"
    [ "unstable" "2022" "02" "21" ]

Change-Id: I0a0add178d95d5a82e112b41ed5f3ca5a19608f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6710
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-21 13:08:07 +00:00
sterni
6e6edcce6a fix(tvix/eval/versions): dash is a divider, not underscore
https: //github.com/NixOS/nix/blob/cd35bbbeef72375873e396b9ffed14a4638693a8/src/libstore/names.cc#L63
Change-Id: I1d5aba6060d11778f3b79089f4b27ef8849e4d37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6709
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-21 13:08:07 +00:00
sterni
0096939bf6 feat(tvix/eval): implement builtins.splitVersion
This was fairly easy, thanks to the work already done by Thomas Frank.
However, the implementation is suboptimal because we parse number parts
only to convert them back to strings afterwards. I've chosen to tackle
this problem in the future, since having an (inefficient) implementation
of splitVersion will be helpful for debugging the slight discrepancies
between C++ Nix and Tvix in the case of compareVersions.

Change-Id: Id6ed8eeb77663ff650c8c53ea952875b1fb7ea84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6688
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-21 13:08:07 +00:00
Vincent Ambo
489395448f feat(tvix/eval): track other type in NotCallable error kind
This makes for slightly nicer error messages if something isn't, well,
callable.

Change-Id: I821c8d7447b93aea9ccaaa52ed329de0cca4b18e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6718
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 23:48:57 +00:00
Vincent Ambo
8f2004d360 refactor(tvix/eval): add VM::call_value helper method
This makes it possible to call a callable value (builtin or
closure/lambda) directly, without unwrapping it first. This is needed
for pretty much all higher-order functions to work correctly.

This is mostly equivalent to the previous code in coerce_to_string for
calling `__toString`, except it expects the argument(s) to already be
placed on the stack.

Note that the span for the `NotCallable` error is not currently
guaranteed to make any sense, will experiment with this.

Change-Id: I821224368d438a28900858b343defc1817e46a0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6717
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 23:48:57 +00:00
Griffin Smith
f600aa5322 test(tvix/eval): Make proptests a bit smaller
Generate smaller recursive values for generated Values, and run fewer
cases for the attrs proptests which are particularly egregious.

Change-Id: Ia35c7c120270feaf045be1deb440c87ebb185c27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6716
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-20 23:41:58 +00:00
Vincent Ambo
876c477256 feat(tvix/eval): implement builtins.map
As we already have a VM passed to the builtins, we can simply execute
the provided closure/lambda in it for each value.

The primary annoyance with this is that we have to clone the upvalues
for each element, but we can try making this cheaper in the
future (it's also a general problem in the VM itself).

Change-Id: I5bcf56d58c509c0eb081e7cf52f6093216451ce4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6714
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 23:37:53 +00:00
Vincent Ambo
8207977648 chore(tvix/eval): clean up this one weird trick ...
... Rust developers don't want you to know about!

Change-Id: Ic43b670ac0982c726bfb8cd27cb57b17934e4b70
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6715
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2022-09-20 23:18:53 +00:00
Griffin Smith
ce9dfad6cb refactor(tvix/eval): Define a Compiler::new function
Change-Id: I6b9283d16447c83dd3978371d9a6ac1beb985926
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6657
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-20 21:19:41 +00:00
sterni
be18df1dab test(tvix/eval): assert flipped compareVersions behaves correctly
Change-Id: I7c239340de5c83a3124dc6a8ba0c70290466966d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6698
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-20 10:59:38 +00:00
sterni
7506983ce2 fix(tvix/eval): force condition of an assert
Change-Id: I3ad2234e8a8e4280e498c6d7af8ea0733ed4c7ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6699
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-20 10:59:38 +00:00
sterni
e04ccc9354 fix(tvix/eval): make sure to deref thunk in type predicate builtins
Previously we only matched the outer constructor after forcing which
would mean that we would always return `false` if the inspected value
was a thunk, regardless what value would be present inside.

Change-Id: I361ea6e855e23ef8e5b59098a50b9cd59253803f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6692
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-20 10:59:38 +00:00
sterni
cd280e0796 fix(tvix/eval): make sure to force before selecting in catAttrs
Previously, this would almost always crash because list items are
thunked more often nowadays and selecting from a thunk would fail. Also
we no longer pop from args, accessing it by index should avoid an
unnecessary clone here.

Change-Id: I4410c4c2e28cc255a2c7cf2a5322db3d2c556a0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6693
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-20 09:59:57 +00:00
sterni
151f4437bc refactor(tvix/eval/builtins): avoid unnecessary popping
Change-Id: I631b442e19e5c05455d705291c11037eae9ed9e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6694
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-20 09:44:48 +00:00
sterni
a01524823a refactor(tvix/eval): don't unnecessarily force in toString
Change-Id: I2bcc720ce7b6aa67ea5f145b1f2381a3ae833ac5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6691
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-20 09:42:57 +00:00
William Carroll
61cce49a8a feat(tvix/eval): Support builtins.getAttr
Support looking-up values from attrsets by their keys.

Change-Id: Ib37a472a511dab145f99ebc849879b3494e8e89f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6554
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-19 16:29:12 +00:00
William Carroll
059f4b964f feat(tvix/eval): Support builtins.hasAttr
See unit tests for examples :)

Change-Id: Ieec51d780a7762cc455ca03a9dc1648a0711924a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6553
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-19 16:18:10 +00:00
sterni
f358be4a1d fix(tvix/eval): move compareVersions test so cargo test finds them
And insert the missing newline the C++ Nix test script needs.

Change-Id: I04ddd7268f9caa1414fd23314c281bb7c1e854cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6689
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-19 09:58:14 +00:00
sterni
4bf38df09c fix(tvix/eval): fix flipped sign in builtins.compareVersions
Change-Id: I42fc0dc633fbd525eaa35689a41ec3717f4352a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6690
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 09:58:14 +00:00
sterni
b79e248959 refactor(tvix/eval): handle forcing in Builtin::apply
Instead of arity, we pass a array reference to Builtin::new that
describes how many arguments there are and which of them need to be
forced, eliminating the need to force manually.

Note that this change doesn't fix some of the instances where the the
Builtin doesn't consider that the value could be a Thunk.

Change-Id: Iadb58bb79886c30dc6b09dcf0ffad8abf28036a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6662
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 09:58:14 +00:00
William Carroll
fefa8c55c4 feat(tvix/eval): Support builtins.tail
TL;DR:
- support `builtins.tail`
- define `ErrorKind::TailEmptyList` and canonical error code
- support basic unit tests

Unsure whether or not the error should be a dedicated `ErrorKind`...

Change-Id: Iae90fda1bb21ce7bdb1aaa2aeb2b8c1e6dcb0f05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6545
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 00:57:11 +00:00
William Carroll
2fe18e4486 feat(tvix/eval): Support builtins.substring
Nix's `builtins.substring`:
- doesn't accept negative indexes for `beg` or `end`. Compare the error messages
  for:
  - `builtins.substring -3  5 "testing"`
  - `builtins.substring  3 -5 "testing"`
  - `builtins.substring -3 -5 "testing"`
- returns an empty string when `beg >= end`
- allows `end` to exceed the length of the input string

Change-Id: I83af7a099d81b6d537ebe5029d946c7031cb7113
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6555
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-19 00:51:10 +00:00
sterni
0c75ce2d3d chore(tvix/eval): add rustfmt to the shell
Change-Id: Ia049bf08feeb1a75f30df8fed7f88fcdf650521b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6663
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-18 23:19:54 +00:00
Griffin Smith
69cbcc1eda refactor(tvix/eval): Simplify forcing in builtins
Refactor the `force!` macro to a method on `Value` which returns a
smart-pointer-esque type, which simplifies the callsite and eliminates
rightward drift, especially for high-arity builtins.

Change-Id: I97a7837580accfb4bbd03b24f2acdbd38645efa5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6656
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-18 22:33:19 +00:00
Vincent Ambo
bcbe1603c8 fix(tvix/eval): ensure all thunks are forced in nested selects
Previously only the first one was guaranteed to be forced, but we need
to do this for all of them.

Fixes b/190

Change-Id: I76b5667dbfb2f3fde3587e7b91d268cbf32aca00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6645
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2022-09-18 22:16:04 +00:00
Griffin Smith
d42d8f3089 docs(tvix/eval): Document OpAttrs
Document the OpAttrs op, paying special attention to the (perhaps
confusing) behavior of taking the number of *pairs*, not the number
of *values*, which will be popped off the stack into the resulting attr
set.

Change-Id: I64df0290308ecae7a5c7e14ead37091d32701507
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6654
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:13:16 +00:00
Griffin Smith
e720545e5b refactor(tvix/eval): use Clap for arg+env parsing
Refactor the environment variable and argument parsing for the tvix repl
to use Clap instead of doing things ad-hoc, and thread through options
obtained from environment variables via explicit arguments rather than
obtaining them from the environment as they're needed. This makes adding
more flags more sustainable, and also makes the binary fully
self-documenting, including supported env vars, via `--help`.

Change-Id: Ib1f6a0cd20056e8c9196760ff755fa5729667760
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6653
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:08:43 +00:00
Griffin Smith
6f70f32513 fix(tvix/eval): Force thunks during equality comparison
Thunks might be encountered deep in equality comparison (eg nested
inside a list or attr-set), at which point we need to force them in
order to compare them for equality (or else we panic when trying to get
at their value).

Fixes: b/192
Change-Id: I912151085f8298f30d5214c7965251c9266443f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6652
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:08:43 +00:00
Griffin Smith
0b76ed5615 chore(tvix/eval): Pass in VM to nix_eq
Pass in, but ignore, a mutable reference to the VM to the `nix_eq`
functions, in preparation for using that VM to force thunks during
comparison.

Change-Id: I565435d8dfb33768f930fdb5a6b0fb1365d7e161
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6651
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:03:42 +00:00
Griffin Smith
915ff5ac2a refactor(tvix/eval): Don't (ab)use PartialEq for Nix equality
Using rust's PartialEq trait to implement Nix equality semantics is
reasonably fraught with peril, both because the actual laws are
different than what nix expects, and (more importantly) because certain
things actually require extra context to compare for equality (for
example, thunks need to be forced). This converts the manual PartialEq
impl for Value (and all its descendants) to a *derived* PartialEq
impl (which requires a lot of extra PartialEq derives on miscellanious
other types within the codebase), and converts the previous
nix-semantics equality comparison into a new `nix_eq` method. This
returns an EvalResult, even though it can't currently return an error,
to allow it to fail when eg forcing thunks (which it will do soon).

Since the PartialEq impls for Value and NixAttrs are now quite boring,
this converts the generated proptests for those into handwritten ones
that cover `nix_eq` instead

Change-Id: If3da7171f88c22eda5b7a60030d8b00c3b76f672
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6650
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 22:03:41 +00:00
Vincent Ambo
0e5baae7ad refactor(tvix/eval): clone the Arc<codemap::File> for the compiler
This disconnects ownership of the `File` reference in a compiler from
the calling scope, which is required for when we implement `import`.

`import` will need to carry an `Rc<RefCell<CodeMap>>` (or maybe, in
the future, Arc) to give us the ability to add new detected code
files at runtime.

Note that the choice of `Arc` over `Rc` here is not ours - it's the
codemap crate's.

Change-Id: I3aeca4ffc167acbd1701846a332d93550b56ba7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6630
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-18 21:12:50 +00:00
Griffin Smith
5dd5c7e254 test(tvix/eval): Add Eq-laws tests for NixAttrs
As before, this limits the cases to a relatively small number because
otherwise things get quite large.

Change-Id: I5371dc56418fca52e1dd1d905b20868f647091ba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6649
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:57:09 +00:00
Griffin Smith
c7faba0c8e test(tvix/eval): Add tests for the Eq laws of Value
Only running 20 cases for now, since Value can get quite big if you let
it run for a while.

Change-Id: I09ef19da22c789c4869793836c98937c44595340
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6648
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:57:09 +00:00
Griffin Smith
96d869923a fix(tvix/value): Properly match on Path for PartialEq
this was found by proptests!

Change-Id: I16d6a6ece3b20cdddd6f78c94cc87befb1b651e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6647
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:55:08 +00:00
Griffin Smith
78d19ff3e9 test(tvix/eval): Allow passing ProptestConfig to generated props
Add an optional config argument to the `<trait>_laws` macros, to allow
configuring the generated tests with a ProptestConfig struct (to limit
the number of cases run)

Change-Id: I2143ddb72c6a870e8be4a9058135b6f9a703039e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6646
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:55:07 +00:00
Griffin Smith
f8b3806720 test(tvix/eval): impl Arbitrary for Value
Impl Arbitrary for Value (and NixAttrs and NixList) in the same way we
did for NixString. Value currently only generates non-"internal"
values (no thunks, AttrNotFound, etc.) and can't generate
functions (builtins or closures), because those'd require full
generation of tvix bytecode, which is a bit more work than I'd like to
do now - there's a `todo!` left in the code for a place where we could
allow opting-in to internal values and functions later.

Change-Id: I07a59e2b1d89cfaa912d4ecebd642caf4ddb040a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6627
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:55:07 +00:00
Griffin Smith
221d3b9485 test(tvix/eval): Add proof-of-concept test for Chunk
This is pretty boring at the moment, but mostly serves as a foot in the
door in the direction of writing more tests

Change-Id: Id88eb4ec7e53ebb2d5b5c254c8f45ff750238811
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6637
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 17:55:06 +00:00
Griffin Smith
d431aa7743 fix(tvix/eval): Emit errors for invalid integers
Invalid integers (eg integers that're too long) end up as error returns
on the `.value()` returned from the literal in the AST - previously we'd
unwrap this error, causing it to panic the compiler, but now we've got a
nice error variant for it (which just unwraps the underlying
std::num::ParseIntError).

Change-Id: I50c3c5ba89407d86659e20d8991b9658415f39a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6635
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-18 16:40:14 +00:00
Vincent Ambo
b3cc4c8c2c chore(tvix/eval): bump rnix-parser to latest HEAD
This bumps rnix-parser to a commit that should be unaffected by the
Nix >= 2.4 bug that prevents it from cloning repositories with filters.

Change-Id: Ie01da95245ec6740fa889eb710819e512202f665
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6634
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-18 16:24:40 +00:00
sterni
a798efc77b docs(tvix/eval): note C++ implementation details for C++ Nix
Just want to note those down somewhere before I forget them again,
we can delete them later if we have it all figured out.

Change-Id: Icafa2d8fc7ca39e38e9637b7eca6f2bbf487c2b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6632
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-18 15:01:34 +00:00
Griffin Smith
3935c34401 test(tvix/eval): Add proptests covering trait impls for String
Add a suite of proptests covering the laws of the handwritten stdlib
trait impls (Eq, Ord, and Hash) for String, generated from a new set of
macros for generating those tests which can be applied to other types.

Change-Id: Ib3276c9e96fca497aece094e5612707d3dc77ccd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6626
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-17 19:51:16 +00:00
Griffin Smith
51f1924f19 test(tvix/eval): Test StringRepr::Smol as well
The From<String> impl for NixString only generates StringRepr::Heap
strings, but we want to make sure we're testing StringRepr::Smol too

Change-Id: I6d04b9cf12ef8462fe2788e0c6414b165f40311d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6629
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-17 19:51:15 +00:00
Griffin Smith
e6fe7b1687 test(tvix/eval): impl Arbitrary for NixString
Change-Id: I3fe2d410c789429493a1278d571ca8fe74c2a69d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6625
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-17 19:04:19 +00:00
Griffin Smith
84def7ef19 chore(tvix/eval): Add deps on proptest+test_strategy
It's proptest time!

Add deps on the proptest and test_strategy crates, gated behind a
new (default-enabled) "arbitrary" feature flag so that they don't affect
dependencies of things like tvixbolt that depend on tvix.

These are going in dependencies, not dev-dependencies, so that we can
impl Arbitrary for stuff outside of test modules (which will be
important for integration suites which want to run proptests)

Change-Id: I1613bd3ea9a835e22986ad4e59700e8736007963
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6624
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-17 19:02:59 +00:00
Vincent Ambo
da9e885e06 refactor(tvix/eval): rename OpAttrsIsSet -> OpHasAttr
This matches the name of the AST node from which it was compiled.
Suggested by sterni in cl/6231

Change-Id: Ia51525158d2f47467c01fce2282005b1a8417a47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6623
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-17 18:58:42 +00:00
Vincent Ambo
337d626f00 refactor(tvix/eval): clean up implementation of compile_literal
Suggested by sterni in cl/6231

Change-Id: I58bbc8a922d360ea79a4dacb76cf8aa1fad93757
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6622
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 18:58:42 +00:00
Vincent Ambo
2f91543a42 test(tvix/eval): add a test for lazy evaluation of with namespace
Suggested by grfn in cl/6222

Change-Id: I8413af6404786bd43e5cb462a307cc236c4fb9ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6621
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 18:58:42 +00:00
sterni
65dea58ef8 chore(tvix/eval): add simple .envrc / shell.nix
Change-Id: I10a44b92391a8ac4d7ec5e4b878953c215f59908
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6620
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-17 17:31:08 +00:00
Griffin Smith
67e796b2e1 test(tvix/eval): Add the start of a nix oracle test suite
Add the start of a test suite that compares tvix eval results against
nix, using the string repr of the value as the comparison. This shells
out to a nix-instantiate binary, which is configurable as an environment
variable, to eval - there's some extra machinery there to setup a new
nix store as a tempdir to allow running this test inside the nix build
for tvix-eval itself.

Currently this has a macro that'll allow writing lots and lots of
hardcoded tests, but going forward I'm also going to be looking into
adding proptest-based generation of expressions to compare.

Change-Id: I9f4895fab1e668ed2b7dfd6f92f8c80de1bbb16b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6307
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-17 17:23:55 +00:00
ThoFrank
ef80d00b06 feat(tvix_eval): Support builtins.compareVersions
Added an Iterator over &str wich yields the VersionParts.

Change-Id: I8043d423127446a173d01d290aab10de0c24a6fc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6619
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-17 14:48:58 +00:00
Vincent Ambo
1ee0f670b9 refactor(tvix/eval): use new ToSpan trait wherever possible
... it would be nice if we could thread it through to the `Scope`
stuff (declaring phantoms & locals).

Change-Id: Id3b84e79032b8fbb12138b719e657565355fbc79
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6616
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 14:11:19 +00:00
Vincent Ambo
337d670d0c feat(tvix/eval): introduce ToSpan trait in compiler module
This trait can be used to convert most structures from rnix-parser
into a codemap::Span. It uses a macro to implement the trait for the
various expression types in the rnix AST, as Rust's silly semantic
versioning restriction stops us from doing a blanket implementation.

This will be used in the next commit to clean up the span handling in
the compiler a bit.

Change-Id: I0a437034e5fa203b5a49c6f25c45932a9f3b2bca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6615
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 14:11:19 +00:00
William Carroll
3a67a14009 feat(tvix/eval): Support builtins.elemAt
(Attempt to) index into a list.

Change-Id: I3592e60a79e64d265e34100d4062041b0b410e00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6551
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-17 12:54:14 +00:00
sterni
53fbc75df9 refactor(tvix/eval): fix current clippy lints
Change-Id: I88482453a62955515a0dcc0b243351b2bbac5236
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6618
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-16 22:58:24 +00:00
Vincent Ambo
0532cb6172 test(tvix/eval): add tests for legacy let syntax
Change-Id: I42d6240056c93cf90986ef3fec88b42f804b06ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6613
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:13:26 +00:00
Vincent Ambo
15c8e8c459 test(tvix/eval): add some tests for recursive attribute sets
Change-Id: I6a4fbe6d5aa6d2ce223bf77330d6c0f512d4300d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6612
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:13:26 +00:00
Vincent Ambo
e86acd3212 feat(tvix/eval): implement legacy let syntax
... and emit a warning if anyone decides to use.

Change-Id: Iaa6fe9fa932340e6d0fa9f357155e78823702576
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6611
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-16 18:13:26 +00:00
Vincent Ambo
20230e1e2d feat(tvix/eval): implement recursive attribute sets
Yep.

This is kind of ugly right now. The idea is that the recursive_scope
compilation function is used for recursive sets as well by emitting
the keys. The rest of the logic is pretty much identical.

There is quite a lot of code here that can be cleaned up (duplication
between attrs and let, duplication inside of the recursive scope
compilation function etc.), but I'd like to get it working first and
then make it nice.

Note that nested keys are *not* supported yet.

Change-Id: I45c7cdd5f0e1d35fd94797093904740af3a97134
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6610
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:12:13 +00:00
Vincent Ambo
a7e280ec00 refactor(tvix/eval): introduce a type for tracking bindings
This type is used in the list temporarily populated by the *second*
pass over all identifiers in a recursive scope. This first pass only
serves to make all bindings known to the compiler, without populating
their values yet.

Having a type here is going to be useful once we implement `rec`,
which needs to thread through slightly more information.

Change-Id: Ie33e0f096c5fcb6c864c991255466748b6f0d1eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6609
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:12:13 +00:00
Vincent Ambo
1cf07051cb refactor(tvix/eval): extract recursive scope logic into a helper
This needs to be reused between let & `rec` attrs.

Change-Id: I4a3bb90af4be32771b0f9e405c19370e105c0fef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6608
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:02:33 +00:00
Vincent Ambo
01a239c955 refactor(tvix/eval): move compile_inherit_attrs to compiler::attrs
Plain move, no other changes.

Change-Id: Ic4f89709f5c2cbc03182a848af080c820e39a0fd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6607
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:02:33 +00:00
Vincent Ambo
6cc9d298aa refactor(tvix/eval): explicitly construct attrs in phases
This makes the phases of attribute set construction that Nix has very
explicit (inherits, static keys, dynamic keys).

This change focuses on the split between dynamic/static keys by
collecting all dynamic ones while compiling the static ones, and then
phasing them in afterwards. It's possible we also need to do some
additional splitting inside of the inherits.

Change-Id: Icae782e2a5c106e3ce0831dda47ed81c923c0a42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6530
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-16 18:02:33 +00:00
William Carroll
f859751c6a feat(tvix/eval): Support builtins.bitXor
Bitwise "exclusive-or" on integers.

Change-Id: I90a0a15afb3a58662d70e82ea14e48b877476e04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6550
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:53:10 +00:00
William Carroll
031ca653a3 feat(tvix/eval): Support builtins.bitOr
Bitwise `or` on integers.

Change-Id: I45b0897331d1a9b6840f9d0feedcf10acc67fcec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6549
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:47:09 +00:00
William Carroll
f5f6bd319d feat(tvix/eval): Support builtins.bitAnd
Bitwise `and` on integers.

Change-Id: I9f2a9182a057af26906683acd97a40dfabbdded8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6548
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 21:41:08 +00:00
William Carroll
5a6d9ee1a0 feat(tvix/eval): Support builtins.attrValues
:)

Change-Id: Idf170b506ed6fab035b1e6f61055fee02e5c9be8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6547
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-15 21:13:03 +00:00
sterni
067f2b16f6 feat(tvix/eval): implement Value::coerce_to_path()
This function is necessary for all builtins that expect some form of
path as an argument. It is merely a wrapper around coerce_to_string that
can shortcut if we already have a path. The absolute path check is done
in the same way as in C++ Nix for compatibility, although it should
probably be revised in the long term (think about Windows, for example).

Since coercing to a path is not an operation possible in the language
directly, this function can live in the builtins module as the only
place it is required.

Change-Id: I69ed5455c00d193fea88b8fa83e28907a761cab5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6574
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 20:58:00 +00:00
William Carroll
e834a2cbc4 feat(tvix/eval): Support builtins.attrNames
Define `.len()` method on `NixAttrs` to preallocate the capacity of the result
vector.

Also anchor an errant comment to its context (I think).

Change-Id: I268f15025d453d7b3ae1146558c80e51433dd2a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6546
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-09-15 20:37:57 +00:00
William Carroll
85b3f17007 feat(tvix/eval): Support builtins.head
TL;DR:
- support `builtins.head`
- define `ErrorKind::IndexOutOfBounds` and canonical error code
- support basic unit tests

Change-Id: I859107ffb4e220cba1be8c2ac41d1913dcca37ff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6544
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 16:48:24 +00:00
sterni
0595870341 refactor(tvix/eval): don't move parts Vec in compile_str_parts
This allows us to get rid of the count local variable which was a bit
confusing. Calling parts.len() multiple times is fine, since the length
doesn't need to be computed.

Change-Id: I4f626729ad1bf23a93cb701385c3f4b50c57456d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6584
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 15:52:53 +00:00
sterni
4eb33e82ff fix(tvix/eval): coerce string interpolation parts to string
With this puzzle piece of string compilation in place, `compile_str`
becomes less redundant, as every part now needs to be compiled the same.
The thunking logic becomes a bit trickier, since we need to thunk even
in the case of `count == 1` if the single part is interpolating.
Splitting the inner (shared) code in a separate function turned out to
be easier for making rustc content.

Change-Id: I6a554ca599926ae5907d7acffce349c9616f568f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6582
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 15:52:53 +00:00
sterni
bcd7e520f0 fix(tvix/eval): thunk string interpolation
If we have multiple string parts, we need to thunk assembling the
string. If we have a single literal, it is strict (like all literals),
but a single interpolation part may compile to a thunk, depending on how
the expression inside is compiled – we can avoid forcing to early here
compared to the previous behavior.

Note that this CL retains the bug that `"${x}"` is erroneously
translated to `x`, implying e.g. `"${12}" == 12`.

The use of `parts.len()` is unproblematic, since normalized_parts()
builds a `Vec` instead of returning an iterator.

Change-Id: I3aecbfefef65cc627b1b8a65be27cbaeada3582b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6580
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 15:52:53 +00:00
sterni
da1d71a4e8 feat(tvix/eval): implement correct toString behavior
Implement C++ Nix's `EvalState::coerceToString` minus some of the Path
/ store handling. This is currently only used for `toString` which does
all possible coercions, but we've already prepared the weaker coercion
variant which is e.g. used for builtins that expect string arguments.

`EvalState::coerceToPath` is still missing for builtins that need a
path, but it'll be easy to build on top of this.

Change-Id: I78d15576b18921791d04b6b1e964b951fdef22c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6571
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-15 11:47:30 +00:00
sterni
162e21f2bb fix(tvix/eval): force left argument of ? before checking for attrs
OpAttrsIsSet and OpAttrsTrySelect will fail silently if the attribute
set value on the stack is actually a thunk, so we need to make sure to
force at every step of the way.

Emitting the force instructions in the compiler because it is easier to
add, but maybe the VM should do this when handling the relevant opcodes?
Comments welcome.

Change-Id: I65c5ef348d59b2d07c9bb06abb24f9f3e6a0fdb2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6540
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-13 22:06:29 +00:00
sterni
7e625afc59 fix(tvix/eval): force value in builtins.typeOf
This prevents Nix programs to observe the "internal" type of thunks.
Possibly .type_of() is also an area of the runtime where we should panic
if "internal" would ever be returned.

Change-Id: I9f358044c48ad64896fb6a1b1a42f00a29efac00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6539
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-13 22:06:29 +00:00
sterni
6d53fb6c52 fix(tvix/eval): force exprs inside string interpolation
The expression inside ${…} may return arbitrary values, including
thunks, so we need to make sure to force them just in case.

Change-Id: Ic11ba00c4c92a10a83becd91233db5f57f6e59c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6541
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-13 22:06:28 +00:00
Vincent Ambo
f69e83ae7b refactor(tvix/eval): cover all Value variants in force_for_output
Avoids accidentally dropping one on the floor if we add more, pointed
out by sterni in cl/6372

Change-Id: Ib7bb0ce9c8331c8337003d20c4d5240dfae1c32a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6570
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
3e188d4f8e fix(tvix/eval): add branch for directly comparing two thunks
Pointed out by sterni in cl/6370

Change-Id: I324d8049a2702ced8f30ad43a64d63ae79dd0eab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6569
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
6fdf55a975 fix(tvix/eval): use fragment span for OpAttrsSelect
Pointed out by sterni in cl/6389

Change-Id: I648056a760266a8cfd7adcdc478c7ff2132991f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6568
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
477015cfe3 refactor(tvix/eval): point OpPushWith span at namespace
Pointed out by sterni in cl/6395

Change-Id: I2dda2bb11fef702df05fd7a4fd93b9e717a85dad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6567
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
62623ef46c refactor(tvix/eval): point OpAssert span at condition
This is more useful than pointing it at the entire assert expression,
as that includes the body as well which is not going to be relevant in
the error.

Pointed out by sterni in cl/6391

Change-Id: I95a5d1edf90df65e7fa53d4d04502afd6e99e89a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6566
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
30de72fefb chore(tvix/eval): do not inherit scope depth in new scopes
This is no longer required, resolution is now more sane. Pointed out
by sterni in cl/6422.

Change-Id: Icc8983c648f864e66813948df6e2d4bad6a7f312
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6565
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
c28ecbee97 refactor(tvix/eval): encapsulate scope_depth in compiler::scope
This field no longer needs to be directly accessible by the compiler.

Addresses a sterni lint from cl/6466

Change-Id: I5e6791943d7f0ab3d9b7a30bb1654c4a6a435b1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6564
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
268605140e fix(tvix/eval): address current clippy lints
Change-Id: I5288849d0e93511b0b5664fa92f1c6882e4a1356
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6563
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
d5ee893fb1 refactor(tvix/eval): use CodeIdx wrapper for instruction pointer
As suggested by sterni in cl/6453.

Change-Id: I3cf80d97c11fd7d085ab510f6be4b5f937c791ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6562
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
0f59fe6601 feat(tvix/eval): implement initial fancy formatting for errors
This very closely follows the way it's done for warnings, but errors
have a lot more information available in some cases which we do not
surface yet.

Note also that due to requiring the `CodeMap`, this is not yet called
from eval.rs as the way that is threaded through needs to be
refactored, so only the method for reporting these errors as strings
is implemented so far.

Next steps for this will be to add a generic diagnostics module that
reduces some of the boilerplate for this between warnings & errors,
and which will also give us a good point in the future to switch to a
fancier diagnostics crate.

Change-Id: If6bb209f8e7a568d866e516a90335b9b2afbf66d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6534
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-13 14:41:57 +00:00
Vincent Ambo
4f67cf221a feat(tvix/eval): implement initial fancy display for warnings
This implements an initial fancy display for warnings emitted by the
tvix compiler, using the codemap_diagnostic crate.

Each warning variant has an associated message, and optionally an
associated annotation for the span displayed to the user.

In theory we could get a lot more fancy with the display for specific
variants if needed (e.g. re-parse the AST and actually add multiple
semantic spans based on context), but this is already a good start.

Example:

  tvix-repl> let toString = https://tvl.fyi; in let inherit toString; in ({}: 42) rec {}
  warning[W004]: declared variable 'toString' shadows a built-in global!
   --> [tvix-repl]:1:5
    |
  1 | let toString = https://tvl.fyi; in let inherit toString; in ({}: 42) rec {}
    |     ^^^^^^^^ variable declared here

  warning[W001]: URL literal syntax is deprecated, use a quoted string instead
   --> [tvix-repl]:1:16
    |
  1 | let toString = https://tvl.fyi; in let inherit toString; in ({}: 42) rec {}
    |                ^^^^^^^^^^^^^^^

  warning[W002]: inherited variable already exists with the same value
   --> [tvix-repl]:1:40
    |
  1 | let toString = https://tvl.fyi; in let inherit toString; in ({}: 42) rec {}
    |                                        ^^^^^^^^^^^^^^^^^

  warning[W999]: feature not yet implemented in tvix: recursive attribute sets
   --> [tvix-repl]:1:70
    |
  1 | let toString = https://tvl.fyi; in let inherit toString; in ({}: 42) rec {}
    |                                                                      ^^^^^^

  warning[W999]: feature not yet implemented in tvix: closed formals
   --> [tvix-repl]:1:62
    |
  1 | let toString = https://tvl.fyi; in let inherit toString; in ({}: 42) rec {}
    |                                                              ^^

  warning[W003]: variable 'toString' is declared, but never used:
   --> [tvix-repl]:1:5
    |
  1 | let toString = https://tvl.fyi; in let inherit toString; in ({}: 42) rec {}
    |     ^^^^^^^^ variable declared here

  => 42 :: int

These are coloured when output to a terminal.

Change-Id: If315648a07e333895db4ae1d0915ee2013806585
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6532
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-09-13 11:12:29 +00:00
Vincent Ambo
dea6d9b73b chore(tvix/eval): add dependency on codemap-diagnostic
This is a crate for source-span based error reporting. Since all of
our spans are already codemap spans, it is a good starting point.

We have to figure out quite a bit of logic for neat error printing;
later on if we want fancier presentation we might want to look at one
of the other libraries in this space like miette.

Change-Id: I4e28886af1ed199b7112d9dbf063c9f29b612bf1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6531
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-09-13 10:54:14 +00:00
Vincent Ambo
2fe193ce41 chore(tvix/eval): address current clippy lints
Change-Id: I76326c20a525044e89d3cd1392a29faa3414ca04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6529
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-11 21:13:24 +00:00
Vincent Ambo
1844c788f5 refactor(tvix/eval): remove todo!() calls in compiler
It is impossible for tvixbolt to recover from panics, so the user
experience of typing an expression using an unsupported feature was
that it would get sad and stop responding to input.

Instead, raise a normal value-level error of a new variant and
continue where possible.

Change-Id: Ibe016c92cacb87b85095c0f83758eddc6468053e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6528
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-11 21:13:24 +00:00