Commit graph

88 commits

Author SHA1 Message Date
Vincent Ambo
94f582341e refactor(tvix/eval): use internal SourceCode field in error printers
Makes use of the SourceCode field now being stored directly in
errors (see parent CL). With this change, the default `Display`
implementation can now format errors correctly, and there is no need
to keep a `SourceCode` around just for error formatting.

Updates dependent crates (CLI, serde, tvixbolt) to use this correctly.

Change-Id: Iddc5d7a6b4bab391f30a999e4c68aca34304c059
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10987
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-02-20 09:18:08 +00:00
Vincent Ambo
3c87687798 refactor(tvix/eval): add SourceCode directly into error types
With this change it's no longer necessary to track the SourceCode
struct separately from the evaluation for error reporting: It's just
stored directly in the errors.

This also ends up resolving an issue in compiler::bindings, where we
cloned the Arc containing file references way too often. In fact those
clones probably compensate for all additional SourceCode clones during
error construction now.

Change-Id: Ice93bf161e61f8ea3d48103435e20c53e6aa8c3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10986
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-02-20 09:18:08 +00:00
Aspen Smith
5d72d3980f refactor(tvix/eval): Box the strings in CatchableErrorKind
These strings are allocated once and never changed, so they don't need
the additional overhead of a capacity given by String - instead, we can
use Box<str> and save on 16 bytes for each of these, *and* for each
Value since this is currently the largest Value variant.

Change-Id: I3e5cb070fe6c5bf82114c92d04f6bae775663a7e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10796
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2024-02-10 20:35:28 +00:00
Aspen Smith
201173afac fix(tvix): Represent strings as byte arrays
C++ nix uses C-style zero-terminated char pointers to represent strings
internally - however, up to this point, tvix has used Rust `String` and
`str` for string values. Since those are required to be valid utf-8, we
haven't been able to properly represent all the string values that Nix
supports.

To fix that, this change converts the internal representation of the
NixString struct from `Box<str>` to `BString`, from the `bstr` crate -
this is a wrapper around a `Vec<u8>` with extra functions for treating
that byte vector as a "morally string-like" value, which is basically
exactly what we need.

Since this changes a pretty fundamental assumption about a pretty core
type, there are a *lot* of changes in a lot of places to make this work,
but I've tried to keep the general philosophy and intent of most of the
code in most places intact. Most notably, there's nothing that's been
done to make the derivation stuff in //tvix/glue work with non-utf8
strings everywhere, instead opting to just convert to String/str when
passing things into that - there *might* be something to be done there,
but I don't know what the rules should be and I don't want to figure
them out in this change.

To deal with OS-native paths in a way that also works in WASM for
tvixbolt, this also adds a dependency on the "os_str_bytes" crate.

Fixes: b/189
Fixes: b/337
Change-Id: I5e6eb29c62f47dd91af954f5e12bfc3d186f5526
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10200
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
2024-01-31 14:51:49 +00:00
Florian Klink
9cdb5e17a4 fix(tvix/eval): fix JSON error types
The error message is misleading. The errors we return can happen both
during serialization or deserialization, though the messages suggested
the latter only.

Change-Id: I2dafe17ec78ee75cab5937a3a81540fda3175eac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10603
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-01-12 09:22:37 +00:00
Ryan Lahfa
20c894e232 feat(tvix/glue): context-aware toFile
This removes the reference tracking and uses instead the context for
references and returns some.

Change-Id: Ic359ca6b903b63f1a9c679c566004c617b792442
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10435
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-01-03 19:43:10 +00:00
sterni
7165ebc43b fix(tvix/eval): remove incorrect imports when coercing
The default behavior of string coercion in C++ Nix is to weakly coerce
and import to store if necessary. There is a flag to make it strongly
coerce (coerceMore) and a flag that controls whether path values have
the corresponding file/directory imported into the store before
returning the (store) path as a string (copyToStore). We need to
implement our equivalent to the copyToStore (import_paths) flag for the
benefit of weak coercions that don't import into the store (dirOf,
baseNameOf, readFile, ...) and strong coercions that don't import into
the store (toString).

This makes coerce_to_string as well as CoercionKind weirder and more
versatile, but prevents us from reimplementing parts of the coercion
logic constantly as can be seen in the case of baseNameOf.

Note that it is not possible to test this properly in //tvix/eval tests
due to the lack of an appropriate EvalIO implementation being available.
Tests should be added to //tvix/glue down the line.

Change-Id: I8fb8ab99c7fe08e311d2ba1c36960746bf22f566
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10361
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-12-14 13:15:23 +00:00
Adam Joseph
8d4aa2c15c fix(tvix/eval): add unimplemented __curPos and builtins.filterSource
This commit adds __curPos (to the global scope, yuck) and
builtins.filterSource.  These are not implemented; forcing them will
produce the same result as `throw "message"`.

Unfortunately these two post-2.3 features are used throughout
nixpkgs.  Since an unresolved indentifier is a catchable error, this
breaks the entire release eval.  With this commit, it simply causes
those broken packages that use these features to appear as they are:
broken.

Change-Id: Ib43dea571f6a9fab4d54869349f80ee4ec5424c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10297
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 17:20:50 +00:00
Adam Joseph
05f42519b5 fix(tvix/eval): fix b/281 by adding Value::Catchable
This commit makes catchable errors a variant of Value.

The main downside of this approach is that we lose the ability to
use Rust's `?` syntax for propagating catchable errors.

Change-Id: Ibe89438d8a70dcec29e016df692b5bf88a5cad13
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9289
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2023-09-24 21:54:10 +00:00
Adam Joseph
926459ce69 refactor(tvix/eval): factor CatchableErrorKind out of ErrorKind
This commit creates a separate enum for "catchable" errors (the kind
that `builtins.tryEval` can detect).

Change-Id: Ie81d1112526d852255d9842f67045f88eab192af
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9287
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-09-24 21:54:09 +00:00
Linus Heckemann
5733274876 fix(tvix/eval): allow negative substring lengths
Nix uses string::substr without checking the sign of the length[1].
The NixOS testing infrastructure relies on this[2], and on the
implicit conversion of that to the maximum possible value for a
size_t.

[1]: ecae62020b/src/libexpr/primops.cc (L3597)
[2]: c7c2984716/nixos/lib/testing/driver.nix (L29)

Change-Id: I6d0caf6830b6bda3fdf44c40c81de6a1befeca7b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8746
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-06-12 08:38:49 +00:00
Vincent Ambo
8c13f18d11 feat(tvix/eval): report all known spans on infinite recursion
This reports the span

1. of the code within a thunk,
2. of the place where the thunk was instantiated,
3. of the place where the thunk was first forced,
4. of the place where the thunk was forced again,

when yielding an infinite recursion error, which hopefully makes it
easier to debug them.

The spans are tracked in the ThunkRepr::Blackhole variant when putting
a thunk under evaluation.

Note that we currently have some loss of span precision in the VM loop
when switching between frame types, so spans 3/4 are currently a bit
wonky. Working on it.

Change-Id: Icbd2a9df903d00e8c2545b3fc46dcd2a9e3e3e55
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8270
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-03-17 19:32:38 +00:00
Vincent Ambo
3fa6b13c1e feat(tvix/eval): track span of first force in a thunk blackhole
This is step 1 towards being able to use all 4 spans that we know when
dealing with infinite recursion. It tracks the span at which the
force of a thunk was first requested when constructing a blackhole, so
that we can highlight the spans of the first and second forces.

These are actually the least relevant spans, but the easiest to put in
place, more coming soon.

Change-Id: I4c7e82f6211b98756439d4148a4191457cc46807
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8269
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-03-17 19:31:37 +00:00
Vincent Ambo
5095e4f269 feat(tvix/eval): add generator "name" to NativeError kind
This produces traces in which we can see what kind of native code was
run. Note that these "names" are named after the generator message, so
these aren't *really* intended for end-user consumption, but we can
give them saner names later.

Example:
https://gist.github.com/tazjin/82b24e92ace8e821008954867ee05057

This already makes the traces a little easier to parse.

Change-Id: Idcd601baf84f492211b732ea0f04b377112e10d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8268
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-03-17 19:31:37 +00:00
Vincent Ambo
ea80e0d3f8 feat(tvix/eval): enrich errors with VM's frame stack information
When emitting an error at runtime, the VM will now use the new
`NativeError` and `BytecodeError` error kinds (which just wrap inner
errors) to create a set of diagnostics to emit.

The primary diagnostic is emitted last, with `error` type (so it will
be coloured red in terminals), the other ones will be emitted with
`note` type, highlighting the causal chain.

Example:
https://gist.github.com/tazjin/25feba7d211702453c9ebd5f8fd378e4

This is currently quite verbose, and we can cut down on this further,
but the purpose of this commit is to surface more information first of
all before worrying about the exact display.

Change-Id: I058104a178c37031c0db6b4b3e4f4170cf76087d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8266
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-17 19:31:37 +00:00
Vincent Ambo
939cebd0f1 fix(tvix/eval): implement cppnix JSON-serialisation semantics
This drops the usage of serde::Serialize, as the trait can not be used
to implement the correct semantics (function colouring!).

Instead, a manual JSON serialisation function is written which
correctly handles toString, outPath and other similar weirdnesses.

Unexpectedly, the eval-okay-tojson test from the C++ Nix test suite
now passes, too.

This fixes an issue where serialising data structures containing
derivations to JSON would fail.

Change-Id: I5c39e3d8356ee93a07eda481410f88610f6dd9f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8209
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-03-13 20:30:59 +00:00
Vincent Ambo
025c67bf4d refactor(tvix/eval): flatten call stack of VM using generators
Warning: This is probably the biggest refactor in tvix-eval history,
so far.

This replaces all instances of trampolines and recursion during
evaluation of the VM loop with generators. A generator is an
asynchronous function that can be suspended to yield a message (in our
case, vm::generators::GeneratorRequest) and receive a
response (vm::generators::GeneratorResponsee).

The `genawaiter` crate provides an interpreter for generators that can
drive their execution and lets us move control flow between the VM and
suspended generators.

To do this, massive changes have occured basically everywhere in the
code. On a high-level:

1. The VM is now organised around a frame stack. A frame is either a
   call frame (execution of Tvix bytecode) or a generator frame (a
   running or suspended generator).

   The VM has an outer loop that pops a frame off the frame stack, and
   then enters an inner loop either driving the execution of the
   bytecode or the execution of a generator.

   Both types of frames have several branches that can result in the
   frame re-enqueuing itself, and enqueuing some other work (in the
   form of a different frame) on top of itself. The VM will eventually
   resume the frame when everything "above" it has been suspended.

   In this way, the VM's new frame stack takes over much of the work
   that was previously achieved by recursion.

2. All methods previously taking a VM have been refactored into async
   functions that instead emit/receive generator messages for
   communication with the VM.

   Notably, this includes *all* builtins.

This has had some other effects:

- Some test have been removed or commented out, either because they
  tested code that was mostly already dead (nix_eq) or because they
  now require generator scaffolding which we do not have in place for
  tests (yet).

- Because generator functions are technically async (though no async
  IO is involved), we lose the ability to use much of the Rust
  standard library e.g. in builtins. This has led to many algorithms
  being unrolled into iterative versions instead of iterator
  combinations, and things like sorting had to be implemented from scratch.

- Many call sites that previously saw a `Result<..., ErrorKind>`
  bubble up now only see the result value, as the error handling is
  encapsulated within the generator loop.

  This reduces number of places inside of builtin implementations
  where error context can be attached to calls that can fail.
  Currently what we gain in this tradeoff is significantly more
  detailed span information (which we still need to bubble up, this
  commit does not change the error display).

  We'll need to do some analysis later of how useful the errors turn
  out to be and potentially introduce some methods for attaching
  context to a generator frame again.

This change is very difficult to do in stages, as it is very much an
"all or nothing" change that affects huge parts of the codebase. I've
tried to isolate changes that can be isolated into the parent CLs of
this one, but this change is still quite difficult to wrap one's mind
and I'm available to discuss it and explain things to any reviewer.

Fixes: b/238, b/237, b/251 and potentially others.
Change-Id: I39244163ff5bbecd169fe7b274df19262b515699
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8104
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2023-03-13 20:30:59 +00:00
Vincent Ambo
91a366af46 fix(tvix/eval): make fields of eval's Error type public
These should be inspectable by callers.

Change-Id: Ia9ef871aa63958d06066aaea61b2aecbd217369b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8089
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-02-13 16:21:46 +00:00
Florian Klink
e0b05c0fa6 feat(tvix/eval): implement builtins.fromTOML
This allows parsing TOML from Tvix. We can enable the eval-okay-fromTOML
testcase from nix_tests. It uses the `toml` crate, and the serde
integration it brings with it.

Change-Id: Ic6f95aacf2aeb890116629b409752deac49dd655
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7920
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-25 07:49:44 +00:00
Vincent Ambo
972c867b36 feat(tvix/eval): add error contexts to annotate error kinds
This makes it possible for users to add additional context to an
error, which will then be rendered as an additional secondary span in
the formatted error output.

We should strive to do this basically anywhere errors are raised that
can occur multiple times, *especially* during type casts. This was
triggered by me debugging a type cast error attached to a fairly
large-ish span (a builtin invocation).

Change-Id: I51be41fabee00cf04de973935daf34fe6424e76f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7849
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-20 15:39:51 +00:00
Vincent Ambo
67addf3b3d feat(tvix/eval): add error variant for threading through errors
This variant is required for external builtins (which in our case
includes `derivation`) to thread through reasonable error messages.

This has some potential for improvement, but it's an improvement over
the status quo of panicking in the external builtins when no
appropriate error is available.

Change-Id: I7e4bdb0a156c7717092dde30aa4785192182dc66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7841
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-16 13:43:14 +00:00
Vincent Ambo
d365b09226 feat(tvix/eval): implement builtins.toXML
Change-Id: I009efc53a8e98f0650ae660c4decd8216e8a06e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7835
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-16 13:43:14 +00:00
Vincent Ambo
02d35e4aa6 feat(tvix/eval): implement builtins.toJSON
Implements `Serialize` for `tvix_eval::Value`. Special care is taken
with serialisation of attribute sets, and forcing of thunks.

The tests should cover both cases well.

Change-Id: I9bb135bacf6f87bc6bd0bd88cef0a42308e6c335
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7803
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-01-12 10:42:44 +00:00
Vincent Ambo
c011a6130c refactor(tvix/eval): impl Display for ErrorKind
This just shuffles the Display implementations around so that
ErrorKind itself is displayable, which is useful in some situations
where errors under construction need to be type-converted.

Change-Id: I7b633d03d0dc34f345c4f20676e0023ecb1db0c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7802
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-01-10 09:52:52 +00:00
Vincent Ambo
49ee3e3b14 chore(tvix/eval): implement std::error::Error for tvix_eval::Error
This makes it easier to interface this error with other crates.

Change-Id: I4947ea6097608f8c0427fb94a819ef748d94ea4b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7711
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-02 22:24:43 +00:00
Vincent Ambo
71174f6626 fix(tvix/eval): fix current clippy warnings
It's been a while since the last time, so quite a lot of stuff has
accumulated here.

Change-Id: I0762827c197b30a917ff470fd8ae8f220f6ba247
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7597
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-25 18:25:06 +00:00
Vincent Ambo
0ef3c2fc2b refactor(tvix/eval): use EvalIO::read_to_string in impure builtins
With this change, the behaviour of reading a string from a file path
is controlled by the provided `EvalIO` structure.

This is a huge step towards abstracting away I/O behaviour correctly.

Change-Id: Ifde8e46cd863b16e0301dca45a434ad27560399f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7567
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 22:37:11 +00:00
Adam Joseph
7606e62a2f feat(tvix/eval): add CoercionKind::ThunksOnly
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: I92acb7e6099a4796d953b2d4d02cca4076ed0fb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7426
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-11-28 20:05:33 +00:00
jhahn
d00030128e feat(tvix/eval): detect division by zero
This detects if the second argument of a division is a zero (either as integer
or as float). If so, an error message is displayed.

This fixes b/219.

Change-Id: I50203d14a71482bc757832a2c8dee08eb7d35c49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7258
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2022-11-10 13:12:05 +00:00
Vincent Ambo
4ff06ba67d feat(tvix/eval): add TvixBug error kind
This can be raised in cases where some panics lurk (e.g. indexed
accesses).

Change-Id: I93f4d60568277e9c5635aa52f378645626d68c5e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7057
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-22 18:30:12 +00:00
sterni
64bb501de1 fix(tvix): distinguish search- and relative path resolution errors
Failures to resolve a nix search path lookup in angle brackets can be
caught using tryEval (if it reaches the runtime). Resolving relative
paths (either to the current directory or the current user's home) can
never be caught, even if they happen inside a thunk at runtime (which is
currently the case for home-relative paths).

Change-Id: I7f73221df66d82a381dd4063358906257826995a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7025
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-21 00:11:29 +00:00
Griffin Smith
2a3d498104 feat(tvix/eval): Validate closed formals
Validate "closed formals" (formal parameters without an ellipsis) via a
new ValidateClosedFormals op, which checks the arguments (in an attr set
at the top of the stack) against the formal parameters on the Lambda in
the current frame, and returns a new UnexpectedArgument error (including
the span of the formals themselves!!) if any arguments aren't allowed

Change-Id: Idcc47a59167a83be1832a6229f137d84e426c56c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7002
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-17 11:29:49 +00:00
Griffin Smith
5eb89be682 feat(tvix/eval): Implement builtins.fromJSON
Using `serde_json` for parsing JSON here, plus an `impl FromJSON for
Value`. The latter is primarily to stay "dependency light" for now -
likely going with an actual serde `Deserialize` impl in the future is
going to be way better as it allows saving significantly on intermediary
allocations.

Change-Id: I152a0448ff7c87cf7ebaac927c38912b99de1c18
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6920
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-15 20:35:22 +00:00
Adam Joseph
2e0a4aaa84 feat(tvix/eval): From<Utf8Error> for ErrorKind
Change-Id: I074d9e862fbdd4e78e443cbaf0e77c7ffe825a10
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6911
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 04:22:44 +00:00
Vincent Ambo
019ea51e5c fix(tvix/eval): format nested compiler errors in fancy output
Instead of just printing the number of errors (useless!) actually emit
separate diagnostics for each nested error.

Change-Id: I97b53c3276c906af5def89077b5b6ba6ec108b37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6933
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 22:07:22 +00:00
Griffin Smith
75f637c1f5 fix(tvix/eval): path resolution errors are catchable
Despite this not being documented, `tryEval` is empirically able to
catch errors caused by a <...> path not resolving (and nixpkgs depends
on this).

Change-Id: Ia3b78a2d9d2d0c603aba829518b351102dc55396
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6926
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-10 20:58:14 +00:00
Griffin Smith
4fd18cbc9a feat(tvix/eval): Implement builtins.tryEval
With asserts compiled using conditional jumps, this ends up being quite
straightforward - the only real tricky bit is that we have to know
whether an error can or can't be handled.

Change-Id: I75617da73b7a9c5cdd888c0e26ae81d2c5c0d714
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6924
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-10-10 20:31:10 +00:00
William Carroll
41ddc37725 feat(tvix/eval): Support builtins.readDir
Co-authored-by: Griffin Smith <root@gws.fyi>
Change-Id: I5ff19efbe87d8f571f22ab0480500505afa624c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6552
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-10 19:36:49 +00:00
Griffin Smith
d0f571dcc0 refactor(tvix/eval): Use Display impl for Error message
This is generally more idiomatic (over just delegating to Debug), and
also allows us to avoid intermediate allocations if we ever end up
using error messages as part of larger strings (because we don't have to
allocate a full String for the return value).

Change-Id: I67e48b44570c72761ed0fcaded9ae4bf3fcbaacf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6896
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-09 17:12:41 +00:00
Vincent Ambo
1677186144 feat(tvix/eval): add some slightly more descriptive span labels
Change-Id: I530c491f60a33fdb97e1553b193de51e7ee57d9a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6873
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 17:27:57 +00:00
Vincent Ambo
9b1a266197 feat(tvix/eval): chain error spans for thunk errors
Adds secondary spans for errors that occur deeply nested within a
thunk.

This is pretty raw right now, there's technically nothing stopping one
of these error chains from being a hundred thunks deep into code,
producing unmanageable error output. We should trim these down
according to some heuristics (e.g. when crossing file boundaries, o r
just - for starters - beginning and end).

Change-Id: Ia73892512737850b6fa3e07cabc37fa9c534c4d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6872
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 17:27:57 +00:00
Vincent Ambo
1e2d323a7c feat(tvix/eval): fancy-format parse errors returned by rnix
This change is quite verbose, so a little bit of explaining:

1. To correctly format parse errors, errors must be able to return
   more than one annotated span (the parser returns a list of errors
   for each span).

   To accomplish this, the structure of how the `Diagnostic` struct
   which formats an error is constructed has changed to delegate the
   creation of the `SpanLabel` vector to the kind of error.

2. The rnix structures don't have human-readable output formats by
   default, so some verbose methods for formatting them in
   human-readable ways have been added in the errors module. We might
   want to move these out into a submodule.

3. In many cases, the errors returned by rnix are a bit strange - so
   while we format them with all information that is easily available
   they may look weird or not necessarily help users. Consider this CL
   only a first step in the right direction.

Change-Id: Ie7dd74751af9e7ecb35d751f8b087aae5ae6e2e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6871
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-08 17:27:57 +00:00
Vincent Ambo
2478a6c5ba feat(tvix/eval): coerce values to paths when importing
This enables the use of string paths (and, in the future,
derivations), as long as their string values represent an absolute
path.

Change-Id: I4b198efeb70415ed52f58bd1da6fa79a24dad14c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6866
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-10-07 14:24:36 +00:00
Vincent Ambo
b530e496a5 feat(tvix/eval): initial implementation of builtins.import
This adds an initial working version of builtins.import which
encapsulates the entire functionality of `import` within the builtin
itself, without requiring any changes in the compiler or VM.

The key insight that enables this is that we can simply return a Thunk
from `import` that is constructed from the output of running the
compiler and - ta-da! - no other component needs to know about it.

A couple of notes:

* builtins.import needs to capture variables like the SourceCode
  structure. This means it can not currently be constructed the same
  way as other builtins and has special handling, which leaks out to
  `eval.rs`. I have postponed dealing with that until we have this
  working a bit more.

* the `globals` are not yet passed through

* the error representation for the new variants is absolutely not done
  yet, we probably want to switch to something that supports
  cause-chaining now (like miette)

* there is no mechanism for emitting warnings at runtime; we need to
  add that

Change-Id: I3117a7ae3ff2432bf44f5ff05ad35f47faca31d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6857
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-06 15:22:32 +00:00
Vincent Ambo
3530404a4a refactor(tvix/eval): introduce source::SourceCode type
This type hides away the lower-level handling of most codemap data
structures, especially to library consumers (see corresponding changes
in tvixbolt).

This will help with implement `import` by giving us central control
over how the codemap works.

Change-Id: Ifcea36776879725871b30c518aeb96ab5fda035a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6855
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-10-05 10:29:47 +00:00
Vincent Ambo
d6643f66b1 feat(tvix/eval): use fancy error formatting in REPL
Change-Id: I8d5652d797b012bc3d5248ca2b2d0c87572dbf2e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6851
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-10-04 21:04:03 +00:00
Vincent Ambo
ee0657c292 fix(tvix/eval): forward thunk error codes from inner errors
Until we can display a chained representatino of errors in thunks, it
is most useful to forward the error code from the innermost error to
the user.

Change-Id: I8d67254d52313be40387f080e57966c001e0d51c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6854
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-04 13:13:20 +00:00
Vincent Ambo
0cee44838c feat(tvix/eval): add error kind for unmergeable nested attributes
Change-Id: Ic5e6d1bf2625c33938360affb0d1a7c922af11bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6799
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 17:46:02 +00:00
Vincent Ambo
3f34af205f feat(tvix/eval): add scaffolding for merging nested attribute sets
This sets up the required logic for finding and merging attribute sets
into nested bindings if they exist. This is absolutely not complete
yet and can, at this commit, probably cause undefined runtime
behaviour if nested attributes are specified.

The basic idea is that a new helper function on the `TrackedBindings`
struct is called with each encountered attribute and determines
whether the new entry can be merged into an existing attribute or not.

Right now the only effect this has in practice is that a new error
becomes available if somebody attempts to cause a merge into an
inherited key.

Change-Id: Id010df3605055eb1ad7fa65241055889dd21bab0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6798
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-29 17:46:02 +00:00
Vincent Ambo
e31f8f735f chore(tvix/eval): fix all current clippy lints
Change-Id: I28d6af8cb408f8427a75d30b9120aaa809a1ea40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6784
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-09-29 11:47:47 +00:00