Commit graph

294 commits

Author SHA1 Message Date
Aspen Smith
7e286aab1a feat(tvix/eval): Box Value::Catchable
This is now the only enum variant for Value that is larger than 8
bytes (it's 16 bytes), so boxing it (especially since it's not
perf-critical) allows us to get the Value size down to only 16 bytes!

Change-Id: I98598e2b762944448bef982e8ff7da6d6683c4aa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10798
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: aspen <root@gws.fyi>
2024-02-13 16:49:53 +00:00
Aspen Smith
dd26177319 revert(tvix/eval): Don't double-box Path values
This reverts commit d3d41552cf.

This was well-intentioned, but now the boxed Path values are actually
the *largest* Value enum variants, at 16 bytes (because they're
fat-pointers, with a len) instead of 8 bytes like all the other values.
Having the double reference is a reasonable price to pay (it seems; more
benchmarks may end up disagreeing) for a smaller Value repr.

Change-Id: I0d3e84f646c8f5ffd0b7259c4e456637eea360f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10797
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2024-02-13 16:49:53 +00:00
Aspen Smith
e3c92ac3b4 fix(tvix/eval): Replace inner NixString repr with Box<Bstr>
Storing a full BString here incurs the extra overhead of the capacity
for the inner byte-vector, which we basically never use as Nix strings
are immutable (and we don't do any mutation / sharing analysis).
Switching to a Box<BStr> cuts us from 72 bytes to 64 bytes per
string (and there are a lot of strings!)

Change-Id: I11f34c14a08fa02759f260b1c78b2a2b981714e4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10794
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-02-13 16:49:53 +00:00
Aspen Smith
5d2ae840f1 refactor(tvix/eval): Box the inside of Value::Json
serde_json::Value is pretty large, and is contributing (albeit not
exclusively) to the large size of the Value repr. Putting it in a box
is *especially* cheap (since it's rarely used) and allows us
to (eventually) cut down on the size of Value.

Change-Id: I005a802d8527b639beb4e938e3320b11ffa1ef23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10795
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
2024-02-10 20:34:25 +00:00
Aspen Smith
3fb0697a71 fix(tvix/eval): Propagate catchables in NixAttrs::construct
Correctly propagate the case where the *key* of an attrset is a
Value::Catchable (eg { "${builtins.throw "c"}" = "b"; }) in
`NixAttrs::construct`, by converting the return type to
`Result<Result<Self, CatchableErrorKind>, ErrorKind>` (ugh!!) and
correctly handling that everywhere (including an `expect` in the
Deserialize impl for NixAttrs, since afaict this is impossible to hit
when deserializing from stuff like JSON).

Change-Id: Ic4bc611fbfdab27c0bd8a40759689a87c4004a17
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10786
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-02-09 19:11:09 +00:00
Aspen Smith
8446cd1c8b fix(tvix/eval): Inline List.sort_by, and propagate errors
In order to correctly propagate errors in the comparator passed to
builtins.sort, we need to do all the sorting in a context where we can
short-circuit return `Value`s (because catchables are Values on the `Ok`
side of the Result , not `Err`s). Unfortunately this means we have
to *inline* the List `sort_by` implementation into the builtin_sort
function - fortunately this is the only place that was called so this is
relatively low cost. This does that, and adds the requisite `try_value!`
invocation to allow us to propagate comparator errors here.

As before, this doesn't include tests, primarily since those are coming
in the next commit.

Change-Id: I8453c3aa2cd82299eae89828e2a2bb118da4cd48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10754
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-02-08 22:50:27 +00:00
Aspen Smith
5f0f4ea374 refactor(tvix/eval): Box Value::String
NixString is *quite* large - like 80 bytes - because of the extra
capacity value for BString and because of the context. We want to keep
Value small since we're passing it around a lot, so let's box the
NixString inside Value::String to save on some memory, and make cloning
ostensibly a little cheaper

Change-Id: I343c8b4e7f61dc3dcbbaba4382efb3b3e5bbabb2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10729
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2024-02-02 16:16:56 +00:00
Aspen Smith
8caa097ba8 feat(tvix/eval): Don't emit OpForce for non-thunk constants
In the compiler, skip emitting an OpForce if the last op was an
OpConstant for a non-thunk constant. This gives a small (~1% on my
machine) perf boost, eg when evaluating hello.outPath:

    ❯ hyperfine \
        "./before --no-warnings -E '(import <nixpkgs> {}).hello.outPath'" \
        "./after --no-warnings -E '(import <nixpkgs> {}).hello.outPath'"
    Benchmark 1: ./before --no-warnings -E '(import <nixpkgs> {}).hello.outPath'
      Time (mean ± σ):      1.151 s ±  0.022 s    [User: 1.003 s, System: 0.151 s]
      Range (min … max):    1.123 s …  1.184 s    10 runs

    Benchmark 2: ./after --no-warnings -E '(import <nixpkgs> {}).hello.outPath'
      Time (mean ± σ):      1.140 s ±  0.022 s    [User: 0.989 s, System: 0.152 s]
      Range (min … max):    1.115 s …  1.175 s    10 runs

    Summary
      ./after --no-warnings -E '(import <nixpkgs> {}).hello.outPath' ran
        1.01 ± 0.03 times faster than ./before --no-warnings -E '(import <nixpkgs> {}).hello.outPath'

Change-Id: I2105fd431d4bad699087907e16c789418e9a4062
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10714
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-02-01 21:08:35 +00:00
Aspen Smith
d3d41552cf refactor(tvix/eval): Don't double-box Path values
PathBuf internally contains a heap pointer (an OsString), so we were in
effect double-boxing here. Removing the extra layer by making
Tvix::Value represented by a Box<Path> rather than a Box<PathBuf> saves
us an indirection, while still avoiding the extra memory overhead of the
capacity which was the reason we were boxing PathBuf in the first place.

Change-Id: I8c185b9d4646161d1921917f83e87421496a3e24
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10725
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
2024-02-01 17:25:42 +00:00
Aspen Smith
201173afac fix(tvix): Represent strings as byte arrays
C++ nix uses C-style zero-terminated char pointers to represent strings
internally - however, up to this point, tvix has used Rust `String` and
`str` for string values. Since those are required to be valid utf-8, we
haven't been able to properly represent all the string values that Nix
supports.

To fix that, this change converts the internal representation of the
NixString struct from `Box<str>` to `BString`, from the `bstr` crate -
this is a wrapper around a `Vec<u8>` with extra functions for treating
that byte vector as a "morally string-like" value, which is basically
exactly what we need.

Since this changes a pretty fundamental assumption about a pretty core
type, there are a *lot* of changes in a lot of places to make this work,
but I've tried to keep the general philosophy and intent of most of the
code in most places intact. Most notably, there's nothing that's been
done to make the derivation stuff in //tvix/glue work with non-utf8
strings everywhere, instead opting to just convert to String/str when
passing things into that - there *might* be something to be done there,
but I don't know what the rules should be and I don't want to figure
them out in this change.

To deal with OS-native paths in a way that also works in WASM for
tvixbolt, this also adds a dependency on the "os_str_bytes" crate.

Fixes: b/189
Fixes: b/337
Change-Id: I5e6eb29c62f47dd91af954f5e12bfc3d186f5526
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10200
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
2024-01-31 14:51:49 +00:00
Florian Klink
023e372583 feat(tvix/eval): track pattern binding names
These need to be preserved at least for builtins.toXML.

Also, we incorrectly only wrote an <attrspat> in case ellipsis was true,
but that's not the case.

Change-Id: I6bff9c47c2922f878d5c43e48280cda9c9ddb692
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10686
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: aspen <root@gws.fyi>
2024-01-25 11:37:35 +00:00
Florian Klink
e1d2589163 fix(tvix/eval/value/function): use BTreeMap for function arg names
At least toXML wants to get these out in a sorted fashion.

Change-Id: I6373d7488fff7c40dc2ddeeecd03ba537c92c4af
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10685
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-01-24 14:00:44 +00:00
sterni
526295a71d chore(3p/sources): Bump channels & overlays
- Adjust to ecl 23.9.9 release

- Regenerate go protos after protoc-gen-go update

- Drop dhall fork which hasn't kept up with 1.42.*

- Address new clippy warnings:

  - Variant naming of Error::ValidationError
  - Simplify .try_into().unwrap()
  - Drop unnecessary identity function
  - Test module must be last in file
  - Drop unused `pub use`

- Update agenix to 0.15.0. Current master has a installCheckPhase that
  doesn't work with C++ Nix 2.3.*:
  a23aa271be (commitcomment-137185861)

Change-Id: Ic29eef20d6fd1362ce1031364a5ca6b4edf195bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10615
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
2024-01-19 21:47:32 +00:00
Ryan Lahfa
2750e1e640 fix(tvix/eval): catchable-aware builtins
A bunch of operations in Tvix are not aware of catchable values
and does not propagate them.

In the meantime, as we wait for a better solution, we just offer this
commit for moving the needle.

Change-Id: Ic3f0e1550126b0847b597dfc1402c35e0eeef469
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10473
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-14 03:37:17 +00:00
Florian Klink
82540717d6 feat(tvix/eval): make into_json public
Allow other crates (like tvix-glue) to look at a Value in JSON, which is
used by the structured attrs feature.

Change-Id: Iba02ace6e11a74c3f9b19dcbef4b008b76dec046
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10602
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-01-12 22:25:35 +00:00
Ryan Lahfa
cc098b9aaa feat(tvix/eval): contextful coercion of files
In the past reference tracking system, `tvix-io` glue was appending
plain paths in the known path state.

Now, we make up for this by just making contextful coercion of file
imports.

Change-Id: Ieb9b04dd83302c77909252d5f7733857ac3cf8fd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10443
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-03 18:50:29 +00:00
Ryan Lahfa
9d43c26576 feat(tvix/eval): contextful == of derivations
Otherwise, you just fail because they are not... contextless strings!

Change-Id: I0b8f63a18cd89c3841b613d41c12ec4ee336f953
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10442
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-01-03 18:49:57 +00:00
Ryan Lahfa
09ec8b6fcf feat(tvix/eval): implement getContext primop
Change-Id: I2c5068a28f9883a01b0ff80a5e5ab32ba18bfc1a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10437
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-01-03 18:20:43 +00:00
Ryan Lahfa
743c362049 feat(tvix/eval): context-aware coerce_to_string
I am still undecided whether we need a CoercionKind to control
the coerced context, here's a simple attempt.

Change-Id: Ibe59d09ef26c519a6acfdfe392014446646dd6d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10426
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-03 16:45:26 +00:00
Ryan Lahfa
3e63d7be42 feat(tvix/eval): context-aware casting to strings
By default, we don't want contextful strings and we almost always want contextless strings.

To this end, we make taking a contextful string a very explicit operation under `to_contextful_str`
and we implement manually the `to_str` cast which requires a `if !s.has_context()` guard that
the macro cannot cover.

Change-Id: I7aae8e57a7d73e547e62b1edb0b1cc7e8c0c69b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10425
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-29 22:00:04 +00:00
Ryan Lahfa
5dfb15d2c8 feat(tvix/eval): introduce NixContext
This prepares the data structures to implement string contexts
in Nix.

Change-Id: Idd913c9c881daeb8d446907f4b940e462e730978
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10420
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-29 21:57:30 +00:00
Adam Joseph
7ddea7340f fix(tvix/eval): catchable in type field of nix_eq()
Change-Id: I165ff77764e272cc94d18cb03ad6cbc9a8ebefde
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10348
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
Adam Joseph
11e35a77a6 refactor(tvix/eval): let OpCoerceToString select the CoercionKind
Change-Id: I92d58ef216d7e0766af70f019b3dcd445284a95d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10344
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-29 21:34:45 +00:00
sterni
d07600dbca fix(tvix/eval/value): correctly emit spaces when coercing lists
r/7176 introduced an incorrect assumption was the benefit of the
nonrecursive coercion algorithm, namely that a coercion operation always
returns a non empty string. This allows to detect whether we are
coercing a list or not by checking if the intermediate result is empty
or not. Unfortunately, coercing null and false yields an empty string,
so we need to explicitly track whether we are coercing a list.

Updated the test case to hopefully catch similar bugs in the future. I'm
not a hundred percent certain I have not introduced a new edge case with
this, so it may be interesting to add a prop test case for this to
nix_oracle down the line. At least lists are the only nested data
structures that can be serialized as nested data structures, so the
problem is kind of limited.

Change-Id: Ia41e904356f1c41a9d35e4e65ec02f2fe5a4100e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10418
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-12-25 15:06:23 +00:00
sterni
7165ebc43b fix(tvix/eval): remove incorrect imports when coercing
The default behavior of string coercion in C++ Nix is to weakly coerce
and import to store if necessary. There is a flag to make it strongly
coerce (coerceMore) and a flag that controls whether path values have
the corresponding file/directory imported into the store before
returning the (store) path as a string (copyToStore). We need to
implement our equivalent to the copyToStore (import_paths) flag for the
benefit of weak coercions that don't import into the store (dirOf,
baseNameOf, readFile, ...) and strong coercions that don't import into
the store (toString).

This makes coerce_to_string as well as CoercionKind weirder and more
versatile, but prevents us from reimplementing parts of the coercion
logic constantly as can be seen in the case of baseNameOf.

Note that it is not possible to test this properly in //tvix/eval tests
due to the lack of an appropriate EvalIO implementation being available.
Tests should be added to //tvix/glue down the line.

Change-Id: I8fb8ab99c7fe08e311d2ba1c36960746bf22f566
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10361
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-12-14 13:15:23 +00:00
Adam Joseph
9792920f8c fix(tvix/eval): fix branching on catchable defaults (b/343)
This commit adds Opcode::OpJumpIfCatchable, which can be inserted
ahead of most VM operations which expect a boolean on the stack, in
order to handle catchables in branching position properly.

Other than remembering to patch the jump, no other changes should be
required.

This commit also fixes b/343 by emitting this new opcode when
compiling if-then-else.  There are probably other places where we
need to do the same thing.

Change-Id: I48de3010014c0bbeba15d34fc0d4800e0bb5a1ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10288
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 14:55:48 +00:00
Adam Joseph
1ac57b0d1c feat(tvix/eval): nonrecursive coerce_to_string()
After this commit, the only non-builtins uses of generators are:

  - coerce_to_string() uses generators::request_enter_lambda()
  - Thunk::force() uses generators::request_enter_lambda()

That's it!  Once those two are taken care of, GenCo can become an
implementation detail of `builtins::BuiltinGen`.  No more crazy
nonlocal flow control within the interpreter: if you've got a GenCo
floating around in your code it's because you're writing a builtin,
which isn't part of the core interpreter.  The interpreter won't
need GenCos to talk to itself anymore.

Technically generators::request_path_import() is also used by
coerce_to_string(), but that's just because the io_handle happens to
be part of the VM.  There's no recursion-depth issue there, so the
call doesn't need to go through the generator mechanism
(request_path_import() doesn't call back to the interpreter!)

Change-Id: I83ce5774d49b88fdafdd61160975b4937a435bb0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10256
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 14:54:46 +00:00
Adam Joseph
3701379745 feat(tvix/eval): nonrecursive deep_force()
This commit implements deep_force() nonrecursively, by maintaining
an explicit stack rather than using the call stack for recursion.

As an added bonus, we don't need to pass around the SharedThunkSet
anymore, and can in fact completely eliminate SharedThunkSet.

Change-Id: I7c4f59f37834d451a28bf6be317eb0a90eac4ee6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10252
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 14:36:02 +00:00
Adam Joseph
c956138ee6 docs(tvix/eval): clarify difference between ThunkSet and Blackhole
The comment explaining ThunkSet makes it seem like it does the same
think as ThunkRepr::Blackhole.  In fact neither one is a substitute
for the other.  Let's explain the difference.

Change-Id: I89ceaaa9d3c499edbc7d48f70ca5d11f97666c43
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10250
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 14:35:00 +00:00
Adam Joseph
ad566999ca fix(tvix/eval): preserve catchables in nix_cmp_ordering(), fix b/338
This commit fixes b/338 by properly propagating catchables through
comparison operations.

Change-Id: I6b0283a40f228ecf9a6398d24c060bdacb1077cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10221
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 14:26:46 +00:00
Adam Joseph
edbd5055a1 feat(tvix/eval): nonrecursive nix_cmp_ordering(), fixes b/339
This commit rewrites Value::nix_cmp_ordering() into an equivalent
nonrecursive form.  Except for calls to Thunk::force(), the new form
no longer uses generators, and is async only because of the fact
that it calls Thunk::force().

I originally believed that this commit would make evaluation faster.
In fact it is slightly slower.  I believe this is due to the added
vec![] allocation.  I am investigating.

Prev-Nixpkgs-Benchmark: {"attrpath":"pkgsCross.aarch64-multiplatform.hello.outPath","peak-kbytes":"460048","system-seconds":"0.68","user-seconds":"5.73"}
This-Nixpkgs-Benchmark: {"attrpath":"pkgsCross.aarch64-multiplatform.hello.outPath","peak-kbytes":"460224","system-seconds":"0.67","user-seconds":"5.84"}
Change-Id: Ic627bc220d9c5aa3c5e68b9b8bf199837cd55af5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10212
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 14:26:46 +00:00
Adam Joseph
8a40f75c2d fix(tvix/eval): never use partial_cmp() (partial fix b/338)
This is part of a fix for b/338.

We should never use PartialOrd::partial_cmp().

All Nix types except floats are obviously totally-ordered.  In
addition, it turns out that because Nix treats division by zero
rather than producing a NaN, and because it does not support
"negative zero", even floats are in fact totally ordered in Nix.

Therefore, every call to PartialOrd::partial_cmp() in tvix is an
error.  We have to *implement* this function, but we should never
call it on built-in types.

Moreover, nix_cmp_ordering() currently returns an Option<Ordering>.
I'm not sure what was going on there, since it's impossible for it
to return None.  This commit fixes it to return simply Ordering
rather than Option<Ordering>.

Change-Id: If5c084164cf19cfb38c5a15554c0422faa5f895d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10218
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-12 14:26:46 +00:00
Adam Joseph
72ece2e518 feat(tvix/eval): nonrecursive nix_eq()
This commit rewrites Value::nix_eq() into an equivalent.  Except for
calls to Thunk::force(), the new form no longer uses generators, and
is async only because of the fact that it calls Thunk::force().

I believed that the nonrecursive form would be faster.  It is, in
fact, slightly slower.  I believe this is due to the vec![]
allocation; I am investigating.

Prev-Nixpkgs-Benchmark: {"attrpath":"pkgsCross.aarch64-multiplatform.hello.outPath","peak-kbytes":"459068","system-seconds":"0.71","user-seconds":"5.39"}
This-Nixpkgs-Benchmark: {"attrpath":"pkgsCross.aarch64-multiplatform.hello.outPath","peak-kbytes":"460048","system-seconds":"0.68","user-seconds":"5.73"}
Change-Id: I10f4868891e4b7475df13f0cbc41ec78dd985dd8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10118
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 14:26:46 +00:00
Florian Klink
d3ecef1a5b refactor(tvix/eval): address clippy lints
Change-Id: Ic2bd4e8291b30ceac9fa0e88a4f56e61ae99b603
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10227
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-12-09 12:58:39 +00:00
Adam Joseph
5fd42a9e70 feat(tvix/eval): impl DoubleEndedIter for OwnedAttrsIterator
Change-Id: I4bd85dbe9c27047f4abbdeff4e2b796e9bcab3a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10211
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2023-12-09 11:35:19 +00:00
Adam Joseph
49b34183e3 feat(tvix/eval): rewrite Thunk::force() in nonrecursive form
This commit rewrites Thunk::force() so that it is not (directly)
self-recursive.  It maintains a Vec of all the
previously-encountered thunks which point to the one it is currently
forcing, rather than recursively calling itself.

Benefits:

- Short term:

  This commit saves the cost of a round-trip through the generator
  machinery for the generators::request_force() which is removed by
  this commit.

- Medium term:

  Once a similar transformation has been applied to nix_cmp(),
  nix_add(), nix_eq(), and coerce_to_string(), those four functions,
  along with Thunk::force(), will make non-tail calls only to each
  other.  They can then be merged into a single tail-recursive
  function which does not use the generator machinery at all:

    enum Task { Cmp, Add, Eq, CoerceToString, Force};

    fn Value::walk(task:Task, v1:Value, v2:Value) {
      // ...

- Long term:

  The long-term goal here is to use generators **only for builtins**
  and [Marionette]-style remote control of the VM.  In other words:
  use `async` for things that actually involve concurrency.  Calls
  from the VM to builtins can then be blocking calls, because even
  cppnix will overflow the stack if you make a MAX_STACK_DEPTH-deep
  recursive call which passes through a builtin at every stack frame
  (e.g. `{ func = builtins.sort (a: b: ... func ...) ...}`).

  This way the inner "tight loop" of the interpreter doesn't pay the
  costs of `async` and generators.  These costs manifest in terms
  of: performance, complex nonlocal control flow, and language
  impediments (async Rust is a restricted subset of real Rust, and
  is missing things like traits).

[Marionette]: https://firefox-source-docs.mozilla.org/testing/marionette/Intro.html

Change-Id: I6179b8abb2ea0492180fcb347f37595a14665777
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10039
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-12-06 06:53:01 +00:00
Aspen Smith
8135a8d38c fix(tvix/eval): Return error rather than panicking on bad substring
If builtins.substring is invoked with (byte!!) offsets that aren't at
codepoint boundaries, return an error rather than panicking. This is
still incorrect (see b/337) but pushes the incorrectness forward a step.

Change-Id: I5a4261f2ff250874cd36489ef598dcf886669d04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10199
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-05 23:12:23 +00:00
Vincent Ambo
5ffb997864 fix(tvix): ensure PartialOrd/Ord agree for StorePath & NixString
This fixes a *future* clippy lint:
https://rust-lang.github.io/rust-clippy/master/index.html#/incorrect_partial_ord_impl_on_ord_type

In essence, because the implementation of *both* Ord and PartialOrd
implies that ordering is not partial, all results of PartialOrd should
simply be those of Ord. This is to avoid subtle bugs in future
refactorings.

Change-Id: I8fc6694010208752dd47746a2aaaeca0c788d574
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10109
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-11-25 15:48:46 +00:00
Adam Joseph
512346ba0b refactor(tvix/eval): add ThunkRepr::is_forced()
Change-Id: I4eab5c81fb82337da06327248845cd2f3a4490d3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10038
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-11-25 02:55:57 +00:00
Adam Joseph
b5a15989cd feat(tvix/eval): add Thunk::unwrap_or_clone()
This commit adds Thunk::unwrap_or_clone(), which uses
Rc::try_unwrap() to avoid cloning the Value out of a an Rc which has
only one strong reference.

Change-Id: Icacefe0c823dcddf046d90c0c5cd5ed59fe976d4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10037
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2023-11-25 02:55:56 +00:00
Vincent Ambo
87d3fac676 chore(tvix): add missing clippy attributes & config
For cases where clippy lints don't apply to us, or something is
misfiring, add appropriate configuration.

Relates to b/321.

Change-Id: I0af453910b4a4112bf685b2a8e9a73de10ec87ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9965
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-11-05 20:28:38 +00:00
Vincent Ambo
2e1399698b refactor(tvix/eval): use IntoIterator trait for owned NixAttrs iter
Uses the standard library IntoIterator trait for the construction of
our iterators. Clippy complains about duplicating this.

While doing this, I opted to rename the `IntoIter` type into something
that is more useful to users, in case somebody ends up working with
these manually.

This fixes a clippy lint, and is related to b/321.

Change-Id: I851fde0d7b8b38d182343a0fd6d9f8dd2a33ee11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9963
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
2023-11-05 20:28:38 +00:00
Vincent Ambo
b3b1f649d6 chore(tvix): fix trivial clippy lints
Relates to b/321.

Change-Id: I37284f89b186e469eb432e2bbedb37aa125a6ad4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9961
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
2023-11-05 20:28:37 +00:00
Vincent Ambo
99f618bcb4 refactor(tvix/eval): delay allocation when comparing attr values
Delays allocation (through cloning) of the values to be compared
until *after* the keys have been compared.

Change-Id: I7d68c27d7a0fbcdcc387db7c092bce50ca4b94ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9900
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-11-03 09:24:31 +00:00
Vincent Ambo
2dd2b844c7 chore(tvix/eval): add a marker for sorted borrowed attrs iteration
Similar to `into_iter_sorted`, add a marker function for call sites
that want *borrowed* sorted iteration.

Change-Id: I7c6f14e1ac43fdb14b861b3da183eb5d12bba139
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9899
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-11-03 09:24:31 +00:00
Adam Joseph
05f42519b5 fix(tvix/eval): fix b/281 by adding Value::Catchable
This commit makes catchable errors a variant of Value.

The main downside of this approach is that we lose the ability to
use Rust's `?` syntax for propagating catchable errors.

Change-Id: Ibe89438d8a70dcec29e016df692b5bf88a5cad13
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9289
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
2023-09-24 21:54:10 +00:00
Florian Klink
731996fbfe docs(tvix/eval): fix some broken docstr references
There's some more left, but they've been renamed/refactored out of
sight.

Change-Id: I41579dedc74342b4c5f8cb39d2995b5b0c90b0f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9372
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
2023-09-22 09:14:33 +00:00
Adam Joseph
9256621009 fix(tvix/eval): update identifier quoting to match cppnix 2.17
In cppnix 2.17, commit b72bc4a972fe568744d98b89d63adcd504cb586c, the
libexpr pretty-printing routine was fixed so that it would no longer
pretty-print attrsets with keywords in their attrnames incorrectly.

This commit implements the corresponding fix for tvix, fixes our
tests to work with cppnix>=2.17 oracles, and expands our test cases
to cover all the keywords.

Change-Id: I4b51389cd3a9c44babc8ab2a84b383b7b0b116ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9283
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-09-15 00:00:59 +00:00
Florian Klink
366e11b48a refactor(tvix/eval): don't use format! in write! args
Change-Id: I0b2fdb418c2e36280d5c551a81634e1742193903
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9105
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-08-20 21:52:21 +00:00
Florian Klink
1f02dc7eba refactor(tvix/eval): cargo clippy &GenCo
Change-Id: I6b1b902ccbc12bf2acdb0fdf406d6ef336ff0b2f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9098
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-08-20 21:51:04 +00:00