mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
tvl-depot/tvix/eval
History
Aspen Smith 6366cee717 feat(tvix/eval): Intern (and leak) small strings, behind a mutex 
This is the most naive version of string interning possible - we store a
map from the string itself to the pointer behind a global mutex, and
memoize the allocation of all strings below a threshold length (16
bytes, for now) into that map. This requires leaking /all/ strings,
since it's not easy to know just from the pointer that a string has been
interned - so interning is disabled if string leaking is also disabled.

In the case where we're leaking strings (the default), even the naive
version of this gets us a pretty nice perfomance boost:

hello outpath           time:   [742.54 ms 745.89 ms 749.14 ms]
                        change: [-2.8722% -2.0135% -1.0654%] (p = 0.00 < 0.05)
                        Performance has improved.

However, in the case where we're not leaking strings, we have to keep
track of which strings have and haven't been interned, which makes this
a little worse:

hello outpath           time:   [779.30 ms 792.82 ms 808.74 ms]
                        change: [+2.5258% +4.0884% +5.8931%] (p = 0.00 < 0.05)
                        Performance has regressed.

Hopefully we can close the gap here a bit with some clever
tricks (coming next).

Change-Id: If08cb48ede703c7fe3bdd8d617443f8a561ad09b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12047
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
2024-08-08 00:02:36 +00:00
..
benches refactor(tvix/eval): Builderize Evaluation 2024-07-06 15:03:46 +00:00
builtin-macros refactor(tvix/eval): Generalize propagation of catchable values 2024-02-08 19:59:21 +00:00
proptest-regressions/value refactor(tvix/eval): flatten call stack of VM using generators 2023-03-13 20:30:59 +00:00
src feat(tvix/eval): Intern (and leak) small strings, behind a mutex 2024-08-08 00:02:36 +00:00
tests refactor(tvix/eval): Builderize Evaluation 2024-07-06 15:03:46 +00:00
.skip-subtree feat(tvix/tests): check in Nix' language test suite 2022-08-24 21:25:41 +00:00
build.rs fix(tvix): make rstest pick up new test case files 2024-06-05 17:52:17 +00:00
Cargo.toml feat(tvix/eval): Forbid Hash{Map,Set}, use Fx instead 2024-08-07 12:38:40 +00:00
clippy.toml feat(tvix/eval): Forbid Hash{Map,Set}, use Fx instead 2024-08-07 12:38:40 +00:00
default.nix feat(tvix): add CI targets for checking crate features powerset 2024-06-03 16:35:51 +00:00
README.md chore: update links to tvixbolt 2024-02-24 10:17:56 +00:00

README.md

Tvix Evaluator

This project implements an interpreter for the Nix programming language. You can experiment with an online version of the evaluator: tvixbolt.

The interpreter aims to be compatible with nixpkgs, on the foundation of Nix 2.3.

Important note: The evaluator is not yet feature-complete, and while the core mechanisms (compiler, runtime, ...) have stabilised somewhat, a lot of components are still changing rapidly.

Please contact TVL with any questions you might have.

Building tvix-eval

Please check the README.md one level up for instructions on how to build this.

The evaluator itself can also be built with standard Rust tooling (i.e. cargo build).

If you would like to clone only the evaluator and build it directly with Rust tooling, you can do:

git clone https://code.tvl.fyi/depot.git:/tvix/eval.git tvix-eval

cd tvix-eval && cargo build

Tests

Tvix currently has three language test suites for tvix-eval:

  • nix_tests and tvix_tests are based on the same mechanism borrowed from the C++ Nix implementation. They consist of Nix files as well as expected output (if applicable). The test cases are split into four categories: eval-okay (evaluates successfully with the expected output), eval-fail (fails to evaluate, no expected output), parse-okay (expression parses successfully, no expected output) and parse-fail (expression fails to parse, no expected output). Tvix currently ignores the last two types of test cases, since it doesn't implement its own parser.

    Both test suites have a notyetpassing directory. All test cases in here test behavior that is not yet supported by Tvix. They are considered to be expected failures, so you can't forget to move them into the test suite proper when fixing the incompatibility.

    Additionally, separate targets in the depot pipeline, under //tvix/verify-lang-tests, check both test suites (including notyetpassing directories) against C++ Nix 2.3 and the default C++ Nix version in nixpkgs. This way we can prevent accidentally introducing test cases for behavior that C++ Nix doesn't exhibit.

    • nix_tests has the test cases from C++ Nix's language test suite and is sporadically updated by manually syncing the directories. The notyetpassing directory shows how far it is until we pass it completely.

    • tvix_tests contains test cases written by the Tvix contributors. Some more or less duplicate test cases contained in nix_tests, but many cover relevant behavior that isn't by nix_tests. Consequently, it'd be nice to eventually merge the two test suites into a jointly maintained, common Nix language test suite.

      It also has a notyetpassing directory for missing behavior that is discovered while working on Tvix and isn't covered by the nix_tests suite.

  • nix_oracle can evaluate Nix expressions in Tvix and compare the result against C++ Nix (2.3) directly. Eventually it should gain the ability to property test generated Nix expressions. An additional feature is that it can evaluate expressions without --strict, so thunking behavior can be verified more easily.

rnix-parser

Tvix is written in memory of jD91mZM2, the author of rnix-parser who sadly passed away.

Tvix makes heavy use of rnix-parser in its bytecode compiler. The parser is now maintained by Nix community members.