Commit graph

1023 commits

Author SHA1 Message Date
Vincent Ambo
1028aff105 chore(tvix/eval): remove dead comment
Change-Id: Icedb7f272e5067569b8dbf1c2d8b0fdd352b8e12
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7936
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
2023-01-26 23:30:43 +00:00
Florian Klink
54b4fb8b16 refactor(tvix/derivation): align error messages with rust style
Change-Id: I9ccd4c043bdddefee98a2c0b3d6eb7d9cb53c454
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7935
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-01-26 16:30:33 +00:00
Vincent Ambo
2e66f7da92 fix(tvix/cli): correctly trim cppnix output in NixCompatIO
We only stripped one of the two uses of this string, leading to
extraneous newlines in the refscanner.

Change-Id: I25d9119be082c487352f0cf66b97ecdcc3e1de06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7932
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-26 10:29:55 +00:00
sterni
daa3721f73 docs(tvix): fix minor spelling problems in pointer equality document
Main one is the its-it's mistake in the last paragraph, the rest was
suggested by LanguageTool.

Change-Id: If1b87a11f480452f312fc2759be7ded782d0a522
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7930
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-01-25 14:30:50 +00:00
sterni
8a8325fb9d docs(tvix/eval): builtins.add is not equivalent to +
While it is in the given example, i.e. for integer addition, to claim
that they are equivalent is a bit misleading: builtins.add is less
overloaded than +, i.e. builtins.add "foo" "bar" will fail whereas
"foo" + "bar" performs string concatenation.

Change-Id: Ib52d530d1ab289b367565b286f06a76dd518d4fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7929
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-25 14:30:50 +00:00
Florian Klink
669496f0ba test(tvix/eval): add test for total_fmt_float
Change-Id: If6c478ee3d2e4ecf5ef92289614f86535ad05cb7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7927
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-01-25 11:24:20 +00:00
Vincent Ambo
164005656d refactor(tvix/eval): extract float formatting into a helper
This keeps the actual TotalDisplay implementation readable, as this
float formatting code suddenly made up the majority of its implementation.

Change-Id: I2c0d00e4a691e0b8ffbc72680f680e16feef4bee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7925
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-25 11:24:19 +00:00
Florian Klink
e0b05c0fa6 feat(tvix/eval): implement builtins.fromTOML
This allows parsing TOML from Tvix. We can enable the eval-okay-fromTOML
testcase from nix_tests. It uses the `toml` crate, and the serde
integration it brings with it.

Change-Id: Ic6f95aacf2aeb890116629b409752deac49dd655
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7920
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-25 07:49:44 +00:00
Florian Klink
1facd889bb feat(tvix/eval): use lexical-core to format float
Apparently our naive implementation of float formatting, which simply
used {:.5}, and trimmed trailing "0" strings not sufficient.

It wrongly trimmed numbers with zeroes but no decimal point, like
`10000` got trimmed to `1`.

Nix uses `std::to_string` on the double, which according to
https://en.cppreference.com/w/cpp/string/basic_string/to_string
is equivalent to `std::sprintf(buf, "%f", value)`.

https://en.cppreference.com/w/cpp/io/c/fprintf mentions this is treated
like this:

> Precision specifies the exact number of digits to appear after
> the decimal point character. The default precision is 6. In the
> alternative implementation decimal point character is written even if
> no digits follow it. For infinity and not-a-number conversion style
> see notes.

This doesn't seem to be the case though, and Nix uses scientific
notation in some cases.

There's a whole bunch of strategies to determine which is a more compact
notation, and which notation should be used for a given number.
https://github.com/rust-lang/rust/issues/24556 provides some pointers
into various rabbit holes for those interested.

This gist seems to be that currently a different formatting is not
exposed in rust directly, at least not for public consumption.

There is the
[lexical-core](https://github.com/Alexhuszagh/rust-lexical) crate
though, which provides a way to format floats with various strategies
and formats.

Change our implementation of `TotalDisplay` for the `Value::Float` case
to use that. We still need to do some post-processing, because Nix
always adds the sign in scientific notation (and there's no way to
configure lexical-core to do that), and lexical-core in some cases keeps
the trailing zeros.

Even with all that in place, there as a difference in `eval-okay-
fromjson.nix` (from tvix-tests), which I couldn't get to work. I updated
the fixture to a less problematic number.

With this, the testsuite passes again, and does for the upcoming CL
introducing builtins.fromTOML, and enabling the nix testsuite bits for
it, too.

Change-Id: Ie6fba5619e1d9fd7ce669a51594658b029057acc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7922
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-25 07:49:44 +00:00
Vincent Ambo
9ecf668452 chore(tvix/eval): delete "useless parenthesis" warning/optimisation
Two main reasons:

1. Traversing the structure to do this optimisation is
   actually *slower* than not optimising it.

2. There are literally hundreds of thousands of incidences of this in
   nixpkgs, and with some of the weird code there some of
   these (functionally) useless parens are actually required for
   readability reasons.

Change-Id: I1044b1c5f9fe20df4b6085851fc3b191277c65dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7917
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-23 17:59:06 +00:00
Vincent Ambo
d2f1a290a4 fix(tvix/eval): force functors before applying them
call_value in the VM expects the callable to be forced when calling
it, which was not the case for functors.

Change-Id: Id55a2fe32a9573be42aef8669e268df519a989cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7909
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-01-23 17:27:53 +00:00
Vincent Ambo
9f75973e4a refactor(tvix/derivation): relax bounds for reference iterator
This makes it easier to pass owned strings if that's what we have.

Change-Id: Ia7351ff2681292d16534ec50fe60b926b683bb9a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7908
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-23 15:08:32 +00:00
Florian Klink
b0d6e748ac refactor(tvix/derivation): remove DOT_FILE_EXT const
This is used in few enough places to just inline it. It felt a bit alien
in the ATerm construction aswell.

write.rs now pleasantly only includes tokens that occur in the ATerm
representation.

Change-Id: I524f8d6c1ce9057ff7fd16c6c3efd98467040a44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7911
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-23 13:53:45 +00:00
Florian Klink
fc177af0c1 refactor(tvix/derivation): pass fingerprint to build_store_path
All of these occurences are actually sha256 digests of some fingerprint
- which means there's not a lot of need to do the hashing outside.
Instead of passing in a digest, keep the sha256 hasher in the function,
and pass in a fingerprint.

This makes it much easier to construct fingerprints using format!() in
all consumers, because we need don't need to juggle with the hasher
anymore.

Change-Id: I2dc3af2cab6cf06f55ae6cbd9a8be95faf2a07b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7907
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-23 13:40:46 +00:00
Vincent Ambo
5d856ac2e9 feat(tvix/derivation): add path_with_references
This allows the calculation of a store path for a plain string that
potentially contains references. These paths are used for
`builtins.toFile` (and potentially other features of C++ Nix).

Change-Id: Ic507c7f264f362b5e6e628255869e5a4fbe4d788
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7906
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-23 12:55:24 +00:00
Vincent Ambo
be7f64ac87 refactor(tvix/derivation): minor structure simplification
Fixes some module comments and embeds the `compress_hash` function in
the `derivation` module, as it was not used outside of this module
anyways.

Change-Id: I6c5c92b3f0c03c2cdcbcfc2f813909a968c4d44c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7905
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-23 12:55:24 +00:00
Vincent Ambo
5719763fd3 feat(tvix/eval): support builtins implemented in Nix itself
This makes it possible to inject builtins into the builtin set that
are written in Nix code, and which at runtime are represented by a
thunk that will compile them the first time they are used.

Change-Id: Ia632367328f66fb2f26cb64ae464f8f3dc9c6d30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7891
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-22 20:48:53 +00:00
Vincent Ambo
8513a58b37 docs(tvix/eval): update some outdated comments
These don't apply anymore since the "antidote-CL".

Change-Id: I40ee73ef43d44bbfc650a8fe6c2b33263dd06959
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7890
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-22 19:39:00 +00:00
Florian Klink
1a84e3e6d8 chore(tvix/store): add fastcdc crate
This is used for content-defined chunking.

Change-Id: I10345372cecb9a643cc51ca45aa5b77d2a05198a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7889
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-21 15:12:45 +00:00
Florian Klink
4614db2f83 docs(tvix): link to IRC channel and mailing list
I omitted the link to the commit history. Most people see this README
through some git interface anyways, so it felt a bit redundant.

Change-Id: I92cf6d08b83ef680fe37df29d3d546cad020955a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7888
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-01-21 15:08:38 +00:00
Florian Klink
758b94f47c docs(tvix): move most of //tvix/eval README up to //tvix
While moving the CLI out of the evaluator, we forgot to update the
README in //tvix/eval. Move this up to //tvix, so people know where
to start.

Keep the instructions on how to build only `//tvix/eval` in `//tvix/
eval/README.md`.

Change-Id: Ie2755e8b5a0056225dbf3a0ee040f70f7f6a1f27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7887
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-21 15:08:38 +00:00
Florian Klink
90979d39f3 feat(tvix/store/directory): validate Directory and sizes
This calls out to Directory::validate() for all received Directory
messages, and also makes sure the sizes we refer a Directory message as
matches the sizes that have been calculated.

Change-Id: I316f9191d5872ee4ba6d78b9a4326f069b22fa63
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7882
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-21 10:48:17 +00:00
Florian Klink
d8e0fa8e5e feat(tvix/store/directory): deduplicate Directory messages
We can omit sending Directory messages to clients that have already been
sent in the same stream.

We can also omit storing a Directory message if we already have it -
they're content-addressed anyways.

Change-Id: Iba44565e07157a83a033177a2ffbdddced64ba5c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7881
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-21 10:48:17 +00:00
Florian Klink
e719da53be feat(tvix/store): add SledDirectoryService
Change-Id: I8e97a59da2c6623f4cca746efa741097a6dbe5b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7871
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-21 10:48:17 +00:00
Adam Joseph
22b9e6ff09 refactor(tvix/eval): administer antidote for poison
The codebase contains a lot of complexity and odd roundabout
handling for shadowing globals.  I'm pretty sure none of this is
necessary, and all of it disappears if you simply make the globals
part of the ordinary identifier resolution chain, with their own
scope up above the root scope.  Then the ordinary shadowing routines
do the right thing, and no special cases or new terminology are
required.

This commit does that.

Note by tazjin: This commit was originally abandoned when Adam decided
not to take away reviewer bandwidth for this at the time (eval was
still in a much earlier stage). As we've recently done some
significant refactoring of globals initialisation this came up again,
and it seems we can easily cover the use-cases of the poison tracking
in other ways now, so I've rebased, updated and resurrected the CL.

Co-Authored-By: Vincent Ambo <tazjin@tvl.su>
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: Ib3309a47a7b31fa5bf10466bade0d876b76ae462
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7089
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-21 10:19:15 +00:00
Florian Klink
ab8486e5b8 chore(tvix/store): add tonic-mock
Upstream seems to be dead, so we're using https://github.com/tyrchen/
tonic-mock/pull/3 here.

According to https://github.com/tyrchen/tonic-mock/pull/1#issuecomment-
1241164173, we might not need this crate at all, but for now, it gets
the job done and is less code to write in the tests.

Change-Id: Ia77fa19b998a5bbabd0311cc714b85a2ee30f36a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7869
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-21 09:34:15 +00:00
Florian Klink
a8b13a0b57 refactor(tvix/store): simplify test a bit
Import more things, and use expect_err to unpack the response.

Change-Id: Ia319dd4d126b8d0e1df585234710d825a33a0002
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7868
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-21 09:34:15 +00:00
Vincent Ambo
7442558b33 refactor(tvix/eval): keep globals alive through VM struct
This forces users to pass the fully constructed set of globals to the
VM, making it harder to accidentally "lose" the set while weak
references to it still exist.

This doesn't modify any functionality, but is laying the foundation
for simplifying some of the builtins behaviour that has grown more
complex again.

Change-Id: I5120f97861c65dc46d90b8a4e2c92ad32cc53e03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7877
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-20 22:48:13 +00:00
Vincent Ambo
6d03e31060 docs(tvix/eval): add doc comments to VM fields
Change-Id: Ia4857c217de15aec8b61e1abd39e22c50e2d816a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7876
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-01-20 22:48:13 +00:00
Vincent Ambo
259faea2b2 feat(tvix/cli): add errors module with drv construction errors
These will be threaded through to eval through the new `TvixError`
variant.

Change-Id: Ia0d3f8710dcf26bb95015cd2a6a2b2911f06343f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7842
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-20 15:39:51 +00:00
Vincent Ambo
7d0456fa0e feat(tvix/eval): let builtin macro capture external state
This adds a feature to the `#[builtins]` macro which lets users
specify an additional state type to (optionally) thread through to
builtins when constructing them.

This makes it possible for builtins-macro users to pass external state
handles (specifically, in our case, known path tracking) into a set of
builtins.

Change-Id: I3ade20d333fc3ba90a80822cdfa5f87a9cfada75
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7840
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-20 15:39:51 +00:00
Vincent Ambo
f12f938166 refactor(tvix/eval): directly return builtin tuples from macro
All invocations of the builtin macro had to previously filter through
the `builtin_tuple` function, but it's more sensible to directly
return these from the macro.

Change-Id: I45600ba84d56c9528d3e92570461c319eea595ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7825
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-20 15:39:51 +00:00
Vincent Ambo
972c867b36 feat(tvix/eval): add error contexts to annotate error kinds
This makes it possible for users to add additional context to an
error, which will then be rendered as an additional secondary span in
the formatted error output.

We should strive to do this basically anywhere errors are raised that
can occur multiple times, *especially* during type casts. This was
triggered by me debugging a type cast error attached to a fairly
large-ish span (a builtin invocation).

Change-Id: I51be41fabee00cf04de973935daf34fe6424e76f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7849
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-20 15:39:51 +00:00
Florian Klink
9822fa387a feat(tvix/derivation): more checks for output hashes and encoding
Change-Id: Idebad60c3bf9daf94d04a36bb73ac0dd767f9e79
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7856
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-18 17:13:23 +00:00
Florian Klink
083e24bbb1 feat(tvix/derivation): add validate_output_paths flag for validate
This allows calling validate() on a derivation that doesn't have its
output paths already calculated yet. It allows offloading some of the
error checking in builtins.derivation* to be offloaded to that function.

Change-Id: Ib4aeadc0eb6583ef8cd765f33e9a9ec32be62729
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7848
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-18 17:13:23 +00:00
Florian Klink
b1e8fe7212 feat(tvix/derivation): check for other invalid output names
This uses the exposed StorePath::validate_name method to check for other
invalid output names (for which it would not be possible to construct a
store path of).

Change-Id: Ia3f65e19a07ef164f9f64013a5f37cbac99eb8e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7855
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-18 17:13:23 +00:00
Florian Klink
4a256dda09 docs(tvix/store): fix docstring for StorePath::to_absolute_path
Change-Id: I8b36c85fe11e4fb62e5d28f6900cd80d89a5cc41
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7854
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-18 17:13:23 +00:00
Florian Klink
0f7e4bec66 feat(tvix/store): rename and expose StorePath::validate_name
Change-Id: I8a16c214c7c644756d9d54187beba8c80ccfb39c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7853
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-18 17:13:23 +00:00
Florian Klink
6e7f06d942 refactor(tvix/derivation): use DerivationError in Output::validate
Change-Id: I7dbd3b8ff9ef92acddde2e579fb24b8311c34d8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7852
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-18 17:13:23 +00:00
Florian Klink
eebb3ce028 refactor(tvix/derivation): rename {ValidateDerivation,}Error
This is now used in more than just validate().

Change-Id: I69c3ad6cb5f3ad60a636fe2ea05d432aebe8e53b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7851
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-18 17:13:23 +00:00
Florian Klink
0aad4e2246 feat(tvix/derivation): also fail if output name is called drv
`drv` is an invalid output name too, as this would cause a
`builtins.derivation` call to return an attrset with a `drvPath` key
(which already exists) and has a different meaning.

Also handle errors during store path construction, and return our own
error type, instead of the ParseStorePathError.

Change-Id: Ib7952dde1d5cf18a0e210928df7c57b5939b7678
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7850
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-01-18 17:13:23 +00:00
Vincent Ambo
0cbdfffea2 refactor(tvix/cli): consistently assert type unity in known_paths
No situation should be allowed in which a path is inserted into
known_paths with different types twice, which we previously enforced
only for some path types.

Change-Id: I8cb47d4b29c0aab3c58694f8b590e131deba7043
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7843
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-17 10:45:09 +00:00
Vincent Ambo
bf6f6a0b3f refactor(tvix/eval): non-hacky suspended native thunks
Instead of having a representation of suspended native thunks that
involves constructing a fake code chunk, make these thunks a
first-class part of the internal thunk representation.

The previous code was not that simple to understand, and actually
contained a critical bug which could lead to Tvix crashes. This
version fixes the particular instance of that bug, but instead
uncovers another (b/238) which can still lead to Tvix crashes.

Fixes: b/237.
Change-Id: I771d03864084d63953bdbb518fec94487481f839
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7750
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-17 10:38:06 +00:00
Vincent Ambo
559a09c5e6 refactor(tvix/eval): remove Box in new_suspended_native
This is unnecessary, Rc already provides all the boxing we need.

Change-Id: I08cf0939c48da43f04c847526c7e5dae5336d528
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7749
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-01-17 10:38:06 +00:00
Vincent Ambo
b9646ab40c feat(tvix/eval): add builtins to builtins
This is a somewhat terrifying hack that enables us to support
`builtins.builtins`, by running a "fake compilation" inside of a
suspended native thunk that can resolve the weak pointer to the
globals.

With this implementation, the thunk at `builtins.builtins` actually
resolves to the "real" `builtins` (verified with a new test).

This is kind of ugly, and it's something users shouldn't use, but
bubbling a warning out of this is difficult at the moment due to a
little bit of trickery with how the spans in suspended native thunks
work (they don't) (see b/237, b/238)

Change-Id: I67d0e93246dd5b279c960aeda00402031aa12af3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7748
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 10:38:06 +00:00
Vincent Ambo
79e54f46ce feat(tvix/cli): add replacement strings tracking to KnownPaths
Replacement strings are some weird internal feature of Nix that is
required for calculating derivation hashes. We need to track these
like other paths, as they need to be re-used on builds with
dependencies on values from previous builds.

Change-Id: Ie955b3fb5ae3685cfadfbe4d06ea6b5e219590c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7828
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-17 10:31:42 +00:00
Vincent Ambo
499e72c1cb feat(tvix/cli): track known plain paths in NixCompatIO
When adding things to a C++ Nix store, ensure that the path is tracked
in the tracker.

Since the mechanism for propagating the tracker instance isn't
finalised yet, I've opted to take an Rc<RefCell> parameter for it. How
exactly that ends up there is going to become clear in the next
commits, but for now it's just instantiated in main with
Default::default.

Change-Id: I90f0b44f2d4f292dedc98ff1aa39041d279b61fd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7833
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 10:31:42 +00:00
Vincent Ambo
9cb3daee20 refactor(tvix/cli): reference scanner owns all the strings
This gets very complex very quickly otherwise, as all the construction
paths for a reference scanner and all the access patterns for the
KnownPaths structure are not yet fully understood.

Change-Id: Ibadf1f18b476695f3c286fc6896ae557760edf63
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7827
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-17 10:31:42 +00:00
Vincent Ambo
940251b87f refactor(tvix/value): use proptest strategies from imbl crate
Instead of going through Vec/BTreeMap for generating our internal
types, use the proptest strategies from imbl.

The one thing I couldn't figure out in the previous implementation is
where the ranges/sizes of generated collections came from. The
strategies in proptest use different types (Range, with an unknown
default value, and SizeRange with 0..100). I've opted to specify
0..100 directly, but we can probably make it configurable.

Change-Id: I749bc4c703fe424099240cab822b1642e5216361
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7791
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 10:20:38 +00:00
Vincent Ambo
f27f5ef0c9 feat(tvix/cli): add known_paths module
This module implements types used to track the set of known paths in
the context of an evaluation.

These are used to determine the build references of a derivation.

Change-Id: I81e15ae33632784e699128916485751613b231a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7816
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 10:20:38 +00:00
Florian Klink
d644ed389a refactor(tvix/derivation): expose proper ValidateDerivationError
Use proper errors, instead of anyhow.

Change-Id: I6db14c72a6319b389b0136aac7b84f50a30fb366
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7847
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-16 23:04:50 +00:00
Florian Klink
95cad95b93 feat(tvix/derivation): make input_sources a BTreeSet
These need to be sorted anyways, so let's use the correct data structure for it.

Change-Id: I009c9989d7647dc1df716170f3680c981db6e4b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7846
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-16 16:45:56 +00:00
Florian Klink
f6b4abac35 chore(tvix/derivation): drop TODOs related to stripping outputs
Different to the go-nix implementation, we don't have a custom "writing
mode" where output paths are omitted from output, but we simply run the
serialization with these fields unpopulated (during construction).

This means, there's no need for a custom writing mode that strips
outputs, so we can drop the comments.

Change-Id: Ic0aafd4e34e0294603490cfce2b1aef4085ff34b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7845
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-01-16 16:45:56 +00:00
Vincent Ambo
67addf3b3d feat(tvix/eval): add error variant for threading through errors
This variant is required for external builtins (which in our case
includes `derivation`) to thread through reasonable error messages.

This has some potential for improvement, but it's an improvement over
the status quo of panicking in the external builtins when no
appropriate error is available.

Change-Id: I7e4bdb0a156c7717092dde30aa4785192182dc66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7841
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-16 13:43:14 +00:00
Vincent Ambo
d365b09226 feat(tvix/eval): implement builtins.toXML
Change-Id: I009efc53a8e98f0650ae660c4decd8216e8a06e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7835
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-16 13:43:14 +00:00
Vincent Ambo
1786b4c835 chore(tvix/eval): add other required items to public API
External implementors of builtins must be able to force values, which
necessitates publishing a bunch more items from the crate.

Change-Id: I8f6b8ae88156aae417dbe630a698d123d0c1c8d4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7830
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-16 13:43:14 +00:00
sterni
ca0bd84589 docs(tvix): add reconstructed changelog for the Nix language
This was put together by digging through the C++ Nix git history as
described in the second paragraph. Note that this is only based on
changes given when the language version was actually increased.

There is no guarantee that there have been no other changes inbetween
that would have warranted an increase as well.

Change-Id: I4ddee0d4ecafa1b3e5e1a867e9700d6c32e936ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7836
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-16 09:32:49 +00:00
Vincent Ambo
43d83274e6 chore(tvix/store): derive Clone for StorePath
Change-Id: Id325afa766b0bd2a3d04cf538f13e77692a22687
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7829
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-14 09:48:37 +00:00
Vincent Ambo
a35dadf9f0 refactor(tvix/derivation): use BTreeSet for derivation outputs
When constructing derivations inside builtins.derivationStrict, we'd
have to very frequently check whether certain outputs have already
been inserted into the derivation inputs.

Using a set type is much easier, especially as this has to be ordered
and the source data that is being inserted also comes from a set,
which might let us pass this more efficiently in the future.

Note that the validate function no longer checks the order of the
entries, as that is now guaranteed by the type.

Change-Id: I2fbb984facba3e668075f6f8df8992092368c63d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7826
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-13 18:01:09 +00:00
Aaqa Ishtyaq
158f4d1d69 fix(tvix/eval): len_without_is_empty clippy warn
This CL addresses clippy warning len_without_is_empty
which expects `.is_empty()` method to be present when
implementing `.len()` method for an item.

Change-Id: I8878db630b9ef5853649a906b764a33299bb5dc8
Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7806
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-12 18:06:49 +00:00
Vincent Ambo
09654ffb80 docs(tvix): add build-references / string-context document
This explains my current thinking on string contexts. Thanks to
everyone who gave input so far.

Change-Id: I773219402a79a9d4753b4e7cfbf3a4a751a993a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7807
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-12 11:01:40 +00:00
Vincent Ambo
02d35e4aa6 feat(tvix/eval): implement builtins.toJSON
Implements `Serialize` for `tvix_eval::Value`. Special care is taken
with serialisation of attribute sets, and forcing of thunks.

The tests should cover both cases well.

Change-Id: I9bb135bacf6f87bc6bd0bd88cef0a42308e6c335
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7803
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-01-12 10:42:44 +00:00
Florian Klink
43f6aec384 feat(tvix/store): implement PathInfoService with sled
This uses [sled](https://github.com/spacejam/sled) to store PathInfo
objects.

Change-Id: I12e8032e5562af8f884efa23a78049fd1108fdbc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7726
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-12 10:11:57 +00:00
Florian Klink
cfa42fd19a chore(tvix/store): update ParseStorePathError messages
Change-Id: I49799393a1e35b4475566819111beb57a628c555
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7801
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-12 10:11:57 +00:00
Florian Klink
ed3b28aa02 chore(tvix/store): update ValidatePathInfoError messages
Change-Id: I8f16e25d998a74ac110ae99eec0edbbfd720c8dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7800
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-12 10:11:57 +00:00
Vincent Ambo
e63bff5545 feat(tvix/refscan): implement reference scanning over data streams
Using yet more machinery from the pretty comprehensive aho_corasick
crate, this makes it possible to pass anything implementing `io::Read`
to the `ReferenceScanner` to accumulate matches.

Change-Id: I5b0e28eb44ea4df24010f40831e29f2cbb8c1f80
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7810
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-11 16:03:51 +00:00
Vincent Ambo
3045645df0 feat(tvix/cli): implement initial refscan module
This module implements a ReferenceScanner struct which uses the
aho_corasick crate to scan string inputs for known, non-overlapping
candidates (store paths, in our case).

I experimented with several different APIs, and landed on this version
with an initial accumulator in the scanner. The scanner is
instantiated from the candidates and "fed" all the strings, then
consumed by the caller to retrieve the result.

Right now only things that look vaguely like bytestrings can be fed to
the scanner, there is no streaming support in the API yet.

Change-Id: I7782f0f0df5fc64bccd813aa14712f5525b0168c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7808
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-11 16:03:51 +00:00
Aaqa Ishtyaq
9382afdb0d fix(tvix/eval): address useless_format clippy warn
This CL address clippy warnings related to use of 'format!' macro
to return unmodified 'String'.

Change-Id: I88726e59d8f39f6a455a8c1f48075b52d167e489
Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7804
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-10 19:13:45 +00:00
Ryan Lahfa
805219a2fa feat(tvix/eval): implement serde::Deserialize for Value
Co-Authored-By: Vincent Ambo <tazjin@tvl.su>

Change-Id: Ib6f7d1f4f4faac36b44f5f75cccc57bf912cf606
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7626
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-10 09:53:21 +00:00
Vincent Ambo
c011a6130c refactor(tvix/eval): impl Display for ErrorKind
This just shuffles the Display implementations around so that
ErrorKind itself is displayable, which is useful in some situations
where errors under construction need to be type-converted.

Change-Id: I7b633d03d0dc34f345c4f20676e0023ecb1db0c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7802
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2023-01-10 09:52:52 +00:00
Vincent Ambo
681800b438 chore(tvix/derivation): expose necessary types in public API
Change-Id: Ia05b8eb3054a39897d3ec5783daecfe4bcf054e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7779
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-01-09 19:48:20 +00:00
Vincent Ambo
1d59d3ba8f fix(tvix/eval): fix last uses of Vec<Value> -> NixList in builtins
Change-Id: I0d71b82eb7ddc1e457b0996b0668006f55f56751
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7790
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-01-08 11:11:52 +00:00
Vincent Ambo
a149a1ee06 fix(tvix/eval): fix typo'd function name in tests
Caught by sterni on cl/7783.

Change-Id: I15d57b893ef22538fdd7e809f3b92861dd2bc1af
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7789
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-07 14:48:42 +00:00
Florian Klink
1e2c859840 feat(tvix/derivation): derive Default for Derivation
Some of the fields in a Derivation struct stay empty, and manually
creating BTreeMap or vec for it is annoying.

Derive Default instead, so we can use the defaults instead of writing
more by hand.

Change-Id: I5d41b4b55c8187cb101eb4266451a470008e0067
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7788
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-07 08:05:41 +00:00
Vincent Ambo
e0c47f32af refactor(tvix/eval): use builtins macro for placeholders
Change-Id: I30bc475e3e36a163fa169083481cdd4b4d0ca456
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7785
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-01-06 20:53:02 +00:00
Vincent Ambo
6b6bd30772 refactor(tvix/eval): move mocked builtins.derivation to tests
This placeholder should not live in the main crate anymore as we will
be injecting the real one from outside of eval, but there are still
language tests that depend on a (simple, mockable) version of it.

Change-Id: I68ea169db15cbdbeed320930d3069e21e376c90d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7783
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-06 20:53:02 +00:00
Florian Klink
396fdde0c0 refactor(tvix/derivation): use StorePath::to_absolute_path()
Instead of concatenating STORE_DIR manually, use
StorePath::to_absolute_path() that does it for us.

Change-Id: Ia288851a05b4e339679db268f3dd7924e7b65586
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7782
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-06 17:59:35 +00:00
Vincent Ambo
1ebda9e13e fix(tvix/derivation): fix build after StorePath rename
This project was not previously covered by CI (fixed in this commit),
so we didn't catch breakage due to a renamed module.

This was noticed while rebasing a CL that has a dependency on this
crate in its Nix build.

Change-Id: Ic48570b9313e5f73e14daab50cf7ea70918c94d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7778
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-06 17:57:06 +00:00
Vincent Ambo
3e03e59893 refactor(tvix/store): move protobuf build config one level up
This embeds the build config directly at the point where `Cargo.nix`
is imported, making it transparent to library consumers.

Change-Id: I5586e12f02ed14587c32d9ef7d93f079366fb127
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7780
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-06 17:57:06 +00:00
Florian Klink
144d010515 feat(tvix/store): document StorePath a bit more.
Change-Id: Ifab28d97ddc22a2073c5df5e6e2cefb51b4b9191
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7777
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-06 15:47:46 +00:00
Florian Klink
c89af03a03 refactor(tvix/store): rename NixPath to StorePath
As discussed in #tvl, this is a more common term for it.

Change-Id: I9b904222b8c076f82192c9b7f0b42be171614ab7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7776
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-06 15:39:35 +00:00
Florian Klink
999afd4be2 test(tvix/derivation): add output_path_construction test
This exercises the output path calculation functions like a constructing
client (an implementation of builtins.derivation) would do.
It first assembles the bar derivation, does the output path calculation
on it, then continues with the foo derivation.

The code ensures the resulting Derivations match our fixtures.

Change-Id: If93f89c6622fac1c1941085083931b6f657c04bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7775
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-06 15:39:35 +00:00
Florian Klink
f73ab76fce feat(tvix/store): add NixPath::to_absolute_path
This provides a function returning a string starting with the store
path prefix, the counterpart of `from_absolute_path`.

Change-Id: I4947f3b00171fe70357c62b3d64dc769b69e7a44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7774
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-06 15:39:35 +00:00
Florian Klink
95bec264d5 feat(tvix/derivation): implement output path calculation
This implement output path calculation for fixed outputs, both fixed-
output and non-fixed-output.

Change-Id: I0a77b99f2ba6b39467cc5dd589ce152a40387f9a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7761
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: jrhahn <mail.jhahn@gmail.com>
2023-01-06 12:25:02 +00:00
Florian Klink
9df9a2f1ab feat(tvix/derivation): add get_fixed_output() helper function
This will return the fixed output of a derivation (and its hash), or
None if the Derivation is not fixed-output.

It will simplify the logic in the output path calculation a bit.

Change-Id: I1066cc18ee4fc419421a8c5995c93ba91b35588f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7760
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-06 12:25:02 +00:00
Florian Klink
d4603fc0af refactor(derivation): return NixPath in calculate_derivation_path
This moves all the hash compression logic into a common helper function.

Also update the docstring, which said "path" here, which could have been
confused with output paths.

Change-Id: Iedfb59aeb24f7638afac669dcd18d57b6cfaece2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7759
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-01-06 12:25:02 +00:00
Vincent Ambo
87c80895cd feat(tvix/eval): skip & warn for useless parenthesis
Change-Id: I567ca0682012b9d09f1217e57a104ac5671f8d82
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7771
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: flokli <flokli@flokli.de>
2023-01-06 12:23:55 +00:00
Vincent Ambo
aadf71a6ed feat(tvix/eval): warn on empty let-bindings
Change-Id: Ib6ef7ce514abbd3e372dfe9df7137aa36dbda9d4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7770
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-06 12:23:55 +00:00
Vincent Ambo
61b8a9b2ba refactor(tvix/eval): short-circuit on empty attrs in compiler
This is marginally more efficient and has simpler bytecode.

Change-Id: Iad37c9aeef24583e8f696911bcd83d43639f2e36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7769
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-06 12:23:55 +00:00
Vincent Ambo
4e98730f38 feat(tvix/eval): warn about empty inherits
Change-Id: I82bec6fe2210bcb88c46fd2fdf3e26bd613d1c1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7768
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2023-01-06 12:23:55 +00:00
Vincent Ambo
e8dcdceb34 fix(tvix/eval): compile but don't emit dead code
This adds a mechanism to the compiler to compile an expression without
emitting any code. This allows for detected dead code to still be
compiled to detect errors & warnings inside of it.

Change-Id: Ie78479173570e9c819d8f32ae683ce34234a4c5d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7767
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-06 12:23:55 +00:00
Vincent Ambo
6a8541e35a feat(tvix/eval): implement initial compiler::optimiser module
This optimiser can rewrite some expressions into more efficient forms,
and warn users about those cases.

As a proof-of-concept, only some simple boolean comparisons are
supported for now.

Change-Id: I7df561118cfbad281fc99523e859bc66e7a1adcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7766
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-06 12:23:55 +00:00
Vincent Ambo
36e5a4cc07 refactor(tvix/eval): take owned ast::Expr in Compiler::compile
This adds a very minimal amount of additional Rc-increments (~1 per
compilation), but makes it a lot easier to add an AST-optimising
compiler pass without incurring a lot of extra cost.

Change-Id: I57208bdfc8882e3ae21c5850e14aa380d3ccea36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7765
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-06 12:23:55 +00:00
Vincent Ambo
0e421a16c5 feat(tvix/cli): add --compile-only "linter" functionality
With this, tvix/cli can be run on files and only produce compiler
errors and warnings.

Change-Id: I5dd9229fc065647787daafd17d7c1540579a1d98
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7764
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-06 12:23:55 +00:00
Vincent Ambo
5926a05f46 feat(tvix/eval): add Evaluation::compile_only method
This would make it possible to implement something like a linter based
on the tvix-eval compiler warnings.

Change-Id: I1feb4e7c4a44be7d1204b0a962ab522fd32b93c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7763
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-01-06 12:23:55 +00:00
Vincent Ambo
27fecee1bc fix(tvix/eval): don't increase with_stack_size in scope inherits
There was probably a misunderstanding somewhere about the
with_stack_size being related to how far away it is from the with, but
it is about whether there is a with at all.

This broke a warning (`UselessInherit`), and may actually have let to
more inefficient codegen in some cases.

Change-Id: I08338ea59ae39dad01ca8a4e09d934a936cdea2f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7762
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-06 12:23:55 +00:00
Vincent Ambo
88432235ae fix(tvix/eval): VM & Builtin* types have to be public
... without them, using the new Builtins API is basically impossible
for library consumers.

Change-Id: Ice0557a2e55e12d812f51bf5a99e6b8c91ad1b91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7755
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-01-06 12:15:23 +00:00
sterni
5946b47ab7 feat(tvix/eval): add builtins.{null,true,false}
Code probably rarely relies on these, but it's not hard to support them.

Change-Id: I8499fec34efaf031f9c013bbd370a13db929a2a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7772
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
2023-01-06 12:14:42 +00:00
sterni
b7090ec874 test(tvix/eval): add test for builtins parity
This will eventually force us to have a base builtins set in common with
C++ Nix, i.e. all 2.3 builtins except the controversial
builtins.valueSize.

Change-Id: I2c767f07d6a14711911658e87da9f18ede57a143
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7747
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-06 12:00:38 +00:00
Florian Klink
95c9c2ae8b feat(tvix/derivation): derive Clone, Debug, Eq and PartialEq
This allows juggling with Derivation structs in unit tests, and makes it
very easy to compare them for equality.

Change-Id: I1faf2ec1eefb1a40fcee3c29f04ec47d29f22691
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7758
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-04 21:58:02 +00:00