Commit graph

18255 commits

Author SHA1 Message Date
Vincent Ambo
a5f28eea94 fix(tvix/eval): print unevaluated thunks like Nix does
Change-Id: Ie4c563e933f571f45cb4f4efe650d1b65f119e8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8324
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2023-03-22 13:44:19 +00:00
sterni
a2c7f7eea1 chore(sterni/machines/edwin): minecraft 1.18.2 -> 1.19.2
Change-Id: I4d002c933929508baab12d3802ac67d42813507b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8333
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-03-21 15:31:07 +00:00
sterni
6a8afb578c chore(sterni/machines/edwin): minecraft 1.18 -> 1.18.2
Starting with 1.18.1 we no longer need to pass an extra flag to work
around the log4j CVE, so baseJvmOpts can be empty.

Change-Id: I6d6c5a366ecbb499b2e3945db81ca0a8b2e2dcbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8332
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
2023-03-21 15:31:06 +00:00
sterni
3c7be2b361 chore(sterni/machines/edwin): minecraft 1.17.1 -> 1.18
Change-Id: If9f6ee1c49a6c427530e56ceff60a2508f6fd9c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8331
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-03-21 15:31:06 +00:00
sterni
2ccaabf9db chore(sterni/machines/edwin): minecraft 1.17 -> 1.17.1
Change-Id: I5e27eb7cfb87cd20a87aa49af114f43aaadc32d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8330
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-03-21 01:31:48 +00:00
sterni
7b7c83e449 feat(sterni/machines/edwin): minecraft 1.16.5 -> 1.17
Change-Id: I3393c16db76fba839ea72612f54e6a2a72d0b69d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8329
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
2023-03-21 01:31:48 +00:00
sterni
012c8e75d0 chore(sterni/machines/edwin): upgrade fabric 0.11.0 -> 0.11.2
Change-Id: I6f833c882e6a5428c9bd94951a5ef5b62ca7c6b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8328
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
2023-03-21 01:31:48 +00:00
sterni
3a74268b6e chore(3p/sources): Bump channels & overlays
* //users/wpcarro/emacs: use top level (ELPA) version of eglot, as it
  was removed from MELPA:

  dc2ead17a8

* //3p/nixpkgs:tdlib: 1.8.11 -> 1.8.12

* //tvix/store: regenerate protos after buf update

Change-Id: I782a8d91fda5ed461788055a3721104e8c032207
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8327
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-03-21 01:31:48 +00:00
Florian Klink
49cb2a2b1f feat(tvix/store/bin): print store path
After ingestion of the contents into the store, this will use the
NonCachingNARCalculationService to create a NAR stream or the contents
of the path, and use our Derivation output path calculation machinery to
determine the output path (using recursive hashing strategy).

In a real-world scenario, we obviously want to cache these calculations,
but this should be sufficient to tinker around with it.

Change-Id: I9b2e69384414f0be1bdcb5a99a4bfd46e8db9932
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8317
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-19 20:57:27 +00:00
Vincent Ambo
80aaadfc19 fix(tvix/eval): use span of set for OpForce in attribute access
Emits the span of the `set` that is being accessed in the `force`
operation of an attribute access.

Looking at traces, it's a lot more useful to get information about
*what* is being forced, as in cases like `foo.bar` it can be
misleading to have an error highlight `bar`, when the error occured
while forcing `foo` to be able to access `bar` in the first place.

Change-Id: Id46ff28f20c67cb4971727ac52cc4811795cea2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8272
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-17 19:40:40 +00:00
Vincent Ambo
8c13f18d11 feat(tvix/eval): report all known spans on infinite recursion
This reports the span

1. of the code within a thunk,
2. of the place where the thunk was instantiated,
3. of the place where the thunk was first forced,
4. of the place where the thunk was forced again,

when yielding an infinite recursion error, which hopefully makes it
easier to debug them.

The spans are tracked in the ThunkRepr::Blackhole variant when putting
a thunk under evaluation.

Note that we currently have some loss of span precision in the VM loop
when switching between frame types, so spans 3/4 are currently a bit
wonky. Working on it.

Change-Id: Icbd2a9df903d00e8c2545b3fc46dcd2a9e3e3e55
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8270
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-03-17 19:32:38 +00:00
Vincent Ambo
3fa6b13c1e feat(tvix/eval): track span of first force in a thunk blackhole
This is step 1 towards being able to use all 4 spans that we know when
dealing with infinite recursion. It tracks the span at which the
force of a thunk was first requested when constructing a blackhole, so
that we can highlight the spans of the first and second forces.

These are actually the least relevant spans, but the easiest to put in
place, more coming soon.

Change-Id: I4c7e82f6211b98756439d4148a4191457cc46807
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8269
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-03-17 19:31:37 +00:00
Vincent Ambo
5095e4f269 feat(tvix/eval): add generator "name" to NativeError kind
This produces traces in which we can see what kind of native code was
run. Note that these "names" are named after the generator message, so
these aren't *really* intended for end-user consumption, but we can
give them saner names later.

Example:
https://gist.github.com/tazjin/82b24e92ace8e821008954867ee05057

This already makes the traces a little easier to parse.

Change-Id: Idcd601baf84f492211b732ea0f04b377112e10d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8268
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-03-17 19:31:37 +00:00
Vincent Ambo
ea80e0d3f8 feat(tvix/eval): enrich errors with VM's frame stack information
When emitting an error at runtime, the VM will now use the new
`NativeError` and `BytecodeError` error kinds (which just wrap inner
errors) to create a set of diagnostics to emit.

The primary diagnostic is emitted last, with `error` type (so it will
be coloured red in terminals), the other ones will be emitted with
`note` type, highlighting the causal chain.

Example:
https://gist.github.com/tazjin/25feba7d211702453c9ebd5f8fd378e4

This is currently quite verbose, and we can cut down on this further,
but the purpose of this commit is to surface more information first of
all before worrying about the exact display.

Change-Id: I058104a178c37031c0db6b4b3e4f4170cf76087d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8266
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-17 19:31:37 +00:00
Vincent Ambo
b78ae941a4 fix(tvix/eval): use coerce_to_string in builtins.substring
This actually uses coercion under the hood in C++ Nix. See the test
for an example.

Change-Id: Id56b364acf269225b6829d0b600e0222f8b3608d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8322
Reviewed-by: andi <andi@notmuch.email>
Tested-by: BuildkiteCI
2023-03-17 19:17:29 +00:00
Griffin Smith
68d287fd82 feat(grfn/gws.fyi): pronouns in bio
Change-Id: Iba14dcec42aba3195dbe28a1cf31acd66919f1b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8323
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2023-03-17 18:59:29 +00:00
Vincent Ambo
b5f5a1595d chore(tvix/eval): remove some dead code
This was commented out and forgotten during the generator refactor, oh
well.

Change-Id: I474b685159a955a846db462da0dd0067af177b04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8321
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-17 11:01:07 +00:00
Florian Klink
b025ebb2a1 refactor(tvix/store/nar): pass in &proto::node::Node
Passing in a &proto::node::Node into all this allows us consumers to
keep ownership of the proto::node::Node.

Change-Id: I44882a86c46826b06a8a8a0b24c18adfc7052662
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8316
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-17 07:49:38 +00:00
Florian Klink
985f842e32 chore(tvix/store/grpcblobsvc): clippy lint
Change-Id: Idb78e0417a962599cdfdef5e7346f7fa41e3fa1b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8320
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-17 07:49:38 +00:00
Florian Klink
23ddf49229 chore(tvix/store/chunkservice): clippy lint
Change-Id: I2073408fe6e87ffadbeb35ef2f9aaa0316c54966
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8319
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-17 07:49:37 +00:00
Florian Klink
7a966b2540 chore(tvix/store/blobreader): clippy lint
Change-Id: I4c85425782878295277abe6dc91a1b570768f7fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8318
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-17 07:49:37 +00:00
Florian Klink
530cb920b5 refactor(tvix/store/chunksvc): use [u8; 32] instead of Vec<u8>
Change-Id: Ie2b94aa5d69ff2c61fb77e13ae844f81f6270273
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8314
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-16 13:47:32 +00:00
Florian Klink
ee23220564 refactor(tvix/store/directorysvc): use [u8; 32] instead of Vec<u8>
Also, simplify the trait interface, only allowing lookups of Directory
objects by their digest.

Change-Id: I6eec28a8cb0557bed9b69df8b8ff99a5e0f8fe35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8313
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-16 13:47:32 +00:00
Vincent Ambo
9c08cbc973 chore(tvix): run cargo update
Change-Id: I710f0920da118cb1b94cd8ff364adadbeef998bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8315
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-03-16 13:44:57 +00:00
Florian Klink
def2d32319 refactor(tvix/nix-compat): remove get_fixed_output() func
The behaviour of this function is a bit unintuitive, and cl/8310 already
inlined the other consumer of it.

Rewrite the last consumer of the function, so we can drop it.

Change-Id: I59c8486037ce3f777667d1d9e4f4a9316d5a0cb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8311
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-15 15:31:55 +00:00
Florian Klink
09b30f895a refactor(tvix/nix-compat): simplify fingerprint calculation
Instead of having two very similar match branches for the FOD and non-
FOD case, detect the FOD case while looping over all outputs.

In the case of anything other than recursive sha256 FODs, the
fingerprint and output path calculation is exactly the same.

Change-Id: Ieb6995653d008766e595cf29d7cd4fb1334e33dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8310
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-15 14:02:24 +00:00
Vincent Ambo
4d465249ef fix(tvix/cli): populate output configuration after walking args
Walking the arguments might encounter an `outputs` output, which might
explicitly (for whatever reason) specify the `out` output.

To prevent dropping FOD settings in this case, we have to populate
that part of the configuration after walking the other attributes.

Change-Id: Iee6a7f0a71e9c9699e79d35e6cb19e1ddb49395d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8312
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-15 13:05:53 +00:00
Florian Klink
b55d1f97ce refactor(tvix/nix-compat): -derivation::Hash, +NixHash
This stops using our own custom Hash structure, which was mostly only
used because we had to parse the JSON representation somehow.

Since cl/8217, there's a `NixHash` struct, which is better suited to
hold this data. Converting the format requires a bit of serde labor
though, but that only really matters when interacting with JSON
representations (which we mostly don't).

Change-Id: Idc5ee511e36e6726c71f66face8300a441b0bf4c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8304
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-15 11:50:12 +00:00
Vincent Ambo
e82385dbe5 style(tvix): add new logo to the Tvix README
Change-Id: Ibb8061be75d6fd16e8f813bc94e3bacaae890454
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8309
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
2023-03-14 22:44:18 +00:00
Florian Klink
36bed4ce57 chore(tvix/store): drop separate bin package
Apparently, having multiple packages with the same path is a bad thing:

```
The bin target `tvix-store` in package `tvix-store-bin v0.1.0 (/home/flokli/tvl/tvix/store)` has the same output filename as the lib target `tvix_store` in package `tvix-store-bin v0.1.0 (/home/flokli/tvl/tvix/store)`.
Colliding filename is: /home/flokli/tvl/tvix/target/doc/tvix_store/index.html
The output filenames should be unique.
This is a known bug where multiple crates with the same name use
the same path; see <https://github.com/rust-lang/cargo/issues/6313>.
```

Change-Id: Ic785c0349070783baf5e8fd23f5fb60603a3c995
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8308
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 22:23:10 +00:00
Florian Klink
0094a1442a fix(tvix): build all tvix crate docs
There's still some shadowing going on, but that's left for a followup
CL.

Change-Id: I02992f1eb494faca99857a3a5ee4dcd47f1b9fd0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8306
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-14 22:23:10 +00:00
Vincent Ambo
0965600fe6 feat(ops): serve Tvix website & docs on (docs.)tvix.dev
Change-Id: I198ea197867f9b9a48e51665d0665f722202e02e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8299
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-14 22:10:40 +00:00
Vincent Ambo
231ed4c11e feat(tvix): build (some of) our Rust documentation in Nix
This will be published on docs.tvix.dev

Change-Id: I348e057351d5295ad20953c9e9a32c257abab089
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8298
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-03-14 21:59:40 +00:00
Vincent Ambo
790eac5a5c feat(ops/glesys): add CNAME for docs.tvix.dev
Change-Id: Ie1994ac4d14344c82ae184f4e3cd9f5292d96c84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8297
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-14 21:59:40 +00:00
Vincent Ambo
19c452875e feat(tvix): add simple initial website
The website itself contains no useful information that we don't
already have published anywhere, but it's a decent landing place.

Mostly I want to have something on the root of tvix.dev, so that we
can start using it for other purposes.

Change-Id: Id2b4dabc7f6e4dd26b61484b86dbde2f39aa1719
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8296
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-03-14 21:59:40 +00:00
Vincent Ambo
3370724062 feat(ops/glesys): point tvix.dev at whitby
Change-Id: Ied022e6c1a8039a9db375a8593afb76edcaa6dbd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8295
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-14 21:59:40 +00:00
Florian Klink
a14880903c refactor(tvix/nix-compat): rename helper func
This only trims the output paths from a Derivation struct, not the
output hashes.

Change-Id: I9250fec4602ed05bb64540c4a89ddb6fb052be1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8303
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-14 20:53:12 +00:00
Florian Klink
84a8ed265c refactor(tvix/nix-compat): remove manual map
Change-Id: I1652e24c9be28112d98683de2d2db51dc46001ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8302
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-14 20:53:12 +00:00
Florian Klink
c699ace11b refactor(tvix/nix-compat): rename test name
Change-Id: Ia6af8d4b53fbd4d9ea26f0094cd98dde66a91f78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8301
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-14 20:53:12 +00:00
Florian Klink
075dcbf1e0 fix(tvix/store/derivation): fix comment
The digest is encoded in LOWERHEX here, not nixbase32.

Change-Id: I8e14742f0135966424073bfdb93ada8cd7bd52e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8300
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 20:53:12 +00:00
Florian Klink
8cfd303203 chore(tvix/cli): don't set COREPKGS anymore
Since cl/8213, tvix bundles corepkgs.

Change-Id: I17fa4452a6a1d554ae67b4aed9d9b1e77cb495f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8305
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-14 20:50:01 +00:00
Florian Klink
5f260edf7f refactor(tvix/nix-compat): replace calculate_drv_replacement_str
Call this function derivation_or_fod_hash, and return a NixHash.

This is more in line with how cppnix calls this, and allows using
to_nix_hash_string() in some places.

Change-Id: Iebf5355f08ed5c9a044844739350f829f874f0ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8293
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-14 18:26:19 +00:00
Adam Joseph
32999cb6f6 docs(tvix/eval): suggested layout adjustment to VM loop diagram
Change-Id: I5467cd66801ad8fe6c4ec0ae337763f1762cea1c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8252
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-03-14 09:56:28 +00:00
Vincent Ambo
466e6dc265 docs(tvix/eval): document inner workings of the new VM loop
Change-Id: I8224bf039f739c401900b5a2ddc839810c87cf6e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8226
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-03-14 09:22:32 +00:00
Vincent Ambo
d456705352 chore(tvix): Generator{Request|Response} -> VM{Request|Response}
We settled on this being the most reasonable name for this construct.

Change-Id: Ic31c45461a842f22aa05f4446123fe3a61dfdbc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8291
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-14 09:22:22 +00:00
Adam Joseph
1e80b9ea8b chore(tvix/eval): mark async functions which are called by the VM
Given Rust's current lack of support for tail calls, we cannot avoid
using `async` for builtins.  This is the only way to avoid
overflowing the cpu stack when we have arbitrarily deep
builtin/interpreted/builtin/interpreted/... "sandwiches"

There are only five `async fn` functions which are not builtins
(some come in multiple "flavors"):

- add_values
- resolve_with
- force, final_deep_force
- nix_eq, nix_cmp_eq
- coerce_to_string

These can be written iteratively rather than recursively (and in
fact nix_eq used to be written that way!).  I volunteer to rewrite
them.  If written iteratively they would no longer need to be
`async`.

There are two motivations for limiting our reliance on `async` to
only the situation (builtins) where we have no other choice:

1. Performance.

   We don't really have any good measurement of the performance hit
   that the Box<dyn Future>s impose on us.  Right now all of our
   large (nixpkgs-eval) tests are swamped by the cost of other
   things (e.g. fork()ing `nix-store`) so we can't really measure
   it.  Builtins tend to be expensive operations anyways
   (regexp-matching, sorting, etc) that are likely to already cost
   more than the `async` overhead.

2. Preserving the ability to switch to `musttail` calls.

   Clang/LLVM recently got `musttail` (mandatory-elimination tail
   calls).  Rust has refused to add this mainly because WASM doesn't
   support, but WASM `tail_call` has been implemented and was
   recently moved to phase 4 (standardization).  It is very likely
   that Rust will get tail calls sometime in the next year; if it
   does, we won't need async anymore.  In the meantime, I'd like to
   avoid adding any further reliance on `async` in places where it
   wouldn't be straightforward to replace it with a tail call.

https://reviews.llvm.org/D99517

https://github.com/WebAssembly/proposals/pull/157

https: //github.com/rust-lang/rfcs/issues/2691#issuecomment-1462152908
Change-Id: Id15945d5a92bf52c16d93456e3437f91d93bdc57
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8290
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-03-13 21:33:58 +00:00
Adam Joseph
e7a534e0c6 refactor(tvix/eval): reduce fetch{forced|captured}_with visibility
This commit moves fetch_forced_with and fetch_captured_with into the
scope of their only caller (resolve_with).

Change-Id: I9a8bc27228888729d591e8cb021c431b2b6468f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8289
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-13 21:33:58 +00:00
Adam Joseph
47895c4c30 feat(tvix/eval): rewrite nix_cmp_ordering to be nonrecursive
This rewrites nix_cmp_ordering as an iterative loop, which
eliminates the extra pinned-boxing helper function.

Change-Id: I33d0ecc913e02affd8fd4c7bc1c9ecfdf4c7deb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8288
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-03-13 21:22:12 +00:00
Vincent Ambo
b3f8d66a6a chore(tvix/eval): prune some dependencies & features
* We no longer need backtrace-on-stack-overflow, as we no longer
  overflow the stack with the recent eval refactorings. This was weird
  voodoo anyways, introduced earlier to debug some cases where stack
  overflows occured.

* default features of genawaiter crate are not needed, as we don't use
  their proc macros

Change-Id: I346fc5a18d7f117ee805909a8be8f535b96be76c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8263
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13 20:30:59 +00:00
Vincent Ambo
94513525b9 refactor(tvix/eval): reorder bytecode operations match by frequency
This reorders the operations in the VM's main `match` statement while
evaluating bytecode according to the frequency with which these
operations appear in some nixpkgs evaluations.

I used raw data that looks like this:
https://gist.github.com/tazjin/63d0788a78eb8575b04defaad4ef610d

This has a small but noticeable impact on evaluation performance.

No operations have changed in any way, this is purely moving code
around.

Change-Id: Iaa4ef4f0577e98144e8905fec88149c41e8c315c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8262
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-13 20:30:59 +00:00