Commit graph

18696 commits

Author SHA1 Message Date
Florian Klink
7f5372327c docs(tvix/store): update invocation in README
This has moved to a `daemon` subcommand.

Change-Id: Iae9778d8a59e6bf84555119fabfd62db3917bb62
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8731
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-09 15:33:14 +00:00
Vincent Ambo
0ddd4ecd9d feat(corp/rih): persist data in storage bucket
Change-Id: Ide7a21b97e845681584378d348cc36aba951d69f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8730
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-09 12:21:59 +00:00
Vincent Ambo
aea8c79ca3 chore(corp/ops): pipe secrets through to backend container
Change-Id: Idcaa4a7213b53fe1e818c6a81754d29b6249e957
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8729
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-09 12:21:59 +00:00
Vincent Ambo
eae70200ce feat(corp/ops): configure data storage bucket
Note that there doesn't seem to be a TF resource type for the IAM
binding between the bucket and the service account itself (other than
applying to all buckets in the folder, which I don't want).

For this reason I've added the `storage.uploader` IAM binding to the
`rih-backend` service account *on the bucket* manually.

Change-Id: I9fb06c7857e61dc642d9ea0d89159a0e343dc984
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8728
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-09 12:21:59 +00:00
Vincent Ambo
fb7db9b692 feat(corp/ops): configure hosting setup for backend domain
Change-Id: Ia0298e3be2e16ac5dbc2b8aec1e840aa3af947e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8719
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-09 12:21:59 +00:00
Vincent Ambo
1e70cd1c4f feat(corp/ops): initial resources for running rih container
Change-Id: Ie7276396bbfcff64d91ca8cc655cad6927fd6599
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8718
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-09 12:21:59 +00:00
Florian Klink
b78a9b7fe1 docs(tvix/store/docs): fix grammar
Change-Id: I1d7d306e5d997a7ba47a83d613edc19a4fba1ed4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8721
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-06-09 09:10:51 +00:00
sterni
a2303da01f fix(tvix/eval): use normal thunking behavior for default in formals
When comparing to C++ Nix, we notice that the thunking of default
expressions in function formals corresponds to their normal thunking,
e.g. literals are not thunked. This means that we can just invoke
compile() without much of a care and trust that it will sort it out
correctly.

If function formals blow up as a result of this, it likely indicates
that the expression is treated incorrectly by compile(), not
compile_param_pattern().

Change-Id: I64acbff2f251423eb72ce43e56a0603379305e1d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8704
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-07 15:19:21 +00:00
sterni
10c6cb7251 fix(tvix/eval): type check function argument with set pattern
C++ Nix forces and typechecks the passed argument even if it is not
necessary in order to compute the return value of the function. I
discovered this when I thought our formals miscompilation might be that
we are too strict, but doesn't look like it in this case.

Change-Id: Ifb3c92592293052c489d1e3ae8c7c54e4b6b4dc6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8701
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-07 15:17:20 +00:00
sterni
617130b088 refactor(tvix/eval): don't track idx twice in compile_param_pattern
Change-Id: I27f9105ddb20d84342550b2a73b479a7764ee3fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8699
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-06-07 15:17:19 +00:00
Griffin Smith
e6a37b27b0 feat(grfn/gws.fyi): Link to bluesky profile
Change-Id: I745f3761c4c91c43bbb7edff946244da242eb48c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8717
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2023-06-06 19:40:20 +00:00
Griffin Smith
ddff7ffab3 fix(grfn/system): Set docker storageDriver on mugwump
A recent upgrade is requiring this now for some reason.

Change-Id: I5908c12d104044897b4969dbd1299a3071d507bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8716
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: grfn <grfn@gws.fyi>
2023-06-06 19:40:20 +00:00
Linus Heckemann
6eedd159ac feat(docs): mention OIDC providers in registration instructions
Change-Id: I698c7b6b1f5f5202c49fcbe026782eb4692d99d3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8706
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-06-06 13:15:19 +00:00
Vincent Ambo
896034cc94 chore(corp/rih): bootstrap backend application with rouille
Change-Id: I48b0c432a843db5a44b3a3de09f18e1232644a44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8715
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-06 11:43:00 +00:00
Vincent Ambo
5dee4780da chore(corp/rih): move frontend to a separate folder
Change-Id: Ic7467f459015c39c73f87c61a048319eaf1243be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8714
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-06 11:43:00 +00:00
Vincent Ambo
6fa6f3a7f4 feat(corp/rih): submit form data to backend
Change-Id: I0c74deea8debf9acbcf6eabf225969dbfe9cc34c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8703
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-06 11:43:00 +00:00
Vincent Ambo
20a6710365 feat(corp/rih): activate submit button when data is complete
The completion detection is very naive for now, but baby steps etc.

Change-Id: Iec6eae77fca13f3fc0ee1225268eef04277b504a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8702
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-06 11:43:00 +00:00
Vincent Ambo
138f1ca1b9 feat(corp/rih): wire up captcha solving callback
This turned out a lot nicer than I expected it to be.

Change-Id: I427670644eba789ea2037423fa9af8e632b19b34
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8695
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-06 11:43:00 +00:00
Vincent Ambo
6f0ddbac06 feat(corp/rih): display & load captcha element inside the form
Change-Id: Ifd0f85d9e4f785c4cb1ae56ae67e6d999ff43c85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8694
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-06 11:43:00 +00:00
Vincent Ambo
b2ff63586a chore(corp/rih): reverse form order
This makes more sense to me.

Change-Id: I013bf9457f20a31a9762768607f4094358e1b7cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8693
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-06 11:43:00 +00:00
Vincent Ambo
ad0fc6549e fix(tazjin/emacs): set font weight back to normal
Whatever broke font rendering in nixpkgs briefly has been fixed.

Change-Id: Ie14aa3e4893c2828b57deacda125561202628b26
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8713
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-06 11:19:16 +00:00
Vincent Ambo
ff96777456 subtree(3p/exwm): update to commit 'b62d5e79'
This is almost one year of changes to EXWM. Note that it undoes our
port of https://github.com/ch11ng/exwm/pull/737

That PR hasn't seen any movement in three years, so it might not be
that relevant anymore. Other stuff has been mainlined in the meantime.

Change-Id: I0845ff8a28a5bb1553855f6d6f0ceeaedcf0809e
2023-06-06 00:40:58 +03:00
Florian Klink
4ddfdc2ae0 feat(nix/buildkite): extraSteps: include parentLabel into hash
Contrary to the normal steps, extra steps only show the attribute name
in their label, not the whole attribute path.

To make sure these are still unique, also incorporate the parentLabel
into the string that's hashed.

Change-Id: Ic7a1f76aaac159ab5bd30054e422234882947990
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8708
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-03 14:09:28 +00:00
Florian Klink
ed159e3a1d feat(nix/buildkite): add step key to release steps
This uses the same logic ("sha1sum of label") as already used in the
"build" chunks.

Fixes https://b.tvl.fyi/issues/276.

Change-Id: I76e59a141bd05f7d1d997a6471ea65e8cc7b4c2a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8707
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-03 13:47:22 +00:00
Linus Heckemann
26fed61750 fix(tvix/cli): fix refscan when no paths are referenced
Before, the construction of a TwoByteWM would panic when no patterns
were provided, as in `tvix --expr 'builtins.toFile "snens" "soos"'`.

Change-Id: I25ed498c475523aec5baf8683b23059fadabb21c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8697
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-02 13:19:56 +00:00
Florian Klink
4cf0ee8ea3 docs(nix/buildkite): escape "release" in warning
When using the buildkite tooling with `postBuild` parameter set, the
following error message is shown:

```
This replaces the previous boolean postBuild API in extra
step definitions. Please remove the postBuild parameter from
this step and instead set phase = release;.
```

However, `release` is no symbol, this should be a string.

Update the message to reflect that.

Change-Id: I341c81d862204db06946512428910cada28481ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8690
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-01 10:34:12 +00:00
Vincent Ambo
70b87c1797 feat(corp/ops): configure bucket hosting configuration
This doesn't have redirects for weird routes yet, but I think that's
doable somehow.

Change-Id: Iaaac711304f9b2bd8ea04302940e9e9259cd67c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8663
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2023-05-31 11:43:17 +00:00
sterni
830bc9961a chore: remove comments formerly meant for alignment
/**/ is a nice way to align if statements which doesn't work with
nixpkgs-fmt, since it'll reflow the comment to the line preceding the
if. Consequently, we can delete these comments now.

Change-Id: Ifa5327f846a903e07607b21f8eedbc32fc36f758
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8689
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-31 10:55:07 +00:00
Florian Klink
2b0b4cadbd feat(tvix/store/bin): use sled for daemon, grpc for import cmd
This now creates different store client, depending on the cli
subcommand.
The `import` command will connect to the gRPC service, and the `daemon`
command will use the sled implementation.

It might make sense to define some URI syntax to make this configurable
by the user, via the CLI.

Change-Id: I72351fcf0e83a013b6aa67a90b64c108cbb01ffd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8619
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-30 18:10:29 +00:00
Vincent Ambo
e3c5eb3313 chore(3p/sources): bump nixpkgs channels & overlays
* 3p/buzz: delete package

  This is unused, old, and uses an insecure version of OpenSSL which
  broke eval.

* 3p/overlays: remove nvd patch

  Newer versions of nvd are compatible with Nix 2.3 by default.

* users/grfn/system/home: remove explicit rust-analyser package

  This conflicts with `rustup`, causing eval failures, as the wrapper
  seems to now be included in `rustup` by default.

* users/grfn/system: temporarily disable ISO builds

  They were broken upstream in nixpkgs by a change to stdenv/setup.sh,
  being fixed in https://github.com/NixOS/nixpkgs/pull/234883

Change-Id: I0eea99ec98f4e73e615c012ffae1d0e37122e73b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8585
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2023-05-30 13:27:52 +00:00
Florian Klink
48b66a8982 feat(tvix/store/bin/import): process all path imports concurrently
Change-Id: I3e1428a4725fc2e552e8f37bc0550121117fcef6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8633
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-30 10:13:07 +00:00
Florian Klink
dbff1289b8 refactor(tvix/store): drop Clone from BS and DS trait bound
We don't need to be able to clone these services in here.

Change-Id: Ifb69450f7ebdc8364cbf9cdfb6464f8560440e4c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8645
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-05-30 10:13:07 +00:00
sterni
b492067c19 test(tvix/eval): check thunking behavior of basic exprs
nix_oracle.rs now gives us the possibility to check this by stuffing the
expressions in a list. In fact, the incorrect behavior fixed in

- cl/8656
- cl/8655
- cl/8662

was discovered using this test suite.

Change-Id: Id0ab01ee6be0b875791214e0a72a2ac941c46c96
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8658
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-29 19:36:42 +00:00
sterni
0ab6494286 refactor(tvix/eval/nix_oracle): allow specifying eval strictness
This will be useful for comparing thunking behavior to C++ Nix. I
considered adding this capability to the tvix_tests/nix_tests
infrastructure, but as it would require changing the test file naming
scheme to do it in a clean way, I've postponed it–it's nice that our
tests are compatible with C++ Nix's test suite.

Change-Id: I60bcdd98ed25140e716f0858f8dc28f21ab957aa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8657
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-29 19:36:41 +00:00
sterni
d09f333d0e fix(tvix/eval): thunk lambda expressions
As cl/8658 and b/274 reveal, lambda expressions are also wrapped in
thunks.

Resolves b/274.

Change-Id: I02fe5c8730ac76748d940e4f4427116587875275
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8662
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-29 19:36:41 +00:00
sterni
2aab01ac29 fix(tvix/eval): thunk HasAttr expressions
HasAttrs was weird because with longer attribute paths it would
sometimes not turn out to be a thunk. If it was a thunk, it'd usually
still do some eval strictly which we'll want to avoid.

Verified against C++ Nix using a new test suite introduced in a later
CL.

Change-Id: I6d047ccc68d046bb268462f170a3c4f3c5ddeffe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8656
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-29 12:44:09 +00:00
sterni
9d0425acc0 fix(tvix/eval): thunk legacy let to match regular one
Probably no real world code broken by this overzealous evaluation, but
let's be thorough!

Change-Id: Ib405a677182eab7940ace940c68e107573473a54
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8655
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-29 12:44:08 +00:00
sterni
385c797884 fix(tvix/eval): thunk unary operator applications
Unary operator applications are thunked which can easily be observed by

  nix-instantiate --eval -E '[ (!true) (-1) ]'

Unfortunately, there are few simple expressions where this makes a
difference in the end result. Thus it only cropped up when using nixpkgs
for cross compilation: Here we would compile the expression

  !(stdenv.cc.isGNU or false)

to assemble python3Minimal's passthru attribute set (at least this seems
to be the most likely explanation from the backtraces I've studied).
This means that an unthunked

    <stdenv.cc.isGNU or false>
    OpForce
    OpInvert

would be performed in order to assemble this attribute set, causing
stdenv.cc to be evaluated too early, causing an infinite recursion.

Resolves b/273.

It seems that having a test suite that doesn't use --strict and relies
on thunks rendered as <CODE> would be beneficial for catching such
issues. I've not been able to find a test case with --strict that
demonstrates the problem fixed in this CL.

Change-Id: I640a5827b963f5b9d0f86fa2142e75e3a6bbee78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8654
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-29 12:44:08 +00:00
Florian Klink
93fa47f2ae feat(tvix/cli): add --no-warnings argument
This will prevent tvix from printing any warnings.

As a followup, we can also thread this parameter through into the
evaluator itself, to prevent warnings from being constructed in first
place.

Change-Id: I15381396f86573484bdd1a73d09034a665638e35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8646
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-05-27 16:44:29 +00:00
Vincent Ambo
c7e7f6d682 feat(corp/rih): implement routing support for privacy policy
Mounts the privacy policy at `/privacy-policy`. Using yew_router
"properly" is difficult in components that don't make use of macros
and context magic, so I've opted to use the gloo history handling
directly to parse the location here.

Change-Id: Icde11485f9947bc860a7b2c43772bb0f4cdf2ea1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8653
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-27 11:40:41 +00:00
Vincent Ambo
f2ed94b9d7 feat(corp/rih): add initial privacy policy
Change-Id: I0c5bade3521300d1a9d91009686770b845f88d74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8652
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-27 11:40:41 +00:00
Vincent Ambo
9c7da22e5b feat(corp/ops): initial hosting bucket & TLS configuration
Doesn't actually have bucket serving or access configuration yet, one
step at a time!

Change-Id: I0ce9b3b077252395bd807fad44cbdca40cdeac49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8649
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-27 11:40:41 +00:00
Vincent Ambo
d419b81ef7 feat(corp/rih): add macro to render Markdown->yew::Html statically
This makes it possible to embed long texts from Markdown files instead
of dealing with writing the weird HTML-tags inside the yew macros,
which will be much easier for content editors to deal with.

Change-Id: Idc4e67404fcfe2b8d5083cf556df1c701ba17660
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8648
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-27 11:40:41 +00:00
Vincent Ambo
99c7896637 feat(corp/rih): implement initial frontend application
This doesn't actually submit anything to the (not-yet-existing)
backend, but will help the designers figure out what we're actually
looking for here.

Change-Id: I680d88151fb0706953f18eb6256da6f205da7ffb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8489
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-27 11:40:41 +00:00
sterni
3b33c19a9c fix(tvix): don't call function eagerly in genList, map & mapAttrs
mapAttrs, map and genList call Nix functions provided by the caller and
store the result of applying them in a Nix data structure that does not
force all of its contents when forced itself. This means that when such
a builtin application is forced, the Nix function calls performed by the
builtin should not be forced: They may be forced later, but it is also
possible that they will never be forced, e.g. in

    builtins.length (builtins.map (builtins.add 2) [ 1 2 3 ])

it is not necessary to compute a single application of builtins.add.

Since request_call_with immediately performs the function call
requested, Tvix would compute function applications unnecessarily before
this change. Because this was not followed by a request_force, the
impact of this was relatively low in Nix code (most functions return a
new thunk after being applied), but it was enough to cause a lot of
bogus builtins.trace applications when evaluating anything from
`lib.modules`. The newly added test includes many cases where Tvix
previously incorrectly applied a builtin, breaking a working expression.

To fix this we add a new helper to construct a Thunk performing a
function application at runtime from a function and argument given as
`Value`s. This mimics the compiler's compile_apply(), but does itself
not require a compiler, since the necessary Lambda can be constructed
independently.

I also looked into other builtins that call a Nix function to verify
that they don't exhibit such a problem:

- Many builtins immediately use the resulting value in a way that makes
  it necessary to compute all the function calls they do as soon as
  the outer builtin application is forced:

  * all
  * any
  * filter
  * groupBy
  * partition

- concatMap needs to (shallowly) force the returned list for
  concatenation.

- foldl' is strict in the application of `op` (I added a comment that
  makes this explicit).

- genericClosure needs to (shallowly) force the resulting list and some
  keys of the attribute sets inside.

Resolves b/272.

Change-Id: I1fa53f744bcffc035da84c1f97ed25d146830446
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8651
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-26 22:35:39 +00:00
Florian Klink
a4d0a80466 refactor(tvix/store/bin): instantiating TvixStoreIO once
Instead of instantiating it once in every loop iteration, put it in an
Arc, and clone that before passing it to the spawned task.

Change-Id: I5d9c838f27048726166fa50206d1edd5ed6849b5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8632
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-25 12:35:59 +00:00
Vincent Ambo
22776280b5 feat(tvix/eval): unthunk empty lists and attribute sets
Change-Id: Ie66cb1b163a544d45d113fd0f866286f230b0188
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7960
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2023-05-25 11:28:13 +00:00
Vincent Ambo
5d3fb33baa feat(tvix/eval): implement unthunking in compiler
This feature allows the compiler to detect situations where the
created thunk is useless and can be merged into the parent chunk
instead.

The only case where the compiler does this initially is when
statically optimising a select expression.

For example, previously the expression `builtins.length` compiled into
two thunks:

1. An "inner" thunk which contained an `OpConstant` that had the
   optimised `length` builtin in it.

2. An "outer" thunk which contained an `OpConstant` to access the
   inner thunk, and the trailing OpForce of the top-level program.

With this change, the inner thunk is skipped completely and the outer
chunk directly contains the `length` builtin access.

This can be applied in several situations, some easier than others,
and we will add them in as we go along.

Change-Id: Ie44521445fce1199f99b5b17712833faea9bc357
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7959
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-05-25 11:28:13 +00:00
Vincent Ambo
3aea9bf527 feat(tvix/eval): implement Chunk::extend method
This method extends the contents of one chunk with that of another,
effectively merging the thunks together.

This will be used for the upcoming "unthunking" functionality.

Change-Id: I6ad74232cd7f3eca198ed921e455205e00d76e6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7958
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-05-25 11:28:13 +00:00
Florian Klink
d25962b9a4 refactor(tvix/eval): stop borrowing &mut self
This does undo cl/8571.

Change-Id: Ib14b4e7404f906e346304b6113860ae811afc94a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8631
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-05-25 11:11:59 +00:00