Adds support for parsing multiple netencode values from stdin.
This is overly complicated for my tastes, but I don’t see a better way
of writing this logic that does not read all of stdin before starting
to parse the first value.
A kingdom for a conduit.
Change-Id: Ia4f849d4096c43e887756b756d2a85d7f9cd380a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6631
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
It was overriding the overrides in haskellPackages, which we didn’t
notice because there is no overrides in the hasell overlay atm.
I also removed the `with` blocks and inlined `overrides`.
Change-Id: Ide72109dbd2568b6cb2435b15024cf1523bec1fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6785
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This was added in preparation for some builtins that don't exist yet
and is producing some noise during compilation.
Change-Id: I51fb0d14c3edf0bd6d9a288d50e44dacf35166c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6769
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This is a little ugly because the plain Iterator::filter method can
not be used (it does not support fallible primitives), so we need to
resort to an `Iterator::filter_map` and deal with the wrapping in
Options everywhere.
This prevents use of `?` which introduces the need for some matching,
but it's not *too* bad.
Change-Id: Ie2c3c0c9756c4c627176f64fb4e0054e717c26d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6765
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
The previous version had a bug where we assumed that the number of
entries in an attribute set AST node would be equivalent to the number
of entries in the runtime attribute set, but due to inherit nodes
containing a variable number of entries, this did not work out.
Fixes b/199
Change-Id: I6f7f7729f3512b297cf29a2e046302ca28477854
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6749
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Fairly straightforward, only thing of note is that we coerce (weakly) to
string here as well.
Change-Id: I03b427e657e402f1f9eb0f795b689bbf5092aba1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6745
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
The simplest solution seems to be to pass references to arithmetic_op!()
which avoids the moving annoyance we had to deal with in the
builtins (no more popping!). We then use .force() to force the values
and dereference any Thunks (which arithmetic_op! doesn't do for us).
Change-Id: I0eb8ad60e80a0b3ba9d9f411e973ef8bcf136989
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6724
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Because `inherit` can take string identifiers, we can access
arbitrarily weird identifiers in scopes using it.
Change-Id: Ic868233221befa160538dd2ffaff1cc7f566585a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6748
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This updates rnix-parser to a version where inherits provide an
iterator over `ast::Attr` instead of `ast::Ident`, which mirrors the
behaviour of Nix (inherits can have (statically known) strings as
their identifiers).
This actually required some fairly significant code reshuffling in the
compiler, as there was an implicit assumption in many places that we
would have an `ast::Ident` node available when dealing with variable
access (which is then explicitly only not true in this case).
Change-Id: I12f1e786c0030c85107b1aa409bd49adb5465546
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6747
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
We delegate to Rust std's Ord trait, so we should at least do a sanity
check here.
Change-Id: I9596068f8ab3e51b79602de0bac79af6d558086e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6723
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Sketch out a new set of "impure" builtins, which supplement the existing
set of "pure" builtins but are gated behind a feature flag, which allows
them to be omitted by crates depending on tvix-eval that only want pure
evaluation, such as tvixbolt.
Change-Id: I2736017b5c9b4776bbba8758e108ec84887abd66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6655
Reviewed-by: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
This was derived from
else if (c1 == "" && n2) return true; // true implies c1 < n2
However, this has no effect since Word always looses out against Number
anyways and the `pre` rules are also unaffected by this change – since
this only affects comparison of an empty Word part with a Number.
Change-Id: Ia04e42ac726352b688c87674b0fdb355f06edbcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6722
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This asserts the not-quite lexicographical property of the comparison.
Change-Id: Iad68081e4b3a7106513f479643de87065dc47739
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6721
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is necessary because builtins.compareVersions compares versions in
a subtly not-quite-but-still-lexicographical way: `pre` for example can
have an effect if it is post-fixed: `2.3 < 2.3pre`. This is a violation
of the rule that in a lexicographical ordering, the longer string is
considered greater if they are otherwise equal. builtins.compareVersion
is comparing lexicographically though, if you do the following
transformation beforehand:
2.3 --split--> [ "2" "3" ] --append--> [ "2" "3" "" ]
2.3pre --split--> [ "2" "3" "pre" ] --append--> [ "2" "3" "pre" "" ]
Comparing the transformed version is then done lexicographically:
2.3 < 2.3.0pre since [ "2" "3" "" ] < [ "2" "3" "0" "pre" ]
Here, the `pre` rule never comes into effect because no comparison on it
happens, instead we use the longer string rule of a lexicographical
comparison.
In the C++ codebase, the reason for this behavior is that the
iterator-esque construct they use always yields the empty string before
it exposes it has been fully consumed. This is probably intentional to
support the postfixed `pre` which is, for example, used by NixOS
versions (e.g. unstable post 22.05 is 22.11-pre). We replicate this
behavior using the `Chain` iterator in `VersionPartsIter::new_for_cmp`.
Change-Id: I021c69aa27b0b7deb949dffe50ed18b6de3a7b1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6720
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is based on the [relevant code] in C++ Nix. Our version has more
branches because the C++ one only checks if it is less than or not, so
can save handling a few cases. We on the other hand, can avoid calling
the algorithm twice. It'd be nice to implement proptests for this in the
future and to make sure that this weird little algorithm doesn't violate
the Ord laws.
[relevant code]: cd35bbbeef/src/libstore/names.cc (L81-L94)
Change-Id: I46642e6da5eac7c0883cdce860622cdba04cd12b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6719
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This was fairly easy, thanks to the work already done by Thomas Frank.
However, the implementation is suboptimal because we parse number parts
only to convert them back to strings afterwards. I've chosen to tackle
this problem in the future, since having an (inefficient) implementation
of splitVersion will be helpful for debugging the slight discrepancies
between C++ Nix and Tvix in the case of compareVersions.
Change-Id: Id6ed8eeb77663ff650c8c53ea952875b1fb7ea84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6688
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This makes it possible to call a callable value (builtin or
closure/lambda) directly, without unwrapping it first. This is needed
for pretty much all higher-order functions to work correctly.
This is mostly equivalent to the previous code in coerce_to_string for
calling `__toString`, except it expects the argument(s) to already be
placed on the stack.
Note that the span for the `NotCallable` error is not currently
guaranteed to make any sense, will experiment with this.
Change-Id: I821224368d438a28900858b343defc1817e46a0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6717
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Generate smaller recursive values for generated Values, and run fewer
cases for the attrs proptests which are particularly egregious.
Change-Id: Ia35c7c120270feaf045be1deb440c87ebb185c27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6716
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
As we already have a VM passed to the builtins, we can simply execute
the provided closure/lambda in it for each value.
The primary annoyance with this is that we have to clone the upvalues
for each element, but we can try making this cheaper in the
future (it's also a general problem in the VM itself).
Change-Id: I5bcf56d58c509c0eb081e7cf52f6093216451ce4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6714
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Previously we only matched the outer constructor after forcing which
would mean that we would always return `false` if the inspected value
was a thunk, regardless what value would be present inside.
Change-Id: I361ea6e855e23ef8e5b59098a50b9cd59253803f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6692
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
There is no need to use an extraStep, actually, and using derivations
reduces noise on CI.
Change-Id: I897c3c3f7e0acee8f051fcc01450ff57176726f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6573
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Previously, this would almost always crash because list items are
thunked more often nowadays and selecting from a thunk would fail. Also
we no longer pop from args, accessing it by index should avoid an
unnecessary clone here.
Change-Id: I4410c4c2e28cc255a2c7cf2a5322db3d2c556a0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6693
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
For some reason Terraform decided that it would otherwise like
to *delete* this configuration, which is undesirable.
Note that there is a "magic" special behaviour when the `alias` and
`provider_id` are set to the name of a built-in supported
provider (github, gitlab etc.), which lets us skip the
authorization_url setup.
Change-Id: Ib66154c2896dda162c57bdc2d7964a9fa4e15f20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6706
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
I think these were set up in the UI and previously not supported in
the Terraform config, now they're supported and Terraform wanted to
delete them ...
Change-Id: I83eb49ceb774ac835dc81638f962e937c7e936c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6707
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Discard string context in prepare-image.nix before parsing input read
with readFile with fromJSON. Required for compatibility with nix >2.3.
Change-Id: I3830707e80fd19a700551a15f1a96d2841d0b022
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6696
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Remove a race condition which appears when uploadHashLayer is called
with the same key from multiple threads simultaneously. This can
easily happen when the same image path is requested by multiple
clients at the same time. When it does, a 500 status is returned and
the following error message is logged:
{
"context": {
"filePath": "github.com/google/nixery/builder/builder.go",
"lineNumber": 440,
"functionName": "github.com/google/nixery/builder.uploadHashLayer"
},
"error": "rename /var/lib/nixery/staging/<hash> /var/lib/nixery/layers/<hash>: no such file or directory",
"eventTime": "...",
"layer": "<hash>",
"message": "failed to move layer from staging",
...
}
To solve this issue, introduce a mutex keyed on the uploaded hash and
move all layer caching into uploadHashLayer. This could additionally
provide a small performance benefit when an already built image is
requested and NIXERY_PKGS_PATH is set, since symlink layers and config
layers are now also cached.
Change-Id: I50788a7ec7940cb5e5760f244692e361019a9bb7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6695
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This is the New Thing that is intended to replace the find-owners
and owners plugins.
In particular:
* It inserts a submit requirement rather than providing a Prolog
predicate.
* The default OWNERS file formats are suspiciously Googley.
* It provides a neat UI for finding OWNERS and tracking approval
state on a per-file basis.
When we fully migrate to using the code-owners plugin, a few
things will need to land, which I will likely do "offline"
directly to the Gerrit backing Git repos:
* Add the corresponding Gerrit config
* Replace OWNERS files depot-wide
* Add OWNERS files to the refs/meta/config branch
* Introduce the Owners-Override label, settable by depot-interventions
The enclosed patch adds two extra pieces of functionality that
we need in tvldepot but aren't upstream:
1. The ability to just specify usernames rather than email addresses
2. The ability to specify `group:GROUPNAME`, _as long as_ that group is
visible to everyone. This is a restriction intended to avoid having
the plugin just leak group membership.
Change-Id: I27d92b6cb7449af83030b9015f09a1571aa8452f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6664
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
And insert the missing newline the C++ Nix test script needs.
Change-Id: I04ddd7268f9caa1414fd23314c281bb7c1e854cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6689
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Instead of arity, we pass a array reference to Builtin::new that
describes how many arguments there are and which of them need to be
forced, eliminating the need to force manually.
Note that this change doesn't fix some of the instances where the the
Builtin doesn't consider that the value could be a Thunk.
Change-Id: Iadb58bb79886c30dc6b09dcf0ffad8abf28036a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6662
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
