This function is inspired by BQN's [⍉] though it is much less elegant
since Nix lacks multi-dimensional arrays. I thought this would be useful
to to avoid multiple `map`s over a single list if we want to return
multiple, separate values from it:
transpose (builtins.map (x: [ (calcA x) (calcB x) ]) myList)
# => [ [ (calcA a) … ] [ (calcB a) … ] ]
While this is quite elegant, it turns out that it is faster to write out
multiple maps:
[ (builtins.map calcA myList) (builtins.map calcB myList) ]
[⍉]: https://mlochbaum.github.io/BQN/doc/transpose.html
Change-Id: Ic333c33af38ab03573b215c9696d75caf2ee18e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11113
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
`#tvl` is the general TVL community channel, `#tvix-dev` is dedicated to
Tvix development discussion.
Change-Id: If899d9ae70dffb4acf6a41ded54f80a1f0551c5d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11110
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
request only contains the outer metadata wrapping, and that's not too
interesting:
> Request { metadata: MetadataMap { headers: {"content-type":
> "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers",
> "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions:
> Extensions }
Drop these fields for now, and rely on the underlying implementations to
add instrumentation for the application-specific fields.
Also, ensure we handle all error cases properly, and log them. We
don't use `err` from instrument, as that'd also log an error on
`Status::not_found`.
Change-Id: Id1b983cb8b059c148c8a376f8802a1d28c59ba97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11103
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This only contains the outer metadata wrapping, and that's not too interesting:
> Request { metadata: MetadataMap { headers: {"content-type":
> "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers",
> "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions:
> Extensions }
Drop these fields for now, and rely on the underlying implementations to
add instrumentation for the application-specific fields.
Clean up the error logging a bit.
Change-Id: Ife1090ed411766a61e1fa60fd4c9570f38de1e98
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11102
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
This only contains the outer metadata wrapping, and that's not too interesting:
> Request { metadata: MetadataMap { headers: {"content-type":
> "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers",
> "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions:
> Extensions }
Drop these fields for now, and rely on the underlying implementations to
add instrumentation for the application-specific fields.
Log errors in some places where we didn't so far.
Change-Id: Ia68d6c526987d3716be62a0809195401cf28512b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11101
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Otherwise this causes conflicts when adding a more recent zstd version
to another crate.
Change-Id: I02d1b1a37f18711c0969cb04e761e967fc4655dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11100
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
In principle we don't want to build any (later) pipeline target during
pipeline evaluation insofar they appear in extraSteps. For this reason,
we have the needsOutput mechanism which prevents the parent target of an
extraStep from being built in 🦙.
Unfortunately, this mechanism is not general purpose enough, as we use
other (i.e. non parent) targets from depot in extraSteps. As a
consequence, kind of expensive builds need to happen during pipeline
construction at the moment. The solution is to use the fact that the
command script we want to run is exposed via the readTree interface to
depot and build the script proper only when the extra step is executed.
To facilitate this, some prerequisite changes need to be made:
- We need to use a symlink different to result in case needsOutput is
true which needs support in mkBuildCommand. We also need to avoid this
symlink being picked up by git, as many extra steps check whether the
tree is dirty or not. (Is there a way to have it outside the depot
tree?)
- Since we rely on the build command printing a single store path we
store in $command_script, we need to avoid it printing two paths
in cases where nix-store(1) is used (nix-store(1) prints the symlink
and readlink(1) would print the store path in a separate line).
Future work would be to remove/deprecate the needsOutput mechanism:
After this change the parent target wouldn't be built right away even if
it appeared in the script via string interpolation. Thus we could,
instead of expecting the target being available as `./result`, make our
extra steps nix-ier.
Change-Id: Idd2e88a865eadabe229ce1e05406e8cc4cb63f94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10850
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
- Set LIBGIT2_NO_VENDOR so that libgit2-sys won't start silently
building the vendored libgit2 version.
- Update to match version distributed by nixpkgs.
- Update related dependencies:
- git2 (reverse dependency)
- bitflags (dependency) which necessitates updating:
- inotify*
- epoll
- libc
Change-Id: I1d9059a16a44b5e02d04f034c8f02bbfdc55b6ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10845
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This introduces a NixOS configuration using config.nix as a NixOS
module, and also checks the firmware is present in the location it's
expected to.
Change-Id: I3ec4333d73fe1b28e10c589c1cf351c59372bb99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11099
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Adds SVG versions of the logo in two colours (using the .ru flag
colours, and using the standard Nix colours).
The flag colours are difficult to use in some contexts because of the
white background of the top, but flokli says the Nix colour version is
"too similar to NixCon". Not yet decided what should go on to swag etc
...
Change-Id: I1e1655e2c7ed4ed6113ddf6b5df76157de5e32cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11093
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
For everything using reqwest here during test cases, we also need to
set SSL_CERT_FILE.
Change-Id: If8aeda65f3d75cb9ac5c9bc64e37a0cb7dffc17c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11092
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
implement From<&nix_compat::nixhash::CAHash> for nar_info::ca::Hash
individually, and make
From<&nix_compat::nixhash::CAHash> for nar_info::Ca a small wrapper that
uses it, as well as the .hash().digest_as_bytes() for the digest.
Change-Id: I7e9b6edd1e3f149eb270faf2928cd846d74e77ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11071
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
If there's an unexpected test failure, print it out, rather than just
saying something is false even though it should be true.
Use .expect() for this, which displays the error if it failed.
We can't use expect_err(), as our stores are not display'able, so use an
assertion with a message there.
Change-Id: I2d88861d979d107edc0717fbdb3cdac9a6bfc5e4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11091
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
This was true while we didn't do string contexts, but drvPath should not
differ anymore.
Change-Id: I58e6ed439519bd4d68259d1577946f3c65110dc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11074
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Some of the docs are still outdated (like architecture and drv path
inconsistencies).
Change-Id: I7a6afceb008ef4cd19a764dd6c637b39fa842a2e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11072
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
The start/end offsets are not necessarily coterminous with the underlying
values array, so even if the stride is fixed, we still we need to slice
the chunks down to match the start/end offsets.
This bug shouldn't affect the correctness of any existing code, since
we're always working with unsliced arrays read directly from Parquet.
Change-Id: I2f7ddc4e66d4d3b2317a44bd436a35bff36bac79
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11081
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
- agenix has not been updated (waiting for
https://github.com/ryantm/agenix/pull/241).
- libgit2_1_5 can be removed (no longer used by cargo-audit).
Change-Id: I96c6a1a4175dc4f2a32b9b2e4ed71caa826a9c42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11077
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
This makes reqwest honor `SSL_CERT_FILE` - previously it was using the
chain bundled in webpki-roots.
`object_store` pulls in `reqwest` with this feature, and the cargo
solver will enable that feature globally as soon as we pull it in, as it
assumes features are additive.
This requires setting `SSL_CERT_FILE` when running tests, otherwise
they'll fail with the unhelpful "NotFound" error.
This was quite some fun to debug, why adding `object_store` to
tvix-castore suddenly made tvix-store tests fail!
Change-Id: I64fc82b4d994715480efdb1ffecb279716456ab9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11090
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
HashingReader wraps an existing AsyncRead, and allows querying for the
digest of all data read "through" it.
The hash function is configurable by type parameter, and we define
B3HashingReader.
Change-Id: Ic08142077566fc08836662218f5ec8c3aff80be5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11087
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
This allows calling .into() to get a B3Digest.
Change-Id: I6e63b496413cd00d84acfcd15c7de0f64c79721f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11086
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
I haven’t thought about how to migrate this when the postgres version
changes, so for now we just pin to 14 and worry about it when we reach
EOL lol
Change-Id: Ib72ec7835c660db636a95fb5af8c9ce40897b1ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11085
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
they switched to TOML & I had a typo in `scolling`
Change-Id: If68408162e7772fbd3b06944ffd9d51dd2134efa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11083
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
This allows peeking at the mode without matching on all enum kinds of
CAHash directly.
Change-Id: Icca147a876f7f4cff0bbaa067ac4dae40d05bee9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11069
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Currently CAHash only deserializes the hash in hex code while
the serializer outputs a nixbase32 hash. This means that you can't currently
deserialize what has been serialized.
This change makes deserialize support any digest format (so hex, nixbase32
and base64) as well as flattens the deserialize code and error handling.
It also implements serde methods of HashAlgo directly using Display and TryFrom
implementations because otherwise these would get serialized as eg. Sha256 instead
of sha256 which also broke CAHash serialize/deserialize.
Change-Id: I1941a72eaec741e4956292adaaf0115b97f260ba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11082
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This doesn't return the digest, but the internal NixHash.
To get the digest, you use `ca_hash.hash().digest_as_bytes()`
(as done in tvix/nix-compat/src/derivation/write.rs).
Change-Id: Ib07918dec63ae42ddf8c8d3f1d10510f9c4df255
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11070
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
The public-consumable thing here is ChunkedReader, not ChunkedBlob.
ChunkedBlob is a helper that can be used to get a new AsyncRead, but
not AsyncSeek. It is used internally by ChunkedReader whenever the
client seeks.
Make this more obvious, by extending the documentation, and putting
ChunkedReader at the top of this file.
Also make ChunkedBlob and its methods private, and give ChunkedReader a
more useful constructor (from_chunks, instead of from_chunked_blob).
Change-Id: I2399867591df923faa73927b924e7c116ad98dc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11079
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
After the previous refactor that made mkBuildExpr use attribute paths,
mkStep and mkDrvMap can now share the code for calculating the attribute
path of any given target.
Change-Id: I50f9687e85c3b4407e4a55ff7e645db882bb80d6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10849
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
mkBuild* is independent of the target abstraction, we can just use
attribute paths. That is useful because we'll want to reuse that code in
order to avoid building derivations (apart from unavoidable IfD) during
pipeline construction for extraSteps.
Breaking Change for mkBuildExpr and mkBuildCommand.
Change-Id: I03646310192087d3e50f358a714472d1ac1a652f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10848
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
When changing the buildkite pipeline code I found that some functions
have a lot of arguments where the order is not necessarily clear. To
ease further refactors / new features, I've ported them over to taking
attribute sets.
Note that this technically is a breaking change, as these functions are
all exposed. Not sure how often they'd be called from the outside,
though.
Change-Id: I118c8c5242922403d12f6e5a61beaf68f636b40a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10847
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
The primary addition is the `builtin_hashFile` function which attempts
value-path coercion and on success reads the file to a nix_string.
Similar to the `builtins_hashString` implementation, the hash is then
derived using the `hash_nix_string` function in the introduced hash
module which has extracted hashing functionality initially specified in
cl/11005.
Change-Id: I5e3ef0317223af99488ebb79efc2fb49b4cbc1b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11007
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Based on the template used by Ocean Sprint and Thaiger Sprint,
initialise the website for the 2024 Nix sprint in Kazan.
This will be published on volgasprint.org, but it's not wired up yet.
Change-Id: I3485084a62a6ea60a90a42b501879092a360612c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11065
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: Mark Shevchenko <markshevchenko@gmail.com>
Tested-by: BuildkiteCI
Simplifies reusing the launch script in other use-cases than the
"official" Nixery image.
Relates to nixery#166
Change-Id: Iaf1dff385ce270792253551081c1b2fca6400037
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11046
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Makes it possible for users to still override this using an envvar.
Relates to nixery#166
Change-Id: Ief2925e03cf2e4351bc38554bf553c8ee259f1f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11045
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
William is rarely active but sometimes has stuff we need to fix for
CI, so I'm removing the blocker on him.
Change-Id: I1de10c0fdf003de2896598519c53e66d29fd9d7e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11035
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
