Commit graph

19880 commits

Author SHA1 Message Date
sterni
b40c4500a3 chore(net/crimp): update dependencies
Change-Id: I30b945859347d8f9b0c6947ecc278a3e9cb7ec2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11133
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-03-11 16:52:45 +00:00
sterni
438db1be30 fix(net/crimp): correctly set content length for PUT requests
Since https://github.com/curl/curl/commit/9c845be2797e20475
(presumably), libcurl will overwrite our previously set request
method to POST if we set .post_field_size(…). The fix is to use
the proper option for PUT/upload, .in_filesize(…). While we're
at it, switch to using .upload(…) instead of the deprecated
.put(…) which should be the same for HTTP.

Change-Id: I393c1a02c70d5b99dff5901cd6e9d9434f68c15b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11132
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2024-03-11 16:36:36 +00:00
Aspen Smith
54609e8c17 feat(tvix/glue): Add AsyncRead wrapper to decompress streams
Add a new AsyncRead wrapper, DecompressedReader, that wraps an
underlying AsyncRead, but sniffs the magic bytes at the start of the
stream to determine which compression format is being used out of the
three that are supported by builtins.fetchTarball, and switches to the
correct decompression algorithm adapter dynamically.

This will be used in the implementation of builtins.fetchTarball

Change-Id: I892a4683d5c93e67d4c173f3d21199bdc6605922
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11019
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-03-11 11:35:18 +00:00
Aspen Smith
de727bccf9 feat(tvix/glue): Implement builtins.fetchurl
Implement the fetchurl builtin, and lay the groundwork for implementing
the fetchTarball builtin (which works very similarly, and is implemented
using almost the same code in C++ nix).

An overview of how this works:

1. First, we check if the store path that *would* result from the
   download already exists in the store - if it does, we just return
   that
2. If we need to download the URL, TvixStoreIO has an `http_client:
   reqwest::Client` field now which we use to make the request
3. As we're downloading the blob, we hash the data incrementally into a
   SHA256 hasher
4. We compare the hash against the expected hash (if any) and bail out
   if it doesn't match
5. Finally, we put the blob in the store and return the store path

Since the logic is very similar, this commit also implements a *chunk*
of `fetchTarball` (though the actual implementation will likely include
a refactor to some of the code reuse here).

The main thing that's missing here is caching of downloaded blobs when
fetchurl is called without a hash - I've opened b/381 to track the TODO
there.

Adding the `SSL_CERT_FILE` here is necessary to teach reqwest how to
load it during tests - see 1c16dee20 (feat(tvix/store): use reqwests'
rustls-native-roots feature, 2024-03-03) for  more info.

Change-Id: I83c4abbc7c0c3bfe92461917e23d6d3430fbf137
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11017
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: aspen <root@gws.fyi>
2024-03-11 02:21:54 +00:00
Aspen Smith
83ad32c481 feat(aspen/emacs): Configure direnv
Change-Id: I541620448137c7e7443d0a7f9c10509fab02bdea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11131
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
2024-03-11 01:58:35 +00:00
Aspen Smith
2466ecaf31 feat(aspen/emacs): Use hotfuzz for completion
This is like orderless but a million times better (no need for spaces!)

Change-Id: Id8d2382e1d83792959580bcbe807cdd9aebdde88
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11130
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
2024-03-11 01:58:35 +00:00
Aspen Smith
a0225431de feat(aspen/emacs): Some more rust bindings
Change-Id: Iabdd9f75438e83e73c7e423a3646d72b9bbe5b83
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11129
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
2024-03-11 01:58:35 +00:00
Aspen Smith
58379302f4 feat(aspen/emacs): Make vertico's faces look nicer with solaire
Change-Id: I7b3befb9f9607a301eaddff9808cdd27dabbfffe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11128
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
2024-03-11 01:58:35 +00:00
sterni
a95ce06cf9 fix(users/wpcarro/nixos): ava's not tarasco
Change-Id: Ib9e28120556abdd81dd17e478617f3a84c0d554f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11125
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-03-10 21:48:05 +00:00
Vincent Ambo
13b0f80e4f feat(treecrumbs): support for TOML
Similar to YAML, but with more weird keys and with sections.

Change-Id: I94d69d2f0e53366855ccb5d99a432c6a0361a910
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11126
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2024-03-10 19:35:46 +00:00
Vincent Ambo
6280706ecd feat(treecrumbs): support for JSON
Fairly similar to YAML, except with only one way of naming keys and
arrays.

Change-Id: Ic2fa539466d69af55b36401bb8c03f2c7edcd91d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11112
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-03-10 19:35:46 +00:00
Vincent Ambo
3b7e546dfc feat(treecrumbs): initial support for C++
Supports namespaces, functions, fields (including trailing return type
syntax) and so on.

One notable issue about this is that the tree-sitter parser for C++
returns the node *following* point if point is on whitespace, which
means that at the top-level of a namespace the crumbs will often show
the *next* function.

I'm against adding workarounds for stuff like that, so I'll just keep
it as is.

Change-Id: If7e71525c4e86e128157dd4eb17c130297ed1e0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11109
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-03-10 19:35:46 +00:00
Vincent Ambo
470f7dfa45 refactor(treecrumbs): macro for language definitions
Adds a macro that provides a more convenient syntax for defining new
languages for treecrumbs (and that can also be easily used outside of
the core treecrumbs code).

This macro automatically takes care of compiling tree-sitter queries
for reuse.

Change-Id: I6a0b892a083c3f243e8b8f0e1c865a9a8a1a5cf5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11107
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-03-10 19:35:46 +00:00
Vincent Ambo
2c9bbe37be feat(tazjin/emacs): install treecrumbs package
Change-Id: Ia903a38df7258c1528f3d4ac3ca4ae7f82edb3e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11106
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2024-03-10 19:27:49 +00:00
Vincent Ambo
eeca6cd891 feat(emacs-pkgs/treecrumbs): support "JSON-style" YAML syntax
Makes crumbs work correctly for weird stuff like:

```yaml
foo:
  - { "bar": [ { baz: 1 }]}
```

Would be nice to have tests for this at some point.

Change-Id: I034e83318435404ef8613e439313dbb08865f228
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11105
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-03-10 19:27:49 +00:00
Vincent Ambo
13cced5b84 feat(tools/emacs-pkgs): configure FSF_OWNERS file
This adds a separate owners file that can be imported in projects with
FSF copyright assignment, so that only people with copyright paperwork
on file can commit (or verify commits before approving).

Change-Id: Ifbe07792572b9d6bf5f7d47c41135892bddea46b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11108
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2024-03-10 19:27:49 +00:00
Vincent Ambo
0be7b938c2 feat(emacs-pkgs/treecrumbs): tree-sitter based breadcrumbs
Adds a new 'treecrumbs' Emacs package that allows displaying fast,
tree-sitter based breadcrumbs. The initial version only supports
YAML (which is what I needed this for!).

The package is documented, please go read the code and comments for
more information.

I assign copyright over this package to the FSF, paperwork is on file.
This means that I can only accept contributions from people with the
right paperwork. If this package ends up being useful, I intend to
eventually submit it to GNU Emacs.

Change-Id: If1297a080f63f402f83b6cc57fa7c970d32d3695
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11104
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2024-03-10 19:27:49 +00:00
Florian Klink
5ce76a1bae feat(ops/machines/whitby): let clbot post in #tvix-dev too
Change-Id: Ic49304291ec2f276e1329ffc7e8b4184d472cbe3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11111
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
2024-03-10 18:35:27 +00:00
sterni
ad9b08e43d feat(sterni/nix/lists): implement transpose
This function is inspired by BQN's [⍉] though it is much less elegant
since Nix lacks multi-dimensional arrays. I thought this would be useful
to to avoid multiple `map`s over a single list if we want to return
multiple, separate values from it:

  transpose (builtins.map (x: [ (calcA x) (calcB x) ]) myList)

  # => [ [ (calcA a) … ] [ (calcB a) … ] ]

While this is quite elegant, it turns out that it is faster to write out
multiple maps:

  [ (builtins.map calcA myList) (builtins.map calcB myList) ]

[⍉]: https://mlochbaum.github.io/BQN/doc/transpose.html

Change-Id: Ic333c33af38ab03573b215c9696d75caf2ee18e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11113
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2024-03-10 18:05:32 +00:00
Florian Klink
37703c9e44 docs(tvix/README): update IRC channels
`#tvl` is the general TVL community channel, `#tvix-dev` is dedicated to
Tvix development discussion.

Change-Id: If899d9ae70dffb4acf6a41ded54f80a1f0551c5d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11110
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-03-10 17:14:41 +00:00
Florian Klink
bd1def3ec4 fix(tvix/store/grpc/pathinfo): skip_all fields, handle errors
request only contains the outer metadata wrapping, and that's not too
interesting:

> Request { metadata: MetadataMap { headers: {"content-type":
> "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers",
> "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions:
> Extensions }

Drop these fields for now, and rely on the underlying implementations to
add instrumentation for the application-specific fields.

Also, ensure we handle all error cases properly, and log them. We
don't use `err` from instrument, as that'd also log an error on
`Status::not_found`.

Change-Id: Id1b983cb8b059c148c8a376f8802a1d28c59ba97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11103
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2024-03-09 05:48:46 +00:00
Florian Klink
4e78de7393 fix(tvix/castore/grpc/directory): skip_all fields in instrument
This only contains the outer metadata wrapping, and that's not too interesting:

> Request { metadata: MetadataMap { headers: {"content-type":
> "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers",
> "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions:
> Extensions }

Drop these fields for now, and rely on the underlying implementations to
add instrumentation for the application-specific fields.

Clean up the error logging a bit.

Change-Id: Ife1090ed411766a61e1fa60fd4c9570f38de1e98
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11102
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-09 05:47:41 +00:00
Florian Klink
05deb37f44 fix(tvix/castore/grpc/blob): skip_all fields in instrument
This only contains the outer metadata wrapping, and that's not too interesting:

> Request { metadata: MetadataMap { headers: {"content-type":
> "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers",
> "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions:
> Extensions }

Drop these fields for now, and rely on the underlying implementations to
add instrumentation for the application-specific fields.

Log errors in some places where we didn't so far.

Change-Id: Ia68d6c526987d3716be62a0809195401cf28512b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11101
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2024-03-09 05:47:36 +00:00
Florian Klink
8f804f5d9c chore(tvix/nix-compat): bump zstd dependency
Otherwise this causes conflicts when adding a more recent zstd version
to another crate.

Change-Id: I02d1b1a37f18711c0969cb04e761e967fc4655dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11100
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
2024-03-09 05:47:03 +00:00
Vincent Ambo
22455cc62c chore(3p/sources): bump channels & overlays (2024-03-07)
* tazjin/nixos: remove unstable ZFS from tverskoy
* skipped agenix update as usual

Change-Id: I21dca17415c147d702ecc14a6c7f1c553ad62b84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11095
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2024-03-08 10:22:22 +00:00
sterni
16a7c4a1be feat(buildkite): avoid building extraSteps in pipeline construction
In principle we don't want to build any (later) pipeline target during
pipeline evaluation insofar they appear in extraSteps. For this reason,
we have the needsOutput mechanism which prevents the parent target of an
extraStep from being built in 🦙.

Unfortunately, this mechanism is not general purpose enough, as we use
other (i.e. non parent) targets from depot in extraSteps. As a
consequence, kind of expensive builds need to happen during pipeline
construction at the moment. The solution is to use the fact that the
command script we want to run is exposed via the readTree interface to
depot and build the script proper only when the extra step is executed.

To facilitate this, some prerequisite changes need to be made:

- We need to use a symlink different to result in case needsOutput is
  true which needs support in mkBuildCommand. We also need to avoid this
  symlink being picked up by git, as many extra steps check whether the
  tree is dirty or not. (Is there a way to have it outside the depot
  tree?)

- Since we rely on the build command printing a single store path we
  store in $command_script, we need to avoid it printing two paths
  in cases where nix-store(1) is used (nix-store(1) prints the symlink
  and readlink(1) would print the store path in a separate line).

Future work would be to remove/deprecate the needsOutput mechanism:
After this change the parent target wouldn't be built right away even if
it appeared in the script via string interpolation. Thus we could,
instead of expecting the target being available as `./result`, make our
extra steps nix-ier.

Change-Id: Idd2e88a865eadabe229ce1e05406e8cc4cb63f94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10850
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2024-03-07 15:39:56 +00:00
sterni
08862432cd chore(3p/rust-crates:libgit2-sys): 0.12.26+1.3.0 -> 0.16.1+1.7.1
- Set LIBGIT2_NO_VENDOR so that libgit2-sys won't start silently
  building the vendored libgit2 version.

- Update to match version distributed by nixpkgs.

- Update related dependencies:

  - git2 (reverse dependency)
  - bitflags (dependency) which necessitates updating:
    - inotify*
    - epoll
  - libc

Change-Id: I1d9059a16a44b5e02d04f034c8f02bbfdc55b6ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10845
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2024-03-07 14:51:30 +00:00
Florian Klink
c499e424d8 feat(users/flokli/ipu6-softisp): provide example closure
This introduces a NixOS configuration using config.nix as a NixOS
module, and also checks the firmware is present in the location it's
expected to.

Change-Id: I3ec4333d73fe1b28e10c589c1cf351c59372bb99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11099
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-03-07 14:43:02 +00:00
Florian Klink
525388fde7 fix(users/flokli/ipu6-softisp): skip subdirs from readTree
There's no readTree targets in there.

Change-Id: I267e3c363ccf1084adb40738361ea1ce702e2a8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11098
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-03-07 14:29:17 +00:00
Florian Klink
4749e2e703 chore(users/flokli/ipu6-softisp): align with linux-firmware
The firmware files ended up in linux-firmware, and
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/commit/?id=dbc294d3e49aa63a61bb4026b5784bae5b75aa2b
moved them to a ipu/ subpath, so update our patches to find them there,
and stop using the pkgs.ipu6-camera-bins package.

https://github.com/NixOS/nixpkgs/pull/290216 bumped our linux-firmware
image past this, so this should work for people tracking nixos-unstable.

Change-Id: Ic3eb9fb364c93ee0d10951451a59f7e98888b5d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11097
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-03-07 13:56:33 +00:00
Vincent Ambo
04ac1c995c feat(web/volgasprint): add sprint logo(s)
Adds SVG versions of the logo in two colours (using the .ru flag
colours, and using the standard Nix colours).

The flag colours are difficult to use in some contexts because of the
white background of the top, but flokli says the Nix colour version is
"too similar to NixCon". Not yet decided what should go on to swag etc
...

Change-Id: I1e1655e2c7ed4ed6113ddf6b5df76157de5e32cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11093
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-03-05 10:26:58 +00:00
Florian Klink
4954a39de4 fix(tvix/castore): also set SSL_CERT_FILE for tests there
For everything using reqwest here during test cases, we also need to
set SSL_CERT_FILE.

Change-Id: If8aeda65f3d75cb9ac5c9bc64e37a0cb7dffc17c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11092
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-03-03 17:43:42 +00:00
Florian Klink
0b396553d6 refactor(tvix/store/protos): more granular nix-compat conversions
implement From<&nix_compat::nixhash::CAHash> for nar_info::ca::Hash
individually, and make
From<&nix_compat::nixhash::CAHash> for nar_info::Ca a small wrapper that
uses it, as well as the .hash().digest_as_bytes() for the digest.

Change-Id: I7e9b6edd1e3f149eb270faf2928cd846d74e77ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11071
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 17:12:51 +00:00
Florian Klink
ef3f8936cb refactor(tvix/*/from_addr): improve test debuggability
If there's an unexpected test failure, print it out, rather than just
saying something is false even though it should be true.

Use .expect() for this, which displays the error if it failed.
We can't use expect_err(), as our stores are not display'able, so use an
assertion with a message there.

Change-Id: I2d88861d979d107edc0717fbdb3cdac9a6bfc5e4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11091
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 16:54:19 +00:00
Florian Klink
b0e5a932ec docs(tvix): remove differences-drv-paths.md
This was true while we didn't do string contexts, but drvPath should not
differ anymore.

Change-Id: I58e6ed439519bd4d68259d1577946f3c65110dc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11074
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
2024-03-03 16:53:41 +00:00
Florian Klink
c9514b7c0c feat(tvix/docs): ship custom highlight.js with Nix support
Selected languages:
 - bash
 - c
 - cpp
 - diff
 - go
 - ini
 - json
 - markdown
 - nix
 - protobuf
 - rust
 - shell
 - xml
 - yaml

Created with https://highlightjs.org/download

Change-Id: I76a87d2c1e9fa582d1474f8ffb9f3129e9a858bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11073
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2024-03-03 16:53:16 +00:00
Florian Klink
65a810fc0b feat(tvix/docs): switch to mdbook
Some of the docs are still outdated (like architecture and drv path
inconsistencies).

Change-Id: I7a6afceb008ef4cd19a764dd6c637b39fa842a2e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11072
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2024-03-03 16:52:38 +00:00
edef
6bdaebcb55 fix(tvix/tools/weave): handle sliced arrays correctly
The start/end offsets are not necessarily coterminous with the underlying
values array, so even if the stride is fixed, we still we need to slice
the chunks down to match the start/end offsets.

This bug shouldn't affect the correctness of any existing code, since
we're always working with unsliced arrays read directly from Parquet.

Change-Id: I2f7ddc4e66d4d3b2317a44bd436a35bff36bac79
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11081
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 16:45:19 +00:00
sterni
ca97e5f485 chore(3p/sources): Bump channels & overlays
- agenix has not been updated (waiting for
  https://github.com/ryantm/agenix/pull/241).

- libgit2_1_5 can be removed (no longer used by cargo-audit).

Change-Id: I96c6a1a4175dc4f2a32b9b2e4ed71caa826a9c42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11077
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 16:00:27 +00:00
Florian Klink
1c16dee207 feat(tvix/store): use reqwests' rustls-native-roots feature
This makes reqwest honor `SSL_CERT_FILE` - previously it was using the
chain bundled in webpki-roots.

`object_store` pulls in `reqwest` with this feature, and the cargo
solver will enable that feature globally as soon as we pull it in, as it
assumes features are additive.

This requires setting `SSL_CERT_FILE` when running tests, otherwise
they'll fail with the unhelpful "NotFound" error.

This was quite some fun to debug, why adding `object_store` to
tvix-castore suddenly made tvix-store tests fail!

Change-Id: I64fc82b4d994715480efdb1ffecb279716456ab9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11090
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-03-03 15:32:38 +00:00
Florian Klink
4b4443240e feat(tvix/castore): add HashingReader, B3HashingReader
HashingReader wraps an existing AsyncRead, and allows querying for the
digest of all data read "through" it.
The hash function is configurable by type parameter, and we define
B3HashingReader.

Change-Id: Ic08142077566fc08836662218f5ec8c3aff80be5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11087
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-03-03 15:31:31 +00:00
Florian Klink
8383e9e02e feat(tvix/castore/digests): impl From digest::Output<_> for B3Digest
This allows calling .into() to get a B3Digest.

Change-Id: I6e63b496413cd00d84acfcd15c7de0f64c79721f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11086
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
2024-03-03 15:18:19 +00:00
Profpatsch
9a7246ea1d chore(users/Profpatsch/whatcd-resolver): slight changes
Change-Id: I57b0fcf9bd3953951dd0cffbee1fbfab5abbeb47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11089
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
2024-03-03 13:31:27 +00:00
Profpatsch
de5790aba8 feat(users/Profpatsch/ytextr): allow extra arguments
Change-Id: Icf246cfb31592afa0a8ada3e0e897f78202d44e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11088
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-03-03 13:30:51 +00:00
Profpatsch
7c2bdf84b9 fix(users/Profpatsch/whatcd-resolver): pin to postgres 14
I haven’t thought about how to migrate this when the postgres version
changes, so for now we just pin to 14 and worry about it when we reach
EOL lol

Change-Id: Ib72ec7835c660db636a95fb5af8c9ce40897b1ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11085
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-03-03 13:30:51 +00:00
Profpatsch
c869f9627e chore(users/Profpatsch): misc small improvements
Change-Id: I8fc128391196da22f03bac76a6c621f2cee73334
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11084
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2024-03-03 13:25:10 +00:00
Profpatsch
5a086020cb fix(users/Profpatsch/nix-home): fix alacritty config
they switched to TOML & I had a typo in `scolling`

Change-Id: If68408162e7772fbd3b06944ffd9d51dd2134efa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11083
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2024-03-03 13:25:10 +00:00
Florian Klink
442eabe9d0 feat(tvix/nix-compat): add HashMode and CAHash::mode()
This allows peeking at the mode without matching on all enum kinds of
CAHash directly.

Change-Id: Icca147a876f7f4cff0bbaa067ac4dae40d05bee9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11069
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2024-03-03 13:17:50 +00:00
Brian Olsen
eff2cc4f68 fix(tvix/nix-compat): Make CAHash deserialize more formats
Currently CAHash only deserializes the hash in hex code while
the serializer outputs a nixbase32 hash. This means that you can't currently
deserialize what has been serialized.

This change makes deserialize support any digest format (so hex, nixbase32
and base64) as well as flattens the deserialize code and error handling.

It also implements serde methods of HashAlgo directly using Display and TryFrom
implementations because otherwise these would get serialized as eg. Sha256 instead
of sha256 which also broke CAHash serialize/deserialize.

Change-Id: I1941a72eaec741e4956292adaaf0115b97f260ba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11082
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 13:07:18 +00:00
Florian Klink
260c2938d4 refactor(tvix/nix-compat): rename NixHash::{digest,hash}
This doesn't return the digest, but the internal NixHash.

To get the digest, you use `ca_hash.hash().digest_as_bytes()`
(as done in tvix/nix-compat/src/derivation/write.rs).

Change-Id: Ib07918dec63ae42ddf8c8d3f1d10510f9c4df255
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11070
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2024-03-03 11:56:18 +00:00