This refactors how the original digest type (Vec<u8>) is converted
to [u8; PATH_HASH_SIZE].
Change-Id: I9441470a3a199620fcf328f2b7c890ca6ae93bde
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7710
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This replaces the hello world example from tvix-store with an actual
gRPC endpoint, implementing all of BlobService, DirectoryService and
PathInfoService.
All RPC methods currently respond with the unimplemented gRPC status.
Co-Authored-By: Márton Boros <martonboros@gmail.com>
Change-Id: Ieba333cca44dc1e3f2ffbe676ba7a99e672b9bfb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7664
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Similar to cl/7682, we also want to make that error configurable.
Change-Id: I64f1a4570b3d75af4741abe10c2855959766e107
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7708
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This implements the NixPath structure. NixPath allow to parse a string to a nix path. If the parsing fails, a DecodeError will be raised.
Change-Id: I28363cdcfb27f04bf21a11c0d130b461667e3720
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7706
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This implements the nix-specific base32 encoding and decoding, exposing
a subset of the API that the data-encoding crate provides.
Nix uses a custom alphabet, no padding, and encodes bytes in reverse
order. The latter one is the reason we can't just use the data-encoding
crate directly.
Three odd corner case tests ported over from go-nix failed. We opened
b/235 to further investigate.
Change-Id: I73fab6ddd67177d882e4c3f2b48761c95853d558
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7683
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We'll be using validate_node_name in other places in a bit, where
returning a ValidateDirectoryError is not appropriate.
Accept a function mapping a string to error as a second argument, and
pass ValidateDirectoryError::InvalidName at the current call sites.
Change-Id: I45cbb0deb4763061ad912c6b18a112c727795a17
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7682
Tested-by: BuildkiteCI
Reviewed-by: jrhahn <mail.jhahn@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
This fix can only be applied after the upstream public-inbox
fix (https://github.com/NixOS/nixpkgs/pull/207693) has been merged.
Change-Id: I957473e2895b7e57baad25c9e908b36aa790f3a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7636
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
* //users/grfn/modules: change deprecated loaOf to attrsOf, the former
is an alias for the latter nowadays.
Change-Id: I6fa71b43f8c1d0adeafb8b78b197e80733f5392a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7679
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is the latest version before 1.0 which has some bigger changes as
brick's EventM gains the ability to be MonadState which requires
adjusting basically all App code.
In 0.72, handleEditorEvent started taking a BrickEvent, so we no longer
need to unwrap the VtyEvent in handlePromptEvent.
Change-Id: Ic6a1ce6e21ba46177d3ce0b8a124abe7d8951464
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7666
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
The main change is that nixpkgs updated to GHC 9.2 and Stackage LTS-20,
so we suffer from a bit of churn.
* //3p/overlays/haskell:
- use updated dhall-nix patch for hnix 0.16
- use superrecord fork with fixes for GHC 9.2
- use graphmod-1.4.5.1 which has support for GHC 9.2
* //users/Profpatsch: relax constraints on base in Haskell pkgs
* //users/Profpatsch/cas-serve: inherit superrecord from 3p
* //users/grfn/xanthous:
- //3p/overlays/haskell for 8.10.7:
* Provide missing dependency of binary-orphans. Fix already commited
upstream as e238c3fdaab710a2ce0135e5a77cd7e6bb023a22, can be
dropped when channel advances.
* Downgrade to brick 0.71.1, the latest version xanthous supports.
- Adjust to generic-arbitrary >= 1.0, providing Arg constraints where
necessary.
- Increase constraint-solver-iterations to 6 (default 4), so
Xanthous.Command and Xanthous.Data can be typechecked.
- Drop NFData instances for Key and Modifier which have been added to
vty upstream.
Change-Id: I2170438c2ce8130b65f1a9fe07c4fecab5683d66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7654
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
The `im::OrdMap` is already small and cheap to copy while sharing
memory, so this is not required anymore.
Only the `KV` variant may have slightly larger content, but in
practice this doesn't seem to make a difference when comparing the two
variants and this one is less complicated.
Change-Id: I64a563b209a2444125653777551373cb2989ca7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7677
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This uses the `im::OrdMap` for `NixAttrs` to enable sharing of memory
between different iterations of a map.
This slightly speeds up eval, but not significantly. Future work might
include benchmarking whether using a `HashMap` and only ordering in
cases where order is actually required would help.
This switches to a fork of `im` that fixes some bugs with its OrdMap
implementation.
Change-Id: I2f6a5ff471b6d508c1e8a98b13f889f49c0d9537
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7676
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
The conversion from im::Vector -> Vec is cheaper for NixList
construction (of course), so where possible we should make use of
that.
This updates most builtins dealing with lists to use Vector directly,
and marks the function constructing NixList from Vec as deprecated so
that we get appropriate warnings in places where it's still in use.
These places are currently inside of JSON serialisation logic which is
in flux right now, so lets leave them as-is until it's stabilised.
Change-Id: I037f12a2800f2576db4d9526bd935efd079163f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7671
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Three small helper functions that can display module graphs of haskell
projects.
Change-Id: I7395ffc8b025f4322efc6c1e494e6a6a0145342c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7675
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Apparently I had forgotten that these already exist on the toplevel.
At one point I should unify the two namespaces, but for now at least
acknowledge that they are the same functions.
Change-Id: Ie7d14de0b65f6c750d97630798c65f777b3eda8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7673
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
This adds a Derivation structure and allows to write it to a structure that implements std::fmt:Write.
The implementation is based on the go-nix version.
Change-Id: Ib54e1202b5c67f5d206b21bc109a751e971064cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7659
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The commit graph can be quite slow for repositories like nixpkgs, so it
is disabled there. For this we refactor the module a bit, allowing us to
set arbitrary cgit settings for repositories. This feature can also
handle all instances of defaultBranch now.
Change-Id: I22e44b7398d2692e8cc16555fb5203ad6a7a69a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7672
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This is a persistent, structurally sharing data structure which is
more efficient in some of our use-cases. I have verified the
efficiency improvement using `hyperfine` repeatedly over expressions
on nixpkgs.
Lists are not the most performance-critical structure in Nix (that
would be attribute sets), but we can already see a small (~5-10%)
improvement.
Note that there are a handful of cases where we still go via `Vec`
that need to be fixed, most notable for `builtins.sort` which can not
currently be implemented directly using `im::Vector` because of a
restrictive type bound.
Change-Id: I237cc50cbd7629a046e5a5e4601fbb40355e551d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7670
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
The only warnings left are unused warnings, but that'll change once we
have a real implementation, and not just tests.
Change-Id: I28912281b5e66735be37e999cc8ef4b8b09028fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7669
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
lazy_static is only used in tests, and anyhow isn't used at all (yet).
This can be dropped.
Change-Id: Ic41ff3f9bb93cfa600c3485e85464f78a3976504
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7668
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Move them from the bottom of src/proto.rs to its own src/tests/mod.rs.
Also drop the test_ prefix, this is not golang.
Change-Id: I2e0b6b9812264f3d9721c0766936f08157fadc66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7667
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
I think what might be going on with b/231 is that the annotations
somehow started conflicting because they don't have contexts set.
Lets try setting a context and see if it changs anything ...
Change-Id: I62ed57f9e24f08e4e7215f05d35cfa769e2e2c24
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7640
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Stat exposes metadata about a given blob,
such as more granular chunking, baos.
It implicitly allows checking for existence too, as asking this for a
non-existing Blob will return a Status::not_found grpc error.
The previous version returned a Status::not_found error on the Get
request too, but there was no chance to prevent the server from starting
to stream (except sending an immediate cancellation).
Being able to check whether something exists in a BlobStore helps to
prevent from uploading in first place.
The granular chunking bits are an optional optimization - if the
BlobStore implements no more granular chunking, the Stat response can
simply contain a single chunk.
Read returns a stream of BlobChunk, which is just a stream of bytes -
not necessarily using the chunking that's returned in the reply of a
Stat() call. It can be used to read blobs or chunks.
Change-Id: I4b6030ef184ace5484c84ca273b49d710433731d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7652
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This changes the RPC methods to return/consume a stream of chunks, instead of a
very big message containing the whole blob, to keep message sizes in manageable
sizes (less than 4MiB).
Change-Id: I2a3a50f07b059d8a2f5196860254adff98c8a352
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7651
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Expose the NAR calculation to a separate `CalculateNAR` method, which
responds with the NAR size and sha256 hash.
Contrary to what cl/7618 and cl/7620 initially did, don't add different
other request types.
In the CalculateNARResponse message, there's now some duplication in the
(optional) `narinfo` field of a PathInfo, but I'm not entirely sure if
we want to drop the fields from there yet.
Change-Id: Id797c56e17efedac115fbd43de9dfde9fa1db140
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7663
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This is gonna get used in another place in a second.
Co-Authored-By: edef <edef@unfathomable.blue>
Change-Id: I347c11c8d24379628b7ed09d2c90670c576e686a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7617
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
BQNLIBS dependency also needs to be provided in the derivation running
all solutions.
Change-Id: I704369127ab92a52c7e4b21de8b7982fb8328f9d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7662
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Didn't end up happening due to a lack of motivation. Will try to finish
the BQN AoC still, though.
Change-Id: Ib296aec9f3cfeef57c79a9ba09fc664c1a19dcff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7661
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This landing page explains how to use the public-inbox.
Change-Id: I37d74decad5173ab35051970593f1d28001af2b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7645
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This script fetches the inbox for depot@tvl.su into the specified
directory in maildir format.
The layout of the folder follows the structure generated by public
inbox, i.e. the directory containing the current maildir will be
`$TARGET/su.tvl.depot.0`, but most mail clients (e.g notmuch) will
figure this out on their own.
----
In addition, we would ideally find a CLI mail client that can be
pointed at an arbitrary maildir (or an IMAP server) and works with
local `sendmail` config so that people can have a single command entry
point to interacting with depot@tvl.su.
Change-Id: Iaf9fcce73e9caa2f202327488c43d0394be26ca6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7644
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This sets up IMAP on inbox.tvl.su:993
I added a hack to work around problems with the NixOS ACME module.
Spent way too much time of my life with problems with that module, so
I only use it with blunt force these days. Others are welcome to make
a cleaner solution.
Change-Id: Ice828766020856cf17d2f0a5b4491f4cec8ad9b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7633
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
While there's currently nothing in here checking the size of the digest,
we should use something that passes the to-be-introduced validate()
function.
Change-Id: I0c515d9e3afc79292dedebce659a32485aa3d936
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7649
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Validate thecks the Directory message for invalid data, such as:
- violations of name restrictions
- invalid digest lengths
- not properly sorted lists
- duplicate names in the three lists
Change-Id: I8d43a13797793c64097e526ef3bd482c9606c87b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7648
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This adds Size() and Digest() functions for the golang version.
Change-Id: If71445a9bb26100bb4076ac4f5c96945b33919f9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7325
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is the new address which leads to the public inbox at inbox.tvl.su
Change-Id: I45d98a373b8acda49b05c4f74669ffb9ad1f1a3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7632
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
