This adds the simpler ChunkService trait, and an implementation for it
using sled, and one using a HashMap.
Change-Id: Icb0fdc41b37b44e9e9e4f548d0f4acae1d83b71e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8086
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
This came up recently again, it makes sense to document the reasoning
behind the decision.
Change-Id: Ic51d5bc7998c70e8b070b6f42877d8e88613935b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8223
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Because they do not use it, and it can not be passed with the coming
generator refactoring.
Change-Id: I0d96f2357a7ee79cd8a0f401583d4286230d4a6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8146
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Instead of using a suspended native thunk, calculate and optionally
insert the storeDir builtin when the VM is constructed.
We already have the IO handle available at this point and can just
check whether a storeDir is present, and insert its absolute value as
a builtin.
Change-Id: If966eee6ff26dc888b6e888e7c46170c0c346b05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8145
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
This is a ThunkSet wrapped to be shareable, which will be required
once ThunkSets are embedded in futures.
Change-Id: I5a067b7972ac86e4d354c75ef05c86b2284c1137
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8144
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Again simplifying some code down the line, where bits of code that
construct attribute sets already have the final structure available.
Change-Id: I0bb7a1daa63298122b51be73d35d695a4f73f8b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8140
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
This simplifies some code down the line.
Change-Id: I58dd71e796e11479f44516cf24932f8061843d23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8139
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Some of these strings are actually just the nix hash representation of
the hash calculated earlier.
There's another one passed around via calculate_drv_replacement_str, but
that's left for a followup.
Change-Id: Id99a2a926a980d679eb49c34ee6a36bf224699b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8218
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
This provides a way to construct a NixHash struct without parsing
strings.
Change-Id: I947d96e15e51e72d5b02929cda8c5fc31d81253a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8217
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
We ironically didn't add support parsing for the "native" format that
Nix uses under the hood.
This extends the from_str method to peek at the prefix of the string to
determine whether to try decoding as SRI, Nix string, or whether it
should be a bare digest.
Change-Id: I33efd24968b16f86eff18305b4ca8f112c7131d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8216
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This will be used for both Nix hash strings and hash strings without the
algo specified.
Change-Id: Iedfe5494fba5f2be00614ba0fc38bf659eafd447
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8215
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Also document how these can be cloned by referring to a custom Gerrit
ref.
Change-Id: I5495a1c29bdd6d78215af7307953437f9cbca5db
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8198
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Because the passing around of externalArgs only happened in the readTree
instantiation in the repo root default.nix, but not in views/kit/
default.nix, it was not possible to get tvl-kit to instantiate the
bundled nixpkgs with a custom system.
This fixes invocations like
```
tvl-kit = import (sources.tvl-kit) { localSystem = "aarch64-linux"; };
```
Change-Id: I3a633e4d695d266459400ba74fc0693ecc5bfb54
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8197
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This allows passing in custom triggers to trigger a (re)deploy.
For example, a caller can put an AWS instance ID into the triggers to
cause a redeploy whenever the instance ID has changed.
The `target_name` terraform variable was doing something similar, but
`triggers` is more generic, allowing multiple triggers, without having
to stringify them.
We also don't need to trigger on the attrpath - it can be changed, and
as long as it still evaluates to the same
`data.external.nixos_system.result.drv` (which is checked on every
plan), no redeploy needs to be made.
Change-Id: I94ce787a50830b87b6f53c08e042e4abe4036bdd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8191
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
This adds an additional parameter `entrypoint`, pointing to a .nix file
(or a directory containing a `default.nix` file) that's providing the
attribute path asked for.
If not set / kept at the default (empty string), it falls back to the
root dir of the repository as before.
Change-Id: I2e63114f21660c842153ac15424b3491d66624d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8190
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This adds addresses of thunk and closure chunks to the debug output
displayed when dumping bytecode.
This makes it possible to see in the dump which thunks are referenced
by constants in other thunks.
Change-Id: I2c98de5227e7cb415666cd3134c947a56979dc80
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8137
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This module makes it fairly easy to deploy NixOS system closures using
Terraform, while properly separating the evaluation of a
derivation (to determine whether a deploy is needed) from the building
and copying of the closure itself.
This has been on my stack for a while. It was originally developed for
Resoptima, who agreed to open-sourcing it in depot back when we
completed our work with them. Their contribution has been acknowledged
in the README.
Co-Authored-By: Florian Klink <flokli@flokli.de>
Change-Id: Ica4c170658cd25f1fb7072c9a45735fcc4351474
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7950
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
* //3p/sources: temporarily switch to nixos-unstable-small, since it
includes:
- evans update we are interested in, allowing us to drop our evans
patches.
- awscli2 update that unbreaks //users/grfn
* //3p/overlays/tvl:
- drop evans patches
- update tdlib to 1.8.11 to make tazjin's emacs happy
- drop obsolete mullvad workaround
* //users/grfn/keyboard: disable -Werror for array-bounds warnings.
Seems like a non-trivial job to resolve the warning properly,
hopefully GCC 12 still generates the same working code as GCC 11 used
to.
* //users/grfn/system/home: remove yubikey-manager-qt.
Yubico can't seem to keep that on pace with yubikey-manager. It
requires a <5 version of the latter which is incompatible with the
recently released cryptography >= 39.
* //3p/gerrit: update changed FOD hash for the fetch step
Change-Id: I590ab996247e69b0ab5059cd173840ef4ebfe939
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8133
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
I didn’t ever get aerc to work, so let’s get rid of this.
Change-Id: Ie59b23fe7d5af70ca82c1c624c1e11d21da735f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8189
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Includes a frontend bug fix and a closure size reduction of the server
application.
Change-Id: I5713a5348281acb7126c1fd85a637a6fff969c98
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8187
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
With this change, readTree gains the ability to notice a `.skip-tree`
marker in addition to the `.skip-subtree` marker.
The behaviour of the new marker will completely ignore the folder that
the marker is located in (i.e. no node will be present for it in the
parent at all).
To make this work, the recursive function in readTree had to be
modified to return a sentinel value (noting that a tree has requested
to be skipped) which is then filtered out when constructing the list
of children. The actual `readTree` function is now a wrapper around
this inner, sentinel-yielding implementation which unwraps the result
set.
For obvious reasons, `.skip-tree` is not allowed at the top-level and
readTree will throw an error if it encounters it there.
Fixes: b/244
Change-Id: Ica731bc1af356e881fd3d31c7109f62ffd2762ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8185
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This CL removes calling into_iter on a reference, as it will
not move out it's content into resulting iterator.
Change-Id: Ifcc10b7cf33b98453570cbcec3eb82ffaba2ffcb
Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8126
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This CL removes redundant clone from value which is
going to be dropped without further use.
Change-Id: Ibd2a724853c5cfbf8ca40bf0b3adf0fab89b9be5
Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8125
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This is an experiment for tvix-eval.
Change-Id: Ic752b5b125cefefeb1343e38a70beb364478e6eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8131
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Instead of using an explicit closure to clone elements, use .cloned().
Change-Id: I31f0f0bad2b4935e1a8d91fa0d14163c94182e1b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8109
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
The size field already is u32, we don't need to convert here.
Change-Id: Ie29819aa2d1d8022e9bd73fcf05b140e45c967a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8107
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We want the address that the Rc is pointing to, not the address of the
Rc.
Change-Id: I8eba21677f242bbe4166c74d4aa4269c316076e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8045
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This CL address clippy warning which expects to
use `writeln` instead of `write` for strings with
new line.
Change-Id: Ia72a07502c60cfd489ecf1e3833b9d42d44a8b17
Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8030
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This CL address clippy warning about finding the zero
length of something using `is_empty()` instead of `len() == 0`.
Change-Id: I2b36c7c7b65b733609fc0dcd33be06f9d772bc9b
Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8029
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This branch was missing, and an assumption elsewhere just executed the
returned (broken) bytecode.
This fixes b/253.
Change-Id: I015023ba921bc08ea03882167f1f560feca25e50
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8090
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
These should be inspectable by callers.
Change-Id: Ia9ef871aa63958d06066aaea61b2aecbd217369b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8089
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Whitby has a lot of memory, but I've still been fighting with the OOM
Killer trying to build a few big packages at the same time. Besides,
it's generally a good idea to always have swap available even if
there's lots of memory for caching optimisation reasons[1], and zram
swap is efficient enough to basically provide bonus memory for free.
[1]: https://haydenjames.io/linux-performance-almost-always-add-swap-space/
Change-Id: I1fbe60f7975ebfa38e341e0de76848ec79b6fcf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8065
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
cl.tvl.fyi only supports the legacy scp protocol, not the sftp
protocol that recent versions of OpenSSH now use by default. So we
need to pass -O to tell it to use scp.
Cc: Vincent Ambo <mail@tazj.in>
Change-Id: I2dcae77fef9126f4f8eebfa574c5a345a5c56615
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8039
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This way, I won't have to teach my name one at a time to every program
that wants to know my it (e.g. git).
Change-Id: I45ddd9c2343a10cd4c13bacd9a97b7470db95c14
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8038
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
