This splits the pure content-addressed layers from tvix-store into a
`castore` crate, and only leaves PathInfo related things, as well as the
CLI entrypoint in the tvix-store crate.
Notable changes:
- `fixtures` and `utils` had to be moved out of the `test` cfg, so they
can be imported from tvix-store.
- Some ad-hoc fixtures in the test were moved to proper fixtures in the
same step.
- The protos are now created by a (more static) recipe in the protos/
directory.
The (now two) golang targets are commented out, as it's not possible to
update them properly in the same CL. This will be done by a followup CL
once this is merged (and whitby deployed)
Bug: https://b.tvl.fyi/issues/301
Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
There's a go.mod in in tvix/store/protos, which sets the module path to
code.tvl.fyi/tvix/store/protos.
While this path makes kinda sense, it's currently not possible to `go
get` it from that location, as we serve the cgit interface from there.
Fortunately, `go get` has a mechanism to determine clone URLs for a
given go module path, as documented in https://go.dev/ref/mod#vcs-find.
We simply need to serve a small HTML file at that path, describing the
proper clone URL.
This points the clone URL for code.tvl.fyi/tvix/store/protos to a josh-
provided subtree of just :/tvix/store/protos, which will contain the
root go.mod file.
We need another layer of indirection as nginx can't have an `alias`
directive inside a conditional block (but can have a redirect).
Contrary to https://b.tvl.fyi/issues/299#comment-464, it seems to work
for our usecase. It might become a problem if we actually serve `go.mod`
files in a nested fashion at some point, but let's look at that once we
get there.
Fixes b/299.
Change-Id: Idcad795105af5d57e6d06de6e232881dccf9110b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9290
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: adisbladis <adisbladis@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
I verified on whitby that the password hashes generated by
//web/pwcrypt are compatible with our OpenLDAP, so it's time to make
this thing public.
Change-Id: Icc2f095ca7ce4acff6de91a1642dea6461177423
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9266
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
The current unstable has a bunch of breakage which people have been
reporting, lets move the public instance to the stable channel until
that is sorted out.
Example breakage: https://github.com/tazjin/nixery/issues/159
Change-Id: Id5eb11ebd235928b85c01c178c32da3badea517f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9126
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
this moved out of whitby some time ago (to koptevo.tazj.in), but is
now causing failures because of ACME cert renewal
Change-Id: I4da5512db0d85d416511a1d10f784e978c5ccc93
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8948
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
in accordnace with similar renaming on other sites
(e.g. GitHub, Exozyme, chaos.social)
My experience with exozyme tells me that fully applying
this change might require manual editing of gerrits database
anyways to fix broken references/patch ownerships.
Change-Id: I024ff264c09b25d8f854c489d93458d1fce7e9f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8919
Autosubmit: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: zseri <zseri.devel@ytrizja.de>
Sadly, this can't quite be an alias (which would be difficult to
automatically set up anyways), since we want to check if an r/number is
part of the (upstream) canon branch.
The test script for the subcommand doubles up as a soundness check for
our pipelines ref creation.
Change-Id: I840af6556e50187c69490668bd8a18dd7dc25a86
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8844
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
This was already removed from whitby a while ago, no reason to keep
this secret.
Change-Id: I4742dd0138a3eff91325c94e44e64b72c644ee3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8915
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Nothing is using this, so it can be removed.
Change-Id: I1b812b6df89d4f79ed313e646e141909519c6083
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8914
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: flokli <flokli@flokli.de>
This was dropped from whitby itself in cl/8905, but didn't drop the
module because we were worried someone else might still be using it.
However, this relies on the "oauth2-proxy" client ID, which only has the
following supported redirect uris (as per ops/keycloak/clients.tf):
- https://login.tvl.fyi/oauth2/callback
- http://localhost:4774/oauth2/callback
… which means, noone can really run this properly anyways, so let's
drop it.
We can always restore it from git.
Change-Id: I7d700f59a62cce1254ad4ba0792a7d7b3960b769
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8913
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
this never worked and was never used, but for now the module itself is
still around in case somebody wants it for something
Change-Id: Id8e449e08c8012786bca0ea57d9c7b97056a1f3d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8905
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This runs a headscale server on sanduny which lets users join their
machines to the TVL tailscale network.
This would theoretically let people communicate with each other on the
internal network, but also more notably joined servers can advertise
exit node capability so that we can have our own "VPN network", for
starters with endpoints in Germany, UK and Russia (whitby, sanduny and
koptevo respectively).
This setup isn't fully stable yet, notably:
* The IP range used by tailscale is just the default one right now,
I'm not sure if that should be changed or what.
* The system is stateful (on sanduny), but the state is not (yet)
backed up anywhere. Use with caution.
* Machine joining is a manual process requiring SSH & root access to
sanduny.
The process is to log in to sanduny, then get a headscale shell with
`sudo -u headscale bash`, and to use the `headscale` CLI within
there to administrate access.
I've opted to create a user account `tvl` for TVL-owned machines,
and a personal account for myself and my machines.
Change-Id: I4f1be1fe8062a6c2e77203ff72fe8709f4e4dec8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8837
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This will host a headscale server for TVL.
Change-Id: I8769852aaaf7a02a2d63f48ecf5adfd86747ff72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8835
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
* //ops/modules/depot-inbox: Adapt to upstream option type declaration.
See nixpkgs commit b6ed3b8f402893df91a8e21ce993520301c2f076.
* //ops/machines/sanduny, //users/tazjin/polyanka:
Remove boot.loader.grub.version options (no longer has any effect).
* //users/sterni/emacs: reflect rename emacsPgtk -> emacs-pgtk
* //3p/overlays: update tdlib to match emacs-overlay
* //3p/overlays: give EXWM from depot a separate name
* //users/grfn/system/home: disable Slack support in ntfy
Change-Id: I03bde088bc70e05b23925f244899807210cb7b20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8547
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
It's actually quite common that a token provider might fail, for
example when fetching a token from instance metadata.
Change-Id: Ie0126fb92c6c613ad36b5583fd68505fdd97f2c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8764
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
These are useful for downstream users of the library, who might not
need all the rest of the tonic stuff.
Change-Id: Iab4d941696ae3c7a33b25815b72f92598aa82b80
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8763
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This makes publishing a bit easier without the build script
interfering and other wonkiness.
Change-Id: Iadb144aabbdeabae8899ebdc62636315239e5f08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8601
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
In case `target_user_ssh_key` points to an empty string, nixos-copy.sh
just doesn't set `IdentityFile=` at all.
This allows using deploy-nixos without any explicitly passed ssh keys,
but picking up whatever ssh setup the user has configured locally.
Change-Id: If335ce8434627e61da13bf6923b9767085af08a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8576
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This uses tonic to generate the full set of gRPC clients for Yandex
Cloud. Includes some utility functions like an authentication
interceptor to make these actually work.
Since the upstream protos are exported regularly I've decided that the
versioning will simply be date-based.
The point of this is journaldriver integration, of course, hence also
the log-centric example code.
Change-Id: I00a615dcba80030e7f9bcfd476b2cfdb298f130d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8525
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
nixpkgs commit 134036f642a7f3ba9efeab509727c0989458b02b moved the
digidoc-tool binary to the `bin` output, so this wasn't actually
providing the digidoc-tool binary anymore.
Change-Id: Id5f7cc69d55b7cc058a6361512cc74de0e7bc1b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8487
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
* Satisfy new assert that the corresponding shell needs to be enabled
via programs.* if it is as the login shell of at least one user.
* //users/tazjin: “Address” removal of hardware.video.hidpi option.
* //3p/gerrit: update fetch sha256
Change-Id: Id0988a0ea7f393d6b7848a7104fc3526ee1177f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8407
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Instead of prepending :unsign to all URLs in josh-proxy, and for all
calls to filteredGitPush, explicitly use it only in the filter we use
for the `export-kit` extraStep.
This means, people cloning tvl-kit via
> https://code.tvl.fyi/depot.git:workspace=views/kit.git
now need to update the URL to point to
> https://code.tvl.fyi/depot.git:unsign:workspace=views/kit.git
instead.
git@github.com:tvlfyi/kit.git will keep the same hashes, as it's updated
to export the unsigned workspace view of it.
This is less invasive than dooming every josh workspace to have to strip
signatures.
Change-Id: I6de05182fad4c3695081388c3bbf37306521d255
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8369
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
I *want* search engines to index our CLs, they might be useful!
Change-Id: I956d92c80d812e1aefefb6daeba77a1588055b94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8361
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
We missed renaming this as well while iterating over
https://cl.tvl.fyi/c/depot/+/7950.
Change-Id: I704d3b60bb3beb1a2148e27bdd4a49075a6649b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8230
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
It's been a long time since we updated josh, almost 400 commits in
between. I read through the entire changelog, and here are relevant
josh commits from in between that might be interesting to us:
38eecee Fix optimisation bug for compose filter (#1159)
e1d10b6 Add :rev(...) filter
0f1a07b Initial implementation of refs locking (#929)
88cea2a Initial work on meta repo support
030ad93 Change magic refs to include "for"
28b1d75 Add split changes feature (#904)
1f908d7 Discover filters only on HEAD (#774)
a368d8f Make --require-auth only apply to push
8d80230 Add :linear filter (#741)
3460ec2 Implement redundant refs filtering (#700)
55b4e50 Implement stacked changes support (#699)
ea1f814 Handle @sha urls by creating magic ref (#690)
883a381 Run filter discovery only on changed refs (#685)
4bb004f Prepend refs/heads to base parameter as default (#664)
Of particular interest is a368d8f, which allows us to drop our
authentication patch and use the standard --require-auth flag again.
The default behaviour of dropping signatures on commits (which are
invalid after filtering) has also been changed in josh, now only
occuring when the `:unsign` filter is present. Since this breaks
commit hashes with our existing exported histories, we are opting to
set a `:unsign` filter prefix on all proxy requests to ensure that the
hashes stay consistent.
During this update we found a bug (josh#1155) which was fixed in the
commit that this CL moves josh to.
Change-Id: I3afac1619f3aa90313a0441da91f0e4a96fe0a3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8186
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This allows passing in custom triggers to trigger a (re)deploy.
For example, a caller can put an AWS instance ID into the triggers to
cause a redeploy whenever the instance ID has changed.
The `target_name` terraform variable was doing something similar, but
`triggers` is more generic, allowing multiple triggers, without having
to stringify them.
We also don't need to trigger on the attrpath - it can be changed, and
as long as it still evaluates to the same
`data.external.nixos_system.result.drv` (which is checked on every
plan), no redeploy needs to be made.
Change-Id: I94ce787a50830b87b6f53c08e042e4abe4036bdd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8191
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
