Initial setup which does not yet include fetching mails at all, this
is for now only going to display a manually populated view of the
existing mailing list while the rest of this stuff is set up.
Change-Id: Ie1235bd257c9056fe37d0740dfca771ebdd880eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7628
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This is a follow up to cl/7191 which neglected to adjust the
status.tvl.su.nix module and re-enable it.
Change-Id: Icc1917004cd50e5eab61a29bc68b393ba9bd6325
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7226
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Upstream nixpkgs removed a lot of aliases this time, so we needed to do
the following transformations. It's a real shame that aliases only
really become discoverable easily when they are removed.
* runCommandNoCC -> runCommand
* gmailieer -> lieer
We also need to work around the fact that home-manager hasn't catched
on to this rename.
* mysql -> mariadb
* pkgconfig -> pkg-config
This also affects our Nix fork which needs to be bumped.
* prometheus_client -> prometheus-client
* rxvt_unicode -> rxvt-unicode-unwrapped
* nix-review -> nixpkgs-review
* oauth2_proxy -> oauth2-proxy
Additionally, some Go-related builders decided to drop support for
passing the sha256 hash in directly, so we need to use the generic hash
arguments.
Change-Id: I84aaa225ef18962937f8616a9ff064822f0d5dc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6792
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
otherwise posting this to reddit's /r/russian is not possible, as they
ban all links to Russian-affiliated sites
Change-Id: I8d23f0961ec7ef097fc2dbdd0aaa178861a19c10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5992
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
at some point in the far past, there was an RSS feed at `/en/rss.xml`.
It seems to still get a single hit or so every hour, which currently
404s.
Change-Id: Ieb13c2c0232861a50a54bc2a4087d9ccb21185cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5962
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
on whitby, cgit runs as the gerrit user to get access to serving
gerrit's repositories directly.
on other machines (e.g. sanduny) this isn't necessary, as we have a
world-readable depot replica.
Change-Id: Ibf7e7cc08e5909e0fa182e561ab0cb472188edcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5932
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
this configures gerrit's built-in replication plugin to push every
change in depot to sanduny.
this allows us to serve a replica of depot from sanduny.
manual config that was needed which needs to be automated:
* system-wide known_hosts does not work, needed one in /var/lib/git
* .ssh/config MUST be present and configured for sanduny.tvl.su
Change-Id: Iba399f2328abb5acb65dae19a36e265eea0952ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5915
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
This module sets up a user with an SSH key and permissions to receive
a (pushed) replica of depot from Gerrit.
This still needs appropriate configuration in Gerrit's replication
plugin on the other end.
This module has been enabled for sanduny. For now it does not (yet)
configure git serving.
Change-Id: I0fb6f7e696609e71008308e855bdf305dcbcd4f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5913
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
It occured to me yesterday that with the config inside of the module
it is kind of difficult to test cgit locally.
This moves it back to a separate location (//web/cgit-tvl) and makes
the most important things configurable via overrides.
Change-Id: I9b0f4c60b75c31441e1718e63b5b55aba3100aae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5893
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This became an "official" module and dropped the `pw-` prefix.
Relates to b/184
Change-Id: I963f83b55b83015b022ab1b8330ea710d2258631
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5751
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
When Keycloak and oauth2_proxy are restarted simultaneously, the
latter might try to come up (repeatedly!) before Keycloak can serve it
properly.
This leads to systemd considering the unit failed.
Since this all happens in the span of a second or so, slightly
increase the restart delay of the service to ensure it comes back
after Keycloak is ready.
A "proper" fix might be to add a script that runs before the actual
service and waits for Keycloak, but I don't want to prioritise that
right now.
Change-Id: I4dadba686de60ffc103fe889ce19f05ca1d7d4fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5695
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Somehow this ended up generating an empty file, with this change it is
fine again. I was looking at the recent commits of the module in
nixpkgs but couldn't quite figure it out, there are also some vague
references to the attribute set key being used as a hostname, but this
doesn't seem to be true in practice.
To be clear, the previous code was wrong, but at some point it
generated a file that accidentally worked.
Change-Id: I42d55730c09daafe6d6fe0eb3647135e84737bca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5670
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
libdigidocpp is a dependency of qdigidoc4(1) already.
This will need https://github.com/NixOS/nixpkgs/pull/174055
"libdigidocpp: Fix PKCS11 module library path" to work, though.
Change-Id: Ic8d671077977b1d1f099a8b4b23cc537b52aa954
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5647
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
The new version brings the new secretsDir setting which means we no
longer have to hardcode /run/agenix everywhere.
Change-Id: I4b579d7233d315a780d7671869d5d06722d769fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5646
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
Nixery is going to gain a new binary (used for building images without
a registry server); to prepare for this the server binary has moved to
cmd/server and the Nix build logic has been updated to wrap this
binary and set the required environment variables.
Change-Id: I9b4f49f47872ae76430463e2fcb8f68114070f72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5603
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Firefox users can add p11-kit-proxy (or other SecurityDevices)
system-wide, by making use of the extraPolicies functionality.
Change-Id: Id58b6cab425199fb0e09e846db2a86d302c0de0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5534
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Cleans up a whole bunch of things I wanted to get out of the door
right away:
* depot internal references to //third_party/nixery have been replaced
with //tools/nixery
* cleaned up files from Github
* fixed SPDX & Copyright headers
* code formatting and inclusion in //tools/depotfmt checks
Change-Id: Iea79f0fdf3aa04f71741d4f4032f88605ae415bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5486
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Someone already packaged the required software, so I didn't have to do
that.
Change-Id: Ifc6a68fd4cd89f4718368a05acb6c6f536e01aab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5431
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
If the Keycloak service is running on the same machine as the oauth2
proxy (spoiler alert: it is!), let the service depend on it.
Change-Id: I30e4222b4cd5589e08849ef6f37cf1fb4369f55a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5421
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
As suggested by sterni, this makes the self-redirect of a machine to
its configuration a generic module working by convention.
In the process of moving this two small fixes have been applied:
* redirect is only applied if the URI is `/`, this is required for
ACME to work
* addSSL = true is added, otherwise we have a certificate but no TLS
listener
Change-Id: Icaef041ff681253a61e36926417bdb2844e3f93d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5313
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This makes the journaldriver configuration machine-independent.
The secret is loaded from agenix instead of being persisted on disk.
Change-Id: I592ae7f5726fcb7f37a406f69dcf5ac498eeb1b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5302
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
With this change, entering just "whitby.tvl.fyi" or "sanduny.tvl.su"
in a browser will redirect users to their machine configurations.
Change-Id: Ibf076a469bcce073e1b1970aa568d6fe16a5c75a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5304
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
This needs to be present on all machines that run ACME stuff.
I've switched the address for a .su one because I have a catchall for
these.
Change-Id: I7af8e1f1cb2fcfbcba4b7d1930ed0edef0106d72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5306
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Rather than defining all system users inline on whitby, move them into
a module that can be imported on multiple machines.
Configuration for terminfos that we've added follows along.
Note that while doing this I've disabled logins for riking and isomer
since they are currently inactive in TVL.
Change-Id: Id18031d355afc34079c5e6e49dc6943e61809a8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5298
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
cgit has its own module now
Change-Id: I9b4cc322374517b8bd3db43345831e2bf43c4bb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5295
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
The ancient `//web/cgit-taz` path stems from the time I had
code.tazj.in serving my initial version of the depot.
I've been meaning to clean this up for forever, so here we go.
Note that this leaves the git-serving module in a strange state where
it only deals with josh. I'll rename it accordingly.
Change-Id: I47ed1e9d90958299b5440a18a1b9075274754e33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5294
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Since our blog index is on the index page, this makes slightly more
sense.
Change-Id: I7b8164490c133e23d892abef21275f8bfed50b66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5123
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
This was already happening without the trailing slash, but needs to
happen separately with it.
Fixes: b/172
Change-Id: Ic3423fd7a2eaf76a073badd80965cee953df4ce9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5121
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
This means it will end up in journaldriver.
Change-Id: I66f781085b5dac9946b3b9a2bf30e447863e1213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5122
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
The cookie secret in the encrypted file was too long, because the
generation command in the oauth2_proxy docs is also wrong. Should
probably fix that upstream as well.
Also noticed that an extra '2' snuck into the service name and fixed
that.
Change-Id: I9a344a75993ab1f98299a8d45e7f5b2e146b7fc5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4957
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Adds a feature to emergency-stop deploys by simply running `touch
/var/lib/auto-deploy/stop`.
This can be useful in some situations, especially if there is a
process that reconciles service state (so that e.g. stopping the
unit's timer would be undone).
Change-Id: I233dfac365a578bfa4110eb605b50be079974ba4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4827
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
The priority of binary caches is decided by the remotes in Nix (???),
and by default nix-serve (which is *very* slow) has a lower priority
than cache.nixos.org (which means that it will be preferred over the
faster cache for paths that exist on both).
To avoid this, override the hardcoded (????) priority by serving the
nix-cache-info response directly from nginx instead.
Change-Id: I15a2d6618386d16edaf69f1c9257a36bd72132d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4823
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
The intent is to configure oauth2_proxy pointing at Keycloak to enable
usage with nginx auth_request directives.
I want to expose this as a function from within the module in which
nginx server configuration blocks can be wrapped, but the function for
that is currently a placeholder.
Change-Id: I5ed7deb9bf1c62818f516e68c33e8c5b632fccfe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4767
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
It looks like we won't need this for oauth2_proxy when combined with
nginx auth_request setups.
Change-Id: I2294aee6226b4f64a27bf6592c2d18092d0268cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4766
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Building nix derivations needs tar (provided by gnutar) and gzip on the
PATH in order to extract .tar.gz archives.
Change-Id: Ia2df7a3a770cfd342dfede58ad34e04805fbd1f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4685
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Note that the login.tvl.fyi WWW configuration is still kind of hanging
around until we've settled where Keycloak lives.
Change-Id: Iaca4e394a7371cafa3716ca66ef09c4eca5b1520
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4626
Autosubmit: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Trialing this as an alternative to CAS that is a little easier to
configure and can help us delegate authentication to other OIDC
services.
Change-Id: Iad63724d349334910af8fed0b148e4ba428f796b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4608
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
I'd like to be able to run extra CI steps that include running docker
containers (to integration test things like webapps that connect to a
database). To do this the buildkite agents themselves need permission to
do docker things.
Change-Id: I3c9a488708f0e12a508754ac41f04148ca7aedac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4408
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
For modules that are gated behind a mkEnableOption, it's reasonable to
just provide them to all Depot-built nixos systems without requiring
people to explicitly import them. This defines a special module called
`default-imports.nix` which imports these modules (currently just
tvl-cache.nix and automatic-gc.nix, as I'm being rather conservative
adding things here to avoid breaking anyone's system), then provides
that module as one of the `modules` passed at the top-level
nixos/eval-config invocation.
Change-Id: I3be299ab10ae4c451ef11c514edb3c89318a2278
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4345
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Add a shared nixos module for configuring whitby as a binary nix cache,
and refactor tverskoy to use this module.
This is enabled via an option to pave the way for including it as an
import in all depot-generated nixos configs at some point in the future.
Change-Id: I6dcc0e8eb48b1ac34457666dceebeedd5da6c526
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4344
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: grfn <grfn@gws.fyi>
https://cl.tvl.fyi/c/depot/+/4264 did move merging config with secrets
into ExecStart=, which is tracked in an RFE upstream:
https://github.com/systemd/systemd/issues/19604#issuecomment-989279884
We didn't link to this so far, neither in the commit message, nor in a
comment.
Let's add a comment, so people know when we can undo this.
Change-Id: I7bed370b671093bb876592b4dccd562f1c256cd2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4326
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Git only allows binary names prefixed with `git-credential-` if the
path to the helper is not absolute.
Why? Who knows.
Change-Id: I216b2a621f62a73f05e21def7ec8016b29ede892
Currently this functionality is provided by a shell script stored in
/etc/secrets (which has the password value hardcoded).
This needs to happen in a separate commit from the one that changes
the pipeline to avoid breaking it (it needs to be deployed first).
Change-Id: I680754c828ccefbacfcf0d5c813a4bc19493ba4c
We already checked this in, but this commit adds the configuration for
making use of it.
There are two copies of besadii's JSON configuration with different
permissions.
Note that the buildkite-graphql-token path needs to be updated in
static-pipeline.yml, but this needs to happen in a separate commit
after deploy because the pipeline will break otherwise.
Change-Id: I6fab4bf1a2e679df7cf76521e2b53bd9dadbac62
... this option really is a pitfall! The list of programs is now the
same as in the upstream module, plus curl and jq.
Change-Id: I29edae4b2400a2724f62df9efa1dc184a8b0af5f
The DynamicUser + Group configuration does not work as planned, thus
the systemd LoadCredentials feature is used instead which makes the
file (which itself is only readable by root) available in a
memory-backed location only readable by the service.
The secret is only available to `ExecStart` commands, so units using
this feature can not be used with pre/post units and the like if those
commands need secrets.
To accommodate this, the merge of configuration files has been moved
into the service launch script, which is now the ExecStart= process.
For details take a look at https://www.freedesktop.org/software/systemd/man/systemd.exec.html#LoadCredential=ID:PATH
Change-Id: I693fe5677cc0d63c7aa485c2c7472457c5262166
It turns out the lib.mkAfter call doesn't behave as expected -
only *some* of the packages that are defaulted end up in the $PATH.
I suspect this is actually something else, e.g. these packages are
always added for some reason or another, and the option is completely
overridden every time.
Change-Id: I854c7198520d82b00e6338ed0fe653836226dc6d
Turns out that the type of this option is not concatenative and it
replaces the packages needed to run Buildkite if set.
Change-Id: I9f52572bc165bccdd8c6518cfdf7b8967f7a50d0
The irccat module uses DynamicUser, so to grant permission to it a new
group has been added for irccat.
I have some vague memory of DynamicUser + Group not behaving as one
would expect, but we'll see what happens.
Change-Id: Iab9f6a3f1a53c4133b635458ce173250cc9a3fac
This is not yet including the secret configuration for gerrit-queue,
and just expects the secret (gerrit username & password) to be
available in /etc/secrets.
Change-Id: Ia465ef7f3f521c70d606d7fdeba9aa83c7e1b98b
This is required for a simplification of the build pipeline (following
CL) and needs to be in a separate commit as it can not be done
atomically (merging the other commit to deploy it would immediately
break pipelines otherwise).
Change-Id: I5d8ec8f3238f79b5518d799486bf98d1d9516c43
Ensure that besadii sees $0 as the correct command name, since that is
the sole mechanism by which its functionality is switched around.
There was a lingering commit that introduced this bug and hadn't been
deployed in a couple of days. Maybe time to tighten deploy cycles soon
...
Change-Id: Ie4284c0f6e5e06d71a71a3702ec7e092260e0ce5
On whitby, the besadii config will live in
/etc/secrets/besadii.json. This CL updates the call sites to pass this
config path to besadii so that it can load Sourcegraph configuration.
Change-Id: Ia139b9fa3b827e7a5f2386214390acc6fe19a75a
Change the Nixery configuration to use the plain nixpkgs package path
instead of the depot path. AFAIK, nobody uses this to fetches depot
packages at the moment - but plenty of people fetch non-depot
packages.
This means that Nixery is cache-busted less often (previously on every
commit => every deploy).
We'll figure out another way to have a depot Nixery later.
Change-Id: Iba632333346181c3d2ce992fbab396ed0d9f86aa
Removes besadii support for the previously used 'ref-updated' hook and
instead introduces support for the 'change-merged' and
'patchset-created' hooks.
These hooks more accurately capture the semantics of when besadii
should trigger CI builds and using them will avoid problems such as
skipping 'canon' builds if chains of CLs are submitted together.
Change-Id: Ib90356c069780bf0c0250e56b927e46a5b31ce7f
Since GCP nuked us, the backups are now moving to GleSYS'
S3-compatible object storage.
This refactors the restic module to support S3-compatible storage
instead of GCP, and switches to the appropriate new secret paths.
The secrets were placed on whitby manually and I verified that the
backups work.
This fixes b/157
Change-Id: I6a9d2b0581967605ce736605a3befb44cdeae7e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3883
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
The setup is explained in the comment, but TL;DR: Use the derivation
hash of static files to create permanent URLs.
Relates to b/151.
Change-Id: Ib1ca3a1a00c90a47f4bf39c29a8b4bbf5b215e7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3664
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
As cschilling explained on cl/3563, there isn't actually anything in
this state that we *need* to persist. We're still keeping it in a
persistent directory on disk as this serves as an optimisation after
restarts of josh.
Change-Id: Ia88886792a5acac34508b5b8a669bd519ca033de
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3631
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This lets each service declare their backup paths together with the
configuration for the service, which is a lot more sensible than what
we had before.
Fixes b/147
Change-Id: If76fe62639f4cc0e6fbb63a2959d584479d8f0fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3583
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
I can never remember which is which.
Change-Id: I69b8235862b8c5b49030a74bfca25aaa113273b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3582
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Relates to b/147.
First step towards giving depot modules the ability to declare their
own backup directories by moving all restic configuration into a new
module and adding a NixOS option for inclusion/exclusion paths for
backups.
This still keeps all backup paths within the whitby config.
Change-Id: Ia96833668f1a3d02da892261153d8b02156b8ac0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3565
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Previously we served the dumb git HTTP protocol from code.tvl.fyi via
cgit. This CL disables this feature and instead runs josh in the same
location (by redirecting appropriately), but while also enabling
partial cloning of all subtrees of the depot.
For example, after this CL the following would result in an
independent clone of //nix/readTree:
git clone https://code.tvl.fyi/depot.git:/nix/readTree.git
Note that there are no josh workspaces configured at all for now,
these references are only for static depot subpaths.
Please refer to the documentation for josh for more information on
available kinds of josh filters.
Josh state is kept in a systemd state directory in /var/lib/josh and
backed up to Restic. Backing this up is necessary, as josh uses
stateful information to do things like tracking merges and rewriting
history per subtree appropriately to avoid cloned repositories ending
up in peculiar states.
Change-Id: I156f0298c2aa42e3bdbf5a0e86109070d640c56e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3563
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
