Commit graph

917 commits

Author SHA1 Message Date
Vincent Ambo
abaa90c3ab fix(ops/pipelines): limit concurrency of 🦙
When pushing a large chain of CLs, builds can fail with OOM issues as
many Nix evaluations of the depot are happening simultaneously.

To work around this, we limit the concurrency of simultaneous Nix
evaluations (i.e. the `:llama` step). This can slow down the start of
builds in a large chain of small changes, but that is a better
tradeoff than failing the builds entirely and making people click
buttons.

Change-Id: If351aaad22d52e2bcf871377f22ab1df594c518d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7501
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-03 14:11:04 +00:00
Lyle Mantooth
4cd4111d0d feat(ops/users): add IslandUsurper to users.
Change-Id: Id6bda45acd33dc4e57775321aa8f318164ca7ee0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7469
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-12-01 10:42:47 +00:00
Márton Boros
c06840ff87 feat(ops/users): Add brainrake to users
Change-Id: I6bb611fd802ed3f1e748d4c75dc2fd4bea9cc91a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7365
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-11-23 13:55:53 +00:00
Vo Minh Thu
888b7faa18 feat(ops/users): Add noteed to users
Change-Id: I40b99a46b76d0df40b811350f3560c629babdbc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7319
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-11-21 13:40:33 +00:00
jhahn
40826e664d feat(ops/users): Add jrhahn to users
Change-Id: I00913a302ecc23fec2e60875dc164b24d73ba4ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7257
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-11-09 21:57:54 +00:00
sterni
83dabf8955 fix(ops/machines/whitby): serve grafana at status.tvl.su again
This is a follow up to cl/7191 which neglected to adjust the
status.tvl.su.nix module and re-enable it.

Change-Id: Icc1917004cd50e5eab61a29bc68b393ba9bd6325
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7226
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-11-07 14:43:18 +00:00
Griffin Smith
8240b2959e chore(whitby): Update grafana config
Uncomment and update the grafana config for whitby based on the new
config format that nixos accepts. I've validated this locally by
visually inspecting the resulting `ini` file, but not actually run it
yet.

Change-Id: I12d78ae48146e1b01bd2a4152276d4c6b16c1a3d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7191
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-11-05 15:06:41 +00:00
sterni
57cf952ea9 chore(3p/sources): Bump channels & overlays (OpenSSL edition)
* //ops/machines/whitby: Disable grafana, since the grafana module was
  changed upstream in a way that our configuration no longer works.
  Since the OpenSSL security update is relatively pressing, adapting the
  grafana configuration beforehand is not a hard requirement. See
  https://github.com/NixOS/nixpkgs/pull/191768.

* //tools/depotfmt: keep Go at version 1.18 to forgo a reformat of the
  tree.

* //nix/buildGo: keep Go at version 1.18, as 1.19 changed the CLI
  interface (?) in a way that breaks buildGo.

* //3p/overlays/tvl: drop upstreamed tdlib upgrade.

* //3p/overlays/tvl: patch buf to work around breakage due to git 2.38.1

TODO items for Go are tracked in b/215.

Change-Id: Ie08fef49cf3db12e6b5225a8b992a990ddc5b642
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7141
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-11-03 15:10:39 +00:00
Florian Klink
79ef6ee283 chore(ops/pipelines/depot/protoCheck): include name in label
Change-Id: I2010bd6e4600e9f1dd6e6af40e81ecbbb72c20d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7054
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2022-10-27 20:34:58 +00:00
sterni
ca3bd5c7ca feat(ops/pipelines): allow accessing the nix store
This is already allowed de facto, since there seems to be a special
exception for reading from derivation outputs. What is forbidden, is
access to files imported to the store (even via builtins.toFile) and
derivation files. The latter is required for doing dependency analysis
on arbitrary derivations, unfortunately.

Access to the store allows kind of evil things, but it should
be (hopefully) hard to do this by accident, and accessing derivation
files is not impure, though it relies on store implementation internals
so to speak.

Change-Id: I33a7de83ef0ee20a7076690329d62f6caffffe5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6835
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-10-08 10:59:45 +00:00
Vincent Ambo
b9bfcf2f33 fix(ops/www): fix port templating for keycloak
Change-Id: I714b12f996d7dbe705f1f553d449f2dbc4910b1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6848
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-10-03 17:42:56 +00:00
Vincent Ambo
e255aff849 chore(ops/whitby): use renamed 'kbdInteractiveAuthentication' option
Relates to b/200

Change-Id: Ica7a32e3d2392aba22c2de93cc9be49c4a57eeb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6838
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-10-02 14:19:33 +00:00
Vincent Ambo
b5cf6b148c chore(ops/whitby): use new keycloak HTTP port option
Relates to b/200

Change-Id: Id8f415d5c4a8947b56031e1671f4f84ac5f2665d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6837
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-10-02 14:19:33 +00:00
sterni
0c178a0ef6 chore(3p/sources): Bump channels & overlays
Upstream nixpkgs removed a lot of aliases this time, so we needed to do
the following transformations. It's a real shame that aliases only
really become discoverable easily when they are removed.

* runCommandNoCC -> runCommand
* gmailieer -> lieer
  We also need to work around the fact that home-manager hasn't catched
  on to this rename.
* mysql -> mariadb
* pkgconfig -> pkg-config
  This also affects our Nix fork which needs to be bumped.
* prometheus_client -> prometheus-client
* rxvt_unicode -> rxvt-unicode-unwrapped
* nix-review -> nixpkgs-review
* oauth2_proxy -> oauth2-proxy

Additionally, some Go-related builders decided to drop support for
passing the sha256 hash in directly, so we need to use the generic hash
arguments.

Change-Id: I84aaa225ef18962937f8616a9ff064822f0d5dc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6792
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-09-28 08:02:31 +00:00
Vincent Ambo
6576c2f15f feat(ops/keycloak): import github identity provider configuration
For some reason Terraform decided that it would otherwise like
to *delete* this configuration, which is undesirable.

Note that there is a "magic" special behaviour when the `alias` and
`provider_id` are set to the name of a built-in supported
provider (github, gitlab etc.), which lets us skip the
authorization_url setup.

Change-Id: Ib66154c2896dda162c57bdc2d7964a9fa4e15f20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6706
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2022-09-20 09:28:45 +00:00
Vincent Ambo
3a1f4831a8 feat(ops/keycloak): add SMTP settings in configuration
I think these were set up in the UI and previously not supported in
the Terraform config, now they're supported and Terraform wanted to
delete them ...

Change-Id: I83eb49ceb774ac835dc81638f962e937c7e936c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6707
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2022-09-20 09:28:45 +00:00
Luke Granger-Brown
adf092a26b feat(monorepo-gerrit): swap owners plugin for code-owners
Change-Id: I9e05384b58dac258bc2da41c22e321b20451ef00
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6686
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
2022-09-19 11:17:07 +00:00
Luke Granger-Brown
f190712b7f chore(gerrit): migrate OWNERS files to code-owners style
Change-Id: Iacc521dfdd4b4a2d5cef3920cf8189bcce35a488
2022-09-19 11:13:28 +00:00
talyz
4b1fb8868d feat(ops/users): Add talyz to users
Change-Id: I3bbc9d31e4d00b26dcef470816c0b44a949ecb7a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6614
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-16 17:30:26 +00:00
Vincent Ambo
beb78c7104 feat(ops/modules): deploy tvixbolt to tvixbolt.tvl.su
Change-Id: I534cf918fc3e03ce8c14cf15f6d3280b6a657c8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6536
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-13 11:05:54 +00:00
Vincent Ambo
20b855042d feat(ops/glesys): add CNAME for tvixbolt.tvl.su
I could not get nginx to serve it from `tvl.su/tvixbolt`, and ran out
of interest in trying to fix it, so lets put it on a subdomain
instead.

Change-Id: I1313d75cc9831d94a894191376534b1e5186a76a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6537
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-09-13 11:05:53 +00:00
sterni
aaa994137a fix: reflect renames of Nix configuration options
Change-Id: I7e28ac3d71acd7d99a1d3ef97bef9422097e4abf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6154
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-08-25 16:34:39 +00:00
Vincent Ambo
20fc7bc0b2 chore(3p/sources): Bump channels & overlays
* tvl-slapd: move database to subdirectory (somehow now required)

Change-Id: I1792b856cf68b11959c0cc9caab4135e556f8c58
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6090
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 14:43:05 +00:00
Vincent Ambo
d2176bb8fb feat(ops/www): add predlozhnik redirect on tazj.in
otherwise posting this to reddit's /r/russian is not possible, as they
ban all links to Russian-affiliated sites

Change-Id: I8d23f0961ec7ef097fc2dbdd0aaa178861a19c10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5992
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-07-28 17:50:18 +00:00
Jean-François Roche
c04316bdc5 feat(ops/users): Add jfroche to users
Change-Id: I60cb0acffd1d21b4660e819799206a0cde4facb0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5970
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-07-22 12:36:08 +00:00
Vincent Ambo
7d3d3b3c8f refactor(ops/glesys): add explicit records pointing to whitby
instead of setting a wildcard record (which causes really weird
behaviour if you set your search domain to tvl.su/tvl.fyi, which I
do), DNS records for services running on whitby are now set
explicitly.

Change-Id: Ia05399b62dad326942fe0efda30782ce153df99d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5961
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-07-22 11:17:58 +00:00
Griffin Smith
7ac9b76c3c feat(grfn/system): Add ogopogo
This is my new work desktop

https://en.wikipedia.org/wiki/Ogopogo

Change-Id: I198d8757ff85eec00a303b990efdd2658cbc3e6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5963
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-07-20 02:48:41 +00:00
Vincent Ambo
482e1b201c fix(ops/www): redirect very old tazj.in feed URLs correctly
at some point in the far past, there was an RSS feed at `/en/rss.xml`.
It seems to still get a single hit or so every hour, which currently
404s.

Change-Id: Ieb13c2c0232861a50a54bc2a4087d9ccb21185cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5962
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-07-19 15:14:29 +00:00
Vincent Ambo
5d65d8e03a fix(ops/www): issue certificate for 'www.tazj.in'
Change-Id: I6179f785bb6bd6168a2a11836b90da5ee93adc69
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5953
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2022-07-18 14:02:58 +00:00
Vincent Ambo
fcfd097e65 refactor(ops/cgit): make user configurable
on whitby, cgit runs as the gerrit user to get access to serving
gerrit's repositories directly.

on other machines (e.g. sanduny) this isn't necessary, as we have a
world-readable depot replica.

Change-Id: Ibf7e7cc08e5909e0fa182e561ab0cb472188edcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5932
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-07-12 08:49:55 +00:00
Vincent Ambo
39d589b84b fix(depot-replica): make the depot replica world readable
Change-Id: Idc0b5210793ab0d83b3ac99cf36d7f7f02a35a37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5931
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-07-12 08:49:55 +00:00
Vincent Ambo
4c25c29505 feat(ops/sanduny): run cgit instance
Change-Id: Id869fa46d74f215a9034e86f795a4cd9e93acb16
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5930
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-07-12 08:49:55 +00:00
Vincent Ambo
c08e47903e feat(ops): configure depot replication to sanduny
this configures gerrit's built-in replication plugin to push every
change in depot to sanduny.

this allows us to serve a replica of depot from sanduny.

manual config that was needed which needs to be automated:

* system-wide known_hosts does not work, needed one in /var/lib/git
* .ssh/config MUST be present and configured for sanduny.tvl.su

Change-Id: Iba399f2328abb5acb65dae19a36e265eea0952ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5915
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-07-03 20:54:06 +00:00
Vincent Ambo
9bc049425a feat(ops/secrets): add private key for depot git replication
Change-Id: Iaf86d1fe635be8fbd9bc8a397999a2cffcc21606
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5914
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-07-03 15:02:21 +00:00
Vincent Ambo
6ab6724e4c feat(ops/modules): add module for receiving a depot replica
This module sets up a user with an SSH key and permissions to receive
a (pushed) replica of depot from Gerrit.

This still needs appropriate configuration in Gerrit's replication
plugin on the other end.

This module has been enabled for sanduny. For now it does not (yet)
configure git serving.

Change-Id: I0fb6f7e696609e71008308e855bdf305dcbcd4f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5913
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-07-03 15:02:21 +00:00
Vincent Ambo
f8ed0088ea fix(ops/sanduny): Enable our binary cache
Change-Id: I53f4c5b667018c0d3b01b307411200b66f6a7de3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5901
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
2022-06-27 18:17:52 +00:00
Vincent Ambo
1094306aa9 refactor(web/cgit-tvl): Move cgit config back out of module
It occured to me yesterday that with the config inside of the module
it is kind of difficult to test cgit locally.

This moves it back to a separate location (//web/cgit-tvl) and makes
the most important things configurable via overrides.

Change-Id: I9b0f4c60b75c31441e1718e63b5b55aba3100aae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5893
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-06-27 14:15:07 +00:00
Åsmund Østvold
2caa50c9bd fix(ops/besadii) test trigger.ref against configured branch
Before this commit besadii only worked for repos having
'refs/heads/canon' as main branch.

Change-Id: Ia2ceb8a720c675be84bc3d81b89338522cea6ebd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5862
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
2022-06-09 15:26:25 +00:00
Vincent Ambo
8e8d6eb1df refactor(ops/keycloak): Use tools.checks.validateTerraform
Remove some ~commit message~ ... uh, code duplication.

Change-Id: Id6e8f2132999e153d3984848f95ccabd52e4f45f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5853
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
2022-06-07 09:32:13 +00:00
Vincent Ambo
92a6e25380 refactor(ops/glesys): Use tools.checks.validateTerraform
Remove some code duplication.

Change-Id: Ia9e0b3b22926eb9e72f302e2c1ebcee68eaa1db9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5852
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
2022-06-07 09:32:13 +00:00
Vincent Ambo
72f91f032b refactor(ops/buildkite): Use tools.checks.validateTerraform
Remove some code duplication.

Change-Id: I7ff49e728e1bd584bca3b84cdc033d93e60aefc2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5851
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
2022-06-07 09:32:13 +00:00
Vincent Ambo
ebd701b221 fix(ops/glesys): Remove now unnecessary workaround
Remove a workaround for a GleSYS provider bug that was fixed in the
last release.

Change-Id: Ibd25de0b4dcccd781518d5d0ae1c75d296f6b05f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5845
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-06-06 11:05:12 +00:00
Vincent Ambo
89ba820059 test(ops/keycloak): Validate Terraform configuration in CI
Change-Id: I5602cf722b9fe9502c9d7610eefc7ba0ab647362
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5844
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-06-06 11:05:12 +00:00
Vincent Ambo
fceb3fef73 test(ops/glesys): Validate Terraform configuration in CI
Change-Id: I8d251d3ee1de77feca865d0a677041c9c485d211
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5843
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-06-06 11:05:12 +00:00
Vincent Ambo
af44916312 test(ops/buildkite): Validate Terraform configuration in CI
Change-Id: Ieef4d7d0a717107ee67432474683f3344b6561f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5842
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-06-06 11:05:12 +00:00
Vincent Ambo
469189afd3 feat(ops/buildkite): Import tvl-kit pipeline
Change-Id: I21f6e0adba3dca3be741761a226ab6810d8bcf8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5841
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-06-06 11:05:12 +00:00
Vincent Ambo
831b04a1e9 feat(ops/buildkite): Import main depot pipeline
Change-Id: Id470750aa90505002c6a7e4f840e56c4939ed391
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5840
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-06-06 11:05:12 +00:00
Vincent Ambo
b29b6a092c docs(ops/buildkite): Add documentation about this config
Change-Id: Ia61b15127c67cdd9dddcab9f3540f1aee949cd6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5839
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-06-06 11:05:12 +00:00
Vincent Ambo
c58cc1e690 feat(ops/buildkite): Bootstrap Buildkite Terraform configuration
In order to run this the secrets needs to be sourced, e.g.:

  eval $(age --decrypt -i ~/.ssh/id_ed25519 $(git rev-parse --show-toplevel)/ops/secrets/tf-buildkite.age)

Change-Id: I9f6a02c0dac22f584181635861ddbb06cf849f14
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5838
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-06-06 11:05:12 +00:00
Vincent Ambo
4288cf961e feat(ops/secrets): Add Buildkite API token for Terraform
Change-Id: I0930f4fb34015ddcaa791b07e4d5d87d069d2b0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5837
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-06-06 11:05:12 +00:00
Vincent Ambo
a027ee9f03 refactor(nix/buildkite): Rename "post" steps to "release" steps
This is in preparation for a subsequent CL that will do much more
significant changes in //nix/buildkite.

Change-Id: I80a8d67d3a7d593854c8d711572483c2581e7881
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5824
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
2022-06-03 17:36:15 +00:00
sterni
e2807ec934 fix(ops/nixos): use builtins.storePath to avoid dumping pkgs.path
This is a less invasive way to achieve the same goal as cl/5681, by
preventing the already existing nixpkgs store path from being dumped
again at the call site. To support nixpkgsBisectPath, we simply check if
pkgs.path is below builtins.storeDir and use builtins.storePath based on
that.

This is actually similar to the approach taken in the nixpkgs
documentation system which tries to limit the amount of nixpkgs that
needs to be dumped by using filterSource on specific subtrees of
nixpkgs. For this to work it has to insist on pkgs.path being an
ordinary Nix path, though.

Change-Id: Idf892f90a5d811184568e4702a901c334d56210e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5787
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-29 10:22:25 +00:00
Vincent Ambo
121fb13648 feat(ops/secrets): Add OAuth2 client secret for panettone
Change-Id: Icc53b161b260632e50b7bdc4c908912fd377bb87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5771
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-05-28 17:03:36 +00:00
Vincent Ambo
38be32c6b0 feat(ops/keycloak): Add OIDC client for panettone
Change-Id: Idb4352e3bbf412df5569aa988a78c6438063f93a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5769
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-05-28 17:03:36 +00:00
Vincent Ambo
aed1fbeb95 fix(gerrit-tvl): Use only one build filter
Buildkite can't handle more than one filter for the query; as of the
last commit it just returned an empty list.

I've verified with curl based on the request the previous attempt
constructed that this works as intended with only setting the commit.

Behaviour is probably undefined if there are two builds for the same
commit (i.e. a retry). Which one will you see? Who knows!

However, since the commit hash contains the Change-Id, we can't get a
situation where the build was for two different CLs at the same
commit. Gerrit wouldn't allow that.

Change-Id: I0dcd0ff44c28d3d15cba23461970bfc8483f4e48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5768
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-28 13:39:58 +00:00
Vincent Ambo
6c3465dc59 chore(ops/sourcegraph): Bump to 3.40.0
Change-Id: I77438201d8ed5237095b3d2e8a855dec3e58b641
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5766
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-28 11:58:34 +00:00
Vincent Ambo
b4c4ea074a chore(ops/sourcegraph): Bump to 3.39.1
Change-Id: I76d0a3ede7cc23a9a6e8db61ed7e9d91670f1699
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5765
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-28 11:57:20 +00:00
Vincent Ambo
e6ed840788 chore(ops/sourcegraph): Bump to 3.38.1
Change-Id: Ib1f4f9591acab537607c9d9c9b123e9c711e331b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5764
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-28 11:55:27 +00:00
Vincent Ambo
291dd44044 chore(ops/sourcegraph): Bump to 3.37.0
Change-Id: If333f28dd0bec4eb965a6e3005ef5aca810c86f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5763
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-28 11:53:41 +00:00
Vincent Ambo
793081905e chore(ops/sourcegraph): Bump to 3.36.3
Change-Id: I3a6caeeb06919b25a9c1200c8f286b0bd34916b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5762
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-28 11:48:27 +00:00
Vincent Ambo
d32fa2bd33 chore(ops/sourcegraph): Bump to 3.35.2
Change-Id: Ia829b4ffa2e7e37438f766d0ff98e504c0d856b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5755
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-28 11:37:15 +00:00
Vincent Ambo
c06d47b787 chore(ops/sourcegraph): Bump to 3.34.2
Change-Id: I865335006a091986f8a98e4d5da7161a25e948d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5754
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-28 11:29:01 +00:00
Vincent Ambo
c6024e7a41 chore(ops/sourcegraph): Bump to 3.33.2
Change-Id: I6568e3226a7ff0796cbf3748c0dab1530fb0fb6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5753
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-28 11:26:03 +00:00
Vincent Ambo
a99e33a107 chore(ops/sourcegraph): Bump to 3.32.1
Change-Id: I8efdf3dbfc5575f24c8e6996a7716d308f1446df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5752
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-28 11:21:27 +00:00
Vincent Ambo
9bff3ae373 fix(tvl-slapd): load argon2 module with new name
This became an "official" module and dropped the `pw-` prefix.

Relates to b/184

Change-Id: I963f83b55b83015b022ab1b8330ea710d2258631
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5751
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-27 23:48:37 +00:00
William Carroll
11a8eea0e3 feat(wpcarro/tarasco): Support tarasco 🇲🇽
Named after the Mexican restaurant, El Tarasco, in El Porto, which I live 3m
walking distance from.

Change-Id: I2cd4b68eaa974ad6c8fec73e0566bc0b831c57a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5743
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-05-27 23:21:25 +00:00
Vincent Ambo
e9e8e38db7 fix(ops/gerrit-tvl): Filter builds by commit hash
The patchsetSha is one of the things passed in to the `fetch()`
interface, and Buildkite's API (now?) supports filtering by the commit
hash in addition.

With this combination, we should not accidentally display builds for
the wrong patch set.

Change-Id: I6bb26dd7387f2dd00291990cadd38629ecda999b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5702
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-27 22:13:58 +00:00
Vincent Ambo
bdccd2c111 fix(ops/modules): Increase RestartSec= of oauth2_proxy service
When Keycloak and oauth2_proxy are restarted simultaneously, the
latter might try to come up (repeatedly!) before Keycloak can serve it
properly.

This leads to systemd considering the unit failed.

Since this all happens in the span of a second or so, slightly
increase the restart delay of the service to ensure it comes back
after Keycloak is ready.

A "proper" fix might be to add a script that runs before the actual
service and waits for Keycloak, but I don't want to prioritise that
right now.

Change-Id: I4dadba686de60ffc103fe889ce19f05ca1d7d4fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5695
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-27 21:10:35 +00:00
Vincent Ambo
772f8f1b90 feat(ops/pipelines): Evaluate depot pipeline in restricted-eval mode
Change-Id: Ic5b98a0777860b68dabb9a9b59e8c682236a71c7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4884
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-05-26 16:57:16 +00:00
Vincent Ambo
7a0a4224a5 refactor(ops/nixos): Prepare for restricted eval
Change-Id: I7b5304dda3040830fe90fc188b35da3fd95451a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5682
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2022-05-26 16:24:34 +00:00
Vincent Ambo
48dfefe40d refactor(sanduny): Prepare for restricted-eval
Change-Id: I83a404dc7dbaf5ca53659d03df4e4de461a9d046
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5688
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-05-26 14:17:33 +00:00
Vincent Ambo
250300f167 refactor(whitby): Prepare for restricted-eval
Change-Id: I7604ca29310d759b0ffee2ffb0048b6365a2894c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5683
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-05-26 14:17:32 +00:00
Vincent Ambo
bc42c5a61b fix(ops/modules): adapt for changed ssh.knownHosts
Somehow this ended up generating an empty file, with this change it is
fine again. I was looking at the recent commits of the module in
nixpkgs but couldn't quite figure it out, there are also some vague
references to the attribute set key being used as a hostname, but this
doesn't seem to be true in practice.

To be clear, the previous code was wrong, but at some point it
generated a file that accidentally worked.

Change-Id: I42d55730c09daafe6d6fe0eb3647135e84737bca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5670
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2022-05-26 10:05:54 +00:00
Vincent Ambo
e3a31b702a feat(whitby): Deploy private SSH key for build agents
Change-Id: I5b1dfaaf28e835cac5b897e18b015d90ac3b2857
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5665
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2022-05-25 23:53:09 +00:00
Vincent Ambo
77f096771d feat(ops/secrets): Add private SSH key for Buildkite agent(s)
The public key is:

  ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIME13zAw3Fk6qsbWCe6mH2zkxOJ+NmG+FwMjLw00mcWt buildkite@tvl

Change-Id: Ia8591e5df42727e4068f26865d83d0af85424fde
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5664
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-25 23:53:09 +00:00
Klemens Nanni
3a53587c2a feat(ops/modules/open_eid.nix): Access all key slots
`onepin-opensc-pkcs11.so` only enables PIN1, but PIN2 is also required.

Change-Id: Ic1c34ca58a46c2978c7e27e7a9b7e6a4d335ac0c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5648
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: kn <klemens@posteo.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-25 20:38:11 +00:00
Klemens Nanni
45c46d4a73 feat(ops/modules/open_eid.nix): Add digidoc-tool(1) to PATH
libdigidocpp is a dependency of qdigidoc4(1) already.

This will need https://github.com/NixOS/nixpkgs/pull/174055
"libdigidocpp: Fix PKCS11 module library path" to work, though.

Change-Id: Ic8d671077977b1d1f099a8b4b23cc537b52aa954
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5647
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-25 20:37:53 +00:00
sterni
03d1986316 feat(3p/agenix): update to 2022-05-16 and add to niv
The new version brings the new secretsDir setting which means we no
longer have to hardcode /run/agenix everywhere.

Change-Id: I4b579d7233d315a780d7671869d5d06722d769fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5646
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
2022-05-25 15:00:37 +00:00
James Landrein
dc346bb8cf feat(ops/users): add j4m3s
I've only been a couple months lurking in the IRC ...

Change-Id: Idebf96d3bf1124f0a97e11e0f854e8c6d4be8d8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5662
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2022-05-24 23:39:44 +00:00
Vincent Ambo
f31edeec1b refactor(nixery): Modernise structure of binaries
Nixery is going to gain a new binary (used for building images without
a registry server); to prepare for this the server binary has moved to
cmd/server and the Nix build logic has been updated to wrap this
binary and set the required environment variables.

Change-Id: I9b4f49f47872ae76430463e2fcb8f68114070f72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5603
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-23 15:04:56 +00:00
Vincent Ambo
045cf244b5 chore(ops/secrets): add key for tazjin/zamalek
Change-Id: Ieb2fe49a67940d7cfbd261edbe10d0a8577a466d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5628
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-17 10:14:02 +00:00
Vincent Ambo
302b754d7a feat(tazjin/nixos): Add system configuration for zamalek
This is my new Huawei MateBook X.

Change-Id: I32a8b77dd8f53b3c89bf63f448cd2880f9a457b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5554
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2022-05-10 13:41:33 +00:00
Florian Klink
e8855f4bef feat(ops/modules/open_eid.nix): document firefox
Firefox users can add p11-kit-proxy (or other SecurityDevices)
system-wide, by making use of the extraPolicies functionality.

Change-Id: Id58b6cab425199fb0e09e846db2a86d302c0de0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5534
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2022-05-08 13:52:27 +00:00
Florian Klink
84c62eb68b feat(ops/modules/open_eid.nix): use p11-kit-proxy
… instead of onepin-opensc-pkcs11.

This acts as a glue to multiple PKCS#11 modules, and reads configuration
files from /etc/pkcs11/modules.

p11-kit is also used to propagate the system trust store to NSS:
https://p11-glue.github.io/p11-glue/sharing-trust-policy.html

See-Also: https://p11-glue.github.io/p11-glue/p11-kit.html
Change-Id: I135c3a80a4eea0bd06f6b00089dc197c82476746
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5533
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2022-05-07 21:29:56 +00:00
sterni
fbbee5584b subtree(3p/cgit): update for git 2.36.0 support
Merge commit '51596ba1c25ff0dbba894153015203b4f1d3947b' into canon

Change-Id: Iaaf7a849d111aebc6bf85cec118439ba1d49f1e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5521
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-04 16:18:51 +00:00
sterni
5fae8b0826 chore(3p/sources): Bump channels & overlays
* //nix/buildLisp: ccl dumped images have fixed themselves… again

* //3p/git: rebase patch on 2.36.0

* //3p/overlays/haskell: remove upstreamed workarounds

* Disable everything depending on cgit temporarily, since it doesn't
  compile with git 2.36 yet.

Change-Id: I9dc11c0846641341adbdcc7162cbf149a15fe0cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5519
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-04 16:18:50 +00:00
Vincent Ambo
96aea32084 fix(ops/besadii): Fix output formatting for non-CL builds
Change-Id: Ie9ffb2d287f6c8a1e3ae45a7ad6671b9b8fa9c8a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5505
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
2022-04-26 09:25:03 +00:00
Vincent Ambo
c05c4995ab chore(3p/sources): Bump channels and overlays
Changes:

* updated keycloak configuration for new version
* migrate to emacs28 outside of //users, re-add emacs27 but with a
  warning attached urging people to migrate

Change-Id: I3e5765a63934541f72f6c4a8673d3b4671850c93
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5501
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-04-21 16:54:07 +00:00
Vincent Ambo
6716bf018c chore(nixery): Housekeeping for depot compatibility
Cleans up a whole bunch of things I wanted to get out of the door
right away:

* depot internal references to //third_party/nixery have been replaced
  with //tools/nixery
* cleaned up files from Github
* fixed SPDX & Copyright headers
* code formatting and inclusion in //tools/depotfmt checks

Change-Id: Iea79f0fdf3aa04f71741d4f4032f88605ae415bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5486
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2022-04-20 15:31:16 +00:00
Vincent Ambo
34e95f514c chore(journaldriver): Bump dependencies
Change-Id: I8819639bf5ddcc52510f20a92ab4b93af873682d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5475
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-04-17 13:16:34 +00:00
William Carroll
d843f0bf4c feat(wpcarro/ava): Support new machine
ava is my new (NixOS!) work machine :)

Change-Id: I1f089f00c02519d5d1d93d011f29075d53500e74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5450
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-04-15 19:07:51 +00:00
Vincent Ambo
e3cd8069ef feat(ops/open_eid): Add script for setting up browser integration
Change-Id: Ib339d62d862fd99dab2fda30376b8e47b337a26b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5441
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
2022-04-14 16:18:43 +00:00
Vincent Ambo
1a2fe4b063 feat(whitby): Increase prometheus retention time to 90d
Change-Id: I67287d7b1d8ee2c3004d381b5bc684bf4fc7d42c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5429
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-04-11 16:18:38 +00:00
Vincent Ambo
186c2822b0 feat(ops/modules): Add module for using Estonian e-residency card
Someone already packaged the required software, so I didn't have to do
that.

Change-Id: Ifc6a68fd4cd89f4718368a05acb6c6f536e01aab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5431
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-04-09 08:49:06 +00:00
Vincent Ambo
017238a1be fix(ops/oauth_proxy): Depend on Keycloak service
If the Keycloak service is running on the same machine as the oauth2
proxy (spoiler alert: it is!), let the service depend on it.

Change-Id: I30e4222b4cd5589e08849ef6f37cf1fb4369f55a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5421
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-03-31 13:27:48 +00:00
Vincent Ambo
e70428e75b refactor(ops/pipelines): Configurable GraphQL token location
For external users of the pipeline construction, the token might be in
a different path than `/run/agenix/buildkite-graphql-token`.

It is made configurable through the BUILDKITE_TOKEN_PATH environment
variable. This should be configured on the pipeline level to apply to
all steps.

Change-Id: I23c52e2d705e4134b8b013f8603f92e5533a6e44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5424
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
2022-03-30 11:06:49 +00:00
sterni
508a62b603 chore(3p/sources): Bump channels & overlays
* Remove use of aliases that have been removed in nixpkgs commit
  a36f455905d55838a0d284656e096fbdb857cf3a:

  - ncat
  - x11
  - nologin
  - dbus_libs
  - emacsPackagesGen
  - man-pages
  - pulseaudioLight

Change-Id: Ide603bf48bc7f77e10e4aa715ba025aece3644fd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5387
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-03-19 17:11:59 +00:00
Vincent Ambo
fb5f21067e fix(modules/quassel): Open firewall port automatically
Change-Id: Ie815495561f789590b5f49ecfd33441822f79047
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5382
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-03-11 21:38:40 +00:00
Vincent Ambo
2cff0712b3 refactor(journaldriver): Replace ureq with crimp
crimp is in TVL (//net/crimp), and it has fewer dependencies than
ureq (including - finally - no more old time or chrono).

Change-Id: I354f8f78b34a85abe3af671ffeffbe6a7fded5ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5318
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-20 15:46:57 +00:00
Vincent Ambo
2626d8348a chore(journaldriver): Bump medallion to 2.5
This version includes my patch for removing chrono from
medallion (https://gitea.cmdln.net/cmdln/medallion/commit/025b143d)

Change-Id: I2b745598538dd34e967e49c2b342be1b04ca9f27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5316
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-20 15:46:57 +00:00
Vincent Ambo
e8756239a8 chore(ops/modules): Remove fix-nginx timer unit
This doesn't seem to be needed anymore.

Change-Id: Id8d4192840e8ab10adb652abc9bd6540009a3dcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5319
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-20 14:26:57 +00:00
Vincent Ambo
71d6a02ca1 refactor(journaldriver): Use time crate directly instead of chrono
With this change, we still depend on chrono (through medallion), but
but I'm going to try and fix that upstream as well.

Change-Id: Iefd3d8578ea8870961107f3222dea7f936c2dd9a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5311
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:52:26 +00:00
Vincent Ambo
af512558e6 test(journaldriver): Add test for serialising timestamps
Change-Id: I5b769f5974546fd4f4f853111bd17c9d22d73a5e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5310
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:52:26 +00:00
Vincent Ambo
19a13e08a8 chore(journaldriver): Migrate to Rust Edition 2021
Change-Id: I858738b6fc554060824bbb4e69d5ccd03789535d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5309
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-18 11:52:25 +00:00
Vincent Ambo
c075a2c541 chore(journaldriver): Update crates within bounds
Basically just `cargo update`.

Change-Id: I86e58d73ff67d69201124d65136773325b240cbe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5308
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:52:24 +00:00
Vincent Ambo
ede837b687 refactor(journaldriver): Use anyhow instead of failure
Apparently failure is not hip anymore, and crate updates are forcing
the use of anyhow now. Whatever.

The functionality basically stays the same, maybe error messages will
look a little bit different now.

Change-Id: I173d644688785339c16161ddeec47a534123710f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5307
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-18 11:52:24 +00:00
Vincent Ambo
4ce2b49cd9 feat(ops/glesys): add DNS record for AAAA sanduny.tvl.su.
Change-Id: I4a74cd173b326941c12b7611841ced2038650137
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5314
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-02-18 11:47:23 +00:00
Vincent Ambo
d68de096c7 feat(ops/glesys): add DNS record for A sanduny.tvl.su.
Change-Id: I12e678f161ca9bfb7e982ed067a0b8bd0325d737
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5296
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-02-18 11:41:13 +00:00
Vincent Ambo
ac6717fe3c fix(ops/modules/www): Make self-redirect to config a generic module
As suggested by sterni, this makes the self-redirect of a machine to
its configuration a generic module working by convention.

In the process of moving this two small fixes have been applied:

* redirect is only applied if the URI is `/`, this is required for
  ACME to work
* addSSL = true is added, otherwise we have a certificate but no TLS
  listener

Change-Id: Icaef041ff681253a61e36926417bdb2844e3f93d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5313
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-02-18 11:39:01 +00:00
Vincent Ambo
5b701ad713 feat(sanduny): Enable journaldriver module
Change-Id: I9026386664000448642ff635bd71a7af5ed546c3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5303
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:38:34 +00:00
Vincent Ambo
536e01e967 refactor(ops/modules): Move journaldriver configuration into module
This makes the journaldriver configuration machine-independent.
The secret is loaded from agenix instead of being persisted on disk.

Change-Id: I592ae7f5726fcb7f37a406f69dcf5ac498eeb1b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5302
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:38:34 +00:00
Vincent Ambo
c72abe04f2 feat(sanduny): Configure Bitfolk nameservers
Change-Id: I81b252aedbf1ce3543a167b6c1942c404d4f1f1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5312
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:35:21 +00:00
Vincent Ambo
95780174e1 feat(ops/machines): Add a module for known SSH keys
Change-Id: I443e479f3edf9c6540de7b5a33bc6f7e2a9c5183
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5305
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-18 08:22:56 +00:00
Vincent Ambo
b936843bb0 feat(ops/modules): Redirect machine base names to their config
With this change, entering just "whitby.tvl.fyi" or "sanduny.tvl.su"
in a browser will redirect users to their machine configurations.

Change-Id: Ibf076a469bcce073e1b1970aa568d6fe16a5c75a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5304
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-18 08:15:56 +00:00
Vincent Ambo
f4f1d97052 refactor(ops/modules): Move ACME base configuration into base.nix
This needs to be present on all machines that run ACME stuff.

I've switched the address for a .su one because I have a catchall for
these.

Change-Id: I7af8e1f1cb2fcfbcba4b7d1930ed0edef0106d72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5306
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 08:15:56 +00:00
Vincent Ambo
6b3eed1fb5 feat(ops/secrets): Add journaldriver key
This changes the structure of secrets.nix a bit to split between
secrets for whitby, and secrets for all TVL machines.

Change-Id: I791f0ce42a16b33051e24a7a6c5b153761ed9eb3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5300
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-17 18:11:58 +00:00
Vincent Ambo
e1353ff2cf feat(ops/machines): Add tvl-users to sanduny.tvl.su
Change-Id: I20f54f4ab298cfee91062f7bf4cdc8b0b3ccb37c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5299
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-17 18:11:58 +00:00
Vincent Ambo
9e9284fc86 feat(ops/machines): add configuration for sanduny.tvl.su
This will be an additional web host / fallback git server for whitby
incidents.

Change-Id: Icd6f7ce574ffd520b5783a50ff317feed7b71fc6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5297
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-17 18:11:58 +00:00
Vincent Ambo
dd5ce78dbd refactor(ops/modules): Move user configuration into module
Rather than defining all system users inline on whitby, move them into
a module that can be imported on multiple machines.

Configuration for terminfos that we've added follows along.

Note that while doing this I've disabled logins for riking and isomer
since they are currently inactive in TVL.

Change-Id: Id18031d355afc34079c5e6e49dc6943e61809a8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5298
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-17 18:08:19 +00:00
Vincent Ambo
c72c1efdeb refactor(ops/modules): Rename git-serving -> josh
cgit has its own module now

Change-Id: I9b4cc322374517b8bd3db43345831e2bf43c4bb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5295
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-16 23:03:06 +00:00
Vincent Ambo
cb8f050b9c refactor(ops/modules): Move cgit configuration into a module
The ancient `//web/cgit-taz` path stems from the time I had
code.tazj.in serving my initial version of the depot.

I've been meaning to clean this up for forever, so here we go.

Note that this leaves the git-serving module in a strange state where
it only deals with josh. I'll rename it accordingly.

Change-Id: I47ed1e9d90958299b5440a18a1b9075274754e33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5294
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-16 23:03:06 +00:00
sterni
235eda6649 chore(3p/sources): Bump channels & overlays
* //nix/buildLisp: re-enable CCL, as the crash has been fixed upstream,
  although it is unclear what exactly caused / fixed it.

* //ops/whitby: the kitty build broke upstream, so we can't install the
  terminfo on whitby for a bit.

Change-Id: I5710acbe837fbc936e334b2e81f9cf00ed6ae280
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5274
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-02-13 09:15:09 +00:00
Vincent Ambo
3d8ee62087 style(rust): Format all Rust code with rustfmt
Change-Id: Iab7e00cc26a4f9727d3ab98691ef379921a33052
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5240
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-02-08 12:06:39 +00:00
Vincent Ambo
3318982f81 chore: move some meta.targets definitions to meta.ci.targets
Change-Id: Icdec1dec89158fb596c5185ac7105892081947f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5252
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-08 12:06:10 +00:00
Vincent Ambo
98b6af02f5 chore(mq_cli): Rename cargo project to mq_cli
I am trying to publish this to crates.io, and `mq` is already taken up
by what seems to be a dead project.

Change-Id: I14d1f5f31f167fde954d9c1e39fc9fec5f4c3d10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5234
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-02-05 13:46:11 +00:00
Vincent Ambo
0971705902 chore(mq_cli): prepare new release
Same trick as in the last commit, trying to hit the right revision.

Change-Id: I0af9e88b4d2fd8239a7819dbe0da13f26cca8d6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5233
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-02-05 13:37:31 +00:00
Vincent Ambo
77fbe57819 chore(posix_mq.rs): Prepare new release to crates.io
... and update some outdated stuff in the README while we're at it.

Change-Id: Ib3a12596bd1ba61e91ac6e1d37106b616da3030d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5232
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-02-05 13:29:04 +00:00
Vincent Ambo
91ef2b671e chore(ops/posix_mq.rs): upgrade to nix 0.23
The previous version had a CVE.

As part of this upgrade, the handling of errors inside of the Nix
crate changed, which we now accommodate.

Change-Id: Iad9a473c1782e0d79919cb5dc3f76316852d8a16
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5226
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-04 15:58:19 +00:00
Vincent Ambo
132798cceb fix(ops/posix_mq.rs): use newer struct mem::MaybeUninit
The previous mem::unitialized method was deprecated in favour of this
struct which carries information about the initialisation state
forward to the compiler.

Change-Id: Ib5f5d1ad91f9957b18eebabc1048f8649bc74049
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5225
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-04 12:47:55 +00:00
Vincent Ambo
054fe8f94b refactor(ops/posix_mq.rs): Update code using deprecated Error trait
The previous Error::description method was deprecated.

Change-Id: I63efd272484c8715b9f324ab09e967fbf53cf55a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5224
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-04 12:47:55 +00:00
Vincent Ambo
aa122cbae7 style: format entire depot with nixpkgs-fmt
This CL can be used to compare the style of nixpkgs-fmt against other
formatters (nixpkgs, alejandra).

Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: cynthia <cynthia@tvl.fyi>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: eta <tvl@eta.st>
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-31 16:11:53 +00:00
Griffin Smith
5f42a160be fix(ops/whitby): enable services.nginx.statusPage
This is necessary for the nginx prometheus exporter to work

Change-Id: I2343d6f5d3d6d6772777d5e14426a537aa1c8ef7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5127
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
2022-01-29 23:07:00 +00:00
Griffin Smith
84304b925a feat(whitby): Enable nginx prometheus exporter
Might be nice to look at rates of requests etc.

Change-Id: I4d12ab0c1a555793e803de4a9614e616951a94e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5125
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
2022-01-29 23:00:25 +00:00
Griffin Smith
63b09a6466 feat(ops/secrets): Add secret for telegram alerts bot token
This isn't actually used by anything that would use agenix, but this
seems like a vaguely sensible way of sharing the token with other people
regardless.

Anyone who finds this commit and wants to be added to the telegram
channel where the alerts go, lmk.

Change-Id: I06d6ed2d4bec099cbf68ede8fd00a5e6f4e7bc60
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5124
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2022-01-29 22:59:58 +00:00
Vincent Ambo
dddeb04679 fix(www/tvl.fyi): Anchor /blog redirects at #blog
Since our blog index is on the index page, this makes slightly more
sense.

Change-Id: I7b8164490c133e23d892abef21275f8bfed50b66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5123
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2022-01-29 22:37:55 +00:00
Griffin Smith
be91bc97a9 fix(tvl.fyi): Redirect /blog/ (with trailing /) to /
This was already happening without the trailing slash, but needs to
happen separately with it.

Fixes: b/172
Change-Id: Ic3423fd7a2eaf76a073badd80965cee953df4ce9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5121
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
2022-01-29 22:27:54 +00:00
Vincent Ambo
7b7dfa3bf2 feat(ops/www): Write JSON access log to journald
This means it will end up in journaldriver.

Change-Id: I66f781085b5dac9946b3b9a2bf30e447863e1213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5122
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2022-01-29 22:24:53 +00:00
Vincent Ambo
88595c23ce feat(ops/glesys): Move tvl.su email over to Yandex 360
This fixes b/158

Change-Id: Ie58737dfe144a342b6d4d30f72ec60d70405af31
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5110
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-01-29 14:51:47 +00:00
Vincent Ambo
19d7ccf87f feat(ops/glesys): Set up Yandex 360 verification record on tvl.su
Relates to b/158

Change-Id: I6738505bcf0088d35bb7c8f9215dcf0b4e27f9ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5109
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-01-29 14:51:47 +00:00
Vincent Ambo
ec2b98d131 refactor(ops/pipelines): Move into postBuildSteps
There is no need for this step to be part of the static pipeline (it
should not run if the build fails anyways).

Change-Id: I71400a452d6f8f4708d146b346eaffda5da2f766
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5049
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-01-22 12:01:09 +00:00
Vincent Ambo
cb107c4fbe feat(ops/pipelines): Upload post-build steps in static pipeline
Change-Id: I5ce6d51837c734951fe10c4f21806cf0fc57ed23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5048
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-01-22 11:59:09 +00:00
Vincent Ambo
5a88e47b71 refactor(ops/pipelines): Split build/post steps into separate chunks
This will create `build-chunk-$n.json` files for steps that should run
_before_ duck, and `post-chunk-$n.json` files for steps that should
run after duck.

The post steps are not yet uploaded to Buildkite, but we also don't
have any right now.

Change-Id: I7e1b59cf55a8bf1d97266f6e988aa496959077bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5047
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
2022-01-22 11:59:08 +00:00
Vincent Ambo
f12ceaa622 refactor(ops/pipelines): Use branches filter for canon-only steps
Using this instead of a conditional leads to nicer output in the UI,
but has no semantic difference.

Change-Id: I5b368d663f417d256e4792d2d46b84fc50d42d0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5045
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2022-01-22 11:59:08 +00:00
Vincent Ambo
a4aabaff68 refactor(ops/pipelines): Move :git: step up in the pipeline
This step is independent of the build result and can be scheduled at
the beginning while pipeline eval is still in progress.

Change-Id: I2ee268e4c333efa654dcb12c0b1562b43231d241
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5044
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-01-22 11:29:04 +00:00
Vincent Ambo
4b9bd8afd7 feat(ops/pipelines): Always upload entire pipeline output
Previously we only stored the drvmap, but we will also need the build
chunks to refactor the generation of dynamic post-steps.

Change-Id: I256fffe13af8f8c4521835257f5d87dda323b248
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5043
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-01-22 11:29:04 +00:00
Vincent Ambo
3e9aa7722d refactor(depotfmt): Move formatting check into an extra step
Change-Id: I7e4cf6bb2351bd11a5396f1663c0d4cc97c0d94e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5009
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-01-21 11:49:03 +00:00
Vincent Ambo
ac7e989571 feat(ops/pipelines): Trigger pipeline for tvl-kit through canon
This CI pipeline in Buildkite verifies the external (josh-provided)
view of the depot at //views/kit.

See https://buildkite.com/tvl/tvl-kit

Note that this always triggers a build of HEAD. This is because we
don't know the transformed commit ID, and we currently have no way to
pass a ref through. The pipeline is configured to skip intermediate
builds.

I asked Buildkite for some ideas on how to improve this, lets see.

Change-Id: I6c60fb1ea7606c1c90219ef04fd7bada64661529
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5010
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-20 17:35:18 +00:00
Vincent Ambo
0652ac0ace refactor(nix/buildkite): Move fetch-parent-targets script here
This is no longer TVL-specific and should live here with the other
generalised stuff.

Change-Id: I95a1b4c0321f34812162d6fd40568269abf639dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5006
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-01-19 17:22:36 +00:00
Vincent Ambo
c2e6c0719c refactor(ops/pipelines): Generalise fetch-parent-targets script
Removes all TVL-specific values in favour of environment variables
supplied by Buildkite.

This makes it possible to reuse this script outside of TVL.

Change-Id: Ic543bc41e4c81e65ee349ad241c515231e97ab30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5005
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-01-19 17:22:36 +00:00
Åsmund Østvold
6e4b0f3cef feat(ops/besadii): make text 'cl' posted BuildKite configurable
Some companies do not know the 'cl' term. They do know of 'change' and
would maybe not like to introduce one more synonym.

This cl introduce an optional entry 'gerritChangeName' in
besadii.json. The string has to match `^[a-z0-9]+$` for readability.

Change-Id: Id70fcb1e45158869f88bf37669be49b8b8a3b295
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4825
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
2022-01-19 10:31:52 +00:00
Vincent Ambo
1f3aa71cf2 fix(ops/oauth2_proxy): Fix cookie secret length
The cookie secret in the encrypted file was too long, because the
generation command in the oauth2_proxy docs is also wrong. Should
probably fix that upstream as well.

Also noticed that an extra '2' snuck into the service name and fixed
that.

Change-Id: I9a344a75993ab1f98299a8d45e7f5b2e146b7fc5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4957
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-17 13:51:47 +00:00
Vincent Ambo
9596c642d5 feat(ops/pipelines): Fetch parent target map for pipeline generation
Change-Id: I1c7d48fc0974549d67146a15f79ddb0b6ddfe805
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4947
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-17 11:49:01 +00:00
Vincent Ambo
0779f96687 feat(nix/buildkite): Check target map of parent to determine skips
This changes the logic for build pipeline generation to inspect
an (optional) parentTargetMap attribute which contains the derivation
map of a target commit.

Targets that existed in a parent commit with the same drv hash will be
skipped, as they are not considered to have changed.

This does not yet wire up any logic for retrieving the target map from
storage, meaning that at this commit all targets are always built.

The intention is that we will have logic to fetch the target
map (initially from Buildkite artefact storage), which we then pass to
the depot via externalArgs when actually generating the pipeline.

Change-Id: I3373c60aaf4b56b94c6ab64e2e5eef68dea9287c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4946
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-17 11:49:01 +00:00
Vincent Ambo
0a21da2bb4 feat(ops/pipelines): Create drvmap structure for each commit
Always create a structure that maps all targets to derivations, and
persist it as a JSON file.

This relates to some of the ideas expressed in:

https://docs.google.com/document/d/16A0a5oUxH1VoiSM8hyFyLW0WiUYpNo2e2D6FTW4BlH8/edit

The file is always uploaded to Buildkite as an artifact. This allows
for retrieving it based on the commit ID in a Buildkite GraphQL query.

By default, Buildkite stores artefacts for 6 months. Storage location
can be overridden (with custom retention) through some environment
variables, but for now at TVL the Buildkite-managed storage is fine.
See also: https://buildkite.com/docs/pipelines/artifacts

In the subsequent filtering implementation, when diffing commits
across a time-range that exceeds artefact retention time, we should
simply default to building everything.

Change-Id: I6d808461cd1c1fdd6983ba8c8ef075736d42caa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3662
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-17 10:26:08 +00:00
Vincent Ambo
005cd4037a fix(gerrit-tvl): Handle "broken" (skipped) jobs correctly
by simply skipping them

Change-Id: I9cbec3b79469ae01b1873d6a42e990b98cc4110a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4921
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2022-01-14 20:44:56 +00:00
Vincent Ambo
a5c9b11a6b fix(gerrit-tvl): Exclude non-command jobs from check results
Change-Id: I13727d30ac7a568f02614a4bbc778afed6a286ba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4891
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-01-14 20:30:53 +00:00
Vincent Ambo
86a205220f fix(gerrit-tvl): Explicitly specify patchset on check runs
Since we now group patchsets inside of Buildkite, the results are no
longer guaranteed to be for the right patchset.

There might be some metadata passed in from Gerrit that would let us
do this with the commit ID instead, but I haven't checked.

Change-Id: I5b74a17697511160fcc89d3dbef23517d974dc6f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4890
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-01-14 20:30:53 +00:00
Vincent Ambo
e8bf17317e fix(gerrit-tvl): Mark job as failed on all failure states
Change-Id: If0fa85d8178b9e457305d0244ddf67d12a4b3051
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4889
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
2022-01-14 20:30:52 +00:00
Vincent Ambo
e1ffaee1dd fix(gerrit-tvl): Support all documented Buildkite job statuses
I'm not sure where the previous list originated, but it was missing
some officially documented statuses. However, the API definitely
returns statuses that are documented to only appear in other types, so
this commit simply maps ALL statuses that Buildkite has documented for
any type.

Also adds a log statement in case we encounter a brand new, unknown,
undocumented status.

Change-Id: Iff003a3bd2608702019ae0f4137958435ad0856f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4888
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2022-01-14 20:30:52 +00:00
Vincent Ambo
058bf61193 fix(gerrit-tvl): Fix construction of ref used in Buildkite
... and remove a spammy log statement.

This changed in besadii a while ago and lead to the behaviour of
failing silently, instead of failing with an error saying "undefined
undefined".

Note that with this change merged the plugin probably still won't work
again, but it gets us a step closer to the real error.

Change-Id: I3db25d246f4b1c634d316cd92574e27fb220d769
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4887
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2022-01-14 20:30:52 +00:00
Vincent Ambo
ee52fbc46c feat(besadii): Skip builds of patchsets with no code changes
Currently Gerrit is configured to copy forward the scores of the
'Verified' label if the tree of the commit does not change (e.g. only
author information or commit message is modified).

Besadii still triggers builds for these patchsets though. With this
change it will inspect the (previously ignored) "kind" of the patchset
and skip patchsets with the same tree as their predecessor.

See Gerrit docs for the semantics of "kind":

https://gerrit-review.googlesource.com/Documentation/json.html#patchSet

Note that an argument can be made that we should do the exact opposite
- stop carrying over 'Verified' at all and always build all patchsets.

I think this depends on whether we intend to use commit metadata in CI
runs at all. Adding a few people to the review for opinions.

Change-Id: I48a96a1ad1e07d92330d84e5cfdc820a39395297
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4867
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: asmundo <asmundo@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-14 17:35:45 +00:00
Vincent Ambo
bce5192566 feat(whitby): Install a handful of systemPackages
Adds more things I keep using via nix-shell, as well as the
deploy-whitby script (which is independent of a particular depot
checkout).

Change-Id: I36f87de7645768a05268c90ba9b3ab833bacca05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4881
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-13 08:39:03 +00:00
Vincent Ambo
9c025d62a0 refactor(deploy-whitby): use nvd instead of nix-diff
nvd only shows us changed versions of packages, as well as
added/removed packages, which means that for the majority of depot
packages nothing will be displayed

however, the current output of nix-diff is not usable anyways, so
having something that can be looked at is better than nothing

Change-Id: Iefbd8139c7ccf5c88ed1209897abdb2ae9302e91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4868
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-13 08:14:00 +00:00
sterni
d99132f45a fix: resolve remaining security.acme.email warnings
These were missed in cl/4784.

Change-Id: I01a5827900c1b3bdfdf9b1c36dcca8d6b59073a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4866
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: sterni <sternenseemann@systemli.org>
2022-01-12 18:30:34 +00:00
Åsmund Østvold
d4b997d832 fix(ops/besadii) no need to ref CL number in post to Gerrit
The comment posted to the Gerrit change do not need to contain the CL
number as it is given by the context of the Gerrit UI.

Change-Id: I172645e7f4d82e2fbebe179578babd42ea29737f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4826
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
2022-01-12 09:37:01 +00:00
William Carroll
1e730c859d fix(wpcarro/all-systems): Remove diogenes from my top-level systems
When `findSystem` attempts to evaluate `system.config.networking.hostName`,
diogenes (because I've refactored its definition) causes the following error:

> You're trying to declare a value of type `string'
> rather than an attribute-set for the option
> `system'!

Change-Id: Ib23cb9aa9cadc1f71ad3369c903e587762d12cc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4830
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-01-08 10:04:00 +00:00
Vincent Ambo
7cac51a995 feat(ops/auto-deploy): Support emergency stops via stop file
Adds a feature to emergency-stop deploys by simply running `touch
/var/lib/auto-deploy/stop`.

This can be useful in some situations, especially if there is a
process that reconciles service state (so that e.g. stopping the
unit's timer would be undone).

Change-Id: I233dfac365a578bfa4110eb605b50be079974ba4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4827
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-01-07 17:17:33 +00:00
Vincent Ambo
b8e011f792 chore(cache.tvl.su): Raise cache priority to 50
The priority of binary caches is decided by the remotes in Nix (???),
and by default nix-serve (which is *very* slow) has a lower priority
than cache.nixos.org (which means that it will be preferred over the
faster cache for paths that exist on both).

To avoid this, override the hardcoded (????) priority by serving the
nix-cache-info response directly from nginx instead.

Change-Id: I15a2d6618386d16edaf69f1c9257a36bd72132d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4823
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-07 01:30:00 +00:00
tazjin
1bd6c2a85b revert: "fix(ops/pipelines): Remove duplicated wait step"
This reverts commit 5e036ed9fc.

Reason for revert: This introduced a logic error since the remaining
step runs at the wrong point in the pipeline. Temporarily reverting to
having duplicated waits in order to clean up later.

Change-Id: Ifa6ece50dd22924f02efd7b790a5863ca1189af7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4841
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2022-01-07 01:06:56 +00:00
Vincent Ambo
73288ba569 feat(ops): Add initial oauth2_proxy configuration
The intent is to configure oauth2_proxy pointing at Keycloak to enable
usage with nginx auth_request directives.

I want to expose this as a function from within the module in which
nginx server configuration blocks can be wrapped, but the function for
that is currently a placeholder.

Change-Id: I5ed7deb9bf1c62818f516e68c33e8c5b632fccfe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4767
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-04 18:04:27 +00:00
Vincent Ambo
3806cea40b chore(ops): Remove login.tvl.fyi module
It looks like we won't need this for oauth2_proxy when combined with
nginx auth_request setups.

Change-Id: I2294aee6226b4f64a27bf6592c2d18092d0268cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4766
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-04 18:04:20 +00:00
Vincent Ambo
4ab061ed98 fix(ops/pipelines): Realise anchor derivation for rooting
Turns the anchor derivation into something that can actually be
built (a call creating a propagated build inputs file), and builds it.

This should fix the anchoring logic we have on canon.

Change-Id: If6a7662b82e2e396388980f65e332cf67a45b46e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4763
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-02 22:25:42 +00:00
Vincent Ambo
5a6f984222 refactor(ops/keycloak): Split out clients & user-sources
Without some kind of physical organisation it's a little difficult to
understand whether things are going "in" (supplying users to Keycloak)
or "out" (getting auth/user info from Keycloak).

Change-Id: I516501081e3448c81c710fcbc79cc68ad2a80f3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4762
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2022-01-02 21:22:17 +00:00
Vincent Ambo
5e036ed9fc fix(ops/pipelines): Remove duplicated wait step
This now happens in //nix/buildkite instead

Change-Id: Ie9e239ee4f28ac34aa4d3279dac55d70a2cb9d86
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4764
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-02 19:09:19 +00:00
Vincent Ambo
2bf39d7101 refactor(modules/smtprelay): Load credentials via agenix
Change-Id: I56f6887e1fd35551cfc83ad08cafebb611f4a341
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4760
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: tazjin <mail@tazj.in>
2022-01-01 15:30:17 +00:00
Vincent Ambo
58c64aa81a feat(ops/secrets): Add smtprelay credentials
Change-Id: I489e611a3fb19b4a374a563aa1afd81a130b2e7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4759
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <mail@tazj.in>
2022-01-01 15:30:17 +00:00
Vincent Ambo
b763f183f7 fix(ops/keycloak): redefine buildkite client, correctly this time
This client definition was previously nonsense. What happened is that
I accidentally imported the client as an OIDC client, which Keycloak
accepted because apparently those are the same entities on the API
level, and that ended up getting mangled into some broken hybrid shape
by Terraform.

This sets up the Buildkite provider again but with the correct
SAML configuration this time.

Change-Id: Id7ba318984d2fcc9e2ca91ed45ccbfd227278bbe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4731
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <mail@tazj.in>
2021-12-28 17:37:22 +00:00
Vincent Ambo
7ecb2a1144 refactor(tools/depotfmt): Move depotfmt check into a real build step
Produces more useful output and also makes for a good target for the
upcoming extraSteps logic.

Change-Id: Ifd389d433d9e27f97940a48999f4fba35646e37a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4727
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-12-28 15:37:10 +00:00
Vincent Ambo
b7ef2a579b refactor: Generalise pipeline generation in //nix/buildkite
Extracts the logic for generating our Buildkite pipeline (which has
been copy&pasted and slightly modified in some places outside of
depot) into a generic //nix/buildkite library.

This should cause no change in functionality.

Change-Id: Iad3201713945de41279b39e4f1b847f697c179f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4726
Autosubmit: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-12-28 15:34:39 +00:00
Vincent Ambo
88d7075b30 fix(ops/users): change my email to the @tvl.su one
Change-Id: Id608fe66b203c1d08958c85be44506a86eec56d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4730
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Autosubmit: tazjin <mail@tazj.in>
2021-12-28 13:00:54 +00:00
zseri
52369a11e3 refactor(ops/secrets): optimize + typecheck mkSecrets
Change-Id: I592c8f2f82cef8fe4509e90a8c48504a0c74d133
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4688
Reviewed-by: zseri <zseri.devel@ytrizja.de>
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: zseri <zseri.devel@ytrizja.de>
Tested-by: BuildkiteCI
2021-12-27 23:16:31 +00:00
Vincent Ambo
d8cdd629f4 feat(ops/glesys): Import DNS records for tvl.su
These records were previously configured manually in the GleSYS web UI
during our DNS outage (b/155).

Note that I could not find a way to `terraform import` these records
and have instead recreated the set and then cleaned up in the UI.

Change-Id: If7de9a7e6dad20953ba8b610589a62dce400e87b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4716
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 16:45:54 +00:00
Vincent Ambo
228138395b feat(ops/glesys): Import DNS records for tvl.fyi
These records were previously configured manually in the GleSYS web UI
during our DNS outage (b/155).

Note that I could not find a way to `terraform import` these records
and have instead recreated the set and then cleaned up in the UI.

Since we often point things at whitby, I have extracted variables for
its IPs in this change.

Change-Id: I09fda94d3734e8aaa278fa858e160d046740da1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4714
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 16:42:53 +00:00
Vincent Ambo
b2c151cebb feat(ops/glesys): Import DNS records for nixery.dev
These records were previously configured manually in the GleSYS web UI
during our DNS outage (b/155).

Note that I could not find a way to `terraform import` these records
and have instead recreated the set and then cleaned up in the UI.

Change-Id: I2b7e0ed0931f50e7fa49c1f6e3400dfe958def04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4713
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 16:42:52 +00:00
Vincent Ambo
df8edcb5f7 feat(ops/secrets): Import secrets for tf-glesys
Adds the secrets and some instructions for deploying the GleSYS
Terraform infrastructure.

Change-Id: I1a10f9cee7648d406b3d27ef45fc74b6923cbc30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4712
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 15:53:57 +00:00
Vincent Ambo
23693ca898 feat(ops/keycloak): Import Buildkite OIDC client
This was previously configured in the UI.

Change-Id: I68361b1489093b76736adab2e38ed7b474b10881
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4711
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 15:53:57 +00:00
Vincent Ambo
fb7d45abc4 feat(ops/keycloak): Import Gerrit OIDC client
This was previously configured in the UI.

Change-Id: Ib15b8ecca96d7814dc85d62199865b22bdb63f95
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4710
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 15:53:57 +00:00
Vincent Ambo
98be390576 fix(ops/keycloak): Move Terraform state to GleSYS bucket
This should never sit around locally the way it does now.

Change-Id: Icfbdaf1949d6d948a796a0759282ea6144af3621
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4709
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 15:53:57 +00:00
Vincent Ambo
e616f978d0 feat(ops/secrets): Add tf-keycloak secrets file
This file can be sourced (somehow, depending on the user) while
working with //ops/keycloak to get the relevant secrets.

Change-Id: Ibb3051c4b019f64824964475451c1c3996db6421
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4708
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 15:53:57 +00:00
Vincent Ambo
4f030f085d feat(ops/keycloak): Add OIDC client for Grafana
Completely forgot about Grafana, so it's currently broken. Oops!

Change-Id: Ia4e6405428ad8e514d6e61635f9692c57f61defe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4705
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <mail@tazj.in>
2021-12-27 15:53:57 +00:00
Vincent Ambo
906d6553c6 fix(whitby): Point grafana at new auth provider
Grafana was still pointing at the (now non-existent) CAS setup. This
changes the endpoints to use Keycloak instead and updates the client
secret.

Change-Id: Ib25d38330aba2ef6d894e8c33d86852c884ab5be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4706
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 14:44:38 +00:00
Griffin Smith
ef62e51b7b refactor(ops/secrets): generalize out a mkSecrets function
Generalize out a reusable mkSecrets function from the
secrets-tree-building that's happening in //ops/secrets, so the same
thing can happen in other places in the depot (I want to use it for my
personal infrastructure).

Change-Id: I059295c8c257d78ad7fa0802859f57c2c105f29b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4679
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: zseri <zseri.devel@ytrizja.de>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2021-12-27 03:46:26 +00:00
Griffin Smith
9b3374b091 feat(ops/machines/all-systems): Add grfn/mugwump
Change-Id: I7770b58c44a5700e86c80d1058e89e9fa65d719b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4686
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
2021-12-26 21:33:36 +00:00
Griffin Smith
7ea212ee07 fix(auto-deploy): Add missing packages to path
Building nix derivations needs tar (provided by gnutar) and gzip on the
PATH in order to extract .tar.gz archives.

Change-Id: Ia2df7a3a770cfd342dfede58ad34e04805fbd1f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4685
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
2021-12-26 21:33:36 +00:00
Vincent Ambo
fc16f1e467 fix(ops/keycloak): set up client for usage with oauth2_proxy
This will be useful for things like panettone, pending a NixOS module
for oauth2-proxy (the upstream one is too complicated and doesn't
support what we need).

Change-Id: I4ca193e10a94a29b1fb9003e945896ff8eb61116
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4662
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: tazjin <mail@tazj.in>
2021-12-26 16:59:01 +00:00
Vincent Ambo
a8923242be fix(ops/keycloak): trust email addresses from LDAP
Verified emails are required for some things, like e.g. oauth2_proxy

Change-Id: Ifb124be40d6d2863cd1b7ed5fbdfcf4827e8808c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4661
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-12-26 16:59:01 +00:00
Vincent Ambo
e8fa347fd1 feat(ops/keycloak): Set up oauth2_proxy client
Change-Id: I996d9644ed7e870d6e5a42af117eafbf841da679
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4640
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-12-26 16:59:01 +00:00
Vincent Ambo
7b3c0b3e2f feat(ops/keycloak): Check in initial Keycloak configuration
This is still missing most of the client configuration etc., in part
due to bugs in the provider which are preventing resource imports.

Change-Id: Ic224ffc001f8e1fe6dcd47b7d002580fdf7b0774
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4628
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-12-26 16:45:59 +00:00
William Carroll
80ef71e995 feat(ops/auto-deploy): Support auto-deploy
Automatically rebuild the current system's NixOS config from the latest checkout
of depot.

Change-Id: I23aa7af50e16e985ac34df214e0905e770316e5e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4390
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: zseri <zseri.devel@ytrizja.de>
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2021-12-26 14:55:42 +00:00
Vincent Ambo
28a80e0251 chore: friendship ended with cas, now keycloak is our best friend
Note that the login.tvl.fyi WWW configuration is still kind of hanging
around until we've settled where Keycloak lives.

Change-Id: Iaca4e394a7371cafa3716ca66ef09c4eca5b1520
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4626
Autosubmit: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-26 00:10:10 +00:00