Commit graph

529 commits

Author SHA1 Message Date
Vincent Ambo
f9bd68e247 fix(ops/secrets): Fix missing file
... okay, this is like the 5th error related to something with this
and file paths. Need to write some validation logic.

Change-Id: I4314818aa1bc25b8cf7bd3593850d3836ccb867c
2021-12-10 23:53:50 +03:00
Vincent Ambo
aa5bf312e8 fix(tvl-buildkite): Use supported credential helper binary name
Git only allows binary names prefixed with `git-credential-` if the
path to the helper is not absolute.

Why? Who knows.

Change-Id: I216b2a621f62a73f05e21def7ec8016b29ede892
2021-12-10 23:37:57 +03:00
Vincent Ambo
2f1c654c14 refactor(ops): Move panettone secrets to agenix
Relates to b/161

Change-Id: I508e5a0eacab668f4bd39a2c888d894b96bed093
2021-12-10 23:19:56 +03:00
Vincent Ambo
2b9be81ea0 refactor(ops/pipelines): Use agenix-deployed besadii secrets
I *think* this is the final step for b/161

Change-Id: Ie7a2198a045f2f1866a245884ab0f5414e205327
2021-12-10 23:14:41 +03:00
Vincent Ambo
60f96d2b17 fix(whitby): Fix typo in buildkite-agents group name
... really would like some assertion helpers for this sort of stuff.

Change-Id: I32d1de18ebfbbdfa5128a8fbdad2efcc511f8514
2021-12-10 23:01:20 +03:00
Vincent Ambo
5baa9b6d87 refactor(tvl-buildkite): Prepare gerrit credentials helper
Currently this functionality is provided by a shell script stored in
/etc/secrets (which has the password value hardcoded).

This needs to happen in a separate commit from the one that changes
the pipeline to avoid breaking it (it needs to be deployed first).

Change-Id: I680754c828ccefbacfcf0d5c813a4bc19493ba4c
2021-12-10 19:52:39 +00:00
Vincent Ambo
2fe8d724d7 refactor(ops): Move Nix cache secret to agenix
... and also the public key, just to keep the distribution mechanism
the same.

Change-Id: Ief14daf9344c0fb99eeb5789c1ec9bfb1f12bee0
2021-12-10 19:48:26 +00:00
Vincent Ambo
82a885a750 refactor(ops): Use besadii configuration from agenix
We already checked this in, but this commit adds the configuration for
making use of it.

There are two copies of besadii's JSON configuration with different
permissions.

Note that the buildkite-graphql-token path needs to be updated in
static-pipeline.yml, but this needs to happen in a separate commit
after deploy because the pipeline will break otherwise.

Change-Id: I6fab4bf1a2e679df7cf76521e2b53bd9dadbac62
2021-12-10 19:31:36 +00:00
Vincent Ambo
b1108821a9 refactor(ops): Move grafana secret into agenix
Change-Id: Id141758135c796881e91d20b950dae74c40d9ab3
2021-12-10 19:31:36 +00:00
Vincent Ambo
b2d46aed2b fix(tvl-buildkite): Add more missing programs to agent path
... this option really is a pitfall! The list of programs is now the
same as in the upstream module, plus curl and jq.

Change-Id: I29edae4b2400a2724f62df9efa1dc184a8b0af5f
2021-12-10 17:13:22 +00:00
Vincent Ambo
b8267c261c fix(ops/irccat): Avoid permissions issue with LoadCredentials=
The DynamicUser + Group configuration does not work as planned, thus
the systemd LoadCredentials feature is used instead which makes the
file (which itself is only readable by root) available in a
memory-backed location only readable by the service.

The secret is only available to `ExecStart` commands, so units using
this feature can not be used with pre/post units and the like if those
commands need secrets.

To accommodate this, the merge of configuration files has been moved
into the service launch script, which is now the ExecStart= process.

For details take a look at https://www.freedesktop.org/software/systemd/man/systemd.exec.html#LoadCredential=ID:PATH

Change-Id: I693fe5677cc0d63c7aa485c2c7472457c5262166
2021-12-10 15:09:09 +00:00
Vincent Ambo
67bde5ecc3 fix(tvl-buildkite): Explicitly set runtimePackages
It turns out the lib.mkAfter call doesn't behave as expected -
only *some* of the packages that are defaulted end up in the $PATH.

I suspect this is actually something else, e.g. these packages are
always added for some reason or another, and the option is completely
overridden every time.

Change-Id: I854c7198520d82b00e6338ed0fe653836226dc6d
2021-12-10 15:06:08 +00:00
Vincent Ambo
2ba481451c chore(ops/secrets): Reencrypt with grfn's key included
Change-Id: I66df150ab5070a81a92f0741334639df9df1f86f
2021-12-10 17:52:08 +03:00
Griffin Smith
a85ab68b12 chore(ops/users): Rotate password for grfn
Just a regular password rotation, plus I wasn't using argon2 unlike
everyone else.

Change-Id: Ic57fe79a2dbfdc15397d20f6b2b47c6aac911d29
2021-12-10 09:45:17 -05:00
Griffin Smith
66a1d3d5d4 feat(ops/secrets): Add key for grfn
Change-Id: I8063ae804932e3815e9a499e0206806818b9b021
2021-12-10 09:44:34 -05:00
Vincent Ambo
bc3d35f3d0 fix(tvl-buildkite): Add missing runtimePackages back
Turns out that the type of this option is not concatenative and it
replaces the packages needed to run Buildkite if set.

Change-Id: I9f52572bc165bccdd8c6518cfdf7b8967f7a50d0
2021-12-10 13:14:11 +00:00
Vincent Ambo
d4403638cf refactor(ops): Move irccat secret into agenix
The irccat module uses DynamicUser, so to grant permission to it a new
group has been added for irccat.

I have some vague memory of DynamicUser + Group not behaving as one
would expect, but we'll see what happens.

Change-Id: Iab9f6a3f1a53c4133b635458ce173250cc9a3fac
2021-12-10 16:13:31 +03:00
Vincent Ambo
002d183876 refactor(ops): Move clbot SSH key into agenix
Change-Id: Iae03ead7dda0509689a76f0d76f9cfeb8434e967
2021-12-10 16:13:31 +03:00
Vincent Ambo
811e6d7d9f chore(whitby): Remove shadowsocks service
No longer required on whitby.

Change-Id: I93951c6b708eae81ddb03df920a4068c1ccde9e7
2021-12-10 13:07:09 +00:00
Vincent Ambo
fc14c21bb9 fix(ops/pipelines): Move to static pipeline
This step would get inserted at the wrong point in the build pipeline
otherwise, causing a dependency cycle and causing the pipeline to fail.

Change-Id: I534568eec77f74ae6c47276820f8a9e99493a3ea
2021-12-10 11:01:21 +03:00
Vincent Ambo
e4231c9816 refactor(ops/pipelines): Move 🦆 logic into static pipeline
This simplifies the fallback logic used in case of Nix evaluation
failure and makes it so that the evaluation step itself is the one
that is marked as failed in Buildkite.

This is possible because the pipeline upload command will insert new
steps at the point where it runs in the pipeline, and not later.

Change-Id: I870534c004ebc457a1602623c4e5f9c0c68e28fc
2021-12-10 07:55:34 +00:00
Vincent Ambo
9ea4d55d81 refactor(ops): Move buildkite-agent-token into agenix
Relates to b/161

Change-Id: I5d3a698d437928966d8b78ce9e0ba226c1437655
2021-12-10 10:32:44 +03:00
Vincent Ambo
a123b9e0a2 refactor(ops): Move owothia secret into agenix
Relates to b/161

Change-Id: I25445281b0dd3c3f3660f8bb0d8337506a1e427b
2021-12-10 10:32:14 +03:00
Vincent Ambo
78744c00f5 refactor(ops): Move clbot secret into agenix
Relates to b/161

Change-Id: I7badf22ff93bb4e8b06e4dd4a8bf880b0bd48f09
2021-12-10 10:32:14 +03:00
Vincent Ambo
496d899428 feat(ops/secrets): Configure secrets for gerrit-queue
Adds a systemd EnvironmentFile secret that contains the Gerrit
username & password for gerrit-queue.

Change-Id: I25acf87764c26774045138402b8a417b6813ee8f
2021-12-10 10:32:14 +03:00
Vincent Ambo
4870b1a2ff feat(ops/modules): Add module for running gerrit-queue
This is not yet including the secret configuration for gerrit-queue,
and just expects the secret (gerrit username & password) to be
available in /etc/secrets.

Change-Id: Ia465ef7f3f521c70d606d7fdeba9aa83c7e1b98b
2021-12-10 10:32:14 +03:00
Vincent Ambo
a9dd719e7c chore(tvl-buildkite): Add jq and curl to agent paths
This is required for a simplification of the build pipeline (following
CL) and needs to be in a separate commit as it can not be done
atomically (merging the other commit to deploy it would immediately
break pipelines otherwise).

Change-Id: I5d8ec8f3238f79b5518d799486bf98d1d9516c43
2021-12-10 10:21:34 +03:00
Vincent Ambo
f1e1f71883 feat(ops/secrets): Bootstrap agenix secrets folder
Sets up the key set and adds an initial secret (besadii config with
tokens) to be deployed to whitby.

Change-Id: Ic07fd5e66b9e7a533013e04c35e052c2aa11f77d
2021-12-08 18:22:00 +00:00
Vincent Ambo
c1479a6221 chore(besadii): Improve error messages on parse failure
Change-Id: I3cc4637aca8a940a0fdeca2d8bd6ac620ea384c0
2021-12-07 18:27:44 +00:00
Vincent Ambo
8a944484f0 fix(ops/besadii): Unquote Gerrit's extra-quotes around emails
Gerrit wraps RFC5322 emails in another layer of quotes when passing
them as flags, and this needs to be unquoted.

Otherwise hook invocations fail with cryptic errors.

Change-Id: Ieeb74c662873d99a4154f8cbc92da77b039cb88e
2021-12-07 18:27:44 +00:00
Vincent Ambo
6faf0edaff fix(ops): Correctly pass command name to besadii invocations
Ensure that besadii sees $0 as the correct command name, since that is
the sole mechanism by which its functionality is switched around.

There was a lingering commit that introduced this bug and hadn't been
deployed in a couple of days. Maybe time to tighten deploy cycles soon
...

Change-Id: Ie4284c0f6e5e06d71a71a3702ec7e092260e0ce5
2021-12-07 18:27:44 +00:00
Vincent Ambo
7f2f5d07f2 fix(ops/besadii): Pass Build.Author to Buildkite
Extracts author information from the flags passed by Gerrit and moves
them along to Buildkite. This should display the owners of builds
correctly in the UI, rather than marking everything as coming from me.

Change-Id: If9efe5553a13f0dbdb8bf3936c1d341ae5922318
2021-12-06 17:42:57 +03:00
Vincent Ambo
b679bb4034 refactor(ops/besadii): Get config from home directory by default
Slightly more ergonomic in some setups.

Change-Id: I565f2d242852ffd299ef5d5740a47520187dd4b4
2021-12-02 17:13:49 +03:00
Vincent Ambo
dcb2410982 refactor(ops/besadii): Generalise for use with non-TVL URLs
This makes it possible to use besadii for any TVL-ish setup using
Gerrit and Buildkite, with the same hook functionality as for TVL.

Change-Id: I1144b68d7ec01c4c8e34f7bee4da590f2ff8c53c
2021-12-02 13:10:21 +03:00
Vincent Ambo
e48ae26e8e feat(ops/besadii): Add other missing configuration keys
Adds configuration keys and rudimentary validation for all other
besadii settings that are currently hardcoded.

This adds the config options:

* repository: Name of the repository in Gerrit.
* branch: Name of the HEAD branch in the repository.
* gerritUrl: Base URL of the Gerrit instance
* gerritUser: Username of the Gerrit user
* gerritPassword: Password of the Gerrit user
* buildkiteOrg: Name of the Buildkite organisation
* buildkiteProject: Name of the pipeline inside the Buildkite
  organisation
* buildkiteToken: Auth token for Buildkite access

All of these configuration options are required.

Change-Id: Ie6b109de9cd8484a3773c6351d7fd140f39a49ed
2021-12-02 13:10:21 +03:00
Vincent Ambo
ee635d4645 chore(ops/modules): Configure besadii call sites to load config
On whitby, the besadii config will live in
/etc/secrets/besadii.json. This CL updates the call sites to pass this
config path to besadii so that it can load Sourcegraph configuration.

Change-Id: Ia139b9fa3b827e7a5f2386214390acc6fe19a75a
2021-12-02 13:10:20 +03:00
Vincent Ambo
168114df52 refactor(ops/besadii): Move Sourcegraph config to a file
Initial step towards moving besadii away from hardcoded values and
onto config files. This is required because I want to reuse besadii
outside of the TVL context.

Change-Id: Id4fa7a49c5d4f876a02b202f04a421ab5ba0dcc4
2021-12-02 13:10:20 +03:00
Vincent Ambo
c53d6d3453 fix(ops/nixery): Temporarily stop serving depot packages in Nixery
Change the Nixery configuration to use the plain nixpkgs package path
instead of the depot path. AFAIK, nobody uses this to fetches depot
packages at the moment - but plenty of people fetch non-depot
packages.

This means that Nixery is cache-busted less often (previously on every
commit => every deploy).

We'll figure out another way to have a depot Nixery later.

Change-Id: Iba632333346181c3d2ce992fbab396ed0d9f86aa
2021-12-02 09:16:52 +03:00
Vincent Ambo
433f0ae5cd fix(ops/www): Redirect tvl.fyi/blog -> tvl.fyi
The blog index page is at the root and people may manually edit the
URL.

Change-Id: I6cdaaaee6223524a9e950584379cfac34f8be160
2021-12-01 23:41:23 +03:00
Vincent Ambo
c1aab56a02 feat(besadii): Support invocation as different Gerrit hooks
Removes besadii support for the previously used 'ref-updated' hook and
instead introduces support for the 'change-merged' and
'patchset-created' hooks.

These hooks more accurately capture the semantics of when besadii
should trigger CI builds and using them will avoid problems such as
skipping 'canon' builds if chains of CLs are submitted together.

Change-Id: Ib90356c069780bf0c0250e56b927e46a5b31ce7f
2021-12-01 12:49:31 +03:00
Vincent Ambo
68d1f834a3 fix(ops/www): Strip .html from TVL blog post URLs
Change-Id: I4d1f9284ec004931c07c04d614b01f28eedea508
2021-11-30 13:56:29 +03:00
Vincent Ambo
6edfdd0773 refactor(ops/pipelines): Query build status from Buildkite API
Instead of manually tracking the build status through Buildkite
metadata, use the Buildkite GraphQL API in the `🦆` build
step (i.e. the one that determines the status of the entire pipeline
to be reported back to Gerrit) to fetch the number of failed jobs.

This way we have less manual state accounting in the pipeline.

The downside is that the GraphQL query embedded here is a little hard
to read.

Notes:

  * This needs an access token for Buildkite. We already have one for
    besadii which is also run by the agents, so I've given it GraphQL
    permissions and reused it.

  * I almost introduced a very rare bug here: My initial intuition was
    to simply `exit $FAILED_JOBS` - in the extremely rare case where
    `$FAILED_JOBS % 256 = 0` this would mean we would ... fail to fail
    the build :)

Change-Id: I61976b11b591d722494d3010a362b544efe2cb25
2021-11-29 23:38:24 +03:00
Vincent Ambo
4d57898af4 chore(ops/users): Update password hash for asmundo
... some issue snuck in on the first one, as is tradition.

Change-Id: I06ce4df82cde26231cd1ab3df500de02e981d9bc
2021-11-29 18:12:19 +03:00
asmundo
78f51edf8c feat(ops/users): Add user asmundo
Change-Id: Ie666b6556d91513babd884b2ed1140cd6c0ed2a9
2021-11-29 14:14:04 +00:00
Vincent Ambo
eca2bc572e refactor(besadii): Rename refUpdated -> buildTrigger
We are changing the Gerrit hooks which invoke besadii, but this
structure will be used for both kinds.

Change-Id: Idb1cb0c640d2c42db8e7af39f3ab372a97bfef91
2021-11-29 13:54:47 +00:00
Vincent Ambo
104f002a07 fix(ops/besadii): Trim whitespace of auth tokens
This is causing failures when trying to update Sourcegraph at least,
for good measure I've trimmed both.

Change-Id: I40266ee83b4e266ffe50f16bb365eb2e51952513
2021-11-28 10:34:36 +00:00
Vincent Ambo
4f1249e46f refactor(readTree): Move 'drvTargets' into readTree
This function is also generally useful for readTree consumers that
have the concept of subtargets.

Change-Id: Ic7fc03380dec6953fb288763a28e50ab3624d233
2021-11-23 14:42:08 +00:00
Vincent Ambo
15cb37f877 fix(ops/restic): Move whitby's backup to GleSYS object storage
Since GCP nuked us, the backups are now moving to GleSYS'
S3-compatible object storage.

This refactors the restic module to support S3-compatible storage
instead of GCP, and switches to the appropriate new secret paths.

The secrets were placed on whitby manually and I verified that the
backups work.

This fixes b/157

Change-Id: I6a9d2b0581967605ce736605a3befb44cdeae7e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3883
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-11-21 12:01:26 +00:00
Vincent Ambo
099f36e5ee fix(ops/pipelines): Fix tagging of commit revisions
It seems that shell variables don't work as expected inside the
Buildkite pipeline, so usage of variables has been removed.

We also don't echo the revision anymore because of that, but it does
still appear in the log of `git push`.

Change-Id: I124e3b09af896da898f2a78715ed371651a1c5f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3780
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-11-06 00:33:23 +00:00
Vincent Ambo
4b33401a36 refactor(ops/pipelines): Move revision tagging into static pipeline
This makes the revision number available much earlier (before the rest
of the pipeline runs, while Nix eval is happening) which should only
be a few seconds after a commit to canon.

It is also more readable in this shape.

Change-Id: Iccbb17dfef6afe68f54fda41e8d10c4dc52b08c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3775
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-11-05 14:24:53 +00:00