Commit graph

15288 commits

Author SHA1 Message Date
Vincent Ambo
3a410a78df feat(ops/secrets): Make (encrypted) secrets part of the tree
Currently in NixOS configuration using agenix secrets there is no
build time validation of secret paths - things fail at runtime (system
activation).

To prevent that, this CL makes the secrets part of the tree based on
the same configuration file used by agenix itself.

This guards against:

* agenix secrets.nix definition for a non-existent file
* age.secrets value in a NixOS config for a non-existent secret

Change-Id: I5b191dcbd5b2522566ff7c38f8a988bbf7679364
2021-12-12 11:19:24 +03:00
Vincent Ambo
8cbb42006a chore(ops/secrets): Reencrypt all secrets with sterni included
Change-Id: I14043c2bd9da43a6b7de65baf0ebb75eaf3c4e22
2021-12-11 18:51:36 +03:00
sterni
40096c2931 feat(ops/secrets): add keys for sterni
Change-Id: Idf13f7737dd51e74e87093e07cdf22ad24407944
2021-12-11 15:41:55 +00:00
Vincent Ambo
40888c9630 fix(tvl.el): Fix use of label command in refs
The l= is part of the command, not of the shape of commands, and the
previous command concatenation logic was wrong because of that.

Fix is done in the most obvious way: Make the l= part of the command.

Change-Id: Ia3c08c3da60fe5fc38f29a2d94adcd123e4f3052
2021-12-10 21:08:48 +00:00
Vincent Ambo
f9bd68e247 fix(ops/secrets): Fix missing file
... okay, this is like the 5th error related to something with this
and file paths. Need to write some validation logic.

Change-Id: I4314818aa1bc25b8cf7bd3593850d3836ccb867c
2021-12-10 23:53:50 +03:00
Vincent Ambo
aa5bf312e8 fix(tvl-buildkite): Use supported credential helper binary name
Git only allows binary names prefixed with `git-credential-` if the
path to the helper is not absolute.

Why? Who knows.

Change-Id: I216b2a621f62a73f05e21def7ec8016b29ede892
2021-12-10 23:37:57 +03:00
Vincent Ambo
2f1c654c14 refactor(ops): Move panettone secrets to agenix
Relates to b/161

Change-Id: I508e5a0eacab668f4bd39a2c888d894b96bed093
2021-12-10 23:19:56 +03:00
Vincent Ambo
2b9be81ea0 refactor(ops/pipelines): Use agenix-deployed besadii secrets
I *think* this is the final step for b/161

Change-Id: Ie7a2198a045f2f1866a245884ab0f5414e205327
2021-12-10 23:14:41 +03:00
Vincent Ambo
60f96d2b17 fix(whitby): Fix typo in buildkite-agents group name
... really would like some assertion helpers for this sort of stuff.

Change-Id: I32d1de18ebfbbdfa5128a8fbdad2efcc511f8514
2021-12-10 23:01:20 +03:00
Vincent Ambo
5baa9b6d87 refactor(tvl-buildkite): Prepare gerrit credentials helper
Currently this functionality is provided by a shell script stored in
/etc/secrets (which has the password value hardcoded).

This needs to happen in a separate commit from the one that changes
the pipeline to avoid breaking it (it needs to be deployed first).

Change-Id: I680754c828ccefbacfcf0d5c813a4bc19493ba4c
2021-12-10 19:52:39 +00:00
Vincent Ambo
2fe8d724d7 refactor(ops): Move Nix cache secret to agenix
... and also the public key, just to keep the distribution mechanism
the same.

Change-Id: Ief14daf9344c0fb99eeb5789c1ec9bfb1f12bee0
2021-12-10 19:48:26 +00:00
Vincent Ambo
82a885a750 refactor(ops): Use besadii configuration from agenix
We already checked this in, but this commit adds the configuration for
making use of it.

There are two copies of besadii's JSON configuration with different
permissions.

Note that the buildkite-graphql-token path needs to be updated in
static-pipeline.yml, but this needs to happen in a separate commit
after deploy because the pipeline will break otherwise.

Change-Id: I6fab4bf1a2e679df7cf76521e2b53bd9dadbac62
2021-12-10 19:31:36 +00:00
Vincent Ambo
b1108821a9 refactor(ops): Move grafana secret into agenix
Change-Id: Id141758135c796881e91d20b950dae74c40d9ab3
2021-12-10 19:31:36 +00:00
Vincent Ambo
1b94d2c0ba feat(tvl.el): Add autosubmit feature to magit-gerrit-rubberstamp
This makes this function a true rubberstamp again, leading to
rubberstamped CLs automatically being merged after CI passes.

This is similar to the initial functionality we had last year, where
this directly submitted changes, but with the addition of the CI
checks.

Change-Id: I946b074b968eb18a64c4edb0043f7a4af28759b4
2021-12-10 22:00:39 +03:00
Vincent Ambo
7cb9b204e9 feat(tvl.el): Add shortcut for push with Autosubmit+1
This almost makes for a sort of fire&forget button, except we don't
have a way to automatically pick reviewers yet :)

Change-Id: I6f446270f8aaf0409ccb6321bdbb5c349079cd19
2021-12-10 18:54:32 +00:00
Vincent Ambo
b2d46aed2b fix(tvl-buildkite): Add more missing programs to agent path
... this option really is a pitfall! The list of programs is now the
same as in the upstream module, plus curl and jq.

Change-Id: I29edae4b2400a2724f62df9efa1dc184a8b0af5f
2021-12-10 17:13:22 +00:00
Bartosz Stebel
f43324e141 fix(3p/apereo-cas): Mitigate CVE-2021-44228
Same approach as in cl/4270.

Change-Id: I3a5a3533ab97513a4b9d8cacc26d013b58441f93
2021-12-10 17:52:49 +01:00
Vincent Ambo
62450bb1c5 feat(depot): Add grfn and sterni to top-level owners
Change-Id: Id2012e3ec6db21ff724245095a99d36ff9d7ad71
2021-12-10 18:11:16 +03:00
Vincent Ambo
b8267c261c fix(ops/irccat): Avoid permissions issue with LoadCredentials=
The DynamicUser + Group configuration does not work as planned, thus
the systemd LoadCredentials feature is used instead which makes the
file (which itself is only readable by root) available in a
memory-backed location only readable by the service.

The secret is only available to `ExecStart` commands, so units using
this feature can not be used with pre/post units and the like if those
commands need secrets.

To accommodate this, the merge of configuration files has been moved
into the service launch script, which is now the ExecStart= process.

For details take a look at https://www.freedesktop.org/software/systemd/man/systemd.exec.html#LoadCredential=ID:PATH

Change-Id: I693fe5677cc0d63c7aa485c2c7472457c5262166
2021-12-10 15:09:09 +00:00
Vincent Ambo
67bde5ecc3 fix(tvl-buildkite): Explicitly set runtimePackages
It turns out the lib.mkAfter call doesn't behave as expected -
only *some* of the packages that are defaulted end up in the $PATH.

I suspect this is actually something else, e.g. these packages are
always added for some reason or another, and the option is completely
overridden every time.

Change-Id: I854c7198520d82b00e6338ed0fe653836226dc6d
2021-12-10 15:06:08 +00:00
Vincent Ambo
2ba481451c chore(ops/secrets): Reencrypt with grfn's key included
Change-Id: I66df150ab5070a81a92f0741334639df9df1f86f
2021-12-10 17:52:08 +03:00
Griffin Smith
a85ab68b12 chore(ops/users): Rotate password for grfn
Just a regular password rotation, plus I wasn't using argon2 unlike
everyone else.

Change-Id: Ic57fe79a2dbfdc15397d20f6b2b47c6aac911d29
2021-12-10 09:45:17 -05:00
Griffin Smith
66a1d3d5d4 feat(ops/secrets): Add key for grfn
Change-Id: I8063ae804932e3815e9a499e0206806818b9b021
2021-12-10 09:44:34 -05:00
Vincent Ambo
2fc64dc277 fix(clbot): Use change *owner* and not *uploader*
In autosubmit cases that require rebases, the change *uploader* might
be clbot which would cause besadii to use clbot as the owner.

This is incorrect, but luckily the change-merged event has an actual
owner field instead.

Change-Id: Ia35b52085f94628e61eb358807b3b85565521b60
2021-12-10 13:50:14 +00:00
Vincent Ambo
bc3d35f3d0 fix(tvl-buildkite): Add missing runtimePackages back
Turns out that the type of this option is not concatenative and it
replaces the packages needed to run Buildkite if set.

Change-Id: I9f52572bc165bccdd8c6518cfdf7b8967f7a50d0
2021-12-10 13:14:11 +00:00
Vincent Ambo
d4403638cf refactor(ops): Move irccat secret into agenix
The irccat module uses DynamicUser, so to grant permission to it a new
group has been added for irccat.

I have some vague memory of DynamicUser + Group not behaving as one
would expect, but we'll see what happens.

Change-Id: Iab9f6a3f1a53c4133b635458ce173250cc9a3fac
2021-12-10 16:13:31 +03:00
Vincent Ambo
002d183876 refactor(ops): Move clbot SSH key into agenix
Change-Id: Iae03ead7dda0509689a76f0d76f9cfeb8434e967
2021-12-10 16:13:31 +03:00
Vincent Ambo
0fa937551e refactor(gerrit-queue): Remove dependency on gin
Unnecessary dependency & complexity.

(Patch originally contributed by an anonymous contributor)

Change-Id: Id49dc362cb0c1b29937404447bb0b1f9794dc117
2021-12-10 13:09:10 +00:00
Vincent Ambo
6e4decf19b refactor(gerrit-queue): Gracefully handle missing changesets
(Patch contributed by an anonymous contributor)

Change-Id: I29fd7dd008d4e509ea074a38d3948946b26da7ab
2021-12-10 13:09:09 +00:00
Vincent Ambo
811e6d7d9f chore(whitby): Remove shadowsocks service
No longer required on whitby.

Change-Id: I93951c6b708eae81ddb03df920a4068c1ccde9e7
2021-12-10 13:07:09 +00:00
Vincent Ambo
7e3308df64 feat(fun/clbot): Add distinct messages for auto-submitted CLs
Detects autosubmitted CLs (other people's CLs submitted by clbot) and
modifies the text submitted to IRC accordingly.

If a CL is autosubmitted, we opt to highlight its author rather than
invoking noping.

Change-Id: Ibc21b7eeb2f0f2087097404baef6976384d68b09
2021-12-10 15:34:53 +03:00
Vincent Ambo
fc14c21bb9 fix(ops/pipelines): Move to static pipeline
This step would get inserted at the wrong point in the build pipeline
otherwise, causing a dependency cycle and causing the pipeline to fail.

Change-Id: I534568eec77f74ae6c47276820f8a9e99493a3ea
2021-12-10 11:01:21 +03:00
Vincent Ambo
e4231c9816 refactor(ops/pipelines): Move 🦆 logic into static pipeline
This simplifies the fallback logic used in case of Nix evaluation
failure and makes it so that the evaluation step itself is the one
that is marked as failed in Buildkite.

This is possible because the pipeline upload command will insert new
steps at the point where it runs in the pipeline, and not later.

Change-Id: I870534c004ebc457a1602623c4e5f9c0c68e28fc
2021-12-10 07:55:34 +00:00
Vincent Ambo
9ea4d55d81 refactor(ops): Move buildkite-agent-token into agenix
Relates to b/161

Change-Id: I5d3a698d437928966d8b78ce9e0ba226c1437655
2021-12-10 10:32:44 +03:00
Vincent Ambo
a123b9e0a2 refactor(ops): Move owothia secret into agenix
Relates to b/161

Change-Id: I25445281b0dd3c3f3660f8bb0d8337506a1e427b
2021-12-10 10:32:14 +03:00
Vincent Ambo
78744c00f5 refactor(ops): Move clbot secret into agenix
Relates to b/161

Change-Id: I7badf22ff93bb4e8b06e4dd4a8bf880b0bd48f09
2021-12-10 10:32:14 +03:00
Vincent Ambo
496d899428 feat(ops/secrets): Configure secrets for gerrit-queue
Adds a systemd EnvironmentFile secret that contains the Gerrit
username & password for gerrit-queue.

Change-Id: I25acf87764c26774045138402b8a417b6813ee8f
2021-12-10 10:32:14 +03:00
Vincent Ambo
4870b1a2ff feat(ops/modules): Add module for running gerrit-queue
This is not yet including the secret configuration for gerrit-queue,
and just expects the secret (gerrit username & password) to be
available in /etc/secrets.

Change-Id: Ia465ef7f3f521c70d606d7fdeba9aa83c7e1b98b
2021-12-10 10:32:14 +03:00
Vincent Ambo
a9dd719e7c chore(tvl-buildkite): Add jq and curl to agent paths
This is required for a simplification of the build pipeline (following
CL) and needs to be in a separate commit as it can not be done
atomically (merging the other commit to deploy it would immediately
break pipelines otherwise).

Change-Id: I5d8ec8f3238f79b5518d799486bf98d1d9516c43
2021-12-10 10:21:34 +03:00
Vincent Ambo
59f97332b3 subtree(3p/gerrit-queue): Vendor at commit '24f5a642'
Imported from github/tvlfyi/gerrit-queue, originally from
github/tweag/gerrit-queue but that upstream is unmaintained.

git-subtree-dir: third_party/gerrit-queue
git-subtree-mainline: ff10b7ab83
git-subtree-split: 24f5a642af
Change-Id: I307cc38185ab9e25eb102c95096298a150ae13a2
2021-12-09 16:13:56 +03:00
Vincent Ambo
ff10b7ab83 chore(3p): Remove gerrit-queue folder in preparation for vendoring
The upstream isn't really maintained anymore, so we may as well take
it over since we're patching it anyways.

Change-Id: I7dddc03ab90b00611520a77a26e73a5be1c2cfb8
2021-12-09 16:11:01 +03:00
Vincent Ambo
afa2d08fe7 feat(3p/gerrit-queue): Patch to use Gerrit 'Autosubmit' label
... instead of a hashtag in Gerrit.

Might be easier to review here:

24f5a642af

Change-Id: I1ae8d4607f7cb858135f88411c82e1a353b28105
2021-12-09 11:16:30 +00:00
Vincent Ambo
417a1ba9eb feat(3p/gerrit-queue): Add derivation for gerrit-queue
This is a Gerrit autosubmit bot (actually written by flokli) which we
intend to use.

For now we're using the plain upstream version, but we'll want to
patch some of the behaviours of it so there's a vendoring on the
horizon.

Change-Id: I021d41b55f9f678435d9aec6d359545577cb9ec0
2021-12-09 11:16:15 +00:00
Vincent Ambo
24f5a642af gerrit: Use a Gerrit label instead of hashtag for autosubmit
This moves to using a Gerrit label ('Autosubmit') with boolean values
for determining whether a developer wants to have a change
automatically submitted.

See also https://cl.tvl.fyi/c/depot/+/4172
2021-12-09 13:49:16 +03:00
Vincent Ambo
5fd3140cf3 fix(tazjin/emacs): Gerrit remote is now 'origin'
Change-Id: I44998510ff4be2fa137ea4c81f888e63ea438a56
2021-12-09 13:24:20 +03:00
Vincent Ambo
623de7920f feat(tazjin/russian): Add words 601-700
Change-Id: I17b1362502952d96b8787ad3c055d66f212fd60b
2021-12-09 08:58:32 +00:00
Griffin Smith
a0fcec54cd feat(grfn/emacs): Tweak rust async_test snippets
Add a stop at the `flavor` argument, since some async tests can use the
default config.

Change-Id: Iffd726b304d0d9dd94938bf23b2688715d1f4e20
2021-12-09 02:08:50 +00:00
Griffin Smith
d4c765743e feat(tvl.el): Add magit-gerrit-cherry-pick
Bound to `A g`, this behaves similarly to `magit-gerrit-checkout` - it
prompts for a CL number, then cherry-picks the latest patchset of that
CL number

Change-Id: Ieef970b99d96170e8c960cc7687ead9022948f8b
2021-12-09 02:08:42 +00:00
Griffin Smith
49d4d12a73 feat(grfn/emacs): Add a prolog use_module snippet
Change-Id: Ib5226a1ad0f084d3755cbfe40bf3556b3fa7fb2b
2021-12-09 02:08:42 +00:00
Griffin Smith
91f33b1a96 feat(grfn/emacs): Add some prolog-mode mappings
Change-Id: If1677024f9a211eee5d42a03413b5058dd797b9a
2021-12-09 02:08:42 +00:00