Commit graph

221 commits

Author SHA1 Message Date
Vincent Ambo
03076c3977 fix(besadii): Do not pass on update values for meta refs
Before this change, besadii would skip further processing of meta refs (which happen for every CL metadata change), but it would still schedule a build by returning an update - which would then inevitably fail.

This change makes besadii skip meta refs the same way it skips non-depot builds, i.e. completely.

Move *on* from meta refs, do *not* collect $100.

Change-Id: I269d2299f4d3cb1f9c041da8c92fa00ae7794b38
Reviewed-on: https://cl.tvl.fyi/c/depot/+/825
Reviewed-by: eta <eta@theta.eu.org>
Reviewed-by: BuildkiteCI
Tested-by: BuildkiteCI
2020-07-01 16:51:50 +00:00
Vincent Ambo
feb3f1a374 feat(nixos/clbot): Add a module for running clbot
Change-Id: I9c10906441c3222b74bcc820a67f11d96462fcfa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/821
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: BuildkiteCI
2020-06-30 23:39:48 +00:00
Cameron Kingsbury
7839b7b7a3 feat(tvl-slapd): update camsbury in slapd
Change-Id: Idce92352ad01f85bd7fbb102decdd1df26dda5f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/823
Reviewed-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-06-30 23:34:04 +00:00
Vincent Ambo
1eb8067bb0 fix(nixos/smtprelay): Only enable if the user asks for it
Change-Id: Ifbdf9bf9e89a1da68e8c823f61a33275183afcb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/822
Reviewed-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
2020-06-30 23:32:45 +00:00
Vincent Ambo
0380841eb1 fix(besadii): Don't log errors for /meta refs
These are updated for all sorts of things and should just be silently
ignored by besadii.

Change-Id: I0a6de373b21d6bef5fd31d0a1d3f72c501073bba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/801
Reviewed-by: BuildkiteCI
Reviewed-by: Kane York <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
2020-06-30 02:51:45 +00:00
Vincent Ambo
dc07977866 chore(ops): Clean up old GCP infrastructure files
This removes almost all of the GCP-infrastructure leftovers from my
previous setup.

The DNS configuration is retained, but moves to my user folder
instead.

Change-Id: I1867acd379443882f11a3c645846c9902eadd5b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/782
Tested-by: BuildkiteCI
Reviewed-by: eta <eta@theta.eu.org>
Reviewed-by: isomer <isomer@tvl.fyi>
2020-06-29 21:24:49 +00:00
Vincent Ambo
d3f9cb0ec3 feat(besadii): Temporarily add Code-Review labels on CLs
Besadii already adds 'Verified'-labels, which are used to signal CI
status on CLs, however we don't actually use these labels (yet) which
also means that they are not displayed in the Gerrit UI.

This change temporarily introduces the Code-Review label *in
addition* (with the same values as Verified), providing a build status
signal on the CL but without being required for submission.

Change-Id: I2c3a37c59aceb426815ad4e400c80ab85be482dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/781
Tested-by: BuildkiteCI
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-29 19:06:37 +00:00
Vincent Ambo
6d3a9e7b5f feat(besadii): Implement support for Buildkite's post-command hook
This hook is invoked by Buildkite (on the runner) after every build
stage. This change adds support in Besadii to run as this hook and
update the build status on a Gerrit CL.

Change-Id: Ie07a94d9b41645a77681cf42f6969d218abf93c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/761
Tested-by: BuildkiteCI
Reviewed-by: Kane York <rikingcoding@gmail.com>
2020-06-29 15:15:19 +00:00
Vincent Ambo
0bb24ed700 feat(besadii): Propagate Gerrit change ID & patchset to Buildkite
I previously implemented this in a CL that ended up being abandoned,
but it turns out we need it for the hook setup, anyways.

These environment variables become available during the build and,
crucially, to the post-build hooks.

Change-Id: Id6c1657947995e8bae1fa7b76184dd8be4c01525
Reviewed-on: https://cl.tvl.fyi/c/depot/+/739
Reviewed-by: Kane York <rikingcoding@gmail.com>
2020-06-29 01:19:51 +00:00
Griffin Smith
93d1ab7a54 feat(pipelines/depot): Run with --show-trace
So if an evaluation fails we get a stacktrace

Change-Id: I54cdc9e93c765ef7cf3a4d0cd79e6d067f4789d3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/733
2020-06-29 00:38:32 +00:00
Vincent Ambo
15335e7b3d feat(besadii): Enable automatic builds for CLs
This expands builds to also be triggered for updates to CL refs.

The message displayed on Buildkite will contain a link back to the
CL (& patchset) from which the build was triggered.

Change-Id: Ib36dee454aeb11d623b89c78b384359ee7ea3477
Reviewed-on: https://cl.tvl.fyi/c/depot/+/708
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
Reviewed-by: isomer <isomer@tvl.fyi>
2020-06-28 17:56:10 +00:00
Vincent Ambo
d3284112f6 refactor(besadii): Rename branchUpdate -> refUpdated
The name of the hook this type represents is 'refUpdated'. Since we're
adding support for additional hooks, it makes sense to to rename this
accordingly.

Change-Id: Ia568c85493813f5e754c77d0b993aaf246d3d595
Reviewed-on: https://cl.tvl.fyi/c/depot/+/667
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-28 17:20:22 +00:00
Vincent Ambo
1380d5998a feat(besadii): Trigger builds on Buildkite instead of Sourcehut
These builds run on runners that we control and disk space is (less
of) an issue there.

Change-Id: Id0a1436b2368418e447f6d5298ab474f829d4c97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/628
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-27 18:02:04 +00:00
Vincent Ambo
22b8a49b87 feat(ops/pipelines): Add Buildkite pipeline configuration
This adds configuration which generates the structure expected for
Buildkite pipelines, which can then be dynamically ingested by
Buildkite when a pipeline is triggered.

Change-Id: I61e3dc3affb19c1f2550ef827fa73b17f8d8ae47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/627
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-27 16:55:18 +00:00
Profpatsch
d640027f66 chore(tvl-slapd): sort alphabetically
bad ericvolp12

Change-Id: I508c7de48d4c2a7c734c38f79d0efeafec5d1e34
Reviewed-on: https://cl.tvl.fyi/c/depot/+/622
Reviewed-by: Profpatsch <mail@profpatsch.de>
2020-06-27 02:39:15 +00:00
Profpatsch
9851063f93 feat(tvl-slapd): add Profpatsch
Change-Id: I2d865a5271e7a3a2fe17009b306fe3f561a1290f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/621
Reviewed-by: tazjin <mail@tazj.in>
2020-06-27 02:38:33 +00:00
Artemis Tosini
41a094bf87 feat(tvl-slapd): add artemist to slapd
Signed-off-by: Artemis Tosini <me@artem.ist>
Change-Id: I11fc0cb58660d3cc55c6cf5489cc872a51454cb5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/609
Reviewed-by: tazjin <mail@tazj.in>
2020-06-26 22:50:29 +00:00
Vincent Ambo
6edf69da4b feat(besadii): Trigger separate builds for build target sets
In CL/570 we split up the build targets into different buckets, with
the idea that this should help us avoid the disk space issues on
Sourcehut.

This commit changes Besadii to read the list of target sets from a
file and trigger a separate build for each one of them.

Change-Id: If280fda5f40cd130c534c40911072e47c2d8f2be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/608
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-26 22:50:29 +00:00
Vincent Ambo
8dedf9d0be chore: Update 'master -> canon' in various places
Change-Id: I901c023f707a0223af673c217c65434e26e2ab1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/568
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 01:33:43 +00:00
Cameron Kingsbury
e7dd5e30e9 feat(tvl-slapd): add camsbury to slapd
add camsbury

From ccd385879ed384389983f4ddc55ef675f40e6119 Mon Sep 17 00:00:00 2001
From: Cameron Kingsbury <camsbury7@gmail.com>
Date: Tue, 23 Jun 2020 14:13:51 -0400
Subject: [PATCH] feat(tvl-slapd): add camsbury to slapd

Change-Id: I0fbf05ca80a006c9b2055509661fc1e93211e30f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/565
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2020-06-23 18:56:19 +00:00
Vincent Ambo
3b05be2fd0 feat(monorepo-gerrit): Use Sourcegraph as the gitweb for Gerrit
This points commit/file/etc. links from Gerrit to Sourcegraph instead
of cgit.

There's a minor problem with this: Some, but not all unsubmitted CLs
are missing in Sourcegraph for unclear reasons so they lead to 404s.

That problem is unrelated to this change and something we need to
investigate separately.

Change-Id: I9b0c1eca8781dc96984ba09b4a71960eb43583bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/541
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-20 17:29:50 +00:00
Vincent Ambo
d18faddba3 chore(nixos/sourcegraph): Configure Sourcegraph to use Cheddar
Change-Id: I2b91bef97c16254ffefcbc4da48ef161a859e7a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/521
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-20 03:00:46 +00:00
Vincent Ambo
a37b4fbef3 feat(besadii): Add Sourcegraph index update triggers
Sourcegraph has a heuristic for determining when to update the
repository that doesn't work for the TVL repository, where commits are
often in irregular short bursts.

This changes besadii to trigger Sourcegraph index updates when invoked
by Gerrit.

Change-Id: Ifcfc25406d9e25bbdc93e79c23608ce4c6b10ba8
2020-06-19 01:35:11 +01:00
Vincent Ambo
2183cfe5de chore(besadii): Make besadii a Go2 project
Change-Id: Id5aef73b131252fb9c880dd95fc72bfe083f1b75
2020-06-18 02:33:23 +01:00
Vincent Ambo
8f6309fe22 fix(monorepo-gerrit): Use displayName attribute as accountFullName
This attribute makes much more sense in this position semantically.

Change-Id: I16cc6304f42c577a2368bd7c9573fcb7dd276a9d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/448
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-17 03:03:22 +00:00
Vincent Ambo
27db1fc86b refactor(tvl-slapd): Move user definitions into Nix code
Implements a function that generates the LDIF record for each user and
templates it into the configuration.

This is slightly more user-friendly and less error-prone (people kept
getting the DNs wrong) than editing the contents manually.

Change-Id: Ic419d2ef464f9a94be5d54b666f7d53134b53eed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/447
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-17 03:03:22 +00:00
Vincent Ambo
9a7a0aa597 chore: Remove traces of Hound
We can always revert this if we want it back.

Change-Id: I1332b6dd541199584b7b5b94a8651172d79e53a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/442
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-16 20:32:18 +00:00
Vincent Ambo
2a764503be fix(monorepo-gerrit): Don't expire sessions unreasonably quickly
Changes the default session timeout to 3 months, which is a lot more
reasonable than the default of 12 hours.

See https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#cache.name.maxAge

Change-Id: I33bce8b072d64ab07f1b954c11068595dca5def7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/431
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-16 17:42:03 +00:00
Vincent Ambo
654f13d405 feat(nixos/sourcegraph): Add a module for running SourceGraph
This module spins up the Sourcegraph container.

Builds:

Note that this is contrary to how our other deployments work, but
packaging Sourcegraph is quite difficult (it's a Gitlab style
deployment with a lot of moving parts and third-party things that it
bundles).

If we decide to keep it around, we will want to look at packaging it
in Nix in the future.

Deployment:

The deployment is a hack. Sourcegraph does not support public
instances, but we want it to be public. To work around this we have
configured HTTP-proxy based authentication (i.e. auth via a header)
and hardcoded a static header.

This works, but lets anonymous users change the "Anonymous" user's
settings. We can expect this to get defaced (profile picture, name
etc), until we figure out how to write some nginx configuration to
drop those requests. See git-bug for details.

The Sourcegraph configuration is also not checked in to the
repository. It's unclear where in the data directory it is stored.

Change-Id: I414ff11c3b49989b6792d697bffc8a0edf96c9cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/425
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-16 13:40:49 +00:00
Eric Volpert
bf911a119e feat(tvl-slapd): Enable ericvolp12 user in LDAP
Thanks.

Change-Id: I5df1e5075b2e056ebde3e66e1cf17b220d650977
Reviewed-on: https://cl.tvl.fyi/c/depot/+/398
Reviewed-by: tazjin <mail@tazj.in>
2020-06-16 02:19:09 +00:00
Vincent Ambo
76c20f6bf7 fix(ops/nixos/tvl-slapd): Sort users & fix glittershark's DN
Change-Id: I33feedacfadaae53da000aff7d42fa06d2189f52
Reviewed-on: https://cl.tvl.fyi/c/depot/+/391
Reviewed-by: tazjin <mail@tazj.in>
2020-06-15 23:18:57 +00:00
Griffin Smith
849afbaeef chore(ops/nixos/tvl-slapd): add glittershark
Change-Id: I2e537079b88a3857964c6b7c66cd9221ca580958
Reviewed-on: https://cl.tvl.fyi/c/depot/+/390
Reviewed-by: tazjin <mail@tazj.in>
2020-06-15 23:17:06 +00:00
Vincent Ambo
a577fd83d6 chore(monorepo-gerrit): Remove 'owners-autoassign' plugin
This plugin just blindly assigns everyone and, as q3k has already
pointed out, just isn't particularly useful.

We might want to roll our own, for example:

19: 40:41 <+Remosi> I want the virtual owner thing, we could call it
 Gerrit Workgroup Synthesizer Queuing, or gwsq for short.
Change-Id: Ib12a921ae4047ac6a734035dd0900c8964fb12d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/350
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-15 00:38:48 +00:00
Vincent Ambo
a4b3f9af93 fix(3p/gerrit): Fix Gerrit derivation name and module configuration
Without these changes, the NixOS module isn't able to use the new
Gerrit derivation.

These changes are already deployed as I needed to make them to get
Gerrit back up.

Change-Id: Iad3aa6158789a014134fddccd40b508b81486100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/301
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-14 18:50:16 +00:00
Cynthia Revström
8dda9e56d7 feat(tvl-slapd): add cynthia to slapd
Change-Id: Ifb55ebd234d15fbaa6ef2e71f97ba7b8203ffcd9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/255
Reviewed-by: tazjin <mail@tazj.in>
2020-06-14 13:37:26 +00:00
Vincent Ambo
268729083e refactor(ops/nixos): Move my NixOS configurations to //users/tazjin
NixOS modules move one level up because it's unlikely that //ops/nixos
will contain actual systems at this point (they're user-specific).

This is the first users folder, so it is also added to the root
readTree invocation for the repository.

Change-Id: I546c701145fa204b7ba7518a8a56a783588629e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/244
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 23:52:35 +00:00
eta
ae85e8a871 feat(tvl-slapd): add eta to slapd
Change-Id: Ib34d59006645b992bd7b6cbd04fc7121ad3f0219
Reviewed-on: https://cl.tvl.fyi/c/depot/+/223
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 19:01:42 +00:00
Vincent Ambo
95e4faf464 feat(monorepo-gerrit): Include owners & owners-autoassign plugins
Change-Id: I62b90fb94293fc5148fe0fd7a06ea3d0e4d44199
Reviewed-on: https://cl.tvl.fyi/c/depot/+/222
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 18:59:12 +00:00
Vincent Ambo
9d01000257 fix(monorepo-gerrit): Do not place hooks in $out/bin
Gerrit does not expect a bin/ there.

Change-Id: I907f96690b8c6bb614dc11889712d7b122c5d5cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/181
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 05:15:15 +00:00
Kane York
665f131dda feat(camden): add builds shortlink
Change-Id: Iedd524d775349f24c13fe7c118830b7d4dfdec49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/81
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 05:06:18 +00:00
Vincent Ambo
b88cbe0dab feat(monorepo-gerrit): Enable Gerrit hooks & configure besadii
Loads the 'hooks' plugin into Gerrit, which - as per my interpretation
of the docs - is going to execute any hooks for which there are
matching binaries.

The intention here is that besadii should implement most of the hooks
we care about. As a start, it is symlinked here to the `ref-updated`
hook.

Change-Id: I6482a9d71cc08908c29dd10f786cbba32b33d04d
2020-06-13 06:04:02 +01:00
Vincent Ambo
b094e65bfc feat(besadii): Adapt into a Gerrit ref-updated-hook
Besadii was previously invoked as a git post-update hook, but Gerrit
does not use these hooks and instead has its own concept of hooks.

This change adapts besadii to be compatible with the way Gerrit hooks
are invoked (arguments being passed as flags, rather than via stdin).

Change-Id: I487b3a9e15810583bc5442fdc024ee2771c580cb
2020-06-13 06:03:45 +01:00
Vincent Ambo
8735c63e97 feat(monorepo-gerrit): Enable download-commands plugin
This enables the display of various download commands on change pages,
which makes things like checking out refs for review locally easier.

Change-Id: I3c29854aa0cf1aa393efb89b7516bbf84e0083d4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/162
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 03:16:24 +00:00
Vincent Ambo
213d637ca9 fix(monorepo-gerrit): Configure advertised address for SSH correctly
This is a prerequisite for setting up the download-commands plugin.

Change-Id: I7803ef18be759f95aec020e4a00ca8e0fb48bfe0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/161
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 03:15:58 +00:00
Vincent Ambo
e09c4a0ae8 chore(monorepo-gerrit): Point SMTP configuration at smtprelay
Change-Id: I33085974fb3764f8a6df7f16245b2f5602f94118
Reviewed-on: https://cl.tvl.fyi/c/depot/+/102
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 02:16:58 +00:00
nyanotech
b994d28ccc feat(tvl-slapd): Add nyanotech to slapd, sort the list
Change-Id: I9ffd2fb3b9ae3f6c8c381f496769eb8977caadeb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/124
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-13 01:34:55 +00:00
Vincent Ambo
c2a5073339 feat(nixos/smtprelay): Add derivation & module for SMTP relay
This adds a little tool that can be used to relay mail to Gmail (and
other SMTP servers). It is intended to be used by Gerrit, which is
incompatible with Gmail's SMTP servers.

Configuration has been tested by performing a few sends through the
tvlbot@tazj.in account.

Note that this is using the standard Gmail SMTP server. Using the
smtp-relay server relies on IP whitelisting, but camden.tazj.in has a
larger number of IPv6 addresses than can be whitelisted (the maximum
is 65k). This means that we are limited to 2000 mails per recipient
per day, which should be fine.

Change-Id: Ie43564d753030f5c800a9cdb4ae98292877d80dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/101
Reviewed-by: edef <edef@edef.eu>
2020-06-13 01:23:01 +00:00
Vincent Ambo
de4f540ed1 feat(monorepo-gerrit): Configure outbound emails for reviews
Configures Gerrit send emails from tvlbot@tazj.in for outgoing review
notifications. Emails are always plain-text and can contain diffs (up
to a maximum size of 256KiB).

The configuration options for this are documented at:

https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#sendemail

Note: The password for this user is stored on the host, in a file that
is not part of version-control and is only readable by the 'git' user.

We should probably figure out a way to do secrets management ...

Change-Id: I2f99b34b1a774c28d814b0aba1f1b78fd512854e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/92
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-12 22:36:10 +00:00
Vincent Ambo
a9f3621fd7 feat(camden): Move hound to cs.tvl.fyi
The old host at cs.tazj.in now redirects there, and I've added a
helper function for creating these redirections.

Change-Id: I66794d752df46c8e795e47aedfaffd8c27c45627
Reviewed-on: https://cl.tvl.fyi/c/depot/+/89
Reviewed-by: riking <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
2020-06-12 02:17:02 +00:00
Vincent Ambo
7bad1fe852 fix(camden): addSSL -> forceSSL for all pages
Change-Id: I451d1bc1a21d4ff25c0c70c963cf17bb924961db
Reviewed-on: https://cl.tvl.fyi/c/depot/+/84
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-12 01:51:32 +00:00