Commit graph

214 commits

Author SHA1 Message Date
Vincent Ambo
6d3a9e7b5f feat(besadii): Implement support for Buildkite's post-command hook
This hook is invoked by Buildkite (on the runner) after every build
stage. This change adds support in Besadii to run as this hook and
update the build status on a Gerrit CL.

Change-Id: Ie07a94d9b41645a77681cf42f6969d218abf93c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/761
Tested-by: BuildkiteCI
Reviewed-by: Kane York <rikingcoding@gmail.com>
2020-06-29 15:15:19 +00:00
Vincent Ambo
0bb24ed700 feat(besadii): Propagate Gerrit change ID & patchset to Buildkite
I previously implemented this in a CL that ended up being abandoned,
but it turns out we need it for the hook setup, anyways.

These environment variables become available during the build and,
crucially, to the post-build hooks.

Change-Id: Id6c1657947995e8bae1fa7b76184dd8be4c01525
Reviewed-on: https://cl.tvl.fyi/c/depot/+/739
Reviewed-by: Kane York <rikingcoding@gmail.com>
2020-06-29 01:19:51 +00:00
Griffin Smith
93d1ab7a54 feat(pipelines/depot): Run with --show-trace
So if an evaluation fails we get a stacktrace

Change-Id: I54cdc9e93c765ef7cf3a4d0cd79e6d067f4789d3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/733
2020-06-29 00:38:32 +00:00
Vincent Ambo
15335e7b3d feat(besadii): Enable automatic builds for CLs
This expands builds to also be triggered for updates to CL refs.

The message displayed on Buildkite will contain a link back to the
CL (& patchset) from which the build was triggered.

Change-Id: Ib36dee454aeb11d623b89c78b384359ee7ea3477
Reviewed-on: https://cl.tvl.fyi/c/depot/+/708
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
Reviewed-by: isomer <isomer@tvl.fyi>
2020-06-28 17:56:10 +00:00
Vincent Ambo
d3284112f6 refactor(besadii): Rename branchUpdate -> refUpdated
The name of the hook this type represents is 'refUpdated'. Since we're
adding support for additional hooks, it makes sense to to rename this
accordingly.

Change-Id: Ia568c85493813f5e754c77d0b993aaf246d3d595
Reviewed-on: https://cl.tvl.fyi/c/depot/+/667
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-28 17:20:22 +00:00
Vincent Ambo
1380d5998a feat(besadii): Trigger builds on Buildkite instead of Sourcehut
These builds run on runners that we control and disk space is (less
of) an issue there.

Change-Id: Id0a1436b2368418e447f6d5298ab474f829d4c97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/628
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-27 18:02:04 +00:00
Vincent Ambo
22b8a49b87 feat(ops/pipelines): Add Buildkite pipeline configuration
This adds configuration which generates the structure expected for
Buildkite pipelines, which can then be dynamically ingested by
Buildkite when a pipeline is triggered.

Change-Id: I61e3dc3affb19c1f2550ef827fa73b17f8d8ae47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/627
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-27 16:55:18 +00:00
Profpatsch
d640027f66 chore(tvl-slapd): sort alphabetically
bad ericvolp12

Change-Id: I508c7de48d4c2a7c734c38f79d0efeafec5d1e34
Reviewed-on: https://cl.tvl.fyi/c/depot/+/622
Reviewed-by: Profpatsch <mail@profpatsch.de>
2020-06-27 02:39:15 +00:00
Profpatsch
9851063f93 feat(tvl-slapd): add Profpatsch
Change-Id: I2d865a5271e7a3a2fe17009b306fe3f561a1290f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/621
Reviewed-by: tazjin <mail@tazj.in>
2020-06-27 02:38:33 +00:00
Artemis Tosini
41a094bf87 feat(tvl-slapd): add artemist to slapd
Signed-off-by: Artemis Tosini <me@artem.ist>
Change-Id: I11fc0cb58660d3cc55c6cf5489cc872a51454cb5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/609
Reviewed-by: tazjin <mail@tazj.in>
2020-06-26 22:50:29 +00:00
Vincent Ambo
6edf69da4b feat(besadii): Trigger separate builds for build target sets
In CL/570 we split up the build targets into different buckets, with
the idea that this should help us avoid the disk space issues on
Sourcehut.

This commit changes Besadii to read the list of target sets from a
file and trigger a separate build for each one of them.

Change-Id: If280fda5f40cd130c534c40911072e47c2d8f2be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/608
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-26 22:50:29 +00:00
Vincent Ambo
8dedf9d0be chore: Update 'master -> canon' in various places
Change-Id: I901c023f707a0223af673c217c65434e26e2ab1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/568
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 01:33:43 +00:00
Cameron Kingsbury
e7dd5e30e9 feat(tvl-slapd): add camsbury to slapd
add camsbury

From ccd385879ed384389983f4ddc55ef675f40e6119 Mon Sep 17 00:00:00 2001
From: Cameron Kingsbury <camsbury7@gmail.com>
Date: Tue, 23 Jun 2020 14:13:51 -0400
Subject: [PATCH] feat(tvl-slapd): add camsbury to slapd

Change-Id: I0fbf05ca80a006c9b2055509661fc1e93211e30f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/565
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2020-06-23 18:56:19 +00:00
Vincent Ambo
3b05be2fd0 feat(monorepo-gerrit): Use Sourcegraph as the gitweb for Gerrit
This points commit/file/etc. links from Gerrit to Sourcegraph instead
of cgit.

There's a minor problem with this: Some, but not all unsubmitted CLs
are missing in Sourcegraph for unclear reasons so they lead to 404s.

That problem is unrelated to this change and something we need to
investigate separately.

Change-Id: I9b0c1eca8781dc96984ba09b4a71960eb43583bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/541
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-20 17:29:50 +00:00
Vincent Ambo
d18faddba3 chore(nixos/sourcegraph): Configure Sourcegraph to use Cheddar
Change-Id: I2b91bef97c16254ffefcbc4da48ef161a859e7a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/521
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-20 03:00:46 +00:00
Vincent Ambo
a37b4fbef3 feat(besadii): Add Sourcegraph index update triggers
Sourcegraph has a heuristic for determining when to update the
repository that doesn't work for the TVL repository, where commits are
often in irregular short bursts.

This changes besadii to trigger Sourcegraph index updates when invoked
by Gerrit.

Change-Id: Ifcfc25406d9e25bbdc93e79c23608ce4c6b10ba8
2020-06-19 01:35:11 +01:00
Vincent Ambo
2183cfe5de chore(besadii): Make besadii a Go2 project
Change-Id: Id5aef73b131252fb9c880dd95fc72bfe083f1b75
2020-06-18 02:33:23 +01:00
Vincent Ambo
8f6309fe22 fix(monorepo-gerrit): Use displayName attribute as accountFullName
This attribute makes much more sense in this position semantically.

Change-Id: I16cc6304f42c577a2368bd7c9573fcb7dd276a9d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/448
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-17 03:03:22 +00:00
Vincent Ambo
27db1fc86b refactor(tvl-slapd): Move user definitions into Nix code
Implements a function that generates the LDIF record for each user and
templates it into the configuration.

This is slightly more user-friendly and less error-prone (people kept
getting the DNs wrong) than editing the contents manually.

Change-Id: Ic419d2ef464f9a94be5d54b666f7d53134b53eed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/447
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-17 03:03:22 +00:00
Vincent Ambo
9a7a0aa597 chore: Remove traces of Hound
We can always revert this if we want it back.

Change-Id: I1332b6dd541199584b7b5b94a8651172d79e53a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/442
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-16 20:32:18 +00:00
Vincent Ambo
2a764503be fix(monorepo-gerrit): Don't expire sessions unreasonably quickly
Changes the default session timeout to 3 months, which is a lot more
reasonable than the default of 12 hours.

See https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#cache.name.maxAge

Change-Id: I33bce8b072d64ab07f1b954c11068595dca5def7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/431
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-16 17:42:03 +00:00
Vincent Ambo
654f13d405 feat(nixos/sourcegraph): Add a module for running SourceGraph
This module spins up the Sourcegraph container.

Builds:

Note that this is contrary to how our other deployments work, but
packaging Sourcegraph is quite difficult (it's a Gitlab style
deployment with a lot of moving parts and third-party things that it
bundles).

If we decide to keep it around, we will want to look at packaging it
in Nix in the future.

Deployment:

The deployment is a hack. Sourcegraph does not support public
instances, but we want it to be public. To work around this we have
configured HTTP-proxy based authentication (i.e. auth via a header)
and hardcoded a static header.

This works, but lets anonymous users change the "Anonymous" user's
settings. We can expect this to get defaced (profile picture, name
etc), until we figure out how to write some nginx configuration to
drop those requests. See git-bug for details.

The Sourcegraph configuration is also not checked in to the
repository. It's unclear where in the data directory it is stored.

Change-Id: I414ff11c3b49989b6792d697bffc8a0edf96c9cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/425
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-16 13:40:49 +00:00
Eric Volpert
bf911a119e feat(tvl-slapd): Enable ericvolp12 user in LDAP
Thanks.

Change-Id: I5df1e5075b2e056ebde3e66e1cf17b220d650977
Reviewed-on: https://cl.tvl.fyi/c/depot/+/398
Reviewed-by: tazjin <mail@tazj.in>
2020-06-16 02:19:09 +00:00
Vincent Ambo
76c20f6bf7 fix(ops/nixos/tvl-slapd): Sort users & fix glittershark's DN
Change-Id: I33feedacfadaae53da000aff7d42fa06d2189f52
Reviewed-on: https://cl.tvl.fyi/c/depot/+/391
Reviewed-by: tazjin <mail@tazj.in>
2020-06-15 23:18:57 +00:00
Griffin Smith
849afbaeef chore(ops/nixos/tvl-slapd): add glittershark
Change-Id: I2e537079b88a3857964c6b7c66cd9221ca580958
Reviewed-on: https://cl.tvl.fyi/c/depot/+/390
Reviewed-by: tazjin <mail@tazj.in>
2020-06-15 23:17:06 +00:00
Vincent Ambo
a577fd83d6 chore(monorepo-gerrit): Remove 'owners-autoassign' plugin
This plugin just blindly assigns everyone and, as q3k has already
pointed out, just isn't particularly useful.

We might want to roll our own, for example:

19: 40:41 <+Remosi> I want the virtual owner thing, we could call it
 Gerrit Workgroup Synthesizer Queuing, or gwsq for short.
Change-Id: Ib12a921ae4047ac6a734035dd0900c8964fb12d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/350
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-15 00:38:48 +00:00
Vincent Ambo
a4b3f9af93 fix(3p/gerrit): Fix Gerrit derivation name and module configuration
Without these changes, the NixOS module isn't able to use the new
Gerrit derivation.

These changes are already deployed as I needed to make them to get
Gerrit back up.

Change-Id: Iad3aa6158789a014134fddccd40b508b81486100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/301
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-14 18:50:16 +00:00
Cynthia Revström
8dda9e56d7 feat(tvl-slapd): add cynthia to slapd
Change-Id: Ifb55ebd234d15fbaa6ef2e71f97ba7b8203ffcd9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/255
Reviewed-by: tazjin <mail@tazj.in>
2020-06-14 13:37:26 +00:00
Vincent Ambo
268729083e refactor(ops/nixos): Move my NixOS configurations to //users/tazjin
NixOS modules move one level up because it's unlikely that //ops/nixos
will contain actual systems at this point (they're user-specific).

This is the first users folder, so it is also added to the root
readTree invocation for the repository.

Change-Id: I546c701145fa204b7ba7518a8a56a783588629e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/244
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 23:52:35 +00:00
eta
ae85e8a871 feat(tvl-slapd): add eta to slapd
Change-Id: Ib34d59006645b992bd7b6cbd04fc7121ad3f0219
Reviewed-on: https://cl.tvl.fyi/c/depot/+/223
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 19:01:42 +00:00
Vincent Ambo
95e4faf464 feat(monorepo-gerrit): Include owners & owners-autoassign plugins
Change-Id: I62b90fb94293fc5148fe0fd7a06ea3d0e4d44199
Reviewed-on: https://cl.tvl.fyi/c/depot/+/222
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 18:59:12 +00:00
Vincent Ambo
9d01000257 fix(monorepo-gerrit): Do not place hooks in $out/bin
Gerrit does not expect a bin/ there.

Change-Id: I907f96690b8c6bb614dc11889712d7b122c5d5cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/181
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 05:15:15 +00:00
Kane York
665f131dda feat(camden): add builds shortlink
Change-Id: Iedd524d775349f24c13fe7c118830b7d4dfdec49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/81
Reviewed-by: tazjin <mail@tazj.in>
2020-06-13 05:06:18 +00:00
Vincent Ambo
b88cbe0dab feat(monorepo-gerrit): Enable Gerrit hooks & configure besadii
Loads the 'hooks' plugin into Gerrit, which - as per my interpretation
of the docs - is going to execute any hooks for which there are
matching binaries.

The intention here is that besadii should implement most of the hooks
we care about. As a start, it is symlinked here to the `ref-updated`
hook.

Change-Id: I6482a9d71cc08908c29dd10f786cbba32b33d04d
2020-06-13 06:04:02 +01:00
Vincent Ambo
b094e65bfc feat(besadii): Adapt into a Gerrit ref-updated-hook
Besadii was previously invoked as a git post-update hook, but Gerrit
does not use these hooks and instead has its own concept of hooks.

This change adapts besadii to be compatible with the way Gerrit hooks
are invoked (arguments being passed as flags, rather than via stdin).

Change-Id: I487b3a9e15810583bc5442fdc024ee2771c580cb
2020-06-13 06:03:45 +01:00
Vincent Ambo
8735c63e97 feat(monorepo-gerrit): Enable download-commands plugin
This enables the display of various download commands on change pages,
which makes things like checking out refs for review locally easier.

Change-Id: I3c29854aa0cf1aa393efb89b7516bbf84e0083d4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/162
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 03:16:24 +00:00
Vincent Ambo
213d637ca9 fix(monorepo-gerrit): Configure advertised address for SSH correctly
This is a prerequisite for setting up the download-commands plugin.

Change-Id: I7803ef18be759f95aec020e4a00ca8e0fb48bfe0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/161
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 03:15:58 +00:00
Vincent Ambo
e09c4a0ae8 chore(monorepo-gerrit): Point SMTP configuration at smtprelay
Change-Id: I33085974fb3764f8a6df7f16245b2f5602f94118
Reviewed-on: https://cl.tvl.fyi/c/depot/+/102
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-13 02:16:58 +00:00
nyanotech
b994d28ccc feat(tvl-slapd): Add nyanotech to slapd, sort the list
Change-Id: I9ffd2fb3b9ae3f6c8c381f496769eb8977caadeb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/124
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-13 01:34:55 +00:00
Vincent Ambo
c2a5073339 feat(nixos/smtprelay): Add derivation & module for SMTP relay
This adds a little tool that can be used to relay mail to Gmail (and
other SMTP servers). It is intended to be used by Gerrit, which is
incompatible with Gmail's SMTP servers.

Configuration has been tested by performing a few sends through the
tvlbot@tazj.in account.

Note that this is using the standard Gmail SMTP server. Using the
smtp-relay server relies on IP whitelisting, but camden.tazj.in has a
larger number of IPv6 addresses than can be whitelisted (the maximum
is 65k). This means that we are limited to 2000 mails per recipient
per day, which should be fine.

Change-Id: Ie43564d753030f5c800a9cdb4ae98292877d80dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/101
Reviewed-by: edef <edef@edef.eu>
2020-06-13 01:23:01 +00:00
Vincent Ambo
de4f540ed1 feat(monorepo-gerrit): Configure outbound emails for reviews
Configures Gerrit send emails from tvlbot@tazj.in for outgoing review
notifications. Emails are always plain-text and can contain diffs (up
to a maximum size of 256KiB).

The configuration options for this are documented at:

https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#sendemail

Note: The password for this user is stored on the host, in a file that
is not part of version-control and is only readable by the 'git' user.

We should probably figure out a way to do secrets management ...

Change-Id: I2f99b34b1a774c28d814b0aba1f1b78fd512854e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/92
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-12 22:36:10 +00:00
Vincent Ambo
a9f3621fd7 feat(camden): Move hound to cs.tvl.fyi
The old host at cs.tazj.in now redirects there, and I've added a
helper function for creating these redirections.

Change-Id: I66794d752df46c8e795e47aedfaffd8c27c45627
Reviewed-on: https://cl.tvl.fyi/c/depot/+/89
Reviewed-by: riking <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
2020-06-12 02:17:02 +00:00
Vincent Ambo
7bad1fe852 fix(camden): addSSL -> forceSSL for all pages
Change-Id: I451d1bc1a21d4ff25c0c70c963cf17bb924961db
Reviewed-on: https://cl.tvl.fyi/c/depot/+/84
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-12 01:51:32 +00:00
edef
52c0be524e chore(ops/nixos/modules): Add edef to slapd
Change-Id: I063a09cdc3bb81397a44f7356f1c11ebd715f74f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/88
Reviewed-by: tazjin <mail@tazj.in>
2020-06-12 01:44:51 +00:00
Kane York
1783239c3f feat(camden): add /irc/ shortlink
Change-Id: If17c758c323aaf00fdf26ddfafaea10acbf1453e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/70
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: riking <rikingcoding@gmail.com>
2020-06-12 01:15:02 +00:00
Vincent Ambo
37bbc43146 feat(camden): Move cgit to code.tvl.fyi
Moves the host at which cgit is served to 'code.tvl.fyi'.

Also updates related projects that link to this, most importantly:

* Hound's & Gerrit's cgit link bases have been updated
* besadii is updated to request CI builds for the new location

Change-Id: I44e3e584010ac29cc913ebb1a197c996eb024d80
Reviewed-on: https://cl.tvl.fyi/c/depot/+/71
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-12 01:14:21 +00:00
Sergiusz Bazanski
79fdb0bb5f chore(ops/nixos/modules): Add q3k to slapd
Change-Id: I083bc4e9283a882e97a6b9098d6a126ca7bb0a93
Reviewed-on: https://cl.tvl.fyi/c/depot/+/68
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-12 00:44:36 +00:00
Vincent Ambo
dc2fd3b521 chore(nixos/camden): Point hound at the depot on gerrit
Change-Id: I19cbffae75017ceefbc19397c54156eb348eda27
Reviewed-on: https://cl.tvl.fyi/c/depot/+/65
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-11 23:47:24 +00:00
Vincent Ambo
4e3d3b6c22 chore(nixos/frog): Move frog to nixos-unstable
There are no remaining traces of Emacs breakage in unstable - as far
as I can tell.

Change-Id: I06c5d78aa3ff9c0cc00c62e6d6966c5079fb3b24
Reviewed-on: https://cl.tvl.fyi/c/depot/+/63
Reviewed-by: tazjin <mail@tazj.in>
2020-06-11 23:20:41 +00:00
Vincent Ambo
80d324b53b feat(nixos/frog): Enable lieer sync for mail@tazj.in
Change-Id: I38a338143d57d5f49532d200910f9406fa49f535
Reviewed-on: https://cl.tvl.fyi/c/depot/+/61
Reviewed-by: tazjin <mail@tazj.in>
2020-06-11 23:18:25 +00:00