I could not get nginx to serve it from `tvl.su/tvixbolt`, and ran out
of interest in trying to fix it, so lets put it on a subdomain
instead.
Change-Id: I1313d75cc9831d94a894191376534b1e5186a76a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6537
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
otherwise posting this to reddit's /r/russian is not possible, as they
ban all links to Russian-affiliated sites
Change-Id: I8d23f0961ec7ef097fc2dbdd0aaa178861a19c10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5992
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
instead of setting a wildcard record (which causes really weird
behaviour if you set your search domain to tvl.su/tvl.fyi, which I
do), DNS records for services running on whitby are now set
explicitly.
Change-Id: Ia05399b62dad326942fe0efda30782ce153df99d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5961
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
at some point in the far past, there was an RSS feed at `/en/rss.xml`.
It seems to still get a single hit or so every hour, which currently
404s.
Change-Id: Ieb13c2c0232861a50a54bc2a4087d9ccb21185cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5962
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
on whitby, cgit runs as the gerrit user to get access to serving
gerrit's repositories directly.
on other machines (e.g. sanduny) this isn't necessary, as we have a
world-readable depot replica.
Change-Id: Ibf7e7cc08e5909e0fa182e561ab0cb472188edcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5932
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
this configures gerrit's built-in replication plugin to push every
change in depot to sanduny.
this allows us to serve a replica of depot from sanduny.
manual config that was needed which needs to be automated:
* system-wide known_hosts does not work, needed one in /var/lib/git
* .ssh/config MUST be present and configured for sanduny.tvl.su
Change-Id: Iba399f2328abb5acb65dae19a36e265eea0952ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5915
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
This module sets up a user with an SSH key and permissions to receive
a (pushed) replica of depot from Gerrit.
This still needs appropriate configuration in Gerrit's replication
plugin on the other end.
This module has been enabled for sanduny. For now it does not (yet)
configure git serving.
Change-Id: I0fb6f7e696609e71008308e855bdf305dcbcd4f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5913
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
It occured to me yesterday that with the config inside of the module
it is kind of difficult to test cgit locally.
This moves it back to a separate location (//web/cgit-tvl) and makes
the most important things configurable via overrides.
Change-Id: I9b0f4c60b75c31441e1718e63b5b55aba3100aae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5893
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Before this commit besadii only worked for repos having
'refs/heads/canon' as main branch.
Change-Id: Ia2ceb8a720c675be84bc3d81b89338522cea6ebd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5862
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
Remove a workaround for a GleSYS provider bug that was fixed in the
last release.
Change-Id: Ibd25de0b4dcccd781518d5d0ae1c75d296f6b05f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5845
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
In order to run this the secrets needs to be sourced, e.g.:
eval $(age --decrypt -i ~/.ssh/id_ed25519 $(git rev-parse --show-toplevel)/ops/secrets/tf-buildkite.age)
Change-Id: I9f6a02c0dac22f584181635861ddbb06cf849f14
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5838
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
This is in preparation for a subsequent CL that will do much more
significant changes in //nix/buildkite.
Change-Id: I80a8d67d3a7d593854c8d711572483c2581e7881
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5824
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
This is a less invasive way to achieve the same goal as cl/5681, by
preventing the already existing nixpkgs store path from being dumped
again at the call site. To support nixpkgsBisectPath, we simply check if
pkgs.path is below builtins.storeDir and use builtins.storePath based on
that.
This is actually similar to the approach taken in the nixpkgs
documentation system which tries to limit the amount of nixpkgs that
needs to be dumped by using filterSource on specific subtrees of
nixpkgs. For this to work it has to insist on pkgs.path being an
ordinary Nix path, though.
Change-Id: Idf892f90a5d811184568e4702a901c334d56210e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5787
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Buildkite can't handle more than one filter for the query; as of the
last commit it just returned an empty list.
I've verified with curl based on the request the previous attempt
constructed that this works as intended with only setting the commit.
Behaviour is probably undefined if there are two builds for the same
commit (i.e. a retry). Which one will you see? Who knows!
However, since the commit hash contains the Change-Id, we can't get a
situation where the build was for two different CLs at the same
commit. Gerrit wouldn't allow that.
Change-Id: I0dcd0ff44c28d3d15cba23461970bfc8483f4e48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5768
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This became an "official" module and dropped the `pw-` prefix.
Relates to b/184
Change-Id: I963f83b55b83015b022ab1b8330ea710d2258631
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5751
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Named after the Mexican restaurant, El Tarasco, in El Porto, which I live 3m
walking distance from.
Change-Id: I2cd4b68eaa974ad6c8fec73e0566bc0b831c57a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5743
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
The patchsetSha is one of the things passed in to the `fetch()`
interface, and Buildkite's API (now?) supports filtering by the commit
hash in addition.
With this combination, we should not accidentally display builds for
the wrong patch set.
Change-Id: I6bb26dd7387f2dd00291990cadd38629ecda999b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5702
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
When Keycloak and oauth2_proxy are restarted simultaneously, the
latter might try to come up (repeatedly!) before Keycloak can serve it
properly.
This leads to systemd considering the unit failed.
Since this all happens in the span of a second or so, slightly
increase the restart delay of the service to ensure it comes back
after Keycloak is ready.
A "proper" fix might be to add a script that runs before the actual
service and waits for Keycloak, but I don't want to prioritise that
right now.
Change-Id: I4dadba686de60ffc103fe889ce19f05ca1d7d4fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5695
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Somehow this ended up generating an empty file, with this change it is
fine again. I was looking at the recent commits of the module in
nixpkgs but couldn't quite figure it out, there are also some vague
references to the attribute set key being used as a hostname, but this
doesn't seem to be true in practice.
To be clear, the previous code was wrong, but at some point it
generated a file that accidentally worked.
Change-Id: I42d55730c09daafe6d6fe0eb3647135e84737bca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5670
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
libdigidocpp is a dependency of qdigidoc4(1) already.
This will need https://github.com/NixOS/nixpkgs/pull/174055
"libdigidocpp: Fix PKCS11 module library path" to work, though.
Change-Id: Ic8d671077977b1d1f099a8b4b23cc537b52aa954
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5647
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
The new version brings the new secretsDir setting which means we no
longer have to hardcode /run/agenix everywhere.
Change-Id: I4b579d7233d315a780d7671869d5d06722d769fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5646
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: sterni <sternenseemann@systemli.org>
I've only been a couple months lurking in the IRC ...
Change-Id: Idebf96d3bf1124f0a97e11e0f854e8c6d4be8d8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5662
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Nixery is going to gain a new binary (used for building images without
a registry server); to prepare for this the server binary has moved to
cmd/server and the Nix build logic has been updated to wrap this
binary and set the required environment variables.
Change-Id: I9b4f49f47872ae76430463e2fcb8f68114070f72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5603
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This is my new Huawei MateBook X.
Change-Id: I32a8b77dd8f53b3c89bf63f448cd2880f9a457b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5554
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Firefox users can add p11-kit-proxy (or other SecurityDevices)
system-wide, by making use of the extraPolicies functionality.
Change-Id: Id58b6cab425199fb0e09e846db2a86d302c0de0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5534
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Changes:
* updated keycloak configuration for new version
* migrate to emacs28 outside of //users, re-add emacs27 but with a
warning attached urging people to migrate
Change-Id: I3e5765a63934541f72f6c4a8673d3b4671850c93
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5501
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Cleans up a whole bunch of things I wanted to get out of the door
right away:
* depot internal references to //third_party/nixery have been replaced
with //tools/nixery
* cleaned up files from Github
* fixed SPDX & Copyright headers
* code formatting and inclusion in //tools/depotfmt checks
Change-Id: Iea79f0fdf3aa04f71741d4f4032f88605ae415bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5486
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
ava is my new (NixOS!) work machine :)
Change-Id: I1f089f00c02519d5d1d93d011f29075d53500e74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5450
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Someone already packaged the required software, so I didn't have to do
that.
Change-Id: Ifc6a68fd4cd89f4718368a05acb6c6f536e01aab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5431
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
If the Keycloak service is running on the same machine as the oauth2
proxy (spoiler alert: it is!), let the service depend on it.
Change-Id: I30e4222b4cd5589e08849ef6f37cf1fb4369f55a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5421
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
For external users of the pipeline construction, the token might be in
a different path than `/run/agenix/buildkite-graphql-token`.
It is made configurable through the BUILDKITE_TOKEN_PATH environment
variable. This should be configured on the pipeline level to apply to
all steps.
Change-Id: I23c52e2d705e4134b8b013f8603f92e5533a6e44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5424
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
crimp is in TVL (//net/crimp), and it has fewer dependencies than
ureq (including - finally - no more old time or chrono).
Change-Id: I354f8f78b34a85abe3af671ffeffbe6a7fded5ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5318
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
With this change, we still depend on chrono (through medallion), but
but I'm going to try and fix that upstream as well.
Change-Id: Iefd3d8578ea8870961107f3222dea7f936c2dd9a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5311
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Apparently failure is not hip anymore, and crate updates are forcing
the use of anyhow now. Whatever.
The functionality basically stays the same, maybe error messages will
look a little bit different now.
Change-Id: I173d644688785339c16161ddeec47a534123710f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5307
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
As suggested by sterni, this makes the self-redirect of a machine to
its configuration a generic module working by convention.
In the process of moving this two small fixes have been applied:
* redirect is only applied if the URI is `/`, this is required for
ACME to work
* addSSL = true is added, otherwise we have a certificate but no TLS
listener
Change-Id: Icaef041ff681253a61e36926417bdb2844e3f93d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5313
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This makes the journaldriver configuration machine-independent.
The secret is loaded from agenix instead of being persisted on disk.
Change-Id: I592ae7f5726fcb7f37a406f69dcf5ac498eeb1b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5302
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
With this change, entering just "whitby.tvl.fyi" or "sanduny.tvl.su"
in a browser will redirect users to their machine configurations.
Change-Id: Ibf076a469bcce073e1b1970aa568d6fe16a5c75a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5304
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
This needs to be present on all machines that run ACME stuff.
I've switched the address for a .su one because I have a catchall for
these.
Change-Id: I7af8e1f1cb2fcfbcba4b7d1930ed0edef0106d72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5306
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This changes the structure of secrets.nix a bit to split between
secrets for whitby, and secrets for all TVL machines.
Change-Id: I791f0ce42a16b33051e24a7a6c5b153761ed9eb3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5300
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This will be an additional web host / fallback git server for whitby
incidents.
Change-Id: Icd6f7ce574ffd520b5783a50ff317feed7b71fc6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5297
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Rather than defining all system users inline on whitby, move them into
a module that can be imported on multiple machines.
Configuration for terminfos that we've added follows along.
Note that while doing this I've disabled logins for riking and isomer
since they are currently inactive in TVL.
Change-Id: Id18031d355afc34079c5e6e49dc6943e61809a8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5298
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
cgit has its own module now
Change-Id: I9b4cc322374517b8bd3db43345831e2bf43c4bb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5295
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
The ancient `//web/cgit-taz` path stems from the time I had
code.tazj.in serving my initial version of the depot.
I've been meaning to clean this up for forever, so here we go.
Note that this leaves the git-serving module in a strange state where
it only deals with josh. I'll rename it accordingly.
Change-Id: I47ed1e9d90958299b5440a18a1b9075274754e33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5294
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
* //nix/buildLisp: re-enable CCL, as the crash has been fixed upstream,
although it is unclear what exactly caused / fixed it.
* //ops/whitby: the kitty build broke upstream, so we can't install the
terminfo on whitby for a bit.
Change-Id: I5710acbe837fbc936e334b2e81f9cf00ed6ae280
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5274
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
I am trying to publish this to crates.io, and `mq` is already taken up
by what seems to be a dead project.
Change-Id: I14d1f5f31f167fde954d9c1e39fc9fec5f4c3d10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5234
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Same trick as in the last commit, trying to hit the right revision.
Change-Id: I0af9e88b4d2fd8239a7819dbe0da13f26cca8d6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5233
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
... and update some outdated stuff in the README while we're at it.
Change-Id: Ib3a12596bd1ba61e91ac6e1d37106b616da3030d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5232
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The previous version had a CVE.
As part of this upgrade, the handling of errors inside of the Nix
crate changed, which we now accommodate.
Change-Id: Iad9a473c1782e0d79919cb5dc3f76316852d8a16
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5226
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
The previous mem::unitialized method was deprecated in favour of this
struct which carries information about the initialisation state
forward to the compiler.
Change-Id: Ib5f5d1ad91f9957b18eebabc1048f8649bc74049
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5225
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This is necessary for the nginx prometheus exporter to work
Change-Id: I2343d6f5d3d6d6772777d5e14426a537aa1c8ef7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5127
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Might be nice to look at rates of requests etc.
Change-Id: I4d12ab0c1a555793e803de4a9614e616951a94e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5125
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
This isn't actually used by anything that would use agenix, but this
seems like a vaguely sensible way of sharing the token with other people
regardless.
Anyone who finds this commit and wants to be added to the telegram
channel where the alerts go, lmk.
Change-Id: I06d6ed2d4bec099cbf68ede8fd00a5e6f4e7bc60
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5124
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Since our blog index is on the index page, this makes slightly more
sense.
Change-Id: I7b8164490c133e23d892abef21275f8bfed50b66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5123
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
This was already happening without the trailing slash, but needs to
happen separately with it.
Fixes: b/172
Change-Id: Ic3423fd7a2eaf76a073badd80965cee953df4ce9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5121
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
This means it will end up in journaldriver.
Change-Id: I66f781085b5dac9946b3b9a2bf30e447863e1213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5122
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
There is no need for this step to be part of the static pipeline (it
should not run if the build fails anyways).
Change-Id: I71400a452d6f8f4708d146b346eaffda5da2f766
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5049
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
This will create `build-chunk-$n.json` files for steps that should run
_before_ duck, and `post-chunk-$n.json` files for steps that should
run after duck.
The post steps are not yet uploaded to Buildkite, but we also don't
have any right now.
Change-Id: I7e1b59cf55a8bf1d97266f6e988aa496959077bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5047
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
Using this instead of a conditional leads to nicer output in the UI,
but has no semantic difference.
Change-Id: I5b368d663f417d256e4792d2d46b84fc50d42d0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5045
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This step is independent of the build result and can be scheduled at
the beginning while pipeline eval is still in progress.
Change-Id: I2ee268e4c333efa654dcb12c0b1562b43231d241
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5044
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Previously we only stored the drvmap, but we will also need the build
chunks to refactor the generation of dynamic post-steps.
Change-Id: I256fffe13af8f8c4521835257f5d87dda323b248
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5043
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
This CI pipeline in Buildkite verifies the external (josh-provided)
view of the depot at //views/kit.
See https://buildkite.com/tvl/tvl-kit
Note that this always triggers a build of HEAD. This is because we
don't know the transformed commit ID, and we currently have no way to
pass a ref through. The pipeline is configured to skip intermediate
builds.
I asked Buildkite for some ideas on how to improve this, lets see.
Change-Id: I6c60fb1ea7606c1c90219ef04fd7bada64661529
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5010
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
This is no longer TVL-specific and should live here with the other
generalised stuff.
Change-Id: I95a1b4c0321f34812162d6fd40568269abf639dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5006
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Removes all TVL-specific values in favour of environment variables
supplied by Buildkite.
This makes it possible to reuse this script outside of TVL.
Change-Id: Ic543bc41e4c81e65ee349ad241c515231e97ab30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5005
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Some companies do not know the 'cl' term. They do know of 'change' and
would maybe not like to introduce one more synonym.
This cl introduce an optional entry 'gerritChangeName' in
besadii.json. The string has to match `^[a-z0-9]+$` for readability.
Change-Id: Id70fcb1e45158869f88bf37669be49b8b8a3b295
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4825
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
The cookie secret in the encrypted file was too long, because the
generation command in the oauth2_proxy docs is also wrong. Should
probably fix that upstream as well.
Also noticed that an extra '2' snuck into the service name and fixed
that.
Change-Id: I9a344a75993ab1f98299a8d45e7f5b2e146b7fc5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4957
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This changes the logic for build pipeline generation to inspect
an (optional) parentTargetMap attribute which contains the derivation
map of a target commit.
Targets that existed in a parent commit with the same drv hash will be
skipped, as they are not considered to have changed.
This does not yet wire up any logic for retrieving the target map from
storage, meaning that at this commit all targets are always built.
The intention is that we will have logic to fetch the target
map (initially from Buildkite artefact storage), which we then pass to
the depot via externalArgs when actually generating the pipeline.
Change-Id: I3373c60aaf4b56b94c6ab64e2e5eef68dea9287c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4946
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Always create a structure that maps all targets to derivations, and
persist it as a JSON file.
This relates to some of the ideas expressed in:
https://docs.google.com/document/d/16A0a5oUxH1VoiSM8hyFyLW0WiUYpNo2e2D6FTW4BlH8/edit
The file is always uploaded to Buildkite as an artifact. This allows
for retrieving it based on the commit ID in a Buildkite GraphQL query.
By default, Buildkite stores artefacts for 6 months. Storage location
can be overridden (with custom retention) through some environment
variables, but for now at TVL the Buildkite-managed storage is fine.
See also: https://buildkite.com/docs/pipelines/artifacts
In the subsequent filtering implementation, when diffing commits
across a time-range that exceeds artefact retention time, we should
simply default to building everything.
Change-Id: I6d808461cd1c1fdd6983ba8c8ef075736d42caa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3662
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Since we now group patchsets inside of Buildkite, the results are no
longer guaranteed to be for the right patchset.
There might be some metadata passed in from Gerrit that would let us
do this with the commit ID instead, but I haven't checked.
Change-Id: I5b74a17697511160fcc89d3dbef23517d974dc6f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4890
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
I'm not sure where the previous list originated, but it was missing
some officially documented statuses. However, the API definitely
returns statuses that are documented to only appear in other types, so
this commit simply maps ALL statuses that Buildkite has documented for
any type.
Also adds a log statement in case we encounter a brand new, unknown,
undocumented status.
Change-Id: Iff003a3bd2608702019ae0f4137958435ad0856f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4888
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
... and remove a spammy log statement.
This changed in besadii a while ago and lead to the behaviour of
failing silently, instead of failing with an error saying "undefined
undefined".
Note that with this change merged the plugin probably still won't work
again, but it gets us a step closer to the real error.
Change-Id: I3db25d246f4b1c634d316cd92574e27fb220d769
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4887
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Currently Gerrit is configured to copy forward the scores of the
'Verified' label if the tree of the commit does not change (e.g. only
author information or commit message is modified).
Besadii still triggers builds for these patchsets though. With this
change it will inspect the (previously ignored) "kind" of the patchset
and skip patchsets with the same tree as their predecessor.
See Gerrit docs for the semantics of "kind":
https://gerrit-review.googlesource.com/Documentation/json.html#patchSet
Note that an argument can be made that we should do the exact opposite
- stop carrying over 'Verified' at all and always build all patchsets.
I think this depends on whether we intend to use commit metadata in CI
runs at all. Adding a few people to the review for opinions.
Change-Id: I48a96a1ad1e07d92330d84e5cfdc820a39395297
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4867
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: asmundo <asmundo@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Adds more things I keep using via nix-shell, as well as the
deploy-whitby script (which is independent of a particular depot
checkout).
Change-Id: I36f87de7645768a05268c90ba9b3ab833bacca05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4881
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
nvd only shows us changed versions of packages, as well as
added/removed packages, which means that for the majority of depot
packages nothing will be displayed
however, the current output of nix-diff is not usable anyways, so
having something that can be looked at is better than nothing
Change-Id: Iefbd8139c7ccf5c88ed1209897abdb2ae9302e91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4868
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
The comment posted to the Gerrit change do not need to contain the CL
number as it is given by the context of the Gerrit UI.
Change-Id: I172645e7f4d82e2fbebe179578babd42ea29737f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4826
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
When `findSystem` attempts to evaluate `system.config.networking.hostName`,
diogenes (because I've refactored its definition) causes the following error:
> You're trying to declare a value of type `string'
> rather than an attribute-set for the option
> `system'!
Change-Id: Ib23cb9aa9cadc1f71ad3369c903e587762d12cc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4830
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Adds a feature to emergency-stop deploys by simply running `touch
/var/lib/auto-deploy/stop`.
This can be useful in some situations, especially if there is a
process that reconciles service state (so that e.g. stopping the
unit's timer would be undone).
Change-Id: I233dfac365a578bfa4110eb605b50be079974ba4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4827
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
The priority of binary caches is decided by the remotes in Nix (???),
and by default nix-serve (which is *very* slow) has a lower priority
than cache.nixos.org (which means that it will be preferred over the
faster cache for paths that exist on both).
To avoid this, override the hardcoded (????) priority by serving the
nix-cache-info response directly from nginx instead.
Change-Id: I15a2d6618386d16edaf69f1c9257a36bd72132d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4823
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
This reverts commit 5e036ed9fc.
Reason for revert: This introduced a logic error since the remaining
step runs at the wrong point in the pipeline. Temporarily reverting to
having duplicated waits in order to clean up later.
Change-Id: Ifa6ece50dd22924f02efd7b790a5863ca1189af7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4841
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
The intent is to configure oauth2_proxy pointing at Keycloak to enable
usage with nginx auth_request directives.
I want to expose this as a function from within the module in which
nginx server configuration blocks can be wrapped, but the function for
that is currently a placeholder.
Change-Id: I5ed7deb9bf1c62818f516e68c33e8c5b632fccfe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4767
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
It looks like we won't need this for oauth2_proxy when combined with
nginx auth_request setups.
Change-Id: I2294aee6226b4f64a27bf6592c2d18092d0268cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4766
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Turns the anchor derivation into something that can actually be
built (a call creating a propagated build inputs file), and builds it.
This should fix the anchoring logic we have on canon.
Change-Id: If6a7662b82e2e396388980f65e332cf67a45b46e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4763
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
Without some kind of physical organisation it's a little difficult to
understand whether things are going "in" (supplying users to Keycloak)
or "out" (getting auth/user info from Keycloak).
Change-Id: I516501081e3448c81c710fcbc79cc68ad2a80f3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4762
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This now happens in //nix/buildkite instead
Change-Id: Ie9e239ee4f28ac34aa4d3279dac55d70a2cb9d86
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4764
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This client definition was previously nonsense. What happened is that
I accidentally imported the client as an OIDC client, which Keycloak
accepted because apparently those are the same entities on the API
level, and that ended up getting mangled into some broken hybrid shape
by Terraform.
This sets up the Buildkite provider again but with the correct
SAML configuration this time.
Change-Id: Id7ba318984d2fcc9e2ca91ed45ccbfd227278bbe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4731
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <mail@tazj.in>
Produces more useful output and also makes for a good target for the
upcoming extraSteps logic.
Change-Id: Ifd389d433d9e27f97940a48999f4fba35646e37a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4727
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
Extracts the logic for generating our Buildkite pipeline (which has
been copy&pasted and slightly modified in some places outside of
depot) into a generic //nix/buildkite library.
This should cause no change in functionality.
Change-Id: Iad3201713945de41279b39e4f1b847f697c179f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4726
Autosubmit: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
These records were previously configured manually in the GleSYS web UI
during our DNS outage (b/155).
Note that I could not find a way to `terraform import` these records
and have instead recreated the set and then cleaned up in the UI.
Change-Id: If7de9a7e6dad20953ba8b610589a62dce400e87b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4716
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
These records were previously configured manually in the GleSYS web UI
during our DNS outage (b/155).
Note that I could not find a way to `terraform import` these records
and have instead recreated the set and then cleaned up in the UI.
Since we often point things at whitby, I have extracted variables for
its IPs in this change.
Change-Id: I09fda94d3734e8aaa278fa858e160d046740da1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4714
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
These records were previously configured manually in the GleSYS web UI
during our DNS outage (b/155).
Note that I could not find a way to `terraform import` these records
and have instead recreated the set and then cleaned up in the UI.
Change-Id: I2b7e0ed0931f50e7fa49c1f6e3400dfe958def04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4713
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Adds the secrets and some instructions for deploying the GleSYS
Terraform infrastructure.
Change-Id: I1a10f9cee7648d406b3d27ef45fc74b6923cbc30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4712
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This was previously configured in the UI.
Change-Id: I68361b1489093b76736adab2e38ed7b474b10881
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4711
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This was previously configured in the UI.
Change-Id: Ib15b8ecca96d7814dc85d62199865b22bdb63f95
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4710
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This should never sit around locally the way it does now.
Change-Id: Icfbdaf1949d6d948a796a0759282ea6144af3621
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4709
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This file can be sourced (somehow, depending on the user) while
working with //ops/keycloak to get the relevant secrets.
Change-Id: Ibb3051c4b019f64824964475451c1c3996db6421
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4708
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Grafana was still pointing at the (now non-existent) CAS setup. This
changes the endpoints to use Keycloak instead and updates the client
secret.
Change-Id: Ib25d38330aba2ef6d894e8c33d86852c884ab5be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4706
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Generalize out a reusable mkSecrets function from the
secrets-tree-building that's happening in //ops/secrets, so the same
thing can happen in other places in the depot (I want to use it for my
personal infrastructure).
Change-Id: I059295c8c257d78ad7fa0802859f57c2c105f29b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4679
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: zseri <zseri.devel@ytrizja.de>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Building nix derivations needs tar (provided by gnutar) and gzip on the
PATH in order to extract .tar.gz archives.
Change-Id: Ia2df7a3a770cfd342dfede58ad34e04805fbd1f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4685
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
This will be useful for things like panettone, pending a NixOS module
for oauth2-proxy (the upstream one is too complicated and doesn't
support what we need).
Change-Id: I4ca193e10a94a29b1fb9003e945896ff8eb61116
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4662
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: tazjin <mail@tazj.in>
Verified emails are required for some things, like e.g. oauth2_proxy
Change-Id: Ifb124be40d6d2863cd1b7ed5fbdfcf4827e8808c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4661
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: Profpatsch <mail@profpatsch.de>
This is still missing most of the client configuration etc., in part
due to bugs in the provider which are preventing resource imports.
Change-Id: Ic224ffc001f8e1fe6dcd47b7d002580fdf7b0774
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4628
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Note that the login.tvl.fyi WWW configuration is still kind of hanging
around until we've settled where Keycloak lives.
Change-Id: Iaca4e394a7371cafa3716ca66ef09c4eca5b1520
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4626
Autosubmit: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Gerrit has OAuth2 and email related secrets which now live in agenix
instead of a random file on disk.
Change-Id: I6220fbb7a2e2ec0102a900b4bcf6150b8b4d32ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4612
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Trialing this as an alternative to CAS that is a little easier to
configure and can help us delegate authentication to other OIDC
services.
Change-Id: Iad63724d349334910af8fed0b148e4ba428f796b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4608
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
This provides the right Terraform provider with a wrapper in $PATH.
Change-Id: Idcb4fa89dff0161e8a73addfce81959e825c331e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4562
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Unclear if this reformat is caused by the channel update or if this file
was ignored previously.
Change-Id: I3498ab181c7fff1b132419783e33a96f7bebfe42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4520
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
If available, provide a link back to Gerrit on the overview page of a
build.
Uses the default style (i.e. style unset), which makes it
non-intrusive visually.
Change-Id: I4271d589d548015b75762fd0584f3958bfcc53e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4442
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
The code in //users/wpcarro/tools/monzo_ynab/ynab/client.go was not
valid Go and has been commented out.
Change-Id: Icb4003607f30294dcbf60132eb7722702c7f0d84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4400
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: Profpatsch <mail@profpatsch.de>
got into some kind of race with different patchsets of this CL
somehow, idk
Change-Id: I3dcdb708f141829b866fbd786483710b43ea9824
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4481
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
If we set this for canon, then stuff starts to fail in non-obvious ways.
Change-Id: I3bf38e29151c6066aaf4eba68ae387272d8a82c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4463
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
apparently this chomps away at things inside of fragment strings
Change-Id: Ie60d52d101dc4281b3a62c228af076791e1c7928
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4462
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This makes it possible to annotate builds with a link back to Gerrit.
Change-Id: If351785d3b631b96753d41f417ca94bc7a95ac54
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4441
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
The branch key for buildkite builds isn't actually used to fetch if a
commit is given - instead, it's just a visual grouping of multiple
builds. This means we can just make the branch key cl/<cl number>, which
is the convention we already use to refer to CLs and gets us a nice
visual grouping of builds of successive patchsets of the same CL number,
even though the ref we're providing isn't a real ref.
Change-Id: Iaa9111297a88f965fda94cd8266240106f58a100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4347
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Autosubmit: grfn <grfn@gws.fyi>
I'd like to be able to run extra CI steps that include running docker
containers (to integration test things like webapps that connect to a
database). To do this the buildkite agents themselves need permission to
do docker things.
Change-Id: I3c9a488708f0e12a508754ac41f04148ca7aedac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4408
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
diogenes "passed" CI because the file was named configuration.nix
(vestage from the NixOS default /etc/nixos/configuration). This CL fixes
some issues I encountered after running depot/bin/rebuild-system.
TL;DR:
- rename configuration.nix -> default.nix to trigger CI
- add diogenes to my systems
- add public SSH key
Change-Id: I24197b8936c201267db6f71f00099dce590eac1d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4388
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
Autosubmit: wpcarro <wpcarro@gmail.com>
Submitted via IRC, instead of the usual Google Groups
due to email issues.
Change-Id: I71a2bdfd10b02370df61bbba4dabc2f45b6c1009
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4384
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <mail@tazj.in>
For modules that are gated behind a mkEnableOption, it's reasonable to
just provide them to all Depot-built nixos systems without requiring
people to explicitly import them. This defines a special module called
`default-imports.nix` which imports these modules (currently just
tvl-cache.nix and automatic-gc.nix, as I'm being rather conservative
adding things here to avoid breaking anyone's system), then provides
that module as one of the `modules` passed at the top-level
nixos/eval-config invocation.
Change-Id: I3be299ab10ae4c451ef11c514edb3c89318a2278
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4345
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Add a shared nixos module for configuring whitby as a binary nix cache,
and refactor tverskoy to use this module.
This is enabled via an option to pave the way for including it as an
import in all depot-generated nixos configs at some point in the future.
Change-Id: I6dcc0e8eb48b1ac34457666dceebeedd5da6c526
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4344
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: grfn <grfn@gws.fyi>
Don't notify reviewers ever on CI status changes, and only notify the
owner if the build fails.
Change-Id: If2cf63581b49e3de77181024ce8a4213031f4bd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4337
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Autosubmit: grfn <grfn@gws.fyi>
This fix was recommended by Buildkite and is explained in the comment.
Change-Id: I3f1c1c07cba0b417857d69c021c8af4750d645c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4334
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
The number of jobs in the depot pipeline is reaching the limits of the
Buildkite backend's ability for a single pipeline upload. Based on a
conversation with their support my understanding is that this has to
do with internal locking mechanisms at Buildkite.
To work around this, we can instead chunk the pipeline into several
smaller chunks that are uploaded serially.
This commit introduces logic to chunk the pipeline accordingly. The
chunk size chosen is 256 for now (a multiple of our number of agents,
which is useful if we can get builds from the first chunk to start
before the next ones are uploaded).
Note that this chunk size is significantly below even the current
number of targets (~460 as of this commit), but choosing a lower chunk
size might alleviate problems we've been seeing with timeouts during
pipeline uploads.
Change-Id: I77030aaf8b874c330218b78c77d15216e13b9af7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4332
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: tazjin <mail@tazj.in>
https://cl.tvl.fyi/c/depot/+/4264 did move merging config with secrets
into ExecStart=, which is tracked in an RFE upstream:
https://github.com/systemd/systemd/issues/19604#issuecomment-989279884
We didn't link to this so far, neither in the commit message, nor in a
comment.
Let's add a comment, so people know when we can undo this.
Change-Id: I7bed370b671093bb876592b4dccd562f1c256cd2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4326
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
We can gcroot the derivation files and drop this step, but have
elected not to do so for the moment, see cl/3436.
Change-Id: I993a1f3921e9f21e18fa260e76d3dd15ffa556bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4327
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <mail@tazj.in>
By default besadii will set the `Verified` label in Gerrit. This adds
a config option to set a different label instead if desired.
Co-authored-by: Vincent Ambo <mail@tazj.in>
Change-Id: I254159e46994e01182987ed5e5e26e27c57f46ce
Currently in NixOS configuration using agenix secrets there is no
build time validation of secret paths - things fail at runtime (system
activation).
To prevent that, this CL makes the secrets part of the tree based on
the same configuration file used by agenix itself.
This guards against:
* agenix secrets.nix definition for a non-existent file
* age.secrets value in a NixOS config for a non-existent secret
Change-Id: I5b191dcbd5b2522566ff7c38f8a988bbf7679364
... okay, this is like the 5th error related to something with this
and file paths. Need to write some validation logic.
Change-Id: I4314818aa1bc25b8cf7bd3593850d3836ccb867c
Git only allows binary names prefixed with `git-credential-` if the
path to the helper is not absolute.
Why? Who knows.
Change-Id: I216b2a621f62a73f05e21def7ec8016b29ede892
Currently this functionality is provided by a shell script stored in
/etc/secrets (which has the password value hardcoded).
This needs to happen in a separate commit from the one that changes
the pipeline to avoid breaking it (it needs to be deployed first).
Change-Id: I680754c828ccefbacfcf0d5c813a4bc19493ba4c
We already checked this in, but this commit adds the configuration for
making use of it.
There are two copies of besadii's JSON configuration with different
permissions.
Note that the buildkite-graphql-token path needs to be updated in
static-pipeline.yml, but this needs to happen in a separate commit
after deploy because the pipeline will break otherwise.
Change-Id: I6fab4bf1a2e679df7cf76521e2b53bd9dadbac62
... this option really is a pitfall! The list of programs is now the
same as in the upstream module, plus curl and jq.
Change-Id: I29edae4b2400a2724f62df9efa1dc184a8b0af5f
The DynamicUser + Group configuration does not work as planned, thus
the systemd LoadCredentials feature is used instead which makes the
file (which itself is only readable by root) available in a
memory-backed location only readable by the service.
The secret is only available to `ExecStart` commands, so units using
this feature can not be used with pre/post units and the like if those
commands need secrets.
To accommodate this, the merge of configuration files has been moved
into the service launch script, which is now the ExecStart= process.
For details take a look at https://www.freedesktop.org/software/systemd/man/systemd.exec.html#LoadCredential=ID:PATH
Change-Id: I693fe5677cc0d63c7aa485c2c7472457c5262166
It turns out the lib.mkAfter call doesn't behave as expected -
only *some* of the packages that are defaulted end up in the $PATH.
I suspect this is actually something else, e.g. these packages are
always added for some reason or another, and the option is completely
overridden every time.
Change-Id: I854c7198520d82b00e6338ed0fe653836226dc6d
Turns out that the type of this option is not concatenative and it
replaces the packages needed to run Buildkite if set.
Change-Id: I9f52572bc165bccdd8c6518cfdf7b8967f7a50d0
The irccat module uses DynamicUser, so to grant permission to it a new
group has been added for irccat.
I have some vague memory of DynamicUser + Group not behaving as one
would expect, but we'll see what happens.
Change-Id: Iab9f6a3f1a53c4133b635458ce173250cc9a3fac
This step would get inserted at the wrong point in the build pipeline
otherwise, causing a dependency cycle and causing the pipeline to fail.
Change-Id: I534568eec77f74ae6c47276820f8a9e99493a3ea
This simplifies the fallback logic used in case of Nix evaluation
failure and makes it so that the evaluation step itself is the one
that is marked as failed in Buildkite.
This is possible because the pipeline upload command will insert new
steps at the point where it runs in the pipeline, and not later.
Change-Id: I870534c004ebc457a1602623c4e5f9c0c68e28fc
Adds a systemd EnvironmentFile secret that contains the Gerrit
username & password for gerrit-queue.
Change-Id: I25acf87764c26774045138402b8a417b6813ee8f
This is not yet including the secret configuration for gerrit-queue,
and just expects the secret (gerrit username & password) to be
available in /etc/secrets.
Change-Id: Ia465ef7f3f521c70d606d7fdeba9aa83c7e1b98b
This is required for a simplification of the build pipeline (following
CL) and needs to be in a separate commit as it can not be done
atomically (merging the other commit to deploy it would immediately
break pipelines otherwise).
Change-Id: I5d8ec8f3238f79b5518d799486bf98d1d9516c43
Sets up the key set and adds an initial secret (besadii config with
tokens) to be deployed to whitby.
Change-Id: Ic07fd5e66b9e7a533013e04c35e052c2aa11f77d
Gerrit wraps RFC5322 emails in another layer of quotes when passing
them as flags, and this needs to be unquoted.
Otherwise hook invocations fail with cryptic errors.
Change-Id: Ieeb74c662873d99a4154f8cbc92da77b039cb88e
Ensure that besadii sees $0 as the correct command name, since that is
the sole mechanism by which its functionality is switched around.
There was a lingering commit that introduced this bug and hadn't been
deployed in a couple of days. Maybe time to tighten deploy cycles soon
...
Change-Id: Ie4284c0f6e5e06d71a71a3702ec7e092260e0ce5
step up in the pipeline
