With this change, entering just "whitby.tvl.fyi" or "sanduny.tvl.su"
in a browser will redirect users to their machine configurations.
Change-Id: Ibf076a469bcce073e1b1970aa568d6fe16a5c75a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5304
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
This needs to be present on all machines that run ACME stuff.
I've switched the address for a .su one because I have a catchall for
these.
Change-Id: I7af8e1f1cb2fcfbcba4b7d1930ed0edef0106d72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5306
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This changes the structure of secrets.nix a bit to split between
secrets for whitby, and secrets for all TVL machines.
Change-Id: I791f0ce42a16b33051e24a7a6c5b153761ed9eb3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5300
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This will be an additional web host / fallback git server for whitby
incidents.
Change-Id: Icd6f7ce574ffd520b5783a50ff317feed7b71fc6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5297
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Rather than defining all system users inline on whitby, move them into
a module that can be imported on multiple machines.
Configuration for terminfos that we've added follows along.
Note that while doing this I've disabled logins for riking and isomer
since they are currently inactive in TVL.
Change-Id: Id18031d355afc34079c5e6e49dc6943e61809a8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5298
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
cgit has its own module now
Change-Id: I9b4cc322374517b8bd3db43345831e2bf43c4bb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5295
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
The ancient `//web/cgit-taz` path stems from the time I had
code.tazj.in serving my initial version of the depot.
I've been meaning to clean this up for forever, so here we go.
Note that this leaves the git-serving module in a strange state where
it only deals with josh. I'll rename it accordingly.
Change-Id: I47ed1e9d90958299b5440a18a1b9075274754e33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5294
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
* //nix/buildLisp: re-enable CCL, as the crash has been fixed upstream,
although it is unclear what exactly caused / fixed it.
* //ops/whitby: the kitty build broke upstream, so we can't install the
terminfo on whitby for a bit.
Change-Id: I5710acbe837fbc936e334b2e81f9cf00ed6ae280
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5274
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
I am trying to publish this to crates.io, and `mq` is already taken up
by what seems to be a dead project.
Change-Id: I14d1f5f31f167fde954d9c1e39fc9fec5f4c3d10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5234
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Same trick as in the last commit, trying to hit the right revision.
Change-Id: I0af9e88b4d2fd8239a7819dbe0da13f26cca8d6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5233
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
... and update some outdated stuff in the README while we're at it.
Change-Id: Ib3a12596bd1ba61e91ac6e1d37106b616da3030d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5232
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The previous version had a CVE.
As part of this upgrade, the handling of errors inside of the Nix
crate changed, which we now accommodate.
Change-Id: Iad9a473c1782e0d79919cb5dc3f76316852d8a16
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5226
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
The previous mem::unitialized method was deprecated in favour of this
struct which carries information about the initialisation state
forward to the compiler.
Change-Id: Ib5f5d1ad91f9957b18eebabc1048f8649bc74049
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5225
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This is necessary for the nginx prometheus exporter to work
Change-Id: I2343d6f5d3d6d6772777d5e14426a537aa1c8ef7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5127
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Might be nice to look at rates of requests etc.
Change-Id: I4d12ab0c1a555793e803de4a9614e616951a94e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5125
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
This isn't actually used by anything that would use agenix, but this
seems like a vaguely sensible way of sharing the token with other people
regardless.
Anyone who finds this commit and wants to be added to the telegram
channel where the alerts go, lmk.
Change-Id: I06d6ed2d4bec099cbf68ede8fd00a5e6f4e7bc60
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5124
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Since our blog index is on the index page, this makes slightly more
sense.
Change-Id: I7b8164490c133e23d892abef21275f8bfed50b66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5123
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
This was already happening without the trailing slash, but needs to
happen separately with it.
Fixes: b/172
Change-Id: Ic3423fd7a2eaf76a073badd80965cee953df4ce9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5121
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
This means it will end up in journaldriver.
Change-Id: I66f781085b5dac9946b3b9a2bf30e447863e1213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5122
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
There is no need for this step to be part of the static pipeline (it
should not run if the build fails anyways).
Change-Id: I71400a452d6f8f4708d146b346eaffda5da2f766
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5049
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
This will create `build-chunk-$n.json` files for steps that should run
_before_ duck, and `post-chunk-$n.json` files for steps that should
run after duck.
The post steps are not yet uploaded to Buildkite, but we also don't
have any right now.
Change-Id: I7e1b59cf55a8bf1d97266f6e988aa496959077bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5047
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
Using this instead of a conditional leads to nicer output in the UI,
but has no semantic difference.
Change-Id: I5b368d663f417d256e4792d2d46b84fc50d42d0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5045
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This step is independent of the build result and can be scheduled at
the beginning while pipeline eval is still in progress.
Change-Id: I2ee268e4c333efa654dcb12c0b1562b43231d241
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5044
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Previously we only stored the drvmap, but we will also need the build
chunks to refactor the generation of dynamic post-steps.
Change-Id: I256fffe13af8f8c4521835257f5d87dda323b248
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5043
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
This CI pipeline in Buildkite verifies the external (josh-provided)
view of the depot at //views/kit.
See https://buildkite.com/tvl/tvl-kit
Note that this always triggers a build of HEAD. This is because we
don't know the transformed commit ID, and we currently have no way to
pass a ref through. The pipeline is configured to skip intermediate
builds.
I asked Buildkite for some ideas on how to improve this, lets see.
Change-Id: I6c60fb1ea7606c1c90219ef04fd7bada64661529
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5010
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
This is no longer TVL-specific and should live here with the other
generalised stuff.
Change-Id: I95a1b4c0321f34812162d6fd40568269abf639dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5006
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Removes all TVL-specific values in favour of environment variables
supplied by Buildkite.
This makes it possible to reuse this script outside of TVL.
Change-Id: Ic543bc41e4c81e65ee349ad241c515231e97ab30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5005
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Some companies do not know the 'cl' term. They do know of 'change' and
would maybe not like to introduce one more synonym.
This cl introduce an optional entry 'gerritChangeName' in
besadii.json. The string has to match `^[a-z0-9]+$` for readability.
Change-Id: Id70fcb1e45158869f88bf37669be49b8b8a3b295
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4825
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
The cookie secret in the encrypted file was too long, because the
generation command in the oauth2_proxy docs is also wrong. Should
probably fix that upstream as well.
Also noticed that an extra '2' snuck into the service name and fixed
that.
Change-Id: I9a344a75993ab1f98299a8d45e7f5b2e146b7fc5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4957
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This changes the logic for build pipeline generation to inspect
an (optional) parentTargetMap attribute which contains the derivation
map of a target commit.
Targets that existed in a parent commit with the same drv hash will be
skipped, as they are not considered to have changed.
This does not yet wire up any logic for retrieving the target map from
storage, meaning that at this commit all targets are always built.
The intention is that we will have logic to fetch the target
map (initially from Buildkite artefact storage), which we then pass to
the depot via externalArgs when actually generating the pipeline.
Change-Id: I3373c60aaf4b56b94c6ab64e2e5eef68dea9287c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4946
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Always create a structure that maps all targets to derivations, and
persist it as a JSON file.
This relates to some of the ideas expressed in:
https://docs.google.com/document/d/16A0a5oUxH1VoiSM8hyFyLW0WiUYpNo2e2D6FTW4BlH8/edit
The file is always uploaded to Buildkite as an artifact. This allows
for retrieving it based on the commit ID in a Buildkite GraphQL query.
By default, Buildkite stores artefacts for 6 months. Storage location
can be overridden (with custom retention) through some environment
variables, but for now at TVL the Buildkite-managed storage is fine.
See also: https://buildkite.com/docs/pipelines/artifacts
In the subsequent filtering implementation, when diffing commits
across a time-range that exceeds artefact retention time, we should
simply default to building everything.
Change-Id: I6d808461cd1c1fdd6983ba8c8ef075736d42caa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3662
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Since we now group patchsets inside of Buildkite, the results are no
longer guaranteed to be for the right patchset.
There might be some metadata passed in from Gerrit that would let us
do this with the commit ID instead, but I haven't checked.
Change-Id: I5b74a17697511160fcc89d3dbef23517d974dc6f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4890
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
I'm not sure where the previous list originated, but it was missing
some officially documented statuses. However, the API definitely
returns statuses that are documented to only appear in other types, so
this commit simply maps ALL statuses that Buildkite has documented for
any type.
Also adds a log statement in case we encounter a brand new, unknown,
undocumented status.
Change-Id: Iff003a3bd2608702019ae0f4137958435ad0856f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4888
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
... and remove a spammy log statement.
This changed in besadii a while ago and lead to the behaviour of
failing silently, instead of failing with an error saying "undefined
undefined".
Note that with this change merged the plugin probably still won't work
again, but it gets us a step closer to the real error.
Change-Id: I3db25d246f4b1c634d316cd92574e27fb220d769
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4887
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Currently Gerrit is configured to copy forward the scores of the
'Verified' label if the tree of the commit does not change (e.g. only
author information or commit message is modified).
Besadii still triggers builds for these patchsets though. With this
change it will inspect the (previously ignored) "kind" of the patchset
and skip patchsets with the same tree as their predecessor.
See Gerrit docs for the semantics of "kind":
https://gerrit-review.googlesource.com/Documentation/json.html#patchSet
Note that an argument can be made that we should do the exact opposite
- stop carrying over 'Verified' at all and always build all patchsets.
I think this depends on whether we intend to use commit metadata in CI
runs at all. Adding a few people to the review for opinions.
Change-Id: I48a96a1ad1e07d92330d84e5cfdc820a39395297
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4867
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: asmundo <asmundo@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Adds more things I keep using via nix-shell, as well as the
deploy-whitby script (which is independent of a particular depot
checkout).
Change-Id: I36f87de7645768a05268c90ba9b3ab833bacca05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4881
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
step up in the pipeline
