Apparently failure is not hip anymore, and crate updates are forcing
the use of anyhow now. Whatever.
The functionality basically stays the same, maybe error messages will
look a little bit different now.
Change-Id: I173d644688785339c16161ddeec47a534123710f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5307
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
As suggested by sterni, this makes the self-redirect of a machine to
its configuration a generic module working by convention.
In the process of moving this two small fixes have been applied:
* redirect is only applied if the URI is `/`, this is required for
ACME to work
* addSSL = true is added, otherwise we have a certificate but no TLS
listener
Change-Id: Icaef041ff681253a61e36926417bdb2844e3f93d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5313
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This makes the journaldriver configuration machine-independent.
The secret is loaded from agenix instead of being persisted on disk.
Change-Id: I592ae7f5726fcb7f37a406f69dcf5ac498eeb1b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5302
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
With this change, entering just "whitby.tvl.fyi" or "sanduny.tvl.su"
in a browser will redirect users to their machine configurations.
Change-Id: Ibf076a469bcce073e1b1970aa568d6fe16a5c75a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5304
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
This needs to be present on all machines that run ACME stuff.
I've switched the address for a .su one because I have a catchall for
these.
Change-Id: I7af8e1f1cb2fcfbcba4b7d1930ed0edef0106d72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5306
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This changes the structure of secrets.nix a bit to split between
secrets for whitby, and secrets for all TVL machines.
Change-Id: I791f0ce42a16b33051e24a7a6c5b153761ed9eb3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5300
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This will be an additional web host / fallback git server for whitby
incidents.
Change-Id: Icd6f7ce574ffd520b5783a50ff317feed7b71fc6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5297
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Rather than defining all system users inline on whitby, move them into
a module that can be imported on multiple machines.
Configuration for terminfos that we've added follows along.
Note that while doing this I've disabled logins for riking and isomer
since they are currently inactive in TVL.
Change-Id: Id18031d355afc34079c5e6e49dc6943e61809a8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5298
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
cgit has its own module now
Change-Id: I9b4cc322374517b8bd3db43345831e2bf43c4bb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5295
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
The ancient `//web/cgit-taz` path stems from the time I had
code.tazj.in serving my initial version of the depot.
I've been meaning to clean this up for forever, so here we go.
Note that this leaves the git-serving module in a strange state where
it only deals with josh. I'll rename it accordingly.
Change-Id: I47ed1e9d90958299b5440a18a1b9075274754e33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5294
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
* //nix/buildLisp: re-enable CCL, as the crash has been fixed upstream,
although it is unclear what exactly caused / fixed it.
* //ops/whitby: the kitty build broke upstream, so we can't install the
terminfo on whitby for a bit.
Change-Id: I5710acbe837fbc936e334b2e81f9cf00ed6ae280
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5274
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
I am trying to publish this to crates.io, and `mq` is already taken up
by what seems to be a dead project.
Change-Id: I14d1f5f31f167fde954d9c1e39fc9fec5f4c3d10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5234
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Same trick as in the last commit, trying to hit the right revision.
Change-Id: I0af9e88b4d2fd8239a7819dbe0da13f26cca8d6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5233
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
... and update some outdated stuff in the README while we're at it.
Change-Id: Ib3a12596bd1ba61e91ac6e1d37106b616da3030d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5232
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The previous version had a CVE.
As part of this upgrade, the handling of errors inside of the Nix
crate changed, which we now accommodate.
Change-Id: Iad9a473c1782e0d79919cb5dc3f76316852d8a16
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5226
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
The previous mem::unitialized method was deprecated in favour of this
struct which carries information about the initialisation state
forward to the compiler.
Change-Id: Ib5f5d1ad91f9957b18eebabc1048f8649bc74049
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5225
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This is necessary for the nginx prometheus exporter to work
Change-Id: I2343d6f5d3d6d6772777d5e14426a537aa1c8ef7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5127
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Might be nice to look at rates of requests etc.
Change-Id: I4d12ab0c1a555793e803de4a9614e616951a94e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5125
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
This isn't actually used by anything that would use agenix, but this
seems like a vaguely sensible way of sharing the token with other people
regardless.
Anyone who finds this commit and wants to be added to the telegram
channel where the alerts go, lmk.
Change-Id: I06d6ed2d4bec099cbf68ede8fd00a5e6f4e7bc60
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5124
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Since our blog index is on the index page, this makes slightly more
sense.
Change-Id: I7b8164490c133e23d892abef21275f8bfed50b66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5123
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
This was already happening without the trailing slash, but needs to
happen separately with it.
Fixes: b/172
Change-Id: Ic3423fd7a2eaf76a073badd80965cee953df4ce9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5121
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
This means it will end up in journaldriver.
Change-Id: I66f781085b5dac9946b3b9a2bf30e447863e1213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5122
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
There is no need for this step to be part of the static pipeline (it
should not run if the build fails anyways).
Change-Id: I71400a452d6f8f4708d146b346eaffda5da2f766
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5049
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
This will create `build-chunk-$n.json` files for steps that should run
_before_ duck, and `post-chunk-$n.json` files for steps that should
run after duck.
The post steps are not yet uploaded to Buildkite, but we also don't
have any right now.
Change-Id: I7e1b59cf55a8bf1d97266f6e988aa496959077bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5047
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
Using this instead of a conditional leads to nicer output in the UI,
but has no semantic difference.
Change-Id: I5b368d663f417d256e4792d2d46b84fc50d42d0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5045
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This step is independent of the build result and can be scheduled at
the beginning while pipeline eval is still in progress.
Change-Id: I2ee268e4c333efa654dcb12c0b1562b43231d241
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5044
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Previously we only stored the drvmap, but we will also need the build
chunks to refactor the generation of dynamic post-steps.
Change-Id: I256fffe13af8f8c4521835257f5d87dda323b248
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5043
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
This CI pipeline in Buildkite verifies the external (josh-provided)
view of the depot at //views/kit.
See https://buildkite.com/tvl/tvl-kit
Note that this always triggers a build of HEAD. This is because we
don't know the transformed commit ID, and we currently have no way to
pass a ref through. The pipeline is configured to skip intermediate
builds.
I asked Buildkite for some ideas on how to improve this, lets see.
Change-Id: I6c60fb1ea7606c1c90219ef04fd7bada64661529
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5010
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
This is no longer TVL-specific and should live here with the other
generalised stuff.
Change-Id: I95a1b4c0321f34812162d6fd40568269abf639dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5006
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Removes all TVL-specific values in favour of environment variables
supplied by Buildkite.
This makes it possible to reuse this script outside of TVL.
Change-Id: Ic543bc41e4c81e65ee349ad241c515231e97ab30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5005
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Some companies do not know the 'cl' term. They do know of 'change' and
would maybe not like to introduce one more synonym.
This cl introduce an optional entry 'gerritChangeName' in
besadii.json. The string has to match `^[a-z0-9]+$` for readability.
Change-Id: Id70fcb1e45158869f88bf37669be49b8b8a3b295
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4825
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
The cookie secret in the encrypted file was too long, because the
generation command in the oauth2_proxy docs is also wrong. Should
probably fix that upstream as well.
Also noticed that an extra '2' snuck into the service name and fixed
that.
Change-Id: I9a344a75993ab1f98299a8d45e7f5b2e146b7fc5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4957
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This changes the logic for build pipeline generation to inspect
an (optional) parentTargetMap attribute which contains the derivation
map of a target commit.
Targets that existed in a parent commit with the same drv hash will be
skipped, as they are not considered to have changed.
This does not yet wire up any logic for retrieving the target map from
storage, meaning that at this commit all targets are always built.
The intention is that we will have logic to fetch the target
map (initially from Buildkite artefact storage), which we then pass to
the depot via externalArgs when actually generating the pipeline.
Change-Id: I3373c60aaf4b56b94c6ab64e2e5eef68dea9287c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4946
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Always create a structure that maps all targets to derivations, and
persist it as a JSON file.
This relates to some of the ideas expressed in:
https://docs.google.com/document/d/16A0a5oUxH1VoiSM8hyFyLW0WiUYpNo2e2D6FTW4BlH8/edit
The file is always uploaded to Buildkite as an artifact. This allows
for retrieving it based on the commit ID in a Buildkite GraphQL query.
By default, Buildkite stores artefacts for 6 months. Storage location
can be overridden (with custom retention) through some environment
variables, but for now at TVL the Buildkite-managed storage is fine.
See also: https://buildkite.com/docs/pipelines/artifacts
In the subsequent filtering implementation, when diffing commits
across a time-range that exceeds artefact retention time, we should
simply default to building everything.
Change-Id: I6d808461cd1c1fdd6983ba8c8ef075736d42caa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3662
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
step up in the pipeline
