The previous `condition` abstraction which allowed the full set of
Buildkite conditionals is way too leaky (it lets users to very
Buildkite-specific things which we may not want to allow, and which
are mostly not relevant to a pure evaluation).
Supporting only the `branches` condition (native to Buildkite) should
make it possible to port this to other future CI systems later.
Change-Id: Ib8adcc41db4f1a3566cbeecf13a4228403105c1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5051
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
Adds support for extra build steps that specify a `prompt`. These
steps will be run at the end of the pipeline and will be gated by
human approval.
This mechanism can be used to, for example, stage releases of software
released from depot that are subject to approval.
Change-Id: I97bb505664a2ccf01142286f14e20a370afaa345
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5033
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
TODO(wpcarro): Figure out how this is sourced on NixOS machines.
Change-Id: Iafddab003699f57bbfd09cd29cb9382dadf6a147
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4995
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
It now resides at `/google-briefcase` 🙂
Change-Id: I64640c7c969f08a687cbc6fde9b69b61a9e0016b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4994
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
This was still showing the build status for a manually created ref on
Buildkite which had not seen any updates since mid-2020.
Change-Id: I42786ce2c0ac598f3d6b8ef9379b18294814a5ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5054
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
This extra nixpkgs import has been around for a while, but seems to no
longer be needed (this software is in nixpkgs).
Change-Id: I9932fe01bda97bd8b50ec9192ab185bea69657f9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5050
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
There is no need for this step to be part of the static pipeline (it
should not run if the build fails anyways).
Change-Id: I71400a452d6f8f4708d146b346eaffda5da2f766
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5049
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
This will create `build-chunk-$n.json` files for steps that should run
_before_ duck, and `post-chunk-$n.json` files for steps that should
run after duck.
The post steps are not yet uploaded to Buildkite, but we also don't
have any right now.
Change-Id: I7e1b59cf55a8bf1d97266f6e988aa496959077bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5047
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Autosubmit: tazjin <tazjin@tvl.su>
Using this instead of a conditional leads to nicer output in the UI,
but has no semantic difference.
Change-Id: I5b368d663f417d256e4792d2d46b84fc50d42d0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5045
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This step is independent of the build result and can be scheduled at
the beginning while pipeline eval is still in progress.
Change-Id: I2ee268e4c333efa654dcb12c0b1562b43231d241
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5044
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Previously we only stored the drvmap, but we will also need the build
chunks to refactor the generation of dynamic post-steps.
Change-Id: I256fffe13af8f8c4521835257f5d87dda323b248
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5043
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
This introduces a new feature to our CI system in which targets can
declare extra steps in `meta.ci.extraSteps`.
See the comment in //nix/buildkite/default.nix for an explanation of
how these extra steps are defined.
Change-Id: Icce2890c743286dd37f43024cd390dcebac8cdba
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5008
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
The --fork-point parameter is dependent on reflog data which may get
garbage collected. This can lead to flaky behaviour where it returns
no results and fails if `git gc` recently ran (Buildkite will do this
occasionally).
Though the parameter is semantically closer to what we're looking for,
the output is *usually* the same commit since we're not dealing with
more than one thing to compare.
Change-Id: Idc31e7a26fda2b7113edfa162d9d3811b1a01bf6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5032
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
This CI pipeline in Buildkite verifies the external (josh-provided)
view of the depot at //views/kit.
See https://buildkite.com/tvl/tvl-kit
Note that this always triggers a build of HEAD. This is because we
don't know the transformed commit ID, and we currently have no way to
pass a ref through. The pipeline is configured to skip intermediate
builds.
I asked Buildkite for some ideas on how to improve this, lets see.
Change-Id: I6c60fb1ea7606c1c90219ef04fd7bada64661529
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5010
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Basically any system I have is going to want to be able to flash an
ergodox, so it makes sense to have these always present
Change-Id: I88f556d4484e282e712062a488321bf80baa87ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5028
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Use agenix for the buildkite ssh key and agent token on mugwump, instead
of storing stuff in /etc/secrets
Change-Id: I56951587b949fc0854e56f5c4e33b601e9cd964e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5027
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
somehow I managed to turn this *on*, but had no way of turning it off as
I didn't have a key.
Change-Id: Icd3d3c23595d73da634fc269fc1881f82650c1e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5026
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Apparently meetup will just make the user ID the username sometimes...
idk what's up with this.
Change-Id: I19eaade2fd3394e6ff81c30692bf4a2899985025
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5022
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
The `local` usage we had before would silently swallow non-zero exit
statuses from the substituted git commands.
For some (as of yet unknown) reason, `git merge-base` seems to
sometimes silently fail and produce no output, which broke the rest of
the script logic.
This change will lead to an earlier error, but we don't know if it is
a fix for the actual cause of the git-merge-base problem because the
shape of that problem is unclear.
Change-Id: I4555c8638da450263fa2fd2c274dfdb69f65578e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5012
Tested-by: BuildkiteCI
Reviewed-by: kn <klemens@posteo.de>
The `or` statement is a *single* expression, leading to an incorrect
negation.
Change-Id: I46fa85c22cf145b9204bf26b85b963494b069d06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5013
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
We need 'meta.ci' to be an attribute set for new CI features.
Change-Id: I83d04e2d74e42e49fe739b049ee4ba799f6d5d86
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5007
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
This is no longer TVL-specific and should live here with the other
generalised stuff.
Change-Id: I95a1b4c0321f34812162d6fd40568269abf639dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5006
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Removes all TVL-specific values in favour of environment variables
supplied by Buildkite.
This makes it possible to reuse this script outside of TVL.
Change-Id: Ic543bc41e4c81e65ee349ad241c515231e97ab30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5005
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
allDeps filters the lisp deps according to the given implementation,
processing any implementation conditional attribute sets. These are not
understood by allNative, so we need to pass it the already filtered
input or evaluation would fail.
Change-Id: I9eb2d0c3b2bf70d759d03490cf31fc585283ce7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5001
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Some companies do not know the 'cl' term. They do know of 'change' and
would maybe not like to introduce one more synonym.
This cl introduce an optional entry 'gerritChangeName' in
besadii.json. The string has to match `^[a-z0-9]+$` for readability.
Change-Id: Id70fcb1e45158869f88bf37669be49b8b8a3b295
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4825
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
This creates a josh workspace called `views/kit` which exports all
dependencies needed by external users of the TVL kit.
The intention of this is to have a single clonable "virtual repo"
which provides everything needed to run a system like TVL, but without
the projects that we develop *in* TVL.
Right now this is limited to the basic Nix code (readTree, pipeline
generation) and things required for Gerrit integration.
The workspace maps depot paths into slightly different (simpler)
locations inside the exported view, to avoid a nested, mostly empty
structure.
Note: This is not an "external API" or some such of depot, it is
simply a convenience for exporting some of our tooling.
Change-Id: Ied24aaef117fc2e0c188ec742ae7cd993a5babd3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4991
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This change is required for using our package set in exported josh
workspaces. It has no functional impact on depot itself.
Change-Id: Id48b40f067b5d53a2b7386a0ba1146a72268b923
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4990
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
TL;DR:
- Ensure that export.sh -> import.sh -> export.sh can round-trip without
intermediate tools.
- Remove default values for variables like ${1}, which only seem to complicate
things.
- Add `trap cleanup EXIT` to scripts.
- Remove noisy full-paths from `zip` (note: a more intuitive, less configurable
`zip`, `unzip` should exist).
Change-Id: Ibbd98d1f0156639138175fcb89e9dfbd17fdae5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4993
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
The variable, `evil-visual-update-x-selection`, must've updated to
`evil-visual-update-x-selection-p`. When I moved from ~/briefcase to /depot I
adopted the newer version of nixpkgs.
Change-Id: I27de45c73833325f64b1e176b8793a0d05233331
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4992
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Found this while looking for non-3p uses of builtins.fetchGit. The
hailgun package is in the package set now, and the other thing was
unused (and it's also in the package set).
Change-Id: I6f519fd6014bbed90fc6cee695bed7afbcf55717
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4984
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
The function is depot specific and thus uses tvl-depot-path, so it
belongs in `tvl.el`. Since non-sly-users won't need it, we tie its
definition to loading the sly package.
Change-Id: I8b104deab455d218d3df6a800e35cc104220a841
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4960
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Moves to the derivation-based git fetchers everywhere in third-party.
This might help with forward-compatibility with newer Nix versions,
though that's not our primary concern right now.
Change-Id: I565bb72585b8639893e9ea3a9e233338aede63a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3903
Tested-by: BuildkiteCI
Reviewed-by: zseri <zseri.devel@ytrizja.de>
step up in the pipeline
