This makes CI utility functions available in TVL kit. For now this is
only the Terraform check, but said check has come up in other repos
before so it's useful to centralise here (and we might add more!)
Change-Id: I18acb19fc3407650ab9bad53dfba022dda498c07
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5858
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
This can be re-used across Terraform environments.
Change-Id: I3d964a17d1cda1aff1df12bd4c0c3ee84b7f7748
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5850
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
Generating a release-only pipeline skips a bigger chunk of eval this
way (the step itself is never actually evaluated, which means we never
actually compute the drv), which can be quite beneficial in terms of
evaluation time.
Change-Id: I2739026ddd1c6a86f82627ac26a046c5fe7359ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5830
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Extra steps that use `depends_on` (e.g. if they need output from their
parent) should not actually depend on their parents build step if the
build phase is not active.
This is required to actually decouple the phases.
Change-Id: I398da9a8a53e97ca3c635342259fc722d54b8e4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5829
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Using the `activePhases` attribute, the set of phases included in an
evaluation can be modified.
This lets users generate e.g. ONLY the release steps of a pipeline.
Change-Id: Ib0c38826dd69666094d619f5f324d1baafce8134
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5828
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Remove a workaround for a GleSYS provider bug that was fixed in the
last release.
Change-Id: Ibd25de0b4dcccd781518d5d0ae1c75d296f6b05f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5845
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
In order to run this the secrets needs to be sourced, e.g.:
eval $(age --decrypt -i ~/.ssh/id_ed25519 $(git rev-parse --show-toplevel)/ops/secrets/tf-buildkite.age)
Change-Id: I9f6a02c0dac22f584181635861ddbb06cf849f14
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5838
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Of course we can't pass the overlays without causing an infinite
recursion, but they are also intended purely for unstable nixpkgs,
so it doesn't matter.
Change-Id: I0e1b42e37ad12872f9420cf59dff6d944b2bc5d3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5847
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This has come up a couple of times. This way system is passed to all
derivations. Maybe we can do something useful with it.
Change-Id: Ia7dfcffbc82abbd3128342a8971a3861865be713
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5832
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
this way the tooling provided by //.envrc will not disappear
Change-Id: Icba1fe85d65316fde939ed3451e0cf80d9064382
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5836
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
This will avoid things like extra steps being accidentally ignored
because of typos.
Change-Id: Ic4fa5925e42a7a449f89b4cde1510e216e91da6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5827
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
This would block CI on human-approval if people were allowed to do it,
so they're just not.
Change-Id: I8a9b657d5c91636a7b4de249b977e24fc0941a1c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5826
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Previously the extra steps were roughly divided into steps that run
"at build time" (i.e. before we publish results to Gerrit), and
"post-build" (i.e. later on).
In practice, these are something like a build/release pairing, where
steps running after the build results are returned are mostly run for
side-effects (e.g. publishing git subtrees to external repos).
This refactoring makes this distinction explicit in //nix/buildkite
and changes the extraSteps API with an explicit `phases` attribute
instead of the previous `postStep` attribute.
In practice the previous API is still supported, but will throw
evaluation warnings until an arbitrarily chosen cutoff date of
2022-10-01 at which point we will change using it into a hard error.
This uncovered a few strange behaviours which we only accidentally
avoided, most of which I have left TODOs about and will clean up in
subsequent commits.
The purpose of this commit is to allow for separate evaluations of
only build or only release steps, for example if release steps are
evaluated in a slightly different context (e.g. with overridden
versioning that is not relevant to standard CI functionality).
Change-Id: I0b0186e3824273c15a774260708702d4a5974dac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5825
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
This is in preparation for a subsequent CL that will do much more
significant changes in //nix/buildkite.
Change-Id: I80a8d67d3a7d593854c8d711572483c2581e7881
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5824
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
The isPowerPC predicate has been [removed], since it was misleadingly
named (it just matches PowerPC, 32bit, little endian). This means the
64bit code path could now actually work.
Not sure about endianess, the CCL docs don't really say much regarding
that topic.
[removed]: https://github.com/NixOS/nixpkgs/pull/168113
Change-Id: Icf4a8c6b1df95fa597ed87508f57aaa73e6185ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5796
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
`mg repl` is essentially a shortcut for nix repl $(mg path //) which
comes up often enough for me. Launching a repl only really makes sense
in the repository root with how readTree works at the moment, so I think
this is a convenient addition.
Change-Id: I32b695885c2e6eaecdcc656c7249afa504439913
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5822
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This is merely a little demonstration of nix#6579:
`users.sterni.nix.misc.isRestrictEval` returns whether the restrict-eval
setting is true or false by exploiting the aforementioned Nix bug.
Change-Id: Icca354d1cd6571cdf0804abae27aac91a18cda1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5692
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Not updating the stable channel to 22.05 yet, since it ships a too
recent bat for us.
Change-Id: Ie8a541e972879f92c62b5e04254cca7b5880c813
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5821
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Resuscitate the configuration for roswell, the semi-portable
configuration I use for ec2 development boxes. Lots of the changes here
are trying to get Tramp working.
Change-Id: I2dc2fd1d9aa76e145fa3f3f847af761cb652ab47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5798
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
This switches upstream from hankhero/cl-json to
sharplispers/cl-json (the former of which had its last commit in 2014).
Sadly the new upstream hasn't decided on an appropriate fix for b/145
yet (due to concern about backwards compatibility, apparently). I did
not look before working on a fix, so I have an 90% finished fix which
is (I think) better than the already proposed ones, so I'll patch it in
here eventually.
Change-Id: I9e39e138fa655794b864db5f268bdfdc35788fcc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5795
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
I keep having this in the user env instead, not good.
Change-Id: I683efc9782281053cb4aee1875c3a664c8dcdae8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5794
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This time, the emacs-overlay seems to have unbroken itself.
* //users/tazjin: use zfs.latestCompatibleLinuxPackages instead of
linuxPackages_latest, since ZFS needs time to catch up (i.e. ZFS is
broken with a 5.18 kernel).
See https://github.com/NixOS/nixpkgs/pull/174091#issuecomment-1137175076
Change-Id: I8d1123af236a5e56618f6ac7a2e22511594b7d4b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5792
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
We needed a derivation for that, but this can also be used in the
Nixery docs building process (which includes nix-1p).
Change-Id: If97cf785a33d703af975da3b41de9b69566dfa81
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5789
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This reverts commit ad7f07e6f1.
Reason for revert: This was just a test of b/167.
Change-Id: I1f709ed1c76c69555bf987370d4e521bd61e915e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5801
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This is a less invasive way to achieve the same goal as cl/5681, by
preventing the already existing nixpkgs store path from being dumped
again at the call site. To support nixpkgsBisectPath, we simply check if
pkgs.path is below builtins.storeDir and use builtins.storePath based on
that.
This is actually similar to the approach taken in the nixpkgs
documentation system which tries to limit the amount of nixpkgs that
needs to be dumped by using filterSource on specific subtrees of
nixpkgs. For this to work it has to insist on pkgs.path being an
ordinary Nix path, though.
Change-Id: Idf892f90a5d811184568e4702a901c334d56210e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5787
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Because of math being upsetting, we were adding 4 padding characters to
an already-properly-padded base64 string, which broke tazjin.
This also breaks this function out into panettone.util, and adds a test
for it.
Change-Id: I7bc8a440ad9d0917272dd9f2e341081ea14693da
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5782
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
The JWT spec apparently specifies that base64 strings in jwts aren't to
be padded - but the common lisp base64 library doesn't know how to
decode unpadded base64 (it signals a condition in that case). This adds
the extra padding characters (a number of `=` characters such that the
length of the string is a multiple of 4) using some FORMAT wizardry (?).
Change-Id: Ic6b66f05db2699bf1f93f870f5dd614c37eccc2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5781
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: grfn <grfn@gws.fyi>
Instead of directly connecting to LDAP and attempting to bind
usernames/password, authenticate users through an OAuth2 flow to
Keycloak.
This has the advantage of reusing the same SSO we already have for
Gerrit, Buildkite, ...
However, much of panettone's functionality makes assumptions about
LDAP being used. As a result there are some warts introduced by
this (for now):
* Since LDAP DNs are used as primary keys for users, we have to
construct fake DNs based on LDAP usernames
It might be sensible to migrate this to the UUIDs used by Keycloak
eventually.
* LDAP is part of the serving path for issues (for fetching user
information), however panettone no longer has a way to fetch
arbitrary user information unless it is persisted in its database.
To work around this, we construct a "fake" user based only on its
DN (i.e. only the username is going to be "correct") and use that to
serve issues.
* Email notifications no longer work (panettone can not access email
addresses)
Some of these need to be worked around by persisting some of that
information in the panettone database instead, as we don't want to
give the service the ability to access arbitrary user information
anymore.
We can probably do this with the user settings feature that already
exists and populate it on launch, but as of this commit email and
displayName functionality is simply broken.
Change-Id: Id32bf5e09d67f0f1e883024c6e013eb342f03b05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5772
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
