Removes all TVL-specific values in favour of environment variables
supplied by Buildkite.
This makes it possible to reuse this script outside of TVL.
Change-Id: Ic543bc41e4c81e65ee349ad241c515231e97ab30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5005
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
allDeps filters the lisp deps according to the given implementation,
processing any implementation conditional attribute sets. These are not
understood by allNative, so we need to pass it the already filtered
input or evaluation would fail.
Change-Id: I9eb2d0c3b2bf70d759d03490cf31fc585283ce7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5001
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Some companies do not know the 'cl' term. They do know of 'change' and
would maybe not like to introduce one more synonym.
This cl introduce an optional entry 'gerritChangeName' in
besadii.json. The string has to match `^[a-z0-9]+$` for readability.
Change-Id: Id70fcb1e45158869f88bf37669be49b8b8a3b295
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4825
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
This creates a josh workspace called `views/kit` which exports all
dependencies needed by external users of the TVL kit.
The intention of this is to have a single clonable "virtual repo"
which provides everything needed to run a system like TVL, but without
the projects that we develop *in* TVL.
Right now this is limited to the basic Nix code (readTree, pipeline
generation) and things required for Gerrit integration.
The workspace maps depot paths into slightly different (simpler)
locations inside the exported view, to avoid a nested, mostly empty
structure.
Note: This is not an "external API" or some such of depot, it is
simply a convenience for exporting some of our tooling.
Change-Id: Ied24aaef117fc2e0c188ec742ae7cd993a5babd3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4991
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This change is required for using our package set in exported josh
workspaces. It has no functional impact on depot itself.
Change-Id: Id48b40f067b5d53a2b7386a0ba1146a72268b923
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4990
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
TL;DR:
- Ensure that export.sh -> import.sh -> export.sh can round-trip without
intermediate tools.
- Remove default values for variables like ${1}, which only seem to complicate
things.
- Add `trap cleanup EXIT` to scripts.
- Remove noisy full-paths from `zip` (note: a more intuitive, less configurable
`zip`, `unzip` should exist).
Change-Id: Ibbd98d1f0156639138175fcb89e9dfbd17fdae5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4993
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
The variable, `evil-visual-update-x-selection`, must've updated to
`evil-visual-update-x-selection-p`. When I moved from ~/briefcase to /depot I
adopted the newer version of nixpkgs.
Change-Id: I27de45c73833325f64b1e176b8793a0d05233331
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4992
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Found this while looking for non-3p uses of builtins.fetchGit. The
hailgun package is in the package set now, and the other thing was
unused (and it's also in the package set).
Change-Id: I6f519fd6014bbed90fc6cee695bed7afbcf55717
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4984
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
The function is depot specific and thus uses tvl-depot-path, so it
belongs in `tvl.el`. Since non-sly-users won't need it, we tie its
definition to loading the sly package.
Change-Id: I8b104deab455d218d3df6a800e35cc104220a841
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4960
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Moves to the derivation-based git fetchers everywhere in third-party.
This might help with forward-compatibility with newer Nix versions,
though that's not our primary concern right now.
Change-Id: I565bb72585b8639893e9ea3a9e233338aede63a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3903
Tested-by: BuildkiteCI
Reviewed-by: zseri <zseri.devel@ytrizja.de>
The cookie secret in the encrypted file was too long, because the
generation command in the oauth2_proxy docs is also wrong. Should
probably fix that upstream as well.
Also noticed that an extra '2' snuck into the service name and fixed
that.
Change-Id: I9a344a75993ab1f98299a8d45e7f5b2e146b7fc5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4957
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This changes the logic for build pipeline generation to inspect
an (optional) parentTargetMap attribute which contains the derivation
map of a target commit.
Targets that existed in a parent commit with the same drv hash will be
skipped, as they are not considered to have changed.
This does not yet wire up any logic for retrieving the target map from
storage, meaning that at this commit all targets are always built.
The intention is that we will have logic to fetch the target
map (initially from Buildkite artefact storage), which we then pass to
the depot via externalArgs when actually generating the pipeline.
Change-Id: I3373c60aaf4b56b94c6ab64e2e5eef68dea9287c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4946
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Always create a structure that maps all targets to derivations, and
persist it as a JSON file.
This relates to some of the ideas expressed in:
https://docs.google.com/document/d/16A0a5oUxH1VoiSM8hyFyLW0WiUYpNo2e2D6FTW4BlH8/edit
The file is always uploaded to Buildkite as an artifact. This allows
for retrieving it based on the commit ID in a Buildkite GraphQL query.
By default, Buildkite stores artefacts for 6 months. Storage location
can be overridden (with custom retention) through some environment
variables, but for now at TVL the Buildkite-managed storage is fine.
See also: https://buildkite.com/docs/pipelines/artifacts
In the subsequent filtering implementation, when diffing commits
across a time-range that exceeds artefact retention time, we should
simply default to building everything.
Change-Id: I6d808461cd1c1fdd6983ba8c8ef075736d42caa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3662
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
It would left-associate in some nix version, this way it’s clear that
you want to join a path segment.
Change-Id: I82a00de720187d1faf908c842e396301ea435a97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4886
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
We can't use cl-lib as that apparently doesn't contain lexical-let*.
Change-Id: I8e65d20215ae5667bb92b71e6318ad9d66125320
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4941
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Since we now group patchsets inside of Buildkite, the results are no
longer guaranteed to be for the right patchset.
There might be some metadata passed in from Gerrit that would let us
do this with the commit ID instead, but I haven't checked.
Change-Id: I5b74a17697511160fcc89d3dbef23517d974dc6f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4890
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
I'm not sure where the previous list originated, but it was missing
some officially documented statuses. However, the API definitely
returns statuses that are documented to only appear in other types, so
this commit simply maps ALL statuses that Buildkite has documented for
any type.
Also adds a log statement in case we encounter a brand new, unknown,
undocumented status.
Change-Id: Iff003a3bd2608702019ae0f4137958435ad0856f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4888
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
... and remove a spammy log statement.
This changed in besadii a while ago and lead to the behaviour of
failing silently, instead of failing with an error saying "undefined
undefined".
Note that with this change merged the plugin probably still won't work
again, but it gets us a step closer to the real error.
Change-Id: I3db25d246f4b1c634d316cd92574e27fb220d769
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4887
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Currently Gerrit is configured to copy forward the scores of the
'Verified' label if the tree of the commit does not change (e.g. only
author information or commit message is modified).
Besadii still triggers builds for these patchsets though. With this
change it will inspect the (previously ignored) "kind" of the patchset
and skip patchsets with the same tree as their predecessor.
See Gerrit docs for the semantics of "kind":
https://gerrit-review.googlesource.com/Documentation/json.html#patchSet
Note that an argument can be made that we should do the exact opposite
- stop carrying over 'Verified' at all and always build all patchsets.
I think this depends on whether we intend to use commit metadata in CI
runs at all. Adding a few people to the review for opinions.
Change-Id: I48a96a1ad1e07d92330d84e5cfdc820a39395297
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4867
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: asmundo <asmundo@gmail.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
(who:html-mode) needs to be set at macro expansion time to properly take
effect which wasn't the case before, but is ensured now by
:compile-toplevel. :load-toplevel ensures that who inside the repl will
behave the same.
Since the :html5 behavior is now actually used, we need to adjust some
of the test cases to account for the different :html5 escaping mode.
Change-Id: I4dfe1d2db38da6a2486fde86596f7e5f50ed8b9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4885
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Adds more things I keep using via nix-shell, as well as the
deploy-whitby script (which is independent of a particular depot
checkout).
Change-Id: I36f87de7645768a05268c90ba9b3ab833bacca05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4881
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
nvd only shows us changed versions of packages, as well as
added/removed packages, which means that for the majority of depot
packages nothing will be displayed
however, the current output of nix-diff is not usable anyways, so
having something that can be looked at is better than nothing
Change-Id: Iefbd8139c7ccf5c88ed1209897abdb2ae9302e91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4868
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Temporarily use a master commit as some interesting failures have been
fixed there (yubikey-manager and notmuch).
* //users/tazjin/nixos/camden: The acme module was moved around, so we
need to adjust the disabled module path.
Change-Id: I21c6a1963ffe4205f3577f531ce10b778a82e2ff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4865
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
