This function is also generally useful for readTree consumers that
have the concept of subtargets.
Change-Id: Ic7fc03380dec6953fb288763a28e50ab3624d233
Since GCP nuked us, the backups are now moving to GleSYS'
S3-compatible object storage.
This refactors the restic module to support S3-compatible storage
instead of GCP, and switches to the appropriate new secret paths.
The secrets were placed on whitby manually and I verified that the
backups work.
This fixes b/157
Change-Id: I6a9d2b0581967605ce736605a3befb44cdeae7e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3883
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
It seems that shell variables don't work as expected inside the
Buildkite pipeline, so usage of variables has been removed.
We also don't echo the revision anymore because of that, but it does
still appear in the log of `git push`.
Change-Id: I124e3b09af896da898f2a78715ed371651a1c5f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3780
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This makes the revision number available much earlier (before the rest
of the pipeline runs, while Nix eval is happening) which should only
be a few seconds after a commit to canon.
It is also more readable in this shape.
Change-Id: Iccbb17dfef6afe68f54fda41e8d10c4dc52b08c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3775
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This automatically pushes a new ref at refs/r/$revision to Gerrit
whenever a CI run completes on canon.
Revision numbers can be fetched from Gerrit with this command:
git fetch gerrit "refs/r/*:refs/r/*"
Note that this build step requires credentials to be provisioned on
the CI runner machine.
Change-Id: I37bb14346832f891240aa47bb55affaace3d5f21
The previous hash had a weird salt length and a trailing newline.
This fixes it.
Change-Id: I1f03238181d0caad38e1f1dbc477356bc20fc32d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3689
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
The setup is explained in the comment, but TL;DR: Use the derivation
hash of static files to create permanent URLs.
Relates to b/151.
Change-Id: Ib1ca3a1a00c90a47f4bf39c29a8b4bbf5b215e7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3664
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This hostname can be used for hosting static assets with aggressive
caching for everything, or potentially CDNing stuff if we ever have
large things here.
Change-Id: I10afdad5eb08125d8d09108e9e099f5573362fe5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3663
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
As cschilling explained on cl/3563, there isn't actually anything in
this state that we *need* to persist. We're still keeping it in a
persistent directory on disk as this serves as an optimisation after
restarts of josh.
Change-Id: Ia88886792a5acac34508b5b8a669bd519ca033de
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3631
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This lets each service declare their backup paths together with the
configuration for the service, which is a lot more sensible than what
we had before.
Fixes b/147
Change-Id: If76fe62639f4cc0e6fbb63a2959d584479d8f0fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3583
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
I can never remember which is which.
Change-Id: I69b8235862b8c5b49030a74bfca25aaa113273b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3582
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This makes it easier to click through to a build from Gerrit after
submitting a CL.
Change-Id: Ic5c6eeb81c87bc4ea23c5c5ca25704434b081fd0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3572
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Currently besadii only posts comments when builds succeed, but it
might be very useful to also have a link to a build when the build is
started.
This just shuffles code around. The only functional change is that the
`labels` field in the review input is marked as `omitempty`, as this
will not be needed when posting the build start comment.
Change-Id: Id4a43fad8817c9a15da02f01ab2b781d48b46978
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3571
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Relates to b/147.
First step towards giving depot modules the ability to declare their
own backup directories by moving all restic configuration into a new
module and adding a NixOS option for inclusion/exclusion paths for
backups.
This still keeps all backup paths within the whitby config.
Change-Id: Ia96833668f1a3d02da892261153d8b02156b8ac0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3565
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Previously we served the dumb git HTTP protocol from code.tvl.fyi via
cgit. This CL disables this feature and instead runs josh in the same
location (by redirecting appropriately), but while also enabling
partial cloning of all subtrees of the depot.
For example, after this CL the following would result in an
independent clone of //nix/readTree:
git clone https://code.tvl.fyi/depot.git:/nix/readTree.git
Note that there are no josh workspaces configured at all for now,
these references are only for static depot subpaths.
Please refer to the documentation for josh for more information on
available kinds of josh filters.
Josh state is kept in a systemd state directory in /var/lib/josh and
backed up to Restic. Backing this up is necessary, as josh uses
stateful information to do things like tracking merges and rewriting
history per subtree appropriately to avoid cloned repositories ending
up in peculiar states.
Change-Id: I156f0298c2aa42e3bdbf5a0e86109070d640c56e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3563
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This one seems a little more involved:
https://docs.sourcegraph.com/admin/migration/3_31
I believe we skip that corruption issue in the previous CL though, by
simply never deploying a version with that weird broken image.
See b/144
Change-Id: I3bbf1b719d00905e08a92011ace5485467f504ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3525
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
We changed away from the default sourcegraph one because it didn't
support Nix, but it seems that there's been a change in the
interaction protocol.
Change-Id: I3a2691df6a87672cf83b819143f25d93d9cd6d13
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3531
Tested-by: BuildkiteCI
Reviewed-by: eta <tvl@eta.st>
Reviewed-by: sterni <sternenseemann@systemli.org>
Add the beginnings of an auto-deploy script for whitby, intended to
be (eventually) suitable for running automatically in a systemd timer.
The current iteration of the script doesn't actually do any deploying,
but instead takes as an argument a revision, creates a new git worktree
in /tmp with that revision checked out, runs a nix-diff of whitby's
system derivation in the running system and at that closure, puts an
html-rendered version of that diff in the public directory used by
deploy.tvl.fyi, and finally sends a message to IRC via irccat with a
link to that HTML page.
Refs: b/110
Change-Id: Id40525567f8845590c909568befd8d00c07a481c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3145
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kn <klemens@posteo.de>
Add a new domain and nginx virtual host at deploys.tvl.fyi, serving out
of a static directory on whitby which is created by systemd-tmpfiles.
This will be used to serve diffs rendered by nix-diff for
pending deploys for whitby
Since this contains stateful data, it is added to the restic backups
on whitby.
Refs: b/110
Change-Id: I5869d40800bbf5fb8fb39878a857f66ff5787830
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3144
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
We changed the configured pipeline in Buildkite to upload
`static-pipeline.yaml` instead of containing the steps of that
pipeline itself.
This makes it easier to test changes to builds and such, but adds
another build step with scheduling overhead etc.
However - we can work around this by killing one of the existing build
steps. There's no reason the failure status zeroing (required for
status reporting) shouldn't be part of the pipeline setup, so I've
moved it there instead and nuked that step.
This should mean that the pipeline is configurable from within the
repo, but without slowing anything down.
Change-Id: I206ecc02647de42a461e33c02879ab84daf5ed2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3461
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Skip build steps if they have already been built, reducing pipelines
to the things that actually changed between builds. On canon all
targets are always built (we require this for anchoring).
Note that this is not perfect, garbage collection and competing
pipelines may affect each other.
Also note that we have some impure targets that change on every
commit.
Change-Id: Ic6bae3b6c8e1e7fd2116ec252f5089f471854ab6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3427
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
We currently evaluate every target twice -- once when the depot pipeline
is built and once when actually running the build step in question. Nix
evaluation is quite slow especially given heavy use of import from
derivation in depot, so avoiding the second evaluation is desireable.
Evaluating a derivation yields a `drv` file in the nix store which can
be passed to `nix-store --realise` in order to build it eliminating the
need to wait for evaluation. We can obtain the path to the `drv` file
while building the pipeline via `target.drvPath` and remember it for the
build later.
However we need to work around a flaw (or oversight) in Nix's dependency
tracking via string context: This is based on derivations, not output
path (because this is what evaluation deals with, likely). This is no
problem per se, but an issue is that Nix can't express a dependency on
a `drv` file without any of its output paths. This means for us that we
either have to build all output paths at evaluation time (which we don't
want, obviously) or to deal with the fact that the `drv` file we need
may be garbage collected at any moment after discarding the string
context -- then nix is unable to track the reference from the pipeline
to the `drv` file in the store.
So to prevent a race condition between the pipeline and the garbage
collector we fall back to the normal nix-build invocation as we did
before.
Change-Id: I9ef8bd233085dc6e30eba54f403ea03ac2d35748
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3426
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This is because I'm bored of CAS gradually consuming all the RAM on Whitby.
Change-Id: Idcc14c19d99a6d3553739c5765be3faf2bdf9d84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3233
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
This is a bit of an under-documented feature, but if the "tag" field for
a gerrit review starts with the string
"autogenerated:<something>~<something-else>", only the last comment per
instance of <something> will be shown by default on the CL page (with
the rest viewable by toggling the "Show all entries" switch). The idea
behind the "<something-else>" tag is to be used for the "type" of
comment within a particular system - gerrit's documentation gives the
example of one tag for "the build is running" and another for "the build
has finished, here's the result".
Change-Id: I9199a6ed97beca1b3a51ec5d6230c6c8358ba2b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3374
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
The dropping of `www.` is intentional, that was unused.
Change-Id: I300f82bb6e5626e2658be8fc5b5e3cf872ab7099
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3384
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Adds a new module for the nixery.dev domain and serves it from whitby.
Note that the DNS records do *not* point to whitby yet, so deploying
this will lead to a failed TLS provisioning unit - but this is
intentional.
Change-Id: I911f67a0aa24f8df3cb52d2cfc49a8b6132cf718
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3383
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
We'll need to do a DNS switchover, likely with a short amount of
downtime due to TLS provisioning.
It would be possible to avoid this by provisioning a cert manually
pre-hoc through the DNS challenge and then configuring whitby to use
that, however I simply don't have time for that right now and the
Google Cloud Project for Nixery is going away in O(days) for $reasons.
Change-Id: I88dface5aaacec5acfa525ae117462f8ad296d92
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3382
Tested-by: BuildkiteCI
Reviewed-by: kn <klemens@posteo.de>
This is required when people change their email addresses (e.g.
cl/3349) as nothing in Gerrit will update that information from the
OAuth provider.
Change-Id: I1eafdf22efd37898dcd0d06bb9a5d1471ffb5e31
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3356
Tested-by: BuildkiteCI
Reviewed-by: eta <eta@theta.eu.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
I got a new domain, etc.
Change-Id: Ic8ffc01f4e5e89dc2458d80a9c38757438cfa764
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3349
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
This record is intended to serve Nixery.
Change-Id: I575dedac18c98f9f4bd5e459babe79e850361651
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3343
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This sets up a very simple Nixery instance with some things lacking:
* no support for garbage-collecting image fragments (yet)
* no popularity setup
The plan is to use this to get the ball rolling on a separate
domain (e.g. images.tvl.fyi), iron things out and then look into
flipping over nixery.dev
Change-Id: Ic594809f9d487fec7a0f632d608752a3f9c61315
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3280
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Break out the configuration for the prometheus fail2ban exporter, which
is a simple python script that exports stats from fail2ban as a
prometheus-scrapable textfile, from Mugwump into a reusable nixos module
in //ops/nixos/modules.
Change-Id: I5451c9c5de6c7bc4431150ae596a9c758bf1b693
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3136
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This fix is essentially the same as the one in cl/1263.
Change-Id: I27be280a610914fcfbb6d7fee7aebaa56b993812
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3158
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
* users/grfn/system/home/yeren: remove obsolete awscli2 overrides
* ops: make new isSystemUser || isNormalUser assertion happy
* users/grfn/system/system/mugwump: make buildkite agents system users
* users/tazjin/nixos/camden: set isSystemUser = true for git
* users/tazjin/emacs: Remove missing & broken packages
* third_party/openldap: remove, as the argon2 module is now enabled upstream
* third_party/gerrit_plugins: Pinned new unstable hashes
* third_party/nix, third_party/grpc: Disabled CI as these are broken
* third_party/overlays/emacs: Bumped version to stay in sync with channel
* third_party/buzz: Update LIBCLANG_PATH to reference libclang.lib,
since libclang's default output no longer contains libclang.so
* users/grfn/system/home: Install julia-stable instead of julia (which
aliases to julia-lts), as the latter depends on an insecure version of
libgit
Change-Id: Iff33b0ecb0ef07a82d1de35e23c40d2f4bf0f8ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3001
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Until we have declarative ZNC config (which requires a solution for
secrets handling in it), make sure we back this up as well.
Change-Id: Idb186327da171eb6d3dbbd83801639f1f9321a40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3159
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Use the new module settings which apply configuration in cn=config
instead of slapd.conf.
The module performed this update via lib.mkChangedModuleOption, I've
applied the transformations contained therein manually. Note that some
of the settings were already in place, which means that the `suffix`
and `database` options seemingly disappear into the void.
Fixes b/105.
Change-Id: I8a968c1eb8cb7827618cb732cdb46006a5d011f9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3157
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This changes the evaluation order for the `depot` argument and ensures
it is partially evaluated before the module system starts resolving
imports.
This way we can import modules from `depot.path` without `depot`
having to come from readTree.
Fixes b/129.
Change-Id: Icf4dd2be15011055dac8b27e991a4ff6a12bf827
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3156
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This time using `tools.hash-password` because login did not work with the
initially created hash.
Change-Id: I1eb62a496d2d8497d27573af47bf8bf70dac9bbb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3153
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
This will be used to serve (nix-) diffs for pending deploys of whitby
Change-Id: Ia864993b1fcb3b7ce5fcc21f32a27528a4c31f08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3149
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
irccat is passing the realname option as the ident of the user, which
doesn't match what is in ZNC.
It hasn't seen any upstream commits in a long time, so I'm just
leaving this as is and fixing it locally in our config.
Change-Id: I3bf865f37b8df9c1cd891a94245ca3fad376bbe1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3150
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
I like running fail2ban on any machine that has stuff like ssh
world-open, to limit the potential for password brute-force attacks etc.
Change-Id: I0c60811ae5a2fddb44f04679fb455e646b8e39c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3138
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This doesn't replace all of them in the repo, but at least the ones
that are relevant to our move.
Change-Id: I842e7594b4c16af30d880272417874f6b29afd22
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3134
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: grfn <grfn@gws.fyi>
This configures owothia to use her new bouncer to HackInt.
Change-Id: I80eb8191c2b0f2a6f8a31d19b60250ade27c1913
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3129
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Points clbot at the new local ZNC instead. This will make it part of
the things happening through the `tvlbot` account.
Relates to b/101
Change-Id: I1c15ffa5720d3af34475c15bee3fdaa537ac659b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3127
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
This adds a shadowsocks service, running on port 8443, tcp and udp.
The password is read from /etc/secrets/shadowsocks-secret.sec, and needs
to be populated externally.
Change-Id: I6797150db108ba14459502dee43d8e4ed6cfa910
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3125
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
The following commit itends to bind on port 8443 on all interfaces,
so let's move this to something else.
Change-Id: Ibb94a0f4e6892b6e543b542b89bcdaaefb617f23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3126
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Bouncer to be used for TVL's IRC bots, see b/101
Change-Id: Ic9f71ecd94365d3baa31e0552b1ce16362f94557
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3124
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Sourcegraph logs warnings about this on startup otherwise. Unclear
to what degree it really affects operation though.
Change-Id: I6ee7c5358631aafd9a7f8155150361bf7499314d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3098
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
The shorter one is going to be more convenient when we get
go-link (or, well, at-link) support.
Change-Id: Ic24adcdad679b893c40c87731add818660259dac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3091
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
.. this is actually likely not disabling it for some pages, that will
need this to be copy & pasted, but it's hard to tell just from the
nginx docs. We'll make sure after deploying.
Change-Id: I2fa6e31ca10835a206673b858594fa071e729d82
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3020
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
This is currently done ad-hoc in a bunch of our systems, but we should
just do it centrally.
The commit message is a bit of a lie, as this doesn't yet update
grfn's systems.
Change-Id: Ic771c1a1da78ec5de9cffbf94c296dce5e11fd84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3047
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
It needs to refer to this by full path of course.
Change-Id: I911c876ba18877681accb722426314d92b9f2318
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3042
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This will require the daemon to be running when launching GC, but
won't start it if it happens to not be running for some reason.
Change-Id: If48fe336030173f028428fc00a81d339ef4b8bce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3015
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Adds a module that automatically collects garbage based on disk space
thresholds, and configures it to run hourly on whitby.
This is implemented as an alternative to cl/2937, which I've been told
uses a Nix feature that doesn't actually work.
Under-the-hood this is simply a systemd timer running a shell script
which checks available disk space and runs GC when necessary.
Change-Id: I3c6b5de85b74ea52e7e16c53f2f900e0911c9805
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3014
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
There's a hard-coded list of Admin usernames for the moment. We should
revisit this and get an actual groups setup in LDAP that's propagated
through...
Change-Id: Ic3601f1a9753573076769f4912038e9f1b60e139
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2982
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Enables a Grafana service pointing to whitby's local Prometheus
instance, accessible at status.tvl.su.
I've no idea how to configure Grafana and if it's possible to link it
to CAS, but we'll see about that later.
Notes:
* the explicit fixpoint for whitby config has been removed as we
have the `config` parameter available now
* backups are enabled for the Grafana storage location
Change-Id: If5ffe0c1a3378d1c88529129487c643642705fd2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2948
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Instead of having two ways of accessing the path to the depot (one of
which was stuttering, depot.depotPath) we settle on only one:
depot.path.
This was mostly used for NixOS module imports.
Co-Authored-By: Florian Klink <flokli@flokli.de>
Change-Id: I2c0db23383fc34f6ca76baaad4cc4af2d9dfae15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2962
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Dropping the message field will make Buildkite use the commit messages
instead, which makes for much more readable build logs.
Change-Id: I1849f811632526893b700f117c9f6cf64888c329
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2949
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Enables Prometheus with a local node exporter, and nothing else for
now.
Some additional collectors have been enabled for things that might be
relevant on whitby:
* systemd: all our services run in systemd
* processes: might be interesting for build-related stats
* logind: might be interesting for interactive usage stats
Change-Id: I48dacdd9c68b4be9edff7b3cb6256dad562498c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2930
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
--show-trace should make it easier to debug tricky evaluation errors
without running nix-build -A ops.pipelines.depot locally again.
Change-Id: Ice540562c3b389fc2a49ec1fc0adacb17db2a528
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2947
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Rename my //users directory and all places that refer to glittershark to
grfn, including nix references and documentation.
This may require some extra attention inside of gerrit's database after
it lands to allow me to actually push things.
Change-Id: I4728b7ec2c60024392c1c1fa6e0d4a59b3e266fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2933
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: glittershark <grfn@gws.fyi>
Splits //ops/nixos into:
* //ops/nixos.nix - utility functions for building systems
* //ops/machines - shared machine definitions (read by readTree)
* //ops/modules - shared NixOS modules (skipped by readTree)
This simplifies working with the configuration fixpoint in whitby, and
is overall a bit more in line with how NixOS systems in user folders
currently work.
Change-Id: I1322ec5cc76c0207c099c05d44828a3df0b3ffc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2931
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: glittershark <grfn@gws.fyi>
This change is required to run the ⚓ step on canon builds.
Change-Id: Ib3cebac67c9f5337b27a948f120b0a9ba834ef2a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2932
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: glittershark <grfn@gws.fyi>
Adds a conditional build step that only runs on the canon branch, and
only if 🦆 (the status reporting step) succeeds, which creates a
new Nix GC root for all depot targets named `depot-canon`.
In practice this might be a bit racey, as canon builds are not
guaranteed to succeed in order (though it is likely). This shouldn't
matter much in practice: We only want to prevent rebuilds of the whole
world.
This fixes b/102
Change-Id: Id3d0bf4158bffcb1ed6929888a29d31609b6ece1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2904
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
This ensures files created by the Buildkite agents are always owned by
the same group, without having to manually chgrp afterwards.
Change-Id: Idbaedec43c16b2ee137d1a95719a05d46db8f900
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2929
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Please read b/108 to make sense of this.
This gets rid of the explicit list of exposed packages from nixpkgs,
and instead makes the entire package set available at
`third_party.nixpkgs`.
To accommodate this, a LOT of things have to be very slightly shuffled
around. Some of this was done in already submitted CLs, but this
change is unfortunately still quite noisy.
Pay extra attention to:
* overlay-like functionality that was partially moved to actual
overlays (partially as in, the minimum required to get a green
build)
* modified uses of the package set path, esp. in NixOS systems
Special notes:
* xanthous has been disabled in CI because of issues with the Haskell
overlay
* //third_party/nix has been disabled because of other unclear
dependency issues
Both of these will be tackled in a followup CL.
Change-Id: I2f9c60a4d275fdb5209264be0addfd7e06c53118
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2910
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This configures accepting requests for b/ and cl/ on plain HTTP ports,
and redirecting to b.tvl.fyi & cl.tvl.fyi appropriately.
Additionally, Panettone request URIs that only contain decimals are
redirected to `/issues/$request_uri` to enable issue short-links.
This fixes b/32.
Change-Id: I56954d8d69a3624267778b467520c509f4daa6c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2908
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
