Some quick testing shows that this improves my data transfer speed to
whitby by roughly 200%.
Change-Id: Id94de975b1ae0930f8d0fe038582dbac0037676c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2659
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: ben <tvl@benjojo.co.uk>
Reading through the changelogs, this includes the following two
changes that may require us to do something:
* For users of single-image Sourcegraph instance, please delete the
secret key file /var/lib/sourcegraph/token inside the container before
attempting to upgrade to 3.21.x.
* A campaigns.restrictToAdmins site configuration option has been
added to prevent non site-admin users from using campaigns.
Change-Id: Ieacf85a9059ad5222800f8d7d4a43435f489a39f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2638
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This was a leftover from the time we were installing.
Change-Id: Id875b907d7f76081a45e7f8f2666b7fba6aefc86
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2632
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
This is my new X13 AMD Thinkpad, on which many fun things will be done.
Change-Id: I4de114a8c5ebb37d2f4844f407d2dc0e7cc9557e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2620
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This commit removes my user directory in the depot, my user account on whitby,
my entry in the LDAP database, and my entry in the website graph. I've had my
fun with TVL, but I want to move on to spending time on some other things.
This additionally removes aranea from the website graph, which they have
requested in private.
Change-Id: I2d098c8fe239f20d9f6c6cbf66a3dfb4a955a4cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2436
Tested-by: BuildkiteCI
Reviewed-by: multi <depot@in-addr.xyz>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Since the slapd data is static and generated using nix, we can simply
move the user list into ops/users, so it's recognized by readTree and we
can use it as ops.users both in ops/nixos/tvl-slapd and web/todolist as
a general purpose user registry for depot.
Update docs/REVIEWS.md as well.
Change-Id: I35caaaab70a5578c47cedc7f33077dd513766290
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2419
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
camden.tazj.in (the host in my flat) is going down as my belongings
are being moved into storage.
Change-Id: Id66512fd2ec6dbdcb6dfc3862af49cfadb15cfa1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2405
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: glittershark <grfn@gws.fyi>
My main workstation is a Thinkpad without a great deal of compute
power available, so enabling the use of whitby as both a substituter
(services.sshServe) and a remote builder (openssh.authorizedKeys) will save me
some time when working on nix things and depot things.
Change-Id: I17bfcbb9860f42fb667603ad819e38e82e6052da
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2399
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
Changes:
* ops/nixos/tvl-slapd: The NixOS module for OpenLDAP has removed the
ability to configure OpenLDAP directly and now forces users to use
some kind of weird Nix->OLC mapping that is mostly undocumented.
This moves the config we need to the new format in a way that may or
may not work and does the other arbitrary dance steps that someone
decided to impose on us. Note that this now throws lots of warnings,
but I can't be bothered to fix them.
* 3p: Random package removals accomodated
* users/glittershark: Pin grfn's kernel to 5.9, because the CK patch
is not yet updated for 5.10
* users/glittershark: Update vendor hash for pg-dump-upsert, I suspect
this changed because of something in the Go build machinery in
nixpkgs. The deleteVendor flag also has no effect anymore and has been
removed.
* users/glittershark: agda build is broken, commenting out development
home-manager environment until it can be fixed
* third_party/haskell_overlay: updating random needs upper boundarles
of a few dependencies relaxed (curse them)
* third_party/gerrit_plugins: for some cursed reason the fixed-output
hash of the gerrit owners plugin fetchgit changed, updated.
Same for the checks plugin.
Change-Id: Ica37995fe8039d3ba80eab643867f98795c56734
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2295
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
This feature can cause object removal to happen while the git folder
is in use in Buildkite, causing CI to fail semi-reegularly.
Change-Id: Ide1a9b2f1761be029e97a058c1983b4cff5e27bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2285
Tested-by: BuildkiteCI
Reviewed-by: multi <depot@in-addr.xyz>
My new work laptop, a dell XPS 13.
Change-Id: Ieab06622c9b280182025edfa63adf649e5fc70d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2205
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Mugwump is too unstable for such an important internet service
Change-Id: Ic714200ce5ce51f366777f538b4a6f443f010960
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2124
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Add the depot.nix module and a depot config option to all nixos system
derivations that're build through the `bin/rebuild-system` machinery.
I can't imagine a scenario where we wouldn't want this level of
integration.
Change-Id: Ieeb98db2eee23919256adb4654bc45d540e055ec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2128
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Adds configuration options for the (inconsistently named) environment
variables that configure irccat integration with Panettone.
The defaults match the irccat setup on whitby.
Change-Id: I6857512a2e3f29f16777493eb981cc69ce3c045f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2080
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
This module configures irccat by creating a JSON configuration file
from a user-supplied Nix struct (this is not checked for correctness),
and merging it recursively with secrets from
`/etc/secrets/irccat.json` at service launch time.
This way we get the ability to configure (most) options declaratively
via Nix, while providing the secrets outside of Nix.
Side note: We need to figure out a secrets distribution mechanism.
Tested: Wrote a dummy config in whitby/default.nix locally and checked
that this builds, but I have not actually run the service yet. I
expect that some minor tweaks will end up being necessary.
Change-Id: I02a2e8dc40a7f8417fd77afcf8a12ac3df117988
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2074
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: glittershark <grfn@gws.fyi>
... I found this location in the logs, because the certs are now valid
for this, but I'm not actually sure if it's right.
Change-Id: I5ac88073e3bf6a95fead4c1d34515622c4416c6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2070
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Sometimes (like today) paroxysm crashes. We'd like it to restart if that
happens.
Change-Id: I98841096bcd6605c4279744ae5c65a9c92092a21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2069
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
The main bottleneck of our builds right now is Nix evaluation, which
means that most of the time is spent idling during builds.
Since we're evaluating in parallel, lets give it a few more builders.
I don't want to go all the way to 64 immediately to first see if we
get any adverse effects from highly concurrent builds running
concurrently (if we do we could group them into different "concurrency
groups" in Buildkite).
Change-Id: Ibc3f89fb59cb4ee471b152ff36887ffe2b39f8f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1932
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Allow 16 things to happen in parallel, which is useful now that the
CI granularity is on a per-target level.
Change-Id: Ie65dd119ea0666618fbb249613e70a68276db834
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1902
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
We have naturally evolved a distinction between logical and physical
targets.
Physical targets are those which correspond directly to a tree
location on disk and can be built with `-A path.to.files`, while
logical targets are those that are exported from within an expression
but do not have a corresponding file on disk.
This change adds support for exporting logical targets from any tree
location by adding a `meta.targets` attribute containing keys into
itself, which will be consumed by the CI target gathering logic and
included in the generated pipeline.
Note that the labels for subtargets are syntactically different to
emphasise that they do not correspond to a file location. For example,
this change enables 'ops.nixos.whitbySystem' as a subtarget, which is
labeled in CI as `ops/nixos:whitbySystem`.
Change-Id: Ied09647a62c2ba98e3914548e3742ad422c63ecf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1893
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Create the pipeline by outputting a file that contains nix-build
invocations for each target's *derivation path*.
Each invocation has a generated Nix expression passed to it with `-E`
which fetches the correct target from the tree while correctly
handling targets with strange characters (such as in Go-packages).
This makes it possible to run target-level granular pipelines. We're
getting somewhere!
Change-Id: Ia6946e389dafd1d4926130bb8891446d6e17133b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1855
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Auto-link b/\d+ in gerrit comments + patch comments to panettone issues.
This is a departure from the previous syntax of #\d+ that we were using
historically, but has been requested by multiple people
Fixes: b/40
Change-Id: I8e556eb6b2f55ae80d945b86b51041751ecb053b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1888
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This is a temporary state (TODO added) to be picked up by the new CI
logic.
Change-Id: Id4702740ffd18325088e2a8a0c6157a8cee7ccf7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1852
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This reverts commit 8fa3087067.
Reason for revert:
This almost worked. We discovered two important issues:
- The daemon startup does not correctly handle the socket passed in by systemd.
- There is some issue with chunking of large calls, running a build for ci-builds resulted in:
tazjin@whitby /depot (canon)> nix-build -A ciBuilds.__allTargets
E20200821 01:42:22.846053 12601 shared.cc:306] error: Rpc call addTextToStore to unix:///nix/var/nix/daemon-socket/socket failed (RESOURCE_EXHAUSTED): Received message larger than max (10889961 vs. 4194304)
Change-Id: Ic5ba4ef06a4953cf71a36b139fe25ea673cb6fee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1802
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
... this is going to break so much stuff. Lets have some fun.
Change-Id: If0185e0323391c7055d47b797083bb5afde57cb5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1829
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
