This uses tonic to generate the full set of gRPC clients for Yandex
Cloud. Includes some utility functions like an authentication
interceptor to make these actually work.
Since the upstream protos are exported regularly I've decided that the
versioning will simply be date-based.
The point of this is journaldriver integration, of course, hence also
the log-centric example code.
Change-Id: I00a615dcba80030e7f9bcfd476b2cfdb298f130d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8525
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
nixpkgs commit 134036f642a7f3ba9efeab509727c0989458b02b moved the
digidoc-tool binary to the `bin` output, so this wasn't actually
providing the digidoc-tool binary anymore.
Change-Id: Id5f7cc69d55b7cc058a6361512cc74de0e7bc1b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8487
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
* Satisfy new assert that the corresponding shell needs to be enabled
via programs.* if it is as the login shell of at least one user.
* //users/tazjin: “Address” removal of hardware.video.hidpi option.
* //3p/gerrit: update fetch sha256
Change-Id: Id0988a0ea7f393d6b7848a7104fc3526ee1177f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8407
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Instead of prepending :unsign to all URLs in josh-proxy, and for all
calls to filteredGitPush, explicitly use it only in the filter we use
for the `export-kit` extraStep.
This means, people cloning tvl-kit via
> https://code.tvl.fyi/depot.git:workspace=views/kit.git
now need to update the URL to point to
> https://code.tvl.fyi/depot.git:unsign:workspace=views/kit.git
instead.
git@github.com:tvlfyi/kit.git will keep the same hashes, as it's updated
to export the unsigned workspace view of it.
This is less invasive than dooming every josh workspace to have to strip
signatures.
Change-Id: I6de05182fad4c3695081388c3bbf37306521d255
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8369
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
I *want* search engines to index our CLs, they might be useful!
Change-Id: I956d92c80d812e1aefefb6daeba77a1588055b94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8361
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
We missed renaming this as well while iterating over
https://cl.tvl.fyi/c/depot/+/7950.
Change-Id: I704d3b60bb3beb1a2148e27bdd4a49075a6649b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8230
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
It's been a long time since we updated josh, almost 400 commits in
between. I read through the entire changelog, and here are relevant
josh commits from in between that might be interesting to us:
38eecee Fix optimisation bug for compose filter (#1159)
e1d10b6 Add :rev(...) filter
0f1a07b Initial implementation of refs locking (#929)
88cea2a Initial work on meta repo support
030ad93 Change magic refs to include "for"
28b1d75 Add split changes feature (#904)
1f908d7 Discover filters only on HEAD (#774)
a368d8f Make --require-auth only apply to push
8d80230 Add :linear filter (#741)
3460ec2 Implement redundant refs filtering (#700)
55b4e50 Implement stacked changes support (#699)
ea1f814 Handle @sha urls by creating magic ref (#690)
883a381 Run filter discovery only on changed refs (#685)
4bb004f Prepend refs/heads to base parameter as default (#664)
Of particular interest is a368d8f, which allows us to drop our
authentication patch and use the standard --require-auth flag again.
The default behaviour of dropping signatures on commits (which are
invalid after filtering) has also been changed in josh, now only
occuring when the `:unsign` filter is present. Since this breaks
commit hashes with our existing exported histories, we are opting to
set a `:unsign` filter prefix on all proxy requests to ensure that the
hashes stay consistent.
During this update we found a bug (josh#1155) which was fixed in the
commit that this CL moves josh to.
Change-Id: I3afac1619f3aa90313a0441da91f0e4a96fe0a3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8186
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This allows passing in custom triggers to trigger a (re)deploy.
For example, a caller can put an AWS instance ID into the triggers to
cause a redeploy whenever the instance ID has changed.
The `target_name` terraform variable was doing something similar, but
`triggers` is more generic, allowing multiple triggers, without having
to stringify them.
We also don't need to trigger on the attrpath - it can be changed, and
as long as it still evaluates to the same
`data.external.nixos_system.result.drv` (which is checked on every
plan), no redeploy needs to be made.
Change-Id: I94ce787a50830b87b6f53c08e042e4abe4036bdd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8191
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
This adds an additional parameter `entrypoint`, pointing to a .nix file
(or a directory containing a `default.nix` file) that's providing the
attribute path asked for.
If not set / kept at the default (empty string), it falls back to the
root dir of the repository as before.
Change-Id: I2e63114f21660c842153ac15424b3491d66624d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8190
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This module makes it fairly easy to deploy NixOS system closures using
Terraform, while properly separating the evaluation of a
derivation (to determine whether a deploy is needed) from the building
and copying of the closure itself.
This has been on my stack for a while. It was originally developed for
Resoptima, who agreed to open-sourcing it in depot back when we
completed our work with them. Their contribution has been acknowledged
in the README.
Co-Authored-By: Florian Klink <flokli@flokli.de>
Change-Id: Ica4c170658cd25f1fb7072c9a45735fcc4351474
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7950
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Whitby has a lot of memory, but I've still been fighting with the OOM
Killer trying to build a few big packages at the same time. Besides,
it's generally a good idea to always have swap available even if
there's lots of memory for caching optimisation reasons[1], and zram
swap is efficient enough to basically provide bonus memory for free.
[1]: https://haydenjames.io/linux-performance-almost-always-add-swap-space/
Change-Id: I1fbe60f7975ebfa38e341e0de76848ec79b6fcf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8065
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This way, I won't have to teach my name one at a time to every program
that wants to know my it (e.g. git).
Change-Id: I45ddd9c2343a10cd4c13bacd9a97b7470db95c14
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8038
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
It looks like this needs to be set for the tvix pipeline to succeed.
It was set to `canon` for `tvl-kit` (not sure if manually, or some
autodetection previously did it for us that's not present anymore).
Anyways, this sets it to how it's set in the web interface, to hopefully
fix it.
Change-Id: Ic3eb60e3f421fa949a84dcdaa928823ff45f679a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8008
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Keycloak seems to have decided today that it will now send headers
that are larger than what the nginx default configuration can handle.
The numbers are a mix of made up and taken from random nginx voodoo
posts on the internet, so they're as good a guess as anyone's.
Change-Id: If037bcba48eee371cc96304b150276c669930c75
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7992
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Changes basically only include dependency bumps. This is r/5656.
Change-Id: If2ad8914c45b61de6525e2640a15d167fef1dfd4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7819
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This bumps all dependencies to their newest version that does not
require code changes.
Change-Id: I7c7f01ce08de0cced86bab93b441327d3061f12d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7818
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This fix can only be applied after the upstream public-inbox
fix (https://github.com/NixOS/nixpkgs/pull/207693) has been merged.
Change-Id: I957473e2895b7e57baad25c9e908b36aa790f3a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7636
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
I think what might be going on with b/231 is that the annotations
somehow started conflicting because they don't have contexts set.
Lets try setting a context and see if it changs anything ...
Change-Id: I62ed57f9e24f08e4e7215f05d35cfa769e2e2c24
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7640
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This landing page explains how to use the public-inbox.
Change-Id: I37d74decad5173ab35051970593f1d28001af2b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7645
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This sets up IMAP on inbox.tvl.su:993
I added a hack to work around problems with the NixOS ACME module.
Spent way too much time of my life with problems with that module, so
I only use it with blunt force these days. Others are welcome to make
a cleaner solution.
Change-Id: Ice828766020856cf17d2f0a5b4491f4cec8ad9b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7633
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is the new address which leads to the public inbox at inbox.tvl.su
Change-Id: I45d98a373b8acda49b05c4f74669ffb9ad1f1a3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7632
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
On the machine running public-inbox, this will start automatically
fetching mails from depot@tvl.su and making them available to
public-inbox.
Change-Id: I2469207bd41d64eba747a74ae5fda9fed548cc83
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7630
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Initial setup which does not yet include fetching mails at all, this
is for now only going to display a manually populated view of the
existing mailing list while the rest of this stuff is set up.
Change-Id: Ie1235bd257c9056fe37d0740dfca771ebdd880eb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7628
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
When pushing a large chain of CLs, builds can fail with OOM issues as
many Nix evaluations of the depot are happening simultaneously.
To work around this, we limit the concurrency of simultaneous Nix
evaluations (i.e. the `:llama` step). This can slow down the start of
builds in a large chain of small changes, but that is a better
tradeoff than failing the builds entirely and making people click
buttons.
Change-Id: If351aaad22d52e2bcf871377f22ab1df594c518d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7501
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This is a follow up to cl/7191 which neglected to adjust the
status.tvl.su.nix module and re-enable it.
Change-Id: Icc1917004cd50e5eab61a29bc68b393ba9bd6325
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7226
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Uncomment and update the grafana config for whitby based on the new
config format that nixos accepts. I've validated this locally by
visually inspecting the resulting `ini` file, but not actually run it
yet.
Change-Id: I12d78ae48146e1b01bd2a4152276d4c6b16c1a3d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7191
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
* //ops/machines/whitby: Disable grafana, since the grafana module was
changed upstream in a way that our configuration no longer works.
Since the OpenSSL security update is relatively pressing, adapting the
grafana configuration beforehand is not a hard requirement. See
https://github.com/NixOS/nixpkgs/pull/191768.
* //tools/depotfmt: keep Go at version 1.18 to forgo a reformat of the
tree.
* //nix/buildGo: keep Go at version 1.18, as 1.19 changed the CLI
interface (?) in a way that breaks buildGo.
* //3p/overlays/tvl: drop upstreamed tdlib upgrade.
* //3p/overlays/tvl: patch buf to work around breakage due to git 2.38.1
TODO items for Go are tracked in b/215.
Change-Id: Ie08fef49cf3db12e6b5225a8b992a990ddc5b642
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7141
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
This is already allowed de facto, since there seems to be a special
exception for reading from derivation outputs. What is forbidden, is
access to files imported to the store (even via builtins.toFile) and
derivation files. The latter is required for doing dependency analysis
on arbitrary derivations, unfortunately.
Access to the store allows kind of evil things, but it should
be (hopefully) hard to do this by accident, and accessing derivation
files is not impure, though it relies on store implementation internals
so to speak.
Change-Id: I33a7de83ef0ee20a7076690329d62f6caffffe5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6835
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Upstream nixpkgs removed a lot of aliases this time, so we needed to do
the following transformations. It's a real shame that aliases only
really become discoverable easily when they are removed.
* runCommandNoCC -> runCommand
* gmailieer -> lieer
We also need to work around the fact that home-manager hasn't catched
on to this rename.
* mysql -> mariadb
* pkgconfig -> pkg-config
This also affects our Nix fork which needs to be bumped.
* prometheus_client -> prometheus-client
* rxvt_unicode -> rxvt-unicode-unwrapped
* nix-review -> nixpkgs-review
* oauth2_proxy -> oauth2-proxy
Additionally, some Go-related builders decided to drop support for
passing the sha256 hash in directly, so we need to use the generic hash
arguments.
Change-Id: I84aaa225ef18962937f8616a9ff064822f0d5dc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6792
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
For some reason Terraform decided that it would otherwise like
to *delete* this configuration, which is undesirable.
Note that there is a "magic" special behaviour when the `alias` and
`provider_id` are set to the name of a built-in supported
provider (github, gitlab etc.), which lets us skip the
authorization_url setup.
Change-Id: Ib66154c2896dda162c57bdc2d7964a9fa4e15f20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6706
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
I think these were set up in the UI and previously not supported in
the Terraform config, now they're supported and Terraform wanted to
delete them ...
Change-Id: I83eb49ceb774ac835dc81638f962e937c7e936c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6707
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: lukegb <lukegb@tvl.fyi>
I could not get nginx to serve it from `tvl.su/tvixbolt`, and ran out
of interest in trying to fix it, so lets put it on a subdomain
instead.
Change-Id: I1313d75cc9831d94a894191376534b1e5186a76a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6537
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
otherwise posting this to reddit's /r/russian is not possible, as they
ban all links to Russian-affiliated sites
Change-Id: I8d23f0961ec7ef097fc2dbdd0aaa178861a19c10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5992
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
instead of setting a wildcard record (which causes really weird
behaviour if you set your search domain to tvl.su/tvl.fyi, which I
do), DNS records for services running on whitby are now set
explicitly.
Change-Id: Ia05399b62dad326942fe0efda30782ce153df99d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5961
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
at some point in the far past, there was an RSS feed at `/en/rss.xml`.
It seems to still get a single hit or so every hour, which currently
404s.
Change-Id: Ieb13c2c0232861a50a54bc2a4087d9ccb21185cf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5962
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
on whitby, cgit runs as the gerrit user to get access to serving
gerrit's repositories directly.
on other machines (e.g. sanduny) this isn't necessary, as we have a
world-readable depot replica.
Change-Id: Ibf7e7cc08e5909e0fa182e561ab0cb472188edcb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5932
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
this configures gerrit's built-in replication plugin to push every
change in depot to sanduny.
this allows us to serve a replica of depot from sanduny.
manual config that was needed which needs to be automated:
* system-wide known_hosts does not work, needed one in /var/lib/git
* .ssh/config MUST be present and configured for sanduny.tvl.su
Change-Id: Iba399f2328abb5acb65dae19a36e265eea0952ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5915
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
This module sets up a user with an SSH key and permissions to receive
a (pushed) replica of depot from Gerrit.
This still needs appropriate configuration in Gerrit's replication
plugin on the other end.
This module has been enabled for sanduny. For now it does not (yet)
configure git serving.
Change-Id: I0fb6f7e696609e71008308e855bdf305dcbcd4f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5913
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
It occured to me yesterday that with the config inside of the module
it is kind of difficult to test cgit locally.
This moves it back to a separate location (//web/cgit-tvl) and makes
the most important things configurable via overrides.
Change-Id: I9b0f4c60b75c31441e1718e63b5b55aba3100aae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5893
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Before this commit besadii only worked for repos having
'refs/heads/canon' as main branch.
Change-Id: Ia2ceb8a720c675be84bc3d81b89338522cea6ebd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5862
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: asmundo <asmundo@gmail.com>
Remove a workaround for a GleSYS provider bug that was fixed in the
last release.
Change-Id: Ibd25de0b4dcccd781518d5d0ae1c75d296f6b05f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5845
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
In order to run this the secrets needs to be sourced, e.g.:
eval $(age --decrypt -i ~/.ssh/id_ed25519 $(git rev-parse --show-toplevel)/ops/secrets/tf-buildkite.age)
Change-Id: I9f6a02c0dac22f584181635861ddbb06cf849f14
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5838
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
This is in preparation for a subsequent CL that will do much more
significant changes in //nix/buildkite.
Change-Id: I80a8d67d3a7d593854c8d711572483c2581e7881
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5824
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
This is a less invasive way to achieve the same goal as cl/5681, by
preventing the already existing nixpkgs store path from being dumped
again at the call site. To support nixpkgsBisectPath, we simply check if
pkgs.path is below builtins.storeDir and use builtins.storePath based on
that.
This is actually similar to the approach taken in the nixpkgs
documentation system which tries to limit the amount of nixpkgs that
needs to be dumped by using filterSource on specific subtrees of
nixpkgs. For this to work it has to insist on pkgs.path being an
ordinary Nix path, though.
Change-Id: Idf892f90a5d811184568e4702a901c334d56210e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5787
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Buildkite can't handle more than one filter for the query; as of the
last commit it just returned an empty list.
I've verified with curl based on the request the previous attempt
constructed that this works as intended with only setting the commit.
Behaviour is probably undefined if there are two builds for the same
commit (i.e. a retry). Which one will you see? Who knows!
However, since the commit hash contains the Change-Id, we can't get a
situation where the build was for two different CLs at the same
commit. Gerrit wouldn't allow that.
Change-Id: I0dcd0ff44c28d3d15cba23461970bfc8483f4e48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5768
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
