Commit graph

128 commits

Author SHA1 Message Date
Vincent Ambo
ac6717fe3c fix(ops/modules/www): Make self-redirect to config a generic module
As suggested by sterni, this makes the self-redirect of a machine to
its configuration a generic module working by convention.

In the process of moving this two small fixes have been applied:

* redirect is only applied if the URI is `/`, this is required for
  ACME to work
* addSSL = true is added, otherwise we have a certificate but no TLS
  listener

Change-Id: Icaef041ff681253a61e36926417bdb2844e3f93d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5313
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2022-02-18 11:39:01 +00:00
Vincent Ambo
5b701ad713 feat(sanduny): Enable journaldriver module
Change-Id: I9026386664000448642ff635bd71a7af5ed546c3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5303
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:38:34 +00:00
Vincent Ambo
536e01e967 refactor(ops/modules): Move journaldriver configuration into module
This makes the journaldriver configuration machine-independent.
The secret is loaded from agenix instead of being persisted on disk.

Change-Id: I592ae7f5726fcb7f37a406f69dcf5ac498eeb1b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5302
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:38:34 +00:00
Vincent Ambo
c72abe04f2 feat(sanduny): Configure Bitfolk nameservers
Change-Id: I81b252aedbf1ce3543a167b6c1942c404d4f1f1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5312
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 11:35:21 +00:00
Vincent Ambo
95780174e1 feat(ops/machines): Add a module for known SSH keys
Change-Id: I443e479f3edf9c6540de7b5a33bc6f7e2a9c5183
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5305
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-18 08:22:56 +00:00
Vincent Ambo
b936843bb0 feat(ops/modules): Redirect machine base names to their config
With this change, entering just "whitby.tvl.fyi" or "sanduny.tvl.su"
in a browser will redirect users to their machine configurations.

Change-Id: Ibf076a469bcce073e1b1970aa568d6fe16a5c75a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5304
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-18 08:15:56 +00:00
Vincent Ambo
f4f1d97052 refactor(ops/modules): Move ACME base configuration into base.nix
This needs to be present on all machines that run ACME stuff.

I've switched the address for a .su one because I have a catchall for
these.

Change-Id: I7af8e1f1cb2fcfbcba4b7d1930ed0edef0106d72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5306
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-18 08:15:56 +00:00
Vincent Ambo
e1353ff2cf feat(ops/machines): Add tvl-users to sanduny.tvl.su
Change-Id: I20f54f4ab298cfee91062f7bf4cdc8b0b3ccb37c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5299
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-17 18:11:58 +00:00
Vincent Ambo
9e9284fc86 feat(ops/machines): add configuration for sanduny.tvl.su
This will be an additional web host / fallback git server for whitby
incidents.

Change-Id: Icd6f7ce574ffd520b5783a50ff317feed7b71fc6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5297
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-17 18:11:58 +00:00
Vincent Ambo
dd5ce78dbd refactor(ops/modules): Move user configuration into module
Rather than defining all system users inline on whitby, move them into
a module that can be imported on multiple machines.

Configuration for terminfos that we've added follows along.

Note that while doing this I've disabled logins for riking and isomer
since they are currently inactive in TVL.

Change-Id: Id18031d355afc34079c5e6e49dc6943e61809a8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5298
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2022-02-17 18:08:19 +00:00
Vincent Ambo
c72c1efdeb refactor(ops/modules): Rename git-serving -> josh
cgit has its own module now

Change-Id: I9b4cc322374517b8bd3db43345831e2bf43c4bb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5295
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-16 23:03:06 +00:00
Vincent Ambo
cb8f050b9c refactor(ops/modules): Move cgit configuration into a module
The ancient `//web/cgit-taz` path stems from the time I had
code.tazj.in serving my initial version of the depot.

I've been meaning to clean this up for forever, so here we go.

Note that this leaves the git-serving module in a strange state where
it only deals with josh. I'll rename it accordingly.

Change-Id: I47ed1e9d90958299b5440a18a1b9075274754e33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5294
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-16 23:03:06 +00:00
sterni
235eda6649 chore(3p/sources): Bump channels & overlays
* //nix/buildLisp: re-enable CCL, as the crash has been fixed upstream,
  although it is unclear what exactly caused / fixed it.

* //ops/whitby: the kitty build broke upstream, so we can't install the
  terminfo on whitby for a bit.

Change-Id: I5710acbe837fbc936e334b2e81f9cf00ed6ae280
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5274
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-02-13 09:15:09 +00:00
Vincent Ambo
aa122cbae7 style: format entire depot with nixpkgs-fmt
This CL can be used to compare the style of nixpkgs-fmt against other
formatters (nixpkgs, alejandra).

Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: cynthia <cynthia@tvl.fyi>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: eta <tvl@eta.st>
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-31 16:11:53 +00:00
Griffin Smith
5f42a160be fix(ops/whitby): enable services.nginx.statusPage
This is necessary for the nginx prometheus exporter to work

Change-Id: I2343d6f5d3d6d6772777d5e14426a537aa1c8ef7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5127
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
2022-01-29 23:07:00 +00:00
Griffin Smith
84304b925a feat(whitby): Enable nginx prometheus exporter
Might be nice to look at rates of requests etc.

Change-Id: I4d12ab0c1a555793e803de4a9614e616951a94e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5125
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
2022-01-29 23:00:25 +00:00
Vincent Ambo
bce5192566 feat(whitby): Install a handful of systemPackages
Adds more things I keep using via nix-shell, as well as the
deploy-whitby script (which is independent of a particular depot
checkout).

Change-Id: I36f87de7645768a05268c90ba9b3ab833bacca05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4881
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-13 08:39:03 +00:00
sterni
d99132f45a fix: resolve remaining security.acme.email warnings
These were missed in cl/4784.

Change-Id: I01a5827900c1b3bdfdf9b1c36dcca8d6b59073a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4866
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: sterni <sternenseemann@systemli.org>
2022-01-12 18:30:34 +00:00
William Carroll
1e730c859d fix(wpcarro/all-systems): Remove diogenes from my top-level systems
When `findSystem` attempts to evaluate `system.config.networking.hostName`,
diogenes (because I've refactored its definition) causes the following error:

> You're trying to declare a value of type `string'
> rather than an attribute-set for the option
> `system'!

Change-Id: Ib23cb9aa9cadc1f71ad3369c903e587762d12cc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4830
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
2022-01-08 10:04:00 +00:00
Vincent Ambo
73288ba569 feat(ops): Add initial oauth2_proxy configuration
The intent is to configure oauth2_proxy pointing at Keycloak to enable
usage with nginx auth_request directives.

I want to expose this as a function from within the module in which
nginx server configuration blocks can be wrapped, but the function for
that is currently a placeholder.

Change-Id: I5ed7deb9bf1c62818f516e68c33e8c5b632fccfe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4767
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-04 18:04:27 +00:00
Vincent Ambo
3806cea40b chore(ops): Remove login.tvl.fyi module
It looks like we won't need this for oauth2_proxy when combined with
nginx auth_request setups.

Change-Id: I2294aee6226b4f64a27bf6592c2d18092d0268cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4766
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-04 18:04:20 +00:00
Vincent Ambo
2bf39d7101 refactor(modules/smtprelay): Load credentials via agenix
Change-Id: I56f6887e1fd35551cfc83ad08cafebb611f4a341
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4760
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: tazjin <mail@tazj.in>
2022-01-01 15:30:17 +00:00
Vincent Ambo
906d6553c6 fix(whitby): Point grafana at new auth provider
Grafana was still pointing at the (now non-existent) CAS setup. This
changes the endpoints to use Keycloak instead and updates the client
secret.

Change-Id: Ib25d38330aba2ef6d894e8c33d86852c884ab5be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4706
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 14:44:38 +00:00
Griffin Smith
9b3374b091 feat(ops/machines/all-systems): Add grfn/mugwump
Change-Id: I7770b58c44a5700e86c80d1058e89e9fa65d719b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4686
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
2021-12-26 21:33:36 +00:00
Vincent Ambo
28a80e0251 chore: friendship ended with cas, now keycloak is our best friend
Note that the login.tvl.fyi WWW configuration is still kind of hanging
around until we've settled where Keycloak lives.

Change-Id: Iaca4e394a7371cafa3716ca66ef09c4eca5b1520
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4626
Autosubmit: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-26 00:10:10 +00:00
Vincent Ambo
e4d20cdaec refactor(ops/whitby): Move Gerrit secrets into agenix
Gerrit has OAuth2 and email related secrets which now live in agenix
instead of a random file on disk.

Change-Id: I6220fbb7a2e2ec0102a900b4bcf6150b8b4d32ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4612
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-25 21:15:41 +00:00
Vincent Ambo
d8a1802b3e feat(whitby): Configure initial Keycloak setup
Trialing this as an alternative to CAS that is a little easier to
configure and can help us delegate authentication to other OIDC
services.

Change-Id: Iad63724d349334910af8fed0b148e4ba428f796b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4608
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-25 21:11:40 +00:00
William Carroll
06a20ba3df fix(ops/diogenes): Ensure diogenes builds
diogenes "passed" CI because the file was named configuration.nix
(vestage from the NixOS default /etc/nixos/configuration). This CL fixes
some issues I encountered after running depot/bin/rebuild-system.

TL;DR:
- rename configuration.nix -> default.nix to trigger CI
- add diogenes to my systems
- add public SSH key

Change-Id: I24197b8936c201267db6f71f00099dce590eac1d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4388
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
Autosubmit: wpcarro <wpcarro@gmail.com>
2021-12-17 22:12:40 +00:00
William Carroll
1ca4c6d2be feat(wpcarro/marcus): Add marcus to the depot
me: marcus, meet depot. depot, meet marcus.
Change-Id: Ic6a25ac85e4c7f6dfea2a42b46a4400f92df70a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4351
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-12-16 22:07:02 +00:00
Griffin Smith
d85c6a1b56 feat(ops/modules): Provide some modules to all nixoses
For modules that are gated behind a mkEnableOption, it's reasonable to
just provide them to all Depot-built nixos systems without requiring
people to explicitly import them. This defines a special module called
`default-imports.nix` which imports these modules (currently just
tvl-cache.nix and automatic-gc.nix, as I'm being rather conservative
adding things here to avoid breaking anyone's system), then provides
that module as one of the `modules` passed at the top-level
nixos/eval-config invocation.

Change-Id: I3be299ab10ae4c451ef11c514edb3c89318a2278
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4345
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2021-12-16 14:17:35 +00:00
sterni
fe0e19ead4 feat(ops/whitby): install alacritty terminfo
alacritty is used by grfn atm.

Change-Id: I10dacd301044f9c37790e22e955cb068fcbd2cfc
2021-12-13 19:40:28 +00:00
sterni
cdf7480662 feat(ops/whitby): add terminfos for other terminals used
* foot (me)
* kitty (lukegb)

Change-Id: I65303e39c4adb05e362792a544134fc2884175bf
2021-12-13 12:56:12 +00:00
Vincent Ambo
961443c23c feat(whitby): Add some more useful programs
I keep using these in nix-shell but really they should just be
installed.

Change-Id: Ic2c36bae8b582fef88029b288accdfd3c8bc0f1b
2021-12-13 15:48:41 +03:00
Vincent Ambo
3a410a78df feat(ops/secrets): Make (encrypted) secrets part of the tree
Currently in NixOS configuration using agenix secrets there is no
build time validation of secret paths - things fail at runtime (system
activation).

To prevent that, this CL makes the secrets part of the tree based on
the same configuration file used by agenix itself.

This guards against:

* agenix secrets.nix definition for a non-existent file
* age.secrets value in a NixOS config for a non-existent secret

Change-Id: I5b191dcbd5b2522566ff7c38f8a988bbf7679364
2021-12-12 11:19:24 +03:00
Vincent Ambo
2f1c654c14 refactor(ops): Move panettone secrets to agenix
Relates to b/161

Change-Id: I508e5a0eacab668f4bd39a2c888d894b96bed093
2021-12-10 23:19:56 +03:00
Vincent Ambo
60f96d2b17 fix(whitby): Fix typo in buildkite-agents group name
... really would like some assertion helpers for this sort of stuff.

Change-Id: I32d1de18ebfbbdfa5128a8fbdad2efcc511f8514
2021-12-10 23:01:20 +03:00
Vincent Ambo
2fe8d724d7 refactor(ops): Move Nix cache secret to agenix
... and also the public key, just to keep the distribution mechanism
the same.

Change-Id: Ief14daf9344c0fb99eeb5789c1ec9bfb1f12bee0
2021-12-10 19:48:26 +00:00
Vincent Ambo
82a885a750 refactor(ops): Use besadii configuration from agenix
We already checked this in, but this commit adds the configuration for
making use of it.

There are two copies of besadii's JSON configuration with different
permissions.

Note that the buildkite-graphql-token path needs to be updated in
static-pipeline.yml, but this needs to happen in a separate commit
after deploy because the pipeline will break otherwise.

Change-Id: I6fab4bf1a2e679df7cf76521e2b53bd9dadbac62
2021-12-10 19:31:36 +00:00
Vincent Ambo
b1108821a9 refactor(ops): Move grafana secret into agenix
Change-Id: Id141758135c796881e91d20b950dae74c40d9ab3
2021-12-10 19:31:36 +00:00
Vincent Ambo
b8267c261c fix(ops/irccat): Avoid permissions issue with LoadCredentials=
The DynamicUser + Group configuration does not work as planned, thus
the systemd LoadCredentials feature is used instead which makes the
file (which itself is only readable by root) available in a
memory-backed location only readable by the service.

The secret is only available to `ExecStart` commands, so units using
this feature can not be used with pre/post units and the like if those
commands need secrets.

To accommodate this, the merge of configuration files has been moved
into the service launch script, which is now the ExecStart= process.

For details take a look at https://www.freedesktop.org/software/systemd/man/systemd.exec.html#LoadCredential=ID:PATH

Change-Id: I693fe5677cc0d63c7aa485c2c7472457c5262166
2021-12-10 15:09:09 +00:00
Vincent Ambo
d4403638cf refactor(ops): Move irccat secret into agenix
The irccat module uses DynamicUser, so to grant permission to it a new
group has been added for irccat.

I have some vague memory of DynamicUser + Group not behaving as one
would expect, but we'll see what happens.

Change-Id: Iab9f6a3f1a53c4133b635458ce173250cc9a3fac
2021-12-10 16:13:31 +03:00
Vincent Ambo
002d183876 refactor(ops): Move clbot SSH key into agenix
Change-Id: Iae03ead7dda0509689a76f0d76f9cfeb8434e967
2021-12-10 16:13:31 +03:00
Vincent Ambo
811e6d7d9f chore(whitby): Remove shadowsocks service
No longer required on whitby.

Change-Id: I93951c6b708eae81ddb03df920a4068c1ccde9e7
2021-12-10 13:07:09 +00:00
Vincent Ambo
9ea4d55d81 refactor(ops): Move buildkite-agent-token into agenix
Relates to b/161

Change-Id: I5d3a698d437928966d8b78ce9e0ba226c1437655
2021-12-10 10:32:44 +03:00
Vincent Ambo
a123b9e0a2 refactor(ops): Move owothia secret into agenix
Relates to b/161

Change-Id: I25445281b0dd3c3f3660f8bb0d8337506a1e427b
2021-12-10 10:32:14 +03:00
Vincent Ambo
78744c00f5 refactor(ops): Move clbot secret into agenix
Relates to b/161

Change-Id: I7badf22ff93bb4e8b06e4dd4a8bf880b0bd48f09
2021-12-10 10:32:14 +03:00
Vincent Ambo
496d899428 feat(ops/secrets): Configure secrets for gerrit-queue
Adds a systemd EnvironmentFile secret that contains the Gerrit
username & password for gerrit-queue.

Change-Id: I25acf87764c26774045138402b8a417b6813ee8f
2021-12-10 10:32:14 +03:00
Vincent Ambo
4870b1a2ff feat(ops/modules): Add module for running gerrit-queue
This is not yet including the secret configuration for gerrit-queue,
and just expects the secret (gerrit username & password) to be
available in /etc/secrets.

Change-Id: Ia465ef7f3f521c70d606d7fdeba9aa83c7e1b98b
2021-12-10 10:32:14 +03:00
Vincent Ambo
15cb37f877 fix(ops/restic): Move whitby's backup to GleSYS object storage
Since GCP nuked us, the backups are now moving to GleSYS'
S3-compatible object storage.

This refactors the restic module to support S3-compatible storage
instead of GCP, and switches to the appropriate new secret paths.

The secrets were placed on whitby manually and I verified that the
backups work.

This fixes b/157

Change-Id: I6a9d2b0581967605ce736605a3befb44cdeae7e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3883
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-11-21 12:01:26 +00:00
Vincent Ambo
0eef0e343f feat(whitby): serve static.tvl.{fyi|su} with max cache settings
The setup is explained in the comment, but TL;DR: Use the derivation
hash of static files to create permanent URLs.

Relates to b/151.

Change-Id: Ib1ca3a1a00c90a47f4bf39c29a8b4bbf5b215e7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3664
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-10-01 20:45:50 +00:00
Vincent Ambo
ce575bf65b feat(whitby): Serve //corp/website on tvl.su
Change-Id: I21e1ddf9a32568cac8ad2595869ac8670867efa9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3658
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-10-01 15:24:35 +00:00
Vincent Ambo
0e3858b5e5 refactor(whitby): Move restic path configuration into modules
This lets each service declare their backup paths together with the
configuration for the service, which is a lot more sensible than what
we had before.

Fixes b/147

Change-Id: If76fe62639f4cc0e6fbb63a2959d584479d8f0fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3583
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-18 15:10:34 +00:00
Vincent Ambo
387c55582b refactor(ops/restic): Move restic configuration into a new module
Relates to b/147.

First step towards giving depot modules the ability to declare their
own backup directories by moving all restic configuration into a new
module and adding a NixOS option for inclusion/exclusion paths for
backups.

This still keeps all backup paths within the whitby config.

Change-Id: Ia96833668f1a3d02da892261153d8b02156b8ac0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3565
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2021-09-16 20:34:05 +00:00
Vincent Ambo
ec38839c33 feat(git-serving): Configure josh to serve the depot over HTTP
Previously we served the dumb git HTTP protocol from code.tvl.fyi via
cgit. This CL disables this feature and instead runs josh in the same
location (by redirecting appropriately), but while also enabling
partial cloning of all subtrees of the depot.

For example, after this CL the following would result in an
independent clone of //nix/readTree:

    git clone https://code.tvl.fyi/depot.git:/nix/readTree.git

Note that there are no josh workspaces configured at all for now,
these references are only for static depot subpaths.

Please refer to the documentation for josh for more information on
available kinds of josh filters.

Josh state is kept in a systemd state directory in /var/lib/josh and
backed up to Restic. Backing this up is necessary, as josh uses
stateful information to do things like tracking merges and rewriting
history per subtree appropriately to avoid cloned repositories ending
up in peculiar states.

Change-Id: I156f0298c2aa42e3bdbf5a0e86109070d640c56e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3563
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2021-09-16 20:34:05 +00:00
Griffin Smith
79b39bb66e feat(whitby): Serve static HTML dir for deploys.tvl.fyi
Add a new domain and nginx virtual host at deploys.tvl.fyi, serving out
of a static directory on whitby which is created by systemd-tmpfiles.

This will be used to serve diffs rendered by nix-diff for
pending deploys for whitby

Since this contains stateful data, it is added to the restic backups
on whitby.

Refs: b/110
Change-Id: I5869d40800bbf5fb8fb39878a857f66ff5787830
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3144
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-09-10 14:37:36 +00:00
Griffin Smith
d857d5ad68 refactor(gs/system): Remove chupacabra
This machine no longer exists

Change-Id: I8e549b8397777a01404bd84c10c195e80f281744
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3431
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 19:41:42 +00:00
Vincent Ambo
f218f2fd56 feat(ops): Serve nixery.dev from whitby
Adds a new module for the nixery.dev domain and serves it from whitby.
Note that the DNS records do *not* point to whitby yet, so deploying
this will lead to a failed TLS provisioning unit - but this is
intentional.

Change-Id: I911f67a0aa24f8df3cb52d2cfc49a8b6132cf718
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3383
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-24 11:30:16 +00:00
Vincent Ambo
7c16a71156 feat(ops/www): Point images.tvl.* at Nixery
Change-Id: I39f979c68e7b74f6da6a7da0f07aaa470886d451
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3346
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-13 10:57:53 +00:00
Vincent Ambo
47409b9610 feat(ops/modules): Add module for running Nixery
This sets up a very simple Nixery instance with some things lacking:

* no support for garbage-collecting image fragments (yet)
* no popularity setup

The plan is to use this to get the ball rolling on a separate
domain (e.g. images.tvl.fyi), iron things out and then look into
flipping over nixery.dev

Change-Id: Ic594809f9d487fec7a0f632d608752a3f9c61315
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3280
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-12 14:55:59 +00:00
Vincent Ambo
65be8f20e0 chore(nixpkgs): Bump channels to 2021-05-25
* users/grfn/system/home/yeren: remove obsolete awscli2 overrides

* ops: make new isSystemUser || isNormalUser assertion happy

* users/grfn/system/system/mugwump: make buildkite agents system users

* users/tazjin/nixos/camden: set isSystemUser = true for git

* users/tazjin/emacs: Remove missing & broken packages

* third_party/openldap: remove, as the argon2 module is now enabled upstream

* third_party/gerrit_plugins: Pinned new unstable hashes

* third_party/nix, third_party/grpc: Disabled CI as these are broken

* third_party/overlays/emacs: Bumped version to stay in sync with channel

* third_party/buzz: Update LIBCLANG_PATH to reference libclang.lib,
  since libclang's default output no longer contains libclang.so

* users/grfn/system/home: Install julia-stable instead of julia (which
  aliases to julia-lts), as the latter depends on an insecure version of
  libgit

Change-Id: Iff33b0ecb0ef07a82d1de35e23c40d2f4bf0f8ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3001
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2021-05-25 17:09:28 +00:00
Vincent Ambo
878957d2f6 chore(whitby): Add ZNC state to Restic backups
Until we have declarative ZNC config (which requires a solution for
secrets handling in it), make sure we back this up as well.

Change-Id: Idb186327da171eb6d3dbbd83801639f1f9321a40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3159
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-05-25 08:15:19 +00:00
Vincent Ambo
4b584e5259 fix(whitby): Fix irccat configuration for incorrectly named option
irccat is passing the realname option as the ident of the user, which
doesn't match what is in ZNC.

It hasn't seen any upstream commits in a long time, so I'm just
leaving this as is and fixing it locally in our config.

Change-Id: I3bf865f37b8df9c1cd891a94245ca3fad376bbe1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3150
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2021-05-23 20:34:10 +00:00
Vincent Ambo
9bb8736a45 feat(whitby): Let sterni bear the wheel
Change-Id: Ib4f7dcbdc754d2fc271f501a9ea270e983a3645f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3147
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-05-23 19:06:15 +00:00
Griffin Smith
75f19a05a1 feat(whitby): Enable fail2ban
I like running fail2ban on any machine that has stuff like ssh
world-open, to limit the potential for password brute-force attacks etc.

Change-Id: I0c60811ae5a2fddb44f04679fb455e646b8e39c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3138
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-05-23 12:40:00 +00:00
Vincent Ambo
687f978b1c feat(ops/owothia): Add owothia module and deploy on whitby
This configures owothia to use her new bouncer to HackInt.

Change-Id: I80eb8191c2b0f2a6f8a31d19b60250ade27c1913
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3129
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-05-22 19:40:45 +00:00
Vincent Ambo
814feed0ce chore(whitby): Move clbot to HackInt
Points clbot at the new local ZNC instead. This will make it part of
the things happening through the `tvlbot` account.

Relates to b/101

Change-Id: I1c15ffa5720d3af34475c15bee3fdaa537ac659b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3127
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2021-05-22 17:57:32 +00:00
Vincent Ambo
1aad1d9beb chore(whitby): Move irccat & panettone notifications to HackInt
Change-Id: I6bd5c183d2c1c28b8c6b0201bdf22a66333d4aea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3131
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-05-22 17:52:01 +00:00
Florian Klink
48b052c1e4 feat(whitby): Add shadowsocks server
This adds a shadowsocks service, running on port 8443, tcp and udp.

The password is read from /etc/secrets/shadowsocks-secret.sec, and needs
to be populated externally.

Change-Id: I6797150db108ba14459502dee43d8e4ed6cfa910
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3125
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-05-22 13:28:36 +00:00
Vincent Ambo
5036ae4376 feat(whitby): Initial ZNC configuration
Bouncer to be used for TVL's IRC bots, see b/101

Change-Id: Ic9f71ecd94365d3baa31e0552b1ce16362f94557
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3124
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2021-05-21 11:30:42 +00:00
Vincent Ambo
7926482f1c feat(ops/www): Configure atward.tvl.fyi and its aliases
Change-Id: I20dfb057f8184899226bcb4527010a6982d426f0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3094
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-05-05 09:02:58 +00:00
Vincent Ambo
790c0d938e feat(ops): Add NixOS module for atward
Very standard, nothing fancy.

Change-Id: Ibb286f221a4752abfb62e971b98e9496357040f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3090
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2021-05-03 23:47:30 +00:00
Vincent Ambo
cc903bc3b0 feat(ops/modules): Add module for automatically collecting garbage
Adds a module that automatically collects garbage based on disk space
thresholds, and configures it to run hourly on whitby.

This is implemented as an alternative to cl/2937, which I've been told
uses a Nix feature that doesn't actually work.

Under-the-hood this is simply a systemd timer running a shell script
which checks available disk space and runs GC when necessary.

Change-Id: I3c6b5de85b74ea52e7e16c53f2f900e0911c9805
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3014
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-04-14 19:37:21 +00:00
Luke Granger-Brown
a0cfa097e0 feat(whitby/grafana): use CAS SSO
There's a hard-coded list of Admin usernames for the moment. We should
revisit this and get an actual groups setup in LDAP that's propagated
through...

Change-Id: Ic3601f1a9753573076769f4912038e9f1b60e139
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2982
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-04-13 00:08:36 +00:00
Vincent Ambo
da5512f2e9 feat(whitby): Enable Grafana at status.tvl.su
Enables a Grafana service pointing to whitby's local Prometheus
instance, accessible at status.tvl.su.

I've no idea how to configure Grafana and if it's possible to link it
to CAS, but we'll see about that later.

Notes:
* the explicit fixpoint for whitby config has been removed as we
  have the `config` parameter available now
* backups are enabled for the Grafana storage location

Change-Id: If5ffe0c1a3378d1c88529129487c643642705fd2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2948
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-04-12 22:01:05 +00:00
Vincent Ambo
f520bd40ca refactor: Replace 'depotPath' with 'depot.path'
Instead of having two ways of accessing the path to the depot (one of
which was stuttering, depot.depotPath) we settle on only one:
depot.path.

This was mostly used for NixOS module imports.

Co-Authored-By: Florian Klink <flokli@flokli.de>
Change-Id: I2c0db23383fc34f6ca76baaad4cc4af2d9dfae15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2962
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-12 21:55:07 +00:00
Vincent Ambo
adc9d026e0 feat(whitby): Enable Prometheus instance on whitby
Enables Prometheus with a local node exporter, and nothing else for
now.

Some additional collectors have been enabled for things that might be
relevant on whitby:

* systemd: all our services run in systemd
* processes: might be interesting for build-related stats
* logind: might be interesting for interactive usage stats

Change-Id: I48dacdd9c68b4be9edff7b3cb6256dad562498c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2930
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-04-12 17:32:48 +00:00
Griffin Smith
6266c5d32f refactor(users/glittershark): Rename to grfn
Rename my //users directory and all places that refer to glittershark to
grfn, including nix references and documentation.

This may require some extra attention inside of gerrit's database after
it lands to allow me to actually push things.

Change-Id: I4728b7ec2c60024392c1c1fa6e0d4a59b3e266fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2933
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: glittershark <grfn@gws.fyi>
2021-04-12 14:45:51 +00:00
Vincent Ambo
90281c4eac refactor(ops): Split //ops/nixos into different locations
Splits //ops/nixos into:

* //ops/nixos.nix - utility functions for building systems
* //ops/machines - shared machine definitions (read by readTree)
* //ops/modules - shared NixOS modules (skipped by readTree)

This simplifies working with the configuration fixpoint in whitby, and
is overall a bit more in line with how NixOS systems in user folders
currently work.

Change-Id: I1322ec5cc76c0207c099c05d44828a3df0b3ffc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2931
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: glittershark <grfn@gws.fyi>
2021-04-11 22:18:22 +00:00