mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
tvl-depot/ci-builds.nix

108 lines
2.5 KiB
Nix
Raw Normal View History

refactor: Fix a variety of filepaths for repo relayouting This fixes readTree and the various project builds, as well as (hopefully) most documentation links inside of the projects.
2019-12-20 21:37:02 +01:00
# This file defines the derivations that should be built by CI.
refactor: Move CI setup to separate Nix file Instead of polluting the repository namespace with the list of CI projects, move that to a separate file. Currently the list of projects to be built by CI is still hardcoded, but this will be fixed soon.
2019-12-09 11:58:09 +01:00
#
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
# The "categories" (i.e. attributes) below exist because we run out of
# space on Sourcehut otherwise.
{ depot, lib, ... }:
let
inherit (builtins) attrNames filter foldl' getAttr substring;
fix(ci-builds): typecheck __allTargets to be (list drv) `nix-build -A` expects a list of derivations, otherwise it will silently skip some targets. We can use yants to ensure we don’t accidentally put test targets that do not run on CI. `depot.users.tazjin.blog` was one such target, the only real drv is in the `rendered` field. `getBins` is not a derivation, rather the `runTestsuite` prints `{}` if it succeeds and aborts the evaluation otherwise. We make it into a derivation, using the `emptyDerivation` primitive we added earlier. We could actually improve that still, see the TODO. Change-Id: I3e7658b21aa4ef84192ac43c11b986bd8570a115 Reviewed-on: https://cl.tvl.fyi/c/depot/+/666 Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: tazjin <mail@tazj.in>
2020-06-28 02:56:47 +02:00
# attach a nix expression to a drv so we can “build” it
# TODO(Profpatsch): instead of failing evaluation if a test fails,
# we can put the expression of the test failure into $out
# and continue with the other CI derivations.
drvify = name: exp: depot.nix.emptyDerivation {
inherit name;
owo = lib.generators.toPretty {} exp;
};
in lib.fix (self: {
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
__apprehendEvaluators = throw ''
Do not evaluate this attribute set directly. It exists only to group builds
for CI runs of different "project groups".
To use the depot, always start from the top-level attribute tree instead.
'';
# Names of all evaluatable attributes in here. This list will be
# used to trigger builds for each key.
__evaluatable = filter (key: (substring 0 2 key) != "__") (attrNames self);
# List of non-public targets, these are only used in local builds
# and not in CI.
__nonpublic = with depot; [
users.tazjin.nixos.camdenSystem
users.tazjin.nixos.frogSystem
];
# Combined list of all the targets, used for building everything locally.
fix(ci-builds): typecheck __allTargets to be (list drv) `nix-build -A` expects a list of derivations, otherwise it will silently skip some targets. We can use yants to ensure we don’t accidentally put test targets that do not run on CI. `depot.users.tazjin.blog` was one such target, the only real drv is in the `rendered` field. `getBins` is not a derivation, rather the `runTestsuite` prints `{}` if it succeeds and aborts the evaluation otherwise. We make it into a derivation, using the `emptyDerivation` primitive we added earlier. We could actually improve that still, see the TODO. Change-Id: I3e7658b21aa4ef84192ac43c11b986bd8570a115 Reviewed-on: https://cl.tvl.fyi/c/depot/+/666 Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: tazjin <mail@tazj.in>
2020-06-28 02:56:47 +02:00
__allTargets =
(with depot.nix.yants; list drv)
(foldl' (x: y: x ++ y) self.__nonpublic
(map (k: getAttr k self) self.__evaluatable));
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
fun = with depot.fun; [
amsterdump
feat(ci-builds): Include //fun/tvlbot and //fun/clbot in CI builds Change-Id: I95daac993808ac9c3de06d20890cca108d8fa961 Reviewed-on: https://cl.tvl.fyi/c/depot/+/828 Reviewed-by: BuildkiteCI Reviewed-by: eta <eta@theta.eu.org> Tested-by: BuildkiteCI
2020-07-01 18:58:24 +02:00
clbot
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
gemma
quinistry
feat(ci-builds): Include //fun/tvlbot and //fun/clbot in CI builds Change-Id: I95daac993808ac9c3de06d20890cca108d8fa961 Reviewed-on: https://cl.tvl.fyi/c/depot/+/828 Reviewed-by: BuildkiteCI Reviewed-by: eta <eta@theta.eu.org> Tested-by: BuildkiteCI
2020-07-01 18:58:24 +02:00
tvldb
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
watchblob
wcl
];
ops = with depot.ops; [
depot.ops."posix_mq.rs"
besadii
journaldriver
kontemplate
mq_cli
feat(nixos/whitby): Hello, World! This adds NixOS configuration for the machine whitby.tvl.fyi. No interesting services are configured yet, so this configuration is quite plain. Change-Id: I67b7c75ebd6e298719b52e6b3bd83cc3be3c45d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/843 Tested-by: BuildkiteCI Reviewed-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-07-02 20:19:08 +02:00
nixos.whitby
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
];
chore(ci-builds): Move third_party builds to their own package group ... and remove a package that doesn't exist anymore (at this location) from the nixpkgs allowlist. Change-Id: I663c84c387fb04bb3b47448132ad768ed5352474 Reviewed-on: https://cl.tvl.fyi/c/depot/+/829 Reviewed-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Tested-by: BuildkiteCI
2020-07-01 19:44:45 +02:00
third_party = with depot.third_party; [
cgit
git
fix(3p/grpc): Bump to 1.30.0 and build with precompiled Abseil gRPC 1.29.0 can not be built with our precompiled libs because it accidentally overrides the C++ standard specified by the users. This was fixed in 1.30.0. Change-Id: I9a0390b3f5ec8dae6c295562f3a94b3dfa681dd1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/925 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi>
2020-07-05 19:04:32 +02:00
grpc
chore(ci-builds): Move third_party builds to their own package group ... and remove a package that doesn't exist anymore (at this location) from the nixpkgs allowlist. Change-Id: I663c84c387fb04bb3b47448132ad768ed5352474 Reviewed-on: https://cl.tvl.fyi/c/depot/+/829 Reviewed-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Tested-by: BuildkiteCI
2020-07-01 19:44:45 +02:00
nix
feat(3p/openldap): Enable slapd-passwd-argon2 module This enables support for the Argon2 password hashing mechanism in OpenLDAP. Note that we also need to configure the LDAP module to load this, so this change is not yet sufficient for actually using Argon2 hashes. Change-Id: I151b854b777daa924b22224a43851432a88a2760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/830 Reviewed-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Tested-by: BuildkiteCI
2020-07-01 19:45:23 +02:00
openldap
feat(3p/apereo-cas): initial Nix derivation (no TVL-specific config yet) Change-Id: Ie7d52370dd554b110bbfa041b943fcf246373b94 Reviewed-on: https://cl.tvl.fyi/c/depot/+/933 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-05 21:08:36 +02:00
apereo-cas
chore(ci-builds): Move third_party builds to their own package group ... and remove a package that doesn't exist anymore (at this location) from the nixpkgs allowlist. Change-Id: I663c84c387fb04bb3b47448132ad768ed5352474 Reviewed-on: https://cl.tvl.fyi/c/depot/+/829 Reviewed-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Tested-by: BuildkiteCI
2020-07-01 19:44:45 +02:00
];
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
various = with depot; [
lisp.dns
nix.buildLisp.example
nix.yants.tests
chore(ci-builds): Update for moved targets Change-Id: I2227d13fd4769f2b4411a43082543a5b56cacfb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/605 Reviewed-by: isomer <isomer@tvl.fyi>
2020-06-26 21:31:01 +02:00
tools.cheddar
tools.nsfv-setup
chore(ci-builds): Move third_party builds to their own package group ... and remove a package that doesn't exist anymore (at this location) from the nixpkgs allowlist. Change-Id: I663c84c387fb04bb3b47448132ad768ed5352474 Reviewed-on: https://cl.tvl.fyi/c/depot/+/829 Reviewed-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Tested-by: BuildkiteCI
2020-07-01 19:44:45 +02:00
web.cgit-taz
web.tvl
fix(ci-builds): typecheck __allTargets to be (list drv) `nix-build -A` expects a list of derivations, otherwise it will silently skip some targets. We can use yants to ensure we don’t accidentally put test targets that do not run on CI. `depot.users.tazjin.blog` was one such target, the only real drv is in the `rendered` field. `getBins` is not a derivation, rather the `runTestsuite` prints `{}` if it succeeds and aborts the evaluation otherwise. We make it into a derivation, using the `emptyDerivation` primitive we added earlier. We could actually improve that still, see the TODO. Change-Id: I3e7658b21aa4ef84192ac43c11b986bd8570a115 Reviewed-on: https://cl.tvl.fyi/c/depot/+/666 Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: tazjin <mail@tazj.in>
2020-06-28 02:56:47 +02:00
(drvify "getBins-tests" nix.getBins.tests)
feat(nix/runExecline): add runExecline runExecline is a primitive that just does not care. It’s similar to `runCommand`, but instead of concatenating bash scripts left and right, it actually *uses* the features of `derivation`, passing things to `args` and making it possible to overwrite the `builder` in a sensible manner. Additionally, it provides a way to pass a nix string to `stdin` of the build script. Similar to `writeExecline`, the passed script is not a string, but a nested list of nix lists representing execline blocks. Escaping is done by the implementation, the user can just use normal nix strings. Change-Id: I890d9e5d921207751cdc8cc4309381395d92742f Reviewed-on: https://cl.tvl.fyi/c/depot/+/701 Reviewed-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-06-28 03:14:11 +02:00
]
++ nix.runExecline.tests
;
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
feat(ci-builds): Add overridden haskell packages Add all the haskell packages we've overridden to the ci targets, so we can check that they build successfully. Change-Id: I3c2f2d61f542cc06ac2266881e182e755fcb3774 Reviewed-on: https://cl.tvl.fyi/c/depot/+/743 Reviewed-by: Kane York <rikingcoding@gmail.com> Reviewed-by: tazjin <mail@tazj.in>
2020-06-29 03:38:41 +02:00
# Haskell packages we've patched locally
haskellPackages = with depot.third_party.haskellPackages; [
generic-arbitrary
hgeometry
hgeometry-combinatorial
vinyl
comonad-extras
feat(3p): add haskell-language-server Add expression for building haskell-language-server, based on a vendored version of https://github.com/korayal/hls-nix with hashes updated to work with our nixpkgs version and ghc 8.8.3. Also add that to CI builds, so whitby will build it for me (thanks whitby). Change-Id: I373f8a7cb67974b8aa043b116436c074591b8d57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/897 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: BuildkiteCI
2020-07-03 17:51:33 +02:00
depot.third_party.haskell-language-server.ghc883
feat(ci-builds): Add overridden haskell packages Add all the haskell packages we've overridden to the ci targets, so we can check that they build successfully. Change-Id: I3c2f2d61f542cc06ac2266881e182e755fcb3774 Reviewed-on: https://cl.tvl.fyi/c/depot/+/743 Reviewed-by: Kane York <rikingcoding@gmail.com> Reviewed-by: tazjin <mail@tazj.in>
2020-06-29 03:38:41 +02:00
];
chore(ci-builds): Update for moved targets Change-Id: I2227d13fd4769f2b4411a43082543a5b56cacfb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/605 Reviewed-by: isomer <isomer@tvl.fyi>
2020-06-26 21:31:01 +02:00
# User-specific build targets
tazjin = with depot.users.tazjin; [
fix(ci-builds): typecheck __allTargets to be (list drv) `nix-build -A` expects a list of derivations, otherwise it will silently skip some targets. We can use yants to ensure we don’t accidentally put test targets that do not run on CI. `depot.users.tazjin.blog` was one such target, the only real drv is in the `rendered` field. `getBins` is not a derivation, rather the `runTestsuite` prints `{}` if it succeeds and aborts the evaluation otherwise. We make it into a derivation, using the `emptyDerivation` primitive we added earlier. We could actually improve that still, see the TODO. Change-Id: I3e7658b21aa4ef84192ac43c11b986bd8570a115 Reviewed-on: https://cl.tvl.fyi/c/depot/+/666 Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: tazjin <mail@tazj.in>
2020-06-28 02:56:47 +02:00
blog.rendered
chore(ci-builds): Update for moved targets Change-Id: I2227d13fd4769f2b4411a43082543a5b56cacfb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/605 Reviewed-by: isomer <isomer@tvl.fyi>
2020-06-26 21:31:01 +02:00
emacs
chore(ci-builds): Build users.tazjin.finito in CI Change-Id: I96bb33e76b373795d8cc5a4a2c7a1f9f49179a36 Reviewed-on: https://cl.tvl.fyi/c/depot/+/804 Reviewed-by: BuildkiteCI Reviewed-by: ericvolp12 <ericvolp12@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-06-30 06:15:00 +02:00
finito
chore(ci-builds): Update for moved targets Change-Id: I2227d13fd4769f2b4411a43082543a5b56cacfb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/605 Reviewed-by: isomer <isomer@tvl.fyi>
2020-06-26 21:31:01 +02:00
homepage
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
];
feat(ci-builds): add chupacabra My laptop, so I can recompile linux-ck not on my machine Change-Id: Ib42e0440628be6a4d03999220ab2c69b19d6e391 Reviewed-on: https://cl.tvl.fyi/c/depot/+/884 Reviewed-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi> Tested-by: BuildkiteCI
2020-07-03 00:47:39 +02:00
glittershark = with depot.users.glittershark; [
system.system.chupacabra
feat(ci-builds): re-add xanthous This reverts commit 2249be39ed4d869ca602b353372f8d9066f0be28. Build should pass now that it's all been properly integrated into the nix tree. Change-Id: If2bfa642302368ac24d06ff7d853c0eda76811a7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/742 Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: Kane York <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-06-29 03:32:41 +02:00
xanthous
feat(ci-builds): add chupacabra My laptop, so I can recompile linux-ck not on my machine Change-Id: Ib42e0440628be6a4d03999220ab2c69b19d6e391 Reviewed-on: https://cl.tvl.fyi/c/depot/+/884 Reviewed-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi> Tested-by: BuildkiteCI
2020-07-03 00:47:39 +02:00
];
refactor(ci-builds): Split up CI builds into multiple buckets These categories separate CI targets, which hopefully avoids the out-of-space errors we have been seeing on Sourcehut. The sets of CI build targets are made available in the depot itself so that besadii can be updated to create a new build for each target group. For convenience, 'ciBuilds' contains an '__allTargets' attribute which combines the contents of each target batch - this makes it possible to still invoke a build for everything by using: nix-build -A ciBuilds.__allTargets Note: Some targets that were previously built in CI aren't anymore, most importantly my NixOS systems which don't fit on Sourcehut. Change-Id: Ia15ed7b743c8add51ae08ce0827a0ddfacd637e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/570 Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-24 04:08:53 +02:00
})
Reference in a new issue Copy permalink