Commit graph

120 commits

Author SHA1 Message Date
Kane York
92d4554b62 feat(tools/tvlc): init project
tvlc is a tool for managing sparse git worktrees of the TVL depot.
It is still in development; near-term tasks include a setup script,
Nix dependency resolution, worktree removal, and the dispatch
script.

See cs.tvl.fyi/depot/docs/designs/SPARSE_CHECKOUTS.md for more info.

Change-Id: Iad96656f0206178980fe7dcadd3dffe70d690f8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1760
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-19 01:23:54 +00:00
Vincent Ambo
4ed096b7d7 chore(3p/emacs): Enable cairo for Emacs 27.1
... this in turn enables support for Harfbuzz, which enables support
for ligatures - maybe this will make grfn happy!

Change-Id: I331c5171a829800fe532ca0b0d4910478e63ef1b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1783
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-18 22:06:16 +00:00
Vincent Ambo
c344e03705 chore(3p): Provide Emacs 27.1
This is now available in the upstream channel, and I need it on one
machine for $reasons.

Normally the Emacs attribute is versioned to make it clear which
version we're dealing with, but not in this case. I've added an assert
to check that it is indeed 27.

Change-Id: I827df0d36350066bf4e177150d32af0c8d123b9b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1764
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-17 18:14:40 +00:00
Vincent Ambo
6f5659c88b chore(3p): Bump nixos-unstable channel
This bumps the channel to a commit that includes fixes for an nginx
issue we have been seeing:

https://github.com/NixOS/nixpkgs/pull/95264

Includes the following compatibility fixes:

- tests disabled in third_party.bufbuild: These were enabled
  unexpectedly by the update, but don't run in the sandbox because
  they want to download things from github

Change-Id: I98a3b5de57f62f1fd3a37701fa1896eddeedff85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1759
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-17 09:54:18 +00:00
Vincent Ambo
d44565d17e chore(3p): Pick Emacs & friends from the default channel
I had previously pinned Emacs to the (older) stable channel, because
of a heretofore undiagnosed issue in newer versions.

It turns out that the older Emacs is linking against X11-related
libraries, namely <unintelligible>, which cause even *weirder*
undiagnosed issues if paired with the rest of the recent packages.

This commit removes the exception for Emacs and just picks it from the
same channel as everything else.

Change-Id: Ifce21c99f0204d4cf131bacb366dd9617fc301b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1751
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-15 00:26:42 +00:00
Vincent Ambo
d42c3dd72f chore(3p): Bump nixpkgs channels
Bumps both nixos-unstable and nixos-20.03 to today's versions, as per
status.nixos.org

Contains minor fixes to things that broke because of the update:

* tazjin/frog: hardware.u2f is a deprecated setting
* glittershark/system: modSha256 in Go modules is now vendorSha256
* glittershark/owothia: removed version constraint on relude

Change-Id: Ib3e9612b1b06ed547b90e4f8b0ffe5ed7fe0a5c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1642
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 00:15:33 +00:00
Griffin Smith
09cb41b7ac fix(3p/nix): Properly configure SANDBOX_SHELL
point the SANDBOX_SHELL macro at the actual path to busybox on the build
machine, or allow it to be configured at build-time with a cmake option.

Change-Id: I044a1315ba9baa3bc9ceddf29f36d14f9f9ccd96
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1632
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 02:15:10 +00:00
edef
8862f3fbaf feat(3p): Whitelist unzip
Change-Id: Iae7e661dc45f8bc8668c1aa4fe82b88e8868c22f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1585
Tested-by: BuildkiteCI
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-03 08:17:51 +00:00
Griffin Smith
696924b5ef feat(web/panettone): Add dev helpers for postgres db
Add a docker-compose file and lorri-based direnv for aiding in
running and connecting to a postgres database during development of
panettone.

Change-Id: I319eee52b52cd48e1f3d2e32c558989768dc19d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1465
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: eta <eta@theta.eu.org>
2020-07-26 21:55:41 +00:00
Vincent Ambo
02066a4bab chore(3p/nix): Enable clang-format check in derivation build
This is the easiest way to get the checks up and running for now, but
we will probably want to separate out things like this into a separate
build step in the future.

Change-Id: I8e1a1095aef09b1eee97abad5b6240bc64d14b8c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1287
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-19 16:54:52 +00:00
Vincent Ambo
a2870d05a3 feat(3p/gtest): Bump version & override stdenv to libcxx
Change-Id: I19a34e80f51add5a7c6cde2650aae60c1919ad43
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1238
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-17 02:59:42 +00:00
Griffin Smith
04ae293360 feat(tools): Add script to flamegraph a command
Add tools.perf-flamegraph, which collects the base case execution of
perf piped through stackcollapse-perf and flamegraph to flamegraph the
execution of an external command via perf.

Change-Id: I671fe254dc374b6cd7deca2d3bdea266164de025
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1176
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-07-15 20:21:34 +00:00
Vincent Ambo
f70afe61a4 feat(3p/rr): Add ThreadRipper compatibility patch
Change-Id: Ia73883075e11bccaa234d3b1212f3ab9363f234c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1129
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-07-13 17:56:22 +00:00
Vincent Ambo
d6bf3207b0 feat(tazjin/frog): Install Quassel client on frog
Change-Id: I5abcf0768a84b2010acefef595d05d4f24f6d622
Reviewed-on: https://cl.tvl.fyi/c/depot/+/956
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
2020-07-08 22:10:08 +00:00
Griffin Smith
9ec0f4d0c8 feat(gs/keyboard): Integrate my layout with the depot
Integrate the keyboard layout for my Ergodox EZ, which is a layout for
QMK, with the depot, including exposing several cross-compilation
packages necessary to compile qmk for avr in third_party.

Change-Id: Idd43169a0a3cf0be2bd1a578fdaff70388a58bfc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/947
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-07-06 22:52:02 +00:00
Luke Granger-Brown
1915fbccd6 feat(3p/apereo-cas): initial Nix derivation (no TVL-specific config yet)
Change-Id: Ie7d52370dd554b110bbfa041b943fcf246373b94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/933
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-07-05 22:42:39 +00:00
Vincent Ambo
d2aaf030bd feat(3p/openldap): Enable slapd-passwd-argon2 module
This enables support for the Argon2 password hashing mechanism in
OpenLDAP. Note that we also need to configure the LDAP module to load
this, so this change is not yet sufficient for actually using Argon2
hashes.

Change-Id: I151b854b777daa924b22224a43851432a88a2760
Reviewed-on: https://cl.tvl.fyi/c/depot/+/830
Reviewed-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Tested-by: BuildkiteCI
2020-07-01 19:10:13 +00:00
Vincent Ambo
6f5211bba8 chore(ci-builds): Move third_party builds to their own package group
... and remove a package that doesn't exist anymore (at this location)
from the nixpkgs allowlist.

Change-Id: I663c84c387fb04bb3b47448132ad768ed5352474
Reviewed-on: https://cl.tvl.fyi/c/depot/+/829
Reviewed-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Tested-by: BuildkiteCI
2020-07-01 19:10:13 +00:00
Profpatsch
7f8184c6b7 refactor(overrides/writeElispBin): use writeScriptBin
Change-Id: Ib9793da985128b4a8f64b04086ea89ce81118188
Reviewed-on: https://cl.tvl.fyi/c/depot/+/705
Tested-by: BuildkiteCI
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: BuildkiteCI
2020-06-30 00:19:27 +00:00
Profpatsch
2f21e0c8c0 refactor(nix): use our own writeScript(Bin)
We have this nice `runExecline` now, so we don’t need to use
`runCommand` (which spawns bash) just to write a simple script.

Change-Id: I2941ed8c1448fa1d7cc02dc18b24a8a945b2c38b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/704
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: BuildkiteCI
2020-06-30 00:19:27 +00:00
Griffin Smith
8091da8f0f feat(3p/haskellPackages): port in patches from xanthous
Add a few relatively uncontroversial patches to fix some broken packages
that I had developed for xanthous to the top-level third_party tree, so
they can be reused by other people in the monorepo

Change-Id: I68740477bda278c5dcc123080029ee4bd2cae37a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/740
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: tazjin <mail@tazj.in>
2020-06-29 01:56:24 +00:00
Griffin Smith
6cd8b8e7e8 feat(3p): Add haskellPackages from nixpkgs
Change-Id: Ic59e86ec2a36f5ee4eaf48592346a084fdef63ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/735
Reviewed-by: glittershark <grfn@gws.fyi>
2020-06-29 00:38:32 +00:00
Profpatsch
cdfae7de48 feat(nix/emptyDerivation) add emptyDerivation
The most trivial of all derivations. It is more useful than it looks.

Can be used to bind nix expressions (e.g. test suites) to a
derivation, so that `nix-build` does not crap itself.

Change-Id: I61c24d8c129c9505733161207f3c30e820f5b15e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/665
Reviewed-by: tazjin <mail@tazj.in>
2020-06-28 17:59:46 +00:00
Profpatsch
98a990c6a6 feat(nix/getBins): add getBins
This is a simple-stupid “unix import system” for nix, for referencing
binaries in `/bin/` by their name and lifting them to a Nix attrset.

Allows for simple aliasing of executable names.

Change-Id: Ifa23cb377201c3b08050c5026e9751e736afaf56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/664
Reviewed-by: tazjin <mail@tazj.in>
2020-06-28 17:58:15 +00:00
Profpatsch
3fd583d27c feat(nix/writeExecline): add writeExecline
This is a writer, similar to `pkgs.writeBashScript` or
`pkgs.writers.writePython3`.

The difference is that we can correctly write all execline scripts by
using nix lists of lists, so the user doesn’t have to care about
escaping arguments (like they have to in bash scripts with
`lib.escapeShellArg` for example).

Change-Id: I2f2874cf61170ddca07b89b692f762725f4a75dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/625
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
2020-06-27 17:52:14 +00:00
Vincent Ambo
22b8a49b87 feat(ops/pipelines): Add Buildkite pipeline configuration
This adds configuration which generates the structure expected for
Buildkite pipelines, which can then be dynamically ingested by
Buildkite when a pipeline is triggered.

Change-Id: I61e3dc3affb19c1f2550ef827fa73b17f8d8ae47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/627
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-27 16:55:18 +00:00
Profpatsch
17eba437a7 feat(nix/binify): add binify
Create a store path where the executable `exe` is linked to
$out/bin/${name}. This is useful for e.g. including it as a “package”
in `buildInputs` of a shell.nix.

For example, if I have the exeutable /nix/store/…-hello, I can make it
into /nix/store/…-binify-hello/bin/hello with
`binify { exe = …; name = "hello" }`.

Change-Id: I600bdcd8f143bca2dd8dfbb165a9a5a8d6397622
Reviewed-on: https://cl.tvl.fyi/c/depot/+/624
Reviewed-by: tazjin <mail@tazj.in>
2020-06-27 03:42:05 +00:00
Vincent Ambo
6944d0c96a feat(tazjin/nsfv-setup): Add script for noise-cancelling others
This script creates a pulseaudio sink that will cancel noise in audio
streams sent to it, and then move it on to the default sink.

This means that other people's crackling, static background, gulping,
keyboard sounds, fan whirring, construction noise etc. are removed.

Some preliminary tests on TVL suggest that this actually works. The
parameter might need some tweaking ("50" is just the default value),
as there is some occasional crackling at the beginning/end of a speech
segment, but this is already *much* better than before.

Change-Id: I9d4e2b39cfc2b878b4b7c5458788b8d46fb801af
Reviewed-on: https://cl.tvl.fyi/c/depot/+/577
Reviewed-by: nyanotech <nyanotechnology@gmail.com>
2020-06-25 03:12:50 +00:00
glittershark
ec84fde6b1 fix(3p/nix): Revert "feat(3p/nix): Add installCheckPhase"
This reverts commit 368e8d1edd.

Reason for revert: Didn't mean to submit, and the phase is currently failing (which breaks the otherwise-functional derivation)

Change-Id: I515b2fb45188dc90f09ae2458453192487c74d71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/581
Reviewed-by: tazjin <mail@tazj.in>
2020-06-23 15:23:04 +00:00
Griffin Smith
368e8d1edd feat(3p/nix): Add installCheckPhase
Add an installCheckPhase that runs the appropriate substituteAll on
common.sh and runs the lang.sh tests with the build artifacts in the
PATH.

Change-Id: I2df5a93b8f3ffdfdc194a0e7d6b6669ef520c345
Reviewed-on: https://cl.tvl.fyi/c/depot/+/561
Reviewed-by: glittershark <grfn@gws.fyi>
2020-06-23 14:25:37 +00:00
Luke Granger-Brown
4de63f784c fix(3p/nix): propagate dependencies which are needed by people linking in nix
This includes absl, which we install into the output, and boost and the boehm GC,
which are moved to propagated deps.

Change-Id: I8f9f9795ff92e26b2320359064241d7fd59c2d33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/549
Reviewed-by: tazjin <mail@tazj.in>
2020-06-21 23:49:19 +00:00
Vincent Ambo
4bcd99d823 feat(3p): Whitelist lutris & its insecure dependency
Change-Id: I9a21b823e65af0a180a8705c58f1cf5d4ba4445c
2020-06-19 01:06:31 +00:00
Luke Granger-Brown
22821b886b chore(3p/go): make hashes more deterministicerer
Because this is using %h, we're subject to the whims of the remote
in trying to make sure that we get a consistent commit hash length
in the VERSION stamp.

Change-Id: I716193c1440ec367880b6a5f7dfa4f85a11c19a9
2020-06-18 01:27:32 +01:00
Vincent Ambo
ab7ef0c547 chore(3p): Use typedGo as the main Go
Because it seems to work and we can.

Change-Id: I0ba3b6541efa8c21ead875368eca18c73db9a416
2020-06-18 00:54:12 +01:00
Vincent Ambo
65f29ee803 feat(3p/typedGo): Add derivation for the Go alpha with a type system
The Go language authors have released an experimental version of Go
that has a type system: https://blog.golang.org/generics-next-step

This overrides the existing Go derivation to build the typed Go. The
next step is a buildTypedGo set of functions that wrap buildGo.

Change-Id: Idb8a4868bca003d821ed5cb324af633398faf002
Reviewed-on: https://cl.tvl.fyi/c/depot/+/443
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-16 22:11:53 +00:00
eta
c3abbb5e2d feat(tvldb): Import the tvldb/paroxysm source, add a Nix derivation
- This imports the tvldb (actually a thing called 'paroxysm') code
  from https://git.theta.eu.org/eta/paroxysm into the monorepo.
- Additionally, I did a nix thing, yay! \o/
  (well, with tazjin's help)
- 3p/default.nix needed modifying to whitelist pgsql.

Change-Id: Icdf13ca221650dde376f632bd2dd8a087af451bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/389
Reviewed-by: tazjin <mail@tazj.in>
2020-06-16 13:30:44 +00:00
Griffin Smith
b570e4c7cf feat(third_party): Add awscli
I need it to deploy my website

Change-Id: I5df8d76d6e0a3d8892ae8bc69d2b46b310f147a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/399
Reviewed-by: tazjin <mail@tazj.in>
2020-06-16 02:28:41 +00:00
Vincent Ambo
69fe278315 feat(3p/git-bug): Add git-bug derivation at unstable-20200614
This builds git-bug, a distributed issue tracker that uses git as its
data store.

It also installs its man pages and shell completions. It is
recommended that users add git-bug to their system closure for these
extras to work, as they will not be picked up by the dispatch script.

Change-Id: I1595368e61b0bae8a9497abd023085cb90a521a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/345
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-06-14 22:27:54 +00:00
Luke Granger-Brown
a73ca3f43d feat(gerrit): Create Gerrit derivation.
This uses the actual Bazel build, using a variety of tricks and hacks to
make it actually work.

Bazel really wants to download linux binaries from the internet and run
them. In lieu of trying to fix the build system to not do this, we
instead put bazel inside an FHS environment, which allows the binaries
to find their dependencies.

We also have to patch a few things:

* We use build --nobuild instead of fetch, so we only fetch the
  dependencies we actually need for the build and not, say, Windows
  binaries.
* We don't remove rules_cc, because we need it as an external
  dependency, not bundled.
* We do some manual fixes on the cache before packing, because we need
  to remove some in-tree sources (so they don't cause the hash to break,
  since the hashes differ each time they're generated), and also remove
  some extraneous files.
* We explicitly turn off the repository and disk caches, because the
  .bazelrc at the root of the Gerrit tree turns them on, with paths
  pointing into the user's home directory.
* detzip is used instead of the zip binary for packing bower_components
  into an archive. detzip doesn't create entries for directories, and
  also doesn't store most metadata (timestamps, etc.), and uses store
  (i.e. uncompressed) compression only. It also sorts the file tree
  before writing them into the file.

Change-Id: I572c43f7175067ecb1b85cdf40dda13a52de1439
Reviewed-on: https://cl.tvl.fyi/c/depot/+/252
Reviewed-by: tazjin <mail@tazj.in>
2020-06-14 16:35:55 +00:00
Vincent Ambo
d6c902de50 chore(3p): Bump nixos-unstable 2020-06-11 21:13:04 +00:00
Vincent Ambo
afe0841e9d feat(ops/nixos): Add module for configuring Gerrit for the repo 2020-06-11 21:13:04 +00:00
Vincent Ambo
5841a49f85 chore(3p): Use nixos from unstable
Machines can still override the *package set*, but not the modules.
2020-06-11 21:13:04 +00:00
Vincent Ambo
430e602336 feat(3p/grpc): Build gRPC using LLVM10 & libcxx 2020-06-07 17:14:48 +01:00
Vincent Ambo
4fb8788c01 feat(3p/protobuf): Check in derivation for proto 3.12 with libcxx 2020-06-07 14:56:56 +01:00
Vincent Ambo
76f4e27386 fix(3p): Force all stdenvs to use LLVM10 2020-05-31 21:58:17 +01:00
Vincent Ambo
86a35d963d refactor(3p/abseil_cpp): Inline Nix derivation 2020-05-27 21:52:44 +01:00
Vincent Ambo
7dc094173b chore(3p): Remove glog override
This library is being vendored, hence the override doesn't matter
anymore.
2020-05-27 01:25:39 +01:00
Vincent Ambo
4947aa9ab3 fix(3p/git): Use git derivation from stable channel as base
unstable is, of course, not working correctly at the moment
2020-05-27 00:06:59 +01:00
Vincent Ambo
fe45079d19 chore(3p): Expose nixpkgs commit to depot 2020-05-26 11:54:04 +01:00
Vincent Ambo
0b23d2e4bf chore(3p/naersk): Update to latest 'master' 2020-05-25 23:36:20 +01:00