Commit graph

12955 commits

Author SHA1 Message Date
sterni
901364869c chore(3p/lisp): import mime4cl source tarball
Used http://wcp.sdf-eu.org/software/mime4cl-20150207T211851.tbz (sha256
5a914669bba7561efe59a4fd0817204c07ad2add98b03ae206ef185ac04affb3).
Importing seems sensible since there's no upstream repo nor has their
been a release since 2015.

This is just an import commit, so the changes made to make it build are
more discoverable as their own commit.

Change-Id: I2ff28c3c7433abdf7857204bc89eaf9edc0b1cbc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3378
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-09-01 22:57:17 +00:00
sterni
de0f0164d2 feat(3p/lisp/npg): build using buildLisp
Change-Id: I9f987dc25d77a829cc0716cbe4cb1b91c36de861
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3269
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-09-01 22:57:17 +00:00
sterni
8e45aace13 chore(3p/lisp): import npg source tarball
Used http://wcp.sdf-eu.org/software/npg-20150517T144652.tbz (sha256
42e88f6067128fbdb3a3d578371c9b0ee2a34f1d36daf80be8a520094132d828).
There's no upstream repository nor a release since 2015, so importing
seems to make a lot of sense.

Since we can't subtree making any depot-related changes in a separate CL
-- this is only the source import.

Change-Id: I64c984ca0a84b9e48c6f496577ffccce1d7bdceb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3377
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-09-01 22:57:17 +00:00
sterni
2e08324484 feat(3p/lisp/sclf): build using buildLisp
Adding the default.nix is quite straightforward, however we have to make
today's SBCL happy: due to package locking it no longer likes sclf using
an sb-impl internal constant for some reason. This is however a good
opportunity to clean up the stat-*-time code: It converted the times in
an implementation specific way even though time.lisp does provide a
generic way to convert between unix and universal time. Note that the
updated ASDF file is untested, but should be a trivial enough change.

Change-Id: If193bf830ac704cc53e0855d8e9fff2b5a5ef291
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3268
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-09-01 22:57:17 +00:00
sterni
a5dbd0f5d9 chore(3p/lisp): import sclf source tarball
Used http://wcp.sdf-eu.org/software/sclf-20150207T213551.tbz (sha256
a231aeecdb9e87c72642292a1e083fffb33e69ec1d34e667326c6c35b8bcc794).
There's no upstream repository nor a release since 2015, so importing
seems to make a lot of sense.

Since we can't subtree making any depot-related changes in a separate CL
to make them more discoverable -- this is only the source import.

Change-Id: Ia51a7f4029dba3abd1eee4eeebcf99aca5c5ba4c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3376
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-09-01 22:57:17 +00:00
sterni
70e5783e22 feat(3p/lisp/closure-html): init at 2017-04-19
This one requires a bit of jumping through hoops. Patching the dtd /
catalog lookup is quite straightforward and similar to cxml, but the
CLOSURE-HTML:*html-dtd* variable gives us a bit of trouble: It is
defined quite late in `html-parser.lisp`, but files that need to be
built first already reference it. SBCL has apparently decided to be
particular about this and emits a `WARNING` (!) condition for this
which is also worthy of `failure-p` of `compile-file` being true,
so that `buildLisp` will abort compilation. We workaround this issue
by injecting an extra source file which `defvar`s the desired symbol.

A similar issue exists with `dump-dtd` which references
`CL-USER:*HTML-DTD*` for some reason. Since this is a helper intended
for development (?) and not exported we just throw it away via a
patch.

Change-Id: Ic0f92815a21f3793925c49a70a72f4a86791efe4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3263
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-09-01 22:57:17 +00:00
sterni
281c09fae7 feat(3p/lisp/closure-common): init at 2018-09-09
Change-Id: Ieec5470fe8fd54851b982c7a380185a3faeaa067
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3258
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-09-01 22:57:17 +00:00
Vincent Ambo
c38873e2e4 fix(3p/overlays): Use notmuch.el from nixpkgs
...  instead of the one from the overlay, which can be out of sync.

This requires the TVL overlay to be applied after the Emacs overlay.

Change-Id: Idac403ea612e334c14b45759dc216699a506678f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3484
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <mail@tazj.in>
2021-09-01 12:27:31 +00:00
Vincent Ambo
885b6d9bb8 refactor(3p/notmuch): Simplify setup
The backported fix is no longer required and we can just apply the
patch in the overlay, this makes everything a little easier.

Change-Id: I654a1bb002eef5c578b8e576e133a159bde3f850
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3483
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-01 12:27:31 +00:00
Vincent Ambo
0dbad73ff1 chore(web/tvl): Update some of the members in the TVL graph
Some leavers, some newcomers (some of which aren't actually new) and
so on. There are some lurkers in the IRC channel who I didn't include.

Change-Id: I9bf6b83ef1fadfb19bc6836f6f5946f115af30f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3482
Tested-by: BuildkiteCI
Reviewed-by: Mike Johnson <mdj@mikejohnson.xyz>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-31 23:29:35 +00:00
Mike Johnson
9fc206f072 fix(ops/users): Another try at a working password hash for mdjnsn
Change-Id: I8b4aea53abb2004585241ad17c5fdfd9186c58f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3481
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-08-31 20:23:00 +00:00
Mike Johnson
fa8dd0f3ab feat(ops/users): Add mdjnsn to users
Change-Id: I94975d848287c32e11b1d3986986f2dbc6c220b9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3466
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2021-08-31 18:43:49 +00:00
Vincent Ambo
0a98d09a93 chore(3p/gerrit): Remove avatar padding patch
In the current config, Gerrit doesn't actually seem to realise that
avatars are not enabled (this changed in 3.4 somehow).

Either way we don't need to maintain this fix since there's an actual
upstream one now: https://gerrit-review.googlesource.com/313982

Change-Id: I7efab7b8fa5e9e38bddae86acd8d8a7852b27bb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3465
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-08-31 15:18:01 +00:00
sterni
83c8716afd chore(3p): update NixOS channels to 2021-08-30
This lets us benefit from the recent OpenSSL security-related
update [1]. Since nixos-unstable is still stuck, we temporarily
use nixos-unstable-small as our unstable channel.

Fixes necessary:

* //users/sterni/nix/char:
  Someone has decided to drop writers.writeC upstream [2],
  so we reimplement it ad-hoc using runCommandCC

[1]: https://www.openssl.org/news/secadv/20210824.txt
[2]: 982f46985e

Change-Id: Id84756e2e370296b7a27e1a3f1744f58f8fe3c47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3463
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-08-30 17:06:40 +00:00
Vincent Ambo
d67b9e3192 refactor(bufCheck): Assume bufCheck is run in depot checkout
Rather than copying the depot path into the store on each commit,
assume bufCheck is run in the depot checkout (which it is, in
Buildkite land).

Change-Id: I4a4af2e5b45acad2d18218e503880ee63b20f078
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3462
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-29 12:41:07 +00:00
Vincent Ambo
43269730e6 refactor(ops/pipelines): Move failure status zeroing to setup
We changed the configured pipeline in Buildkite to upload
`static-pipeline.yaml` instead of containing the steps of that
pipeline itself.

This makes it easier to test changes to builds and such, but adds
another build step with scheduling overhead etc.

However - we can work around this by killing one of the existing build
steps. There's no reason the failure status zeroing (required for
status reporting) shouldn't be part of the pipeline setup, so I've
moved it there instead and nuked that step.

This should mean that the pipeline is configurable from within the
repo, but without slowing anything down.

Change-Id: I206ecc02647de42a461e33c02879ab84daf5ed2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3461
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-29 12:37:04 +00:00
Vincent Ambo
fe225d48a1 feat(3p/gerrit): Upgrade Gerrit and plugins to v3.4.0
Brings us back to a stable version of Gerrit instead of a random
commit. Note that Gerrit 3.4.1 is out, but due to a bug it can not be
built publicly because it accidentally points at a private
submodule (this is being fixed upstream).

Change-Id: I0376c63a649498cef999dfa99bfccba511f2c8da
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3444
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-08-28 15:32:53 +00:00
Vincent Ambo
0c1b8f954e chore(tazjin/keys): Use ecdsa-sk key for tverskoy
Unfortunately this doesn't work with Gerrit yet, but it's fine for SSH auth.

Change-Id: Idcfebb117ca39e47ef5595f5bb64ea31dbef3af7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3442
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-27 14:43:43 +00:00
Vincent Ambo
1af8bf2b02 fix(ops): Add another target that needs a //users exception
Change-Id: I3edede16890893e14727283a12c7fa3310cb4dc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3441
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2021-08-26 21:04:34 +00:00
Vincent Ambo
f6638ce62c fix(web): tazjin.css -> tvl.css
Bug introduced by the previous static asset move.

Change-Id: I827976e468e4ce877a12dfbca6126b3a7445e783
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3440
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 20:57:02 +00:00
Vincent Ambo
df566e9b4a style(depot): Use depot path format for args filter errors
As pointed out by sterni, we don't need lib for this.

Change-Id: Ide9719641098f770a098a938d047afa0dcb5dc6d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3439
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2021-08-26 20:54:20 +00:00
Vincent Ambo
e1f83cc086 feat(depot): Disallow access to //users from outside of it
Code under this depot path is essentially unstable and potentially
unreviewed - this is a good thing (people can play around with cursed
stuff all they want), but we should not make the rest of the
repository depend on any of it.

Any cursed things that are required outside of users can be moved to a
different depot path if people agree with that.

Change-Id: I46a34a0e9662069c01b43d9a653e5545e325e587
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3434
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-08-26 20:39:07 +00:00
Vincent Ambo
8b851956ad feat(readTree): Add support for path-dependent args filtering
Adds another argument to readTree itself which can be passed when
importing readTree (e.g. in our default.nix) to filter the arguments
passed to a target based on that target's location in the tree.

This is intentionally not yet mentioned in the docs, and also
intentionally implemented in such a way that the API surface of
readTree doesn't change. The reason for this is that I want to figure
out whether these filter functions are actually useful, e.g. within
depot by filtering user-folder passing, and then refactor the readTree
API to find a public way of exposing this as part of the readTree
function itself (and not its import).

Relates to b/143.

Change-Id: I2cdf09f67916527d2337f4bfb578749aeac51a6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3433
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 20:33:52 +00:00
Griffin Smith
d857d5ad68 refactor(gs/system): Remove chupacabra
This machine no longer exists

Change-Id: I8e549b8397777a01404bd84c10c195e80f281744
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3431
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 19:41:42 +00:00
Griffin Smith
8201f0f3d8 refactor(gs/system): Remove rebuilder script
I no longer use this, I just use the rebuild-system that all nixos
systems get now.

Change-Id: I2272ff13b21b3194c06b51dbc340c19b8bb336a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3430
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2021-08-26 19:10:19 +00:00
Vincent Ambo
72ebd3411b fix(atward): Redirect // queries to depot root
Makes it possible to open the default code viewer for the user at the
depot root by searching for `//`.

Fixes b/134.

Change-Id: I409ad36cea28de27cd1789a84eda71f8979d3133
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3437
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 19:02:23 +00:00
Vincent Ambo
da0b330756 chore(web/tvl): Move TVL static assets out of //users/tazjin
It's now more like my personal homepage depends on TVL assets, not the
other way around.

Change-Id: Ifb9d61aa8ec2cab549e25de3a3dfbbd08f3d336c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3435
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 17:46:06 +00:00
Vincent Ambo
60b25b49de fix(ops/pipelines/depot): Buildkite branches use full ref names
... otherwise the filtering also applies to canon.

Change-Id: Ia1c67b99282fb8fd0e4d22e997535170f0326e33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3432
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2021-08-26 16:35:44 +00:00
Vincent Ambo
d5ddfb7b96 feat(pipelines/depot): Skip build steps if their out paths exist
Skip build steps if they have already been built, reducing pipelines
to the things that actually changed between builds. On canon all
targets are always built (we require this for anchoring).

Note that this is not perfect, garbage collection and competing
pipelines may affect each other.

Also note that we have some impure targets that change on every
commit.

Change-Id: Ic6bae3b6c8e1e7fd2116ec252f5089f471854ab6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3427
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
2021-08-26 16:29:32 +00:00
sterni
094dfa2ed3 feat(users/sterni/emacs): add ariadne conill's blog to subscriptions
Change-Id: I6d5935279069c8af7e7a5f21f9d221c93a533d8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3428
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 15:34:58 +00:00
sterni
9ed439bfbd feat(users/sterni/nix): cursed nix html DSL
Couldn't sleep, so I made a surprisingly neat way to render HTML
documents in Nix using our favorite feature __findFile:

  let
    inherit (depot.users.sterni.nix.html) __findFile esc;

  in

  <html> {} [
    (<head> {} [
      (<meta> { charset = "utf-8"; } null)
      (<title> {} (esc "hello"))
    ])
    (<body> {} [
      (<h1> {} (esc "hello world"))
    ])
  ]

=> "<html><head><meta charset=\"utf-8\"/><title>hello</title></head><body><h1>hello world</h1></body></html>"

Change-Id: Id36808a56ae3da3b5263c06f29342fc22d105c21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3410
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 15:34:58 +00:00
sterni
17d78867bb feat(ops/pipelines/depot): only evaluate once if possible
We currently evaluate every target twice -- once when the depot pipeline
is built and once when actually running the build step in question. Nix
evaluation is quite slow especially given heavy use of import from
derivation in depot, so avoiding the second evaluation is desireable.

Evaluating a derivation yields a `drv` file in the nix store which can
be passed to `nix-store --realise` in order to build it eliminating the
need to wait for evaluation. We can obtain the path to the `drv` file
while building the pipeline via `target.drvPath` and remember it for the
build later.

However we need to work around a flaw (or oversight) in Nix's dependency
tracking via string context: This is based on derivations, not output
path (because this is what evaluation deals with, likely). This is no
problem per se, but an issue is that Nix can't express a dependency on
a `drv` file without any of its output paths. This means for us that we
either have to build all output paths at evaluation time (which we don't
want, obviously) or to deal with the fact that the `drv` file we need
may be garbage collected at any moment after discarding the string
context -- then nix is unable to track the reference from the pipeline
to the `drv` file in the store.

So to prevent a race condition between the pipeline and the garbage
collector we fall back to the normal nix-build invocation as we did
before.

Change-Id: I9ef8bd233085dc6e30eba54f403ea03ac2d35748
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3426
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 15:24:33 +00:00
Griffin Smith
aad7e7e74b fix(gs/emacs): Don't repeat flycheck nav with evil .
Prevent flycheck-next-error and flycheck-prev-error from being repeated
by evil, since they're movement commands rather than editing commands.
This lets me spam `]e.` if I have to do the same thing to all the errors
in a buffer, for example.

Change-Id: I5993f6d19b71b63e5f4be1f3ce9e0cfd0357cc6e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3425
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2021-08-26 13:45:27 +00:00
Griffin Smith
5ba19ec2a2 fix(grfn/system): Fix rust-analyzer target dir
This was missing a path segment, plus calling it rust-analyzer makes it
clearer what's going on

Change-Id: I8f71fe1b438d72f743472ab10ec939f686ad0da1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3424
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2021-08-26 13:45:27 +00:00
Griffin Smith
a1205f0565 feat(gs/home): Alias tf->terraform
Change-Id: Ic79fec6b19c185a6e094dfc9571ea783a1e07148
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3423
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2021-08-26 13:45:27 +00:00
sterni
2f44743824 docs(nix/buildLisp): document NIX_BUILDLISP_LISP_ARGS
Change-Id: I54294da08ee08bcf6cba6c792a2a2235b988a778
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3422
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 11:26:52 +00:00
sterni
ea6ee1e219 docs(nix/buildLisp): move example further up
Make "Example" the second section again since it got a bit buried under
a lot of detailed documentation you won't necessarily need right away.

Change-Id: I481354d1761c590e5872dfce8c3cf9934e278673
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3421
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-08-26 11:26:52 +00:00
sterni
2dd8bc7977 fix(nix/buildLisp): prevent image loader from parsing arguments
CCL and SBCL create executables by dumping their image. As a
consequence, some part of the respective compiler is embedded in the
resulting executable which is executed and loads the image. For CCL and
SBCL this piece of software seems to unconditionally parse arguments
which can't be prevented since it happens before any lisp is loaded.

Luckily in both cases the parsing stops at `--`, so we can just pass
this via the wrapper — we just need to work around the problem that this
will of course be left in argv and confuse any later code. This can be
rectified by deleting everything prior to the first `--` in the global
argument list on startup in both cases.

In cases we do want to pass arguments to the image loader, we can use
the special NIX_BUILDLISP_LISP_ARGS environment variable which is
understood by the wrapper.

Note: This fix doesn't interfere with ECL since it is not using the
wrapper script at the moment.

Fixes b/136.

Change-Id: I3f95aa61e945e51428021ca18232ff15c923f870
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3357
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-08-26 11:26:52 +00:00
Vincent Ambo
a0be4fd902 chore(3p/nixery): Bump to latest version
Change-Id: Idd5b5646cd9c7e6b97c3957f31f7419258e0097c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3411
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-08-25 13:55:02 +00:00
Vincent Ambo
da5a5ab2e4 feat(tverskoy): Attempt #2 to install Steam
Change-Id: I79bcd37f498ec8a337b65b069d085c672b830ee9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3247
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-25 09:53:04 +00:00
sterni
2fa32b563f docs(nix/buildLisp): document recent changes to buildLisp
Doing this in a separate CL to avoid having to track the intermediate
changes no one will ever see in documentation as well which would be
unnecessary effort.

* Multi-implementation support introduced in cl/3292 and refined in
  cl/3368 in terms of the user interface.

* Implementation specific srcs and deps introduced in cl/3321

* Implementation passthru attrs and rename from .sbcl -> .repl was done
  in cl/3359

* ECL added in cl/3297, CCL in cl/3350

Change-Id: Ia13f2aea4e7e091c00991fcbfc601de364413979
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3380
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
7df7cd6257 feat(nix/buildLisp): pass implementation description instead of name
Instead of using a string to refer to an internal set defined in
buildLisp, we just expose the relevant sets (as nix.buildLisp.sbcl,
nix.buildLisp.ecl, …) and receive them as the `implementation`
argument directly. This has several advantages:

* It becomes easier to extend buildLisp, even for downstream users:
  Since you can just pass your own set, there's nothing stopping you
  from adding support for another implementation in a downstream
  derivation without having to edit the buildLisp file in any way which
  is great if you're using e. g. builtins.fetchGit to import it.

* Users can mess with the implementation set by changing out some parts
  of it for customization purposes. Note that currently the sets use a
  lot of self-references which aren't even bound by a fix-point, so to
  make this work smoothly, we'd need to add some overriding mechanism.

* The buildLisp code becomes quite a bit clearer. Since we're now always
  dealing with the implementation set, the confusing distinction between
  `impl`, `impl.name` and `implementation` no longer exists. `impl` is
  now exclusively an abbreviation of `implementation` (we could make
  this more consistent in the future even).

Change-Id: I36d68069dd1315610b2f7159941507b465469b7c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3368
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
2021-08-24 22:00:15 +00:00
sterni
d7e70b1d72 feat(nix/buildLisp): add ccl
This adds support for Clozure's CL implementation to buildLisp. This is
quite trivial in comparison to ECL since SBCL and CCL have very similar
in how they work (so much so that CCL also suffers from b/136).

Also the similarities in the code actually added here are striking, so
I'll try to make an effort to reduce the code duplication in the
future.

To fix builds with CCL the following changes were made:

* //3p/lisp/nibbles: The double inclusion of the types.lisp file was
  fixed. CCL doesn't like double definitions and refuses to compile
  otherwise.

* //3p/lisp/physical-quantities: Update to a new bug fix release which
  contains a compilation fix for CCL.

* //3p/lisp/routes: apply a patch fixing the build which was previously
  failing due to a double definition.

* //3p/lisp/usocket: only depend on sb-bsd-sockets for SBCL and ECL, the
  latter of which seems to have a SBCL compatible implementation of the
  package.

* Conditionally include a few CCL-specific source files and add
  `badImplementation` entries for the remaining failures which are
  //fun/gemma (to be expected) and //web/panettone which fails with an
  incredibly vague message.

Change-Id: I666efdc39a0f16ee1bb6e23225784c709b04e740
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3350
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
02566cdcfb feat(nix/buildLisp): add ecl
Adds ECL as a second supported implementation, specifically a statically
linked ECL. This is interesting because we can create statically linked
binaries, but has a few drawbacks which doesn't make it generally
useful:

* Loading things is very slow: The statically linked ECL only has byte
  compilation available, so when we do load things or use the REPL it is
  significantly worse than with e. g. SBCL.

* We can't load shared objects via the FFI since ECL's dffi is not
  available when linked statically. This means that as it stands, we
  can't build a statically linked //web/panettone for example.

Since ECL is quite slow anyways, I think these drawbacks are worth it
since the biggest reason for using ECL would be to get a statically
linked binary. If we change our minds, it shouldn't be too hard to
provide ecl-static and ecl-dynamic as separate implementations.

ECL is LGPL and some libraries it uses as part of its runtime are as
well. I've outlined in the ecl-static overlay why this should be of no
concern in the context of depot even though we are statically linking.

Currently everything is building except projects that are using cffi to
load shared libaries which have gotten an appropriate
`badImplementations` entry. To get the rest building the following
changes were made:

* Anywhere a dependency on UIOP is expressed as `bundled "uiop"` we now
  use `bundled "asdf"` for all implementations except SBCL. From my
  testing, SBCL seems to be the only implementation to support using
  `(require 'uiop)` to only load the UIOP package. Where both a
  dependency on ASDF and UIOP exists, we just delete the UIOP one.
  `(require 'asdf)` always causes UIOP to be available.

* Where appropriate only conditionally compile SBCL-specific code and
  if any build the corresponding files for ECL.

* //lisp/klatre: Use the standard condition parse-error for all
  implementations except SBCL in try-parse-integer.

* //3p/lisp/ironclad: disable SBCL assembly optimization hack for all
  other platforms as it may interfere with compilation.

* //3p/lisp/trivial-mimes: prevent call to asdf function by substituting
  it out of the source since it always errors out in ECL and we hardcode
  the correct path elsewhere anyways.

As it stands ECL still suffers from a very weird problem which happens
when compiling postmodern and moptilities:
https://gitlab.com/embeddable-common-lisp/ecl/-/issues/651

Change-Id: I0285924f92ac154126b4c42145073c3fb33702ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3297
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: eta <tvl@eta.st>
2021-08-24 22:00:15 +00:00
sterni
0285ea7eac feat(nix/buildLisp): expose drvs built w/ the other implementations
For every implementation we support an extra passthru attribute with the
name of the implementation is created which points to a version of the
derivation built with that implementation. E. g. if we support CCL, ECL
and SBCL, third_party.lisp.alexandria would have:

* third_party.lisp.alexandria.sbcl
* third_party.lisp.alexandria.ecl
* third_party.lisp.alexandria.ccl

To make this possible, the REPL derivation which was called `sbcl`
originally has been renamed to `repl`.

Since some things won't build with all implementations, we introduce a
brokenOn argument which influences the meta.targets list that is
created, but won't prevent the passthru attrs from being created to
ease debugging failures.

Change-Id: Icd6af345143593fac30ded10deabf31172e5d48a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3359
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
ee6b2003fc feat(nix/buildLisp): implementation specific deps and srcs
Both the deps and srcs arguments may now have special “filter sets” in
the lists they receive as arguments. When building, buildLisp checks if
such sets either have a attribute named like the current implementation
or a "default" attribute. If yes, the set is replaced by the respective
attribute's value. If no, the set is removed from the list without
replacement.

This can be used to add elements for (a) specific implementation(s):

  { sbcl = buildLisp.bundled "sb-posix"; }

  { sbcl = ./sbcl/optional-sbcl.lisp; }

or to switch between files for different implementations:

  # If a implementation case is missing and no default set present,
  # no file will be added. Compilation will likely fail as a result.
  {
    ecl = ./tf-ecl.lisp;
    ccl = ./tf-ccl.lisp;
    sbcl = ./tf-sbcl.lisp;
  }

or to account for special behavior for a certain implementation:

  {
    ccl = ./ccl-quirk-impl.lisp
    default = ./ansi-impl.lisp;
  }

Change-Id: I082c3701d1f5063b92100bf336a83425471c269d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3321
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
acb5994e87 feat(nix/buildLisp): allow implementation-specifc bundled functions
By implementing a bundled function for an implementation, we can use a
custom one for a specific implementation. This is useful for
implementations like ECL where a require will be compiled as an
instruction rather than importing all new symbols into a dump, so using
the underlying static or shared object directly would be beneficial.

overrideLisp for bundled libraries now only allows overriding the name
and implementation arguments.

Change-Id: I9036b29157e8daa4d86ff87d603b044373711dbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3301
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
sterni
d344637fe2 refactor(nix/buildLisp): prepare multi implementation support
Concept is roughly:

* receive extra argument `implementation` that refers to the name of an
  implementation or rather an attribute in an internal attribute set
  telling buildLisp how to do certain build steps.

* We assume an implementation can execute lisp files as scripts and that
  we can implement the following main tasks in lisp:

  - Building a library (`genCompileLisp`)

  - Building an executable (`genDumpLisp`)

  - Loading a library dynamically (`genLoadLisp`)

  Based on that we can implement:

  - Running a test suite (`genTestLisp`)

  - A REPL preloaded with a libraries and their dependencies (`lispWith`)

  Additional attributes for implementing these parts genericly  are
  added as needed (`faslExt` and `runScript`).

* `genCompileLisp` no longer prints a shell script which concatenates
  the individual FASLs. Instead it does the step previously done by the
  shell script itself. In essence `genCompileLisp` now writes a lisp
  script which compiles and installs the library to build.
  This will allow us extra freedom for different implementations, e. g.
  for ECL we'll want to build a object file archive additionally to fasl
  files in order to be able to link proper executables.

* `genLoadLisp` and `genTestLisp` are almost generic (the former just
  sometimes would need to use different file extensions), but we
  integrate them into the implementation “API” to facilitate minor
  tweaks we need to do like the `fasc` extension for ECL's native FASL
  files.

Change-Id: I1b8ccc0063159638ec7af534e9a6b5384e750193
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3292
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 22:00:15 +00:00
Vincent Ambo
708fba53c3 fix(3p/gerrit): Fix hash mismatch in dependency blob
Change-Id: I8c3392e6b524b3868013df91fe5a7d3094ee757c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3409
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2021-08-24 21:58:24 +00:00
sterni
1b731abe55 chore(3p): Bump NixOS channels to 2021-08-24
Change-Id: I727660fda72e4274304d56d2d4bd35c3164ae73c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3402
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
2021-08-24 19:23:59 +00:00