Commit graph

11214 commits

Author SHA1 Message Date
Griffin Smith
053a98a7d7 revert(tvix): "Disable sandboxing by default until its time comes"
This reverts commit a41c3dedb1.

Reason for revert: I believe its time has come.

Change-Id: I4133dc8a3b57b40b280646d1622ed7a0409c5246
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1941
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-06 21:12:11 +00:00
Griffin Smith
8b7dcfaeba feat(tvix): Write even more build logs to client
Write *all* build logs (essentially everything that wasn't a DLOG), not
just the stdout of the builder, to the client-bound log_sink_ that's
threaded through build.cc. These logs are significantly more useful on
the client, both when debugging tvix itself and (more importantly) when
building things with it.

To make this all work, this commit also refactors the passing-around of
the log sink in build.cc slightly - the main reference is now owned by
the Worker class, which already stores pointers to itself on the Goals
it creates, so those Goals can then load the log sink from the Worker.
This change was originally made to allow for logging to the log_sink
from instance methods of the worker, but actually feels significantly
more natural than storing the sink on the goal anyway.

Change-Id: I461685bc7f5c0a326054940ac5971b568dd43f8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1925
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-09-06 21:12:11 +00:00
Vincent Ambo
87bfc53d7b chore(tazjin/keys): Add S10e key
Change-Id: I3efdd49e23bdc357f15ea189b890e25843cc0255
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1933
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-09-06 17:15:14 +00:00
Vincent Ambo
afc07b1b74 chore(whitby): Double the number of build agents again
The main bottleneck of our builds right now is Nix evaluation, which
means that most of the time is spent idling during builds.

Since we're evaluating in parallel, lets give it a few more builders.

I don't want to go all the way to 64 immediately to first see if we
get any adverse effects from highly concurrent builds running
concurrently (if we do we could group them into different "concurrency
groups" in Buildkite).

Change-Id: Ibc3f89fb59cb4ee471b152ff36887ffe2b39f8f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1932
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-09-06 14:42:40 +00:00
Vincent Ambo
7f1c9bc234 fix(tazjin/emacs): Kill vterm buffers if their process exits
This has somehow stopped being a thing in newer vterm versions,
causing the weird behaviour with my term switcher - buffers with the
correct name were sticking around, but no longer in the right mode.

Change-Id: Ie641eb3db91808d7d1016de1e8ef3ad271c8995e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1931
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-09-05 23:38:12 +00:00
Vincent Ambo
11430f4a4b fix(term-switcher.el): Explicitly fail if buffers are missing
Since upgrading to Emacs 27 I have observed a strange behaviour where
this terminal switcher sometimes fails to select a valid buffer, in
which case it falls through to the case that just opens a new buffer
instead.

This is kind of annoying and to aid in debugging this change makes the
creation of new buffers explicit and fails if no matching buffer is
found.

Note that this is likely not a fix for the issue itself, but it will
help debug what is going on.

Change-Id: I906869aba7d25156aaf92c090b169ce02785b85e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1930
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-09-05 11:45:33 +00:00
Kane York
67e8f477a1 fix(u/riking/dotfiles): source rbenv config correctly
Change-Id: I1cf3dfc2d1edd2cb97ca432de5eb03c434b566e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1929
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
2020-09-03 22:03:16 +00:00
Kane York
68cc9b8224 feat(u/riking/dotfiles): port non-functions of bashrc to fish
Change-Id: I5fff28f67fe721b7662c5931e3b1b2226644598a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1928
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-03 22:03:16 +00:00
Kane York
dd94d3036a feat(u/riking/dotfiles): convert bashrc functions to fish
Change-Id: I8a7d1806a8920d31c0b572a4259eef908339e449
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1927
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-03 22:03:16 +00:00
Griffin Smith
2f7b688389 feat(gs/emacs): Generate org agenda commands from project tags
Auto-generate org agenda commands from project tags of the form
`project__foo_bar`, prefixed with `p` and named based on the first
letters of the words in the project. there is (obviously) some overlap
here but that can be fixed by just adding extra underscores to things to
disambiguate.

Change-Id: If07b15a21d8bcb6df6245e8c6e4731041930ecc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1926
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-09-01 23:06:42 +00:00
Griffin Smith
c4136447b3 refactor(tvix): Remove default buildMode arguments from libstore
These were not used anyway, and *may* have been causing a weird series
of problems leading to things sometimes aborting. Or maybe it has
nothing to do with it - the only correlation we've found is from
disassembling the output of a core dump. Regardless, this feels
stylistically like a good idea.

Change-Id: I288b50945e74ea90c118a21767fa4387c9f47f7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1921
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-01 22:05:21 +00:00
Griffin Smith
adebfbbf24 feat(tvix): Print "Tvix" in the version string
Rather than printing "Nix" in the version string, print "Tvix". This is
cosmetically nice, but also is very useful when making sure I actually
have installed tvix on a system successfully.

Change-Id: Idd1a9954bd66509327f52b7457cfba2f090ab30c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1924
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
2020-09-01 03:30:12 +00:00
Eelco Dolstra
785cb3a754 refactor(tvix): getEnv(): Return std::optional
This allows distinguishing between an empty value and no value.

Patch ported from upstream at
ba87b08f85

Change-Id: I061cc8e16b1a7a0341adfc3b0edca1c0c51d5c97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1884
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-01 02:21:53 +00:00
Griffin Smith
c5f3b12f04 feat(tvix): Support systemd socket activation
When the nix daemon starts up, first check (using sd_listen_fds) whether
we have been systemd socket-activated. If so, instead of passing the nix
daemon socket path to grpc, start a manual accept(2) loop, passing the
client file descriptors to grpc via AddInsecureChannelFromFd. There's an
open grpc issue at https://github.com/grpc/grpc/issues/19133 for
building support into grpc to do this automatically, but as of right now
this appears to be the only way to make this happen.

Making this happen, by the way, was a bit of a journey - at one point I
attempted to ServerBuilder's experimental AddExternalConnectionAcceptor
API, and that didn't work either - it appears that the final missing
piece to getting this working was explicitly fcntl(2)ing the client file
descriptors to set O_NONBLOCK before passing them into gRPC. With that
set, this all works inside of the test vm.

Fixes: b/56
Change-Id: I5d2ab2b5b02eb570249b30a9674e115c61b0ab0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1882
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-09-01 01:07:11 +00:00
Griffin Smith
e472aa016e feat(tvix): Expose depot in the test vm
Expose depot to the test vm via NIX_PATH, so we can test the build of
depot packages from inside the test vm.

Change-Id: I26314634895ab1339f5715c0b0b261028eefdaa3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1923
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-09-01 01:07:11 +00:00
Griffin Smith
912268ed5e feat(gs/system): Add nixbuild.net as remote builder
Change-Id: I3b071cc91af5ee896e88c10d6594333ff4eddf77
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1922
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-09-01 01:07:11 +00:00
Griffin Smith
7a83b8fe40 fix(ops/monorepo-gerrit): Escape escape characters with more escape characters
Escape!

Change-Id: Idec7d17feda13ccf1ec03c41106ec6ed581d9e47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1901
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-31 23:21:45 +00:00
Vincent Ambo
2afa22e512 feat(whitby): Double the number of Buildkite agents
Allow 16 things to happen in parallel, which is useful now that the
CI granularity is on a per-target level.

Change-Id: Ie65dd119ea0666618fbb249613e70a68276db834
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1902
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-31 23:21:16 +00:00
Vincent Ambo
fcb014e43d chore(glittershark/system): Include chupacabraSystem in CI builds
Change-Id: Ic8a63f76a0b19f6b1f9ab836d1418849cce06aa0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1895
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-31 23:14:11 +00:00
Vincent Ambo
77a47fa85c chore(tazjin/nixos): Include camdenSystem and frogSystem in CI
Change-Id: I83c99a853e7a14a41ba87fed9d7988d4a2556920
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1894
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-31 23:14:11 +00:00
Vincent Ambo
9c482d6238 feat(ci): Add subtarget support for builds
We have naturally evolved a distinction between logical and physical
targets.

Physical targets are those which correspond directly to a tree
location on disk and can be built with `-A path.to.files`, while
logical targets are those that are exported from within an expression
but do not have a corresponding file on disk.

This change adds support for exporting logical targets from any tree
location by adding a `meta.targets` attribute containing keys into
itself, which will be consumed by the CI target gathering logic and
included in the generated pipeline.

Note that the labels for subtargets are syntactically different to
emphasise that they do not correspond to a file location. For example,
this change enables 'ops.nixos.whitbySystem' as a subtarget, which is
labeled in CI as `ops/nixos:whitbySystem`.

Change-Id: Ied09647a62c2ba98e3914548e3742ad422c63ecf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1893
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-31 23:14:11 +00:00
Vincent Ambo
61d2d2d503 feat(ops/pipelines): Dynamically generate CI pipeline from targets
Create the pipeline by outputting a file that contains nix-build
invocations for each target's *derivation path*.

Each invocation has a generated Nix expression passed to it with `-E`
which fetches the correct target from the tree while correctly
handling targets with strange characters (such as in Go-packages).

This makes it possible to run target-level granular pipelines. We're
getting somewhere!

Change-Id: Ia6946e389dafd1d4926130bb8891446d6e17133b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1855
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-31 23:14:11 +00:00
Griffin Smith
21690c644b fix(panettone): Automatically reconnect to ldap
Wrap all ldap access in a macro that automatically reconnects and
retries operations that fail due to a connection error, to handle the
case where the ldap server restarts while we still have an open
connection.

Fixes: #44
Change-Id: I4859cf509106e480f97fed17e7f08e0eea909352
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1871
Tested-by: BuildkiteCI
Reviewed-by: eta <eta@theta.eu.org>
2020-08-31 23:03:45 +00:00
Griffin Smith
2bc564bd0d chore(tvix): Add riking to owners
Change-Id: Icefafe6967cb83a43c9e417a29ce8cc97fb3432d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1892
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-31 23:01:44 +00:00
Griffin Smith
2279419ede refactor(tvix): Drop meson.build files
These are unused, and I'm not sure they ever actually worked.

Change-Id: I6bb6b4257b7815932581ded7929437dba80961c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1885
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-31 15:38:16 +00:00
Griffin Smith
a9ca270710 feat(gs/emacs): Add binding for slack-all-threads
Change-Id: I5509be5c8ea10e1bb5f8aad890ecc16716c3f1fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1896
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-31 15:35:22 +00:00
Vincent Ambo
29613e7494 feat(readTree): Use tree location as the value of __readTree
Change-Id: I6a087279b5d819db475f2179cb77c0f55cbd0ac6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1860
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-31 00:46:35 +00:00
Griffin Smith
9b4986c5ed feat(monorepo-gerrit): Auto-link panettone issues
Auto-link b/\d+ in gerrit comments + patch comments to panettone issues.
This is a departure from the previous syntax of #\d+ that we were using
historically, but has been requested by multiple people

Fixes: b/40
Change-Id: I8e556eb6b2f55ae80d945b86b51041751ecb053b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1888
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-30 22:16:50 +00:00
Griffin Smith
de84e491cf fix(tvix): localstatedir should be /nix/var
The various bits (including the unix socket that we listen on) append
/nix to this - so it isn't expected to be in there.

Change-Id: I0d95f2cc5641f610c1bdb03b036fac61dac9842d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1890
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-08-30 16:39:25 +00:00
Griffin Smith
1c64930e9f feat(tvix): Install profile scripts into the output
The nix package is expected to install /etc/profile.d/nix.sh and
/etc/profile.d/nix-daemon.sh into its output - these set the environment
variables expected by the client-side nix user commands, eg NIX_DAEMON.

Fixes: b/52
Change-Id: I5fb964c076f092d0ef7e1d49aca37875c866fb90
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1889
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-08-30 16:39:25 +00:00
Griffin Smith
1ab6e658af refactor(gs/emacs): Break clojure config into its own file
Change-Id: I1638b2026c67820e10719a28d303684a87210f81
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1887
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-30 15:40:53 +00:00
Griffin Smith
de851ec08b feat(panettone): Add nav to the top of the page as well
The absence of the navbar containing the "all issues" and "log out"
links from the top of the page has been a common complaint - initially I
disagreed, but after some time thinking about it I've come around. This
adds the same nav - with the "All Issues" link and the "Log Out" link -
to the top of every page, and also fixes a bug where query params would
prevent the "All Issues" link from being hidden on the "All Issues"
page, which looked especially weird when they were right next to each other.

Change-Id: I1d07175fa07aee057ddd140a6864d01342fbb7ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1868
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
2020-08-29 16:43:59 +00:00
Griffin Smith
74a8c3d359 fix(tvix): Chunk the AddTextToStore request
Rather than sending the entire AddTextToStore request along in a single
message, send it in a stream of chunks using the same metadata-first
approach we've been using for the other store gRPC requests. This fixes
a bug where certain builds could send more data than the maximum gRPC
request size (4194304 bytes, it would appear), resulting in a
RESOURCE_EXHAUSTED error.

The initial chunk size, which is currently constant but should be made
dynamic at some point in the future, has been chosen based on the IPC
bandwidth delay product for tazjin's desktop, rounded up.

Change-Id: I6f0232cdbc98653484816b39855126873fc59a03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1835
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-29 14:29:45 +00:00
Griffin Smith
059d90dd6d feat(tvix): Add scripts for local nix daemon testing
These are the scripts I use to test the nix daemon interaction with a
non-/nix store directory during development, copied almost verbatim from
my cmake build directory. As such, there's likely a *lot* of cleanup and
deduplication to be done here, but I'm committing these as is in the
hope that others can benefit from them somehow.

Change-Id: I42a10a85e6731fa2014c7ea9738224d678a8376b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1881
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
2020-08-29 01:52:29 +00:00
Griffin Smith
96ea76bd59 feat(panettone): Add styling for blockquotes
Style blockquotes (which show up in rendered markdown) similarly to how
github does, by rendering a 5px-wide margin to the left with some
padding.

Fixes: #48
Change-Id: I79aa3b6cda5d928885c2cc36f504009232252c17
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1869
Tested-by: BuildkiteCI
Reviewed-by: eta <eta@theta.eu.org>
2020-08-29 01:47:33 +00:00
eta
9e118c6d69 feat(paroxysm): upload queries for all quotes to eta's pastebin
- Currently, asking for all your quotes either stalls the bot, or
  doesn't result in you getting all your quotes, or both. This aims
  to resolve this oversight by shoving them all in a pastebin.
- This uses the lovely `crimp` library by tazjin, which is really
  good at just doing HTTP stuff with minimal fuss. Amazing!
  (although we should probably actually use the depot version)
- Everything is hard coded for now, but we probably don't care.
- Stuff expires after 24 hours, for privacy reasons?
- We also had to add a function to format entries without colours,
  and took the opportunity to clean up the format!() a bit.

Change-Id: I6e75968c7da48a51fff327355b8fa2c025d0db75
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1872
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-28 23:57:51 +00:00
Griffin Smith
13088392e2 fix(gs/emacs): Load slack after defining color vars
Change-Id: Ife6164bd4b4d68f82a01d34354c19c1a90c943ff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1870
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-28 22:32:46 +00:00
Griffin Smith
74a6a9c817 feat(panettone): Increase session timeout to 90 days
The default was really annoyingly short - 90 days feels perfectly fine
for what we want, though we may want to increase even further.

Fixes: #19
Change-Id: I917abd95c4925f8491cd2be7cd87d91bb6621153
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1867
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-28 21:51:23 +00:00
Griffin Smith
1d9a2cac09 feat(panettone): Configure a session secret
Load a SESSION_SECRET env var and set it as the hunchentoot session
secret if present, so that restarting panettone doesn't destroy all
sessions due to the secret getting regenerated.

Refs: #19
Change-Id: Ia2c633fa998e128ecece66e824df01c430da8235
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1866
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
2020-08-28 21:51:23 +00:00
Griffin Smith
d8701f7e21 feat(gs/home): Alias grom = git rebase origin/master
Change-Id: I8dfc3efc393dd5a3a3ad20cfb84969a8d8805f87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1865
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-28 21:24:36 +00:00
Griffin Smith
a7cd92770d revert(glittershark): "Temporarily skip grfn's subtree"
This reverts commit 475d41f698. I'd like
my derivations back, please.

Changes necessary to get this working:

- Don't depend on `nixpkgs` being in the NIX_PATH for my website - it's
  not necessary anyway since emacs 27 is mainline now
- .skip-subtrees on things that shouldn't be evaluated anyway
- Get rid of system/pkgs, and move the one thing in there that *wasn't*
  already in third_party (alsi) to third_party
- Drop notifymuch for now - it's not working, and I'll probably get it
  landed in nixpkgs before I manage to get it working
- Add __readTree = true to my systems so they get built.
- explicitly disable ci for xanthous, which is failing to build and had
  been omitted previously

Change-Id: I20f5e81d6eb7ffe040091a08d75d0cb15304f707
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1864
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-28 21:24:36 +00:00
Griffin Smith
fb0528c5da fix(tvix): Pass all args when auto-calling a function with an ellipsis
The command line options --arg and --argstr that are used by a bunch of
CLI commands to pass arguments to top-level functions in files go
through the same code-path as auto-calling top-level functions with
their default arguments - this, however, was only passing the arguments
that were *explicitly* mentioned in the formals of the function - in the
case of an as-pattern with an ellipsis (eg args @ { ... }) extra passed
arguments would get omitted. This fixes that to instead pass *all*
specified auto args in the case that our function has an ellipsis.

Submitted upstream at https://github.com/NixOS/nix/pull/3965

Fixes: #46
Change-Id: I32b7ee0e5bacf75b2bc43a3f0796f533f4bd5959
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1863
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-08-28 01:18:19 +00:00
Griffin Smith
3bada1d41e refactor(tvix): Clean up MixEvalArgs internals
Previously, MixEvalArgs (a generic data type used to handle --arg,
--argstr, and -I arguments to `nix-build`, `nix eval`, etc.) was storing
the difference between --arg and --argstr by prepending a single
character (either 'E' or 'S') to the value of the arg. This is messy and
un-type-safe, so this commit refactors that to use a proper enum and a
std::pair, which allows us to add a switch and get totality checking.
yay, types!

Change-Id: I883f8db3136d05dda190fac0a1b494386c2ff87b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1862
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-28 01:18:19 +00:00
Vincent Ambo
4ff9d5dee8 feat: Implement automatic CI target detection for the depot
Automatically walk the entire depot tree and pick out things that are
"buildable", then include them in the attribute `ci.targets` (which is
now also the target for CI builds).

A long time ago, in a land far away, we (well, I, at the time) had a
prototype of this which ran into constant issues with infinite
recursions while trying to walk the tree. In fact, this is why
readTree originally gained the `__readTree`-attribute which marks
things that were imported automatically.

Based on some code edef whipped up earlier (with the breakthrough
being that we also add the attribute to top-level folders, which
suddenly resolves a whole bunch of problems), I've now implemented
this actually working version.

At the moment all builds still happen as one big bag of builds, but at
some point we will granularise this.

Change-Id: I86f12ce7f63dae98e7e5c6646a4e9d220de783f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1854
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-26 23:49:32 +00:00
Vincent Ambo
88317aea0d fix(besadii): Remove unused Nix code from the derivation
This was used previously when build granularity was besadii's task,
which it no longer is.

Change-Id: I6df2db1ed4730a7953199b7b48aa9ad916418b22
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1853
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 23:49:32 +00:00
Vincent Ambo
5ad61ee408 chore: Add __readTree attributes on systems that need to be built
This is a temporary state (TODO added) to be picked up by the new CI
logic.

Change-Id: Id4702740ffd18325088e2a8a0c6157a8cee7ccf7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1852
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-26 23:49:32 +00:00
Vincent Ambo
eb3c4a3c11 fix(3p/haskell_overlay): Fix imports and remove unused file
Change-Id: I1973bc1b11442dc8150c3b07bd06a6f94c4c4380
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1851
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 23:49:32 +00:00
Vincent Ambo
7e6b3e1b42 chore: Add meta.ci=false to various broken projects
These projects, which are not currently included in CI runs, don't
build at the moment.

Upcoming logic changes would mean that we would start including them
in CI, which is undesirable until they're fixed - but I'm not going to
be doing that now.

Change-Id: I7c337e098be8bff00db6d99fc7236a695f5a85f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1850
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 23:49:32 +00:00
Vincent Ambo
970f90c711 fix(multi/home-manager): Prevent readTree failures on home-manager.nix
This file is not a readTree-compatible Nix file, but rather a NixOS
module. At some point it should be moved elsewhere and .skip-subtree'd
to avoid this issue.

Change-Id: If1b3f7cc80084af1f44036b8b9272f7b76438c2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1849
Tested-by: BuildkiteCI
Reviewed-by: multi <depot@in-addr.xyz>
2020-08-26 23:49:32 +00:00
Vincent Ambo
dbda84bac3 fix(readTree): Add __readTree attribute to folders
This is required to automatically walk the tree (see subsequent
commits).

Note: Lisp packages are removed from the CI builds in this commit
because the attrValues of third_party.lisp will contain an element
that is simply `true`, which causes a type error.

These packages are re-added when CI refactoring is complete.

Change-Id: I21e2b719e6c7161c23d2867a216f4daa1c6c8394
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1848
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-26 23:49:32 +00:00