You can now provide a list of Nix derivations to tvlc to get a git worktree + sparse-checkout containing only the paths needed to build the specified derivations.
Known bugs: even though //third_party is only passed to readdir(), git doesn't know this and includes all of //third_party/*.
Change-Id: I9dccebd3fbff4bb04ebd568175cf0a7e37d71ab3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1826
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This will look up a file in the current worktree of the git repository
enclosing `default-directory'.
In combination with project-find-file this lets me toggle between
switching to a file within a project, and within the whole depot.
Change-Id: Ie1011f10051fc2c4bd4279b0944a79c7edf92f3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1838
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
If this ends up working well I'll extract it to tvl.el
Change-Id: I83722abf33a3346ccc7957c8d64d6381b15c6ee9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1837
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Expose an `sbcl` attribute on packages and programs, to allow for easier
development either with SLY or on a REPL.
Change-Id: Ide4d087a5223561e1fe192ef32dc593c54b5a20e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1834
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This is the clang-tidy lint 'google-explicit-constructor'.
There's a whole bunch of breakage that was introduced by this, and we
had to opt out a few types of this (esp. the string formatting crap).
In some cases minor other changes have been done to keep the code
working, instead of converting between types (e.g. an explicit
comparison operator implementation for nix::Pid).
Change-Id: I12e1ca51a6bc2c882dba81a2526b9729d26988e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1832
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
The previous clang-tidy invocation missed some header files, which has
now been rectified.
Change-Id: I31547754fbf52f439dc7aeefb08ab90bd50c4156
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1831
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
This reverts commit 8fa3087067.
Reason for revert:
This almost worked. We discovered two important issues:
- The daemon startup does not correctly handle the socket passed in by systemd.
- There is some issue with chunking of large calls, running a build for ci-builds resulted in:
tazjin@whitby /depot (canon)> nix-build -A ciBuilds.__allTargets
E20200821 01:42:22.846053 12601 shared.cc:306] error: Rpc call addTextToStore to unix:///nix/var/nix/daemon-socket/socket failed (RESOURCE_EXHAUSTED): Received message larger than max (10889961 vs. 4194304)
Change-Id: Ic5ba4ef06a4953cf71a36b139fe25ea673cb6fee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1802
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
... this is going to break so much stuff. Lets have some fun.
Change-Id: If0185e0323391c7055d47b797083bb5afde57cb5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1829
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
.. turns out producing an entire build log on a single line is not
useful.
This does not use `<< std::endl` because we have run into buffering
issues with the implementation of the logs->gRPC sink, but intend to
replace this in the future using a structured sink for BuildEvent
protos rather than a raw stream.
Change-Id: Ia9b05fa804391d389e2ef53ab4436c0ec5cc452e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1828
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
Introduces the actual log sink in several places where we actually
want the build logs to thread through correctly.
This should cover *most* build paths.
Change-Id: I735dff8a79f7e35a5874eb89b4abb980f9703dc2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1827
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
This *should* wire up the builder's logs all the way back through the
gRPC client, where they are then conveniently discarded.
Change-Id: I65f22526d0b5a8b8d90f28665bc1b4bc7f7c802a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1825
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
This passes an output stream for build logs to almost all relevant
functions inside of build.cc by threading it through the
`Goal`-abstraction.
Store calls that create goals but don't have a sink available use the
DiscardLogsSink().
Change-Id: I2c0cb1aec1f9150f33113f4752055cea518ede8b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1824
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
Only having a third has left me feeling like I need a little more space.
Change-Id: I9424ec8bf2eeb55a62f21dd72b5ee6251670b0b4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1800
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
Disable the prompt for sending thread replies to the channel, tweak the
slack popup buffer settings, add some bindings, and move all the slack
config to its own file.
Change-Id: I670394942bd9b7e4cfc22953227c21d50a315c22
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1798
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
This part of the store API needs to carry a handle to the log sink
from now on, so that it can be passed in as appropriate from the gRPC
handlers.
In all places where there is no such handler available at the moment,
the discarding log sink has been inserted. This can be used as a
convenient grep target in the future.
Change-Id: I26628e30b4c6437dccdf8f722ca2e8ed827dfc19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1797
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
this shuts up a few lints ...
Change-Id: I51ee9418767e97247f98b170cdad137e5b12b44d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1796
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
This is not actually legal code, but it kind of ... works. There are
more of these around, these were just the ones Griffin stumbled upon
while working on the build logs.
Change-Id: Iff9821d8fe145dd426648a8ff4510a73f67c9b7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1795
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
Introduces a class which implements std::streambuf by sending build
log lines to the provided gRPC stream writer as individual messages.
This can be used in the implementations of calls which trigger builds
to forward logs back to the clients.
Change-Id: I3cecba2219cc24d56692056079c7d7e4e0fc1e2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1794
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
Introduces a new `BuildEvent` proto type which is streamed in response
to calls that trigger builds of derivations.
This type can currently supply build statuses, log lines and
information about builds starting.
This is in preparation for threading build logs through the processes.
Since we have nowhere to send the logs (yet), a null sink is used
instead.
Co-authored-by: Griffin Smith <grfn@gws.fyi>
Change-Id: If7332337b89506c7e404cd20174acdaa1a3be4e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1793
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
In some cases we don't have anywhere for the build logs to go. Until
we understand those cases fully and can get rid of them, this null
sink implementation can be used.
Change-Id: Ib93c43caf268e2c01c43d59737a829e8c43d223e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1792
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
depot-scanner is a tool that runs Nix and parses the --trace-file-access output to deduce what files are necessary to evaluate a derivation.
Take DEPOT_ROOT from the environment. If depotRoot doesn't exist, print an error early.
Fix the build of the protobuf library. Switch to the GRPC build rule, as a service is in this proto file.
Create the PathType enum and parse it from cmdline flags.
Change-Id: I537b5c6bceecf76ca510f7ac04ab9dad7785feb1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1769
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
tvlc is a tool for managing sparse git worktrees of the TVL depot.
It is still in development; near-term tasks include a setup script,
Nix dependency resolution, worktree removal, and the dispatch
script.
See cs.tvl.fyi/depot/docs/designs/SPARSE_CHECKOUTS.md for more info.
Change-Id: Iad96656f0206178980fe7dcadd3dffe70d690f8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1760
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
The font weight rendering issue on frog is, for now, undefeatable and
I'm resorting to just using a lighter font base weight which makes it
possible to actually see bold things.
Change-Id: Ida10f2e8d728039c9ab76bfab1fd0d36340fbac7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1785
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
This reverts commit 4d113ae9f9.
Reason for revert: Source repository link has been added to the homepage
Change-Id: I21667fab928621ba8973c8598d01920ecbf512ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1801
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
... this in turn enables support for Harfbuzz, which enables support
for ligatures - maybe this will make grfn happy!
Change-Id: I331c5171a829800fe532ca0b0d4910478e63ef1b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1783
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
If the global nix config, or any available user nix config location, does not exist, then the loadConfFile() function will throw and not finish initalizing the Nix configuration.
Change-Id: I6db83bca54d9b66699356d107720603476e32c23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1657
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Now that we have access to a store in tests, we can enable the tests that needed a store.
Additionally, move the expected output files for disabled tests into the disabled folder.
Change-Id: I2492d49d43b93c7c9b0463e4d3d2855a5a51365d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1758
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Adds a lot more information about what is actually going on here, as
well as links to relevant things such as our Monorepo document.
I'm aware that I didn't make all `//...` links clickable, but I
realised at some point that it might be easier to just update cheddar
to support highlighting those natively :-)
Change-Id: Icbb212a6c07a5cf38ab8e65d83a64bec783eb8d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1768
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Adds the ability to post to multiple channels by simply running
multiple instances of clbot.
We should probably implement support for this in clbot itself, but
right now I can't be bothered to write Go.
Change-Id: I5cffd0dc10a7f6cc19c37c5834c5610166b4ae23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1771
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
This is now available in the upstream channel, and I need it on one
machine for $reasons.
Normally the Emacs attribute is versioned to make it clear which
version we're dealing with, but not in this case. I've added an assert
to check that it is indeed 27.
Change-Id: I827df0d36350066bf4e177150d32af0c8d123b9b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1764
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
This bumps the channel to a commit that includes fixes for an nginx
issue we have been seeing:
https://github.com/NixOS/nixpkgs/pull/95264
Includes the following compatibility fixes:
- tests disabled in third_party.bufbuild: These were enabled
unexpectedly by the update, but don't run in the sandbox because
they want to download things from github
Change-Id: I98a3b5de57f62f1fd3a37701fa1896eddeedff85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1759
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
