Main motivation for this is to get the openldap update that fixes
10 CVEs: CVE-2020-36221 to including CVE-2020-36230. See also this
issue which lists them all: https://github.com/NixOS/nixpkgs/issues/113490
Someone should also redeploy whitby as soon as this lands in canon and
all build failures have been fixed.
Things done to resolve upstream breakages:
* grpc no longer takes abseil-cpp as an input, it has also been removed
in the override.
* Upgrade glittershark's kernel to 5.11 since the linuxPackages_5_9
attribute has been removed by upstream and the patch used by them is
available for 5.11 as well.
* The fixed output hash for third_patry.apereo-cas changed for some reason.
* Remove the pin of haskellPackages.vector from the haskell overlay. It
broke as the most recent version of vector in nixos-unstable no longer
depends on semigroups. This effectively updates vector from 0.12.1.2
to 0.12.2.0.
* Align two comments in tvix/libstore/worker-protocol.hh because the
updated clang-format now demands that.
Change-Id: I2ecf10a98de935e9222acf1feaea447d4c11ed2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2538
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
Nix internally differentiates between lambdas and primops, but their
type in the nix expression language is the same (lambda). The
implementation of builtins.functionArgs only checks if the given
expression is of type tLambda and fails if the type is tPrimop or
tPrimopApp which are also functions. This most notably breaks
lib.generators.toPretty when called on a builtin making for example
yants fail if a primop is typechecked and an error message is
generated.
This fix generates an empty set for primops like for plain lambdas
and is based upstream commit b2748c6e99239ff6803ba0da76c362790c8be192.
Additionally we add to two tests:
* eval-okay-functionargs now includes a few test cases checking that
builtins.functionArgs always returns an empty set for builtins and
also works as expected for normal functions.
* eval-okay-types now also checks if builtins are functions.
Future work would be to make builtins.functionArgs work as users would
expect for builtins like builtins.fetchurl, builtins.fetchGit etc. which
take a set as an argument. These currently don't register as formal
arguments, but it would be an usability improvement at least if they
did.
See also https://github.com/NixOS/nix/pull/3626#issuecomment-698546704
Change-Id: I2bf4cb80d44a4b72ade13d3e0dbd7dfb1d049f32
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2477
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: glittershark <grfn@gws.fyi>
Once again the sha256 of the fetchgit fixed output derivation for check
changed which was brought to light by the recent GC on whitby.
Change-Id: Ib3c3b5b489717ac6d73631282f27e4363d4ac5c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2481
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Nobody has actually done any experimentation with typed Go, so we're
getting rid of it for now - it's causing annoying IFD during build
graph generation.
Change-Id: Ibac3dea98ebed1b3ee08acda184d24c500cf695d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2458
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Recent channel updates have caused some issues for telega.el, mostly
because the version of tdlib (the C++ library for Telegram) and the
Emacs package are out of sync.
This overrides the version used in the Emacs package to a "known good"
commit. It would be useful to change the tdlib derivation in nixpkgs
to make this version mismatch a hard build error.
Change-Id: I9c994f783e1cc17e933432507cd13b65697efd4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2445
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Uses inotify to watch a file and print when it is modified, so we can
update the parser and display the sexp on the terminal.
Now the setup is good enough to start experiementing with queries on
the syntax tree.
Change-Id: I091587fc495ff627c79a69a52915aaaa8c51fcd2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2411
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Your regularly scheduled channel update, but slightly more regular
than before.
Included fixes:
* 3p/emacs: Pick telega.el from stable channel, unstable is broken.
* glittershark/fprintd: Compile with gcc9, since build fails with the
new default of gcc10
* glittershark/fprintd: Use a global overlay for the fprintd package
until https://github.com/NixOS/nixpkgs/pull/108962 lands in
nixos-unstable
* glittershark/home: Don't install rr, as it's not building with gcc10
Co-Author: Griffin Smith <grfn@gws.fyi>
Change-Id: Ia715fef64a405a220049fc540017356fa7370e0b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2341
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
The owners plugin should in theory be able to match on subdirs (at
least according to its documentation, but it doesn’t and nobody has
any idea how to debug it.
We already know that subdirectories work just fine, so let’s go the
path of least resistance because frankly, I couldn’t care any less.
The haskell overlay also moves to the subdir, this way both can be
changed in the same go by the same people.
Change-Id: I7d98f48afa649ad2c58e38e674e1c4df09039c1c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2347
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Running this after a codified refactor acts as a good smoke test,
if a big subset of packages is broken or any central packages are
broken, this should find them quite quickly, thanks to randomness™.
Just let it run for a few minutes and check the errors that pop up.
Change-Id: I1505dd31ca25b29254474a15cd6cb71d9743038a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2346
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: lukegb <lukegb@tvl.fyi>
In recent Chrome versions, EXWM has some issue around handing focus
back to the application. There is a Github issue about this and this
commit implements the suggested workaround, which I've verified
locally.
Change-Id: Ib451e8d8b34921665c3015853850d12e04612929
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2342
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Changes:
* ops/nixos/tvl-slapd: The NixOS module for OpenLDAP has removed the
ability to configure OpenLDAP directly and now forces users to use
some kind of weird Nix->OLC mapping that is mostly undocumented.
This moves the config we need to the new format in a way that may or
may not work and does the other arbitrary dance steps that someone
decided to impose on us. Note that this now throws lots of warnings,
but I can't be bothered to fix them.
* 3p: Random package removals accomodated
* users/glittershark: Pin grfn's kernel to 5.9, because the CK patch
is not yet updated for 5.10
* users/glittershark: Update vendor hash for pg-dump-upsert, I suspect
this changed because of something in the Go build machinery in
nixpkgs. The deleteVendor flag also has no effect anymore and has been
removed.
* users/glittershark: agda build is broken, commenting out development
home-manager environment until it can be fixed
* third_party/haskell_overlay: updating random needs upper boundarles
of a few dependencies relaxed (curse them)
* third_party/gerrit_plugins: for some cursed reason the fixed-output
hash of the gerrit owners plugin fetchgit changed, updated.
Same for the checks plugin.
Change-Id: Ica37995fe8039d3ba80eab643867f98795c56734
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2295
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
The exposed package list has to be changed/amended quite frequently,
every time somebody wants to use a package not yet in that list and
thus has to whitelist it here.
This effectively requires a superowner review every single time, which
is an unreasonable blocker for many CLs.
I thus propose moving the list into a separate file (I called it
`nixpkgs-whitelist.nix` which is more descriptive than `exposed.nix`
and letting anybody add themselves to the OWNERS on that file.
Change-Id: Ied8bac066e4b9a91ddd642db805fe33dc37872c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2323
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
A bunch of writer functions wrapping the `buildRustCrate`
functionality of nixpkgs. Can be used to write inline rust code, or
rust code read from files with `builtins.readFile`.
Change-Id: I9d74e9381b858b485925e4dc3fbb7fc392877c0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2318
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Smol writer to create a python lib directly from a nix string.
The resulting library can be consumed by the writePython3 writer.
Change-Id: Id3d793564d230b38a08f65140bda4287285e1a72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2310
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Has a little setup to get the cursor position and map it onto a tree
sitter node. The current node is saved in a cursor variable, and a
highlight overlay marks the range of the current node in the buffer.
Change-Id: I0af56115f928732e993fbefe978a246ca7c757ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2258
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
We ended up dropping the use of this library again.
Change-Id: I2c44cd22a6128d23f87a582402bf5fb84991d608
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2292
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Since we don't have a Bindings implementation with unstable order this
function is not required, as its callers can just iterate over the
attributes instead.
Change-Id: I01b35277b5a2dde69d684bc881dbd7c0701bcbb3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2291
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
We don't have an avatar provider at the moment, so it's wonky. I'll make
this a CL upstream as well once my work laptop has charged.
Change-Id: I79754560b2de6981508ba7e10faf6b50cb9c3f8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2266
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
* rebases the send scope patch
* removes the API key override patch, lieer now has a configuration
option for this
Change-Id: I198e8b61855f6cdb2b1439a1c8f2d9d69261c1b5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2242
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
As the comment above the inherit statement says it is supposed to only
contain packages required for //third_party/nix. As much as I like IRC I
can't find a usage of Quassel within Nix itself. Perhaps a plugin to
report build status at some point?
Until that plugin mainfests we might as well move it to the correct
location.
Change-Id: I14fd6b698bb779b80829e7bebd5daa69cacdde78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2177
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
To make it easier for future tests to use the arbitrary specialisations
we've defined for some nix types, centralize them all in a single
arbitrary.hh header file.
Change-Id: I767f27949cfe7ec55c79901f7d7aa538d2f98c6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2182
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Add missing break statements to the big switch block in
BuildResult::FromProto, and cover the whole thing with a rapidcheck
round-trip test.
Change-Id: I7a07ca398cc5f02ca4fd8e6256fd563c6f3aea9d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2178
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Similarly to how we did for buildPaths, add a std::ostream& log_sink
parameter to the build_derivation method on Store, and pass it std::cerr
when called at the top level by nix commands - most notably, the
build-remote hook binary, so that we get build logs when using tvix as a
remote builder.
Change-Id: I0f8f729ba8429d4838a0a135a5c2ac1e1a95d575
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2176
Tested-by: BuildkiteCI
Reviewed-by: andi <andi@notmuch.email>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Upstream doesn't, and for good reason - this might not be a store path,
for example if it's /bin/sh
Fixes: b/73
Change-Id: If9aa96de2cd8ab941c098a9f122b8b302a92ec38
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2175
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
The --quiet and -v/--verbose flags were removed during the glog
conversion in d0c44425e1, which has
previously broken programs like e.g. home-manager, which passes --quiet
to nix-build.
A nix-build-specific workaround was added in
24f9354d5b, which manipulates the
FLAGS_stderrthreshold global variable from glog. This commit moves the
--quiet logic back into the argument handling code in libmain, and adds
corresponding handling for -v/--verbose.
Change-Id: I13d860ebbb78541d9f1236691a1efe8bd2163c67
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2170
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
This was referencing a nonexistent note in buildPaths, for one, but for
another let's get log outputs when calling this RPC.
Change-Id: Ic9d17834b356ea84d69692ccc0249d09777e833b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2173
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Subclasses of std::streambuf are expected to override overflow in
addition to xsputn, as it's called in certain cases by the non-virtual
methods. In our case, this was preventing endlines from getting sent
over the log stream.
Change-Id: I70d00f0c7cb8f8cf2f744f58974c21e7a70a715b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2172
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
- A couple of log messages had double-newlines still
- Remove the [ drvPath ] bit from the "building path" output, since the
derivation path is already present in the message itself.
Change-Id: Ib4b514a18fcb63c7191f4d7cf0a9b857aaa3f3fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2171
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
This allows users to use C-A C to access the QEMU console while within
the VM terminal. From there you can control the process (e.g. kill it).
Change-Id: If2eec2e385fd27fc4e20921dcad78be7390f9d2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2167
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
For the build of tvix we can safely ignore all the files in src/protos
as well as all the nix files in the root of the repository. By passing
the input sources through builtins.filterSource we can make a per-file
decision whether or not we want to copy it into the store (in other
words: if we consider it relevant for the build).
This enables much faster development on the actual build expression.
Change-Id: Ib4c01dbe9cbfa9770922f6257af2a7259814ce0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2159
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
The different signedness of level and withLevel was causing implicit conversions. Use a nullopt instead of a -1 sentinel value.
third_party/nix/src/libexpr/nixexpr.cc:242:21: warning: narrowing conversion from 'unsigned int' to signed type 'int' is implementation-defined [bugprone-narrowing-conversions]
Change-Id: I7c2cadb6fd6bbff6c5b84028651ad4ebba423297
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2157
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
Implicit signedness conversion was causing warnings. Perform a static_cast; this value should never be out of range.
third_party/nix/src/libmain/shared.cc:310:12: warning: narrowing conversion from 'unsigned int' to signed type 'int' is implementation-defined [bugprone-narrowing-conversions]
Change-Id: I34fe53d186ae33f970147135bbe3894fb39a0ec7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2156
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
third_party/nix/src/libstore/sqlite.cc:89:33: warning: narrowing conversion from 'unsigned int' to signed type 'int' is implementation-defined [bugprone-narrowing-conversions]
Change-Id: I1dbc01acaecc4f382f63e7108429cd814ff05f4f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2155
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
This is passed by some third-party tools, notably home-manager. At some
point we should fully revamp argument parsing and log verbosity setting,
but for now this gets us to more of a working state
Fixes: b/78
Change-Id: I4134b17315e30145fd43d3659f407d7121ce8876
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2136
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
It seems like the amount of logging is causing things to move a little
slower - even if that's not really the case, it gets in the way of
debugging things.
Refs: b/76
Change-Id: I9ea99a3b16e3307a0b0371bad22d03b0e2175af6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2134
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
To aid in both debugging and (eventually) printing stacktraces, add a
Pos member to a few more Expr variants.
Change-Id: Ic1d2a056fc7e6c07bc3e79fa38845cb4a5da5ca5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2133
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Included fixes for random breakage:
* 3p/awscli: pick from the stable channel; it is broken on unstable
* 3p/googletest: bumped version & removed patches that nixpkgs applies
* 3p/lisp/cffi: bumped library version for SBCL compat
* 3p/nix: fix libsystemd attribute
* 3p/nix: reformatted (clang-format handling of ternaries changed)
* glittershark/home: Use home-manager from nixkpgs
* glittershark/kernel: bumped linux-ck patch hash
* glittershark/kernel: removed "patch patch"
* multi/whitby: Use home-manager from nixpkgs
* tazjin/frog: drop Sourcetrail (it doesn't build currently)
Note that in addition to these changes, some previous CLs updated the
versions of git and cgit which was necessary for this channel bump,
but which could not be done in the same commit due to the nature of
the subtree merges.
Change-Id: If2563e8a68e2750c4b913a976ff7b93b42e8b7f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2110
Tested-by: BuildkiteCI
Reviewed-by: multi <depot@in-addr.xyz>
Reviewed-by: glittershark <grfn@gws.fyi>
This also bumps the stable nixpkgs to 20.09 as of 2020-11-21, because
there is some breakage in the git build related to the netrc
credentials helper which someone has taken care of in nixpkgs.
The stable channel is not used for anything other than git, so this
should be fine.
Change-Id: I3575a19dab09e1e9556cf8231d717de9890484fb
... notably, this includes Abseil's own StatusOr type, which
conflicted with our implementation (that was taken from TensorFlow).
Change-Id: Ie7d6764b64055caaeb8dc7b6b9d066291e6b538f
The Abseil version of `StatusOr` does not come with the status macros
or the `Consume*` family of functions.
This change modifies the existing code to use the common denominator
of the API that is available between Abseil's own implementation of
`StatusOr` and the one from Tensorflow that we are currently using.
Change-Id: I5c37f68636a1fd54d153f95d7303ab8644abb774
Passing a string directly to add_paths like this causes the proto class
to take ownership over the string, meaning when it is destructed it
will *explicitly* free the string. When the string's actual owner (the
derivation struct) then goes out of scope it'll get freed again, causing
a double-free. This fixes that to instead use the copy constructor to
assign to a pointer to a new path, and covers the whole to_proto method
with a rapidcheck test.
Fixes: b/64
Change-Id: I84235bed9104ff430a0acf686d4a96f1e2e9a897
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2106
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
This was accidentally using the proto arena API to assign the derivation
field of a BuildDerivationRequest. We *thought* this was causing a
double free, but even with this change that's still happening. That
said, this change is probably still a good idea since it's using the
proto API as intended.
References: b/64
Change-Id: I950a4eafb214e9113639ea54d2dfd4659b7be931
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2104
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This is to be used for forwarding messages to our IRC channels.
Change-Id: I6362c6f50a22f504588a7161d41a170f4e7a6edc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2073
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
... this should also update my system EXWM.
Change-Id: Idfbbda67613ac678dc2d5f82533e1c6176ab4a28
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2072
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
* bumps to HEAD of master
* include the zen_workaround script in the drv output
Change-Id: I8f5f33a61b9395295c02f3ea5e930a81b3ffa7ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1990
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
The code that calls queryPathInfoUncached explicitly catches the
InvalidPath exception and translates it into a null result - but the RPC
code was throwing a regular old Error for invalid paths. At some point
we should get rid of all the exception-driven control flow in this whole
thing, but in the meantime this gets us back to functional.
Change-Id: I2a38790ee0c691ab0c8394c7738d7693fa42aa10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1980
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
The static_assert is present to force us to change the constructor when a member grows.
Change-Id: Ifa4f4a03eb7ce13cff109cf26ecf0032045905e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1972
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Make all static std::strings constexpr std::string_views, and replace
concatenation with absl::StrCat where necessary.
Technically all of these are constant, so they really don't need to be
top-level statics - and since I'm trying to get rid of as much global
state as possible in preparation for making the nix daemon properly
multithreaded I figured I'd knock these out while I was at it.
Change-Id: Ibd3ad9ef68f0a0eacb135541b39fdb13dae042e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1939
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This is constant, so let's make it a constexpr.
Change-Id: I6a9eb9f740979740f730ebe142d2cbf23dbcfd70
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1961
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Since the daemon is running in threads now rather than forking a process
per connection (thanks to grpc) this static flag to prevent accidentally
initializing a new worker during a build is getting stepped on by
multiple threads. This converts it to thread-local, and also adds an
actual message to the check so that if it hits in the future we know
what's going on.
Fixes: b/58
Change-Id: I07a2f1582e56709c104f79935e5405fa24888f59
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1940
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: V <v@anomalous.eu>
This is mostly to help in tracing through the core dumps for b/58, but
is also generally a good idea while we're still in hardcore
debugging-mode.
Fixes: b/63
Change-Id: Ib816a2a98bf3b8afa6e8b255a74ccb93af6508c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1937
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
This reverts commit a41c3dedb1.
Reason for revert: I believe its time has come.
Change-Id: I4133dc8a3b57b40b280646d1622ed7a0409c5246
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1941
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Write *all* build logs (essentially everything that wasn't a DLOG), not
just the stdout of the builder, to the client-bound log_sink_ that's
threaded through build.cc. These logs are significantly more useful on
the client, both when debugging tvix itself and (more importantly) when
building things with it.
To make this all work, this commit also refactors the passing-around of
the log sink in build.cc slightly - the main reference is now owned by
the Worker class, which already stores pointers to itself on the Goals
it creates, so those Goals can then load the log sink from the Worker.
This change was originally made to allow for logging to the log_sink
from instance methods of the worker, but actually feels significantly
more natural than storing the sink on the goal anyway.
Change-Id: I461685bc7f5c0a326054940ac5971b568dd43f8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1925
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
These were not used anyway, and *may* have been causing a weird series
of problems leading to things sometimes aborting. Or maybe it has
nothing to do with it - the only correlation we've found is from
disassembling the output of a core dump. Regardless, this feels
stylistically like a good idea.
Change-Id: I288b50945e74ea90c118a21767fa4387c9f47f7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1921
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Rather than printing "Nix" in the version string, print "Tvix". This is
cosmetically nice, but also is very useful when making sure I actually
have installed tvix on a system successfully.
Change-Id: Idd1a9954bd66509327f52b7457cfba2f090ab30c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1924
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
This allows distinguishing between an empty value and no value.
Patch ported from upstream at
ba87b08f85
Change-Id: I061cc8e16b1a7a0341adfc3b0edca1c0c51d5c97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1884
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
When the nix daemon starts up, first check (using sd_listen_fds) whether
we have been systemd socket-activated. If so, instead of passing the nix
daemon socket path to grpc, start a manual accept(2) loop, passing the
client file descriptors to grpc via AddInsecureChannelFromFd. There's an
open grpc issue at https://github.com/grpc/grpc/issues/19133 for
building support into grpc to do this automatically, but as of right now
this appears to be the only way to make this happen.
Making this happen, by the way, was a bit of a journey - at one point I
attempted to ServerBuilder's experimental AddExternalConnectionAcceptor
API, and that didn't work either - it appears that the final missing
piece to getting this working was explicitly fcntl(2)ing the client file
descriptors to set O_NONBLOCK before passing them into gRPC. With that
set, this all works inside of the test vm.
Fixes: b/56
Change-Id: I5d2ab2b5b02eb570249b30a9674e115c61b0ab0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1882
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Expose depot to the test vm via NIX_PATH, so we can test the build of
depot packages from inside the test vm.
Change-Id: I26314634895ab1339f5715c0b0b261028eefdaa3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1923
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
These are unused, and I'm not sure they ever actually worked.
Change-Id: I6bb6b4257b7815932581ded7929437dba80961c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1885
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
The various bits (including the unix socket that we listen on) append
/nix to this - so it isn't expected to be in there.
Change-Id: I0d95f2cc5641f610c1bdb03b036fac61dac9842d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1890
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
The nix package is expected to install /etc/profile.d/nix.sh and
/etc/profile.d/nix-daemon.sh into its output - these set the environment
variables expected by the client-side nix user commands, eg NIX_DAEMON.
Fixes: b/52
Change-Id: I5fb964c076f092d0ef7e1d49aca37875c866fb90
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1889
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Rather than sending the entire AddTextToStore request along in a single
message, send it in a stream of chunks using the same metadata-first
approach we've been using for the other store gRPC requests. This fixes
a bug where certain builds could send more data than the maximum gRPC
request size (4194304 bytes, it would appear), resulting in a
RESOURCE_EXHAUSTED error.
The initial chunk size, which is currently constant but should be made
dynamic at some point in the future, has been chosen based on the IPC
bandwidth delay product for tazjin's desktop, rounded up.
Change-Id: I6f0232cdbc98653484816b39855126873fc59a03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1835
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
These are the scripts I use to test the nix daemon interaction with a
non-/nix store directory during development, copied almost verbatim from
my cmake build directory. As such, there's likely a *lot* of cleanup and
deduplication to be done here, but I'm committing these as is in the
hope that others can benefit from them somehow.
Change-Id: I42a10a85e6731fa2014c7ea9738224d678a8376b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1881
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
This reverts commit 475d41f698. I'd like
my derivations back, please.
Changes necessary to get this working:
- Don't depend on `nixpkgs` being in the NIX_PATH for my website - it's
not necessary anyway since emacs 27 is mainline now
- .skip-subtrees on things that shouldn't be evaluated anyway
- Get rid of system/pkgs, and move the one thing in there that *wasn't*
already in third_party (alsi) to third_party
- Drop notifymuch for now - it's not working, and I'll probably get it
landed in nixpkgs before I manage to get it working
- Add __readTree = true to my systems so they get built.
- explicitly disable ci for xanthous, which is failing to build and had
been omitted previously
Change-Id: I20f5e81d6eb7ffe040091a08d75d0cb15304f707
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1864
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
The command line options --arg and --argstr that are used by a bunch of
CLI commands to pass arguments to top-level functions in files go
through the same code-path as auto-calling top-level functions with
their default arguments - this, however, was only passing the arguments
that were *explicitly* mentioned in the formals of the function - in the
case of an as-pattern with an ellipsis (eg args @ { ... }) extra passed
arguments would get omitted. This fixes that to instead pass *all*
specified auto args in the case that our function has an ellipsis.
Submitted upstream at https://github.com/NixOS/nix/pull/3965Fixes: #46
Change-Id: I32b7ee0e5bacf75b2bc43a3f0796f533f4bd5959
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1863
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Previously, MixEvalArgs (a generic data type used to handle --arg,
--argstr, and -I arguments to `nix-build`, `nix eval`, etc.) was storing
the difference between --arg and --argstr by prepending a single
character (either 'E' or 'S') to the value of the arg. This is messy and
un-type-safe, so this commit refactors that to use a proper enum and a
std::pair, which allows us to add a switch and get totality checking.
yay, types!
Change-Id: I883f8db3136d05dda190fac0a1b494386c2ff87b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1862
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
These projects, which are not currently included in CI runs, don't
build at the moment.
Upcoming logic changes would mean that we would start including them
in CI, which is undesirable until they're fixed - but I'm not going to
be doing that now.
Change-Id: I7c337e098be8bff00db6d99fc7236a695f5a85f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1850
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
This is the clang-tidy lint 'google-explicit-constructor'.
There's a whole bunch of breakage that was introduced by this, and we
had to opt out a few types of this (esp. the string formatting crap).
In some cases minor other changes have been done to keep the code
working, instead of converting between types (e.g. an explicit
comparison operator implementation for nix::Pid).
Change-Id: I12e1ca51a6bc2c882dba81a2526b9729d26988e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1832
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
The previous clang-tidy invocation missed some header files, which has
now been rectified.
Change-Id: I31547754fbf52f439dc7aeefb08ab90bd50c4156
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1831
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
.. turns out producing an entire build log on a single line is not
useful.
This does not use `<< std::endl` because we have run into buffering
issues with the implementation of the logs->gRPC sink, but intend to
replace this in the future using a structured sink for BuildEvent
protos rather than a raw stream.
Change-Id: Ia9b05fa804391d389e2ef53ab4436c0ec5cc452e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1828
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
Introduces the actual log sink in several places where we actually
want the build logs to thread through correctly.
This should cover *most* build paths.
Change-Id: I735dff8a79f7e35a5874eb89b4abb980f9703dc2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1827
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
This *should* wire up the builder's logs all the way back through the
gRPC client, where they are then conveniently discarded.
Change-Id: I65f22526d0b5a8b8d90f28665bc1b4bc7f7c802a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1825
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
This passes an output stream for build logs to almost all relevant
functions inside of build.cc by threading it through the
`Goal`-abstraction.
Store calls that create goals but don't have a sink available use the
DiscardLogsSink().
Change-Id: I2c0cb1aec1f9150f33113f4752055cea518ede8b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1824
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
