This also incidentally includes a fix for _not_ loading fonts from
Google Fonts, but I don't really care about that.
Change-Id: I6e00791d0ba06cb1e3c029e1b8617c33000d2ab1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1041
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Backport of b244e65cdb
Fixes https://github.com/NixOS/nix/issues/3175
------
Original fault description by puck:
I suspect I found the issue: The Nix Command structs are stored on the
heap (using ref<T>, a wrapper around std::shared_ptr<T>), which means
that any pointers that the NixRepl struct contains are eligible to be
reaped by the GC. This includes, but is not limited to, the Env
pointer, which seems to cause most of the random segfaults, or random
other values in the environment, which seems to be what @arianvp
experienced too.
Change-Id: I376d7cfd432daaa6f1fbbf77788ff048082f34e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1001
Tested-by: BuildkiteCI
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
Previously the nix-daemon would crash if a user fed it invalid store
paths for drv files. The crash was due to the changed assertion
triggering. Whenever that assertion would hit the nix-daemon process
along with all it's current childs (running builds from all users) would
be interrupted.
Before this patch:
$ nix-store --realise /nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.drv
don't know how to build these paths:
/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.drv
error: unexpected end-of-file
< nix-daemon terminates >
With this patch:
$ nix-store --realise /nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.drv
don't know how to build these paths:
/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.drv
error: path '/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.drv' is not a valid store path
< nix-daemon does *NOT* terminate >
Change-Id: I01c5048c8a43a8b9154bdeb781d05b7744869ec0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/981
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
This takes us about halfway through worker-protocol.hh
I have left out the documentation strings for some of these items
because I don't feel that I can currently write an unambigous
description of them. For now I am just attempting to match the types.
Change-Id: Iae64b1676152fe4ea069e2021b75ad76465cf368
Reviewed-on: https://cl.tvl.fyi/c/depot/+/960
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
- X-Forwarded-Proto support so it knows it's behind TLS
- Remove extraneous logs and just log to stdout so it's caught be systemd
Change-Id: I650777bbfd24a1922f26967ffff7da06d14b6639
Reviewed-on: https://cl.tvl.fyi/c/depot/+/952
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
This reverts commit 8b58593193.
Reason for revert: Google has fixed the upstream issue and this is not necessary anymore.
Change-Id: Ib1f7cfbc0e7e0997239514e9be75d2deab8c3c67
Reviewed-on: https://cl.tvl.fyi/c/depot/+/961
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Integrate the keyboard layout for my Ergodox EZ, which is a layout for
QMK, with the depot, including exposing several cross-compilation
packages necessary to compile qmk for avr in third_party.
Change-Id: Idd43169a0a3cf0be2bd1a578fdaff70388a58bfc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/947
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
There is a temporary issue affecting the retrieval of discovery
documents in the Google Mail API:
https://issuetracker.google.com/issues/160441983
It's possible to work around this by hardcoding the document instead
of retrieving it over the network, as all other API calls still work
as they should.
This does exactly that by generating a patch to apply to lieer, from
the file checked in to the depot with this commit.
This workaround should be reverted once Google has fixed the issue
upstream.
Change-Id: I0063d0bc67753ffa5261c2aa059c7bfd09136ba0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/945
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
This isn't actually used yet, but forces the protos to be included in
the build which is useful for iteration.
Change-Id: I2abcaf297f34ae741f00ad0c929b226d5603c9d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/928
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Adds dependencies on the gRPC & protobuf libraries, and implements Nix
code to generate the C++ sources from the included proto definitions.
This is theoretically supported via CMake, but practically doesn't
work and I don't care to debug why.
Doing it like this lets us instead add a CMake library target for our
proto definitions based on the sources generated by Nix.
Pros:
* no need to deal with the gRPC CMake mess
* it works!
Cons: * iteration requires nix-shell restart
Change-Id: Ie1fe9807fc96c49cb8f7161ba59d093456062b15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/927
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Adds initial gRPC definitions for the Nix worker protocol, which is
currently defined messily across the following files:
src/libstore/worker-protocol.hh
src/libstore/remote-store.cc
src/nix-daemon/nix-daemon.cc
The protocol definition is basically a big enum with the signatures of
the calls being implicit in the various client/server implementation
functions.
The definitions in this file are slowly reversed from these implicit
signatures, and are likely to contain an error or two which will be
weeded out when this is taken into use.
Only a handful of the calls are included in this commit, it is
intended to get us up and running first.
Change-Id: Ibc9b2ab4b91a064c8935f09f7ac72bb8150fb476
Reviewed-on: https://cl.tvl.fyi/c/depot/+/926
Reviewed-by: isomer <isomer@tvl.fyi>
Tested-by: BuildkiteCI
gRPC 1.29.0 can not be built with our precompiled libs because it
accidentally overrides the C++ standard specified by the users.
This was fixed in 1.30.0.
Change-Id: I9a0390b3f5ec8dae6c295562f3a94b3dfa681dd1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/925
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Same as cl/921, it seems that the trick to making this work is indeed
overriding the C++ standard used in the dependency.
Change-Id: I3c5984d71014d774c161ecc283844f504fd44719
Reviewed-on: https://cl.tvl.fyi/c/depot/+/922
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
This didn't work previously ... but now it does. I think setting the
standard explicitly is what did the trick, but it's slightly unclear
to me why.
Either way this means that Abseil is no longer constantly getting
recompiled when building Nix, which is nice.
Change-Id: I377f7b68bf1ef9045df6a2eee8fdd0c92f243547
Reviewed-on: https://cl.tvl.fyi/c/depot/+/921
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
These bits are no longer required with the hashmap-backed
implementation of attribute sets.
Change-Id: I8b936d8d438a00bad4ccf8e0b4dd719c559ce8c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/912
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Add expression for building haskell-language-server, based on a vendored
version of https://github.com/korayal/hls-nix with hashes updated to
work with our nixpkgs version and ghc 8.8.3. Also add that to CI
builds, so whitby will build it for me (thanks whitby).
Change-Id: I373f8a7cb67974b8aa043b116436c074591b8d57
Reviewed-on: https://cl.tvl.fyi/c/depot/+/897
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: BuildkiteCI
At present, we don't return HTML titles if there's a trailing slash,
or a patchset. Instead, just consume the / and anything after it.
This also fixes /123, because this is HTTP redirected to the full path
*with a trailing slash* which otherwise wouldn't get the title
injected.
Change-Id: Idfd0e67752880a37dce0b400a3c1cfc53fac2912
Reviewed-on: https://cl.tvl.fyi/c/depot/+/859
Reviewed-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
This enables support for the Argon2 password hashing mechanism in
OpenLDAP. Note that we also need to configure the LDAP module to load
this, so this change is not yet sufficient for actually using Argon2
hashes.
Change-Id: I151b854b777daa924b22224a43851432a88a2760
Reviewed-on: https://cl.tvl.fyi/c/depot/+/830
Reviewed-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Tested-by: BuildkiteCI
... and remove a package that doesn't exist anymore (at this location)
from the nixpkgs allowlist.
Change-Id: I663c84c387fb04bb3b47448132ad768ed5352474
Reviewed-on: https://cl.tvl.fyi/c/depot/+/829
Reviewed-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Tested-by: BuildkiteCI
We have this nice `runExecline` now, so we don’t need to use
`runCommand` (which spawns bash) just to write a simple script.
Change-Id: I2941ed8c1448fa1d7cc02dc18b24a8a945b2c38b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/704
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: BuildkiteCI
I've done a small amount of investigation and settled on this as my
favorite gitignore source filter function out of the several that are
available.
Change-Id: Idf1f2f643acc7f8e44de6c0c8702b16e0d37face
Reviewed-on: https://cl.tvl.fyi/c/depot/+/762
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Add a few relatively uncontroversial patches to fix some broken packages
that I had developed for xanthous to the top-level third_party tree, so
they can be reused by other people in the monorepo
Change-Id: I68740477bda278c5dcc123080029ee4bd2cae37a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/740
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: tazjin <mail@tazj.in>
The most trivial of all derivations. It is more useful than it looks.
Can be used to bind nix expressions (e.g. test suites) to a
derivation, so that `nix-build` does not crap itself.
Change-Id: I61c24d8c129c9505733161207f3c30e820f5b15e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/665
Reviewed-by: tazjin <mail@tazj.in>
This is a simple-stupid “unix import system” for nix, for referencing
binaries in `/bin/` by their name and lifting them to a Nix attrset.
Allows for simple aliasing of executable names.
Change-Id: Ifa23cb377201c3b08050c5026e9751e736afaf56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/664
Reviewed-by: tazjin <mail@tazj.in>
This is a writer, similar to `pkgs.writeBashScript` or
`pkgs.writers.writePython3`.
The difference is that we can correctly write all execline scripts by
using nix lists of lists, so the user doesn’t have to care about
escaping arguments (like they have to in bash scripts with
`lib.escapeShellArg` for example).
Change-Id: I2f2874cf61170ddca07b89b692f762725f4a75dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/625
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
This adds configuration which generates the structure expected for
Buildkite pipelines, which can then be dynamically ingested by
Buildkite when a pipeline is triggered.
Change-Id: I61e3dc3affb19c1f2550ef827fa73b17f8d8ae47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/627
Reviewed-by: ericvolp12 <ericvolp12@gmail.com>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Create a store path where the executable `exe` is linked to
$out/bin/${name}. This is useful for e.g. including it as a “package”
in `buildInputs` of a shell.nix.
For example, if I have the exeutable /nix/store/…-hello, I can make it
into /nix/store/…-binify-hello/bin/hello with
`binify { exe = …; name = "hello" }`.
Change-Id: I600bdcd8f143bca2dd8dfbb165a9a5a8d6397622
Reviewed-on: https://cl.tvl.fyi/c/depot/+/624
Reviewed-by: tazjin <mail@tazj.in>
This script creates a pulseaudio sink that will cancel noise in audio
streams sent to it, and then move it on to the default sink.
This means that other people's crackling, static background, gulping,
keyboard sounds, fan whirring, construction noise etc. are removed.
Some preliminary tests on TVL suggest that this actually works. The
parameter might need some tweaking ("50" is just the default value),
as there is some occasional crackling at the beginning/end of a speech
segment, but this is already *much* better than before.
Change-Id: I9d4e2b39cfc2b878b4b7c5458788b8d46fb801af
Reviewed-on: https://cl.tvl.fyi/c/depot/+/577
Reviewed-by: nyanotech <nyanotechnology@gmail.com>
This is a real-time noise suppression plugin for pulseaudio. I want to
cancel other people's noise with this.
Change-Id: Ia3031435e0db35eb80b626ab60d7a62b892d295d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/576
Reviewed-by: Kane York <rikingcoding@gmail.com>
This reverts commit 368e8d1edd.
Reason for revert: Didn't mean to submit, and the phase is currently failing (which breaks the otherwise-functional derivation)
Change-Id: I515b2fb45188dc90f09ae2458453192487c74d71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/581
Reviewed-by: tazjin <mail@tazj.in>
Add an installCheckPhase that runs the appropriate substituteAll on
common.sh and runs the lang.sh tests with the build artifacts in the
PATH.
Change-Id: I2df5a93b8f3ffdfdc194a0e7d6b6669ef520c345
Reviewed-on: https://cl.tvl.fyi/c/depot/+/561
Reviewed-by: glittershark <grfn@gws.fyi>
We don't want traces compiled out since they're an actual language
feature that're used in userspace - also their absence is breaking the
tests
Change-Id: Icaefca8f52e94001785f724fdc0c10a7586b24e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/562
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: lukegbot <bot@lukegb.com>
This includes absl, which we install into the output, and boost and the boehm GC,
which are moved to propagated deps.
Change-Id: I8f9f9795ff92e26b2320359064241d7fd59c2d33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/549
Reviewed-by: tazjin <mail@tazj.in>
This also installs the rest of corepkgs as a side-effect.
Change-Id: I67a42d45793d5e8fdad51c1f306eebf63e9c2868
Reviewed-on: https://cl.tvl.fyi/c/depot/+/548
Reviewed-by: tazjin <mail@tazj.in>
Also fixes the pkgconfig files to use the corresponding CMake variables.
Change-Id: I8095b8aff39ad91e592f3edc95555c9f1f1f153d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/545
Reviewed-by: tazjin <mail@tazj.in>
This ensures that we install both glog's .a and all the .so files we
generate into a single consistent output lib path (which is, err,
lib64, but whatever).
Change-Id: Ib6ac6eacf5f56e4b719cfb586db731efc122c31b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/544
Reviewed-by: tazjin <mail@tazj.in>
Without this patch, this board is not included in the pin map
overrides and audio basically doesn't work.
With this patch, the audio still doesn't work, but it can at least
correctly detect what is and isn't plugged in - so that's progress.
Change-Id: I66ca7d7a1e0e25a0212b9659381875ad4c590ffc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/542
Reviewed-by: lukegb <lukegb@tvl.fyi>
Because this is using %h, we're subject to the whims of the remote
in trying to make sure that we get a consistent commit hash length
in the VERSION stamp.
Change-Id: I716193c1440ec367880b6a5f7dfa4f85a11c19a9
The Go language authors have released an experimental version of Go
that has a type system: https://blog.golang.org/generics-next-step
This overrides the existing Go derivation to build the typed Go. The
next step is a buildTypedGo set of functions that wrap buildGo.
Change-Id: Idb8a4868bca003d821ed5cb324af633398faf002
Reviewed-on: https://cl.tvl.fyi/c/depot/+/443
Reviewed-by: lukegb <lukegb@tvl.fyi>
We can always revert this if we want it back.
Change-Id: I1332b6dd541199584b7b5b94a8651172d79e53a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/442
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>
- This imports the tvldb (actually a thing called 'paroxysm') code
from https://git.theta.eu.org/eta/paroxysm into the monorepo.
- Additionally, I did a nix thing, yay! \o/
(well, with tazjin's help)
- 3p/default.nix needed modifying to whitelist pgsql.
Change-Id: Icdf13ca221650dde376f632bd2dd8a087af451bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/389
Reviewed-by: tazjin <mail@tazj.in>
I need it to deploy my website
Change-Id: I5df8d76d6e0a3d8892ae8bc69d2b46b310f147a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/399
Reviewed-by: tazjin <mail@tazj.in>
This derivation can add arbitrary new Sublime syntaxes to bat's syntax
file, which is used by cheddar.
Included is a Prolog syntax. It is kind of mediocre, but better than
nothing.
Change-Id: I6ceecbc86a5dfba5886cd27bd5114721845446a7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/348
Reviewed-by: lukegb <lukegb@tvl.fyi>
This plugin just blindly assigns everyone and, as q3k has already
pointed out, just isn't particularly useful.
We might want to roll our own, for example:
19: 40:41 <+Remosi> I want the virtual owner thing, we could call it
Gerrit Workgroup Synthesizer Queuing, or gwsq for short.
Change-Id: Ib12a921ae4047ac6a734035dd0900c8964fb12d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/350
Reviewed-by: riking <rikingcoding@gmail.com>
This builds git-bug, a distributed issue tracker that uses git as its
data store.
It also installs its man pages and shell completions. It is
recommended that users add git-bug to their system closure for these
extras to work, as they will not be picked up by the dispatch script.
Change-Id: I1595368e61b0bae8a9497abd023085cb90a521a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/345
Reviewed-by: lukegb <lukegb@tvl.fyi>
Without these changes, the NixOS module isn't able to use the new
Gerrit derivation.
These changes are already deployed as I needed to make them to get
Gerrit back up.
Change-Id: Iad3aa6158789a014134fddccd40b508b81486100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/301
Reviewed-by: lukegb <lukegb@tvl.fyi>
This adds support for overriding the detected languages based on the
filename - we assume here that rules.pl will always map to a Prolog
file.
I could've overridden the entire default language to Prolog, since it's
unlikely that we'll have any Perl here, but given the relative
popularity of the two languages I opted to just override the file we
know we'll have (because it's used by Gerrit itself).
https://usercontent.irccloud-cdn.com/file/yhZZx1nd/highlighted_prolog.png
Change-Id: I26a7e6dab191e0b80a027b026f884020a1f07178
Reviewed-on: https://cl.tvl.fyi/c/depot/+/254
Reviewed-by: tazjin <mail@tazj.in>
highlight.js supports syntax highlighting .nix files, but the Gerrit
diff components doesn't map the .nix mimetype onto the nix language.
.nix appears to already be taken by another mimetype that isn't
Nix-related, but we just map that onto nix anyway.
https://usercontent.irccloud-cdn.com/file/AVhoPvrb/highlighted.png
Change-Id: I842b29c78355e5bec580e711e25d693284ab6f59
Reviewed-on: https://cl.tvl.fyi/c/depot/+/253
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: q3k <q3k@q3k.org>
gerrit.Watcher is a class which watches the Gerrit stream-events SSH
connection and produces events.
There's a basic CLBot binary as well, to demonstrate driving it to
produce messages on the logging output. It doesn't really do anything
else.
Change-Id: I274fe0a77c8329f79456425405e2fbdc3ca2edf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/245
Reviewed-by: tazjin <mail@tazj.in>
This uses the actual Bazel build, using a variety of tricks and hacks to
make it actually work.
Bazel really wants to download linux binaries from the internet and run
them. In lieu of trying to fix the build system to not do this, we
instead put bazel inside an FHS environment, which allows the binaries
to find their dependencies.
We also have to patch a few things:
* We use build --nobuild instead of fetch, so we only fetch the
dependencies we actually need for the build and not, say, Windows
binaries.
* We don't remove rules_cc, because we need it as an external
dependency, not bundled.
* We do some manual fixes on the cache before packing, because we need
to remove some in-tree sources (so they don't cause the hash to break,
since the hashes differ each time they're generated), and also remove
some extraneous files.
* We explicitly turn off the repository and disk caches, because the
.bazelrc at the root of the Gerrit tree turns them on, with paths
pointing into the user's home directory.
* detzip is used instead of the zip binary for packing bower_components
into an archive. detzip doesn't create entries for directories, and
also doesn't store most metadata (timestamps, etc.), and uses store
(i.e. uncompressed) compression only. It also sorts the file tree
before writing them into the file.
Change-Id: I572c43f7175067ecb1b85cdf40dda13a52de1439
Reviewed-on: https://cl.tvl.fyi/c/depot/+/252
Reviewed-by: tazjin <mail@tazj.in>
detzip will be used in a patch for the Gerrit bower repository helper, which allows
us to get consistent hashes for the output of fetching the dependencies for the
Bazel build.
Change-Id: I6c87b19815b9d747064108aecbb57ed875d2623b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/251
Reviewed-by: lukegb <lukegb@tvl.fyi>
This adds a little tool that can be used to relay mail to Gmail (and
other SMTP servers). It is intended to be used by Gerrit, which is
incompatible with Gmail's SMTP servers.
Configuration has been tested by performing a few sends through the
tvlbot@tazj.in account.
Note that this is using the standard Gmail SMTP server. Using the
smtp-relay server relies on IP whitelisting, but camden.tazj.in has a
larger number of IPv6 addresses than can be whitelisted (the maximum
is 65k). This means that we are limited to 2000 mails per recipient
per day, which should be fine.
Change-Id: Ie43564d753030f5c800a9cdb4ae98292877d80dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/101
Reviewed-by: edef <edef@edef.eu>
Clang treats function-like macros "correctly", in that, per the C11 spec:
"Each subsequent instance of the function-like macro name followed by a (
[...] is replaced by the replacement list [...]".
Additionally, fprintf is also permitted to be defined as a function-like
macro rather than as a true function: "Any function declared in a header
may be additionally implemented as a function-like macro defined in the
header [...]". The specification then suggests surrounding the name of the
function in parens to avoid this, which is the technique we use here to avoid
the function-like macro being invoked.
The other fix here is to use uintptr_t for some arithmetic, since Git
is expecting an int as the value here and not a pointer.
Having a colon in the path may cause issues, and having the hash
function indicated isn't actually necessary. We now verify the path
format in the tests to prevent regressions.
(cherry picked from commit c65a6fa86aef7bdf51fb4fba7bd31d265619ba3f)
This makes the paths consistent without relying on ordering.
Co-authored-by: edef <edef@edef.eu>
(cherry picked from commit 515c0a263e137a00e82f7d981284dbe54db23247)
Completes the switch from Meson to CMake for the core build system in
Nix.
Meson was added originally because someone else had already done the
work for integrating it in Nix and it was an upgrade from the previous
setup.
However over time it became clear that Meson is not quite mature
enough for projects like Nix that have occasionally peculiar
configuration constraints.
Some issues encountered with Meson (some of these are due to the Meson
setup in Nix):
* Difficulty with generating correct compile_commands.json for
external tools like clangd
* Difficulty linking to libc++ when using clang
* Ugly shell invocations for certain parts of the build system (I want
these to be gone!!!)
This CMake setup mimics the Meson configuration, but there are some
differences (some temporary):
* headers are now included separately for each library (see a previous
commit that changes includes appropriately)
* autoheaders-style configuration is currently hardcoded. Before
blindly copying this I want to evaluate how much of it actually exists
for portability concerns that I don't have (such as support for OS
X).
* Nix is built with libc++ by default.
* [libstore] SQL schema is now inlined via a generated header, not an
included string literal
Abseil is still built as part of this build, rather than an external
dependency, because it chokes on differently configured compiler
invocations.
Note that because of the move to libc++ an unwanted behaviour is
introduced: glog log messages no longer have a body. I have yet to
debug what is going on there.
Previously all includes were anchored in one global mess of header
files. This moves the includes into filesystem "namespaces" (if you
will) for each sub-package of Nix.
Note: This commit does not introduce the relevant build system changes.
This function was a custom (and inefficient in the case of
single-character delimiters) string splitter which was used all over
the codebase. Abseil provides an appropriate replacement function.
Replaces these functions with corresponding functions from Abseil,
namely absl::StripAsciiWhitespace and absl::SimpleAtoi.
In the course of doing this some minor things I encountered along the
way were also refactored.
This also changes the signatures of the various custom readFile
functions to use absl::string_view types.
It is considered bad form to use things from includes in headers, as
these directives propagate to everywhere else and can make it
confusing.
types.hh (which is includes almost literally everywhere) had some of
these directives, which this commit removes.
Suppose I have a path /nix/store/[hash]-[name]/a/a/a/a/a/[...]/a,
long enough that everything after "/nix/store/" is longer than 4096
(MAX_PATH) bytes.
Nix will happily allow such a path to be inserted into the store,
because it doesn't look at all the nested structure. It just cares
about the /nix/store/[hash]-[name] part. But, when the path is deleted,
we encounter a problem. Nix will move the path to /nix/store/trash, but
then when it's trying to recursively delete the trash directory, it will
at some point try to unlink
/nix/store/trash/[hash]-[name]/a/a/a/a/a/[...]/a. This will fail,
because the path is too long. After this has failed, any store deletion
operation will never work again, because Nix needs to delete the trash
directory before recreating it to move new things to it. (I assume this
is because otherwise a path being deleted could already exist in the
trash, and then moving it would fail.)
This means that if I can trick somebody into just fetching a tarball
containing a path of the right length, they won't be able to delete
store paths or garbage collect ever again, until the offending path is
manually removed from /nix/store/trash. (And even fixing this manually
is quite difficult if you don't understand the issue, because the
absolute path that Nix says it failed to remove is also too long for
rm(1).)
This patch fixes the issue by making Nix's recursive delete operation
use unlinkat(2). This function takes a relative path and a directory
file descriptor. We ensure that the relative path is always just the
name of the directory entry, and therefore its length will never exceed
255 bytes. This means that it will never even come close to AX_PATH,
and Nix will therefore be able to handle removing arbitrarily deep
directory hierachies.
Since the directory file descriptor is used for recursion after being
used in readDirectory, I made a variant of readDirectory that takes an
already open directory stream, to avoid the directory being opened
multiple times. As we have seen from this issue, the less we have to
interact with paths, the better, and so it's good to reuse file
descriptors where possible.
I left _deletePath as succeeding even if the parent directory doesn't
exist, even though that feels wrong to me, because without that early
return, the linux-sandbox test failed.
Reported-by: Alyssa Ross <hi@alyssa.is>
Thanks-to: Puck Meerburg <puck@puckipedia.com>
Tested-by: Puck Meerburg <puck@puckipedia.com>
Reviewed-by: Puck Meerburg <puck@puckipedia.com>
(cherry picked from commit c05e20daa1abb3446e378331697938b78af2b3d7)
Replaces the previous implementations which performed sorting with one
that instead walks through the map (which is already sorted) and
yields values from it.
This fixes a handful of language tests because the previous
implementation did not actually yield useful values on the new implementation.
In the change to the backing structure of attribute sets, the
requirement to manually balance the capacity of the structure went
away.
This is a) because Abseil's data structures manage this on their own,
and b) because the new Bindings class is allocated using `new (GC)`
rather than writing into a predefined memory area.
As part of this change functions related to the capacity were
deprecated and set to 0 values, which in turn caused the creation of
new attribute sets to return the same (mutable!) default value in
various cases, leading to "side effects" that caused evaluation
failures.
FWIW, I'm not sure if this optimisation had noticeable performance
impact, but while untangling libexpr it definitely doesn't help trying
to follow what it's doing - so bye, bye!
This wrapper derivation (which assumes that the depot is available at
~/depot) can be used to actually get clangd working with
//third_party/nix.
In my setup I can launch this with M-x eglot, followed by
env
CLANGD_FLAGS='--compile-commands-dir=/home/tazjin/projects/nix-build'
nix-shell -A third_party.nix --run 'nix-clangd' /home/tazjin/depot
This is closer to bug-for-bug compatibility with the previous version,
which would put new elements at the end of the array and (due to the
linear scan) return previous ones.
Emacs is currently subtly broken on nixos-unstable, but I don't care
about debugging that.
To work around it, this reintroduces the NixOS stable channel (20.03)
but as a separate attribute set from which attributes like Emacs can be
picked into //third_party.
The new attribute set API uses the iterators of the btree_map
directly. This requires changes in various files because the internals
of libexpr are very entangled.
This code runs and compiles, but there is a bug causing empty
attribute sets to be assigned incorrectly.
Instead of doing some sort of inline merge-sort of the two attribute
sets, use the attribute sets merge function.
This commit alone does not build and is not supposed to.
This is the first step towards replacing the implementation of
attribute sets with an absl::btree_map.
Currently many access are done using array offsets and pointer
arithmetic, so this change is currently causing Nix to fail in various
ways.
Replaces most uses of `string` with `std::string`.
This came up because I removed the "types.hh" import from
"symbol-table.hh", which percolated through a bunch of files where
`string` was suddenly no longer defined ... *sigh*
