Previously all includes were anchored in one global mess of header
files. This moves the includes into filesystem "namespaces" (if you
will) for each sub-package of Nix.
Note: This commit does not introduce the relevant build system changes.
This function was a custom (and inefficient in the case of
single-character delimiters) string splitter which was used all over
the codebase. Abseil provides an appropriate replacement function.
Replaces these functions with corresponding functions from Abseil,
namely absl::StripAsciiWhitespace and absl::SimpleAtoi.
In the course of doing this some minor things I encountered along the
way were also refactored.
This also changes the signatures of the various custom readFile
functions to use absl::string_view types.
It is considered bad form to use things from includes in headers, as
these directives propagate to everywhere else and can make it
confusing.
types.hh (which is includes almost literally everywhere) had some of
these directives, which this commit removes.
Suppose I have a path /nix/store/[hash]-[name]/a/a/a/a/a/[...]/a,
long enough that everything after "/nix/store/" is longer than 4096
(MAX_PATH) bytes.
Nix will happily allow such a path to be inserted into the store,
because it doesn't look at all the nested structure. It just cares
about the /nix/store/[hash]-[name] part. But, when the path is deleted,
we encounter a problem. Nix will move the path to /nix/store/trash, but
then when it's trying to recursively delete the trash directory, it will
at some point try to unlink
/nix/store/trash/[hash]-[name]/a/a/a/a/a/[...]/a. This will fail,
because the path is too long. After this has failed, any store deletion
operation will never work again, because Nix needs to delete the trash
directory before recreating it to move new things to it. (I assume this
is because otherwise a path being deleted could already exist in the
trash, and then moving it would fail.)
This means that if I can trick somebody into just fetching a tarball
containing a path of the right length, they won't be able to delete
store paths or garbage collect ever again, until the offending path is
manually removed from /nix/store/trash. (And even fixing this manually
is quite difficult if you don't understand the issue, because the
absolute path that Nix says it failed to remove is also too long for
rm(1).)
This patch fixes the issue by making Nix's recursive delete operation
use unlinkat(2). This function takes a relative path and a directory
file descriptor. We ensure that the relative path is always just the
name of the directory entry, and therefore its length will never exceed
255 bytes. This means that it will never even come close to AX_PATH,
and Nix will therefore be able to handle removing arbitrarily deep
directory hierachies.
Since the directory file descriptor is used for recursion after being
used in readDirectory, I made a variant of readDirectory that takes an
already open directory stream, to avoid the directory being opened
multiple times. As we have seen from this issue, the less we have to
interact with paths, the better, and so it's good to reuse file
descriptors where possible.
I left _deletePath as succeeding even if the parent directory doesn't
exist, even though that feels wrong to me, because without that early
return, the linux-sandbox test failed.
Reported-by: Alyssa Ross <hi@alyssa.is>
Thanks-to: Puck Meerburg <puck@puckipedia.com>
Tested-by: Puck Meerburg <puck@puckipedia.com>
Reviewed-by: Puck Meerburg <puck@puckipedia.com>
(cherry picked from commit c05e20daa1abb3446e378331697938b78af2b3d7)
Replaces the previous implementations which performed sorting with one
that instead walks through the map (which is already sorted) and
yields values from it.
This fixes a handful of language tests because the previous
implementation did not actually yield useful values on the new implementation.
In the change to the backing structure of attribute sets, the
requirement to manually balance the capacity of the structure went
away.
This is a) because Abseil's data structures manage this on their own,
and b) because the new Bindings class is allocated using `new (GC)`
rather than writing into a predefined memory area.
As part of this change functions related to the capacity were
deprecated and set to 0 values, which in turn caused the creation of
new attribute sets to return the same (mutable!) default value in
various cases, leading to "side effects" that caused evaluation
failures.
FWIW, I'm not sure if this optimisation had noticeable performance
impact, but while untangling libexpr it definitely doesn't help trying
to follow what it's doing - so bye, bye!
This wrapper derivation (which assumes that the depot is available at
~/depot) can be used to actually get clangd working with
//third_party/nix.
In my setup I can launch this with M-x eglot, followed by
env
CLANGD_FLAGS='--compile-commands-dir=/home/tazjin/projects/nix-build'
nix-shell -A third_party.nix --run 'nix-clangd' /home/tazjin/depot
This is closer to bug-for-bug compatibility with the previous version,
which would put new elements at the end of the array and (due to the
linear scan) return previous ones.
Emacs is currently subtly broken on nixos-unstable, but I don't care
about debugging that.
To work around it, this reintroduces the NixOS stable channel (20.03)
but as a separate attribute set from which attributes like Emacs can be
picked into //third_party.
The new attribute set API uses the iterators of the btree_map
directly. This requires changes in various files because the internals
of libexpr are very entangled.
This code runs and compiles, but there is a bug causing empty
attribute sets to be assigned incorrectly.
Instead of doing some sort of inline merge-sort of the two attribute
sets, use the attribute sets merge function.
This commit alone does not build and is not supposed to.
This is the first step towards replacing the implementation of
attribute sets with an absl::btree_map.
Currently many access are done using array offsets and pointer
arithmetic, so this change is currently causing Nix to fail in various
ways.
Replaces most uses of `string` with `std::string`.
This came up because I removed the "types.hh" import from
"symbol-table.hh", which percolated through a bunch of files where
`string` was suddenly no longer defined ... *sigh*
This replaces the previous use of std::unordered_set with
absl::node_hash_set.
This type was chosen because the current implementation requires
pointer stability.
This does not yet touch the 'Attr' struct.
As a bonus, the implementation of the SymbolTable struct is now
consolidated into a single header/implementation file pair.
Meson is unable to use CMake in Nix to determine the internal
structure of the Abseil libraries.
This commit adds an explicit list of most of the Abseil targets that
are relevant (so far) and bundles them into a list that is linked
together.
cmake automatically runs a configure hook which breaks the build,
since this isn't actually a cmake project. This hook is now disabled.
Additionally Abseil's sources are linked to an absolute derivation
path when the build launches, as opposed to the relative path used for
development builds.
This applies the modernization fixes listed here:
https://clang.llvm.org/extra/clang-tidy/checks/list.html
The 'modernize-use-trailing-return-type' fix was excluded due to my
personal preference (more specifically, I think the 'auto' keyword is
misleading in that position).
This last change set was generated by a full clang-tidy run (including
compilation):
clang-tidy -p ~/projects/nix-build/ \
-checks=-*,readability-braces-around-statements -fix src/*/*.cc
Actually running clang-tidy requires some massaging to make it play
nice with Nix + meson, I'll be adding a wrapper or something for that soon.
These were not caught by the previous clang-tidy invocation, but were
instead sorted out using amber[0] as such:
ambr --regex 'for (\(.+\))\s([a-z].*;)' 'for $1 { $2 }'
[0]: https://github.com/dalance/amber
These were not caught by the previous clang-tidy invocation, but were
instead sorted out using amber[0] as such:
ambr --regex 'if (\(.+\))\s([a-z].*;)' 'if $1 { $2 }'
[0]: https://github.com/dalance/amber
Previously these structs were declared anonymously inside of the -
anonymous - union. This is not actually supported by the C++ standard,
but is merely a compiler-specific extension.
Unfortunately untangling this required a forward-declaration of the
Value type.
This change was generated with:
fd -e cc -e hh | xargs -I{} clang-tidy {} -p ~/projects/nix-build/ \
--checks='-*,readability-braces-around-statements' --fix \
-fix-errors
Some manual fixes were applied because some convoluted unbraced
statements couldn't be untangled by clang-tidy.
This commit still includes invalid files, but I decided to clean them
up in a subsequent commit so that it becomes more obvious where
clang-tidy failed. Maybe this will allow for a bug-report to
clang-tidy.
Changes the configuration to regroup all includes. The include groups
will be (in this order):
1. (in .cc): Include of the corresponding header
2. Includes of C++ standard library headers
3. Includes of other external headers
4. Includes of local headers
Removes the activity transfer that was previously nulled out from the
daemon protocol completely.
This might actually break Nix completely, I haven't tried yet, but
that's fine because this will be replaced with gRPC.
The progress bar has lots of complexity for little benefit. The
previous activity tracking stuff has been deleted as part of the
logging refactoring and I am not going to implement support for this
again for now.
1. First of all, this doesn't work in nixpkgs. Per [1], gcc ignores `-L`
for purposes of `--print-file-dirs`, which breaks horribly on linux. But
if we don't pass extra dirs, meosn first just tries `-l...`, which does
work.
2. Even if it did work, `libdir` means where we are installing libs, not
where libs are expected to be found. Those are not necessarily the
same (again, nixpkgs), and even when they are and non-standard, it is
better to use DESTDIR or have a modified toolchain.
[1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87758
(cherry picked from commit a142164e746644e20f66908c156ca913bef4664f)
Reverts "Bump channel to a NixOS 20.03 release commit".
This reverts commit a629d7fdd4.
This commit moves to a 20.03 release commit, in which building
Chromium with VAAPI is broken.
These patches enable hardware-accelerated video decoding, which is
useful for Stadia.
The main issue with this is that Hydra doesn't currently cache
Chromium with these patches, which means that it is built from scratch
which takes in the order of 5 hours on an otherwise unused nugget.
Builds ffmpeg with CUDA Toolkit as a dependency, which includes a
library called "libnpp" that provides something related to hardware
accelerated video stream resizing.
v0v
Adds the proto definitions required for the Stackdriver Logging API.
This compiles, but I'm unsure whether it's actually correct because
there seems to be a lot of copy & paste in the build setup.
Updates the build process for googleapis in C++ to read the proto
sources from the GOOGLEAPIS_DIR environment variable (injected by Nix)
instead of attempting to download them at build time.
This adds very basic capability[0] and message tag[1] support to rcirc
which is used to implement support for the IRCv3 server-time[2] spec.
During connection setup, the server is asked to list its capabilities
and the `server-time` capability is then blindly requested from
it (the CAP handler code does not check whether server-time is
actually part of the listed capabilities). rcirc does not need to know
whether this negotiation succeeded, because server time tags will
either be sent or not.
By default rcirc prints all timestamps at current-time. A new variable
`rcirc-last-message-time` has been added which, if set, overrides this
timestamp. It is set by the message handler after parsing IRCv3 tags.
Thanks to William Cummings for nudging me in the direction of his post
about adding ZNC playback support to rcirc[4], from which some parts
of this code were taken.
This has been tested with IRCCloud's bouncers.
[0]: https://ircv3.net/specs/core/capability-negotiation
[1]: https://ircv3.net/specs/extensions/message-tags
[2]: https://ircv3.net/specs/extensions/server-time-3.2.html
This change makes cgit pass the current repo and vpath inside of the
repo on to the about cmd, which makes it possible for it to correctly
render `tree`, `log` and other links to the same vpath.
This removes the ASDF system definition for Gemma and switches the
code over to buildLisp.
The program builds (including some terrifying hacks to get the
frontend to work), but there are some bizarre runtime issues that I
need to debug.
This is pretty much exactly the derivation from nixpkgs, with some
things removed to reduce the closure size (e.g. the various formatters
used in the default cgit version, which are replaced by cheddar in my
setup).
This overrides the upstream derivation to:
* use local sources
* build `git send-email`
It also calls autoreconf before building because files that are
included in the git distribution tarball (which the normal derivation
uses) are missing from source.
If you're on tree/foo/bar, the about link will now point to
about/foo/bar.
Currently the annoying thing about this is that it will also do it for
files.
This means that individual subtrees of a repository will also have
their READMEs rendered on the about page, for example:
/foo/bar/README.md
Will render on:
/about/foo/bar/
This is useful for monorepo setups in which subtrees represent
individual projects.
Generates URLs that do not include the repository name.
On git.tazj.in, only one repository (depot) is served - hence URLs
generated by cgit need not include the name.
Adds dottime (as defined on https://dotti.me) as a timestamp format.
This format is designed to simplify working with timestamps across
many different timezones by keeping the timestamp format itself in
UTC (and indicating this with a dot character), but appending the
local offset.
This is implemented as a new format because the timestamp needs to be
rendered both as UTC and including the offset, an implementation using
a strftime formatting string is not sufficient.
Similar to buildGo.nix, the library derivations carry information
about their dependencies which is merged when a load file is
instantiated.
The load files are created when compiling libraries, but will in the
future also be created when wrapping SBCL and dumping images.
It turns out I'm going to need multiple different OAuth clients for a
variety of reasons. This defaults to the client for tazj.in accounts,
but I use a different one in my work overlay.
This uses Nix to inject the path to the syntax highlighting assets
that ship with the bat source code into the cheddar build at compile
time, where the Rust compiler then inserts it into the binary via
macros.
bat has a lot of custom syntax highlighting definitions that they
collected from all over the place (including for languages like Nix!)
and this makes them accessible to cheddar.
Also if you're reading this, can you just take a moment to appreciate
how incredible it is that Nix just lets us do something like this?!
Packages the telega-server binary and adds the required mode into
Emacs.
Unread message count is displayed in the modeline, which is neat.
Probably need to figure out some key bindings for this.
This enables usage of __dispatch.sh from anywhere, even outside of the
depot.
Specifically this means I can add `~/depot/bin` to my $PATH and all
the registered tools work from anywhere.
These packages contain the Cloud SDK for Go. There is currently a
linker issue (presumably due to something in `buildGo.nix`) that means
that projects using them can not actually be built.
Instead of exposing the entire package tree from nixpkgs, whitelist
individual packages explicitly so that they show up in
`pkgs.third_party`.
This makes it much easier to control external dependencies used by my
projects.
Bonus: It even includes a working `third_party.callPackage` with only
the whitelisted packages!
This is not the final layout yet, but makes it so that my top-level
attribute set is no longer overlaid into nixpkgs itself.
This is useful for other people who are importing my monorepo.
Broadly speaking, the following things are included:
* there is now a uniform `args` struct that is passed to all
derivations, package headers have been changed appropriately
* overrides are now loaded from a separate `override` folder just
using read-tree.nix
* third-party packages have moved into the `third_party` attribute set
