highlight.js supports syntax highlighting .nix files, but the Gerrit
diff components doesn't map the .nix mimetype onto the nix language.
.nix appears to already be taken by another mimetype that isn't
Nix-related, but we just map that onto nix anyway.
https://usercontent.irccloud-cdn.com/file/AVhoPvrb/highlighted.png
Change-Id: I842b29c78355e5bec580e711e25d693284ab6f59
Reviewed-on: https://cl.tvl.fyi/c/depot/+/253
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: q3k <q3k@q3k.org>
gerrit.Watcher is a class which watches the Gerrit stream-events SSH
connection and produces events.
There's a basic CLBot binary as well, to demonstrate driving it to
produce messages on the logging output. It doesn't really do anything
else.
Change-Id: I274fe0a77c8329f79456425405e2fbdc3ca2edf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/245
Reviewed-by: tazjin <mail@tazj.in>
This uses the actual Bazel build, using a variety of tricks and hacks to
make it actually work.
Bazel really wants to download linux binaries from the internet and run
them. In lieu of trying to fix the build system to not do this, we
instead put bazel inside an FHS environment, which allows the binaries
to find their dependencies.
We also have to patch a few things:
* We use build --nobuild instead of fetch, so we only fetch the
dependencies we actually need for the build and not, say, Windows
binaries.
* We don't remove rules_cc, because we need it as an external
dependency, not bundled.
* We do some manual fixes on the cache before packing, because we need
to remove some in-tree sources (so they don't cause the hash to break,
since the hashes differ each time they're generated), and also remove
some extraneous files.
* We explicitly turn off the repository and disk caches, because the
.bazelrc at the root of the Gerrit tree turns them on, with paths
pointing into the user's home directory.
* detzip is used instead of the zip binary for packing bower_components
into an archive. detzip doesn't create entries for directories, and
also doesn't store most metadata (timestamps, etc.), and uses store
(i.e. uncompressed) compression only. It also sorts the file tree
before writing them into the file.
Change-Id: I572c43f7175067ecb1b85cdf40dda13a52de1439
Reviewed-on: https://cl.tvl.fyi/c/depot/+/252
Reviewed-by: tazjin <mail@tazj.in>
detzip will be used in a patch for the Gerrit bower repository helper, which allows
us to get consistent hashes for the output of fetching the dependencies for the
Bazel build.
Change-Id: I6c87b19815b9d747064108aecbb57ed875d2623b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/251
Reviewed-by: lukegb <lukegb@tvl.fyi>
This adds a little tool that can be used to relay mail to Gmail (and
other SMTP servers). It is intended to be used by Gerrit, which is
incompatible with Gmail's SMTP servers.
Configuration has been tested by performing a few sends through the
tvlbot@tazj.in account.
Note that this is using the standard Gmail SMTP server. Using the
smtp-relay server relies on IP whitelisting, but camden.tazj.in has a
larger number of IPv6 addresses than can be whitelisted (the maximum
is 65k). This means that we are limited to 2000 mails per recipient
per day, which should be fine.
Change-Id: Ie43564d753030f5c800a9cdb4ae98292877d80dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/101
Reviewed-by: edef <edef@edef.eu>
Clang treats function-like macros "correctly", in that, per the C11 spec:
"Each subsequent instance of the function-like macro name followed by a (
[...] is replaced by the replacement list [...]".
Additionally, fprintf is also permitted to be defined as a function-like
macro rather than as a true function: "Any function declared in a header
may be additionally implemented as a function-like macro defined in the
header [...]". The specification then suggests surrounding the name of the
function in parens to avoid this, which is the technique we use here to avoid
the function-like macro being invoked.
The other fix here is to use uintptr_t for some arithmetic, since Git
is expecting an int as the value here and not a pointer.
Having a colon in the path may cause issues, and having the hash
function indicated isn't actually necessary. We now verify the path
format in the tests to prevent regressions.
(cherry picked from commit c65a6fa86aef7bdf51fb4fba7bd31d265619ba3f)
This makes the paths consistent without relying on ordering.
Co-authored-by: edef <edef@edef.eu>
(cherry picked from commit 515c0a263e137a00e82f7d981284dbe54db23247)
Completes the switch from Meson to CMake for the core build system in
Nix.
Meson was added originally because someone else had already done the
work for integrating it in Nix and it was an upgrade from the previous
setup.
However over time it became clear that Meson is not quite mature
enough for projects like Nix that have occasionally peculiar
configuration constraints.
Some issues encountered with Meson (some of these are due to the Meson
setup in Nix):
* Difficulty with generating correct compile_commands.json for
external tools like clangd
* Difficulty linking to libc++ when using clang
* Ugly shell invocations for certain parts of the build system (I want
these to be gone!!!)
This CMake setup mimics the Meson configuration, but there are some
differences (some temporary):
* headers are now included separately for each library (see a previous
commit that changes includes appropriately)
* autoheaders-style configuration is currently hardcoded. Before
blindly copying this I want to evaluate how much of it actually exists
for portability concerns that I don't have (such as support for OS
X).
* Nix is built with libc++ by default.
* [libstore] SQL schema is now inlined via a generated header, not an
included string literal
Abseil is still built as part of this build, rather than an external
dependency, because it chokes on differently configured compiler
invocations.
Note that because of the move to libc++ an unwanted behaviour is
introduced: glog log messages no longer have a body. I have yet to
debug what is going on there.
Previously all includes were anchored in one global mess of header
files. This moves the includes into filesystem "namespaces" (if you
will) for each sub-package of Nix.
Note: This commit does not introduce the relevant build system changes.
This function was a custom (and inefficient in the case of
single-character delimiters) string splitter which was used all over
the codebase. Abseil provides an appropriate replacement function.
Replaces these functions with corresponding functions from Abseil,
namely absl::StripAsciiWhitespace and absl::SimpleAtoi.
In the course of doing this some minor things I encountered along the
way were also refactored.
This also changes the signatures of the various custom readFile
functions to use absl::string_view types.
It is considered bad form to use things from includes in headers, as
these directives propagate to everywhere else and can make it
confusing.
types.hh (which is includes almost literally everywhere) had some of
these directives, which this commit removes.
Suppose I have a path /nix/store/[hash]-[name]/a/a/a/a/a/[...]/a,
long enough that everything after "/nix/store/" is longer than 4096
(MAX_PATH) bytes.
Nix will happily allow such a path to be inserted into the store,
because it doesn't look at all the nested structure. It just cares
about the /nix/store/[hash]-[name] part. But, when the path is deleted,
we encounter a problem. Nix will move the path to /nix/store/trash, but
then when it's trying to recursively delete the trash directory, it will
at some point try to unlink
/nix/store/trash/[hash]-[name]/a/a/a/a/a/[...]/a. This will fail,
because the path is too long. After this has failed, any store deletion
operation will never work again, because Nix needs to delete the trash
directory before recreating it to move new things to it. (I assume this
is because otherwise a path being deleted could already exist in the
trash, and then moving it would fail.)
This means that if I can trick somebody into just fetching a tarball
containing a path of the right length, they won't be able to delete
store paths or garbage collect ever again, until the offending path is
manually removed from /nix/store/trash. (And even fixing this manually
is quite difficult if you don't understand the issue, because the
absolute path that Nix says it failed to remove is also too long for
rm(1).)
This patch fixes the issue by making Nix's recursive delete operation
use unlinkat(2). This function takes a relative path and a directory
file descriptor. We ensure that the relative path is always just the
name of the directory entry, and therefore its length will never exceed
255 bytes. This means that it will never even come close to AX_PATH,
and Nix will therefore be able to handle removing arbitrarily deep
directory hierachies.
Since the directory file descriptor is used for recursion after being
used in readDirectory, I made a variant of readDirectory that takes an
already open directory stream, to avoid the directory being opened
multiple times. As we have seen from this issue, the less we have to
interact with paths, the better, and so it's good to reuse file
descriptors where possible.
I left _deletePath as succeeding even if the parent directory doesn't
exist, even though that feels wrong to me, because without that early
return, the linux-sandbox test failed.
Reported-by: Alyssa Ross <hi@alyssa.is>
Thanks-to: Puck Meerburg <puck@puckipedia.com>
Tested-by: Puck Meerburg <puck@puckipedia.com>
Reviewed-by: Puck Meerburg <puck@puckipedia.com>
(cherry picked from commit c05e20daa1abb3446e378331697938b78af2b3d7)
Replaces the previous implementations which performed sorting with one
that instead walks through the map (which is already sorted) and
yields values from it.
This fixes a handful of language tests because the previous
implementation did not actually yield useful values on the new implementation.
In the change to the backing structure of attribute sets, the
requirement to manually balance the capacity of the structure went
away.
This is a) because Abseil's data structures manage this on their own,
and b) because the new Bindings class is allocated using `new (GC)`
rather than writing into a predefined memory area.
As part of this change functions related to the capacity were
deprecated and set to 0 values, which in turn caused the creation of
new attribute sets to return the same (mutable!) default value in
various cases, leading to "side effects" that caused evaluation
failures.
FWIW, I'm not sure if this optimisation had noticeable performance
impact, but while untangling libexpr it definitely doesn't help trying
to follow what it's doing - so bye, bye!
This wrapper derivation (which assumes that the depot is available at
~/depot) can be used to actually get clangd working with
//third_party/nix.
In my setup I can launch this with M-x eglot, followed by
env
CLANGD_FLAGS='--compile-commands-dir=/home/tazjin/projects/nix-build'
nix-shell -A third_party.nix --run 'nix-clangd' /home/tazjin/depot
This is closer to bug-for-bug compatibility with the previous version,
which would put new elements at the end of the array and (due to the
linear scan) return previous ones.
Emacs is currently subtly broken on nixos-unstable, but I don't care
about debugging that.
To work around it, this reintroduces the NixOS stable channel (20.03)
but as a separate attribute set from which attributes like Emacs can be
picked into //third_party.
The new attribute set API uses the iterators of the btree_map
directly. This requires changes in various files because the internals
of libexpr are very entangled.
This code runs and compiles, but there is a bug causing empty
attribute sets to be assigned incorrectly.
Instead of doing some sort of inline merge-sort of the two attribute
sets, use the attribute sets merge function.
This commit alone does not build and is not supposed to.
This is the first step towards replacing the implementation of
attribute sets with an absl::btree_map.
Currently many access are done using array offsets and pointer
arithmetic, so this change is currently causing Nix to fail in various
ways.
Replaces most uses of `string` with `std::string`.
This came up because I removed the "types.hh" import from
"symbol-table.hh", which percolated through a bunch of files where
`string` was suddenly no longer defined ... *sigh*
This replaces the previous use of std::unordered_set with
absl::node_hash_set.
This type was chosen because the current implementation requires
pointer stability.
This does not yet touch the 'Attr' struct.
As a bonus, the implementation of the SymbolTable struct is now
consolidated into a single header/implementation file pair.
Meson is unable to use CMake in Nix to determine the internal
structure of the Abseil libraries.
This commit adds an explicit list of most of the Abseil targets that
are relevant (so far) and bundles them into a list that is linked
together.
cmake automatically runs a configure hook which breaks the build,
since this isn't actually a cmake project. This hook is now disabled.
Additionally Abseil's sources are linked to an absolute derivation
path when the build launches, as opposed to the relative path used for
development builds.
This applies the modernization fixes listed here:
https://clang.llvm.org/extra/clang-tidy/checks/list.html
The 'modernize-use-trailing-return-type' fix was excluded due to my
personal preference (more specifically, I think the 'auto' keyword is
misleading in that position).
This last change set was generated by a full clang-tidy run (including
compilation):
clang-tidy -p ~/projects/nix-build/ \
-checks=-*,readability-braces-around-statements -fix src/*/*.cc
Actually running clang-tidy requires some massaging to make it play
nice with Nix + meson, I'll be adding a wrapper or something for that soon.
