This adds support for overriding the detected languages based on the
filename - we assume here that rules.pl will always map to a Prolog
file.
I could've overridden the entire default language to Prolog, since it's
unlikely that we'll have any Perl here, but given the relative
popularity of the two languages I opted to just override the file we
know we'll have (because it's used by Gerrit itself).
https://usercontent.irccloud-cdn.com/file/yhZZx1nd/highlighted_prolog.png
Change-Id: I26a7e6dab191e0b80a027b026f884020a1f07178
Reviewed-on: https://cl.tvl.fyi/c/depot/+/254
Reviewed-by: tazjin <mail@tazj.in>
highlight.js supports syntax highlighting .nix files, but the Gerrit
diff components doesn't map the .nix mimetype onto the nix language.
.nix appears to already be taken by another mimetype that isn't
Nix-related, but we just map that onto nix anyway.
https://usercontent.irccloud-cdn.com/file/AVhoPvrb/highlighted.png
Change-Id: I842b29c78355e5bec580e711e25d693284ab6f59
Reviewed-on: https://cl.tvl.fyi/c/depot/+/253
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: q3k <q3k@q3k.org>
gerrit.Watcher is a class which watches the Gerrit stream-events SSH
connection and produces events.
There's a basic CLBot binary as well, to demonstrate driving it to
produce messages on the logging output. It doesn't really do anything
else.
Change-Id: I274fe0a77c8329f79456425405e2fbdc3ca2edf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/245
Reviewed-by: tazjin <mail@tazj.in>
This uses the actual Bazel build, using a variety of tricks and hacks to
make it actually work.
Bazel really wants to download linux binaries from the internet and run
them. In lieu of trying to fix the build system to not do this, we
instead put bazel inside an FHS environment, which allows the binaries
to find their dependencies.
We also have to patch a few things:
* We use build --nobuild instead of fetch, so we only fetch the
dependencies we actually need for the build and not, say, Windows
binaries.
* We don't remove rules_cc, because we need it as an external
dependency, not bundled.
* We do some manual fixes on the cache before packing, because we need
to remove some in-tree sources (so they don't cause the hash to break,
since the hashes differ each time they're generated), and also remove
some extraneous files.
* We explicitly turn off the repository and disk caches, because the
.bazelrc at the root of the Gerrit tree turns them on, with paths
pointing into the user's home directory.
* detzip is used instead of the zip binary for packing bower_components
into an archive. detzip doesn't create entries for directories, and
also doesn't store most metadata (timestamps, etc.), and uses store
(i.e. uncompressed) compression only. It also sorts the file tree
before writing them into the file.
Change-Id: I572c43f7175067ecb1b85cdf40dda13a52de1439
Reviewed-on: https://cl.tvl.fyi/c/depot/+/252
Reviewed-by: tazjin <mail@tazj.in>
detzip will be used in a patch for the Gerrit bower repository helper, which allows
us to get consistent hashes for the output of fetching the dependencies for the
Bazel build.
Change-Id: I6c87b19815b9d747064108aecbb57ed875d2623b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/251
Reviewed-by: lukegb <lukegb@tvl.fyi>
This adds a little tool that can be used to relay mail to Gmail (and
other SMTP servers). It is intended to be used by Gerrit, which is
incompatible with Gmail's SMTP servers.
Configuration has been tested by performing a few sends through the
tvlbot@tazj.in account.
Note that this is using the standard Gmail SMTP server. Using the
smtp-relay server relies on IP whitelisting, but camden.tazj.in has a
larger number of IPv6 addresses than can be whitelisted (the maximum
is 65k). This means that we are limited to 2000 mails per recipient
per day, which should be fine.
Change-Id: Ie43564d753030f5c800a9cdb4ae98292877d80dc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/101
Reviewed-by: edef <edef@edef.eu>
Clang treats function-like macros "correctly", in that, per the C11 spec:
"Each subsequent instance of the function-like macro name followed by a (
[...] is replaced by the replacement list [...]".
Additionally, fprintf is also permitted to be defined as a function-like
macro rather than as a true function: "Any function declared in a header
may be additionally implemented as a function-like macro defined in the
header [...]". The specification then suggests surrounding the name of the
function in parens to avoid this, which is the technique we use here to avoid
the function-like macro being invoked.
The other fix here is to use uintptr_t for some arithmetic, since Git
is expecting an int as the value here and not a pointer.
Having a colon in the path may cause issues, and having the hash
function indicated isn't actually necessary. We now verify the path
format in the tests to prevent regressions.
(cherry picked from commit c65a6fa86aef7bdf51fb4fba7bd31d265619ba3f)
This makes the paths consistent without relying on ordering.
Co-authored-by: edef <edef@edef.eu>
(cherry picked from commit 515c0a263e137a00e82f7d981284dbe54db23247)
Completes the switch from Meson to CMake for the core build system in
Nix.
Meson was added originally because someone else had already done the
work for integrating it in Nix and it was an upgrade from the previous
setup.
However over time it became clear that Meson is not quite mature
enough for projects like Nix that have occasionally peculiar
configuration constraints.
Some issues encountered with Meson (some of these are due to the Meson
setup in Nix):
* Difficulty with generating correct compile_commands.json for
external tools like clangd
* Difficulty linking to libc++ when using clang
* Ugly shell invocations for certain parts of the build system (I want
these to be gone!!!)
This CMake setup mimics the Meson configuration, but there are some
differences (some temporary):
* headers are now included separately for each library (see a previous
commit that changes includes appropriately)
* autoheaders-style configuration is currently hardcoded. Before
blindly copying this I want to evaluate how much of it actually exists
for portability concerns that I don't have (such as support for OS
X).
* Nix is built with libc++ by default.
* [libstore] SQL schema is now inlined via a generated header, not an
included string literal
Abseil is still built as part of this build, rather than an external
dependency, because it chokes on differently configured compiler
invocations.
Note that because of the move to libc++ an unwanted behaviour is
introduced: glog log messages no longer have a body. I have yet to
debug what is going on there.
Previously all includes were anchored in one global mess of header
files. This moves the includes into filesystem "namespaces" (if you
will) for each sub-package of Nix.
Note: This commit does not introduce the relevant build system changes.
This function was a custom (and inefficient in the case of
single-character delimiters) string splitter which was used all over
the codebase. Abseil provides an appropriate replacement function.
Replaces these functions with corresponding functions from Abseil,
namely absl::StripAsciiWhitespace and absl::SimpleAtoi.
In the course of doing this some minor things I encountered along the
way were also refactored.
This also changes the signatures of the various custom readFile
functions to use absl::string_view types.
It is considered bad form to use things from includes in headers, as
these directives propagate to everywhere else and can make it
confusing.
types.hh (which is includes almost literally everywhere) had some of
these directives, which this commit removes.
Suppose I have a path /nix/store/[hash]-[name]/a/a/a/a/a/[...]/a,
long enough that everything after "/nix/store/" is longer than 4096
(MAX_PATH) bytes.
Nix will happily allow such a path to be inserted into the store,
because it doesn't look at all the nested structure. It just cares
about the /nix/store/[hash]-[name] part. But, when the path is deleted,
we encounter a problem. Nix will move the path to /nix/store/trash, but
then when it's trying to recursively delete the trash directory, it will
at some point try to unlink
/nix/store/trash/[hash]-[name]/a/a/a/a/a/[...]/a. This will fail,
because the path is too long. After this has failed, any store deletion
operation will never work again, because Nix needs to delete the trash
directory before recreating it to move new things to it. (I assume this
is because otherwise a path being deleted could already exist in the
trash, and then moving it would fail.)
This means that if I can trick somebody into just fetching a tarball
containing a path of the right length, they won't be able to delete
store paths or garbage collect ever again, until the offending path is
manually removed from /nix/store/trash. (And even fixing this manually
is quite difficult if you don't understand the issue, because the
absolute path that Nix says it failed to remove is also too long for
rm(1).)
This patch fixes the issue by making Nix's recursive delete operation
use unlinkat(2). This function takes a relative path and a directory
file descriptor. We ensure that the relative path is always just the
name of the directory entry, and therefore its length will never exceed
255 bytes. This means that it will never even come close to AX_PATH,
and Nix will therefore be able to handle removing arbitrarily deep
directory hierachies.
Since the directory file descriptor is used for recursion after being
used in readDirectory, I made a variant of readDirectory that takes an
already open directory stream, to avoid the directory being opened
multiple times. As we have seen from this issue, the less we have to
interact with paths, the better, and so it's good to reuse file
descriptors where possible.
I left _deletePath as succeeding even if the parent directory doesn't
exist, even though that feels wrong to me, because without that early
return, the linux-sandbox test failed.
Reported-by: Alyssa Ross <hi@alyssa.is>
Thanks-to: Puck Meerburg <puck@puckipedia.com>
Tested-by: Puck Meerburg <puck@puckipedia.com>
Reviewed-by: Puck Meerburg <puck@puckipedia.com>
(cherry picked from commit c05e20daa1abb3446e378331697938b78af2b3d7)
Replaces the previous implementations which performed sorting with one
that instead walks through the map (which is already sorted) and
yields values from it.
This fixes a handful of language tests because the previous
implementation did not actually yield useful values on the new implementation.
In the change to the backing structure of attribute sets, the
requirement to manually balance the capacity of the structure went
away.
This is a) because Abseil's data structures manage this on their own,
and b) because the new Bindings class is allocated using `new (GC)`
rather than writing into a predefined memory area.
As part of this change functions related to the capacity were
deprecated and set to 0 values, which in turn caused the creation of
new attribute sets to return the same (mutable!) default value in
various cases, leading to "side effects" that caused evaluation
failures.
FWIW, I'm not sure if this optimisation had noticeable performance
impact, but while untangling libexpr it definitely doesn't help trying
to follow what it's doing - so bye, bye!
This wrapper derivation (which assumes that the depot is available at
~/depot) can be used to actually get clangd working with
//third_party/nix.
In my setup I can launch this with M-x eglot, followed by
env
CLANGD_FLAGS='--compile-commands-dir=/home/tazjin/projects/nix-build'
nix-shell -A third_party.nix --run 'nix-clangd' /home/tazjin/depot
This is closer to bug-for-bug compatibility with the previous version,
which would put new elements at the end of the array and (due to the
linear scan) return previous ones.
Emacs is currently subtly broken on nixos-unstable, but I don't care
about debugging that.
To work around it, this reintroduces the NixOS stable channel (20.03)
but as a separate attribute set from which attributes like Emacs can be
picked into //third_party.
The new attribute set API uses the iterators of the btree_map
directly. This requires changes in various files because the internals
of libexpr are very entangled.
This code runs and compiles, but there is a bug causing empty
attribute sets to be assigned incorrectly.
Instead of doing some sort of inline merge-sort of the two attribute
sets, use the attribute sets merge function.
This commit alone does not build and is not supposed to.
This is the first step towards replacing the implementation of
attribute sets with an absl::btree_map.
Currently many access are done using array offsets and pointer
arithmetic, so this change is currently causing Nix to fail in various
ways.
Replaces most uses of `string` with `std::string`.
This came up because I removed the "types.hh" import from
"symbol-table.hh", which percolated through a bunch of files where
`string` was suddenly no longer defined ... *sigh*
This replaces the previous use of std::unordered_set with
absl::node_hash_set.
This type was chosen because the current implementation requires
pointer stability.
This does not yet touch the 'Attr' struct.
As a bonus, the implementation of the SymbolTable struct is now
consolidated into a single header/implementation file pair.
Meson is unable to use CMake in Nix to determine the internal
structure of the Abseil libraries.
This commit adds an explicit list of most of the Abseil targets that
are relevant (so far) and bundles them into a list that is linked
together.
cmake automatically runs a configure hook which breaks the build,
since this isn't actually a cmake project. This hook is now disabled.
Additionally Abseil's sources are linked to an absolute derivation
path when the build launches, as opposed to the relative path used for
development builds.
This applies the modernization fixes listed here:
https://clang.llvm.org/extra/clang-tidy/checks/list.html
The 'modernize-use-trailing-return-type' fix was excluded due to my
personal preference (more specifically, I think the 'auto' keyword is
misleading in that position).
This last change set was generated by a full clang-tidy run (including
compilation):
clang-tidy -p ~/projects/nix-build/ \
-checks=-*,readability-braces-around-statements -fix src/*/*.cc
Actually running clang-tidy requires some massaging to make it play
nice with Nix + meson, I'll be adding a wrapper or something for that soon.
These were not caught by the previous clang-tidy invocation, but were
instead sorted out using amber[0] as such:
ambr --regex 'for (\(.+\))\s([a-z].*;)' 'for $1 { $2 }'
[0]: https://github.com/dalance/amber
These were not caught by the previous clang-tidy invocation, but were
instead sorted out using amber[0] as such:
ambr --regex 'if (\(.+\))\s([a-z].*;)' 'if $1 { $2 }'
[0]: https://github.com/dalance/amber
Previously these structs were declared anonymously inside of the -
anonymous - union. This is not actually supported by the C++ standard,
but is merely a compiler-specific extension.
Unfortunately untangling this required a forward-declaration of the
Value type.
This change was generated with:
fd -e cc -e hh | xargs -I{} clang-tidy {} -p ~/projects/nix-build/ \
--checks='-*,readability-braces-around-statements' --fix \
-fix-errors
Some manual fixes were applied because some convoluted unbraced
statements couldn't be untangled by clang-tidy.
This commit still includes invalid files, but I decided to clean them
up in a subsequent commit so that it becomes more obvious where
clang-tidy failed. Maybe this will allow for a bug-report to
clang-tidy.
Changes the configuration to regroup all includes. The include groups
will be (in this order):
1. (in .cc): Include of the corresponding header
2. Includes of C++ standard library headers
3. Includes of other external headers
4. Includes of local headers
Removes the activity transfer that was previously nulled out from the
daemon protocol completely.
This might actually break Nix completely, I haven't tried yet, but
that's fine because this will be replaced with gRPC.
The progress bar has lots of complexity for little benefit. The
previous activity tracking stuff has been deleted as part of the
logging refactoring and I am not going to implement support for this
again for now.
1. First of all, this doesn't work in nixpkgs. Per [1], gcc ignores `-L`
for purposes of `--print-file-dirs`, which breaks horribly on linux. But
if we don't pass extra dirs, meosn first just tries `-l...`, which does
work.
2. Even if it did work, `libdir` means where we are installing libs, not
where libs are expected to be found. Those are not necessarily the
same (again, nixpkgs), and even when they are and non-standard, it is
better to use DESTDIR or have a modified toolchain.
[1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87758
(cherry picked from commit a142164e746644e20f66908c156ca913bef4664f)
Reverts "Bump channel to a NixOS 20.03 release commit".
This reverts commit a629d7fdd4.
This commit moves to a 20.03 release commit, in which building
Chromium with VAAPI is broken.
These patches enable hardware-accelerated video decoding, which is
useful for Stadia.
The main issue with this is that Hydra doesn't currently cache
Chromium with these patches, which means that it is built from scratch
which takes in the order of 5 hours on an otherwise unused nugget.
Builds ffmpeg with CUDA Toolkit as a dependency, which includes a
library called "libnpp" that provides something related to hardware
accelerated video stream resizing.
v0v
Adds the proto definitions required for the Stackdriver Logging API.
This compiles, but I'm unsure whether it's actually correct because
there seems to be a lot of copy & paste in the build setup.
Updates the build process for googleapis in C++ to read the proto
sources from the GOOGLEAPIS_DIR environment variable (injected by Nix)
instead of attempting to download them at build time.
This adds very basic capability[0] and message tag[1] support to rcirc
which is used to implement support for the IRCv3 server-time[2] spec.
During connection setup, the server is asked to list its capabilities
and the `server-time` capability is then blindly requested from
it (the CAP handler code does not check whether server-time is
actually part of the listed capabilities). rcirc does not need to know
whether this negotiation succeeded, because server time tags will
either be sent or not.
By default rcirc prints all timestamps at current-time. A new variable
`rcirc-last-message-time` has been added which, if set, overrides this
timestamp. It is set by the message handler after parsing IRCv3 tags.
Thanks to William Cummings for nudging me in the direction of his post
about adding ZNC playback support to rcirc[4], from which some parts
of this code were taken.
This has been tested with IRCCloud's bouncers.
[0]: https://ircv3.net/specs/core/capability-negotiation
[1]: https://ircv3.net/specs/extensions/message-tags
[2]: https://ircv3.net/specs/extensions/server-time-3.2.html
This change makes cgit pass the current repo and vpath inside of the
repo on to the about cmd, which makes it possible for it to correctly
render `tree`, `log` and other links to the same vpath.
This removes the ASDF system definition for Gemma and switches the
code over to buildLisp.
The program builds (including some terrifying hacks to get the
frontend to work), but there are some bizarre runtime issues that I
need to debug.
This is pretty much exactly the derivation from nixpkgs, with some
things removed to reduce the closure size (e.g. the various formatters
used in the default cgit version, which are replaced by cheddar in my
setup).
This overrides the upstream derivation to:
* use local sources
* build `git send-email`
It also calls autoreconf before building because files that are
included in the git distribution tarball (which the normal derivation
uses) are missing from source.
If you're on tree/foo/bar, the about link will now point to
about/foo/bar.
Currently the annoying thing about this is that it will also do it for
files.
This means that individual subtrees of a repository will also have
their READMEs rendered on the about page, for example:
/foo/bar/README.md
Will render on:
/about/foo/bar/
This is useful for monorepo setups in which subtrees represent
individual projects.
Generates URLs that do not include the repository name.
On git.tazj.in, only one repository (depot) is served - hence URLs
generated by cgit need not include the name.
Adds dottime (as defined on https://dotti.me) as a timestamp format.
This format is designed to simplify working with timestamps across
many different timezones by keeping the timestamp format itself in
UTC (and indicating this with a dot character), but appending the
local offset.
This is implemented as a new format because the timestamp needs to be
rendered both as UTC and including the offset, an implementation using
a strftime formatting string is not sufficient.
Similar to buildGo.nix, the library derivations carry information
about their dependencies which is merged when a load file is
instantiated.
The load files are created when compiling libraries, but will in the
future also be created when wrapping SBCL and dumping images.
It turns out I'm going to need multiple different OAuth clients for a
variety of reasons. This defaults to the client for tazj.in accounts,
but I use a different one in my work overlay.
This uses Nix to inject the path to the syntax highlighting assets
that ship with the bat source code into the cheddar build at compile
time, where the Rust compiler then inserts it into the binary via
macros.
bat has a lot of custom syntax highlighting definitions that they
collected from all over the place (including for languages like Nix!)
and this makes them accessible to cheddar.
Also if you're reading this, can you just take a moment to appreciate
how incredible it is that Nix just lets us do something like this?!
Packages the telega-server binary and adds the required mode into
Emacs.
Unread message count is displayed in the modeline, which is neat.
Probably need to figure out some key bindings for this.
This enables usage of __dispatch.sh from anywhere, even outside of the
depot.
Specifically this means I can add `~/depot/bin` to my $PATH and all
the registered tools work from anywhere.
These packages contain the Cloud SDK for Go. There is currently a
linker issue (presumably due to something in `buildGo.nix`) that means
that projects using them can not actually be built.
Instead of exposing the entire package tree from nixpkgs, whitelist
individual packages explicitly so that they show up in
`pkgs.third_party`.
This makes it much easier to control external dependencies used by my
projects.
Bonus: It even includes a working `third_party.callPackage` with only
the whitelisted packages!
This is not the final layout yet, but makes it so that my top-level
attribute set is no longer overlaid into nixpkgs itself.
This is useful for other people who are importing my monorepo.
Broadly speaking, the following things are included:
* there is now a uniform `args` struct that is passed to all
derivations, package headers have been changed appropriately
* overrides are now loaded from a separate `override` folder just
using read-tree.nix
* third-party packages have moved into the `third_party` attribute set
