I got the message proto wrong on this one as well - it needs both a path
and a signatures.
Change-Id: I9a489b1285bda61c15b2a3b47d9cfc3b50e387da
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1270
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Implement the proto handler on the server side for
Worker::BuildDerivation. This includes several additions to the proto
which I had missed on the first pass, including the actual proto
definition for the Derivation itself and a few sequence number
reorderings which are fine because this is all provisional and not
deployed yet.
A couple things to note
- I implemented a couple constructors for nix classes that initialize
themselves based on their proto variants, which felt nice and didn't
end up causing any issues.
- I've made the conversions between the enum types in nix and in proto
explicit via switch statements rather than using a static_cast, out of
an abundance of caution that the error would get mismatched in the
future and we'd convert the wrong thing to the wrong thing - this is
verbose, but exceptionally future proof.
Change-Id: Iecf6b88e76bc37e49efa05fd65d6cd0cb0deffed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1249
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: Kane York <rikingcoding@gmail.com>
This silences a -Winconsistent-missing-override
Change-Id: I2492a8ab3c0b0a33a794f012bfee6bbd0c2f0a8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1271
Reviewed-by: Kane York <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
Interpolating a path into a string will copy the referenced path into
the Nix store, so this got a dependency on all of src. By first
constructing a path to the src/proto directory using the + operator,
and then interpolating it, we limit what is copied to the store, and
therefore what code triggers a nix-proto-srcs rebuild.
Change-Id: I8dd750f6bc5902b74ffb56470bc8a5f2c01c8cf1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1263
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
This store implementation is required in all unit tests that use the
evaluator.
Change-Id: I1cfe8cecab8722cd66dc803747821a2be2b2619f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1269
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
This change does away with the previous special-casing of lists of
certain element sizes, and the use of raw C-style arrays.
Lists are now backed by a std::vector of nix::Value*, which uses the
traceable GC allocator.
This change is unfortunately quite noisy because the accessor methods
were updated/removed accordingly, so all callsites of Nix-related
lists have changed.
For some operations in primops.cc where keeping the previous code
structure would have been more difficult with a "proper" vector, the
implementation has been replaced with std::vector methods. For
example, list concatenation now uses appropriate range inserts.
Anecdotally the performance of this is about equal, to even slightly
better, than the previous implementation.
All language tests pass and the depot paths I've used for testing
still evaluate.
Change-Id: Ib5eca6c0207429cb323a330c838c3a2200b2c693
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1266
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
This has several advantages:
* we can ensure that the vector is traced by the GC
* we don't need to unsafely allocate memory to make an Env
Note that there was previously a check about the size of the
environment, but it's unclear why this was the case (git history
yielded nothing interesting) and it seems to have no effect.
Change-Id: I4998b879a728a6fb68e1bd187c521e2304e5047e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1265
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
This breaks tooling for some people, and we aren't actually using the
git submodules.
Change-Id: I5b4dfd4ad76bf72e9dbc7de29f17f28ebf6b8ba0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1255
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
all of the executables that get built during regular development depend
on this being set to a directory that contains the nix directory -
previously we had been doing it manually every time, this automates it.
Change-Id: I4c957c0abf0a92ca7122a47d3b141a8ede280e13
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1258
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Configures the CMake build to load & run the GoogleTest tests.
I (grfn) also updated this to get the tests running as part of the nix
derivation, which required defining our own manual configurePhase and
installCheckPhase, rather than depending on the one provided by stdenv.
Not doing this would cause cmake to attempt to *run* the tests as part
of the buildPhase, which wouldn't work because the dynamic libraries
hadn't been put into a place where the test executables knew where to
find them. We're not sure *why* this fixes it, and for some reason
fixing this also breaks the automatic behavior of nixpkgs of passing
-j$NIX_BUILD_CORES -l$NIX_BUILD_CORES to make, but that's eaasy enough
to fix manually in a preBuild
Paired-With: Griffin Smith <grfn@gws.fyi>
Change-Id: I79d61854a3ff47301cdce8a40c76820a97bdf901
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1240
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Also adds the missing check_contents field to the VerifyStoreRequest
proto message, since it was missed in the original pass. This is done
using a renumbering, which is fine in this case since the proto hasn't
been deployed yet
Change-Id: I92bf4e48a71a25ae02ae02b3deaf6e7c71fe5da7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1237
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Reviewed-by: Kane York <rikingcoding@gmail.com>
Change-Id: I8b1d84799a608a516d0b4980022a7edd545a1ca1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1235
Tested-by: BuildkiteCI
Reviewed-by: Kane York <rikingcoding@gmail.com>
Change-Id: I243d67b0bce29d54c7d6e08f5eee70bd395cf9a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1234
Tested-by: BuildkiteCI
Reviewed-by: Kane York <rikingcoding@gmail.com>
Backported from:
b3e5eea4a9fcd048a526
Intentionally skipped because we have not backported the JSON changes:
9f46f54de4
Did not apply changes ni primops.cc, because those look suspect and
are also based on something that we don't have in our tree.
Change-Id: I837787ce9f2c90267bc39fce15177980d209d4e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1253
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
We want to *trace* the 'Value *' arrays, not garbage-collect them!
Otherwise the vectors/maps can end up pointing to nowhere.
Backported from:
10e17eaa58
Change-Id: I30dc94caa80c9d982e7a14bc67ba2d065e8203aa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1252
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Previously the memory would occasionally be collected during eval since
the GC doesn't consider the member variable as alive / doesn't scan the
region of memory where the pointer lives.
By using the traceable_allocator<T> allocator provided by Boehm GC we
can ensure the memory isn't collected. It should be properly freed when
SourceExprCommand goes out of scope.
Backported from:
d2c371927e
Change-Id: I1f7c745dbc66c7164bee50f4d9b0d437dbc7dd51
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1251
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
This is a blocker to enabling the linter.
Change-Id: I9f8d3cc19c7539086f53474a505362230fc56c04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1245
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
This is a bad idea, it shouldn't exist, nixpkgs doesn't use it.
Change-Id: Ic4d1b936d8f059d5c40f0567af165b02427d7e36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1241
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
These are the queries that are handled in the confusing case statement
in the old daemon implementation, because they have very similar
structure.
Change-Id: Ie7143354f66cef4336dff8072ede9a56271a7e89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1228
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
What it does I will never know. It takes a path and returns a path.
Something happens in the middle.
Change-Id: I499a9df700e5b954c9aaf6d694753ff34e773dfd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1210
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
These files were always implicitly depended upon by worker-protocol.hh, but just so happened also always be included in places where the other ones were already present.
We are likely getting rid of this file sooner rather than later, but in
the meantime this will cause clangd to shut up while I'm editing the
file.
Change-Id: I0d765f8b86828b6612e5483417e452221ea3c5b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1207
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Add QueryValidPaths and QuerySubstitutablePaths, both of which filter a
list of paths based on a set of criteria.
Change-Id: I6aa4647efe82b82dc9582a311643d5f9b6d521d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1203
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Replaces the previous uses of the (ordered!) std::map and std::set
with absl::flat_hash_{map|set}.
After some careful reading it seems that there is actually no
ordering dependency on these types, and the (drop-in) replacements
perform slightly better.
Overall this is not fixing a bottleneck, just a driveby thing.
Change-Id: Ided695dc75676bd58515aa9382df0be0a09c565e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1220
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: isomer <isomer@tvl.fyi>
Tested-by: BuildkiteCI
Several definitions of functions declared in eval.hh were previously
implemented in parser.y, this moves them over to parser.cc.
While this still isn't a reasonable place to keep them, the long-term
fix is more likely to be that eval.hh needs to be split up.
Before we get to that point however, this already gives us the ability
to use tooling with this code.
Change-Id: If06fb655325fe281564047ffab0a0a640428a0ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1219
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Moves several of the static helper functions into a new parser.cc
file.
Once the rest of the code is usefully extracted, these will be moved
to a private namespace.
Change-Id: I0d7b53dcefe31bb5c6bad3ad7f5fcb48276bf799
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1218
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
First step (of many?) towards extracting all the inline code from the
Yacc file and keeping it somewhere more accessible instead.
Note that none of this code has previously been touched by a linter or
formatter, pretty much ever, so as it is extracted it also undergoes
similar changes to the whole codebase after the initial fork.
Change-Id: If3b7181f22e3b3fd8c58dfa9befa7ee2896ea06d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1217
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Instead of manually iterating over the two bindings to be combined,
this adds a new static method on the Bindings class which merges two
attribute sets by calling the range insertion operator over them.
In some anecdotal tests, this can lead to a ~10% speed bump -
depending on the specific operation.
Change-Id: I5dea03b0589a83a789d3a8a0fc81d0d9e6598371
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1216
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Changes the derivation name & README overview to say "Tvix" instead of
"tazjix".
The previous name was mostly intended as a joke, and a way for me to
distinguish output paths. It's certainly not the intention to have a
portmanteau with my name here, especially now that several people are
contributing to the fork.
Change-Id: Icface5484d52355111eca23b2f6bd3b9e5567275
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1212
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
