tvl-depot/ops
sterni 17d78867bb feat(ops/pipelines/depot): only evaluate once if possible
We currently evaluate every target twice -- once when the depot pipeline
is built and once when actually running the build step in question. Nix
evaluation is quite slow especially given heavy use of import from
derivation in depot, so avoiding the second evaluation is desireable.

Evaluating a derivation yields a `drv` file in the nix store which can
be passed to `nix-store --realise` in order to build it eliminating the
need to wait for evaluation. We can obtain the path to the `drv` file
while building the pipeline via `target.drvPath` and remember it for the
build later.

However we need to work around a flaw (or oversight) in Nix's dependency
tracking via string context: This is based on derivations, not output
path (because this is what evaluation deals with, likely). This is no
problem per se, but an issue is that Nix can't express a dependency on
a `drv` file without any of its output paths. This means for us that we
either have to build all output paths at evaluation time (which we don't
want, obviously) or to deal with the fact that the `drv` file we need
may be garbage collected at any moment after discarding the string
context -- then nix is unable to track the reference from the pipeline
to the `drv` file in the store.

So to prevent a race condition between the pipeline and the garbage
collector we fall back to the normal nix-build invocation as we did
before.

Change-Id: I9ef8bd233085dc6e30eba54f403ea03ac2d35748
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3426
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 15:24:33 +00:00
..
besadii feat(besadii): Tag gerrit comments as autogenerated 2021-08-24 13:00:59 +00:00
dns feat(ops/dns): Point nixery.dev to whitby 2021-08-24 11:53:10 +00:00
gerrit-tvl feat(gerrit-tvl): add Buildkite-backed Checks plugin implementation 2021-04-07 11:19:04 +00:00
journaldriver refactor(ops): Consistent use of depot.third_party vs. pkgs 2021-04-10 12:09:20 +00:00
kontemplate refactor(ops): Consistent use of depot.third_party vs. pkgs 2021-04-10 12:09:20 +00:00
machines feat(ops): Serve nixery.dev from whitby 2021-08-24 11:30:16 +00:00
modules fix(tvl-sso): set memory limit to 512M 2021-08-24 16:28:14 +00:00
mq_cli chore: Rename pkgs->depot in all Nix file headers 2020-02-21 13:54:53 +00:00
pipelines feat(ops/pipelines/depot): only evaluate once if possible 2021-08-26 15:24:33 +00:00
posix_mq.rs chore: Rename pkgs->depot in all Nix file headers 2020-02-21 13:54:53 +00:00
users chore(ops/users): change my email address to tvl@eta.st 2021-08-13 11:21:19 +00:00
nixos.nix refactor(ops/nixos): Pass depot as a special argument 2021-05-24 21:48:37 +00:00