mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(pipelines/depot): Skip build steps if their out paths exist

Browse source 
Skip build steps if they have already been built, reducing pipelines
to the things that actually changed between builds. On canon all
targets are always built (we require this for anchoring).

Note that this is not perfect, garbage collection and competing
pipelines may affect each other.

Also note that we have some impure targets that change on every
commit.

Change-Id: Ic6bae3b6c8e1e7fd2116ec252f5089f471854ab6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3427
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
This commit is contained in:
Vincent Ambo 2021-08-26 18:02:52 +03:00 committed by tazjin
parent 094dfa2ed3
commit d5ddfb7b96
1 changed files with 12 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
ops/pipelines/depot.nix
View file

@ -48,6 +48,18 @@ let
"|| (buildkite-agent meta-data set 'failure' '1'; exit 1)" "|| (buildkite-agent meta-data set 'failure' '1'; exit 1)"
]; ];
label = ":nix: ${mkLabel target}"; label = ":nix: ${mkLabel target}";
# Skip build steps if their out path has already been built.
skip = let
shouldSkip = with builtins;
# Only skip in real Buildkite builds
(getEnv "BUILDKITE_BUILD_ID" != "") &&
# Always build everything for the canon branch.
(getEnv "BUILDKITE_BRANCH" != "canon") &&
# Discard string context to avoid realising the store path during
# pipeline construction.
(pathExists (unsafeDiscardStringContext target.outPath));
in if shouldSkip then "Target was already built." else false;
}; };
# Protobuf check step which validates that changes to .proto files # Protobuf check step which validates that changes to .proto files