This test, after performing the usual Nixery build, loads the built
image into Docker, runs it, pulls an image from Nixery and runs that
image.
To make this work, there is some configuration on the Travis side.
Most importantly, the following environment variables have special
values:
* `GOOGLE_KEY`: This is set to a base64-encoded service account key to
be used in the test.
* `GCS_SIGNING_PEM`: This is set to a base64-encoded signing key (in
PEM) that is used for signing URLs.
Both of these are available to all branches in the Nixery repository.
With this patch, and this file I called `log.py`:
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p python3 --pure
import sys
from pprint import pprint
stack = []
timestack = []
for line in open(sys.argv[1]):
components = line.strip().split(" ", 2)
if components[0] != "function-trace":
continue
direction = components[1]
components = components[2].rsplit(" ", 2)
loc = components[0]
_at = components[1]
time = int(components[2])
if direction == "entered":
stack.append(loc)
timestack.append(time)
elif direction == "exited":
dur = time - timestack.pop()
vst = ";".join(stack)
print(f"{vst} {dur}")
stack.pop()
and:
nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
./log.py ./log.matthewbauer > log.matthewbauer.folded
flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg
I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg
---
Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
Implements a cache that keeps track of:
a) Manifests that have already been built (for up to 6 hours)
b) Layers that have already been seen (and uploaded to GCS)
This significantly speeds up response times for images that are full
or partial matches with previous images served by an instance.
Some upcoming changes might require the Nix build to be split into
multiple separate nix-build invocations of different expressions, thus
splitting this out is useful.
It also fixes an issue where `build-image/default.nix` might be called
in an environment where no Nix channels are configured.
ALl the ones except for build-image.nix are considered trivial. On the
latter, nixfmt makes some useful changes but by-and-large it is not
ready for that code yet.
Removes usage of the old layering algorithm and replaces it with the
new one.
Apart from the new layer layout this means that each layer is now
built in a separate derivation, which hopefully leads to better
cacheability.
Instead of requiring the server component to be made aware of the
location of the Nix builder via environment variables, this commit
introduces a wrapper script for the builder that can simply exist on
the builders $PATH.
This is one step towards a slightly nicer out-of-the-box experience
when using `nix-build -A nixery-bin`.
This commit adds the actual logic for extracting layer groupings and
merging them until the layer budget is satisfied.
The implementation conforms to the design doc as of the time of this
commit.
This script generates an entry in a text file for each time a
derivation is referred to by another in nixpkgs.
For initial testing, this output can be turned into group-layers
compatible JSON with this *trivial* invocation:
cat output | awk '{ print "{\"" $2 "\":" $1 "}"}' | jq -s '. | add | with_entries(.key |= sub("/nix/store/[a-z0-9]+-";""))' > test-data.json
As described in the design document, this adds considerations for
closure size and popularity. All closures meeting a certain threshold
for either value will have an extra edge from the image root to
themselves inserted in the graph, which will cause them to be
considered for inclusion in a separate layer.
This is preliminary because popularity is considered as a boolean
toggle (the input I generated only contains the top ~200 most popular
packages), but it should be using either absolute popularity values or
percentiles (needs some experimentation).
--
2dd3b23ea940804de727e396b300cfae4b1b71a1 by Derek Mauro <dmauro@google.com>:
Upgrade to LLVM r366207 and Bazel 0.28.1 for Linux-Clang testing.
PiperOrigin-RevId: 263162761
--
f03ae9e4e9f42c075745d28b4ced78071e73724a by Derek Mauro <dmauro@google.com>:
Remove unneeded dependencies on //absl/base from targets that no
longer depend on it.
PiperOrigin-RevId: 263129193
--
8ec2aab1eea50e7f71a6a687a07e5ae0e0945f5e by Derek Mauro <dmauro@google.com>:
Move raw_logging to a separate target (raw_logging_internal)
PiperOrigin-RevId: 262972007
GitOrigin-RevId: 2dd3b23ea940804de727e396b300cfae4b1b71a1
Change-Id: I3f4580e87797386b0b5e90c8ced74cbf078f61c1
--
d3d344e214e80b766d3e5c355e16124eb8602ff2 by Tom Manshreck <shreck@google.com>:
Add LTS Branch to LTS docs
PiperOrigin-RevId: 262904704
--
a9b10e6959209e8ded66e5fb041b4f1811a3f375 by CJ Johnson <johnsoncj@google.com>:
Minor cleanup on InlinedVector headers
PiperOrigin-RevId: 262632981
GitOrigin-RevId: d3d344e214e80b766d3e5c355e16124eb8602ff2
Change-Id: I3d0c078c08520b5c6a1a46c4b681ae0d98297f24
* exwm-input.el (exwm-input--on-buffer-list-update): The way of
detecting a switch from a `*temp*' buffer does not always work.
Disable it until we find a better way.
* exwm-systemtray.el (exwm-systemtray--on-workspace-switch)
(exwm-systemtray--on-randr-refresh): Instead of retrieving the real
frame height, manually calculate it with workarea height and
menu-bar/tool-bar size.
--
3dbb096e4662311f81df1017a8e0975e903936cf by Derek Mauro <dmauro@google.com>:
Document and workaround a known MSVC bug doing constexpr pointer arithmetic
PiperOrigin-RevId: 262604652
--
b5fa7f1a0c776f6ba20d52772a1679ec42ad21fd by Derek Mauro <dmauro@google.com>:
Fix typo in macos_xcode_bazel.sh
PiperOrigin-RevId: 262591285
--
89dd77ab5bb44d76b6cb6b2f288e21536e16a85a by Derek Mauro <dmauro@google.com>:
Internal change
PiperOrigin-RevId: 262582747
--
32295ed9a0c6c8ab143a912194040eede05d3ea3 by Abseil Team <absl-team@google.com>:
Internal change
PiperOrigin-RevId: 262569140
--
7f0f5b94197369228024529022d727439d2c894f by Abseil Team <absl-team@google.com>:
Internal change
PiperOrigin-RevId: 262563554
--
314aed043639abbd221074125c57b7c68616de7e by Derek Mauro <dmauro@google.com>:
Release absl::btree
PiperOrigin-RevId: 262553526
--
72b44056c6ce9000c4a6cd9aec58b82067c82a13 by CJ Johnson <johnsoncj@google.com>:
Internal change
PiperOrigin-RevId: 262421185
--
4e2c12151edf534f929e8e810f1334073f90489a by Abseil Team <absl-team@google.com>:
Update documentation to make it less likely for users to write `Hours(24)` without considering using civil dates instead.
PiperOrigin-RevId: 262420758
--
92b85b9573e800bd96b019408eefbc5ce4f68780 by Derek Mauro <dmauro@google.com>:
Add the ability to override the bazel version in the macos_xcode_bazel.sh
test script.
PiperOrigin-RevId: 262412063
GitOrigin-RevId: 3dbb096e4662311f81df1017a8e0975e903936cf
Change-Id: I423b2b829dc0c5f814e37bec4d68c7470f43f041
This uses a significantly larger percentage of the total available
layers (125) than before, which means that cache hits for layers
become more likely between images.
--
f51743aa96e19aa3dda96d09d313b4390f1d61e7 by CJ Johnson <johnsoncj@google.com>:
Minor touchup on the InlinedVector/Storage internal header file
PiperOrigin-RevId: 262389640
--
e2f54c1f7142e40d30ff0fda43bef050625821a5 by Abseil Team <absl-team@google.com>:
Update the absl codebase to use proper marketing names for macOS and Xcode
PiperOrigin-RevId: 262389450
--
f29aae774edd0d00e2daa1fb96694a6dc3565a55 by CJ Johnson <johnsoncj@google.com>:
Blocks code generator script from being affected by LTS inline namespaces
PiperOrigin-RevId: 262376791
GitOrigin-RevId: f51743aa96e19aa3dda96d09d313b4390f1d61e7
Change-Id: I33be7f5a708ce8a2b7111b00151e43d73c5e0009
--
38bc0644e17bf9fe4d78d3db92cd06f585b99ba7 by Andy Soffer <asoffer@google.com>:
Change benchmark to be cc_binary instead of cc_test, and fix a bug in the zipf_distribution benchmark in which arguments were passed in the wrong order.
PiperOrigin-RevId: 262227159
--
3b5411d8f285a758a1713f7ef0dbfa3518f2b38b by CJ Johnson <johnsoncj@google.com>:
Updates Simple<*>() overload to match the name schema of the others
PiperOrigin-RevId: 262211217
--
0cb6812cb8b6e3bf0386b9354189ffcf46c4c094 by Andy Soffer <asoffer@google.com>:
Removing period in trailing namespace comments.
PiperOrigin-RevId: 262210952
--
c903feae3a881be81adf37e9fccd558ee3ed1e64 by CJ Johnson <johnsoncj@google.com>:
This is a cleanup on the public header of InlinedVector to be more presentable
PiperOrigin-RevId: 262207691
--
9a94384dc79cdcf38f6153894f337ebb744e2d76 by Tom Manshreck <shreck@google.com>:
Fix incorrect doc on operator()[] for flat_hash_set
PiperOrigin-RevId: 262206962
--
17e88ee10b727af82c04f8150b6d246eaac836cb by Derek Mauro <dmauro@google.com>:
Fix gcc-5 build error
PiperOrigin-RevId: 262198236
GitOrigin-RevId: 38bc0644e17bf9fe4d78d3db92cd06f585b99ba7
Change-Id: I77cababa47ba3ee8b6cebb2c2cfc9f60a331f6b7
Make curl's low speed limit configurable via stalled-download-timeout.
Before, this limit was five minutes without receiving a single byte.
This is much too long as if the remote end may not have even
acknowledged the HTTP request.
