The camelCase variant of `rebuild-system` doesn't exist, but the
kebab-case version does.
Side note: this `lazy-dispatch` upgrade is pretty cool. TIL `direnv`
supports `watch_file` and `PATH_add`.
Change-Id: Idc9109a9b0de327ddf7b9c6a4368b7bebb551196
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5565
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
nix-shell pollutes the environment with all sorts of variables. Let's
just add the tools to the PATH?
This also papers over the various differences in users `use_nix`
implementations by not using it at all.
Change-Id: If4282531fd6b7453b3611fe50217beacadc08bb5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5524
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This modifies the envrc configuration to add the result of building
//tools/depot-deps to $PATH, instead of dispatching through the
manually maintained list of symlinks.
While at it, I've cleaned up some stuff from that list that is no
longer actually used.
Change-Id: If345c44da75b23c06b7c7f435be0cb02f99aaac5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5513
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
in some cases, users might want to pass through flags for
nix-build (such as `-j`).
magrathea now accepts these as arguments to `mg build`, as long as
they are separated by `--`.
the arguments passed to `mg build` are parsed into a proper record,
which enables us to show users very clear error messages in case they
forget to use the `--` separator and keeping us future-compatible with
more potential arguments to magrathea itself.
Change-Id: I81f5d9db52779a5cc3b8bbdd975316274fffe5fc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5507
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Reviewed-by: asmundo <asmundo@gmail.com>
Cleans up a whole bunch of things I wanted to get out of the door
right away:
* depot internal references to //third_party/nixery have been replaced
with //tools/nixery
* cleaned up files from Github
* fixed SPDX & Copyright headers
* code formatting and inclusion in //tools/depotfmt checks
Change-Id: Iea79f0fdf3aa04f71741d4f4032f88605ae415bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5486
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
This does not fully change the build structure of Nixery to be
depot-compatible yet, but should allow most targets to be built in
depot CI.
This contains some hacks to work around surface incompatibilities
which we'll clear away later.
Change-Id: I84e7734334abbe299983956f528c0897f49fa8c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5485
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This absorbs a josh-filtered Nix subtree into depot, at
//tools/nixery.
This subtree was created through `josh-filter ':prefix=tools/nixery'`,
which allows a filter on tools/nixery to yield the same commit hashes
as the original Nixery repository (allowing for history continuity).
Change-Id: Icc1a99bf1248226b91f437b0a90361d36fb0d327
This tool has been replaced by niv.
Change-Id: I011059b7d8890d0456b22f066e723584cc1d9a2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5329
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This has been superseded by magrathea.
Change-Id: Ief4a3d1b81e51e7a9c9a0112584fa7efc8aca63f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5328
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This command builds the desired target and runs the executable produced
by it. If a directory is produced, it looks for a single (!) executable
in the bin directory. Dot files are ignored, so wrappers should
generally work. In the future we could provide a flag to select one of
multiple executables.
All arguments following the target are passed to the executable as is.
Examples:
mg run ops/mq_cli ls
mg run web/bubblegum:examples
Change-Id: I6490668af68e028520973196d9daa5f1d58969ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5277
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Instead of the strict check-all-our-crates, generate a fake Cargo.lock
and add it to the report generated by check-all-our-lock-files.
check-all-our-crates was a reimplementation of cargo-audit anyways and
prevented us from updating the advisory db due to its strict
model (failing on any advisory).
Change-Id: I264a7f1a5058a527cbc46d26225352ecd437a22b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5230
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Rename check-all-our-lock-files to tree-lock-file-report and pull out
all the buildkite-specific code which makes the code less awkward.
check-all-our-lock-files is then only executed in extraSteps and runs
tree-lock-file-report on depot, adding it as a warning to the pipeline
if it is non-empty.
Change-Id: If6bd236d90cc680cba0ed4e988f2f28ddb8012d6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5229
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This script is somewhat usable by humans (it even has a help screen!)
and can be reused in //users/sterni/nixpkgs-crate-holes. We are using
bash since that allows us to exit with the actual exit code of
cargo-audit - something that's not possible in execline.
Change-Id: I3331ae8222a20e23b8e30dc920ab48af78f0247c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5228
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
I want to add a shortcut to build and run e.g. scripts that are depot
targets - for which it would be useful to not have stdout polluted by
magrathea itself.
Change-Id: Ic58fe28eafb4d0715e53beae041bfaa5d1745812
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5276
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
The rustfmt configuration of the depot is moved to `rustfmt.toml` (it
is recognised more reliably from this path than from the hidden
.rustfmt.toml).
Nested configuration is theoretically possible, but detection of
nested config files is flaky. Paths with nested config files need to
be disabled in the top-level check (I've excluded my user directory).
Change-Id: I385ce3ef529bda28fac03bfba86fc204c81b8a61
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5241
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
apparently the cache can get out-of-sync or something; either way we
had a build where it missed a file that was misformated.
Change-Id: I2967aec99ee1c7c8b978a3dfdfed4ff213bb6591
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5249
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
In some cases we want to be able to "emergency approve" something on
behalf of a different user.
Example cases:
* clean up of abandoned directories with restrictive OWNERS
* security fixes blocked on people in different timezones
This script can be used to perform these approvals if the user is a
member of depot-interventions. Note that access to depot-interventions
is audit logged.
The user on behalf of whom approval is performed is always added to
the attention set to ensure that they are made aware of the CRFO
approval.
Note: This depends on nixpkgs#156466. Keeping WIP until we have a
channel with that patch.
Change-Id: I16e5f9d7baa9daab49c88b629bb8f024aad9d94c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5085
Tested-by: BuildkiteCI
Reviewed-by: kn <klemens@posteo.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Many of the vulnerabilities (in the respective crates) reported are not
actually exploitable vulnerabilties of the packages we report them for.
Consequently it is more accurate to state that they are advisories.
Change-Id: I02932125b77fc9c71e583ae49e822fd3438dce05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5202
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Buildkite doesn't understand GitHub Flavored Markdown and having a read
only checklist in there is probably not much use.
Change-Id: I41538487087e8c817b1a5e653f077bb0fbe6eb47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5201
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
In the spirit of the readTree filter we should also not include files in
user directories from the outside.
Change-Id: I1abe36a721048900d2758b5986063b68b8d1af93
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5200
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
check-all-our-lock-files works very similarly to
//users/sterni/nixpkgs-crate-holes, even reusing some parts of it, but
is much simpler since we don't need to extract the lock files — they are
already in tree.
It is implemented as a very simple script which just traverses the
subtree of the current directory, collecting all warnings. When
executing this script in buildkite via extraSteps, it never fails,
instead annotating the pipeline run with a warning.
Change-Id: I0a0bc26deffe7b20b99f5aa7238fb3c3bb9deb92
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3721
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Any other cgit configuration in depot would need this script wrapper as
well.
Change-Id: Ifa04e1c9de9c925eb3f60c5d3854221ae02ef06c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5206
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
The function is depot specific and thus uses tvl-depot-path, so it
belongs in `tvl.el`. Since non-sly-users won't need it, we tie its
definition to loading the sly package.
Change-Id: I8b104deab455d218d3df6a800e35cc104220a841
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4960
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
We can't use cl-lib as that apparently doesn't contain lexical-let*.
Change-Id: I8e65d20215ae5667bb92b71e6318ad9d66125320
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4941
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
A formatting error broke this at some point (the let clauses were
outside of the definition list).
Change-Id: Iaa2dc9ad02d2f7e909ca9bf28705e782ad26060b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4765
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Produces more useful output and also makes for a good target for the
upcoming extraSteps logic.
Change-Id: Ifd389d433d9e27f97940a48999f4fba35646e37a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4727
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
Two minor "quality of life" improvements:
- automatically set SSL_CERT_FILE environment variable,
so that programs relying on OpenSSL for certificate
validation can actually validate certificates
(the certificates are included no matter what since
we add the "cacert" package to all iamges)
- if the requested image includes an interactive shell
(e.g. if it includes the "shell" metapackage), set
the image Cmd to "bash", which allows to execute
"docker run nixery.dev/shell" and get a shell)
I'm happy to split this PR in two if you'd like, but
since both features touch the Config structure and are
rather small, I thought it would make sense to bundle
them together.
`terraform fmt` can only handle a single path, but treefmt expects
formatters to be able to handle multiple paths at once.
this wraps it in a small shell script that calls `terraform fmt` with
at most one path at a time.
Change-Id: I2b9c1b89b5a276f3d4915b95608ce36b2509e334
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4639
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
previously, depot-scanner swallowed/ignored all non-processed
stderr output of nix-instantiate, which makes diagnosing
failures of nix-instantiate (e.g. failed with exit status 1)
difficult. This commit fixes that by always forwarding
the remaining stderr messages.
Example previous error message:
panic: nix-instantiate failed: exit status 1
goroutine 1 [running]:
main.main()
/nix/store/8vb2j13bd7j5ipl7dhsnwvgr7nrrsqsi-main.go:160 +0xeb4
Example new error message:
nix-inst> error: unrecognised flag '--trace-file-access'
nix-inst> Try '/run/current-system/sw/bin/nix-instantiate --help' for more information.
panic: nix-instantiate failed: exit status 1
goroutine 1 [running]:
main.main()
/nix/store/qy7v79a3harddirzmc0432vbzqhyf91i-main.go:165 +0xeb4
Change-Id: I666f3490fc648f77a5384b95edd74f6115f7920d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4553
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Right now this only includes Go, but more is to come.
Change-Id: Idd8fc27c0eb25e82688ef8337ba20810d834f4b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4504
Reviewed-by: zseri <zseri.devel@ytrizja.de>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
These instructions were not up-to-date (they didn't mention
the different storage backends, and some variables were
tagged as optional while they were mandatory). With this
update, they should (hopefully) be more accurate! :)
I also added instructions if someone wants to run Nixery
outside of the container image (I found it convenient when
working on Nixery's code).
(zseri:) It seems like relativePath should be replaced with relativeFlag,
as no variable with the former name exists, a boolean is as far as I can
tell expected, and a boolean with a similar name exists. Lets give it a
try in the CI.
Change-Id: I0e7e522a41a517a38222dcda3b66731344613c1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3581
Reviewed-by: tazjin <mail@tazj.in>
Autosubmit: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Prompting with "Checkout CL" for the gerrit cherry-pick command doesn't
make any sense.
Change-Id: I51495e7975202146fae2da0807e525596f2d490e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4516
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Right now the only included formatter is gofmt, but we will extend
this over time.
The version of treefmt is bumped to 0.3.0 (which supports custom
config files) until this lands in nixpkgs.
Change-Id: I1e1aafd05ec7427c616f90c90490c528ecb2615c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4399
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
All targets would have no trailing slash, so it was at least ugly that
// had one as the only legal target.
Change-Id: I1b60850ac86d8c550f262841694fb00c518413b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4404
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
r/3000 will now be shortlinked to show the commit in cgit. Going via
atward probably doesn't make sense at this point, since the depot refs
are not available in sourcegraph at all (for reasons I can't
repeat). Switching to atward might be interesting when/if we introduce
support for shortlinking // paths.
Fixes: b/163
Change-Id: I57c1a7d02d881e4f8b3ee1f71755dd7930925dc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4402
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <mail@tazj.in>
for global installations of magrathea, setting MG_ROOT can be a way to
switch quickly between different repositories (esp. in combination
with `cd (mg path)`).
Change-Id: I4627fe78b7cc112b75ab57e7806ffd85c6d38aee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4396
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
this command prints the absolute directory for a given target. it can
be combined with shell aliases to add quick navigation commands.
unfortunately due to the nature of computers implementing something
like `mg cd` directly is not possible.
Change-Id: Icc88eb97384812c620c49fe2de8fa331f4d7153b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4395
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
magrathea now does what it says on the tin - build and shell commands
can be used with the targets specified on the command line.
implementation notes:
* string representation of target has been changed to look like the
target spec format, this is now used in user-facing messages
* errors returned by the target parser make the program exit with
status 1
* normalisation could be done better (for example, maybe it makes
sense to always do it) but it's good enough for now
Change-Id: Ib85f389a5cec92b3c2f3b9c0b40764435bbcc68b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4394
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
i'm not sure what happened here, but it works (yes, the fancy target
printing is completely unnecessary, but oh well):
#;152> (parse-target "foo")
#target(foo)
#;153> (parse-target "//foo")
#target(//foo)
#;154> (parse-target "//foo/bar")
#target(//foo/bar)
#;155> (parse-target "//foo/bar/")
#target(//foo/bar)
#;156> (parse-target "//foo/bar:baz")
#target(//foo/bar:baz)
#;157> (parse-target "//foo/bar/:baz")
#target(//foo/bar:baz)
#;158> (parse-target "//foo/bar:")
(error . "unexpected end of input while parsing virtual target")
#;159> (parse-target "//foo//")
(error . "unexpected root-anchor while parsing normal target")
the most notable thing is that trailing slashes are allowed in the
physical targets, since people may be autocompleting these on the
shell from folder names.
Change-Id: I32975ad77fe2a327130dc9574011fe92cce49f84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4393
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
this is going to be a serious version of //tools/depot-build.
right now it doesn't support parsing any target specs yet, so only
shells and builds for the physical project of the current folder work.
Change-Id: I4308e29da940571622ff9e539fbb8ededd27aca7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4335
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
as before, fifth doesn't exist on all emacsen, but nth definitely does
Change-Id: Ic0e4e3790402d960d1546d37187758a4d9ca33c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4346
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
The l= is part of the command, not of the shape of commands, and the
previous command concatenation logic was wrong because of that.
Fix is done in the most obvious way: Make the l= part of the command.
Change-Id: Ia3c08c3da60fe5fc38f29a2d94adcd123e4f3052
This makes this function a true rubberstamp again, leading to
rubberstamped CLs automatically being merged after CI passes.
This is similar to the initial functionality we had last year, where
this directly submitted changes, but with the addition of the CI
checks.
Change-Id: I946b074b968eb18a64c4edb0043f7a4af28759b4
This almost makes for a sort of fire&forget button, except we don't
have a way to automatically pick reviewers yet :)
Change-Id: I6f446270f8aaf0409ccb6321bdbb5c349079cd19
Bound to `A g`, this behaves similarly to `magit-gerrit-checkout` - it
prompts for a CL number, then cherry-picks the latest patchset of that
CL number
Change-Id: Ieef970b99d96170e8c960cc7687ead9022948f8b
Adds all the functionality described in the README in cl/4066.
This code is very closely related to //users/tazjin/russian/russian.el
Change-Id: I14f1052cebfbe4886e75e8efc730eacbf8773f29
Passively is a tool to help people learn information via Emacs,
designed for language learning.
As of this CL, the actual implementation still lives in
//users/tazjin/russian/russian.el but I am generalising it here.
Change-Id: Iac5a8cfc78415496637a7ba5ddc4c2a1aa6bee26
This function is also generally useful for readTree consumers that
have the concept of subtargets.
Change-Id: Ic7fc03380dec6953fb288763a28e50ab3624d233
Add a new magit-gerrit-checkout command, which prompts for a CL number
then fetches and checks out the latest patchset of that CL with a
detached HEAD.
Change-Id: I88b8209d40017479d97ed40ecbd5fd1ccd7cf650
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3880
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
`our-crates` can just check if the attributes in question are
derivation (i. e. have an `outPath`) instead of blacklisting the
`__readTree` attribute specifically.
Change-Id: I472692e89c0e9eff551372c72a73ab765b0b6599
This makes it possible for users of cheddar as a library to supply
their own shortlinks. In practice it is unlikely anyone will do this,
but the change also allows us to (relatively) easily add additional
shortlinks to the set used by TVL.
Note that Cheddar is primarily intended for use by TVL and the default
rendering function interfaces have not changed, and will default to
using TVL shortlinks.
A new public function `format_markdown_with_shortlinks` has been added
with which users can use an alternative set of shortlinks. This
function should not be used in TVL depot code.
Change-Id: I4ddab28cbcf45d07c51323b7b730b96e62922816
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3083
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
This is required by common patterns in shell scripts.
There are some caveats around this. Adding logic to filter whether
coreutils is included in an image would slow down the Nix evaluation,
so the link is currently created even in cases where it doesn't point
to anything.
Fixes#109
This fixes all compilation warnings except the one about 'tvl lacking
a parent group, which we can look into later (it doesn't matter that much).
Change-Id: Iaff5e7f5f251f0670afb0a47031ccf197de69818
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3408
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
I'm using gerrit at work now, and would like to use tvl.el to interact
with it via Emacs, but we use a different default branch than "canon".
This makes it configurable, and also marks it as safe so I can configure
it in .dir-locals.el
Change-Id: I66d4c7ce94351f2df863ec49dbc3e1d1d6d1547a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3369
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Very simplistic tool to aid with updating our nixpkgs channel pins:
This tool prints the correct two `*Hashes` sets to stdout, so you can
easily delete the appropriate lines in `third_party/nixpkgs/default.nix`,
run
./bin/depot-nixpkgs-update | wl-copy
and paste it into your editor.
Doing this fully automatically would be possible, but would either
a) require changing `default.nix`, so it is regex-able more easily,
b) touching something like rnix-parser which I have no motivation to
at the moment or
c) searching for the old hashes and replacing them with the new
ones. This may be a simple and worthwhile improvement in the future.
Change-Id: I4df44e3827ce9ff6e4fe2d336c08016d799e21a7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3252
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
After the discussion in #116, this stores the blob content types
in extended attributes when using the filesystem backend.
If the underlying filesystem doesn't support extended attributes,
storing blobs won't work; also, if extended attributes get removed,
blobs won't be served anymore. We can relax this behavior if
needed (i.e. log errors but still accept to store or serve blobs).
However, since the Docker Engine (and possibly other container
engines) won't accept to pull images from a registry that doesn't
use correct content types for manifest files, it could be argued
that it's better to give a hard fail. (Otherwise, the container
engine gives cryptic error messages like "missing signature key".)
I can change that behavior (and log errors but still store/serve
blobs to the filesystem) if you think it's better.
With https://github.com/google/nixery/pull/127, nixery will use extended
attributes to store metadata (when using local storage).
Right now, our integration test mounts a tmpfs to /var/cache/nixery.
However, *user* xattrs aren't supported with tmpfs [1], so setting
xattrs would fail.
To workaround this, use a folder in the current working directory and
hope it's backed by something supporting user xattrs (which is the case
for GitHub Actions).
[1]: https://man7.org/linux/man-pages/man5/tmpfs.5.html#NOTES
* users/grfn/system/home/yeren: remove obsolete awscli2 overrides
* ops: make new isSystemUser || isNormalUser assertion happy
* users/grfn/system/system/mugwump: make buildkite agents system users
* users/tazjin/nixos/camden: set isSystemUser = true for git
* users/tazjin/emacs: Remove missing & broken packages
* third_party/openldap: remove, as the argon2 module is now enabled upstream
* third_party/gerrit_plugins: Pinned new unstable hashes
* third_party/nix, third_party/grpc: Disabled CI as these are broken
* third_party/overlays/emacs: Bumped version to stay in sync with channel
* third_party/buzz: Update LIBCLANG_PATH to reference libclang.lib,
since libclang's default output no longer contains libclang.so
* users/grfn/system/home: Install julia-stable instead of julia (which
aliases to julia-lts), as the latter depends on an insecure version of
libgit
Change-Id: Iff33b0ecb0ef07a82d1de35e23c40d2f4bf0f8ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3001
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
* This was mostly for //third_party/nix and its dependencies which now
have been set to use llvmPackages_11 manually.
* For //users/grfn/achilles we also manually select the newer LLVM version.
* //tools/cheddar doesn't seem to need llvm anymore.
* //third_party/buzz also compiles with clang 7.1.0
* replace clang-tools everywhere with new attribute clang-tools_11
For the future we may want to have something similar again, but it may
not be necessary to invest too much time into it: nixpkgs is set to
upgrade their default llvmPackages to LLVM 11 as well at some point in
the near future.
Co-Authored-By: sterni <sternenseemann@systemli.org>
Change-Id: Id83868dbc476a6c776b59518b856c933f30ea79d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3135
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
This is fixed in upstream nixpkgs, but we're not yet at a commit where
it's used, so it's important to use the OpenLDAP from //third_party
Change-Id: I7c033cd23f45a95c4a4af864ffe561c496833a0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3143
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
We have a bunch of crates in `third_party/rust-crates`; it would be
great if we could check them for existing CVEs.
This tool does that, it takes the rust security advisory database,
parses the applicable CVEs, and cross-checks them against the actual
crate versions we list in our package database.
The dumb parser we wrote is tested against all entries in the
database, so we will notice when upstream breaks their shit.
Checking the semver stuff is easy enough with the semver crate.
If an advisory matches, it prints the whole thing and fails the build.
Change-Id: I9e912c43d37a685d9d7a4424defc467a171ea3c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2818
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
Implements highlighting of shortlinks like cl/123, or b/123.
Highlighting works by replacing the input Markdown using a simple
regular expression replacement.
We also considered parsing and replacing these links in the Markdown
AST, but it would have been significantly more complex (due to arena
allocation) for little upside and no obvious performance benefit.
Change-Id: I53f03fb17491046d89d0b7f605929571c11ee9a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3082
Tested-by: BuildkiteCI
Reviewed-by: eta <eta@theta.eu.org>
The BAT_SYNTAXES environment variable is injected during Nix builds
and points to the build result of //third_party/bat_syntaxes, which is
included in the cheddar binary.
During development this is a bit annoying: Opening a Nix shell will
set the right variable, but it will also mess with the Cargo
environment in ways that make it annoying to use (*and* require a
pre-build of all dependencies as required by naersk).
For an easier development flow, this adds a build script that
automatically runs Nix, fetches the path to the syntax set and sets
the environment variable. It is a no-op if the variable is already
set.
This makes for a smoother development experience when working on
cheddar in a depot checkout.
Change-Id: Ib5435518897e3443247edd9f362821c0b9ff756b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3081
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Splits `main.rs` into `lib.rs` and `bin/cheddar.rs`, which enables
reuse of cheddar's rendering logic in other Rust applications.
Change-Id: Ifd1a44a8d1620c595550a0a497a25b0563e917ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3060
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Drops the go2nix configuration in favour of pkgs.buildGoModule.
Note that the go.sum file is bloated by issues with cyclic
dependencies in some Google projects, but this large number of
dependencies is not actually built.
* remove a step that was not supposed to be committed ("Do we have
Docker?")
* remove setup of old temporary storage directory (now done in
integration script test instead)
* skip creation of out-link for initial Nixery build (to avoid
cache-busting on the second build)
In case the `GOOGLE_APPLICATION_CREDENTIALS` environment variable is not
set, a redirect to storage.googleapis.com is issued, which means the
underlying bucket objects need to be publicly accessible.
This wasn't really obvious until now, so further clarify it.
This copies the integration tests from `.travis.yaml` into a script,
documents the assumptions it makes, and wires it into GitHub Actions.
Contrary to the travis version, we don't use Nixery's GCS backend, as
handing out access to the bucket used, especially for PRs, needs to be
done carefully.
Adding back GCS to the integration test can be done at a later point,
either by using a mock server, or by only exposing the credentials for
master builds (and have the test script decide on whether
GOOGLE_APPLICATION_CREDENTIALS is set or not).
The previous travis version had some complicated post-mortem log
gathering - instead of doing this, we can just `docker run` nixery, but
fork it into the background with the shell - causing it to still be able
to log its output as it's running.
An additional `--rm` is appended, so the container gets cleaned up on
termination - this allows subsequent runs on non-CI infrastructure (like
developer laptops), without having to manually clean up containers.
Fixes#119.
