These packages are invalid in Nix, and worked around in nixpkgs with
underscores, but the underscores are invalid in the Docker registry
protocol.
We work around this by detecting this case and adding the underscore
to yield the correct package reference. There is no case where this
workaround can break something, as there can be no valid package
matching the regular expression.
This relates to https://github.com/tazjin/nixery/issues/158
Change-Id: I7990cdb534a8e86c2ceee2c589a2636af70a4a03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8531
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Usually the current behavior is best: You are dropped in a REPL with the
package(s) you are working on already available. As you are working on
them, you recompile individual files and your changes become available.
However, I've found that there are some occasions when this is not
desireable, e.g.: When you are working on something and have broken the
test suite intermittently, it becomes impossible to start a new REPL.
Not sure how the yes-or-no-p question should be phrased, its negation
may be better?
Change-Id: I6a37ebc02f3121f628fc9206e0de650851824cd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8415
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Instead of prepending :unsign to all URLs in josh-proxy, and for all
calls to filteredGitPush, explicitly use it only in the filter we use
for the `export-kit` extraStep.
This means, people cloning tvl-kit via
> https://code.tvl.fyi/depot.git:workspace=views/kit.git
now need to update the URL to point to
> https://code.tvl.fyi/depot.git:unsign:workspace=views/kit.git
instead.
git@github.com:tvlfyi/kit.git will keep the same hashes, as it's updated
to export the unsigned workspace view of it.
This is less invasive than dooming every josh workspace to have to strip
signatures.
Change-Id: I6de05182fad4c3695081388c3bbf37306521d255
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8369
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
We need to followup on cl/8186, and use the same workaround for josh-
filter that we use for josh-proxy, so we can push again.
This could potentially become an optional function argument, if new
subtrees are to be carved out, and we want to keep signatures on commits
in there.
Change-Id: I7a9b821a7365767e8a4188e8200fa3c37463142c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8368
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: flokli <flokli@flokli.de>
Upgrade to syntect 5.0 and load the new kind of syntax set
serialisation with the new helper function for that purpose.
Includes other minor API fixes as well, note that the things that are
now calling `expect` previously failed internally at those points and
we're reasonably confident they don't fail in production.
This has been waiting for a long time ...
Change-Id: I8af4fef995ff64bfbe24e1f13917fa50ecb6e4ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7787
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This is a semi-manual bump of all dependencies (except syntect, which
is more complex to update).
Change-Id: I8c678a16d779f3f896b95f7d161710ac39d38e88
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7786
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
I think what might be going on with b/231 is that the annotations
somehow started conflicting because they don't have contexts set.
Lets try setting a context and see if it changs anything ...
Change-Id: I62ed57f9e24f08e4e7215f05d35cfa769e2e2c24
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7640
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This script fetches the inbox for depot@tvl.su into the specified
directory in maildir format.
The layout of the folder follows the structure generated by public
inbox, i.e. the directory containing the current maildir will be
`$TARGET/su.tvl.depot.0`, but most mail clients (e.g notmuch) will
figure this out on their own.
----
In addition, we would ideally find a CLI mail client that can be
pointed at an arbitrary maildir (or an IMAP server) and works with
local `sendmail` config so that people can have a single command entry
point to interacting with depot@tvl.su.
Change-Id: Iaf9fcce73e9caa2f202327488c43d0394be26ca6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7644
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is the new address which leads to the public inbox at inbox.tvl.su
Change-Id: I45d98a373b8acda49b05c4f74669ffb9ad1f1a3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7632
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
These are both unused things from a long time ago, which we don't need
to keep around anymore.
Their design doc has been marked as archived.
Change-Id: Icd2744e511e78ec95ec8f39e5f79ed1fe98e9e4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7639
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The //tvix README already steers to `mg run //tvix:crate2nixGenerate
--`, there's no point in /also/ having a non-formatting version of
crate2nix in ``$PATH`.
Change-Id: Idc6409799ae5f0629376eef6eeff6eb9eaa4fb99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7613
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
As we start using this in more parts of depot, it makes sense to add
it to the available tools.
Change-Id: I148902714167b36bc51aeca4a241c79ad8a59285
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7562
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
With this, we can disambiguate the following `mg run` invocations:
```shell
$ mg run :foo # run the virtual target, foo, with no args
$ mg run -- :foo # run the empty target and pass the arg, :foo, to it
```
Change-Id: Id6395b36a4d8ef3f325937e322e1c27b8630b556
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7408
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Something I missed last time reading through the process documentation
is that you can use a combination of `process` and `process-wait` to
determine the exit status of a child process *and* read from its
standard output. With `process*` we could even capture stderr, but we
probably want it mounted to the parent process' stderr anyways.
Change-Id: I9840f607df465caa80d28109e344e5fc1402949d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7259
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Profpatsch originally implemented an advisory checker from scratch in
Rust. We now ended up just using cargo-audit for the global checks
exposed via CI and the custom implementation is unused. To clean up
//tools/rust-crates-advisory a bit, we can move the unused parts to his
user directory.
Change-Id: Iacbd27c163edd07c804220fd1b3569c23aebd3e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7171
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
I've elected to split the check-all-our-lock-files script into two new
scripts: One very simple script which generates the report by invoking
lock-file-report on the fake lock file for //third_party/rust-crates and
all lock files in depot, and one which executes this and adds it as a
buildkite annotation if there are any warnings (which is reported by the
report generating script using a non zero exit code).
The latter script could become the basis for generalizing buildkite
annotations, a slight attempt at making it easily reusable in the future
has been made. So far we expect a report generating script to exit non
zero if a report should be made and to print commonmark to stdout. In
the future we may want to use a JSON format for generating the report,
allowing us to filter it by buildkite target (using the drvmap to
exclude certain reports, potentially).
Change-Id: I1df9e440509d69adff5b8e6304105a45dc62c018
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5260
Reviewed-by: kn <klemens@posteo.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
I think migrating the execline scripts over to bash makes sense:
1. Ever since nixpkgs-fmt, execline scripts in depot have become a huge
pain to write and edit and I can't think of a satisfying solution to
this problem.
2. The scripts here require remembering things across loop cycles (i. e.
the status variable) which is not possible in pure execline. As a a
workaround we used to read the entire report into memory first and
check if it was empty (tying us to the argv limit for the report
length).
Change-Id: I954b08b982ef947f9014a685676d2b83a2aec4d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5259
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
* //ops/machines/whitby: Disable grafana, since the grafana module was
changed upstream in a way that our configuration no longer works.
Since the OpenSSL security update is relatively pressing, adapting the
grafana configuration beforehand is not a hard requirement. See
https://github.com/NixOS/nixpkgs/pull/191768.
* //tools/depotfmt: keep Go at version 1.18 to forgo a reformat of the
tree.
* //nix/buildGo: keep Go at version 1.18, as 1.19 changed the CLI
interface (?) in a way that breaks buildGo.
* //3p/overlays/tvl: drop upstreamed tdlib upgrade.
* //3p/overlays/tvl: patch buf to work around breakage due to git 2.38.1
TODO items for Go are tracked in b/215.
Change-Id: Ie08fef49cf3db12e6b5225a8b992a990ddc5b642
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7141
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
The version of buf used is quite old.
nixpkgs provides a more recent version, but it requires us to migrate
config to the latest version.
depot_scanner.proto doesn't honor some of the conventions, so we need
allow_comment_ignores and drop a bunch of comments in there.
Change-Id: Ic978fe92fb7c8471f58c137497528f18aad8f3ab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7053
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: tazjin <tazjin@tvl.su>
Nixery's previous landing page was an mdBook that was basically
unmaintained and full of incorrect information. It also duplicated
some things (like nix-1p) which actually live elsewhere.
This commit removes the mdBook completely and reduces it down to a
simple TVL-style landing page. The landing page has been checked in
in its entirety because Nixery is frequently cloned through josh
without the entirety of depot, however the page has been created by
building it through depot's //web/tvl/template.
See also https://github.com/tazjin/nixery/issues/156
Change-Id: I20e1d58f1e6608377207e80345c169f7d92d3847
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6930
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
The latter has been deprecated in nixpkgs.
Relates to b/200
Change-Id: I42871ce3eb54ebf092909f033b43936b9610d982
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6836
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Upstream nixpkgs removed a lot of aliases this time, so we needed to do
the following transformations. It's a real shame that aliases only
really become discoverable easily when they are removed.
* runCommandNoCC -> runCommand
* gmailieer -> lieer
We also need to work around the fact that home-manager hasn't catched
on to this rename.
* mysql -> mariadb
* pkgconfig -> pkg-config
This also affects our Nix fork which needs to be bumped.
* prometheus_client -> prometheus-client
* rxvt_unicode -> rxvt-unicode-unwrapped
* nix-review -> nixpkgs-review
* oauth2_proxy -> oauth2-proxy
Additionally, some Go-related builders decided to drop support for
passing the sha256 hash in directly, so we need to use the generic hash
arguments.
Change-Id: I84aaa225ef18962937f8616a9ff064822f0d5dc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6792
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Discard string context in prepare-image.nix before parsing input read
with readFile with fromJSON. Required for compatibility with nix >2.3.
Change-Id: I3830707e80fd19a700551a15f1a96d2841d0b022
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6696
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Remove a race condition which appears when uploadHashLayer is called
with the same key from multiple threads simultaneously. This can
easily happen when the same image path is requested by multiple
clients at the same time. When it does, a 500 status is returned and
the following error message is logged:
{
"context": {
"filePath": "github.com/google/nixery/builder/builder.go",
"lineNumber": 440,
"functionName": "github.com/google/nixery/builder.uploadHashLayer"
},
"error": "rename /var/lib/nixery/staging/<hash> /var/lib/nixery/layers/<hash>: no such file or directory",
"eventTime": "...",
"layer": "<hash>",
"message": "failed to move layer from staging",
...
}
To solve this issue, introduce a mutex keyed on the uploaded hash and
move all layer caching into uploadHashLayer. This could additionally
provide a small performance benefit when an already built image is
requested and NIXERY_PKGS_PATH is set, since symlink layers and config
layers are now also cached.
Change-Id: I50788a7ec7940cb5e5760f244692e361019a9bb7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6695
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This adds scaffolding code for running the Nix language test suite.
The majority of eval-okay-* tests should eventually be runnable as-is
by Tvix, however the eval-fail-* tests might not as we intend to have
more useful error messages than upstream Nix.
Change-Id: I4f3227f0889c55e4274b804a3072850fb78dd1bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6126
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
This adds a new function (intentionally bound to a rare key (Q)) in
the push menu which can push a *private* change to Gerrit.
A private change is one that, until submitted, is only visible to its
owner and all explicitly added people (reviewers, CC).
Change-Id: I6ee13dbbad099584475d3efac96e5d9b86efbc26
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6061
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Since the source of nix-1p is checked in under //nix/nix-1p, we should
use it from there if Nixery is being built inside of depot.
Change-Id: Iddd54f7b93b398b2f909db6ee105366a9914a2ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5882
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
People occasionally ask what the current nixpkgs commit is on
nixery.dev (see e.g. https://github.com/tazjin/nixery/issues/153).
With this change, the commit is displayed on nixery.dev if Nixery is
built for the TVL deployment.
Change-Id: I795220214db5a367a126c9b4bd03754e9f144940
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5881
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
This can be re-used across Terraform environments.
Change-Id: I3d964a17d1cda1aff1df12bd4c0c3ee84b7f7748
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5850
Tested-by: BuildkiteCI
Reviewed-by: asmundo <asmundo@gmail.com>
In order to run this the secrets needs to be sourced, e.g.:
eval $(age --decrypt -i ~/.ssh/id_ed25519 $(git rev-parse --show-toplevel)/ops/secrets/tf-buildkite.age)
Change-Id: I9f6a02c0dac22f584181635861ddbb06cf849f14
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5838
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
`mg repl` is essentially a shortcut for nix repl $(mg path //) which
comes up often enough for me. Launching a repl only really makes sense
in the repository root with how readTree works at the moment, so I think
this is a convenient addition.
Change-Id: I32b695885c2e6eaecdcc656c7249afa504439913
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5822
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This exports the `:/tools/nixery` subtree to Github automatically
after merges to `canon`.
Due to the way the project was imported this continues the existing
git history in the external repository.
Change-Id: Ie871c14ad5d8f1019f8be86adecbe9b130ffb01a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5667
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This adds an extra step definition which can push the result of
running a josh filter on the repository to Github.
Change-Id: I1f93ae78e1bf452fbd1b21ce943a60acc85c944f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5666
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Nixery is going to gain a new binary (used for building images without
a registry server); to prepare for this the server binary has moved to
cmd/server and the Nix build logic has been updated to wrap this
binary and set the required environment variables.
Change-Id: I9b4f49f47872ae76430463e2fcb8f68114070f72
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5603
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This will be required for making a standalone, Nixery-style image
builder function usable from Nix.
Change-Id: I5e36348bd4c32d249d56f6628cd046916691319f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5601
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
The camelCase variant of `rebuild-system` doesn't exist, but the
kebab-case version does.
Side note: this `lazy-dispatch` upgrade is pretty cool. TIL `direnv`
supports `watch_file` and `PATH_add`.
Change-Id: Idc9109a9b0de327ddf7b9c6a4368b7bebb551196
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5565
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
nix-shell pollutes the environment with all sorts of variables. Let's
just add the tools to the PATH?
This also papers over the various differences in users `use_nix`
implementations by not using it at all.
Change-Id: If4282531fd6b7453b3611fe50217beacadc08bb5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5524
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This modifies the envrc configuration to add the result of building
//tools/depot-deps to $PATH, instead of dispatching through the
manually maintained list of symlinks.
While at it, I've cleaned up some stuff from that list that is no
longer actually used.
Change-Id: If345c44da75b23c06b7c7f435be0cb02f99aaac5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5513
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
in some cases, users might want to pass through flags for
nix-build (such as `-j`).
magrathea now accepts these as arguments to `mg build`, as long as
they are separated by `--`.
the arguments passed to `mg build` are parsed into a proper record,
which enables us to show users very clear error messages in case they
forget to use the `--` separator and keeping us future-compatible with
more potential arguments to magrathea itself.
Change-Id: I81f5d9db52779a5cc3b8bbdd975316274fffe5fc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5507
Tested-by: BuildkiteCI
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Reviewed-by: asmundo <asmundo@gmail.com>
Cleans up a whole bunch of things I wanted to get out of the door
right away:
* depot internal references to //third_party/nixery have been replaced
with //tools/nixery
* cleaned up files from Github
* fixed SPDX & Copyright headers
* code formatting and inclusion in //tools/depotfmt checks
Change-Id: Iea79f0fdf3aa04f71741d4f4032f88605ae415bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5486
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
This does not fully change the build structure of Nixery to be
depot-compatible yet, but should allow most targets to be built in
depot CI.
This contains some hacks to work around surface incompatibilities
which we'll clear away later.
Change-Id: I84e7734334abbe299983956f528c0897f49fa8c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5485
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This absorbs a josh-filtered Nix subtree into depot, at
//tools/nixery.
This subtree was created through `josh-filter ':prefix=tools/nixery'`,
which allows a filter on tools/nixery to yield the same commit hashes
as the original Nixery repository (allowing for history continuity).
Change-Id: Icc1a99bf1248226b91f437b0a90361d36fb0d327
This tool has been replaced by niv.
Change-Id: I011059b7d8890d0456b22f066e723584cc1d9a2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5329
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This has been superseded by magrathea.
Change-Id: Ief4a3d1b81e51e7a9c9a0112584fa7efc8aca63f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5328
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This command builds the desired target and runs the executable produced
by it. If a directory is produced, it looks for a single (!) executable
in the bin directory. Dot files are ignored, so wrappers should
generally work. In the future we could provide a flag to select one of
multiple executables.
All arguments following the target are passed to the executable as is.
Examples:
mg run ops/mq_cli ls
mg run web/bubblegum:examples
Change-Id: I6490668af68e028520973196d9daa5f1d58969ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5277
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Instead of the strict check-all-our-crates, generate a fake Cargo.lock
and add it to the report generated by check-all-our-lock-files.
check-all-our-crates was a reimplementation of cargo-audit anyways and
prevented us from updating the advisory db due to its strict
model (failing on any advisory).
Change-Id: I264a7f1a5058a527cbc46d26225352ecd437a22b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5230
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Rename check-all-our-lock-files to tree-lock-file-report and pull out
all the buildkite-specific code which makes the code less awkward.
check-all-our-lock-files is then only executed in extraSteps and runs
tree-lock-file-report on depot, adding it as a warning to the pipeline
if it is non-empty.
Change-Id: If6bd236d90cc680cba0ed4e988f2f28ddb8012d6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5229
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
This script is somewhat usable by humans (it even has a help screen!)
and can be reused in //users/sterni/nixpkgs-crate-holes. We are using
bash since that allows us to exit with the actual exit code of
cargo-audit - something that's not possible in execline.
Change-Id: I3331ae8222a20e23b8e30dc920ab48af78f0247c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5228
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
I want to add a shortcut to build and run e.g. scripts that are depot
targets - for which it would be useful to not have stdout polluted by
magrathea itself.
Change-Id: Ic58fe28eafb4d0715e53beae041bfaa5d1745812
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5276
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
The rustfmt configuration of the depot is moved to `rustfmt.toml` (it
is recognised more reliably from this path than from the hidden
.rustfmt.toml).
Nested configuration is theoretically possible, but detection of
nested config files is flaky. Paths with nested config files need to
be disabled in the top-level check (I've excluded my user directory).
Change-Id: I385ce3ef529bda28fac03bfba86fc204c81b8a61
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5241
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
apparently the cache can get out-of-sync or something; either way we
had a build where it missed a file that was misformated.
Change-Id: I2967aec99ee1c7c8b978a3dfdfed4ff213bb6591
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5249
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
In some cases we want to be able to "emergency approve" something on
behalf of a different user.
Example cases:
* clean up of abandoned directories with restrictive OWNERS
* security fixes blocked on people in different timezones
This script can be used to perform these approvals if the user is a
member of depot-interventions. Note that access to depot-interventions
is audit logged.
The user on behalf of whom approval is performed is always added to
the attention set to ensure that they are made aware of the CRFO
approval.
Note: This depends on nixpkgs#156466. Keeping WIP until we have a
channel with that patch.
Change-Id: I16e5f9d7baa9daab49c88b629bb8f024aad9d94c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5085
Tested-by: BuildkiteCI
Reviewed-by: kn <klemens@posteo.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Many of the vulnerabilities (in the respective crates) reported are not
actually exploitable vulnerabilties of the packages we report them for.
Consequently it is more accurate to state that they are advisories.
Change-Id: I02932125b77fc9c71e583ae49e822fd3438dce05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5202
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Buildkite doesn't understand GitHub Flavored Markdown and having a read
only checklist in there is probably not much use.
Change-Id: I41538487087e8c817b1a5e653f077bb0fbe6eb47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5201
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
In the spirit of the readTree filter we should also not include files in
user directories from the outside.
Change-Id: I1abe36a721048900d2758b5986063b68b8d1af93
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5200
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
check-all-our-lock-files works very similarly to
//users/sterni/nixpkgs-crate-holes, even reusing some parts of it, but
is much simpler since we don't need to extract the lock files — they are
already in tree.
It is implemented as a very simple script which just traverses the
subtree of the current directory, collecting all warnings. When
executing this script in buildkite via extraSteps, it never fails,
instead annotating the pipeline run with a warning.
Change-Id: I0a0bc26deffe7b20b99f5aa7238fb3c3bb9deb92
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3721
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Any other cgit configuration in depot would need this script wrapper as
well.
Change-Id: Ifa04e1c9de9c925eb3f60c5d3854221ae02ef06c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5206
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
The function is depot specific and thus uses tvl-depot-path, so it
belongs in `tvl.el`. Since non-sly-users won't need it, we tie its
definition to loading the sly package.
Change-Id: I8b104deab455d218d3df6a800e35cc104220a841
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4960
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
We can't use cl-lib as that apparently doesn't contain lexical-let*.
Change-Id: I8e65d20215ae5667bb92b71e6318ad9d66125320
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4941
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
A formatting error broke this at some point (the let clauses were
outside of the definition list).
Change-Id: Iaa2dc9ad02d2f7e909ca9bf28705e782ad26060b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4765
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Produces more useful output and also makes for a good target for the
upcoming extraSteps logic.
Change-Id: Ifd389d433d9e27f97940a48999f4fba35646e37a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4727
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: sterni <sternenseemann@systemli.org>
Two minor "quality of life" improvements:
- automatically set SSL_CERT_FILE environment variable,
so that programs relying on OpenSSL for certificate
validation can actually validate certificates
(the certificates are included no matter what since
we add the "cacert" package to all iamges)
- if the requested image includes an interactive shell
(e.g. if it includes the "shell" metapackage), set
the image Cmd to "bash", which allows to execute
"docker run nixery.dev/shell" and get a shell)
I'm happy to split this PR in two if you'd like, but
since both features touch the Config structure and are
rather small, I thought it would make sense to bundle
them together.
`terraform fmt` can only handle a single path, but treefmt expects
formatters to be able to handle multiple paths at once.
this wraps it in a small shell script that calls `terraform fmt` with
at most one path at a time.
Change-Id: I2b9c1b89b5a276f3d4915b95608ce36b2509e334
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4639
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
Reviewed-by: grfn <grfn@gws.fyi>
previously, depot-scanner swallowed/ignored all non-processed
stderr output of nix-instantiate, which makes diagnosing
failures of nix-instantiate (e.g. failed with exit status 1)
difficult. This commit fixes that by always forwarding
the remaining stderr messages.
Example previous error message:
panic: nix-instantiate failed: exit status 1
goroutine 1 [running]:
main.main()
/nix/store/8vb2j13bd7j5ipl7dhsnwvgr7nrrsqsi-main.go:160 +0xeb4
Example new error message:
nix-inst> error: unrecognised flag '--trace-file-access'
nix-inst> Try '/run/current-system/sw/bin/nix-instantiate --help' for more information.
panic: nix-instantiate failed: exit status 1
goroutine 1 [running]:
main.main()
/nix/store/qy7v79a3harddirzmc0432vbzqhyf91i-main.go:165 +0xeb4
Change-Id: I666f3490fc648f77a5384b95edd74f6115f7920d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4553
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Right now this only includes Go, but more is to come.
Change-Id: Idd8fc27c0eb25e82688ef8337ba20810d834f4b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4504
Reviewed-by: zseri <zseri.devel@ytrizja.de>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Autosubmit: tazjin <mail@tazj.in>
These instructions were not up-to-date (they didn't mention
the different storage backends, and some variables were
tagged as optional while they were mandatory). With this
update, they should (hopefully) be more accurate! :)
I also added instructions if someone wants to run Nixery
outside of the container image (I found it convenient when
working on Nixery's code).
(zseri:) It seems like relativePath should be replaced with relativeFlag,
as no variable with the former name exists, a boolean is as far as I can
tell expected, and a boolean with a similar name exists. Lets give it a
try in the CI.
Change-Id: I0e7e522a41a517a38222dcda3b66731344613c1e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3581
Reviewed-by: tazjin <mail@tazj.in>
Autosubmit: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
Prompting with "Checkout CL" for the gerrit cherry-pick command doesn't
make any sense.
Change-Id: I51495e7975202146fae2da0807e525596f2d490e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4516
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Right now the only included formatter is gofmt, but we will extend
this over time.
The version of treefmt is bumped to 0.3.0 (which supports custom
config files) until this lands in nixpkgs.
Change-Id: I1e1aafd05ec7427c616f90c90490c528ecb2615c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4399
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
All targets would have no trailing slash, so it was at least ugly that
// had one as the only legal target.
Change-Id: I1b60850ac86d8c550f262841694fb00c518413b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4404
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
r/3000 will now be shortlinked to show the commit in cgit. Going via
atward probably doesn't make sense at this point, since the depot refs
are not available in sourcegraph at all (for reasons I can't
repeat). Switching to atward might be interesting when/if we introduce
support for shortlinking // paths.
Fixes: b/163
Change-Id: I57c1a7d02d881e4f8b3ee1f71755dd7930925dc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4402
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <mail@tazj.in>
for global installations of magrathea, setting MG_ROOT can be a way to
switch quickly between different repositories (esp. in combination
with `cd (mg path)`).
Change-Id: I4627fe78b7cc112b75ab57e7806ffd85c6d38aee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4396
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
this command prints the absolute directory for a given target. it can
be combined with shell aliases to add quick navigation commands.
unfortunately due to the nature of computers implementing something
like `mg cd` directly is not possible.
Change-Id: Icc88eb97384812c620c49fe2de8fa331f4d7153b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4395
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
magrathea now does what it says on the tin - build and shell commands
can be used with the targets specified on the command line.
implementation notes:
* string representation of target has been changed to look like the
target spec format, this is now used in user-facing messages
* errors returned by the target parser make the program exit with
status 1
* normalisation could be done better (for example, maybe it makes
sense to always do it) but it's good enough for now
Change-Id: Ib85f389a5cec92b3c2f3b9c0b40764435bbcc68b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4394
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
i'm not sure what happened here, but it works (yes, the fancy target
printing is completely unnecessary, but oh well):
#;152> (parse-target "foo")
#target(foo)
#;153> (parse-target "//foo")
#target(//foo)
#;154> (parse-target "//foo/bar")
#target(//foo/bar)
#;155> (parse-target "//foo/bar/")
#target(//foo/bar)
#;156> (parse-target "//foo/bar:baz")
#target(//foo/bar:baz)
#;157> (parse-target "//foo/bar/:baz")
#target(//foo/bar:baz)
#;158> (parse-target "//foo/bar:")
(error . "unexpected end of input while parsing virtual target")
#;159> (parse-target "//foo//")
(error . "unexpected root-anchor while parsing normal target")
the most notable thing is that trailing slashes are allowed in the
physical targets, since people may be autocompleting these on the
shell from folder names.
Change-Id: I32975ad77fe2a327130dc9574011fe92cce49f84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4393
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
