Something I missed last time reading through the process documentation
is that you can use a combination of `process` and `process-wait` to
determine the exit status of a child process *and* read from its
standard output. With `process*` we could even capture stderr, but we
probably want it mounted to the parent process' stderr anyways.
Change-Id: I9840f607df465caa80d28109e344e5fc1402949d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7259
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
**TL;DR:**
Most of these changes predicate behavior on the platform:
- At buildtime this is `localSystem == $something`. (`localSystem` is from
`readTree`)
- At runtime this is `(memq window-system '(ns))`.
- Add `coreutils` so `dired` listing with `--group-directories-first` works
because that flag depends on the GNU version of `ls`.
**Background:**
I need to support a bunch of OSX users at $WORK. As such, I'm planning
on using my MBP for the next few weeks to build empathy for our
userbase and polish some currently rough edges.
If I'm going to get an serious work done, I need my Emacs setup. Step
one is making sure it can build and run.
Change-Id: I918efccfa5f149e218aeea476c2c7df1c7b64ae8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7309
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Drop into a new shell environment with the same variables defined in a systemd
unit file (for debugging purposes).
Change-Id: Iaf513809b524f3f3e845b512450da71694bb7c7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7308
Autosubmit: wpcarro <wpcarro@gmail.com>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
It's impossible to log into many public wifi networks otherwise when
the login depends on DNS hijacking.
Change-Id: I09f8b504810eebeb788997d2100a6db4777a8725
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7307
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Proof-reading much easier when Markdown is rendered in the browser.
Change-Id: Ia173dea817866d214547546bc3428ac6fe6782bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7305
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
By re-using the nix_2_3 expression from nixpkgs we are no longer stuck
with the 2021 expression from Nix's release.nix and the resulting
derivation has a shape that other expressions in nixpkgs expect (e.g.
nix-serve), so we can actually overlay our fork into the nix_2_3
attribute. This should reduce duplication on e.g. whitby.
Since there is no nixUnstable expression in nixpkgs anymore, it expects
a Nix release tarball which we produle using the appropriate release.nix
job from the repository. Sadly there is some trickery involved in
getting it to respect localSystem, since the tarball job is not based on
the passed in systems list.
Change-Id: Ib49f298334d166327f91559a06b0a37b2488bc63
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7262
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
Since cl/7260 has eliminated all uses of builtins.currentSystem
in the main evaluation path of depot, we can ensure that we use
localSystem consistently in the future by making it impossible
to access the value of builtins.currentSystem in readTree nodes.
Change-Id: I6ded54021c42fcf31a80268149179f95f22bad88
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7261
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
cl/5832 added a global system parameter to depot which allowed specifying
what `system` should be used for nixpkgs and all depot derivations
(assuming a native compilation case) which was implemented in cl/5846.
This allows instantiating derivations for a different system than
whatever builtins.currentSystem happens to be. This is useful for
debugging, allows you to schedule builds on build servers for
other platforms or build for architectures that are a subset of
the one you are running (e.g. i686-linux).
This change eliminates all remaining uses of builtins.currentSystem
which could lead to an inconsistent combination of `system` values
when passing `localSystem`.
Change-Id: I0f824f4f0afa88ef1ddd9a8cecb24bf94bacde7a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7260
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
Both //tvix/eval and //tvix/nix_cli have need to for rust tooling available
in $PATH.
Move this one level up, so it's accessible in all subdirectories.
Change-Id: I0763bbe9cefdc962f3a8f86c51e8f67cde8b4b04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7248
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
This detects if the second argument of a division is a zero (either as integer
or as float). If so, an error message is displayed.
This fixes b/219.
Change-Id: I50203d14a71482bc757832a2c8dee08eb7d35c49
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7258
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This reverts commit 6813598c17.
Reason for revert: This function is a re-implementation of
builtins.unsafeDiscardOutputDependency which I missed at the time.
Change-Id: I5bb52bfd5e8d51defaf90ee795b0fe99be84f6db
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7265
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
As per the discussion in
https://cl.tvl.fyi/c/depot/+/7128/2..5/tvix/proto/castore.proto#b39, ref
sounds more like an external, stateful ID. Call this `digest`, to make
clear it's precisely this.
Change-Id: I81dd3769e2ce017de470ae92f72a38fb72015f10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7134
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
`buf` doesn't like protos with different package names in the same
directory.
Change-Id: I30806b46b88f103779faa40466461091a4a01e06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7130
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Make it clear these are symlinks, not hardlinks.
The term "link" is too heavily correlated to other meanings in IPFS/IPLD
world, and calling this symlink removes this confusion.
Change-Id: Id3f1eaa32098510b05f3e1a1348344503bcb4d5a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7129
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Add a new `documentation: Option<&'static str>` field to Builtin, and
populate it in the `#[builtins]` macro with the docstring of the builtin
function, if any.
Change-Id: Ic68fdf9b314d15a780731974234e2ae43f6a44b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7205
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Refactor the arguments of a Builtin to be a vec of a new BuiltinArgument
struct, which contains the old strictness boolean and also a static
`name` str - this is automatically determined via the ident for the
corresponding function argument in the proc-macro case, and passed in in
the cases where we're still manually calling Builtin::new.
Currently this name is unused, but in the future this can be used as
part of a documentation system for builtins.
Change-Id: Ib9dadb15b69bf8c9ea1983a4f4f197294a2394a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7204
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Some new top-level re-exports (specifically VM, Builtin, and ErrorKind)
were added to lib.rs in tvix/eval to allow the builtin-macros tests to
work - we should be clear which of these are part of the public
interface (I think it's reasonable for ErrorKind to be) and which
aren't (specifically I'm not sure VM and Builtin necessarily should be,
at least yet).
Change-Id: I3bbeaa63cdda9227224cd3bc298a9bb8da4deb7c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7203
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Similar to what we did with pure builtins, define the impure builtins
within a module at the top-level using the new #[builtins] attribute
macro
Change-Id: Ie5d5135d00bb65e651531df6eadba642cd4eb08e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7202
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Break out all pure builtin functions to top-level functions defined
within the `pure_builtins` module in `builtins/mod.rs`.
Change-Id: I9a10660446d557b1a86da4c45a463e9a1a9b4f2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7201
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Mostly as a proof-of-concept of the new proc-macros for defining
builtins, define a single builtin (the first in the list, `abort`) at
the top-level of a child module within builtins/mod.rs, and add it to
the list of builtins returned from `pure_builtins`.
If this works nicely, we can start breaking out the rest of the builtins
into the top-level too, in addition to introducing additional sets of
builtins (to differentiate between pure and impure builtins).
Change-Id: I5bdd57c57fecf8d63c9fed4fc6b1460f533b20f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7199
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Add a single new proc macro to a new proc-macro crate,
`tvix-eval-proc-macros` for defining an inline module containing nix
builtins, and automatically generating a function within that module
which returns a list of those builtins as `tvix_eval::value::Builtin`.
Change-Id: Ie4afae438914d2af93d15637151a49b4c68aa352
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7198
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
This commit adds a markdown document which explains how the
thread-local VM infrastructure works, in case it is useful in the
future.
Change-Id: Id10e32a9e3c5fa38a15d4bec9800f7234c59234a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7193
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: tazjin <tazjin@tvl.su>
This is a follow up to cl/7191 which neglected to adjust the
status.tvl.su.nix module and re-enable it.
Change-Id: Icc1917004cd50e5eab61a29bc68b393ba9bd6325
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7226
Tested-by: BuildkiteCI
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Profpatsch originally implemented an advisory checker from scratch in
Rust. We now ended up just using cargo-audit for the global checks
exposed via CI and the custom implementation is unused. To clean up
//tools/rust-crates-advisory a bit, we can move the unused parts to his
user directory.
Change-Id: Iacbd27c163edd07c804220fd1b3569c23aebd3e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7171
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
We need a user in the context when we ask the groups backend to look up
groups by name, so for now if we don't have a _real_ user in the context
(such as during change indexing), then populate the context with the
anonymous user just for the duration of the groups backend calls.
Change-Id: If961d84fe57443cb95deb59628802658585ed1cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7172
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
This is more generally useful than just inside the VM, until it is
stabilised in Rust itself.
Change-Id: Id9aa3d5b533ff38e3d2c6b85ad484394fdd05dcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7186
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: grfn <grfn@gws.fyi>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Reference the non-deprecated version of the grafana http port option in
the proxyPass for the nginx config of mugwump.
Change-Id: Ic7f370c7f7a451fe95a046d491d7b1cdf5f728cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7200
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
This fixes a mistake I made in d978b556e6.
Change-Id: I88db697105a7149e9785f6aface03bff68566d2b
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7085
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Scope_depth and with_stack_depth were being reset to zero for nested
function abstractions. Fortunately nothing depends on them being
computed correctly in these cases, but it sure was confusing.
Change-Id: I59980b6a5aff043f60079f97211220b0086eb97d
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7091
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
It is very confusing that this opcode is called DataLocalIdx, but it
carries a StackIdx rather than a LocalIdx. It seems like this
really ought to be called DataStackIdx, but maybe I've
misunderstood; if so please explain it to me.
Change-Id: I91f6ffa759412beef0b91d3c19ec0d873fe51b99
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7088
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Uncomment and update the grafana config for whitby based on the new
config format that nixos accepts. I've validated this locally by
visually inspecting the resulting `ini` file, but not actually run it
yet.
Change-Id: I12d78ae48146e1b01bd2a4152276d4c6b16c1a3d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7191
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Lots of deprecations were made in the new nixos version for the grafana
config - this updates all of those settings in mugwump's system config
Change-Id: I69cdc9d2d59702c38d6334a4d27a04bef4e8c132
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7190
Reviewed-by: grfn <grfn@gws.fyi>
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
I've elected to split the check-all-our-lock-files script into two new
scripts: One very simple script which generates the report by invoking
lock-file-report on the fake lock file for //third_party/rust-crates and
all lock files in depot, and one which executes this and adds it as a
buildkite annotation if there are any warnings (which is reported by the
report generating script using a non zero exit code).
The latter script could become the basis for generalizing buildkite
annotations, a slight attempt at making it easily reusable in the future
has been made. So far we expect a report generating script to exit non
zero if a report should be made and to print commonmark to stdout. In
the future we may want to use a JSON format for generating the report,
allowing us to filter it by buildkite target (using the drvmap to
exclude certain reports, potentially).
Change-Id: I1df9e440509d69adff5b8e6304105a45dc62c018
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5260
Reviewed-by: kn <klemens@posteo.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
I think migrating the execline scripts over to bash makes sense:
1. Ever since nixpkgs-fmt, execline scripts in depot have become a huge
pain to write and edit and I can't think of a satisfying solution to
this problem.
2. The scripts here require remembering things across loop cycles (i. e.
the status variable) which is not possible in pure execline. As a a
workaround we used to read the entire report into memory first and
check if it was empty (tying us to the argv limit for the report
length).
Change-Id: I954b08b982ef947f9014a685676d2b83a2aec4d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5259
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This implements builtins.split, and passes eval-okay-regex-split.nix
(which is moved out of notyetpassing).
Signed-off-by: Adam Joseph <adam@westernsemico.com>
Change-Id: Ieb0975da2058966c697ee0e2f5b3f26ccabfae57
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7143
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
This was broken in my blog for way too long.
Change-Id: I03c45c666d67006a4608a4b19d6167ab692e321d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5905
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
