Commit graph

18683 commits

Author SHA1 Message Date
John Ericson
26c68f8e89 refactor(nix-compat): Properly encapsulate store path construction
Before there was code scattered about (e.g. text hashing module and
derivation output computation) constructing store paths from low level
building blocks --- there was some duplication and it was easy to make
nonsense store paths.

Now, we have roughly the same "safe-ish" ways of constructing them as
C++ Nix, and only those are exposed:

- Make text hashed content-addressed store paths

- Make other content-addressed store paths

- Make input-addressed fixed output hashes

Change-Id: I122a3ee0802b4f45ae386306b95b698991be89c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8411
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-04-09 15:12:21 +00:00
sterni
b4670bfbd1 chore(3p/sources): Bump channels & overlays
* //3p/gerrit{,_plugins}: adjust for API change to buildBazelPackage

  440b4de588

* //3p/gerrit_plugins: update hash of deps jar

Change-Id: I131d5846acbce718126fb47671893a568d1020dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8445
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-04-09 13:41:21 +00:00
sterni
ba4ea9aa88 chore: another year of licensing
Change-Id: I1cec0d3ee832ab102c883abaf0169dfd7ef38e5e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8473
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-04-09 13:34:18 +00:00
Profpatsch
56b2ae0e59 docs(users/Profpatsch): Add toplevel README
Change-Id: Iae4b46932cb964c5668ed679ad068c20d480cdf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8472
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-04-08 13:16:00 +00:00
Profpatsch
970fd0a891 feat(users/Profpatsch): Actually add the mentioned .hlint file
Change-Id: I6d08379835160dfb941fe45e708cfc2f942acfbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8471
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-04-08 13:16:00 +00:00
Profpatsch
80e04da5d4 docs(users/Profpatsch): Add small README to most subdirs
Change-Id: I2912e32fe1b2d3d3aff7bafba809634ec4c9adb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8470
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-04-08 12:25:07 +00:00
Profpatsch
260626e016 refactor(users/Profpatsch/lorri-wait-for-eval): use MyPrelude
Change-Id: I45aec0cd178cd51b1857b576e9c3a8ccc1866100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8469
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-04-08 12:25:07 +00:00
Profpatsch
ca7c1a4eae chore(users/Profpatsch): delete struct-edit
Change-Id: I17c6c1b2659267a74ed252e18363c46ea89d99fc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8468
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-04-08 12:25:06 +00:00
Profpatsch
34d89cd6a8 chore(users/Profpatsch): delete nixpkgs-rewriter
Change-Id: I0e8a1f4e6e23cbeead46ea60129cad66d3ffe844
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8467
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-04-08 12:25:06 +00:00
Profpatsch
0d4ca2c31b chore(users/Profpatsch): delete check-crate-advisory
Change-Id: Iefd7d8bcd699c9bef5c88e730e1d2dc5ec407ee0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8466
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-04-08 12:25:06 +00:00
Profpatsch
5e400b5b24 chore(users/Profpatsch): bring reverse-haskell-deps into shape
Add a cabal file and move into subdir.
Use MyPrelude & fix a few linter warnings.

Change-Id: I19d5ba47be789fc24f8e02ee8721f73c706ae3e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8465
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-04-08 12:25:05 +00:00
Florian Klink
6edc2182d1 docs(tvix/store): update comment regarding reachability check
This might be more understandable.

Change-Id: I91371d304a9a316a2934ce41929998eaef67e11c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8416
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-04-07 09:35:39 +00:00
Florian Klink
60751b70eb docs(tvix/store/grpc_directorysvc_wrapper): add TODO
The logic validating connectivity of Directory nodes should be moved
to SimplePutter, and this use whatever DirectoryPutter the store comes
with.

Change-Id: Id68a86a96cc49ff73920017839788859ea9c5161
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8358
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-04-07 09:35:39 +00:00
sterni
0cb2057a76 feat(emacs-pkgs/tvl): allow starting sly only including dependencies
Usually the current behavior is best: You are dropped in a REPL with the
package(s) you are working on already available. As you are working on
them, you recompile individual files and your changes become available.

However, I've found that there are some occasions when this is not
desireable, e.g.: When you are working on something and have broken the
test suite intermittently, it becomes impossible to start a new REPL.

Not sure how the yes-or-no-p question should be phrased, its negation
may be better?

Change-Id: I6a37ebc02f3121f628fc9206e0de650851824cd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8415
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-04-07 09:32:38 +00:00
Florian Klink
8263c024c9 refactor(tvix/store/import): use DirectoryPutter in import.rs
This should allow import_path to communicate to a gRPC remote store,
that actually verifies the Directory nodes are interconnected.

Change-Id: Ic5d28c33518f50dedec15f1732d81579a3afaff1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8357
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-04-07 09:26:36 +00:00
Florian Klink
0836450006 feat(tvix/store/directorysvc): add put_multiple_start
This provides a handle to upload multiple proto::Directory as part of
the same closure.

Change-Id: I9213dde257a260c8622239918ea541064b270484
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8356
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-04-07 09:26:36 +00:00
sterni
96d7f4f0ac chore(3p/sources): Bump channels & overlays
* Satisfy new assert that the corresponding shell needs to be enabled
  via programs.* if it is as the login shell of at least one user.

* //users/tazjin: “Address” removal of hardware.video.hidpi option.

* //3p/gerrit: update fetch sha256

Change-Id: Id0988a0ea7f393d6b7848a7104fc3526ee1177f4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8407
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-04-07 09:20:33 +00:00
sterni
9b78aa1ad6 chore(sterni/machines/edwin): minecraft 1.19.2 -> 1.19.4
Change-Id: Ib6141a6d905220395d822ac1d8e4f47aa89161a3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8425
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-04-05 16:56:49 +00:00
Vincent Ambo
f687f5c3a3 fix(tazjin/nixos): scrot is out, maim is in
Scrot doesn't work very well anyways, it keeps producing crazy
artifacts. Maim seems more sane.

Change-Id: I0823b1755b7810c8b35a9c53f232d2e7f1722943
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8414
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-04-01 15:21:30 +00:00
John Ericson
a8bd926e0f refactor(nix-compat): Avoid encoding round trip
When building store paths we can just construct the thing.

Change-Id: Ife5d461d6a440ecbb22f32a86a6d51d212a2035b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8409
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-31 18:46:33 +00:00
John Ericson
b932cf2d85 refactor(nix-compat) Make nixhash_* mods hierarchical
They can go under `nixhash`

Change-Id: Ia15835c57130b66d58f5df80ae9595dceee00941
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8408
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-31 18:46:18 +00:00
John Ericson
df78a4d24e refactor(tvix/nix-compat): Inline texthash mod
It is moved into `store_path::utils` with the other path builders.

Change-Id: I3257170e442af5d83bcf79e63fa7387dd914597c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8410
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-31 18:46:08 +00:00
Vincent Ambo
ce502bdc89 refactor(tvix/eval): improve representation of chunk/span mapping
This switches out the previous compressed representation (count of
instructions per span) with a representation where the chunk's span
list stores the index of the first operation that belongs to a span,
and finds the right span by using a binary search when looking them
up.

This improves the lookup complexity from O(n) to O(log n).

This improvement was suggested and (mostly) implemented by GPT-4. I
only fixed up some names and updated the logic for deleting
spans (which it only did not do because I didn't tell it about that).

The code was verified by producing a complex error before/after the
change and ensuring that all spans in the error match exactly.

Co-Authored-By: GPT-4
Change-Id: Ibfa12cc6973af1c9b0ae55bb464d1975209771f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8385
Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-03-31 16:46:19 +00:00
Florian Klink
5bf9134324 refactor(tvix/store/directorysvc): move DirectoryTraverser to utils
Change-Id: Ie60a660e0fda7c80a6c7de20404c1965fe0e0d63
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8355
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-03-31 10:54:48 +00:00
Florian Klink
e9686f84d9 fix(views/kit): communicate :unsign in the tvl-kit URL directly
Instead of prepending :unsign to all URLs in josh-proxy, and for all
calls to filteredGitPush, explicitly use it only in the filter we use
for the `export-kit` extraStep.

This means, people cloning tvl-kit via

> https://code.tvl.fyi/depot.git:workspace=views/kit.git

now need to update the URL to point to

> https://code.tvl.fyi/depot.git:unsign:workspace=views/kit.git

instead.

git@github.com:tvlfyi/kit.git will keep the same hashes, as it's updated
to export the unsigned workspace view of it.

This is less invasive than dooming every josh workspace to have to strip
signatures.

Change-Id: I6de05182fad4c3695081388c3bbf37306521d255
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8369
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-31 08:46:01 +00:00
sterni
1139109063 refactor(3p/lisp/mime4cl): replace babel with flexi-streams
decode-RFC2047 used babel's octets-to-string, but we can replace it with
the function of the same name from flexi-streams. This doesn't make a
difference for the moment, but will be useful in the future:
flexi-streams provides de- and encoding streams that we'll be able to
use to replace and augment some of the stream based MIME part handling
code in mime4cl. babel doesn't have as powerful stream functionality
although it seems to be planned.

Another big upside of flexi-streams is that we'll be able to replace
delimited-input-string using it. This should allow us to slowly work
towards correct and more efficient decoding of MIME bodies.

Change-Id: I17174f1c96c5be7d103d396564e6aa0fe24c80fc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8371
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-03-30 16:11:54 +00:00
sterni
e292d82d83 feat(sterni/emacs): disable warning 8 (dash len) in chktex flychecks
Change-Id: I394c48395cdc50bea3747f55d95f169e599018ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8370
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
2023-03-30 16:11:54 +00:00
Florian Klink
e8fd2aabb3 fix(tools/releases): add :unsign to filteredGitPush
We need to followup on cl/8186, and use the same workaround for josh-
filter that we use for josh-proxy, so we can push again.

This could potentially become an optional function argument, if new
subtrees are to be carved out, and we want to keep signatures on commits
in there.

Change-Id: I7a9b821a7365767e8a4188e8200fa3c37463142c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8368
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: flokli <flokli@flokli.de>
2023-03-30 14:28:39 +00:00
Florian Klink
5f2c2e79e1 refactor(tvix/nix-compat): move build_store_path out of derivation
This doesn't have anything to do with ATerms, we just happen to be using
the aterm representation of a Derivation as contents.

Moving this into store_path/utils.rs makes these things much cleaner -
Have a build_store_path_from_references function, and a
build_store_path_from_fingerprint helper function that makes use of it.

build_store_path_from_references is invoked from the derivation module
which can be used to calculate the derivation path.

In the derivation module, we also invoke
build_store_path_from_fingerprint during the output path calculation.

Change-Id: Ia8d61a5e8e5d3f396f93593676ed3f5d1a3f1d66
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8367
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-30 14:03:35 +00:00
Florian Klink
4ab180b1eb refactor(tvix/nix-compat): use Sha256::new_with_prefix
Change-Id: I328983008c0680e0448ccff604dbdae312f7988b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8366
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-30 14:03:34 +00:00
Florian Klink
6454769eef refactor(tvix/nix-compat): drop is_derivation in build_store_path
This only added a suffix to the input argument, if build_store_path was
building the path of a Derivation.

As we need to also add the `.drv` suffix to the name we pass into
text_hash_string inside calculate_derivation_path, we can simply add the
suffix there and drop the parameter from build_store_path.

Change-Id: Icd5343dd1458f112b9296b389e81ce2ebdd16a9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8365
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-30 14:03:34 +00:00
Florian Klink
971080c912 refactor(tvix/nix-compat): add text_hash_string function
Use it to calculate the text_hash_string, which is then used in the
calculate_derivation_path and path_with_references functions.

Relates to b/263.

Change-Id: I7478825e2a23a11224212fea5e3fd06daa97d5e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8364
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-30 14:03:34 +00:00
Florian Klink
14c5781389 refactor(tvix/nix-compat): move HashAlgo to separate file
… and keep the pub exports as is.

Change-Id: I2ad21660577553395f05b5ba71083626429b0dfc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8363
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-03-30 13:47:31 +00:00
Florian Klink
e53152866e refactor(tvix/nix-compat): move NixHashWithMode into separate file
… and keep the pub exports as is.

Change-Id: I9f89a738c508c478ddba61303c21ea294f01ee9f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8362
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-30 13:47:31 +00:00
Vincent Ambo
73c12f7c4f fix(ops/www): allow all indexing on cl.tvl.fyi
I *want* search engines to index our CLs, they might be useful!

Change-Id: I956d92c80d812e1aefefb6daeba77a1588055b94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8361
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
2023-03-29 12:17:56 +00:00
Vincent Ambo
bf88421dc7 feat(views): export tvl-kit view
Change-Id: I739adb7d32d03294f8ec9962c3d93e35842af83b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8360
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-03-28 09:42:13 +00:00
Vincent Ambo
08bf03a3d8 refactor(views): move view export configuration to //views
Change-Id: I064b996fd52134b7e83541ea1190774c51a8ba30
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8359
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-03-28 09:42:12 +00:00
sterni
b340b8a57f refactor(sterni/mblog): drop redundant <style> element
Change-Id: I7cf9681130cc2f99050ebc0c736a4e1387ce8fbc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8350
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2023-03-27 17:39:56 +00:00
Florian Klink
d884890e56 docs(tvix/store/directorysvc/grpc): fix comment
We do compare for equality. This comment probably was when I tried to
compare the `Result<T, E>`, and as `E` doesn't derive PartialEq it was
annoying.

Change-Id: I18bb19528c76af91c9d24d88d55dd46d0c092d20
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8354
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 14:37:33 +00:00
sterni
34553dfac5 chore(3p/sources): bump home-manager
Previously there was an issue in nmd which prevents it from working with
Nix 2.3 (it passed --store dummy://) which is why we hadn't updated
home-manager for a while. home-manager has now [included] the [fix] for
nmd I proposed, so we can finally update!

[fix]: f5a1859425
[included]: db37c53760

Change-Id: Ia1447549c0f97aa754ac1842eb453e95838c00c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8346
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 14:32:31 +00:00
sterni
deccd8d39e chore(3p/sources): Bump channels & overlays
* //users/wpcarro/avaSystem: disable hidpi

  Recent changes have made nixpkgs adopt the position that hidpi
  optimization can't be done generically and at the very least needs to
  know a specific DPI number to optimize for. In addition to knowledge
  of the display(s) in question (i.e. wpcarro needs to do this) the
  issue <https://github.com/NixOS/nixpkgs/issues/222805> can give
  guidance as to how to restore the desired hidpi look and feel.

Change-Id: Ia4b079a06dcb710050619f350cd0655216b4a42f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8345
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-27 14:32:31 +00:00
Florian Klink
5553a175e6 chore(tvix): bump Cargo dependencies
Change-Id: I421f7a9440bbf18646143cd9fbfe790db22a0e80
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8352
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 09:03:34 +00:00
Florian Klink
2fe53cce40 feat(tvix/store/directorysvc): add DirectoryService::get_recursive()
This moves the recursive BFS traversal of Directory closures from the
GRPCDirectoryServiceWrapper out into a a DirectoryTraverser struct
implementing Iterator.

It is then used from various implementors of DirectoryService in the
`get_recursive()` method.

This allows distinguishing between recursive requests and non-recursive
requests in the gRPC client trait implementation.

Change-Id: I50bfd4a0d9eb11832847329b78c587ec7c9dc7b1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8351
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-27 09:03:34 +00:00
Vincent Ambo
2d305fd5b3 refactor(tvix/eval): retain call frames when entering calls
This grows the frame stack as the call stack grows, which yields *much*
better user-facing error messages.

I haven't measured the performance impact this has yet, for now I'm
still just trying to add more information to errors and then cut down
again where necessary.

Change-Id: I89f058ef31979edacf4667775d460b60704ce4d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8334
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-03-27 09:02:43 +00:00
Florian Klink
367a5e9922 feat(tvix/store/directorysvc): add gRPC client
This provides a GRPCDirectoryService struct implementing
DirectoryService, allowing a client to Directory objects from a (remote)
tvix-store.

Remote in this case is anything outside the current process, be it
another process, or an endpoint on the network.

To keep the sync interface in the `DirectoryService` trait, a handle to
some tokio runtime needs to be passed into the constructor, and the two
methods use `self.tokio_handle.spawn` to start an async function, and
`self.tokio_handle.block_on` to wait for its completion.

The client handle, called `grpc_client` itself is easy to clone, and
treats concurrent requests internally. This means, even though we keep
the `DirectoryService` trait sync, there's nothing preventing it from
being used concurrently, let's say from multiple threads.

There's still two limitations for now:

1) The trait doesn't make use of the `recursive` request, which
   currently leads to a N+1 query problem. This can be fixed
   by `GRPCDirectoryService` having a reference to another
   `DirectoryService` acting as the local side.
   I want to wait for general store composition code to pop up before
   manually coding this here.

2) It's currently only possible to put() leaf directory nodes, as the
   request normally requires uploading a whole closure. We might want
   to add another batch function to upload a whole closure, and/or do
   this batching in certain cases. This still needs some more thinking.

Change-Id: I7ffec791610b72c0960cf5307cefbb12ec946dc9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8336
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 08:48:41 +00:00
Florian Klink
b919f29752 feat(tvix/store/pathinfosvc): add gRPC client
Change-Id: Ie8e205c691bd11db99fcf097357c1e49161c6e19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8349
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-27 08:48:41 +00:00
Florian Klink
93f3964cbc docs(tvix/store/blobreader): fix docstring
Change-Id: Ia8ce1dd1663663903b9f60feada12e6a16fb4af2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8353
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-03-27 08:44:40 +00:00
Florian Klink
bde10682a5 docs(tvix/store): fix typo in comment
We query the blob service for detailled blob info, not the chunk
service.

Change-Id: I85a6a57b1dae74a950f734be7d4455c5c35ae355
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8348
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 08:44:39 +00:00
sterni
80e168b22d feat(sterni/mblog): implement environment based config mechanism
Change-Id: I091c0d5decc0a1eb3d24e81b713434ab391c677d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8347
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-03-24 18:32:50 +00:00
Vincent Ambo
fee0c83915 fix(tazjin/blog): fix moved link in a blog post
Change-Id: Ia0fb7638f94cfa7aaeac3044939b9efa20ef3507
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8337
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-03-24 09:06:24 +00:00