Commit graph

18543 commits

Author SHA1 Message Date
sterni
d5b989ddc0 test(tvix/eval): re-enable blackhole test
Change-Id: Id933f3bd708aa3342b9fd6a5584e65ee11751ff8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8773
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: sterni <sternenseemann@systemli.org>
2023-06-14 15:59:45 +00:00
sterni
8e5551329a fix(tvix/eval): don't thunk home relative paths
C++ Nix resolves home relative paths at [parse] time. This is not an
option for us, since it prevents being able to separate the compilation
and execution phase later (e.g. precompiled nix expressions). However, a
practical consequence of this is that paths expressions are always
literals (strict) and never thunks.

[parse]: 7066d21a0d/src/libexpr/parser.y (L518-L527)

Change-Id: Ie4b9dc68f62c86d6c7fd5f1c9460c850d97ed1ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7041
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-14 10:57:28 +00:00
sterni
0128323c55 docs(tvix): mention langVersion >= 5 check in nixpkgs
Change-Id: I7aae900012e901fa4d886b1a4bf0657d593e4983
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8771
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-14 10:55:55 +00:00
Vincent Ambo
3d3890ebcd chore(tazjin/yddns): update yandex-cloud version
Change-Id: Iaf246014601bf7ab3fb1af4961c2fe26f66e45be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8770
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-14 10:06:37 +00:00
Vincent Ambo
b3c9619f8c feat(corp/ops): configure DNS records for email handling
Change-Id: I19e41e559d475af78744c4623691578e8ead4327
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8769
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-14 10:06:37 +00:00
Vincent Ambo
5753e2393a fix(tazjin/nixos): fix 32-bit support for games
Change-Id: I04cb7c448228771536f9f87155183bdaa8faf4e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8768
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-14 10:06:37 +00:00
Vincent Ambo
d5748475d8 fix(ops/yandex-cloud-rs): fix dev-dependencies for examples
Change-Id: Ib99755d2b49464a6a30442b696ecfeda03038066
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8767
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-14 10:06:37 +00:00
Vincent Ambo
ded30c1ba9 docs(ops/yandex-cloud-rs): link to folder with usage examples
Change-Id: If2596b5a3dc542dca9a06a51a5a0f509034665c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8766
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-14 10:06:37 +00:00
Vincent Ambo
1bdb3e5ba8 chore(ops/yandex-cloud-rs): bump API definitions to 2023-06-13
Change-Id: Iad2d85eaffe96de0cf9ecb490fe5ba87209e1005
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8765
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-14 10:06:37 +00:00
Vincent Ambo
2e893dca1d refactor(ops/yandex-cloud-rs): allow TokenProvider impls to fail
It's actually quite common that a token provider might fail, for
example when fetching a token from instance metadata.

Change-Id: Ie0126fb92c6c613ad36b5583fd68505fdd97f2c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8764
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-14 10:06:37 +00:00
Vincent Ambo
ed388f019a chore(ops/yandex-cloud-rs): re-export some tonic types
These are useful for downstream users of the library, who might not
need all the rest of the tonic stuff.

Change-Id: Iab4d941696ae3c7a33b25815b72f92598aa82b80
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8763
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-14 10:06:37 +00:00
Vincent Ambo
6f912f5ecf fix(ops/yandex-cloud-rs): add Bearer prefix to auth token
Change-Id: I27d23de0598e3ca926a85cba3022f2dfff25f6be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8762
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-14 10:06:37 +00:00
Vincent Ambo
597d1a0aa2 feat(tazjin/yddns): hack together a quick dyndns update script
Change-Id: I4114a1b83e5f1edbb04d0ab920f107245ec998bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8761
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-14 10:06:37 +00:00
Vincent Ambo
8d632af36e fix(views/tvix): add fuse dependencies to external shell view
Change-Id: Ie7b8c5b983e5b41bffe0748d0047ffcfd82072e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8759
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-13 12:50:45 +00:00
Vincent Ambo
b16a2f302e chore(tazjin/emacs): disable winner-mode
I don't actually use this, and it might be responsible for some EXWM
bugs in the latest version.

Change-Id: I8817e0cdd9d1b68ed6a9bdd072616f63f9f20d21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8758
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-13 10:13:51 +00:00
Vincent Ambo
3e15d7b82a fix(tazjin/blog): link to reddit via internet archive
reddit is dying now, but the response there was actually quite
interesting, so I'm keeping an archive link to it around.

Change-Id: I419276428ce141fc526124d9746e3ab13df4bbc8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8756
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-13 10:03:02 +00:00
Florian Klink
c947a73b8e chore(tvix/store/pathinfosvc): remove GRPCPathInfoService::new
There's very little reason to instantiate a GRPCPathInfoService in a
context where we are not already in a tokio context.

Change-Id: Ib81d649387717cb98de8a8039f92472f727b10c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8755
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-12 21:33:59 +00:00
Florian Klink
011fbe2214 chore(tvix/store/directorysvc): remove GRPCDirectoryService::new
The only place where we did use new was also already where we've been in
a tokio context, so just using from_client is easier.

Change-Id: I39dbc18f6aaa3abc342409be623395647f968530
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8754
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-12 21:33:59 +00:00
Florian Klink
37e7742ba9 chore(tvix/store/blobsvc): remove GRPCBlobService::new
There's very little reason to instantiate a GRPCBlobService in a context
where we are not already in a tokio context.

Change-Id: Ic6e18809a9f2a76f1c098ed330118d8dcfba5137
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8753
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-06-12 21:33:59 +00:00
Florian Klink
b5e37869e6 refactor(tvix/store/pathinfosvc): use Arc<dyn …>
This removes the use of generics, like previously done with Blob and
Directory services.

Change-Id: I7cc8bd1439b026c88e80c11e38aafc63c74e5e84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8751
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-12 13:27:05 +00:00
Florian Klink
64a4f6185c refactor(tvix/store/blobsvc): drop Result<_,_> around open_write
We never returned Err here anyways, and we can still return an error
during the first (or subsequent) write(s).

Change-Id: I4b4cd3d35f6ea008e9ffe2f7b71bfc9187309e2f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8750
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-12 12:43:34 +00:00
Florian Klink
b49f7cfec6 docs(tvix/store/blobservice): fix docstrings
These are outdated.

Change-Id: I1a1ae130a55847f57a48d5e244e7e029c1ecae7b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8749
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-12 12:43:34 +00:00
Ryan Lahfa
5de04dbc12 feat(tvix/store): increase blob chunk size
From 64 bytes to 100 KBytes.
We need to provide a custom wrapper with a different Default instance.

Change-Id: Id7c6c437b8183b355a9e388f98cef1622b363f64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8748
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-06-12 12:36:56 +00:00
Florian Klink
09c5ca0a0d refactor(tvix/store/fuse): use Arc<dyn …> instead of generics
Change-Id: I5685379bd6f89d17da6843d31bef4c1fc4dc0a18
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8745
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-12 10:28:59 +00:00
Florian Klink
38a7caaada refactor(tvix/store/blobsvc/sled): cargo clippy
Change-Id: I00b8b567509d5e0847270f36dadb5dcb534b9b73
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8739
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-06-12 10:28:59 +00:00
Florian Klink
aa7bdc1199 refactor(tvix/store): use Arc instead of Box
This allows us to blob services without closing them before putting them
in a box.
We currently need to use Arc<_>, not Rc<_>, because the GRPC wrappers
require Sync.

Change-Id: I679c5f06b62304f5b0456cfefe25a0a881de7c84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8738
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-06-12 10:24:24 +00:00
Florian Klink
7725eb53ad refactor(tvix/store): use Box<dyn DirectoryService>
Once we support configuring services at runtime, we don't know what
DirectoryService we're using at compile time.

This also means, we can't explicitly use the is_closed method from
GRPCPutter, without making it part of the DirectoryPutter itself.

Change-Id: Icd2a1ec4fc5649a6cd15c9cc7db4c2b473630431
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8727
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-12 10:24:24 +00:00
Florian Klink
6f85dbfc06 feat(tvix/store/pathinfosvc): add calculate_nar method
Putting this in the PathInfoService trait makes much more sense, we can
have direct control over where/how to cache the results in the
implementation.

This now requires each PathInfoService to hold pointers to BlobService
and DirectoryService.

Change-Id: I4faae780d43eae4beeb57bd5e190e6d1a5d3314e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8724
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-12 10:15:20 +00:00
Florian Klink
8d05c0ceaa refactor(tvix/src/nar): drop NARCalculationService
There's only one way to calculate NAR files, by walking through them.

Things like caching such replies should be done closer to where we use
these, composing NARCalculationService doesn't actually give us much.

Instead, expose two functions, `nar::calculate_size_and_sha256` and
`nar::writer_nar`, the latter writing NAR to a writer, the former using
write_nar to only keeping the NAR size and digest.

Change-Id: Ie5d2cfea35470fdbb5cbf9da1136b0cdf0250266
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8723
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-06-12 10:15:20 +00:00
Florian Klink
27ff98000b feat(tvix/store): eliminate generics in BlobStore
To construct various stores at runtime, we need to eliminate associated
types from the BlobService trait, and return Box<dyn …> instead of
specific types.

This also means we can't consume self in the close() method, so
everything we write to is put in an Option<>, and during the first close
we take from there.

Change-Id: Ia523b6ab2f2a5276f51cb5d17e81a5925bce69b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8647
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-12 10:15:20 +00:00
sterni
5139cc45c2 test(tvix/eval): builtins.substring's behavior with negative args
Change-Id: Ie8b97d174e9d58e33bf08c9b9e0afeeddd089ba8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8700
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-06-12 09:38:08 +00:00
Linus Heckemann
5733274876 fix(tvix/eval): allow negative substring lengths
Nix uses string::substr without checking the sign of the length[1].
The NixOS testing infrastructure relies on this[2], and on the
implicit conversion of that to the maximum possible value for a
size_t.

[1]: ecae62020b/src/libexpr/primops.cc (L3597)
[2]: c7c2984716/nixos/lib/testing/driver.nix (L29)

Change-Id: I6d0caf6830b6bda3fdf44c40c81de6a1befeca7b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8746
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-06-12 08:38:49 +00:00
Linus Heckemann
2b4ad47c16 fix(tvix/eval): emit only warnings on shadowed outputs
Unfortunately, nixpkgs has at least one case[1] where the out environment
variable is shadowed -- though it doesn't cause a problem, since it's
shadowed with the correct value, odd as this may be!

[1]: c7c2984716/pkgs/development/python-modules/pybind11/default.nix (L19)

Change-Id: Ibf6790d2484dc9cce8e424feeb5886664d498dc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8696
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-11 20:59:34 +00:00
Vincent Ambo
7604833d2a fix(views/kit): provide lib argument to lazy-deps
Change-Id: I6687bda77a3160b922156de9045d5f8565be0a61
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8744
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
2023-06-11 17:05:56 +00:00
Vincent Ambo
ba36a15b31 feat(corp/rih): implement backend captcha validation
Change-Id: Ia80a6aeb8c20bdacbf93356be31592ca4ba7fcdc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8741
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-11 16:59:15 +00:00
Florian Klink
b3ca1a78eb feat(tvix/store): add mount command to entrypoint and fuse mod
`tvix-store mount PATH` will mount the tvix-store to the given path.

Change-Id: Icb82a6b3cb8a22eec856c375a28ae5580403833f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8665
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-06-10 15:33:07 +00:00
Florian Klink
365937cd08 feat(tvix/nix-compat): derive Hash
This allows using a StorePath as a key in a hashmap.

Change-Id: Id3eed623da4e1fc44a970a3982c7caa21d2495c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8666
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-10 14:54:36 +00:00
Florian Klink
83c1546b9a chore(tvix/store): add fuse feature
This brings in fuse (via the `fuser` crate), and adds pkg-config and
libfuse to the dev shell, so `cargo build` can link against it.

Change-Id: I0d11607490e27d946bdf92b0b9e45f9ab644ba74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8664
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-10 14:48:48 +00:00
DavHau
723186c5ad docs(REVIEWS.md): improve gerrit workflow explanation
Change-Id: I03c156c72831931f5a1984b6ff80bdc239f75e8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8726
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-10 11:57:17 +00:00
Vincent Ambo
9e61373096 feat(corp/ops): provide captcha verification key to backend
Change-Id: I960aa604488a27945ae6e69e6f3212a8dfd356cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8740
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-10 11:23:04 +00:00
Vincent Ambo
e3778ff6bc fix(corp/ops): let service account use encryption key
Change-Id: Idd68e849457ecf600b1d9a318846557adfce8575
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8737
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-10 11:23:04 +00:00
Vincent Ambo
75ffea3fe6 feat(corp/rih/backend): sprinkle some logging all over the place
Change-Id: Ifd55a0bf75070b1d47c2d65c32960f05ad7040a0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8736
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-10 11:23:04 +00:00
Vincent Ambo
f72d1f459d fix(corp/ops): add CORS configuration for backend
Change-Id: Ia953523eff463bb98189735bf4cfad6390b8eb71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8735
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-10 11:23:04 +00:00
Vincent Ambo
fd98f0644f fix(corp/rih): correct structure of frontend request
Gotta use a shared crate for this ..

Change-Id: I57669bdcace9676ff1311f3171c015702c934f56
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8734
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-10 11:23:04 +00:00
Vincent Ambo
a970de3365 feat(corp/rih): add UUIDs to records from frontend
This UUID stays the same even if a user submits the form multiple
times (unless they edit it manually in local storage, of course).

Change-Id: I4190fbfeb1027ce8a8d87bc283099539e8722b39
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8733
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-10 11:23:04 +00:00
Florian Klink
d925ec34cd docs(src/tests/nar_renderer): fix comment
This testcase tests a missing blob fails the rendering, the comment has
been copied from elsewhere.

Change-Id: I48fa3fa454e12506590fa14a3591d156bafa8b5e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8722
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-06-09 15:33:14 +00:00
Florian Klink
485fd26334 docs(tvix/store): drop nar-bridge references
The CLs did bitrot a bit, they're based on an older version of the
protocol, and it's unclear if they'll be a separate Go Binary, or just
another HTTP handler inside tvix-store itself, considering we now have
way more NAR juggling code than before.

Change-Id: I3632035cda8d75a8ff23b3132312f0f086d9e02f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8732
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-09 15:33:14 +00:00
Florian Klink
7f5372327c docs(tvix/store): update invocation in README
This has moved to a `daemon` subcommand.

Change-Id: Iae9778d8a59e6bf84555119fabfd62db3917bb62
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8731
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-09 15:33:14 +00:00
Vincent Ambo
0ddd4ecd9d feat(corp/rih): persist data in storage bucket
Change-Id: Ide7a21b97e845681584378d348cc36aba951d69f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8730
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-09 12:21:59 +00:00
Vincent Ambo
aea8c79ca3 chore(corp/ops): pipe secrets through to backend container
Change-Id: Idcaa4a7213b53fe1e818c6a81754d29b6249e957
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8729
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-06-09 12:21:59 +00:00