Commit graph

18310 commits

Author SHA1 Message Date
Florian Klink
5774117d5e feat(tvix/store): implement TvixStoreIO
This providesEvalIO, asking given PathInfoService, DirectoryService
and BlobService.

Change-Id: I32f210f5a7aa8173ad9a7d53e8a5ac03619f527a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8561
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-24 09:25:02 +00:00
Florian Klink
4b00f1d7ac feat(tvix/cli): check nixpkgs eval for pkgs.hello.outPath
We already evaluate this the same as Nix does, so worth adding it into
CI.

Change-Id: I529faccac6e5e16e0bc985ab4c3e0cd07bb23293
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8624
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-05-24 09:17:42 +00:00
Florian Klink
1fb5008237 feat(tvix/cli): introduce mkNixpkgsEvalCheck, add outPath check
This introduces a function that can be instantiated with an attribute
path to instantiate, as well as the expected path (normally interpolated
with the nix evaluator).

Check both pkgs.stdenv.drvPath and pkgs.stdenv.outPath to match.

Change-Id: Id667ed35fa159ff83fedb3017ef8d3271aa42695
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8606
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-05-24 09:17:41 +00:00
Florian Klink
7bd9df0133 feat(tvix/store/pathinfosvc): GRPCPathInfoService::from_client
Change-Id: I4a4f6f713dd76bf95f393c094c0eb9b6d15c5436
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8613
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-05-24 09:01:37 +00:00
Florian Klink
8d392221c6 feat(tvix/store/directorysvc): GRPCDirectoryService::from_client
Change-Id: Ifa274a380683b01f0f24cd9ff1f50d22f13b38bd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8612
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-05-24 09:01:36 +00:00
Florian Klink
f4a7b325e9 feat(tvix/store/blobsvc): add constructors
Change-Id: I55e06bf4e8a11dc2caf92c597558f1b820b42566
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8610
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-05-24 09:01:36 +00:00
Florian Klink
2d214eacc5 fix(tvix/store/directorysvc): stop renaming spans
We have code.namespace as a field, set to
tvix_store::directoryservice::sled, so there's no need to repeat the
name here.

Change-Id: Ic1aa8a2b24de439c6a189966bd773e9acf49d1e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8623
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-24 09:01:35 +00:00
Florian Klink
457fd4c325 feat(tvix/store/nar): add GRPCNARCalculationService
This asks a remote tvix-store for the nar size and digest of a given
root node.

Change-Id: If9f916d9bfc5f8dc3166e2c6c1671c0f0124d1c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8611
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-24 09:01:35 +00:00
Gabor Greif
01e8fa6529 chore(tvix/website): fix spelling mistake
Change-Id: Id1b2287bebef3384676f89817030c0a00f8d5935
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8622
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
2023-05-23 12:43:43 +00:00
sterni
8f199c65df docs(3p/lisp/mime4cl): describe changes compared to original version
Spell out that “may diverge” is more of a “has diverged by now”. We are
essentially maintaining a fork of mime4cl.

Change-Id: I9049e8296a666c3d1b08eae28813147f360771ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8621
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-23 12:43:43 +00:00
Florian Klink
79bbf25cb2 refactor(tvix/store/nar): clippy
Change-Id: I0d864fd1b9248fe4d5c88e3d8fa396ea759fea09
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8616
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-23 12:28:34 +00:00
Florian Klink
4dd954f8f5 refactor(tvix/store/directorysvc): clippy
Change-Id: I30f7e98fe79ba22d218b4aabaef88c84b6085d82
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8617
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-23 12:28:34 +00:00
Florian Klink
e51d85e078 refactor(tvix/store/digests): clippy
useless conversion to the same type: `std::vec::Vec<u8>`

Change-Id: Idcb16679bb2a3350784965d8d9bbb593c760634e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8615
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-23 12:28:34 +00:00
Florian Klink
066179651c refactor(tvix/store/blobsvc): move from Vec<u8> to B3Digest
Change-Id: I809bab75221f81b6023cfe75c2fe9e589c1e9192
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8605
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-23 10:48:55 +00:00
Florian Klink
b8ff08b1b0 refactor(tvix/store/directorysvc): move from Vec<u8> to B3Digest
This introduces a new struct, B3Digest, which internally holds a
Vec<u8>, but only allows construction with 32 bytes.

It also implements display, which will print the base64 representation.
This should reduce some boilerplate when parsing Vec<u8>.

Change-Id: Ia91aa40cb691916773abc8f93e6ed79a5fd34863
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8592
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-05-23 10:48:55 +00:00
Vincent Ambo
e779b866cc chore(corp/ops): yc-cli: 0.104 -> 0.106
Change-Id: If783a7a4315ecab70f20347a66fb72f682dbd97c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8609
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-23 10:17:24 +00:00
Vincent Ambo
6d9ebd7b7c chore(ops/yandex-cloud-rs): bump API specs to 2023-05-23
Change-Id: Ibc98d3878690099d6d95dfe1a2741d551ed7a72a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8608
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-23 10:17:24 +00:00
Vincent Ambo
b8cbf3456b chore(corp): update copyright years
Change-Id: I59a93c5c41562f5c7c65c9fd6a4d4409df904beb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8607
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
2023-05-23 10:17:24 +00:00
Florian Klink
11771a06ae refactor(tvix/eval): use &Path instead of PathBuf
This allows getting rid of some clones in eval/src/vm/generators.rs.

Change-Id: I330390307d3bcfeef19c98954c753ee55b1ccee3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8604
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-22 09:43:33 +00:00
Florian Klink
b4bb9062ea fix(tvix/eval): add path where useful to ErrorKind::IO
These two places didn't add the path from the context to the
ErrorKind, but simply relied on the
impl From<std::io::Error> for tvix_eval::ErrorKind, which doesn't add
the path.

Change-Id: Ifc7dbbe305d24242b0705de1dea34e8923e9d2cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8603
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2023-05-22 09:43:21 +00:00
Florian Klink
b7ab6c0856 refactor(tvix/eval/io): use io::Error instead of tvix_eval errors
We didn't return anything useful other than ErrorKind::IO anyways.

We can use io::ErrorKind::Unsupported for DummyIO.

Fixes b/271.

Change-Id: Icb231e9b38168e8b6fa473bfa405d160357b317f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8602
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-22 09:42:58 +00:00
Vincent Ambo
03958a5446 chore(ops/yandex-cloud-rs): explicitly set include in manifest
This makes publishing a bit easier without the build script
interfering and other wonkiness.

Change-Id: Iadb144aabbdeabae8899ebdc62636315239e5f08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8601
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2023-05-19 09:42:47 +00:00
Vincent Ambo
fd92ea9be3 fix(ops/yandex-cloud-rs): set license in Cargo manifest
Change-Id: Icc15953557585cbb2708a1267ab509caca8b258e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8600
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-19 09:29:44 +00:00
Vincent Ambo
1076b509ae chore(ops/yandex-cloud-rs): bump API definitions (2023-05-19)
Change-Id: I0c4e77587b9fac14017449eb6a4630265b07950e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8599
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-19 09:23:08 +00:00
Vincent Ambo
e9efa579a7 docs(ops/yandex-cloud-rs): add developer-facing README
Mostly to remind myself about the wonky release process.

Change-Id: I76ea8d9a2ed600ebb31f4b1a5368f3cefa0556d6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8598
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-19 09:23:08 +00:00
Florian Klink
e31008db14 fix(tvix/store): fix timing sensitivity in gRPC directorysvc test
One test in here assumed the server was fast, but when very busy, a
computer running these tests might not be fast.

Treat both cases that can occur separately.

Change-Id: Iba168ad39f2458c4fb8873211df33beeaff7c6c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8595
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-18 19:38:54 +00:00
Florian Klink
103d2eb909 refactor(tvix/store): bump CalculateNARResponse.nar_size to u64
Change-Id: I5f41981820363a5eb9982c3d19830916207c62cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8591
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-18 19:29:53 +00:00
Florian Klink
0e7bbf0d01 fix(tvix/store/proto/grpc_blobsvc): clippy
Change-Id: I0473562793f5fa2211cb10329ba1bca14b1acab9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8594
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-18 19:29:53 +00:00
Florian Klink
a0801ae220 refactor(tvix/store/blobsvc/sled): clippy
Change-Id: Icfea8ceb6b827a645c6f10efae741f6fa2114fad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8590
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-18 19:29:53 +00:00
Florian Klink
2e09e94aad refactor(tvix/store/directorysvc/traverse): clippy, use NamedNode
Also, get rid of the explicit byte comparison here, but unwrap like in
the rest of the codebase. We'll deal with this in a generic manner in
b/267 and b/189.

Change-Id: Ie5f3d27ab35b7e88d67a2796c29cdd7bc7df71f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8589
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-18 19:29:53 +00:00
Florian Klink
dcbcac8955 refactor(tvix/nix-compat): clippy
Change-Id: Ie5277a5c15d9dfe543ca41fa7c6a1eedf22a9f64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8593
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-18 19:29:53 +00:00
Vincent Ambo
64ee21ec86 chore(tvix/cli): remove unnecessary RefCell
This remained around from some previous time where this type was being
cloned around directly, but it's not anymore.

Change-Id: I2c61cf9cecb8e5d7d08644ed20642ee61357bc21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8580
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-05-18 19:12:11 +00:00
Vincent Ambo
ba4807e1de refactor(tvix/cli): split CLI-specific IO logic into TvixIO type
This adds a wrapper type TvixIO<T: EvalIO>, which can wrap around an
arbitrary EvalIO implementation and perform actions needed for the
Tvix CLI (marking imported paths as known, and handling __corepkgs__).

Change-Id: I5fc1ca199b9f94b21a89103b84575e0f8f58dff9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8579
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2023-05-18 19:12:11 +00:00
sterni
7f99eb44a5 test(3p/lisp/mime4cl): test decoding RFC2047 examples
Change-Id: I32abb00e8cec697adb45b9a175cd753e807d33d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8588
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-18 16:18:44 +00:00
sterni
8fa67a0d6f refactor(3p/lisp/mime4cl): remove unused DECODE-STREAM
This function has apparently been unused ever since we imported mime4cl
into depot.

Change-Id: I224c9b2947ffd641e01e8cd5454d7a7fa75278d4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8587
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
2023-05-18 16:18:43 +00:00
sterni
b388354c4d refactor(3p/lisp/mime4cl): port remaining base64 decoding to qbase64
DECODE-BASE64-STREAM-TO-SEQUENCE is the only thing that requires
anything fancy: We read into an adjustable array. Alternative could be
using REDIRECT-STREAM and WITH-OUTPUT-TO-STRING, but that is likely
slower (untested).

Test cases are kept for now to confirm that qbase64 is conforming to our
expectations, but can probably dropped in favor of a few more sample
messages in the test suite.

:START and :END are sadly no longer supported and need to be replaced by
SUBSEQ.

Change-Id: I5928aed7551b0dea32ee09518ea6f604b40c2863
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8586
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
2023-05-18 16:18:43 +00:00
sterni
02684f3ac6 refactor(3p/lisp/mime4cl): remove be and be*
Seems simple enough to use standard LET and a few parentheses more which
stock emacs can indent probably.

Change-Id: I0137a532186194f62f3a36f9bf05630af1afcdae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8584
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-18 16:18:42 +00:00
sterni
a06e30e73b refactor(sterni/mblog): move REDIRECT-STREAM into mime4cl
Eventually, we'll want to replace dump-stream-binary with something more
efficient—given that we have flexi-streams we can use something that
only does matching element types no problem. REDIRECT-STREAM is much
more efficient thanks to using an internal buffer.

streams.lisp gets a new section at the beginning for grouping utilities
that don't have any real (internal) dependencies.

Change-Id: I141cd36440d532131f389be2768fdaa54e7c7218
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8583
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-18 16:16:39 +00:00
sterni
734cec2e3b refactor(3p/lisp/mime4cl): use qbase64 for decoding FILE-PORTIONs
Porting over the rest of the decoding (RFC2047) and especially encoding
over to qbase64 is still pending, as it is a little trickier.

Change-Id: Id4740eb074a387aeea2cb94b781e204248530799
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8582
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-18 16:16:39 +00:00
sterni
3d2e55ad53 refactor(mime4cl): replace *-input-adapter-stream with flexi-streams
The input adapter streams were input streams yielding either binary or
character data that could be constructed from a variable data source.
The stream would take care not to destroy the underlying data
source (i.e. not close it if it was a stream), so similar to with
FILE-PORTIONs, but simpler.

Unfortunately, the implementation was quite inefficient: They are
ultimately defined in terms of a function that retrieves the next
character in the source. This only allows for an implementation of
READ-CHAR (and READ-BYTE). Thanks to cl/8559, READ-SEQUENCE can be used
on e.g. FILE-PORTION, but this was still negated by a input adapter
based on one—then, READ-SEQUENCE would need to fall back on READ-CHAR or
READ-BYTE again.

Luckily, we can replace BINARY-INPUT-ADAPTER-STREAM and
CHARACTER-INPUT-ADAPTER-STREAM with a much simpler abstraction: Instead
of extra stream classes, we have a function, MAKE-INPUT-ADAPTER, which
returns an appropriate instance of FLEXI-STREAM based on a given source.
This way, the need for a distinction between binary and character input
adapter is eliminated, since FLEXI-STREAMS supports both binary and
character reads (external format is not yet handled, though).
Consequently, the :binary keyword argument to MIME-BODY-STREAM can be
dropped.

flexi-streams provides stream classes for everything except a stream
that doesn't close the underlying one. Since we have already implemented
this in POSITIONED-FLEXI-INPUT-STREAM, we can split this functionality
into a new superclass ADAPTER-FLEXI-INPUT-STREAM.

This change also allows addressing the performance regression
encountered in cl/8559: It seems that flexi-streams performs worse when
we are reading byte by byte or char by char. (After this change mblog is
still two times slower than on r/6150.) By eliminating the adapter
streams, we can start utilizing READ-SEQUENCE via decoding code that
supports it (i.e. qbase64) and bring performance on par with r/6150
again. Surely there are also ways to gain back even more performance
which has to be determined using profiling. Buffering more aggressively
seems like a sure bet, though.

Switching to flexi-streams still seems like a no-brainer, as it allows
us to drop a lot of code that was quite hacky (e.g. DELIMITED-INPUT-
STREAM) and implements en/decoding handling we did not support before,
but would need for improved correctness.

Change-Id: Ie2d1f4e42b47512a5660a1ccc0deeec2bff9788d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8581
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-18 16:14:38 +00:00
sterni
b379e44dfb refactor(3p/lisp/mime4cl): use flexi-streams and binary input
This refactor is driven by the following (ultimate) aims:

- Get rid of as much of the custom stream code in mime4cl which makes
  less code to maintain in the future.

- Lay the groundwork for correct handling of 8bit transfer encoding:
  The mime4cl we inherited assumes that any MIME message can be decoded
  completely by the CL implementation (in SBCL's case using latin1)
  into CHARACTERs. This is not necessarily the case. flexi-streams
  allows changing how the stream is decoded on the fly and also has
  support for reading the underlying bytes which is perfect for the
  requirements decoding MIME has.

- Since flexi-streams uses trivial-gray-streams, it supports
  READ-SEQUENCE. Taking advantage of this may improve decoding
  performance significantly in the future.

This incurs the following changes:

- Naturally we now open given files as binary files in MIME-MESSAGE.
  Given strings are encoded using STRING-TO-OCTETS and then passed on
  to a new octet vector method. Instead of MY-STRING-INPUT-STREAM this
  now uses flexi-streams' WITH-INPUT-FROM-SEQUENCE.

- OPEN-FILE-PORTION and OPEN-DECODED-FILE-PORTION need to be merged,
  since the transfer encoding not only implies an extra decoder stream
  that needs to be attached after file portion stream, but also imply a
  certain encoding of the stream itself (mostly binary vs. ASCII).
  As flexi-streams can change their encoding on the fly this could be
  untangled again, but it is not strictly necessary.

  As before, we use the DATA slot of the file portion to create a fresh
  stream if possible. Instead of strings we now use an vector of octets
  to match MIME-MESSAGE.

  The actual portioned stream relies on POSITIONED-FLEXI-INPUT-STREAM, a
  subclass of the stock FLEXI-INPUT-STREAM class, described below.

- POSITIONED-FLEXI-INPUT-STREAM replaces DELIMITED-INPUT-STREAM. It is
  created using MAKE-POSITIONED-FLEXI-INPUT-STREAM which accepts the
  same arguments as MAKE-FLEXI-STREAMS and, additionally, :IGNORE-CLOSE.
  A POSITIONED-FLEXI-INPUT-STREAM works the same as an
  FLEXI-INPUT-STREAM, but upon creation, the underlying stream is
  rewinded or forwarded to the argument given by :POSITION using
  FILE-POSITION.

  If :IGNORE-CLOSE is T, a call to CLOSE is not forwarded to the
  underlying stream.

Change-Id: I2d48c769bb110ca0b7cf52441bd63c1e1c2ccd04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8559
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-18 16:14:37 +00:00
Florian Klink
a1324513ad feat(tvix/store/directorysvc): add traverse_to
This walks from a node further down until it reaches the requested path.

Change-Id: I2f9a15a8601db4d06c95d7b47cd6153264e203e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8568
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-05-18 06:26:55 +00:00
Florian Klink
3a4e29c261 refactor(tvix/store): rename import::{import_path -> ingest_path}
This distinguishes it better from the EvalIO::import_path method.
Also update the docstring to explain what it does (and what it doesn't).

Change-Id: I32a8b2869fa67a894df28532b22bf170961a2abf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8578
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2023-05-17 13:06:53 +00:00
Florian Klink
a6c7869393 fix(tvix/store/src/proto/mod.rs): fix error string
This was missed while renaming NixPath to StorePath.

Change-Id: Ibcc929c43b111e4370e8222c1dd86d403548367f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8577
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-05-17 13:06:52 +00:00
sterni
a4d740af2e refactor(3p/lisp/mime4cl/test): create one test case per sample file
Since rt.lisp seems to start tests in parallel, the informational output
about which sample file is being tested gets mangled in all sorts of
ways. The solution is to just loop over the sample files outside a test
and schedule a single test case per sample file from there.

Change-Id: I4494e4a526ce6d92a298cf7daf06c8013c7ca605
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8569
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-16 16:25:11 +00:00
Florian Klink
e815b680c0 refactor(tvix/store/pathinfosvc): drop ByWhat, use digest directly
We currently only support querying by the output hash digest.
This makes the interface a bit simpler.

Change-Id: I80b285373f1923e85cb0e404c4b15d51a7f259ef
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8570
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-16 10:39:06 +00:00
Florian Klink
71c29d0f4c feat(tvix/nix-compat): add StorePath::from_absolute_path_full
This allows decomposing a path consisting of a store path AND a suffix
into a StorePath, and a PathBuf containing the rest.

Change-Id: I81290e2fd804cdc9d1e88c71cb22c0fb882d7936
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8567
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-16 10:01:25 +00:00
Florian Klink
1a2190cd3b docs(tvix/nix-compat): update docstrings
Make it cleaner that StorePath only does encode the first path component
after the STORE_DIR prefix. Also, move some of the comments around a
bit, so it makes more sense what's using what.

Change-Id: Ibb57373a13526e30c58ad561ca50e1336b091d94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8566
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2023-05-16 10:01:25 +00:00
Florian Klink
14a8ea9eab feat(ops/terraform/deploy-nixos): make target_user_ssh_key optional
In case `target_user_ssh_key` points to an empty string, nixos-copy.sh
just doesn't set `IdentityFile=` at all.

This allows using deploy-nixos without any explicitly passed ssh keys,
but picking up whatever ssh setup the user has configured locally.

Change-Id: If335ce8434627e61da13bf6923b9767085af08a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8576
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-16 09:55:23 +00:00
Griffin Smith
bb4d807974 feat(grfn/system): Install dropbox on all systems
Change-Id: I35aaf174c7193c6fa6610989d8334ceabcdb6ced
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8575
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
2023-05-15 14:12:23 +00:00