Commit graph

20466 commits

Author SHA1 Message Date
Profpatsch
54ea3eeacb fix(users/Profpatsch/whatcd-resolver): add page chrome to artist
Change-Id: Id13553c450603b04c6bba0085d7f3246fe0d54af
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11682
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
c5555f25da fix(users/Profpatsch/whatcd-resolver): speed up table query
This took a while to figure out, but essentially sorting a thing with
one million queries takes a long time, as compared to doing it on a
small subset of things and then joining against the final ordering.

The generated column helps, too.

Change-Id: I1bf283e2be060748eebda92576e3d062c51a6777
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11681
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
5da968070e chore(users/Profpatsch/openlab-tools): warnings & format
Change-Id: Iab582c4cd9dc623449940920d9ac4864b515b5c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11680
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
bd5c49c23a chore(users/Profpatsch/.hlint.yaml): ignore Use maybe
Change-Id: I302b4b66b0c3a3646fb84afd0140bf0ef92e541d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11679
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
87d806c20f fix(users/Profpatsch/whatcd-resolver): store seeding_weight
Generated columns!!

This reduces the query time for distinct on somewhat!

Change-Id: Ic004d327b719a9f44a224ec87709992ce3d8f160
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11676
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
b54ad3e580 feat(users/Profpatsch/whatcd-resolver): add artist albums
Simple artist album page that only shows albums the artist was
involved with.

Change-Id: Icff34afc6d1b39b6fb17765c1b3ea500dd4b4d95
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11675
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
a3a03a5a80 fix(users/Profpatsch/whatcd-resolver): index seeding_weight calc
The seeding weight would slow down the query quite a bit, so let’s
move it into a procedure and add an index onto the torrents table that
caches the result.

Baba’s first pl/SQL function!

Change-Id: I3bc6919b115c02b9c9aa74702fac0a8bbc66d2c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11674
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
2ac89bb480 fix(users/Profpatsch/whatcd-resolver): fix transmission session
The logic around transmission session handling was f*cked, this fixes
that.

We use an IORef instead of an MVar, since we want to unconditionally
write the new value. Even if multiple requests race, I *hope* that
transmission returns the same session id, otherwise we might get a
request loop. But it should be fine. (The semantics is not nicely
documented in the RPC docs.)

Additionally, log the session ids in the requests.

Change-Id: Id7d33f8cb74cb349e502331cad5eb5abe8a624cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11673
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2024-06-03 14:55:13 +00:00
Profpatsch
3b8b47baba feat(users/Profpatsch/whatcd-resolver): add status to http trace
Change-Id: Ic83a79c18129dd195e808d1c78758dbf0be8ff76
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11672
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
44d8bf80f5 fix(users/Profpatsch/whatcd-resolver): remove trace debugging
Accidentally left that in.

Change-Id: Id744ea0f2281e0e2a0fe3324efa3f4c13b6ce9fc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11671
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
c1598cd2d2 feat(users/Profpatsch/whatcd-resolver): display all artists
apparently the `torrent_group` json only contains one artist, while
the torrent itself contains a list of them.

This is important, because we need the artist id and the
`torrent_group` does not provide it, only the torrent one.

Change-Id: I3db45f454f14e89ea8c8dafba2065ecd55f5bcda
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11670
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
7197d28098 feat(users/Profpatsch/whatcd-resolver): add query args to log
Change-Id: I162a438bc9d072adf952295c93839b3050bfff40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11669
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
f13c5dddb5 chore(users/Profpatsch/shell.nix): comment out broken haskell pkg
Change-Id: I73286424ee07e22b156096bfb582b22daf05c819
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11661
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Profpatsch
5c709131de fix(users/Profpatsch/whatcd-resolver): show query error as html
We want the user thingy to see which error happened; it also gets
logged in the traces as before.

There’s another function which we should replace as well at one point.

Change-Id: I3d49edccd0e2088a45ac0138af9536b40dfa6848
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11660
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2024-06-03 14:55:13 +00:00
Aspen Smith
c2f649f62e feat(tvix/repl): Add a command to recursively print
Add a command, :p, to evaluate an expression and recursively print the
result, as if `--strict` had been passed on the command line.

Demonstration of this working:

    ❯ cargo r --bin tvix
        Finished dev [unoptimized + debuginfo] target(s) in 0.27s
        Running `target/debug/tvix`
    tvix-repl> { x = (x: x) 1; }
    => { x = <CODE>; } :: set
    tvix-repl> :p { x = (x: x) 1; }
    => { x = 1; } :: set

Change-Id: I1a81d7481160c30d2a4483c6308e25fa45f2dfdf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11738
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 09:04:59 +00:00
Aspen Smith
4a489d930c feat(tvix/repl): Add a help command
Add a command to display help for the REPL, which can be either :? or
:h.

Change-Id: Ifdfd8c31130ca5afcde05a4c4276b768eb54c06f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11737
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 09:04:59 +00:00
Aspen Smith
55ec06b26b feat(tvix/repl): Implement :q, to quit
Change-Id: Ib3b314b21f4d8c30bfd674e79771179d51d4f2e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11736
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-06-01 09:04:59 +00:00
Aspen Smith
70c4b512c2 refactor(tvix/repl): Abstract out REPL command handling
Prepare for introducing additional REPL commands by splitting out
the *parsing* of the repl commands, which returns a new ReplCommand
type, from the actual *handling* of the commands.

Change-Id: If81a53c1e2d90204d26ce3bb2ea9eebf7bb3fd51
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11734
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 09:04:59 +00:00
Aspen Smith
23589a1db3 refactor(tvix): Break out REPL into its own module
In preparation for adding some new functionality to the tvix REPL, break
it out into its own module.

Change-Id: I4fb78320e92562e3474a3724536cb22c1d893e57
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11733
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 09:04:59 +00:00
Aspen Smith
96ce9aea04 feat(tvix/repl): Support multiline input
Transparently support multiline input in the Tvix REPL, by handling the
UnexpectedEOF error returned by the parser and using it to progressively
build up an input expr over multiple iterations of the REPL's outer
loop.

This works quite nicely:

❯ cargo r --bin tvix
   Compiling tvix-cli v0.1.0 (/home/aspen/code/depot/tvix/cli)
     Finished dev [unoptimized + debuginfo] target(s) in 1.72s
     Running `target/debug/tvix`
tvix-repl> { foo
         >   =
         >   1;
         > }
=> { foo = 1; } :: set
<press up arrow>
tvix-repl> { foo
  =
  1;
}
=> { foo = 1; } :: set

Change-Id: Ib0ed4766b13e8231d696cdc27281ac158e20a777
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11732
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-06-01 09:04:59 +00:00
Matthew Tromp
bbced9222c feat(ops/users): Add chickadee
Change-Id: I7b06473f67ee630a02676b19ff42ef02dd4014ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11742
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
2024-05-31 18:03:22 +00:00
Ilan Joselevich
be624daf13 fix(tvix/eval): nix_tests.rs's eval_test requires impure flag
This fixes the following command:
'cargo test --no-default-features --features nix_tests'

Change-Id: I9883c39e1e428c72a0e7e0b75a73c8ed734abd3b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11740
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-05-30 21:52:16 +00:00
Florian Klink
5405ed9dc6 fix(tvix/eval): proptests require arbitrary feature
`cargo test --no-default-features` fails, if we don't conditionalize
this on the `arbitrary` feature too.

Change-Id: I81a277810119fed0cfc37c942c422f731aa14b2e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11726
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-30 16:18:30 +00:00
Florian Klink
baa39d9d09 fix(tvix/eval/tests/one_offs): test_source_builtin can be pure
`Evaluation::new_impure()` would require the test to be impure, but
there's nothing in this test specifically requiring us to make use of
impure features.

Change-Id: Idb24981195d1a94f51053ae04403eb5f0e27f3d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11725
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-30 16:18:30 +00:00
Florian Klink
c83f2e765c fix(tvix/eval/tests/nix_oracle): tests are impure-only
This uses StdIO, which is only available when the `impure` feature is
enabled.

Change-Id: I039b1f45f6619dd099fa943e58322ff521482dfa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11724
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-30 16:18:30 +00:00
Florian Klink
2c628f6e1c refactor(tvix/eval): move nix_tests to separate module
Gate this behind the nix_tests feature flag (which wasn't applied
consistently).

Also, at least right now all of these use StdIO internally, either by
creating it on their own, or through the `eval_test` helper.

Once we have some proper classification system for tests we can probably
split this up further, but for now only run them if the impure feature
is enabled.

Change-Id: I3236cf09b3391585df99073367c4e4832c5e7898
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11723
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-30 16:18:30 +00:00
Florian Klink
543c103903 fix(tvix/eval/vm/generators): allow unused rq_{path_exists,read_dir}
These VM requests are only emitted by code gated behind the impure feature.

To prevent warning from popping up when building without default
features, allow these to be unused if `impure` is not enabled.

Change-Id: Id871a5215e9a0f09aa78edecdd111369ee7ffe34
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11722
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2024-05-30 16:18:30 +00:00
Vincent Ambo
e8324073eb chore(3p/teleirc): use TVL fork temporarily
This contains a fix for an issue where the bridge would forward
messages from the wrong channel, because the source channel was not
checked.

tvlbot runs in 3 channels, so this caused issues.

Change-Id: Icc85406b273f375ac90287364df83fb76f028b59
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11731
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2024-05-27 12:47:08 +00:00
Florian Klink
60f189897e fix(fun/clbot): address sprintf warning
%q is only defined for maps containing booleans, not strings. It should
be OK to use %v here.

Change-Id: I7475eb89032653a588ecdf70809eb35bbe7e04ab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11730
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
2024-05-27 08:20:05 +00:00
Florian Klink
6fb542aae6 fix(tvix/eval/nix_search_path): gate tests on impure feature
These use StdIO, which is only available if the impure feature is
enabled.

Change-Id: I18d8e191a7eba6ba5bd59f43631973eaa796c7bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11721
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-26 19:53:51 +00:00
Florian Klink
db68c104af fix(tvix/eval/io): OsStringExt and std::fs::File import are impure-only
These are only needed when building with the impure feature enabled.

This removes some warnings when building with --no-default-features.

Change-Id: I3139d9133d4846aeb1b1b5f3830c0d078d047292
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11720
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-26 19:52:49 +00:00
Florian Klink
8089682bb5 fix(tvix/nix-compat): async nar reader requires wire feature
It uses it internally.

Change-Id: I8cb8fc9f567260d57f3a203407333d83beddf537
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11719
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-26 19:52:49 +00:00
Florian Klink
ca542440a5 fix(tvix/build): tonic-reflection feature needs castore with it too
If building tvix-build with the tonic-reflection feature, it needs
to import `tvix_castore::proto::FILE_DESCRIPTOR_SET`, which is only
available if tvix-castore is built with the `tonic-reflection` feature.

Change-Id: I355b4c5b4c1333d5cc56335de47ad5d2f1db6337
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11716
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
2024-05-26 19:46:47 +00:00
Florian Klink
9586e5c30d refactor(tvix/castore): move src/fs/test into fuse mod
These tests only interact with the FUSE layer, and
import super::fuse to do its work.

However, this only works if the `fuse` feature is enabled, which we
don't do if we enable the `virtiofs` feature only, causing the tests
to fail:

```
❯ cargo test --no-default-features --features virtiofs
   Compiling tvix-castore v0.1.0 (/home/flokli/dev/nixos/code.tvl.fyi-submit2/tvix/castore)
error[E0432]: unresolved import `super::fuse`
  --> castore/src/fs/tests.rs:14:13
   |
14 | use super::{fuse::FuseDaemon, TvixStoreFs};
   |             ^^^^ could not find `fuse` in `super`
```

We move src/fs/tests.rs to src/fs/fuse/tests.rs
(and src/fs/fuse.rs to src/fs/fuse/mod.rs) to better structure this,
which will automatically cause both tests and code to only be built if
we have the `fuse` feature enabled.

Change-Id: I8fbbad3e4457e326bdfd171aa5c43d25d3187b5b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11715
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-26 19:46:47 +00:00
Vincent Ambo
61cf4905fe feat(ops/modules): launch teleirc for Volga Sprint
For the duration of the sprint, this bot will take care of
synchronising the IRC channel with the Telegram group.

After the sprint, it will be removed again.

Change-Id: I6d5b1316fc85ddd26adf55e31f6bff742907fc24
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11727
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-05-26 19:36:29 +00:00
Vincent Ambo
73092db85a feat(3p/teleirc): add derivation for irc<>tg bridge
We need this for the Volga Sprint channel.

Change-Id: I77ff3172e3a8eca55de51e45cb824ec0f40d4065
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11718
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-05-26 19:36:29 +00:00
Ilan Joselevich
3ebcf30b64 feat(tvix): filter src via lib.fileset
Previously changing any file (including default.nix or README.md) would
cause Nix to compute a different derivation path, resulting in needing
to (very often redundantly) rebuild the crate/package. With this change
you can expect less rebuilds.

Crate2nix currently does its own insufficient src filtering and it does
not expose an API to override the filtering, so instead I created a
function that we can use to override the src to have stricter filtering.

I implemented the filtering for all of the workspace members, if you
want to modify any of them, please read the lib.fileset docs
https://nixos.org/manual/nixpkgs/unstable/#sec-functions-library-fileset

Change-Id: I7ab5a0064a76938d14f7f65801be9f3a5c3bad04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11714
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-05-26 14:27:50 +00:00
edef
edd93b1962 fix(tvix/eval): drop superfluous string context check
cl/11712 simultaneously introduced this check and made it unnecessary,
since NixString::context should never return `Some` for empty contexts
now.

Change-Id: I41a655ff33910e8326cbb7d7526eb91bd19e9585
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11713
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-05-25 11:33:22 +00:00
edef
49750fa1e7 fix(tvix/eval): disallow empty but allocated string contexts
Both `Some(NixContext::new())` and `None` represent empty contexts,
but the former trips up `NixString::has_context`, and seems likely
to trip up other things.

We could hide the difference in the accessors, but we don't really
*want* the distinction to exist, since heap-allocating a null value
is pretty much always a mistake.

Change-Id: Ie84d26fb0d4b59e68354891ba13bde3bae40ab6e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11712
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-05-25 08:42:32 +00:00
Daniel Mendler
3e6bfe36af
Bump version to 0.30
Same version as 0.29. Bump only required to trigger ELPA rebuild.
2024-05-24 09:19:18 +02:00
Daniel Mendler
3b28c051f5
Bump version to 0.29 2024-05-23 19:19:41 +02:00
Daniel Mendler
956e7eb175
Depend on xelb 0.19 2024-05-23 19:19:19 +02:00
Florian Klink
c4c42c8b6c docs(tvix/docs/TODO): add item for string context rework
Change-Id: I4592490a75fb05093b3fd8649db3b57bff748c0d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11707
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2024-05-23 15:58:22 +00:00
Florian Klink
6ad3780734 docs(tvix/TODO): toXML string context and self-closing tags is done
Change-Id: I83f6b1863a023ea0bf8518c67eb51f740c3ea89a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11706
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Alyssa Ross <hi@alyssa.is>
2024-05-23 14:58:41 +00:00
Florian Klink
649a862ae1 feat(tvix/eval): rm NixContext::join, add take_context & IntoIterator
In places where we want to extend context with that from another
NixString, use take_context() to split it off, then call .extend(),
making use of IntoIterator to avoid a bunch of clones.

Change-Id: I2460141a3ed776c64c36132b2203b6a1d710b922
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11705
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
2024-05-23 14:50:38 +00:00
Florian Klink
ec8d79f3db feat(tvix/eval): teach builtins.toXML context
XmlEmitter gains a NixContext field, and `write_typed_value` extends it
with all context elements present in the passed value.

Once all serialization is done, a into_context() function returns the
collected context, so we can construct a NixString with context.

Tests for this live in tvix-glue, as we use builtins.derivation, which
is not present in the tvix-eval crate.

Fixes b/398.

Change-Id: I85feaaa17b753885f8a017a54e419ec4e602af21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11704
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
2024-05-23 14:49:07 +00:00
Florian Klink
a4a313cdd2 feat(tvix/eval): add NixContext::extend
This is a slightly less annoying version of `join`, which does not
consume self. It's more consistent with HashSet::extend().

Change-Id: Ifd0872da36fe8e7b2aa6948674cb8e4023abe9d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11703
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2024-05-23 14:49:07 +00:00
Florian Klink
48e045299d fix(tvix/eval): NixString are bytes
This is not necessarily valid UTF-8.

Change-Id: I72f3157240772eb9c558e5699b4785e44d256fd4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11702
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2024-05-23 09:54:26 +00:00
Profpatsch
5b2ba0efa1 refactor(tvix/eval): rewrite xml emitter to be simple-stupid
In order to be compatible with the nix XML generator, it’s easier to
generate the XML directly, instead of going through a library which we
have to bend to do what we need.

Removes dependency on `xml-rs`, which came with a full XML parser that
we didn’t use. Only takes a tiny bit of code for the XML escaping,
somewhat simplified.

I add a little escaping value, to make sure we have the same behaviour
as nix proper.

Interestingly enough, we never need to escape XML attribute names,
because the `builtins.toXML` format encodes user-defined values as
attribute keys only. So we only escape attribute values.

Fixes: https://b.tvl.fyi/issues/399
Change-Id: If4d407d324864b3bb9aa3160e2ec6889f7727127
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11697
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
2024-05-22 10:32:55 +00:00
Connor Brewster
e7be342256 feat(tvix/store): concurrently upload small blobs during nar ingestion
Currently all blobs are uploaded serially when ingesting NARs. If a NAR
contains many, small blobs, ingestion may become slow if there is a lot
of round-trip latency to the blob service.

This makes the NAR ingester use the ConcurrentBlobUploader which allows
for buffering small blobs in memory so they can be uploaded concurrently
to the blob service without blocking further deserialization.

Change-Id: I093a73770232df12d9a11e5d901b99c08505c3cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11694
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-05-20 15:21:51 +00:00