Commit graph

5 commits

Author SHA1 Message Date
edef
84a82f6f41 feat(users/edef/weave): use tracing_indicatif for progress
Progress bars :3

Change-Id: I770d0f8381521b6efc8b38c0db4d59c771887fee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12673
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-10-20 21:22:23 +00:00
edef
201d8f0cf2 feat(users/edef/weave): use FxHashSet and dedupe early
Deduping early saves a fair bit of memory, but the extra hashing is
costly.

We switch to FxHash, since we don't need a DoS-proof hash, but we do
need it to be *fast*.

Change-Id: Ic6b7010874c417862baa9b882593208c8dd1d5e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12648
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-10-19 17:01:38 +00:00
edef
313899c291 refactor(users/edef/weave/swizzle): use polars streaming
This vastly reduces the memory requirements, so we can run in ~40G RAM.

Change-Id: I4952a780df294bd852a8b4682ba2fd59b9bae675
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12667
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-10-19 17:01:38 +00:00
edef
dfff592784 fix(users/edef/weave): use safer_owning_ref
owning_ref has serious unsoundness.

Change-Id: Ie760697cd6399e6bc75f1ad17c9bb74adc077a35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12656
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: edef <edef@edef.eu>
Reviewed-by: flokli <flokli@flokli.de>
2024-10-17 22:47:41 +00:00
Vincent Ambo
bb34210a64 chore(tvix/tools): move weave to //users/edef
This is not a core Tvix tool, it's a tool that uses a Tvix component.

Change-Id: I705f2c4ab87f1512e005007c933e16b84ed4279f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12605
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2024-10-12 23:11:25 +00:00
Renamed from tvix/tools/weave/Cargo.toml (Browse further)