Commit graph

31 commits

Author SHA1 Message Date
Profpatsch
7129749968 feat(users/Profpatsch/whatcd-resolver): also link json ld fields
We can cross-reference all of these to schema.org, it should work for
most of the fields.

Change-Id: I38d8dbc7e964764886ddd156c4148bcf3ee376f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9744
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-10-15 23:05:28 +00:00
Profpatsch
81b790af1d feat(users/Profpatsch/whatcd-resolver): start checking musicbrainz
Ideally we can figure out how to search for single songs by grepping
through musicbrainz. For this we kinda need the jsonld results, so
this is a first step which visualizes the structure and makes it
easy-ish to lazily traverse it.

Change-Id: Ieca21674dee8e8c2dacbab4f2f15ccbe067da647
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9743
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-10-15 22:57:25 +00:00
Profpatsch
0a98f8ec3b chore(third_party/haskell): update pa packages
Change-Id: I8abcb479b0f5c0bd6ed1abc3c9618c2362ff835a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9740
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-10-15 18:58:41 +00:00
Profpatsch
8e811fe625 feat(users/Profpatsch/whatcd-resolver): more otel traces
Change-Id: I5094b64f202eeedb57510a25850bba2edd9ec36f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9725
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-10-14 18:01:11 +00:00
Vincent Ambo
47bc5bb4bc chore(3p/sources): bump channels & overlays (2023-10-08)
Change-Id: I9eeaffad53aba278cd9082c5b4c3e7b1c5f6cc22
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9585
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: tazjin <tazjin@tvl.su>
2023-10-09 09:09:10 +00:00
Profpatsch
acfc49efc2 feat(users/Profpatsch/MonadPostgres): trace db queries
Experiment of how to instrument a lib I’m using to trace instead of
log.

Now that we added MonadTracer to Transaction, we can drop the unlifted `inSpanT`.

Change-Id: Iea891a58cfb33a0837978611456c33aefcccf0d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9491
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-09-29 17:25:26 +00:00
Profpatsch
0dcc72a31c feat(users/Profpatsch/whatcd-resolver): add basic otel tracing
For it to work, you need otel (e.g. jaeger) to run on port 4317.

Change-Id: I36f0493b9be26af256769ae5af8916029036a76e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9488
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-09-29 17:25:26 +00:00
Profpatsch
3863a2ebd6 refactor(users/Profpatsch/whatcd-resolver): more efficient inserts
Instead of inserting torrents and every tour group seperately, insert
the tour groups and then the torrents in one go (unzipped).

I finally found a good use for list comprehensions, flattening nested
lists.

Change-Id: I7dfc765ad058dff3afb3b03887141b334a4b1988
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9486
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
2023-09-29 17:11:20 +00:00
Profpatsch
6c360f2b64 refactor(users/Profpatsch/whatcd-resolver): naive combined insert
Still n+1, but now we got the IO where we want it to make the database
inserts efficient.

Change-Id: I2ee36ea41bf186cbeb5581b0df802bb3611769fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9485
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-09-29 17:11:20 +00:00
Profpatsch
7bc739ec5f refactor(users/Profpatsch/whatcd-resolver): move inserts out
Now we can move the I/O into a where block.

Change-Id: Ib5334948f3d11ca120ce0b7a46c67f8500fdab3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9484
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
2023-09-29 17:08:16 +00:00
Profpatsch
7157e2baed refactor(users/Profpatsch/whatcd-resolver): prepare to split IO
Returning an I/O action was a good first approximation, but leads to a
n+1 query problem, making the whole shebang pretty slow after doing a
search.

Thus we need to split data & I/O, so we can be more clever in the next
commit.

Change-Id: Ieb2f8d5445f1258047da9b121b977c0b8d2dd7f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9483
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-09-29 17:08:15 +00:00
Profpatsch
3de9601764 feat(users/Profpatsch/whatcd-resolver): add executable
Change-Id: I28f5c26443b9a878f8f374af2c7935831d17ab12
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9355
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-09-17 18:05:42 +00:00
Profpatsch
07b0dff209 refactor(users/Profpatsch/whatcd-resolver): rename main function
Change-Id: I8ce8ef1cc8a35358d20610be7c8bdadefd3aa30f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9354
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-09-17 18:05:42 +00:00
Profpatsch
d111a0fda8 feat(users/Profpatsch/whatcd-resolver): misc improvements
* run on port 9092 (transmission runs on 9091)
* run postgres on port 5431 instead of 5432 (to not interfere)
* only search for albums for now
* correctly handle missing torrent file in SELECT

Change-Id: I20125f7731c9b80a9e8ea05b726adfb1244a24bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9335
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-09-15 12:44:06 +00:00
Profpatsch
8cfe6bc99b feat(users/Profpatsch/whatcd-resolver): query all pages in search
If more than one page is returned by the search, query all of them.

Also add an ON CONFLICT clause in case the torrent group already
exists, to update it.

This function is getting a bit unwieldy (plus it suffers from an n+1
problem), but ok.

Change-Id: Ib505a2be8286d658ae44a3fe124a4fb42d0fc0c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9334
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-09-15 12:34:04 +00:00
Profpatsch
33fa42a1a3 chore(users/Profpatsch): Update postgres module n stuff
Improvements from “upstream”, fresh served.

Change-Id: I60e02835730f6a65739eaa729f3e3eed1a0693e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9025
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-08-08 20:03:22 +00:00
Profpatsch
8dac21b444 feat(users/Profpatsch/whatcd-resolver): Prefer remasters
Usually remasters just … sound better. Thus let’s give them a boost in
the selection score. In case they are horrible they should have
significantly less people seeding them and less snatches than their
originals anyway (maybe the multiplier should be 1.5 or 2 instead?)

Change-Id: I6817091f42b7bcb75eaf57a6aa635b5ba468f99e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9014
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
2023-08-07 14:03:01 +00:00
Profpatsch
3e5a2ea57f chore(users/Profpatsch): Move Multipart2 into new webstuff package
Change-Id: I903f1b554beed1240d2a9cf14ff44d1f3cb41ec5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9013
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-08-06 16:08:23 +00:00
Profpatsch
b41af857ae chore(users/Profpatsch/whatcd-resolver): build with nix
Change-Id: I2d6ee30671bb60a96bcbfede1fd7a17fcf7c3ca8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9012
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-08-06 11:21:38 +00:00
Profpatsch
57bab040ed chore(users/Profpatsch): move utils to my-prelude
I want to use these in multiple projects.

Change-Id: I5dfdad8614bc5835e59df06f724de78acae78d42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8971
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-07-16 20:15:49 +00:00
Profpatsch
43feacb64b feat(users/Profpatsch/whatcd-resolver): Show percent done
Change-Id: I6d7852570bdca807e4d4fff01d72de9f1084fd42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8910
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-07-14 08:03:14 +00:00
Profpatsch
50c27b6ba1 feat(users/Profpatsch/whatcd-resolver): Somewhat fix torrent state
I feel like I’m slowly but steadily coding myself into a corner here,
have to rething the whole state thing.

Anyway, now the refresh will display roughly the same information as
the interactive one, which is *a* first step I guess.

Change-Id: I8820c2e321e6e8c9eba0f2f1cc70ce07a044621c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8906
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-07-14 08:03:14 +00:00
Profpatsch
12d23b3e64 feat(users/Profpatsch/whatcd-resolver): Add torrent & basic status
This is a bit dirty, ideally we have a single polling loop that uses
`hx-swap-oob` to fill all status fields in the table (to avoid O(n)
looping requests).

Change-Id: I78ab392964cf00e39424002fe48cb35a60af184a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8875
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-07-14 08:03:14 +00:00
Profpatsch
9504914a59 feat(users/Profpatsch/whatcd-resolver): Download torrent file
Change-Id: I75422a1fc4f94e8aa856f1ea1b2dbec42360c7ac
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8874
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-07-14 08:03:14 +00:00
Profpatsch
4ec27ed088 feat(users/Profpatsch/whatcd-resolver): Display transmission torrent
Change-Id: I1a45dd4c7fa798c161545abf545017be1f83a8f9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8873
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-07-14 08:03:14 +00:00
Profpatsch
fa0b7d0804 feat(users/Profpatsch/whatcd-resolver): Remove dynatable
In favor of all-server-side rendering.

Change-Id: I439c31cc6184cd1f6f77843819eebfb396e0ace8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8872
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-07-14 08:03:14 +00:00
Profpatsch
68a9037d17 feat(users/Profpatsch/whatcd-resolver): Add server-side search
Change-Id: Ifbbe3bca6988b0a090f456ae8d9dbaa808c89e19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8867
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
2023-07-14 08:03:14 +00:00
Profpatsch
5cfdd259df feat(users/Profpatsch/whatcd-resolver): Cache searches & web UI
When looking up stuff on the tracker, cache the results in our
database and display the best torrent matches in a simple web UI.

Change-Id: Iba8417fbdd3ea812765ab0289a1d5b03b7c2be81
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8857
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-07-14 08:03:14 +00:00
Profpatsch
70da4318f5 feat(users/Profpatsch/whatcd-resolver): INSERT red search results
Change-Id: Ice7fdb2e265cfb99734ed41d17b62ac98f7a4869
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8840
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-07-14 08:03:14 +00:00
Profpatsch
98e38339f2 feat(users/Profpatsch/whatcd-resolver): add initial redacted API
Change-Id: I3143a932646d61b7661df4e4da4edf24a360f6cc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8833
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-07-14 08:03:14 +00:00
Profpatsch
07b976ccd8 feat(users/Profpatsch): init whatcd-resolver
Change-Id: Ieb377fb8caa60e716703153dfeca5173f9a6779d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8830
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2023-07-14 08:03:14 +00:00