Commit graph

17 commits

Author SHA1 Message Date
Vincent Ambo
38042ea445 feat(tazjin/tgsa): add gpt-3 powered message translation feature
this is slow and often overloaded, but it's kind of cool when it
works. this translation method deals much better with the kind of
slang you'd see in telegram posts than any other method.

Change-Id: I7e4c845eb382f0eac627c4237b492c8e40dae574
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8625
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-05-24 16:28:33 +00:00
Vincent Ambo
eadcfbbfab feat(tazjin/tgsa): support extracting fallback message from preview
some telegram channels do not allow embedding of messages, but do
allow a preview to be shown on twitter. this preview is just embedded
in the html, and can be scraped out if no message was found.

technically this preview also contains image links, but they are to
very low resolution, thumbnail-style images so i decided not to
include them here.

Change-Id: Ifb89f9fbde8140d577a5ee3af6e60b04232e53e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8480
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2023-04-12 12:31:12 +00:00
Vincent Ambo
9b395edaa5 chore(tazjin/tgsa): remove note about long image links
this is no longer the case since image links are now forwarded by tgsa
with a cache

Change-Id: I36216efb5f372115e082a11a51c04736f102debb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5612
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-16 08:20:22 +00:00
Vincent Ambo
73c08943b8 feat(tazjin/tgsa): use permanent media links in bbcode
uses the tgsa-proxied media links in the message bbcode, leading to
stable image serving even if telegram swaps cdn addresses around, as
long as their embed page is någorlunda the same.

Change-Id: I50af1b3512d4e429fae4a2b3d10395664169e7a1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5611
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-15 00:01:31 +00:00
Vincent Ambo
b308361a73 feat(tazjin/tgsa): add stable redirects to telegram images
telegram occasionally changes the upstream urls at which images are
served, with old/new partially overlapping in time afaict.

with this commit, a new endpoint is added to tgsa at

  /img/$user/$post_id/$img_idx

which serves a redirect to the actual image url that telegram had
returned within the last hour (i.e. as per the cache).

next step is to use these image urls in the bbcode itself.

Change-Id: I9aa5cb56bc444cbe796868346c67f2e1e1b79413
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5610
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-14 23:50:50 +00:00
Vincent Ambo
f200b1265f refactor(tazjin/tgsa): move error handling one layer up
this lays the groundwork for adding another handler and returning
handler results as `anyhow::Result<rouille::Response>`.

needed for the image redirect stuff.

Change-Id: I909bd9c2f46f42ea759d50662d7bc36c1f408ed3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5609
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-14 23:50:50 +00:00
Vincent Ambo
e3d09c3446 refactor(tazjin/tgsa): factor out cache access helper
factor out a function to access telegram posts from the cache,
fetching them anew if required.

a small behavioural change means that the program now takes a write
lock when fetching a post, to avoid simultaneously fetching the same
post many times (quite likely once it serves image redirects).

Change-Id: If9c1429f86fb118dab90834f349cb222709c3a31
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5608
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-14 23:50:50 +00:00
Vincent Ambo
89f25b431d feat(tazjin/tgsa): Expire cached telegram entries after one hour
Telegram expires certain links in messages after some (unknown) amount
of time; this commit is the first step to working around that by
providing an image URL proxy.

In order to do that, we tick two important boxes here:

1. Store an extensible data structure in the cache.
2. Expire them periodically.

Change-Id: Iba192d8b71db4493c942d1baf5680bd086f8f60b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5607
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-05-14 22:31:31 +00:00
Vincent Ambo
e0b9d9b1cd fix(tazjin/tgsa): Preserve newlines in messages
Change-Id: I9e2d7038357a5510898d153740ae0c0e1f7a8a3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5483
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-04-20 11:02:44 +00:00
Vincent Ambo
094b8cc7af style(tazjin/tgsa): Move attribution outside of quote body
When quoting a post containing a tgsa quote, the entire quote was
previously erased. Doing it this way will leave the Telegram link in it.

Change-Id: I53bfadd0615734f7e219b20509858d9b73c908a4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5482
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-04-20 09:37:34 +00:00
Vincent Ambo
eca7954758 chore(tgsa): Remove shortening of image links
The link shortening was in place because posts with image URLs that
were too long would not get recognised as valid image links.

astral has fixed this after my report:

  https://forums.somethingawful.com/showthread.php?threadid=3999866

I verified this with the example post that has a bunch of images and
such in it.

Thanks astral!

Change-Id: I60e90fd52020ee5464c931a4dde91382c8a149ab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5481
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-04-19 22:07:13 +00:00
Vincent Ambo
856dfa62a1 feat(tazjin/tgsa): Handle messages with audio attached
Audio can not be embedded on these stupid dead comedy forums, but
people can click through to listen.

Change-Id: I6e28636e69e424bb8cbc6b92963d1b28b3c04bf6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5478
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-04-17 22:12:43 +00:00
Vincent Ambo
6c8841c176 chore(tazjin/tgsa): Update quote footer text
Change-Id: I39ea4a40d9265d06193e3bddede157b10066da06
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5477
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-04-17 20:01:27 +00:00
Vincent Ambo
91198685b4 feat(tazjin/tgsa): cache generated bbcode in-memory per message
Change-Id: Id36266ef20585bfbdffb197ab08aba2978cb86cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5471
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2022-04-16 21:31:50 +00:00
Vincent Ambo
792de59c64 feat(tazjin/tgsa): add "web interface" for this
Change-Id: I1619f0ceec2f83f4728d7de006a08b2ce6148e59
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5468
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-04-16 20:35:01 +00:00
Vincent Ambo
f6b8bf8b41 feat(tazjin/tgsa): Support linked video thumbnails
I couldn't figure out how to get the ID for the `single=1` mode for
linking to a video directly, but linking to the post in embed mode
should also work.

Change-Id: Iebbd62724c36cee227a7c24968617ab418e8bd0c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5467
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-04-16 20:35:01 +00:00
Vincent Ambo
3d58dfb4ab feat(tazjin/tgsa): Implement initial tg embed -> BBcode features
Supports only posts with plain photos, and loses all message
formatting, but it's getting there.

Change-Id: I9e4afcf3072d1e0724521ccbdc1338fe4f8d5ebe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5466
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2022-04-16 20:35:01 +00:00