Eventually, we'll want to replace dump-stream-binary with something more
efficient—given that we have flexi-streams we can use something that
only does matching element types no problem. REDIRECT-STREAM is much
more efficient thanks to using an internal buffer.
streams.lisp gets a new section at the beginning for grouping utilities
that don't have any real (internal) dependencies.
Change-Id: I141cd36440d532131f389be2768fdaa54e7c7218
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8583
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Porting over the rest of the decoding (RFC2047) and especially encoding
over to qbase64 is still pending, as it is a little trickier.
Change-Id: Id4740eb074a387aeea2cb94b781e204248530799
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8582
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
The input adapter streams were input streams yielding either binary or
character data that could be constructed from a variable data source.
The stream would take care not to destroy the underlying data
source (i.e. not close it if it was a stream), so similar to with
FILE-PORTIONs, but simpler.
Unfortunately, the implementation was quite inefficient: They are
ultimately defined in terms of a function that retrieves the next
character in the source. This only allows for an implementation of
READ-CHAR (and READ-BYTE). Thanks to cl/8559, READ-SEQUENCE can be used
on e.g. FILE-PORTION, but this was still negated by a input adapter
based on one—then, READ-SEQUENCE would need to fall back on READ-CHAR or
READ-BYTE again.
Luckily, we can replace BINARY-INPUT-ADAPTER-STREAM and
CHARACTER-INPUT-ADAPTER-STREAM with a much simpler abstraction: Instead
of extra stream classes, we have a function, MAKE-INPUT-ADAPTER, which
returns an appropriate instance of FLEXI-STREAM based on a given source.
This way, the need for a distinction between binary and character input
adapter is eliminated, since FLEXI-STREAMS supports both binary and
character reads (external format is not yet handled, though).
Consequently, the :binary keyword argument to MIME-BODY-STREAM can be
dropped.
flexi-streams provides stream classes for everything except a stream
that doesn't close the underlying one. Since we have already implemented
this in POSITIONED-FLEXI-INPUT-STREAM, we can split this functionality
into a new superclass ADAPTER-FLEXI-INPUT-STREAM.
This change also allows addressing the performance regression
encountered in cl/8559: It seems that flexi-streams performs worse when
we are reading byte by byte or char by char. (After this change mblog is
still two times slower than on r/6150.) By eliminating the adapter
streams, we can start utilizing READ-SEQUENCE via decoding code that
supports it (i.e. qbase64) and bring performance on par with r/6150
again. Surely there are also ways to gain back even more performance
which has to be determined using profiling. Buffering more aggressively
seems like a sure bet, though.
Switching to flexi-streams still seems like a no-brainer, as it allows
us to drop a lot of code that was quite hacky (e.g. DELIMITED-INPUT-
STREAM) and implements en/decoding handling we did not support before,
but would need for improved correctness.
Change-Id: Ie2d1f4e42b47512a5660a1ccc0deeec2bff9788d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8581
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
This is possible since all the commits have been made by me. The code
taken from SCLF (which is licensed LGPL-2.1-or-later) can also be
included since the LGPL 2.1 is [compatible] with the GPL 3.0.
compatible: https://www.gnu.org/licenses/license-list.en.html#LGPLv2.1
Change-Id: I2d274c29378679c489dc667a53b234642c3da817
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5928
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
SCLF is quite a big utility library (almost 3€ LOC) with limited
portability (CMUCL, SBCL and CLISP to an extent). Continuing to maintain
it is an unnecessary burden, as depot only uses a fraction of it which
is now inlined into the respective users (mime4cl and mblog).
In the future trimming down ex-sclf.lisp may make sense either by
refactoring the code that uses it or by moving interesting utilities
into e.g. klatre.
Change-Id: I2e73825b6bfa372e97847f25c30731a5aad4a1b5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5922
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Accessing the headers of a MIME message feels like something mime4cl
should handle. We implemented this ad hoc in mblog before in order to
not need to worry about doing it in a sensible way. Now we introduce a
decent-ish interface for getting a header from a MIME message,
mime-message-header-values:
* It returns a list because MIME message headers may appear multiple
times.
* It decodes RFC2047 only upon request, as you may want to be stricter
about parsing certain fields.
* It checks header name equality case insensitively.
The code for decoding the RFC2047 string is retained and still uses
babel for doing the actual decoding.
Change-Id: I58bbbe4b46dbded04160b481a28a40d14775673d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5150
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Depending on the stream backing this, read-sequence should be more
efficient.
Change-Id: I5d0461f76f4b132ac6e6c3a2e503f0173d5f4114
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5194
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This change finally sort of puts the parts together: We take a maildir,
render all its note messages as standalone HTML, extract the attachments
alongside and finally generate a global index page linking all notes.
The new executable and mnote-html are both contained in the same image
and we dispatch the right functionality based on argv[0].
Change-Id: I5a5bdbfaca79199f92e73ea4a2f070fa900d2bc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5113
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This is the only thing we need from that package and it avoids having
to solve the annoying conflict between closure-html and who.
Change-Id: Iacfb8d4948d1987e767ffc456b8e141b468ef6d9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5111
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Non ASCII Subjects will use RFC2047 to encode their content. Using
mime4cl's parse-RFC2047-text we obtain a list of ASCII strings and byte
vectors tagged with their encoding. Using babel we can then decode the
byte sequence, assuming the encoding is named the same in babel and
RFC2047 (which it is for UTF-8 at least…).
Change-Id: I2840672409452bd194fb1635721e338364d9b484
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5078
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
* Upon creation of an apple-note object we can check if certain fields
we are interested in are present and of the right type etc.
These currently are:
- UUID (for links later)
- Subject (title)
- Time
- Text part with supported MIME type
These are then put into their own shortcut fields in the apple-note
subclass which allows for easier access and forces us to make sure
they are present.
* Split out everything note related into its own package. Using the new
type, we can expose an interface which sort of makes sense.
Change-Id: Ic9d67518354e61a3cc8388bb0e566fce661e90d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5072
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
For now mblog only contains the mnote-html executable which takes a mime
message from a maildir and prints the equivalent HTML fragment to
stdout. It is intended to work with the mblaze(7) utilities,
i. e. mnote-html resolves all `object` tags to proper `img` inclusions
with the correct filename, so mshow(1)'s -x version can supply the
needed image files. A note created using Apple's Notes app (tested with
the iOS version) can be converted in a viewable HTML file like this:
$ mnote-html path/to/msg > fragment.html
$ mshow -x path/to/msg
$ cat <(echo "<!DOCTYPE html>") fragment.html > document.html
$ xdg-open document.html
Note that only the limited feature set of Apple Notes when using the
IMAP backend is supported. The iCloud-based one has more (quite neat)
features, but its notes can only accessed via an internal API as far as
I know.
This CLI is a bit impractical due to the big startup overhead of loading
the lisp image. mblog should be become a fully fletched static site
generator in the future, but this is a good starting point and providing
the mnote-html tool is certainly useful.
Change-Id: Iee6d1558e939b932da1e70ca2d2ae75638d855df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3271
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
