tvl-depot/third_party/lisp
sterni b379e44dfb refactor(3p/lisp/mime4cl): use flexi-streams and binary input
This refactor is driven by the following (ultimate) aims:

- Get rid of as much of the custom stream code in mime4cl which makes
  less code to maintain in the future.

- Lay the groundwork for correct handling of 8bit transfer encoding:
  The mime4cl we inherited assumes that any MIME message can be decoded
  completely by the CL implementation (in SBCL's case using latin1)
  into CHARACTERs. This is not necessarily the case. flexi-streams
  allows changing how the stream is decoded on the fly and also has
  support for reading the underlying bytes which is perfect for the
  requirements decoding MIME has.

- Since flexi-streams uses trivial-gray-streams, it supports
  READ-SEQUENCE. Taking advantage of this may improve decoding
  performance significantly in the future.

This incurs the following changes:

- Naturally we now open given files as binary files in MIME-MESSAGE.
  Given strings are encoded using STRING-TO-OCTETS and then passed on
  to a new octet vector method. Instead of MY-STRING-INPUT-STREAM this
  now uses flexi-streams' WITH-INPUT-FROM-SEQUENCE.

- OPEN-FILE-PORTION and OPEN-DECODED-FILE-PORTION need to be merged,
  since the transfer encoding not only implies an extra decoder stream
  that needs to be attached after file portion stream, but also imply a
  certain encoding of the stream itself (mostly binary vs. ASCII).
  As flexi-streams can change their encoding on the fly this could be
  untangled again, but it is not strictly necessary.

  As before, we use the DATA slot of the file portion to create a fresh
  stream if possible. Instead of strings we now use an vector of octets
  to match MIME-MESSAGE.

  The actual portioned stream relies on POSITIONED-FLEXI-INPUT-STREAM, a
  subclass of the stock FLEXI-INPUT-STREAM class, described below.

- POSITIONED-FLEXI-INPUT-STREAM replaces DELIMITED-INPUT-STREAM. It is
  created using MAKE-POSITIONED-FLEXI-INPUT-STREAM which accepts the
  same arguments as MAKE-FLEXI-STREAMS and, additionally, :IGNORE-CLOSE.
  A POSITIONED-FLEXI-INPUT-STREAM works the same as an
  FLEXI-INPUT-STREAM, but upon creation, the underlying stream is
  rewinded or forwarded to the argument given by :POSITION using
  FILE-POSITION.

  If :IGNORE-CLOSE is T, a call to CLOSE is not forwarded to the
  underlying stream.

Change-Id: I2d48c769bb110ca0b7cf52441bd63c1e1c2ccd04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8559
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2023-05-18 16:14:37 +00:00
..
asdf-flv chore: Rename pkgs->depot in all Nix file headers 2020-02-21 13:54:53 +00:00
closure-html chore(3p/lisp): use nixpkgs sources for closure-html 2021-12-15 21:02:10 +00:00
mime4cl refactor(3p/lisp/mime4cl): use flexi-streams and binary input 2023-05-18 16:14:37 +00:00
npg chore(gerrit): migrate OWNERS files to code-owners style 2022-09-19 11:13:28 +00:00
qbase64 feat(3p/lisp): add qbase64 2023-05-14 14:42:43 +00:00
quasiquote_2 chore: Rename pkgs->depot in all Nix file headers 2020-02-21 13:54:53 +00:00
s-xml style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
alexandria.nix chore(3p/lisp): Unvendor alexandria and use nixpkgs sources 2021-12-15 21:02:10 +00:00
anaphora.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
babel.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
bordeaux-threads.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
cffi.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
chipz.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
chunga.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-ansi-text.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-base64.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-colors.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-colors2.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-date-time-parser.nix feat(3p/lisp/cl-date-time-parser): init at 00d6fc7 2022-01-26 17:43:54 +00:00
cl-fad.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
cl-json.nix feat(nix/buildLisp): re-enable CCL 2022-08-30 17:51:35 +00:00
cl-plus-ssl.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
cl-ppcre.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-prevalence.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-smtp.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-unicode.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
cl-who.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
cl-yacc.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
closer-mop.nix feat(3p/overlays/ecl-static): 21.2.1 -> 1c98924 2022-01-28 10:43:01 +00:00
closure-common.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
defclass-std.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
drakma.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
easy-routes.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
fiveam.nix chore(3p/lisp): use nixpkgs sources for fiveam 2021-12-15 21:02:11 +00:00
flexi-streams.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
global-vars.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
hunchentoot.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
ironclad.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
iterate.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
lass.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
let-plus.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
lisp-binary.nix feat(3p/lisp/lisp-binary): 2022-04-10 -> 2022-09-19 2022-12-19 13:13:00 +00:00
local-time.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
marshal.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
md5.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
metabang-bind.nix feat(3p/lisp): add metabang-bind 2022-01-19 13:13:09 +00:00
moptilities.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
nibbles.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
OWNERS chore(gerrit): migrate OWNERS files to code-owners style 2022-09-19 11:13:28 +00:00
parse-float.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
parse-number.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
parseq.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
physical-quantities.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
postmodern.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
prove.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
puri.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
rfc2388.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
routes.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
s-sysdeps.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
split-sequence.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
trivial-backtrace.nix chore(3p/lisp): use nixpkgs sources for trivial-backtrace 2021-12-15 21:14:12 +00:00
trivial-features.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
trivial-garbage.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
trivial-gray-streams.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
trivial-indent.nix refactor(3p/lisp): Use sources from nixpkgs where possible 2021-12-15 10:34:02 +00:00
trivial-ldap.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
trivial-mimes.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
uax-15.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
unix-opts.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
usocket-server.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00
usocket.nix style: format entire depot with nixpkgs-fmt 2022-01-31 16:11:53 +00:00