Commit graph

21 commits

Author SHA1 Message Date
sterni
36f6322d16 feat(sterni/nix/fun): implement tail call “optimization” for Nix
I've had the notion that builtins.genericClosure can be used to express
any recursive algorithm, but a proof is much better than a notion of
course! In this case we can easily show this by implementing a function
that converts a tail recursive function into an application of
builtins.genericClosure.

This is possible if the function resolves its self reference using a
fixed point which allows us to pass a function that encodes the call to
self in a returned attribute set, leaving the actual call to
genericClosure's operator. Additionally, some tools for collecting meta
data about functions (argCount) and calling arbitrary functions (apply,
unapply) are necessary.

Change-Id: I7d455db66d0a55e8639856ccc207639d371a5eb8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5292
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
2022-02-15 12:52:19 +00:00
Vincent Ambo
aa122cbae7 style: format entire depot with nixpkgs-fmt
This CL can be used to compare the style of nixpkgs-fmt against other
formatters (nixpkgs, alejandra).

Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: cynthia <cynthia@tvl.fyi>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: eta <tvl@eta.st>
Reviewed-by: grfn <grfn@gws.fyi>
2022-01-31 16:11:53 +00:00
sterni
750ef6c693 feat(sterni/nix/utf8): check if codepoint valid/encodeable
* Enforce the U+0000 to U+10FFFF range in `count` and throw an error if
  the given codepoint exceeds the range (encoding U+0000 won't work of
  course, but this is Nix's fault…).

* Check if the produced bytes are well formed and output an error if
  not. This indicates that the codepoint can't be encoded as UTF-8, like
  U+D800 which is reserved for UTF-16.

Change-Id: I18336e527484580f28cbfe784d51718ee15c5477
2021-11-25 12:15:35 +01:00
sterni
0e9c770972 refactor(sterni/nix/utf8): let wellFormedByte check first byte
Previously we would check the first byte only when trying to figure out
the predicate for the second byte. If the first byte was invalid, we'd
then throw with a helpful error message. However this made
wellFormedByte a very weird function.

At the expense of doing the same check twice, we now check the first
byte, when it is first passed, and always return a boolean.

Change-Id: I32ab6051c844711849e5b4a115e2511b53682baa
2021-11-25 12:15:35 +01:00
sterni
87a0aaa77d feat(sterni/nix/utf8): implement UTF-8 encoding
This implementation is still a bit rough as it doesn't check if the
produced string is valid UTF-8 which may happen if an invalid Unicode
codepoint is passed.

Change-Id: Ibaa91dafa8937142ef704a175efe967b62e3ee7b
2021-11-25 12:15:35 +01:00
sterni
9370ea5e33 chore(sterni/nix/utf8): remove decodeSafe
This is not really used anywhere and kind of useless. A better
decodeSafe would never return null and instead make use of replacement
characters to represent invalid bytes in the input.

Change-Id: Ib4111529bf0e472dbfa720a5d0b939c2d2511de5
2021-11-25 12:15:35 +01:00
sterni
ab92c42f59 feat(sterni/nix/utf8): allow decoding the empty string
Change-Id: I8de9cd28c822ac5befbcd16e118440cd13cd86e9
2021-11-23 14:23:41 +01:00
sterni
8615322bc8 refactor(sterni/nix/utf8): use genericClosure for decoding iteration
builtins.genericClosure is a quite powerful (and undocumented) Nix
primop: It repeatedly applies a function to values it produces and
collects them into a list. Additionally individual results can be
identified via a key attribute.

Since genericClosure only ever creates a single list value internally,
we can eliminate a huge performance bottleneck when building a list in a
recursive algorithm: list concatenation. Because Nix needs to copy the
entire chunk of memory used internally to represent the list, building
big lists one element at a time grinds Nix to a halt.

After rewriting decode using genericClosure decoding the LaTeX source
of my 20 page term paper now takes 2s instead of 14min.

Change-Id: I33847e4e7dd95d7f4d78ac83eb0d74a9867bfe80
2021-11-23 14:22:24 +01:00
sterni
e507b84291 feat(users/sterni/nix/string): very simple printf implementation
This is mostly to yet another silly idea which turns out to be
possible. This may be actually useful should I implement more
sophisticated format specifiers like "%xd" or "%f".

Change-Id: Ia56cd6f5793a09fe5e19c91a8e8f9098f3244d57
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3537
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-12 09:25:30 +00:00
sterni
83c8716afd chore(3p): update NixOS channels to 2021-08-30
This lets us benefit from the recent OpenSSL security-related
update [1]. Since nixos-unstable is still stuck, we temporarily
use nixos-unstable-small as our unstable channel.

Fixes necessary:

* //users/sterni/nix/char:
  Someone has decided to drop writers.writeC upstream [2],
  so we reimplement it ad-hoc using runCommandCC

[1]: https://www.openssl.org/news/secadv/20210824.txt
[2]: 982f46985e

Change-Id: Id84756e2e370296b7a27e1a3f1744f58f8fe3c47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3463
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2021-08-30 17:06:40 +00:00
sterni
9ed439bfbd feat(users/sterni/nix): cursed nix html DSL
Couldn't sleep, so I made a surprisingly neat way to render HTML
documents in Nix using our favorite feature __findFile:

  let
    inherit (depot.users.sterni.nix.html) __findFile esc;

  in

  <html> {} [
    (<head> {} [
      (<meta> { charset = "utf-8"; } null)
      (<title> {} (esc "hello"))
    ])
    (<body> {} [
      (<h1> {} (esc "hello world"))
    ])
  ]

=> "<html><head><meta charset=\"utf-8\"/><title>hello</title></head><body><h1>hello world</h1></body></html>"

Change-Id: Id36808a56ae3da3b5263c06f29342fc22d105c21
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3410
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 15:34:58 +00:00
Profpatsch
eb41eef612 chore(nix): move rustSimple from users.Profpatsch.writers
I think it’s solid enough to use in a wider context.

Change-Id: If53e8bbb6b90fa88d73fb42730db470e822ea182
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3055
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-04-24 10:23:55 +00:00
sterni
f59ab9aba5 feat(sterni/nix/fun): add hasEllipsis
As a complementation to builtins.functionArgs this function checks if
the function has a set pattern that contains an ellipsis
(i. e. `{ [arg, [ arg1, [ … ]]] ... }:`). The implementation of this is
pretty cursed however since there is no clean way to do this in vanilla
nix: We need to match on the output of builtins.toXML which does try to
serialize functions by outputting their argument and information about
it (whether it is a normal argument or a attribute set pattern, in the
latter case it also serialize every component of the pattern).

Change-Id: I0f33721811a3180cec205a0c98e6d92e10e92075
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2950
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-12 17:47:53 +00:00
sterni
8d4b2f3d54 refactor(sterni): use pkgs over third_party to import from nixpkgs
This should ease migrating to a distinction between depot.third_party
and pkgs (as in nixpkgs) in the future.

Ref cl/2910, b/108.

Change-Id: I53a854071fddd7c0d0526cc4c5b16998202082c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2913
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2021-04-10 11:40:18 +00:00
sterni
a5f2b446aa feat(sterni/nix/url): implement urldecoding
We use builtins.split directly as it should be a bit more efficient as
lib.splitStrings. Also its returning of a list for every regex match is
useful to update the state while parsing the tokens:

* The tokens are obtained by splitting the string at every '%'
* Everytime we see a boundary (that is a list in the returned
  list of builtins.split), we know that the first two chars of
  the next string are a percent encoded character.

One implementation flaw is that it will currently crash if it encounters
mal-formed URLs (since int.fromHex chrashes if it encounters any non
hex digit characters) and accepts some malformed urlencoding like
"foo %A".

Change-Id: I90d08d7a71b16b4f4a4879214abd7aeff46c20c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2744
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-01 13:09:46 +00:00
sterni
b810c46a45 feat(users/sterni/nix/utf8): pure nix utf-8 decoder
users.sterni.nix.utf8 implements UTF-8 decoding in pure nix. We
implement the decoding as a simple state machine which is fed one byte
at a time. Decoding whole strings is possible by subsequently calling
step. This is done in decode which uses builtins.foldl' to get around
recursion restrictions and a neat trick using builtins.deepSeq puck
showed me limiting the size of the thunks in a foldl' (which can also
cause a stack overflow).

This makes decoding arbitrarily large UTF-8 files into codepoints using
nix theoretically possible, but it is not really practical: Decoding a
36KB LaTeX file I had lying around takes ~160s on my laptop.

Change-Id: Iab8c973dac89074ec280b4880a7408e0b3d19bc7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2590
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-03-05 11:07:41 +00:00
sterni
5ae1d3fd7b feat(users/sterni/nix/flow): add switch conditional
switch would probably otherwise be called match, but has been renamed so
it isn't confused with string.match and the enum matching capabilities
yants has.

It implements the closest to pattern matching nix can come which is
still flexible enough to not be painful: Syntactically it works like
cond, but is given a value. Instead of booleans it checks passed
predicates or equality if simple values are passed. Both types of checks
can be mixed.

Change-Id: I40f000979cfd469316e15fd58d6c3a80312c1cc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2589
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-03-05 11:07:41 +00:00
sterni
ef40a8621f feat(users/sterni/nix/fun): make lrs read left to right completely
Change-Id: I57d290f770bc1d6bd88a46924889b919d68201e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2588
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-03-05 11:07:41 +00:00
sterni
7af0fb1066 refactor(users/sterni/nix/string): don't calculate length for drop
Since nix ends the substring at the end of the string anyways we can
just statically use the largest nix integer as the length of the string.
According to my testing this it ever so slightly faster as well.

Change-Id: I64566e91c7b223f03dcebe3bc5710696dc4261bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2587
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-03-05 11:07:41 +00:00
sterni
8ff14cacb6 feat(users/sterni/nix): move flow.match to string.match
After all it only matches strings.

Change-Id: I3d2e5221ef43f692de69028e78ed98b6b11f82d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2586
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2021-03-05 11:07:41 +00:00
sterni
3b33c1bd76 feat(users/sterni/nix): add sternis nix lib
What you see here is mostly the fallout of me implementing a correct
urlencode implementation in nix for Profpatsch's blog implementation
(although they'll probably keep it at arm's length).

Where I want to go from here:

* Extend this library towards general purpose nix™, mainly by
  implementing missing interfaces which you'd still have to use
  <nixpkgs/lib> for right now. Reexposing parts of <nixpkgs/lib>
  with better naming is fine for now, at some point I'd contemplate
  making this depend on nothing outside of depot, maybe even itself
  (should be easy we only use yants for an easily replaceable check).

* Improve error messages possibly by carefully reintroducing yants. I
  originally typed essentially everything using yants, but turns out
  this can a) be dangerous when stuff you are handling throws because
  type checking means evaluating and b) has a incredible performance
  cost in some cases.

* Reexpose builtins with better naming and slightly wrapped so they
  don't unrecoverably throw in cases where a null or something would
  suffice.

Change-Id: I33ab08ca4e62dbc16b86c66c653935686e6b0e79
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2541
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2021-03-01 17:34:35 +00:00