mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
tvl-depot/web/bubblegum/default.nix

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

274 lines
7.2 KiB
Nix
Raw Normal View History

feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
{ depot, lib, pkgs, ... }:
let
inherit (depot.nix)
runExecline
getBins
feat(nix/utils): add storePathName, a more generic baseNameOf This is a wrapper around baseNameOf which also can deal with derivations. Added to //nix/utils since I've found myself introducing an ad-hoc implementation of this for both //web/bubblegum and //nix/buildC. Change-Id: I2fcd97a150d6eda21ab323fa0d881ff7442a892e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3049 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-04-20 10:18:28 +02:00
utils
refactor(web/bubblegum): only rebuild examples on dependency change Using sparseTree we can make a (surprisingly long) list of things from depot the examples depend on and create a stripped down depot version which only contains them. As a result the examples are no longer rebuilt on every commit. Change-Id: I3693570ca4bdbbf9da795e552f278f3b1b1b77a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3504 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 00:20:31 +02:00
sparseTree
chore(nint): move from //users/sterni to //nix Since //web/bubblegum depends on nint, we need to move it to a non user directory to conform with the policy established via cl/3434. Note that this likely doesn't mean greater stability (which isn't really implied in depot anyways), since I still would like to use a more elaborate calling convention to allow for additional useful features. Change-Id: I616f905d8df13e3363674aab69a797b0d39fdd79 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3506 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 18:26:03 +02:00
nint
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
;
feat(nix/sparseTree): add optional `name` argument This allows pinning the name of the sparse tree derivation, which stops the continous rebuilding of tvix-store-proto dependents. I've opted to let the function take an attribute set instead and refactored the call sites appropriately. Change-Id: I3e57785094b1adbfffa24caf9f1c3384844fa200 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8965 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-07-13 23:12:01 +02:00
minimalDepot = sparseTree {
root = depot.path.origSrc;
name = "minimal-depot";
paths = [
# general depot things
"default.nix"
"nix/readTree"
# nixpkgs for lib and packages
"third_party/nixpkgs"
"third_party/overlays"
fix(web/bubblegum): third_party/sources is required in bubblegum Change-Id: Ifb63ab95a0a11f07e8ad2346aff5893fe7c268cd Reviewed-on: https://cl.tvl.fyi/c/depot/+/12351 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-08-26 14:27:22 +02:00
"third_party/sources"
feat(nix/sparseTree): add optional `name` argument This allows pinning the name of the sparse tree derivation, which stops the continous rebuilding of tvix-store-proto dependents. I've opted to let the function take an attribute set instead and refactored the call sites appropriately. Change-Id: I3e57785094b1adbfffa24caf9f1c3384844fa200 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8965 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-07-13 23:12:01 +02:00
# bubblegum and its dependencies
"web/bubblegum"
"nix/runExecline"
"nix/utils"
"nix/sparseTree"
# tvix docs for svg demo
"tvix/docs"
# for blog.nix
"users/sterni/nix"
];
};
refactor(web/bubblegum): only rebuild examples on dependency change Using sparseTree we can make a (surprisingly long) list of things from depot the examples depend on and create a stripped down depot version which only contains them. As a result the examples are no longer rebuilt on every commit. Change-Id: I3693570ca4bdbbf9da795e552f278f3b1b1b77a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3504 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 00:20:31 +02:00
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
statusCodes = {
# 1xx
"Continue" = 100;
"Switching Protocols" = 101;
"Processing" = 102;
"Early Hints" = 103;
# 2xx
"OK" = 200;
"Created" = 201;
"Accepted" = 202;
"Non-Authoritative Information" = 203;
"No Content" = 204;
"Reset Content" = 205;
"Partial Content" = 206;
"Multi Status" = 207;
"Already Reported" = 208;
"IM Used" = 226;
# 3xx
"Multiple Choices" = 300;
"Moved Permanently" = 301;
"Found" = 302;
"See Other" = 303;
"Not Modified" = 304;
"Use Proxy" = 305;
"Switch Proxy" = 306;
"Temporary Redirect" = 307;
"Permanent Redirect" = 308;
# 4xx
"Bad Request" = 400;
"Unauthorized" = 401;
"Payment Required" = 402;
"Forbidden" = 403;
"Not Found" = 404;
"Method Not Allowed" = 405;
"Not Acceptable" = 406;
"Proxy Authentication Required" = 407;
"Request Timeout" = 408;
"Conflict" = 409;
"Gone" = 410;
"Length Required" = 411;
"Precondition Failed" = 412;
"Payload Too Large" = 413;
"URI Too Long" = 414;
"Unsupported Media Type" = 415;
"Range Not Satisfiable" = 416;
"Expectation Failed" = 417;
"I'm a teapot" = 418;
"Misdirected Request" = 421;
"Unprocessable Entity" = 422;
"Locked" = 423;
"Failed Dependency" = 424;
"Too Early" = 425;
"Upgrade Required" = 426;
"Precondition Required" = 428;
"Too Many Requests" = 429;
"Request Header Fields Too Large" = 431;
"Unavailable For Legal Reasons" = 451;
# 5xx
"Internal Server Error" = 500;
"Not Implemented" = 501;
"Bad Gateway" = 502;
"Service Unavailable" = 503;
"Gateway Timeout" = 504;
"HTTP Version Not Supported" = 505;
"Variant Also Negotiates" = 506;
"Insufficient Storage" = 507;
"Loop Detected" = 508;
"Not Extended" = 510;
"Network Authentication Required" = 511;
};
/* Generate a CGI response. Takes three arguments:
1. Status of the response as a string which is
the descriptive name in the protocol, e. g.
`"OK"`, `"Not Found"` etc.
2. Attribute set describing extra headers to
send, keys and values should both be strings.
3. Response content as a string.
See the [README](./README.md) for an example.
feat(web/bubblegum): allow passing status as an int The whole pass the name of the status as a string thing was mostly born out of an overeager use of yants. It is still very neat especially for common cases like "OK", so we'll keep it, but also allow passing the integer variant of the status as well which probably feels more natural for a lot of people, especially over getting the casing right for "I'm a teapot". Change-Id: I3f012a291447ef385efdd28132292a8b331998c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2850 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-05 01:58:26 +02:00
Type: either int string -> attrs string -> string -> string
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
*/
feat(web/bubblegum): report some errors to the user via HTTP We can actually catch some errors that may be generated in bubblegum applications where we can report them to the user in a way that doesn't require curl -vv: * Type errors in the status argument: By removing yants completely we not only (presumably) gain some performance, but also the ability to return an internal server error on an unexpected type instead of throwing. * User generated evaluation errors: by using builtins.tryEval we can catch throws and asserts the user inserted when generating the body and report to the user that something went wrong. To do: also support for the headers. Change-Id: I8363b9825c6c730e624eb8016a5482d63cbc1890 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2849 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-05 01:53:21 +02:00
respond =
feat(web/bubblegum): allow passing status as an int The whole pass the name of the status as a string thing was mostly born out of an overeager use of yants. It is still very neat especially for common cases like "OK", so we'll keep it, but also allow passing the integer variant of the status as well which probably feels more natural for a lot of people, especially over getting the casing right for "I'm a teapot". Change-Id: I3f012a291447ef385efdd28132292a8b331998c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2850 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-05 01:58:26 +02:00
# response status as an integer (status code) or its
# textual representation in the HTTP protocol.
# See `statusCodes` for a list of valid options.
feat(web/bubblegum): report some errors to the user via HTTP We can actually catch some errors that may be generated in bubblegum applications where we can report them to the user in a way that doesn't require curl -vv: * Type errors in the status argument: By removing yants completely we not only (presumably) gain some performance, but also the ability to return an internal server error on an unexpected type instead of throwing. * User generated evaluation errors: by using builtins.tryEval we can catch throws and asserts the user inserted when generating the body and report to the user that something went wrong. To do: also support for the headers. Change-Id: I8363b9825c6c730e624eb8016a5482d63cbc1890 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2849 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-05 01:53:21 +02:00
statusArg:
# headers as an attribute set of strings
headers:
# response body as a string
bodyArg:
let
status =
feat(web/bubblegum): allow passing status as an int The whole pass the name of the status as a string thing was mostly born out of an overeager use of yants. It is still very neat especially for common cases like "OK", so we'll keep it, but also allow passing the integer variant of the status as well which probably feels more natural for a lot of people, especially over getting the casing right for "I'm a teapot". Change-Id: I3f012a291447ef385efdd28132292a8b331998c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2850 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-05 01:58:26 +02:00
if builtins.isInt statusArg
then {
code = statusArg;
line = lib.findFirst
(line: statusCodes."${line}" == statusArg)
null
(builtins.attrNames statusCodes);
} else if builtins.isString statusArg then {
feat(web/bubblegum): report some errors to the user via HTTP We can actually catch some errors that may be generated in bubblegum applications where we can report them to the user in a way that doesn't require curl -vv: * Type errors in the status argument: By removing yants completely we not only (presumably) gain some performance, but also the ability to return an internal server error on an unexpected type instead of throwing. * User generated evaluation errors: by using builtins.tryEval we can catch throws and asserts the user inserted when generating the body and report to the user that something went wrong. To do: also support for the headers. Change-Id: I8363b9825c6c730e624eb8016a5482d63cbc1890 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2849 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-05 01:53:21 +02:00
code = statusCodes."${statusArg}" or null;
line = statusArg;
} else {
code = null;
line = null;
};
renderedHeaders = lib.concatStrings
(lib.mapAttrsToList (n: v: "${n}: ${toString v}\r\n") headers);
internalError = msg: respond 500
{
Content-type = "text/plain";
} "bubblegum error: ${msg}";
body = builtins.tryEval bodyArg;
in
if status.code == null || status.line == null
then internalError "Invalid status ${lib.generators.toPretty {} statusArg}."
else if !body.success
then internalError "Unknown evaluation error in user code"
else
lib.concatStrings [
"Status: ${toString status.code} ${status.line}\r\n"
renderedHeaders
"\r\n"
body.value
];
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
/* Returns the value of the `SCRIPT_NAME` environment
variable used by CGI.
*/
scriptName = builtins.getEnv "SCRIPT_NAME";
/* Returns the value of the `PATH_INFO` environment
variable used by CGI. All cases that could be
considered as the CGI script's root (i. e.
`PATH_INFO` is empty or `/`) is mapped to `"/"`
for convenience.
*/
pathInfo =
let
p = builtins.getEnv "PATH_INFO";
in
if builtins.stringLength p == 0
then "/"
else p;
/* Helper function which converts a path from the
root of the CGI script (i. e. something which
could be the content of `PATH_INFO`) to an
absolute path from the web root by also
utilizing `scriptName`.
Type: string -> string
*/
feat(web/bubblegum): report some errors to the user via HTTP We can actually catch some errors that may be generated in bubblegum applications where we can report them to the user in a way that doesn't require curl -vv: * Type errors in the status argument: By removing yants completely we not only (presumably) gain some performance, but also the ability to return an internal server error on an unexpected type instead of throwing. * User generated evaluation errors: by using builtins.tryEval we can catch throws and asserts the user inserted when generating the body and report to the user that something went wrong. To do: also support for the headers. Change-Id: I8363b9825c6c730e624eb8016a5482d63cbc1890 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2849 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-05 01:53:21 +02:00
absolutePath = path:
if builtins.substring 0 1 path == "/"
then "${scriptName}${path}"
else "${scriptName}/${path}";
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
bins = getBins pkgs.coreutils [ "env" "tee" "cat" "printf" "chmod" ]
chore(nint): move from //users/sterni to //nix Since //web/bubblegum depends on nint, we need to move it to a non user directory to conform with the policy established via cl/3434. Note that this likely doesn't mean greater stability (which isn't really implied in depot anyways), since I still would like to use a more elaborate calling convention to allow for additional useful features. Change-Id: I616f905d8df13e3363674aab69a797b0d39fdd79 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3506 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 18:26:03 +02:00
// getBins nint [ "nint" ];
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
/* Type: args -> either path derivation string -> derivation
*/
writeCGI =
{
# if given sets the `PATH` to search for `nix-instantiate`
# Useful when using for example thttpd which unsets `PATH`
# in the CGI environment.
binPath ? ""
# name of the resulting derivation. Defaults to `baseNameOf`
# the input path or name of the input derivation.
# Must be given if the input is a string.
, name ? null
feat(nix/utils): add storePathName, a more generic baseNameOf This is a wrapper around baseNameOf which also can deal with derivations. Added to //nix/utils since I've found myself introducing an ad-hoc implementation of this for both //web/bubblegum and //nix/buildC. Change-Id: I2fcd97a150d6eda21ab323fa0d881ff7442a892e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3049 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-04-20 10:18:28 +02:00
, ...
}@args:
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
input:
let
drvName =
feat(nix/utils): add storePathName, a more generic baseNameOf This is a wrapper around baseNameOf which also can deal with derivations. Added to //nix/utils since I've found myself introducing an ad-hoc implementation of this for both //web/bubblegum and //nix/buildC. Change-Id: I2fcd97a150d6eda21ab323fa0d881ff7442a892e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3049 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-04-20 10:18:28 +02:00
if builtins.isString input || args ? name
then args.name
else utils.storePathName input;
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
script =
if builtins.isPath input || lib.isDerivation input
then input
else if builtins.isString input
then pkgs.writeText "${drvName}-source" input
else builtins.throw "Unsupported input: ${lib.generators.toPretty {} input}";
shebang = lib.concatStringsSep " " ([
"#!${bins.env}"
# use the slightly cursed /usr/bin/env -S which allows us
# to pass any number of arguments to our interpreter
# instead of maximum one using plain shebang which considers
# everything after the first space as the second argument.
"-S"
] ++ lib.optionals (builtins.stringLength binPath > 0) [
"PATH=${binPath}"
] ++ [
"${bins.nint}"
# always pass depot so scripts can use this library
refactor(web/bubblegum): only rebuild examples on dependency change Using sparseTree we can make a (surprisingly long) list of things from depot the examples depend on and create a stripped down depot version which only contains them. As a result the examples are no longer rebuilt on every commit. Change-Id: I3693570ca4bdbbf9da795e552f278f3b1b1b77a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3504 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 00:20:31 +02:00
"--arg depot '(import ${minimalDepot} {})'"
feat(web/bubblegum): nix CGI programming framework So here is what has been keeping me up at night: At some point I realized that nix actually made a somewhat passable language for CGI programming: * That `builtins.getEnv` exists as one of the impurities of Nix is perfect as environment variables are the main way of communication from the web server to the CGI application. * We can actually read from the filesystem via builtins.readDir and builtins.readFile with bearable overhead if we avoid importing the used paths into the nix store. * Templating and routing are convenient to implement via indented strings and attribute sets respectively. Of course there are obvious limitation: * The overhead of derivations is probably much to great for them to be useful via IfD. * Even without derivations, nix evaluation is very slow to the point were a trivial application takes between 100ms and 400ms to produce a response. * We can't really cause effects other than producing a response which makes it not viable for a lot of applications. There are some ways around this: * With a custom interpreter we could have streaming and multiplexed I/O (using lazy lists emulated via attrsets) to cause such effects, but it would probably perform terribly. * We can use builtins.fetchurl to call other HTTP-based microservices, but only in very limited constraints, i. e. only GET, no headers, and only if the tarball ttl is set to 0 in the global nix.conf. * Terrible error handling capabilities because builtins.tryEval actually doesn't catch a lot of errors. To prove that it actually works, there are some demo applications, which I invite you to run and potentially break horribly: nix-build -A web.bubblegum.examples && ./result # navigate to http://localhost:9000 The setup uses thttpd and executes the nix CGI scripts using users.sterni.nint which automatically passed `depot`, so they can import the cgi library. Change-Id: I3a22a749612211627e5f8301c31ec2e7a872812c Reviewed-on: https://cl.tvl.fyi/c/depot/+/2746 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-21 12:57:40 +01:00
]);
in
runExecline.local drvName { } [
"importas"
"out"
"out"
"pipeline"
[
"foreground"
[
"if"
[ bins.printf "%s\n" shebang ]
]
"if"
[ bins.cat script ]
]
"if"
[ bins.tee "$out" ]
"if"
[ bins.chmod "+x" "$out" ]
"exit"
"0"
];
in
{
inherit
respond
pathInfo
scriptName
absolutePath
writeCGI
;
}
Reference in a new issue Copy permalink