Commit graph

15238 commits

Author SHA1 Message Date
William Carroll
4ff1ea291c Drop support for ServantT transformer type for server
After burning a few hours wrestling with the type system, I decided to revert to
the simpler `Server API` type instead of the `ServantT` transformer type.

The problem is that I couldn't write a MonadError instance for `RIO Context`,
which is my `AppM` (i.e. application monad). Using `throwIO` in the server
handlers results in 500 errors, which is not what I wanted. I'm still pretty
fuzzy about what's happening; I now know that exception handling in Haskell is
pretty gnaryly. I may revisit this at a later time when my knowledge is more
extensive. For now: time to fry bigger fish.

An easier abstract is for me to pass `T.Context` into `server` as an argument,
which after all is what a Reader does.

TL;DR:
- Read server, client ports from .envrc
- Define a top-level Failure type (empty for now)
- Define a top-level Success type
- Define App as RIO Context (Either Failure Success)
2020-08-10 15:02:05 +01:00
William Carroll
f61ed25755 Prefer ServantT for server to consume App context
Long story -> short: I'd like to access my App monad from within my Servant
handlers.

While this code type-checks, I'm not sure it's working as intended. Needing to
change throwError to throwIO fails the "smell test". I expect to refactor this
code, but I'm calling it a night for now.
2020-08-09 23:15:12 +01:00
William Carroll
bbcd0bf27d Replace Prelude with RIO
I believe RIO stands for: "ReaderT <something-something> IO", which is a nod to
the top-level application data type:

```haskell
-- This is a simplification
newtype RIO env a = RIO { runRIO :: ReaderT env a () }
```

I read about RIO from an FP-Complete blog post a few months ago, and now I'm
excited to try it out for a real project. Bon voyage!
2020-08-09 22:17:19 +01:00
William Carroll
7d85ba559d Move Haskell-related shell.nix code into its own shell.nix
I'm getting tired of:

```shell
$ cd <project-root>
$ nix-shell
$ cd src/server
$ ghci Main.hs
```

Instead:

```shell
$ cd <project-root>/src/server
$ ghci Main.hs
```
2020-08-09 22:11:39 +01:00
multi
5e58c8bc28 feat(whitby): add multi
Change-Id: Ibfc2a5fcf73099b8414b8c46958007374d14fd0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1701
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: isomer <isomer@tvl.fyi>
2020-08-09 19:42:07 +00:00
glittershark
1fd633dffd fix(web/panettone): revert "implement shorthand issue URLs"
This reverts commit 3115113854.

Reason for revert: this is causing all issues to return a 404 - reverting until we can get it working.

Change-Id: I5f3c5ec3b24f245a1f7ef12645200d16ed0f1b35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1721
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2020-08-09 19:02:07 +00:00
Griffin Smith
d1653533a6 test(tvix): Cover scanForReferences in a test
Aded a few test cases covering the scanForReferences function, which had
been accidentally broken in 976a36c (which is now partially-reverted).
As part of this, since the test needed to generate hashes for store
paths, the logic in MakeStorePath to compress a sha256 hash down to 20
bytes and convert it to base32 has been extracted to a member function
on the Hash class.

Fixes: #34
Change-Id: Ie2d914688a80f42d0234d351a7cc0714fd15709e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1698
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 17:26:19 +00:00
multi
dfab786653 feat(users/multi): init, add SSH keys
Change-Id: I93d6733595c96e0d4e67281e4781e2e0ff424a63
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1700
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-09 17:20:43 +00:00
multi
c0baddeed0 feat(tvl-slapd): add multi
Change-Id: I2fec9b5dd92da0343426c4a129d882fa87d92e6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1699
Reviewed-by: cynthia <cynthia@tvl.fyi>
Reviewed-by: eta <eta@theta.eu.org>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: lukegb <lukegb@tvl.fyi>
Tested-by: BuildkiteCI
2020-08-09 15:44:04 +00:00
William Carroll
26b7237aab Sketch database schema
Defining a few tables in init.sql to sketch a few records that I need to
persist.
2020-08-09 10:22:12 +01:00
William Carroll
9df2b49afd Initialize a default.nix for nix-build
As the previous commit mentions, I'm attempting to build and deploy this project
with `nix-shell` and `nix-build` instead of `cabal` and `stack`.

I'm in the Hamburg airport right now, and my internet connection isn't stable
enough to test this, so I'm committing it until I can more robustly test it.
2020-08-09 10:19:36 +01:00
William Carroll
119c8e9df9 Add common language extensions to .ghci
I'd like to see if I can avoid using `cabal` and `stack` and build and deploy
this application using `nix-shell` and `nix-build` only. Let's see how that
goes.
2020-08-09 10:18:46 +01:00
Griffin Smith
ec7d368546 fix(tvix): Make nix value structs inherit from GC
All of the miscellanious structs that make up a nix Value should inherit
from gc, as they contain pointers to GC'ed things as members

Fixes: #42
Change-Id: I057d8c9f7dafbee7de7644ff152c9cba1aa6bc03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1696
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 02:22:44 +00:00
Griffin Smith
059468ba15 refactor(tvix): Remove UDSRemoteStore
Now that we've fully implemented the RPC-based store client, we can get
rid of the UDSRemoteStore, whose only use was connecting to the locally
running nix daemon. The RemoteStore still needs to be around to connect
to remote upstream nix stores over SSH.

Change-Id: I0699819803cbfe966b9a46786f2c927d8e4bf1a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1693
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 02:22:44 +00:00
Griffin Smith
e440f60b6c feat(tvix): Implement all remaining RPC calls
Implement all remaining RPC calls on the RpcSstore client, remove a few
stub methods we had added that weren't actually present in the old
RemoteStore implementation, and add one more RPC call for getBuildLog
that is present in the store API, but that we hadn't added as a
stub *or* to the proto.

Fixes: #29
Change-Id: Id827f51a393ece4bc7bbecaf38aee9eb4b329770
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1692
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
2020-08-09 02:22:44 +00:00
Griffin Smith
747dc65154 refactor(tvix): Factor-out proto utilities
Factor out the shared utilities for interacting with protobufs/grpc from
libstore to a proto.hh header in libproto.

Change-Id: I1cb8d94867d5d4b63a9994be0b53f8f612eb8e3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1691
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 01:52:59 +00:00
Griffin Smith
ef7894273b feat(tvix): Pass function name to SuccessOrThrow
To aid in debugging RPC call failures, pass the name of the function
being called to the SuccessOrThrow util funcion in the RpcStore

Change-Id: I523dacfab896b85a3dbe6050c07ee6bd5906fa44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1690
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 01:17:09 +00:00
Kane York
1de00e6c42 chore(3p/nix): apply google-readability-casting
Command run: jq <compile_commands.json -r 'map(.file)|.[]' | grep -v '/generated/' | parallel clang-tidy -p compile_commands.json -checks=-*,google-readability-casting --fix

Manual fixes applied in src/nix-env/nix-env.cc, src/libstore/store-api.cc

Change-Id: I406b4be9368c557ca59329bf6f7002704e955f8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1557
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-08 22:16:02 +00:00
Griffin Smith
053a138002 fix(tvix): Wrap remaining RPCs in HandleExceptions
Wrap the BuildPaths and AddTextToStore RPC handlers in HandleExceptions.
These were missed in the original pass due to a merge.

Change-Id: Ie5be45e6098fba7a2b6b1c1be81578cb742c2880
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1689
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-08 20:03:59 +00:00
edef
3115113854 feat(web/panettone): implement shorthand issue URLs
Fix #32

Change-Id: I6ccec959201673850b4b56a44734a2874aad5856
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1648
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
2020-08-08 18:23:53 +00:00
edef
3049f31d28 chore(3p/nix): don't optional-wrap EvalState::file_access_trace_fn further
std::function has a natural null we can't eliminate anyway, so this was
effectively std::optional<std::optional<non_nullable_function>>.

Change-Id: If99f139146021edb25d133dad7f0f6e125ef53df
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1688
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-08 18:01:11 +00:00
Griffin Smith
7db734afad feat(tvix): Implement AddToStoreNar
Implement both the client and server sides of AddToStoreNar, using a
templated generalization of the sources and sinks we were using for
AddToStore on both ends.

Change-Id: I73d0ed34118c711b125851dff99a7518ced4af35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1686
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-08 17:51:13 +00:00
William Carroll
e8f35f0d10 Consume GoogleSignIn.validateJWT
TL;DR:
- Consume GoogleSignIn.validateJWT in the Handler for /verify
- Rename validation fn to validateJWT
- Prefer Text to String type
2020-08-08 17:57:11 +01:00
William Carroll
8a7a3b29a9 Add tests for "exp" field of the JWT
Assert that the exp field of the JWT is "fresh".
2020-08-08 14:47:39 +01:00
William Carroll
f1883b2790 Test that the JWT's iss field meets our expectations
The JWT should match "accounts.google.com" or "https://accounts.google.com". If
it doesn't, we produce a validation error.

TL;DR:
- Group all failed stringOrURI function calls as StringOrURIParseFailure errors
2020-08-08 14:08:11 +01:00
William Carroll
526728eb89 Test that an improperly encoded JWT returns a DecodeError
The subject of this commit message says it all.
2020-08-08 13:46:57 +01:00
William Carroll
d34b146702 Tests valid and invalid JWTs for the "aud" field
Test that when the JWT contains the client ID for my Google app, the JWT is
valid, and when it doesn't, it's invalid.
2020-08-08 13:44:22 +01:00
William Carroll
926d8e643e Update jwtIsValid API to return IO Bool
I need IO for:
- Getting the current time to validate `exp`
- Making an HTTP request to Google's token verifier endpoint
2020-08-08 11:18:49 +01:00
Kane York
cc01059d40 docs(sparse): Add RFC design for depot worktree tool
This is meant to be a high-velocity alternative to writing an entire
FUSE filesystem to facilitate smaller checkouts of the depot.

The script mostly works today, except that remote dependency detection is not set up.

Change-Id: Idf9048df4f7ee39eaaedcdafcd9621f3d7479947
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1644
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-08 10:11:27 +00:00
William Carroll
3eaf6e5aea Remove redundant deps from API.hs
Thank you, -Wall. You are truly an unsung hero.
2020-08-08 11:10:28 +01:00
William Carroll
7b8ec4170a Begin work for supporting GoogleSignIn server-side
I'm attempting to be an obedient boy and implement this and future features
using TDD.

TL;DR:
- Defined a few tests
- Defined an empty GoogleSignIn module
- Defined a Fixtures module to quickly create JWTs to test
2020-08-08 11:10:19 +01:00
William Carroll
9dcbd0d067 Define Utils module
Dumping grounds for personal, stylistic functions intended to improve readabily
and writability (in that order).
2020-08-08 11:06:53 +01:00
William Carroll
a7ddb56b9b Support echo server to test POST /verify
TL;DR:
- Add common dependencies like Servant, Aeson, Warp, Cors
- Define a POST /verify endpoint for our client to hit
- POST to /verify client-side onSignIn
2020-08-06 22:23:06 +01:00
William Carroll
1fc1087014 Support Google Sign-in client-side
TODO: Support Google Sign-in server-side

Also:
- Add Haskell to project's shell.nix
- Add stubbed Main.hs and Spec.hs
- Add common .ghci file
2020-08-06 21:54:25 +01:00
Kane York
a5dae62e85 fix(3p/nix): Use SkipEmpty in all calls to absl::StrSplit
The behavior to return a list containing a single empty string when
provided an empty string is a behavior that absl inherited from legacy
code. However, the behavior expected by legacy code in Nix is the
behavior provided by the SkipEmpty option. Switch all calls to use
SkipEmpty, except for the call already using SkipWhitespace.

See also commit 26a59482d2, with the
partly-prophetic message: "there may be other places we need to
fix this as well."

Change-Id: I6e94856a12cfb1b7e4a3b4e221769ed446648861
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1687
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 20:50:47 +00:00
Vincent Ambo
6a97206ceb refactor(tvix): Use absl::btree_map for BasicDerivation's env
Change-Id: I111a9a268debea322f23fdced3bed9ff3e8ed3b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1685
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-06 02:49:58 +00:00
Vincent Ambo
3ec3661678 refactor(tvix): Use absl::btree_map for DerivationInputs
Change-Id: If160ab1b09161969d9080d5d0d6f82e44a953e3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1684
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-06 02:30:28 +00:00
Vincent Ambo
91bd7ce73a refactor(tvix): Use absl::btree_map for DerivationOutputs
This container implementation is much faster than std::map. We have
stuck to an ordered container because it's unclear whether the
accesses of this field (of which there are *many*) are actually
ordering dependent.

Also includes an Arbitrary implementation for absl::btree_map (for any
K, V that are also Arbitrary).

Change-Id: I04f58ca0ce32b9ae1759313b01508b0e44bae793
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1683
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-06 02:20:36 +00:00
Vincent Ambo
a41c3dedb1 chore(tvix): Disable sandboxing by default until its time comes
Change-Id: Ifcc776841924abcb43fb115f3de569fe500756c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1682
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-06 02:20:36 +00:00
Vincent Ambo
820a4fd46d feat(tazjin/frog): Install sourcetrail
Change-Id: I4858a1e374162ea1cf35aa906e0d723fe8c2c392
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1681
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-08-06 02:03:36 +00:00
Griffin Smith
79052a61cc docs: Add the RULE
Change-Id: I90eddabb7e5e917524b300d5dfe86ddab4271432
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1660
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-08-06 00:23:13 +00:00
Vincent Ambo
81f09b2dcd fix(whitby): Increase nrBuildUsers to 128
Change-Id: I3a444e163745d17d10f923c0be7565840937c53a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1662
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 00:21:23 +00:00
Vincent Ambo
bc1293b944 fix(whitby): I'm a trusted user, owo
Change-Id: I2666b3cf8bdefcb5d4caeddf191dc65f6a8cb05f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1661
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 00:21:23 +00:00
Vincent Ambo
d42c3dd72f chore(3p): Bump nixpkgs channels
Bumps both nixos-unstable and nixos-20.03 to today's versions, as per
status.nixos.org

Contains minor fixes to things that broke because of the update:

* tazjin/frog: hardware.u2f is a deprecated setting
* glittershark/system: modSha256 in Go modules is now vendorSha256
* glittershark/owothia: removed version constraint on relude

Change-Id: Ib3e9612b1b06ed547b90e4f8b0ffe5ed7fe0a5c4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1642
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 00:15:33 +00:00
William Carroll
5f52077492 Re-type type using the altered keyboard
Remember: always read the instructions; that's the most important part.
2020-08-06 00:18:44 +01:00
William Carroll
e14fff7d4b Support Transforms.optimize
Partially optimize inputs and document rules for further optimizations we can
make.
2020-08-06 00:15:31 +01:00
Kane York
68b5306c56 feat(3p/nix): add --trace-file-access to nix-instantiate
This builds on edef's work with depot-scan by adding a dedicated
flag to the command. We piggyback on upstream's restricted-mode
implementation, the checkSourcePath function.

Change-Id: I52bb613549f40dbca1e8caa036635910c1a3d6d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1654
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 22:49:22 +00:00
William Carroll
d45685e245 Apply a series of transformation to a QWERTY keyboard
TL;DR:
- Accept input from the CLI
- Add a project README.md
2020-08-05 23:36:04 +01:00
edef
6a128fc162 chore(whitby): add edef
Change-Id: I7265259bc87594bd481c7bd455187c09b1effd1c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1650
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-05 22:32:54 +00:00
edef
e6ccbc9f35 fix(tools/depot-build): handle multiple levels of subdirectories
Change-Id: I99511c17d635b0222e1c933432007e6f8ea8cf85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1655
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-08-05 22:31:47 +00:00