Lorri does not cleanly integrate with my corporate device, which cannot run
NixOS. To expose dependencies to Emacs buffers, I will use nix-buffer.el, which
reads its values from dir-locals.nix. To easily expose dependencies from my
existing shell.nix files into dir-locals.nix, I wrote a Nix utility function.
I'm not sure if this commit breaks everything in my monorepo. I think it
will.
Why am I doing this? Perhaps it's a bad idea. I don't fully understand how
readTree works. My ignorance is costing me hours of time spent debugging. In an
effort to better understand readTree, I'm removing the default values for my Nix
expression parameters, which I believe have preventing errors from surfacing.
- Move state "gen server" to the top of main/0
- Initialize it as empty
- Ensure that persistTokens/2 is called whenever the state changes
- Support setState/2 (similar in spirit to getState/0)
Problem:
When SIGINT signals we're sent to the token server, it would shut down without
completing the shutdown procedure. The shutdown procedure would persist the
application state (i.e. access and refresh tokens).
This is problematic for the following sequence of events:
t0. Access and refresh tokens retrieved from kv.json and used as app state.
t1. Tokens are refreshed but not persisted. (I'm still unsure how this
happens). Remember that this means the previous access and refresh tokens
from t0 are now invalid.
t2. User sends a SIGINT.
t3. Token server shuts down.
t4. Token server is restarted, kv.json is used as the app state even though its
tokens are now invalid.
t5. Tokens are attempted to refresh, Monzo API rejects the tokens because
they're invalid.
Now we need to provide the token server with valid access and refresh tokens
otherwise we will repeat the loop described above. This means going through the
client authorization flow again or copying and pasting the tokens logged from
the token server into kv.json. Either scenario is more manual than I'd prefer.
Solution:
Use a buffered channel to receive the os.Signal. I got this idea after reading
these docs: https://golang.org/pkg/os/signal/#Notify and I debugged this issue
shortly thereafter.
I also rearranged the order of operations in main/0 to ensure that
handleInterrupts/0, which registers the event listeners, occurs before
scheduleTokenRefresh/2 is called. This allows the token server to gracefully
shutdown even if it's in the middle of the scheduleTokenRefresh/2 call.
Consume the newly relocated auth package.
Additionally:
- Debugged error where JSON was properly decoding but not populating the
refreshTokenResponse struct, so my application was signaling false positive
messages about token refresh events.
- Logging more often and more data to help my troubleshooting
- Refreshing tokens as soon as the app starts just to be safe
- Clean up the code in general
Relocated the logic for authorizing clients into a separate package that the
tokens server now depends on. Moving this helped me separate concerns. I removed
a few top-level variables and tried to write more pure versions of the
authorization functions to avoid leaking Monzo-specific details.
Attempting to read the persisted tokens from the key-value store when the server
begins. The server currently fails when those values are empty.
TODO
- Consider adding logic for knowing if the cached tokens are expired and prompt
the user to reauthorize the client using a web browser.
- Created a gopkgs directory and registered it with default.nix's readTree
- Moved monzo_ynab/utils -> gopkgs
- Consumed utils.go in main.go
- Renamed monzo_ynab -> job
I created a server to manage my access and refresh tokens. This server exposes a
larger API than it needs to at the moment, but that should change. The goal is
to expose a GET at /token to retrieve a valid access token. The server should
take care of refreshing tokens before they expire and getting entirely new
tokens, should they become so stale that I need to re-authorize my application.
A lot of my development of this project has been clumsy. I'm new to Go; I didn't
understand OAuth2.0; I'm learning concurrent programming (outside of the context
of comfortable Elixir/Erlang).
My habits for writing programs in compiled languages feels amateurish. I find
myself dropping log.Println's all over the source code when I should be using
proper debugging tools like Delve and properly logging with things like
httputil.Dump{Request,Response}.
The application right now is in a transitional state. There is still plenty of
code in main.go that belongs in tokens.go. For instance, the client
authorization code belongs in the tokens server.
Another question I haven't answered is where is the monzo client that I can use
to make function calls like `monzo.Transactions` or `monzo.Accounts`?
The benefit of having a tokens server is that it allows me to maintain state of
the tokens while I'm developing. This way, I can stop and start main.go without
disturbing the state of the access tokens. Of course this isn't the primary
benefit, which is to abstract over the OAuth details and expose an API
that gives me an access token whenever I request one.
The first benefit that I listed could and perhaps should be solved by
introducing some simple persistence. I'd like to write the access tokens to disk
when I shutdown the tokens server and read them from disk when I start the
tokens server. This will come. I could have done this before introducing the
tokens server, and it would have saved me a few hours I think.
Where has my time gone? Mostly I've been re-authorizing my client
unnecessarily. This process is expensive because it opens a web browser, asks me
to enter my email address, sends me an email, I then click the link in that
email. Overall this takes maybe 1-3 minutes in total. Before my tokens server
existed, however, I was doing this about 10-20 times per hour. It's a little
disappointing that I didn't rectify this earlier. I'd like to remain vigilant
and avoid making similar workflow mistakes as I move ahead.
I've been using restclient.el and `restclient-mode` lately to test API calls,
and I'm enjoying. I think it might make sense to track these scratch files in
the repo. Who knows? They may serve as a form of documentation.
Define transaction structs for both Monzo and YNAB. Each package has a `main`
function that runs some shallow but preliminary round-trip tests for the
serializers and decoders.
The fixtures.json file that each of them is referencing has been ignored in case
either contains confidential data of which I'm unaware.
Define my YNAB personal access token as an environment variable. Prefix Monzo
environment variables with "monzo_" to more easily differentiate between Monzo
credentials and YNAB credentials.
From what I currently understand, lorri is a tool (sponsored by Target) that
uses nix and direnv to build and switch between environments quickly and
easily.
When you run `lorri init` inside of a directory, lorri creates a shell.nix and
an .envrc file. The .envrc file calls `eval "$(lorri direnv)"` and the shell.nix
calls `<nixpkgs>.mkShell`, which creates a shell environment exposing
dependencies on $PATH and environment variables. lorri uses direnv to ensure
that $PATH and the environment variables are available depending on your CWD.
lorri becomes especially powerful because of Emacs's `direnv-mode`, which
ensures that Emacs buffers can access anything exposed by direnv as well.
I still need to learn more about how lorri works and how it will affect my
workflow, but I'm enjoying what I've seen thus far, and I'm optimistic about the
road ahead.
After some toil and lots of learning, monzo_ynab is receiving access and refresh
tokens from Monzo. I can now use these tokens to fetch my transactions from the
past 24 hours and then forward them along to YNAB.
If YNAB's API requires OAuth 2.0 login flow for authorization, I should be able
to set that up in about an hour, which would be much faster than it took me to
setup the login flow for Monzo. Learning can be a powerful thing.
See the TODOs scattered around for a general idea of some (but not all) of the
work that remains.
TL;DR
- Package monzo_ynab with buildGo
- Move some utility functions to sibling packages
- Add a README with a project overview, installation instructions, and a brief
note about my ideas for deployment
Note: I have some outstanding questions about how to manage state in Go. Should
I use channels? Should I use a library? Are top-level variables enough? Answers
to some or all of these questions and more coming soon...
I discovered direnv's convenient `source_up` function today. I needed it to
inherit the values defined in ~/briefcase/.envrc, and it's working exactly as I
expected it would. What a fine piece of software direnv is.
