Problem:
When SIGINT signals we're sent to the token server, it would shut down without
completing the shutdown procedure. The shutdown procedure would persist the
application state (i.e. access and refresh tokens).
This is problematic for the following sequence of events:
t0. Access and refresh tokens retrieved from kv.json and used as app state.
t1. Tokens are refreshed but not persisted. (I'm still unsure how this
happens). Remember that this means the previous access and refresh tokens
from t0 are now invalid.
t2. User sends a SIGINT.
t3. Token server shuts down.
t4. Token server is restarted, kv.json is used as the app state even though its
tokens are now invalid.
t5. Tokens are attempted to refresh, Monzo API rejects the tokens because
they're invalid.
Now we need to provide the token server with valid access and refresh tokens
otherwise we will repeat the loop described above. This means going through the
client authorization flow again or copying and pasting the tokens logged from
the token server into kv.json. Either scenario is more manual than I'd prefer.
Solution:
Use a buffered channel to receive the os.Signal. I got this idea after reading
these docs: https://golang.org/pkg/os/signal/#Notify and I debugged this issue
shortly thereafter.
I also rearranged the order of operations in main/0 to ensure that
handleInterrupts/0, which registers the event listeners, occurs before
scheduleTokenRefresh/2 is called. This allows the token server to gracefully
shutdown even if it's in the middle of the scheduleTokenRefresh/2 call.
Consume the newly relocated auth package.
Additionally:
- Debugged error where JSON was properly decoding but not populating the
refreshTokenResponse struct, so my application was signaling false positive
messages about token refresh events.
- Logging more often and more data to help my troubleshooting
- Refreshing tokens as soon as the app starts just to be safe
- Clean up the code in general
Relocated the logic for authorizing clients into a separate package that the
tokens server now depends on. Moving this helped me separate concerns. I removed
a few top-level variables and tried to write more pure versions of the
authorization functions to avoid leaking Monzo-specific details.
I'm writing sensitive data here, so I'd like to ignore it instead of encrypting
it and publishing it. Perhaps later on, I can extend the key-value store to
handle encryption and decryption but that feels like overkill for now.
Attempting to read the persisted tokens from the key-value store when the server
begins. The server currently fails when those values are empty.
TODO
- Consider adding logic for knowing if the cached tokens are expired and prompt
the user to reauthorize the client using a web browser.
- Created a gopkgs directory and registered it with default.nix's readTree
- Moved monzo_ynab/utils -> gopkgs
- Consumed utils.go in main.go
- Renamed monzo_ynab -> job
In order to persist my access and refresh tokens, I needed a store. I think
using a database like SQLite may have been fine for this but was heavier weight
than what I wanted.
I decided to write a simple key-value store when the state is encoded and JSON
in a file called kv.json.
TODO:
- Support field nesting
- Support better error handling
- Support parameterizing the store path (i.e. ./kv.json)
I created a server to manage my access and refresh tokens. This server exposes a
larger API than it needs to at the moment, but that should change. The goal is
to expose a GET at /token to retrieve a valid access token. The server should
take care of refreshing tokens before they expire and getting entirely new
tokens, should they become so stale that I need to re-authorize my application.
A lot of my development of this project has been clumsy. I'm new to Go; I didn't
understand OAuth2.0; I'm learning concurrent programming (outside of the context
of comfortable Elixir/Erlang).
My habits for writing programs in compiled languages feels amateurish. I find
myself dropping log.Println's all over the source code when I should be using
proper debugging tools like Delve and properly logging with things like
httputil.Dump{Request,Response}.
The application right now is in a transitional state. There is still plenty of
code in main.go that belongs in tokens.go. For instance, the client
authorization code belongs in the tokens server.
Another question I haven't answered is where is the monzo client that I can use
to make function calls like `monzo.Transactions` or `monzo.Accounts`?
The benefit of having a tokens server is that it allows me to maintain state of
the tokens while I'm developing. This way, I can stop and start main.go without
disturbing the state of the access tokens. Of course this isn't the primary
benefit, which is to abstract over the OAuth details and expose an API
that gives me an access token whenever I request one.
The first benefit that I listed could and perhaps should be solved by
introducing some simple persistence. I'd like to write the access tokens to disk
when I shutdown the tokens server and read them from disk when I start the
tokens server. This will come. I could have done this before introducing the
tokens server, and it would have saved me a few hours I think.
Where has my time gone? Mostly I've been re-authorizing my client
unnecessarily. This process is expensive because it opens a web browser, asks me
to enter my email address, sends me an email, I then click the link in that
email. Overall this takes maybe 1-3 minutes in total. Before my tokens server
existed, however, I was doing this about 10-20 times per hour. It's a little
disappointing that I didn't rectify this earlier. I'd like to remain vigilant
and avoid making similar workflow mistakes as I move ahead.
I enjoyed using term-switcher so much that I ended up adopting vterm as my
primary terminal. After reaching for vterm as often as I did, I realized that I
would enjoy supporting cycling through instances, creating new instances,
deleting existing instances, renaming instances. Thus spawned vterm-mgt.el.
I'm particularly excited about the KBD to toggle between vterm instances and
source code buffers.
Supporting these functions was a little tricky. For example, how should we
handle calling cycle/remove on the item that is currently focused? After
attempting to be clever, I decided to just set the value to nil and let the
consumer decide what is best for them. I can always support a more opinionated
version that fallsback to previous-index if previous-index is set. But until I
have a better idea of how I'm going to consume this, I think nil is the best
option.
Add predicate for determining if a cycle contains items.
Updated cycle/{new,from-list} to support setting current-index to nil when a
consumer calls it with an empty list.
- generate_board: writing
- print_board: reading
- neighbords: reading
I'm working up to creating a function to initialize a game board where no three
adjacent cells either vertically or horizontally should be the same value.
Recently I've been asked a few interview questions that involve reading from or
writing to a grid, matrix, game board, etc. I am not as fast as I'd like to be
at this, so I'm going practice.
Here I'm practicing reading from existing matrices. I should practice writing to
empty boards, reading neigboring cells, wrapping around the board (in the case
of Conway's Game of Life), and other useful practices.
I've been using restclient.el and `restclient-mode` lately to test API calls,
and I'm enjoying. I think it might make sense to track these scratch files in
the repo. Who knows? They may serve as a form of documentation.
Define transaction structs for both Monzo and YNAB. Each package has a `main`
function that runs some shallow but preliminary round-trip tests for the
serializers and decoders.
The fixtures.json file that each of them is referencing has been ignored in case
either contains confidential data of which I'm unaware.
Define my YNAB personal access token as an environment variable. Prefix Monzo
environment variables with "monzo_" to more easily differentiate between Monzo
credentials and YNAB credentials.
I removed most of the packages that I install with `nix-env`. You can view these
with `nix-env --query`. This is one small step in a grander project to migrate
entirely to a declarative config managed by Nix.
This does two things:
1. Starts lorri daemon
2. Moves ssh-agent and docker daemon startup calls to ~/.profile
I'm still not entirely sure when ~/.profile is evaluated... I'd like to use
systemd to startup and manage these background services, but I currently don't
have a strong enough desire to do this.
dkish was an idea to quickly create REPLs for all sorts of languages like
Haskell, Elixir, Clojure. I haven't used these, and if I started wanting these
with my newfound comfort with Nix, I think I'd reach for that instead.
I'm in the midst of transitioning onto a few new tools.
My previous workflow just used `nix-env` to install *some* packages. I didn't
have a prescribed methodology for which packages I would install using `nix-env`
and which ones I would install using `sudo apt-get install`. Sometimes if a
package would be available in my aptitude repositories, I'd use that; other
times when it wasn't available I'd use `nix-env`. One complication about being
on gLinux intead of NixOS is that some packages (e.g. nixpkgs.terminator) is
available via `nix-env -iA nixpkgs.terminator`, but the installation won't
actually run on my gLinux. In these instances, I would install terminator from
the aptitude repositories.
Then @tazjin introduced me to his Emacs configuration that he builds using
Nix. What appealed to me about his built Emacs is that it worked as expected on
either a NixOS machine and on gLinux (and presumably on other non-NixOS machines
as well).
A setup towards which I'm working is to own one or a few NixOS machines whose
configurations are entirely managed with Nix. On devices like my work machines,
which cannot run NixOS, I can build as much of the software that I need using
Nix and attempt to minimize the ad hoc configuration either with shell scripts,
python, golang, or more Nix code... it's clear that I still don't have a clear
idea of how that part will work.
For now, I'm adopting nix, nix-env, lorri, direnv, and weening off of aptitude
as much as I can. Things are a bit messy, but my general trend feels
positive. Stay tuned for more updates.
From what I currently understand, lorri is a tool (sponsored by Target) that
uses nix and direnv to build and switch between environments quickly and
easily.
When you run `lorri init` inside of a directory, lorri creates a shell.nix and
an .envrc file. The .envrc file calls `eval "$(lorri direnv)"` and the shell.nix
calls `<nixpkgs>.mkShell`, which creates a shell environment exposing
dependencies on $PATH and environment variables. lorri uses direnv to ensure
that $PATH and the environment variables are available depending on your CWD.
lorri becomes especially powerful because of Emacs's `direnv-mode`, which
ensures that Emacs buffers can access anything exposed by direnv as well.
I still need to learn more about how lorri works and how it will affect my
workflow, but I'm enjoying what I've seen thus far, and I'm optimistic about the
road ahead.
actors.go is my attempt to better understand golang's channels. I'm mapping my
understanding of concurrency from my experience with Elixir / Erlang and actors
onto golang until I have more opinions.
Since I did not pass my one-site interview with DM, but I have been invited to
attempt again, I decided to partition this directory into two parts:
1. part_one: Hosting the exercises that I completed before my first attempt at
earning the job.
2. part_two: Hosting the exercise that I will complete before my second attempt
at earning the job.
After some toil and lots of learning, monzo_ynab is receiving access and refresh
tokens from Monzo. I can now use these tokens to fetch my transactions from the
past 24 hours and then forward them along to YNAB.
If YNAB's API requires OAuth 2.0 login flow for authorization, I should be able
to set that up in about an hour, which would be much faster than it took me to
setup the login flow for Monzo. Learning can be a powerful thing.
See the TODOs scattered around for a general idea of some (but not all) of the
work that remains.
TL;DR
- Package monzo_ynab with buildGo
- Move some utility functions to sibling packages
- Add a README with a project overview, installation instructions, and a brief
note about my ideas for deployment
Note: I have some outstanding questions about how to manage state in Go. Should
I use channels? Should I use a library? Are top-level variables enough? Answers
to some or all of these questions and more coming soon...
I discovered direnv's convenient `source_up` function today. I needed it to
inherit the values defined in ~/briefcase/.envrc, and it's working exactly as I
expected it would. What a fine piece of software direnv is.
I'm now pulling the authorization code off of Monzo's request to my redirect
URI. I intend to use exchange that code for an access and refresh token. Once I
have these two items, I should be able to interact with Monzo's API much more
easily.
- Prefer goimports to gofmt. goimports calls gofmt; it also adds and removes
dependencies.
- Assert the presence of goimports, godoc, godef
- KBD godef to M-.
- Support the M-x compile command for calling `go build -v`
Support a Mercurial alias for listing the files that have changed on a
particular branch.
This commit is particularly noisy because I reformatted the above aliases to
align with the new width.
What's done:
- Basic support of the client authorization grant stage of the OAuth login
flow:
- Open Google Chrome to point the user to Monzo's client authorization page.
- Created a web server to retrieve the authorization code from Monzo.
What's left:
- Pulling the authorization grant (i.e. code) from Monzo's request and
exchanging it for an access token and a refresh token, which can be used to
make subsequent requests.
Unanswered question:
- Assuming this is a stateless app, where should I store the access token and
refresh token to avoid the authorization flow. I'd like to avoid the client
authorization flow because ideally I could run this app as a job that runs
periodically throughout the day without requiring my interactions with it.
Some interesting notes:
- Notice how in the .envrc file, it's possible to make calls to `pass`. This
allows me to check in the .envrc files without obscuring their content. It
also allows me to consume these values in my app by using
`os.Getenv("client_secret")`, which I find straightforward. Overall, I'm quite
pleased to have stumbled upon this pattern - assuming that it's secure.
I mistakenly mapped one of my dual-function keys on my Ergodox to send Shift+CMD
instead of CMD. When some of my Emacs keybindings weren't firing, I noticed that
the key event they received was some like `C-S-s-<char>` instead of say
`C-s-<char>`. As a quick fix, I duplicated each of my keybindings that relied on
the CMD key to support Shift+CMD as well until I remapped the key on my
Ergodox. This morning, I remapped the Shift+CMD key to CMD, so I'm bidding adieu
to this code.
Today I learned that you can email your Kindle files to read them using the
paperwhite display. I'm attempting to read RFCs, so after reading 1/4 of the way
through RFC6479 (on OAuth2.0), I realized that it might be easier to read on my
Kindle instead of on my computer screen. Out of this, rfcToKindle.go was born.
I'm not sure if I'd like to publish this or not.
Press `<M-escape.` to display a list of buffers hosting X applications. Use
`completing-read` to select and focus one of these.
See the function docs and TODOs for more information.
Currently, after I connect my monitor to my laptop, I run `display/enable-4k`,
which will use `xrandr` to enable the display. The scaling of the enabled
display is not what I expect. So I've habituated re-running the same function,
`display/enable-4k`, which scales the display and meets my expectations.
What's strange is that if instead of running `display/enable-4k` the first time
from Emacs, I call `xrandr ...` from a terminal, this enables the display and
scales it properly on the first invocation.
I'm unsure how to explain this behavior. It's possible that a environment
variable is set properly in the terminal that isn't set in my Emacs, but this is
just a guess.
I'm going to using a different invocation in display.el that explicitly passes
the monitors dimensions. Let's see if that works.
I'm currently quite unfamiliar with golang. As an exercise to help me onboard
onto golang, and as a proof-of-concept to see if golang is a viable substitute
for Python as a scripting language, I decided to port my delete_dotfile_symlinks
to golang.
In the process, renamed ./python -> ./scripts, which is a more accommodating
name for a directory.
