Commit graph

1362 commits

Author SHA1 Message Date
William Carroll
098ca0da15 Ensure that the build step "depends on" the lint step
This way, if the lint step fails, the build step doesn't run. Nice!
2020-08-22 23:27:44 +01:00
William Carroll
3f9352b118 Remove --add-provider step from briefcase lint
So it turns out that I was wrong and that .git/config is stateful. Multiple
calls to --add-provider will append the same provider each time...

Instead I'm defining secret-patterns.txt and version-controlling it.

Then:
- dev-side: I'm adding `providers = cat ci/secret-patterns.txt` to .git/config
- ci-side:  I'm adding `providers = cat ci/secret-patterns.txt` to .git/config

Unfortunately this is ad-hoc configuration ci-side, which I would like to
avoid. The good news is that my pre-commit hooks and failures from git-secrets
should now align with my CI, since they're both reading from
secret-patterns.txt. One step backwards... two steps forwards?
2020-08-22 23:27:44 +01:00
William Carroll
92fa785e75 Call --add-provider during lint stage
I'm also `cat .git/config` because I think the Buildkite destroys the
.git/config file for each build, but I want to verify that. If it does, I prefer
that because it seems to share the spirit of the "Destroy Your Darlings" essay.
2020-08-22 23:27:44 +01:00
William Carroll
aeeaed605e Log git information during briefcase's lint stage
I would like to find out what the state of the repo is during pre-receive hook.
2020-08-22 23:27:44 +01:00
William Carroll
0644e1f740 Replace build badge
Changed pipelines = new badge.
2020-08-22 23:27:44 +01:00
William Carroll
fa645acf5c Prefer :nix: emoji
Buildkite support language extensions as emojis!
2020-08-21 14:00:11 +01:00
William Carroll
36399df006 Use emojis for build, lint steps
Y'know... the important stuff
2020-08-21 13:51:47 +01:00
William Carroll
fd626d907e Remove debugging information
Problem: my dev machine returns a different value for `git config --get-all
secrets.patterns` than my CI machine... I ran `git-secrets --register-aws` to
get additional coverage, but it's still not the same. I created an issue on the
git-secrets GH repo to get better troubleshooting advice, but I don't need the
logging info. anymore, so I'm removing it.
2020-08-21 12:31:03 +01:00
William Carroll
56e4d9f9b5 Debugging briefcase pipeline
Somehow `git-secrets --scan-history` is exiting non-zero, when I don't think it
should. Logging some environment information to get a better idea of what's
going on.
2020-08-21 11:25:16 +01:00
William Carroll
f53b08e4a0 Call --scan-history
My current pipeline is succeeding with a false-positive. After this change, it
should return a true-negative.
2020-08-21 11:22:52 +01:00
William Carroll
3f3cba7c32 Define BuildKite pipelines in Nix
After a handful of failed attempts to run lint-secrets.sh due to a missing
`git-secrets` executable on my git server, I decided that now was a good time to
use Nix to define my BuildKite pipelines.

TL;DR:
- Delete ci/scripts directory
- Define ci/pipelines/{briefcase,socrates}.nix

Outside of this repository:
- I logged into my admin account at git.wpcarro.dev and changed my Gitea
  post-receive hook to trigger the briefcase pipeline
- I logged into my BuildKite account, deleted my build-briefcase pipeline,
  created a new briefcase pipeline that called:

  ```shell
  nix-build -A ci.pipelines.briefcase -o briefcase.yaml
  buildkite-agent pipeline upload briefcase.yaml
  ```

One day I will audit all of my ad-hoc, non-mono-repo activity (like the steps I
listed above) and attempt to fit everything herein... one step at a time,
though!
2020-08-21 11:12:57 +01:00
William Carroll
13daa560ba Testing new CI lint-secrets step
Adding a fake secret to test to the new CI build step. I'm not sure I expect
this to fail the step because it relies on a pattern that I defined in
.git/config... let's see!
2020-08-20 23:32:16 +01:00
William Carroll
5b6862ea39 Call git secret hide whenever //secrets.json is saved
Having `git secret hide` as a pre-commit hook doesn't make much sense to me. I
will detail why when/if I write a blog post on briefcase's secret mgt setup.

The problem is, if I change secrets.json and then run `git status`, I won't see
any pending changes. This is because secrets.json is gitignore'd. If I run `git
secret hide` everytime I save secrets.json, I can rest assured that my `git
status` will be consistent with any updates to secrets.json.
2020-08-20 18:41:39 +01:00
William Carroll
17c68d654b Prefer reading secrets.json to using pass show
I'm attempting to maintain a top-level secrets.json that defines all of the
sensitive data that I'd like to version-control without exposing everything in
cleartext to the world. To that end, I'm using `git secret`, which will use
`gpg` to encrypt secrets.json everytime I call `git secret hide` and decrypt
everytime I call `git secret reveal`.

I'm going to try this until I don't like it anymore... if that day comes...

I should write a blog post about my setup to solicit useful feedback and share
my ideas with others.
2020-08-20 18:31:37 +01:00
William Carroll
392832a1ca Testing git-secret
Adding a dummy, top-level secrets.json file using `git-secret`. It might be nice
to have a mono-secrets file in json because then I can use it with `jq` like:

```shell
$ jq '.secret' --join-output < ~/briefcase/secrets.json
```
2020-08-20 14:50:53 +01:00
William Carroll
d71776a9ae Remove 2x-newlines from .gitignore
I saw an issue on GitHub that claims that git-secret doesn't like 2x-newlines in
.gitignore files. Let's see if that helps...
2020-08-20 12:18:35 +01:00
William Carroll
cbaf74de0e Setup git-secret
This morning I'm attempting to secure my monorepo. How?

- `git secret`:  DONE: To version-control sensitive data
- `git secrets`: TODO: Lint code for sensitive data

I will probably update the CI to call `git secrets --scan` or some similar
command to fail when that exists non-zero. I have much to learn, but doing is
the best way to learn it.
2020-08-20 11:53:14 +01:00
William Carroll
4d8ce34a7e Simplify EXWM init hook
Anytime something before or during window-manager.el fails to evaluate,
I lose the ability to type, but I *can* still click.

@tazjin recommended that I use the mouse to cycle to the *Warnings*
buffer, which led me to another bug in a series of bugs that I'm
uncovering: ~/briefcase/org didn't exist.

A simple mistake like this should break my WM startup, so I decided to
remove most of my init hook logic.
2020-08-20 11:26:32 +01:00
William Carroll
c9da7c5214 Add XMODIFIERS=emacs
This fixes the latest segfault I encountered after
/usr/bin/{google-emacs,emacs} was updated...
2020-08-20 11:26:32 +01:00
William Carroll
9d7331d63b Debug evil-want-keybindings issue
Problem: dependency loading order

I originally assumed that keybindings.el was the first module to `require
'evil` because init.el shows:

```elisp
(require 'keybindings)
(require 'window-manager)
```

The problem is that keybindings.el calls `require 'window-manager` and
window-manager.el requires evil! I admit, I've created a bit of a birds nest for
myself.

A few thoughts:
- keybindings.el doesn't need to `require 'window-manager`. Fixed!
- window-manager.el shouldn't need to `require 'evil`. TODO...
2020-08-20 11:26:32 +01:00
William Carroll
53be40b1a6 Drop use-package in keybindings.el
I'm attempting to kill that zombie bug about evil-want-keybinding...
2020-08-20 11:26:32 +01:00
William Carroll
39622607ff Add missing dependencies to emacs/default.nix
While debugging some broken Emacs config, I ran Emacs in X, where for some
reason my PATH doesn't have my nix-env dependencies... because of this, when I
call `~/.nix-profile/bin/wpcarros-emacs` to start my Emacs, I saw warnings about
missing packages that I hadn't seen before. Nice!
2020-08-20 11:26:32 +01:00
William Carroll
3dc7f7758b Drop support for wpc-keybindings
In favor of keybindings.el!

Now I have:
- kbd.el: There are no keybindings in this file. It's just a library for working
  with keybindings in Emacs.
- keybindings.el: (hopefully) all of my keybindings for EXWM, evil, etc.
2020-08-20 11:26:32 +01:00
William Carroll
4ae6eff607 Remove unused kbd/install-kbds?
In another refactor, I'd like to move all ad-hoc keybindings out of individual
modules and into keybindings.el.
2020-08-20 11:26:32 +01:00
William Carroll
23cf5d43d8 Centralize <SPC> in normal mode KBDs
Merging keybinding and wpc-keybindings step-by-step...
2020-08-20 11:26:32 +01:00
William Carroll
108da34dd9 Delete unused KBDs
Now that everything is in my monorepo, it's easy for me to use <SPC>jd to search
for these files.
2020-08-20 11:26:32 +01:00
William Carroll
29b1694a76 Add --no-out-link to ci/scripts
I don't need the ./result symlinks...
2020-08-20 11:26:32 +01:00
William Carroll
784de28be4 Remove <unstable> from briefcase
I don't use this anywhere, so it's time to shed more weight.
2020-08-20 11:26:32 +01:00
William Carroll
33890d8a8b Move scratch/brilliant into //assessments
Where it belongs...
2020-08-20 11:26:32 +01:00
William Carroll
0c71fc9d1d Drop support for dir-locals.nix, <nixpkgs>, etc.
In the spirit of Marie Kondo, I'm tidying up!

TL;DR:
- Prefer .envrc `use_nix` and delete all dir-locals.nix files
- Remove ~all references to <nixpkgs>, <unstable>, <depot> and prefer
  referencing each with briefcase.third_party.{pkgs,unstable,depot}
- Delete nixBufferFromShell function since I was only using that in
  dir-locals.nix files
2020-08-20 11:26:32 +01:00
William Carroll
51ec9e0d88 Move /home/wpcarro/nixpkgs-channels to /var/lib
My builds are still failing. This time with...
```
error: getting status of /home/wpcarro/nixpkgs-channels: Permission denied
```
...what confused me was the following:

```shell
$ sudo -u buildkite-agent-socrates stat /home/wpcarro/nixpkgs-channels
permission denied
```

But `ls -al /home/wpcarro | grep nixpkgs-channels` showed `r-w` for all users...

Thankfully @riking on ##tvl told me that I should check the permissions for
/home/wpcarro and /home...

After running `ls -al /home`, I saw `---` for all user... I then reproduced the
error by running:

```shell
$ sudo -u buildkite-agent-socrates stat /home
permission denied
```

Great!

So then I moved nixpkgs-channels to /var/lib/buildkite-agent-socrates. @edef
recommended that I read more about DynamicUser= setting for systemd, which looks
relevant after I took a cursory glance.

I'll also want a more declarative way to manager this, but I'm making small
improvements every day.
2020-08-20 11:26:32 +01:00
William Carroll
79904056af Move buildkite's SSH key out of /home/wpcarro into /etc/ssh
After enabling buildkite-agent using NixOS, it runs as its own user,
buildkite-agent-socrates, which does not have its own home directory. I moved
the SSH key that I made when running buildkite-agent as wpcarro into /etc/ssh
and `chown`'d it for buildkite-agent-socrates.
2020-08-20 11:26:32 +01:00
William Carroll
60b8b83376 Enable services.buildkite-agents
Instead of enabling `buildkite-agent` ad hoc, use NixOS to configure it.
2020-08-20 11:26:32 +01:00
William Carroll
890d4606cf Add CI build status badge to top-level README
Wahoo!
2020-08-20 11:26:32 +01:00
William Carroll
42efb3b08a Support build-briefcase.sh
For now, I'm supporting two CI pipelines:
- build-socrates
- build-briefcase

Conceptually, build-briefcase should cover what build-socrates does now, but
eventually I would like build-socrates to call `switch-to-configuration` so that
all of my websites, etc. stay fresh.
2020-08-20 11:26:32 +01:00
William Carroll
1bb32b1bcc Disable failing goals/default.nix
Disabling failing packages until I can get a working CI build.
2020-08-20 11:26:31 +01:00
William Carroll
59f7481411 Revise previous opinions about absolute paths GT <bracket-notation>
Unforeseen problem: `buildkite-agent` runs its builds in a separate directory,
so if I want the `nix-build` command to build the newly checked out code, I need
to set <briefcase> to the CWD.
2020-08-20 11:26:31 +01:00
William Carroll
7833632a79 Attempt nix-build instead of nixos-rebuild switch
I've encountered a few problems with attempting to support nixos-rebuild:
- the activation step requires `sudo` privileges
- the `buildkite-agent` runs on the same machine, socrates, that is rebuilding
  itself. This means that when the activation step runs, it will attempt to
  restart `buildkite-agent` when the agent is still working

I'm slowly removing places in my nix code that rely on '<bracket>' notation, so
that I no longer depend on NIX_PATH being set. I still have more work to do.

{covid-uk,sandbox}/default.nix are breaking when I attempt to run my
build-socrates.sh script locally, so I'm temporarily disabling them until I can
get CI working as I expect.
2020-08-20 11:26:31 +01:00
William Carroll
51cdb87223 Prefer nixos-rebuild to the rebuild script
The rebuild script calls sudo, which I won't need as I test running
buildkite-agent prefixed with `sudo` or as the root user.
2020-08-20 11:26:31 +01:00
William Carroll
0a0b09c714 Define buildkite-agent user
I'd like my buildkite-agent to run as its own user, so I'm defining that user
here.
2020-08-20 11:26:31 +01:00
William Carroll
2bf3c6c926 Log all polkit actions to find action.id for nixos-rebuild
I would like to setup a polkit rule to allow `buildkite-agent` (i.e. a
forthcoming user) to call `nixos-rebuild`. I need to know the `action.id` before
I can write a reliable rule.
2020-08-20 11:26:31 +01:00
William Carroll
9a6d2c3343 Debug $USER in build-socrates.sh
Attempting to see what $USER the buildkite-agent is when it runs.
2020-08-20 11:26:31 +01:00
William Carroll
d56b2210cd Debug build-socrates.sh
- using `set -euo pipefail` for setting recommended failure-modes
- using `set -x` and `echo "$PATH"` to debug my failing build

Sidenote: I find BuildKite's documentation quite helpful!
2020-08-20 11:26:31 +01:00
William Carroll
208883243f Attempt to build Socrates using BuildKite
Let's see what happens...
2020-08-20 11:26:31 +01:00
William Carroll
334616392b Set -Wall and cleanup warnings
I should have done this from the start.
2020-08-20 11:26:31 +01:00
William Carroll
6c0777aada Create an assessment directory
I've been doing a few take-home assessment recently, all of which I've attempted
to solve using Haskell. I'm having a good time, and I'm noticing strong and weak
points with my Haskell programming. I always attempt to apply any feedback a
reviewer gives me, and I'm storing my first drafts, second attempts, and
feedback here for now.

This recently attempt was for a role at Jane Street.
2020-08-20 11:26:31 +01:00
William Carroll
f032eee79d List directories first in dired view
Ideally I'd use exa insted of ls, but I cannot seem to support that yet.
2020-08-20 11:26:31 +01:00
William Carroll
375c4a4c85 Restore shell.nix
I believe `use_nix` looks for shell.nix and then default.nix files, so I was
wrong earlier when I said that I could get rid of shell.nix files altogether.

I need to learn more about `use_nix` and getting environments from default.nix
files.
2020-08-20 11:26:31 +01:00
William Carroll
7e2dcc97cc Drop support for wpc/find-file
Cleaning things up...
2020-08-20 11:26:31 +01:00
William Carroll
56b503c573 Prefer direnv's use_nix to shell.nix
I find it bothersome to share the list of Haskell dependencies between my
default.nix and shell.nix files. A few days ago, I created a THIRD file,
shared.nix, that defined the shared code b/w default.nix and shell.nix. This
DRY'd things up, but it also added a new file, which I didn't like.

Today I learned that direnv integrates with Nix using a function called
`use_nix`. Voila! I typically already have .envrc files per-project, so this
doesn't add any unnecessary files, and it allows me to delete my shell.nix
files.

I would use `lorri`, except that I encountered issues using Lorri on my work
computer, which I'm not interested in attempting to resolve now.
2020-08-20 11:26:31 +01:00