Most of the errors at the moment are related to line-lengths exceeding my 80
character limit. While these are valid, the linter doesn't currently support
disabling these checks, which means that my Elisp code always fails this CI
step. This creates too much noise and conditions me to care less about CI
failures.
When the Elisp linter support this feature, I will gladly re-enable this step.
Problem: `(bufferp "*Warnings*")` always returns `nil` because it expects a
buffer object. Thankfully I wrote a function called `buffer-exists?`, which is a
more DWIM alternative of `bufferp`.
Hopefully now CI should fail!
TL;DR:
- Assert that the path to the init.el exists
- Check *Errors* buffer in case an error is uncaught but logged
- Log a message when Emacs successfully initializes
These were hard-coded as $HOME/BRIEFCASE, which won't work in CI, since CI runs
as the user buildkite-agent-socrates, whose $HOME directory doesn't exist.
TL;DR:
- Define runEmacsScript to emacs/default.nix for ci/pipelines/post-receive
- Write script.el to call (load init.el) and catch any errors
- Lint Elisp with gonewest818/elisp-lint
Also nice how Buildkite supports :gnu: emojis!
I wanted Gitea to call Buildkite's pre-receive pipeline and either accept or
reject the incoming code depending on the outcome. The problem is that I can
only *create* builds from Gitea's pre-receive hook.
Now I'm left with two options:
1. run the lint-secrets step in post-receive
2. run `/nix/store/<hash>/git-secrets --scan-history $REPO_PATH` in Gitea
As far as I can tell, I cannot define Gitea hooks in Nix, which is unfortunate;
otherwise, option 2 would appeal more.
I'm doing option one for now.
So it turns out that I was wrong and that .git/config is stateful. Multiple
calls to --add-provider will append the same provider each time...
Instead I'm defining secret-patterns.txt and version-controlling it.
Then:
- dev-side: I'm adding `providers = cat ci/secret-patterns.txt` to .git/config
- ci-side: I'm adding `providers = cat ci/secret-patterns.txt` to .git/config
Unfortunately this is ad-hoc configuration ci-side, which I would like to
avoid. The good news is that my pre-commit hooks and failures from git-secrets
should now align with my CI, since they're both reading from
secret-patterns.txt. One step backwards... two steps forwards?
I'm also `cat .git/config` because I think the Buildkite destroys the
.git/config file for each build, but I want to verify that. If it does, I prefer
that because it seems to share the spirit of the "Destroy Your Darlings" essay.
Problem: my dev machine returns a different value for `git config --get-all
secrets.patterns` than my CI machine... I ran `git-secrets --register-aws` to
get additional coverage, but it's still not the same. I created an issue on the
git-secrets GH repo to get better troubleshooting advice, but I don't need the
logging info. anymore, so I'm removing it.
Somehow `git-secrets --scan-history` is exiting non-zero, when I don't think it
should. Logging some environment information to get a better idea of what's
going on.
After a handful of failed attempts to run lint-secrets.sh due to a missing
`git-secrets` executable on my git server, I decided that now was a good time to
use Nix to define my BuildKite pipelines.
TL;DR:
- Delete ci/scripts directory
- Define ci/pipelines/{briefcase,socrates}.nix
Outside of this repository:
- I logged into my admin account at git.wpcarro.dev and changed my Gitea
post-receive hook to trigger the briefcase pipeline
- I logged into my BuildKite account, deleted my build-briefcase pipeline,
created a new briefcase pipeline that called:
```shell
nix-build -A ci.pipelines.briefcase -o briefcase.yaml
buildkite-agent pipeline upload briefcase.yaml
```
One day I will audit all of my ad-hoc, non-mono-repo activity (like the steps I
listed above) and attempt to fit everything herein... one step at a time,
though!
Adding a fake secret to test to the new CI build step. I'm not sure I expect
this to fail the step because it relies on a pattern that I defined in
.git/config... let's see!
My builds are still failing. This time with...
```
error: getting status of /home/wpcarro/nixpkgs-channels: Permission denied
```
...what confused me was the following:
```shell
$ sudo -u buildkite-agent-socrates stat /home/wpcarro/nixpkgs-channels
permission denied
```
But `ls -al /home/wpcarro | grep nixpkgs-channels` showed `r-w` for all users...
Thankfully @riking on ##tvl told me that I should check the permissions for
/home/wpcarro and /home...
After running `ls -al /home`, I saw `---` for all user... I then reproduced the
error by running:
```shell
$ sudo -u buildkite-agent-socrates stat /home
permission denied
```
Great!
So then I moved nixpkgs-channels to /var/lib/buildkite-agent-socrates. @edef
recommended that I read more about DynamicUser= setting for systemd, which looks
relevant after I took a cursory glance.
I'll also want a more declarative way to manager this, but I'm making small
improvements every day.
For now, I'm supporting two CI pipelines:
- build-socrates
- build-briefcase
Conceptually, build-briefcase should cover what build-socrates does now, but
eventually I would like build-socrates to call `switch-to-configuration` so that
all of my websites, etc. stay fresh.
Unforeseen problem: `buildkite-agent` runs its builds in a separate directory,
so if I want the `nix-build` command to build the newly checked out code, I need
to set <briefcase> to the CWD.
I've encountered a few problems with attempting to support nixos-rebuild:
- the activation step requires `sudo` privileges
- the `buildkite-agent` runs on the same machine, socrates, that is rebuilding
itself. This means that when the activation step runs, it will attempt to
restart `buildkite-agent` when the agent is still working
I'm slowly removing places in my nix code that rely on '<bracket>' notation, so
that I no longer depend on NIX_PATH being set. I still have more work to do.
{covid-uk,sandbox}/default.nix are breaking when I attempt to run my
build-socrates.sh script locally, so I'm temporarily disabling them until I can
get CI working as I expect.
- using `set -euo pipefail` for setting recommended failure-modes
- using `set -x` and `echo "$PATH"` to debug my failing build
Sidenote: I find BuildKite's documentation quite helpful!
