mdebray/tvl-depot
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(wpcarro/blog): Create short post about restic

Browse source 
Read the blog post for more information :)

Change-Id: I46a2f8ec9dc0a6546062e645f34186aa9195c70f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5907
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
This commit is contained in:
William Carroll 2022-06-30 20:50:51 -07:00 committed by clbot
parent 2ec0d36119
commit aaa0f92c45
2 changed files with 98 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
users/wpcarro/website/blog/posts.nix
View file

@ -36,4 +36,11 @@
content = ./posts/csharp-unused-variables.md;
draft = false;
}
{
key = "restic-field-guide";
title = "Beginner's Field Guide to restic";
date = 1656645093;
content = ./posts/restic.md;
draft = false;
}
]

91
users/wpcarro/website/blog/posts/restic.md Normal file
View file

@ -0,0 +1,91 @@
Continuing along the trend that [Profpatsch][2] recently inspired in me: writing
short notes to myself instead of fully fledged blog posts aimed at some
unknowable audience. Today we're looking at how I burned myself by only
*partially* RTFD.
## Background
I recently started using [restic][4] and NixOS thanks to the help of [TVL's
`restic.nix` module][1]. I setup `1x/h` backups to [MinIO][3] (S3-compatible
storage) for just a handful of `/var/lib` directories (`~9GiB` total), but after
a few days MinIO reported that my bucket size was `O(100GiB)`!
> What's going on?
> -- me
```shell
$ restic stats
repository 763bfe37 opened successfully, password is correct
scanning...
Stats in restore-size mode:
Snapshots processed: 175
Total File Count: 31369384
Total Size: 21.027 GiB
```
> Wait: 20GiB... wat?
> -- me (moments later)
Maybe we're snapshotting our MinIO buckets, and that's contributing to our
bucket size. Checking the logs proved that `restic` was backing-up `1.5GiB/h`,
which supported MinIO's reports.
> Ah maybe `restic stats` isn't reporting what I *think* it's reporting...
> -- me (again)
Let's consult Le Docs:
```shell
$ restic stats -h
The "stats" command walks one or multiple snapshots in a repository
and accumulates statistics about the data stored therein. It reports
on the number of unique files and their sizes, according to one of
the counting modes as given by the --mode flag.
It operates on all snapshots matching the selection criteria or all
snapshots if nothing is specified. The special snapshot ID "latest"
is also supported. Some modes make more sense over
just a single snapshot, while others are useful across all snapshots,
depending on what you are trying to calculate.
[to be continued]
```
This is where I stopped reading (the first time). But then I returned a second
time as I was running low on theories...
```shell
[continued]
The modes are:
* restore-size: (default) Counts the size of the restored files.
* files-by-contents: Counts total size of files, where a file is
considered unique if it has unique contents.
* raw-data: Counts the size of blobs in the repository, regardless of
how many files reference them.
* blobs-per-file: A combination of files-by-contents and raw-data.
```
Bingo: `--mode=raw-data` **not** `--mode=restore-size`.
## Solution
```shell
$ restic stats --mode=raw-data
repository 763bfe37 opened successfully, password is correct
scanning...
Stats in raw-data mode:
Snapshots processed: 175
Total Blob Count: 710988
Total Size: 303.216 GiB
```
> Ah... the world agrees again.
> -- me
[1]: https://cs.tvl.fyi/depot@2ec0d3611960b163a7052e8554ba065f3c89a8cc/-/blob/ops/modules/restic.nix?L9
[2]: https://github.com/profpatsch
[3]: https://min.io/
[4]: https://restic.net/