The previous change, which makes journaldriver write the cursor
position in two steps, requires that journaldriver can write files
adjacent to the cursor position file itself.
Instead of simply guessing that this is possible (e.g. by changing the
file suffix), expect the user to provide a directory that
journaldriver can work with.
Exits from `persist_cursor` early if the cursor received from journald
is an empty string.
We don't currently know if this actually happens (please see #2 for
more details), so an error message has been added that asks users to
report this if it ever occurs.
This deals with a potential issue where creating a new file in place
of an existing cursor position file may cause position files to be
empty.
The cause for this is that the newly created file will truncate the
previous content, if journaldriver is then terminated before it
completes the cursor write to this file, it will not have written a
valid cursor (or anything at all).
Potentially relates to #2
This replaces reqwest with the more simplistic ureq library for
performing required HTTP requests.
Reqwest comes with a lot of (tokio-based) machinery for
high-performance requesting that is a bit out of scope for
journaldriver's needs.
This clocks in at 62 (!) fewer dependencies after the change, with
equivalent functionality. Wew.
(kjære barn har mange navn :P)
Adds a small piece of documentation about the conversion between
journald priorities and Stackdriver severities to the README, as well
as information about how to easily emit messages at different
priorities from applications logging to journald.
If a priority is present, it is passed as-is into the Stackdriver API.
This allows filtering by severity in the logs UI. Conveniently, the
levels are the same between journald and Stackdriver.
Fixes#11.
This directory is the default for state directories managed by
systemd, which will be the main use-case of journaldriver when running
it on NixOS.
Relates to nixos/nixpkgs#42134
Lets users specify environment variables to configure the monitored
resource descriptor. In combination with service account support this
can be used to configure journaldriver on non-GCP machines.
The supported environment variables are:
* `GOOGLE_CLOUD_PROJECT`: Project ID of the project to which to report
logs
* `LOG_NAME`: Name of the Stackdriver Logging log into which to write
entries
Adds support for authenticating towards Google's APIs by using static
service account credentials which can be retrieved from the GCP
console.
Users can toggle the behaviour by specifying the
`GOOGLE_APPLICATION_CREDENTIALS` environment variable.
Still slightly below the Stackdriver limit. However, previous errors
seem to have mostly been related to JSON payloads - which has been
fixed in the previous commit.
In some cases Stackdriver seems to return error responses for batched
inserts.
This change will log the error response body and status from
Stackdriver for all insertion errors.
Instead of relying on Stackdriver's ingestion timestamps, parse
timestamps out of journal entries and provide those to Stackdriver.
If a timestamp could not be parsed out of a log entry, the ingestion
time is used as the fallback.
In some cases sending 1000 entries seemingly results in a vague "Bad
request" error.
This reduces the chunk size to something that should be more
manageable.
Adds support for persisting the cursor position in a file (by default
`/var/journaldriver/cursor.pos`, overridable via the environment
variable `CURSOR_POSITION_FILE`).
This lets journaldriver resume from a known journal position after
service restarts.
This closes#3
Stackdriver supports structured JSON payloads in addition to simple
plain-text payloads.
This commit introduces a new feature in which journaldriver will
attempt to parse incoming log messages into JSON vaues and forward
them as structured payloads if they are JSON objects.
Messages that can not be parsed into JSON objects will continue to be
forwarded as plain text messages.
This implements the "meat" of the initial version of journaldriver.
Records from journald are converted into the representation required
by Stackdriver and forwarded to the API.
In this initial version journaldriver is only supported on instances
running on GCP.
