The audit log schema — the SIEM parser's contract

This document is the contract a SIEM (or any other external log consumer) parses against: the file's location and permissions, the JSON shape of one entry, the versioning rule that governs when that shape is allowed to change, and the rotation behavior an ingesting agent must tolerate. internal/audit/audit.go is the implementation; this file is what an integrator should read instead of that source.

1. The file

Canonical path: $ISG_HOME/audit.log. In the isolated-CA production layout (see docs/provisioning-isolated-ca.md) ISG_HOME is /var/lib/isg, so the file is /var/lib/isg/audit.log.

• Mode: 0640.
• Owner:group: isg:isg-operators. A SIEM forwarding agent must run as a member of the isg-operators group (or as isg itself) to read it — a plain operator login account is not enough by design.
• Format: JSONL — one JSON object per line, UTF-8, newline-terminated. Each line is independently parseable; a consumer must not assume anything about relationships between lines beyond file order.

The file is opened, appended to, and closed on every single write (see §4) — there is no long-lived file handle inside isg holding it open.

2. The fields

Every line is an object with exactly these top-level keys:

details is intentionally an open, event-dependent map — a command-denied entry carries command and rule, a cert-issued entry carries different keys entirely. A parser must not assume a fixed key set inside details, and must not fail on keys it doesn't recognize there either. The one documented cross-event convention: events that record who acted use either details.actor or details.user depending on the event (see Query's actor filter in internal/audit/audit.go for the exact split) — a parser matching on "who did this" should check both.

2.1 Acting on someone else's behalf: as_user, and why it is not principal

An operator can open a session for a different directory identity (isg connect <host> --as <user>). The events that know about that distinction carry it in as_user, and they carry it always — including when nobody was impersonated, in which case as_user holds the acting user's own name. Absence is never the signal: a rule looking for impersonation compares as_user against the event's own subject (user, or caller_user where the event has one) rather than testing whether a key is there.

That is a change of contract, made deliberately before any external consumer existed. Until v0.231.0 this axis was written as principal, omitted when empty — so a missing key meant both "no --as was passed" and "no impersonation happened", and nothing in the line told the two apart. Worse, principal was carrying four unrelated meanings across the log at once:

A rule filtering on principal to answer "who acted for whom" therefore pulled in certificate issuance and — inverting its own question — the identity of the requester in the issuance-authorization events.

Since v0.231.0 the first group writes as_user, and the third writes requester_directory_user, which pairs with the requester_user already taken from the peer credential: which OS user asked, and who the directory says that is. Rows two and four keep principal, because there the word is the correct SSH term for what it names — a certificate's subject, and the login name a probe authenticated as. principal in this log never means "on whose behalf".

requester_directory_user keeps the omit-when-empty rule, and there the absence is real information rather than ambiguity: it means the directory lookup never got that far. caller_user is a third, separate axis — which OS user opened the session — and is untouched: it is written only when it differs from the subject, and that remains a deliberate choice recorded at its own events.

2.2 One event records a QUERY, not an execution

Every other event in this log names something that happened: a command ran, a certificate was issued, a session closed. COMMAND_COMPLETION is the exception, and a consumer that reads it as an executed command will draw conclusions the line does not support.

It records a Tab-completion lookup from the brokered prompt. The operator typed a path fragment and pressed Tab; the daemon ran a fixed listing command with that fragment as a single quoted argument and sent the names back to the client. Nothing the operator typed was executed, and the command gate never evaluated it — the gate decides about running a command, and here nothing is run on the operator's behalf.

{"event":"COMMAND_COMPLETION","session_id":"SESS-2026-000251",
 "details":{"prefix":"/etc/ssh/ssh_h","matches":6,"truncated":false,
            "cwd":"/mnt/cdrom","as_user":"peter"}}

prefix is the point of the event: it is the only field that says what was probed. matches is a count and never the list — the names would add nothing a reader needs, while writing a directory's whole contents into the log on every Tab. truncated says the listing hit its cap, so the count is a floor rather than a total.

2.3 Why it is a separate event, and what to alert on

The lookup could have been folded into COMMAND_ALLOWED. It was not, deliberately: three Tab presses while typing one command would write three times as many rows as the command itself, and a log that used to say "this operator ran these seven commands" would dilute into noise.

The split also gives a consumer something better than noise. This is the one place in ISG where a directory tree can be examined without running a command, so volume is the signal: a burst of completions — say a hundred in a minute, or a walk down successive path components — looks nothing like a person finishing a filename, and looks exactly like someone mapping a filesystem. Alert on the rate, not on any single row; a single completion is an operator pressing Tab.

Two limits worth knowing before writing that rule. The lookup is gated at the session level — it can only happen inside an open, authorized session on a host that operator was already allowed to reach — so it widens no host boundary. And a completion leaves no trace on the target beyond that host's own sshd/auditd records of the listing command; this log is where the intent is recorded.

2.4 broker: an absent key and an empty value are different facts

A parser sees three cases on this key, and only two of them are about the same thing:

The first case exists forever, because the log is append-only (§5) and old lines are never rewritten to add the key. The second is why the field carries no "omit when empty" rule: on a line this code writes, the empty string is a statement — this broker has no mark — and dropping the key would merge that statement with the silence of the pre-field lines.

The distinction is only visible in the raw JSON, on the key's presence. A consumer that unmarshals into a struct with a plain string field gets "" for both and cannot tell them apart afterwards; if a rule needs to, it must check for the key itself.

One thing the field does not claim: it names the broker that WROTE the line, which for a daemon-forwarded event (the client measures, the daemon records) is the daemon — not whatever process first observed the event. That is deliberate; a client-supplied mark would be a claim rather than a fact.

2.5 prev: the hash chain, and what it does and does not prove

Every line written since this landed carries the hex SHA-256 of the line physically above it in the same file — the line's bytes without the trailing newline, which is exactly what a line-oriented reader hands back. Nothing is canonicalized anywhere: isg serializes the record once, and that one byte sequence is what goes into audit.log, what is hashed, and what is re-rendered into the syslog export's MSG (§7). There is no second representation for a verifier to disagree with.

The same three-case reading broker has (§2.4) applies here, for the same reason and with the same "the key's presence decides" caveat:

The chain flows across rotation (§4): the first line written into a fresh audit.log links to the last line of the piece that was just renamed away, so the pieces and the live file are one chain, walked oldest-first. It also flows across a daemon restart, which re-reads the tail on startup.

What it proves. An edit or a deletion anywhere in the trail breaks the chain from that point on. isg system audit-chain verify walks the pieces and the live file in one pass and names the first line that no longer follows from the one above it.

What it does not. The file is on the broker's own disk, so whoever can rewrite it can also recompute every hash after their edit and hand back a chain that verifies perfectly. The chain is worth something because its head leaves the machine: isg serve writes an AUDIT_CHAIN_HEAD event at startup and every hour, carrying the current head in details.head, and that event goes out on the ordinary fan-out — into the SIEM, over the syslog export. A rewritten log has to reproduce every head exported before the edit, over bytes that are no longer there.

So the head event is the one an ingesting agent should keep and be able to look up. Two properties make it self-checking: details.head equals that same line's own prev (the head is read immediately before the entry is appended), and a missing hourly entry is not by itself evidence of tampering — a daemon that was down writes none.

Per-record authentication — a MAC proving the line came from this ISG rather than from anyone with write access — is a separate, not-yet-built step. This one detects modification of a sequence; it does not authenticate an individual record.

2.6 The SSO login events, and the one field to alert on

Three events describe a login that went through an external OIDC identity provider. They sit beside the LOGIN_MFA_* events rather than replacing them, because they record a different party's decision.

surface says which front door, and it is on all three events, from both fronts, always — never omitted, so a rule never has to infer it from which other keys happen to be present. Two values: webui (the browser's authorization-code callback) and gateway (the CLI's RFC 8628 device flow, ssh isg-login@gateway).

The two fronts carry different companion keys, and the differences are deliberate rather than an inconsistency:

protocol says which SSO protocol (B272), and it is on all three events, from both fronts, always: oidc or saml. The gateway front is always oidc — under protocol: saml the CLI login stays password plus the local second factor. On saml, issuer is the identity provider's entityID, sub is the assertion's NameID, and LOGIN_SSO_OK carries authn_context_class_ref and session_index instead of sid/amr.

mfa_source on LOGIN_SSO_OK is the field worth a rule. It has three values:

local_fallback is the one to alert on. It is not a failure — the login is still fully authenticated — but it means the identity provider is not asserting what the deployment believes it asserts, and the operator should find out why before anyone widens the trust.

reason on LOGIN_SSO_FAILED is a closed set. Most values belong to one surface, because they name a step only that surface has:

Under protocol: saml the webui front has its own steps (B272). state, not_in_directory, no_resolver and locked mean what they mean above; a state here is an ACS post whose RelayState does not match its flow cookie, or whose flow is unknown, expired or already used. The rest name the check the SAML Response failed:

signature, in_response_to and replay deserve a rule of their own: nothing legitimate produces them.

Three of them deserve their own rules. not_in_directory means a valid token for a non-user, which is what a federation misconfiguration and an IdP compromise both look like from here. verify means a token that did not survive signature/issuer/audience/expiry checking, which nothing legitimate produces. user_mismatch means somebody approved a browser step as an account other than the one they claimed at the terminal — a run of them is either an operator with two accounts or an IdP whose username claim does not map to directory names.

None of these events counts toward a lockout, on either surface. There is no password in either flow, so nothing was guessed; counting a refusal here would let anyone lock a real user out by replaying a malformed callback or by refusing somebody else's device authorization. Both fronts read the lockout counters the password and MFA fronts write — a user locked out there is refused here too — and neither ever adds to them. A rule correlating brute force must therefore not expect LOGIN_LOCKED to follow a run of LOGIN_SSO_FAILED.

All three carry an empty session_id: an ISG session (SESS-…) is a brokered SSH session, and a login is not one — not a browser login, and not the gateway login that goes on to open one (that session's own events carry its id, and login_attempt_id is what joins them to these).

The logout events (B273, B463)

idp_logout on LOGOUT is a closed set: redirect — the browser was sent to the identity provider's end_session_endpoint, so the IdP session ends too (rp_logout: on, OIDC session, a kept id_token, and an IdP that publishes the endpoint) — or none — only the gateway's own session ended (a password or SAML session, rp_logout: off, or no endpoint/id_token).

reason on LOGOUT_SSO_BACKCHANNEL_FAILED is a closed set:

verify and replay deserve a rule: nothing legitimate produces them. A request without any logout_token is answered 400 and writes no line.

2.7 Target names: host is canonical, host_input is what was typed (B457)

Every target name that enters the gateway — a session request, a gateway-login target, an issue request on the issuance socket, an mTLS renewal — is brought to one canonical form before any decision: surrounding space and one trailing dot removed, lower case, checked against the RFC 1123 label rule, and, when connect.yaml sets target_domain, a dot-less name qualified with that domain. From then on host always holds the canonical name — on the decision events, on SESSION_STARTED, on every in-session event. Before this, an alias of a host (its FQDN, upper case, its IP) was a different string to every gate, and could resolve to a lower lockdown level than the host itself.

host_input is written next to host when, and only when, the typed name differs from the canonical one. Absent means the operator typed the canonical name. A rule correlating with what a user actually entered reads host_input when present and host otherwise; a rule grouping by target reads host only.

A name that cannot be canonicalized is refused on the spot with reason: invalid_target_name: empty, a character outside the RFC 1123 label rule, or an IP literal while target_domain is set. That line carries host_input and no host — there is no canonical name to write — plus a detail with the parser's message. The event is the one the surface already uses for a refusal:

invalid_target_name is added to the closed set of reason values those two events carry. The issuance-path CERT_ISSUED, CERT_DENIED and HOST_GROUP_* lines written by the shared signing core carry the canonical host only; on that path host_input appears on the socket's own ISSUE_REQUEST_* lines.

The lockdown-level source cmdb-miss (the servers level command, the webUI and JIRA ticket bodies) names a host the CMDB does not know; its level is cmdb.yaml's miss_level.

2.8 HTTP brokering: the application-session events (B277)

When apps.yaml is present, ISG brokers web applications behind the webUI login. A browser reaches an application's own host name, is handed to the webUI for a single-use code, and gets an application session of its own (cookie isg_app) — one per user and application, a child of the webUI session it came from. Four events describe it; an allowed proxied request writes nothing to the audit log.

• app_id (B491) is the application's id (APP-<year>-<sequence>): the apps.yaml id, or for a dynamic application the one the record store minted. It is what every Web Access reference names — approvals, session records, locks, tickets — so two applications of the same name (a deleted and re-added one) never mix; filter on it. app stays the technical name, for reading. Lines written before B491 have no app_id.

• APP_UPSTREAM_LOGIN / APP_UPSTREAM_LOGIN_FAILED (B468) are a kind: form account's logins: the proxy itself posts the login form for one application session and keeps the upstream's cookies. relogin is true when the upstream had reported the session logged out. On a failure step is the failed step's index (step == steps: the request retried after a re-login was logged out again) and status its HTTP status (0: no answer). No password, token, cookie value or response body is ever written.

• APP_RDP_UPSTREAM_CONNECTED / APP_RDP_UPSTREAM_FAILED (B278) are an RDP target's connections: guacd's RDP reaches the broker through the RBI host's agent, and the broker itself dials the configured upstream — the only address it ever dials for that application. rbi_id joins them to the APP_RBI_SESSION_* lines. The injected account's name and secret are never written here.

• APP_RDP_UPSTREAM_CLOSED (B538) is that connection's end. ended_by: target means the record was still active and the isolated session still running — the target (a logoff, the network, a Windows takeover) ended it, and the session closes with target_closed; any close the ISG made itself is ended_by: isg. first_eof is diagnostic only: guacd often closes first.

• APP_RDP_CHANNEL_TRANSFER (B278 F2) is one clipboard transfer or one print of an RDP session, written when its stream ends (or when the view's tunnel ends with it still open). bytes is the decoded size; blocked true means the channel was off for the user and the broker's tunnel dropped the stream — the attempt is logged all the same. The content is never written. Sound (audio_out) is continuous and has no per-stream line; the connection's channels shows whether it was on.

• parent_auth_source is the webUI session's auth_source (password or sso): how the person behind this application session logged in.

• parent_ended means the webUI session is gone (logout, expiry); the application session is closed on its next request, at most a few seconds after the webUI logout.

• access_revoked is written together with an APP_ACCESS_DENIED carrying the actual reason: the group gate is re-evaluated on every request against a directory answer at most DefaultIdentityCacheTTL (5 s) old, so a group taken away closes the session at most 5 s later.

• reserved_username refuses a name the application would map onto a built-in account (measured: admin became the Grafana super-admin).

• handoff_invalid is an unknown, expired, already used or other-application code; handoff_unbound is a valid code presented by a browser that does not hold the state cookie it was minted for — a code someone else sent.

• no_credential (B277 a3): an identity: credential application, the user passes allow_groups but no credentials entry's groups meets theirs. There is no default account. account on APP_SESSION_STARTED is the name of the entry the handoff chose (the first match); every request chooses again and its transcript line carries http_account. The secret itself is never written anywhere; an unreadable one answers 503 without a line here.

• Approval (B277 b). An app with require_approval: true hands off only under an approved request of that user for that app whose window is open. The request is an ordinary approval record and COMMAND_APPROVAL_REQUESTED line with protocol: http, user, app, app_id (B491; the request's target is the app id), grant_seconds and justification_len — the justification text itself is never written here, it is free text. The decision is the usual approve/deny line. Both carry approval_mode: four_eyes (B496; an SSH approval line has no approval_mode). An app with approval_mode: self (B496) has no request and no approver: the grant is one COMMAND_APPROVAL_GRANTED line with approval_mode: self, protocol, user, app, app_id, grant_seconds and justification_len, and no by — the stored request is approved with an empty decision_by and mode: self, its decided_at the moment of the grant. The window is [decided_at, decided_at + grant_seconds); approval_id and approval_until on APP_SESSION_STARTED name it, and the session record carries approval_id too (also on its SESSION_STARTED). approval_window_ended is a session the window's end closed (it ended before max_duration would have). approval_missing is a require_approval app's code minted without an approval — a webUI broker with another apps.yaml, or an older one.

• admin_close (B277 b): an administrator closed the Web Access session record (webUI or isg sessions close). The application session is deleted on every broker and the approval window, if any, is closed with it; the broker serving the user acts on its next request (at most 5 s later), any other broker on its next sweep.

• directory_unavailable is written by the webUI handoff; a directory outage during a proxied request answers 503 without closing the session or writing a line.

• APP_REQUEST_DENIED is one request refused by a policy.yaml http_rules entry; the application session stays open. path is the decoded, normalized path without the query string (a query can carry secrets). A request whose path has a ./.. segment, an inner empty segment or a control character is answered 400 and writes no line — it is a malformed request, not a policy decision. The verdict sees a WebSocket upgrade request, not the frames after it.

• Allow-list mode (B497). mode is the application's http_app_modes mode (allow, allow_audit, or empty). Under allow a request no action: allow rule matches is refused with rule: http_mode:allow. APP_REQUEST_WOULD_DENY is the allow_audit trial: the same request is let through and this line records what allow mode would have refused — its path set is the application's missing allow rules. Lines written before B497 have no mode.

• Step-up (B495). APP_STEP_UP_REQUIRED is one request an action: step_up rule held because the application session has no live confirmation; a page load is sent to the webUI's step-up page, anything else is answered 403. The webUI writes APP_STEP_UP_FAILED (no_factor: the user has no factor mfa_mode can use; invalid_code with the challenge's attempts; too_many_attempts spends the challenge; expired; locked_out: the login lockout holds the user or the address). APP_STEP_UP_OK is written by the application host once the confirmation marked the session: valid_until is the end of the window — the rule's valid_for, at most the session's own end. step_up_mismatch is a step-up code redeemed by a browser whose isg_app session, user or application is not the one it was minted for. A wrong code also writes the usual LOGIN_LOCKED when it trips the lockout.

• An allowed Web Access request is not an audit line but a transcript line of a protocol: http session (below); so are the refused ones, next to their APP_REQUEST_DENIED.

The Web Access session record (B277 a2-2). Each application session gets an ordinary session record on every broker that serves it, opened by its first request there, so SESSION_STARTED and SESSION_CLOSED describe it with the usual fields plus protocol: http (absent on an SSH session). target is the app id (B491; before it, the app name) and target_label the app's friendly name at the start, auth is web_access, and commands_allowed / commands_denied count requests. The reason of its SESSION_CLOSED is client_exit (logout, or the application session is gone), idle_timeout, max_duration, external_close (the webUI session ended, or an administrator closed the record — then APP_SESSION_CLOSED says admin_close), access_revoked (the group gate failed mid-session), locked (an incident lock, §2.10 — then APP_SESSION_CLOSED says locked too), app_deleted (an administrator deleted the dynamic app, below — APP_SESSION_CLOSED says app_deleted too) or daemon_shutdown. A failed transcript write is one RECORDING_UNAVAILABLE (session_id, app, app_id, user, reason: transcript_write_failed) per failure spell; with require_recording: true the session's requests are refused (503) until a write succeeds again.

Dynamic applications (B475). With dynamic_apps_domain in apps.yaml, an administrator past the session_admin_group gate (§2.9) adds, changes and deletes applications on the webUI's /apps/manage page; the apps.yaml ones cannot be changed there. Three events, via is webui:

• A change takes effect without closing anything: the group gate runs per request, so a group taken away refuses the next request (access_revoked above). Other brokers see a change within 5 s.
• Deleting closes the application's active HTTP session records on the deleting broker (SESSION_CLOSED reason: app_deleted, closed_by, via) and its application sessions on every broker; afterwards its host answers 421.

2.9 webUI admin writes refused by a group gate (B480)

Two ldap.yaml groups gate the webUI's administrative writes, checked against the logged-in user. A refusal changes nothing and writes one line:

• SESSION_ADMIN_DENIED with via: cli is isg sessions close refused on the issuance socket (B475); user is the SO_PEERCRED-attested caller.
• Closing one's own session (the caller is its user or caller_user) asks no gate and never writes SESSION_ADMIN_DENIED, on either surface.
• An administrator's close writes SESSION_CLOSED with closed_by (who) and via (webui or cli) in its details (B483).
• CA_ADMIN_DENIED is the issuance socket's event too; there it carries group and identity_source and no via. Filter on via to tell the two surfaces apart.
• reason is the gate's own error text, free text for a human, not a closed set.
• A gate set to required (ca_admin_group_required, session_admin_group_required) without a group or an identity source refuses the write but writes no line: that is a deployment error, not a decision about the user.

2.10 Incident locks on a user or an application (B477)

An administrator past the session_admin_group gate can lock a user or a Web Access application (its app id, B491; isg lock add --app also takes the name and resolves it) — on the webUI's /locks page or with isg lock add. A lock made on an app name before B491 still holds. The lock is shared by every broker on the state root; creating it closes what it locks out, and while it lives every entry point refuses.

• A user lock closes every active session record the user owns or called (SSH and HTTP) with SESSION_CLOSED reason: locked, closed_by and via, and logs the user out of the webUI and every application. An app lock closes the application's HTTP records and application sessions. A live SSH connection on another broker is cut by that broker's watchdog within seconds: its SESSION_TERMINATED says locked.
• The refusals while a lock lives: GATEWAY_AUTH_DENIED reason: user_locked (after the directory lookup, before any code is sent); a session-socket request for a locked principal or caller gets Result reason user_locked; a webUI password, MFA or first-enrollment step writes LOGIN_LOCKED with reason: user_locked and step (password, mfa, enroll) — the brute-force LOGIN_LOCKED carries no reason; an SSO callback writes LOGIN_SSO_FAILED reason: user_locked; Web Access writes APP_ACCESS_DENIED reason: locked.
• An expired lock (until in the past) refuses nothing and writes nothing; it stays listed until removed.
• The lock mail (B503, notify.yaml lock_mail.enabled) writes one NOTIFY_SENT, NOTIFY_FAILED or NOTIFY_SKIPPED per lock created or removed on the broker that made the change, with kind lock_created or lock_removed, sinks: ["mail"], lock_kind, target, recipients (the number of addresses mailed), recipients_without_mail (recipients the directory has no address for) and, on failure, error. No address is logged. NOTIFY_SKIPPED means nobody could be mailed — e.g. an empty session_admin_group. With lock_mail off nothing is written.

2.11 The license (B467)

The broker reads an offline, signed license.yaml (re-checked every 30 seconds, no restart needed). Without one it runs in a 30-day evaluation mode whose clock is one record in the shared record store. Only a NEW session is gated: running sessions and break-glass are never refused or cut.

• LICENSE_LOADED is written when a broker loads a license it has not loaded before — so once per broker start, and once per replacement.
• LICENSE_REJECTED is written once per file state: an unchanged bad file writes nothing more. A rejected replacement leaves the license loaded before in force until restart.
• LICENSE_EVAL_STARTED is written once per fleet: the record is never rewritten, so a restart does not start a new evaluation.
• state is one of eval, eval_expired, valid, warning, grace, expired, invalid. A refused SSH session answers Result reason license; a refused Web Access session writes APP_ACCESS_DENIED reason: license next to LICENSE_DENIED.

3. The versioning rule

schema_version states which version of this contract — the meaning and type of the fields in §2 — the line was written under. Current value: "1".

• It changes when an existing field's meaning or type changes — for example, if session_id ever stopped being a plain string, or if timestamp changed format.
• It does not change when a field is added. New fields are additive and backward compatible; a compliant parser tolerates unknown keys without failing. Do not gate new-field handling on a schema_version bump — there won't be one. broker (§2.4) is this rule's first application: it was added after schema_version existed, it changed no existing field's meaning or type, and the version therefore stayed "1". A parser must not read "1" as "this line has no broker key" — the key's own presence is what answers that. prev (§2.5) is the second application, under the same rule and with the same caveat.
• Its absence means the line predates versioning, not that it's malformed. schema_version was introduced after the audit log had already been in production use; existing lines were not, and will never be, rewritten to add it retroactively (the log is append-only — see §5). A parser must treat a missing schema_version as "written before this contract existed" and process the line normally, not as a parse error.

4. The rotation contract

Rotation is configured in packaging/isg-audit.logrotate and is load-bearing for how an ingesting agent must track the file:

/var/lib/isg/audit.log {
    monthly
    maxsize 50M
    rotate 9999
    compress
    delaycompress
    missingok
    notifempty
    create 0640 isg isg-operators
    su isg isg-operators
}

• Rename-based rotation (create), not copytruncate. isg's audit Logger opens audit.log by path, appends, and closes the handle on every write, so a rotation that renames the old file and creates a fresh one loses nothing — the next write simply lands in the new file. A forwarding agent must therefore be inode-aware (follow the file by watching for rotation, e.g. via inotify or a stat-based inode check), not just tail a file descriptor it opened once — a descriptor held across a rename keeps reading the old, now-frozen file.
• delaycompress: the most recently rotated piece is left uncompressed for one more rotation cycle, so an agent that reads compressed rotated pieces has one full cycle of lead time before a given piece is gzipped.
• rotate 9999: rotated pieces are, in effect, never deleted. Nothing is ever silently dropped from the trail by the rotation policy itself — an agent that falls behind does not race a deletion window.
• monthly + maxsize 50M: whichever threshold is hit first triggers rotation.
• su isg isg-operators: rotation runs as the isg user/group, not root. On the documented layout (/var/lib/isg at 0750 isg:isg-operators, as the PIC §3 matrix and the isg_gateway role set it) the directive is not what makes rotation run — logrotate 3.18.0 was measured to run the drop-in on that root without it. It stays for two reasons: root never writes into the isg tree, so the piece that create 0640 isg isg-operators makes has ownership consistent with the rest of the tree; and the same drop-in keeps working on a setgid/group-writable root (2770, as a hand-built or older install may have it), where logrotate refuses a log under a group-writable directory owned by a group other than root and, without su, would skip it silently (skipping ... because parent directory has insecure permissions).

5. The fail-closed limit — file is, and must remain, the audit-of-record

Writing to audit.log is fail-closed as the default posture: internal/audit's Logger.Log returns an error on a failed write, and the main session/command path (internal/session's session-start, command-allowed/denied/error, and session-close audit calls) propagates that error up rather than continuing as if the write had succeeded. A narrow, explicit exception exists — LogOrWarn, used only by call sites where the state change already happened before the audit write (lazy expiry of stale approvals) or where failing the operation would be worse than continuing without the record (some sshbroker probes) — and even there the failure is never silent: it's logged to the process's own standard logger so the gap is at least visible operationally, per that function's own doc comment in internal/audit/audit.go.

Any future forwarding path this project might add — syslog, HTTP, a message queue — would necessarily be fire-and-forget or otherwise fail-open at the point where it hands off to a remote receiver that might be unreachable, slow, or down. Because of that, no future forwarding path may ever become the sole audit sink. The on-disk file written by internal/audit must remain authoritative; disabling the file write in favor of a forward-only path would silently open an audit gap the moment that path's target is unreachable, with no local record left to reconstruct what was missed. Anyone integrating a SIEM forwarder should tail/ship this file (§4) rather than have isg push to them directly, precisely so the local file stays the fail-closed source of truth regardless of what happens downstream.

Such a forwarding path now exists — the optional syslog export described in §7 — and it was built to this rule rather than around it: it is a copy, it can never rescue a failed file write, and it cannot be made the sole sink. Read §7.4 before treating its stream as complete.

6. A second, different file: the collected break-glass log

Everything above is about audit.log, which isg writes. There is a second file a SIEM will encounter in an ISG deployment, and it obeys none of the rules above:

/var/log/isg-breakglass-remote.log

It is not written by isg. It is a verbatim rsyslog copy of the target hosts' own sshd lines, forwarded to the gateway so that the emergency (break-glass) door leaves a trace on a machine the person using that door does not control. Plain RFC3164 syslog text, not JSONL; no schema_version, no details, none of §2's guarantees. This section exists because the lines in it are easy to misread in a way that inverts their meaning.

6.1 Which line means somebody got in

sshd writes two different lines during a certificate login, and only one of them is an entry:

Measured (2026-08-15, on a target host): a login attempt whose private key could not sign — the operator key was passphrase-protected and ssh ran with BatchMode=yes — produced

Accepted certificate ID "isg-bg:peter@INC-9001" (serial 9001) signed by CA ED25519 SHA256:…
Postponed publickey for isg-breakglass … [preauth]
Failed publickey for isg-breakglass … ED25519 SHA256:…

No login happened. A rule that alerts on Accepted certificate as "someone used the emergency door" would have reported successful root access for an attempt that never got in.

6.2 Why Accepted certificate is not evidence of anything an attacker cannot fake

The offering phase of SSH public-key authentication does not require the private key: the client may present a certificate and only later be asked to prove possession. A certificate is public data by design — ISG's own break-glass workflow copies certs around with scp, legitimately.

Therefore anyone who obtains a copy of an issued certificate can produce an unlimited number of Accepted certificate lines on any target that trusts the signing CA, without being able to log in at all. Treat the line as "a certificate was presented", never as "a certificate was used".

One further limit, measured: the message itself carries no username, no source IP and no port — only the ID, the serial and the CA fingerprint — so the break-glass account can be recognised in it solely through the isg-bg: identity prefix.

The syslog frame, however, does carry a connection identifier that the message body lacks: the tag field holds the sshd session's pid (sshd-session[7516]), and one sshd session serves exactly one connection and writes every line of it — the presentations, the Postponed publickey, and the final Accepted/Failed — under that same pid. A presentation and the login (or failure) that followed it can therefore be correlated on (host, pid).

An earlier revision of this section stated the opposite, that no shared identifier existed. That was wrong: it described the message body and overlooked the frame around it. The correction matters for anyone writing rules here, so it is recorded rather than quietly edited away.

Two costs come with that key, and a consumer must pick which error it prefers, exactly as ISG did (§6.3): pids are reused, so two connections far enough apart can merge — hiding a "presented, never got in" behind a later success — while adding a date to the key splits a connection that crosses midnight and invents a "never got in" that did not happen.

6.3 What ISG itself does with these lines

internal/breakglasslog has two parsers, and the split is the same one this section draws:

• ParseAccepted matches only Accepted publickey for isg-breakglass. Every ISG-derived freshness claim — the system check drill-trace row, isg breakglass commands — is built from this parser alone, so no ISG output has ever conflated a presentation with an entry.
• ParsePresented matches Accepted certificate lines and exists purely to count them.

Both parsers also expose the tag's pid, and the drill-trace row groups the lines by (host, pid) before counting anything, so what it reports is connections, not lines:

cert presentations: 14 connections, 10 entered, 4 without a login

Counting the lines would roughly double the truth. A successful certificate login writes the Accepted certificate line twice — once when the certificate is offered, once when the signature is checked — while an attempt that presents a certificate and then cannot sign writes it once. Measured on this project's own collected log: every one of its connections had one of exactly two shapes, 2 presentations + 1 login (10 of them) or 1 presentation + 1 failure (4 of them). A line-based reading of the same file reports "24 presentations, 15 entries" and invites the conclusion that nine attempts failed; four did.

The row keeps the key without a date and therefore undercounts rather than raising a false alarm (§6.2). On a log whose lines carry no pid at all it falls back to one connection per line, which reproduces the old line-based reading on exactly the input where nothing better exists.

The row deliberately never changes its status because of these numbers. The collected log is append-only: one old failed drill would otherwise hold the row at WARN forever, and a warning that is always on is one operators learn to skip. Windowed alerting on this signal belongs in the consuming SIEM, which has the retention and the time windows to do it properly — this section is what such a rule should be written against.

A presentation without a matching entry is not suppressed anywhere, and it should not be: "a valid certificate was presented and nobody got in" is a real signal. It is a failed drill, or a copied certificate being probed against the fleet. Both are worth seeing.

6.4 Counting caveat

All three numbers describe the collected log as it stands. The file rotates, and a target that stops forwarding is indistinguishable from a target nobody touched. Pid reuse merges connections rather than splitting them (§6.2). Every one of those effects pushes the counts down, so treat them as lower bounds — never as a complete history.

7. The syslog export — what a receiving parser gets

Since B208's phase 1b, internal/audit fans every entry out to zero or more sinks past the file, and phase 1b-2 ships the first one: internal/auditsyslog, an opt-in copy of every entry forwarded to a SIEM. It is off unless audit.yaml says otherwise (see audit.example.yaml); with it off, nothing in this section exists — no goroutine, no queue, no connection.

An earlier revision of this document argued that no Sink interface should exist yet, precisely because an interface with no real implementation invites someone to wire up a stub and switch the file writer off (§5). The interface exists now, and that concern is answered in code rather than in prose: a sink has to declare, through a method the compiler demands, whether a successful write means the record reached durable storage. The syslog sink answers no, permanently — see §7.4.

7.1 Transport and framing

• TCP with TLS. No UDP, and no switch to ask for it: RFC 5424 over UDP has no framing and no delivery signal, so a message longer than the path MTU is truncated in silence. Audit lines are regularly long, and a truncated audit line is worse than a missing one because it still parses.
• Certificate verification is not optional. An empty tls_ca_file means the system trust store, never "skip verification"; no such switch exists anywhere in this project.
• Framing is RFC 6587 octet-counting — MSG-LEN SP SYSLOG-MSG, where MSG-LEN is the decimal byte count of everything after the space. Not the newline-terminated (non-transparent) framing: octet-counting is deterministic regardless of the payload. Today's audit lines contain no newline, but that is a by-product of JSON encoding rather than an enforced invariant, and the framing must not depend on it.

7.2 The RFC 5424 envelope

<PRI>1 TIMESTAMP HOSTNAME APP-NAME PROCID MSGID STRUCTURED-DATA MSG

There is no BOM. RFC 5424 §6.4 says a UTF-8 MSG SHOULD begin with one; this is a deliberate departure from that SHOULD. Our MSG is JSON that collectors will parse, and a leading U+FEFF makes naive JSON parsers fail on an otherwise valid document. The encoding is unambiguous without it.

7.3 Correlate on broker, not on HOSTNAME

They answer different questions, and the difference bites exactly when it matters — a trail merged from several gateways. HOSTNAME names the machine; broker (§2.4) names the ISG instance. Two brokers can share a host, one broker can be moved between hosts, and a containerized gateway's hostname may be an ephemeral id nobody chose. A rule that groups a session's entries, or that asks "which gateway served this", must read the broker field inside MSG.

7.4 What a delivered message does and does not prove

It does not prove the record was stored. There is no application-level acknowledgement in RFC 5424 over RFC 6587 framing: the receiver sends nothing back, and a completed TCP write only means the local kernel accepted the bytes. This is why the sink reports itself not durable, and why a successful syslog send can never make a failed audit.log write look successful — §5's rule, enforced at the fan-out rather than merely documented. A deployment with strict: cert_issued: true still requires the file write to succeed.

Entries can be dropped, and dropping is reported, not silent. The sink never blocks the gateway: an audit copy that could stall a session is worse than an audit copy with a hole in it. Writes go into a bounded queue (queue_size), and when it is full the new entry is refused — never a queued one, so what does arrive stays in chronological order. Every drop and every send failure increments isg_audit_target_failure_total{audit_target="syslog"} and produces a process-log line (rate-limited to one per minute per target, carrying the count of the ones folded into it; the counter itself is never folded).

The file remains the complete record. Everything §5 says still holds: audit.log is the audit-of-record, this export is a copy, and a consumer that needs completeness should reconcile against the file (§4) rather than assume the stream had no gaps.