Command reference

Every isg command, its help text and its flags, as the binary actually defines them.

Order is cobra's own — alphabetical within each level, which is also the order isg --help prints. (Measured 2026-08-23: cobra.EnableCommandSorting is left at its default true, so (*cobra.Command).Commands() sorts by name; the declaration order in newRootCmd is not what a reader sees anywhere.)

Hidden commands are omitted with their subtrees — Hidden: true is this project's marker for a command driven by a machine rather than by an operator. Cobra's own help and completion commands, and the --help flag every command carries, are omitted for the same reason: they are cobra's surface, not this program's.

isg

Inshipia Security Gateway

A privileged SSH access broker: audited, policy-gated, MFA-fronted access to production hosts.

Paths in this program's help are written relative to $ISG_HOME, the config and state root. It resolves in this order:

  1. the ISG_HOME environment variable, when set
  2. the "home:" key of /etc/isg/isg.conf, when that file exists (typically /var/lib/isg on a system-unit installation)
  3. ~/.isg — the per-user fallback, reached only when neither of the above is present

Read $ISG_HOME as "whichever of those three won here"; run "isg system check" to see the resolved root and where it came from.

Flag Type Default Description
--version, -v bool false version for isg

isg agent

Service-account renewal agent (runs on the service machine)

isg agent run

Start the renewal loop (blocks; suitable for systemd)

Starts the ISG service-account agent.

Prerequisites (one-time, on the gateway): isg ca join-token redeem --token <token> scp $ISG_HOME/ca/renewable/<account>.{crt,key} <service-machine>:$ISG_HOME/agent/renewable/ scp $ISG_HOME/pub/x509_ca_cert.pem <service-machine>:$ISG_HOME/agent/

Then configure $ISG_HOME/agent.yaml and run this command (or enable the systemd user service: systemctl --user enable --now isg-agent.service).

Flag Type Default Description
--config string — path to agent.yaml (default: $ISG_HOME/agent.yaml)

isg agent status

Show the current access cert status

Flag Type Default Description
--config string — path to agent.yaml (default: $ISG_HOME/agent.yaml)

isg approvals

Manage approval-required command requests

isg approvals approve

Approve a pending request (the requester's shell unblocks)

isg approvals approve <APPROVAL-ID> [flags]
Flag Type Default Description
--reason string — Optional reason for the approval (recorded in audit log)

isg approvals deny

Deny a pending request

isg approvals deny <APPROVAL-ID> [flags]
Flag Type Default Description
--reason string — Optional reason for the denial

isg approvals list

List approval requests

Flag Type Default Description
--pending bool false Only show pending approvals

isg approvals show

Show one approval request in detail

isg approvals show <APPROVAL-ID> [flags]

isg audit

Show audit log entries

Flag Type Default Description
--session string — Filter by session ID

isg breakglass

Emergency access that bypasses ONLY the four-eyes approval gate (all other controls still apply)

isg breakglass commands

Show what a break-glass operator ran on <host>, with the session machinery labelled (not dropped)

isg breakglass commands <host> [flags]
Flag Type Default Description
--operator-only bool false Hide the [login-machinery] and [session-machinery] records instead of labelling them
--since duration 24h0m0s Only show records newer than this (session headers still report the real session start)

isg breakglass connect

Open a break-glass session via the isg serve daemon (--reason is required; fully audited)

isg breakglass connect [target] [flags]
Flag Type Default Description
--as string — OS user to SSH as on the target (required)
--reason string — Reason for the emergency break-glass access (required)

isg breakglass ledger-check

Join the break-glass signing ledger to the collected logins: what was signed and never used, and what was used with no ledger row

isg breakglass ledger-check --ledger <path> [flags]
Flag Type Default Description
--ledger string — Path to a copy of the break-glass signing ledger from the CA host (required)
--since duration 0s Only classify ledger rows signed within this window (default: the whole ledger)

isg ca

Certificate authority management (service account access certs)

isg ca host-cert

Manage SSH host certificates for target-host authentication (ADR 640811011)

isg ca host-cert issue

Issue an SSH host certificate for a target host's existing host key

isg ca host-cert issue --host <host> --key <file> [--principals a,b,c] [--ttl <duration>] [flags]

Issues a host certificate over the target's OWN existing SSH host public key (--key) — the CA never generates or sees the host's private key. The daemon records the certificate in the host-cert ledger (the shared record store, so every broker of a multi-broker gateway lists it), mirrors it to $ISG_HOME/ca/hosts/<host>-cert.pub, and the operator scps the published copy at $ISG_HOME/pub/hosts/<host>-cert.pub to the target.

--principals defaults to --host. It must list every hostname/alias isg connect (or the inventory) dials the target by: ssh.CertChecker checks the certificate's principals against the exact dial hostname, so a cert issued for the wrong name will be rejected even though it's otherwise valid.

Issuance is refused if another host already has a live certificate over the same host key — usually a golden image or a cloned VM snapshot that shipped one /etc/ssh/ssh_host_*_key to every machine. Regenerate the target's host keys instead; --allow-duplicate-key overrides the refusal and is audited.

Flag Type Default Description
--allow-duplicate-key bool false issue even though another host already has a live cert over this key — two hosts sharing one key can present each other's identity (audited)
--host string — target host name (required)
--key string — path to the target's existing SSH host public key (required)
--principals string — comma-separated principals (default: --host)
--ttl duration 0s cert validity (default: ca.yaml host_cert_ttl, else 8760h)

isg ca host-cert list

List issued host certificates

isg ca host-cert revoke

Revoke a host certificate by host or by serial (gateway-side serial list)

isg ca host-cert revoke (--host <host> | --serial <serial>) [flags]

Adds the host certificate's serial to $ISG_HOME/ca/revoked.yaml's ssh_host_serials list. isg connect (host_key_check: strict) and the system check's live host probe read this list on every dial — no cache, no staleness. The target host's sshd config is not touched: this is a gateway-side control, the same split as "isg ca revoke" for X.509 renewable identities.

--host resolves the serial from the host's CURRENTLY recorded certificate — the host-cert ledger record in the shared record store, which every broker reads, not this broker's ca/hosts/<host>-cert.pub mirror file (that file may be a stale copy on a multi-broker gateway). After a re-issue that is the NEW cert, so --host would revoke the live one and the superseded serial would be unreachable — the ledger keeps one certificate per host. Use --serial (the decimal value "isg ca host-cert list" prints) to revoke that superseded certificate. Exactly one of the two is required.

--purge additionally removes the host's issuance record once it is revoked: the ledger record in the shared store, the gateway's own ca/hosts/<host>-cert.pub mirror and the published copy under pub/hosts/. Use it for a certificate issued by mistake, which would otherwise sit in this list as "revoked" forever. It does NOT undo the revocation — the serial stays on ssh_host_serials permanently, because a certificate that was once signed may exist somewhere and cannot be called back. The purged serial then appears in the orphaned-serials section of "isg ca host-cert list", and the HOST_CERT_RECORD_PURGED audit event is what says it got there on purpose. --purge requires --host: a bare serial does not name a record to remove.

Flag Type Default Description
--host string — target host name (revokes its current certificate)
--purge bool false also remove the host's issuance record (the ledger record, ca/hosts/ and pub/hosts/) after revoking it — the serial stays on the revocation list; requires --host
--serial string — certificate serial to revoke (decimal, as printed by "isg ca host-cert list")

isg ca init

Initialize the ISG CA key pairs (idempotent; SSH and X.509 CA are independent)

Create both CA key pairs under the CA directory (default $ISG_HOME/ca/), if they are not already there:

ca_key ed25519 SSH CA. Signs the short-lived user certificates a target host accepts through TrustedUserCAKeys. x509_ca_key ECDSA P-256 X.509 CA. Issues the mTLS certificates for the agent renewal endpoint and the admin UI's server certificate.

Idempotent, and the two CAs are independent: an existing key pair is loaded rather than overwritten, so one CA can already exist while the other is created. The public halves are also mirrored into the operator-readable pub/ directory, which is what lets 'isg ca pubkey' work without traversing ca/.

NAME CONSTRAINTS ON THE X.509 CA ARE MANDATORY FOR A NEW CA (BACKLOG B378). Anybody who wants the admin UI to open without a browser warning imports this X.509 CA into a trust store, and an unconstrained CA key can impersonate ANY website on that machine. So a NEW X.509 CA is only created with an explicit list of the names it may certify:

isg ca init --permitted-dns isg.example.local --permitted-ip 192.168.57.0/24

Both flags take a comma-separated list and may be repeated. --permitted-dns permits the domain AND its subdomains; --permitted-ip takes CIDRs (use /32 for a single address). At least one entry is required: with neither flag and no X.509 CA on disk this command writes NOTHING and says so. The deliberate way out is --no-name-constraints, which creates the pre-B378 unconstrained CA and says in one line that it did.

The constraint can only be set on a NEW certificate. If the X.509 CA already exists these flags change nothing — the existing CA is loaded untouched and the output states which constraints it actually carries. Replacing a CA to change them is a decision about every trust store it was imported into. The SSH CA is unaffected by all three flags.

Flag Type Default Description
--no-name-constraints bool false Create the NEW X.509 CA with NO name constraints — it may then certify ANY name (B378 opt-out)
--permitted-dns stringSlice [] DNS name a NEW X.509 CA may certify (the domain and its subdomains); comma-separated, repeatable
--permitted-ip stringSlice [] CIDR a NEW X.509 CA may certify (use /32 for a single address); comma-separated, repeatable

isg ca issue

Issue a short-lived SSH access certificate for a service account

isg ca issue --account <account> --host <host> [--pubkey <file>] [--out <dir>] [flags]
Flag Type Default Description
--account string — service account name (required)
--host string — target host (required)
--out string — ephemeral key/cert output directory (default: $ISG_ISSUED_DIR, else $XDG_STATE_HOME/isg/issued, else ~/.local/state/isg/issued); "-" writes the key to stdout instead of a file
--pubkey string — path to existing public key file (optional; generates ephemeral key if omitted)

isg ca join-token

Manage one-time bootstrap tokens for service account enrollment

isg ca join-token create

Create a one-time join token for service account enrollment

isg ca join-token create --account <account> [--ttl <duration>] [flags]
Flag Type Default Description
--account string — service account name (required)
--ttl duration 15m0s token validity duration (e.g. 15m, 1h)

isg ca join-token list

List issued join tokens (without the raw token value)

isg ca join-token redeem

Redeem a join token and issue a renewable identity certificate

isg ca join-token redeem --token <token> [--out <dir>] [flags]

Validates the one-time join token and issues an X.509 client certificate (renewable identity) for the associated service account.

The private key is generated HERE, on this machine, and never leaves it — only a PKIX-PEM public key crosses the local issuance socket to the daemon, which signs a certificate over it and never sees the private key. The daemon separately keeps its own recording copy of the signed certificate under $ISG_HOME/ca/renewable/<account>.crt (read by 'isg ca list' and 'isg ca revoke'); this command writes the operator-owned working copy to --out (default "."): <out>/<account>.crt (0644) — X.509 client cert <out>/<account>.key (0600) — private key (PKCS8 PEM)

This is the Phase 3a local test path. In Phase 3b the agent will call the /ca/renew HTTP endpoint instead of this CLI command.

Flag Type Default Description
--out string . directory to write the renewable identity cert+key to
--token string — raw join token from isg ca join-token create (required)

isg ca list

List renewable identities and revoked serials

Shows the renewable identity certificates in $ISG_HOME/ca/renewable/ (account, serial, expiry, revoked status) and, separately, any revoked serials in revoked.yaml that no longer have a matching cert file.

Read-only: daemon-mediated since ADR 670892035 C2c (the "list-renewable" op over the local issuance socket — see issuesock.ListRenewable) — the CLI no longer reads caDir directly.

isg ca pubkey

Print the CA public key (authorized_keys format)

Print the SSH CA's public key in OpenSSH authorized_keys format: the line a target host has to trust through TrustedUserCAKeys in its sshd_config before it will accept any ISG-issued user certificate.

Reads the published, operator-readable copy in pub/, never the private ca_key. The output does not need the private half, and on the isolated CA layout the ca/ directory is not traversable by the invoking operator at all. Reading the raw ca/ path is upgrade-skew handling only, for a deployment where the published mirror has not been backfilled yet.

No daemon is contacted, which is deliberate: provisioning a target's TrustedUserCAKeys has to be possible before 'isg serve' is running.

isg ca revoke

Revoke a service account's renewable identity certificate

isg ca revoke --account <account> [flags]

Revokes the renewable identity certificate for the given service account.

The certificate's serial is added to $ISG_HOME/ca/revoked.yaml. The /ca/renew endpoint checks this list on every request — a revoked certificate is denied with 403 immediately, before any policy or gate check.

This is a gateway-side control: the target host sshd config is not touched.

Flag Type Default Description
--account string — service account name (required)

isg ca server-cert

Manage this gateway's own TLS serving certificates (webUI, CA renew endpoint)

isg ca server-cert renew

Re-issue this gateway's TLS serving certificate for one hostname

isg ca server-cert renew --hostname <name> [flags]

Replaces the TLS serving certificate this gateway presents for <name>, along with its private key: a renewal generates a fresh ECDSA P-256 key and a fresh serial, it does not merely re-date the old one.

THIS IS THE ONLY THING THAT RENEWS THESE CERTIFICATES. The daemon never replaces one on its own — not at startup, not on a timer, not when the deadline gets close. isg serve issues a certificate the first time it needs one for a hostname and then serves it for as long as it exists. Renewing is an operator's explicit act, and it is audited as one (SERVER_CERT_RENEWED).

THE NEW CERTIFICATE REACHES THE WIRE ONLY AT THE NEXT isg serve START. Both listeners load their certificate once, when they start, so until the daemon is restarted the file on disk and the certificate being served are two different certificates — and isg system check measures them separately, which is how you see that window rather than guess at it:

server_cert_expiry reads the files under ca/server/ (what you just wrote) webui_cert_expiry what the webUI port actually presents ca_renew_cert_expiry what the CA renew port actually presents

A disagreement between the first row and the other two, right after this command, is the expected state and not a fault. It closes on restart.

WHICH HOSTNAME. Each listener has its own name and therefore its own certificate file, and this command renews exactly one of them:

--ui-hostname the webUI's name (isg serve) --ca-renew-hostname the renew endpoint's name (isg serve)

Pass the same value here that isg serve was given. A name isg serve does not use is not an error — it issues a certificate nothing serves.

Flag Type Default Description
--hostname string — the serving name whose certificate to renew — the value of isg serve's --ui-hostname or --ca-renew-hostname (required)

isg connect

Connect to a target host through the ISG

isg connect [target] [flags]
Flag Type Default Description
--as string — OS user to SSH as on the target (required)
--change string — Change-management ID to associate with this session (free-form, optional)
--command string — run a single command through the daemon-mediated session and exit with its exit code, instead of opening an interactive shell

isg jira

JIRA ticketing integration helpers

isg jira verify

Verify JIRA connection (auth + project key)

Runs two read-only checks against the JIRA instance configured in $ISG_HOME/jira.yaml without opening or modifying any issues:

  1. auth — GET /rest/api/2/myself (verifies credentials)
  2. project — GET /rest/api/2/project/<key> (verifies project_key exists)

Credentials are read from jira.yaml and never printed.

isg license

Show the ISG license in force

isg license show

Verify the local license.yaml with the built-in keys and show it, with the evaluation clock

isg lock

Lock a user or a Web Access application out of every entry point

isg lock add

Lock a user or an app and close its running sessions

isg lock add (--user NAME | --app ID|NAME) --reason TEXT [--for 2h] [--force] [flags]
Flag Type Default Description
--app string — Web Access application to lock: its id (APP-…) or its apps.yaml/webUI name, which the daemon resolves to the id; an unknown one is an error
--for duration 0s lock duration (e.g. 2h); 0 = until removed
--force bool false lock a --user name the identity source does not know (without it, an unknown name is an error)
--reason string — why (required, recorded in the audit log)
--user string — username to lock

isg lock list

List the locks (expired ones marked)

isg lock rm

Lift a lock

isg lock rm (--user NAME | --app ID|NAME) [flags]
Flag Type Default Description
--app string — locked Web Access application: its id (APP-…) or its name (also lifts a lock made on the name before B491)
--user string — locked username

isg mfa

Manage TOTP multi-factor enrollment (ADR 631439362)

isg mfa enroll

Enroll (or re-enroll) a user for TOTP-based MFA

Generates a new TOTP secret + 10 recovery codes for the target user, shows a QR code for an authenticator app (plus the secret as text, for manual entry), and asks for one confirmation code before persisting anything — an enrollment that isn't confirmed against a real authenticator app is discarded, not saved half-done.

This is self-service only: you can only enroll yourself. The --user flag, if given, must match the current OS user (it exists to name your own identity explicitly); enrolling another user is refused.

Daemon-mediated (B108, phase 4b): this dials 'isg serve's local issuance socket in two steps (mfa-enroll-begin, then mfa-enroll-confirm) rather than generating the secret and writing $ISG_HOME/totp or audit.log itself — there is no in-process fallback, so 'isg serve' must be running. The daemon determines who is being enrolled from the connecting OS user (SO_PEERCRED), never from a client-supplied field, so the self-service rule holds even though the --user check below is only a client-side convenience.

Flag Type Default Description
--force bool false overwrite an existing enrollment without confirmation (invalidates the old authenticator app entry)
--user string — your own identity to enroll (must match the current OS user; enrolling another user is not allowed)

isg mfa reset

Reset (delete) a user's TOTP enrollment — admin operation

isg mfa reset <user> [flags]

Deletes the target user's TOTP secret and recovery codes. The user must run "isg mfa enroll" again before they can log in under totp or totp-with-email-fallback mode.

Daemon-mediated (B108, phase 1): this dials 'isg serve's local issuance socket rather than touching $ISG_HOME/totp or audit.log itself — there is no in-process fallback, so 'isg serve' must be running. Authorization is gated daemon-side on the mfa_reset_group directory group (like approver_group gates decisions). When no LDAP or no mfa_reset_group is configured, authorization falls back to OS-level trust (whoever can run isg on the gateway host). Both this path and the webUI share authorizeMFAReset.

isg policy

Inspect and validate command policies

isg policy check

Show how the policy evaluates one command line, without running it

isg policy check -- <command line> [flags]

Evaluate one command line against a policy and print the whole reasoning: how the line decomposes into simple commands, which constructs cannot be evaluated before it runs, every candidate text the engine matches rules against, and the rule that decided.

The engine evaluates the raw line as typed, each simple command inside it (either side of ';', '&&', '||', '|', '&', inside '( )', '{ }', a loop or conditional body, and inside a command substitution), and each of those with a leading 'sudo' and the known command-prefix wrappers stepped over. The STRICTEST outcome wins: deny beats require_approval beats no match.

Nothing is run, no daemon is contacted and nothing is written. Pass the line after '--' so its own flags and quoting reach this command intact.

--file evaluates against a policy file other than the gateway's own; --level and --role evaluate as a different host lockdown level or user role, which is how a rule carrying min_level or roles is exercised from a workstation.

Exit status is the decision:

0 allowed — no rule matched 1 denied — a deny rule matched 2 approval — a require_approval rule matched

Flag Type Default Description
--file string — policy file to evaluate against (default: $ISG_HOME/policy.yaml)
--level int 0 server lockdown level to evaluate at (min_level rules below it are inactive)
--role string — user role to evaluate as (empty: the role gate is open and every rule fires)

isg policy show

Show the effective policy (file or defaults)

isg policy validate

Parse and validate a policy file without loading it

isg policy validate <file> [flags]

Load the file the way 'isg serve' would, and report the first error instead of starting anything. Format follows the extension: .yaml, .yml or .json, and anything else is itself an error.

What that covers, beyond the file's own syntax: the sudo allow-list spec, an unknown rule category, an empty pattern, and the compilation of every rule's regex. A pattern that does not compile is a load error here, not a rule that silently never matches later.

Nothing is applied and no daemon is contacted. The policy a running gateway is actually enforcing is 'isg policy show'.

isg rbi-agent

Web Access isolation agent (runs on an RBI host)

isg rbi-agent run

Poll the brokers and run the isolated browser sessions (blocks; suitable for systemd)

Starts the RBI agent (B499). It polls every broker in rbi-agent.yaml over the CA renew listener's mTLS, runs each browser session in its own podman container and carries the container's app connections to the broker that owns the session.

The identity is a service account's renewable certificate (isg ca join-token create + redeem); its CN must be an agent in the brokers' rbi.yaml.

Flag Type Default Description
--config string — path to rbi-agent.yaml (default: $ISG_HOME/rbi-agent.yaml)

isg retention

Inspect and apply the stated data retention rule

isg retention apply

Delete the recordings, transcripts and state records the retention rule has expired (dry run unless --commit)

Flag Type Default Description
--commit bool false actually delete the expired items; without it nothing is removed and only the plan is printed

isg retention status

Show the retention rule and how the stored state classifies against it

isg serve

Launch the browser admin view

Flag Type Default Description
--bind string 127.0.0.1 Bind address for admin UI
--ca-renew-bind string 0.0.0.0 Bind address for the mTLS CA renew endpoint
--ca-renew-hostname string — Hostname for the renew endpoint server cert SAN (flag > $ISG_CA_RENEW_HOSTNAME > os.Hostname())
--ca-renew-port int 8766 Bind port for the mTLS CA renew endpoint
--gateway-sock-group string isg-gateway Group the local gateway auth UNIX socket is bound to (separate from isg-operators)
--gateway-sock-mode string 0660 Permission mode (octal) for the local gateway auth UNIX socket
--issue-sock-mode string 0600 Permission mode (octal) for the local cert-issuance UNIX socket
--port int 8765 Bind port for admin UI
--secure-cookie bool false Set Secure flag on session cookies (auto-enabled when TLS is active)
--session-sock-mode string 0660 Permission mode (octal) for the local session UNIX socket
--ui-hostname string — Hostname for the admin UI TLS cert SAN (flag > $ISG_UI_HOSTNAME > os.Hostname())

isg servers

Inspect per-server lockdown levels

isg servers level

Show the resolved lockdown level for a host

isg servers level <host> [flags]

isg servers show

Show the configured server inventory, the CMDB settings behind it, and both default levels

Show where the lockdown levels come from: servers.yaml's source, its default level and its pinned hosts, and — when cmdb.yaml is present — the resolver, the CMDB's own default_level (which is what a host NOT pinned in servers.yaml actually resolves to), the label_rules count, the on_unavailable posture, the cache TTL, the static resolver's mock_file and the state of the CMDB cache.

Read-only in the strict sense: this command never writes the CMDB cache file, even when it resolves a host through the CMDB (B396). That file belongs to the daemon.

--hosts additionally lists the hosts a static resolver's mock_file names. Those are not shown by default because a real CMDB connector has no such list to offer: it answers one host at a time, so only the static dev/test resolver can ever enumerate its hosts, and the host-level question for any resolver is answered by "isg servers level <host>".

Flag Type Default Description
--hosts bool false also list the hosts a static CMDB mock_file names

isg session

Session-socket diagnostics (daemon-mediated session transport, ADR 653328395)

isg session ping

Show how the daemon resolves this caller's identity over the session socket

Dials the local session socket (ADR 653328395, S1), performs the protocol-version handshake, and prints the identity the DAEMON established for this caller from SO_PEERCRED plus the directory lookup it drives — not anything this command claims about itself. That is the whole point of this round: today the CLI resolves its own LDAP identity and the daemon has no way to check it; here the daemon decides, and this command just reports what it decided.

This is transport-only today: no SSH, no MFA, no authorization decision, no session yet. Session mediation itself is a later round under the same ADR.

isg sessions

Manage ISG sessions

isg sessions close

Mark an active session as closed

isg sessions close <SESSION-ID> [flags]

isg sessions list

List all sessions

isg sessions show

Show details of one session

isg sessions show <SESSION-ID> [flags]

isg storage

Move closed sessions' recordings to the stream store, and verify they are there

isg storage archive

Move closed sessions' recordings into the stream store (dry run unless --commit)

Flag Type Default Description
--commit bool false actually move the recordings; without it nothing is written, nothing is deleted, and only the plan is printed

isg storage verify

Check that every recording a session record calls archived is in the stream store, at the size it was archived at

Flag Type Default Description
--deep bool false also read every archived recording back in full and compare its SHA-256 to the one recorded at archival time; without it only existence and size are checked. Off by default because the read-back is disk I/O on a directory backend and a per-byte, per-request DOWNLOAD of every archived recording on an object store

isg system

ISG installation diagnostics

isg system audit-chain

Show the audit log's hash chain head and how much of the log is chained

Reports the audit log's hash chain (B216): the current head, how many lines carry a chain link, and how many predate the chaining.

WHAT THE CHAIN IS. Every audit line written since this feature landed carries the SHA-256 of the line physically above it, so an edit or a deletion in the middle of the file breaks the chain from that point on — 'isg system audit-chain verify' walks the trail and names the first line that no longer follows from the one above it.

WHY THE HEAD MATTERS MORE THAN THE LOCAL CHECK. The file lives on this broker's disk, so anyone who can rewrite it can recompute every hash after their edit and hand back a chain that verifies here. What they cannot reach is the heads that already left the machine: 'isg serve' writes an AUDIT_CHAIN_HEAD audit entry at startup and every hour, and that entry goes out on the normal audit fan-out — into the SIEM, over the syslog export. A rewritten log has to reproduce every head exported before the edit, over bytes that are no longer there.

So the head printed here is the value to compare against the last one your SIEM received. An empty head means an empty log, not a problem.

The counts describe the LIVE file only. The rotated pieces beside it are named at the bottom; 'verify' is what walks them.

Changes nothing.

isg system audit-chain verify

Walk the rotated pieces and the live audit log, checking every chain link

Walks the audit trail on this disk — the rotated pieces oldest first, then the live audit.log — and checks every line that carries a chain link against the line physically above it, ACROSS FILE BOUNDARIES.

The pieces are walked in one pass on purpose: the link a rotation created, from the first line of a new file to the last line of the piece just rotated away, is exactly the one a per-file check could never see.

Lines from before the chaining existed are counted and reported as a summary, not as an error — there is nothing in them to verify.

Exits NON-ZERO on the first break, naming the file and the line number, so this can be wired into external monitoring (cron + alert on non-zero). The walk stops there: everything after a break is measured against a starting point already known to be wrong.

A GREEN RESULT IS NOT A PROOF OF NON-TAMPERING. This file is on this broker's own disk, and whoever can rewrite it can recompute the hashes after their edit. The check that cannot be defeated locally is comparing the head against the AUDIT_CHAIN_HEAD entries your SIEM already received — see 'isg system audit-chain'.

Changes nothing.

isg system brokers

List the brokers writing to this state root, and whether each is still alive

Reads the broker leases and prints one row per broker: who it is, where it runs, and how long ago it last said so.

WHY THIS EXISTS. On a shared record store the orphan sweep cannot use a boot id or a PID to judge another broker's session records — those are readings from a machine it cannot see. It uses these leases instead: a broker whose heartbeat is older than the TTL is treated as gone, and its still-active sessions are marked orphaned by whichever broker noticed (audited as BROKER_LEASE_EXPIRED).

The state column is therefore the one to read during an incident. "live" means that broker's sessions are being left alone; "expired" means they are being, or are about to be, reaped.

A "DRAINING" marker on that column means the broker was put out of service on purpose ('isg system drain on'): it is refusing NEW sessions and serving out the ones it has, ahead of a planned restart. It says nothing about whether the broker is healthy — live/expired still does.

A record that cannot be parsed is listed separately rather than hiding the rest: its broker is treated as LIVE (so its sessions are left alone) until 'isg system brokers release' clears it.

WHERE THE LEASES COME FROM (--source). By default this asks the RUNNING DAEMON, over its diag endpoint, and only falls back to reading the record store directly if no daemon answers. That default exists so listing the brokers needs no database credential of your own: the daemon already holds the store open, and the diag token is something an isg-operators member can already read. Pass --source store to read the store yourself (what this command always did, and the only path when the daemon is down), or --source diag to require the daemon and fail rather than fall back.

Either way the first line of the output says which source answered.

Changes nothing. A lease is written only by the broker it belongs to.

Flag Type Default Description
--source string auto where to read the leases from: auto (the running daemon's diag endpoint, silently falling back to the record store), diag (the daemon only — no fallback), or store (the record store only)

isg system brokers release

Remove one broker's lease record, so a broker blocked behind it can start

isg system brokers release <broker-id> [flags]

Removes a single broker lease record from the store.

WHY YOU WOULD: a lease record that cannot be parsed blocks the broker it names from starting at all — an unreadable lease is treated as HELD, never as free — and before this command the only tool for it was 'rm', which does neither the check nor the audit entry. A corrupt record is removed without --force: it has no holder to protect.

An EXPIRED lease is also removed without --force; everything on this store already treats it as a dead broker.

A LEASE THAT IS STILL BEATING IS REFUSED unless you pass --force. Removing it tells every other broker that this one is gone, and their orphan sweeps will close its live sessions. The refusal names the holder — host, pid, heartbeat age — so you can go and check it before overriding.

The broker id is the record id: the broker's 'id:' from broker.yaml, or unmarked-broker for a broker that has none. 'isg system brokers' lists them.

Writes a BROKER_LEASE_RELEASED audit entry naming you and what was cleared.

Flag Type Default Description
--force bool false remove the lease even though its heartbeat is still inside the TTL — its brokers' live sessions will be orphaned by the next sweep

isg system check

Validate config files and probe connectivity to configured systems

Validates every known $ISG_HOME config file (existence, syntax, and dedicated loader/validator where one exists) and probes live connectivity to the systems ISG is configured to talk to (LDAP, X.509 CA, JIRA, SMTP, target hosts).

The report is MEASURED BY THE DAEMON: this command asks the running 'isg serve' for it over the session socket (the same measurement its webUI /system page and its /readyz make) and only renders the answer, so the output does not depend on which account runs it. The first line says so ("Measured by the daemon (pid N, user isg) over /run/isg/session.sock"). When the daemon is not running, not reachable from this account (not in the isg-operators group), or too old to know the request, the report is measured locally instead and the first line says that too — and rows that read files only the daemon may read (the record store's credential) then say "not measurable from this account" rather than failing.

Exits non-zero if any check FAILs, so this command can be wired into external monitoring (cron + alert on non-zero, Nagios/Zabbix, etc.).

--verbose breaks each live check down into its individual sub-steps (e.g. JIRA's auth/project/search), in the same style as "isg jira verify", and prints each row's CAVEAT — what that check did NOT measure — as an indented line under the detail. Without --verbose a row that has one ends in " …", and the report closes with a single line saying how many are hidden.

Rows stay one line each when the output is not a terminal (a pipe, a file, cron): no wrapping and no color, so monitoring integrations that parse this output are unaffected.

--color=auto|always|never asks for the color separately from that wrapping. auto (the default) is the rule above PLUS CLICOLOR_FORCE; always colors even on a pipe; never colors nothing at all, a real terminal included. Inside auto the order is: NO_COLOR (present with ANY value, the empty string included) beats CLICOLOR_FORCE (set to anything but "0"), which in turn beats the terminal test and TERM=dumb. An explicit --color=always or --color=never is stronger than all three — the flag is the operator saying it on purpose.

COLOR DOES NOT BRING THE WRAPPING WITH IT. The width is still measured only on a real terminal, so --color=always on a pipe gives colored but UNWRAPPED rows: one row per line, exactly what a parser reads today. That is the point of two separate decisions rather than one — asking for color must not also reshape the monitoring form.

For a logged, non-interactive run from another host, either ask for the color:

ssh gateway isg system check --color=always | tee check.log

or allocate a terminal (ssh -t gateway isg system check), which is what makes the default auto say yes on its own.

--fleet groups its rows under a per-broker heading ("-- broker R1 — redhat1 --"). Headings are NOT check rows: a parser reads the lines starting with [OK, [WARN or [FAIL, which is what the section titles, the separator lines and the Summary line have always required. Each check row keeps its own broker prefix in its name, so a single line grepped out of a pipe still names the broker it is about.

Flag Type Default Description
--color string auto when the report's status markers may carry an ANSI color: auto|always|never. auto (the default) colors a real terminal, honouring NO_COLOR (present with any value) and TERM=dumb, and on a NON-terminal colors when CLICOLOR_FORCE is set to anything but "0" — inside auto, NO_COLOR beats CLICOLOR_FORCE, which beats the terminal test. always colors regardless of the terminal, of NO_COLOR and of TERM (an explicit flag is the stronger statement); never colors nothing at all, a terminal included. Color never brings the wrapping with it: the report still wraps only on a real terminal at its measured width, so --color=always on a pipe gives colored but unwrapped one-line rows — use it for a logged non-interactive run (ssh gateway isg system check --color=always), the alternative being ssh -t. Any other value is an error and no report is printed.
--deep bool false extend the "CA key isolation" row with a content-based scan for stray copies of the CA's signing keys: walks a fixed set of common locations ($HOME, /home, /root, /tmp, /var/tmp, /opt, /srv; depth- and size-limited, no symlinks followed, no mount crossing) and identifies a match by comparing each candidate file's key material against the CA's own public key/certificate — a copy simply renamed or moved evades the default check (which only stats one known alternate path) but not this one. Time-bounded (15s by default); an operator account can only see what it has read access to, so this is necessarily partial — run as root (sudo isg system check --deep) for a fuller answer. Local filesystem only, no network probe, no audit entry.
--fleet bool false add a per-broker section read from the broker leases in the shared record store: one row per broker (live/EXPIRED plus heartbeat age), one row per endpoint that broker advertised having opened (TCP connect: OPEN/CLOSED — an OPEN port beside an EXPIRED lease is a FAIL, since something is holding the socket while the broker that owns it stopped writing its heartbeat), one row comparing the LOCAL broker's actually-open TCP ports against the set it advertised (read from /proc, not from ss/netstat; needs permission to read the daemon's own /proc entry, otherwise the row says so), and one row per ISG systemd unit on this host. A remote host's full port scan and a remote systemd's state are NOT in scope — the rows say so as caveats rather than passing silently. Changes nothing; opens no session.
--local bool false measure from THIS account instead of asking the running daemon for its report (B276). The debug path, not the default: the daemon's report is the authoritative one (it reads files this account may not, and it is what the webUI /system page and /readyz show); --local exists to compare the two, or to see what a given login account can measure on its own. The report's first line says which path was taken either way.
--probe-hosts bool false also attempt live SSH connections to each target host (servers.yaml entries plus hosts with an issued host cert, minus the brokers' own front certs — a broker is not a brokered target unless servers.yaml lists it): reachability, the bypass probe, human-cert coverage, and the break-glass auth-method probe (does the isg-breakglass account's sshd Match block still refuse password-family auth). All four log in to the target; each is audited separately (REACHABILITY_PROBE, BYPASS_PROBE, HUMAN_CERT_PROBE, BREAK_GLASS_AUTH_METHOD_PROBE). Default: DNS resolution only.
--probe-keyset bool false read and parse each target's break-glass keyset (isg-breakglass account, /etc/ssh/breakglass_authorized_keys: mode, owner, and per-key options such as restrict) via a daemon-mediated SSH probe, then offer each key to isg-breakglass and observe whether the target accepts it (audited as BREAK_GLASS_PROBE) — this second step is also what feeds the readiness row's break_glass condition (isg system check's per-host "Ready to broker" row). Independent of --probe-hosts. Unlike every --probe-hosts probe (which deliberately stop short of a completed login), the keyset read itself performs a REAL login to each target — an "Accepted publickey" line in the target's own sshd log, not a preauth close — and is audited as BREAK_GLASS_KEYSET_READ. A cert-authority line (B149, the offline break-glass CA the target trusts) is never offered this way — a CA public key is not an authentication key — it is instead verified by comparing its fingerprint against breakglass_ca_fingerprint. Default: off.
--probe-without-audit bool false run --probe-hosts' probes even though the gateway won't record an audit entry for them. The probes open real SSH connections to target hosts, which the target's own sshd log will show; this flag runs them without that being backed by an ISG audit entry, since 'isg serve' is not running to record one. Only for an urgent measurement when the daemon is down — requires --probe-hosts.
--verbose, -v bool false show sub-steps behind each live check (jira verify style) and each row's caveat — what the check did NOT measure

isg system drain

Stop accepting new sessions on this broker, ahead of a planned restart

isg system drain on|off [flags]

Puts this broker into (or out of) DRAIN MODE.

WHAT DRAIN DOES: the broker refuses NEW sessions and keeps serving the ones it already has. Nothing is closed, nothing is moved to another broker, and no session is interrupted. An operator who is about to restart the gateway drains it first, waits for the running sessions to end on their own, and only then stops the daemon — instead of cutting every live shell at once, which is what a restart does today.

WHAT KEEPS RUNNING: commands on already-open sessions, approvals, session closes, the admin UI, and the CA operations. A drain that froze those would make the state impossible to get out of.

WHAT ELSE CHANGES: /readyz answers 503, so a load balancer takes this broker out of rotation and sends new connections to a sibling. The response says explicitly that the failure is an intended operator state, not a fault. isg_broker_draining goes to 1. 'isg system brokers' (and 'isg system check --fleet') mark the row.

THERE IS NO DEADLINE and no automatic shutdown. Drain is a state; stopping the daemon is your decision, and nothing here makes it for you.

IT DOES NOT SURVIVE A RESTART. The gate is in-memory and the lease's flag is cleared by the next start, so the broker comes back serving — which is what you want, since a restarted broker that silently stayed drained would be a gateway missing from the fleet with nothing to show why.

Requires 'isg serve' to be running: the state lives in the daemon.

Writes a DRAIN_ON / DRAIN_OFF audit entry naming you.

isg system lease

Inspect or clear the state root's exclusive-ownership lease

The state root ($ISG_HOME) is written by exactly one 'isg serve' at a time, and serve.lease is what enforces it: every sessions.json and approvals.json update is a full-file rewrite, so a second daemon over the same directory would not corrupt those files, it would silently discard the first daemon's sessions.

'status' shows who holds the lease and whether a start would be allowed right now. 'release' clears a lease left behind by a broker that is gone, after checking that its holder is not a live process on this machine.

isg system lease release

Clear a state root lease whose holder is gone

Removes serve.lease so a new 'isg serve' can take this state root.

REFUSES while the holder is a live process on THIS machine: removing that lease would let a second daemon start beside a running one, which is the split-brain the lease exists to prevent. Stop that process first.

It does NOT refuse when the lease names another host or a previous boot — those are exactly the cases a machine cannot settle for itself and a human can. That is also why this is a command and not a timeout: a lease that expired on its own would hand back the split-brain.

Writes a STATE_LEASE_RELEASED audit entry naming you and the holder you cleared. Use this instead of 'rm': rm does neither the check nor the entry.

isg system lease status

Show who holds this state root's lease, and whether a start would be allowed

Reads serve.lease from disk (never from a running daemon's memory) and reports its holder, the age of its heartbeat, and what 'isg serve' would do about it if started here right now.

Changes nothing. The heartbeat's age is evidence for you, not a timer: no age, however old, ever unlocks the lease by itself.

isg system provision

Create the OS group and account the gateway login path needs (B23)

Idempotently creates the gateway socket's OS group and the gateway login account (the account sshd's ForceCommand admits a human as — see packaging/sshd_config.d/60-isg-gateway.conf), and adds the daemon's own OS account to that group so it can chown the socket to it.

Refuses to touch anything else: it never writes /etc/pam.d/sshd or /etc/ssh/sshd_config.d/, and never restarts a service. Run 'isg system check' afterward to confirm the posture; the two remaining manual steps are printed at the end of this command's own output.

Flag Type Default Description
--dry-run bool false print what would be done without changing anything

isg system tasks

List the gateway-level background tasks and which broker is running each

Reads the gateway task leases from the record store and prints one row per task: which broker holds it, where that broker runs, and how long ago it last claimed a tick.

WHY THIS EXISTS. Two of 'isg serve''s loops are not per-broker work: the recording archival and the retention sweep. Both walk every session record on a shared store and both write, so on several brokers they must run exactly once per tick across the whole fleet, not once per broker. Each tick, every broker tries to claim the task's lease and only the winner runs the pass — so an empty listing means nobody has run these tasks since the state root was created, and a stale heartbeat means the holder stopped asking.

The heartbeat column is the one to read during an incident. It is NOT a background beat: it is the last tick on which some broker claimed the task, so an age far past the tick interval means the task is not running.

A record that cannot be read is listed separately: while it is there NO broker runs that task — a record that cannot say who is running the pass cannot be assumed free — and 'isg system tasks release' clears it.

Changes nothing.

isg system tasks release

Remove one gateway task's lease record, so the task can run again

isg system tasks release <task> [flags]

Removes a single gateway task lease record from the store.

WHY YOU WOULD: a lease record that cannot be read stops that task on EVERY broker, permanently — an unreadable lease is treated as HELD, never as free, and nothing ages out a record whose heartbeat cannot be parsed. Before this command the only tool for it was 'rm', which does neither the liveness check nor the audit entry. A corrupt record is removed without --force: it has no holder to protect.

An EXPIRED lease is also removed without --force; the next tick on any broker would have taken it over anyway.

A LEASE WHOSE HOLDER IS STILL CLAIMING TICKS IS REFUSED unless you pass --force. That broker may be running the pass right now, and clearing the record lets the next broker's tick start the same pass alongside it — which is exactly what the lease exists to prevent. The refusal names the holder — host, pid, heartbeat age — so you can go and check it before overriding.

The task name is the record id: archive, purge. 'isg system tasks' lists them.

Writes a GWTASK_LEASE_RELEASED audit entry naming you and what was cleared.

Flag Type Default Description
--force bool false remove the lease even though its holder claimed a tick inside the TTL — if that broker is in fact running the pass, another broker will start the same pass alongside it