Architecture

The Inshipia Security Gateway (ISG) is a daemon, isg serve. Operators reach it over a local socket or a browser; it reaches the target hosts over SSH, and — with Web Access — the web admin consoles over HTTP(S). Everything else — the identity source, the mail relay, the optional back-ends — hangs off that daemon and can be left out.

One daemon on one host is the shipped default, and everything below describes it. More than one daemon can serve the same estate at the same time, sharing their state rather than dividing the work; what that changes is set out under Running more than one gateway, and nothing else on this page assumes one or the other.

Components

System overview: operator surfaces, the ISG gateway daemon, target hosts and the systems the gateway talks to

The gateway is the only thing that holds a credential for a target host, and it holds each one for minutes. An operator's isg connect does not open an SSH connection itself: it hands the request to the daemon over a local UNIX socket, and the daemon establishes who is asking from the kernel — not from anything the client says about itself. The browser reaches the same daemon on its own TCP port for the admin view and the approval screens.

Web Access is the same gateway for web admin consoles — BMC interfaces, monitoring and storage consoles. The browser reaches the gateway's application listener (apps.yaml: listen) under the application's own host name; the gateway checks the user against the application's groups, approval and incident locks, evaluates every request against policy.yaml's http_rules, records it in the session transcript, and only then forwards it to the console (upstream). The console learns who the user is from a signed assertion (identity: jwt), from nothing at all (identity: none), or not at all because the gateway signs in with a shared account from OpenBao (identity: credential — a header, a basic login, or the console's own login form with kind: form). With a form login the console's session cookies stay in the gateway's memory: the browser holds only the gateway's own application cookie.

An application set to mode: isolated does not open in the user's browser at all. The gateway has an isolation (RBI) host start a disposable browser for the session, sends the user only its picture over the Guacamole protocol (guacd, rbi.yaml: agents), and takes the user's input back. The isolated browser reaches the console only through the gateway. The RBI agent connects to the gateways' renewal listener with mutual TLS (rbi-agent.yaml: brokers): its commands, the browser's traffic and the recording upload all go that way, and the RBI host holds no credential and no storage key.

Three systems are ordinary dependencies rather than options: the target hosts (SSH), the identity source that resolves an operator to a directory identity and its group memberships, and — if multi-factor codes go by e-mail rather than TOTP — a mail relay. The audit log is a local file that a SIEM or log shipper reads; an opt-in syslog copy (audit.yaml: syslog) can push every entry to a SIEM as well, but the file stays the audit of record either way.

A fourth external system is optional and supplies only the login: an OIDC identity provider, named in sso.yaml, behind the browser login to the admin view and behind the device flow of the gateway login. It never supplies the identity or the authorization — the name it returns is a lookup key, and the directory gate stands exactly as it does for a password login.

The three back-ends drawn dashed are off unless a configuration file names them: an OpenBao instance as the certificate signer instead of the built-in one, PostgreSQL as the store for session and approval records instead of the local directory, and an S3-compatible object store as the archive for session recordings.

Dashed means optional for a single gateway. For more than one they are not: PostgreSQL is what the gateways share, and OpenBao becomes mandatory rather than merely available. A gateway that is named and uses PostgreSQL but not OpenBao refuses to start — see Running more than one gateway.

Connecting

Sequence: connect, identity resolution, access gates, MFA, certificate issuance, SSH to the target

A connect is settled by the daemon in a fixed order, and the order is the point. The caller's identity is established first, from the kernel-attested credentials of the socket connection, and resolved against the directory. The two non-interactive access gates — human_access in ca.yaml, then the per-host required group in hostgroup.yaml — run next, so a request that was never going to be allowed does not cost the operator a multi-factor challenge. Only then is the challenge issued, and only after it succeeds does a credential exist at all.

Approving a command

Flowchart: policy evaluation, approval request, the second person's decision, and the three outcomes

Every command in a session is evaluated on its own, against the policy file as it reads at that moment — policy.yaml is loaded per command, so an edited policy takes effect on the next command without restarting the gateway or the session. A rule can deny the command outright or send it to a second person. A command that matches nothing runs.

The built-in deny rules are part of every policy, including the policy of a gateway with no policy.yaml at all, and a user rule cannot override them: the file's own rules are evaluated between the built-in denials and the built-in approval rules, and the first match wins.

Opening a web console

A user opens My applications in the admin view, which lists only the applications their groups allow, and follows the application's link. The application host has no session yet, so it sends the browser to the admin view for a one-time hand-off: the user is already signed in there, and the hand-off checks the groups, the incident locks and the licence again. An application with require_approval asks for a duration first and waits for a second person's decision (approval_mode: four_eyes), or opens at once on a stated reason (approval_mode: self); either way the window has an end, and the application session closes with it. From then on every request is decided on its own: an incident lock, a lost group or an http_rules deny refuses it before it reaches the console, a step_up rule asks for a fresh second factor, and every request — allowed or not — is a line in the session's transcript.

State

Where state lives: the state root, the record back-end, the recording archive and the retention sweep

Running more than one gateway

Several gateways can serve one estate at the same time. They are peers: each one runs every role — the admin view, session brokering, CA operations, approvals — and none of them is a standby. What makes them one system rather than several is that they share their state.

More than one gateway: peer gateways sharing one record store, one object store and one OpenBao, brokering SSH to the same target hosts

What that buys is that losing a gateway does not close the door: operators keep opening sessions through the ones still standing. What it does not buy is the survival of sessions already running on the gateway that was lost. A live session is an SSH stream and a PTY on that particular machine; when the machine goes, the stream goes with it, and the operator opens a new session through a peer. The record does survive — a surviving gateway marks the session ended rather than leaving it open — so the audit trail has no gap.

Three things carry that shared state, and two things stay local no matter how many gateways there are:

What One gateway — local, under $ISG_HOME Shared layout Switch
Session and approval records sessions/, approvals/ PostgreSQL storage.yaml: records.backend
Broker leases, lockouts, revocations, login state, gateway tasks one directory per record family PostgreSQL — the same store, the same switch storage.yaml: records.backend
Recordings and transcripts of closed sessions recording-<SESSION-ID>.cast, transcript-<SESSION-ID>.jsonl S3-compatible object store storage.yaml: recordings.backend
The SSH certificate signer ca_key under ca/ OpenBao — the private key is never opened or required on a gateway: a seeded secondary has no ca_key at all, only the pinned ca_key.pub; the primary keeps the copy it imported into the OpenBao ca.yaml: signer
Multi-factor secrets totp/ OpenBao (KV) mfa.yaml: store
The audit log audit.log local on every gateway, always —
The X.509 CA, the public mirror, the state-root lease, the event ticket outbox ca/, pub/, serve.lease, ticket_outbox/ local, always —

A local directory in place of any of the shared rows is a single-gateway layout, not a shared one, and putting that directory on a network file system does not make it shared: serve.lease records which host owns the state root, and a gateway on another host refuses to start on it.

Each gateway carries a name of its own — broker.yaml's id, one to eight alphanumeric characters — and stamps it onto every session record and every audit line, so a record read from the shared store says which gateway produced it. isg system brokers lists the fleet.

Two of the startup refusals in Fail-closed posture exist for this layout. The first is that an unnamed gateway will not join a store another gateway is already using. The second is that a gateway which is both named and pointed at PostgreSQL must also use OpenBao for signing and for multi-factor secrets. The reason is that the alternatives are correct alone and quietly wrong together: the built-in signer counts certificate serials per process, so two gateways can issue the same serial to different certificates and a revocation list stops naming one certificate; and a multi-factor secret written to one gateway's disk does not exist on its peers, so an operator who enrolled on one cannot log in through another. Both failures would appear far from their cause — at a revocation that does not bite, at a login that does not work — which is why the check happens at startup instead.

Some things stay local on purpose. Each gateway writes its own audit log: the file is append-only by contract, and a shared writer would trade that contract for convenience. The fleet-wide view is assembled by exporting the lines, not by pooling the file. The event ticket outbox is local for the same reason a shared one would need a shared record to hang the work on, and a revocation has none. A gateway that goes down therefore leaves its own event tickets until it returns — a delay, not a loss, and the audit log covers the gap. Session tickets are the opposite case: they hang off the shared session record, so a peer picks up an interrupted one after fifteen minutes.

For the operator, two consequences are worth planning around. The admin view keeps its login state in the shared store, so gateways can sit behind one name or a load balancer with no sticky-session requirement. And the probe rate against a target multiplies by the number of gateways: the interval that keeps probes from tripping a target's lockout is enforced per gateway, deliberately, because it is a load property rather than a correctness one.

Fleet topology

Everything above describes one gateway. An estate in production is a small fleet, installed by the Ansible playbooks: the gateways, plus the hosts that carry their shared state, their identity source and their monitoring. This section names those host roles, which port each one opens, and to whom. The gateway's own ports and sockets stay under Network and ports below.

A typical estate: two brokers, one shared store, one directory host, one monitoring host and any number of target hosts.

Host roles

Role Inventory group What runs there How many
Broker isg_brokers The gateway itself (isg_gateway, isg_broker_e1) and the break-glass log receiver (isg_breakglass_receiver). When a VIP is configured, isg_keepalived holds it — installed after the main run, once every broker is healthy Two or more; they share state, not work
Shared store isg_store PostgreSQL for the session records, RustFS (S3) for the recordings, and OpenBao (isg_shared_store, isg_bao_init) One
Directory host isg_infra OpenLDAP, Keycloak and Mailpit (isg_directory). Mailpit is a test mail sink only: it receives the multi-factor codes over SMTP One
Monitoring host isg_monitoring Prometheus and Grafana (isg_monitoring) One
Isolation (RBI) host isg_rbi The RBI agent, guacd and the disposable browser containers (isg_rbi) — only when an application uses mode: isolated Optional; one or more
Target host isg_targets The customer's servers. The playbooks set them up for certificate login and break-glass access (isg_target_host, isg_breakglass_target); their host firewall stays with the customer's own baseline Any number
Jump host isg_jumphost An optional host that testers log in from. Its playbook is separate and is not part of the gateway installation Optional
Controller — The machine that runs Ansible. It is not an inventory host One

Every broker, store, directory and monitoring host also gets the firewall baseline (isg_firewall) and the node exporter (isg_node_exporter). The target hosts get neither: they belong to the customer.

Service, port and source

Each row is a firewall rule per source, opened by the role that owns the port. An empty source list opens nothing — it never means "everyone".

Port Where Opened by Source
22 (SSH) brokers isg_firewall isg_net_admin_sources and isg_net_user_sources — on a broker, port 22 is also the gateway login
22 (SSH) store, directory, monitoring isg_firewall isg_net_admin_sources
VRRP (IP protocol 112) brokers isg_firewall the other broker, only when isg_vip_address is set
8765 (admin view) brokers isg_gateway isg_net_user_sources and the other brokers (for the fleet-wide self-check)
8444 (Web Access application listener, isg_apps_port) brokers (and the VIP) isg_gateway isg_net_user_sources — only when Web Access is on (isg_cfg_apps or a dynamic-app domain is set)
8766 (certificate renewal, mutual TLS) brokers isg_gateway isg_net_target_sources, the RBI hosts (isg_rbi) and the other brokers
8767 (diagnostics and metrics) brokers isg_broker_e1 (from diag.yaml's listen) isg_monitoring_host and the other brokers
9100 (node exporter) every fleet host isg_node_exporter isg_monitoring_host
5432 (PostgreSQL, session records) store isg_shared_store the brokers
9000 (RustFS S3, recordings) store isg_shared_store the brokers
8200 (OpenBao) store isg_shared_store the brokers
4822 (guacd, TLS) RBI hosts isg_rbi the brokers only; with an empty list the role does not open the port, and says so
636 (LDAP over TLS) directory host isg_directory the brokers
389 (LDAP, unencrypted) directory host isg_directory the brokers — only during the transition phase; once the directory is TLS-only, the port is not published at all
1025 (SMTP, multi-factor codes) directory host isg_directory the brokers
8443 (Keycloak HTTPS) directory host isg_directory isg_net_user_sources and the brokers (the browser, and the daemon's discovery and key fetches)
8025 (Mailpit web view) directory host isg_directory isg_directory_firewall_admin_sources (default isg_net_admin_sources) and, when Mailpit is a Web Access application, the brokers (isg_directory_firewall_mail_ui_app_sources)
3000 (Grafana) monitoring host isg_monitoring isg_monitoring_firewall_admin_sources (default isg_net_admin_sources) and, when Grafana is a Web Access application, the brokers (isg_monitoring_firewall_app_sources) — the reference environment opens it to the brokers only
9091 (Prometheus) monitoring host isg_monitoring the same two lists as Grafana
20514 (break-glass log receiver, TCP syslog) brokers isg_breakglass_receiver isg_net_target_sources — the target hosts send the break-glass lines. Here an empty list stops the run: a receiver no line can reach would only show up as a warning that never clears

SSH from a broker to a target host depends on the target's firewall, which is the customer's. Grafana reaches Prometheus on its own host over localhost, so no rule is needed for it.

Network

The installation has one network input: four values and a switch in the environment's group variables.

Variable Meaning
isg_firewall_enabled Whether the firewall baseline runs in this environment
isg_net_admin_sources Sources for SSH and the admin consoles
isg_net_user_sources Sources for the browser (admin view, Keycloak) and the gateway login on the brokers' port 22
isg_net_target_sources The target hosts, for certificate renewal and the break-glass log
isg_vip_address When set: VRRP is allowed between the brokers, and this is the address isg_keepalived brings up

Every other address comes from the inventory: the brokers' addresses and the monitoring host. Host names may resolve through any resolver, but the addresses the roles use do not depend on it: they are written out in the inventory. A host certificate's IP principal is not something the run can discover on the host, and on a host with more than one network leg the default address is not necessarily the one a broker dials.

Network and ports

Ports are the shipped defaults; each is a flag on isg serve or a key in a configuration file. The gateway's own local sockets are listed with them, because from an operator's point of view they are the same kind of thing: a door into the daemon. Where more than one gateway runs, each binds its own copy of every row below.

Direction Port or path Protocol When it is needed
Browser → gateway TCP 8765 (--bind 127.0.0.1, --port 8765) HTTPS once the CA is initialized, plain HTTP before that Always — the admin view and the approval screens
Browser → gateway (Web Access) TCP 8443 (apps.yaml: listen; the playbooks set 8444) HTTPS, with the certificate in apps.yaml: tls_cert_file (a *.<dynamic_apps_domain> SAN for webUI-managed applications) Only when apps.yaml names an application or a dynamic-app domain
Renewal agent → gateway TCP 8766 (--ca-renew-bind 0.0.0.0, --ca-renew-port 8766; the agent dials agent.yaml: ca_renew_url) HTTPS with mutual TLS Only for service-account certificate renewal, and only started once the CA is initialized
Monitoring → gateway TCP 127.0.0.1:8767 (diag.yaml: listen) HTTP — /healthz, /readyz, /metrics Only when diag.yaml sets enabled: true; no socket is bound otherwise
Operator CLI → gateway /run/isg/session.sock (mode 0660) local UNIX socket Always — this is the session path
Operator CLI → gateway /run/isg/issue.sock (mode 0600) local UNIX socket Certificate issuance, MFA enrolment and reset, approval decisions, closing a session
SSH login account → gateway /run/isg/gateway.sock (mode 0660, group isg-gateway) local UNIX socket Only when an identity source is configured
Gateway → target host TCP 22 SSH Always
Target host → break-glass log receiver breakglass.yaml: breakglass_log_target (the brokers' receiver: TCP 20514, or a SIEM) syslog, over TCP unless breakglass_log_protocol is udp When break-glass forwarding is set up: the target hosts' break-glass login and command trace
Gateway → identity source the port in ldap.yaml's url LDAPS (ldaps://, 636) is the recommended mode; LDAP with StartTLS (start_tls: true, 389) is equivalent in strength. An unencrypted connection needs allow_plaintext: true to say so, and the daemon refuses to start without it — so a directory should not offer its unencrypted port to the network at all When ldap.yaml is present
Gateway → mail relay smtp.yaml: host, port SMTP, with STARTTLS when tls: true When multi-factor codes are delivered by e-mail
Gateway → OpenBao vault.yaml: address HTTPS When ca.yaml sets signer: vault, or mfa.yaml sets store: vault
Gateway → PostgreSQL the connection string named by storage.yaml's records.postgres.dsn_file PostgreSQL When records.backend is postgres
Gateway → object store storage.yaml: recordings.s3.endpoint HTTPS (S3 API) When recordings.backend is s3
Gateway → SIEM audit.yaml: syslog.address RFC 5424 syslog over TCP with TLS When syslog.enabled is true — an opt-in copy of the audit log, never the record itself
Gateway → identity provider sso.yaml: issuer (OIDC) or saml_idp_metadata_url (SAML) HTTPS (OIDC discovery, token and device endpoints; the SAML IdP metadata) When sso.yaml sets enabled: true
Gateway → web console apps.yaml: each application's upstream (apps[].upstream) HTTP or HTTPS (upstream_ca_file for a private CA) For every proxied Web Access request, and for the gateway's own sign-in of a kind: form account
Gateway → OpenBao (Web Access accounts) vault.yaml: address, path <kv_mount>/data/<prefix>/<app>/<name> (<prefix> is apps.yaml: credential_kv_prefix) HTTPS When an application uses identity: credential; the broker only reads
Gateway → RBI host rbi.yaml: agents[].guacd (TCP 4822) Guacamole protocol over TLS, the server checked against guacd_ca_file Only when an application uses mode: isolated: the isolated browser's picture and the user's input
RBI agent → gateway rbi-agent.yaml: brokers (TCP 8766, the renewal listener) HTTPS with mutual TLS Only on an RBI host: the agent's commands, the isolated browser's traffic to the console (through the gateway) and the recording upload
Gateway → Jira jira.yaml: endpoint HTTPS (Jira REST API) When jira.yaml is configured and not disabled: the session tickets
Gateway → webhook notify.yaml: webhook.url HTTPS, a JSON POST When notify.yaml sets a webhook: each new approval request

The socket paths above are the system-service layout. When XDG_RUNTIME_DIR is set — a per-user daemon, typically during evaluation — the daemon binds $XDG_RUNTIME_DIR/isg/<name>.sock instead, and the CLI looks for /run/isg/<name>.sock first and falls back to the per-user path.

Two of these ports face the network by default and two do not, and the difference is deliberate. The admin view binds loopback until you widen it, and widening it is refused unless authentication and TLS are both in place (see Fail-closed posture below). The renew endpoint binds all interfaces because a renewal agent runs elsewhere by definition; its protection is mutual TLS, not the bind address. The diagnostic listener is loopback and disabled, on the assumption that the monitoring agent runs on the same host.

Certificate lifecycle

The gateway runs its own certificate authority, initialized once with isg ca init. It signs four different things, with four different lifetimes, and the differences follow from what each credential is for.

The human access certificate is what a session is actually built on. When a connect passes every gate, the daemon generates a fresh ed25519 key pair in its own memory, signs an SSH user certificate over it, and dials the target with that. It is valid for exactly ten minutes — five minutes either side of issuance, a window sized for clock drift during the handshake and nothing else — carries the permit-pty extension and nothing more, and is never written to disk. Neither the key nor the certificate is ever handed to the operator.

A service-account access certificate (isg ca issue --account <account> --host <host>) is the machine equivalent: one hour by default, or whatever ca.yaml's service_accounts[].ttl_seconds says for that account.

A renewable identity is the X.509 client certificate a service machine holds so its agent can renew access certificates over the mutual-TLS endpoint. It is issued by redeeming a one-time join token (isg ca join-token redeem --token <token>; the token itself defaults to fifteen minutes) and lasts thirty days unless ca.yaml's renewable_ttl_days says otherwise. The private key is generated on the service machine and never crosses to the gateway.

A host certificate (isg ca host-cert issue --host <host> --key <file>) is what lets the gateway recognise a target host. It is signed over the target's existing host public key — the CA never sees the host's private key — and lasts one year by default (ca.yaml: host_cert_ttl). There is no renewal agent on the target side, so the model here is a long lifetime plus revocation rather than short-lived rotation.

The signer behind all of this is chosen by ca.yaml's signer key: builtin, which opens the CA private key on the gateway, or vault, which delegates the signing to OpenBao and never opens a local key at all. The gateway's own TLS server certificates — for the admin view and for the renew endpoint — are derived from the same CA and last ninety days.

What happens at expiry differs by credential, and one case surprises people. SSH checks a certificate's validity only during the handshake. A session that connected inside its ten-minute window therefore keeps working indefinitely afterwards; there is no renewal and no mid-session re-issuance, by design. The lever for cutting a human off is refusing the next issuance — that is what the access gates do — not invalidating a certificate already in use. For the long-lived credentials it is the other way round: isg ca revoke --account <account> and isg ca host-cert revoke add a serial to the gateway's revocation list, which is consulted on every renew request and every dial, with no cache to go stale.

Who signs what: the SSH CA's builtin and vault branches with the three SSH certificates, the always-local X.509 CA, and the offline break-glass CA

The diagram keeps apart what the paragraphs above necessarily run together. The left column is one switch and everything downstream of it. The right column is an authority that switch never moves, which is why a gateway signing through OpenBao still keeps a private key of its own on disk. The dashed band is a third CA, drawn outside both columns because that is where it lives — on a machine this gateway cannot reach, and whose signatures it never produces.

Data at rest

Everything the gateway persists lives under one directory: $ISG_HOME, which is /var/lib/isg in the system-service layout. Configuration and state share it. Every configuration file is optional — a state root with no files at all is a valid first-run state — and each loader defines what its own file's absence means. The configuration reference lists every file and every key in it.

The state, as opposed to the configuration:

Path What it holds
sessions/ One JSON file per session record, mode 0640
approvals/ One JSON file per approval record, mode 0640
audit.log The audit log: append-only JSONL, one line per event, mode 0640
recording-<SESSION-ID>.cast The terminal recording of one session, capped at 5 MiB
transcript-<SESSION-ID>.jsonl The commands of one session — for a Web Access session, one line per request (method, path, status, the brokered account; with record_body, the masked body of a changing request)
apps-dynamic/ The applications managed on the web UI's Manage apps page (a record family, PostgreSQL in a fleet)
ca/ The CA's own key material — private keys mode 0600
pub/ The operator-readable mirror of the CA's public artifacts, mode 0644
serve.lease Which process currently owns this state root
totp_outbox.txt Multi-factor codes, when no mail relay is configured
ticket_outbox/ Ticket intents waiting to be retried

A kind: form account's upstream session cookies are deliberately not on this list: they live in the gateway's memory only, per application session, and a restart or a move to another gateway signs in again.

Which of these leave the directory for a shared store, and which never do, is the table under Running more than one gateway. retention.yaml turns on a periodic sweep that deletes expired artifacts — every 24 hours unless sweep.interval says otherwise, and off entirely unless sweep.enabled is set. Which gateways are alive across the fleet is a record in the shared store, and isg system brokers is what reads it.

The audit log is the exception to all of it. It is never swept, never rewritten and never moved: it is append-only by contract, and the contract — the JSON shape of a line, the file's mode, the rotation behaviour an ingesting agent must tolerate — is set out in the audit log schema.

Fail-closed posture

The gateway refuses to start rather than start in a state that looks correct and is not. That is a deliberate trade: a refusal is seen by the operator doing the restart, whereas a silently disabled control is seen by nobody until it matters.

isg serve will not come up when:

Two more cases are refusals at request time rather than at startup. If the directory lookup for a requested --as principal fails for an infrastructure reason — not "no such user", which is survivable — the daemon denies the session rather than continuing with empty group memberships and hoping a rule lets it through. And break-glass, the mechanism that bypasses the four-eyes gate, is denied to everyone when no breakglass_group is configured; there is no reading of "unset" that makes it permissive.

Two things are not fatal. The first is an uninitialized CA: the daemon starts, says so, and does not open the renew endpoint; the CA administration operations stay unavailable until isg ca init has run. The second is an identity provider that cannot be reached at startup: the daemon starts without SSO — the password form stays, even under web: required — and the SSO (OIDC) row of isg system check says so until the provider is back. Whether anyone can enter a running gateway must not depend on a JSON document being fetchable.