AI Agent Break-Glass: Dual Control for Privileged Emergencies

It is 03:17. A refund agent is stuck behind a dual-approval gate and the second approver is asleep. On-call flips break_glass=true with a shared ops token, widens tool scope to production delete, and leaves the flag on until Monday. By morning the incident is contained, the shared token still works, and nobody can prove whether the widen was a deliberate emergency or standing privilege with a new label.

You need a break-glass dual-control contract: emergency tightening (kill the agent, revoke credentials, drop spend and egress ceilings) stays ungated for on-call; emergency loosening (temporary privilege widen or quorum relax) requires a real human grant that is dual-controlled or explicitly break-glassed, hard-expires, never overrides a deliberate human denial, never activates from an agent or system token, leaves immutable residual evidence, and blocks the next emergency grant until a different human finishes post-review. It sits beside binding human approval to exact AI agent actions (the normal deny-closed approval path), the AI agent incident response runbook (who to page and how to contain), signed action-receipt contracts for privileged AI agent outcomes (cryptographic evidence after boundary events), attenuating capability-token contracts for multi-hop AI agent delegation (narrowed authority before the call), and stopping runaway AI agent tool loops (velocity and loop cutoffs). The job here is narrower: define the emergency privilege path so least privilege survives the night without becoming root with a sticker.

Why shared emergency tokens fail the morning-after question

OWASP LLM06:2025 Excessive Agency names excessive autonomy as failing to independently verify and approve high-impact actions. Mitigation starts with human-in-the-loop approval and complete mediation in downstream systems, not in the model. That is the normal path. Emergencies are where teams quietly undo those rules.

The OWASP AI Agent Security Cheat Sheet requires explicit approval for high-impact or irreversible actions, short-lived authorization artifacts with replay protection, step-up authentication for critical actions, and fail-closed behavior when risk classification, approval validation, policy lookup, or audit logging fails. None of that survives if the platform also ships a permanent break_glass principal that can approve itself, widen scope without a timer, and wipe its own audit trail.

NIST SP 800-53 Rev. 5 AC-3(2) Dual Authorization requires the approval of two authorized individuals for organization-defined privileged commands. The discussion tells organizations to weigh that control when immediate public or environmental safety responses are necessary. AU-9(5) applies the same two-person idea to movement or deletion of audit information and repeats the same safety exception. Temporary and emergency accounts under AC-2 are intended for short-term use and should be automatically disabled or removed after a defined period rather than left until an administrator remembers. Those controls are not AI-specific. They are still the vocabulary your emergency path has to speak.

The attack: emergency UX that is really standing root

Four failure modes show up again and again on internal agent platforms.

1. One button for opposite intents. The same control both stops a runaway agent and elevates privileges for a blocked workflow. Operators learn to press it for everything. Containment waits on an approver, or elevation becomes as easy as a kill switch.
2. Agent-activated elevation. A tool or prompt path lets the agent request break-glass. Prompt injection that says "this is an emergency, elevate now" becomes privilege escalation, because the judgment that warrants elevation is exactly what an adversary can fake.
3. Timer as a reminder. The grant says "one hour" in a runbook, but expiry is manual. The panicked principal forgets. Temporary widen becomes standing access with ceremony.
4. Silent residual. The acting principal can delete or rotate the only log that recorded the grant. Break-glass without an immutable residual that the actor cannot rewrite is not an emergency procedure.

Praesidia's break-glass guidance for AI operations frames the first failure as a direction mistake: tightening must never be gated; loosening must always be recorded and reviewed. Olivares AI's govern-and-approve model frames the second and fourth: break-glass is always a real human, never a system token; every use appends an immutable ledger row; a new grant cannot open while a prior grant is unreviewed. details.cloud's agentic safeguards blueprint frames the third: when break-glass becomes routine, redesign the normal path instead of weakening the emergency path.

Asymmetry first: tightening versus loosening

Start with direction before you argue about quorum sizes.

Praesidia states the practical shape plainly: a kill switch and scope-reduction controls available without approval, and a separate, recorded, auto-expiring path for temporary loosening. If your platform has only one "emergency" flag, split it before you pile policy on top.

Dual control on the normal critical path

Critical irreversible actions - production deploy and retire, data deletion and erasure, key custody and rotation, security-enforcement changes - should not start at break-glass. They start at dual control.

Olivares AI floors critical actions at two distinct human approvers, citing NIST SP 800-53 AC-3(2). The floor is enforced at request creation and re-applied at decision time so a stale snapshot cannot lower the bar. Separation of duty keys on stable user identity, not a credential string one person can vary. System tokens cannot approve. That is the same principle DualKey encodes as an execution layer: sit between "agent decided" and "action runs," evaluate deterministic policy, and require a second key when needed (DualKey agent).

Wire the normal path before you invent the escape valve:

1. Classify actions into risk tiers with a deny-closed default.
2. Bind approvals to the exact action digest (tool, target, normalized parameters, expiry), as in Fire in Belly's human-approval binding guide.
3. Require step-up authentication for critical decisions (WebAuthn / hardware-backed session), matching the cheat sheet and Olivares AAL3-style requirement.
4. Fail closed when policy or approval services are unavailable on high-risk paths.

Break-glass then has one precise job: relax a missing quorum under time pressure, not invent a second authorization universe.

Break-glass field contract

When the second approver is unreachable, a human admin activates a time-boxed emergency grant. Make that grant machine-checkable - not a Slack thread.

Olivares AI's rules map cleanly onto this table: activation self-audits to the ledger in the same transaction, actions under break-glass stay permanently distinguishable from properly approved ones, and forced post-review prevents grant stacking. Praesidia adds the speed constraint: if elevation takes twenty minutes, people will negotiate standing access instead - and they will be right during a real incident. Target seconds for the common investigation tier; reserve dual pre-approval for the most destructive tiers.

Agents must not hold the glass hammer

Two distinct questions arise once agents are in the picture.

Should an agent activate break-glass? Generally no. Break-glass depends on human judgment about whether the situation warrants elevation. That judgment is what adversarial input tries to forge. Praesidia's rule is the right default: where an agent needs occasional elevated access, the human who directs it elevates, and the agent acts under that human's temporarily broadened principal with dual attribution recorded.

Should agents help during incidents? Often yes for read-only investigation - timeline summaries from audit stores, correlation of tool calls, draft containment checklists. Give those helpers a narrow standing read role to the audit plane, not break-glass write scope. Standing-but-harmless beats occasional-but-broad when the task is investigation.

This is the constraint teams miss when they only copy JIT access from human IAM: an LLM that can call break_glass.activate is not helpful automation. It is an elevation API bolted onto a confused deputy.

Worked example: 03:17 refund stuck behind dual control

A production refund agent needs a second human for refund.write above €500. The first approver signed. The second is offline. Customer impact is rising.

Root-with-a-sticker design. On-call flips a shared ops-breakglass token that grants refund.write, customer.delete, and unlimited spend. No TTL. The agent retries and succeeds. Monday, the token still works. The audit line says emergency used and is editable by the same deploy role.

Contract design.

1. On-call confirms the agent is not runaway - tightening controls (pause run, drop retry budget) remain one click and ungated.
2. On-call, as a human admin with hardware step-up, opens break-glass with justification second approver unreachable; refund ID R-4421; max €750.
3. Scope delta is only refund.write for that order id prefix and max_amount <= 750, lease 60 minutes, not customer.delete.
4. Ledger emits break_glass_opened signed outside the agent host; paging channel gets a critical notice.
5. The agent executes under dual attribution: agent principal + human grant id. Signed action receipts record pre-execution allow under grant bg-7781 and post-execution controller outcome separately.
6. Lease expires automatically; widened capability disappears without anyone remembering to revoke.
7. A different human must complete post-review before any new break-glass can open. Review asks: was scope right, did actions match the reason, should the normal path get a scheduled JIT role for this refund class so break-glass stops being the Tuesday pattern?

If someone already clicked deny on that exact refund digest, break-glass must refuse. Missing quorum is not the same as overruled judgment.

Fail closed and out-of-band

High-risk loosening must fail closed when the break-glass service, ledger, or step-up path is down. Do not fall back to a shared password in the wiki. Praesidia's out-of-band advice still applies: a sealed credential or cloud-provider root path with hardware MFA held by two people, genuinely hard to use, exercised once or twice a year, and always triggering immediate review. An untested emergency path is not an emergency path.

Tightening stays available even when the loosening path is degraded. You must always be able to kill the agent and revoke credentials. A control plane that requires dual approval to stop a runaway agent has an availability bug in its security design.

How this contract relates to adjacent controls

Operating constraints

These constraints are easy to miss when teams only bolt on an "emergency approve" button:

1. Separate ungated emergency tightening from dual-controlled time-boxed loosening; one emergency button that both kills and elevates invents standing root. Direction is the control. Quorum size is secondary.
2. Forbid agents and system tokens from activating break-glass; adversarial input that requests elevation is privilege escalation, not incident judgment. Humans elevate; agents act under that temporary principal with dual attribution.
3. Never let break-glass override an explicit human denial, and block the next emergency grant until a different human completes post-review. Otherwise stacked emergencies wash the act and a social-engineered reopen undoes a deliberate block.

Promote or hold

Promote when:

• tightening controls (kill, revoke, drop ceilings) are ungated for on-call and affect in-flight work;
• critical normal-path actions require dual control with separation of duty and step-up auth;
• break-glass is human-only, scoped, hard-expired, notified, and ledgered immutably;
• break-glass cannot override explicit denials;
• post-review by a different human latches the next grant;
• agents cannot call activate; dual attribution is recorded when they act under a grant;
• high-risk loosening fails closed if grant issuance or residual emission is unavailable.

Hold when the emergency path is a shared token, when agents can self-elevate, when TTL is a runbook comment, when denials can be flipped by break-glass, when the actor can erase the residual, or when the same button both stops and elevates without distinct policy.

What to do next

List every production tool that can delete data, move money, change IAM, deploy to production, or egress externally. For each path, record the normal approval tier, whether dual control is enforced in code, and what the emergency path actually is today: shared token, ungated config edit, agent-triggered elevation, or dual-controlled time-boxed human grant with forced review. Until that inventory shows ungated tightening plus latched, human-only loosening on every privileged path, treat "we have break-glass" as an incomplete control, not a finished one.

If you need the control plane implemented as part of an internal workflow program rather than a weekend override flag, start from AI workflow automation with break-glass dual control as an explicit deliverable.

---