Adversarial Containment Foundations
A pre-session discussion brief defining the system boundary, threat assumptions, control layers, and open questions for containing capable autonomous agents.
Purpose and status
This note establishes a technical baseline for PESAEAS Morocco #1 in Rabat. It is a discussion brief, not a finished specification, security claim, or record of community consensus.
The scope is containment: limiting what an autonomous system can observe, change, invoke, and transmit when its behavior cannot be assumed cooperative. Model alignment, capability evaluation, and general application security matter, but they are not substitutes for a defensible containment boundary.
The containment problem
A conventional sandbox often assumes that the workload is untrusted but mechanically predictable. An autonomous agent changes the operating assumption: it may search for alternate paths, combine benign capabilities into a harmful sequence, influence operators, or adapt after a control blocks its first attempt.
Containment therefore has to be treated as a system property. The model, tools, credentials, runtime, network, data stores, operators, monitoring path, and shutdown mechanism all participate in the boundary. A strong control at one layer does not compensate for an unrestricted path at another.
Threat assumptions
The discussion begins from the following assumptions. They are intentionally conservative and should be narrowed for a concrete deployment rather than accepted as universal facts.
- The contained system may discover strategies that were not anticipated by its designers.
- Tool calls, external services, plugins, and human operators can become indirect escape paths.
- Credentials and permissions may be broader than the task actually requires.
- Telemetry can be incomplete, delayed, disabled, or manipulated during an incident.
- A component failure should not automatically expand the system’s authority or external reach.
Minimum control structure
A credible enclave design should explain how the following controls work together and which component remains trusted when another component fails.
- Explicit assets and trust boundaries: identify what must be protected and what the workload can reach.
- Default-deny authority: grant narrow capabilities for a defined task, duration, and destination.
- Complete mediation: inspect relevant tool, data, network, filesystem, and execution transitions.
- Resource budgets: bound time, compute, storage, process creation, retries, and external calls.
- Independent evidence: record policy decisions and boundary events outside the workload’s control.
- Out-of-band recovery: preserve a shutdown and restore path that the contained system cannot authorize.
Failure modes to examine
The brief does not prescribe one implementation. It asks each implementation to account for recurring failure classes.
- Direct boundary escape through a runtime, kernel, hypervisor, or hardware vulnerability.
- Confused-deputy behavior in which a permitted tool performs an action the agent could not perform directly.
- Data exfiltration through allowed outputs, metadata, timing, or encoded content.
- Resource exhaustion that degrades monitoring, forces unsafe fallback behavior, or denies operator control.
- Compromise of the policy, telemetry, update, or emergency-control plane.
Questions for the Rabat session
The session will use these questions to test the baseline against practical systems.
- Which actions require mediation in a minimal autonomous-agent deployment?
- Where should policy live so that the agent cannot reinterpret or modify it?
- What evidence is necessary to distinguish a blocked attempt from an undetected attempt?
- Which failures must fail closed, and where is a controlled degraded mode safer?
- What is the smallest useful containment profile that an engineer could implement and inspect?
Revision path
After the Rabat session, the note will be revised against the concrete objections, examples, and unresolved questions raised in the room. Revisions will distinguish changes to the threat assumptions from changes to the proposed control structure.