More Agents Are Not More Secure

The trap

The intuition is almost irresistible: if one model can be fooled, surely three models cross-checking each other are harder to fool. Put a council of agents on the problem, let them debate, take the consensus, and you have redundancy, diversity, and error correction for free. This is the argument behind agent councils, orchestrator-plus-subagent architectures, and the "have the models check each other" pattern now shipping in production frameworks.

That intuition is not generally true under adversarial evaluation. Multi-agent systems may improve benign reasoning while introducing attack surfaces that make many configurations less robust than their single-agent baselines. The correction is the subject of this article, and its short form is a trade: multi-agent architectures buy error correction and pay for it in adversarial robustness, and whether the trade is worth it depends on engineering discipline that common reference architectures do not enforce by default. Ship "several LLMs talking in a shared scratchpad with an LLM judge," and you have built distributed prompt-injection infrastructure and called it defense in depth.

None of this is a new claim, and the piece is stronger for saying so up front. A 2025 research agenda named multi-agent security as a distinct field (Schröder de Witt et al., "Open Challenges in Multi-Agent Security," arXiv:2505.02077). Since then the field has measured the trade rather than argued it: structured jailbreaks raise the average harmfulness of a multi-agent debate from 28 percent to 80 percent (arXiv:2504.16489); architecture alone shapes whether the same fleet refuses the same harmful task, with many configurations more vulnerable than their single-agent baselines and attack success varying by as much as 3.8 times across configurations, though no architecture was universally safer (arXiv:2604.23459); and self-replicating injection propagates across an agent society like a worm (Prompt Infection, Lee and Tiwari, arXiv:2410.07283). What follows organizes that diagnosis into a decision framework and pairs it with a buildable governance layer, crediting both to their sources and claiming only the small number of framings that are genuinely new here.

Diagnosis: what multi-agent actually trades

The reason practitioners and security researchers talk past each other is that they are answering different questions, and separating those questions is the single most useful move in this whole area.

The capability question: does collaboration reduce benign errors, hallucinations, and reasoning mistakes? The positive multi-agent literature (debate, self-consistency, critique-and-revise) is almost entirely evidence for the capability question, and it is real. Agents that critique each other do catch each other's mistakes on reasoning-heavy tasks with verifiable intermediate steps.

The security question: does collaboration reduce exploitability under an adaptive attacker who is optimizing against the system? Here the multi-agent setting introduces new attack surface rather than removing it: an injected payload in one agent's output is read by the next agent as a trusted peer instruction, untrusted content becomes trusted after a summarizing agent passes it along, and the central aggregator becomes the single highest-value target.

Multi-agent architectures trade capability gains for security costs. "Multi-agent is safer" (the practitioner, answering the capability question) and "multi-agent is a net-new attack surface" (the security researcher, answering the security question) are both true, and they are answers to different questions. Keeping them apart turns a stalled "it depends" into a decision procedure, and every conclusion below follows from it.

The trade also has a quantitative form that corrects a common deployment error. The tempting way to evaluate a defense is by attack-success rate: if the council's per-step ASR is lower than the single model's, the council wins. That accounting is incomplete, because expected loss depends on consequence, not probability alone:

E[L] = Σ P(F_i | A, S) × C(F_i, S)

for failure modes F_i under architecture A and system S, where the consequence term C is not a scalar but decomposes into the authority a compromised action exercises, the data and resource scope it touches, its persistence, its propagation to other agents, its reversibility, and its impact on external systems. A multi-agent architecture typically raises several of those consequence terms at once, because it can attempt more actions, with more delegated authority, that persist and propagate further. So the corrected statement is: a lower per-step ASR is not necessarily a security improvement when the architecture increases the number, authority, persistence, or blast radius of attempted actions. This is authority-adjusted expected loss, standard in form; the agent-specific point is that multi-agent architectures move the consequence terms in the wrong direction while advertising movement in the probability term. The reversibility and blast-radius side of the same accounting is developed in a companion article on single-agent authorization, and this piece cross-references rather than re-derives it.

Diagnosis: the failure-mode map

The failure modes sort into three types by the question they attack, and each has published attacks behind it, which is the honest way to present a taxonomy that is otherwise a competent import of classical security engineering.

Type A, security-question failures (new to, or worse in, multi-agent):

• Inter-agent injection and deception laundering. An adversarial payload in one agent's output is treated downstream as a trusted instruction, and untrusted content acquires trust after a seemingly-trusted agent summarizes it. This is not hypothetical: Prompt Infection (arXiv:2410.07283) demonstrates a self-replicating injection that spreads across a society of ten to fifty agents. The source deliberation cited only single-agent indirect injection (Greshake et al.); the multi-agent-specific citation is Prompt Infection, and it anchors this row. The countermeasure is typed, provenance-carrying messages in which the payload is always data, never instruction, and trust never rises through transformation.
• Mediator or orchestrator capture. The central aggregation point holds the most authority and is the most valuable single target, and a role-prompted LLM critic cannot be the security boundary because RLHF-trained agreeableness makes it soft-pedal. Separate process authority from content authority, and give any blocking critic structural, non-model teeth.
• Trust-boundary erosion and topology amplification. Provenance degrades along agent chains until external data carries "a trusted agent said so" authority, and a fully-connected graph creates order-k-squared inter-agent channels. That topology matters is measured: Architecture Matters (arXiv:2604.23459) varies roles, communication topology, and memory across thirteen configurations in three environments and finds the architecture shapes the trade between task performance and attack resistance, with attack success varying by as much as 3.8 times, and no configuration safe across the board. Prefer the sparsest graph the task allows.

Type B, capability-question failures (the correlated version of a single-model failure, often with added false confidence): correlated factual failure, sycophancy cascade, premature convergence, and groupthink on a poisoned premise. These are the same underlying problem, that agents with shared pretraining and shared training norms fail together, so a vote among them is not the independent vote it appears to be. That correlated failure defeats redundancy is not new; it is Knight and Leveson's 1986 result on N-version programming, where 27 independently developed implementations produced substantially more coincident failures than statistical independence predicted, and the subsequent fault analysis identified 45 faults and numerous statistically correlated fault pairs. The multi-agent version is that consensus is not evidence, and fast, smooth agreement is a failure signal rather than a safety one.

Type C, emergent failures (not predictable from per-agent analysis): the hardest and least-solved item is implicit collusion, where individually-safe actions compose into a globally-unsafe outcome and per-agent monitoring is blind to it by construction. The only proposed answer is global action-trace analysis over the whole sequence, and no mature version exists. This blind spot appears three times across this research portfolio under three names: emergent collusion here, cross-action aggregation in the governance half below (harmful trajectories built from individually sub-threshold actions), and cumulative memory effects in the memory work. Gating single commits is blind by construction to harmful sequences, and that is the field's shared open problem.

The independence question

The whole case for a council rests on one assumption, and it is the same assumption in the diagnosis (the diversity requirement) and in the prescription below (the quorum): that using different models produces independent failures. If it holds, redundancy buys real security; if it fails, the council is theater. It is the most load-bearing unvalidated claim in either half of this work, so it deserves a section.

The right way to operationalize it is a measurement: pairwise failure correlation on a domain-relevant adversarial probe set, with a pre-deployment gate that proceeds only if the correlation is low. The specific cut-points people reach for (proceed below roughly 0.3, treat above roughly 0.5 as redundant) are illustrative proposals, not derived values; the contribution is that the quantity is measured and gated at all.

The published evidence says the assumption is in trouble against exactly the threat it is supposed to cover. Adversarial attacks transfer across model families: gradient-optimized suffixes built on open models carry over to closed ones (Zou et al., GCG, arXiv:2307.15043), and recent work traces that transferability to shared internal representations rather than coincidence (Angell et al., "Jailbreak Transferability Emerges from Shared Representations," arXiv:2506.12913, 2025), which is the mechanism that would make "different vendor" a weak defense. Attackers can also exploit the topology itself: Agents Under Siege (Khan et al., arXiv:2504.00218) reports an attack optimized for constrained multi-agent topologies outperforming conventional attacks by as much as 7 times. And the classic result the source debate reached for actually points the same way once cited correctly: Tramer et al. (NeurIPS 2020) showed that ensemble and diversity-based defenses systematically overestimate their own robustness and fall to attacks adapted to them. The debate cited that paper for a fabricated figure pointing the opposite direction; read correctly, it supports the thesis.

Two qualifications keep this from overreaching. Transfer is not uniform: some families are far more resilient than others on the same prompts, so vendor diversity is not worthless, only weaker than the intuition assumes. And whatever genuine diversity exists at the first independent pass erodes once the agents interact and their contexts correlate, so a correlation measured on isolated first-passes overstates deployment-time independence. The load-bearing conclusion is that the deterministic, non-model controls (the effect-floor gate, sealed voting, the mandatory non-model check described next) carry the real adversarial weight, and model diversity is a supplement to them rather than the foundation. The clean confirmation is a runnable experiment, worth stating as a protocol: measure jailbreak and injection transfer across families in an actual governance-vote setting, before and after inter-agent interaction. Its result decides whether a diversity-based quorum is sound; present it as the crux, not a settled number.

Prescription: the missing layer

If the diagnosis is that naive multi-agent is a net-new attack surface, the prescription is the layer that changes the sign, and it starts from a distinction the current interoperability stack does not make.

MCP and A2A provide communication, discovery, task, authentication, and authorization machinery. They do not standardize application-level effect classification, collective commitment semantics, dissent authority, quorum policy, or approval of irreversible actions. A consequential action and a trivial one look the same at that level, and you cannot govern what you cannot distinguish. That gap does not make every fleet unsafe; it makes unsafe, by design, any fleet that lets these protocols mediate effectful autonomous action past an irreversible boundary without a governance interception layer.

The honest framing is that the agent ecosystem is re-encountering problems for which classical multi-agent research developed substantially more explicit formal machinery decades ago: speech-act-based agent communication with explicit performatives (FIPA-ACL), and electronic institutions that specify who may commit to what under which norms (Shoham and Leyton-Brown; the electronic-institutions line of work). That machinery did not solve governance for injection-prone LLM agents, but it named the primitives (performatives, commitments, roles, institutional norms) that the coordination protocols currently lack. The fix is not to bake governance into the coordination protocols, which loses on adoption, but to add a mandatory-adjacent governance plane that enforces complete mediation at effectful commit boundaries, in the sense Saltzer and Schroeder gave that term in 1975.

The unit of enforcement is a typed message envelope in which authority never derives from the fact that content arrived through a peer. An illustrative envelope:

sender_principal      workload identity of the sender
source_provenance     origin of the content and the chain it passed through
message_performative  the speech act: inform, request, propose, commit
content_type          declared as data, never as instruction
trust_label           integrity level of the content
taint_labels          untrusted sources this content descends from
requested_effect      the tool and effect class, if any
allowed_consumers     which roles may act on this
signature             over the whole envelope
correlation_id        for audit and replay

The point is not that natural-language content can never contain instructions; it obviously can. The point is that downstream policy never grants authority merely because those instructions arrived through a peer. This is governance for agent communities, the layer between agents; single-agent authorization (capability scoping, revocation, deterministic intent enforcement) is developed in the companion authorization article, and this piece defers to it.

Prescription: the deterministic core

The governance plane works because one principle is applied twice: remove injectable, model-based judgment from every safety-critical gate, enforce deterministically outside the model, and permit semantic judgment only where it can fail safe.

The classifier is the spine, and it has a precondition worth stating before the parts: deterministic effect classification requires narrow, effect-transparent tools. A general-purpose tool such as http.request or code.execute can express arbitrary effects while presenting a single innocuous signature, so the declared signature stops being a reliable proxy for the eventual consequence. Such tools must be sandboxed, recursively mediated, decomposed into narrow tools, or excluded above a defined autonomy tier; the classifier's guarantees hold only over effect-transparent capabilities. With that precondition, the classifier has four parts, each closing a gap the others leave open. A deterministic effect floor classifies an action by its tool's effect signature, not the model's stated intent: payment.execute, iam.grant_role, and email.send_external are classified by what they do, and no agent can talk its way into a lower risk class by narrating that the action is routine. An isolated, monotonic semantic tier handles novel or composite actions by running a classifier on a different model family with stripped context, seeing only the tool signature, parameters, and policy schema, never the orchestration conversation, and it may raise the risk class but never lower it. That monotonicity is the safety-by-construction property: a compromised semantic classifier can only over-escalate, costing human attention, never under-escalate into harm. It is the same Biba-style integrity move the memory work applies to memory labels, imported here to a classifier. A runtime effect monitor compares actual API side-effects against the declared manifest and quarantines tools that exceed it, and default-deny blocks unrecognized effectful tools until they are registered.

The same trick resolves dissent adjudication, which otherwise smuggles a model judgment back into the gating path. Two invariants keep it clean: the semantic content of a dissent is preserved always, signed and immutable, but never gates the decision; and the operational force of a dissent is a deterministic function of the action class it targets and the dissenter's role, never of the dissent's content. You never ask "is this objection correct?", only "what does it target, and does this role have blocking authority there?", and both are deterministic. The veto taxonomy follows the same discipline: a soft veto delays and is quorum-overridable only if the quorum saw the rationale, a hard veto belongs to institutional roles and is not peer-overridable, and a break-glass path is human-only and rate-limited. The renaming discipline matters, because an agent-supermajority-overridable veto is not a veto, it is a weighted vote with a misleading label, and the guarantee "this action class requires compliance sign-off" dissolves if peers can override it.

The quorum has to satisfy the independence constraint from the independence section, which is why the diagnosis and the prescription belong in one article. State the requirement as the property, not the input: critical actions require a correlation-gated quorum spanning independently measured evidence paths and failure domains. Vendor diversity may contribute to that independence, but vendor count is not itself evidence of it. Count independent evidence paths and failure domains, not agent heads (grounded in Knight and Leveson), require sealed voting so agents commit before seeing each other, and require at least one deterministic non-model check.

Two substrates sit underneath, and they are not the same component. A Zanzibar-style relationship engine (the model introduced by Pang et al., USENIX ATC 2019, and implemented by OpenFGA) establishes principals, roles, delegations, and resource relationships: who stands in what relation to which resource. A separate policy or verification layer (OPA, Cedar, or typed custom logic) evaluates action class, parameter constraints, effect budgets, quorum state, and dissent authority. The control-and-data separation and deterministic privilege control they enforce come from CaMeL (Debenedetti et al., arXiv:2503.18813, which completes 77 percent of AgentDojo tasks with provable security versus 84 percent undefended) and Progent (arXiv:2504.11703, 2025). None of these substrates is new here; the assembly at the council level is the only part this piece puts together. Consensus itself then becomes an integrity signal rather than a guarantee: it counts as evidence only when the independence conditions hold and the dissent record is preserved, and suspiciously perfect agreement is something to alarm on, not to trust.

Prescription: the binding constraint

The governance plane's scarce resource is not compute; it is human attention, and treating it as a first-class budget line is what separates a real design from a diagram.

The arithmetic is queueing theory applied to human review, and it is worth showing in full so the numbers are derived rather than asserted. Let N be the fleet size, r the effectful decisions per agent per hour, e the fraction of those that escalate to a human, t the mean review time in minutes, ρ the maximum sustainable reviewer utilization, and λ_bg the break-glass arrival rate. Then escalation arrival and the required number of concurrent reviewers are:

λ = N × r × e + λ_bg   (escalations per hour)
c ≥ (λ × t) / (60 × ρ)   (concurrent reviewers)

Worked with one illustrative set of inputs (N = 1000, r = 2, t = 5 minutes, ρ = 0.6, λ_bg ≈ 0; all placeholders, not measured):

The escalation fraction is therefore a design parameter, not an afterthought, and moving it from 1 percent to 10 percent moves oversight from a few reviewers to dozens (concurrent; round-the-clock coverage multiplies this by the shift factor). The circuit-breaker rule follows from the same accounting and is the security-critical part: under overload the system must tighten, not loosen. Critical actions default to deny on timeout, never approve, because allow-on-timeout is a denial-of-service attack, flood the review queue and you clear the path for harm.

Enforcement is mandatory-adjacent, which resolves the tension between "governance must not be optional" and "do not rewrite the protocols." Near-term, enforce at the infrastructure layer the agent cannot route around, a service mesh, an admission controller, a network policy; longer-term, register against a standard governance hook in the coordination protocols. Wrap, do not modify. The components all map to off-the-shelf infrastructure (policy engines, append-only signed logs, workload identity), which is why the feasibility verdict is that this is buildable this year and the hard part is organizational adoption, not technology.

Where it stops

A framework is only as credible as the gaps it admits, and four are load-bearing enough that a team should treat them as the active research program, because they are exactly where a real adversary will attack.

Cross-action aggregation. Every mechanism above gates a single commit boundary and is blind by construction to harmful trajectories built from individually sub-threshold actions: fifty small reads that jointly exfiltrate, a sequence of reversible writes that jointly establish persistence. The direction is cumulative effect budgets per agent and session with deny-on-exceed, but whether that stays deterministic rather than reintroducing a judgment about when accumulation becomes a decision is unresolved. This is Type C emergent collusion seen from the defensive side.

General-purpose tools. As the core section flagged, an agent with http.request or code.execute performs any effect while the effect-signature classifier sees only "HTTP call." This is not merely a limitation of the design, it is a precondition on it: the effect floor is only as sound as the effect-transparency of the tools it classifies, and general-purpose tools break that transparency. Banning, recursive mediation, and forced decomposition are the options, and none is clearly right.

The independence assumption. The crux above is unresolved in the direction that matters: cross-family transfer is measured high enough to doubt that model diversity delivers adversarial independence, which is why the deterministic controls carry the real weight and the quorum's diversity is a supplement.

The scalable mediator. External structural enforcement is what surfaces real dissent, but the enforcer in the deliberations that produced this framework was a scarce, non-parallelizable human reviewer. Who plays that role for a ten-thousand-agent fleet without being either a model (a soft center) or a human (unscalable) is unanswered, and it belongs in the staffing arithmetic as a second-order cost that was never added.

Bottom line for a team acting today

• Agent multiplicity is not a security control. Build the deterministic effect gate, provenance-typed inter-agent messages (payload is always data), a correlation-gated quorum with sealed voting and at least one non-model check, and a human-attention budget sized to your fleet's autonomy. These are off-the-shelf and worth doing now.
• Measure your own failure correlation on a domain-relevant adversarial probe set before you trust a quorum, and treat model diversity as a supplement to the deterministic controls, not a substitute.
• Do not evaluate a multi-agent defense on per-step attack-success rate alone; weight it by consequence, because a lower rate over more numerous, higher-authority, more persistent actions can be a larger expected loss.
• Confine the effect gate to effect-transparent tools; sandbox, decompose, or tier-limit http.request and code.execute, or the classifier's guarantees do not hold.
• Treat three red flags as reasons for external review: same-family debate cited as a security argument, a majority vote treated as safety evidence, and a mediator with broad content authority.
• Set the circuit-breaker to deny on timeout for critical actions, so overloading the reviewers does not clear the path for harm.
• Do not let a council certify its own soundness. The one thing the deliberations behind this framework proved is that a group cannot see its own groupthink from the inside.

What this piece claims, and what it does not

Author's note on method

This analysis draws on two structured multi-model deliberations, a diagnosis run and a prescription run, each with three frontier models under a judge across several rounds. Three disciplines from that process are worth stating. First, the convergence of the two deliberations on one spine is method, not evidence: the models share training corpora and are not independent witnesses, and the published literature had already reached the same place. Second, one provenance quirk belongs on the record: the diagnosis run's configured judge timed out on every call, so its authoritative synthesis was produced out-of-band by a separate model reading the full transcript, and no per-round convergence was ever certified; the run's "convergence" is not a verification signal. Third, the reflexive specimen is worth one paragraph. The diagnosis council was asked to judge its own architecture and did so while visibly exhibiting the pseudo-diversity and premature convergence it was cataloguing, and its judge ruled that a council cannot self-certify its freedom from groupthink because the motion was under-contested, with no assigned steelman or red-team. That is a memorable demonstration, and its payload, that dissent must be enforced structurally from outside the deliberating agents, supports the mandatory-interceptor design in the core section. But it is a caution, not a confirmation; the evidence that the thesis is true comes from the published measurements cited throughout. Every statistic and citation here was verified against primary sources independently of the transcripts, which mattered: one recurring failure in the source material was a fabricated statistic (a claimed 40 to 60 percent council attack-success drop attributed to nonexistent internal reports), and a citation offered for the opposite of what the cited paper actually found.

References

Multi-agent security: agenda, measurements, attacks

• Schröder de Witt et al. "Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents." arXiv:2505.02077 (2025).
• "Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debate." arXiv:2504.16489 (2025). Average harmfulness 28.14% to 80.34% under structured rewriting.
• "Architecture Matters for Multi-Agent Security." arXiv:2604.23459 (2026). Roles by topology by memory, 13 configurations, 3 environments; attack success varies by as much as 3.8x, no architecture universally safer.
• Lee, Tiwari. "Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems." arXiv:2410.07283 (ICLR 2025 submission).
• Khan et al. "Agents Under Siege." arXiv:2504.00218 (2025). An attack optimized for constrained multi-agent topologies outperforms conventional attacks by up to ~7x.

Cross-family adversarial transfer (the independence crux)

• Zou, Wang, Kolter, Fredrikson. "Universal and Transferable Adversarial Attacks on Aligned Language Models" (GCG). arXiv:2307.15043 (2023).
• Angell et al. "Jailbreak Transferability Emerges from Shared Representations." arXiv:2506.12913 (2025).
• Tramer, Carlini, Brendel, Madry. "On Adaptive Attacks to Adversarial Example Defenses." NeurIPS 2020. Ensemble and diversity defenses overestimate robustness under adaptive attack.

Classical foundations (imported, credited)

• Saltzer, Schroeder. "The Protection of Information in Computer Systems." Proc. IEEE 63(9), 1278-1308 (1975). Complete mediation, least privilege.
• Knight, Leveson. "An Experimental Evaluation of the Assumption of Independence in Multiversion Programming." IEEE TSE 12(1), 96-109 (1986). 27 versions, 45 faults, numerous correlated fault pairs.
• Lamport, Shostak, Pease. "The Byzantine Generals Problem." ACM TOPLAS 4(3), 382-401 (1982).
• Pang et al. "Zanzibar: Google's Consistent, Global Authorization System." USENIX ATC 2019. (OpenFGA is the open-source ReBAC implementation.)
• Biba. "Integrity Considerations for Secure Computer Systems." ESD-TR-76-372 (1977).
• FIPA-ACL Message Structure Specification (fipa.org). Shoham, Leyton-Brown. "Multiagent Systems." Cambridge (2009).

The information-flow family (cross-referenced to the companion authorization and memory articles)

• Debenedetti et al. "Defeating Prompt Injections by Design" (CaMeL). arXiv:2503.18813 (2025). 77% of AgentDojo with provable security vs 84% undefended.
• Shi et al. "Progent: Securing AI Agents with Privilege Control." arXiv:2504.11703 (2025).
• Greshake et al. "Not What You've Signed Up For." AISec '23. arXiv:2302.12173.
• Debenedetti et al. "AgentDojo." arXiv:2406.13352 (2024).

Systematizations

• "SoK: The Attack Surface of Agentic AI." arXiv:2603.22928 (2026).
• "A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents." arXiv:2604.23338 (2026).