Pith. Sign in

REVIEW

Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffs

Artificial Intelligence · Submitted 29 Aug 2026 · 2 references

Pith's verdict: Solid evidence for a real privacy failure mode in multi-agent LLM handoffs; the core qualitative pattern is robust even if the exact magnitudes may shift under independent measurement.

T0 · AI review

How this review was made: An AI referee (deepseek/deepseek-v4-flash) read the paper on September 9, 2026 and answered a fixed list of questions about it. T0 is the starting level every paper gets. The levels describe how deeply Pith has checked the paper, never the authors.

  1. T0 AI review An AI read the paper and answered a fixed list of questions.
  2. T1 Audit Automatic checks that the references and files the paper relies on exist and match.
  3. T2 Compute The paper's code or calculations re-run with the same results.
  4. T3 Formal statement The claims written out in a formal math language a computer can read.
  5. T4 Proof Those claims proved, and the proof checked by computer.

How Pith checks papers →

· 2 major objections · 11 minor · Ask about this paper · The compiled record · Read the full review →
Challenge this review Comment (0) Read on arXiv ↗
Cite ▾
arXiv ID2608.29028 (version 1)
Pith ID3JVQA4ZC, Pith's permanent record of this paper and its review
CitationAgam Goyal, Eshwar Chandrasekharan, Hari Sundaram, Yian Wang. "Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffs." arXiv:2608.29028 (2026). https://pith.science/paper/3JVQA4ZC
Cite this reviewBibTeX for Pith's review of this paper
Claim paper

Why Pith says this

Highlight any text to comment on it.

Referee report
The paper

The paper claims

Multi-agent LLM handoffs suffer from 'summary collapse': operational facts survive compression while the privacy rules that limit their use are weakened or dropped, causing systematic downstream leakage.

The reading

The paper identifies a structural privacy failure in multi-agent LLM systems: when one agent compresses an upstream interaction into a handoff artifact for a downstream agent, the summary preferentially preserves the operational facts the next agent needs while weakening or dropping the boundary metadata that governed how those facts could be used. The authors call this selective loss "summary collapse" and measure it with two survival scores—boundary-marker survival (σ_b) and operational-fact survival (σ_op)—showing they decouple under compression, especially under tight token budgets. Controlled experiments demonstrate that the strength of the boundary marker (vague hedges vs. explicit constraints) causally determines whether leaked content appears in audience-facing outputs, and that typed, machine-readable audience allowlists suppress leakage almost entirely when correctly populated. The work moves the privacy question from a single agent's decision to the artifact passed between agents, arguing that the real leak happens upstream of the final output.

What carries the argument

The key mechanism is the decoupling of boundary-marker survival (σ_b) from operational-fact survival (σ_op) during handoff compression, quantified via a human-validated LLM judge (κ=0.74). The central causal chain is marker strength: a five-level gradient from no marker to hard imperative shows a sharp drop in leakage at the transition from vague hedge to soft suggestion (L1→L2), indicating that the marker must be readable as an operational instruction, not merely privacy-sounding language. The typed audience-allowlist schema operationalizes the boundary as a machine-actionable constraint, which nearly eliminates leakage when correctly populated.

Extended reading notes

Core claim

The central discovery is that handoff compression in multi-agent LLM systems systematically discards the rules that constrain sensitive information while preserving the information itself, a failure mode termed summary collapse. Using a custom testbed (BOUND-HANDOFF) of 36 scenarios across seven domains, the authors show that boundary-marker survival (σ_b) and operational-fact survival (σ_op) are nearly uncorrelated at the individual-handoff level, and a substantial fraction of handoffs land in the low-σ_b, high-σ_op quadrant. The decoupling widens under hard compression (e.g., a 25-word budget reduces σ_b to approximately 0.57 while σ_op stays above 0.97). Through matched downstream tests, they prove that marker explicitness—not mere presence—is the causal determinant of leakage: vague language like "use discretion" leaks in 73% of GPT-5-mini cases, whereas explicit constraints reduce leakage to under 15%. A gold-derived typed audience-allowlist schema eliminates leakage on two primary models, but only when the boundary field is correct; a corrupted allowlist restores leakage to near no-marker levels.

Load-bearing premise

The experimental scenarios, though spanning seven domains and independently replicated on PrivacyLens, are hand-authored and semi-synthetic; the load-bearing premise is that the decoupling of σ_b and σ_op and the causal role of marker strength generalize to production multi-agent systems with real users, long-lived memory, and natural boundary-marking language.

Editorial extensions

If this is right

  • Multi-agent system designers should encode privacy boundaries as typed fields (e.g., allowed_audiences) rather than rely on natural-language prompts, as the latter are stripped by compression.
  • Privacy evaluation in multi-agent systems must measure handoff artifacts, not just final outputs, because boundary loss occurs upstream and may be hidden by later information loss.
  • The marker-strength gradient provides a design principle: vague hedges are functionally equivalent to no marker for leakage risk; only explicit, instruction-like constraints offer meaningful protection.
  • Boundary extraction becomes a bottleneck: systems need automated, paraphrase-robust validation of boundary fields with confidence scoring and abstention mechanisms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The selective loss pattern may generalize beyond handoff compression to any LLM summarization or memory retrieval task where operational content competes with meta-constraints, suggesting a broader vulnerability in agentic systems.
  • The typed allowlist approach, while effective in the controlled setting, requires accurate extraction of boundaries from upstream dialogue, which is itself an open problem that could introduce new failure modes (e.g., extraction errors amplifying leakage).
  • Cross-cultural or multilingual deployments may require recalibrating the marker-strength gradient, as politeness conventions differ across languages.
Original abstract

Multi-agent LLM systems often coordinate by compressing an upstream interaction into a handoff artifact that downstream agents treat as shared state. We show that this handoff step is a structural source of privacy leakage: summaries preferentially preserve operational facts while weakening the boundary metadata that governs how those facts may be used---a failure mode we call \emph{summary collapse}. On a controlled multi-agent coordination testbed we measure marker survival with a human-validated judge ($\kappa = 0.74$), where $\sigma_b = 1$ means every boundary marker survives verbatim and $\sigma_b = 0$ means all are lost. Boundary-marker and operational-fact survival are nearly uncorrelated at the handoff level on both GPT-5-mini and DeepSeek-R1-32B (Pearson $r$ near zero): uncompressed free-text handoffs preserve boundaries at $\sigma_b \approx 0.80$, whereas a $25$-word budget drops $\sigma_b$ to ${\approx}0.57$ while operational-fact survival stays near ceiling. Controlled downstream tests reveal that protection depends on \emph{boundary explicitness}: vague languages leak in $73\%$ of GPT and $50\%$ of DeepSeek cases, while explicit constraints reduce leakage to under $15\%$ across all three tested models. A no-handoff single-agent control further shows the failure is not reducible to multi-agent topology as direct full-marker access still leaks more often than the operationalized handoff. Prompt-only mitigation and exact-string redaction only partially address the problem, while a gold-derived audience allowlist nearly eliminates leakage across models, showing that correctly identifying audience boundaries is the key factor.

The compiled record

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,083 input tokens.

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.35 · human participants 0.53 · controlled comparison 0.98 · randomized assignment 0.04 · states quantitative result 0.99 · compares with prior results 0.91 · mentions code or data availability 0.03.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; .

Multi-agent LLM systems often coordinate by compressing an upstream interaction into a handoff artifact that downstream agents treat as shared state. We show that this handoff step is a structural source of privacy leakage: summaries preferentially preserve operational facts while weakening the boundary metadata that governs how those facts may be used---a failure mode we call \emph{summary collapse}. On a controlled multi-agent coordination testbed we measure marker survival with a human-validated judge ($\kappa = 0.74$), where $\sigma_b = 1$ means every boundary marker survives verbatim and $\sigma_b = 0$ means all are lost. Boundary-marker and operational-fact survival are nearly uncorrelated at the handoff level on both GPT-5-mini and DeepSeek-R1-32B (Pearson $r$ near zero): uncompressed free-text handoffs preserve boundaries at $\sigma_b \approx 0.80$, whereas a $25$-word budget drops $\sigma_b$ to ${\approx}0.57$ while operational-fact survival stays near ceiling. Controlled downstream tests reveal that protection depends on \emph{boundary explicitness}: vague languages leak in $73\%$ of GPT and $50\%$ of DeepSeek cases, while explicit constraints reduce leakage to under $15\%$ across all three tested models. A no-handoff single-agent control further shows the failure is not reducible to multi-agent topology as direct full-marker access still leaks more often than the operationalized handoff. Prompt-only mitigation and exact-string redaction only partially address the problem, while a gold-derived audience allowlist nearly eliminates leakage across models, showing that correctly identifying audience boundaries is the key factor.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#sentencerolethe authors' own workstates the main conclusionstrength
s0 Multi-agent LLM systems often coordinate by compressing an upstream interaction into a handoff artifact that downstream agents treat as shared state. background 1.00 0.06 0.03 asserted 0.87
s1 We show that this handoff step is a structural source of privacy leakage: summaries preferentially preserve operational facts while weakening the boundary metadata that governs how those facts may be used---a failure mode we call \emph{summary collapse}. contribution 0.53 0.89 0.92 asserted 1.00
s2 On a controlled multi-agent coordination testbed we measure marker survival with a human-validated judge ($\kappa = 0.74$), where $\sigma_b = 1$ means every boundary marker survives verbatim and $\sigma_b = 0$ means all are lost. method 0.98 0.96 0.06 asserted 0.97
s3 Boundary-marker and operational-fact survival are nearly uncorrelated at the handoff level on both GPT-5-mini and DeepSeek-R1-32B (Pearson $r$ near zero): uncompressed free-text handoffs preserve boundaries at $\sigma_b \approx 0.80$, whereas a $25$-word budget drops $\sigma_b$ to ${\approx}0.57$ while operational-fact survival stays near ceiling. result 1.00 0.96 0.19 asserted 1.00
s4 Controlled downstream tests reveal that protection depends on \emph{boundary explicitness}: vague languages leak in $73\%$ of GPT and $50\%$ of DeepSeek cases, while explicit constraints reduce leakage to under $15\%$ across all three tested models. result 1.00 0.94 0.14 asserted 1.00
s5 A no-handoff single-agent control further shows the failure is not reducible to multi-agent topology as direct full-marker access still leaks more often than the operationalized handoff. result 0.90 0.94 0.05 asserted 1.00
s6 Prompt-only mitigation and exact-string redaction only partially address the problem, while a gold-derived audience allowlist nearly eliminates leakage across models, showing that correctly identifying audience boundaries is the key factor. result 0.67 0.91 0.45 asserted 1.00

What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 0.99.

What was examined, and what was not

sources read
  • arXiv listing abstract v1
not examined
  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers
levels run
  • P abstract
not run
  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

Figures
Figure 1. Summary collapse. A task-relevant opera￾tional fact f and the boundary rule b that licenses its use both enter the handoff; under compression, f survives while b is weakened or omitted, so the downstream agent treats f as unconstrained. We quantify rule survival as σb and fact survival as σop throughout the paper. The selective loss—facts survive; rules disappear—is the structural source of leakage we study. that time,” they share just enough—the slot is unavailable—while withholding the therapy ap￾pointment, the job interview, or the court hearing. This selective disclosure is the…
Figure 2. Privacy–utility frontier under graded audience

Figures reproduced from arXiv:2608.29028 by the authors.

References
How the literature uses it

What this paper does with the arXiv papers it cites

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

What it cites · 3 sentences without a confident label

Uses the method or tool of 4 papers
  • GPT-5-mini [ref] and DeepSeek-R1-32B [ref] are the two primary models on which every experiment is run; Qwen3-32B [ref] is included as a targeted third-model replication, covering the four experiments where cross-pipeline robustness is essential for the headline claims: E1 baseline handoff survival, E2 compression stress, E8 marker-strength gradient, and the E9 corrupted-allowlist stress test.

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning · Models · 0.83

  • GPT-5-mini [ref] and DeepSeek-R1-32B [ref] are the two primary models on which every experiment is run; Qwen3-32B [ref] is included as a targeted third-model replication, covering the four experiments where cross-pipeline robustness is essential for the headline claims: E1 baseline handoff survival, E2 compression stress, E8 marker-strength gradient, and the E9 corrupted-allowlist stress test.

    Qwen3 Technical Report · Models · 0.83

  • GPT-5-mini [ref] and DeepSeek-R1-32B [ref] are the two primary models on which every experiment is run; Qwen3-32B [ref] is included as a targeted third-model replication, covering the four experiments where cross-pipeline robustness is essential for the headline claims: E1 baseline handoff survival, E2 compression stress, E8 marker-strength gradient, and the E9 corrupted-allowlist stress test.

    OpenAI GPT-5 System Card · Models · 0.83

Cites as background 6 papers
  • AgentLeak shows violations frequently occur in internal inter-agent channels rather than user-facing outputs [ref] ; SumLeak formalizes compositional leakage in which individually innocuous responses aggregate into a sensitive attribute [ref] ; and prompt injection or environmental manipulation can elicit sensitive context at execution time [ref] .

    2509.14284 · Related Work · 0.87

  • AgentLeak shows violations frequently occur in internal inter-agent channels rather than user-facing outputs [ref] ; SumLeak formalizes compositional leakage in which individually innocuous responses aggregate into a sensitive attribute [ref] ; and prompt injection or environmental manipulation can elicit sensitive context at execution time [ref] .

    Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution · Related Work · 0.87

  • MEXTRA demonstrates black-box extraction of stored user–agent interactions [ref] ; Prompt Infection [ref] , MASLeak [ref] , and Agent-in-the-Middle [ref] show inter-agent channels can be exploited to self-replicate instructions, extract proprietary system information, or compromise an entire MAS.

    IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems · Related Work · 0.86

Discussion

Disagree with the review, or know something it missed? Leave a comment, highlight any passage above to comment on it, or challenge this review formally.