AI Agents Do Not Fail Alone:The Context Fails First

Fouad Bousetouane

REVIEW 4 major objections 5 minor 1 cited by

This paper claims that AI agent reliability is predictable from the quality of the operating context alone, before behavioral testing, by measuring seven context criteria and showing they correspond to distinct downstream failures.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 02:34 UTC pith:TCLJEOI4

load-bearing objection The core claim—that context quality independently predicts behavior—is undercut by the shared LLM-juror scoring; the rubric is still useful packaging. the 4 major comments →

arxiv 2607.14275 v1 pith:TCLJEOI4 submitted 2026-07-15 cs.AI cs.MA

AI Agents Do Not Fail Alone:The Context Fails First

Fouad Bousetouane This is my paper

classification cs.AI cs.MA

keywords context engineeringAI agent reliabilityagent evaluationhallucination resistanceprompt injectionguardrailstool usepreflight diagnostic

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that AI agents do not fail merely because of their base models; they fail because the context they reason inside—instructions, tool schemas, grounding evidence, guardrails, and untrusted inputs—is poorly engineered. It defines context-engineering quality as a measurable seven-criterion construct (role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, token efficiency), implemented in an open-source harness with multi-juror consensus scoring. Crucially, the context score is kept separate from behavioral metrics, so the study can test whether context quality predicts behavior without circularity. Holding frontier LLMs fixed and varying only the context across three controlled conditions, the paper reports that each context criterion predicts its corresponding behavioral signal—grounding predicts hallucination resistance, guardrails predict manipulation resistance, instruction consistency predicts instruction following, and tool schemas predict tool use. If correct, this reframes context engineering as an auditable, pre-behavioral diagnostic layer of agent evaluation and governance.

Core claim

A controlled study of frontier LLM agents across customer-support, healthcare-claims, and legal-drafting domains (300 multi-turn evaluations, 7,500 turns, three context conditions) shows that variations in context quality alone—with the model held fixed—produce large behavioral differences. The isolated context score tracks the intended context changes, and individual criteria correlate with their hypothesized behavioral targets: grounding sufficiency with hallucination resistance (r=0.63), guardrail coverage with manipulation resistance (r=0.60), instruction consistency with instruction following (r=0.57), and tool schema quality with tool use (r=0.47). The paper claims this establishes con

What carries the argument

The central object is Q_CE, a context-quality function that scores an agent's assembled context X on seven criteria, producing an isolated score that deliberately does not enter the behavioral metric B(A,X). The argument runs on the claim that because Q_CE is computed separately and never feeds the behavior score, any observed correlation between the two is non-circular evidence that context quality is a genuine preflight signal.

Load-bearing premise

The behavioral scores are produced by the same multi-juror LLM infrastructure over the same interaction traces as the context scores, and the paper never states that behavior jurors are blind to the context or that the behavior rubrics were designed to avoid overlap with the context criteria; if juror behavior ratings are influenced by seeing a weak context, the reported correlations are partly an evaluator effect.

What would settle it

Run the same 300 evaluations but have behavioral jurors score agent outputs with the context condition blinded (e.g., stripped system prompts and tool definitions, or randomized condition labels). If the correlation between the context score and behavior ratings collapses or drops sharply, the claimed predictive signal is confounded by juror expectations rather than a property of the context itself.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

Teams could audit an agent's context before running expensive adversarial behavior tests, using criterion-specific scores to spot missing grounding, weak guardrails, or conflicting instructions in advance.
Context engineering becomes a repeatable engineering loop: revise tool schemas, add grounding, harden injection boundaries, rescore the context, and only then evaluate behavior.
The artifact study suggests the same signal extends to generated deliverables (runbooks, contracts, reports), not just live dialogue.
The finding that the weakest context is the cheapest per call but the most dangerous reframes token cost as a reliability concern, not a budgeting one.
Context scores could serve as a governance artifact in regulated domains, providing an auditable record of the operating environment an agent was given.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If this holds up, context quality scores could be used as a pre-deployment gate in regulated industries—without yet proving that a strong context guarantees safe behavior, only that a weak one is a red flag.
The criterion-to-behavior map suggests a natural extension: deliberately degrade one context dimension at a time and measure the isolated behavioral effect, which would sharpen the causal claim beyond correlational evidence.
A testable extension for generalizability: apply the same context ladder to open-weight models and small models to check whether the predictive signal strength varies with model capability.
An evaluator-effect test is implied by the design: behavior jurors should be blinded to context condition to rule out contamination of behavior ratings by perceived context quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The core claim—that context quality independently predicts behavior—is undercut by the shared LLM-juror scoring; the rubric is still useful packaging.

read the letter

Quick take: This is a cleanly written, practically motivated paper with a useful seven-criterion rubric for context quality, but the headline validation is not trustworthy as presented. The same multi-juror LLM infrastructure scores both context and behavior over the same traces, and the paper never says the behavior jurors are blind to the context. So the correlations in Table 6 (e.g., 0.63 for grounding vs. hallucination resistance) may be inflated by the juror's perception of the context rather than an independent predictive relationship. That's a load-bearing flaw.

What's good: the paper names a real gap—context engineering is done but not measured—and packages known failure surfaces (role clarity, guardrails, grounding, etc.) into a coherent questionnaire. The controlled context ladder (poor/structured/hardened) is a sensible design, and the artifact study is a nice extension beyond dialogue. The writing is clear and the mapping from criteria to behavioral signals is plausible and falsifiable. If the measurement were clean, this would be a useful preflight diagnostic for agent teams.

Soft spots: besides the blinding issue, the statistical reporting is thin. Tables 4–6 give means and correlations with no variance, confidence intervals, p-values, or within-condition breakdowns. With 300 evaluations you could do better. The C3 result (hardened context lowers behavioral scores despite higher context quality) is explained as a tradeoff, which is plausible, but without blind scoring it's also consistent with juror bias. No data or code are shipped in the paper; the GitHub link is to a harness, not the study artifacts. The paper also leans on the author's own prior work for the harness and HOB evaluation, which is fine if those are real, but the validation here doesn't independently establish them.

Who it's for: anyone building agent evaluation frameworks or thinking about pre-behavioral context checks. The rubric is worth borrowing even if you ignore the empirical claims.

Recommendation: yes, send to peer review—the question matters and the construct is worth discussing—but the reviewers should demand a redesign with blinded behavioral scoring, human calibration, inter-rater reliability, and proper statistics. My guess is the correlations will shrink but not vanish.

Referee Report

4 major / 5 minor

Summary. AI Agents Do Not Fail Alone proposes that an agent's operating context can be measured independently of its behavior via a seven-criterion construct (role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, token efficiency) implemented in ProofAgent-Harness with multi-juror consensus scoring. The central claim is that context-engineering quality (Q_CE) is a valid preflight reliability signal: although Q_CE is excluded from the behavioral score B(A,X), it should predict behavioral outcomes. The authors test this with a controlled study across customer support, healthcare claims, and legal drafting, holding the LLM fixed and varying context across poor, structured, and hardened conditions. They report that structure improves behavior, that context scores separate the three conditions, and that criteria correlate with behavioral outcomes (e.g., r=0.63 for grounding sufficiency vs. hallucination resistance). An artifact study and token-cost analysis supplement the result.

Significance. The paper addresses an important practical problem and deserves credit for making a concrete proposal: a multi-criteria context score, an open-source harness, a falsifiable mapping from context criteria to behavioral signals, and a controlled manipulation of context while holding the model fixed. If the validity of the measurement were established, the contribution would be meaningful for agent evaluation and governance. However, the validation as presented is not yet sound: the alleged independence between context and behavior judgments is not established, and the statistical reporting is insufficient. The contribution is therefore conditional on substantial additional evidence.

major comments (4)

[§3.3–3.4, §4.1, §4.4] The paper's separation of Q_CE from B(A,X) is computational, not empirical. The same multi-juror LLM infrastructure scores both context and behavior on the same full interaction traces; no blinding of behavior jurors to the context, and no inter-rater reliability or human calibration, is reported. Since constructs are conceptually adjacent (grounding sufficiency vs. hallucination resistance; guardrail coverage vs. manipulation resistance), a behavior juror who reads a weak context may rate behavior lower even for identical agent responses. This evaluator effect would inflate the Table 6 correlations and make the central validation circular in practice. The manuscript must either document a blinding protocol (e.g., behavior jurors see traces with the context section removed or scrambled) or show empirically that behavior ratings are insensitive to perceived context quality.
[§4.4, Table 6] The correlational claim lacks the statistical support needed to sustain it. No confidence intervals, p-values, within-condition correlations, or per-domain/per-backbone breakdowns are reported. With only three context conditions and large mean differences, the r-values in Table 6 could be driven by the experimental manipulation rather than by criterion-level predictive validity across evaluations. Please report within-condition analyses (e.g., correlations within C2/C3), a hierarchical/mixed-effects model with condition as a factor, and variance metrics for Tables 4 and 5.
[§4.3, §4.5, Tables 5 and 7] Table 5 mostly confirms that the manipulation changed the context in the way the authors designed it; this is a manipulation check, not an independent validation of the instrument. The artifact study (Table 7) repeats the same shared-evaluator problem: CE grounding and hallucination resistance are generated by the same harness with no separate or blind evaluator. Without evidence of agreement with human expert raters or stability across judge configurations, the paper cannot claim the measurement is reproducible.
[§3.2, Eq. (1), Table 4] The aggregate Q_CE depends on weights w_k and thresholds τ_s/τ_a that are untested. No sensitivity analysis is reported, so the reported relationships might be sensitive to default equal weighting. Additionally, the C3 effect (higher context quality, slightly lower behavior) is interpreted as a tradeoff; without variance and a pre-specified hypothesis, this post hoc explanation is weak.

minor comments (5)

[Table 3] The number of evaluations per condition is not stated. If 100 per domain is the total across C1/C2/C3, per-cell n is about 33; please clarify.
[Table 6] The column label 'Expected Pearson r' should be 'Observed Pearson r' and should include n, confidence intervals, and the exact correlation method used.
[Table 4] Define the 'Critical' and 'CE' columns. The table is labeled behavioral outcomes, but 'CE' appears to be the context-engineering score; including it in this table is confusing.
[References] Reference [13] spells 'OW ASP' as two words; correct to 'OWASP'.
[§3.3] Specify the default consensus operator Γ and the number of jurors m. The text lists options ('median aggregation, debate-and-revote, Delphi-style') but never states which was used, nor whether the juror configuration is the same as the agent backbones.

Circularity Check

1 steps flagged

Shared multi-juror LLM evaluator for both context and behavior makes Table 6's 'independent prediction' partially circular.

specific steps

other [§3.3, §3.4, §4.1, §4.4 (Table 6)]
"the same infrastructure is used to assess the quality of the agent’s context before or alongside behavioral evaluation ... ProofAgent-Harness supplies adversarial pressure through multi-turn evaluation scenarios and uses a separate frontier LLM juror configuration, with multi-juror consensus scoring, to assess both behavior and context. ... By keeping Q_CE outside the behavioral scoring pipeline, we can test a stronger claim: QCE(X) predicts B(A,X) without being part of B(A,X)."

The paper's only stated isolation is algebraic: QCE is not summed into the behavioral grade. But the measurements are not independent: the same multi-juror LLM scoring infrastructure produces both the context scores and the behavior scores from the same traces, and the paper never states that behavior jurors are blind to the context or that behavior rubrics avoid the construct vocabulary of the context criteria (Table 2 pairs grounding vs. hallucination, guardrails vs. manipulation, etc.). A behavior juror who sees a weak context can rate hallucination resistance or safety lower because of perceived context weakness, even if the agent's outputs contain no unsupported claims. The Table 6 r-values therefore reflect, at least in part, agreement within a single evaluation pipeline rather than

full rationale

Formally, the paper separates Q_CE from B by not including it in the behavioral grade, and no parameter is fitted from behavior and renamed as a prediction. The self-citations to ProofAgent-Harness and HOB are not the circular core, since the harness is open-source and no uniqueness theorem is invoked. The circular step is at the measurement layer: the same ProofAgent-Harness multi-juror LLM pipeline is stated to 'assess both behavior and context' from the same traces, with no blinding of behavior jurors to the context. Because the context criteria and behavior targets are deliberately paired (grounding/hallucination, guardrails/manipulation, instruction consistency/instruction following), a juror's behavior score can be influenced by the very context quality the paper claims to validate as an independent predictor. This makes the Table 6 correlations partly an evaluator effect rather than a clean cross-source prediction. The controlled C1/C2/C3 manipulation and directionally consistent findings give the central claim some independent empirical content, so the paper is not fully circular, but the headline claim of 'non-circular validation' is overstated given the shared measurement design.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

No new physical or computational entities are postulated. The paper's contributions are a composite measurement construct (Q_CE) and an evaluation harness (ProofAgent-Harness), which are methods rather than entities with independent falsifiable handles. The construct's validity is exactly what the paper claims to demonstrate but does not.

free parameters (5)

Criterion weights w_k = Equal weights by default
§3.2 sets the aggregate Q_CE as a weighted average with all criteria equally weighted in the default profile; the grade and any use of the overall score depend on this hand-set choice, though the per-criterion correlations do not.
Grade thresholds τ_s and τ_a = Not specified numerically
§3.2 defines 'Strong', 'Adequate', and 'Weak' grades using thresholds τ_s and τ_a, but no values are given, so the grade output is not reproducible.
Juror count m and consensus operator Γ = Unspecified (median/debate/Delphi options)
§3.3 defines consensus scores via Γ(s_{1,k}, ..., s_{m,k}) but never fixes m or the specific consensus operator; both context and behavior scores depend on these choices.
Evaluator LLM configuration = 'Separate frontier LLM juror configuration'
§4.1 says ProofAgent-Harness uses a separate frontier LLM juror configuration for both context and behavior, but the models, prompts, temperature, and other settings are not reported, so the scoring instrument is underspecified.
Context ladder design (C1/C2/C3) = Hand-constructed
The three context conditions are authored to differ on the criteria being measured; the context score's ability to separate them (§4.3) is therefore partly a manipulation check rather than an independent discovery.

axioms (5)

domain assumption Multi-juror LLM consensus produces valid, objective scores for context quality and behavior without human ground-truth calibration.
§3.3–3.4: no inter-rater reliability, no comparison to human labels, and no analysis of known LLM-judge biases is provided.
domain assumption Behavioral scoring is not influenced by the manipulated context (no halo/evaluator-prior effect).
§4.1: the paper states the context score is isolated from the behavioral score, but never states that behavior jurors are blind to the context; the entire non-circular validation depends on this.
domain assumption The seven criteria are non-overlapping and cover the relevant context-failure surfaces.
§3.2 Table 1: no factor analysis or ablation is reported; e.g., role clarity and instruction consistency overlap conceptually, and injection hardening overlaps with guardrail coverage.
domain assumption Pooling 300 evaluations across three domains and three conditions and computing Pearson correlations is a valid test of the prediction map.
§4.4 Table 6: correlations may be driven by the three-condition differences (ecological correlation) rather than by within-condition signal; no multilevel or per-domain analysis is provided.
domain assumption GPT-5.5 and Claude Opus 4.8 are representative frontier agents whose behavior under context variation generalizes.
§4.1: only two closed backbones are used, with no version details, no open-weight comparison, and no evidence of generalizability.

pith-pipeline@v1.3.0-alltime-deepseek · 13879 in / 14884 out tokens · 153252 ms · 2026-08-02T02:34:25.174610+00:00 · methodology

0 comments

read the original abstract

Context engineering has become central to building reliable AI agents, yet it remains largely unmeasured. Agents do not fail in isolation: their behavior is shaped by the instructions, tools, memory, retrieved knowledge, guardrails, and untrusted inputs accumulated in their context. When this context is weak, agents drift, hallucinate, misuse tools, ignore constraints, become vulnerable to injection, and waste tokens. This paper validates context-engineering quality as an independent leading indicator of agent reliability. We implement the measurement in ProofAgent-Harness, an open-source infrastructure for AI agent evaluation that uses multi-juror, consensus-based scoring. The harness assesses context across seven criteria: role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, and token efficiency. Crucially, the context score is isolated from behavioral metrics and release decisions, enabling a non-circular validation. Through a controlled context-quality study across regulated agent domains, holding frontier LLM agents fixed and varying only their operating context, we show that context-quality criteria consistently predict their corresponding behavioral outcomes. Grounding sufficiency predicts hallucination resistance, guardrail coverage predicts manipulation resistance, instruction consistency predicts instruction following, and tool-schema quality predicts tool use. These findings establish context measurement as a validated preflight signal for agent reliability and position context engineering as an auditable layer of agent evaluation and governance.

Figures

Figures reproduced from arXiv: 2607.14275 by Fouad Bousetouane.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness
cs.MA 2026-07 conditional novelty 5.0

The ProofAgent Index (PAI) adds context, compliance, and governance to behavior scores for AI agent release decisions and ranks risky configurations with AUC 0.98 inside the author's ProofAgent Harness.

Reference graph

Works this paper leans on

26 extracted references · 16 linked inside Pith · cited by 1 Pith paper

[1]

Introducing the model context protocol, 11 2024

Anthropic. Introducing the model context protocol, 11 2024. URL https://www. anthropic.com/news/model-context-protocol. Anthropic announcement

2024
[2]

Effective context engineering for ai agents, 9 2025

Anthropic. Effective context engineering for ai agents, 9 2025. URL https://www. anthropic.com/engineering/effective-context-engineering-for-ai-agents. Anthropic En- gineering blog

2025
[3]

Ai agents need memory control over more context, 2026

Fouad Bousetouane. Ai agents need memory control over more context, 2026. URL https://arxiv.org/abs/2601.11653

arXiv 2026
[4]

Human-on-the-bridge: Scalable evaluation for ai agents, 2026

Fouad Bousetouane. Human-on-the-bridge: Scalable evaluation for ai agents, 2026. URL https://arxiv.org/abs/2606.16871

arXiv 2026
[5]

Proofagent harness: Open infrastructure for adversarial evaluation of ai agents, 2026

Fouad Bousetouane. Proofagent harness: Open infrastructure for adversarial evaluation of ai agents, 2026. URL https://arxiv.org/abs/2605.24134

Pith/arXiv arXiv 2026
[6]

Inan, Lukas Wutschitz, Yanzhi Chen, Robert Sim, and Saravan Rajmohan

Minki Kang, Wei-Ning Chen, Dongge Han, Huseyin A. Inan, Lukas Wutschitz, Yanzhi Chen, Robert Sim, and Saravan Rajmohan. Acon: Optimizing context compression for long-horizon llm agents, 2025. URL https://arxiv.org/abs/2510.00615

Pith/arXiv arXiv 2025
[7]

Retrieval-augmented generation for knowledge-intensive nlp tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K¨ uttler, Mike Lewis, Wen-tau Yih, Tim Rockt¨ aschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. InAdvances in Neural Information Processing Systems, 2020. doi: 10.48550/ arXiv.2005.11401. URL h...

Pith/arXiv arXiv doi:10.48550/arxiv.2005.11401 2020
[8]

Camel: Communicative agents for “mind” exploration of large scale language model society

Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for “mind” exploration of large scale language model society. InAdvances in Neural Information Processing Systems, 2023. doi: 10.48550/arXiv.2303.17760. URL https://arxiv.org/abs/2303.17760

Pith/arXiv arXiv doi:10.48550/arxiv.2303.17760 2023
[9]

Prompt compression for large language models: A survey

Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Collier. Prompt compression for large language models: A survey. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics, 2025. doi: 10.48550/arXiv.2410.12388. URL https://aclanthology.org/2025.naacl-long.368/

Pith/arXiv arXiv doi:10.48550/arxiv.2410.12388 2025
[10]

Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024. doi: 10.1162/tacl a 00638. URL https://aclanthology.org/2024.tacl-1.9/

doi:10.1162/tacl 2024
[11]

Prompt injection attack against llm-integrated applications, 2023

Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and Yang Liu. Prompt injection attack against llm-integrated applications, 2023. URL https://arxiv.org/abs/2306.05499. 22

Pith/arXiv arXiv 2023
[12]

A survey of context engineering for large language models,

Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, Chenlin Zhou, Jiayi Mao, Tianze Xia, Jiafeng Guo, and Shenghua Liu. A survey of context engineering for large language models,
[13]

Llm01:2025 prompt injection, 2025

OW ASP GenAI Security Project. Llm01:2025 prompt injection, 2025. URL https:// genai.owasp.org/llmrisk/llm01-prompt-injection/. OW ASP Top 10 for Large Language Model Applications

2025
[14]

Patil, Ion Stoica, and Joseph E

Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. Memgpt: Towards llms as operating systems, 2023. URL https://arxiv.org/abs/2310.08560

Pith/arXiv arXiv 2023
[15]

A systematic survey of prompt engineering in large language models: Techniques and applications, 2024

Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. A systematic survey of prompt engineering in large language models: Techniques and applications, 2024. URL https://arxiv.org/abs/2402.07927

Pith/arXiv arXiv 2024
[16]

Toolformer: Language models can teach themselves to use tools

Timo Schick, Jane Dwivedi-Yu, Roberto Dess ` ı, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. InAdvances in Neural Information Processing Systems,
[17]

Active context compression: Autonomous memory management in llm agents, 2026

Nikhil Verma. Active context compression: Autonomous memory management in llm agents, 2026. URL https://arxiv.org/abs/2601.07190

arXiv 2026
[18]

Le, Ed H

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. InInternational Conference on Learning Representations,
[19]

Chi, Quoc V

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InAdvances in Neural Information Processing Systems, 2022. doi: 10.48550/arXiv.2201.11903. URL https://arxiv.org/abs/2201.11903

Pith/arXiv arXiv doi:10.48550/arxiv.2201.11903 2022
[20]

White, Doug Burger, and Chi Wang

Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W. White, Doug Burger, and Chi Wang. Autogen: Enabling next-gen llm applications via multi- agent conversation, 2023. URL https://arxiv.org/abs/2308.08155

Pith/arXiv arXiv 2023
[21]

URL https://arxiv.org/abs/2203.11171

doi: 10.48550/arXiv.2203.11171. URL https://arxiv.org/abs/2203.11171

Pith/arXiv arXiv doi:10.48550/arxiv.2203.11171
[22]

Agentic context engineering: Evolving contexts for self-improving language models, 2025

Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsid- har Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic context engineering: Evolving contexts for self-improving language models, 2025. URL https://arxiv.org/abs/2510.04618. 23

Pith/arXiv arXiv 2025
[23]

Large language models are human-level prompt engineers, 2022

Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers, 2022. URL https://arxiv.org/abs/2211.01910. 24

Pith/arXiv arXiv 2022
[24]

ReAct: Synergizing reasoning and acting in language models

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InInterna- tional Conference on Learning Representations, 2023. doi: 10.48550/arXiv.2210.03629. URL https://arxiv.org/abs/2210.03629

Pith/arXiv arXiv doi:10.48550/arxiv.2210.03629 2023
[2023]

URL https://arxiv.org/abs/2302.04761

doi: 10.48550/arXiv.2302.04761. URL https://arxiv.org/abs/2302.04761

Pith/arXiv arXiv doi:10.48550/arxiv.2302.04761
[2025]

URL https://arxiv.org/abs/2507.13334

Pith/arXiv arXiv

[1] [1]

Introducing the model context protocol, 11 2024

Anthropic. Introducing the model context protocol, 11 2024. URL https://www. anthropic.com/news/model-context-protocol. Anthropic announcement

2024

[2] [2]

Effective context engineering for ai agents, 9 2025

Anthropic. Effective context engineering for ai agents, 9 2025. URL https://www. anthropic.com/engineering/effective-context-engineering-for-ai-agents. Anthropic En- gineering blog

2025

[3] [3]

Ai agents need memory control over more context, 2026

Fouad Bousetouane. Ai agents need memory control over more context, 2026. URL https://arxiv.org/abs/2601.11653

arXiv 2026

[4] [4]

Human-on-the-bridge: Scalable evaluation for ai agents, 2026

Fouad Bousetouane. Human-on-the-bridge: Scalable evaluation for ai agents, 2026. URL https://arxiv.org/abs/2606.16871

arXiv 2026

[5] [5]

Proofagent harness: Open infrastructure for adversarial evaluation of ai agents, 2026

Fouad Bousetouane. Proofagent harness: Open infrastructure for adversarial evaluation of ai agents, 2026. URL https://arxiv.org/abs/2605.24134

Pith/arXiv arXiv 2026

[6] [6]

Inan, Lukas Wutschitz, Yanzhi Chen, Robert Sim, and Saravan Rajmohan

Minki Kang, Wei-Ning Chen, Dongge Han, Huseyin A. Inan, Lukas Wutschitz, Yanzhi Chen, Robert Sim, and Saravan Rajmohan. Acon: Optimizing context compression for long-horizon llm agents, 2025. URL https://arxiv.org/abs/2510.00615

Pith/arXiv arXiv 2025

[7] [7]

Retrieval-augmented generation for knowledge-intensive nlp tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K¨ uttler, Mike Lewis, Wen-tau Yih, Tim Rockt¨ aschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. InAdvances in Neural Information Processing Systems, 2020. doi: 10.48550/ arXiv.2005.11401. URL h...

Pith/arXiv arXiv doi:10.48550/arxiv.2005.11401 2020

[8] [8]

Camel: Communicative agents for “mind” exploration of large scale language model society

Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for “mind” exploration of large scale language model society. InAdvances in Neural Information Processing Systems, 2023. doi: 10.48550/arXiv.2303.17760. URL https://arxiv.org/abs/2303.17760

Pith/arXiv arXiv doi:10.48550/arxiv.2303.17760 2023

[9] [9]

Prompt compression for large language models: A survey

Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Collier. Prompt compression for large language models: A survey. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics, 2025. doi: 10.48550/arXiv.2410.12388. URL https://aclanthology.org/2025.naacl-long.368/

Pith/arXiv arXiv doi:10.48550/arxiv.2410.12388 2025

[10] [10]

Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024. doi: 10.1162/tacl a 00638. URL https://aclanthology.org/2024.tacl-1.9/

doi:10.1162/tacl 2024

[11] [11]

Prompt injection attack against llm-integrated applications, 2023

Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and Yang Liu. Prompt injection attack against llm-integrated applications, 2023. URL https://arxiv.org/abs/2306.05499. 22

Pith/arXiv arXiv 2023

[12] [12]

A survey of context engineering for large language models,

Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, Chenlin Zhou, Jiayi Mao, Tianze Xia, Jiafeng Guo, and Shenghua Liu. A survey of context engineering for large language models,

[13] [13]

Llm01:2025 prompt injection, 2025

OW ASP GenAI Security Project. Llm01:2025 prompt injection, 2025. URL https:// genai.owasp.org/llmrisk/llm01-prompt-injection/. OW ASP Top 10 for Large Language Model Applications

2025

[14] [14]

Patil, Ion Stoica, and Joseph E

Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. Memgpt: Towards llms as operating systems, 2023. URL https://arxiv.org/abs/2310.08560

Pith/arXiv arXiv 2023

[15] [15]

A systematic survey of prompt engineering in large language models: Techniques and applications, 2024

Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. A systematic survey of prompt engineering in large language models: Techniques and applications, 2024. URL https://arxiv.org/abs/2402.07927

Pith/arXiv arXiv 2024

[16] [16]

Toolformer: Language models can teach themselves to use tools

Timo Schick, Jane Dwivedi-Yu, Roberto Dess ` ı, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. InAdvances in Neural Information Processing Systems,

[17] [17]

Active context compression: Autonomous memory management in llm agents, 2026

Nikhil Verma. Active context compression: Autonomous memory management in llm agents, 2026. URL https://arxiv.org/abs/2601.07190

arXiv 2026

[18] [18]

Le, Ed H

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. InInternational Conference on Learning Representations,

[19] [19]

Chi, Quoc V

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InAdvances in Neural Information Processing Systems, 2022. doi: 10.48550/arXiv.2201.11903. URL https://arxiv.org/abs/2201.11903

Pith/arXiv arXiv doi:10.48550/arxiv.2201.11903 2022

[20] [20]

White, Doug Burger, and Chi Wang

Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W. White, Doug Burger, and Chi Wang. Autogen: Enabling next-gen llm applications via multi- agent conversation, 2023. URL https://arxiv.org/abs/2308.08155

Pith/arXiv arXiv 2023

[21] [21]

URL https://arxiv.org/abs/2203.11171

doi: 10.48550/arXiv.2203.11171. URL https://arxiv.org/abs/2203.11171

Pith/arXiv arXiv doi:10.48550/arxiv.2203.11171

[22] [22]

Agentic context engineering: Evolving contexts for self-improving language models, 2025

Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsid- har Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic context engineering: Evolving contexts for self-improving language models, 2025. URL https://arxiv.org/abs/2510.04618. 23

Pith/arXiv arXiv 2025

[23] [23]

Large language models are human-level prompt engineers, 2022

Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers, 2022. URL https://arxiv.org/abs/2211.01910. 24

Pith/arXiv arXiv 2022

[24] [24]

ReAct: Synergizing reasoning and acting in language models

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InInterna- tional Conference on Learning Representations, 2023. doi: 10.48550/arXiv.2210.03629. URL https://arxiv.org/abs/2210.03629

Pith/arXiv arXiv doi:10.48550/arxiv.2210.03629 2023

[25] [2023]

URL https://arxiv.org/abs/2302.04761

doi: 10.48550/arXiv.2302.04761. URL https://arxiv.org/abs/2302.04761

Pith/arXiv arXiv doi:10.48550/arxiv.2302.04761

[26] [2025]

URL https://arxiv.org/abs/2507.13334

Pith/arXiv arXiv