pith. sign in

arxiv: 2605.22102 · v1 · pith:UNWIOWJFnew · submitted 2026-05-21 · 💻 cs.AI

ExComm: Exploration-Stage Communication for Error-Resilient Agentic Test-Time Scaling

Woomin Song , Beomjun Kim , Daewon Choi , Sai Muralidhar Jayanthi , Saket Dingliwal , Jinwoo Shin , Aram Galstyan This is my paper

Pith reviewed 2026-05-22 05:49 UTC · model grok-4.3

classification 💻 cs.AI
keywords agentic test-time scalingerror propagationmulti-agent communicationbelief state auditingtool-based verificationtrajectory diversificationAIME benchmarkGAIA benchmark
0
0 comments X

The pith

ExComm detects cross-agent factual conflicts to prevent error propagation in agentic test-time scaling.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes ExComm, a communication protocol designed to make agentic test-time scaling more resilient to errors. It is based on the idea that many mistakes made by agents working in parallel show up as conflicting facts between their belief states. By periodically checking for these conflicts and fixing them with tool-assisted verification, the method corrects issues early rather than letting them affect later steps. Soft updates add the corrections without erasing what the agents already know, and a diversification step keeps different reasoning paths distinct. This approach is shown to deliver better results than previous methods on math and general agent benchmarks.

Core claim

ExComm periodically audits agent belief states to detect such conflicts, resolves them through a dedicated tool-based verification loop, and returns concise, targeted feedback to the involved agents. Corrections are incorporated through soft belief updates, which append verified feedback rather than overwriting existing beliefs. To prevent collapsing trajectory diversity due to communication, ExComm further introduces a trajectory diversification module that redirects redundant trajectories toward orthogonal strategies.

What carries the argument

ExComm, the exploration-stage communication protocol that audits belief states for cross-agent factual conflicts and resolves them via tool-based verification with soft updates.

Load-bearing premise

The majority of intermediate errors in parallel agentic reasoning produce detectable cross-agent factual conflicts that can be resolved through a dedicated tool-based verification loop.

What would settle it

An experiment that measures the frequency of detectable cross-agent factual conflicts from intermediate errors and finds them to be rare would undermine the protocol's core motivation.

Figures

Figures reproduced from arXiv: 2605.22102 by Aram Galstyan, Beomjun Kim, Daewon Choi, Jinwoo Shin, Sai Muralidhar Jayanthi, Saket Dingliwal, Woomin Song.

Figure 1
Figure 1. Figure 1: Overview of ExComm. ExComm augments a standard agentic test-time scaling loop with an exploration-stage communication step. After each execution step, the Online Belief Consistency Module gathers agent belief states {Bi} N i=1, detects factual conflicts, resolves them through tool￾augmented verification, and produces a set of targeted resolutions R. Each agent Ai receives only the relevant subset Ri ⊆ R, w… view at source ↗
Figure 2
Figure 2. Figure 2: Performance-Cost Trade-Off. We plot majority-voting accuracy versus normal￾ized API cost on AIME 2024 using Gemini￾2.5-Flash-Lite. Error bars are standard errors. To assess the efficiency of ExComm, we analyze the per-sample API cost of each method and summa￾rize the performance-cost trade-off in [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The prompt template used for the critic model. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Prompt template for error type classification. Categories 1 and 2 are merged as “Conflicting” [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
read the original abstract

A common failure mode in long-horizon agentic test-time scaling is error propagation, where factual errors or invalid deductions introduced at intermediate steps persist in the agent's belief state and contaminate later reasoning. Existing test-time scaling methods provide limited control over this process, as they often rely on agents to detect their own mistakes, select among flawed trajectories, or refine solutions only after errors have already shaped the reasoning path. We propose ExComm, a communication protocol for exploration-stage agentic test-time scaling. ExComm is motivated by the empirical observation that the majority of intermediate errors in parallel agentic reasoning produce detectable cross-agent factual conflicts. Leveraging the iterative structure of agentic workflows, ExComm periodically audits agent belief states to detect such conflicts, resolves them through a dedicated tool-based verification loop, and returns concise, targeted feedback to the involved agents. Corrections are incorporated through soft belief updates, which append verified feedback rather than overwriting existing beliefs. Furthermore, to prevent collapsing trajectory diversity due to communication, ExComm further introduces a trajectory diversification module that redirects redundant trajectories toward orthogonal strategies. Experiments on AIME 2024, AIME 2025, and GAIA with Gemini-2.5-Flash-Lite and Qwen3.5-4B show that ExComm consistently outperforms strong test-time scaling baselines, achieving average performance gains of 5.7% and 5.0% over the best-performing baselines, respectively. Further analyses demonstrate improved error recovery, favorable scaling behavior, stronger diversity than adapted communication baselines, and the best performance-cost trade-off among the evaluated methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes ExComm, a communication protocol for exploration-stage agentic test-time scaling to address error propagation in long-horizon agentic workflows. Motivated by the claim that most intermediate errors produce detectable cross-agent factual conflicts, ExComm periodically audits belief states, resolves conflicts via a tool-based verification loop, applies soft belief updates by appending verified feedback, and adds a trajectory diversification module to avoid collapse in diversity. Experiments on AIME 2024/2025 and GAIA using Gemini-2.5-Flash-Lite and Qwen3.5-4B report average gains of 5.7% and 5.0% over the best-performing test-time scaling baselines, with additional analyses on error recovery, scaling behavior, diversity, and performance-cost trade-offs.

Significance. If the results hold and the motivating observation is substantiated, ExComm could provide a practical mechanism for improving error resilience in parallel agentic systems by leveraging inter-agent communication and external verification rather than relying solely on self-correction. The reported gains, favorable scaling, and emphasis on maintaining trajectory diversity represent a targeted contribution to test-time scaling methods; the performance-cost analysis is a strength that could inform deployment decisions.

major comments (2)
  1. [§1 / Motivation] The central motivation (§1 and abstract) states that 'the majority of intermediate errors in parallel agentic reasoning produce detectable cross-agent factual conflicts' but supplies no quantitative breakdown by error type (factual vs. logical/deductive), no preliminary statistics on conflict frequency, and no description of the detection heuristic (e.g., embedding similarity threshold, exact match, or LLM judge). This assumption is load-bearing for the verification loop; if factual conflicts are not dominant or detection is noisy, gains may stem primarily from diversification or extra compute rather than the claimed mechanism.
  2. [§4 / Experiments] Table 1 and §4.2 report average gains of 5.7% (AIME) and 5.0% (GAIA) over best baselines, yet the manuscript does not detail variance across runs, number of seeds, exact baseline re-implementations (including how communication baselines were adapted), or data-exclusion rules. Without these, it is impossible to determine whether the improvements are robust or sensitive to post-hoc choices.
minor comments (2)
  1. [§3.2] The description of 'soft belief updates' in §3.2 is high-level; a short pseudocode snippet or concrete example of how verified feedback is appended without overwriting prior beliefs would improve reproducibility.
  2. [Figure 4] Figure 4 (scaling curves) lacks error bars or shaded regions indicating run-to-run variability, making it harder to assess the reliability of the observed scaling trends.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback on our manuscript. We appreciate the opportunity to clarify the motivating assumptions and experimental reporting in ExComm. We address each major comment below and are prepared to revise the paper accordingly to strengthen its presentation.

read point-by-point responses

  1. Referee: [§1 / Motivation] The central motivation (§1 and abstract) states that 'the majority of intermediate errors in parallel agentic reasoning produce detectable cross-agent factual conflicts' but supplies no quantitative breakdown by error type (factual vs. logical/deductive), no preliminary statistics on conflict frequency, and no description of the detection heuristic (e.g., embedding similarity threshold, exact match, or LLM judge). This assumption is load-bearing for the verification loop; if factual conflicts are not dominant or detection is noisy, gains may stem primarily from diversification or extra compute rather than the claimed mechanism.

    Authors: We agree that the manuscript would be strengthened by a more explicit quantitative grounding of this motivating observation. The claim stems from observations made during system development on sample trajectories, but we did not include a dedicated breakdown or heuristic description in the submitted version to keep the focus on the core method. In revision, we will add a concise description of the detection heuristic (an LLM judge for factual consistency between belief states, using semantic equivalence with a fixed threshold) and preliminary statistics from a pilot analysis of error types and conflict rates. This addition will help substantiate that the reported gains arise from targeted conflict resolution rather than diversification or compute alone. We will place this material in Section 1 or as a short subsection in the method. revision: yes

  2. Referee: [§4 / Experiments] Table 1 and §4.2 report average gains of 5.7% (AIME) and 5.0% (GAIA) over best baselines, yet the manuscript does not detail variance across runs, number of seeds, exact baseline re-implementations (including how communication baselines were adapted), or data-exclusion rules. Without these, it is impossible to determine whether the improvements are robust or sensitive to post-hoc choices.

    Authors: We concur that these details are necessary for evaluating robustness and reproducibility. The current manuscript reports only aggregate averages. In the revised version, we will specify the number of random seeds, report variance or standard deviations (with error bars added to Table 1), provide precise descriptions of baseline re-implementations including adaptations made to existing communication baselines, and clarify that data exclusion followed official benchmark protocols with no additional post-hoc filtering. These changes will be incorporated into Section 4.2. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical claims rest on experimental validation

full rationale

The paper introduces ExComm as a protocol motivated by an empirical observation about cross-agent factual conflicts in agentic reasoning, then demonstrates performance gains via experiments on AIME 2024/2025 and GAIA. No mathematical derivation chain, equations, or first-principles results are presented that reduce by construction to fitted inputs, self-citations, or renamed patterns. The central claims rely on reported benchmark improvements rather than any self-referential loop. The motivating observation is stated as empirical but is not used as a load-bearing derivation; it functions as motivation for the method design. This is a standard empirical contribution with no detectable circularity per the specified patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The approach depends on the unverified empirical observation that most errors create detectable factual conflicts and on the assumption that tool-based verification can resolve them without introducing new errors. No free parameters or invented physical entities are mentioned.

axioms (1)
  • domain assumption Majority of intermediate errors produce detectable cross-agent factual conflicts
    Stated as the motivating empirical observation in the abstract; used to justify the conflict-auditing step.

pith-pipeline@v0.9.0 · 5846 in / 1268 out tokens · 39789 ms · 2026-05-22T05:49:19.713405+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages · 9 internal anchors

  1. [1]

    Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute opti- mally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  2. [2]

    Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems, 36:46534–46594, 2023

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems, 36:46534–46594, 2023

    work page 2023

  3. [3]

    Evolving deeper llm thinking.arXiv preprint arXiv:2501.09891, 2025

    Kuang-Huei Lee, Ian Fischer, Yueh-Hua Wu, Dave Marwood, Shumeet Baluja, Dale Schu- urmans, and Xinyun Chen. Evolving deeper llm thinking.arXiv preprint arXiv:2501.09891, 2025

    work page arXiv 2025

  4. [4]

    ACM Transactions on Intelligent Systems and Technology, 15(3)

    Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language model generation.arXiv preprint arXiv:2311.17311, 2023

    work page arXiv 2023

  5. [5]

    Large Language Monkeys: Scaling Inference Compute with Repeated Sampling

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809–11822, 2023

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809–11822, 2023

    work page 2023

  7. [7]

    Scaling test time com- pute for open models

    Edward Beeching, Lewis Tunstall, and Sasha Rush. Scaling test time com- pute for open models. https://huggingface.co/spaces/HuggingFaceH4/ blogpost-scaling-test-time-compute , 2024. Hugging Face Blog, Accessed: 2025-09-20

    work page 2024

  8. [8]

    Scaling test-time compute for LLM agents

    King Zhu, Hanhao Li, Siwei Wu, Tianshun Xing, Dehua Ma, Xiangru Tang, Minghao Liu, Jian Yang, Jiaheng Liu, Yuchen Eleanor Jiang, et al. Scaling test-time compute for llm agents.arXiv preprint arXiv:2506.12928, 2025

    work page arXiv 2025

  9. [9]

    Co-sight: Enhancing llm-based agents via conflict-aware meta-verification and trustworthy reasoning with structured facts.arXiv preprint arXiv:2510.21557, 2025

    Hongwei Zhang, Ji Lu, Shiqing Jiang, Chenxiang Zhu, Li Xie, Chen Zhong, Haoran Chen, Yurui Zhu, Yongsheng Du, Yanqin Gao, et al. Co-sight: Enhancing llm-based agents via conflict-aware meta-verification and trustworthy reasoning with structured facts.arXiv preprint arXiv:2510.21557, 2025

    work page arXiv 2025

  10. [10]

    Large Language Models Cannot Self-Correct Reasoning Yet

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet.arXiv preprint arXiv:2310.01798, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  11. [11]

    When can llms actually correct their own mistakes? a critical survey of self-correction of llms.Transactions of the Association for Computational Linguistics, 12:1417–1440, 2024

    Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can llms actually correct their own mistakes? a critical survey of self-correction of llms.Transactions of the Association for Computational Linguistics, 12:1417–1440, 2024

    work page 2024

  12. [12]

    MathArena: Evaluating LLMs on Uncontaminated Math Competitions

    Mislav Balunovi´c, Jasper Dekoninck, Ivo Petrov, Nikola Jovanovi´c, and Martin Vechev. Math- arena: Evaluating llms on uncontaminated math competitions.arXiv preprint arXiv:2505.23281, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  13. [13]

    Gaia: a benchmark for general ai assistants

    Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. InThe Twelfth International Conference on Learning Representations, 2024

    work page 2024

  14. [14]

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  15. [15]

    Qwen3.5: Towards native multimodal agents.Qwen

    Qwen Team. Qwen3.5: Towards native multimodal agents.Qwen. URL: https://qwen.ai/blog?id=qwen3.5 (Access Date: 07. 05. 2026), 2026. 10

    work page 2026

  16. [16]

    Improv- ing factuality and reasoning in language models through multiagent debate

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improv- ing factuality and reasoning in language models through multiagent debate. InForty-first international conference on machine learning, 2024

    work page 2024

  17. [17]

    Mixture-of-agents enhances large language model capabilities.International Conference on Learning Representa- tions, 2025

    Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agents enhances large language model capabilities.International Conference on Learning Representa- tions, 2025

    work page 2025

  18. [18]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InThe eleventh international conference on learning representations, 2023

    work page 2023

  19. [19]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.Advances in Neural Information Processing Systems, 36:38154–38180, 2023

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.Advances in Neural Information Processing Systems, 36:38154–38180, 2023

    work page 2023

  20. [20]

    OpenHands: An Open Platform for AI Software Developers as Generalist Agents

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. Openhands: An open platform for ai software developers as generalist agents, 2024b.URL https://arxiv. org/abs/2407.16741, 2(4):9, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  21. [21]

    Toolformer: Language models can teach themselves to use tools.Advances in Neural Information Processing Systems 36 (2023): 68539-68551., 2023

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettle- moyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools.Advances in Neural Information Processing Systems 36 (2023): 68539-68551., 2023

    work page 2023

  22. [22]

    Easytool: Enhancing llm-based agents with concise tool instruction

    Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Kan Ren, Dongsheng Li, and Deqing Yang. Easytool: Enhancing llm-based agents with concise tool instruction. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages ...

    work page 2025

  23. [23]

    Toward self-improvement of llms via imagination, searching, and criticizing.Advances in Neural Information Processing Systems, 37:52723–52748, 2024

    Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Lei Han, Haitao Mi, and Dong Yu. Toward self-improvement of llms via imagination, searching, and criticizing.Advances in Neural Information Processing Systems, 37:52723–52748, 2024

    work page 2024

  24. [24]

    Processbench: Identifying process errors in mathematical reasoning

    Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Processbench: Identifying process errors in mathematical reasoning, 2025. URLhttps://arxiv.org/abs/2412.06559

    work page arXiv 2025

  25. [25]

    A new era of intelligence with gemini 3.Google

    Sundar Pichai, Demis Hassabis, and Koray Kavukcuoglu. A new era of intelligence with gemini 3.Google. URL: https://blog.google/products-and-platforms/products/gemini/gemini-3/(Access Date: 07. 05. 2026), 2025

    work page 2026

  26. [26]

    OpenAI GPT-5 System Card

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  27. [27]

    Qwen3 Technical Report

    Qwen Team. Qwen3 technical report, 2025. URLhttps://arxiv.org/abs/2505.09388

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  28. [28]

    Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi- agent debate, 2024. URLhttps://arxiv.org/abs/2305.19118

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  29. [29]

    Improving diversity of commonsense generation by large language models via in-context learning, 2024

    Tianhui Zhang, Bei Peng, and Danushka Bollegala. Improving diversity of commonsense generation by large language models via in-context learning, 2024. URL https://arxiv. org/abs/2404.16807. 11 A Further Discussions A.1 Broader Impact Statement This work studies methods for improving the reliability of agentic test-time scaling through exploration-stage co...

    work page arXiv 2024

  30. [30]

    To this end, we provide the critic with a ground-truth reference solution

    Additional Use of Reference Solutions.Unlike ProcessBench, which aimed to benchmark the performance of the critic itself, our objective is to maximize the accuracy of the critique. To this end, we provide the critic with a ground-truth reference solution. This is particularly beneficial for tool-augmented benchmarks like GAIA; without a reference solution...

    work page

  31. [31]

    Although detecting error recovery is a more complex task, the provision of a reference solution significantly mitigates the difficulty

    Identification of Error Recovery.While ProcessBench focused solely on detecting the existence and location of the first error, we extend this scope to determine if the error is also recovered in subsequent steps. Although detecting error recovery is a more complex task, the provision of a reference solution significantly mitigates the difficulty. By treat...

    work page

  32. [32]

    Action" and

    Analyze the "Action" and "Memory Diff" of {agent_id}

    work page

  33. [33]

    Factual Errors

    Identify "Factual Errors" committed by {agent_id}. - Definition: A Factual Error is where {agent_id} incorrectly derives an intermediate result or maintains an incorrect intermediate result that impacts decision-making. - Exclusions: Minor tool errors, or errors made by other agents

    work page

  34. [34]

    - Occurrence Step: Step number where {agent_id} introduced the error

    For each Factual Error: - Error Type: Description. - Occurrence Step: Step number where {agent_id} introduced the error. - Recovered Step: Step number where the error was corrected (or "N/A")

    work page

  35. [35]

    N/A" (Unrecovered). - Do NOT mark an error as

    Final Check: - If the agent's final answer is INCORRECT (differs from Reference Answer), there MUST be at least one Factual Error that is "N/A" (Unrecovered). - Do NOT mark an error as "Recovered" if the agent proceeded to a wrong conclusion based on a related misconception. # Output Format Provide your analysis in the following XML format: <error_recover...

    work page 2024