Pith.

Artificial Intelligence · arXiv:2610.00372 v1 · submitted 30 Sep 2026 · 37 references

When Harnesses Lose the Signal: Causal Evaluation of Recovery in LLM Agents

Shuyao Xiao, Shengling Wang, Xuan Chen, Ke Chao, Ming Cui, Feifei Qian, Chaoyang Mei, Fanlin Meng

show 2 more authors

Ziming Yu, Junxi Yin

The paper says

“Large language model agents rely on external harnesses to pass information between the model and its environment and to recover from execution errors. Yet recovery is usually judged only by average task success. This hides an important tension.”

more of the abstract

The same operation can rescue a failing trajectory or disrupt one that would otherwise succeed. We frame recovery as a causal decision problem. Starting from the same execution state, we compare what happens with and without recovery, separate rescue from harm, and study how the value of recovery changes over time. We then introduce the Causal Intervention Router (CIR), a lightweight policy that uses information available before recovery to decide when intervention is worthwhile. On long-horizon ALFWorld tasks with Qwen3-14B, CIR raises success from 70.33% to 73.33%, a gain of 3.00 percentage points. It leaves all evaluated trajectories with correct observations untouched. Additional controls show that the benefit of recovery cannot be explained solely by the new observation returned by the environment. These results provide a practical way to evaluate recovery and apply it selectively.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 3 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A careful, honest empirical study whose paired rescue/harm framing is the real contribution; the headline gain is real but modest, and the causal interpretation rests on an unverified post-branch determinism assumption. This line is wrong

2 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 37 references say.

The paper says, at 3.2, 5.1, Appendix A

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The causal attribution in Eqs. (1)–(3) and the CIR supervision labels of Section 4.2 both require the two branches to be identical except for the recovery operation. The paper verifies identical histories before the branch point (Section 5.1) and records no environment-replay mismatches (Appendix A), but it never checks that the two branches remain token-identical after the branch point under greedy decoding.

more of objection 1

Section 5.2 additionally relies on the same assumption when it claims that the clean-observation no-recovery branch reproduces the factual success, making Y0_i(clean)=1 'by construction'. This is a load-bearing, unverified step: any post-branch token-level divergence confounds the rescue/harm decomposition and contaminates the training labels. I recommend a concrete reproducibility check, e.g., re-executing each branch (or a subsample of branches) twice from the same state and confirming identical token sequences, or reporting a divergence bound and a sensitivity analysis of the reported counts and the p=0.034 result. Without this, the central causal claims rest on an assertion that is stated but not demonstrated.

The paper says, at 5.4, Table 1, Section 6

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The headline gain over never-refreshing (+3.00 pp, p=0.034) is supported, but the paper's framing that anomaly detection 'is insufficient' and that outcome prediction drives the benefit is not statistically supported. CIR's margins over the two anomaly baselines are 1.33 pp (95% CI [−0.33, 3.33]) and 1.00 pp (95% CI [−0.67, 2.67]), both intervals including zero, with only 11 vs 6 rescues at similar intervention rates.

more of objection 2

The directional pattern is consistent with the design, but the Discussion and Conclusion claim more than these intervals warrant. Either a power/sensitivity analysis should be provided, or the claim should be moderated to state that CIR is not worse than the baselines while producing the highest point estimate. The mechanism claim (part of Section 5.3) is, by contrast, well supported by the shared-rescue counts.

4 minor objections

The paper says, at 5.1

Pith says: minor objection 1 of 4

The naming 'valid seen' and 'valid unseen' is confusing given that fitting uses one split and evaluation the other; please clarify explicitly which ALFWorld split is used for fitting and which for evaluation, and that no task crosses splits.

The paper says, at 5.4, Table 1

Pith says: minor objection 2 of 4

The gains under one-step stale and missing observations are both +1.33 pp with lower confidence bound exactly 0.00; consider stating explicitly that the overall benefit is concentrated in the two-step stale condition (9.33 pp) and that evidence under the other conditions is weak.

show 2 more

The paper says, at Fig. 2

Pith says: minor objection 3 of 4

In Figure 2(b), the 'Primary (n=106)' and 'Independent (n=100)' labels overlap the plotted points; please adjust the layout so the panel labels are legible.

The paper says, at 4.3

Pith says: minor objection 4 of 4

The thresholds α=0.876, β=0.052, γ=0 and the choices λ=2, ϵ=0.02, d_min=1 are stated without any description of how sensitive the reported results are to these settings; a short sensitivity or robustness note would strengthen the claim that thresholds were selected without test-set tuning.

What would settle it

Select at least 10 tasks from the evaluation cohort. For each task, under the same injected error condition and at the same execution state, run the no-recovery branch 5 times with identical input, model weights, greedy decoding, and hardware. Record whether the 5 runs produce token-for-token identical action sequences and identical final outcomes. If any run diverges, quantify the divergence rate and estimate how many of the reported rescue/harm counts (e.g., 11 rescues and 2 harms in Section 5.4) could change under that rate; then check whether the +3.00 pp CIR gain and its 95% CI still exclude zero after simulating the noise. The check should also verify that the harness's environment replay is deterministic by comparing environment states at each step across the 5 runs.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 3 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 37 s · 15,390 tokens read, 5,110 written
  3. Editor wrote the verdict above 10 s · 16,402 tokens read, 1,293 written
  4. Referee major revision: 2 major objections, 4 minor 29 s · 15,734 tokens read, 4,370 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Worth reading. The genuinely new thing here is the paired rescue/harm framing for harness recovery: run two branches from the same state, one with refresh, one without, then count rescues and harms separately instead of averaging over success rates. That decomposition is useful for anyone designing agent harnesses, and the content-ablated control is a nice touch—it shows the benefit of refresh is not just about showing the agent a new observation, since 8 of 10 rescues survive when the returned content is hidden. CIR, the selective policy, is a sensible extension: train T-learner outcome models on the paired data, threshold on harm, and intervene only when the predicted benefit clears the bar. The experimental design is genuinely careful: task-disjoint fitting and evaluation, out-of-fold threshold selection, cluster bootstrap, sign-flip test, and honest reporting of both positive and negative effects. The held-out gain of +3.00 pp (95% CI [0.67, 5.67], sign-flip p=0.034) over never-refreshing is supported and is not a fitted quantity.

The soft spots are real but proportionate. The load-bearing assumption is deterministic replay after the branch point. The paper verifies identical histories before branching and reports no environment-replay mismatches, but nothing checks that the two branches stay identical after the recovery decision. With greedy decoding on fixed hardware this is usually reproducible, but it is asserted, not demonstrated, and every rescue/harm count and every CIR training label depends on it. A cautious referee should ask for a determinism check or at least an acknowledgment that token-level divergence would confound the instance-level effects. Second, CIR's advantage over the two anomaly-detection baselines is not statistically significant—margins of 1.33 and 1.00 pp with CIs that include negative values. So the specific claim that outcome prediction adds value over simple anomaly detection is not established, though the point estimates favor CIR. Third, no code or data is shipped, and sample sizes are small (75–106 tasks), which is why the wide CIs around the 9.33 pp effect in the most severely outdated condition deserve caution. The paper itself acknowledges that its scores are not rescue or harm probabilities and that averages hide countervailing effects.

Who is this for: harness researchers and anyone evaluating recovery or self-correction mechanisms in LLM agents. It deserves a serious referee, with requests for the determinism evidence, baseline-significance clarity, and code or data release. My own verdict would be conditional acceptance, not rejection. The framing is the contribution, and it is solid.

The strongest objection

The causal framework rests on the claim in Section 3.2 that the two branches 'use the same state, history, model, and decoding settings, their difference is attributable to the recovery decision for the states studied here.' The paper verifies identical histories before branching (Section 5.1: 'We also verify that the two runs have identical histories before they branch') and reports 'No environment-replay mismatches were recorded' in Appendix A, but nothing checks whether the two branches remain identical after the branch point. Greedy decoding is deterministic only if the token-selection procedure is exactly reproducible; in practice, floating-point accumulation order, GPU nondeterminism, or subtle sampler settings (e.g., order of logits) can cause divergence. Environment replay may also have nondeterministic elements not captured by the prefix check. If any divergence occurs, the instance-level effects in Equations 1-3 mix the recovery operation's causal effect with independent noise. More insidiously, CIR is trained on supervision labels derived from these paired outcomes (Section 4.2). If those labels are corrupted by nondeterminism, CIR may learn spurious patterns that nevertheless yield a positive held-out gain by chance, especially given the small sample (75 tasks, wide CIs around the +3.00 pp effect). The paper states the deterministic-replay condition as an assumption and partially verifies it, but the unverified part—post-branch determinism—is precisely what carries the causal interpretation. This is the single most load-bearing assumption because every rescue/harm count, every training label, and the CIR policy's learned utility depend on it.

The reader identified deterministic replay as the weakest assumption, and my reading agrees. The paper explicitly states the deterministic condition, so the concern is not that the authors are unaware; it is that they provide only branch-point verification, not post-branch verification. A concrete test of post-branch reproducibility is the natural settlement: if the branch appears exactly once yet the run is nondeterministic, the causal attribution and the learned policy both inherit ambiguity. My proposed test directly checks this. If the test passes (5 identical runs per task), the causal framing stands and the verdict remains CONDITIONAL (code/data still missing, CIR-vs-baseline margins not significant). If it fails, the verdict should drop to REJECT or at least UNVERDICTED, because the central claim would be unsupported. Since the test has not been run, I do not change the verdict now; the unresolved nature of the assumption is a reason that CONDITIONAL is appropriate.

The referee's report

If the paired counterfactual framing holds, recovery in LLM agent harnesses is shown to be a state- and time-dependent decision rather than an unconditionally beneficial operation, and a lightweight selective policy is demonstrated to beat never-refreshing on a task-disjoint held-out set. The design is notably careful: paired runs from the same state with verified pre-branch prefix identity, task-disjoint evaluation, out-of-fold threshold selection that avoids leakage, task-level cluster bootstrap confidence intervals, and a sign-flip test. The reporting is honest about wide confidence intervals, including the baseline comparisons. The empirical gains are modest but real under the stated assumptions, and the paired-evaluation methodology is a useful contribution on its own.

Pith's reading of the paper (a machine's summary, not the authors')

The paper tries to establish that harness recovery in LLM agents is best framed as a causal decision rather than a fixed operation. Its core move is a paired counterfactual comparison: from the same execution state, the authors branch the run once with recovery and once without, and classify each trajectory as rescued, harmed, or unchanged. They use those paired outcomes to train CIR, a lightweight policy that refreshes only when predicted benefit outweighs predicted risk. The central empirical claim is that this selective policy beats both never-refreshing and a pure anomaly detector on held-out ALFWorld tasks, and that it does so without ever touching a trajectory whose observations were correct.

What carries the argument. The load-bearing device is the paired counterfactual comparison with a rescue–harm decomposition. From the same execution state, delay d, and observation condition e, one branch continues without recovery and the other applies the recovery operation m; because the branches share state, history, model, and decoding settings, the outcome difference τ_i^m(e,d) = Y_i^m(e,d) − $Y_i^{0}$(e) is attributed to the recovery decision. Outcomes are binned into rescue (failure→success), harm (success→failure), stable success, and stable failure, with the average effect equal to rescue rate minus harm rate. CIR then estimates three quantities from information available before intervention — the error probability p_err(x), success without recovery p_0(x), and success after refresh p_1(x) — and forms rescue score s_R(x) = (1−p_0(x))·p_1(x) and harm score s_H(x) = p_0(x)·(1−p_1(x)), refreshing at the first candidate delay where error, failure, and utility thresholds are all met. The separation of error detection from outcome prediction is what lets the policy leave clean trajectories alone while still recovering stale ones.

Core claim. The central discovery is that the value of a refresh operation is not fixed: it depends on the observation condition, the state reached after an error, and the time at which recovery is attempted. Under clean observations an immediate refresh harms success by 6.45 percentage points, while under two-step-stale observations it raises success by 7.53; the clean-to-stale difference stays positive at every tested delay. Given that evidence, the Causal Intervention Router (CIR) — which estimates the probability of error, of success without recovery, and of success after refresh, and combines them into rescue and harm scores — raises held-out success from 70.33% to 73.33% (+3.00 pp, 95% CI [0.67, 5.67], sign-flip p = 0.034), with 11 rescues and 2 harms across 300 episodes. It never refreshes any of the 75 clean episodes, and its largest gain (+9.33 pp) appears exactly where observations are most severely outdated. A separate mechanism control shows that the benefit cannot be explained solely by the newly returned observation: content-ablated refresh — which queries the environment but hides the returned content — shares 8 of 10 full-refresh rescues.

The premise it rests on. The paired counterfactual method assumes deterministic replay: the no-recovery and recovery branches, launched from the same execution state, are identical except for the recovery operation, so any outcome difference is attributable to that operation alone.

If the paper is right (Pith's inferences, not the authors')

Recovery should be evaluated per-trajectory in terms of rescue and harm, because the same average gain can hide very different mixes of rescues and harms.

Anomaly detection alone is not enough for recovery decisions: an unusual observation does not imply that a refresh will help, so a policy must also predict the intervention's outcome.

A selective policy can concentrate recovery where its benefit is largest — here, on two-step-stale observations — while preserving the never-refresh success rate on clean trajectories.

The content of the newly returned observation is not the sole driver of refresh's benefit, since content-ablated refresh shares 8 of 10 rescues, meaning the act of querying itself contributes value.

The same paired evidence used for causal evaluation can serve as supervision for training a deployment-time decision rule, connecting measurement to policy.

The paired design and its rescue–harm decomposition likely extend beyond a single refresh — to replanning, reflection, tool calls, or memory rewrites — wherever the harness can branch from a fixed state and ask whether an intervention changes the final outcome.

The hard requirement of deterministic replay means the method is most trustworthy in environments with reproducible state transitions; in stochastic environments the two branches would need to account for sampling noise before the rescue–harm counts can be read causally.

The tunable harm penalty λ and clean-trajectory limit ϵ together define an explicit risk budget for deployment, letting a practitioner trade recovery aggressiveness against the acceptable rate of damaging trajectories that would otherwise succeed.

The content-ablated result hints that part of refresh's value comes from re-synchronizing the agent's internal belief with the environment, a mechanism that a purely informational account of recovery would miss.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Large language model agents rely on external harnesses to pass information between the model and its environment and to recover from execution errors. Yet recovery is usually judged only by average task success. This hides an important tension. The same operation can rescue a failing trajectory or disrupt one that would otherwise succeed. We frame recovery as a causal decision problem. Starting from the same execution state, we compare what happens with and without recovery, separate rescue from harm, and study how the value of recovery changes over time. We then introduce the Causal Intervention Router (CIR), a lightweight policy that uses information available before recovery to decide when intervention is worthwhile. On long-horizon ALFWorld tasks with Qwen3-14B, CIR raises success from 70.33% to 73.33%, a gain of 3.00 percentage points. It leaves all evaluated trajectories with correct observations untouched. Additional controls show that the benefit of recovery cannot be explained solely by the new observation returned by the environment. These results provide a practical way to evaluate recovery and apply it selectively.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-02

The authors' main claim, as the classifier picked it out 0.95

“These results provide a practical way to evaluate recovery and apply it selectively.”

How the main claim is stated: asserted 0.98

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.23 · human participants 0.03 · controlled comparison 0.95 · randomized assignment 0.03 · states quantitative result 0.99 · compares with prior results 0.72 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Large language model agents rely on external harnesses to pass information between the model and its environment and to recover from execution errors. background 1.00 0.05 0.02 asserted 1.00
Yet recovery is usually judged only by average task success. gap 0.96 0.09 0.03 asserted 0.99
This hides an important tension. gap 0.85 0.09 0.03 asserted 0.94
The same operation can rescue a failing trajectory or disrupt one that would otherwise succeed. gap 0.56 0.14 0.06 asserted 0.88
We frame recovery as a causal decision problem. contribution 0.72 0.89 0.04 asserted 0.97
Starting from the same execution state, we compare what happens with and without recovery, separate rescue from harm, and study how the value of recovery changes over time. method 0.88 0.93 0.07 asserted 0.90
We then introduce the Causal Intervention Router (CIR), a lightweight policy that uses information available before recovery to decide when intervention is worthwhile. method 0.55 0.92 0.10 asserted 0.99
On long-horizon ALFWorld tasks with Qwen3-14B, CIR raises success from 70.33% to 73.33%, a gain of 3.00 percentage points. result 1.00 0.96 0.27 asserted 1.00
It leaves all evaluated trajectories with correct observations untouched. result 1.00 0.93 0.09 asserted 1.00
Additional controls show that the benefit of recovery cannot be explained solely by the new observation returned by the environment. result 0.88 0.93 0.10 asserted 1.00
These results provide a practical way to evaluate recovery and apply it selectively. interpretation 0.98 0.75 0.69 asserted 0.98

What kind of paper the classifier reports this is: method or tool 0.97 · evidence from observation or experiment 0.85

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-02: 1 request, 7,922 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (6)
Figure 1 of the paper
Figure 1. Overview of our framework. (A) We introduce controlled observation errors and try
Figure 2 of the paper
Figure 2. Paired recovery effects in the 93-task factual-success cohort across delays (a) and in the
Figure 3 of the paper
Figure 3. Mechanism controls under clean observations at delay 0 (a) and two-step stale observations
Figure 4 of the paper
Figure 4. Selective recovery on 75 held-out prefix-feasible tasks. Each task is evaluated under four
Figure 5 of the paper
Figure 5. Joint recovery outcomes under two-step stale observations at delay
Figure 6 of the paper
Figure 6. Paired outcomes at recovery delay d = 1 in the 93-task factual-success cohort. Each bar partitions trajectories into stable success, rescue, harm, and stable failure. 15

Figures are the authors', reproduced from the paper.

References (37)
  1. [1] Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards on Pith
  2. [2] Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume = on Pith
  3. [3] Deep learning , author=. 2016 , publisher= on Pith
  4. [4] Packer, Charles and Wooders, Sarah and Lin, Kevin and Fang, Vivian and Patil, Shishir G. and Stoica, Ion and Gonzalez, Joseph E. , note =. 2023 , eprint = on Pith
  5. [5] Wu, Qingyun and Bansal, Gagan and Zhang, Jieyu and Wu, Yiran and Li, Beibin and Zhu, Erkang and Jiang, Li and Zhang, Xiaoyun and Zhang, Shaokun and Liu, Jiale and Awadallah, Ahmed Hassan and White, Ryen W. and Burger, Doug and Wang, Chi , booktitle =. 2024 , url = on Pith
  6. [6] Yang, John and Jimenez, Carlos E. and Wettig, Alexander and Lieret, Kilian and Yao, Shunyu and Narasimhan, Karthik and Press, Ofir , booktitle =. 2024 , url = on Pith
  7. [7] Cemri, Mert and Pan, Melissa Z. and Yang, Shuyi and Agrawal, Lakshya A. and Chopra, Bhavya and Tiwari, Rishabh and Keutzer, Kurt and Parameswaran, Aditya and Klein, Dan and Ramchandran, Kannan and Zaharia, Matei A. and Gonzalez, Joseph E. and Stoica, Ion , booktitle =. Why Do ... on Pith
  8. [8] Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , booktitle =. 2023 , url = on Pith
  9. [9] Shah, Jaineet , note =. 2026 , eprint =. doi:10.48550/arXiv.2606.08275 , url = on PithPith's reviewarXivdoi:10.48550/arxiv.2606.08275
  10. [10] Holland, Paul W. , title =. Journal of the American Statistical Association , year =. doi:10.1080/01621459.1986.10478354 , url = on Pithdoi:10.1080/01621459.1986.10478354
  11. [11] Wu, Yiran and Yue, Tianwei and Zhang, Shaokun and Wang, Chi and Wu, Qingyun , booktitle =. 2024 , url = on Pith
  12. [12] Shinn, Noah and Cassano, Federico and Gopinath, Ashwin and Narasimhan, Karthik and Yao, Shunyu , booktitle =. 2023 , url = on Pith
  13. [13] Madaan, Aman and Tandon, Niket and Gupta, Prakhar and Hallinan, Skyler and Gao, Luyu and Wiegreffe, Sarah and Alon, Uri and Dziri, Nouha and Prabhumoye, Shrimai and Yang, Yiming and Gupta, Shashank and Majumder, Bodhisattwa Prasad and Hermann, Katherine and Welleck, Sean and Y... on Pith
  14. [14] Gou, Zhibin and Shao, Zhihong and Gong, Yeyun and Shen, Yelong and Yang, Yujiu and Duan, Nan and Chen, Weizhu , booktitle =. 2024 , url = on Pith
  15. [15] Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , url = on Pith
  16. [16] Large Language Models Cannot Self-Correct Reasoning Yet , author =. The Twelfth International Conference on Learning Representations , year = on Pith
  17. [17] Ma, Chang and Zhang, Junlei and Zhu, Zhihao and Yang, Cheng and Yang, Yujiu and Jin, Yaohui and Lan, Zhenzhong and Kong, Lingpeng and He, Junxian , booktitle =. 2024 , url = on Pith
  18. [18] Zhang, Shaokun and Yin, Ming and Zhang, Jieyu and Liu, Jiale and Han, Zhiguang and Zhang, Jingyang and Li, Beibin and Wang, Chi and Wang, Huazheng and Chen, Yiran and Wu, Qingyun , booktitle =. Which Agent Causes Task Failures and When? On Automated Failure Attribution of. 202... on Pith
  19. [19] Zhao, Yue and Li, Mengyuan and Li, Ruolin and Wang, Prince Zizhuang and Jiang, Shuli and Pang, Linsey and Xiao, Xiongye and Hu, Xiyang , note =. 2026 , eprint =. doi:10.48550/arXiv.2608.22808 , url = on PithPith's reviewarXivdoi:10.48550/arxiv.2608.22808
  20. [20] Bonagiri, Akash and Borkar, Devang and Anderias, Gerard Janno and Rafatirad, Setareh and Homayoun, Houman , note =. 2026 , eprint =. doi:10.48550/arXiv.2605.25338 , url = on PithPith's reviewarXivdoi:10.48550/arxiv.2605.25338
show 17 more
  1. [21] Chen, Mengzhuo and Wang, Junjie and Liu, Zhe and Wang, Yawen and Zheng, Haiming and Wang, Qing , note =. From Failed Trajectories to Reliable. 2026 , eprint =. doi:10.48550/arXiv.2606.06324 , url = on PithPith's reviewarXivdoi:10.48550/arxiv.2606.06324
  2. [22] Ma, Ming and Zhang, Jue and Yang, Fangkai and Kang, Yu and Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei , note =. 2025 , eprint =. doi:10.48550/arXiv.2512.06749 , url = on Pithdoi:10.48550/arxiv.2512.06749
  3. [23] Rubin, Donald B. , title =. Journal of Educational Psychology , year =. doi:10.1037/h0037350 , url = on Pithdoi:10.1037/h0037350
  4. [24] Kitagawa, Toru and Tetenov, Aleksey , title =. Econometrica , year =. doi:10.3982/ECTA13288 , url = on Pithdoi:10.3982/ecta13288
  5. [25] K. Metalearners for Estimating Heterogeneous Treatment Effects Using Machine Learning , journal =. 2019 , volume =. doi:10.1073/pnas.1804597116 , url = on Pithdoi:10.1073/pnas.1804597116
  6. [26] Tian, Jin and Pearl, Judea , title =. Annals of Mathematics and Artificial Intelligence , year =. doi:10.1023/A:1018912507879 , url = on Pithdoi:10.1023/a:1018912507879
  7. [27] Shridhar, Mohit and Yuan, Xingdi and C. International Conference on Learning Representations , year = on Pith
  8. [28] Yang, An and Li, Anfeng and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Gao, Chang and Huang, Chengen and Lv, Chenxu and Zheng, Chujie and Liu, Dayiheng and Zhou, Fan and Huang, Fei and Hu, Feng and Ge, Hao and Wei, Haoran and Lin, Huan an... on Pith
  9. [29] Efron, Bradley , title =. The Annals of Statistics , year =. doi:10.1214/aos/1176344552 , url = on Pithdoi:10.1214/aos/1176344552
  10. [30] McNemar, Quinn , title =. Psychometrika , year =. doi:10.1007/BF02295996 , url = on Pithdoi:10.1007/bf02295996
  11. [31] Fagerland, Morten W. and Lydersen, Stian and Laake, Petter , title =. BMC Medical Research Methodology , year =. doi:10.1186/1471-2288-13-91 , url = on Pithdoi:10.1186/1471-2288-13-91
  12. [32] Clopper, C. J. and Pearson, E. S. , title =. Biometrika , year =. doi:10.1093/biomet/26.4.404 , url = on Pithdoi:10.1093/biomet/26.4.404
  13. [33] Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows , author=. 2026 , eprint= on Pith
  14. [34] AgentRx: Diagnosing AI Agent Failures from Execution Trajectories , author=. 2026 , eprint= on Pith
  15. [35] LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks , author=. 2026 , eprint= on Pith
  16. [36] ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards , author=. Proceedings of the 43rd International Conference on Machine Learning , year= on Pith
  17. [37] When Errors Become Memories: Causal Pathway Tracing in Multi-Turn Memory-Augmented LLMs , author=. 2026 , eprint= on Pith
Cited by 3 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment