Artificial Intelligence · arXiv:2609.34603 v2 · submitted 28 Sep 2026 · 21 references
After the Fix: Transfer of Corrected Agent Experience
The paper says
“Does repairing an episode make its experience a better memory for the next task? We transfer the same failed source before and after accepted repair to a fixed target, alongside independent execution.”
more of the abstract
Our 3,300 runs cover 100 ThinkingBox pairs and the same 100 APEX pairs with and without source-state inheritance, under eleven conditions. ThinkingBox's Full/Skill/Hybrid correction gains are 44/29/32 percentage points, with corrected performance 25/22/18 points above independence; inference weakens at the task-family level. Yet 12 of Full's 15-point larger correction gap over Skill come from worse uncorrected performance, not better corrected memory. Moreover, 22 of Full's 46 upward transitions restore observed baseline success. Neither APEX regime establishes comparable aggregate correction benefits. Action evidence connects workflow gains with reusable obligations and convention conflicts with source-local choices. Text APEX's accepted execution reaches 52% versus its summary's 40%, without robust global/group-level superiority or an estab- lished advantage over independence. Smaller handoffs reduce input but increase calls. The value of repairing experience is therefore distinct from the value of reusing it: memory updates require both a previous-version reference and a fresh-start reference.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 30 Sep 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. Genuinely new pre/post-repair same-episode axis and a durable design principle, but headline magnitudes rest on one execution per cell and fail global adjustment. This line is wrong
3 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 21 references say.
The paper says, at B.1, Eq. (B.1); Sections 5.1–5.3 and abstract
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.89):
“Group-level inference addresses task dependence, not repeat-run reliability; repeti- tions and blinded artifact adjudication would distinguish persistent gains from execution and scoring variation.”
Pith says: Objection 1 of 3 · major
The single-execution-per-cell design makes every headline magnitude — the +44/+29/+32pp correction gains, the 25/22/18pp baseline-relative gains, the 22-of-46 restoration split, and the 12-of-15 decomposition — a point estimate from one trajectory per condition, and Eq. (B.1) shows explicitly that a single run cannot separate a condition's task-specific effect from execution variation.
more of objection 1
The paper acknowledges this at b217 ('Group-level inference addresses task dependence, not repeat-run reliability'), and Table 16 confirms the instability: B-only outcomes differ across regime executions for 17 of 100 targets. Given that the abstract and Section 5 state these magnitudes flatly, the load-bearing empirical claims need either repeated runs on the core ThinkingBox and APEX cells or the magnitudes must be visibly re-framed throughout (including the abstract) as single-run observations whose sampling distribution is unknown.
The paper says, at Section 5.2, Fig. 3, Table 11; abstract
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.76):
“Group-level inference addresses task dependence, not repeat-run reliability; repeti- tions and blinded artifact adjudication would distinguish persistent gains from execution and scoring variation.”
Pith says: Objection 2 of 3 · major
The claim that 22 of Full's 46 upward transitions 'restore observed baseline success' is a central empirical support for the paper's thesis that repairing experience is distinct from reusing it, but each target is classified n_101 (restoration) versus n_001 (new success) from a single B-only run.
more of objection 2
For targets whose per-target acceptance probability is near 0.5, Eq. (B.1) shows this label is close to a coin flip, so the 22-of-46 split could swing widely on repeated runs. The paper's own Table 16 shows exactly this fragility for B-only outcomes. The restoration/new-success distinction should be presented as a within-this-run observation, not as a measured property of the condition, or supported by repeated B-only runs on the relevant targets.
The paper says, at Section 5.3, Table 9
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The conclusion that Full's 15-point larger correction gap 'mostly reflects its weaker starting point' (12 of the 15 points from U-Full's worse performance) rests on the U-Full versus U-Skill contrast of 35 vs 23 targets, which does not survive even target-level family adjustment (Table 9: U-S–U-F target pH = 0.3782).
more of objection 3
The paper hedges only the 3-point C-Full advantage ('not established by the adjusted representation tests') but not the 12-point U leg. Since both legs are single-run point estimates and the U contrast is not statistically established, the interpretive claim that Full's apparent superiority is an artifact of a weaker predecessor should be stated as an arithmetic observation on this run's data, not as a robust finding.
4 minor objections
The paper says, at Abstract and Section B.2
Pith says: minor objection 1 of 4
The abstract says inference 'weakens at the task-family level,' while Section B.2 states that no primary contrast survives the group-level global-24 adjustment. The abstract should state the stronger fact directly: all headline correction effects fail to survive the combined group-level and family adjustment, even though the ThinkingBox target-level effects survive the global-24 target-level adjustment.
The paper says, at Section 2 and Appendix G
Pith says: minor objection 2 of 4
The distinction between this study and prior negative-transfer work (Feng et al. 2026; Li et al. 2026a) would be clearer if the paper explicitly noted that the two-reference evaluation (C–U and C–B-only) is its novel axis, and briefly stated whether any prior benchmark reported the restoration-vs-new-success decomposition on a same-episode pair.
show 2 more
The paper says, at Appendix F.3
Pith says: minor objection 3 of 4
The sensitivity layer that changes Text Execution–Summary from 52/40 with 14/2 discordances to 50/38 with 12/0 discordances is presented clearly, but the retention of 'first valid judgments' as the main result should be flagged in the main text's Section 6.1 discussion, where the +12pp figure is stated without reference to this audit.
The paper says, at Throughout
Pith says: minor objection 4 of 4
Several tables mix percentage-point gaps with raw counts in the same display (e.g., Fig. 2, Tables 7–8). A consistent notation — e.g., always placing 'pp' adjacent to gaps and leaving counts unlabeled — would reduce the risk of misreading the exploratory intervals as error bars on the counts.
What would settle it
Run three independent repetitions of ThinkingBox B-only, U-Full, and C-Full on the same 100 pairs with the frozen Appendix A settings (900 additional B-stage executions). From the 3×3 cross-repetition table, compute the spread of the C–U gap (observed +44pp), the C–B-only gap (+25pp), and the restoration count n_101 (observed 22 of Full's 46 upward transitions); also estimate per-target w_i = 2p_i(1−p_i) and evaluate how much of the observed 46/2 discordance Eq. B.1 attributes to execution noise at equal probabilities. If the gaps spread by more than roughly ±10pp across repetitions, or n_101 ranges outside about 13–31, the empirical magnitudes and the restoration/new-success split are single-draw artifacts; the design principle survives, but the empirical section should be downgraded to indicative pending repetition.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 30 Sep 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The new thing here is the axis, not the method. Comparing the same failed episode before and after accepted repair, on a fixed target, against both the uncorrected predecessor (C-U) and independent execution (C-B-only), is a real addition to the ExpeL/CLIN/AWM lineage. The restoration-versus-new-success decomposition (101 vs 001) is the useful part: it makes explicit that a correction gain can just reflect a weak predecessor, and that half of Full's upward transitions only recover a success the memory itself displaced. That distinction is worth keeping regardless of the magnitudes. The '12 of 15 points from worse U' point is also a good cautionary result. Credit where due: the paper is unusually honest. Section B.1 states plainly that one execution per cell cannot separate task effects from execution variation, b217 flags the repeat-run reliability issue, and the non-blind audits and cohort conditioning on repairability are all disclosed. The arithmetic checks out - I verified the 22-of-46 split and the 12-of-15 decomposition against Table 11. No fitted parameters, no circularity. The soft spots are real but proportionate. First, every headline quantity - +44/+29/+32pp, the 22/46 restoration split - is a point estimate from one trajectory per cell. For targets near p = 0.5, the B-only classification that determines whether a gain is 'restoration' or 'new success' is close to a coin flip, and Table 16 shows B-only outcomes shifting across executions. The 22-of-46 split could swing noticeably on the next run set. Second, none of the 24 primary contrasts survive the global-24 adjustment; the ThinkingBox effects hold at target-level family adjustment but not group-level. The paper reports this honestly, but it means the empirical case is conditional, not established. These are acknowledged limitations, and I am not pretending to have found them - but they are load-bearing for the magnitude claims. What survives all this is the principle: evaluate a memory revision against both its predecessor and a fresh start. That is a solid, citable design contribution that does not depend on the point estimates. The empirical magnitudes need repeated core-cell runs and blinded audits before they should be quoted as effects. This deserves a serious referee. It is new, transparent, and the methodological conclusion stands even if the empirical chapters need revision. I'd engage with it, and I'd cite the design principle in my own work. The reviewer's job is to insist on the repetition, not to reject the question.
The strongest objection
The paper's central claim has two parts: the design principle that C–U and C–B-only answer different questions and that both references are needed, and the empirical demonstration that ThinkingBox corrected memories beat both references (44/29/32pp correction, 25/22/18pp base) while APEX does not, with 22 of Full's 46 upward transitions restoring baseline success. The principle is logically secure; every empirical magnitude is a point estimate from one trajectory per cell (Section B.1; acknowledged at b217). The most exposed quantity is the restoration decomposition, which carries the abstract's 'distinct from the value of reusing it': each target is classified n_101 (restoration) or n_001 (new success) from a single B-only run, so for targets whose per-target success probability is near 0.5 the label is close to a coin flip. The 22-of-46 split could swing widely on the next set of runs, and with it the paper's headline interpretation. The same instability propagates into the aggregate gaps and into the '12 of 15' decomposition: the 12pp leg is U-Full (23) versus U-Skill (35), a contrast that does not survive family adjustment (Table 9: target pH = 0.3782). Eq. B.1 makes the mechanics explicit: Pr[Y(c)≠Y(c')] = (p_c−p_c')² + ½(w_c+w_c'), so a single run cannot separate task effects from execution variation. The paper's own Table 16 shows B-only observed outcomes differing across executions (37 pass in both regimes, 17 in exactly one), confirming single-run acceptance instability. Repetition is the one check that would tell whether the +44pp gap and the 22/46 split are persistent effects or single draws.
The reader's weakest-assumption pick (one execution per cell, Section B.1 and b217) is exactly the load-bearing concern: I agree with it, so the verdict is unchanged. I verified that the decomposition arithmetic is correct: 44−29 = (67−64)−(23−35) = 3+12, and the transition counts in Table 11 sum to the net gaps (46 gains − 2 losses = +44 for ThinkingBox Full). The statistical reporting is internally consistent and carefully hedged: Table 7 shows all six ThinkingBox primary contrasts surviving global-24 at target level, while Table 8's group-level sensitivity does not, and the paper flags this explicitly. I also credit the transparency: full transition tables, leave-one-cluster-out ranges, frozen outcome records, recomputed interval sensitivity, and a candid disagreement audit. The central methodological conclusion — memory revisions must be evaluated against both predecessor and fresh start — is a design principle that holds regardless of any single cell's outcome, so my concern does not overturn the paper's conceptual contribution. What is conditional is the empirical demonstration: the ThinkingBox magnitudes and, especially, the 22-of-46 restoration interpretation, which assigns each target a restoration/new-success label based on one B-only run whose per-target probability is unestimated. Judging non-blindness (acknowledged at b581) and repairability-conditioned cohort selection (disclosed in Appendix A) are secondary because they mainly affect the already-null APEX contrasts and the scope, not the executable-assertion ThinkingBox results that carry the positive claims. Hence CONDITIONAL stands: accept the design principle, treat the empirical magnitudes as conditional on a repeated-run check of the ThinkingBox core cells.
The referee's report
If the empirical magnitudes were reliable, the paper would make a useful methodological contribution: the recommendation to evaluate any agent memory revision against both the previous-version memory and independent execution is clear, actionable, and backed by a sound paired same-episode design that holds provenance fixed. The paper is unusually transparent — it ships frozen records, reproducible scripts, and explicit statements of its own limitations (e.g., b217, b400, b433–b435, b581), and its decomposition claims are exact arithmetic on observed counts rather than fitted quantities. However, every headline magnitude is a point estimate from one execution per cell, a fact the paper acknowledges but does not fully propagate into the abstract and conclusions, which present the single-run values as settled effects. The contribution is therefore a defensible evaluation principle with an empirical demonstration of limited inferential strength.
Pith's reading of the paper (a machine's summary, not the authors')
Most agent-memory systems assume that once a failed run is repaired and accepted, the repaired episode is the better experience to carry into the next task. This paper argues that repairing an episode and reusing it are different achievements, and that the repaired memory must be tested against two references at once: the uncorrected failure it replaces and an independent run of the next task with no source memory at all. In the ThinkingBox workflow benchmark the corrected full-history handoff beats the failed original by 44 percentage points, but twelve of the fifteen points of its advantage over the skill abstraction come from the failed full-history handoff being worse, not from the fix being better, and 22 of its 46 improvements only restore successes that an independent run already achieved. In the two APEX regimes the same comparisons show no reliable aggregate correction benefit, and text-Hybrid is directionally negative. The paper's conclusion is that a memory update should be judged by both a correction gain and a fresh-start gain, because a large correction gain alone can reflect a weak old memory rather than a superior repaired one.
What carries the argument. The load-bearing object is a paired three-branch design: the same failed episode is transferred to the same fixed target as an uncorrected record (U), as the first accepted repaired record (C), and not at all (B-only), with the same 100 source–target pairs serving every condition in each regime. The argument is carried by an outcome-pattern decomposition, $n_{buc}$, the number of targets whose B-only/U/C acceptance bits are $(b,u,c)$, which makes the net correction gain $N\,\widehat{\Delta}^{corr} = n_{001} + n_{101} - n_{010} - n_{110}$. That identity is what separates 'new success beyond both references' ($n_{001}$) from 'restoration of a success the old memory displaced' ($n_{101}$), and it is what lets the paper show that a large correction gain can be mostly a weak uncorrected starting point rather than a strong fix, converting an apparent 44-point win into a decision-relevant breakdown.
Core claim. The paper's central claim is that the value of repairing an episode and the value of reusing it are separate quantities, so an accepted repair should not automatically replace the failed memory it came from. The authors test this with a same-source, fixed-target design: for each of 100 source–target pairs in each of three regimes, the same failed source A is handed to the same new task B in an uncorrected state (U), after its first accepted repair (C), and not at all (B-only), holding provenance and target fixed. The evidence is that correction gains are easy to overstate in one regime and absent in another: in ThinkingBox, C beats U by 44/29/32 points for Full/Skill/Hybrid, yet twelve of Full's fifteen extra points over Skill come from U-Full's weaker starting point, and 22 of Full's 46 upward transitions only restore a B-only success that the uncorrected memory had displaced; in both APEX regimes no correction gain is established, with text-Hybrid at −8 points. The paper concludes that reporting only a correction gain conflates a weak predecessor with a strong fix, that neither the predecessor comparison nor the fresh-start comparison alone is enough, and that memory design should preserve the conditions under which a repair's advice was valid.
The premise it rests on. The load-bearing premise is that one execution of each condition on each target gives a trustworthy estimate of that condition's acceptance behavior; the paper acknowledges that its resampling accounts for task-composition uncertainty but not repeated-run reliability, so the headline numbers — the 44-point ThinkingBox gain, the 23-versus-35 starting gap behind Full's fifteen extra points, and the 22-of-46 restoration count — could shift if the same cells were run again.
If the paper is right (Pith's inferences, not the authors')
Any evaluation of an agent-memory revision should report both a correction gain (corrected memory versus the uncorrected version it replaces) and a fresh-start gain (versus independent execution), because a large correction gain alone can reflect a weak predecessor rather than a superior corrected memory, as when twelve of Full's fifteen extra points over Skill came from U-Full being worse, not C-
A corrected memory that beats its predecessor can still lose to a fresh start: 22 of Full's 46 upward transitions only restored a B-only success that the old memory had displaced, so adopting a repair should hinge on beating independence, not merely on beating the failure.
Target-level significance does not establish family-level generality: under group-level sign-flip sensitivity, no primary contrast survives global adjustment, so the ThinkingBox gains weaken when task families rather than individual targets are the unit of analysis.
The benefit does not generalize across regimes: neither APEX set establishes aggregate correction gains, and APEX-text's accepted-Execution advantage (52% versus 40% for Summary) is only five points above B-only's 47%.
Smaller handoffs trade one cost for another: Execution cut cumulative input by 12–32% while increasing model calls in every regime, so memory selection must weigh success, context reading, and downstream interaction demand jointly.
Editorial extension: the two-reference principle transfers to any system that swaps an old artifact for a repaired one — a fine-tuned model, a rewritten prompt, an updated skill library — where an apparent improvement over the old artifact should likewise be checked against a fresh baseline so that restorations are not counted as new gains.
Editorial extension: the outcome-pattern decomposition $n_{001}, n_{101}, n_{010}, n_{110}$ would be a cheap standard diagnostic for agent-memory benchmarks, letting readers separate newly unlocked successes from restored successes without re-running experiments.
Editorial extension: the paper's memory-design sketch — retain what changed, the requirement that justified the change, and the inputs it depended on — implies a concrete testable artifact: store each repair with an applicability predicate evaluated against the new request, and check whether the APEX convention-conflict cases (such as mid-year versus full-year discounting) stop failing.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
How Pith checks papers · The full audit of this paper · Review history (2 versions)
The paper
Abstract
Does repairing an episode make its experience a better memory for the next task? We transfer the same failed source before and after accepted repair to a fixed target, alongside independent execution. Our 3,300 runs cover 100 ThinkingBox pairs and the same 100 APEX pairs with and without source-state inheritance, under eleven conditions. ThinkingBox's Full/Skill/Hybrid correction gains are 44/29/32 percentage points, with corrected performance 25/22/18 points above independence; inference weakens at the task-family level. Yet 12 of Full's 15-point larger correction gap over Skill come from worse uncorrected performance, not better corrected memory. Moreover, 22 of Full's 46 upward transitions restore observed baseline success. Neither APEX regime establishes comparable aggregate correction benefits. Action evidence connects workflow gains with reusable obligations and convention conflicts with source-local choices. Text APEX's accepted execution reaches 52% versus its summary's 40%, without robust global/group-level superiority or an estab- lished advantage over independence. Smaller handoffs reduce input but increase calls. The value of repairing experience is therefore distinct from the value of reusing it: memory updates require both a previous-version reference and a fresh-start reference.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-29
The authors' main claim, as the classifier picked it out 1.00
“The value of repairing experience is therefore distinct from the value of reusing it: memory updates require both a previous-version reference and a fresh-start reference.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.03 · states limitation 0.91 · human participants 0.06 · controlled comparison 0.97 · randomized assignment 0.04 · states quantitative result 0.99 · compares with prior results 0.91 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Does repairing an episode make its experience a better memory for the next task? | background 0.56 | 0.07 | 0.03 | unclear 0.92 |
| We transfer the same failed source before and after accepted repair to a fixed target, alongside independent execution. | method 0.91 | 0.94 | 0.03 | asserted 0.77 |
| Our 3,300 runs cover 100 ThinkingBox pairs and the same 100 APEX pairs with and without source-state inheritance, under eleven conditions. | method 0.99 | 0.95 | 0.03 | asserted 0.97 |
| ThinkingBox's Full/Skill/Hybrid correction gains are 44/29/32 percentage points, with corrected performance 25/22/18 points above independence; inference weakens at the task-family level. | result 1.00 | 0.95 | 0.16 | asserted 1.00 |
| Yet 12 of Full's 15-point larger correction gap over Skill come from worse uncorrected performance, not better corrected memory. | result 0.74 | 0.94 | 0.09 | asserted 1.00 |
| Moreover, 22 of Full's 46 upward transitions restore observed baseline success. | result 0.97 | 0.94 | 0.05 | asserted 1.00 |
| Neither APEX regime establishes comparable aggregate correction benefits. | result 1.00 | 0.93 | 0.09 | asserted 1.00 |
| Action evidence connects workflow gains with reusable obligations and convention conflicts with source-local choices. | interpretation 0.91 | 0.77 | 0.06 | asserted 0.96 |
| Text APEX's accepted execution reaches 52% versus its summary's 40%, without robust global/group-level superiority or an estab- lished advantage over independence. | result 1.00 | 0.92 | 0.06 | asserted 0.99 |
| Smaller handoffs reduce input but increase calls. | result 0.93 | 0.72 | 0.03 | asserted 0.99 |
| The value of repairing experience is therefore distinct from the value of reusing it: memory updates require both a previous-version reference and a fresh-start reference. | interpretation 1.00 | 0.77 | 0.83 | asserted 1.00 |
What kind of paper the classifier reports this is: empirical study 0.93 · evidence from observation or experiment 0.72
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-29: 1 request, 8,081 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (6)
Figures are the authors', reproduced from the paper.
References (21)
- [1] Saleema Amershi, Dan Weld, Mihaela V orvoreanu, Adam Fourney, Besmira Nushi, Penny Collis- son, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. Guidelines for human-AI interaction. InProceedings of the 2019 CHI Conference o... on Pith
- [2] The script also recomputes Execution–Full resource intervals and five matched regime contrasts. Table 8: Primary contrasts under group-level inference. CI: freshly recomputed pointwise cluster interval (pp); pH uses the 9/9/6 families,p 24 all primary contrasts. Regime Hypothe... on Pith
- [4] URLhttps://arxiv.org/abs/2602.11988. Zexue He, Yu Wang, Churan Zhi, Yuanzhe Hu, Tzu-Ping Chen, Lang Yin, Ze Chen, Tong Arthur Wu, Siru Ouyang, Zihan Wang, Jiaxin Pei, Julian McAuley, Yejin Choi, and Alex Pentland. Mem- oryArena: Benchmarking agent memory in interdependent mult... on PithPith's reviewarXiv
- [7] Regime Comparison Clusters Minimum Maximum ThinkingBox C-F / U-F 15 +40.66 +48.42 ThinkingBox C-S / U-S 15 +25.27 +31.96 ThinkingBox C-H / U-H 15 +27.78 +34.74 ThinkingBox E / S 15 +8.64 +13.33 APEX-state C-F / U-F 16 -2.22 +1.08 APEX-state C-S / U-S 16 +1.05 +4.30 APEX-state ... on Pith
- [8] URLhttps://arxiv.org/abs/ 2508.21433. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics, 12:157–173, on PitharXiv
- [9] doi: 10.1162/tacl_a_00638. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter C... on Pithdoi:10.1162/tacl_a_00638
- [10] doi: 10.52202/075280-2019. Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, and Peter Clark. CLIN: A continually learning language agent for rapid task adaptation and generalization. InConference on ... on Pithdoi:10.52202/075280-2019
- [12] URLhttps://arxiv. org/abs/2603.25158. Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. ReasoningB... on PithPith's reviewarXiv
- [14] URLhttps://arxiv.org/ abs/2601.14242. Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 ofProceed- ings of Machine Learning Research, pp. 63897–63911, on PitharXiv
- [15] URLhttps://proceedings.iclr.cc/paper_files/paper/2026/hash/ 2e2960f2fe9e981f33f51c78656e3ca2-Abstract-Conference.html. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: LLM agents are experiential learners. InProceedings of the AAAI Confe... on Pith
- [16] doi: 10.1609/aaai.v38i17.29936. Xu Zou and Jie Tang. Trace as state: Reasoning traces as conditional states for long-context trans- formers.arXiv preprint arXiv:2609.02702, on PithPith's reviewarXiv
- [21] Accep- tance, partial credit, backend restriction, and action-level audit answer different questions and are reported separately. G EXTENDEDLITERATURECOMPARISON Correction in continuing human–AI collaboration.Amershi et al. (2019) distinguish making errors easy to correct, rem... on Pith
- [22] turns task feedback into re- flections for later attempts. Self-correction without external feedback need not improve reasoning (Huang et al., 2024), and early mistaken assumptions can persist across turns (Laban et al., 2025). We examine what happensafterA is accepted: how co... on Pith
- [23] evaluates memory- guided action across interdependent sessions; we additionally vary source correction. Micro- benchmarking examines ranking reliability under small gaps (Yauney et al., 2026). We decompose outcomes through acceptance, rubric satisfaction, and paired trajectori... on Pith
- [24] Strategies from success and fail- ure Memory; exploration via MaTTS Feng et al. (2026) Task- or subtask-induced skills Granularity×text/code SkillsBench (Li et al., 2026a) Curated or generated skills Skill source, domain, and utility This study The same source failure with and... on Pith
- [95] Eight sources remain unaccepted after ten feedback rounds and are replaced by eight unused natural pilot failures. The final cohort contains 87 formal-screen and 13 pilot sources, not a held-out cohort. Its accepted repairs occur at rounds one through four for 78/13/6/3 source... on Pith
- [100] A/Q denotes acceptance percentage / task-normalized rubric percentage. Condition ThinkingBox APEX-state A/Q APEX-text A/Q B-only 42 44 / 55.54 47 / 57.40 U-None 37 45 / 54.79 45 / 53.50 U-Full 23 42 / 54.00 45 / 55.83 U-Skill 35 39 / 50.89 44 / 54.29 U-Hybrid 28 41 / 53.11 48 ... on PitharXiv
- [2019] doi: 10.1145/3290605.3300233. Yiyang Feng, Biddut Sarker Bijoy, Niranjan Balasubramanian, and Jiawei Zhou. Break it down, pass it on: Cross-task skill transfer in LLM agents.arXiv preprint arXiv:2608.20274, on PitharXiv
- [2024] URLhttps://openreview.net/forum? id=IkmD3fKBPQ. Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. LLMLingua: Compressing prompts for accelerated inference of large language models. InProceedings of the 2023 Con- ference on Empirical Methods in Natural Langua... on Pith
- [2025] URLhttps://arxiv.org/abs/ 2505.06120. Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, Yifeng He, Shenghan Zheng, Kyoung Whan Choe, Jiankai Sun, Shuyi Wang, et al. SkillsBench: Benchmarking how well agent skills work across diverse tasks.arXiv preprint arXiv:2602.12... on PithPith's reviewarXiv
show 1 more
- [2026] URL https://arxiv.org/abs/2608.20274. Thibaud Gloaguen, Niels Mündler, Mark Müller, Veselin Raychev, and Martin Vechev. Evaluat- ing AGENTS.md: Are repository-level context files helpful for coding agents?arXiv preprint arXiv:2602.11988, on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment