Artificial Intelligence · arXiv:2609.08832 v1 · submitted 8 Sep 2026 · 50 references
Closing the Consistency Gap: Self-Evolving Agents That Learn to Stay on Course
The paper says
“Large language model (LLM)-powered agents can be accurate on average yet unreliable in production, a discrepancy that has been observed but remains largely unaddressed.”
more of the abstract
When given the same task five times, a ReAct agent on the AppWorld benchmark using GPT-4.1 succeeds in all five runs only 53% of the time, even though its per-run pass rate averages 77%. We call this 24-point shortfall the consistency gap, and we argue that addressing it is a precondition for trustworthy AI agent deployment. We present a self-evolving agent framework that reduces this gap by identifying unstable, low-consistency steps in agent trajectories and converting them into episodic memory the agent can draw on in future runs. At its core is a Consistency Analyzer that pinpoints where and why a trajectory is likely to flip across executions, and a Guideline Generator that converts the diagnosis into targeted guidelines, committed to memory and injected into future agent executions on similar tasks. On AppWorld with ReAct/GPT-4.1, our framework raises the fraction of tasks that succeed in all five runs by +16 points on same-task evaluation and +13 points on similar-task generalization.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 9 Sep 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. Solid empirical paper on an important problem, but the central claim—that targeting inconsistent steps drives gains—is not isolated; the results could come from any prompt augmentation. This line is wrong
3 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 50 references say.
The paper says, at §4.2, Tables 1–2, and §6
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The paper lacks a control condition that isolates the effect of the Consistency Analyzer's targeting. The evaluation compares baseline (no guidelines) against the full pipeline (guidelines from flagged steps). There is no ablation such as: (a) guidelines generated from randomly selected non-flagged steps, (b) generic guidelines unrelated to the trajectory, or (c) guidelines from high-consistency steps.
more of objection 1
Without such controls, the +16pp gain may reflect any additional prompt structure rather than specifically targeting flip-prone decisions. This issue is load-bearing because the contribution claims that the consistency signal guides generation; if a control matches the gains, the method reduces to 'add post-hoc instructions'. The paper acknowledges this gap in §6 (guideline validation) but does not test it, leaving the central causal claim unsubstantiated.
The paper says, at §3.1, §4.4
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The aggregate trajectory consistency achieves AUROC 0.699 as a pass/fail predictor, which is only modest. The paper claims this supports the generator's judgment when no ground truth is available (end of §4.4), but at this AUROC level, the signal is noisy; about 30% of classification decisions would be wrong.
more of objection 2
The paper does not demonstrate that using this predictor actually improves guideline quality over a baseline that simply treats all trajectories as failures. An ablation study (guidelines generated with vs. without the consistency predictor) would clarify whether the modest correlation is practically useful.
The paper says, at §3.1, §4.1
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The free parameters (resampling budget N=30, temperature 0.5, threshold θ_C=0.85, field importance weights δ_f) are set ad-hoc. The paper studies only the resampling budget (sensitivity study mentioned but not shown in the main text; it is in a brief note).
more of objection 3
There is no sensitivity analysis for θ_C or δ_f, nor any justification for the chosen values. Given that the step-consistency scores directly determine which steps are flagged for guideline generation, the results may be sensitive to these parameters. The authors should either provide evidence that results are stable over reasonable ranges or state how to set these parameters in practice.
6 minor objections
The paper says, at §1, Figure 1
Pith says: minor objection 1 of 6
Figure 1's pipeline diagram would benefit from explicit labels for the two loops (operational and offline analysis) mentioned in §3. The current figure is abstract and does not convey the self-evolving cycle clearly.
The paper says, at §2.1, Eq. (3)
Pith says: minor objection 2 of 6
The notation Pass^k (with caret) is nonstandard and resembles an exponent. Consider using 'PassAll_k' or 'Consistent_k' to avoid confusion with the standard Pass@k notation used in Eq. (1).
show 4 more
The paper says, at §3.1, Eqs. (4)–(6)
Pith says: minor objection 3 of 6
The description of Jaccard similarity for categorical responses (Eq. 5) should clarify whether the tokenization is word-level or subword-level, and whether it handles multi-token tool names consistently. A brief example would help reproducibility.
The paper says, at §4.2, Tables 1–2
Pith says: minor objection 4 of 6
Tables 1 and 2 in Appendix B are central to the main claims but are placed in the appendix. Consider moving them to the main text (§4.2) or at least summarizing the full per-difficulty breakdown prominently in the main text, as Figure 3 only shows normalized consistency, not the raw Pass^5 values.
The paper says, at §6
Pith says: minor objection 5 of 6
The limitation section correctly notes that token cost is multiplied ~30×. However, it does not estimate the total API cost for the 168-task benchmark. Adding a rough cost estimate would help practitioners assess practical deployability.
The paper says, at §3.2, Appendix A
Pith says: minor objection 6 of 6
The example guidelines in Appendix A are informative, but they appear to be manually selected. The paper should state whether these are actual generated outputs or illustrative.
What would settle it
Run the same AppWorld 168-task evaluation with ReAct/GPT-4.1 under three additional conditions: (A) guidelines generated from 30 randomly selected steps (not flagged by the analyzer); (B) generic guidelines generated from a zero-shot prompt 'Give general advice for AppWorld tasks' without any trajectory; (C) guidelines from high-consistency steps (C(s_i) > 0.95). Compare Pass^5 and Mean@5 to the paper's current same-task results. If conditions A/B/C match or exceed the reported +16pp Pass^5 gain, the consistency targeting is not the driver.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 9 Sep 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
You asked about 2609.08832. Here's my read.
The paper defines a new metric (the consistency gap: Mean@k − Passk) and shows it's large on AppWorld (24pp for GPT-4.1). That alone is useful for the field, because standard accuracy numbers overstate reliability. The framework—resampling-based consistency analysis plus guideline generation—is clean, black-box, and agent-agnostic. The empirical gains are real and decent: +16pp Pass5 same-task, +13pp similar-task generalization. The similar-task transfer is the strongest evidence that the method captures recurring patterns rather than overfitting.
But the stress-test concern is load-bearing: there is no ablation to isolate whether the gains come from targeting inconsistent steps or just from adding any post-hoc instructions. Without a control condition—e.g., guidelines from randomly selected steps or generic hints—the +16pp could be due to the agent receiving any helpful reminder, not specifically the consistency-driven ones. The paper's Section 6 acknowledges the lack of validation but doesn't test this. The AUROC of 0.699 for trajectory consistency as a pass/fail predictor is modest, leaving substantial unexplained variance. Free parameters (resampling budget, threshold) are set heuristically, though sensitivity is partially explored.
That said, the problem is important, the framing is clear, and the gains on similar tasks suggest something real is happening. I'd send this to peer review with a strong request for the ablation controls. If the authors can show that consistency-targeted guidelines significantly outperform generic ones, the paper becomes a solid contribution. Without that, it's an interesting but incomplete case study.
I would cite this if I were working on agent reliability, and I'd bring it to reading group because the definition and framing are worth discussing even if the causal claim needs tightening.
The strongest objection
The central claim is that the Consistency Analyzer's targeting of inconsistent steps drives Pass^5 improvement. However, the evaluation compares baseline (no guidelines) against a full pipeline (guidelines derived from flagged steps). There is no ablation: (1) guidelines generated from randomly selected (non-flagged) steps, (2) generic guidelines from the same LLM unrelated to the trajectory, or (3) guidelines from steps with high consistency scores. Without such controls, the +16pp gain may reflect any additional prompt structure—e.g., the agent being reminded to 'verify API parameters' regardless of the step's actual instability. The similar-task condition reduces overfitting concerns but does not isolate the targeting mechanism. The paper itself flags lack of validation (Section 6) but does not test whether the consistency signal is causally necessary. This is load-bearing because the entire contribution hinges on the analyzer guiding generation; if a generic guideline baseline matches or exceeds the reported gains, the methodology reduces to 'add post-hoc instructions,' which is not a novel contribution.
The reader's weakest assumption focused on the consistency estimator's accuracy and lack of guideline validation. I agree that validation is missing, but I identify a more fundamental design gap: the evaluation does not isolate the causal role of the consistency analyzer, leaving open the possibility that any prompt-level guidance—even from random steps—could yield similar gains. This is a distinct concern because it targets the attribution of improvement rather than the estimator's precision. The reader's AUROC concern (0.699) implies the analyzer is noisy, but my concern is that even a noisy analyzer's outputs might not be necessary for the observed effect. A control experiment with non-flagged or generic guidelines would disambiguate. I keep verdict UNCHANGED because the current evidence, while incomplete, is still suggestive and the proposed test is straightforward to run; the paper's limitations section acknowledges the broader validation issue, so no verdict change is warranted yet.
The referee's report
If the reported gains hold, the framework offers a practical, model-agnostic method to improve the run-to-run reliability of LLM agents—a key concern for production deployment. The consistency gap formalization (Pass^k vs. Mean@k) is a clear metric, and the black-box resampling approach avoids requiring model internals. The generalization to similar tasks (same-scenario siblings) suggests the guidelines capture reusable patterns rather than overfitting to task idiosyncrasies.
Pith's reading of the paper (a machine's summary, not the authors')
The paper identifies a 'consistency gap' in LLM-powered agents: the same task succeeds on average 77% of the time but in all five runs only 53% of the time, a gap that matters for production reliability. It proposes a self-evolving framework that detects unstable inference steps by resampling responses (30 times per step) and generates natural-language guidelines that are stored in memory and injected into future runs. On the AppWorld benchmark with a ReAct/GPT-4.1 agent, the framework lifts five-run consistency by 16 percentage points on the same task and 13 points on similar tasks, with mean accuracy actually rising slightly. The approach requires no access to model internals or agent code, only black-box API calls.
What carries the argument. The key machinery is a two-stage pipeline: (1) a black-box Consistency Analyzer that resamples each agent inference step N=30 times at temperature 0.5 and computes a step-level score via pairwise Jaccard similarity (for categorical actions) or cosine similarity over SBERT embeddings (for text/code), flagging steps with score below 0.85; (2) a Guideline Generator that converts flagged steps into natural-language consistency guidelines (strategy, recovery, optimization) and commits them to a dual-indexed episodic memory for retrieval by cosine similarity into future prompts.
Core claim. The paper's central claim is that run-to-run inconsistency in LLM agents has a measurable token-level fingerprint — flat response distributions at inference points — and that this inconsistency can be reduced by resampling each step offline, identifying flip-prone points, and generating prompt-level guidelines that stabilize those decisions. The key empirical discovery is that injecting such guidelines raises the fraction of tasks that pass all five runs ($Pass^{5}$) from 53% to 69% on AppWorld with GPT-4.1, with no degradation in mean pass rate (Mean@5 rises from 77.4% to 81.0%).
The premise it rests on. The step-level consistency score computed from 30 resamples at temperature 0.5 using Jaccard similarity over tokenized actions reliably identifies which decisions will flip in future runs under platform noise, but the reported AUROC of 0.699 shows substantial unexplained variance.
If the paper is right (Pith's inferences, not the authors')
Same-task repeat reliability (Pass^5) on AppWorld with GPT-4.1 rises by +16 points relative to baseline.
Guidelines generalize to similar tasks within the same scenario: Pass^5 rises by +13 points, only 3 points below same-task transfer.
The framework works with weaker models too: GPT-OSS-120B's aggregate Pass^5 rises by +6 points same-task and +8.7 points similar-task.
Aggregate trajectory consistency alone (AUROC 0.699) can serve as a proxy for trajectory pass/fail, enabling the generator to judge trajectory outcome without ground-truth ratings.
If the analyzer's step-level consistency prediction can be improved (AUROC 0.699 leaves room), the gap reduction might be larger. Adaptive resampling that focuses on candidate flat steps could cut the 30× cost.
The fact that guidelines generated by the same model that produced the trajectory still improve consistency suggests that self-judging is partially effective, but a validation stage (as noted in limitations) could prevent overfit or vague guidelines.
The framework's reliance on cosine-similarity retrieval means that tasks far from any stored experience may see little benefit; scaling to diverse task distributions likely requires dynamic retrieval strategies.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Large language model (LLM)-powered agents can be accurate on average yet unreliable in production, a discrepancy that has been observed but remains largely unaddressed. When given the same task five times, a ReAct agent on the AppWorld benchmark using GPT-4.1 succeeds in all five runs only 53% of the time, even though its per-run pass rate averages 77%. We call this 24-point shortfall the consistency gap, and we argue that addressing it is a precondition for trustworthy AI agent deployment. We present a self-evolving agent framework that reduces this gap by identifying unstable, low-consistency steps in agent trajectories and converting them into episodic memory the agent can draw on in future runs. At its core is a Consistency Analyzer that pinpoints where and why a trajectory is likely to flip across executions, and a Guideline Generator that converts the diagnosis into targeted guidelines, committed to memory and injected into future agent executions on similar tasks. On AppWorld with ReAct/GPT-4.1, our framework raises the fraction of tasks that succeed in all five runs by +16 points on same-task evaluation and +13 points on similar-task generalization.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-23
The authors' main claim, as the classifier picked it out 0.58
“On AppWorld with ReAct/GPT-4.1, our framework raises the fraction of tasks that succeed in all five runs by +16 points on same-task evaluation and +13 points on similar-task generalization.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.11 · human participants 0.04 · controlled comparison 0.75 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.87 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Large language model (LLM)-powered agents can be accurate on average yet unreliable in production, a discrepancy that has been observed but remains largely unaddressed. | background 0.92 | 0.08 | 0.04 | asserted 0.97 |
| When given the same task five times, a ReAct agent on the AppWorld benchmark using GPT-4.1 succeeds in all five runs only 53% of the time, even though its per-run pass rate averages 77%. | result 0.52 | 0.68 | 0.03 | asserted 1.00 |
| We call this 24-point shortfall the consistency gap, and we argue that addressing it is a precondition for trustworthy AI agent deployment. | gap 0.47 | 0.67 | 0.23 | asserted 0.99 |
| We present a self-evolving agent framework that reduces this gap by identifying unstable, low-consistency steps in agent trajectories and converting them into episodic memory the agent can draw on in future runs. | contribution 0.78 | 0.91 | 0.31 | asserted 1.00 |
| At its core is a Consistency Analyzer that pinpoints where and why a trajectory is likely to flip across executions, and a Guideline Generator that converts the diagnosis into targeted guidelines, committed to memory and injected into future agent executions on similar tasks. | method 0.98 | 0.84 | 0.07 | asserted 1.00 |
| On AppWorld with ReAct/GPT-4.1, our framework raises the fraction of tasks that succeed in all five runs by +16 points on same-task evaluation and +13 points on similar-task generalization. | result 1.00 | 0.96 | 0.54 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 5,157 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (3)
Figures are the authors', reproduced from the paper.
References (50)
- [1] altk-evolve: On-the-job learning for. 2026 , howpublished= on Pith
- [2] A Survey on the Memory Mechanism of Large Language Model based Agents , author=. ACM Transactions on Information Systems (TOIS) , year= on Pith
- [3] Du, Yiming and Huang, Wenyu and Zheng, Danna and Wang, Zhaowei and Montella, S. Rethinking Memory in. arXiv preprint arXiv:2505.00675 , year= on PitharXiv
- [4] Chhikara, Prateek and Khant, Dev and Aryan, Saket and Singh, Taranjeet and Yadav, Deshraj , journal=. Mem0: Building Production-Ready on Pith
- [5] Larimar: Large Language Models with Episodic Memory Control , author=. Proceedings of the International Conference on Machine Learning (ICML) , year= on Pith
- [6] Packer, Charles and Wooders, Sarah and Lin, Kevin and Fang, Vivian and Patil, Shishir G. and Stoica, Ion and Gonzalez, Joseph E. , journal= on Pith
- [7] Xu, Wujiang and Liang, Zujie and Mei, Kai and Gao, Hang and Tan, Juntao and Zhang, Yongfeng , journal= on Pith
- [8] Agent Workflow Memory , author=. arXiv preprint arXiv:2409.07429 , year= on PithPith's reviewarXiv
- [9] Anokhin, Petr and Semenov, Nikita and Sorokin, Artyom and Evseev, Dmitry and Kravchenko, Andrey and Burtsev, Mikhail and Burnaev, Evgeny , booktitle=. 2024 , note= on Pith
- [10] Feng, Erhu and Zhou, Wenbo and Liu, Zibin and Chen, Le and Dong, Yunpeng and Zhang, Cheng and Zhao, Yisheng and Du, Dong and Hua, Zhichao and Xia, Yubin and Chen, Haibo , journal=. Get Experience from Practice: on Pith
- [11] DeChant, Chad , journal=. Episodic Memory in on Pith
- [12] DeChant, Chad , booktitle=. On the Risks and Benefits of Episodic Memory in on Pith
- [13] Dunteman, Mike , year=. Muscle Memory: A Behavior Cache for on Pith
- [14] Xiong, Zidi and Lin, Yuping and Xie, Wenya and He, Pengfei and Tang, Jiliang and Lakkaraju, Himabindu and Xiang, Zhen , journal=. How Memory Management Impacts on Pith
- [15] Fang, Runnan and Liang, Yuan and Wang, Xiaobin and Wu, Jialong and Qiao, Shuofei and Xie, Pengjun and Huang, Fei and Chen, Huajun and Zhang, Ningyu , journal= on Pith
- [16] Zhou, Huichi and Chen, Yihang and Guo, Siyuan and Yan, Xue and Lee, Kin Hei and Wang, Zihan and Lee, Ka Yiu and Zhang, Guchun and Shao, Kun and Yang, Linyi and Wang, Jun , journal=. Memento: Fine-tuning on Pith
- [17] Ouyang, Siru and Yan, Jun and Hsu, I-Hung and Chen, Yanfei and Jiang, Ke and Wang, Zifeng and Han, Rujun and Le, Long T. and Daruki, Samira and Tang, Xiangru and Tirumalashetty, Vishy and Lee, George and Rofouei, Mahsan and Lin, Hangfei and Han, Jiawei and Lee, Chen-Yu and Pfi... on Pith
- [18] Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models , author=. arXiv preprint arXiv:2510.04618 , year= on PithPith's reviewarXiv
- [19] Chaudhury, Subhajit and Das, Payel and Swaminathan, Sarathkrishna and Kollias, Georgios and Nelson, Elliot and Pahwa, Khushbu and Pedapati, Tejaswini and Melnyk, Igor and Riemer, Matthew , booktitle=. 2025 , note= on Pith
- [20] Ye, Ye , journal=. Task Memory Engine ( on Pith
show 30 more
- [21] He, Zexue and Karlinsky, Leonid and Kim, Donghyun and McAuley, Julian and Krotov, Dmitry and Feris, Rogerio , journal= on Pith
- [22] Lei, Mingcong and Zhao, Yiming and Wang, Ge and Mai, Zhixin and Cui, Shuguang and Han, Yatong and Ren, Jinke , journal= on Pith
- [23] Modarressi, Ali and Imani, Ayyoob and Fayyaz, Mohsen and Sch. arXiv preprint arXiv:2404.11672 , year= on PithPith's reviewarXiv
- [24] Pink, Mathis and Wu, Qinyuan and Vo, Vy Ai and Turek, Javier and Mu, Jianing and Huth, Alexander and Toneva, Mariya , journal=. Position: Episodic Memory is the Missing Piece for Long-Term on Pith
- [25] Li, Zhiyu and Song, Shichao and Xi, Chenyang and Wang, Hanyu and Tang, Chen and Niu, Simin and Chen, Ding and Yang, Jiawei and Li, Chunyu and Yu, Qingchen and Zhao, Jihao and Wang, Yezhaohui and Liu, Peng and Lin, Zehao and Wang, Pengyuan and Huo, Jiahao and Chen, Tianyi and C... on Pith
- [26] Zhang, Zeyu and Bo, Xiaohe and Ma, Chen and Li, Rui and Chen, Xu and Wen, Ji-Rong , booktitle=. 2025 , note= on Pith
- [27] Qwen2.5 Technical Report , author=. arXiv preprint arXiv:2412.15115 , year= on PithPith's reviewarXiv
- [28] arXiv preprint arXiv:2508.10925 , year= on PithPith's reviewarXiv
- [29] 2025 , howpublished= on Pith
- [30] Towards Enterprise-Ready Computer Using Generalist Agent , author=. arXiv preprint arXiv:2503.01861 , year= on PithPith's reviewarXiv
- [31] Episodic Memories Generation and Evaluation Benchmark for Large Language Models , author=. Proceedings of ICLR , year= on Pith
- [32] Towards Large Language Models with Human-Like Episodic Memory , author=. Trends in Cognitive Sciences , volume=. 2025 , note= on Pith
- [33] Zhao, Yifan and others , journal=. 2025 , note= on Pith
- [34] Jha, Saurabh and Arora, Rohan and Bhavya and Zheutlin, Noah and Toro Isaza, Paulina and Shwartz, Laura and Deng, Yu and Sow, Daby and Mahindru, Ruchi and Puri, Ruchir , journal=. Think Locally, Explain Globally: Graph-Guided on Pith
- [35] Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems , author=. arXiv preprint arXiv:2503.06745 , year= on PithPith's reviewarXiv
- [36] Fournier, Fabiana and Limonad, Lior and David, Yuval , journal=. Agentic on Pith
- [37] Liu, Yibing and Zhang, Chong and Han, Zhongyi and Liu, Hansong and Wang, Yong and Yu, Yang and Wang, Xiaoyan and Yin, Yilong , journal= on Pith
- [38] Hutter, Robert and Pradel, Michael , journal= on Pith
- [39] Trajectory-Informed Memory Generation for Self-Improving Agent Systems , author=. arXiv preprint arXiv:2603.10600 , year= on PitharXiv
- [40] Reimers, Nils and Gurevych, Iryna , booktitle=. 2019 , publisher= on Pith
- [41] Trivedi, Harsh and Khot, Tushar and Hartmann, Mareike and Manku, Ruskin and Dong, Vinty and Li, Edward and Gupta, Shashank and Sabharwal, Ashish and Balasubramanian, Niranjan , booktitle= on Pith
- [42] Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , booktitle= on Pith
- [43] Self-Consistency Improves Chain of Thought Reasoning in Language Models , author=. Proceedings of the 11th International Conference on Learning Representations (ICLR) , year= on Pith
- [44] Snell, Charlie and Lee, Jaehoon and Xu, Kelvin and Kumar, Aviral , journal=. Scaling on Pith
- [45] Uncertainty Estimation in Autoregressive Structured Prediction , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year= on Pith
- [46] Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation , author=. Proceedings of the 11th International Conference on Learning Representations (ICLR) , year= on Pith
- [47] Detecting Hallucinations in Large Language Models Using Semantic Entropy , author=. Nature , volume= on Pith
- [48] Evaluating Large Language Models Trained on Code , author=. arXiv preprint arXiv:2107.03374 , year= on PithPith's reviewarXiv
- [49] On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification , author=. arXiv preprint arXiv:2608.18066 , year= on PithPith's reviewarXiv
- [50] Rabanser, Stephan and Kapoor, Sayash and Kirgis, Peter and Liu, Kangheng and Utpala, Saiteja and Narayanan, Arvind , journal=. Towards a Science of on Pith
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment