Pith. sign in

REVIEW 3 major objections 5 minor 32 references

Self-Critique-Guided Curiosity Refinement: Enhancing Honesty and Helpfulness in Large Language Models via In-Context Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding a self-critique pass to curiosity-driven prompting improves GPT-4o-judged honesty and helpfulness scores across all ten models tested, with relative gains from 1.4% to 4.3%.

desk verdict A useful benchmark and a plausible prompting extension, but the headline gains rest on a single judge that the refinement prompt is explicitly optimized against. read the letter →

arxiv 2506.16064 v1 pith:AN4QUOOU submitted 2025-06-19 cs.CL

Duc Hieu Ho , Chenglin Fan This is my paper · ORCID
classification cs.CL
keywords largelanguagemodelshonestyhelpfulnessself-critiquein-contextlearningpromptingHONESETLLM-as-a-judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that large language models become more honest and more helpful, without any fine-tuning, if after producing an answer they are prompted to critique themselves on specific honesty and helpfulness criteria and then make minimal edits in response to that critique. The authors evaluate this self-critique-guided curiosity refinement strategy on the HONESET benchmark across ten proprietary and open-weight models, using GPT-4o as a judge. Compared with the earlier curiosity-driven prompting approach, the added self-critique and refinement steps improve the mean H2 score for every model, with relative gains between 1.4% and 4.3%. The result matters because it points to a scalable, training-free way to make already-deployed models more trustworthy in settings where retuning or fine-tuning is impractical.

What carries the argument

The load-bearing mechanism is a five-step in-context prompting pipeline. Step 1 asks the query directly. Step 2, following curiosity-driven prompting, has the model state what is uncertain or missing. Step 3 produces an optimized answer. Step 4 is the new self-critique pass: a prompt asks the model to judge that answer on exactly the three dimensions the evaluation uses—explanation and honesty, guidance and helpfulness, solution appropriateness—giving justified scores from 1 to 10 and one key improvement suggestion. Step 5 is the refinement pass: a second prompt asks the model to make minimal targeted edits that fix the critique's flagged weaknesses while preserving everything else. The argument is that the critique makes weak points explicit and the refinement constraint prevents the model from destabilizing already-good content, so the output improves without retraining.

What would settle it

Take a random sample of HONESET responses from curiosity-driven prompting and from the proposed refinement, have human annotators score each response on the same three honesty–helpfulness dimensions, and compare the order of the two methods. If human scores show no reliably positive difference for the refined responses—or if a different judge model, e.g., one not among the ten evaluated models, reverses the ranking—then the paper's central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that a model's own critique can be turned into a concrete editing instruction that improves the honesty and helpfulness of its already-optimized answer. Starting from the curiosity-driven prompting pipeline—raw answer, followed by an exploration of uncertainties and external-resource gaps, followed by an optimized answer—the paper adds two in-context steps: the model scores its own optimized response on three dimensions (explanation and honesty, guidance and helpfulness, solution appropriateness) and writes an overall suggestion; then, using that critique, it produces a revised response with precise, minimal edits rather than a rewrite. On all ten models this second-stage refinement increased the share of excellent responses and reduced poor-quality responses, producing relative H2 gains from 1.4% (GPT-4o) to 4.3% (Llama 4 Maverick) over curiosity-driven prompting alone.

Load-bearing premise

The result stands on GPT-4o's judgments being a valid measure of honesty and helpfulness; since GPT-4o is also one of the ten models being judged, and the critique prompt asks models to improve on the exact dimensions the judge scores, the reported relative gains could partly reflect the judge preferring outputs that resemble its own refined style.

Editorial extensions

If this is right

  • Every one of the ten models tested—from GPT-4o to Llama 3 8B—scores higher on the GPT-4o-judged H2 metric after the self-critique and refinement steps than after curiosity-driven prompting alone.
  • The number of poor-quality responses (scores 1–3) falls in every model, and the number of excellent responses (7–10) rises, so the method does not just shift the mean through a few outliers.
  • The gains are achieved entirely through in-context learning at temperature 0, meaning no additional training data or parameter updates are needed.
  • Models with weaker raw-prompting baselines, such as Llama 4 Scout and Gemma 3 27B, show some of the largest relative improvements, suggesting the refinement step is especially helpful for open-weight models.
  • The paper frames the approach as a general trustworthiness lever, stating it could extend to other alignment dimensions such as harmlessness and fairness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the effect is driven by the critique making targeted weaknesses explicit, then substituting a strong external critic for self-critique should preserve or exceed the gains; that is a testable comparison the paper does not run.
  • Because GPT-4o is both the judge and one of the ten evaluated models, and the self-critique prompt asks the model to improve on the same three dimensions the judge scores, some of the reported gain for GPT-4o could reflect judge self-preference rather than a genuine change in response quality; re-scoring with a different judge or human raters would settle this.
  • The 1.4–4.3% range is for a single refinement pass; running the critique–refine loop multiple times might yield diminishing returns or drift, so the one-pass design is a practical choice but not an explored optimum.
  • Latency and inference cost roughly double with the two added steps, so in time-sensitive applications the benefit would need to be weighed against that cost—the paper notes this as a limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a training-free prompting strategy, self-critique-guided curiosity refinement, that extends Gao et al.'s curiosity-driven prompting with two additional in-context steps: a self-critique step in which the model scores its own optimized response on three dimensions (explanation & honesty, guidance & helpfulness, solution appropriateness) and a refinement step in which the model revises the response to address the critique. The authors benchmark ten proprietary and open-weight LLMs (GPT-4o, GPT-4o-mini, GPT-o3-mini, Gemini 2.0 Flash, Gemma 2/3, Llama 3 8B/70B, Llama 4 Scout/Maverick) on the 930-query HONESET, using GPT-4o as judge for both the purely honest rate and the H2 (honesty & helpfulness) score. They report that curiosity-driven prompting improves both metrics over raw prompting, and that the proposed refinement further improves H2 scores by 1.4% to 4.3% relative to curiosity-driven prompting across all ten models (Table 6).

Significance. If the reported gains are genuine, the paper offers a simple, training-free method to improve LLM honesty and helpfulness, with potential practical value for deployment. The paper's strengths include its systematic coverage of ten diverse models, fully specified prompt templates (Figures 3 and 4), reproducible inference settings (temperature 0, top-p 1, max tokens 2500), and internally consistent tables: spot-checks of the relative gains in Tables 3, 5, and 6 are arithmetically correct. However, the central claim rests on the validity of GPT-4o as a judge of H2. The evaluation is subject to a circularity concern: the self-critique and refinement prompts ask the model to optimize exactly the dimensions on which the GPT-4o judge scores, and GPT-4o is itself one of the evaluated models. No human validation of the H2 scale or alternate-judge cross-check is provided, and no confidence intervals or significance tests are reported. The contribution is therefore plausible but not yet established at the level required for the claims made.

major comments (3)
  1. [§4.3.2, §3.2 (Figures 3 and 4), Table 6] The H2 evaluation is the basis for the paper's central claim, but the judge, GPT-4o, is also one of the ten evaluated models (Section 4.2), and the self-critique prompt (Figure 3) and refinement prompt (Figure 4) explicitly instruct the model to improve its scores on the same three dimensions that the H2 judge uses (Explanation & honesty, Guidance & helpfulness, Solution appropriateness). The final responses are therefore optimized against the judge's rubric at inference time. The only validation cited (91.43% agreement, Section 4.3.1) is for GPT-4 as a purely honest judge, not for GPT-4o on the H2 composite scale. Without human evaluation or an independent judge, the reported 1.4% to 4.3% relative gains in Table 6 are plausibly inflated by rubric-conforming surface features (e.g., explicit disclaimers, structured action items, phrasing that mirrors the rubric) rather than by substantive improvements in honesty and helpfulness. The paper must provide a human-validated sample or an alternate-judge cross-check to support the central claim.
  2. [§4.5, Tables 3, 5, and 6] All comparison metrics are single-run point estimates with no confidence intervals, standard errors, or significance tests. Because the judge's scores may be sensitive to prompt wording, ordering, and any randomness in the judge's decoding, the assertion of 'consistent improvements across all models' is not statistically established. Please report bootstrap confidence intervals or paired tests (e.g., Wilcoxon signed-rank) over the 930 queries, and state the judge's sampling configuration.
  3. [§4.2, Table 6 (GPT-4o row)] For the GPT-4o row, the same model generates the response, critiques it, refines it, and judges the final result. This is the strongest case of judge self-preference, and the reported 1.4% relative gain for GPT-4o is particularly vulnerable to bias. The authors should either exclude the GPT-4o row from the central comparison or confirm the result with an external judge; as written, one of the ten 'consistent' improvements is essentially a self-assessment.
minor comments (5)
  1. [§4.5, Table 5 discussion] The text refers to 'Gemini 3 27B' but the corresponding table row is 'Gemma 3 27B'; please correct this typo.
  2. [§4.4, Table 1 discussion] The text refers to 'Gemina 2.0 Flash'; this should be 'Gemini 2.0 Flash'.
  3. [§4.3.1 and §4.3.2] The judge's own inference configuration (temperature, top-p, token limit, API version) is not stated. The text says the evaluation uses 'the same inference configuration settings' as the generation steps, but the judge is a separate component and needs its own specification for reproducibility.
  4. [§4.2] Please list the exact model versions and access dates (e.g., GPT-4o-2024-05-13 or similar) for all proprietary models; this is needed for reproducibility because API models change over time.
  5. [§5.4] The limitations section acknowledges increased latency and computational cost but does not mention the judge-dependence of the evaluation, which is the main threat to the paper's central claim; this omission should be addressed.

Circularity Check

1 steps flagged · score 6.0 of 10

H2 gains are partly by construction: the refinement prompt targets the same three dimensions that the GPT-4o judge scores, so Table 6 measures rubric optimization rather than independently validated honesty/helpfulness.

  1. self definitional [Section 3.2 (Figures 3 and 4), evaluated in Section 4.3.2 and Table 6]
    "The prompt instructs the model to evaluate its output based on the three core dimensions consisting of explanation and honesty, guidance and helpfulness, and solution appropriateness by the system prompt outlined in Figure 3. ... You are an expert editor tasked with making targeted improvements to an existing LLM’s response based on a specific critique with the primary goal of enhancing its score according to evaluation standards while preserving its strengths."

    The H2 judge in Section 4.3.2 scores responses on the H2 framework from Gao et al.: rationality of explanations for honesty/disclaimer, quality of further guidance, and potential solution (Section 2.2). Figure 3’s self-critique rubric is the same three dimensions ('Explanation & honesty', 'Guidance & helpfulness', 'Solution appropriateness'), and Figure 4 tells the model to revise 'with the primary goal of enhancing its score according to evaluation standards.' The refinement step is therefore an in-context optimizer for the exact rubric that the GPT-4o judge later uses to compute H2.

full rationale

The paper is not built on a self-citation chain: its main external dependency, the HONESET dataset and H2 framework, comes from Gao et al. and is openly cited, with no author overlap. There is also no fitted parameter renamed as a prediction; the method is a purely in-context prompting pipeline. The central circularity risk is definitional: the proposed self-critique and refinement prompts explicitly adopt the same three quality dimensions that the GPT-4o H2 judge uses, and the refinement prompt states that its primary goal is to enhance the response's score according to those evaluation standards. Asking a model to improve on a rubric and then reporting improvements on that same rubric is a partially constructed result: the H2 gain is unsurprising and does not by itself validate honesty or helpfulness beyond the rubric. Additional validity concerns, such as GPT-4o serving as both judge and one of the evaluated models and the absence of human re-validation of the judge for refined outputs, are correctness risks rather than circularity per se. Section 5.4 lists only latency/cost and missing harmlessness/fairness as limitations, so the judge-validity gap is unaddressed. Overall, the benchmarking of raw versus curiosity-driven prompting is independent, but the paper's headline 1.4–4.3% refinement gain reduces substantially to rubric optimization, warranting a partial circularity score.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new free parameters or invented entities. It relies on the HONESET dataset, the H2 framework, and the GPT-4o judge as background assumptions, plus the claim that the curiosity-driven prompts are faithfully reproduced. These are domain assumptions drawn from prior work without independent validation in this paper.

assumptions (3)
  • domain assumption The HONESET dataset validly measures honesty and helpfulness across its six categories.
    The paper relies on HONESET from Gao et al. (2024) as the ground-truth evaluation set without independent validation, as described in Section 4.1.
  • domain assumption The GPT-4o judge reliably evaluates honesty and helpfulness, with agreement close to the 91.43% reported for GPT-4.
    The paper cites Gao et al.'s GPT-4 human-agreement figure in Section 4.3.1 but does not re-validate the GPT-4o judge, which also evaluates GPT-4o's own outputs.
  • domain assumption The curiosity-driven prompting prompts and inference settings from Gao et al. are faithfully reproduced.
    The paper states this in Section 3.1 but does not include the actual prompts, so a reproduction cannot verify the match.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Critique-Guided Curiosity Refinement: Enhancing Honesty and Helpfulness in Large Language Models via In-Context Learning." pith.science (2026). https://pith.science/paper/AN4QUOOU

@misc{pith2026250616064,
  author       = {Pith},
  title        = {Pith review of: Self-Critique-Guided Curiosity Refinement: Enhancing Honesty and Helpfulness in Large Language Models via In-Context Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AN4QUOOU}},
  note         = {Machine review of arXiv:2506.16064}
}
abstract

Large language models (LLMs) have demonstrated robust capabilities across various natural language tasks. However, producing outputs that are consistently honest and helpful remains an open challenge. To overcome this challenge, this paper tackles the problem through two complementary directions. It conducts a comprehensive benchmark evaluation of ten widely used large language models, including both proprietary and open-weight models from OpenAI, Meta, and Google. In parallel, it proposes a novel prompting strategy, self-critique-guided curiosity refinement prompting. The key idea behind this strategy is enabling models to self-critique and refine their responses without additional training. The proposed method extends the curiosity-driven prompting strategy by incorporating two lightweight in-context steps including self-critique step and refinement step. The experiment results on the HONESET dataset evaluated using the framework $\mathrm{H}^2$ (honesty and helpfulness), which was executed with GPT-4o as a judge of honesty and helpfulness, show consistent improvements across all models. The approach reduces the number of poor-quality responses, increases high-quality responses, and achieves relative gains in $\mathrm{H}^2$ scores ranging from 1.4% to 4.3% compared to curiosity-driven prompting across evaluated models. These results highlight the effectiveness of structured self-refinement as a scalable and training-free strategy to improve the trustworthiness of LLMs outputs.

Figures

Figures reproduced from arXiv: 2506.16064 by the authors.

Figure 1
Figure 1. The overall pipeline of the curiosity-driven approach. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 5
Figure 5. Each bar illustrates the number of input [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 2
Figure 2. The overall pipeline of the self-critique-guided curiosity refinement approach. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: Prompt template for large language model to self-critique its responses. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]
Figure 4
Figure 4. Figure 4: Prompt template for large language model to refine its responses. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Distribution of query categories in the HONESET. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Visualization of purely honest rate between raw output and curiosity-driven prompting across ten [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 14 canonical work pages

  1. [1]

    Gpt-4 technical report,

    OpenAI, J. Achiam, S. Adler, et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2024. [Online]. Available: https://arxiv.org/abs/2303.08774

    arXiv 2024

  2. [2]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, et al., “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024. [Online]. Available: https://arxiv.org/abs/2407.21783. 18

    arXiv 2024

  3. [3]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, et al. , “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  4. [4]

    Aligning ai with shared human values,

    D. Hendrycks, C. Burns, S. Basart, et al., “Aligning ai with shared human values,” Pro- ceedings of the International Conference on Learning Representations (ICLR), 2021

    work page 2021

  5. [5]

    Hon- estllm: Toward an honest and helpful large language model,

    C. Gao, S. Wu, Y. Huang, et al. , “Hon- estllm: Toward an honest and helpful large language model,” in Advances in Neural In- formation Processing Systems, A. Globerson, L. Mackey, D. Belgrave, et al., Eds., vol. 37, Curran Associates, Inc., 2024, pp. 7213–7255. [Online]. Available: https : / / proceedings . neurips . cc / paper _ files / paper / 2024 / f...

    work page 2024

  6. [6]

    Ai align- ment: A comprehensive survey,

    J. Ji, T. Qiu, B. Chen, et al. , “Ai align- ment: A comprehensive survey,” arXiv preprint arXiv:2310.19852, 2023. [Online]. Available: https://arxiv.org/abs/2310.19852

    arXiv 2023

  7. [7]

    A general language assistant as a laboratory for align- ment,

    A. Askell, Y. Bai, A. Chen, et al., “A general language assistant as a laboratory for align- ment,” arXiv preprint arXiv:2112.00861, 2021. [Online]. Available: https://arxiv.org/abs/ 2112.00861

    arXiv 2021

  8. [8]

    A survey of hallucination in large foundation models,

    V. Rawte, A. Sheth, and A. Das, “A survey of hallucination in large foundation models,” arXiv preprint arXiv:2309.05922, 2023

    arXiv 2023

Show all 32 references
  1. [9]

    To- wards understanding sycophancy in language models,

    M. Sharma, M. Tong, T. Korbak, et al., “To- wards understanding sycophancy in language models,” in Proceedings of the 12th Inter- national Conference on Learning Representa- tions (ICLR), arXiv:2310.13548, 2024. [Online]. Available: https : / / arxiv . org / abs / 2310 . 13548

    2024 arXiv

  2. [10]

    Lan- guage models (mostly) know what they know,

    S. Kadavath, T. Conerly, A. Askell, et al., “Lan- guage models (mostly) know what they know,” arXiv preprint arXiv:2207.05221, 2022

    2022 arXiv

  3. [11]

    Emergent abilities of large language models,

    J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, et al., “Emergent abilities of large language models,” Transactions on Machine Learning Research, 2022. [Online]. Available: https://arxiv.org/abs/2206.07682

    2022 arXiv

  4. [12]

    I think, therefore i am: Benchmarking awareness of large language models using awarebench,

    Y. Li, Y. Huang, Y. Lin, S. Wu, Y. Wan, and L. Sun, “I think, therefore i am: Benchmarking awareness of large language models using awarebench,” arXiv preprint arXiv:2401.17882, 2024. [Online]. Available: https://arxiv.org/abs/2401.17882

    2024 arXiv

  5. [13]

    Trustllm: Trustworthiness in large language models,

    Y. Huang, L. Sun, H. Wang, et al., “Trustllm: Trustworthiness in large language models,” ver- sion v6, arXiv preprint arXiv:2401.05561, 2024. [Online]. Available: https://arxiv.org/abs/ 2401.05561

    2024 arXiv

  6. [14]

    When large language models contradict humans? large language models’ sycophantic behaviour,

    L. Ranaldi and G. Pucci, “When large language models contradict humans? large language models’ sycophantic behaviour,” arXiv preprint arXiv:2311.09410, 2024. [Online]. Available: https://arxiv.org/abs/2311.09410

    2024 arXiv

  7. [15]

    Simple synthetic data reduces sycophancy in large language models,

    J. Wei, D. Huang, Y. Lu, D. Zhou, and Q. V. Le, “Simple synthetic data reduces sycophancy in large language models,” arXiv preprint arXiv:2308.03958, 2024. [Online]. Available: https://arxiv.org/abs/2308.03958

    2024 arXiv

  8. [16]

    Aligning language models to explicitly handle ambigu- ity,

    H. J. Kim, Y. Kim, C. Park, et al., “Aligning language models to explicitly handle ambigu- ity,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Pro- cessing, Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds., Miami, Florida, USA: Associa- tion for ...

    2024

  9. [17]

    MM- LLMs: Recent advances in MultiModal large language models,

    D. Zhang, Y. Yu, J. Dong, et al. , “MM- LLMs: Recent advances in MultiModal large language models,” in Findings of the Associa- tion for Computational Linguistics: ACL 2024, L.-W. Ku, A. Martins, and V. Srikumar, Eds., Bangkok, Thailand: Association for Compu- tational Linguis...

    2024

  10. [18]

    Large language models are human-level prompt engi- neers,

    Y. Zhou, A. I. Muresanu, Z. Han, et al., “Large language models are human-level prompt engi- neers,” in International Conference on Learn- ing Representations (ICLR) , 2023. [Online]. Available: https : / / arxiv . org / abs / 2211 . 01910

    2023

  11. [19]

    Promptchainer: Chaining large language model prompts through visual programming,

    T. Wu, E. Jiang, A. Donsbach, et al. , “Promptchainer: Chaining large language model prompts through visual programming,” in CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA ’22), ser. LBW-022, New Orleans, LA, USA: ACM, 2022, pp. 1–7. doi: 10 . ...

    2022

  12. [20]

    Least-to- most prompting enables complex reasoning in large language models,

    D. Zhou, N. Sch¨ arli, L. Hou, et al., “Least-to- most prompting enables complex reasoning in large language models,” in International Con- ference on Learning Representations (ICLR), 2023

    2023

  13. [21]

    Lan- guage models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, et al., “Lan- guage models are few-shot learners,” in Ad- vances in Neural Information Processing Sys- tems, vol. 33, 2020, pp. 1877–1901

    2020

  14. [22]

    Rethinking the role of demonstra- tions: What makes in-context learning work?

    S. Min, M. Lewis, L. Zettlemoyer, and H. Ha- jishirzi, “Rethinking the role of demonstra- tions: What makes in-context learning work?” ”Proceedings of the 2022 Conference on Em- pirical Methods in Natural Language Process- ing, 2022. [Online]. Available: https://arxiv. org/abs...

    2022 arXiv

  15. [23]

    BERT: Pre-training of deep bidi- rectional transformers for language under- standing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidi- rectional transformers for language under- standing,” in Proceedings of the 2019 Confer- ence of the North American Chapter of the As- sociation for Computational Linguistics: Hu- man Language ...

    2019

  16. [24]

    A survey on in-context learning,

    Q. Dong, L. Li, D. Dai, et al., “A survey on in-context learning,” in Proceedings of the 2024 Conference on Empirical Methods in Nat- ural Language Processing, Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds., Miami, Florida, USA: Association for Computational Linguis- tics, Nov...

    2024

  17. [25]

    Let’s verify step by step,

    H. Lightman, V. Kosaraju, Y. Burda, et al., “Let’s verify step by step,” in Advances in Neu- ral Information Processing Systems (NeurIPS), vol. 36, 2023, pp. 38 963–38 979

    2023

  18. [26]

    Self-refine: Iterative refinement with self- feedback,

    A. Madaan, N. Tandon, P. Gupta, et al. , “Self-refine: Iterative refinement with self- feedback,” in Advances in Neural Informa- tion Processing Systems , A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds., vol. 36, Curran As- sociates, Inc., 2023, pp....

    2023

  19. [27]

    Reflexion: Language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,” in Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds., vol. 36, Curran Assoc...

    2023

  20. [28]

    Self-criticism: Aligning large language models with their un- derstanding of helpfulness, honesty, and harm- lessness,

    X. Tan, S. Shi, X. Qiu, et al., “Self-criticism: Aligning large language models with their un- derstanding of helpfulness, honesty, and harm- lessness,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, M. Wang and I. Zi...

    2023 doi

  21. [29]

    Con- stitutional ai: Harmlessness from ai feedback,

    Y. Bai, S. Kadavath, S. Kundu, et al., “Con- stitutional ai: Harmlessness from ai feedback,” arXiv preprint arXiv:2212.08073, 2022. [On- line]. Available: https : / / arxiv . org / abs / 2212.08073

    2022 arXiv

  22. [31]

    Openai o3-mini system card,

    OpenAI, “Openai o3-mini system card,” Ope- nAI, Tech. Rep., 2025. [Online]. Available: https://cdn.openai.com/o3-mini-system- card-feb10.pdf

    2025

  23. [32]

    Gemma: Open models based on gemini research and technology,

    G. Team, T. Mesnard, C. Hardin, et al. , “Gemma: Open models based on gemini research and technology,” arXiv preprint arXiv:2403.08295, 2024. [Online]. Available: https://arxiv.org/abs/2403.08295. 21

    2024 arXiv

  24. [2024]

    Available: https : / / arxiv

    [Online]. Available: https : / / arxiv . org/abs/2410.21276

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.