Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Introspection of Thought Helps AI Agents

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A code-integrated prompt that makes a single LLM internally play two debating agents improves reasoning across six benchmarks and lowers token cost.

desk verdict A plausible prompt-engineering idea with a load-bearing code bug and missing ablation; needs major revisions before I'd trust the claims. read the letter →

arxiv 2507.08664 v1 pith:Q73WN5I5 submitted 2025-07-11 cs.AI

Haoran Sun , Shaoning Zeng This is my paper
classification cs.AI
keywords IntrospectionofThoughtPromptCodeengineeringmulti-agentdebateself-reflectiontokenefficiencyzero-shotreasoningmultimodalquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Introspection of Thought (INoT) is a prompt-only framework that embeds a small LLM-readable programming language, PromptCode, in the prompt and asks the model to execute a two-agent debate entirely inside one context: initial reasoning, mutual critique, rebuttal, adjustment, and an agreement check. The paper claims this internalizes the iterative reflection that other agent frameworks perform through multiple external API calls, so the same task is done in fewer exchanges. Across six benchmarks spanning code generation, math, and question answering, INoT reports an average score of 81.3, surpassing seven baselines by 7.95% on average and beating the strongest baseline, Tree-of-Thought, by 6.41 points, while using on average 58.3% fewer tokens than the best-performing baseline. The framework also transfers to multimodal image QA, where an Image Augment module contributes further gains. The practical upshot is a zero-shot prompting route to better LLM and MLLM reasoning without retraining, at lower inference cost.

What carries the argument

The load-bearing object is PromptCode, an LLM-oriented hybrid of Python and natural language embedded in an XML-structured prompt. The prompt defines a role, rules, an optional Image Augment module, and a Reasoning Logic code block that instructs the LLM to instantiate Agent_A and Agent_B, run up to 10 rounds of argument, critique, rebuttal, and adjustment, and check whether both agents' final results agree before outputting the answer. The machinery's work is to compress an external agent-debate loop into instructions executed inside one model context, which the paper argues both raises answer quality and reduces token consumption because the model does not repeatedly send and receive full prompts.

What would settle it

A decisive check is to replace the Reasoning Logic code in the INoT prompt with an equivalent natural-language description of the same two-agent debate, holding output format and round count fixed; if accuracy or token cost stays the same, the claim that code-form PromptCode execution drives the gains is false. A second check is to inspect the model's outputs for explicit critique and rebuttal turns and for a genuine agreement comparison before the final answer, since the paper does not report transcript-level compliance.

Watch

Extended reading notes

Core claim

The paper's central claim is that an LLM can follow pseudo-code written in PromptCode to simulate a virtual multi-agent debate in a single context, and that this internal simulation is what improves accuracy and cuts cost. The debate loop in the Reasoning Logic code has the model produce independent initial answers from two agents, then cycle through critiques, rebuttals, and adjustments up to ten rounds until both agents converge on the same result, outputting only the final answer without explanation. The paper reports average gains of 7.95% over seven prompting baselines across HumanEval, MBPP, MATH, GSM8K, HotpotQA, and SQuAD, with the largest jumps on code tasks (e.g., pass@1 of 95.9 on HumanEval versus 90.6 for the best baseline), and 58.3% lower token cost than the best baseline. It further reports that the framework is model-agnostic in the tested range, with performance spread under 5% across six LLMs, and that it improves three image-QA benchmarks when combined with the Image Augment module. The intended consequence is that reflection and self-denial happen inside the model during one pass, rather than through external loops or multi-agent orchestration.

Load-bearing premise

The method assumes the LLM actually obeys the PromptCode and genuinely plays two independent debating agents inside one context, so that the measured gains come from that emulation rather than from the prompt being longer or from generic self-checking instructions; the paper's own ablation experiment intended to test this is referenced with a missing figure, so this premise is not yet directly evidenced.

Editorial extensions

If this is right

  • Iterative reasoning frameworks need not make multiple external API calls: a single prompt can encode a full debate loop, so agent builders can cut per-task token spend substantially.
  • Prompt-based internal reflection transfers across domains: the same INoT prompt, with the Image Augment module switched on only when images are present, improves math, code, QA, and multimodal QA results.
  • The approach is model-agnostic in the reported experiments: six different LLMs all stay within five points on the same benchmarks, suggesting the benefit is not tied to one proprietary model.
  • Because the final response is produced without explanation after the internal debate, downstream parsing of agent outputs becomes simpler and shorter, reinforcing the cost reduction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: a natural next test is whether the code form itself matters; a structurally identical prompt written in YAML, JSON, or natural-language numbered steps might yield the same gains, which would mean the mechanism is structured self-debate rather than PromptCode syntax.
  • Editorial: the paper does not measure how often the model actually produces divergent Agent_A and Agent_B answers, so transcript-level analysis could show whether convergence happens through genuine rebuttal or through the model trivially agreeing with itself.
  • Editorial: the 58.3% token-cost figure compares only against the best-performing baseline, not against all baselines; reporting accuracy per token or cost to reach a target score would give a fuller resource comparison.
  • Editorial: varying MaxRounds and the number of virtual agents could reveal an optimal debate budget, something the fixed 10-round design leaves unexplored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper introduces Introspection of Thought (INoT), a zero-shot prompting framework that places an LLM-readable pseudo-code language (PromptCode) inside an XML-structured prompt. The prompt instructs the model to simulate two virtual debate agents, Agent_A and Agent_B, that independently reason, critique each other, rebut, adjust, and check agreement within a single context. The authors evaluate INoT on six text benchmarks (HumanEval, MBPP, MATH, GSM8K, HotpotQA, SQuAD) using seven baselines and six LLMs, and on three image-QA benchmarks using three MLLM backbones. They report an average performance improvement of 7.95% over the baselines and a 58.3% token-cost reduction relative to the best baseline, and they attribute these gains to the internalized debate mechanism enabled by PromptCode.

Significance. INoT is a potentially valuable contribution to inference-time prompting: if the reported gains are reproducible, the method offers a zero-shot, post-training-free way to improve LLM/MLLM reasoning while reducing external multi-agent iteration overhead. The PromptCode idea is concrete and testable, and the evaluation across six LLMs and three MLLMs gives the empirical claims some generality. The paper also makes a falsifiable prediction about cost efficiency. However, the current manuscript does not yet establish the mechanism: the central ablation is missing, the code in Listing 3 contradicts the claimed early-exit behavior, and the cost claim rests on one dataset. The performance numbers are also insufficiently supported by statistical detail. These issues are correctable within the scope of a revision.

major comments (4)
  1. [§3.3, Listing 3] The debate loop in Listing 3 is written as `While not agreement or Counter < MaxRounds`, with MaxRounds=10 and Counter initialized to 0. Under standard control-flow semantics this loop cannot exit when agreement is reached before Counter reaches 10 (because `Counter < MaxRounds` stays true), and it does not stop at the iteration limit when agreement is false (because `not agreement` is true). This contradicts the accompanying text, which says the loop ends 'until either an agreement is reached or the iteration limit is met.' Since the paper attributes both the performance gain and the 58.3% token-cost reduction to internal convergence of the debate, this inconsistency is load-bearing: if the model follows the code literally it should run 10 rounds or more, making the low reported cost implausible; if the model does not follow the code, the mechanism is not actually tested. Please correct the PromptCode (for example, `while not agreement and Counter < MaxRounds`) or clarify that PromptCode is not executed literally, and report measured round counts and token costs on all benchmarks.
  2. [§4.2, Ablation Study] The ablation study text says 'As shown in Figure ??' and then asserts that removing the PromptCode lowers performance and that 'the impact of the execution of the PromptCode logic is greater than that of the design of PromptCode and PromptComplier.' No figure is present in the manuscript, and no numeric results for this ablation are reported anywhere. This is the only experiment that directly tests whether the code-driven debate mechanism, rather than the longer prompt or other confounds, is responsible for the gains. The missing evidence leaves the central causal claim unverified. Please add the full ablation results (at least the removed-code condition on all six text benchmarks) or remove the unsupported quantitative statement.
  3. [§4.2, Cost Analysis; Figure 3] The 58.3% token-cost reduction appears to be computed from Figure 3 alone, whose caption states that the comparison is on the HumanEval test set only. The abstract and conclusion, however, present the reduction as a general property of INoT. In addition, the paper does not specify how token cost is measured (input, output, or total; whether the long PromptCode is counted; number of API calls), which matters because INoT's prompt is substantially longer than the baselines' prompts. Please report cost on all six text benchmarks (and ideally on the image benchmarks), state the exact measurement procedure, and qualify the abstract and conclusion accordingly.
  4. [Table 1 and §4.2, Comparison to Baselines] Several reported numbers are internally inconsistent or not supported by the table. The text says INoT surpasses the best baseline ToT by '6.41 percentage points,' but Table 1 shows 81.3 - 76.4 = 4.9 points. The text reports an SQuAD F1 of 87.8 for INoT, while Table 1 lists 88.8. The abstract reports a 7.95% average improvement, while the contribution bullet in §1 reports 11.6%, without defining which baseline set each refers to. Finally, the paper states that all final results are averages of five experiments, but it reports no variance, error bars, or significance tests; without these, it is impossible to tell whether the observed gaps are meaningful or within noise. Please correct the numbers, define the averaging procedure precisely, and add variance or statistical testing, or justify why it is omitted.
minor comments (5)
  1. [§4.1, Baselines] The baseline list says 'Logical Cain-of-Thought,' which should be 'Logical Chain-of-Thought' (LogiCoT).
  2. [References] Reference [1] cites 'Claude-3.2-sonnet,' while the text consistently uses 'Claude-3.5-sonnet'; please align the reference with the model actually used.
  3. [§4.2, Ablation Study] The term 'PromptComplier' appears without definition; if this refers to a component that compiles PromptCode, it should be named and described, and the spelling should be corrected to 'PromptCompiler.'
  4. [Figure 3; §4.2] There are unresolved 'Figure ??' references in the Introduction and in the Ablation subsection, and the Figure 3 caption uses abbreviations CS, LC, GI, AI that are not introduced in the body text; please fix the references and define the abbreviations at first use.
  5. [Table 3] The column headers 'LLaV A qa' and 'LLaV Aw' are cryptic; spell them out as 'LLaVA-Bench (COCO)' and 'LLaVA-Bench (In-the-Wild)' in the header or in a footnote.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; INoT is an empirical prompting evaluation against external benchmarks with no fitted constants, no self-derived predictions, and no load-bearing self-citations.

full rationale

The paper derives no formal result: INoT is a prompt template (PromptCode) applied zero-shot to six text benchmarks and three image QA benchmarks. The headline numbers (7.95% average improvement and 58.3% token-cost reduction) are computed from tabulated experimental measurements against external baselines (Table 1, Figure 3); no parameter is fitted to a subset and then renamed a prediction. The mechanism (intra-LLM debate) is an asserted interpretation of prompt design, not a mathematical prediction from assumptions that contain the conclusion. There are no self-citations: baselines and datasets are external (CoT, ToT, IoT, ProgCo, etc.), so no uniqueness theorem or prior work by the same authors is invoked to force the outcome. The manuscript has separate evidentiary problems that do not constitute circularity: the ablation supporting 'execution of PromptCode logic' is missing ('As shown in Figure ??', Section 4.2), and the cost analysis is presented on one dataset (Figure 3 caption: 'on the HumanEval dataset test set') while the abstract reports an average cost claim. Additionally, Listing 3's loop condition ('While not agreement or Counter < MaxRounds') cannot terminate early on agreement under Python semantics, so the described early-exit mechanism is internally inconsistent. These are correctness and evidence risks, not circular reductions: the performance and cost claims are not defined in terms of each other or fitted from the data they predict. Therefore, per the hard rules, no circular step can be quoted with an exhibited reduction, and the score is 0.

Assumptions & free parameters 1 free parameters · 5 assumptions · 2 invented entities

The method adds a hand-designed prompt language and virtual agents, but no fitted constants are used to construct the claim. The main load-bearing assumptions concern whether LLMs actually follow the pseudo-code and whether the token-cost comparison generalizes beyond one benchmark.

free parameters (1)
  • MaxRounds = 10
    Maximum debate rounds in the Reasoning Logic code; hand-chosen, not tuned or justified.
assumptions (5)
  • domain assumption The LLM will follow PromptCode instructions literally and will simulate two independent debating agents inside one context.
    Section 3.3 and Listing 3 assume sequential Agent_A/Agent_B reasoning, critique, rebuttal, and adjustment purely from prompt text; there is no mechanism or measurement of compliance or independence.
  • domain assumption Internal self-critique and rebuttal within a single decoding improve answer accuracy.
    The empirical gains in Table 1 are the only support; no causal decomposition or ablation numbers are provided, and the Ablation Study references a missing figure.
  • domain assumption One internal debate pass consumes fewer tokens than external iterative frameworks.
    The cost claim in Section 4.2 is based on Figure 3, which is captioned as HumanEval only; the inference that token savings generalize to all six benchmarks is assumed.
  • domain assumption XML-structured prompts improve LLM parsing compared with natural-language frameworks.
    Section 3 states XML is intuitive and easy to parse without citation or controlled experiment.
  • domain assumption Python-like code in prompts yields stricter instruction following than natural language.
    Section 3.1 asserts this; no evidence isolates the code format from the content of the reasoning instructions.
invented entities (2)
  • PromptCode language
    purpose: A pseudo-code notation embedded in the prompt that the LLM is asked to 'execute' to structure reasoning.
    Defined only in Listing 1 and Listing 3; has no formal grammar, compiler, or external semantics; its behavior is entirely the LLM's interpretation.
  • Agent_A and Agent_B virtual debate agents
    purpose: Role-play personas within a single LLM context that critique and rebut each other's answers.
    No separate models or processes exist; 'independence' is asserted by prompt instruction only, with no measurement of whether two distinct perspectives actually emerge.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Introspection of Thought Helps AI Agents." pith.science (2026). https://pith.science/paper/Q73WN5I5

@misc{pith2026250708664,
  author       = {Pith},
  title        = {Pith review of: Introspection of Thought Helps AI Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q73WN5I5}},
  note         = {Machine review of arXiv:2507.08664}
}
read the original abstract

AI Agents rely on Large Language Models (LLMs) and Multimodal-LLMs (MLLMs) to perform interpretation and inference in text and image tasks without post-training, where LLMs and MLLMs play the most critical role and determine the initial ability and limitations of AI Agents. Usually, AI Agents utilize sophisticated prompt engineering and external reasoning framework to obtain a promising interaction with LLMs, e.g., Chain-of-Thought, Iteration of Thought and Image-of-Thought. However, they are still constrained by the inherent limitations of LLM in understanding natural language, and the iterative reasoning process will generate a large amount of inference cost. To this end, we propose a novel AI Agent Reasoning Framework with Introspection of Thought (INoT) by designing a new LLM-Read code in prompt. It enables LLM to execute programmatic dialogue reasoning processes following the code in prompt. Therefore, self-denial and reflection occur within LLM instead of outside LLM, which can reduce token cost effectively. Through our experiments on six benchmarks for three different tasks, the effectiveness of INoT is verified, with an average improvement of 7.95\% in performance, exceeding the baselines. Furthermore, the token cost of INoT is lower on average than the best performing method at baseline by 58.3\%. In addition, we demonstrate the versatility of INoT in image interpretation and inference through verification experiments.

Figures

Figures reproduced from arXiv: 2507.08664 by the authors.

Figure 1
Figure 1. We use PromptCode to write reasoning logic code, and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. An Uncertainty-Driven Adaptive Self-Alignment Framework for Large Language Models

    cs.AI 2025-07 conditional novelty 5.0 of 10

    A training pipeline that scores a model's own responses for semantic, factual, and safety uncertainty, builds preference pairs from those scores, and trains in three difficulty stages improves reported alignment score...

  2. A Novel Self-Evolution Framework for Large Language Models

    cs.CL 2025-07 reject novelty 4.0 of 10

    A dual-phase framework that uses a Censor satisfaction scorer to expand training data and then applies SFT plus frequency-weighted DPO, reporting benchmark gains over SFT, PO, and memory baselines.

Reference graph

Works this paper leans on

39 extracted references · 15 canonical work pages · cited by 2 Pith papers

  1. [1]

    Claude-3.2-sonnet

    Anthropic. Claude-3.2-sonnet. 2024. URL https://www.anthropic.com/ news/claude-3-5-sonnet

    work page 2024

  2. [2]

    Austin, A

    J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, and C. Sutton. Program synthesis with large language models, 2021. URL https://arxiv.org/abs/2108.07732

    arXiv 2021

  3. [3]

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Ka- plan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. H...

    arXiv 2021

  4. [4]

    W. Chen, X. Ma, X. Wang, and W. W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks, 2023. URL https://arxiv.org/abs/2211.12588

    arXiv 2023

  5. [5]

    Cobbe, V

    K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schul- man. Training verifiers to solve math word problems, 2021. URL https://arxiv.org/abs/2110.14168

    arXiv 2021

  6. [6]

    DeepSeek-V2.5

    DeepSeek. DeepSeek-V2.5. 2024. URL https://huggingface.co/ deepseek-ai/DeepSeek-V2.5

    work page 2024

  7. [7]

    N. Ding, Y . Qin, G. Yang, F. Wei, Z. Yang, Y . Su, S. Hu, Y . Chen, C.-M. Chan, W. Chen, et al. Parameter-efficient fine-tuning of large-scale pre- trained language models. Nature Machine Intelligence, 5(3):220–235, 2023

    2023

  8. [8]

    Y . Dong, X. Jiang, Z. Jin, and G. Li. Self-collaboration code generation via chatgpt. ACM Transactions on Software Engineering and Method- ology, 33(7):1–38, 2024

    work page 2024

Show all 39 references
  1. [9]

    T. M. Gemma Team, C. Hardin, R. Dadashi, S. Bhupatiraju, L. Sifre, M. Rivière, M. S. Kale, J. Love, P. Tafti, L. Hussenot, and et al. Gemma

  2. [10]

    Hendrycks, C

    D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. URL https://arxiv.org/abs/2103.03874

    2021 arXiv

  3. [11]

    W. Hu, H. Liu, L. Chen, F. Zhou, C. Xiao, Q. Yang, and C. Zhang. Socratic questioning: Learn to self-guide multimodal reasoning in the wild, 2025. URL https://arxiv.org/abs/2501.02964

    2025 arXiv

  4. [12]

    Huang, W

    X. Huang, W. Liu, X. Chen, X. Wang, H. Wang, D. Lian, Y . Wang, R. Tang, and E. Chen. Understanding the planning of llm agents: A survey. arXiv preprint arXiv:2402.02716, 2024

    2024 arXiv

  5. [13]

    Latif and X

    E. Latif and X. Zhai. Fine-tuning chatgpt for automatic scoring. Com- puters and Education: Artificial Intelligence, 6:100210, 2024

    2024

  6. [14]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning, 2023. URL https://arxiv.org/abs/2304.08485

    2023 arXiv

  7. [15]

    H. Liu, Z. Teng, L. Cui, C. Zhang, Q. Zhou, and Y . Zhang. Logicot: Logical chain-of-thought instruction-tuning, 2023. URL https://arxiv. org/abs/2305.12147

    2023 arXiv

  8. [16]

    P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in nat- ural language processing. ACM Computing Surveys, 55(9):1–35, 2023

    2023

  9. [17]

    Z. Liu, W. Yao, J. Zhang, L. Yang, Z. Liu, J. Tan, P. K. Choubey, T. Lan, J. Wu, H. Wang, et al. Agentlite: A lightweight library for building and advancing task-oriented llm agent system. arXiv preprint arXiv:2402.15538, 2024

    2024 arXiv

  10. [18]

    Z. Liu, Y . Zhang, P. Li, Y . Liu, and D. Yang. A dynamic llm-powered agent network for task-oriented agent collaboration, 2024. URL https: //arxiv.org/abs/2310.02170

    2024 arXiv

  11. [19]

    Llama3.2

    Llama. Llama3.2. 2024. URL https://ollama.com/library/llama3.2

    2024

  12. [20]

    J. Long. Large language model guided tree-of-thought, 2023. URL https://arxiv.org/abs/2305.08291

    2023 arXiv

  13. [21]

    P. Lu, S. Mishra, T. Xia, L. Qiu, K.-W. Chang, S.-C. Zhu, O. Tafjord, P. Clark, and A. Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507–2521, 2022

    2022

  14. [22]

    Marvin, N

    G. Marvin, N. Hellen, D. Jjingo, and J. Nakatumba-Nabende. Prompt engineering in large language models. In International conference on data intelligence and cognitive informatics , pages 387–402. Springer, 2023

    2023

  15. [23]

    Qwen2.5-coder

    Qwen. Qwen2.5-coder. 2024. URL https://ollama.com/library/qwen2. 5-coder

    2024

  16. [24]

    S. K. Radha, Y . N. Jelyani, A. Ghukasyan, and O. Goktas. Iteration of thought: Leveraging inner dialogue for autonomous large language model reasoning, 2024. URL https://arxiv.org/abs/2409.12618

    2024 arXiv

  17. [25]

    Rajpurkar, J

    P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang. SQuAD: 100,000+ questions for machine comprehension of text. In J. Su, K. Duh, and X. Carreras, editors, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages 2383–2392, Austin, Texas, N...

    2016

  18. [26]

    K. I. Roumeliotis and N. D. Tselikas. Chatgpt and open-ai models: A preliminary review. Future Internet, 15(6):192, 2023

    2023

  19. [27]

    Saad-Falcon, A

    J. Saad-Falcon, A. G. Lafuente, S. Natarajan, N. Maru, H. Todorov, E. Guha, E. K. Buchanan, M. Chen, N. Guha, C. Ré, et al. Archon: An architecture search framework for inference-time techniques. arXiv preprint arXiv:2409.15254, 2024

    2024 arXiv

  20. [28]

    Sahoo, A

    P. Sahoo, A. K. Singh, S. Saha, V . Jain, S. Mondal, and A. Chadha. A systematic survey of prompt engineering in large language mod- els: Techniques and applications, 2024. URL https://arxiv.org/abs/2402. 07927

    2024

  21. [29]

    X. Song, Y . Wu, W. Wang, J. Liu, W. Su, and B. Zheng. Progco: Program helps self-correction of large language models, 2025. URL https://arxiv.org/abs/2501.01264

    2025 arXiv

  22. [30]

    P. Wang, Z. Wang, Z. Li, Y . Gao, B. Yin, and X. Ren. Scott: Self- consistent chain-of-thought distillation, 2023. URL https://arxiv.org/ abs/2305.01879

    2023 arXiv

  23. [31]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, b. ichter, F. Xia, E. Chi, Q. V . Le, and D. Zhou. Chain-of-thought prompting elicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Bel- grave, K. Cho, and A. Oh, editors,Advances in Neural Information Pr...

    2022

  24. [32]

    Z. Yang, P. Qi, S. Zhang, Y . Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning. Hotpotqa: A dataset for diverse, explainable multi- hop question answering, 2018. URL https://arxiv.org/abs/1809.09600

    2018 arXiv

  25. [33]

    Y . Yao, J. Duan, K. Xu, Y . Cai, Z. Sun, and Y . Zhang. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211, 2024

    2024

  26. [34]

    Zhang, J

    J. Zhang, J. Xiang, Z. Yu, F. Teng, X. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, B. Zheng, B. Liu, Y . Luo, and C. Wu. Aflow: Automating agentic workflow generation, 2024. URL https: //arxiv.org/abs/2410.10762

    2024 arXiv

  27. [35]

    Zhang, J

    R. Zhang, J. Han, C. Liu, P. Gao, A. Zhou, X. Hu, S. Yan, P. Lu, H. Li, and Y . Qiao. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199, 2023

    2023 arXiv

  28. [36]

    Zhang, A

    Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola. Mul- timodal chain-of-thought reasoning in language models, 2024. URL https://arxiv.org/abs/2302.00923

    2024 arXiv

  29. [37]

    Q. Zhou, R. Zhou, Z. Hu, P. Lu, S. Gao, and Y . Zhang. Image-of-thought prompting for visual reasoning refinement in multimodal large language models. arXiv preprint arXiv:2405.13872, 2024

    2024 arXiv

  30. [38]

    W. Zhou, Y . E. Jiang, L. Li, J. Wu, T. Wang, S. Qiu, J. Zhang, J. Chen, R. Wu, S. Wang, et al. Agents: An open-source framework for au- tonomous language agents. arXiv preprint arXiv:2309.07870, 2023

    2023 arXiv

  31. [2024]

    URL https://www.kaggle

    doi: 10.34740/KAGGLE/M/3301. URL https://www.kaggle. com/m/3301

    doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.