Reasoning as a Resource: Optimizing Fast and Slow Thinking in Code Generation Models

Shuai Wang; Zongjie Li

REVIEW 4 major objections 5 minor 1 cited by

Reasoning as a Resource: Optimizing Fast and Slow Thinking in Code Generation Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This position paper argues that the depth of a code model's reasoning should be managed explicitly as a tunable resource across data generation, evaluation, and deployment, citing adaptive controllers that save 50–70% of tokens without…

desk verdict A coherent position paper that names a real problem but leans on out-of-domain results and sloppy citations; worth reviewing only if reframed as an agenda, not a result. read the letter →

arxiv 2506.09396 v1 pith:UUOM6PQD submitted 2025-06-11 cs.SE cs.AI

Zongjie Li , Shuai Wang This is my paper

classification cs.SEcs.AI

keywords reasoningdepthchain-of-thoughtcodegenerationlargelanguagemodelsadaptiveinferencetokenbudgetsyntheticdatabenchmarking

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the depth of reasoning a code-generation model performs before answering should be treated as an explicit, controllable resource rather than an incidental side effect of prompting. It proposes managing that depth—measured as the length and cost of the model's written chain-of-thought—across all three stages of a model's life: synthetic data creation, benchmarking, and deployment. The motivation is the three-way tension among correctness, latency, and token cost: if reasoning budgets can be scheduled adaptively, models could deliver equal accuracy at much lower cost. The support comes from cited adaptive controllers that save 50–70% of tokens without hurting success rates on general reasoning tasks, and from scaling laws showing accuracy rises predictably with token count. The paper then outlines concrete changes: curriculum-style scheduling of reasoning depth in data synthesis, benchmarks that report latency and cost alongside pass@k, and policy engines that allocate thinking budgets per request.

What carries the argument

The central object is the reasoning depth controller: a learned scheduler that decides, per prompt, whether to emit a fast direct answer or a slow chain-of-thought trace. The paper treats this controller as the hinge of the entire lifecycle, since the same mechanism can schedule CoT length during synthetic data generation, make token budgets visible during evaluation, and act as a policy engine during deployment. The measurable artifact is the CoT trace itself, whose token length quantifies both the depth and the cost of reasoning. A secondary mechanism is the diagnostic matrix that cross-tabulates reasoning quality against solution correctness, designed to expose overfitting, memorization, and execution errors that pass@k hides.

What would settle it

Run an adaptive reasoning controller on a code benchmark such as HumanEval and compare pass@k at matched token budgets against a fixed full-reasoning baseline; if the controller cannot match accuracy while cutting tokens, the claimed Pareto improvement fails. A second decisive test is to vary only the length of written reasoning for the same underlying model and task: if accuracy does not track token count in the way the scaling laws predict, then token length is not a reliable handle on reasoning depth.

Watch

Extended reading notes

Core claim

The paper's central claim is that reasoning depth—the extent of step-by-step deliberation a model performs before producing a solution, operationalized as chain-of-thought token length—should be a first-class design parameter, managed explicitly across the entire code-model lifecycle. It contends that synthetic data pipelines should schedule reasoning depth as a curriculum variable, generating short traces for pattern-matching tasks and long traces for complex or security-critical examples, balancing supervision richness against token economy and intellectual-property protection. It contends that benchmarks should log timing, token counts, and reasoning traces so that accuracy, latency, and cost are reported together, exposing a multi-dimensional performance envelope and enabling a diagnostic matrix that cross-tabulates reasoning quality against solution correctness. It contends that deployment should use adaptive reasoning controllers as policy engines that map service-level objectives to per-request chain-of-thought budgets, treating CoT as both a cost and a potential confidentiality liability. The supporting evidence is cited rather than produced here: adaptive controllers that recover baseline accuracy with 50–70% fewer tokens, inference-time scaling laws that tie accuracy to token count, and audits showing that unconstrained CoT can leak proprietary logic.

Load-bearing premise

The agenda depends on adaptive controllers that decide when a model should write out full step-by-step reasoning, demonstrated on general puzzles, working just as well for coding tasks and for data-generation and deployment stages—and on the length of that written reasoning being a true measure of thinking depth.

Editorial extensions

If this is right

Synthetic data pipelines that schedule chain-of-thought length by task difficulty could produce the same supervision richness at a fraction of the token cost, changing how code models are trained.
Benchmarks that report latency and token usage alongside pass@k would reveal Pareto improvements that are currently invisible, changing how code models are compared and selected.
Production systems could allocate thinking budgets per request—deep reasoning for complex or security-critical tasks, fast answers for boilerplate—while using chain-of-thought limits as a defensive control against intellectual-property leakage.
Code agents built this way would be optimized around an accuracy-latency-cost envelope, making them cheaper to operate at scale and easier to align with service-level objectives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If reasoning depth is genuinely controllable, the same scheduling idea should transfer beyond code generation to other token-cost-sensitive domains such as mathematical reasoning or general agentic tasks, where adaptive budgets could similarly cut inference cost.
A concrete testable extension is to build a code benchmark that logs chain-of-thought traces and applies the paper's diagnostic matrix; if models that succeed via coincidental correctness (wrong reasoning, right answer) turn out to be less robust to perturbation, that would validate the need for reasoning-aware evaluation.
The paper's security argument implies a defensive payoff that is only implicit: deliberately restricting or sanitizing chain-of-thought in production could serve dual duty as both a cost-saving measure and a safeguard against IP leakage, which may make adaptive budgets attractive to security-conscious adopters.
If token length turns out to be a poor proxy for reasoning depth—for instance, if a model can reason internally without externalizing tokens—then the framework's controllers would have no reliable signal to optimize, and the agenda would need a different operationalization of depth before it can be engineered.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a position paper, not a research paper, and the authors mostly know that. The idea — treat reasoning depth as a controllable budget across synthetic data, evaluation, and deployment — is clearly stated and genuinely worth discussing. The three-stage framing is tidy, the security angle (CoT as leak surface) adds something, and the paper does cite the relevant adaptive-CoT work (AdaCoT, AdaptThink) rather than pretending the idea is new. I'd credit it as a useful synthesis. That said, the load-bearing numbers don't transfer. The 50–70% token savings come from AdaCoT and AdaptThink, both evaluated on general reasoning benchmarks where the answer is a short string and suppressing CoT doesn't change the output surface. Code generation is open-ended and execution-graded; the CoT often contains the algorithmic content, so a trigger policy that withholds it on 'easy' prompts may withhold exactly what's needed to produce a working program. The paper extends the same result to synthetic data generation, claiming lower synthesis costs, while also citing evidence that longer step-wise rationales help Code Llama. Those two claims sit in tension, and the paper never tests either. There are also concrete citation problems. Reference [30] is labeled Self-Instruct but points to the Self-Consistency paper. Reference [26] is described as MasLeak, benchmarking IP leakage in multi-agent systems, but the cited arXiv ID (2404.13945) is a paper on image mutation for testing visual deep learning systems. Those are not minor glitches; they undermine trust in exactly the places where the argument leans on external evidence. The diagnostic matrix in Table 1 is fine as a classification tool, and the call for latency/token-aware benchmarks is reasonable. But none of the three proposals is demonstrated here. If the authors reframe this as an agenda with explicit hypotheses — and fix the references — it could be a decent roadmap for the code-LLM community. As it stands, it's a well-organized abstract of other people's results with an untested extrapolation at the center. Who should read it: researchers thinking about evaluation methodology or deployment cost controls for code models. I'd not cite the 50–70% numbers from this paper. For peer review: I'd send it out only after the authors correct the references and hedge the transfer claims; the core framing is worth a serious look, but the current manuscript would waste a referee's time if those errors stay.

Referee Report

4 major / 5 minor

Summary. This position paper argues that reasoning depth, operationalized as the length of chain-of-thought (CoT) traces, should be treated as an explicitly manageable resource throughout the lifecycle of code generation LLMs. It proposes three concrete shifts: scheduling CoT length as a curriculum variable in synthetic data generation, augmenting benchmark evaluation with latency and token-cost metrics alongside pass@k, and deploying adaptive reasoning-budget controllers that align thinking effort with task difficulty and security constraints. The paper cites adaptive chain-of-thought controllers AdaCoT and AdaptThink as evidence that per-prompt reasoning budgets can reduce token use by 50–70% without accuracy loss, and it sketches a unifying architecture centered on a 'Reasoning Depth Controller.' The contribution is conceptual; the paper contains no new experiments, derivations, or empirical analyses.

Significance. If the proposed agenda is feasible, it could reshape how code LLMs are trained, evaluated, and deployed, connecting efficiency engineering with security and cost considerations. The paper is timely given the emergence of hybrid fast/slow models such as Qwen-3, and it usefully identifies concrete gaps in current benchmark culture and in synthetic-data pipelines. Its strength is the clarity of the framework and the breadth of related-work connections. However, the central feasibility claim currently rests entirely on cited results from general-domain reasoning controllers, not on any direct evidence for code generation; the significance of the proposal is therefore conditional on a transfer that the paper does not establish.

major comments (4)

[Section 2 and Section 4] The paper's central quantitative support for Pareto improvements is the claim that AdaCoT and AdaptThink achieve 50–70% token savings without accuracy loss. Both cited controllers were developed and evaluated on general reasoning benchmarks with short, exact-match answer surfaces, not on open-ended, execution-graded code generation. In code generation, the CoT trace often contains the algorithmic design steps needed to produce a compilable program, so a triggering policy that withholds CoT on 'easy' prompts may withhold exactly the content required for correctness. Please provide a pilot experiment, a dataset-specific analysis, or at least a detailed argument grounded in code-generation characteristics showing that adaptive CoT triggering transfers to code. Without this, the load-bearing premise of the paper is unsupported.
[Section 2] The paper extrapolates the inference-time token savings of AdaCoT/AdaptThink to the synthetic-data generation stage, claiming that such controllers 'could directly lower synthesis costs.' This is an unverified leap: inference-time triggering decides whether to emit CoT for a given prompt on an already-trained model, whereas synthetic-data curation decides what traces to include in training data, which affects the student model's learned behavior. These are different optimization problems with different feedback loops. The paper should either provide evidence that adaptive trace selection improves or maintains downstream model quality, or explicitly re-frame this as an open hypothesis rather than a direct consequence of the cited controllers.
[Section 1 and Section 3, Table 1] The definition of 'reasoning depth' is inconsistent across the paper. Section 1 equates it with the token length and computational cost of CoT traces, yet Section 3 cites structured CoT (SCoT) as evidence that 'reasoning content, not just length, matters,' and Table 1 requires judging whether a CoT is 'Correct' or 'Incorrect.' The diagnostic matrix in Table 1 lacks any operational definition of how CoT correctness is determined independently of the final solution. If reasoning depth is meant to be token count, then Table 1's quality axis is undefined; if it is meant to be a semantic property, then the paper's reliance on token-budget metrics is insufficient. Please clarify the intended operationalization and how each quadrant of Table 1 would be measured in practice.
[General] The paper makes prescriptive claims about the entire model lifecycle but provides no direct evidence for the central mechanism. As a position paper, this is acceptable if the claims are framed as hypotheses; however, the abstract and Section 1 state that the proposed approach 'can unlock superior trade-offs' as a matter of fact. The manuscript should distinguish established results (AdaCoT/AdaptThink on general reasoning) from conjectures (transfer to code generation, utility for synthetic data, and deployment policy effectiveness), and should state what evidence would validate or falsify each conjecture.

minor comments (5)

[References, [30]] The text in Section 2 says 'Self-Instruct demonstrated that models can bootstrap their own instruction-response pairs,' but reference [30] is 'Self-Consistency Improves Chain of Thought Reasoning in Language Models.' The Self-Instruct paper is not cited; please correct the citation or the claim.
[References, [26]] The text in Section 4 attributes the 'MasLeak' framework to reference [26], but reference [26] is 'Benchmarking Multi-Modal LLMs for Testing Visual Deep Learning Systems Through the Lens of Image Mutation.' The cited paper does not appear to be about prompt leakage in multi-agent systems; please replace with the correct MasLeak reference or revise the text.
[Abstract] There is a typo in the abstract: 'deploymen' should be 'deployment.'
[Formatting] The manuscript uses a conference template with placeholder dates ('Conference’17, July 2017') and ACM DOI placeholders; these should be updated to the actual submission venue and year.
[Section 3] The proposal to log and report CoT traces lacks discussion of practical reproducibility challenges, such as how to compare traces across models with different tokenizers or how to handle proprietary models that do not expose internal reasoning. A brief note on these limitations would strengthen the evaluation proposal.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the position paper's claims rest on external adaptive-CoT results; the misattributed self-citation [26] is a reliability issue, not circularity.

full rationale

This is a position paper with no fitted parameters, learned quantities, or equations, so there is no prediction loop in which an output is equivalent to an input by construction. The central proposal—treating reasoning depth as a controllable resource across data synthesis, evaluation, and deployment—is supported by external works (AdaCoT [20], AdaptThink [37], inference-scaling-law analysis [35]) rather than by the authors' own prior results, so the core argument has independent content. The many self-citations support peripheral observations (e.g., SAST blind-spot testing [12], code-completion repair [14]) and do not carry the central claim. One flagged reliability issue: Sections 1, 2, and 4 attribute the MasLeak result—'unconstrained CoT can leak proprietary logic in multi-agent systems'—to reference [26], but [26] is a same-author paper on benchmarking multi-modal LLMs for visual deep-learning testing, not the MasLeak framework; this is a missing/misattributed reference and a correctness concern, but not circular, because the paper does not define reasoning-depth control in terms of [26] and the main agenda does not reduce to that citation. Accordingly, no significant circularity is present and the score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper's central agenda rests on a small number of domain assumptions inherited from cited work: that token length measures reasoning depth, that adaptive CoT controllers save 50-70% tokens without accuracy loss, and that these benefits transfer to code and to the proposed lifecycle stages. Since no new evidence is provided, these remain axioms rather than demonstrated facts.

assumptions (4)

domain assumption CoT token length is a valid and sufficient measure of reasoning depth.
The paper equates reasoning depth with token length and cost throughout (Section 1: 'quantifies this depth in terms of token length and computational cost'). This is an unvalidated modeling choice.
domain assumption Adaptive CoT controllers (AdaCoT, AdaptThink) achieve 50-70% token savings while recovering baseline accuracy, and this transfers to code generation and to synthetic-data and deployment stages.
Used in Sections 2-4 as the existence proof for the proposed Pareto improvements; these are external results not reproduced here.
domain assumption Unconstrained chain-of-thought in code models can leak proprietary information, as evidenced by the MasLeak framework.
Relied on in Sections 2 and 4 to motivate security-conscious budget control; the cited reference [26] appears mismatched, so this supporting evidence is not properly anchored.
domain assumption Inference-time scaling laws imply a smooth, predictable trade-off between token count and accuracy.
Invoked in Section 3 to argue that token budgets matter; external result cited as [35].

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reasoning as a Resource: Optimizing Fast and Slow Thinking in Code Generation Models." pith.science (2026). https://pith.science/paper/UUOM6PQD

@misc{pith2026250609396,
  author       = {Pith},
  title        = {Pith review of: Reasoning as a Resource: Optimizing Fast and Slow Thinking in Code Generation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UUOM6PQD}},
  note         = {Machine review of arXiv:2506.09396}
}

read the original abstract

This position paper proposes a fundamental shift in designing code generation models: treating reasoning depth as a controllable resource. Rather than being an incidental byproduct of prompting, we argue that the trade-off between rapid, direct answers ("fast thinking") and elaborate, chain-of-thought deliberation ("slow thinking") must be explicitly managed. We contend that optimizing reasoning budgets across the entire model lifecycle - from synthetic data creation and benchmarking to real-world deploymen - can unlock superior trade-offs among accuracy, latency, and cost. This paper outlines how adaptive control over reasoning can enrich supervision signals, motivate new multi-dimensional benchmarks, and inform cost-aware, security-conscious deployment policies. By viewing fast and slow thinking as complementary modes to be scheduled, we envision coding agents that think deep when necessary and act fast when possible.

Figures

Figures reproduced from arXiv: 2506.09396 by the authors.

**Figure 1.** An overarching framework for treating reasoning as a controllable resource. The top panel illustrates how the Reasoning Depth Controller modulates model behavior by choosing between “Fast Thinking” (direct answer generation) and “Slow Thinking” (generating an explicit chain of thought). The bottom panel depicts how this controller is applied to three critical stages of the Model Lifecycle: (1) strategically schedul… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Position Paper: Programming Language Techniques for Bridging LLM Code Generation Semantic Gaps
cs.SE 2025-07 unverdicted novelty 2.0 of 10

A position paper arguing that PL techniques, especially formal verification and structure-aware representations, should be deeply integrated into LLM code generation.

Reference graph

Works this paper leans on

43 extracted references · 19 canonical work pages · cited by 1 Pith paper

[30]

Le, Ed H

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self- Consistency Improves Chain of Thought Reasoning in Language Mod- els. InThe Eleventh International Conference on Learning Representa- tions, ICLR 2023, Kigali, Rwanda, May 1-5, 2023

work page 2023
[26]

Zhao Tian, Junjie Chen, and Xiangyu Zhang. 2025. Fixing Large Lan- guage Models’ Specification Misunderstanding for Better Code Gener- ation. InProceedings of the 47th IEEE/ACM International Conference on Software Engineering (ICSE). arXiv:2309.16120

arXiv 2025
[1]

[n. d.]. qwen3blog.https://qwenlm.github.io/blog/qwen3

work page
[2]

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. Conference’17, July 2017, Washington, DC, USA Zongjie Li and Shuai Wang arXiv preprint arXiv:2108.07732(2021)

arXiv 2021
[3]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

arXiv 2021
[4]

Sourav Deb, Kush Jain, Rijnard van Tonder, Claire Le Goues, and Alex Groce. 2024. Syntax Is All You Need: A Universal-Language Approach to Mutant Generation. InProceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (FSE ’24)

work page 2024
[5]

Hadeel Eladawy, Claire Le Goues, and Yuriy Brun. 2024. Automated Program Repair, What Is It Good For? Not Absolutely Nothing!. In Proceedings of the 46th International Conference on Software Engineering (ICSE). 84:1–84:13. doi:10.1145/3597503.3639095

arXiv 2024
[6]

Zequan Fan, Hao Ruan, Sergey Mechtaev, and Abhik Roychoudhury

work page

Show all 43 references

[7]

Jinyao Guo, Chengpeng Wang, Xiangzhe Xu, Zian Su, and Xiangyu Zhang. 2025. RepoAudit: An Autonomous LLM-Agent for Repository- Level Code Auditing. InInternational Conference on Machine Learning (ICML). arXiv:2501.18160

2025 arXiv
[8]

Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes.arXiv preprint ar...

2023 arXiv
[9]

Zimo Ji, Daoyuan Wu, Wenyuan Jiang, Pingchuan Ma, Zongjie Li, and Shuai Wang. 2025. Measuring and Augmenting Large Language Models for Solving Offensive Security Challenges. InProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, CCS 2025, Taip...

2025
[10]

Jia Li, Ge Li, Yongmin Li, and Zhi Jin. 2025. Structured chain-of- thought prompting for code generation.ACM Transactions on Software Engineering and Methodology34, 2 (2025), 1–23

2025
[11]

Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023. StarCoder: may the source be with you!arXiv preprint arXiv:2305.06161(2023)

2023 arXiv
[12]

Zongjie Li, Zhibo Liu, Wai Kin Wong, Pingchuan Ma, and Shuai Wang
[13]

Zongjie Li, Pingchuan Ma, Huaijin Wang, Shuai Wang, Qiyi Tang, Sen Nie, and Shi Wu. 2022. Unleashing the Power of Compiler Intermedi- ate Representation to Enhance Neural Program Embeddings. In44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pitt...

2022
[14]

Evaluating C/C++ Vulnerability Detectability of Query-Based Static Application Security Testing Tools.IEEE Transactions on De- pendable and Secure Computing(2024)

2024
[15]

Zongjie Li, Chaozheng Wang, Pingchuan Ma, Chaowei Liu, Shuai Wang, Daoyuan Wu, and Cuiyun Gao. 2023. On the feasibility of specialized ability stealing for large language code models. (2023)

2023
[16]

Zongjie Li, Chaozheng Wang, Zhibo Liu, Haoxuan Wang, Dong Chen, Shuai Wang, and Cuiyun Gao. 2023. CCTEST: Testing and Repairing Code Completion Systems. In45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20,

2023
[17]

Zongjie Li, Chaozheng Wang, Shuai Wang, and Gao Cuiyun. 2023. Protecting Intellectual Property of Large Language Model-Based Code Generation APIs via Watermarks. InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS 2023, Copenhagen, Denm...

2023
[18]

Zongjie Li, Chaozheng Wang, Pingchuan Ma, Daoyuan Wu, Shuai Wang, Cuiyun Gao, and Yang Liu. 2024. Split and Merge: Aligning Position Biases in LLM-based Evaluators. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohi...

2024
[19]

Zongjie Li, Daoyuan Wu, Shuai Wang, and Su Zhendong. 2025. Differentiation-Based Extraction of Proprietary Data from Fine-tuned LLMs. InProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, CCS 2025, Taipei, Taiwan, October 13-17, 2025

2025
[20]

Zongjie Li, Daoyuan Wu, Shuai Wang, and Zhendong Su. 2025. Api- guided dataset synthesis to finetune large code models.Proceedings of the ACM on Programming Languages9, OOPSLA1 (2025), 786–815

2025
[21]

Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. 2024. StarCoder 2 and The Stack v2: The Next Generation.arXiv preprint arXiv:2402.19173(2024)

2024 arXiv
[22]

Chenwei Lou, Zewei Sun, Xinnian Liang, Meng Qu, Wei Shen, Wenqi Wang, Yuntao Li, Qingping Yang, and Shuangzhi Wu. 2025. AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforce- ment Learning.arXiv preprint arXiv:2505.11896(2025)

2025 arXiv
[23]

Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. 2023. Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950(2023)

2023 arXiv
[24]

Pingchuan Ma, Zhaoyu Wang, Zongjie Li, Zhenlan Ji, Ao Sun, Juergen Rahmel, and Shuai Wang. 2025. Reeq: Testing and Mitigating Ethically Inconsistent Suggestions of Large Language Models with Reflective Equilibrium. (2025).https://doi.org/10.1145/3722554

2025 doi
[25]

Chaozheng Wang, Zongjie Li, Cuiyun Gao, Wenxuan Wang, Ting Peng, Hailiang Huang, Yuetang Deng, Shuai Wang, and Michael R Lyu
[27]

Ruoke Wang, Zongjie Li, Chaozheng Wang, Yang Xiao, and Cuiyun Gao. 2024. NAVRepair: Node-type Aware C/C++ Code Vulnerability Repair.arXiv preprint arXiv:2405.04994(2024)

2024
[28]

Exploring Multi-Lingual Bias of Large Code Models in Code Generation.arXiv preprint arXiv:2404.19368(2024)

2024 arXiv
[29]

Liwen Wang, Yuanyuan Yuan, Ao Sun, Zongjie Li, Pingchuan Ma, Daoyuan Wu, and Shuai Wang. 2024. Benchmarking Multi-Modal LLMs for Testing Visual Deep Learning Systems Through the Lens of Image Mutation.arXiv preprint arXiv:2404.13945(2024)

2024 arXiv
[31]

Shangwen Wang, Bo Lin, Zhensu Sun, Ming Wen, Yepang Liu, Yan Lei, and Xiaoguang Mao. 2023. Two Birds with One Stone: Boosting Code Generation and Code Search via a Generative Adversarial Network. Proc. ACM Program. Lang.7, OOPSLA2, Article 239 (oct 2023), 30 pages. doi:10.1145/3622815

2023 doi
[32]

Xunguang Wang, Wenxuan Wang, Zhenlan Ji, Zongjie Li, Pingchuan Ma, Daoyuan Wu, and Shuai Wang. 2025. STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Models. arXiv preprint arXiv:2503.17932(2025). Reasoning as a Resource: Optimizing Fast and ...

2025 arXiv
[33]

Wai Kin Wong, Huaijin Wang, Zongjie Li, Zhibo Liu, Shuai Wang, Qiyi Tang, Sen Nie, and Shi Wu. 2023. Refining decompiled c code with large language models.arXiv preprint arXiv:2310.06530(2023)

2023 arXiv
[34]

Xunguang Wang, Daoyuan Wu, Zhenlan Ji, Zongjie Li, Pingchuan Ma, Shuai Wang, Yingjiu Li, Yang Liu, Ning Liu, and Juergen Rahmel. 2024. SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner.arXiv preprint arXiv:2406.05498(2024)

2024 arXiv
[35]

Zhijie Wang, Zijie Zhou, Da Song, Yuheng Huang, Shengmai Chen, Lei Ma, and Tianyi Zhang. 2025. Towards Understanding the Charac- teristics of Code Generation Errors Made by Large Language Models. InProceedings of the 47th IEEE/ACM International Conference on Soft- ware Enginee...

2025 arXiv
[36]

Zhaojian Yu, Yilun Zhao, Arman Cohan, and Xiao-Ping Zhang. 2024. HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation.arXiv preprint arXiv:2412.21199 (2024)

2024 arXiv
[37]

Wai Kin Wong, Daoyuan Wu, Huaijin Wang, Zongjie Li, Zhibo Liu, Shuai Wang, Qiyi Tang, Sen Nie, and Shi Wu. 2025. DecLLM: LLM- Augmented Recompilable Decompilation for Enabling Programmatic Use of Decompiled Code. InProceedings of the 34th ACM SIGSOFT International Symposium on...

2025
[38]

Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024. Inference scaling laws: An empirical analysis of compute- optimal inference for problem-solving with language models.arXiv preprint arXiv:2408.00724(2024)

2024 arXiv
[39]

Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, and Lidong Bing. 2023. Verify-and-edit: A knowledge-enhanced chain-of-thought framework.arXiv preprint arXiv:2305.03268(2023)

2023 arXiv
[40]

Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. 2025. Adaptthink: Reasoning models can learn when to think.arXiv preprint arXiv:2505.13417(2025)

2025 arXiv
[41]

Kunpeng Zhang, Zongjie Li, Daoyuan Wu, Shuai Wang, and Xin Xia
[2024]

InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’24)

Oracle-guided Program Selection from Large Language Models. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’24). ACM, 628–640. doi:10.1145/ 3652212.3652272
[2025]

Low-Cost and Comprehensive Non-textual Input Fuzzing with LLM-Synthesized Input Generators.arXiv preprint arXiv:2501.19282 (2025)

2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [30]

Le, Ed H

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self- Consistency Improves Chain of Thought Reasoning in Language Mod- els. InThe Eleventh International Conference on Learning Representa- tions, ICLR 2023, Kigali, Rwanda, May 1-5, 2023

work page 2023

[2] [26]

Zhao Tian, Junjie Chen, and Xiangyu Zhang. 2025. Fixing Large Lan- guage Models’ Specification Misunderstanding for Better Code Gener- ation. InProceedings of the 47th IEEE/ACM International Conference on Software Engineering (ICSE). arXiv:2309.16120

arXiv 2025

[3] [1]

[n. d.]. qwen3blog.https://qwenlm.github.io/blog/qwen3

work page

[4] [2]

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. Conference’17, July 2017, Washington, DC, USA Zongjie Li and Shuai Wang arXiv preprint arXiv:2108.07732(2021)

arXiv 2021

[5] [3]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

arXiv 2021

[6] [4]

Sourav Deb, Kush Jain, Rijnard van Tonder, Claire Le Goues, and Alex Groce. 2024. Syntax Is All You Need: A Universal-Language Approach to Mutant Generation. InProceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (FSE ’24)

work page 2024

[7] [5]

Hadeel Eladawy, Claire Le Goues, and Yuriy Brun. 2024. Automated Program Repair, What Is It Good For? Not Absolutely Nothing!. In Proceedings of the 46th International Conference on Software Engineering (ICSE). 84:1–84:13. doi:10.1145/3597503.3639095

arXiv 2024

[8] [6]

Zequan Fan, Hao Ruan, Sergey Mechtaev, and Abhik Roychoudhury

work page