Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

CodeV: Issue Resolving with Visual Data

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Converting the visual data in GitHub issues into text lets a text-only LLM resolve over 50% more issues, the paper reports.

desk verdict Visual SWE-bench is a useful new artifact, but the paper's central claim that visual data drive the improvement is missing a key control. read the letter →

arxiv 2412.17315 v1 pith:R2UFKSHE submitted 2024-12-23 cs.SE cs.AIcs.CL

Linhao Zhang , Daoguang Zan , Quanshun Yang , Zhirong Huang , Dong Chen , Bo Shen , Tianyu Liu , Yongshun Gong
show 5 more authors
Pengjie Huang Xudong Lu Guangtai Liang Lizhen Cui Qianxiang Wang
This is my paper · ORCID
classification cs.SEcs.AIcs.CL
keywords GitHubissueresolvingvisualdatavision-languagemodelSWE-benchbenchmarkconstructionpatchgenerationsoftwareengineering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that existing GitHub issue-resolving systems leave useful evidence on the table: screenshots, diagrams, and videos attached to issues. CodeV is a two-phase pipeline that first uses a vision-language model to turn each piece of visual data into a fine-grained text description and to write a structured summary of the whole issue, then feeds that enriched text into an ordinary text-only issue resolver. On a new benchmark called Visual SWE-bench, built from 133 visual issues drawn from SWE-bench and visualization-library pull requests, CodeV plus Agentless resolves 12.78 percent of instances compared with 7.52 percent for Agentless alone, a relative improvement of more than 50 percent. The authors argue this shows visual data carries information that text alone cannot, and that it can be exploited cheaply with open-source models.

What carries the argument

The machinery is the visual-to-text translation step. Fine-grained description is generated in two passes: an independent description that records the raw content of each image or video, and a contextual description that explains what the visual data demonstrates relative to the reported bug. Structured summarization then reorganises the entire issue into a template with clear fields such as problemSummary, stepsToReproduce, expectedResults, and actualResults. These textual artifacts replace the original visual data in the prompt, allowing a text-only LLM operating through Agentless to reason about evidence it could never see directly.

What would settle it

Have humans compare each generated description and summary against the original image or video for all 133 instances, scoring omitted or hallucinated visual facts, and then check whether the seven instances CodeV resolves beyond Agentless are exactly the ones with accurate descriptions; if those instances contain key description errors, the visual-to-text fidelity mechanism is undercut.

Watch

Extended reading notes

Core claim

The central claim is that visual data in an issue can be converted into text and that this conversion, not the coding model's innate multimodal ability, is what unlocks better issue resolution. CodeV's vision-language model produces two artifacts: fine-grained descriptions (an independent description of each image or video, then a contextual description that interprets the image's role in the issue) and a structured summary with problem summary, reproduction steps, expected results, and actual results. These are spliced into the original issue text and passed to Agentless, an open-source localization-and-repair pipeline. The result is 17 of 133 Visual SWE-bench instances resolved (12.78 percent) versus 10 (7.52 percent) for Agentless alone; a variant that feeds images directly to a vision-language model for repair resolves only one instance, which the authors attribute to the VLM's weak coding ability. Ablations show that removing any component lowers performance, with structured summarization removal having the largest effect.

Load-bearing premise

The entire improvement depends on the assumption that the vision-language model's text descriptions and structured summary preserve every task-relevant detail in the screenshot, diagram, or video, because the original visual data is discarded and never checked for accuracy.

Editorial extensions

If this is right

  • Visual GitHub issues can be attacked with open-source text-only coding LLMs, avoiding the estimated thousands of dollars needed to run closed multimodal agents over a benchmark.
  • Existing text-only issue resolvers such as SWE-agent, AutoCodeRover, and RAG can be given visual awareness simply by preprocessing issues with CodeV's pipeline, without changing the resolver.
  • Structured summarization is the highest-value component: removing it cuts resolved instances from 15 to 10, more than removing either description step.
  • VLM size is not decisive: Qwen2-VL-2B and 7B pipelines resolved 17 instances, matching or beating the 72B version's 15, so cheap small VLMs can carry the translation step.
  • Visual SWE-bench provides a reusable, test-verified set of 133 visual issue instances, filling a gap left by existing benchmarks whose evaluation scripts are not public.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors discard the original visual data after converting it to text; a natural extension is to keep both modalities and let the LLM refer back to the image, which could recover details the VLM misreads or omits.
  • Because the benchmark is dominated by visualization and documentation libraries (matplotlib, sphinx, seaborn), the observed gain may be largest for output- and plot-oriented bugs; other software domains with different visual genres may see a smaller effect.
  • The headline gain rests on 7 additional resolved instances (17 versus 10), so the over-50-percent relative improvement is a small absolute effect; replicating on a larger benchmark would test whether the advantage is stable.
  • A direct test of the fidelity assumption would be to have humans mark the factual accuracy of the VLM's descriptions on all 133 instances and check whether the instances CodeV uniquely solves are exactly those with accurate descriptions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CodeV, a two-phase approach for automatically resolving GitHub issues that contain visual data such as screenshots and videos. In the data-processing phase, a vision-language model (Qwen2-VL) generates fine-grained descriptions of each image/video and a structured summary of the entire issue. In the patch-generation phase, these textual artifacts are spliced into the original issue and passed to an existing textual issue-resolving approach (Agentless) to produce a patch. The authors construct a new benchmark, Visual SWE-bench, containing 133 instances from 11 repositories, selected so that visual data is essential for resolution. Experiments on this benchmark show that CodeV + Agentless resolves 12.78% of instances versus 7.52% for Agentless alone (17 vs. 10 instances) in the best configuration, and ablations indicate that removing structured summarization drops performance back to the Agentless baseline.

Significance. If the central claim were established, the paper would make a useful contribution by drawing attention to a previously ignored modality in issue resolution and by providing a benchmark for future work on visual GitHub issues. The construction of Visual SWE-bench, with its execution verification and human filtering, is a substantial effort and could serve as a community resource. The paper also provides transparent limitation statements and uses only open-source models, which improves reproducibility. However, the significance is tempered by the fact that the ablation evidence does not isolate the contribution of visual data, and the benchmark is constructed in a way that may amplify the apparent benefit of visual processing.

major comments (3)
  1. [Section 4.2.2, Table 3]
  2. [Section 3.1, construction step 4]
  3. [Table 2 and Table 3]
minor comments (5)
  1. [Section 1, Figure 1 caption]
  2. [Table 1]
  3. [Section 2.1, structured summarization]
  4. [Section 4.2.1 and Appendix C]
  5. [Figure 4]

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CodeV's results are empirical comparisons on a fixed benchmark, not reductions of outputs to inputs.

full rationale

The paper contains no derivation chain in which a claimed prediction reduces to an input or a fitted parameter. CodeV is an empirical pipeline; its central result (Table 2) compares resolved instances of CodeV+Agentless against Agentless and other baselines on the same benchmark. No parameter is fitted from the benchmark and then re-predicted. The ablation in Table 3 is an empirical decomposition, not a circular definition. The benchmark construction includes human-verification criteria that retain only instances where visual data is 'essential' for resolution (Section 3.1), which makes the benchmark aligned with the paper's thesis, but this is a selection/external-validity concern rather than circularity: the baselines are evaluated on the same instances and the improvement could in principle have gone either way. The absence of a text-only control for structured summarization is a potential internal-validity limitation for the causal claim, but it is not a case where an output is equivalent to an input by construction. No self-citation is load-bearing; citations to the authors' prior work appear only in related-work or baseline context. Thus no circular step meets the required evidentiary standard.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper has no fitted numerical free parameters and invents no entities. It relies on domain assumptions about VLM fidelity, benchmark representativeness, and the SWE-bench harness. The most consequential assumption is that human filtering for 'visual data essential' creates a fair evaluation.

assumptions (4)
  • domain assumption VLM descriptions preserve task-relevant visual information.
    Section 2.1 converts all visual data into text; if the VLM omits or misreads details, the text-only LLM cannot recover them and no human check is reported.
  • ad hoc to paper Human-filtered benchmark is representative of visual issue resolving.
    Section 3.1 excludes instances where visual data is not essential or is OCR-able, which may create a set that favors visual-data methods.
  • domain assumption SWE-bench evaluation harness (FAIL_TO_PASS tests) is ground truth.
    The benchmark relies on applying gold and test patches to determine resolved status; this is inherited from SWE-bench.
  • domain assumption Agentless v1.0 is an appropriate and correctly configured text-only baseline.
    Section 4.1 uses Agentless 1.0; the paper does not provide code-level verification that implementation differences do not drive the comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CodeV: Issue Resolving with Visual Data." pith.science (2026). https://pith.science/paper/R2UFKSHE

@misc{pith2026241217315,
  author       = {Pith},
  title        = {Pith review of: CodeV: Issue Resolving with Visual Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R2UFKSHE}},
  note         = {Machine review of arXiv:2412.17315}
}
read the original abstract

Large Language Models (LLMs) have advanced rapidly in recent years, with their applications in software engineering expanding to more complex repository-level tasks. GitHub issue resolving is a key challenge among these tasks. While recent approaches have made progress on this task, they focus on textual data within issues, neglecting visual data. However, this visual data is crucial for resolving issues as it conveys additional knowledge that text alone cannot. We propose CodeV, the first approach to leveraging visual data to enhance the issue-resolving capabilities of LLMs. CodeV resolves each issue by following a two-phase process: data processing and patch generation. To evaluate CodeV, we construct a benchmark for visual issue resolving, namely Visual SWE-bench. Through extensive experiments, we demonstrate the effectiveness of CodeV, as well as provide valuable insights into leveraging visual data to resolve GitHub issues.

Figures

Figures reproduced from arXiv: 2412.17315 by the authors.

Figure 1
Figure 1. An example of a visual GitHub issue from [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of CODEV. Through these two steps, we obtain fine-grained descriptions of all visual data in the issue. These descriptions capture not only the intrinsic features of the visual data but also its critical function in the problem’s context. 2.1.2 Structured Summarization Some GitHub issues are described in a structured format, including reproduction steps, expected re￾sults, actual results, and so on. This fo… view at source ↗
Figure 3
Figure 3. Distribution of Visual SWE-bench task in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Venn diagrams of issues resolved from Visual [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Structured summarization example for the [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Prompt for generating descriptions of images [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Prompt for analyzing the function of images [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Prompt for generating a structured summary. [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: An example of a processed visual issue. The [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multilingual Multimodal Software Developer for Code Generation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A 7B vision-language model trained on synthetic diagram-to-code data outperforms several larger open-weight models on a new 10-language UML/flowchart code-generation benchmark.

  2. VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models

    cs.CL 2025-08 unverdicted novelty 5.0 of 10

    Merging a coding LLM into a vision-language model via task vectors yields an open-source multimodal coder that reaches near-GPT-4o performance on the authors' new benchmark.

  3. SlideCoder: Layout-aware RAG-enhanced Hierarchical Slide Generation from Design

    cs.CV 2025-06 conditional novelty 5.0 of 10

    SlideCoder converts slide design images to editable python-pptx code and reports large gains over prior baselines on a new difficulty-tiered benchmark.

Reference graph

Works this paper leans on

33 extracted references · 9 canonical work pages · cited by 3 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Anthropic. 2024. Introducing claude 3.5 sonnet. https://www.anthropic.com/news/claude-3-5-sonnet

    work page 2024

  4. [4]

    AppMap. 2024. Appmap speedruns to the top of the swe bench leaderboard. https://appmap.io/blog/2024/06/20/appmap-navie-swe-bench-leader

    work page 2024

  5. [5]

    Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, and Ramesh Nallapati. 2023. https://openreview.net/foru...

    work page 2023

  6. [6]

    Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J

    Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021. https://arxiv.org/abs/2108.07732 Program synthesis with large language models . CoRR, abs/2108.07732

    arXiv 2021

  7. [7]

    AWS. 2024. Amazon q developer the most capable generative ai–powered assistant for software development. https://aws.amazon.com/q/developer

    work page 2024

  8. [8]

    Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Costin, Donald Pinckney, Ming-Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, et al. 2022. Multipl-e: A scalable and extensible approach to benchmarking neural code generation. arXiv preprint arXiv:2208.08227

    arXiv 2022

Show all 33 references
  1. [9]

    Dong Chen, Shaoxin Lin, Muhan Zeng, Daoguang Zan, Jian - Gang Wang, Anton Cheshkov, Jun Sun, Hao Yu, Guoliang Dong, Artem Aliev, Jie Wang, Xiao Cheng, Guangtai Liang, Yuchi Ma, Pan Bian, Tao Xie, and Qianxiang Wang. 2024 a . https://doi.org/10.48550/ARXIV.2406.01304 Coder: Iss...

    2024 doi arXiv

  2. [10]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Sco...

    2021 arXiv

  3. [11]

    Xiangping Chen, Xing Hu, Yuan Huang, He Jiang, Weixing Ji, Yanjie Jiang, Yanyan Jiang, Bo Liu, Hui Liu, Xiaochen Li, et al. 2024 b . Deep learning-based software engineering: Progress, challenges, and opportunities. arXiv preprint arXiv:2410.13110

    2024 arXiv

  4. [12]

    DeepSeek-AI. 2024. https://arxiv.org/abs/2405.04434 Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model . Preprint, arXiv:2405.04434

    2024 arXiv

  5. [13]

    Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2023. https://doi.org/10.48550/ARXIV.2308.01861 Classeval: A manually-crafted benchmark for evaluating llms on class-level code generation . CoRR, ab...

    2023 doi arXiv

  6. [14]

    Factory. 2024. Factory bringing autonomy to software engineering. https://www.factory.ai

    2024

  7. [15]

    Honeycomb. 2024. Honeycomb. https://honeycomb.sh

    2024

  8. [16]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. 2024. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186

    2024 arXiv

  9. [17]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. 2024. https://openreview.net/forum?id=VTF8yNQM66 Swe-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning R...

    2024

  10. [18]

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html Is your code generated by chatgpt really correct? rigorous evaluation of large language models for...

    2023

  11. [19]

    Wei Liu, Ailun Yu, Daoguang Zan, Bo Shen, Wei Zhang, Haiyan Zhao, Zhi Jin, and Qianxiang Wang. 2024. https://doi.org/10.48550/ARXIV.2406.07003 Graphcoder: Enhancing repository-level code completion via code context graph-based retrieval and language model . CoRR, abs/2406.07003

    2024 doi arXiv

  12. [20]

    Niklas Muennighoff, Qian Liu, Armel Randy Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro von Werra, and Shayne Longpre. 2024. https://openreview.net/forum?id=mw1PWNSWZP Octopack: Instruction tuning code large language models . In The Twe...

    2024

  13. [21]

    OpenAI. 2024. Hello gpt-4o. https://openai.com/index/hello-gpt-4o

    2024

  14. [22]

    Gabriel Orlanski, Kefan Xiao, Xavier Garcia, Jeffrey Hui, Joshua Howland, Jonathan Malmaud, Jacob Austin, Rishabh Singh, and Michele Catasta. 2023. https://proceedings.mlr.press/v202/orlanski23a.html Measuring the impact of programming language distribution . In International ...

    2023

  15. [23]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. Qwen2-vl: Enhancing vision-language mode...

    2024 arXiv

  16. [24]

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. https://doi.org/10.48550/ARXIV.2407.01489 Agentless: Demystifying llm-based software engineering agents . CoRR, abs/2407.01489

    2024 doi arXiv

  17. [25]

    Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024 a . https://doi.org/10.48550/ARXIV.2405.15793 Swe-agent: Agent-computer interfaces enable automated software engineering . CoRR, abs/2405.15793

    2024 doi arXiv

  18. [26]

    Jimenez, Alex L

    John Yang, Carlos E. Jimenez, Alex L. Zhang, Kilian Lieret, Joyce Yang, Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R. Narasimhan, Diyi Yang, Sida I. Wang, and Ofir Press. 2024 b . https://doi.org/10.48550/ARXIV.2410.03859 Swe-bench multimodal: Do AI sys...

    2024 doi arXiv

  19. [27]

    Hao Yu, Bo Shen, Dezhi Ran, Jiaxin Zhang, Qi Zhang, Yuchi Ma, Guangtai Liang, Ying Li, Qianxiang Wang, and Tao Xie. 2024. https://doi.org/10.1145/3597503.3623316 Codereval: A benchmark of pragmatic code generation with generative pre-trained models . In Proceedings of the 46th...

    2024

  20. [28]

    Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang, and Jian - Guang Lou. 2023. https://doi.org/10.18653/V1/2023.ACL-LONG.411 Large language models meet nl2code: A survey . In Proceedings of the 61st Annual Meeting of the Association for Compu...

    2023 doi

  21. [29]

    Daoguang Zan, Zhirong Huang, Ailun Yu, Shaoxin Lin, Yifan Shi, Wei Liu, Dong Chen, Zongshuai Qi, Hao Yu, Lei Yu, Dezhi Ran, Muhan Zeng, Bo Shen, Pan Bian, Guangtai Liang, Bei Guan, Pengjie Huang, Tao Xie, Yongji Wang, and Qianxiang Wang. 2024. https://doi.org/10.48550/ARXIV.24...

    2024 doi arXiv

  22. [30]

    Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian - Guang Lou, and Weizhu Chen. 2023 a . https://doi.org/10.18653/V1/2023.EMNLP-MAIN.151 Repocoder: Repository-level code completion through iterative retrieval and generation . In Proceedings of...

    2023 doi

  23. [31]

    Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. https://doi.org/10.1145/3650212.3680384 Autocoderover: Autonomous program improvement . In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2024, Vienna, Au...

    2024

  24. [32]

    Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, and Rui Wang. 2023 b . https://doi.org/10.48550/ARXIV.2311.07989 A survey on language models for code . CoRR, abs/2311.07989

    2023 doi arXiv

  25. [33]

    Zibin Zheng, Kaiwen Ning, Yanlin Wang, Jingwen Zhang, Dewu Zheng, Mingxi Ye, and Jiachi Chen. 2023. https://doi.org/10.48550/ARXIV.2311.10372 A survey of large language models for code: Evolution, benchmarking, and future trends . CoRR, abs/2311.10372

    2023 doi arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.