Pith. sign in

REVIEW 4 major objections 5 minor 34 references

Analysis on LLMs Performance for Code Summarization

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mistral-7b and Phi-3-medium lead open LLMs at code summarization, the paper claims, with Mistral best overall across six programming languages.

desk verdict A routine benchmark comparison whose central ranking is not supported by the reported numbers: duplicate scores and an inconsistent split make the evidence impossible to trust. read the letter →

arxiv 2412.17094 v2 pith:5S3WQPLE submitted 2024-12-22 cs.SE cs.AI

Md. Ahnaf Akib , Md. Muktadir Mazumder , Salman Ahsan This is my paper · ORCID
classification cs.SEcs.AI
keywords CodesummarizationLargelanguagemodelsComparativeanalysisMistral-7bPhi-3-mediumBLEUROUGE-LXGLUE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to compare four open-source large language models — Llama-3-8b, Phi-3-medium, Mistral-7b, and Gemma-7b — on the task of generating natural-language summaries of source code. After fine-tuning each model on the CodeXGLUE benchmark across six programming languages and scoring the outputs with BLEU and ROUGE-L, the paper claims that Mistral-7b is the best overall performer and that Phi-3-medium is a close and consistent alternative, with Llama-3 and Gemma trailing on most datasets. The practical point is that developers building code summarization or documentation tools can make an informed open-source model choice rather than relying on general-purpose assumptions.

What carries the argument

The comparative setup itself is the load-bearing mechanism: a fixed public benchmark (CodeXGLUE, a filtered subset of CodeSearchNet), the same supervised fine-tuning pipeline applied to all four models, and two standard n-gram-based metrics (BLEU and ROUGE-L) computed on the same test splits. This lets the paper turn "which model is better at code summarization" into a direct score comparison across models and programming languages. BLEU measures n-gram overlap with reference summaries, while ROUGE-L measures the longest common subsequence and thus rewards fluency and contextual relevance.

What would settle it

Re-run the four fine-tunings on the CodeXGLUE splits with the paper's hyperparameters and recompute BLEU and ROUGE-L for each language. If the new tables do not reproduce the reported ranking — in particular, if another model equals or beats Mistral on JavaScript and PHP, or if identical scores reappear across models — the "Mistral best overall" conclusion does not survive.

Watch

Extended reading notes

Core claim

The central discovery, stated in the paper's own terms, is that fine-tuning changes the ranking: among the four open LLMs tested, Mistral-7b produces summaries with the highest combined BLEU and ROUGE-L scores across most programming languages, with especially strong results on JavaScript and PHP, while Phi-3-medium is the best on Ruby and Go. The paper concludes that Mistral and Phi-3 are the top performers and that Llama-3-8b and Gemma-7b generally lag behind. Section 4.4 states this verdict explicitly: "Mistral emerges as the best overall performer."

Load-bearing premise

The reported BLEU and ROUGE-L tables are accurate measurements from the fine-tuning runs described, with no duplicated or rounded-away scores, so the ranking they produce is a true reflection of model performance.

Editorial extensions

If this is right

  • For automatic code documentation, Mistral-7b is the paper's recommended default open-source model, with Phi-3-medium as the leading alternative.
  • Mistral's margin is largest on JavaScript and PHP; Phi-3-medium is the model to choose for Ruby and Go.
  • Llama-3-8b and Gemma-7b are not the paper's first choice for summarization, even where their individual scores are respectable.
  • Fine-tuning with a small batch size and a single epoch on CodeXGLUE is sufficient to separate the models, suggesting the ranking can be reproduced on modest hardware.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reader who accepts the ranking should still treat the per-language margins carefully: BLEU and ROUGE-L measure surface overlap, so a model that paraphrases accurately can score lower than one that copies reference phrasing.
  • The same comparison could be extended to code generation, code repair, or human judgment of summary usefulness; the paper's conclusion is specifically about n-gram match on this benchmark.
  • Several reported scores are identical across different models on the same dataset (for example, Phi-3-medium and Llama-3-8b share exact Go scores), so independent re-runs with error bars would confirm whether the ranking's fine distinctions are real.
  • A natural next experiment is to test whether the Mistral advantage persists on project-specific few-shot prompts, which related work suggests can beat fine-tuning for code summarization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports a comparative evaluation of four open-source large language models (Gemma-7b, Phi-3-medium, Llama-3-8b, and Mistral-7b) for code summarization, using BLEU and ROUGE-L scores on the CodeXGLUE benchmark across six programming languages. The central claim, stated in Section 4.4 and the conclusion, is that Mistral-7b is the best overall performer, with Phi-3-medium also strong, and that practitioners should therefore prefer these models for code summarization. The paper includes background on RNN-, GNN-, and Transformer-based methods, a description of the fine-tuning setup (LoRA, SFTTrainer, specific hyperparameters), and per-language result tables. No code, data, or trained models are released, and no statistical significance testing is reported.

Significance. If the reported measurements were trustworthy, the comparison would be of practical interest to software engineers selecting open-source LLMs for code summarization, and the CodeXGLUE-based evaluation framework could serve as a useful reference. The paper's strengths are its timely research question and the sensible choice of well-known open models and a standard benchmark. However, the empirical core is not reproducible: no artifacts are provided, no error bars or significance tests are given, and the reported tables contain exact duplicate scores across distinct models. The central ranking claim is therefore not supported by the evidence presented as it stands.

major comments (4)
  1. [Section 4.2, Tables 4.3–4.8] The central ranking claim rests entirely on the BLEU and ROUGE-L scores in these tables, but the tables contain exact duplicate values across different models and languages. For example, Table 4.3 lists Gemma-7b and Llama-3-8b both at BLEU 7.23 and ROUGE-L 18.95, and Table 4.5 lists Phi-3-medium and Llama-3-8b both at BLEU 1.37 and ROUGE-L 7.69. After independent fine-tuning of different architectures, exact equality on both metrics is implausibly coincidental, and the paper provides no error bars, confidence intervals, or significance tests to address this. These duplicates undermine the trustworthiness of the measurements on which the paper's conclusion depends.
  2. [Sections 3.2 and 4.1.2] The data split is described inconsistently. Section 4.1.2 states that a 70-20-10 train-test-development split was adopted, while Section 3.2 reports the official CodeXGLUE counts (for Python, 251,820 training, 13,914 dev, and 14,918 test examples), which are incompatible with a 70/20/10 partition of that corpus. Because the exact training and test sets determine the validity of all reported scores, this inconsistency makes the experimental setup irreproducible and prevents an independent check of the results.
  3. [Section 4.4 and Table 4.8] The 'Final Verdict' asserts that Mistral emerges as the best overall performer, but the presented tables do not support an unambiguous overall ranking. In Table 4.8 (Ruby), Mistral has the lowest BLEU (2.52) and ROUGE-L (7.74) of the four models, and in Table 4.4 (Java) it is not first on BLEU; the method by which per-language scores are aggregated (e.g., unweighted average, specific weighting, or a qualitative judgment) is never defined. Without such a definition, the central conclusion is not derivable from the reported results.
  4. [Section 4.1.1 and Section 4.1.4] The paper claims to fine-tune 7B–8B parameter models with LoRA on CodeXGLUE-scale data using the free tier of Google Colab with NVIDIA Tesla K80 GPUs, but provides no runtime, memory, or throughput measurements, and no code or data are released. This description makes the experiments difficult to verify or replicate, especially given the scale of the datasets and models; the absence of any implementation artifact is a load-bearing reproducibility gap for an empirical paper.
minor comments (5)
  1. [Section 4.1.3 and Table 4.2] The hyperparameter table does not include the LoRA rank or alpha, even though Section 4.1.4 states that a LoRA adapter is added; the optimizer is described as 'Adam' in Section 4.1.4 but listed as 'adamw_8bit' in Table 4.2.
  2. [Section 3.2, Table 3.1] The counts in Table 3.1 are rounded (e.g., PHP 241k, 14k, 13k) and use 'Validation' while Section 4.1.2 uses 'development'; the text also gives precise counts (251,820, 13,914, 14,918) that differ from the rounded table entries, which is confusing for readers trying to reconstruct the data.
  3. [Abstract and Section 1] The abstract mentions 'BLEU 3.1' and 'ROUGE 3.2' and Section 1 lists F1, precision, and accuracy as metrics, but the experiments report only BLEU and ROUGE-L; this inconsistency between the stated and actual evaluation metrics should be resolved.
  4. [Section 4.5] The Limitation section acknowledges that models may require individualized training strategies and that data quality matters, but it does not mention the lack of code/data release or the absence of statistical tests, which are the most pressing limitations for this study.
  5. [References] Several references are incomplete or contain formatting errors, e.g., reference [2] lacks a year and reference [29] lists the author as 'G.Team'; these should be corrected for consistency.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the Mistral-best claim is a direct empirical reading of reported BLEU/ROUGE-L scores against the external CodeXGLUE benchmark, not a derivation from its own inputs.

full rationale

The paper makes no derivation chain that could reduce to its own inputs. It reports measured BLEU and ROUGE-L scores for four open-source LLMs on the CodeXGLUE benchmark and then ranks the models by those scores. The central claim, that Mistral is the best overall performer, is a direct reading of the reported tables (e.g., Sections 4.2 and 4.4), not a quantity derived from a fitted parameter, a self-citation chain, a uniqueness theorem, or an ansatz smuggled in via citation. The scores are presented as empirical observations against an external benchmark, so there is no fitted input being renamed as a prediction. Concerns that identical scores across different models (e.g., Phi-3-medium and Llama-3-8b both scoring BLEU 1.37 and ROUGE-L 7.69 on Go) indicate copy-pasted or non-genuine measurements are data-integrity and reproducibility concerns, not circularity. Likewise, the inconsistency between the stated 70-20-10 split in Section 4.1.2 and the official CodeXGLUE counts in Section 3.2 is an internal consistency problem, not a circular derivation. The paper's own Section 4.5 lists limitations about training strategies, data scarcity, bias, and model staleness, but none of these assert that the results are predetermined by the evaluation setup. Under the requirement that circularity be exhibited by quoting a specific reduction, no such reduction can be found here. The appropriate finding is therefore no significant circularity.

Assumptions & free parameters 8 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities. It depends on standard benchmark assumptions and a set of manually chosen fine-tuning hyperparameters that are not justified and are partly under-specified (LoRA rank missing).

free parameters (8)
  • learning_rate = 2e-4
    Chosen by hand for all fine-tuning runs; directly affects convergence and final scores.
  • per_device_train_batch_size = 2
    Chosen to fit GPU memory; influences gradient noise and model updates.
  • gradient_accumulation_steps = 4
    Chosen to simulate a larger batch; affects optimization dynamics.
  • num_epochs = 1
    Only one pass over data; may underfit models, especially larger ones.
  • warmup_steps = 5
    Chosen by hand; small number given limited training steps.
  • weight_decay = 0.01
    Standard value chosen by hand.
  • seed = 3407
    Fixed random seed, but no multiple seeds are used to estimate variance.
  • LoRA rank
    Not reported in the paper; required to reproduce the fine-tuning.
assumptions (3)
  • domain assumption CodeXGLUE is a reliable benchmark for code summarization
    The paper uses CodeXGLUE as ground truth without questioning its quality or representativeness.
  • domain assumption BLEU and ROUGE-L are adequate evaluation metrics for summary quality
    The paper does not justify that these metrics reflect human-judged quality for code summaries.
  • domain assumption The fine-tuning and evaluation pipeline is correctly implemented
    The paper provides no artifacts to verify that models were loaded, fine-tuned, and evaluated as intended; the identical scores cast doubt on this premise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Analysis on LLMs Performance for Code Summarization." pith.science (2026). https://pith.science/paper/5S3WQPLE

@misc{pith2026241217094,
  author       = {Pith},
  title        = {Pith review of: Analysis on LLMs Performance for Code Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5S3WQPLE}},
  note         = {Machine review of arXiv:2412.17094}
}
read the original abstract

Code summarization aims to generate concise natural language descriptions for source code. Deep learning has been used more and more recently in software engineering, particularly for tasks like code creation and summarization. Specifically, it appears that the most current Large Language Models with coding perform well on these tasks. Large Language Models (LLMs) have significantly advanced the field of code summarization, providing sophisticated methods for generating concise and accurate summaries of source code. This study aims to perform a comparative analysis of several open-source LLMs, namely LLaMA-3, Phi-3, Mistral, and Gemma. These models' performance is assessed using important metrics such as BLEU\textsubscript{3.1} and ROUGE\textsubscript{3.2}. Through this analysis, we seek to identify the strengths and weaknesses of each model, offering insights into their applicability and effectiveness in code summarization tasks. Our findings contribute to the ongoing development and refinement of LLMs, supporting their integration into tools that enhance software development and maintenance processes.

Figures

Figures reproduced from arXiv: 2412.17094 by the authors.

Figure 1.1
Figure 1.1. A Simple block of code or function 2 [PITH_FULL_IMAGE:figures/full_fig_p012_1_1.png] view at source ↗
Figure 1.2
Figure 1.2. [PITH_FULL_IMAGE:figures/full_fig_p013_1_2.png] view at source ↗
Figure 2.1
Figure 2.1. Illustration of Vanishing and Exploding Gradients in RNNs mary, while Code2Seq represents a significant advancement in source code process￾ing, particularly in the context of seq2seq models, its applicability may be hindered by training complexities, gradient instability, and scalability issues when handling ex￾tensive codebases. Furhtermore, Iyer, Konstas, Cheung, et al. [18] were the first to propose CODE-NN. Subs… view at source ↗
Figures from the paper (10 more)
Figure 2.2
Figure 2.2. Figure 2.2: Preprocessing in GNN The challenge of precisely extracting and encoding the data from the AST into a matrix form while preserving the syntactic and semantic links seen in the source code is most likely the cause of the preprocessing step’s difficulty. A multi-way Tre…
Figure 2.3
Figure 2.3. Figure 2.3: Transformer-based Architecture The model may unnecessarily attend to the same pieces of information more than once due to redundancy in the attention mechanisms. Despite the Transformer’s many advantages, it is still difficult to incorporate code structure informatio…
Figure 2
Figure 2. Figure 2: (a) and its corresponding AST in Figure 2.4(b), showing how the method, [PITH_FULL_IMAGE:figures/full_fig_p021_2.png]
Figure 2.4
Figure 2.4. Figure 2.4: Representation of code snippet and its Abstract Syntax Tree are logically and syntactically organized. One drawback of the approach is that, in comparison to other models, it performs worse on Python datasets [PITH_FULL_IMAGE:figures/full_fig_p022_2_4.png]
Figure 2.5
Figure 2.5. Figure 2.5: Performance of AST Transformer in Python and Java dataset This could mean that, in comparison to how it handles other languages like Java, the method performs less well when dealing with the syntax or coding patterns of Python. The histogram in [PITH_FULL_IMAGE:figu…
Figure 2.6
Figure 2.6. Figure 2.6: The encoder takes a graph 𝐺 as input and is composed of several GN￾Transformer blocks. "+" is what is referred to as a residual connection that comes after a normalizing layer. The changed node properties are output in graph 𝐺 ′ by the encoder. Only the token nodes (…
Figure 2.7
Figure 2.7. Figure 2.7: The description and the first line of code are tokenized by the Llama 2 to￾kenizer. This provides helpful information by displaying the tokens that are included in the description. In 2024, Guo et al. created GraphcodeBERT, a new way to understand semantic code that …
Figure 3.1
Figure 3.1. Figure 3.1: Proposed Methodology For investigating the answers we have employed a robust and structured methodology for our research. The overall pipeline of the methodology is illustrated in the figure 3.1 16 [PITH_FULL_IMAGE:figures/full_fig_p026_3_1.png]
Figure 4.1
Figure 4.1. Figure 4.1: Visualization for LLMs Performance in BLEU scores ROUGE-L Scores of Models Across Different Datasets The ROUGE-L scores for the same models over the same datasets are displayed in the chart 4.2, which measures the longest common subsequence (LCS) between generated te…
Figure 4.2
Figure 4.2. Figure 4.2: Visualization for LLMs Performance in ROUGE-L scores 4.4 Final Verdict In our comparative analysis of large language models for code summarization, Mistral emerges as the best overall performer, demonstrating high scores across most datasets, with exceptional perform…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 18 canonical work pages

  1. [1]

    Phi-3technicalreport:Ahighlyca- pablelanguagemodellocallyonyourphone,

    M.Abdin,S.A.Jacobs,A.A.Awan, et al.,“Phi-3technicalreport:Ahighlyca- pablelanguagemodellocallyonyourphone,” arXiv preprint arXiv:2404.14219, 2024

    arXiv 2024

  2. [2]

    Gpt-4 technical report,(2023),

    O. Achiam, S. Adler, S. Agarwal,et al., “Gpt-4 technical report,(2023),”URL https://api. semanticscholar. org/CorpusID,vol.257532815,

    work page 2023

  3. [3]

    Few-shot training llms for project-specific code- summarization,

    T. Ahmed and P. Devanbu, “Few-shot training llms for project-specific code- summarization,”in 37th IEEE/ACM International Conference on Automated Soft- ware Engineering (ASE ’22),ACM,Rochester,MI,USA,2022,pp.1–5. doi: 10. 1145/3551349.3559555

    arXiv 2022

  4. [4]

    Llama3modelcard,

    AI@Meta,“Llama3modelcard,”2024.[Online].Available: https://github. com/meta-llama/llama3/blob/main/MODEL_CARD.md

    work page 2024

  5. [5]

    Code2seq: Generating sequences fromstructuredrepresentationsofcode,

    U. Alon, S. Brody, O. Levy, and E. Yahav, “Code2seq: Generating sequences fromstructuredrepresentationsofcode,” arXiv preprint arXiv:1808.01400,2018

    arXiv 2018

  6. [6]

    A parallel corpus of python functions and documentation strings for automated code documentation and code genera- tion,

    A. V. M. Barone and R. Sennrich, “A parallel corpus of python functions and documentation strings for automated code documentation and code genera- tion,” arXiv preprint arXiv:1707.02275,2017

    arXiv 2017

  7. [7]

    Gn-transformer: Fusing sequence and graph representation for improved code summarization,

    J. Cheng, I. Fostiropoulos, and B. Boehm, “Gn-transformer: Fusing sequence and graph representation for improved code summarization,”arXiv preprint arXiv:2111.08874,2021

    arXiv 2021

  8. [8]

    Learning phrase represen- tations using rnn encoder-decoder for statistical machine translation,

    K. Cho, B. Van Merriënboer, C. Gulcehre,et al., “Learning phrase represen- tations using rnn encoder-decoder for statistical machine translation,”arXiv preprint arXiv:1406.1078,2014

    arXiv 2014

Show all 34 references
  1. [9]

    Structuredneuralsumma- rization,

    P.Fernandes,M.Allamanis,andM.Brockschmidt,“Structuredneuralsumma- rization,” arXiv preprint arXiv:1811.01824,2018

    2018 arXiv

  2. [10]

    Code structure guided transformerforsourcecodesummarization.corrabs/2104.09340(2021),

    S. Gao, C. Gao, Y. He, J. Zeng, L. Y. Nie, and X. Xia, “Code structure guided transformerforsourcecodesummarization.corrabs/2104.09340(2021),” arXiv preprint arXiv:2104.09340,2021. 32

    2021 arXiv

  3. [11]

    M2ts: Multi-scale multi-modal approach based on trans- former for source code summarization,

    Y. Gao and C. Lyu, “M2ts: Multi-scale multi-modal approach based on trans- former for source code summarization,” inProceedings of the 30th IEEE/ACM International Conference on Program Comprehension,2022,pp.24–35

    2022

  4. [12]

    Graphcodebert:Pre-trainingcoderepresentationswithdataflow,

    D.Guo et al.,“Graphcodebert:Pre-trainingcoderepresentationswithdataflow,” CoRR,vol.abs/2009.08366,2024

    2009 arXiv

  5. [13]

    Analyzing the performance of large language modelsoncodesummarization,

    R. Haldar and J. Hockenmaier, “Analyzing the performance of large language modelsoncodesummarization,” University of Illinois Urbana-Champaign,2024

    2024

  6. [14]

    Measuringcodingchallengecom- petencewithapps,

    D.Hendrycks,S.Basart,S.Kadavath, et al.,“Measuringcodingchallengecom- petencewithapps,” arXiv preprint arXiv:2105.09938,2021

    2021 arXiv

  7. [15]

    Deep code comment generation,

    X. Hu, G. Li, X. Xia, D. Lo, and Z. Jin, “Deep code comment generation,” in Proceedings of the 26th conference on program comprehension , 2018, pp. 200– 210

    2018

  8. [16]

    Summarizing source code with transferredapiknowledge,

    X. Hu, G. Li, X. Xia, D. Lo, S. Lu, and Z. Jin, “Summarizing source code with transferredapiknowledge,”2018

    2018

  9. [17]

    Code- searchnetchallenge:Evaluatingthestateofsemanticcodesearch,

    H. Husain, H.-H. Wu, T. Gazit, M. Allamanis, and M. Brockschmidt, “Code- searchnetchallenge:Evaluatingthestateofsemanticcodesearch,” arXiv preprint arXiv:1909.09436,2019

    1909 arXiv

  10. [18]

    Summarizing source code using a neural attention model,

    S. Iyer, I. Konstas, A. Cheung, and L. Zettlemoyer, “Summarizing source code using a neural attention model,” in54th Annual Meeting of the Association for Computational Linguistics 2016,AssociationforComputationalLinguistics,2016, pp.2073–2083

    2016

  11. [19]

    Mistral7b,

    A.Q.Jiang,A.Sablayrolles,A.Mensch, et al.,“Mistral7b,” arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  12. [20]

    Improvedcodesummarization viaagraphneuralnetwork,

    A.LeClair,S.Haque,L.Wu,andC.McMillan,“Improvedcodesummarization viaagraphneuralnetwork,”in Proceedings of the 28th international conference on program comprehension,2020,pp.184–195

    2020

  13. [21]

    A neural model for generating natural languagesummariesofprogramsubroutines,

    A. LeClair, S. Jiang, and C. McMillan, “A neural model for generating natural languagesummariesofprogramsubroutines,”in 2019 IEEE/ACM 41st Interna- tional Conference on Software Engineering (ICSE),IEEE,2019,pp.795–806

    2019

  14. [22]

    Retrieval-augmentedgenerationfor codesummarizationviahybridgnn,

    S.Liu,Y.Chen,X.Xie,J.Siow,andY.Liu,“Retrieval-augmentedgenerationfor codesummarizationviahybridgnn,” arXiv preprint arXiv:2006.05405,2020

    2006 arXiv

  15. [23]

    Codexglue:Amachinelearningbenchmarkdataset forcodeunderstandingandgeneration,

    S.Lu,D.Guo,S.Ren, et al.,“Codexglue:Amachinelearningbenchmarkdataset forcodeunderstandingandgeneration,” arXiv preprint arXiv:2102.04664,2021. 33

    2021 arXiv

  16. [24]

    Codegen:Anopenlargelanguagemodelforcodewithmulti- turnprogramsynthesis,

    E.Nijkamp et al.,“Codegen:Anopenlargelanguagemodelforcodewithmulti- turnprogramsynthesis,”in ICLR,2024

    2024

  17. [25]

    Codellama:Openfoundationmodels forcode,

    B.Roziere,J.Gehring,F.Gloeckle, et al.,“Codellama:Openfoundationmodels forcode,” arXiv preprint arXiv:2308.12950,2023

    2023 arXiv

  18. [26]

    Au- tomaticsourcecodesummarizationwithextendedtree-lstm,

    Y. Shido, Y. Kobayashi, A. Yamamoto, A. Miyamoto, and T. Matsumura, “Au- tomaticsourcecodesummarizationwithextendedtree-lstm,”in 2019 Interna- tional Joint Conference on Neural Networks (IJCNN),IEEE,2019,pp.1–8

    2019

  19. [27]

    Sequencetosequencelearningwithneu- ralnetworks,

    I.Sutskever,O.Vinyals,andQ.V.Le,“Sequencetosequencelearningwithneu- ralnetworks,” Advances in neural information processing systems,vol.27,2014

    2014

  20. [28]

    Ast-trans: Code summarization with efficient tree-structured attention,

    Z. Tang, X. Shen, C. Li,et al., “Ast-trans: Code summarization with efficient tree-structured attention,” inProceedings of the 44th International Conference on Software Engineering,2022,pp.150–162

    2022

  21. [29]

    Gemini:Afamilyofhighlycapablemul- timodalmodels,

    G.Team,R.Anil,S.Borgeaud, et al.,“Gemini:Afamilyofhighlycapablemul- timodalmodels,” arXiv preprint arXiv:2312.11805,2023

    2023 arXiv

  22. [30]

    Gemma:Openmodelsbasedongemini researchandtechnology,

    G.Team,T.Mesnard,C.Hardin, et al.,“Gemma:Openmodelsbasedongemini researchandtechnology,” arXiv preprint arXiv:2403.08295,2024

    2024 arXiv

  23. [31]

    Attentionisallyouneed,

    A.Vaswani,N.Shazeer,N.Parmar, et al.,“Attentionisallyouneed,” Advances in neural information processing systems,vol.30,2017

    2017

  24. [32]

    Improving automatic source code summa- rizationviadeepreinforcementlearning,

    Y. Wan, Z. Zhao, M. Yang,et al., “Improving automatic source code summa- rizationviadeepreinforcementlearning,”in Proceedings of the 33rd ACM/IEEE international conference on automated software engineering,2018,pp.397–407

    2018

  25. [33]

    Asurveyofautomaticsourcecodesumma- rization,

    C.Zhang,J.Wang,Q.Zhou, et al.,“Asurveyofautomaticsourcecodesumma- rization,” Symmetry,vol.14,no.3,p.471,2022

    2022

  26. [34]

    Retrieval-basedneuralsource codesummarization,

    J.Zhang,X.Wang,H.Zhang,H.Sun,andX.Liu,“Retrieval-basedneuralsource codesummarization,”in Proceedings of the ACM/IEEE 42nd International Con- ference on Software Engineering,2020,pp.1385–1397. 34

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.