When Retriever Meets Generator: A Joint Model for Code Comment Generation

Alessio Bucaioni; Anh M. T. Bui; Huy N. D. Pham; Phuong T. Nguyen; Tien P. T. Le

REVIEW 4 major objections 5 minor 35 references

When Retriever Meets Generator: A Joint Model for Code Comment Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that fusing retrieval and generation into a single CodeT5 encoder-decoder, trained with contrastive retrieval pre-training, joint fine-tuning, and a self-refinement loop, outperforms three established baselines on Java…

desk verdict A coherent incremental RAG system for code comment generation with a solid replication package, but the SOTA claim rests on point estimates without error bars and a stacked LLM baseline. read the letter →

arxiv 2507.12558 v2 pith:MHLGVDR7 submitted 2025-07-16 cs.SE

Tien P. T. Le , Anh M. T. Bui , Huy N. D. Pham , Alessio Bucaioni , Phuong T. Nguyen This is my paper

classification cs.SE

keywords codecommentgenerationretrieval-augmentedjointretriever-generatortrainingcontrastivelearningT5summarizationself-refinement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RAGSum is a code-comment generator built on one CodeT5 backbone, a pre-trained encoder-decoder for code, so that retrieval and generation are trained as a single system rather than as separate stages. The recipe has three parts: contrastive pre-training shapes the encoder's embeddings for nearest-neighbor search; joint fine-tuning weights each retrieved exemplar's generation loss by its retrieval similarity, so the retriever learns from how well the generator does; and a self-refinement loop re-trains the model on its own most ROUGE-L-faithful outputs. In preliminary results on Java, Python, and C benchmarks, the paper reports that RAGSum beats CMR-Sum, JOINTCOM, and a Llama-3.1-8B-based RAG baseline on every metric considered. The paper argues that this points to a cheaper route to better documentation automation: tighter coupling of retrieval and generation instead of larger models.

What carries the argument

The central object is a single CodeT5 encoder-decoder whose encoder doubles as the retriever and whose decoder acts as the generator. The load-bearing mechanism is the joint fine-tuning loss: for each code query, the retriever returns the top-$k$ code-comment pairs, the decoder computes a cross-entropy loss for each augmented input, and the per-exemplar losses are averaged with weights $\nu_j = sim(q_i, q_j^r)$, the cosine similarity between the query and the retrieved code. That weighting lets generation feedback shape retrieval. Before that, a contrastive pre-training phase aligns code and comment embeddings with in-batch negatives; after that, a self-refinement loop fine-tunes the model on its own highest-scoring candidates, selected by ROUGE-L against the reference comment.

What would settle it

Retrain the baselines with the same CodeT5 backbone, the same retrieval corpus, and the same number of fine-tuning epochs as RAGSum, without freezing the LLM generator, and compare on identical test splits; if the BLEU and ROUGE-L gap shrinks to noise or reverses, the claimed advantage is training budget rather than joint modeling.

Watch

Extended reading notes

Core claim

The central claim is that retrieval and generation for code comments should not be optimized in isolation: when the retriever is trained through the generator's loss, it stops fetching irrelevant neighbors, and the decoder conditions on exemplars that actually help. RAGSum realizes this by pre-training the CodeT5 encoder with code-to-code and code-to-comment contrastive losses, joint fine-tuning encoder and decoder with a similarity-weighted composite loss, and refining the model on self-generated candidates scored by ROUGE-L. The paper reports that this achieves the best results among the compared methods on all three datasets, with the clearest margin on Python, where corpus-BLEU rises to 33.0 against 28.89 for CMR-Sum, and with smaller but consistent gains on Java and C.

Load-bearing premise

The central comparison assumes the reproduced baselines—CMR-Sum, JOINTCOM, and Llama-3.1-8B in a RAG setup—are configured at a fair, standard setting, so the reported gains come from joint fine-tuning rather than from unequal training effort or frozen baseline weights.

Editorial extensions

If this is right

On the JCSD, PCSD, and CCSD benchmarks, RAGSum outperforms CMR-Sum, JOINTCOM, and Llama-3.1-8B with RAG prompting across corpus-BLEU, sentence-BLEU, ROUGE-L, METEOR, and CIDEr.
Joint fine-tuning is the main driver of the gain: ablating it lowers corpus-BLEU by 13.07% on Java, 15.02% on Python, and 20.99% on C relative to the full model.
Contrastive encoder pre-training and self-refinement each add measurable improvements; on PCSD, removing self-refinement drops corpus-BLEU from 33.0 to 32.57.
At inference, RAGSum needs only the single highest-scoring retrieved exemplar, so the joint model adds accuracy without a heavy retrieval-time ensemble.
The best number of retrieved exemplars for training is dataset-dependent: $k=4$ for Java and $k=3$ for Python and C.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same three-phase recipe—contrastive retriever pre-training, similarity-weighted joint fine-tuning, and self-refinement—could transfer to other retrieval-plus-generation tasks such as commit message generation, API documentation, or code search, since none of the components is comment-specific.
Editorial inference: the self-refinement step selects candidates by ROUGE-L against the ground-truth comment during training; a deployment variant would need a learned reward model or reference-free scorer, and whether the gain survives that substitution is untested here.
Editorial inference: because the optimal $k$ varies across languages, an adaptive retrieval-width controller could squeeze out further gains; the paper does not propose one.
Editorial inference: comparing RAGSum against a fully fine-tuned instruction-tuned model of similar parameter count would clarify whether the reported edge comes from joint modeling or from the baselines' frozen weights.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a decent empirical paper that combines known pieces — contrastive retriever pre-training, joint fine-tuning of retriever and generator on one CodeT5 backbone, and a self-refinement pass — and reports consistent gains on three standard benchmarks (JCSD, PCSD, CCSD). The combination is new enough, and the ablation study cleanly shows that each component contributes, with joint fine-tuning carrying most of the weight. The replication package and use of the standard datasets are real pluses.

Where it gets soft: the central claim of a new SOTA is not actually supported by the numbers as reported. Table II gives single point estimates with no error bars, no repeated seeds, no significance tests. On CCSD, the decisive comparison against JOINTCOM is 27.95 vs 26.32 C-BLEU, 47.35 vs 46.15 ROUGE-L — margins that could easily be run-to-run noise. The abstract calls the results preliminary, and Section V-B then says 'substantially outperforms' and 'new state-of-the-art.' Those claims need to be scaled back or backed with variance information. The Llama-3.1-8B baseline is also not a fair opponent: it is a frozen generator in the RAG framework while RAGSum is fully fine-tuned, so the headline comparison against it mostly shows that fine-tuning helps. There is also a plain typo in Section III-A where the q2c contrastive loss is described as 'minimize the similarity' between code and its comment, when the formula does the opposite; that needs fixing, as does 'ROUTE-L' in the abstract.

None of this is fatal. The core idea is sensible and the engineering appears to be honest. I just would not let the 'new state-of-the-art' sentence survive. My recommendation for peer review: send it out. A serious referee can usefully push on the evaluation rigor, and the paper is short enough and the replication package concrete enough that a revision cycle could make it work. I would want to see either error bars or repeated-seed runs, a descoped claim, and an LLM baseline that gets the same training budget or is explicitly positioned as a lower bound.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes RAGSum, a code-comment generation model that couples retrieval and generation in a single CodeT5 backbone. The approach has three stages: contrastive pretraining of the encoder for code-code and code-comment retrieval, joint fine-tuning of the retriever and generator with a weighted generation loss, and a self-refinement loop that augments training data with model-generated comments selected by ROUGE-L against ground truth. The authors evaluate on JCSD, PCSD, and CCSD, comparing against CMR-Sum, JOINTCOM, and Llama-3.1-8B in a RAG setting, and report improvements on BLEU, ROUGE-L, METEOR, and CIDEr, with ablations showing the contribution of each component. The paper is explicitly framed as preliminary but also claims to set a new state of the art in comment generation.

Significance. If the reported gains are reproducible, RAGSum would be a useful contribution to retrieval-augmented code summarization: unifying retriever and generator in one backbone is a natural design, the contrastive pretraining objective is reasonable, and the authors provide a replication package. The significance is moderated, however, by the absence of any uncertainty quantification, the small margins over the strongest fine-tuned baseline on CCSD, and an unbalanced comparison with the frozen Llama baseline. These issues mean the central state-of-the-art claim is not yet established, although the underlying architecture and evaluation framework are sound enough to warrant further work.

major comments (4)

[Section V-B, Table II] The central claim that RAGSum substantially outperforms the baselines rests on single point estimates with no error bars, no repeated seeds, and no significance testing. On CCSD, the margin over JOINTCOM is C-BLEU 27.95 vs. 26.32 and ROUGE-L 47.35 vs. 46.15, which could plausibly fall within run-to-run variation for transformer fine-tuning. The abstract and the answer to RQ2 use the word 'substantially' even though the paper elsewhere calls the results preliminary. Please report at least 3-5 independent runs and include paired significance tests (e.g., bootstrap or Wilcoxon) over the test set, at least for the decisive RAGSum vs. JOINTCOM comparison.
[Section IV-A, Table II] The Llama-3.1-8B baselines are not trained on the benchmark data: the model is used as a frozen generator with CodeT5 embeddings for retrieval, while RAGSum is fine-tuned on the full training set. This is an unequal comparison, and the resulting gap does not establish that RAGSum is superior to an equivalently trained LLM. The paper should either fine-tune or adapt the LLM baseline under the same training budget, or explicitly relabel these rows as zero-shot/few-shot upper-bound references and exclude them from the state-of-the-art comparison.
[Section III-B, Eq. (3); also Abstract] The abstract states that the composite objective 'rewards accurate top-k retrieval,' but the joint loss in Eq. (3) is a generation cross-entropy loss weighted by retrieval similarity; there is no retrieval-ranking term in the joint phase. Retrieval-specific learning appears only in the contrastive pretraining of Eqs. (1)-(2). This discrepancy is load-bearing for the claimed tight coupling of retriever and generator, and the paper should either add an explicit retrieval loss to the joint objective or revise the claim.
[Section III-C and Table III] The self-refinement process is under-specified and its reported contribution is small: Table III shows that removing self-refinement changes C-BLEU by 0.10-0.43 points across datasets. The number of candidates K, the sampling/decoding strategy, and the temperature during candidate generation are not given, which limits reproducibility. Also, because the selection criterion is ROUGE-L against the ground-truth comment on the training set, the practice is not circular, but the authors should clarify that this is a self-training procedure and report the variance of the refinement gain, as the observed differences may not be significant.

minor comments (5)

[Abstract] The abstract contains the typo 'ROUTE-L' and should read 'ROUGE-L'; also, 'substantially outperforms' is inconsistent with the paper's own characterization of the results as preliminary.
[Section III-A, Eq. (2)] The text says L_q2c 'aims to minimize the similarity between the input code q_i and its corresponding comment c_i,' but the equation maximizes this similarity; the wording should be corrected.
[Section IV-D] The implementation details do not state the number of retrieved exemplars k used in the main experiments; the k-sensitivity analysis in Figure 4 reports optimal values per dataset, so the paper should explicitly state which k was used to produce Table II.
[Table III] The ablation labels are hard to parse: 'RAGSum w/o pretrained + SR' and 'RAGSum w/o combined' should be explicitly defined in the text so that readers know exactly which components are removed in each row.
[Section V-B and V-C] The narrative says the C-dataset performance 'remains competitive' while the numbers are described as gains; consider aligning the wording with the actual magnitude of the differences, especially given the lack of significance testing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RAGSum's components are trained against ground-truth comments and external benchmarks, and no load-bearing claim reduces to its own inputs.

full rationale

The derivation chain is self-contained. Contrastive pretraining (Eqs. 1-2) shapes embeddings using in-batch negatives on training code-comment pairs; joint fine-tuning (Eq. 3) weights generation loss by cosine similarity to retrieved exemplars; self-refinement selects candidates by ROUGE-L against the training reference and fine-tunes on the selected outputs. None of these steps defines the target metric in terms of the model output: the final evaluation is against held-out ground-truth comments on JCSD, PCSD, and CCSD, and the baselines are external (JOINTCOM, CMR-Sum, Llama-3.1-8B). The self-refinement loop is a standard self-training / metric-guided selection procedure, not a circular definition, since the selection signal is the ground-truth reference and the training objective remains cross-entropy. There are no load-bearing self-citations: reference [28] on self-improvement is not authored by the current paper's authors, and Reference [17] is a replication-package link, not a cited result used to justify the central claim. The paper's own caveats—'preliminary results' in the abstract and the Section V-E note that automatic metrics 'may not fully capture semantic equivalence'—are statistical-validity concerns (no error bars or significance tests), not evidence of circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method introduces no new theoretical entities; it relies on standard assumptions about pre-trained models and evaluation metrics.

free parameters (4)

top-k retrieved exemplars = k=4 for JCSD; k=3 for PCSD and CCSD
Tuned per dataset and shown in Figure 4; the model's final performance depends on this choice.
temperature tau = 0.2
Set for contrastive losses; standard hyperparameter but affects embedding separation.
learning rates = 5e-5 fine-tuning, 1e-5 self-improvement
Chosen without reported search; affects convergence.
epoch counts = 1 pretraining, 10 fine-tuning, 5 self-refinement
Manual settings that influence final model quality.

assumptions (3)

domain assumption CodeT5-base provides suitable shared representations for both retrieval and generation.
The entire method is built on the pre-trained CodeT5 backbone; no comparison with other backbones is reported.
domain assumption Retrieval of similar code-comment pairs from the training set improves comment generation.
This motivates the whole RAG design; the paper does not test retrieval with out-of-domain or larger pools.
domain assumption BLEU, ROUGE-L, METEOR, and CIDEr are valid proxies for comment quality.
The evaluation relies entirely on these metrics; the threats-to-validity section admits they may lack semantic equivalence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of When Retriever Meets Generator: A Joint Model for Code Comment Generation." pith.science (2026). https://pith.science/paper/MHLGVDR7

@misc{pith2026250712558,
  author       = {Pith},
  title        = {Pith review of: When Retriever Meets Generator: A Joint Model for Code Comment Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MHLGVDR7}},
  note         = {Machine review of arXiv:2507.12558}
}

read the original abstract

Automatically generating concise, informative comments for source code can lighten documentation effort and accelerate program comprehension. Retrieval-augmented approaches first fetch code snippets with existing comments and then synthesize a new comment, yet retrieval and generation are typically optimized in isolation, allowing irrelevant neighbors topropagate noise downstream. To tackle the issue, we propose a novel approach named RAGSum with the aim of both effectiveness and efficiency in recommendations. RAGSum is built on top offuse retrieval and generation using a single CodeT5 backbone. We report preliminary results on a unified retrieval-generation framework built on CodeT5. A contrastive pre-training phase shapes code embeddings for nearest-neighbor search; these weights then seed end-to-end training with a composite loss that (i) rewards accurate top-k retrieval; and (ii) minimizes comment-generation error. More importantly, a lightweight self-refinement loop is deployed to polish the final output. We evaluated theframework on three cross-language benchmarks (Java, Python, C), and compared it with three well-established baselines. The results show that our approach substantially outperforms thebaselines with respect to BLEU, METEOR, and ROUTE-L. These findings indicate that tightly coupling retrieval and generationcan raise the ceiling for comment automation and motivateforthcoming replications and qualitative developer studies.

Figures

Figures reproduced from arXiv: 2507.12558 by the authors.

**Figure 1.** Example of retrieved comments by CMR-Sum and JOINTCOM. self-refinement loop further polishes the output. To study RAGSum, we evaluated it on three cross-language benchmarks, i.e., Java, Python, C and compared it with three well-established baselines, i.e., CMR-Sum [14], JOINTCOM [13], and LLama-3.1-8B [16]. The experimental results showed that RAGSum gains significant improvements with respect to the baselines. The… view at source ↗

**Figure 2.** The overall architecture of RAGSum. twice to produce two representation vectors, qi and q + i . The model is trained to minimize the distance between these two representations while maximizing the distance to other code queries in the batch B, which serve as negative samples (q − i ). sim(·) denotes the cosine similarity of two vectors. τ is the temperature from contrastive learning [11]. The second loss function, L… view at source ↗

**Figure 3.** Distribution of Retrieved Comments and Targets Across Methods. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Top-k Impact Scores [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 6.** Figure 6: Example of Python code. to evaluate generated summaries from additional perspectives, such as human evaluation [35]. ▷ External validity. Potential threat to validity lies in the variation of results and performance of our approach with different coding styles, program…

**Figure 5.** Figure 5: Example of Java code. This highlights the impact of relevant code comment on the quality of generated comment [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 21 canonical work pages

[1]

Exploiting method names to improve code summarization: A deliberation multi-task learning ap- proach,

R. Xie, W. Ye, J. Sun, and S. Zhang, “Exploiting method names to improve code summarization: A deliberation multi-task learning ap- proach,” in 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC). IEEE, 2021, pp. 138–148

work page 2021
[2]

Few-shot training llms for project-specific code-summarization,

T. Ahmed and P. Devanbu, “Few-shot training llms for project-specific code-summarization,” in Proceedings of the 37th IEEE/ACM interna- tional conference on automated software engineering , 2022, pp. 1–5

work page 2022
[3]

Towards automatically generating summary comments for java meth- ods,

G. Sridhara, E. Hill, D. Muppaneni, L. Pollock, and K. Vijay-Shanker, “Towards automatically generating summary comments for java meth- ods,” in Proceedings of the 25th IEEE/ACM international conference on Automated software engineering , 2010, pp. 43–52

work page 2010
[4]

Automatic source code summa- rization of context for java methods,

P. W. McBurney and C. McMillan, “Automatic source code summa- rization of context for java methods,” IEEE Transactions on Software Engineering, vol. 42, no. 2, pp. 103–119, 2015

work page 2015
[5]

Autocomment: Mining question and an- swer sites for automatic comment generation,

E. Wong, J. Yang, and L. Tan, “Autocomment: Mining question and an- swer sites for automatic comment generation,” in 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 2013, pp. 562–567

work page 2013
[6]

Clocom: Mining existing source code for automatic comment generation,

E. Wong, T. Liu, and L. Tan, “Clocom: Mining existing source code for automatic comment generation,” in 2015 IEEE 22nd International conference on software analysis, evolution, and reengineering (SANER). IEEE, 2015, pp. 380–389

work page 2015
[7]

Summarizing source code using a neural attention model,

S. Iyer, I. Konstas, A. Cheung, and L. Zettlemoyer, “Summarizing source code using a neural attention model,” in 54th Annual Meeting of the Association for Computational Linguistics 2016 . Association for Computational Linguistics, 2016, pp. 2073–2083

work page 2016
[8]

Deep code comment genera- tion,

X. Hu, G. Li, X. Xia, D. Lo, and Z. Jin, “Deep code comment genera- tion,” in Proceedings of the 26th conference on program comprehension, 2018, pp. 200–210

work page 2018

Show all 35 references

[9]

A neural model for gener- ating natural language summaries of program subroutines,

A. LeClair, S. Jiang, and C. McMillan, “A neural model for gener- ating natural language summaries of program subroutines,” in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 2019, pp. 795–806

2019
[10]

Leveraging code generation to improve code retrieval and summarization via dual learning,

W. Ye, R. Xie, J. Zhang, T. Hu, X. Wang, and S. Zhang, “Leveraging code generation to improve code retrieval and summarization via dual learning,” in Proceedings of The Web Conference 2020, 2020, pp. 2309– 2319

2020
[11]

Retrieval augmented code generation and summarization,

M. R. Parvez, W. Ahmad, S. Chakraborty, B. Ray, and K.-W. Chang, “Retrieval augmented code generation and summarization,” in Findings of the Association for Computational Linguistics: EMNLP 2021 , 2021, pp. 2719–2734

2021
[12]

Editsum: A retrieve-and-edit framework for source code summarization,

J. A. Li, Y . Li, G. Li, X. Hu, X. Xia, and Z. Jin, “Editsum: A retrieve-and-edit framework for source code summarization,” in 2021 36th IEEE/ACM International Conference on Automated Software En- gineering (ASE). IEEE, 2021, pp. 155–166

2021
[13]

Improving retrieval-augmented code comment generation by retrieving for generation,

H. Lu and Z. Liu, “Improving retrieval-augmented code comment generation by retrieving for generation,” in 2024 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 2024, pp. 350–362

2024
[14]

Cross-modal retrieval-enhanced code summarization based on joint learning for retrieval and generation,

L. Li, B. Liang, L. Chen, and X. Zhang, “Cross-modal retrieval-enhanced code summarization based on joint learning for retrieval and generation,” Information and Software Technology , vol. 175, p. 107527, 2024

2024
[15]

Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,

Y . Wang, W. Wang, S. Joty, and S. C. Hoi, “Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021, pp. 8696–8708

2021
[16]

The llama 3 herd of models,

A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan et al. , “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783 , 2024

2024 arXiv
[17]

When Retriever Meets Generator: A Joint Model for Code Comment Generation

A. Authors, “When Retriever Meets Generator: A Joint Model for Code Comment Generation.” Figshare, May 2025. [Online]. Available: https://figshare.com/s/9654cd9dfabe9332b0e3

2025
[18]

Improving automatic source code summarization via deep reinforce- ment learning,

Y . Wan, Z. Zhao, M. Yang, G. Xu, H. Ying, J. Wu, and P. S. Yu, “Improving automatic source code summarization via deep reinforce- ment learning,” in Proceedings of the 33rd ACM/IEEE international conference on automated software engineering , 2018, pp. 397–407

2018
[19]

Codebert: A pre-trained model for programming and natural languages,

Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang et al., “Codebert: A pre-trained model for programming and natural languages,” arXiv preprint arXiv:2002.08155 , 2020

2002 arXiv
[20]

Graphcodebert: Pre-training code repre- sentations with data flow,

D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. Liu, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu et al., “Graphcodebert: Pre-training code repre- sentations with data flow,” arXiv preprint arXiv:2009.08366 , 2020

2009 arXiv
[21]

CAST: Enhancing code summarization with hierarchical splitting and reconstruction of abstract syntax trees,

E. Shi, Y . Wang, L. Du, H. Zhang, S. Han, D. Zhang, and H. Sun, “CAST: Enhancing code summarization with hierarchical splitting and reconstruction of abstract syntax trees,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, M.-F. Moens...

2021
[22]

Retrieval-based neural source code summarization,

J. Zhang, X. Wang, H. Zhang, H. Sun, and X. Liu, “Retrieval-based neural source code summarization,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering , 2020, pp. 1385–1397

2020
[23]

Automatic comment generation via multi-pass deliberation,

F. Mu, X. Chen, L. Shi, S. Wang, and Q. Wang, “Automatic comment generation via multi-pass deliberation,” in Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineer- ing, 2022, pp. 1–12

2022
[24]

Source code summarization in the era of large language models,

W. Sun, Y . Miao, Y . Li, H. Zhang, C. Fang, Y . Liu, G. Deng, Y . Liu, and Z. Chen, “Source code summarization in the era of large language models,” arXiv preprint arXiv:2407.07959 , 2024

2024 arXiv
[25]

Automatic code summarization via chatgpt: How far are we?

W. Sun, C. Fang, Y . You, Y . Miao, Y . Liu, Y . Li, G. Deng, S. Huang, Y . Chen, Q. Zhang et al., “Automatic code summarization via chatgpt: How far are we?” arXiv preprint arXiv:2305.12865 , 2023

2023 arXiv
[26]

Cocosoda: Effective contrastive learning for code search,

E. Shi, Y . Wang, W. Gu, L. Du, H. Zhang, S. Han, D. Zhang, and H. Sun, “Cocosoda: Effective contrastive learning for code search,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2023, pp. 2198–2210

2023
[27]

Retrieval-augmented generation for code summarization via hybrid gnn,

S. Liu, Y . Chen, X. Xie, J. Siow, and Y . Liu, “Retrieval-augmented generation for code summarization via hybrid gnn,” arXiv preprint arXiv:2006.05405, 2020

2006 arXiv
[28]

Better language models of code through self-improvement,

H. Q. To, N. D. Q. Bui, J. Guo, and T. N. Nguyen, “Better language models of code through self-improvement,” 2023

2023
[29]

A parallel corpus of python functions and documentation strings for automated code documentation and code generation,

A. V . M. Barone and R. Sennrich, “A parallel corpus of python functions and documentation strings for automated code documentation and code generation,” arXiv preprint arXiv:1707.02275 , 2017

2017 arXiv
[30]

M2ts: Multi-scale multi-modal approach based on transformer for source code summarization,

Y . Gao and C. Lyu, “M2ts: Multi-scale multi-modal approach based on transformer for source code summarization,” in Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension, 2022, pp. 24–35

2022
[31]

Bleu: a method for automatic evaluation of machine translation,

K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318

2002
[32]

Rouge: A package for automatic evaluation of summaries,

C.-Y . Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out , 2004, pp. 74–81

2004
[33]

Meteor: An automatic metric for mt evalua- tion with improved correlation with human judgments,

S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evalua- tion with improved correlation with human judgments,” in Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization , 2005, pp. 65–72

2005
[34]

Cider: Consensus- based image description evaluation,

R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “Cider: Consensus- based image description evaluation,” in Proceedings of the IEEE confer- ence on computer vision and pattern recognition , 2015, pp. 4566–4575

2015
[35]

Deep code comment generation with hybrid lexical and syntactical information,

X. Hu, G. Li, X. Xia, D. Lo, and Z. Jin, “Deep code comment generation with hybrid lexical and syntactical information,” Empirical Software Engineering, vol. 25, pp. 2179–2217, 2020

2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Exploiting method names to improve code summarization: A deliberation multi-task learning ap- proach,

R. Xie, W. Ye, J. Sun, and S. Zhang, “Exploiting method names to improve code summarization: A deliberation multi-task learning ap- proach,” in 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC). IEEE, 2021, pp. 138–148

work page 2021

[2] [2]

Few-shot training llms for project-specific code-summarization,

T. Ahmed and P. Devanbu, “Few-shot training llms for project-specific code-summarization,” in Proceedings of the 37th IEEE/ACM interna- tional conference on automated software engineering , 2022, pp. 1–5

work page 2022

[3] [3]

Towards automatically generating summary comments for java meth- ods,

G. Sridhara, E. Hill, D. Muppaneni, L. Pollock, and K. Vijay-Shanker, “Towards automatically generating summary comments for java meth- ods,” in Proceedings of the 25th IEEE/ACM international conference on Automated software engineering , 2010, pp. 43–52

work page 2010

[4] [4]

Automatic source code summa- rization of context for java methods,

P. W. McBurney and C. McMillan, “Automatic source code summa- rization of context for java methods,” IEEE Transactions on Software Engineering, vol. 42, no. 2, pp. 103–119, 2015

work page 2015

[5] [5]

Autocomment: Mining question and an- swer sites for automatic comment generation,

E. Wong, J. Yang, and L. Tan, “Autocomment: Mining question and an- swer sites for automatic comment generation,” in 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 2013, pp. 562–567

work page 2013

[6] [6]

Clocom: Mining existing source code for automatic comment generation,

E. Wong, T. Liu, and L. Tan, “Clocom: Mining existing source code for automatic comment generation,” in 2015 IEEE 22nd International conference on software analysis, evolution, and reengineering (SANER). IEEE, 2015, pp. 380–389

work page 2015

[7] [7]

Summarizing source code using a neural attention model,

S. Iyer, I. Konstas, A. Cheung, and L. Zettlemoyer, “Summarizing source code using a neural attention model,” in 54th Annual Meeting of the Association for Computational Linguistics 2016 . Association for Computational Linguistics, 2016, pp. 2073–2083

work page 2016

[8] [8]

Deep code comment genera- tion,

X. Hu, G. Li, X. Xia, D. Lo, and Z. Jin, “Deep code comment genera- tion,” in Proceedings of the 26th conference on program comprehension, 2018, pp. 200–210

work page 2018