Pith. sign in

REVIEW 2 major objections 7 minor 89 references

Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models

T0 review · 2 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Variational prefix tuning makes code summarization models generate sets of summaries that are both more diverse and more accurate than sampling or beam search.

desk verdict A solid, parameter-efficient CVAE-prefix method for diverse code summarization, with real gains over sampling and beam search, but the evaluation asymmetries and an untested core mechanism need referee attention. read the letter →

arxiv 2505.09062 v1 pith:Q2YWZ3OW submitted 2025-05-14 cs.SE cs.AIcs.LG

Junda Zhao , Yuliang Song , Eldan Cohen This is my paper · ORCID
classification cs.SEcs.AIcs.LG
keywords sourcecodesummarizationdiversegenerationvariationalprefixtuningconditionalautoencoderparameter-efficientfine-tuningoraclemetricspre-trainedmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Variational Prefix Tuning (VPT), a way to make pre-trained code summarization models emit a set of candidate summaries instead of one, so that a developer who finds the first output inadequate has alternatives that are still on target. VPT bolts a Conditional Variational Autoencoder onto a frozen backbone model: a small variational encoder learns a distribution over summary embeddings, and each sampled latent vector is used as a prefix that steers decoding toward a different yet plausible summary. The paper claims this raises both accuracy and diversity when evaluated by oracle metrics, which score a set by its single best member; with the CodeT5+ backbone, the best of ten VPT candidates reaches Oracle BLEU 46.40 on Python and 49.22 on Java, beating sampling and beam search. The claim matters because it offers a parameter-efficient route to multiple adequate outputs in code summarization, a setting the authors argue prior work has not addressed.

What carries the argument

The load-bearing object is the variational prefix: a continuous latent vector $z$ sampled from a Conditional Variational Autoencoder and prepended to the frozen decoder's attention, steering each decoding run toward a different plausible summary. During training, a posterior network encodes the concatenation of code and reference summary into $q_\varphi(z|x,y)$, a prior network encodes the code alone into $p_\theta(z|x)$, and the objective maximizes the ELBO with a KL term that keeps the posterior near the code-conditioned prior; both networks reuse the frozen encoder with small trainable pooling tokens, so only a few million parameters change. During inference $z$ is drawn from the prior, each sample becomes a prefix, and a per-prefix beam search produces candidates. A final bi-criteria subset selection chooses $U$ summaries from the pool by trading normalized log-probability (quality) against one minus pairwise BLEU-4 (diversity), and the paper adds two posterior-collapse countermeasures, cyclical KL weighting and batch normalization on the VAE heads, so the decoder does not ignore $z$.

What would settle it

Run the trained VPT twice on the same code with different latent samples, and also run it once with the same latent sample reset; if the between-sample output distance is no larger than the within-sample distance, or if the KL term collapses for a large fraction of examples, the steering premise is false.

Watch

Extended reading notes

Core claim

The paper's central claim is that injecting stochastic latent variables into a frozen pre-trained code summarizer, instead of learning fixed prefixes or retraining the whole model, lets the model produce a set of candidate summaries that is, by oracle metrics, both more accurate and more diverse than what sampling, beam search, stochastic beam search, or diverse beam search produce. On the CodeT5+ backbone, the best of ten VPT candidates reaches Oracle BLEU 46.40 on Python and 49.22 on Java, versus 44.31 and 47.46 for sampling at the same set size; widening the set to twenty candidates widens the lead, and the full 100-candidate pool reaches 51.86 on Python. VPT achieves this while training only about 10.8% of the parameters needed for full fine-tuning, because the backbone stays frozen and only the variational prefix module and pooling tokens are learned. The authors further claim that the method adapts across four different pre-trained models and, paired with a bi-criteria subset selector, is a general way to present users with several adequate summaries rather than a single one.

Load-bearing premise

The argument assumes that the frozen decoder genuinely treats a sampled latent vector as a control signal, so that different samples produce different yet plausible summaries; if the decoder ignores the prefix, the measured gains could be mostly an artifact of running many beams and then reranking by quality and diversity.

Editorial extensions

If this is right

  • With a frozen backbone and about 24 million trainable parameters, any existing code summarization model can be upgraded to return a ranked set of candidate summaries, so deployment cost is close to that of a small adapter rather than a full retrain.
  • Because oracle scores rise with candidate count (46.40 at ten, 48.62 at twenty, 51.86 at 100 on Python), the main ceiling on user-facing quality is the reranker, and better subset selection should keep improving results.
  • The ablation shows the bi-criteria selector alone contributes roughly one BLEU point, so the reranking step is a separable improvement that could be applied on top of other decoders and other generative models.
  • Gains appear on all four tested backbone models and on both Java and Python, which supports treating VPT as a modular add-on rather than a model-specific tuning trick.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same stochastic-prefix mechanism should transfer to code generation, unit test generation, and program repair, where multiple candidate solutions are already collected; a variational prefix could make the candidate sets cheaper and more diverse than temperature sampling.
  • Editorial inference: the paper's diversity metrics are lexical, so the claim 'diverse yet accurate' is strongest at the surface level; a testable extension is to measure whether different latent samples correspond to semantically distinct descriptions by clustering summary embeddings.
  • Editorial inference: the reported 100-candidate oracle numbers suggest that if a good reranker existed, VPT's practical value would grow; a human study measuring how often a developer finds an adequate summary in the final 10- or 20-set would be the direct test of the user-facing claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper proposes Variational Prefix Tuning (VPT), a parameter-efficient method for generating diverse yet accurate sets of source-code summaries from pre-trained encoder-decoder models. VPT attaches a Conditional Variational Autoencoder (CVAE) to a frozen backbone: during training, a posterior network encodes the code-plus-reference-summary pair into a latent distribution, while a prior network encodes only the code; at inference, latent vectors sampled from the prior are inserted as continuous prefixes to steer decoding. Each prefix is decoded with beam search to form a candidate pool, and a bi-criteria subset selection step (balancing backbone log-probability and pairwise BLEU diversity) returns the final set of 10 or 20 summaries. The paper evaluates VPT on Java and Python benchmark datasets with CodeT5+, PLBART, NeuralCodeSum, and SCRIPT backbones, reporting oracle BLEU, ROUGE-L, METEOR, BERTScore, SIDE, and diversity metrics, and compares against beam search, sampling, stochastic and diverse beam search, LoRA-fine-tuned CodeLlama-7B, and GPT-4o with few-shot retrieval. An ablation study isolates the contributions of beam search, the code-conditioned prior, and the subset selector, with Wilcoxon signed-rank tests.

Significance. If the mechanism is substantiated, VPT is a timely and useful contribution: it is the first work to treat code summarization as diverse-set generation with large code models, and its modular design is attractive because only 24.2M parameters are trainable versus 222.9M for full fine-tuning. The evaluation is broad (four backbones, two datasets, seven metrics), the replication package is provided, and the ablation uses paired significance tests. The cleanest result is the 100-candidate pool comparison in Table 3, where VPT reaches Oracle BLEU 51.86 versus 48.92 for sampling and 47.13 for beam search on Python; that result is not confounded by reranking. The main risk is that the paper's distinctive mechanism—latent prefixes steering the frozen decoder—is not directly tested, and part of the smaller-set gains may come from the reranker rather than the generator.

major comments (2)
  1. [3.4-3.6, Table 9] The load-bearing claim that latent variables steer generation is not directly evidenced. Section 3.4 states that sampled z are 'used as prefixes to guide summary sequence generation during decoding,' and Section 3.5 adds beam search per prefix, but no experiment isolates the effect of z. The paper reports no KL(q||p) values or training curves, no statistics on how many distinct outputs correspond to distinct z samples, and no control replacing sampled z with a mean or fixed prefix while holding the 100-candidate pool, per-prefix beam search, and bi-criteria subset selection fixed. Table 9 ablates beam search, the prior network, and the subset selector, yet the '× × ×' row still uses sampled latents; latent sampling itself is never removed. Because the frozen decoder could partially ignore the prefix (posterior collapse is only mitigated, not removed, by Cyclic KL and VAE BatchNorm in Section 3.4), the observed gains could in principle stem mostly from running beam search per prefix and then reranking a 100-candidate pool. To support the paper's central attribution, please add (a) a sampled-versus-mean or fixed-prefix control, (b) per-different-z output diversity statistics, and (c) KL-divergence monitoring during training, or explicitly reinterpret the contribution as the combined VPT-plus-reranking pipeline.
  2. [4.5, Table 3] The headline comparison in Table 3 is not controlled across methods. Section 4.5 states that VPT and Sampling generate 100 summaries and then apply bi-criteria subset selection, whereas Beam, SBS, and DBS directly produce the final 10 or 20 summaries. The reported gains of VPT over Beam, SBS, and DBS at #U=10 and #U=20 therefore include the effect of the reranker, which those baselines do not receive. The sentence explaining that beam size 100 with subset selection decreased performance gives no numbers, so the reader cannot separate generator quality from selector quality. Please report all baselines under identical post-processing: for instance, give Beam, SBS, and DBS the same 100-candidate pool plus reranking (using repeated smaller beams where memory is a concern), or restrict the controlled #U=10/20 comparison to VPT versus Sampling and rely on the #U=100 rows for the generator-level claim.
minor comments (7)
  1. [3.5, Eq. (8)] Equation (8) is formally circular: Y_t is defined in terms of Y_t on the right-hand side, and the constraint y_i != y_j is applied to variables that are not clearly the beam elements from the previous step. Please rewrite the update in terms of extensions of Y_{t-1} and clarify what is being selected.
  2. [3.6] The paper does not specify how the bi-criteria subset selection objective in Eq. (9) is solved; citing Zhong et al. is not sufficient for reproducibility. Please state whether the selection is exact or greedy and describe the algorithm or provide a pointer to the implementation.
  3. [4.5] The tuned values for the sampling temperature T, the prior standard-deviation scaling factor, and the subset-selection weights alpha and beta are not reported. Since these are free parameters tuned on the validation set, please report the chosen values or confirm that they are included in the replication package.
  4. [Tables 3 and 9] There are small inconsistencies between Table 3 and Table 9 for what appears to be the same final VPT configuration (e.g., Python #U=10 SIDE 95.21 vs 95.11 and Python #U=20 SIDE 96.27 vs 96.21). Please reconcile these numbers or explain the difference.
  5. [Throughout] The term 'uni-model' appears in Sections 2.2, 3.3, and 4.4.2; it should be 'uni-modal' for consistency with standard usage.
  6. [Table 8] Table 8 is based on a 500-example subset, and the VPT scores differ from the full-test-set scores in Table 3. Please state this explicitly in the caption and report confidence intervals or variance estimates, since single-point estimates on a small subset make the comparison hard to interpret.
  7. [Table 4] Sampling's diversity scores differ substantially between Python and Java (e.g., D-1 at #U=10 is 38.80 for Python but 56.08 for Java), yet no explanation is offered. A brief comment on this dataset-dependent behavior would help the reader interpret the diversity results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the VPT derivation is a standard CVAE ELBO with independent held-out test evaluation, and the only self-citation is a non-load-bearing external subset-selection algorithm.

full rationale

The derivation chain is self-contained against external benchmarks. The core training objective (Eq. 7) is the standard CVAE ELBO (Eq. 5), with the reconstruction term optimizing next-token cross-entropy and the KL term regularizing the posterior toward the prior; no term is defined in terms of the evaluation metrics (Oracle BLEU, ROUGE-L, METEOR, BERTScore, SIDE) and no parameter is fitted to the test set. During inference, latent variables are sampled from the prior and used as prefixes, which is a mechanistic design rather than a metric; whether the frozen decoder is actually steered by z is an empirical correctness question, not a circularity, because the reported oracle metrics are computed against ground-truth summaries on held-out test data (Section 5). Hyperparameters, including sampling temperature, latent scaling, and subset-selection weights, are tuned on a held-out validation set (Section 4.5). The bi-criteria subset selection (Eqs. 9-11) is a reranker applied to candidate summaries: the quality term uses the backbone model's own length-normalized log-probability and the diversity term uses pairwise BLEU-4 distance, neither of which is the oracle target, and the same selection procedure is applied to sampling baselines and to CodeLlama/GPT-4o where applicable. The ablation study (Table 9) further shows that VPT with beam search and prior net but without bi-criteria selection still exceeds the beam-search and sampling baselines, so the central claim does not reduce to the reranker. The only self-citation, [45] Zhong et al. (co-authored by Cohen), supplies the subset-selection algorithm; it is a published, externally usable combinatorial optimization method, not a uniqueness theorem or an ansatz that pre-commits the results, and it does not raise the circularity score.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The 'variational prefix' is a method component, not a postulated entity. The load-bearing assumptions are about the informativeness of frozen encoder embeddings, the decoder's use of prefixes, and the validity of oracle metrics.

free parameters (4)
  • prior sigma scaling factor = not reported; tuned per dataset
    Section 3.2: prior std fixed at 1 during training, 'can be scaled' during inference; Section 4.5: scaling factor tuned on validation.
  • alpha and beta in subset selection = not reported; tuned per dataset
    Eq. 9 parameters controlling the trade-off; Section 4.5 says they are tuned on a held-out validation set.
  • sampling temperature T for baselines = not reported; tuned per dataset
    Section 4.5: temperature for sampling is tuned on the validation set.
  • cyclic KL weight schedule = cycled from 0 to 1; schedule details not reported
    Section 3.4: used to avoid posterior collapse; the schedule parameters are hand-chosen and unstated.
assumptions (5)
  • standard math ELBO with Gaussian prior and posterior, reparameterization trick
    Eq. (5) and (7); the CVAE framework from Sohn et al. 2015.
  • domain assumption The frozen backbone encoder embeddings are informative enough to parameterize both prior and posterior distributions
    Sections 3.2 and 3.3 rely on CodeT5+ encoder outputs for the prior mean and posterior parameters.
  • domain assumption Prepending variational prefixes to the decoder attention guides generation as intended
    Section 3.4 and Figure 2 assume the frozen decoder uses the sampled latent prefixes, rather than ignoring them.
  • domain assumption Oracle metrics, selecting the best candidate per reference-based metric, are a valid proxy for 'at least one adequate summary'
    Section 4.3.1; the paper notes in threats to validity that human judgment is not incorporated.
  • domain assumption The Java (Hu et al.) and Python (Wan et al.) datasets are representative for code summarization
    Section 4.2; threats to validity note the limited language coverage.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models." pith.science (2026). https://pith.science/paper/Q2YWZ3OW

@misc{pith2026250509062,
  author       = {Pith},
  title        = {Pith review of: Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q2YWZ3OW}},
  note         = {Machine review of arXiv:2505.09062}
}
read the original abstract

Recent advancements in source code summarization have leveraged transformer-based pre-trained models, including Large Language Models of Code (LLMCs), to automate and improve the generation of code summaries. However, existing methods often focus on generating a single high-quality summary for a given source code, neglecting scenarios where the generated summary might be inadequate and alternative options are needed. In this paper, we introduce Variational Prefix Tuning (VPT), a novel approach that enhances pre-trained models' ability to generate diverse yet accurate sets of summaries, allowing the user to choose the most suitable one for the given source code. Our method integrates a Conditional Variational Autoencoder (CVAE) framework as a modular component into pre-trained models, enabling us to model the distribution of observed target summaries and sample continuous embeddings to be used as prefixes to steer the generation of diverse outputs during decoding. Importantly, we construct our method in a parameter-efficient manner, eliminating the need for expensive model retraining, especially when using LLMCs. Furthermore, we employ a bi-criteria reranking method to select a subset of generated summaries, optimizing both the diversity and the accuracy of the options presented to users. We present extensive experimental evaluations using widely used datasets and current state-of-the-art pre-trained code summarization models to demonstrate the effectiveness of our approach and its adaptability across models.

Figures

Figures reproduced from arXiv: 2505.09062 by the authors.

Figure 1
Figure 1. An overview for using VPT to generate a diverse and accurate set of code [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Detailed Model Architecture of VPT. 3.1. Backbone Model For our approach, we adopted pre-trained transformer-based encoder￾decoder models commonly used for code summarization tasks [10, 5, 32, 6, 37] as the backbone of our method. The backbone model is responsible for pro￾cessing source code tokens x into contextual embeddings used as the condi￾tional input and decodes the summaries out of the contextual embeddings … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 31 canonical work pages

  1. [1]

    X. Xia, L. Bao, D. Lo, Z. Xing, A. E. Hassan, S. Li, Measur- ing program comprehension: A large-scale field study with profession- als, IEEE Transactions on Software Engineering 44 (2018) 951–976. doi:10.1109/TSE.2017.2734091

    arXiv 2018

  2. [2]

    Stapleton, Y

    S. Stapleton, Y. Gambhir, A. LeClair, Z. Eberhart, W. Weimer, K. Leach, Y. Huang, A human study of comprehension and code sum- marization, in: The 28th International Conference on Program Com- prehension, ICPC ’20, 2020, p. 2–13

    2020

  3. [3]

    Sridhara, E

    G. Sridhara, E. Hill, D. Muppaneni, L. Pollock, K. Vijay-Shanker, To- wards automatically generating summary comments for java methods, in: Proceedings of the 25th IEEE/ACM International Conference on Automated Software Engineering, ASE ’10, 2010, p. 43–52

    2010

  4. [4]

    X. Hu, G. Li, X. Xia, D. Lo, Z. Jin, Deep code comment generation, in: Proceedings of the 26th Conference on Program Comprehension, ICPC ’18, Association for Computing Machinery, New York, NY, USA, 2018, p. 200–210. URL: https://doi.org/10.1145/3196321.3196334. doi:10.1145/3196321.3196334. 2https://github.com/jundaz/VPT.git 31

    arXiv 2018

  5. [5]

    W. U. Ahmad, S. Chakraborty, B. Ray, K.-W. Chang, A transformer- based approach for source code summarization., in: ACL, Association for Computational Linguistics, 2020, pp. 4998–5007

    2020

  6. [6]

    Z. Gong, C. Gao, Y. Wang, W. Gu, Y. Peng, Z. Xu, Source code summarization with structural relative position guided trans- former, in: 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE Computer Society, Los Alamitos, CA, USA, 2022, pp. 13–24. URL: https:// doi.ieeecomputersociety.org/10.1109/SANER53432.2022...

    arXiv 2022

  7. [7]

    S. Gao, C. Gao, Y. He, J. Zeng, L. Nie, X. Xia, M. Lyu, Code struc- ture–guided transformer for source code summarization, ACM Trans. Softw. Eng. Methodol. 32 (2023). doi: 10.1145/3522674

    doi:10.1145/3522674 2023

  8. [8]

    H. Wu, H. Zhao, M. Zhang, Code summarization with structure-induced transformer, in: C. Zong, F. Xia, W. Li, R. Navigli (Eds.), Find- ings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Association for Computational Linguistics, Online, 2021, pp. 1078–1090. URL: https://aclanthology.org/2021.findings-acl

    2021

Show all 89 references
  1. [9]

    Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, M. Zhou, CodeBERT: A pre-trained model for programming and natural languages, in: T. Cohn, Y. He, Y. Liu (Eds.), Findings of the Association for Computational Linguis- tics: EMNLP 2020, Ass...

    2020 doi

  2. [10]

    Y. Wang, H. Le, A. D. Gotmare, N. D. Q. Bui, J. Li, S. C. H. Hoi, Codet5+: Open code large language models for code understanding and generation, 2023. arXiv:2305.07922

    2023 arXiv

  3. [11]

    Rozi` ere, J

    B. Rozi` ere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, R. Sauvestre, T. Remez, J. Rapin, A. Kozhevnikov, I. Evtimov, J. Bitton, M. Bhatt, C. C. Ferrer, A. Grattafiori, W. Xiong, A. D´ efossez, J. Copet, F. Azhar, H. Touvron, L. Martin, N. Usunier,...

    2024 arXiv

  4. [12]

    Zhang, X

    X. Zhang, X. Hou, X. Qiao, W. Song, A review of automatic source code summarization, Empirical Softw. Engg. 29 (2024). URL: https://doi.org/10.1007/s10664-024-10553-6 . doi:10.1007/ s10664-024-10553-6

    2024 doi

  5. [13]

    Macherey, F

    W. Macherey, F. Och, I. Thayer, J. Uszkoreit, Lattice-based minimum error rate training for statistical machine translation, in: The 2008 Conference on Empirical Methods in Natural Language Processing, As- sociation for Computational Linguistics, Honolulu, Hawaii, 2008, pp. 725–734

    2008

  6. [14]

    Tromble, S

    R. Tromble, S. Kumar, F. Och, W. Macherey, Lattice Minimum Bayes- Risk decoding for statistical machine translation, in: Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Honolulu, Hawaii, 2008, p...

    2008

  7. [15]

    Kumar, W

    S. Kumar, W. Byrne, Minimum Bayes-risk decoding for statistical ma- chine translation, in: Proceedings of the Human Language Technol- ogy Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, Association for Com- putational ...

    2004

  8. [16]

    D. P. Kingma, M. Welling, Auto-Encoding Variational Bayes, in: 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings,

    2014

  9. [17]

    K. Sohn, H. Lee, X. Yan, Learning structured output representation using deep conditional generative models, in: C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, R. Garnett (Eds.), Advances in Neural Informa- tion Processing Systems, volume 28, Curran Associates, Inc., 2015

    2015

  10. [18]

    L. Wang, A. Schwing, S. Lazebnik, Diverse and accurate image descrip- tion using a variational auto-encoder with an additive gaussian encoding space, in: Advances in Neural Information Processing Systems, 2017. 33

    2017

  11. [19]

    Z. Lin, G. I. Winata, P. Xu, Z. Liu, P. Fung, Variational transformers for diverse response generation, 2020. arXiv:2003.12738

    2020 arXiv

  12. [20]

    U. Jain, Z. Zhang, A. Schwing, Creativity: Generating diverse questions using variational autoencoders, in: 2017 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, Los Alamitos, CA, USA, 2017, pp. 5415–5424. URL: https://doi.ieeecompute...

    2017 doi

  13. [21]

    T. Wang, X. Wan, T-cvae: Transformer-based conditioned variational autoencoder for story completion, in: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, 2019, pp. 5233–5239. URL: https://doi.org/10.24963/ijcai.2019/727. doi...

    2019 doi

  14. [22]

    L. Fang, T. Zeng, C. Liu, L. Bo, W. Dong, C. Chen, Transformer- based conditional variational autoencoder for controllable story genera- tion, 2021. arXiv:2101.00828

    2021 arXiv

  15. [23]

    Jiang, G

    J. Jiang, G. G. Xia, D. B. Carlton, C. N. Anderson, R. H. Miyakawa, Transformer vae: A hierarchical model for structure-aware and in- terpretable music representation learning, in: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (I...

    2020

  16. [24]

    A. K. Vijayakumar, M. Cogswell, R. R. Selvaraju, Q. Sun, S. Lee, D. Crandall, D. Batra, Diverse beam search: Decoding diverse solutions from neural sequence models, 2018. arXiv:1610.02424

    2018 arXiv

  17. [25]

    Aneja, H

    J. Aneja, H. Agrawal, D. Batra, A. Schwing, Sequential latent spaces for modeling the intention during diverse image captioning, 2019. arXiv:1908.08529

    2019 arXiv

  18. [26]

    Ruan, Z.-H

    Y.-P. Ruan, Z.-H. Ling, X. Zhu, Q. Liu, J.-C. Gu, Gen- erating diverse conversation responses by creating and rank- ing multiple candidates, Computer Speech & Language 62 (2020) 101071. URL: https://www.sciencedirect.com/science/ 34 article/pii/S0885230820300048. doi:https://d...

    2020

  19. [27]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems 30,...

    2017

  20. [28]

    D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. Liu, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu, M. Tufano, S. K. Deng, C. Clement, D. Drain, N. Sundaresan, J. Yin, D. Jiang, M. Zhou, Graphcodebert: Pre-training code representations with data flow, 2021. arXiv:2009.08366

    2021 arXiv

  21. [29]

    S. Liu, B. Wu, X. Xie, G. Meng, Y. Liu, Contrabert: Enhancing code pre-trained models via contrastive learning, 2023. arXiv:2301.09072

    2023 arXiv

  22. [30]

    S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. Clement, D. Drain, D. Jiang, D. Tang, G. Li, L. Zhou, L. Shou, L. Zhou, M. Tufano, M. Gong, M. Zhou, N. Duan, N. Sundaresan, S. K. Deng, S. Fu, S. Liu, Codexglue: A machine learning benchmark dataset for code unde...

    2021 arXiv

  23. [31]

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Ka- plan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Win- t...

    2021 arXiv

  24. [32]

    W. U. Ahmad, S. Chakraborty, B. Ray, K.-W. Chang, Uni- fied pre-training for program understanding and generation, 2021. arXiv:2103.06333

    2021 arXiv

  25. [33]

    Lachaux, B

    M.-A. Lachaux, B. Roziere, L. Chanussot, G. Lample, Unsupervised translation of programming languages, 2020. arXiv:2006.03511. 35

    2020 arXiv

  26. [34]

    Raffel, N

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, 2023. arXiv:1910.10683

    2023 arXiv

  27. [35]

    S. R. Bowman, L. Vilnis, O. Vinyals, A. Dai, R. Jozefowicz, S. Ben- gio, Generating sentences from a continuous space, in: S. Rie- zler, Y. Goldberg (Eds.), Proceedings of the 20th SIGNLL Confer- ence on Computational Natural Language Learning, Association for Computational Li...

    2016 doi

  28. [36]

    X. L. Li, P. Liang, Prefix-tuning: Optimizing continuous prompts for generation, 2021. arXiv:2101.00190

    2021 arXiv

  29. [37]

    Z. Tang, X. Shen, C. Li, J. Ge, L. Huang, Z. Zhu, B. Luo, Ast-trans: code summarization with efficient tree-structured attention, in: Pro- ceedings of the 44th International Conference on Software Engineering, ICSE ’22, Association for Computing Machinery, New York, NY, USA, 2...

    2022

  30. [38]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. arXiv:1810.04805

    2019 arXiv

  31. [39]

    Lucas, G

    J. Lucas, G. Tucker, R. Grosse, M. Norouzi, Don’t Blame the ELBO! A Linear VAE Perspective on Posterior Collapse, Curran Associates Inc., Red Hook, NY, USA, 2019

    2019

  32. [40]

    Goyal, A

    A. Goyal, A. Sordoni, M.-A. Cˆ ot´ e, N. R. Ke, Y. Bengio, Z-forcing: Training stochastic recurrent networks, 2017. URL: https://arxiv. org/abs/1711.05411. arXiv:1711.05411

    2017 arXiv

  33. [41]

    Q. Zhu, W. Bi, X. Liu, X. Ma, X. Li, D. Wu, A batch normalized in- ference network keeps the KL vanishing away, in: D. Jurafsky, J. Chai, N. Schluter, J. Tetreault (Eds.), The 58th Annual Meeting of the Associ- ation for Computational Linguistics, Association for Computational...

    2020 doi

  34. [42]

    H. Fu, C. Li, X. Liu, J. Gao, A. C ¸ elikyilmaz, L. Carin, Cyclical an- nealing schedule: A simple approach to mitigating kl vanishing., in: 36 J. Burstein, C. Doran, T. Solorio (Eds.), NAACL-HLT (1), Association for Computational Linguistics, 2019, pp. 240–250

    2019

  35. [43]

    Ioffe, C

    S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep net- work training by reducing internal covariate shift, 2015. URL: https: //arxiv.org/abs/1502.03167. arXiv:1502.03167

    2015 arXiv

  36. [44]

    Su, Variational autoencoder (5): Vae+bn=better vae, 2020

    J. Su, Variational autoencoder (5): Vae+bn=better vae, 2020. URL: https://www.spaces.ac.cn/archives/7381

    2020

  37. [45]

    Zhong, P

    S. Zhong, P. Shati, E. Cohen, Bi-criteria diverse plan selection via beam search approximation, in: Proceedings of the International Symposium on Combinatorial Search, volume 17, 2024, pp. 188–196

    2024

  38. [46]

    Papineni, S

    K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, Bleu: a method for automatic evaluation of machine translation, in: Association for Com- putational Linguistics, 2002, pp. 311–318

    2002

  39. [47]

    X. Hu, G. Li, X. Xia, D. Lo, S. Lu, Z. Jin, Summarizing source code with transferred api knowledge, in: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, 2018, pp. 2269–2275. URL: https://doi.org/10.24963/ijcai.2018/314. doi...

    2018 doi

  40. [48]

    A. V. M. Barone, R. Sennrich, A parallel corpus of python functions and documentation strings for automated code documentation and code generation., in: IJCNLP(2), 2017, pp. 314–319

    2017

  41. [49]

    Y. Wan, Z. Zhao, M. Yang, G. Xu, H. Ying, J. Wu, P. S. Yu, Improving automatic source code summarization via deep reinforcement learning, in: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, 2018, p. 397–407. URL: https:// doi.org/10...

    2018

  42. [50]

    W. Sun, C. Fang, Y. Chen, Q. Zhang, G. Tao, T. Han, Y. Ge, Y. You, B. Luo, An extractive-and-abstractive framework for source code sum- marization, 2023. arXiv:2206.07245

    2023 arXiv

  43. [51]

    Lin, ROUGE: A package for automatic evaluation of summaries, in: Text Summarization Branches Out, Association for Computational Linguistics, Barcelona, Spain, 2004, pp

    C.-Y. Lin, ROUGE: A package for automatic evaluation of summaries, in: Text Summarization Branches Out, Association for Computational Linguistics, Barcelona, Spain, 2004, pp. 74–81. 37

    2004

  44. [52]

    Banerjee, A

    S. Banerjee, A. Lavie, Meteor: An automatic metric for mt evaluation with improved correlation with human judgments., in: J. Goldstein, A. Lavie, C.-Y. Lin, C. R. Voss (Eds.), IEEvaluation@ACL, Association for Computational Linguistics, 2005, pp. 65–72

    2005

  45. [53]

    Zhang, V

    T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, Y. Artzi, Bertscore: Evaluating text generation with bert., in: ICLR, OpenReview.net, 2020

    2020

  46. [55]

    W. Kool, H. van Hoof, M. Welling, Stochastic beams and where to find them: The gumbel-top-k trick for sampling sequences without replace- ment, 2019. arXiv:1903.06059

    2019 arXiv

  47. [56]

    J. Xu, B. Liu, Y. Zhou, M. Liu, R. Yao, Z. Shao, Diverse image caption- ing via conditional variational autoencoder and dual contrastive learn- ing, ACM Trans. Multimedia Comput. Commun. Appl. 20 (2023). URL: https://doi.org/10.1145/3614435. doi:10.1145/3614435

    2023 doi

  48. [57]

    J. Mao, W. Xu, Y. Yang, J. Wang, Z. Huang, A. Yuille, Deep captioning with multimodal recurrent neural networks (m-rnn), 2015. arXiv:1412.6632

    2015 arXiv

  49. [58]

    J. Li, M. Galley, C. Brockett, J. Gao, B. Dolan, A diversity-promoting objective function for neural conversation models., in: K. Knight, A. Nenkova, O. Rambow (Eds.), HLT-NAACL, The Association for Computational Linguistics, 2016, pp. 110–119

    2016

  50. [59]

    Y. Zhu, S. Lu, L. Zheng, J. Guo, W. Zhang, J. Wang, Y. Yu, Texygen: A benchmarking platform for text generation models, 2018. arXiv:1802.01886

    2018 arXiv

  51. [60]

    S. Iyer, I. Konstas, A. Cheung, L. Zettlemoyer, Summarizing source code using a neural attention model, in: K. Erk, N. A. Smith (Eds.), The 54th Annual Meeting of the Association for Computational Linguistics 38 (Volume 1: Long Papers), Association for Computational Linguistic...

    2016 doi

  52. [61]

    Eriguchi, K

    A. Eriguchi, K. Hashimoto, Y. Tsuruoka, Tree-to-sequence attentional neural machine translation, in: Proceedings of the 54th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Berlin, Germany, 2...

    2016 doi

  53. [62]

    B. Wei, G. Li, X. Xia, Z. Fu, Z. Jin, Code Generation as a Dual Task of Code Summarization, Curran Associates Inc., Red Hook, NY, USA, 2019

    2019

  54. [63]

    Lewis, Y

    M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, L. Zettlemoyer, Bart: Denoising sequence-to-sequence pre- training for natural language generation, translation, and comprehen- sion, 2019. arXiv:1910.13461

    2019 arXiv

  55. [64]

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, LoRA: Low-rank adaptation of large language models, in: International Conference on Learning Representations, 2022. URL: https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  56. [65]

    Accessed: 2024-10-15

    OpenAI, Gpt-4o, https://platform.openai.com/docs/models/ gpt-4o, 2024. Accessed: 2024-10-15

    2024

  57. [66]

    D. P. Kingma, J. Ba, Adam: A method for stochastic optimization,

  58. [67]

    Ahmed, K

    T. Ahmed, K. S. Pai, P. Devanbu, E. Barr, Automatic semantic augmentation of language model prompts (for code summarization), in: Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE ’24, Association for Computing Machin- ery, New York, NY, U...

    2024

  59. [68]

    Haiduc, J

    S. Haiduc, J. Aponte, L. Moreno, A. Marcus, On the use of auto- mated text summarization techniques for summarizing source code, in: 2010 17th Working Conference on Reverse Engineering, 2010, pp. 35–44. doi:10.1109/WCRE.2010.13. 39

    2010 doi

  60. [69]

    B. P. Eddy, J. A. Robinson, N. A. Kraft, J. C. Carver, Evaluating source code summarization techniques: Replication and expansion, in: 2013 21st International Conference on Program Comprehension (ICPC), 2013, pp. 13–22

    2013

  61. [70]

    E. Wong, T. Liu, L. Tan, Clocom: Mining existing source code for automatic comment generation, in: 2015 IEEE 22nd International Con- ference on Software Analysis, Evolution, and Reengineering (SANER), 2015, pp. 380–389. doi: 10.1109/SANER.2015.7081848

    2015

  62. [71]

    Zhang, J

    C. Zhang, J. Wang, Q. Zhou, T. Xu, K. Tang, H. Gui, F. Liu, A survey of automatic source code summarization, Symmetry 14 (2022). URL: https://www.mdpi.com/2073-8994/14/3/471. doi: 10. 3390/sym14030471

    2022

  63. [72]

    LeClair, S

    A. LeClair, S. Jiang, C. McMillan, A neural model for generating natural language summaries of program subroutines, in: The 41st International Conference on Software Engineering, ICSE ’19, IEEE Press, 2019, p. 795–806. URL: https://doi.org/10.1109/ICSE.2019.00087. doi:10. 1109...

    2019

  64. [73]

    LeClair, S

    A. LeClair, S. Haque, L. Wu, C. McMillan, Improved code summariza- tion via a graph neural network, in: The 28th International Conference on Program Comprehension, ICPC ’20, Association for Computing Ma- chinery, New York, NY, USA, 2020, p. 184–195. URL: https://doi. org/10.11...

    2020

  65. [74]

    Z. Yu, W. Zheng, J. Wang, Q. Tang, S. Nie, S. Wu, Codecmr: Cross- modal retrieval for function-level binary source code matching, in: Ad- vances in Neural Information Processing Systems, volume 33, 2020, pp. 3872–3883

    2020

  66. [75]

    Y. Zhou, J. Shen, X. Zhang, W. Yang, T. Han, T. Chen, Automatic source code summarization with graph attention networks, Journal of Systems and Software 188 (2022) 111257. URL: https://www. sciencedirect.com/science/article/pii/S0164121222000279. doi:https://doi.org/10.1016/j....

    2022

  67. [76]

    Tevet, J

    G. Tevet, J. Berant, Evaluating the evaluation of diversity in natural language generation, in: The 16th Conference of the European Chapter 40 of the Association for Computational Linguistics: Main Volume, Asso- ciation for Computational Linguistics, Online, 2021, pp. 326–346

    2021

  68. [77]

    Holtzman, J

    A. Holtzman, J. Buys, L. Du, M. Forbes, Y. Choi, The curious case of neural text degeneration, 2020. arXiv:1904.09751

    2020 arXiv

  69. [78]

    Z. Li, X. Ding, T. Liu, Generating reasonable and diversified story end- ing using sequence to sequence model with adversarial training, in: E. M. Bender, L. Derczynski, P. Isabelle (Eds.), Proceedings of the 27th International Conference on Computational Linguistics, Associa-...

    2018

  70. [79]

    J. Xu, X. Ren, J. Lin, X. Sun, Diversity-promoting GAN: A cross- entropy based generative adversarial network for diversified text gener- ation, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 3940–3949. URL: https: //aclan...

    2018 doi

  71. [80]

    I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, 2014. arXiv:1406.2661

    2014 arXiv

  72. [81]

    X. Gu, K. Cho, J.-W. Ha, S. Kim, Dialogwae: Multimodal re- sponse generation with conditional wasserstein auto-encoder, 2019. arXiv:1805.12352

    2019 arXiv

  73. [82]

    Luo, J.-T

    T.-C. Luo, J.-T. Chien, Variational dialogue generation with normal- izing flows, in: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 7778– 7782

    2021

  74. [83]

    Cho, Noisy parallel approximate decoding for conditional recurrent language model, 2016

    K. Cho, Noisy parallel approximate decoding for conditional recurrent language model, 2016. arXiv:1605.03835

    2016 arXiv

  75. [84]

    J. Li, W. Monroe, D. Jurafsky, A simple, fast diverse decoding algorithm for neural generation, 2016. arXiv:1611.08562

    2016 arXiv

  76. [85]

    Tam, Cluster-based beam search for pointer-generator chat- bot grounded by knowledge, Computer Speech & Language 64 (2020) 101094

    Y.-C. Tam, Cluster-based beam search for pointer-generator chat- bot grounded by knowledge, Computer Speech & Language 64 (2020) 101094. URL: https://www.sciencedirect.com/science/ 41 article/pii/S0885230820300279. doi:https://doi.org/10.1016/j. csl.2020.101094

    2020

  77. [86]

    Zhang, C

    Q. Zhang, C. Fang, Y. Ma, W. Sun, Z. Chen, A survey of learning- based automated program repair, ACM Trans. Softw. Eng. Methodol. 33 (2023). URL: https://doi.org/10.1145/3631974. doi: 10.1145/ 3631974

    2023 doi

  78. [87]

    Kessel, C

    M. Kessel, C. Atkinson, Diversity-driven unit test generation, Journal of Systems and Software 193 (2022) 111442. URL: https://www. sciencedirect.com/science/article/pii/S0164121222001406. doi:https://doi.org/10.1016/j.jss.2022.111442. 42

    2022

  79. [93]

    doi:10.18653/v1/2021.findings-acl.93

    2021 doi

  80. [575]

    doi:10.1109/CVPR.2017.575

    2017 doi

  81. [2014]

    arXiv:http://arxiv.org/abs/1312.6114v10

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.