Pith. sign in

REVIEW 4 major objections 4 minor 46 references

Code search rerankers can learn to prefer fast, lean, readable code—not just relevant matches—via a three-level ranking objective trained on LLM-synthesized quality pairs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 18:53 UTC pith:3AXKT5WB

load-bearing objection Useful benchmark and clear writing, but the core training signal is unvalidated synthetic data; without execution checks on those labels, the QPA gains could be measuring prompt artifacts instead of code quality. the 4 major comments →

arxiv 2607.17139 v1 pith:3AXKT5WB submitted 2026-07-19 cs.SE cs.CL

SynH-Rank: Quality-Aware Code Search via Diverse Data Synthesis and Hierarchical Ranking Training

Keyu Liang , Haoye Wang , Yanfu Yan , Zhiyuan Wan , Zhongxin Liu This is my paper
classification cs.SE cs.CL MSC 68T5068N30
keywords code searchrerankingcode qualitynon-functional requirementsdata synthesislearning-to-rankhierarchical rankingbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that code search rerankers, which currently rank only by semantic relevance, should also prefer code that is fast, memory-efficient, and maintainable. It introduces SynH-Rank, a framework that trains rerankers on LLM-generated high- and low-quality code pairs using a hierarchical ranking loss that explicitly orders high-quality relevant above low-quality relevant above irrelevant. The authors build a new benchmark, QualCode, and show that SynH-Rank improves quality-preference accuracy by 20.15% over vanilla rerankers and 15.80% over relevance-only contrastive training, while also improving traditional relevance metrics on two of three backbones. The central claim is that quality awareness can be injected into existing rerankers without sacrificing relevance, and that it generalizes to multi-condition queries.

Core claim

SynH-Rank combines guideline-driven LLM data synthesis with a three-level hierarchical ranking training objective to make code rerankers quality-aware. The key insight is that standard contrastive learning, which treats all negatives equally, cannot express the ordinal relationship among high-quality relevant, low-quality relevant, and irrelevant code. By assigning relevance labels 2, 1, and 0 and training with a listwise learning-to-rank loss weighted by nDCG deltas, the reranker learns to prefer high-quality, functionally correct code over lower-quality but still relevant code, while preserving relevance discrimination. On the new QualCode benchmark spanning speed, memory, and maintainabil

What carries the argument

The central mechanism is the three-level labeling scheme (high-quality relevant > low-quality relevant > irrelevant, with relevance labels 2, 1, 0) combined with a hierarchical ranking loss: a sum over ordered pairs of the nDCG-weighted logistic loss, |ΔnDCG|·log(1+exp(−(s_i−s_j))). This explicitly encodes the preference order and up-weights errors at the top of the ranking. The second component is guideline-driven diversity in data synthesis: instead of nucleus sampling, the LLM is prompted with specific quality guideline groups (e.g., 'precomputation, closed-form formulas' vs 'brute-force, exhaustive search') to generate semantically distinct high- and low-quality variants, which the autho

Load-bearing premise

The training signal assumes that the LLM's guideline-prompted 'high-quality' and 'low-quality' code variants actually differ in real execution speed, memory usage, and maintainability; the paper filters only invalid or duplicate outputs and never executes or statically validates the generated variants.

What would settle it

Run the SynH-Rank training pipeline but replace the LLM's quality labels with a random assignment of high/low quality to the synthesized variants. If QPA on QualCode still improves substantially over the vanilla backbone (rather than collapsing to near 0.5), the claimed gains would not be attributable to genuine quality awareness. A lighter check: execute a sample of the synthesized positive/negative pairs and measure whether the positives are actually faster/lower-memory/fewer-smells as claimed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Quality-aware reranking can be trained without human-annotated quality labels, using only LLM-synthesized pairs guided by explicit quality dimensions.
  • Injecting the three-level hierarchy improves not just quality preference but also relevance metrics on most backbones, suggesting that quality and relevance signals can be learned jointly rather than traded off.
  • The framework transfers to multi-condition queries (e.g., 'fast and memory-efficient') without explicit multi-condition supervision, indicating that per-dimension quality signals can combine additively.
  • Rerankers, being plug-and-play in retrieve-then-rerank pipelines, allow quality awareness to be added without re-indexing code corpora, making the approach practical for existing systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If SynH-Rank's gains reflect genuine quality awareness rather than prompt artifacts, a natural extension is to verify generated variants by execution (runtime/memory profiling) and static analysis, which could produce even cleaner training signals and reveal how much of the QPA gain is due to learnable quality cues versus surface style.
  • The additive generalization to multi-condition queries suggests a testable hypothesis: that quality signals are internally represented as independent scoring dimensions; probing the model's per-dimension scores could confirm this and inform specialized multi-condition training if the additive assumption breaks down at higher N.
  • The framework's reliance on a single synthesis LLM (Qwen3-Coder) leaves open whether quality labels generalize across model families; training with multiple synthesizers and measuring QPA variance would quantify the model-specific bias the authors acknowledge.
  • Because the benchmark derives positives/negatives from CodeNet's measured CPU time and memory, the evaluation itself is execution-grounded; extending the same construction to other languages (Java, C++) with their own quality metrics would test whether the framework's Python-specific guidelines transfer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SynH-Rank, a quality-aware code reranking framework. The authors construct QualCode, a benchmark of 4,209 query/code pairs derived from CodeNet, with execution-speed, memory, and maintainability labels obtained from CodeNet metadata and SonarQube; they also construct MC-QualCode for multi-condition evaluation and introduce QPA and MCA metrics. Training data are synthesized with Qwen3-Coder using dimension-specific guidelines that ask for high- or low-quality variants, and the reranker is trained with a three-level hierarchical ranking loss (high-quality relevant > low-quality relevant > irrelevant). Experiments on three rerankers report average QPA gains of 20.15% over the backbones and 15.80% over relevance-only contrastive training, plus multi-condition MCA gains.

Significance. The problem addressed—incorporating non-functional code quality into code search reranking—is timely and practically relevant. QualCode and MC-QualCode are potentially useful evaluation resources, and the synthetic-data-plus-hierarchical-ranking recipe is simple and reproducible. The ablation study usefully separates the contributions of the guideline-driven synthesis and the loss function. However, the central claim of 'quality awareness' depends on an unvalidated bridge between LLM-prompt-assigned synthetic labels and the measured qualities in QualCode, and the experimental section lacks statistical significance tests. If the synthetic labels are validated, this would be a solid contribution; in its current form the evidence is conditional.

major comments (4)
  1. [§4.1, §5.4, §8] The synthetic training labels are never validated. The paper uses Qwen3-Coder outputs generated with 'most efficient' vs 'intentionally inefficient' prompts and only removes empty/duplicate outputs and tasks with <3 unique candidates (§4.1). No execution, memory profiling, or SonarQube is applied to the synthesized code, while QualCode's evaluation labels are measured (CodeNet CPU/memory, SonarQube smells, §3.1). This is load-bearing because Table 4 shows the hierarchical loss adds only ~0.57% QPA over InfoNCE with identical samples, so the headline gains are attributed to the synthetic data; if those data encode prompt artifacts (brute-force markers, length, unused helpers—see Fig. 4), the QPA improvements do not establish genuine quality awareness. §8 concedes possible model-specific bias. Please add an execution/static-analysis validation of a sample of synthetic pairs and report per-
  2. [Tables 3–5] All results are single runs with no error bars or significance tests. Several core comparisons are small: Jina-v2-base SynH-Rank speed QPA is 0.5166 vs SRCL 0.5031 and maintainability QPA is 0.7644 vs 0.7612 (Table 3); MCA differences for Jina in Table 5 are similarly narrow. Without variance estimates or paired tests, the 20.15% headline—an average over backbones—may not be a robust effect. Report mean±std over at least three seeds and a paired significance test (e.g., bootstrap or Wilcoxon) for QPA and MCA.
  3. [Table 3, RQ1 answer] The claim that SynH-Rank 'simultaneously enhances traditional relevance metrics' is not supported across all backbones. On BGE-M3, SynH-Rank's MRR@10 is 0.3409 vs 0.4367 for SRCL (Speed row) and nDCG@10 is similarly lower. The paper's backbone-capacity explanation is plausible, but the abstract and RQ1 answer state the improvement as general. Please qualify the relevance claim to the models/dimensions where it holds, or discuss the trade-off explicitly in the abstract.
  4. [§3.2, Table 5] MC-QualCode construction uses different quality thresholds than QualCode (top/bottom 33% vs min/max with 2× separation) and hard negatives satisfying N−1 conditions. While the controlled-variable design is a strength, the paper does not report the number of distinct problems underlying the 3,000 triplets per condition level. If the triplets come from a small problem set, the MCA results may reflect problem memorization rather than generalizable multi-condition reasoning. Please report the number of unique problems and the average number of triplets per problem, and consider a held-out-problem split for the generalization claim.
minor comments (4)
  1. [Eq. (6)] The ΔnDCG_ij weight uses rank_i and rank_j, but after decomposing the training list into ordered pairs S, the ranks are not defined for the pair decomposition. Please specify how ranks are assigned during training (e.g., fixed positions from the original list) so the loss is unambiguous.
  2. [§5.4] The de-contamination step filters by 10-gram overlap, but it is unclear whether this is applied to problem descriptions or to generated code, and how many problems were removed. Please specify and quantify the filtering.
  3. [Table 3] The row labels 'BGE-M3SRCL' and 'BGE-M3SynH-Rank' are visually confusing. Use subscripts or clearly separated blocks, and consider highlighting the best result per column.
  4. [References / Related Work] Several related-work references are from the same research group (e.g., [7], [20], [23], [39]). This is not problematic per se, but the positioning of the quality-aware ranking contribution would benefit from a broader comparison with independent work on quality-aware IR and learning-to-rank with ordinal labels.

Circularity Check

0 steps flagged

No significant circularity; the headline QPA gain is an empirical transfer result from synthesized training data to independently measured benchmark labels.

full rationale

SynH-Rank's claimed QPA gains are not circular. The training signal is LLM-synthesized (TACO queries + Qwen3-Coder, Sections 4.1 and 5.4), whereas QualCode and MC-QualCode evaluation labels come from external, measured sources: CodeNet CPU-time/memory metadata and SonarQube code-smell analysis (Sections 3.1-3.2), so no parameter is fitted to the data on which it is evaluated. The hierarchical ranking loss in Equation (5) explicitly encodes the same ordering that QPA measures, but that is ordinary supervised learning on a training distribution, not a reduction: QPA is measured on held-out real triplets, and the benchmark construction never uses the model's own outputs. The self-citations ([7], [20], [23], [39], [41]) appear in background discussions of retrievers and rerankers, are corroborated by independent external references, and none is invoked as a uniqueness theorem or as justification for the central result. Section 8's conceded threat, that Qwen3-Coder's synthetic high/low labels are not validated by execution or static measurement, is a validity/externalizability concern about the training signal, not a circularity: nothing in the paper defines the evaluation labels in terms of the generated labels. No reduction of the claimed result to its own inputs is present.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The paper's contribution rests mainly on domain assumptions about the validity of CodeNet/SonarQube labels and LLM-synthesized quality variants. No new physical or conceptual entities are introduced; QPA and MCA are metrics, not entities. The free parameters are design choices for benchmark construction and training, none fitted to a hold-out external target.

free parameters (6)
  • Speed/memory 2× separation threshold = 2 × cpu_time_positive < cpu_time_negative
    Hand-chosen in §3.1 to ensure a clear positive/negative quality gap in QualCode.
  • MC-QualCode top/bottom 33% cutoffs = top 33% = 1, bottom 33% = 0
    Hand-chosen binary quality thresholds in §3.2; middle tier discarded.
  • Synthesis decoding parameters = temperature=0.7, top_p=0.8, top_k=20, max_tokens=1024
    Chosen in §5.4 to balance creativity and correctness; no tuning study reported.
  • Fine-tuning hyperparameters = batch_size=16, lr=6e-6, one epoch
    Chosen in §5.4; no sweep or cross-validation reported.
  • Negative composition per query = 4 low-quality relevant + 1 irrelevant
    Chosen in §5.4; directly shapes the hierarchy supervision.
  • Seed problem count = 1000 TACO problems
    Randomly sampled in §5.4 after decontamination; arbitrary scale decision.
axioms (6)
  • domain assumption CodeNet accepted submissions are functionally correct and CPU time/memory are reliable quality indicators.
    §3 selects only accepted submissions and uses min/max CPU time and memory to define high/low quality.
  • domain assumption SonarQube code-smell count and diversity are a valid proxy for maintainability.
    §3.1 uses SonarQube code smells as the maintainability metric without human validation.
  • ad hoc to paper LLM-synthesized variants carry the requested quality levels without execution verification.
    §4.1 generates c+ and c− from guidelines and filters only invalid/duplicate outputs; §8 admits model-specific bias.
  • domain assumption The three-level hierarchy high-quality relevant > low-quality relevant > irrelevant matches developer preferences.
    §1 and §4.2 define rel ∈ {2,1,0}; no user study in this paper validates the preference order.
  • domain assumption Per-dimension quality signals combine additively for multi-condition queries.
    §6.3 offers an additive-score explanation for MC-QualCode generalization; it is not tested directly.
  • domain assumption The 10-gram overlap decontamination fully prevents training/test leakage.
    §5.4 applies a 10-gram filter from DeepSeek-Coder; no leakage analysis is reported.

pith-pipeline@v1.3.0-alltime-deepseek · 18039 in / 12081 out tokens · 103234 ms · 2026-08-01T18:53:16.751743+00:00 · methodology

0 comments
read the original abstract

Code search enhances developer productivity by enabling efficient code reuse. Current code search systems often use a retrieve-then-rerank pipeline, where rerankers focus on modeling semantic relevance between queries and code. However, these rerankers overlook critical non-functional qualities like execution speed, memory usage, and maintainability, which are essential for practical software development. Studies reveal developers expect results to maintain high coding standards and satisfy specific needs, such as resource optimization, highlighting the importance of quality-aware code search. Achieving quality-aware code search faces two major challenges: the scarcity of quality-annotated datasets for effective training and the limitations of standard contrastive learning objectives, which fail to capture the ordinal relationships among high-quality, low-quality, and irrelevant code. Although contrastive learning excels in distinguishing relevant from irrelevant code, its binary objective does not support nuanced quality distinctions.To address these challenges, we propose SynH-Rank, a quality-aware code reranking framework that combines LLM-driven diverse data synthesis with hierarchical ranking training. SynH-Rank employs a three-level labeling scheme to explicitly model the hierarchy: high-quality relevant > low-quality relevant > irrelevant. Additionally, we introduce a new benchmark with 4,209 pairs and two novel metrics: Quality Preference Accuracy (QPA) for assessing prioritization of high-quality code and Multi-Condition Accuracy (MCA) for evaluating performance under complex constraints.Experimental results show SynH-Rank improves QPA by 20.15\% over backbone models and outperforms standard relevance-only contrastive training by 15.80\%, while simultaneously enhancing traditional relevance metrics and multi-condition generalizability.

Figures

Figures reproduced from arXiv: 2607.17139 by Haoye Wang, Keyu Liang, Yanfu Yan, Zhiyuan Wan, Zhongxin Liu.

Figure 1
Figure 1. Figure 1: Overall Framework of SynH-Rank. candidates of varying quality and relevance. In this section, we describe each component in turn. 4.1 Diverse Data Synthesis A fundamental bottleneck in quality-aware code search is the scarcity of labeled data covering non-functional properties. To overcome this, we synthesize paired code samples that differ in quality across three non-functional dimensions: speed, memory, … view at source ↗
Figure 2
Figure 2. Figure 2: Synthesis prompts for the speed dimension [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Intra-query diversity comparison under unfiltered [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Case study of reranking results between the back [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 14 linked inside Pith

  1. [1]

    Jina AI. 2024. Jina Reranker v2: base-multilingual model. https://huggingface.co/ jinaai/jina-reranker-v2-base-multilingual. Accessed: 2026-04-03

    2024

  2. [2]

    Alison Fernandez Blanco, Alexandre Bergel, and Juan Pablo Sandoval Alcocer

  3. [3]

    Gong Chen, Xiaoyuan Xie, Daniel Tang, Qi Xin, and Wenjie Liu. 2025. HedgeCode: A Multi-Task Hedging Contrastive Learning Framework for Code Search. In2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE, 1857–1868

    2025

  4. [4]

    Luca Di Grazia and Michael Pradel. 2023. Code search: A survey of techniques for finding code.Comput. Surveys55, 11 (2023), 1–31

    2023

  5. [5]

    Mingzhe Du, Luu A Tuan, Bin Ji, Qian Liu, and See-Kiong Ng. 2024. Mercury: A code efficiency benchmark for code large language models.Advances in Neural Information Processing Systems37 (2024), 16601–16622

    2024

  6. [6]

    Guodong Fan, Shizhan Chen, Cuiyun Gao, Jianmao Xiao, Tao Zhang, and Zhiyong Feng. 2024. Rapid: Zero-shot domain adaptation for code search with pre-trained models.ACM Transactions on Software Engineering and Methodology33, 5 (2024), 1–35

    2024

  7. [7]

    Lishui Fan, Jiakun Liu, Zhongxin Liu, David Lo, Xin Xia, and Shanping Li. 2025. Exploring the capabilities of llms for code-change-related tasks.ACM Transactions on Software Engineering and Methodology34, 6 (2025), 1–36

    2025

  8. [8]

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020. Codebert: A pre-trained model for programming and natural languages. InFindings of the association for computational linguistics: EMNLP 2020. 1536–1547

    2020

  9. [9]

    Jiahui Geng, Fengyu Cai, Shaobo Cui, Qing Li, Liangwei Chen, Chenyang Lyu, Haonan Li, Derui Zhu, Walter Pretschner, Heinz Koeppl, et al. 2025. CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval. arXiv preprint arXiv:2506.11066(2025)

    Pith/arXiv arXiv 2025

  10. [10]

    Akhilesh Deepak Gotmare, Junnan Li, Shafiq Joty, and Steven CH Hoi. 2021. Cascaded fast and slow models for efficient semantic code search.arXiv preprint arXiv:2110.07811(2021)

    Pith/arXiv arXiv 2021

  11. [11]

    Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. Unixcoder: Unified cross-modal pre-training for code representation. InProceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 7212–7225

    2022

  12. [12]

    Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. 2020. Graphcodebert: Pre-training code representations with data flow.arXiv preprint arXiv:2009.08366 (2020)

    Pith/arXiv arXiv 2020

  13. [13]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yifan Wu, YK Li, et al. 2024. DeepSeek-Coder: when the large language model meets programming–the rise of code intelligence.arXiv preprint arXiv:2401.14196(2024)

    Pith/arXiv arXiv 2024

  14. [14]

    Jiawei Guo, Ziming Li, Xueling Liu, Kaijing Ma, Tianyu Zheng, Zhouliang Yu, Ding Pan, Yizhi Li, Ruibo Liu, Yue Wang, et al. 2024. Codeeditorbench: Evaluating code editing capability of large language models.arXiv preprint arXiv:2404.03543 (2024)

    Pith/arXiv arXiv 2024

  15. [15]

    Fan Hu, Yanlin Wang, Lun Du, Xirong Li, Hongyu Zhang, Shi Han, and Dongmei Zhang. 2023. Revisiting code search in a two-stage paradigm. InProceedings Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al. of the sixteenth ACM international conference on Web search and data mining. 994–1002

    2023

  16. [16]

    Dong Huang, Jianbo Dai, Han Weng, Puzhen Wu, Yuhao Qing, Heming Cui, Zhi- jiang Guo, and Jie M Zhang. 2024. Effilearner: Enhancing efficiency of generated code via self-optimization.Advances in Neural Information Processing Systems37 (2024), 84482–84522

    2024

  17. [17]

    Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019. Codesearchnet challenge: Evaluating the state of semantic code search.arXiv preprint arXiv:1909.09436(2019)

    Pith/arXiv arXiv 2019

  18. [18]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. InProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP). 6769–6781

    2020

  19. [19]

    Rongao Li, Jie Fu, Bo-Wen Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, and Ge Li. 2023. Taco: Topics in algorithmic code generation dataset. arXiv preprint arXiv:2312.14852(2023)

    Pith/arXiv arXiv 2023

  20. [20]

    Keyu Liang, Zhongxin Liu, Chao Liu, Zhiyuan Wan, David Lo, and Xiaohu Yang

  21. [21]

    Chao Liu, Xindong Zhang, Hongyu Zhang, Zhiyuan Wan, Zhan Huang, and Meng Yan. 2024. An empirical study of code search in intelligent coding assistant: Perceptions, expectations, and directions. InCompanion Proceedings of the 32nd ACM international conference on the foundations of software engineering. 283–293

    2024

  22. [22]

    Zhijie Liu, Yutian Tang, Xiapu Luo, Yuming Zhou, and Liang Feng Zhang. 2024. No need to lift a finger anymore? assessing the quality of code generation by chatgpt.IEEE Transactions on Software Engineering50, 6 (2024), 1548–1584

    2024

  23. [23]

    Zhongxin Liu, Zhijie Tang, Junwei Zhang, Xin Xia, and Xiaohu Yang. 2024. Pre- training by predicting program dependencies for vulnerability analysis tasks. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13

    2024

  24. [24]

    Xuan Lu, Sifan Liu, Bochao Yin, Yongqi Li, Xinghao Chen, Hui Su, Yaohui Jin, Wenjun Zeng, and Xiaoyu Shen. 2025. Multiconir: Towards multi-condition information retrieval.Preprint(2025)

    2025

  25. [25]

    Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. Docu- ment ranking with a pretrained sequence-to-sequence model. InFindings of the association for computational linguistics: EMNLP 2020. 708–718

    2020

  26. [26]

    Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-stage document ranking with BERT.arXiv preprint arXiv:1910.14424(2019)

    Pith/arXiv arXiv 2019

  27. [27]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748(2018)

    Pith/arXiv arXiv 2018

  28. [28]

    Yun Peng, Akhilesh Deepak Gotmare, Michael R Lyu, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. 2025. Perfcodegen: Improving performance of llm generated code with execution feedback. In2025 IEEE/ACM Second International Conference on AI Foundation Models and Software Engineering (Forge). IEEE, 1–13

    2025

  29. [29]

    Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, et al

  30. [30]

    Revanth Gangi Reddy, JaeHyeok Doo, Yifei Xu, Md Arafat Sultan, Deevya Swain, Avirup Sil, and Heng Ji. 2024. FIRST: Faster improved listwise reranking with single token decoding. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 8642–8652

    2024

  31. [31]

    Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundare- san, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020. Codebleu: a method for automatic evaluation of code synthesis.arXiv preprint arXiv:2009.10297(2020)

    Pith/arXiv arXiv 2020

  32. [32]

    Liangying Shao, Yanfu Yan, Denys Poshyvanyk, and Jinsong Su. 2025. UniGen- Coder: Merging SEQ2SEQ and SEQ2TREE Paradigms for Unified Code Genera- tion. In2025 IEEE/ACM 47th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). IEEE, 71–75

    2025

  33. [33]

    Ensheng Shi, Yanlin Wang, Wenchao Gu, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, and Hongbin Sun. 2023. Cocosoda: Effective contrastive learning for code search. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2198–2210

    2023

  34. [34]

    Manav Singhal, Tushar Aggarwal, Abhijeet Awasthi, Nagarajan Natarajan, and Aditya Kanade. 2024. Nofuneval: Funny how code lms falter on requirements beyond functional correctness.arXiv preprint arXiv:2401.15963(2024)

    Pith/arXiv arXiv 2024

  35. [35]

    Adam Tornhill and Markus Borg. 2022. Code red: the business impact of code quality-a quantitative study of 39 proprietary production codebases. InProceed- ings of the International Conference on Technical Debt. 11–20

    2022

  36. [36]

    Kexin Wang, Nandan Thakur, Nils Reimers, and Iryna Gurevych. 2022. GPL: Gen- erative pseudo labeling for unsupervised domain adaptation of dense retrieval. In Proceedings of the 2022 conference of the North American chapter of the association for computational linguistics: human language technologies. 2345–2360

    2022

  37. [37]

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-pack: Packed resources for general chinese embeddings. InProceedings of the 47th international ACM SIGIR conference on research and development in information retrieval. 641–649

    2024

  38. [38]

    Zhipeng Xue, Xiaoting Zhang, Zhipeng Gao, Xing Hu, Shan Gao, Xin Xia, and Shanping Li. 2025. Clean Code, Better Models: Enhancing LLM Performance with Smell-Cleaned Dataset.ACM Transactions on Software Engineering and Methodology(2025)

    2025

  39. [39]

    Yanfu Yan, Nathan Cooper, Kevin Moran, Gabriele Bavota, Denys Poshyvanyk, and Steve Rich. 2024. Enhancing code understanding for impact analysis by combining transformers and program dependence graphs.Proceedings of the ACM on Software Engineering1, FSE (2024), 972–995

    2024

  40. [40]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

    Pith/arXiv arXiv 2025

  41. [41]

    Junwei Zhang, Zhongxin Liu, Xing Hu, Xin Xia, and Shanping Li. 2023. Vulner- ability detection by learning from syntax-based execution paths of code.IEEE Transactions on Software Engineering49, 8 (2023), 4196–4212

    2023

  42. [42]

    Sheng Zhang, Yifan Ding, Shuquan Lian, Shun Song, and Hui Li. 2025. CodeRAG: Finding Relevant and Necessary Knowledge for Retrieval-Augmented Repository- Level Code Completion. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 23289–23299

    2025

  43. [43]

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models.arXiv preprint arXiv:2506.05176(2025). Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

    Pith/arXiv arXiv 2025

  44. [2021]

    Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks.arXiv preprint arXiv:2105.12655(2021)

    Pith/arXiv arXiv 2021

  45. [2022]

    Software visualizations to analyze memory consumption: A literature review.ACM Computing Surveys (CSUR)55, 1 (2022), 1–34

    2022

  46. [2025]

    Zero-shot cross-domain code search without fine-tuning.Proceedings of the ACM on Software Engineering2, FSE (2025), 1937–1959

    2025