Pith. sign in

REVIEW 3 major objections 6 minor 27 references

Cluster Purge Loss: Structuring Transformer Embeddings for Equivalent Mutants Detection

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new loss function, Cluster Purge Loss, added to cross-entropy during fine-tuning, re-organises transformer embeddings inside each mutant class and lifts equivalent-mutant detection F1 on the Java benchmark from 87.22% to 89.46%.

desk verdict CPL is a genuinely new metric loss with a clean ablation, but the headline SOTA gain is best-case from test-set tuning with train/test classes overlapping; the robust median gain is much smaller. read the letter →

arxiv 2507.20078 v1 pith:GEZ3L727 submitted 2025-07-26 cs.LG

Adelaide Danilov , Aria Nourbakhsh , Christoph Schommer This is my paper · ORCID
classification cs.LG
keywords equivalentmutantdetectionClusterPurgeLossdeepmetriclearningembeddingspacestructuringmutationtestingtransformerfine-tuningcontrastivecoderepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Equivalent code mutants are altered programs that still behave exactly like the original, and they quietly distort mutation-testing scores. This paper argues that transformer fine-tuning fails on them because the embedding space separates mutant classes but leaves the two kinds of mutants inside a class overlapping. To fix that, it adds Cluster Purge Loss to the usual cross-entropy objective: the loss tracks per-class running averages of distances from the origin program to equivalent and to non-equivalent mutants, then pushes each mutant past the other group's boundary. On the Java benchmark the joint loss raises F1-score from 87.22% to 89.46%, and the same ranking over baselines holds on a smaller C-language dataset. The paper's claim is that the missing ingredient was semantic organisation of intra-class distances, not a stronger classifier head.

What carries the argument

The central object is the Cluster Purge Loss itself. A 'verge' is a dynamically adjusted boundary for one mutant class, computed as the Exponential Moving Average of distances from the origin embedding to one type of mutant. The loss maintains two verges per class—a positive one for equivalent mutants and a negative one for non-equivalent mutants—and uses their crossing as the decision point. It penalises any equivalent mutant farther from the origin than the negative verge by more than a margin $\zeta$, and any non-equivalent mutant closer than the positive verge by more than $\zeta$; the exponents $\alpha$ and $\beta$ make the two penalties asymmetric. This machinery is what lets a fixed classifier head exploit semantic distances it could not otherwise see.

What would settle it

Hold out entire origin programs when building the test set and train on the rest; if Cluster Purge Loss then stops beating cross-entropy-only fine-tuning, the claim that it structures intra-class semantics generally fails. A second check is a multi-seed rerun at the best hyperparameters: if the 2.24 pp F1 margin over the baseline does not survive, the reported gain is within run-to-run noise.

Watch

Extended reading notes

Core claim

Cluster Purge Loss structures the embedding space at the level of individual mutant classes rather than between classes. For each class—mutants descended from one origin program—the loss keeps two Exponential Moving Averages of distances to the origin embedding: the positive verge for equivalent mutants and the negative verge for non-equivalent mutants. On each minibatch, a ReLU (max-with-zero) penalty term penalises an equivalent mutant that lies beyond the negative verge (plus margin $\zeta$) and penalises a non-equivalent mutant inside the positive verge (minus $\zeta$), with exponents $\alpha$ and $\beta$ shaping the two sides. Combined with cross-entropy as $L = L_{\mathrm{CPL}}\cdot\lambda + L_{\mathrm{CE}}$, this moves non-equivalent mutants much farther from their origin (mean distance $0.105\pm0.133$ to $0.398\pm0.303$, $p<0.0001$) while leaving equivalent mutants comparatively close, and it produces the highest F1-score on the benchmark. The paper concludes that the improvement is attributable to the way CPL reorganises the embedding space.

Load-bearing premise

The load-bearing premise is that the random train/test split tests the ability that matters; since the same origin programs appear in both sets, the method has not been shown to generalise to unseen programs.

Editorial extensions

If this is right

  • CPL can be dropped into an existing fine-tuning pipeline without touching the model architecture: it only needs class ids and an origin embedding per class.
  • The method keeps the same ranking on datasets with opposite class balance (15% vs 83% equivalent mutants), so the gain is not tied to a particular distribution of labels.
  • After training, distance to the origin becomes a usable semantic signal: non-equivalent mutants sit about twice as far as equivalent ones, which could support ranking or explanation of mutants.
  • The best hyperparameters put $\lambda$ just above 1 and $\zeta$ negative, which the paper reads as CPL acting as a corrective regulariser rather than the dominant loss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: a class-disjoint train/test split—training on some origin programs, testing on others—would show whether the reported gain is genuine structuring of intra-class semantics or familiarity with the 52 origin programs that already appear in both splits.
  • Extension: the same per-class verge mechanism could transfer to other binary semantic-difference tasks, such as clone detection or bug-fix verification, wherever a class has a canonical representative embedding to measure distances from.
  • Extension: because each hyperparameter configuration was run once, a multi-seed rerun at the best configuration would reveal how much of the 2.24 pp F1 margin over the baseline is stable and how much is seed noise.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes Cluster Purge Loss (CPL), an auxiliary deep-metric-learning term added to cross-entropy for fine-tuning UniXCoder on equivalent-mutant detection. CPL maintains per-class exponential moving averages of distances from the origin to equivalent and non-equivalent mutants ('verges') and applies hinge-like penalties so equivalent mutants are pulled inside the non-equivalent verge while non-equivalents are pushed beyond the equivalent verge, with margin ζ and exponents α, β, λ. On a Java mutant benchmark the authors report F1 = 89.46% at λ=1.15, ζ=-0.05, versus 87.22% for the re-run baseline and 88.18% for a contrastive-loss variant; a smaller C-language dataset shows the same ranking. The paper also reports distance statistics and t-SNE visualizations to argue that CPL reorganizes the embedding space.

Significance. If the reported gains were robust, the paper would make a modest but useful contribution: a simple, resource-lean loss term that improves a strong code model on a practically relevant binary classification task, with an interpretable geometric meaning. The inclusion of a contrastive baseline and a second language is a good-faith attempt at controlled ablation. However, the headline F1 is a test-set maximum over 56 configurations from a single trial per configuration, and the pair-level data split leaves open a memorization-based alternative explanation; these issues currently put the SOTA claim on fragile ground.

major comments (3)
  1. [§3.3.1 and §5] The reported 89.46% F1 is the best of 56 (λ, ζ) configurations evaluated on the test set, with only one run per configuration (Section 5). A test-set-selected maximum over a grid is not a reliable estimate of the method's performance; the median CPL F1 across the grid is 87.52% (Appendix B), only 0.30 pp above the 87.22% baseline. The contrastive baseline was also selected from 42 configurations but on its own margin range, so the comparison is best versus best. I request a validation split for hyperparameter selection, multiple seeds per configuration, and reporting of mean ± std, with the baseline given comparable tuning or the comparison explicitly framed as a tuned-model comparison with that caveat.
  2. [§3.1 and Appendix C.1] The train/test split is made at the level of mutant pairs, not origin programs. Class IDs are then introduced for all 52 origins; because the 1652/1650 pair split (later 1590/1580 after deduplication) is random, the same origin classes very likely occur in both training and test. Since CPL's training signal is explicitly class-conditioned through per-class verges v+_c and v−_c (Equations 7–9), the model may be learning per-origin calibration rather than a transferable notion of semantic equivalence. The paper does not report whether test classes were seen in training, and the C dataset (Appendix C.1) uses the same pair-level split. I request a class-disjoint (leave-origin-out) evaluation and a report of train/test class overlap; without it, the SOTA claim is conditional.
  3. [§3.3.2] The embedding-space result is presented as evidence that CPL improves the model (p < 0.0001 for non-equivalents; ratio 0.95→2.11). But the loss in Equation (9) directly enforces exactly this separation (equivalents inside v−, non-equivalents beyond v+), so the observed distance change is a check that the objective was optimized, not independent evidence for the mechanism. The visualization is also limited to two origin classes. I suggest reframing this section as a sanity check of the objective and moving the evidential weight onto the classification F1 under a properly held-out split.
minor comments (6)
  1. [§2.1, Eq. (9)] Equation (9) introduces the ReLU notation [·]_+ without defining it; define it before first use, and also rewrite Equation (7), where `v^+_c = 0 =⇒ v^+_c = d^+_c 0` is ungrammatical and ambiguous between initialization and update.
  2. [§3.1] The reduction from 1652 to 1590 training pairs and 1650 to 1580 test pairs after 'duplicates being removed' is not explained; state the deduplication criterion.
  3. [§3.3.2] Report which statistical test produced the p-values, and clarify whether the equivalent-mutant comparison (p = 0.83) is a test of no difference.
  4. [Appendix E] Appendix E says 'the 98 reported experiments' when the total across Java and C is 196; align this wording with Section 5.
  5. [Throughout] Capitalization of model names is inconsistent (modelbase/modelCPL/modelcontrast versus modelcpl/modelbase); unify the notation.
  6. [§2.1] The text says 'we can compute the loss' but the symmetric equation for v−_c is not written out; include the negative-verge definition explicitly for readability.

Circularity Check

2 steps flagged · score 6.0 of 10

Headline F1 is selected from a 56-point test-set grid, and the embedding-space confirmation merely checks CPL's own objective; no self-citation chain.

  1. self definitional [Section 3.3.2 vs. Eq. (9) in Section 2.1]
    "To prove the hypothesis about Cluster Purge Loss promoting more organized embedding space, which is beneficial for EMD, the embeddings of mutants with origin 1408 and 2001 were extracted from the best performing modelcpl and plotted after applying T-SNE (Figure 3). ... Thus, we can conclude that our hypothesis holds and the introduction of CPL improved the performance on the equivalent mutant detection task by promoting the semantic meaning on distances between embeddings in the intra-class context."

    Eq. (9) defines LCPL as a sum of ReLU penalties on (dist(oki,si) - v-ki + zeta) for equivalents and (v+ki - dist(oki,si) + zeta) for non-equivalents. By construction, optimizing this loss pushes non-equivalent mutants farther from the origin and equivalent mutants closer to it. The reported increase in mean non-equivalent distance (0.105 to 0.398) and the ratio change from 0.95 to 2.11 are therefore direct checks that the loss's own objective was optimized, not independent evidence that the embedding space became semantically better organized. The conclusion that CPL 'promotes semantic meaning on distances' simply restates the assumption encoded in the loss.

  2. fitted input called prediction [Section 3.3.1, Table 1, and Section 5]
    "We conducted 56 experiments by fine-tuning modelcpl on traincpl and evaluating on testcpl with dist being normalized cosine distance, gamma = 12, alpha = 2, beta = 1/2, zeta in [-0.06, 0.01] with a step 0.01 and lambda in [1.00, 1.30] with a step 0.05. ... The best result in terms of F1-score is (P: 95.31%, R: 85.41%, F1: 89.46%) at lambda = 1.15 and zeta = -0.05. ... The main limitation of our work is that we ran only one trial for each of the 196 hyperparameter experiments due to limited computational resources."

    The headline 89.46% F1 is the maximum over 56 hyperparameter configurations, all evaluated on the same testcpl set used to choose the best configuration. The selected (lambda, zeta) are therefore fitted to the test labels, and the reported number is a selection-maximized estimate rather than an unbiased prediction on unseen data. Comparing this best-of-56 result against a single re-run of modelbase (87.22%) inflates the claimed 2.24 pp gain; the paper's own limitation note confirms only one trial per configuration, so the advantage may be noise. This is a fitted choice presented as a predicted generalization result.

full rationale

The paper's core classification experiments are a held-out evaluation and are not circular in the equation-reduction sense, but two load-bearing parts of the argument are. First, the embedding-space evidence in Section 3.3.2 is self-definitional: the loss in Eq. (9) explicitly minimizes distances of equivalents to the origin and maximizes distances of non-equivalents, so observing those distances change after training simply verifies that the loss was optimized; it does not independently confirm the mechanism. Second, the SOTA claim depends on hyperparameters selected on the test set: 56 configurations are evaluated on testcpl and the best F1 is reported, which is a fitted maximum rather than a robust prediction, especially given the single-run limitation admitted in Section 5. The pair-level train/test split and the resulting reuse of origin classes is a real generalization risk, but it is a data-leakage concern rather than a definitional circularity, so it does not enter the score. There is no load-bearing self-citation chain: the baseline and dataset come from an external source (Tian et al.), and no uniqueness theorem or prior-work ansatz is invoked to force the choice of CPL. Overall, partial circularity in the reported evidence justifies a score of 6.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The CPL loss introduces five tunable hyperparameters (lambda, zeta, gamma, alpha, beta), of which lambda and zeta are fitted by grid search on the test set and the rest are hand-set with stated assumptions. The method relies on the correctness of equivalence labels and on the assumption that origin-embedding distance is a valid proxy for semantic equivalence. No new physical or conceptual entities are introduced besides the loss function's internal EMA vectors (verges), which are computational state, not independent postulates.

free parameters (5)
  • lambda (CPL weight) = 1.15 (Java best), 1.30 (C best)
    Grid-searched over [1.00, 1.30] with step 0.05; selected by maximizing test-set F1 (Section 3.3.1, Appendix C.2.1).
  • zeta (margin) = -0.05 (Java best), -0.01 (C best)
    Grid-searched over [-0.06, 0.01] with step 0.01; selected by maximizing test-set F1 (Section 3.3.1, Appendix A).
  • gamma (EMA smoothing factor) = 12
    Fixed at 12 based on preliminary experiments; not grid-searched (Appendix A).
  • alpha (exponent for equivalent term) = 2
    Chosen by hand based on the assumption that equivalent mutants are already close to the origin (Appendix A); not grid-searched.
  • beta (exponent for non-equivalent term) = 0.5
    Chosen by hand to emphasize changing the distribution of non-equivalent mutants (Appendix A); not grid-searched.
assumptions (5)
  • domain assumption Ground-truth equivalence labels in the Java and C datasets are correct.
    The dataset is taken from MutantBench and Tian et al. The paper does not validate the labels; the entire training and evaluation depend on their correctness (Section 3.1).
  • domain assumption The train/test split is by mutant pair, so origin programs appear in both sets.
    Section 3.1 describes a 50/50 split of pairs without class-level separation. This means the test set likely contains classes seen during training, affecting the generalization interpretation.
  • domain assumption Embedding distance to the origin is a meaningful proxy for semantic equivalence.
    This is the premise of the CPL objective (Section 2.1). If false, the loss would not provide useful structure.
  • domain assumption The UniXCoder model with GGMA and RoBERTa head from Tian et al. is a sound base for the task.
    The paper inherits the base model, input construction, and preprocessing from prior work (Section 3.2) without re-benchmarking those components.
  • standard math The EMA update formulas (Eqs. 5-8) correctly compute exponential moving averages.
    The closed form for batched EMA updates is a standard algebraic identity; no errors found.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cluster Purge Loss: Structuring Transformer Embeddings for Equivalent Mutants Detection." pith.science (2026). https://pith.science/paper/GEZ3L727

@misc{pith2026250720078,
  author       = {Pith},
  title        = {Pith review of: Cluster Purge Loss: Structuring Transformer Embeddings for Equivalent Mutants Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GEZ3L727}},
  note         = {Machine review of arXiv:2507.20078}
}
read the original abstract

Recent pre-trained transformer models achieve superior performance in various code processing objectives. However, although effective at optimizing decision boundaries, common approaches for fine-tuning them for downstream classification tasks - distance-based methods or training an additional classification head - often fail to thoroughly structure the embedding space to reflect nuanced intra-class semantic relationships. Equivalent code mutant detection is one of these tasks, where the quality of the embedding space is crucial to the performance of the models. We introduce a novel framework that integrates cross-entropy loss with a deep metric learning objective, termed Cluster Purge Loss. This objective, unlike conventional approaches, concentrates on adjusting fine-grained differences within each class, encouraging the separation of instances based on semantical equivalency to the class center using dynamically adjusted borders. Employing UniXCoder as the base model, our approach demonstrates state-of-the-art performance in the domain of equivalent mutant detection and produces a more interpretable embedding space.

Figures

Figures reproduced from arXiv: 2507.20078 by the authors.

Figure 1
Figure 1. CPL conditions mutants to cross the Expo [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Baseline embeddings for classes with origins 1408 and 2001 [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Embeddings after Cross-Entropy and Cluster Purge Loss for classes with origins 1408 and 2001 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The process of mutation testing of the test suite for the original program. Equivalent mutants don’t let [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Model performance matrix for different values of [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Performance matrix for the selected range of [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 8 canonical work pages

  1. [1]

    Michael Baer, Norbert Oster, and Michael Philippsen. 2020. https://doi.org/10.1109/ICSTW50294.2020.00055 MutantDistiller : Using Symbolic Execution for Automatic Detection of Equivalent Mutants and Generation of Mutant Killing Tests . In 2020 IEEE International Conference on Software Testing , Verification and Validation Workshops ( ICSTW ) , pages 294--3...

    arXiv 2020

  2. [2]

    Chopra, R

    S. Chopra, R. Hadsell, and Y. LeCun. 2005. https://doi.org/10.1109/CVPR.2005.202 Learning a similarity metric discriminatively, with application to face verification . In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), volume 1, pages 539--546 vol. 1

    doi:10.1109/cvpr.2005.202 2005

  3. [3]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre -training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the 2019 Conference of the North , pages 4171--4186, Minneapolis, Minnesota. Association for Computational Linguistics

    doi:10.18653/v1/n19-1423 2019

  4. [4]

    Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. https://doi.org/10.18653/v1/2022.acl-long.499 UniXcoder : Unified Cross - Modal Pre -training for Code Representation . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( Volume 1: Long Papers ) , pages 7212--7225, Dublin, Ireland. Associati...

    doi:10.18653/v1/2022.acl-long.499 2022

  5. [5]

    Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou

    Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin B. Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. https://openreview.net/forum?id=jLoC4ez43PZ Graphcodebert: Pre-training code representations with data flow

    work page 2021

  6. [6]

    Yue Jia and Mark Harman. 2011. https://doi.org/10.1109/TSE.2010.62 An Analysis and Survey of the Development of Mutation Testing . IEEE Transactions on Software Engineering, 37(5):649--678

    doi:10.1109/tse.2010.62 2011

  7. [7]

    Marinos Kintis, Mike Papadakis, Yue Jia, Nicos Malevris, Yves Le Traon, and Mark Harman. 2018. https://doi.org/10.1109/TSE.2017.2684805 Detecting Trivial Mutant Equivalences via Compiler Optimisations . IEEE Transactions on Software Engineering, 44(4):308--333

    arXiv 2018

  8. [8]

    Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/1be5bc25d50895ee656b8c2d9eb89d6a-Paper-Conference.pdf Diffusion-lm improves controllable text generation . In Advances in Neural Information Processing Systems, volume 35, pages 4328--4343. Curran Associates, Inc

    work page 2022

Show all 27 references
  1. [9]

    Yiling Lou, Dan Hao, and Lu Zhang. 2015. https://doi.org/10.1109/ISSRE.2015.7381798 Mutation-based test-case prioritization in software evolution . In 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), pages 46--57

    2015

  2. [10]

    Hao Luo, Youzhi Gu, Xingyu Liao, Shenqi Lai, and Wei Jiang. 2019. https://doi.org/10.1109/CVPRW.2019.00190 Bag of tricks and a strong baseline for deep person re-identification . In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1487--1495

    2019

  3. [11]

    Deen Dayal Mohan, Bhavin Jawade, Srirangaraj Setlur, and Venu Govindaraju. 2023. https://doi.org/10.1016/bs.host.2023.01.003 Chapter 4 - deep metric learning for computer vision: A brief overview . In Venu Govindaraju, Arni S.R. Srinivasa Rao , and C.R. Rao, editors, Deep Lear...

    2023 doi

  4. [12]

    Muhammad Rashid Naeem, Tao Lin, Hamad Naeem, and Hailu Liu. 2020. https://doi.org/10.1002/smr.2238 A machine learning approach for classification of equivalent mutants . Journal of Software: Evolution and Process, 32(5):e2238. E2238 smr.2238

    2020 doi

  5. [13]

    Erik Nijkamp, Hiroaki Hayashi, Caiming Xiong, Silvio Savarese, and Yingbo Zhou. 2023. https://doi.org/10.48550/ARXIV.2305.02309 Codegen2: Lessons for training llms on programming and natural languages . CoRR, arXiv:2305.02309

    2023 doi arXiv

  6. [14]

    Mike Papadakis, Yue Jia, Mark Harman, and Yves Le Traon. 2015. https://doi.org/10.1109/ICSE.2015.103 Trivial compiler equivalence: A large scale empirical study of a simple, fast and effective equivalent mutant detection technique . In 2015 IEEE/ACM 37th IEEE International Con...

    2015 doi

  7. [15]

    Mike Papadakis and Yves Le Traon. 2015. https://doi.org/10.1002/stvr.1509 Metallaxis-fl: mutation-based fault localization . Softw. Test. Verif. Reliab., 25(5–7):605–628

    2015 doi

  8. [16]

    Venkatesh Balavadhani Parthasarathy, Ahtsham Zafar, Aafaq Khan, and Arsalan Shahid. 2024. https://doi.org/10.48550/arXiv.2408.13296 The Ultimate Guide to Fine - Tuning LLMs from Basics to Breakthroughs : An Exhaustive Review of Technologies , Research , Best Practices , Applie...

    2024 doi arXiv

  9. [17]

    Samuel Peacock, Lin Deng, Josh Dehlinger, and Suranjan Chakraborty. 2021. https://doi.org/10.1109/ICSTW52544.2021.00016 Automatic Equivalent Mutants Classification Using Abstract Syntax Tree Neural Networks . In 2021 IEEE International Conference on Software Testing , Verifica...

    2021

  10. [18]

    Michael Pradel and Koushik Sen. 2018. https://doi.org/10.1145/3276517 Deepbugs: a learning approach to name-based bug detection . Proc. ACM Program. Lang., 2(OOPSLA)

    2018 doi

  11. [19]

    Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most Marufatul Jannat Mim, Jubaer Ahmad, Mohammed Eunus Ali, and Sami Azam

    Mohaimenul Azam Khan Raiaan, Md. Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most Marufatul Jannat Mim, Jubaer Ahmad, Mohammed Eunus Ali, and Sami Azam. 2024. https://doi.org/10.1109/ACCESS.2024.3365742 A Review on Large Language Models : Architecture...

    2024

  12. [20]

    Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. https://doi.org/10.1109/CVPR.2015.7298682 Facenet: A unified embedding for face recognition and clustering . In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 201...

    2015

  13. [21]

    Eu Wern Teh, Terrance DeVries, and Graham W. Taylor. 2020. https://doi.org/10.1007/978-3-030-58586-0_27 ProxyNCA ++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis . In Computer Vision – ECCV 2020: 16th European Conference , Glasgow , UK , August 23–28, 2020...

    2020 doi

  14. [22]

    Zhao Tian, Honglin Shu, Dong Wang, Xuejie Cao, Yasutaka Kamei, and Junjie Chen. 2024. https://doi.org/10.48550/arXiv.2408.01760 Large Language Models for Equivalent Mutant Detection : How Far Are We ? arXiv preprint. ArXiv:2408.01760

    2024 doi arXiv

  15. [23]

    Lars van Hijfte and Ana Oprescu. 2021. https://icst2021.icmc.usp.br/details/mutation-2021-papers/4/MutantBench-an-Equivalent-Mutant-Problem-Comparison-Framework Mutantbench: an equivalent mutant problem comparison framework . In n 2021 IEEE International Conference on Software...

    2021

  16. [24]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Attention is All you Need . In Advances in Neur...

    2017

  17. [25]

    Zibin Zheng, Kaiwen Ning, Yanlin Wang, Jingwen Zhang, Dewu Zheng, Mingxi Ye, and Jiachi Chen. 2023. https://doi.org/10.48550/ARXIV.2311.10372 A Survey of Large Language Models for Code : Evolution , Benchmarking , and Future Trends . arXiv preprint

    2023 doi arXiv

  18. [26]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  19. [27]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.