Pith. sign in

REVIEW 4 major objections 5 minor 48 references

On The Impact of Merge Request Deviations on Code Review Practices

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that over a third of industrial merge requests are workflow deviations, that few-shot learning can flag them at 91 percent accuracy, and that removing them changes both the performance and interpretation of review-time…

desk verdict Useful taxonomy and credible prevalence estimates, but the headline impact numbers depend on an unvalidated classifier deployment. read the letter →

arxiv 2506.08860 v2 pith:XHMUE4LC submitted 2025-06-10 cs.SE cs.AIcs.LG

Samah Kansab , Francis Bordeleau , Ali Tizghadam This is my paper
classification cs.SEcs.AIcs.LG
keywords codereviewmergerequestsdeviationdetectionfew-shotlearningSetFitcompletiontimefeatureimportanceindustrialcasestudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most code review research treats every merge request as a genuine attempt to evaluate code. The authors argue that industrial practice is messier: across four teams and 26.7k merge requests, up to 37.02 percent of merge requests are deviations such as library updates, build or configuration tweaks, code cleaning, work-in-progress drafts, reverts, huge changes, or empty change sets, which bypass or do not need formal review. They build a few-shot SetFit classifier that detects these deviations with up to 91 percent accuracy using only 15 examples per class. They then show that removing deviations from datasets used to predict review completion time improves model performance in 53.33 percent of cases, by up to 2.25x, and substantially changes feature-importance rankings in 47 to 60 percent of cases. The upshot is that review analytics and ML-based effort models should filter workflow deviations before drawing conclusions about how reviews really work.

What carries the argument

The central object is the deviation label: a binary distinction between a standard review MR and an MR that breaks the expected review workflow, defined by seven categories named EOW, CC, LU, BOCA, RC, HC, and ECS. The detection machinery is SetFit, a few-shot sentence-transformer framework that fine-tunes a T5 model on small labeled sets, which the paper uses with 15 examples per class to label the full dataset. The impact machinery is a 100-resample bootstrap comparison of XGBoost, ExtraTree, and Random Forest models predicting review completion time, with Cliff's $\Delta$ and Wilcoxon tests for performance differences and Kendall's tau plus Top-K overlap for feature-importance shifts.

What would settle it

Manually label a random sample of a few hundred MRs from the same four projects, compare the SetFit predictions to human labels, and re-run the RQ3 comparison using only human-verified deviation labels; if the classifier misses or over-predicts particular categories, the reported performance gains and feature-importance shifts will shrink or disappear.

Watch

Extended reading notes

Core claim

The paper's central claim is that a substantial share of industrial merge requests are not genuine code reviews, and that treating them as such biases both predictive models and their interpretation. The authors define seven deviation categories, ranging from library updates and configuration adjustments to work-in-progress experiments and empty change sets, and show that these account for up to 37.02 percent of merge requests in their industrial dataset. They demonstrate that a few-shot SetFit model, trained on 15 examples per class, detects these deviations with up to 91 percent accuracy and outperforms fine-tuned BERT, making large-scale automatic filtering practical. When the classifier-labeled deviations are excluded, XGBoost, ExtraTree, and Random Forest models of review completion time show statistically significant improvements in 53.33 percent of cases (up to 2.25x on the SA metric), with overall feature-importance rankings changing in 47 percent of cases and top-k rankings in 60 percent. The authors read this as evidence that deviation removal mitigates biases that would otherwise lead to misleading process optimizations.

Load-bearing premise

The RQ3 conclusions assume the SetFit classifier's deviation labels for all 26.7k MRs are accurate, but the paper reports only that a random sample of predictions was validated with industrial partners, without giving sample size or agreement.

Editorial extensions

If this is right

  • Review analytics pipelines should screen merge requests for deviations before using them to estimate review effort, since leaving deviations in can distort both performance and interpretation.
  • ML models predicting review completion time improve or stay unchanged in 77.57 percent of cases when deviations are removed, with statistically significant gains in 53.33 percent of cases, making deviation filtering a cheap preprocessing step.
  • Feature-importance rankings used to explain what drives review time change substantially after deviation removal, so conclusions about which factors matter most may be unreliable on unfiltered data.
  • A few-shot classifier with 15 examples per class reaches up to 91 percent accuracy, meaning teams can automate deviation detection without large manual labeling efforts.
  • The taxonomy is not closed: applying the approach to an open-source project revealed an additional documentation-update deviation category, so the categories likely need context-specific extension.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the deviation rates reported here generalize to other industrial GitLab settings, prior studies of review effort built on raw MR logs may have partly learned to predict whether an MR is a real review rather than how much effort real reviews take; re-running those studies with a deviation filter would be a direct test.
  • The paper stops at filtering deviations out; a natural extension it does not pursue is using deviation type as a feature, which could let models separate review effort from workflow noise instead of discarding a third of the data.
  • Because an open-source project showed a much lower deviation rate (10.69 percent) and an extra category, deviation thresholds and taxonomies likely need per-project calibration rather than a single global rule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces the concept of "MR deviations"—merge requests that do not follow the standard code review workflow, such as library updates, configuration changes, work-in-progress drafts, or empty change sets. It proposes a seven-category taxonomy, reports that deviations account for up to 37.02% of MRs in an industrial dataset of 26.7k MRs, and develops a few-shot SetFit-based detector achieving up to 91% accuracy. The main impact study (RQ3) trains ML models to predict code review completion time before and after removing MRs classified as deviations, reporting performance improvements in 53.33% of cases (up to 2.25x) and substantial shifts in feature importance rankings (47% overall and 60% for top-k). External validation is attempted on a second industrial partner and on the Inkscape open-source project.

Significance. If the findings hold, the taxonomy and the empirical evidence that deviations distort ML-based review analytics would be a useful contribution to software engineering research and practice. The paper ships a replication package, uses bootstrap-based evaluation with significance tests, and includes external validation on a second company and an open-source project, which are notable strengths. The central impact claim, however, relies on classifier-predicted deviation labels applied to the full dataset without reporting validation statistics for that deployment, and the annotation process lacks inter-rater reliability reporting. These gaps are load-bearing for RQ3 and for the credibility of the taxonomy.

major comments (4)
  1. [V.A.1] The RQ3 impact analysis applies the best-performing SetFit model to the full 26.7k MRs, but the paper reports no validation statistics for this deployment. The sentence "we randomly sample and validate predictions with our industrial partners" provides no sample size, agreement metric, or confusion matrix. The metrics in Table IV come from the 80/20 split of the annotated sample, not from the full data used in RQ3. Since per-team precision is 0.71-0.78 and deviation prevalence is roughly 22-37%, the false-positive rate implies that the "without deviations" training set is contaminated by non-deviations and the "with deviations" set includes false negatives. The reported 53.33% improvement rate and the feature-importance shifts (47% and 60%) could therefore be artifacts of classifier label noise rather than of true deviations. Please report full-data validation (sample size, precision/recall, and error breakdown) or restrict RQ3 to manually validated subsets if such validation cannot be provided.
  2. [III.C] The taxonomy and prevalence results in RQ1 rest on manual annotation by two researchers with a supervising third, but the paper does not report any inter-rater reliability statistic (e.g., Cohen's kappa) or a quantitative account of disagreement resolution. Since these same annotations serve as training labels for the RQ2 detector, annotation unreliability propagates directly into the full-data predictions used in RQ3. Please report agreement measures and clearly describe how disagreements were resolved during the collaborative validation phase.
  3. [IV (Table IV)] The comparison between SetFit and BERT is not matched in training-data size. BERT is fine-tuned on the full annotated sample (or "all the datasets," as stated in the text), whereas SetFit uses 5, 10, or 15 instances per class. The conclusion that SetFit "outperforms BERT" is not supported because BERT is not evaluated in the same few-shot regime. Please add matched few-shot baselines for BERT (e.g., BERT with 15 instances per class) or explicitly reframe the comparison as SetFit with few shots versus BERT with the full sample, and avoid the blanket superiority claim.
  4. [V.A.2 (Table V)] The paper states that "all the metrics can be collected at the moment of creation of the MR," but several features appear to require a temporal window that is not specified. For instance, "Source branch approval time" and "Target branch approval time" are described as averages over MRs on the respective branches; if these averages are computed over all MRs including future ones, the completion-time prediction model suffers from temporal leakage, which would affect both the full-data and deviation-excluded models and could confound the RQ3 comparison. Please clarify the computation window for each historical feature and, if needed, perform a temporal split validation.
minor comments (5)
  1. [IV.A.2] The description of SetFit is incorrect: the text says "we fine-tune a T5 model to generate the deviation (or not) for each MR," but SetFit is based on sentence transformers with contrastive learning, not on T5. Please correct the model description.
  2. [Table III] The #MR column contains stray "k" suffixes (e.g., "6,344k"), which is inconsistent with the stated total of 26.7k MRs; the values appear to be 6,344, 8,396, 7,416, 735, and 4,004. Please fix the table formatting.
  3. [Table IV] The column header "SetFit (5 Epochs)" is misleading because the rows actually vary the number of instances per class (5, 10, 15), not the number of epochs. Please rename the header to reflect the number of training instances per class.
  4. [Table VI] The notation in the performance columns is hard to interpret: the "+/++" markers appear to indicate effect size (Cohen's d) while the Wilcoxon test is defined in the footnote but never marked in the table. Please add explicit significance markers and clarify in the caption or a footnote whether the reported numbers are ratios where values greater than 1 always indicate improvement (and how the transformation for MSE/MAE versus SA works).
  5. [VI (Model Validation)] The cross-project validation described in the Discussion reports accuracy between 0.54 and 0.72, but the text does not state whether this cross-project model or the within-project model was serialized and used for the RQ3 full-data predictions. Please clarify which model is used in the impact analysis.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the taxonomy, detection evaluation, and impact study are empirical and externally anchored to human annotation and independent benchmarks.

full rationale

The paper's derivation chain is empirical rather than definitional. RQ1 defines deviation categories through collaborative annotation with industrial partners, and the prevalence figures (up to 37.02%) come from manually annotated samples, not from the detection model. RQ2 evaluates SetFit and BERT against held-out validation splits of the annotated data and reports accuracy/precision/recall/F1, so the 91% accuracy claim is an empirical measurement, not an assumption. RQ3 uses the best RQ2 model to label the full 26.7k MR dataset and then compares review-completion-time models with and without those labeled deviations; this makes the RQ3 result dependent on classifier quality, which the paper itself flags only as 'we randomly sample and validate predictions with our industrial partners' without reporting agreement statistics. That is a validity threat concerning label noise, not circularity: the removed set is not the same quantity being predicted, and the comparison is made against an independent target (review completion time) using external modeling practices and benchmarks. No load-bearing premise is justified solely by self-citation; the cited prior work is external and the uniqueness of the taxonomy is not imported from a self-authored theorem. No equation reduces to its own input by construction. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The main commitments are the author-defined taxonomy, the unquantified annotation reliability, and the classifier-based labeling of the full dataset. The HC threshold and the SetFit 15-shot choice are free parameters that affect downstream results.

free parameters (2)
  • Huge Changes (HC) cutoff = 500 commits or 10,000+ lines changed
    Table II sets the HC definition at 'More than 500 commits or 10,000+ lines changed' without empirical justification; this threshold determines which MRs are labeled as deviations and thus affects all downstream percentages.
  • Number of SetFit training instances per class = 15
    The 15-shot configuration is chosen based on validation performance in RQ2; it is a model selection choice, not an independently justified parameter, and it is the configuration used to label all data for RQ3.
assumptions (4)
  • domain assumption The seven deviation categories (EOW, CC, LU, BOCA, RC, HC, ECS) are complete and mutually exclusive for the studied contexts.
    Categories are author- and partner-defined in Section III.C; no exhaustiveness procedure is shown, and Section VI adds an eighth category (Documentation Updates) for Inkscape, showing the set is context dependent.
  • domain assumption Manual annotation by two researchers with third-party supervision and partner validation produces reliable ground truth.
    Section III.C describes the process but reports no inter-rater agreement metric (e.g., Cohen's kappa), so label reliability is unquantified.
  • domain assumption All features used to predict review completion time are available at MR creation time.
    Stated in Section V.A.2 ('we ensured that all the metrics can be collected at the moment of creation'), but no timestamp check is shown for features like approval histories and historical MR stats.
  • domain assumption GitLab API metrics accurately reflect review activity.
    Review duration, comments, and approvals are read from the API; in-person or oral reviews noted in the introduction are not captured, which could blur the deviation definitions.
invented entities (1)
  • Seven-category MR deviation taxonomy (EOW, CC, LU, BOCA, RC, HC, ECS)
    purpose: Labels MRs that bypass standard review workflows, used as the basis for detection and impact analysis
    The categories are introduced by the authors and industrial partners; they are conceptual constructs rather than falsifiable predictions, and the later addition of Documentation Updates shows the set is not fixed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On The Impact of Merge Request Deviations on Code Review Practices." pith.science (2026). https://pith.science/paper/XHMUE4LC

@misc{pith2026250608860,
  author       = {Pith},
  title        = {Pith review of: On The Impact of Merge Request Deviations on Code Review Practices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XHMUE4LC}},
  note         = {Machine review of arXiv:2506.08860}
}
read the original abstract

Code review is a key practice in software engineering, ensuring quality and collaboration. However, industrial Merge Request (MR) workflows often deviate from standardized review processes, with many MRs serving non-review purposes (e.g., drafts, rebases, or dependency updates). We term these cases deviations and hypothesize that ignoring them biases analytics and undermines ML models for review analysis. We identify seven deviation categories, occurring in 37.02% of MRs, and propose a few-shot learning detection method (91% accuracy). By excluding deviations, ML models predicting review completion time improve performance in 53.33% of cases (up to 2.25x) and exhibit significant shifts in feature importance (47% overall, 60% top-*k*). Our contributions include: (1) a taxonomy of MR deviations, (2) an AI-driven detection approach, and (3) empirical evidence of their impact on ML-based review analytics. This work aids practitioners in optimizing review efforts and ensuring reliable insights.

Figures

Figures reproduced from arXiv: 2506.08860 by the authors.

Figure 3
Figure 3. Percentage Distribution of deviation Types Across Projects (MP, CP, DP, PF, FPGA). Note: BOCA = Build or Configuration Adjustments, EOW = Experimental or Work in Progress, CC = Code Cleaning, LU = Library Update, RC = Revert Commits, HC = Huge Changes, ECS = Empty Change Set. Furthermore, experimental or work-in-progress MR deviations, particularly prominent in the FPGA project at 31.1%, represent another instance w… view at source ↗
Figure 4
Figure 4. Percentage distribution of deviation types in the project from the other industrial partner. Note: BOCA = Build or Configuration Adjustments, EOW = Experimental or Work in Progress, CC = Code Cleaning, LU = Library Update, RC = Revert Commits, HC = Huge Changes, ECS = Empty Change Set. a Software-Defined Network (SDN) solution, allows us to extend our analysis beyond our initial dataset. Following the same process o… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 25 canonical work pages

  1. [1]

    User’s guide to correlation coeffi - cients

    Haldun Akoglu. “User’s guide to correlation coeffi - cients”. In: Turkish journal of emergency medicine 18.3 (2018), pp. 91–93

    work page 2018

  2. [2]

    When pro- cess data quality affects the number of bugs: Correla - tions in software engineering datasets

    Adrian Bachmann and Abraham Bernstein. “When pro- cess data quality affects the number of bugs: Correla - tions in software engineering datasets”. In: 2010 7th IEEE Working Conference on Mining Software Reposi - tories (MSR 2010). IEEE. 2010, pp. 62–71

    work page 2010

  3. [3]

    Four eyes are bet- ter than two: On the impact of code reviews on software quality

    Gabriele Bavota and Barbara Russo. “Four eyes are bet- ter than two: On the impact of code reviews on software quality”. In: 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE. 2015, pp. 81–90

    2015

  4. [4]

    Investigating technical and non - technical factors influencing modern code review

    Olga Baysal et al. “Investigating technical and non - technical factors influencing modern code review”. In: Empirical Software Engineering 21 (2016), pp. 932– 959

    2016

  5. [5]

    Modern code reviews in open - source projects: Which problems do they fix?

    Moritz Beller et al. “Modern code reviews in open - source projects: Which problems do they fix?” In: Proceedings of the 11th working conference on mining software repositories. 2014, pp. 202–211

    work page 2014

  6. [6]

    Identifying software met- rics thresholds for safety critical system

    Sˇ teˇ pa´ n Cais and Petr P´ıcha. “Identifying software met- rics thresholds for safety critical system”. In: The Third International Conference on Informatics Engineering and Information Science (ICIEIS2014), The Society of Digital Information and Wireless Communications . 2014, pp. 67–78. pp. 794–805

    work page 2014

  7. [7]

    Representative subset selection and outlier detection via isolation forest

    Wo-Ruo Chen et al. “Representative subset selection and outlier detection via isolation forest”. In: Analytical methods 8.39 (2016), pp. 7225–7231

    work page 2016

  8. [8]

    Outlier detection using isolation forest and local outlier factor

    Zhangyu Cheng, Chengming Zou, and Jianwei Dong. “Outlier detection using isolation forest and local outlier factor”. In: Proceedings of the conference on research in adaptive and convergent systems. 2019, pp. 161–168

    work page 2019

Show all 48 references
  1. [9]

    Learning to Predict Code Review Completion Time In Modern Code Review

    Moataz Chouchen et al. “Learning to Predict Code Review Completion Time In Modern Code Review”. In: Empirical Software Engineering 28.4 (2023), p. 82

    2023

  2. [10]

    Explaining mispredictions of ma - chine learning models using rule induction

    Ju¨rgen Cito et al. “Explaining mispredictions of ma - chine learning models using rule induction”. In: Pro- ceedings of the 29th ACM Joint Meeting on Euro- pean Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2021, pp. 716–727

    2021

  3. [11]

    Bert: Pre -training of deep bidirectional transformers for language understanding

    Jacob Devlin. “Bert: Pre -training of deep bidirectional transformers for language understanding”. In: arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  4. [12]

    Early prediction of merged code changes to prioritize reviewing tasks

    Yuanrui Fan et al. “Early prediction of merged code changes to prioritize reviewing tasks”. In: Empirical Software Engineering 23 (2018), pp. 3346–3393

    2018

  5. [13]

    Intelligent code reviews using deep learning

    Anshul Gupta and Neel Sundaresan. “Intelligent code reviews using deep learning”. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’18) Deep Learning Day. 2018. pp. 1–39

    2018

  6. [14]

    Understanding the Time to First Response In GitHub Pull Requests

    Kazi Amit Hasan et al. “Understanding the Time to First Response In GitHub Pull Requests”. In: arXiv preprint arXiv:2304.08426 (2023)

    2023 arXiv

  7. [15]

    Early prediction for merged vs abandoned code changes in modern code reviews

    Khairul Islam et al. “Early prediction for merged vs abandoned code changes in modern code reviews”. In: Information and Software Technology 142 (2022), p. 106756

    2022

  8. [16]

    E ´ tude comparative de la distribution florale dans une portion des Alpes et des Jura

    Paul Jaccard. “E ´ tude comparative de la distribution florale dans une portion des Alpes et des Jura”. In: Bull Soc Vaudoise Sci Nat 37 (1901), pp. 547–579

    1901

  9. [17]

    Will my patch make it? and how fast? case study on the linux kernel

    Yujuan Jiang, Bram Adams, and Daniel M German. “Will my patch make it? and how fast? case study on the linux kernel”. In: 2013 10th Working Conference on Mining Software Repositories (MSR). IEEE. 2013, pp. 101–110

    2013

  10. [18]

    The Impact of Correlated Metrics on the Interpretation of Defect Prediction Models

    J Jiarpakdee, C Tantithamthavorn, and AE Hassan. “The Impact of Correlated Metrics on the Interpretation of Defect Prediction Models”. In: IEEE Trans. Software Eng. early access 10 (2019)

    2019

  11. [19]

    Code review effectiveness: an empirical study on se - lected factors influence

    Marian Jureczko, Łukasz Kajda, and Paweł Go´recki. “Code review effectiveness: an empirical study on se - lected factors influence”. In: IET Software 14.7 (2020),

    2020

  12. [20]

    On Wasted Contri - butions: Understanding the Dynamics of Contributor - Abandoned Pull Requests –A Mixed -Methods Study of 10 Large Open-Source Projects

    SayedHassan Khatoonabadi et al. “On Wasted Contri - butions: Understanding the Dynamics of Contributor - Abandoned Pull Requests –A Mixed -Methods Study of 10 Large Open-Source Projects”. In: ACM Transactions on Software Engineering and Methodology 32.1 (2023),

    2023

  13. [21]

    Do code review measures ex- plain the incidence of post -release defects? Case study replications and bayesian networks

    Andrey Krutauz et al. “Do code review measures ex- plain the incidence of post -release defects? Case study replications and bayesian networks”. In: Empirical Soft- ware Engineering 25 (2020), pp. 3323–3356

    2020

  14. [22]

    An empirical study of the characteris- tics of popular Minecraft mods

    Daniel Lee et al. “An empirical study of the characteris- tics of popular Minecraft mods”. In: Empirical Software Engineering 25 (2020), pp. 3396–3429

    2020

  15. [23]

    Redundancy, context, and preference: An empirical study of duplicate pull requests in OSS projects

    Zhixing Li et al. “Redundancy, context, and preference: An empirical study of duplicate pull requests in OSS projects”. In: IEEE Transactions on Software Engineer- ing 48.4 (2020), pp. 1309–1335

    2020

  16. [24]

    Cupcleaner: A data cleaning approach for comment updating

    Qingyuan Liang et al. “Cupcleaner: A data cleaning approach for comment updating”. In: arXiv preprint arXiv:2308.06898 (2023)

    2023

  17. [25]

    Too Noisy To Learn: Enhancing Data Quality for Code Review C

    Chunhua Liu, Hong Yi Lin, and Patanamon Thong - tanunam. “Too Noisy To Learn: Enhancing Data Quality for Code Review C”. In: arXiv preprint arXiv:2502.02757 (2025)

    2025 arXiv

  18. [26]

    Nudge: Accelerating Overdue Pull Requests toward Completion

    Chandra Maddila et al. “Nudge: Accelerating Overdue Pull Requests toward Completion”. In: ACM Transac - tions on Software Engineering and Methodology 32.2 (2023), pp. 1–30

    2023

  19. [27]

    An empirical study of the impact of modern code review practices on software quality

    Shane McIntosh et al. “An empirical study of the impact of modern code review practices on software quality”. In: Empirical Software Engineering 21 (2016), pp. 2146–2189

    2016

  20. [28]

    The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects

    Shane McIntosh et al. “The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects”. In: Proceedings of the 11th working conference on mining software repositories. 2014, pp. 192–201

    2014

  21. [29]

    Patient classification as an outlier detection problem: an application of the one - class support vector machine

    Janaina Moura˜o-Miranda et al. “Patient classification as an outlier detection problem: an application of the one - class support vector machine”. In: Neuroimage 58.3 (2011), pp. 793–804

    2011

  22. [30]

    Automatic Categorization of GitHub Actions with Transformers and Few-shot Learn- ing

    Phuong T Nguyen et al. “Automatic Categorization of GitHub Actions with Transformers and Few-shot Learn- ing”. In: Proceedings of the 18th ACM/IEEE Interna - tional Symposium on Empirical Software Engineering and Measurement. 2024, pp. 468–474

    2024

  23. [31]

    An empirical study on log level prediction for multi -component systems

    Youssef Esseddiq Ouatiti et al. “An empirical study on log level prediction for multi -component systems”. In: IEEE Transactions on Software Engineering 01 (2022), pp. 1–1

    2022

  24. [32]

    Impact of discretiza - tion noise of the dependent variable on machine learning classifiers in software engineering

    Gopi Krishnan Rajbahadur et al. “Impact of discretiza - tion noise of the dependent variable on machine learning classifiers in software engineering”. In: IEEE Transac- tions on Software Engineering 47.7 (2019), pp. 1414– 1430

    2019

  25. [33]

    The impact of feature importance methods on the interpretation of defect clas- sifiers

    Gopi Krishnan Rajbahadur et al. “The impact of feature importance methods on the interpretation of defect clas- sifiers”. In: IEEE Transactions on Software Engineering 48.7 (2021), pp. 2245–2261

    2021

  26. [34]

    The impact of using regression models to build defect classifiers

    Gopi Krishnan Rajbahadur et al. “The impact of using regression models to build defect classifiers”. In: 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE. 2017, pp. 135–145

    2017

  27. [35]

    Using machine intelligence to prioritise code review requests

    Nishrith Saini and Ricardo Britto. “Using machine intelligence to prioritise code review requests”. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE - SEIP). IEEE. 2021, pp. 11–20

    2021

  28. [36]

    Leveraging Zero -Shot and Few - Shot Learning for Enhanced Sentiment and Require - ment Classification in Software Engineering

    Md Shafikuzzaman. “Leveraging Zero -Shot and Few - Shot Learning for Enhanced Sentiment and Require - ment Classification in Software Engineering”. MA the - sis. Lamar University-Beaumont, 2024

    2024

  29. [37]

    An exploratory study on code attention in BERT

    Rishab Sharma et al. “An exploratory study on code attention in BERT”. In: Proceedings of the 30th IEEE/ACM International Conference on Program Com- prehension. 2022, pp. 437–448

    2022

  30. [38]

    An experience report on defect modelling in practice: Pit - falls and challenges

    Chakkrit Tantithamthavorn and Ahmed E Hassan. “An experience report on defect modelling in practice: Pit - falls and challenges”. In: Proceedings of the 40th Inter - national conference on software engineering: Software engineering in practice. 2018, pp. 286–295

    2018

  31. [39]

    An empirical compar- ison of model validation techniques for defect prediction models

    Chakkrit Tantithamthavorn et al. “An empirical compar- ison of model validation techniques for defect prediction models”. In: IEEE Transactions on Software Engineer - ing 43.1 (2016), pp. 1–18

    2016

  32. [40]

    The impact of auto - mated parameter optimization on defect prediction mod- els

    Chakkrit Tantithamthavorn et al. “The impact of auto - mated parameter optimization on defect prediction mod- els”. In: IEEE Transactions on Software Engineering 45.7 (2018), pp. 683–711

    2018

  33. [41]

    Investigating code review practices in defective files: An empirical study of the qt system

    Patanamon Thongtanunam et al. “Investigating code review practices in defective files: An empirical study of the qt system”. In: 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories . IEEE. 2015, pp. 168–179

    2015

  34. [42]

    Review participation in modern code review: An empirical study of the android, Qt, and OpenStack projects

    Patanamon Thongtanunam et al. “Review participation in modern code review: An empirical study of the android, Qt, and OpenStack projects”. In: Empirical Software Engineering 22 (2017), pp. 768–817

    2017

  35. [43]

    Towards automating code review activities

    Rosalia Tufano et al. “Towards automating code review activities”. In: 2021 IEEE/ACM 43rd International Con- ference on Software Engineering (ICSE). IEEE. 2021, pp. 163–174

    2021

  36. [44]

    Efficient few -shot learning with - out prompts

    Lewis Tunstall et al. “Efficient few -shot learning with - out prompts”. In: arXiv preprint arXiv:2209.11055 (2022)

    2022 arXiv

  37. [45]

    Large -scale intent analysis for identifying large - review-effort code changes

    Song Wang, Chetan Bansal, and Nachiappan Nagap- pan. “Large -scale intent analysis for identifying large - review-effort code changes”. In: Information and Soft - ware Technology 130 (2021), p. 106408

    2021

  38. [46]

    EvaCRC: Evaluating Code Review Comments

    Lanxin Yang et al. “EvaCRC: Evaluating Code Review Comments”. In: Proceedings of the 31st ACM Joint European Software Engineering Conference and Sym - posium on the Foundations of Software Engineering . 2023, pp. 275–287

    2023

  39. [47]

    Automatic code review by learning the structure information of code graph

    Ying Yin et al. “Automatic code review by learning the structure information of code graph”. In: Sensors 23.5 (2023), p. 2551

    2023

  40. [48]

    Which Features are Learned by Code- Bert: An Empirical Study of the BERT -based Source Code Representation Learning

    Lan Zhang et al. “Which Features are Learned by Code- Bert: An Empirical Study of the BERT -based Source Code Representation Learning”. In: arXiv preprint arXiv:2301.08427 (2023)

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.