REVIEW 3 major objections 6 minor 1 cited by

An Empirical Study on the Amount of Changes Required for Merge Request Acceptance

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Code review effort in GitLab merge requests is measurable as the volume of lines changed after submission, and a random forest classifier can identify the high-rework MRs before acceptance.

desk verdict A useful new metric for code review effort, with a construct validity caveat and some overstated claims; worth engaging on the merits. read the letter →

arxiv 2507.23640 v1 pith:GAGIQ2J2 submitted 2025-07-31 cs.SE

Samah Kansab , Mohammed Sayagh , Francis Bordeleau , Ali Tizghadam This is my paper · ORCID

classification cs.SE

keywords codereviewmergerequestsGitLabeffortmeasurementmachinelearningrandomforestempiricalsoftwareengineeringreworkprediction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that code review effort in GitLab is best read through the volume of code rewritten after a merge request is opened, not through how long the review lasts or how many people join it. It defines the amount of changes as the total added and removed lines in commits made after MR creation, and reports that across 23,600+ MRs from four open-source projects, up to 73% of MRs undergo such post-submission changes, with 28% of those involving at least 200 changed lines. The paper then shows this rework volume is only weakly correlated with review duration and contributor count, but more strongly with comment activity, and that a random forest classifier can separate large-rework MRs from small ones with high discriminative power (median area under the ROC curve, AUC, between 0.84 and 0.88). If the claim holds, teams can estimate and triage review effort from signals available at MR creation rather than waiting for outcomes.

What carries the argument

The load-bearing object is the 'amount of changes for accepting an MR,' computed as additions plus deletions across all commits committed after MR creation (authored date ignored because authored commits are not visible until pushed). This single number turns review effort into a measurable count. Around it the paper builds a supervised pipeline: median-based discretization splits MRs into 'large' and 'small' rework; correlation and redundancy analysis prunes collinear features; 100 out-of-sample bootstrap samples train and evaluate random forest (compared with CART, logistic regression, and k-nearest-neighbour); permutation importance and accumulated local effects (ALE) plots identify and interpret the predictors. The identity doing the work is the definition itself: by anchoring effort to post-creation committed lines, the metric is computable from GitLab data alone and is independent of subjective time accounting.

What would settle it

Collect a sample of GitLab MRs and compare the paper's line-based effort count against a ground-truth record of which changes actually responded to reviewer comments, for example by linking each post-creation commit to discussion threads and checking whether large pushes are pre-authored and merely pushed after opening. If a substantial share of post-creation lines are not review-driven, the reported rates of rework and the classifier's target are measuring something other than review effort.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that the amount of changes required to accept a merge request—defined as the sum of added and deleted lines in all commits committed after the MR is created—is a valid measure of code review effort and a learnable target. In four GitLab projects (Omnibus, GitLab-runner, Inkscape, Gitaly; over 23,600 MRs), between 66% and 73% of MRs required post-submission changes, and 28% of those revised MRs touched at least 200 lines. This effort did not move with review duration (Spearman 0.42–0.48) or participant count (0.19–0.29), but did track comment volume (0.64–0.71), leading the authors to conclude that time and people metrics miss a distinct dimension of effort. A random forest classifier, trained on features from text, code complexity, developer experience, review history, collaboration, and branching, classified MRs as large- or small-rework with median AUC 0.84–0.88 across projects; the top predictors were the number of initial files and initial MR size, with experience and text features also in the top five, and historical project metrics mattering in some projects. The paper concludes that the metric can be explained and anticipated at MR creation time.

Load-bearing premise

The argument assumes that every line committed after a merge request is opened is rework caused by the review, and that work done before opening the request—or pre-written commits pushed later—carries no review effort; if teams prepare substantial code before submitting, the effort measure is miscalibrated and all downstream findings inherit that bias.

Editorial extensions

If this is right

Review effort can be quantified without waiting for the review to finish, using commits already visible at MR creation plus the MR's initial complexity.
Teams should not infer rework from review duration or headcount, since the paper finds these are weak proxies.
Comment volume is a stronger correlate of rework, so high-discussion MRs deserve closer scrutiny.
Complexity of the initial submission, especially the number of initial files and initial size, is the strongest early warning sign of a large rework, while experienced authors tend to need fewer changes.
An MR can be flagged at submission time as likely to require large changes, enabling prioritisation and resource planning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If line-based rework becomes an adopted metric, teams may start gaming it by splitting or deferring commits, so the measure is most reliable as a diagnostic before it becomes a target.
The weak correlation with duration and participant count suggests effort has at least two independent axes; a natural next step is a two-dimensional model that predicts both calendar time and code rework together.
Because the data come from four open-source GitLab projects, the same pipeline could be run on industrial GitLab instances or on GitHub pull requests to test whether the 0.84–0.88 AUC and feature rankings transfer.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Sam,

The headline: this is the first paper I know of that measures code review effort as lines changed after MR creation on GitLab, and it has a solid empirical base. The closest prior work counts patchset iterations on Gerrit, so the measured object is genuinely new. The descriptive results—66–73% of MRs get post-submission changes, 28% of those involve at least 200 lines—are simple, robust, and plausibly useful for triage. The ML pipeline is standard for the field, but it is done carefully: bootstrap resampling, temporal validation, multiple metrics, and a public replication package. That deserves credit.

The soft spots are real but manageable. First, the abstract says review effort is 'not correlated' with review time; the reported Spearman values are 0.42–0.48, which the paper itself classifies as low correlation. Low is not zero. Second, the intro claims the model can predict at MR creation, but several features (#commits, #discussions, mean time between commits) only exist after creation. That claim needs to be scoped to classification after submission, or the features need to be made purely pre-submission. Third, the median label threshold is computed from the full dataset including the validation MRs, and the noise-reduction procedure is described only as 'maximizing AUC' without saying how. Both are fixable but as written they hurt reproducibility. Finally, the stress-test concern about authored vs committed dates is legitimate: the measure counts all commits pushed after MR creation, and some of those could be pre-authored or bot/merge commits. The paper states this choice explicitly but does not test how much it matters. A robustness check using authored_date, or at least excluding merge commits, would strengthen the construct validity argument. None of these sink the paper. The central claim—that post-submission churn is a useful, predictable proxy for review effort—holds up, with the caveat that it is a proxy, not effort itself.

Who this is for: anyone mining code review data, or building review triage tools. It is not a landmark paper, but it is a solid empirical contribution with a new metric and a clean dataset. I would give it a serious referee slot. If I were handling it, I'd ask for the wording fixes and the robustness check, not for a rejection.

My recommendation: engage with it. It deserves reviewer time, and the authors have the data to answer the hard questions.

Referee Report

3 major / 6 minor

Summary. The paper defines code review effort as the total number of added and deleted lines in all commits made after a GitLab merge request is created, and studies this quantity on 23.6k MRs from four open-source projects. It reports that 66–73% of MRs require post-submission changes, that a substantial share of revised MRs exceed 200 changed lines, that this effort is only weakly correlated with review duration and participant counts, and that a random forest model classifies 'large effort' MRs with median AUC 0.84–0.88. It further analyzes feature importance, finding complexity, experience, and text metrics most predictive. The paper includes a replication package and uses standard bootstrap and validation techniques.

Significance. If the measure is accepted as a proxy for review effort, the study provides a new, time-independent effort perspective that is complementary to prior work on review duration and iteration counts. The dataset size, the public replication package, and the explicit temporal validation split are strengths. The main risk is construct validity: the effort measure conflates review-driven revisions with pre-authored commits pushed after MR creation, and the reported AUC may be optimistic due to label leakage from the full dataset. These issues are fixable in principle, and the paper's descriptive statistics and feature analyses are valuable even if the predictive claim needs tightening.

major comments (3)

[Section V (Approach) and Section VII] The central measure counts all added and deleted lines in commits committed after MR creation, regardless of authored date. As the paper states, 'authored commits are not visible until they are committed,' so pre-existing work that is pushed, amended, or rebased after MR creation is counted as review-driven rework; merge commits and bot or maintainer commits can also be included. No analysis is presented on the proportion of such commits, and Section VII does not list this as a threat. This is load-bearing because every RQ1 percentage, the correlation results, and the RQ2 labels are computed from this measure. Please quantify the share of post-creation commits with authored-date before MR creation and, if feasible, rerun the core statistics with a stricter rework definition (e.g., commits whose authored date is after MR creation, or commits touching files that received reviewer comments).
[Section IV (Data Processing) and Section VI (Approach)] The median-based label threshold is computed from the full dataset, including the temporally held-out validation set ('Median values are computed from the full dataset'). This leaks label information into the validation phase and weakens the claim that the validation AUC reflects out-of-sample performance. Additionally, the 'noise reduction process based on maximizing AUC' is applied to the training set, but the details are not specified; if the AUC-optimizing criterion is evaluated on the same training data, it constitutes target leakage that can inflate the reported AUC. Please compute thresholds and noise-reduction parameters inside each bootstrap training fold (or at least on training data only) and report the difference in validation AUC.
[Section V (Results) and Abstract] The abstract's 'not correlated with review time' overstates the reported Spearman values of 0.42–0.48, which the paper itself classifies as 'Low Correlation' on its own scale (0.3–0.5). The wording should be 'weakly correlated' or 'low correlation.' In addition, the stated 'up to 28% of these involve at least 200 lines of code' is inconsistent with the counts presented in Figure 1 and Table I: for Omnibus, 1,878 of 4,739 revised MRs (39.6%) exceed 200 changed lines, while for Gitaly the analogous fraction is about 24%. Please reconcile the definition of the reported percentage and the underlying counts.

minor comments (6)

[Section V (Approach)] The correlation interpretation list contains a typo: 'Negligible Correlation, if 0.3¡ |r— ≤ 5' should presumably read |r| ≤ 0.3 (or a value below 0.3).
[Section IV] The text references 'Figure ??' in the correlation/redundancy analysis; this placeholder appears to be a missing figure reference.
[Abstract and Introduction] The abstract states 'up to 71% of MRs require adjustments,' while the introduction states 'up to 73%;' the numbers should be made consistent.
[Section VI (Approach)] There is a duplicated article in 'the the amount of changes' in the approach paragraph; the text should be proofread for such typos.
[Table II and figures] Feature names are inconsistent between text, table, and figures (e.g., 'mean_rewok_size' vs. 'mean amount of changes for accepting MRs hist'); use one consistent naming scheme throughout.
[Section VIII (Conclusion)] The conclusion contains garbled text ('MRs are amount of changes for accepting MRed before integration'), which should be rewritten.

Circularity Check

1 steps flagged · score 4.0 of 10

The central effort metric is definitional, not circular; however, the ML evaluation has a target-leakage loop because the 'large/small' labels are thresholded by a median computed on the full dataset, including the validation MRs.

other [Section IV, Data Processing, 'Effort Labeling and Noise Reduction'; Section VI, Model Training]
"We convert the effort data into ”large” or ”small” classes using median -based discretization to enable binary classification, as our goal is to explain the factors driving high or low review effort. ... Median values are computed from the full dataset (see Table I)."

The validation set is a held-out subset (the most recent MRs), but the median that defines 'large' vs 'small' is computed over the full dataset, so each validation MR's label depends on the outcome distribution that includes that same MR. The reported validation AUC (0.84–0.88) is therefore not an evaluation against an independently defined target; the target labels were constructed with access to the validation outcomes. The median is a fitted threshold from the full data, and the model's 'prediction' is a classification of labels that were partially determined by that threshold.

full rationale

The paper's central measure—added+deleted lines in commits after MR creation—is a definitional construct, not a derived prediction, so the RQ1 percentages and the low correlations with review duration/participant counts are empirical statements about that proxy and are not circular. No load-bearing self-citations or imported uniqueness theorems appear; the authors cite prior work for standard techniques (e.g., [23] for discretization noise) without making the argument depend on those citations. The main circular flaw is in the RQ2 evaluation: the median threshold for the binary 'large/small' label is computed on the full dataset, which includes the validation MRs. That makes the validation AUC a measure of predicting labels that were partially defined by the validation outcomes themselves. A related but less explicit concern is that the noise-reduction step is 'based on maximizing AUC,' so the evaluation metric is also used as a data-cleaning objective; however, the paper states test/validation sets are untouched, so this is not as directly circular as the median leakage. Overall, the prediction task does not reduce to a single fitted equation, but the validation performance is partially constructed through the label threshold, so a moderate circularity score is warranted.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The study's conclusions rest on the validity of the effort metric, the completeness of GitLab API data, and the interpretability of random forest explanations. The label threshold and noise-reduction procedure are data-derived and under-specified, and one model feature is a lagged version of the outcome itself.

free parameters (2)

Median-based label threshold = Project-specific median of amount of changes, not explicitly reported
Used to binarize the target into large or small classes. It is computed from the full dataset, including test MRs, and determines all classification results.
Noise reduction tuning criterion = Not specified
Training-set instances are removed to maximize AUC; the amount and rule for removal are not given, so the target preprocessing includes data-dependent tuning that can affect reported performance.

assumptions (3)

domain assumption Amount of changes, defined as added plus deleted lines in commits after MR creation, is a valid operationalization of code review effort.
Central construct defined in Section IV. It excludes changes discussed but never committed and includes pushed commits that may not be responses to review.
domain assumption GitLab API returns complete commit, discussion, and change history for the studied MRs.
All metrics rely on API extraction; missing or filtered data would bias every distribution, correlation, and ML result.
domain assumption Random forest permutation importance and Accumulated Local Effects plots represent explanatory impact of features on rework size.
RQ3 interprets these model explanations as causes of large rework, but they are correlational model explanations, not causal estimates.

invented entities (1)

Amount of changes for accepting MRs independent evidence
purpose: A metric defined as the total added and deleted lines in all commits made after MR creation, used as both the study outcome and the label for ML classification.
The metric is operationally defined from Git history and is independently measurable from the same data. It is a new construct, not a physical entity, and its validity as an effort proxy is assumed rather than externally validated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of An Empirical Study on the Amount of Changes Required for Merge Request Acceptance." pith.science (2026). https://pith.science/paper/GAGIQ2J2

@misc{pith2026250723640,
  author       = {Pith},
  title        = {Pith review of: An Empirical Study on the Amount of Changes Required for Merge Request Acceptance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GAGIQ2J2}},
  note         = {Machine review of arXiv:2507.23640}
}

read the original abstract

Code review (CR) is essential to software development, helping ensure that new code is properly integrated. However, the CR process often involves significant effort, including code adjustments, responses to reviewers, and continued implementation. While past studies have examined CR delays and iteration counts, few have investigated the effort based on the volume of code changes required, especially in the context of GitLab Merge Requests (MRs), which remains underexplored. In this paper, we define and measure CR effort as the amount of code modified after submission, using a dataset of over 23,600 MRs from four GitLab projects. We find that up to 71% of MRs require adjustments after submission, and 28% of these involve changes to more than 200 lines of code. Surprisingly, this effort is not correlated with review time or the number of participants. To better understand and predict CR effort, we train an interpretable machine learning model using metrics across multiple dimensions: text features, code complexity, developer experience, review history, and branching. Our model achieves strong performance (AUC 0.84-0.88) and reveals that complexity, experience, and text features are key predictors. Historical project characteristics also influence current review effort. Our findings highlight the feasibility of using machine learning to explain and anticipate the effort needed to integrate code changes during review.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Humans Integrate, Agents Fix: How Agent-Authored Pull Requests Are Referenced in Practice
cs.SE 2026-04 conditional novelty 6.0 of 10

Humans reference agent PRs mainly to build features; agents self-reference mainly to fix bugs; linked agent PRs require substantially more review effort.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · cited by 1 Pith paper

[1]

Four eyes are bet- ter than two: On the impact of code reviews on software quality

Gabriele Bavota and Barbara Russo. “Four eyes are bet- ter than two: On the impact of code reviews on software quality”. In: 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE. 2015, pp. 81–90

work page 2015
[2]

Investigating technical and non - technical factors influencing modern code review

Olga Baysal et al. “Investigating technical and non - technical factors influencing modern code review”. In: Empirical Software Engineering 21 (2016), pp. 932– 959

work page 2016
[3]

Impact of developer reputation on code review outcomes in oss projects: An empirical investigation

Amiangshu Bosu and Jeffrey C Carver. “Impact of developer reputation on code review outcomes in oss projects: An empirical investigation”. In: Proceedings of the 8th ACM/IEEE international symposium on em - pirical software engineering and measurement. 2014, pp. 1–10

work page 2014
[4]

Learning to Predict Code Review Completion Time In Modern Code Review

Moataz Chouchen et al. “Learning to Predict Code Review Completion Time In Modern Code Review”. In: Empirical Software Engineering 28.4 (2023), p. 82

work page 2023
[5]

Explaining mispredictions of ma - chine learning models using rule induction

Ju¨rgen Cito et al. “Explaining mispredictions of ma - chine learning models using rule induction”. In: Pro- ceedings of the 29th ACM Joint Meeting on Euro- pean Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2021, pp. 716–727

work page 2021
[6]

Towards a taxonomy of code review smells

Emre Dog˘an and Eray Tu¨zu¨n. “Towards a taxonomy of code review smells”. In: Information and Software Technology 142 (2022), p. 106737

work page 2022
[7]

Early prediction of merged code changes to prioritize reviewing tasks

Yuanrui Fan et al. “Early prediction of merged code changes to prioritize reviewing tasks”. In: Empirical Software Engineering 23 (2018), pp. 3346–3393

work page 2018
[8]

Revisiting the impact of depen - dency network metrics on software defect prediction

Lina Gong et al. “Revisiting the impact of depen - dency network metrics on software defect prediction”. In: IEEE Transactions on Software Engineering 48.12 (2021), pp. 5030–5049

work page 2021

Show all 37 references

[9]

Chongomweru Halimu, Asem Kasem, and SH Shah Newaz. “Empirical comparison of area under ROC curve (AUC) and Mathew correlation coefficient (MCC) for evaluating machine learning algorithms on imbal - anced datasets for binary classification”. In: Proceed- ings of the 3rd interna...

2019
[10]

Understanding the Time to First Response In GitHub Pull Requests

Kazi Amit Hasan et al. “Understanding the Time to First Response In GitHub Pull Requests”. In: arXiv preprint arXiv:2304.08426 (2023)

2023 arXiv
[11]

Using a balanced scorecard to identify opportunities to improve code review effective- ness: An industrial experience report

Masum Hasan et al. “Using a balanced scorecard to identify opportunities to improve code review effective- ness: An industrial experience report”. In: Empirical Software Engineering 26 (2021), pp. 1–34

2021
[12]

Using AUC and accuracy in evaluating learning algorithms

Jin Huang and Charles X Ling. “Using AUC and accuracy in evaluating learning algorithms”. In: IEEE Transactions on knowledge and Data Engineering 17.3 (2005), pp. 299–310

2005
[13]

Would the patch be quickly merged?

Yuan Huang et al. “Would the patch be quickly merged?” In: Blockchain and Trustworthy Systems: First International Conference, BlockSys 2019, Guangzhou, China, December 7–8, 2019, Proceedings

2019
[14]

2020, pp

Springer. 2020, pp. 461–475

2020
[15]

Early prediction for merged vs abandoned code changes in modern code reviews

Khairul Islam et al. “Early prediction for merged vs abandoned code changes in modern code reviews”. In: Information and Software Technology 142 (2022), p. 106756

2022
[16]

Will my patch make it? and how fast? case study on the linux kernel

Yujuan Jiang, Bram Adams, and Daniel M German. “Will my patch make it? and how fast? case study on the linux kernel”. In: 2013 10th Working Conference on Mining Software Repositories (MSR). IEEE. 2013, pp. 101–110

2013
[17]

The Impact of Correlated Metrics on the Interpretation of Defect Prediction Models

J Jiarpakdee, C Tantithamthavorn, and AE Hassan. “The Impact of Correlated Metrics on the Interpretation of Defect Prediction Models”. In: IEEE Trans. Software Eng. early access 10 (2019)

2019
[18]

On Wasted Contri - butions: Understanding the Dynamics of Contributor - Abandoned Pull Requests –A Mixed -Methods Study of 10 Large Open-Source Projects

SayedHassan Khatoonabadi et al. “On Wasted Contri - butions: Understanding the Dynamics of Contributor - Abandoned Pull Requests –A Mixed -Methods Study of 10 Large Open-Source Projects”. In: ACM Transactions on Software Engineering and Methodology 32.1 (2023), pp. 1–39

2023
[19]

An empirical study of the characteris- tics of popular Minecraft mods

Daniel Lee et al. “An empirical study of the characteris- tics of popular Minecraft mods”. In: Empirical Software Engineering 25 (2020), pp. 3396–3429

2020
[20]

Redundancy, context, and preference: An empirical study of duplicate pull requests in OSS projects

Zhixing Li et al. “Redundancy, context, and preference: An empirical study of duplicate pull requests in OSS projects”. In: IEEE Transactions on Software Engineer- ing 48.4 (2020), pp. 1309–1335

2020
[21]

Nudge: Accelerating Overdue Pull Requests toward Completion

Chandra Maddila et al. “Nudge: Accelerating Overdue Pull Requests toward Completion”. In: ACM Transac - tions on Software Engineering and Methodology 32.2 (2023), pp. 1–30

2023
[22]

The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects

Shane McIntosh et al. “The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects”. In: Proceedings of the 11th working conference on mining software repositories. 2014, pp. 192–201

2014
[23]

Assessing predictions of violence: being accurate about accuracy

Douglas Mossman. “Assessing predictions of violence: being accurate about accuracy.” In: Journal of consult - ing and clinical psychology 62.4 (1994), p. 783

1994
[24]

Impact of discretiza - tion noise of the dependent variable on machine learning classifiers in software engineering

Gopi Krishnan Rajbahadur et al. “Impact of discretiza - tion noise of the dependent variable on machine learning classifiers in software engineering”. In: IEEE Transac- tions on Software Engineering 47.7 (2019), pp. 1414– 1430

2019
[25]

The impact of using regression models to build defect classifiers

Gopi Krishnan Rajbahadur et al. “The impact of using regression models to build defect classifiers”. In: 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE. 2017, pp. 135–145

2017
[26]

Using machine intelligence to prioritise code review requests

Nishrith Saini and Ricardo Britto. “Using machine intelligence to prioritise code review requests”. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE- SEIP). IEEE. 2021, pp. 11–20

2021
[27]

The proof and measurement of association between two things

Charles Spearman. “The proof and measurement of association between two things.” In: (1961)

1961
[28]

Measuring and visualizing code stability–a case study at three companies

Miroslaw Staron et al. “Measuring and visualizing code stability–a case study at three companies”. In: 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Con - ference on Software Process and Product Measurement. IEEE. 20...

2013
[29]

An experience report on defect modelling in practice: Pit - falls and challenges

Chakkrit Tantithamthavorn and Ahmed E Hassan. “An experience report on defect modelling in practice: Pit - falls and challenges”. In: Proceedings of the 40th Inter - national conference on software engineering: Software engineering in practice. 2018, pp. 286–295

2018
[30]

An empirical compar- ison of model validation techniques for defect prediction models

Chakkrit Tantithamthavorn et al. “An empirical compar- ison of model validation techniques for defect prediction models”. In: IEEE Transactions on Software Engineer - ing 43.1 (2016), pp. 1–18

2016
[31]

The impact of auto - mated parameter optimization on defect prediction mod- els

Chakkrit Tantithamthavorn et al. “The impact of auto - mated parameter optimization on defect prediction mod- els”. In: IEEE Transactions on Software Engineering 45.7 (2018), pp. 683–711

2018
[32]

Review participation in modern code review: An empirical study of the android, Qt, and OpenStack projects

Patanamon Thongtanunam et al. “Review participation in modern code review: An empirical study of the android, Qt, and OpenStack projects”. In: Empirical Software Engineering 22 (2017), pp. 768–817

2017
[33]

Towards automating code review activities

Rosalia Tufano et al. “Towards automating code review activities”. In: 2021 IEEE/ACM 43rd International Con- ference on Software Engineering (ICSE). IEEE. 2021, pp. 163–174

2021
[34]

Large -scale intent analysis for identifying large - review-effort code changes

Song Wang, Chetan Bansal, and Nachiappan Nagap- pan. “Large -scale intent analysis for identifying large - review-effort code changes”. In: Information and Soft - ware Technology 130 (2021), p. 106408

2021
[35]

Leveraging change intents for char - acterizing and identifying large-review-effort changes

Song Wang et al. “Leveraging change intents for char - acterizing and identifying large-review-effort changes”. In: Proceedings of the Fifteenth International Con - ference on Predictive Models and Data Analytics in Software Engineering. 2019, pp. 46–55

2019
[36]

Pull request latency explained: An empirical overview

Xunhui Zhang et al. “Pull request latency explained: An empirical overview”. In: Empirical Software Engineer - ing 27.6 (2022), p. 126

2022
[37]

Pre- dicting defects using network analysis on dependency graphs

Thomas Zimmermann and Nachiappan Nagappan. “Pre- dicting defects using network analysis on dependency graphs”. In: Proceedings of the 30th international con - ference on Software engineering. 2008, pp. 531–540

2008

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.