Pith. sign in

REVIEW 3 major objections 6 minor 95 references

Revisiting Heterogeneous Defect Prediction: How Far Are We?

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Across 962 setups, no-label baselines beat trained defect predictors

desk verdict Solid replication with a cautionary result that probably holds for simple baselines, but the strongest 'all five unsupervised methods win' claim rides on an oracle baseline and should be fixed before publication. read the letter →

arxiv 1908.06560 v1 pith:FTQCPYOZ submitted 2019-08-19 cs.SE

Xiang Chen , Yanzhou Mu , Chao Ni , Zhanqi Cui This is my paper · ORCID
classification cs.SE
keywords softwaredefectpredictionheterogeneouscross-projectunsupervisedbaselinesManualUp/ManualDownreplicationstudyeffort-awareperformancemeasuresMcNemardiversityanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks a question the heterogeneous defect prediction (HDP) literature had not systematically asked: do supervised transfer-learning methods that predict defects across projects with different metric sets actually beat methods that ignore labels entirely? The authors compare five state-of-the-art HDP methods with five unsupervised methods on 962 source-to-target combinations built from 34 projects across five dataset groups, and find that HDP methods do not significantly outperform the unsupervised baselines on two non-effort-aware measures (F1 and AUC) or four effort-aware measures (ACC, Popt, PMI@20%, and IFA); the simple LOC-based ManualUp/ManualDown ranking is especially hard to beat. In the Scott-Knott rankings, all five unsupervised methods sit significantly above the HDP methods on F1 and AUC. A diversity analysis using McNemar's test shows the two families flag different defective modules, so they are partly complementary rather than redundant. If the conclusion holds, new HDP methods should be evaluated against unsupervised baselines before any claim of progress.

What carries the argument

The machinery that carries the argument is a comparison protocol rather than a single theorem. Each HDP method and each unsupervised method is evaluated under one shared experimental setup on 962 source-to-target combinations, and performance is ranked by the Scott-Knott test (a recursive clustering of methods into statistically distinct rank groups), with pairwise differences checked by the Benjamini-Hochberg-corrected Wilcoxon signed-rank test and Cliff's delta effect sizes. The load-bearing baselines are ManualUp/ManualDown from Zhou et al., which sort modules by lines of code and label the top half defective, plus UDP5, a variant that picks the metric achieving the best performance on each target project and performance measure. The diversity analysis uses McNemar's test on paired predictions to decide whether two methods identify the same defective modules. What this machinery does is convert the research question into a falsifiable ranking: if a supervised HDP method cannot land in a top Scott-Knott rank group against methods that ignore labels, then its transfer machinery is not delivering measurable value on these datasets.

What would settle it

Re-run the same 962 comparisons with UDP5 either deleted or redefined to pick its metric using only source-project information or a fixed label-free rule; the central claim would be falsified if, without UDP5, HDP methods occupy the top Scott-Knott rank group on F1 or AUC, and it would be strengthened if the unsupervised methods still dominate.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the current generation of heterogeneous defect prediction methods has not established an advantage over methods that need no labeled training data. Across 962 cross-project combinations, the five HDP methods—metric selection and matching, EMKCA, CTKCCA, TSEL, and distribution-characteristic mapping—fail to rank above the five unsupervised methods in Scott-Knott analyses for any of the six performance measures. For F1 and AUC specifically, every unsupervised method lands in a significantly better rank group than every HDP method, with the win/tie/loss tables showing the unsupervised methods winning the large majority of the 34 target-project comparisons. For effort-aware measures, the two size-based ranking methods (ManualUp/ManualDown and its metric-selection variant UDP5) are the clear leaders, while the other unsupervised methods remain competitive. The authors also find through McNemar's test that HDP and unsupervised predictions diverge on defective modules more than within-family comparisons do, and that a substantial fraction of defective modules are missed by both families, which they read as evidence that HDP research still has a long way to go.

Load-bearing premise

Section 3.2 introduces UDP5 by letting it choose, for every target project and performance measure, the metric that performs best using that target project's labels; if that oracle-like construction is not a legitimate baseline, the global Scott-Knott rankings and the claim that all five unsupervised methods beat HDP collapse, though the simpler LOC-based baselines may still stand.

Editorial extensions

If this is right

  • Future HDP studies that report gains over older supervised methods but omit unsupervised baselines should not be read as progress; the default comparison should include ManualUp/ManualDown and comparable label-free rankers.
  • Because HDP and unsupervised predictions flag different defective modules, ensembling the two families is a concrete, testable route to better recall; the paper explicitly points toward ensemble learning as a likely next step.
  • The low satisfactory ratios under both criteria (precision and recall above 75%, or recall above 70% with precision above 50%) imply that HDP models are not yet dependable enough to drive defect-inspection effort on the studied project types.
  • The result extends the earlier cross-project finding that simple size-based ranking beats complex models: the same pattern survives when source and target projects use different metric sets, which is the harder heterogeneous setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the headline result leans on UDP5, which uses target-project labels to choose its metric; if UDP5 is excluded, the strong claim that all five unsupervised methods beat HDP weakens, though the simpler LOC-based baselines would still be competitive.
  • Editorial inference: the diversity result implies a cheap experiment the paper does not run—combining one HDP model with a LOC-based ranker in a simple ensemble and testing whether the union beats either alone; the McNemar tables suggest headroom for exactly this kind of hybrid.
  • Editorial inference: if the pattern replicates beyond these 34 projects, the practical implication is that research effort may be better spent on deciding when labels help, rather than on ever more elaborate kernel or domain-adaptation machinery.
  • Editorial inference: a further testable extension would be to run the same comparison with effort-aware measures on change-level (just-in-time) data, since the earlier unsupervised-baseline debates originated there and the present study's effort-aware results may or may not carry over.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper reports a large-scale replication study comparing five heterogeneous defect prediction (HDP) methods with five unsupervised baselines on 962 source-target combinations drawn from 34 projects in five dataset groups. Across F1, AUC, ACC, Popt, PMI@20%, and IFA, the authors conclude that HDP methods do not significantly outperform the unsupervised methods, and that simple baselines such as ManualUp/ManualDown (UDP4) should be included in future HDP evaluations. The paper also reports McNemar-based diversity analyses suggesting that HDP and unsupervised methods identify partially complementary defective modules, and a satisfactory-ratio analysis indicating that HDP methods rarely meet previously proposed precision/recall criteria.

Significance. If the main result is correct, the paper provides a valuable and potentially field-changing negative result: it would show that the gains attributed to HDP methods over the past five years are not visible relative to simple unsupervised baselines, and it would justify changing standard evaluation practice. The study has a large and diverse subject pool (962 combinations, 34 projects, five groups), considers both effort-aware and non-effort-aware measures, and uses standard statistical tools such as Scott-Knott, Wilcoxon with BH correction, and Cliff's delta. The central weakness is that one of the five 'unsupervised' baselines, UDP5, is not actually unsupervised: its metric is selected using target-project labels. Because UDP5 appears in the global rankings, win/tie/loss tables, diversity analysis, and satisfactory-ratio analysis, several of the strongest claims currently overstate what the data show. The UDP4-based conclusion is more defensible and may survive revision. The paper would also benefit from releasing artifacts; no permanent code or data link is currently provided.

major comments (3)
  1. [Section 3.2, Section 5.1, Figures 2-3, Tables 4-7] UDP5 is defined in Section 3.2 as the variant that chooses, for each target project and each performance measure, the metric that achieves the best performance in the target project. This is an oracle baseline because the selection uses the target labels, not just the target feature values. The paper nonetheless counts UDP5 among the 'five unsupervised methods' and uses it in the Scott-Knott rankings of Figures 2 and 3 and in the win/tie/loss tables (Tables 4-7). Section 5.1's claim that 'all the 5 unsupervised methods can significantly perform better than supervised methods' therefore depends on a baseline that no practitioner could deploy. Please exclude UDP5 from the main comparisons, or explicitly re-label it as an upper-bound reference, and recompute all rankings, win/tie/loss counts, and the statements in the abstract and Section 5.1 accordingly.
  2. [Tables 4 and 5, Section 5.3] The influence of UDP5 is visible in the 34/34 win counts reported in Tables 4 and 5. Since UDP5's per-measure metric is selected using target labels, these wins are partly self-fulfilling and cannot be used as evidence that an unsupervised method outperforms the HDP methods. The same dependence affects the diversity analysis in Section 5.3, where UDP5-A and UDP5-F choose metrics by AUC and F1 performance on the target; the resulting pairwise diversity counts (Tables 11 and 12) may reflect the oracle's adaptation to the target rather than a property of unsupervised predictions. The paper should re-run the analysis with only the deployable unsupervised baselines (UDP1-UDP4) and state what changes.
  3. [Section 5.4] Table 14 includes UM5 in the satisfactory-ratio analysis. The high ratios for UM5 (e.g., 66.67% on PROMISE under SC2) are not achievable by an unsupervised method under the same protocol, because the metric is selected using the target labels. The qualitative conclusion that the satisfactory ratios of HDP methods are pessimistic may survive, but the UM5 rows should be removed or explicitly marked as oracle results.
minor comments (6)
  1. [Section 5.1, Table 5 text] The sentence 'the UDP methods can win HDP1, HDP2, HDP3, HDP4 and HDP5 at least 26, 30, 29, 30, 30 and 26 times' lists six minima for five HDP methods; please correct the enumeration.
  2. [Footnote 2] Footnote 2 is a placeholder and no artifact link is provided; because this is a replication study, please provide a permanent URL to code, data, and detailed results.
  3. [Section 5.3] The sentence about the mcnemar function mixing 'statsmodel' and 'R package exact2 × 26' is garbled; please clarify which implementation and version was used.
  4. [Section 4.3.2, Figures 6-7] For IFA and PMI@20%, state explicitly whether larger values are better or worse; the current text says UM4 has the 'highest IFA value' without noting that high IFA is usually considered undesirable.
  5. [Section 4.2] The derivation of 962 HDP combinations is spread over two sentences and depends on per-project metric sets inside NASA; a short formula or table would help reproducibility.
  6. [Section 5.3, Tables 10-12] Report whether any multiple-comparison correction was applied to the McNemar tests; uncorrected p<0.05 on hundreds of comparisons per method pair will produce many 'significant' combinations by chance.

Circularity Check

1 steps flagged · score 6.0 of 10

Oracle-based UDP5 makes the 'all five unsupervised methods beat HDP' finding self-fulfilling; the UDP4-based conclusion remains independent.

  1. fitted input called prediction [Section 3.2 (UDP5), applied in Section 5.1, Figures 2-3, Tables 4-5, and Section 5.3 (UDP5-A/UDP5-F)]
    "Therefore, we consider the fifth unsupervised methods, which is a variant of the method UDP4. In particular, given a specific performance measure, we choose the metric, which can achieve the best performance in the target project. We use UDP5 to denote this method."

    UDP5 is constructed by selecting, for each target project and each evaluation measure, the metric that performs best on the target project's labels. It is therefore an oracle baseline, not a label-free unsupervised method. Section 5.1 then reports that 'all the 5 unsupervised methods can significantly perform better than supervised methods' on AUC and F1 and that 'UM5 can achieve the best performance'; Tables 4 and 5 show UDP5 winning 34/34 target combinations. These wins are forced by the construction: the metric choice was fitted to the same labels and the same performance measure used in the evaluation. The Scott-Knott rankings in Figures 2-3 and the RQ3 diversity analysis using UDP5-A/UDP5-F also inherit this label-based selection.

full rationale

The paper's central negative result has two layers. The layer supported by UDP4 (ManualDown/ManualUp) is a legitimate, label-free size-based ranking baseline proposed by Zhou et al.; if the paper had only claimed that HDP methods do not beat such simple baselines, the study would be self-contained and non-circular. The circularity is concentrated in UDP5: Section 3.2 defines UDP5 as choosing, for each target project and each performance measure, the metric that gives the best performance on the target project's known labels. This is an oracle selection, yet the paper counts UDP5 among its five unsupervised methods and uses it for the strongest comparative claims: that all five unsupervised methods significantly outperform HDP methods on AUC and F1, that UM5 achieves the best performance, and the Scott-Knott rankings in Figures 2 and 3. UDP5's perfect 34/34 win counts in Tables 4 and 5 are a direct consequence of the construction, not an independent empirical discovery. Removing UDP5 does not destroy the UDP4-based conclusion, but it does invalidate the 'all five' and 'best' statements and could change the global rankings. No other load-bearing step reduces to its inputs: the HDP methods are replicated from external prior work, and self-citations in the related work are not load-bearing. The lack of released code/data is a reproducibility concern, not an additional circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The study contributes no new model or derivation; its only fitted parameter is the oracle-based UDP5 metric choice. The conclusions rest on assumptions about defect-proneness metric values, faithful re-implementations of prior HDP methods, and dataset representativeness.

free parameters (1)
  • UDP5 optimal metric selection = per target project and per performance measure; the metric with highest performance on the target labels
    The method chooses its ranking metric by evaluating on the target project's ground truth, so its reported performance is an upper bound fitted to the test data, not an independent baseline.
assumptions (3)
  • domain assumption Defective modules tend to have larger metric values than non-defective modules, used by UDP1-UDP4.
    This assumption underlies the unsupervised baselines cited from Nam and Kim 2015, Zhang et al. 2016, and Zhou et al. 2018; if false, the baselines are meaningless.
  • domain assumption The implementations of HDP1-HDP4 obtained from previous studies are correct and faithful to the original methods.
    Section 6 states the authors used shared implementations for HDP1-HDP4; the negative result depends on these replications being accurate.
  • domain assumption The five public dataset groups, AEEEM, ReLink, PROMISE, NASA, and SOFTLAB, are representative and their labels are reliable.
    Section 4.2 selects these datasets as widely used in prior HDP studies; the generalization of the negative result depends on their representativeness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revisiting Heterogeneous Defect Prediction: How Far Are We?." pith.science (2026). https://pith.science/paper/FTQCPYOZ

@misc{pith2026190806560,
  author       = {Pith},
  title        = {Pith review of: Revisiting Heterogeneous Defect Prediction: How Far Are We?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FTQCPYOZ}},
  note         = {Machine review of arXiv:1908.06560}
}
read the original abstract

Until now, researchers have proposed several novel heterogeneous defect prediction HDP methods with promising performance. To the best of our knowledge, whether HDP methods can perform significantly better than unsupervised methods has not yet been thoroughly investigated. In this article, we perform a replication study to have a holistic look in this issue. In particular, we compare state-of-the-art five HDP methods with five unsupervised methods. Final results surprisingly show that these HDP methods do not perform significantly better than some of unsupervised methods (especially the simple unsupervised methods proposed by Zhou et al.) in terms of two non-effort-aware performance measures and four effort-aware performance measures. Then, we perform diversity analysis on defective modules via McNemar's test and find the prediction diversity is more obvious when the comparison is performed between the HDP methods and the unsupervised methods than the comparisons only between the HDP methods or between the unsupervised methods. This shows the HDP methods and the unsupervised methods are complementary to each other in identifying defective models to some extent. Finally, we investigate the feasibility of five HDP methods by considering two satisfactory criteria recommended by previous CPDP studies and find the satisfactory ratio of these HDP methods is still pessimistic. The above empirical results implicate there is still a long way for heterogeneous defect prediction to go. More effective HDP methods need to be designed and the unsupervised methods should be considered as baselines.

Figures

Figures reproduced from arXiv: 1908.06560 by the authors.

Figure 1
Figure 1. The Illustrative Diagram for Popt Performance Measure means more false positives (i.e., non-defective modules are predicted as de￾fective modules) before detecting the first defective module and may have a non-ignorable impact on developers’ confidence and tolerance [64,38]. 5 Empirical Results 5.1 Results Analysis for RQ1 Motivation. In this RQ, we want to compare existing HDP methods with unsupervised methods in t… view at source ↗
Figure 2
Figure 2. Scott-Knott Test Results in terms of AUC Performance Measure [PITH_FULL_IMAGE:figures/full_fig_p021_2.png] view at source ↗
Figure 3
Figure 3. Scott-Knott Test Results in terms of F1 Performance Measure [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Scott-Knott Test Results in terms of Popt Performance Measure [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]
Figure 5
Figure 5. Figure 5: Scott-Knott Test Results in terms of ACC Performance Measure The Scott-Knott test results in terms of IFA can be found in [PITH_FULL_IMAGE:figures/full_fig_p025_5.png]
Figure 6
Figure 6. Figure 6: Scott-Knott Test Results in terms of IFA Performance Measure [PITH_FULL_IMAGE:figures/full_fig_p026_6.png]
Figure 7
Figure 7. Figure 7: Scott-Knott Test Results in terms of PMI@20% Performance Measure 5.3 Results Analysis for RQ3 Motivation. In previous two RQs, we compare and rank the performance of the HDP methods and the unsupervised methods in terms of effort-aware performance measures and non-effo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

95 extracted references · 80 canonical work pages

  1. [1]

    Journal of the royal statistical society

    Benjamini, Y., Hochberg, Y.: Controlling the false discovery rate: a practical and pow- erful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological) pp. 289–300 (1995)

    1995

  2. [2]

    In: Proceedings of the Working Conference on Mining Software Repositories, pp

    Bettenburg, N., Nagappan, M., Hassan, A.E.: Think locally, act globally: Improving defect and effort prediction models. In: Proceedings of the Working Conference on Mining Software Repositories, pp. 60–69 (2012)

    2012

  3. [3]

    Bettenburg, N., Nagappan, M., Hassan, A.E.: Towards improving statistical modeling of software engineering data: think locally, act globally! Empirical Software Engineering 20(2), 294–335 (2015)

    2015

  4. [4]

    354–363 (2017)

    Bin, Y., Zhou, K., Lu, H., Zhou, Y., Xu, B.: Training data selection for cross-project defection prediction: which approach is better? In: Proceedings of the International Symposium on Empirical Software Engineering and Measurement, pp. 354–363 (2017)

    2017

  5. [5]

    Bowes, D., Hall, T., Petri´ c, J.: Software defect prediction: do different classifiers find the same defects? Software Quality Journal 26(2), 525–552 (2018)

    2018

  6. [6]

    In: Proceedings of the International Sympo- sium on Empirical Software Engineering and Measurement, pp

    Camargo Cruz, A.E., Ochimizu, K.: Towards logistic regression models for predicting fault-prone code across software projects. In: Proceedings of the International Sympo- sium on Empirical Software Engineering and Measurement, pp. 460–463 (2009) 2 The website will be given when the manuscript is accepted. 36 Xiang Chen et al

    2009

  7. [7]

    In: Proceedings of the International Conference on Software Testing, Verification and Validation, pp

    Canfora, G., De Lucia, A., Di Penta, M., Oliveto, R., Panichella, A., Panichella, S.: Multi-objective cross-project defect prediction. In: Proceedings of the International Conference on Software Testing, Verification and Validation, pp. 252–261 (2013)

    2013

  8. [8]

    Software Testing, Verification and Reliability 25(4), 426–459 (2015)

    Canfora, G., Lucia, A.D., Penta, M.D., Oliveto, R., Panichella, A., Panichella, S.: Defect prediction as a multiobjective optimization problem. Software Testing, Verification and Reliability 25(4), 426–459 (2015)

    2015

Show all 95 references
  1. [9]

    Journal of Artificial Intelligence Research 16(1), 321–357 (2002)

    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research 16(1), 321–357 (2002)

    2002

  2. [10]

    Information and Software Technology 62, 67–77 (2015)

    Chen, L., Fang, B., Shang, Z., Tang, Y.: Negative samples reduction in cross-company software defects prediction. Information and Software Technology 62, 67–77 (2015)

    2015

  3. [11]

    Information and Software Technology 93, 1–13 (2018)

    Chen, X., Zhao, Y., Wang, Q., Yuan, Z.: Multi: Multi-objective effort-aware just-in-time software defect prediction. Information and Software Technology 93, 1–13 (2018)

    2018

  4. [12]

    IEEE Transactions on Software Engineering 43(7), 641–657 (2016)

    Da Costa, D.A., McIntosh, S., Shang, W., Kulesza, U., Coelho, R., Hassan, A.E.: A framework for evaluating the results of the szz approach for identifying bug-introducing changes. IEEE Transactions on Software Engineering 43(7), 641–657 (2016)

    2016

  5. [13]

    In: Proceedings of the Working Conference on Mining Software Repositories, pp

    D’Ambros, M., Lanza, M., Robbes, R.: An extensive comparison of bug prediction ap- proaches. In: Proceedings of the Working Conference on Mining Software Repositories, pp. 31–41 (2010)

    2010

  6. [14]

    Neural computation 10(7), 1895–1923 (1998)

    Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural computation 10(7), 1895–1923 (1998)

    1998

  7. [15]

    In: Proceedings of the Asia-Pacific Software Engineering Conference (2018)

    Feng Wang, J.H., Ma, Y.: A top-k learning to rank approach to cross-project software defect prediction. In: Proceedings of the Asia-Pacific Software Engineering Conference (2018)

    2018

  8. [16]

    In: Proceedings of the Working Conference on Mining Software Repositories, pp

    Fukushima, T., Kamei, Y., McIntosh, S., Yamashita, K., Ubayashi, N.: An empirical study of just-in-time defect prediction using cross-project models. In: Proceedings of the Working Conference on Mining Software Repositories, pp. 172–181 (2014)

    2014

  9. [17]

    In: Proceedings of the 37th International Conference on Software Engineering-Volume 1, pp

    Ghotra, B., McIntosh, S., Hassan, A.E.: Revisiting the impact of classification techniques on the performance of defect prediction models. In: Proceedings of the 37th International Conference on Software Engineering-Volume 1, pp. 789–800 (2015)

    2015

  10. [18]

    IEEE Transactions on Software Engineering 38(6), 1276–1304 (2011)

    Hall, T., Beecham, S., Bowes, D., Gray, D., Counsell, S.: A systematic literature review on fault prediction performance in software engineering. IEEE Transactions on Software Engineering 38(6), 1276–1304 (2011)

    2011

  11. [19]

    Information & Software Technology 59, 170–190 (2015)

    He, P., Li, B., Liu, X., Chen, J., Ma, Y.: An empirical study on software defect prediction with a simplified metric set. Information & Software Technology 59, 170–190 (2015)

    2015

  12. [20]

    CoRR abs/1411.4228 (2014)

    He, P., Li, B., Ma, Y.: Towards cross-project defect prediction with imbalanced feature sets. CoRR abs/1411.4228 (2014)

    2014 arXiv

  13. [21]

    Automated Software Engineering 19(2), 167–199 (2012)

    He, Z., Shu, F., Yang, Y., Li, M., Wang, Q.: An investigation on the feasibility of cross- project defect prediction. Automated Software Engineering 19(2), 167–199 (2012)

    2012

  14. [22]

    In: Proceedings of International Conference on Predictive Models in Software Engineering, pp

    Herbold, S.: Training data selection for cross-project defect prediction. In: Proceedings of International Conference on Predictive Models in Software Engineering, pp. 6:1–6:10 (2013)

    2013

  15. [23]

    local models for cross-project defect prediction

    Herbold, S., Trautsch, A., Grabowski, J.: Global vs. local models for cross-project defect prediction. Empirical Software Engineering 22(4), 1866–1902 (2017)

    2017

  16. [24]

    IEEE Transactions on Software Engineering44(9), 811 – 833 (2018)

    Herbold, S., Trautsch, A., Grabowski, J.: A comparative study to benchmark cross- project defect prediction approaches. IEEE Transactions on Software Engineering44(9), 811 – 833 (2018)

    2018

  17. [25]

    IEEE Transactions on Software Engineering 45(2), 111 – 147 (2019)

    Hosseini, S., Turhan, B., Gunarathna, D.: A systematic literature review and meta- analysis on cross project defect prediction. IEEE Transactions on Software Engineering 45(2), 111 – 147 (2019)

    2019

  18. [26]

    In: Proceedings of the International Conference on Predictive Models in Software Engineering, pp

    Hosseini, S., Turhan, B., M¨ antyl¨ a, M.: Search based training data selection for cross project defect prediction. In: Proceedings of the International Conference on Predictive Models in Software Engineering, pp. 3:1–3:10 (2016)

    2016

  19. [27]

    Information and Software Technology 95, 296–312 (2018)

    Hosseini, S., Turhan, B., M¨ antyl¨ a, M.: A benchmark study on the effectiveness of search- based data selection and feature selection for cross project defect prediction. Information and Software Technology 95, 296–312 (2018)

    2018

  20. [28]

    In: Proceedings of the International Conference on Software Maintenance and Evolution, pp

    Huang, Q., Xia, X., Lo, D.: Supervised vs unsupervised models: A holistic look at effort- aware just-in-time defect prediction. In: Proceedings of the International Conference on Software Maintenance and Evolution, pp. 159–170 (2017) Revisiting Heterogeneous Defect Prediction: ...

    2017

  21. [29]

    Empirical Software Engineering pp

    Huang, Q., Xia, X., Lo, D.: Revisiting supervised and unsupervised models for effort- aware just-in-time defect prediction. Empirical Software Engineering pp. 1–40 (2018)

    2018

  22. [30]

    TEMA (S˜ ao Carlos)15(1), 3–17 (2014)

    Jelihovschi, E.G., Faria, J.C., Allaman, I.B.: Scottknott: a package for performing the scott-knott clustering algorithm in r. TEMA (S˜ ao Carlos)15(1), 3–17 (2014)

    2014

  23. [31]

    In: Proceedings of Joint Meeting of the European Software Engineering Conference and the Symposium on Foundations of Software Engineering (2015)

    Jing, X., Fei, W., Dong, X., Qi, F., Xu, B.: Heterogeneous cross-company defect predic- tion by unified metric representation and cca-based transfer learning. In: Proceedings of Joint Meeting of the European Software Engineering Conference and the Symposium on Foundations of So...

    2015

  24. [32]

    IEEE Trans- actions on Software Engineering 43(4), 321–339 (2017)

    Jing, X.Y., Wu, F., Dong, X., Xu, B.: An improved sda based defect prediction frame- work for both within-project and cross-project class-imbalance problems. IEEE Trans- actions on Software Engineering 43(4), 321–339 (2017)

    2017

  25. [33]

    In: Proceedings of the International Conference on Predictive Models in Software Engineering, pp

    Jureczko, M., Madeyski, L.: Towards identifying software project clusters with regard to defect prediction. In: Proceedings of the International Conference on Predictive Models in Software Engineering, pp. 9:1–9:10 (2010)

    2010

  26. [34]

    Empirical Software Engineering 21(5), 2072–2106 (2016)

    Kamei, Y., Fukushima, T., McIntosh, S., Yamashita, K., Ubayashi, N., Hassan, A.E.: Studying just-in-time defect prediction using cross-project models. Empirical Software Engineering 21(5), 2072–2106 (2016)

    2016

  27. [35]

    In: Proceedings of the International Conference on Software Maintenance, pp

    Kamei, Y., Matsumoto, S., Monden, A., Matsumoto, K.i., Adams, B., Hassan, A.E.: Revisiting common bug prediction findings using effort-aware models. In: Proceedings of the International Conference on Software Maintenance, pp. 1–10 (2010)

    2010

  28. [36]

    In: Proceedings of the International Conference on Software Analysis, Evolution, and Reengineering, pp

    Kamei, Y., Shihab, E.: Defect prediction: Accomplishments and future challenges. In: Proceedings of the International Conference on Software Analysis, Evolution, and Reengineering, pp. 33–45 (2016)

    2016

  29. [37]

    IEEE Transactions on Software Engineering 39(6), 757–773 (2013)

    Kamei, Y., Shihab, E., Adams, B., Hassan, A.E., Mockus, A., Sinha, A., Ubayashi, N.: A large-scale empirical study of just-in-time quality assurance. IEEE Transactions on Software Engineering 39(6), 757–773 (2013)

    2013

  30. [38]

    In: Proceedings of the International Symposium on Software Testing and Analysis, pp

    Kochhar, P.S., Xia, X., Lo, D., Li, S.: Practitioners’ expectations on automated fault localization. In: Proceedings of the International Symposium on Software Testing and Analysis, pp. 165–176 (2016)

    2016

  31. [39]

    IEEE Transactions on Software Engineering (2018)

    Krishna, R., Menzies, T.: Bellwethers: A baseline method for transfer learning. IEEE Transactions on Software Engineering (2018)

    2018

  32. [40]

    In: Proceedings of International Conference on Automated Software Engineering, pp

    Krishna, R., Menzies, T., Fu, W.: Too much automation? the bellwether effect and its implications for transfer learning. In: Proceedings of International Conference on Automated Software Engineering, pp. 122–131 (2016)

    2016

  33. [41]

    Automated Software Engineering 25(2), 201–245 (2018)

    Li, Z., Jing, X.Y., Wu, F., Zhu, X., Xu, B., Ying, S.: Cost-sensitive transfer kernel canonical correlation analysis for heterogeneous defect prediction. Automated Software Engineering 25(2), 201–245 (2018)

    2018

  34. [42]

    Software Testing Verification and Reliability 28(1), 1–22 (2018)

    Li, Z., Jing, X.Y., Zhu, X.: Heterogeneous fault prediction with cost sensitive domain adaptation. Software Testing Verification and Reliability 28(1), 1–22 (2018)

    2018

  35. [43]

    In: Proceedings of the International Conference on Software Maintenance and Evolution, pp

    Li, Z., Jing, X.Y., Zhu, X., Zhang, H.: Heterogeneous defect prediction through multiple kernel learning and ensemble learning. In: Proceedings of the International Conference on Software Maintenance and Evolution, pp. 91–102 (2017)

    2017

  36. [44]

    IEEE Transactions on Software Engineering PP(99), 1–1 (2017)

    Li, Z., Jing, X.Y., Zhu, X., Zhang, H., Xu, B., Shi, Y.: On the multiple sources and privacy preservation issues for heterogeneous defect prediction. IEEE Transactions on Software Engineering PP(99), 1–1 (2017)

    2017

  37. [45]

    Automated Software Engineering (2019)

    Li, Z., Jing, X.Y., Zhu, X., Zhang, H., Xu, B., Ying, S.: Heterogeneous defect prediction with two-stage ensemble learning. Automated Software Engineering (2019)

    2019

  38. [46]

    Information and Software Technology 100, 87–102 (2018)

    Limsettho, N., Bennin, K.E., Keung, J.W., Hata, H., Matsumoto, K.: Cross project defect prediction using class distribution estimation and oversampling. Information and Software Technology 100, 87–102 (2018)

    2018

  39. [47]

    Information and Software Technology 107, 125–136 (2019)

    Liu, C., Yang, D., Xia, X., Yan, M., Zhang, X.: A two-phase transfer learning model for cross-project defect prediction. Information and Software Technology 107, 125–136 (2019)

    2019

  40. [48]

    IEEE Transactions on Reliability 65(1), 38–53 (2015)

    Liu, W., Liu, S., Gu, Q., Chen, J., Chen, X., Chen, D.: Empirical studies of a two- stage data preprocessing approach for software fault prediction. IEEE Transactions on Reliability 65(1), 38–53 (2015)

    2015

  41. [49]

    Information and Software Technology 54(3), 248–256 (2012) 38 Xiang Chen et al

    Ma, Y., Luo, G., Zeng, X., Chen, A.: Transfer learning for cross-company software defect prediction. Information and Software Technology 54(3), 248–256 (2012) 38 Xiang Chen et al

    2012

  42. [50]

    IEEE Transactions on software engineering 39(6), 822–834 (2013)

    Menzies, T., Butcher, A., Cok, D., Marcus, A., Layman, L., Shull, F., Turhan, B., Zimmermann, T.: Local versus global lessons for defect prediction and effort estimation. IEEE Transactions on software engineering 39(6), 822–834 (2013)

    2013

  43. [51]

    global models for effort estimation and defect prediction

    Menzies, T., Butcher, A., Marcus, A., Zimmermann, T., Cok, D.: Local vs. global models for effort estimation and defect prediction. In: Proceedings of International Conference on Automated Software Engineering, pp. 343–351 (2011)

    2011

  44. [52]

    IEEE Transactions on Software Engineering 33(1), 2–13 (2007)

    Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Transactions on Software Engineering 33(1), 2–13 (2007)

    2007

  45. [53]

    In: Proceedings of the International Symposium on Foundations of Software Engineering, pp

    Meyer, A.N., Fritz, T., Murphy, G.C., Zimmermann, T.: Software developers’ percep- tions of productivity. In: Proceedings of the International Symposium on Foundations of Software Engineering, pp. 19–29 (2014)

    2014

  46. [54]

    IEEE Transactions on Software Engineering (2017)

    Nam, J., Fu, W., Kim, S., Menzies, T., Tan, L.: Heterogeneous defect prediction. IEEE Transactions on Software Engineering (2017)

    2017

  47. [55]

    In: Proceedings of International Conference on Automated Software Engineering, pp

    Nam, J., Kim, S.: Clami: Defect prediction on unlabeled datasets. In: Proceedings of International Conference on Automated Software Engineering, pp. 452–463 (2015)

    2015

  48. [56]

    In: Proceedings of Joint Meeting of the European Software Engineering Conference and the Symposium on Foundations of Software Engineering, pp

    Nam, J., Kim, S.: Heterogeneous defect prediction. In: Proceedings of Joint Meeting of the European Software Engineering Conference and the Symposium on Foundations of Software Engineering, pp. 508–519 (2015)

    2015

  49. [57]

    In: Proceedings of the International Conference on Software Engineering, pp

    Nam, J., Pan, S.J., Kim, S.: Transfer defect learning. In: Proceedings of the International Conference on Software Engineering, pp. 382–391 (2013)

    2013

  50. [58]

    Journal of Systems and Soft- ware 152, 215–238 (2019)

    Ni, C., Chen, X., Wu, F., Shen, Y., Gu, Q.: An empirical study on pareto based multi- objective feature selection for software defect prediction. Journal of Systems and Soft- ware 152, 215–238 (2019)

    2019

  51. [59]

    In: Proceedings of Annual Computer Software and Applications Conference, pp

    Ni, C., Liu, W., Gu, Q., Chen, X., Chen, D.: Fesch: A feature selection method using clusters of hybrid-data for cross-project defect prediction. In: Proceedings of Annual Computer Software and Applications Conference, pp. 51–56 (2017)

    2017

  52. [60]

    Journal of Computer Science and Technology 32(6), 1090–1107 (2017)

    Ni, C., Liu, W.S., Chen, X., Gu, Q., Chen, D.X., Huang, Q.G.: A cluster based feature selection method for cross-project software defect prediction. Journal of Computer Science and Technology 32(6), 1090–1107 (2017)

    2017

  53. [61]

    IEEE Transactions on Neural Networks 22(2), 199–210 (2011)

    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer compo- nent analysis. IEEE Transactions on Neural Networks 22(2), 199–210 (2011)

    2011

  54. [62]

    IEEE Transactions on Knowledge and Data Engineering 22(10), 1345–1359 (2010)

    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22(10), 1345–1359 (2010)

    2010

  55. [63]

    In: Proceedings of the International Conferencce on Software Maintenance, Reengineering and Reverse Engineering, pp

    Panichella, A., Oliveto, R., De Lucia, A.: Cross-project defect prediction models: L’union fait la force. In: Proceedings of the International Conferencce on Software Maintenance, Reengineering and Reverse Engineering, pp. 164–173 (2014)

    2014

  56. [64]

    199–209 (2011)

    Parnin, C., Orso, A.: Are automated debugging techniques actually helping program- mers? In: Proceedings of the international symposium on software testing and analysis, pp. 199–209 (2011)

    2011

  57. [65]

    In: Proceed- ings of the Working Conference on Mining Software Repositories, pp

    Peters, F., Menzies, T., Marcus, A.: Better cross company defect prediction. In: Proceed- ings of the Working Conference on Mining Software Repositories, pp. 409–418 (2013)

    2013

  58. [66]

    In: Proceedings of the International Conference on Software Quality, Reliability and Security, pp

    Poon, W.N., Bennin, K.E., Huang, J., Phannachitta, P., Keung, J.W.: Cross-project defect prediction using a credibility theory based naive bayes classifier. In: Proceedings of the International Conference on Software Quality, Reliability and Security, pp. 434– 441 (2017)

    2017

  59. [67]

    Information and software technology 55(8), 1397–1418 (2013)

    Radjenovi´ c, D., Heriˇ cko, M., Torkar, R.,ˇZivkoviˇ c, A.: Software fault prediction metrics: A systematic literature review. Information and software technology 55(8), 1397–1418 (2013)

    2013

  60. [68]

    In: Proceedings of the International Symposium on Foundations of Software Engineering, pp

    Rahman, F., Posnett, D., Devanbu, P.: Recalling the imprecision of cross-project defect prediction. In: Proceedings of the International Symposium on Foundations of Software Engineering, pp. 61:1–61:11 (2012)

    2012

  61. [69]

    Applied Soft Computing 49, 1062–1077 (2016)

    Ryu, D., Baik, J.: Effective multi-objective na¨ ıve bayes learning for cross-project defect prediction. Applied Soft Computing 49, 1062–1077 (2016)

    2016

  62. [70]

    Empirical Software Engineering 21(1), 43–71 (2016)

    Ryu, D., Choi, O., Baik, J.: Value-cognitive boosting with a support vector machine for cross-project defect prediction. Empirical Software Engineering 21(1), 43–71 (2016)

    2016

  63. [71]

    Journal of Computer Science and Technology 30(5), 969–980 (2015) Revisiting Heterogeneous Defect Prediction: How Far Are We? 39

    Ryu, D., Jang, J.I., Baik, J.: A hybrid instance selection using nearest-neighbor for cross-project defect prediction. Journal of Computer Science and Technology 30(5), 969–980 (2015) Revisiting Heterogeneous Defect Prediction: How Far Are We? 39

    2015

  64. [72]

    Software Quality Journal 25(1), 235–272 (2017)

    Ryu, D., Jang, J.I., Baik, J.: A transfer cost-sensitive boosting approach for cross-project defect prediction. Software Quality Journal 25(1), 235–272 (2017)

    2017

  65. [73]

    IEEE Transactions on Software Engineering 39(9), 1208–1215 (2013)

    Shepperd, M., Song, Q., Sun, Z., Mair, C.: Data quality: Some comments on the nasa software defect datasets. IEEE Transactions on Software Engineering 39(9), 1208–1215 (2013)

    2013

  66. [74]

    1–5 (2005)

    ´Sliwerski, J., Zimmermann, T., Zeller, A.: When do changes induce fixes? In: Proceed- ings of the International Workshop on Mining Software Repositories, pp. 1–5 (2005)

    2005

  67. [75]

    In: Proceedings of the International Conference on Software Engineering, pp

    Tantithamthavorn, C., McIntosh, S., Hassan, A.E., Matsumoto, K.: Automated parame- ter optimization of classification techniques for defect prediction models. In: Proceedings of the International Conference on Software Engineering, pp. 321–332 (2016)

    2016

  68. [76]

    Empirical Software Engineering 17(1-2), 62–74 (2012)

    Turhan, B.: On the dataset shift problem in software engineering prediction models. Empirical Software Engineering 17(1-2), 62–74 (2012)

    2012

  69. [77]

    Empirical Software Engineer- ing 14(5), 540–578 (2009)

    Turhan, B., Menzies, T., Bener, A.B., Di Stefano, J.: On the relative value of cross- company and within-company data for defect prediction. Empirical Software Engineer- ing 14(5), 540–578 (2009)

    2009

  70. [78]

    Information and Software Technology 55(6), 1101– 1118 (2013)

    Turhan, B., Mısırlı, A.T., Bener, A.: Empirical evaluation of the effects of mixed project data on learning defect predictors. Information and Software Technology 55(6), 1101– 1118 (2013)

    2013

  71. [79]

    Universidad de Cantabria (2010)

    Vaerenbergh, S.v., et al.: Kernel methods for nonlinear identification, equalization and separation of signals. Universidad de Cantabria (2010)

    2010

  72. [80]

    IEEE Transactions on Software Engineering (2018)

    Wan, Z., Xia, X., Hassan, A.E., Lo, D., Yin, J., Yang, X.: Perceptions, expectations, and challenges in defect prediction. IEEE Transactions on Software Engineering (2018)

    2018

  73. [81]

    IEEE Transactions on Software Engineering pp

    Wang, S., Liu, T., Nam, J., Tan, L.: Deep semantic feature learning for software defect prediction. IEEE Transactions on Software Engineering pp. 1–1 (2018)

    2018

  74. [82]

    IEEE Transactions on Software Engineering 42(8), 707–740 (2016)

    Wong, W.E., Gao, R., Li, Y., Abreu, R., Wotawa, F.: A survey on software fault local- ization. IEEE Transactions on Software Engineering 42(8), 707–740 (2016)

    2016

  75. [83]

    IEEE Transactions on Reliability 67(2), 581 – 597 (2018)

    Wu, F., Jing, X.Y., Sun, Y., Sun, J., Huang, L., Cui, F., Sun, Y.: Cross-project and within-project semisupervised software defect prediction: A unified approach. IEEE Transactions on Reliability 67(2), 581 – 597 (2018)

    2018

  76. [84]

    In: Proceedings of Joint Meeting of the European Software Engineering Con- ference and the Symposium on Foundations of Software Engineering, pp

    Wu, R., Zhang, H., Kim, S., Cheung, S.C.: Relink: recovering links between bugs and changes. In: Proceedings of Joint Meeting of the European Software Engineering Con- ference and the Symposium on Foundations of Software Engineering, pp. 15–25 (2011)

    2011

  77. [85]

    IEEE Transactions on Software Engineering 42(10), 977– 998 (2016)

    Xia, X., David, L., Pan, S.J., Nagappan, N., Wang, X.: Hydra: Massively compositional model for cross-project defect prediction. IEEE Transactions on Software Engineering 42(10), 977– 998 (2016)

    2016

  78. [86]

    In: Proceedings of the International Conference on Program Comprehension, pp

    Xu, Z., Li, S., Tang, Y., Luo, X., Zhang, T., Liu, J., Xu, J.: Cross version defect pre- diction with representative data via sparse subset selection. In: Proceedings of the International Conference on Program Comprehension, pp. 132–143 (2018)

    2018

  79. [87]

    In: Proceedings of the International Symposium on Foundations of Software Engineering, pp

    Yang, Y., Zhou, Y., Liu, J., Zhao, Y., Lu, H., Xu, L., Xu, B., Leung, H.: Effort-aware just-in-time defect prediction: simple unsupervised models could be better than su- pervised models. In: Proceedings of the International Symposium on Foundations of Software Engineering, pp....

    2016

  80. [88]

    Journal of Systems and Software 132, 366–378 (2017)

    Yu, Q., Jiang, S., Zhang, Y.: A feature matching and transfer approach for cross- company defect prediction. Journal of Systems and Software 132, 366–378 (2017)

    2017

  81. [89]

    Empirical Software Engineering 22(6), 3186–3218 (2017)

    Zhang, F., Keivanloo, I., Zou, Y.: Data transformation in cross-project defect prediction. Empirical Software Engineering 22(6), 3186–3218 (2017)

    2017

  82. [90]

    In: Proceedings of the International Confer- ence on Software Engineering, pp

    Zhang, F., Zheng, Q., Zou, Y., Hassan, A.E.: Cross-project defect prediction using a connectivity-based unsupervised classifier. In: Proceedings of the International Confer- ence on Software Engineering, pp. 309–320 (2016)

    2016

  83. [91]

    In: Proceedings of Annual Computer Software and Ap- plications Conference, pp

    Zhang, Y., Lo, D., Xia, X., Sun, J.: An empirical study of classifier combination for cross-project defect prediction. In: Proceedings of Annual Computer Software and Ap- plications Conference, pp. 264–269 (2015)

    2015

  84. [92]

    Automated Software Engineering 24(1), 47–69 (2017)

    Zhang, Z.W., Jing, X.Y., Wang, T.J.: Label propagation based semi-supervised learning for software defect prediction. Automated Software Engineering 24(1), 47–69 (2017)

    2017

  85. [93]

    ACM Transactions on Software Engineering and Methodology 27(1), 1:1–1:51 (2018) 40 Xiang Chen et al

    Zhou, Y., Yang, Y., Lu, H., Chen, L., Li, Y., Zhao, Y., Qian, J., Xu, B.: How far we have progressed in the journey? an examination of cross-project defect prediction. ACM Transactions on Software Engineering and Methodology 27(1), 1:1–1:51 (2018) 40 Xiang Chen et al

    2018

  86. [94]

    Chapman and Hall/CRC (2012)

    Zhou, Z.H.: Ensemble methods: foundations and algorithms. Chapman and Hall/CRC (2012)

    2012

  87. [95]

    domain vs

    Zimmermann, T., Nagappan, N., Gall, H., Giger, E., Murphy, B.: Cross-project defect prediction: a large scale experiment on data vs. domain vs. process. In: Proceedings of Joint Meeting of the European Software Engineering Conference and the Symposium on Foundations of Softwar...

    2009

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.