Pith. sign in

REVIEW 3 major objections 4 minor 68 references

Incorporating fault-proneness estimations into coverage-based test case prioritization methods

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that test case prioritization improves when the coverage contribution of each code unit is weighted by its estimated fault-proneness, with evidence that the modified additional strategy detects faults significantly faster…

desk verdict Useful incremental TCP paper with a public dataset, but the headline significance claim is compromised by post-selection of P0. read the letter →

arxiv 1908.06502 v2 pith:UPDW7O3U submitted 2019-08-18 cs.SE

Mostafa Mahdieh , Seyed-Hassan Mirian-Hosseinabadi , Khashayar Etemadi , Ali Nosrati , Sajad Jalali This is my paper · ORCID
classification cs.SE
keywords testcaseprioritizationregressiontestingdefectpredictionfault-pronenesscoverage-basedadditionalstrategyAPFDfault-basedcoverage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that regression test case prioritization can be improved by weighting code coverage with the estimated fault-proneness of code units, instead of treating every covered unit as equally important. The authors define a fault-based coverage score and feed it into the standard additional and total greedy prioritization strategies. Across 160 evaluation versions drawn from five real-world Java projects, the modified additional strategy achieves a mean APFD improvement of 4.63% over the traditional additional strategy, a difference that survives a signed-rank test ($p = 0.00268$); the modified total strategy shows no significant difference ($p = 0.406$). If the claim holds, teams can get earlier fault detection from data they already have, coverage traces plus bug history, without changing how often tests run.

What carries the argument

The load-bearing object is the fault-based coverage formula (Equation 4), which replaces the raw coverage sum with $\sum_j \mathrm{Cover}(i,j) \times \mathrm{Prob}(F_j)$, so that units more likely to contain faults contribute more to a test case's score. The probability $\mathrm{Prob}(F_j)$ is a linear blend of a baseline constant $P_0$ and a per-unit fault-proneness score $P_{\mathrm{dp}}(j)$ from a two-layer neural network (104 input metrics, 300 sigmoid hidden neurons, sigmoid output, trained with F1 loss and negative subsampling to handle scarce buggy classes). This reweighted coverage is what carries the argument: it changes the objective of the greedy additional algorithm, causing test cases that cover likely-faulty units to be selected earlier.

What would settle it

Fix $P_0$ before the evaluation, say at 0.5 or tuned on versions outside the five projects, then rerun the 160-version comparison and recompute the signed-rank test on the modified additional strategy; if the mean APFD improvement drops below significance or reverses, the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the coverage objective in test case prioritization should be fault-weighted, and that this reweighting pays off in the additional strategy. The paper defines fault-based coverage as $\sum_j \mathrm{Cover}(i,j) \times \mathrm{Prob}(F_j)$, with $\mathrm{Prob}(F_j) = P_0 + (1-P_0) P_{\mathrm{dp}}(j)$, where $P_{\mathrm{dp}}(j)$ is a neural-network defect-prediction score for unit $j$. Running the standard additional greedy algorithm on this weighted coverage, the paper reports a mean APFD of 60.50% against 59.54% for traditional coverage across 160 evaluation versions, a 4.63% mean improvement that is statistically significant under a signed-rank test ($p = 0.00268$). The same modification of the total strategy yields no statistically significant difference ($p = 0.406$), so the paper's positive claim is specifically about the additional strategy.

Load-bearing premise

The headline gain depends on the value $P_0 = 0.3$, which the authors selected after preliminary experiments on the same five projects used for the evaluation; if that selection was tuned to the evaluation data, the 4.63% improvement is an overfitted optimum rather than a general property of the method.

Editorial extensions

If this is right

  • The modified additional strategy, not the modified total strategy, carries the reported benefit; teams using greedy additional prioritization are the ones who should expect earlier fault detection.
  • The improvement is modest in absolute terms (mean APFD from 59.54% to 60.50%) but statistically significant, so the practical payoff is faster failure feedback rather than many more faults found.
  • The modification costs almost nothing at prioritization time: the added defect-prediction step is $O(mf)$ with $f = 104$ fixed, leaving the asymptotic complexity of both strategies unchanged, and observed runtime differences were under 40 milliseconds.
  • The method requires a usable bug history for the same project; when bug data is scarce, the weighting can hurt rather than help, which is why the paper recommends tuning the confidence parameter $P_0$ to the project.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the value $P_0 = 0.3$ was chosen after preliminary experiments on the same five projects used for evaluation, the 4.63% gain is probably optimistic; a fair estimate would require tuning $P_0$ on held-out versions or projects.
  • Because the approach is a weighting layer on top of coverage, better defect predictions, from richer features, more history, or cross-project models, should translate into better prioritization; this could be tested by swapping the predictor while holding the greedy strategy fixed.
  • The paper predicts fault-proneness at class level and then uses those scores for method-level coverage units, so a granularity mismatch may be diluting the effect; deriving scores at the same granularity as the coverage units could make the improvement larger.
  • The weighting is independent of test execution cost, so it should compose naturally with time-aware or cost-aware prioritization, a combination the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a modification of coverage-based test case prioritization (TCP) in which the coverage weight of a code unit is multiplied by an estimated fault-proneness probability. Specifically, Eq. (4) defines FaultBasedCover(i) = sum_j Cover(i,j) * Prob(F_j), and Eq. (7) sets Prob(F_j) = P0 + (1 - P0) * Pdp(j), where Pdp(j) is the output of a two-layer neural network defect predictor. The modified additional and total strategies are evaluated on an extended Defects4J dataset (Defects4J+M) covering 160 evaluation versions across five projects. The central reported result is that the modified additional strategy significantly outperforms the traditional additional strategy, with a mean APFD improvement of 4.63% and Wilcoxon signed-rank p = 0.00268, while the modified total strategy does not show a significant difference (p = 0.406). The paper also contributes the extended dataset, the defect prediction model, and an open-source implementation.

Significance. If the reported improvement is genuine, the proposal is a simple and potentially practical way to inject defect-prediction information into existing TCP heuristics without changing their asymptotic complexity. The temporal split (training on versions 1..i-1 and testing on version i) is a sound way to avoid circularity in the defect prediction evaluation, and the use of real Defects4J bugs is a strength. The release of the Defects4J+M dataset and the implementation is valuable for replication. However, the headline statistical claim is currently conditional on a parameter value selected using the same evaluation data, so the true out-of-sample improvement is not established by the experiments as reported.

major comments (3)
  1. [Section 5.1 and Eq. (7)] The choice P0 = 0.3 is stated in Section 5.1 to come from 'preliminary experiments,' and Section 5.2 then sweeps Cdp = 1 - P0 over the full [0,1] range on the same evaluation versions. Because the same data are used to select P0 and to compute the reported 4.63% mean APFD improvement and Wilcoxon p = 0.00268, the hypothesis test evaluates a post-selected configuration rather than a fixed, pre-specified method. The authors should provide a validation protocol that avoids this selection bias: either fix P0 using only training versions, use nested cross-validation over versions, or report the improvement over the whole P0 range together with an appropriate multiple-comparison correction. Without this, RQ1's central claim is not valid as stated.
  2. [Table 4 and Section 3.2.3] The defect prediction component identifies only 71 of 160 evaluation bugs (at the stated >0.1 threshold), and no ablation is reported that removes the fault-proneness contribution (e.g., setting Pdp(j) = 0 for all j while keeping Eq. (7)). The evaluation therefore does not isolate whether the observed APFD improvement comes from the learned fault-proneness estimates or from the constant P0 weighting alone. A comparison against a history-free weighting baseline, such as weights derived from previous bug counts or from a constant Prob(F_j), would be needed to support the claim that the learned model contributes value.
  3. [Section 5.1, Table 5] The per-project results are non-uniform: the modified additional strategy is slightly worse on Closure (-0.14%), and the modified total strategy is worse on Chart (-2.68%). The Wilcoxon test pools 160 versions from five projects without accounting for within-project correlation or providing per-project effect sizes. Please report per-project significance tests or a mixed-effects model, and accompany the overall p-value with an effect-size measure, so that the reader can assess whether the 'overall significant' result is driven by a small number of versions or by a consistent effect.
minor comments (4)
  1. [Section 3.2.2, Eq. (7)] The sentence following Eq. (7) is reversed: setting P0 = 1 gives Prob(F_j) = 1, which matches Eq. (6), while setting P0 = 0 gives Prob(F_j) = Pdp(j), which matches Eq. (5).
  2. [Table 4] The column header 'Min input samples' is duplicated; the second occurrence should read 'Max input samples.'
  3. [Figure 5] The curves in Figure 5 show only mean APFD against Cdp; adding per-version variance or confidence bands would make the sensitivity analysis more informative.
  4. [Section 5.1, Eq. (8)] The definition of 'impr' is clear, but the accompanying text could more explicitly state that the improvement column is not the arithmetic difference of the two mean APFD columns. Consider adding a one-sentence numerical illustration to avoid a common misinterpretation.

Circularity Check

1 steps flagged · score 4.0 of 10

The headline APFD improvement is evaluated at P0=0.3, a value selected by preliminary experiments on the same data, making the reported significance a post-selection estimate rather than a pre-specified prediction.

  1. fitted input called prediction [Section 5.1 (RQ1/RQ2), P0 selection; Section 5.2 (RQ3) sweeps P0 over the same data.]
    "Our preliminary experiments showed that our approach works the best when P0 is set to 0.3. Therefore, we compare the modified strategies with traditional ones using this setting."

    The only tunable parameter of the proposed modification is P0 in Equation 7. The paper states that P0=0.3 was selected because preliminary experiments showed it works best, and no separate validation set, nested tuning procedure, or multiple-comparison correction is described. The same evaluation versions are then used to compute the mean APFD improvement of 4.63% and the Wilcoxon p-value of 0.00268 that support the headline claim. The reported significance therefore tests the best-performing member of a one-parameter family selected on the same data, rather than a fixed, pre-specified method; the improvement is a post-selection estimate, not an independent out-of-sample prediction.

full rationale

The paper's core derivation is not circular in the self-definitional sense: Equation 4 is an explicit definition of fault-based coverage, and the defect prediction model is trained on versions 1 through i-1 and applied to version i (Section 4.4), so the fault-proneness scores are not constructed from the target APFD values. The one genuinely circular element is the choice of P0=0.3. Section 5.1 states that this value was selected because preliminary experiments showed it works best, and Section 5.2 sweeps Cdp over the same data; the paper then reports the mean APFD improvement (4.63%) and Wilcoxon p=0.00268 at that selected value as evidence for RQ1. No separate validation set or correction for the parameter search is described, so the significance test is evaluating a data-dependent configuration. This is a fitted input called prediction, but it affects only the strength of the headline significance claim, not the derivation of the method itself. The temporal split and the use of real Defects4J faults give the underlying comparison independent content, so the overall circularity is moderate rather than total.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central method is a definitional modification of coverage; it introduces one tunable parameter P0 and relies on domain assumptions about defect prediction and APFD. No new physical or conceptual entities are postulated.

free parameters (1)
  • P0 = 0.3
    Global weight in Eq. 7 blending coverage and predicted fault-proneness; chosen after preliminary experiments on the same five projects (Section 5.1), so the reported improvement is conditional on this value.
assumptions (4)
  • domain assumption Defects4J provides real, representative bugs for evaluating regression test prioritization.
    Used in Section 4.2 to justify the subject projects; the validity of the evaluation rests on this.
  • domain assumption Class-level fault-proneness scores can be extrapolated to method-level coverage units.
    Stated in Section 3.2.3; also a threat in Section 6.2.
  • domain assumption APFD captures the goal of test case prioritization.
    Adopted in Section 2.1 as the evaluation metric; a different goal could change the conclusions.
  • domain assumption A neural network trained on past versions of a project can predict fault-proneness of later versions.
    The entire approach depends on this; Section 3.2.3 and empirical results in Table 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Incorporating fault-proneness estimations into coverage-based test case prioritization methods." pith.science (2026). https://pith.science/paper/UPDW7O3U

@misc{pith2026190806502,
  author       = {Pith},
  title        = {Pith review of: Incorporating fault-proneness estimations into coverage-based test case prioritization methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UPDW7O3U}},
  note         = {Machine review of arXiv:1908.06502}
}
read the original abstract

Context: During the development process of a software program, regression testing is used to ensure that the correct behavior of the software is retained after updates to the source code. This regression testing becomes costly over time as the number of test cases increases and it makes sense to prioritize test cases in order to execute fault-detecting test cases as soon as possible. There are many coverage-based test case prioritization (TCP) methods that only use the code coverage data to prioritize test cases. By incorporating the fault-proneness estimations of code units into the coverage-based TCP methods, we can improve such techniques. Objective: In this paper, we aim to propose an approach which improves coverage-based TCP methods by considering the fault-proneness distribution over code units. Further, we present the results of an empirical study that shows using our proposed approach significantly improves the additional strategy, which is a widely used coverage-based TCP method. Method: The approach presented in this study uses the bug history of the software in order to introduce a defect prediction method to learn a neural network model. This model is then used to estimate fault-proneness of each area of the source code and then the estimations are incorporated into coverage-based TCP methods. Our proposed approach is a general idea that can be applied to many coverage-based methods, such as the additional and total TCP methods. Results: The proposed methods are evaluated on datasets collected from the development history of five real-world projects including 357 versions in total. The experiments show that using an appropriate bug history can improve coverage-based TCP methods.

Figures

Figures reproduced from arXiv: 1908.06502 by the authors.

Figure 1
Figure 1. Overview of the defect prediction method. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed test case prioritization algorithm [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Evaluation results of traditional and modified additional TCP strategies (RQ1) [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Evaluation results of traditional and modified total TCP strategies (RQ2) [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: The APFD performance of the modified strategies with respect to varying [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 67 canonical work pages

  1. [1]

    S. Yoo, M. Harman, Regression testing minimization, selection and pri- oritization: a survey, Software Testing, Verification and Reliability 22 (2) (2012) 67–120

    work page 2012

  2. [2]

    D. Hao, L. Zhang, L. Zhang, G. Rothermel, H. Mei, A unified test case prioritization approach, ACM Transactions on Software Engineering and Methodology (TOSEM) 24 (2) (2014) 10

    work page 2014

  3. [3]

    Rothermel, R

    G. Rothermel, R. H. Untch, C. Chu, M. J. Harrold, Test case prioriti- zation: An empirical study, in: Software Maintenance, 1999.(ICSM’99) Proceedings. IEEE International Conference on, IEEE, 1999, pp. 179– 188

    work page 1999

  4. [4]

    D. Hao, L. Zhang, L. Zang, Y . Wang, X. Wu, T. Xie, To be optimal or not in test-case prioritization, IEEE Transactions on Software Engineering 42 (5) (2016) 490–505

    work page 2016

  5. [5]

    R. Just, D. Jalali, M. D. Ernst, Defects4j: A database of existing faults to enable controlled testing studies for java programs, in: Proceedings of the 2014 International Symposium on Software Testing and Analysis, ACM, 2014, pp. 437–440

    work page 2014

  6. [6]

    Elbaum, A

    S. Elbaum, A. G. Malishevsky, G. Rothermel, Test case prioritization: A family of empirical studies, Software Engineering, IEEE Transactions on 28 (2) (2002) 159–182

    work page 2002

  7. [7]

    Woodcock, J

    J. Woodcock, J. Davies, Using Z: specification, refinement, and proof, V ol. 39, Prentice Hall Englewood Cliffs, 1996

    work page 1996

  8. [8]

    Engstrom, P

    E. Engstrom, P. Runeson, A. Ljung, Improving regression testing trans- parency and efficiency with history-based prioritization–an industrial case study, in: Software Testing, Verification and Validation (ICST), 2011 IEEE Fourth International Conference on, IEEE, 2011, pp. 367–376

    work page 2011

Show all 68 references
  1. [9]

    Catal, D

    C. Catal, D. Mishra, Test case prioritization: a systematic mapping study, Software Quality Journal 21 (3) (2013) 445–478

    2013

  2. [10]

    Rothermel, R

    G. Rothermel, R. H. Untch, C. Chu, M. J. Harrold, Prioritizing test cases for regression testing, Software Engineering, IEEE Transactions on 27 (10) (2001) 929–948

    2001

  3. [11]

    H. Mei, D. Hao, L. Zhang, L. Zhang, J. Zhou, G. Rothermel, A static ap- proach to prioritizing junit test cases, Software Engineering, IEEE Trans- actions on 38 (6) (2012) 1258–1275

    2012

  4. [12]

    Nam, Survey on software defect prediction, Department of Compter Science and Engineerning, The Hong Kong University of Science and Technology, Tech

    J. Nam, Survey on software defect prediction, Department of Compter Science and Engineerning, The Hong Kong University of Science and Technology, Tech. Rep

  5. [13]

    Menzies, Z

    T. Menzies, Z. Milton, B. Turhan, B. Cukic, Y . Jiang, A. Bener, Defect prediction from static code features: current results, limitations, new ap- proaches, Automated Software Engineering 17 (4) (2010) 375–407

    2010

  6. [14]

    Menzies, J

    T. Menzies, J. Greenwald, A. Frank, Data mining static code attributes to learn defect predictors, Software Engineering, IEEE Transactions on 33 (1) (2007) 2–13

    2007

  7. [15]

    Zimmermann, R

    T. Zimmermann, R. Premraj, A. Zeller, Predicting defects for eclipse, in: Predictor Models in Software Engineering, 2007. PROMISE’07: ICSE Workshops 2007. International Workshop on, IEEE, 2007, pp. 9–9

    2007

  8. [16]

    Kl ¨as, F

    M. Kl ¨as, F. Elberzhager, J. M ¨unch, K. Hartjes, O. V on Graevemeyer, Transparent combination of expert and measurement data for defect pre- diction: an industrial case study, in: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering-V olume 2, AC...

    2010

  9. [17]

    T. J. Ostrand, E. J. Weyuker, R. M. Bell, Predicting the location and number of faults in large software systems, Software Engineering, IEEE Transactions on 31 (4) (2005) 340–355

    2005

  10. [18]

    A. E. Hassan, R. C. Holt, The top ten list: Dynamic fault prediction, in: Software Maintenance, 2005. ICSM’05. Proceedings of the 21st IEEE International Conference on, IEEE, 2005, pp. 263–272

    2005

  11. [19]

    Pinzger, N

    M. Pinzger, N. Nagappan, B. Murphy, Can developer-module networks predict failures?, in: Proceedings of the 16th ACM SIGSOFT Interna- tional Symposium on Foundations of software engineering, ACM, 2008, pp. 2–12

    2008

  12. [20]

    Meneely, L

    A. Meneely, L. Williams, W. Snipes, J. Osborne, Predicting failures with developer networks and social network analysis, in: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of soft- ware engineering, ACM, 2008, pp. 13–23

    2008

  13. [21]

    Moser, W

    R. Moser, W. Pedrycz, G. Succi, A comparative analysis of the e fficiency of change metrics and static code attributes for defect prediction, in: Soft- 13 ware Engineering, 2008. ICSE’08. ACM/IEEE 30th International Confer- ence on, IEEE, 2008, pp. 181–190

    2008

  14. [22]

    E. J. Weyuker, T. J. Ostrand, R. M. Bell, Do too many cooks spoil the broth? using the number of developers to enhance defect prediction mod- els, Empirical Software Engineering 13 (5) (2008) 539–559

    2008

  15. [23]

    T. L. Graves, A. F. Karr, J. S. Marron, H. Siy, Predicting fault incidence using software change history, Software Engineering, IEEE Transactions on 26 (7) (2000) 653–661

    2000

  16. [24]

    Ashraf, A

    E. Ashraf, A. Rauf, K. Mahmood, Value based regression test case prior- itization, in: Proceedings of the world congress on engineering and com- puter science, V ol. 1, 2012, pp. 24–26

    2012

  17. [25]

    Elbaum, G

    S. Elbaum, G. Rothermel, S. Kanduri, A. G. Malishevsky, Selecting a cost-effective test case prioritization technique, Software Quality Journal 12 (3) (2004) 185–210

    2004

  18. [26]

    Lewis, Z

    C. Lewis, Z. Lin, C. Sadowski, X. Zhu, R. Ou, E. J. Whitehead Jr, Does bug prediction support human developers? findings from a google case study, in: Proceedings of the 2013 International Conference on Software Engineering, IEEE Press, 2013, pp. 372–381

    2013

  19. [27]

    H. Hata, O. Mizuno, T. Kikuno, Bug prediction based on fine-grained module histories, in: Proceedings of the 34th International Conference on Software Engineering, IEEE Press, 2012, pp. 200–210

    2012

  20. [28]

    Kamei, E

    Y . Kamei, E. Shihab, B. Adams, A. E. Hassan, A. Mockus, A. Sinha, N. Ubayashi, A large-scale empirical study of just-in-time quality assur- ance, IEEE Transactions on Software Engineering 39 (6) (2013) 757–773

    2013

  21. [29]

    Lessmann, B

    S. Lessmann, B. Baesens, C. Mues, S. Pietsch, Benchmarking classi- fication models for software defect prediction: A proposed framework and novel findings, Software Engineering, IEEE Transactions on 34 (4) (2008) 485–496

    2008

  22. [30]

    Gamma, K

    E. Gamma, K. Beck, Junit: A cooks tour, Java Report 4 (5) (1999) 27–38

    1999

  23. [31]

    M. R. Ho ffmann, Eclemma-jacoco java code coverage library, http:// eclemma.org/jacoco/index.html (2011)

    2011

  24. [32]

    Ltd., Sourcemeter - free-to-use, advanced source code analysis tool, https://www.sourcemeter.com/resources/java (2011)

    F. Ltd., Sourcemeter - free-to-use, advanced source code analysis tool, https://www.sourcemeter.com/resources/java (2011)

    2011

  25. [33]

    Wilcoxon, Individual comparisons by ranking methods, in: Break- throughs in statistics, Springer, 1992, pp

    F. Wilcoxon, Individual comparisons by ranking methods, in: Break- throughs in statistics, Springer, 1992, pp. 196–202

    1992

  26. [34]

    A. J. O ffutt, J. Pan, J. M. V oas, Procedures for reducing the size of coverage-based test sets, in: Proceedings of the Twelfth International Conference on Testing Computer Software, ACM, 1995, pp. 111–123

    1995

  27. [35]

    Z. Li, M. Harman, R. M. Hierons, Search Algorithms for Regression Test Case Prioritization, IEEE Transactions on Software Engineering - TSE 33 (4) (2007) 225–237. doi:10.1109/TSE.2007.38

    2007 doi

  28. [36]

    J. Zhou, D. Hao, Impact of static and dynamic coverage on test-case pri- oritization: An empirical study, in: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), IEEE, 2017, pp. 392–394

    2017

  29. [37]

    Zhang, J

    L. Zhang, J. Zhou, D. Hao, L. Zhang, H. Mei, Prioritizing junit test cases in absence of coverage information, in: 2009 IEEE International Confer- ence on Software Maintenance, IEEE, 2009, pp. 19–28

    2009

  30. [38]

    Elbaum, A

    S. Elbaum, A. G. Malishevsky, G. Rothermel, Prioritizing test cases for regression testing, in: Proceedings of the 2000 ACM SIGSOFT Interna- tional Symposium on Software Testing and Analysis, ISSTA ’00, ACM, New York, NY , USA, 2000, pp. 102–112. doi:10.1145/347324. 348910. UR...

    2000

  31. [39]

    J. A. Jones, M. J. Harrold, Test-suite reduction and prioritization for mod- ified condition/decision coverage, IEEE Transactions on software Engi- neering 29 (3) (2003) 195–209

    2003

  32. [40]

    Kov ´acs, G

    G. Kov ´acs, G. ´A. N´emeth, M. Subramaniam, Z. Pap, Optimal string edit distance based test suite reduction for sdl specifications, in: International SDL Forum, Springer, 2009, pp. 82–97

    2009

  33. [41]

    C. Fang, Z. Chen, K. Wu, Z. Zhao, Similarity-based test case prioritiza- tion using ordered sequences of program entities, Software Quality Jour- nal 22 (2) (2014) 335–361

    2014

  34. [42]

    C. Fang, Z. Chen, B. Xu, Comparing logic coverage criteria on test case prioritization, Science China Information Sciences 55 (12) (2012) 2826– 2840

    2012

  35. [43]

    S. G. Elbaum, J. C. Munson, Software evolution and the code fault intro- duction process, Empirical Software Engineering 4 (3) (1999) 241–262

    1999

  36. [44]

    S. Li, N. Bian, Z. Chen, D. You, Y . He, A simulation study on some search algorithms for regression test case prioritization, in: Quality Software (QSIC), 2010 10th International Conference on, IEEE, 2010, pp. 72–81

    2010

  37. [45]

    Zhang, D

    L. Zhang, D. Hao, L. Zhang, G. Rothermel, H. Mei, Bridging the gap be- tween the total and additional test-case prioritization strategies, in: Pro- ceedings of the 2013 International Conference on Software Engineering, IEEE Press, 2013, pp. 192–201

    2013

  38. [46]

    Jiang, Z

    B. Jiang, Z. Zhang, W. K. Chan, T. Tse, Adaptive random test case pri- oritization, in: Proceedings of the 2009 IEEE/ACM International Confer- ence on Automated Software Engineering, IEEE Computer Society, 2009, pp. 233–244

    2009

  39. [47]

    D. Hao, X. Zhao, L. Zhang, Adaptive test-case prioritization guided by output inspection, in: 2013 IEEE 37th Annual Computer Software and Applications Conference, IEEE, 2013, pp. 169–179

    2013

  40. [48]

    R. K. Saha, L. Zhang, S. Khurshid, D. E. Perry, An information retrieval approach for regression test prioritization based on program changes, in: 2015 IEEE/ACM 37th IEEE International Conference on Software Engi- neering, V ol. 1, IEEE, 2015, pp. 268–279

    2015

  41. [49]

    T. B. Noor, H. Hemmati, A similarity-based approach for test case pri- oritization using historical failure data, in: 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), IEEE, 2015, pp. 58–68

    2015

  42. [50]

    M. J. Arafeen, H. Do, Test case prioritization using requirements-based clustering, in: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, IEEE, 2013, pp. 312–321

    2013

  43. [51]

    Ledru, A

    Y . Ledru, A. Petrenko, S. Boroday, N. Mandran, Prioritizing test cases with string distances, Automated Software Engineering 19 (1) (2012) 65– 95

    2012

  44. [52]

    Korel, L

    B. Korel, L. H. Tahat, M. Harman, Test prioritization using system mod- els, in: Software Maintenance, 2005. ICSM’05. Proceedings of the 21st IEEE International Conference on, IEEE, 2005, pp. 559–568

    2005

  45. [53]

    Korel, G

    B. Korel, G. Koutsogiannakis, L. H. Tahat, Application of system models in regression test suite prioritization, in: Software Maintenance, 2008. ICSM 2008. IEEE International Conference on, IEEE, 2008, pp. 247– 256

    2008

  46. [54]

    Zhang, S.-S

    L. Zhang, S.-S. Hou, C. Guo, T. Xie, H. Mei, Time-aware test-case prior- itization using integer linear programming, in: Proceedings of the eigh- teenth international symposium on Software testing and analysis, ACM, 2009, pp. 213–224

    2009

  47. [55]

    B. Suri, S. Singhal, Analyzing test case selection & prioritization using aco, ACM SIGSOFT Software Engineering Notes 36 (6) (2011) 1–5

    2011

  48. [56]

    H. Do, S. Mirarab, L. Tahvildari, G. Rothermel, The e ffects of time con- straints on test case prioritization: A series of controlled experiments, IEEE Transactions on Software Engineering 36 (5) (2010) 593–617

    2010

  49. [57]

    Marijan, A

    D. Marijan, A. Gotlieb, S. Sen, Test case prioritization for continuous regression testing: An industrial case study, in: 2013 IEEE International Conference on Software Maintenance, IEEE, 2013, pp. 540–543

    2013

  50. [58]

    Zhang, C

    X. Zhang, C. Nie, B. Xu, B. Qu, Test case prioritization based on varying testing requirement priorities and test case costs, in: Seventh International Conference on Quality Software (QSIC 2007), IEEE, 2007, pp. 15–24

    2007

  51. [59]

    K. R. Walcott, M. L. So ffa, G. M. Kapfhammer, R. S. Roos, Timeaware test suite prioritization, in: Proceedings of the 2006 international sympo- sium on Software testing and analysis, ACM, 2006, pp. 1–12

    2006

  52. [60]

    Huang, K.-L

    Y .-C. Huang, K.-L. Peng, C.-Y . Huang, A history-based cost-cognizant test case prioritization technique in regression testing, Journal of Systems and Software 85 (3) (2012) 626–637

    2012

  53. [61]

    Engstr ¨om, P

    E. Engstr ¨om, P. Runeson, G. Wikstrand, An empirical evaluation of re- gression testing based on fix-cache recommendations, in: 2010 Third In- ternational Conference on Software Testing, Verification and Validation, IEEE, 2010, pp. 75–78

    2010

  54. [62]

    Laali, H

    M. Laali, H. Liu, M. Hamilton, M. Spichkova, H. W. Schmidt, Test case prioritization using online fault detection information, in: Ada-Europe International Conference on Reliable Software Technologies, Springer, 2016, pp. 78–93

    2016

  55. [63]

    Anderson, S

    J. Anderson, S. Salem, H. Do, Improving the e ffectiveness of test suite through mining historical data, in: Proceedings of the 11th Working Con- ference on Mining Software Repositories, ACM, 2014, pp. 142–151

    2014

  56. [64]

    S. Kim, J. Baik, An e ffective fault aware test case prioritization by in- corporating a fault localization technique, in: Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineer- ing and Measurement, ACM, 2010, p. 5

    2010

  57. [65]

    S. Wang, J. Nam, L. Tan, Qtep: quality-aware test case prioritization, in: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ACM, 2017, pp. 523–534. 14

    2017

  58. [66]

    J. Nam, S. Kim, Clami: Defect prediction on unlabeled datasets (t), in: 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), IEEE, 2015, pp. 452–463

    2015

  59. [67]

    Paterson, J

    D. Paterson, J. Campos, R. Abreu, G. M. Kapfhammer, G. Fraser, P. McMinn, An empirical study on the use of defect prediction for test case prioritization, in: 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), IEEE, 2019, pp. 346–357

    2019

  60. [68]

    Zimmermann, N

    T. Zimmermann, N. Nagappan, H. Gall, E. Giger, B. Murphy, Cross- project defect prediction: a large scale experiment on data vs. domain vs. process, in: Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on Th...

    2009

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.