Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Practical Pipeline-Aware Regression Test Optimization for Continuous Integration

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pipeline-aware Deep-Q-Learning approach schedules the first failing test within the first 16% of the test suite and catches 87% of relevant transitions at half the budget.

desk verdict Solid industrial RL test-optimization paper with useful pipeline-aware rewards, but post-submit headline numbers hinge on a self-defined flakiness threshold. read the letter →

arxiv 2501.11550 v1 pith:3YKIGLH3 submitted 2025-01-20 cs.SE

Daniel Schwendner , Maximilian Jungwirth , Martin Gruber , Martin Knoche , Daniel Merget , Gordon Fraser This is my paper · ORCID
classification cs.SE
keywords regressiontestprioritizationselectionContinuousIntegrationDeepQ-LearningReinforcementLearningflakinesspipeline-awareindustrialcasestudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that regression test optimization for large-scale continuous integration can be made practical by letting a Deep-Q-Learning agent learn pipeline-specific notions of which tests matter, using only lightweight, language-agnostic features. For pre-submit pipelines, where a failure blocks a merge, the paper defines a cost-aware reward (CostRank) and reports that the first failing test is scheduled within the first 16% of tests on average, beating a state-of-the-art bandit baseline and a heuristic baseline. For post-submit pipelines, where developers care about tests whose outcome changes between builds, the paper defines a transition-aware reward (CostChangeRank) that penalizes flaky transitions, and reports that 87% of relevant transitions are selected within half the test execution budget and 99.78% within five CI cycles. If these results hold, the approach would cut CI resource use and feedback latency in monorepo settings without needing per-test coverage data.

What carries the argument

The mechanism that carries the argument is a Deep-Q-Network agent that ranks each test target independently (pointwise ranking) and a pair of pipeline-specific reward functions. The pre-submit reward, CostRank, returns a positive value that increases the earlier a failing test appears and the shorter its execution time, and a negative value for passing tests, so the agent learns to surface likely failures quickly. The post-submit reward, CostChangeRank, returns 1 for a relevant transition, −1 for a flaky transition defined as one that flips back within three consecutive CI cycles, and the negative normalized duration otherwise, so the agent learns to prefer short, information-rich tests while avoiding flaky ones. Input features are limited to the last 25 test verdicts, a PCA-reduced bag-of-words encoding of the test target name, cycles since last failure, cycles since last execution, and average execution duration. The agent is trained with experience replay and an ε-greedy-style Gaussian exploration on action values.

What would settle it

Re-run the post-submit evaluation on the same dataset while changing the flaky-transition threshold from three cycles to one cycle and to five cycles; if the reported 87% and 99.78% detection rates change by more than a few percentage points, the central result is an artifact of the chosen threshold. A second check is to inspect whether tests that flip twice within three cycles but then stabilize and stay flipped are nevertheless labeled flaky, which would mean genuine regressions are being discarded by the reward.

Watch

Extended reading notes

Core claim

The central claim is that a single Deep-Q-Learning framework, trained on historical test results and test names, can outperform dedicated regression test optimization baselines in both pre-submit and post-submit industrial CI pipelines when the reward function matches the pipeline's goal. In pre-submit pipelines, where the goal is fast failure feedback, the authors define CostRank, which rewards failing tests more when they are scheduled earlier and when their execution cost is low; with this reward the approach reaches a mean NAPFD of 0.75 at a full budget and 0.53 even at a 10% budget, with the first failure appearing after 16% of the test suite on average. In post-submit pipelines, where the goal is to detect non-flaky pass-to-fail or fail-to-pass transitions, the authors define CostChangeRank, which rewards relevant transitions and assigns −1 to transitions that flip again within three CI cycles; with this reward the approach selects 87% of relevant transitions within half the execution budget and detects 99.78% within five cycles. The paper further shows that the cost-aware rewards outperform their non-cost-aware counterparts (RNFail and RNChange) and random ordering.

Load-bearing premise

The paper's post-submit numbers depend on defining a flaky transition as one that flips again within three consecutive CI cycles; that same definition is used both as the reward signal during training and as the ground truth for evaluating whether a detected transition is relevant, so if a different threshold better matches what developers actually care about, the reported 87% and 99.78% figures would not be measuring that.

Editorial extensions

If this is right

  • Pre-submit pipelines can shorten developer feedback loops: with a 50% budget the first failing test appears at 16% of the suite on average, and even at a 10% budget the NAPFD stays at 0.53.
  • Post-submit pipelines can halve their resource consumption while still catching 87% of developer-relevant test transitions immediately and over 98% within three CI cycles.
  • Because the features are just historical verdicts and test names, the technique can be dropped into a multi-language monorepo without instrumentation or coverage harvesting.
  • The agent keeps learning from new execution results, so it tracks a CI environment where tests are added, renamed, and removed daily without manual retraining.
  • The cost-aware rewards generalize: any pipeline whose objective is fast failure feedback can reuse CostRank, and any pipeline where outcome changes matter can reuse CostChangeRank.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The three-cycle flakiness threshold is a business parameter, not a law of nature; a deployment with a faster or slower merge cadence should recalibrate it, and the paper's headline post-submit numbers would likely shift if recalibrated.
  • The pipeline-aware reward design transfers beyond CI to other gating loops, such as choosing which system-level tests to run before promoting a release candidate, where the same pass-to-fail/fail-to-pass distinction applies.
  • The 99.78% five-cycle figure is a recovery guarantee with delay tolerance, not instant detection; teams that need to revert a regression before several merges accumulate must budget for that latency or require a dedicated fail-fast stage.
  • Combining the test-name embedding with code-change signals (e.g., which files a PR touches) would likely sharpen the ranking further, but that extension is not explored in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes PR-DQL, a Deep-Q-Learning-based test prioritization and selection approach for large-scale CI systems, with two pipeline-specific reward functions: CostRank for pre-submit pipelines and CostChangeRank for post-submit pipelines. Using only language-agnostic features (historical test results, test-name embeddings, recency, and average duration), the authors evaluate on a 20-week industrial dataset from BMW, replaying CI cycles against RANDOM, ROCKET, COLEMAN, and a custom RNChange reward baseline. The central claims are that PR-DQL schedules the first failing test within the first 16% of tests in pre-submit pipelines, and that in post-submit pipelines it selects 87% of developer-relevant transitions within half the execution time and detects 99.78% of relevant transitions within five CI cycles. The paper contributes an open dataset and a lightweight, language-agnostic approach, but the post-submit evaluation relies on a self-defined flaky-transition threshold that is used both as the training reward label and as the evaluation ground truth, and the comparative claims lack significance testing.

Significance. If the results hold, this is a valuable industrial contribution: the feature set is genuinely lightweight and language-agnostic, the pipeline-aware reward design is a sensible departure from failure-only prioritization, and the CI replay methodology avoids harming production. The open dataset is a concrete asset for the regression-test-optimization community, and the explicit treatment of post-submit flakiness as a transition-detection problem addresses a real gap between academic benchmarks and industrial practice. However, the strength of the empirical support is currently limited by the self-referential definition of relevant transitions in RQ2 and by the absence of statistical tests for the RQ1 comparisons; these issues must be addressed before the headline numbers can be taken at face value.

major comments (4)
  1. [Section IV.C and Eq. (2), Figures 7 and 8] The three-CI-cycle flaky-transition threshold is used both to define the CostChangeRank reward labels during training and to construct the evaluation ground truth for the RQ2 claims. Because the threshold is selected from the churn rate of the same dataset, the reported 87% and 99.78% figures measure performance against a self-generated target rather than an externally validated notion of developer relevance. A flaky test that stabilizes only after four or more cycles would be labeled relevant, and a real regression fixed within three cycles would be labeled flaky, so both directions of misclassification are possible. Please provide a robustness analysis varying the threshold, or an external validation (e.g., developer-confirmed regressions and bug fixes), to demonstrate that the numbers are not an artifact of label construction.
  2. [Tables IV and V, Section V.A] The claim that PR-DQL 'outperforms existing approaches' is not supported by any significance testing. In Table V, PR-DQL and COLEMAN both report NAPFD 0.71 at an 80% budget, and at a 50% budget the difference is 0.71 versus 0.68 with standard deviations around 0.31; ROCKET and RANDOM are clearly worse, but the PR-DQL-versus-COLEMAN advantage is within one standard deviation. Reporting only means and standard deviations is insufficient to justify 'consistently achieving the highest NAPFD values.' Paired significance tests (e.g., Wilcoxon signed-rank) and effect sizes, or confidence intervals, are needed to support the comparative conclusion.
  3. [Section IV.A and Section V.B] The ROCKET baseline was implemented by the authors because the original replication package is missing, but no validation of this reimplementation is provided. An unvalidated baseline implementation can bias the comparison, especially because ROCKET's heuristic involves several configurable weights. Please make the reimplementation publicly available and validate it against published ROCKET results on a benchmark dataset, or at least report the exact configuration used. In addition, RQ2 compares only against RANDOM and the authors' own RNChange variant; no established transition-based selection method, such as the Google approach cited as reference [35], is included, which weakens the claim of outperforming existing techniques in the post-submit setting.
  4. [Abstract and Section V.A] The headline claim that PR-DQL 'scheduled the first failing test within the first 16% of tests' is inconsistent with Table IV, where PR-DQL's NFR is reported as 0.17 ± 0.22. Table III reports 0.16 for CostRank, but the abstract and the conclusion attribute 16% to the full PR-DQL approach. Please reconcile whether the 16% figure refers to the reward-function comparison in Table III or to the final PR-DQL configuration in Table IV, and correct the abstract and conclusion accordingly.
minor comments (5)
  1. [Abstract] The sentence 'there not only a need but also potential for further improvements' is missing a verb and should read 'there is not only a need but also potential for further improvements.'
  2. [Figure 7] The y-axis label 'Ratio of Detected Transitions per CI Cycle [%]' is ambiguous; the surrounding text describes the percentage of executed test cases that reveal a relevant transition, so the axis label should be reworded to match the text.
  3. [Section III.C] The subsection heading 'Per-submit pipelines' appears to be a typo for 'Pre-submit pipelines'; the text immediately following it discusses pre-submit testing and the CostRank function.
  4. [Section IV.A] The statement 'We use the publicly available open-source implementations of COLEMAN' is not accompanied by version or commit information; please include exact versions for reproducibility, particularly since the dataset is provided openly.
  5. [Section VI] The paragraph on safeguard mechanisms correctly acknowledges the risk of incorrect omissions, but it should be tied to the evaluation by stating whether any gate failures or incorrect culprit assignments were observed during the 20-week study period; as written, the deployment risk is acknowledged but not quantified.

Circularity Check

1 steps flagged · score 4.0 of 10

Post-submit 87%/99.78% claims are measured against a three-cycle flaky-transition label that is also the reward definition; pre-submit results are independent.

  1. self definitional [Section IV.C (Test Flakiness) and Section V.B (RQ2), with Eq. (2) in Section III.C]
    "Following other industry practitioners [35], we regard rapidly changing transitions as flaky. Namely, we define a flaky transition as one that switches signals again within three consecutive CI cycles. We chose this threshold based on our churn rate ... Most of our tests transitions switch their signal again within the chosen threshold as can be seen in Fig. 6, indicating flaky transitions. ... In this context, we consider all test cases with a change in the execution result and a positive information gain for developers as relevant. We do not regard transitions due to flakiness as relevant."

    Eq. (2) defines the CostChangeRank reward as +1 for 'relevant transition(ti)', -1 for 'flaky transition(ti)', and -ci otherwise, using the three-cycle rule from Section IV.C. RQ2 then evaluates selection by the percentage of executed tests that reveal a 'relevant test result transition' and by the number of CI cycles needed to detect such transitions, using the same three-cycle rule as the ground truth. Because the threshold was chosen from the churn rate of the same dataset (Fig. 6), the post-submit headline numbers are recall against a self-defined label rather than an externally validated notion of developer relevance.

full rationale

The pre-submit evaluation is self-contained: PR-DQL is compared to COLEMAN, ROCKET, and RANDOM on standard NAPFD/NFR/NTTF metrics, and those results do not reduce to the reward function by construction. The post-submit evaluation, however, uses the same flaky-transition rule (three consecutive CI cycles) both to build the CostChangeRank reward (Eq. 2) and to define the ground truth 'relevant transition' in Section V.B. The threshold is chosen after inspecting the same dataset's churn rate (Section IV.C, Fig. 6), so the 87% and 99.78% figures are conditional on a self-defined label rather than an external validation of developer relevance. This is a definitional circularity for the post-submit target, but it does not make the whole derivation vacuous: the model must still learn to rank tests from language-agnostic features, and the comparison against RNChange and RANDOM under the same label is meaningful. There is no load-bearing self-citation chain or uniqueness argument. Score 4 reflects one partially circular target definition while the central methodology retains independent content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central approach relies on three fitted parameters: k (history length), alpha (cost weight), and the flaky-transition threshold. The latter is especially significant because it defines the ground-truth labels for both training and evaluation. The assumptions are mostly domain-specific: that test names carry semantic signal, that historical results alone are predictive, and that excluding pass-only cycles does not bias the comparison. No new physical or conceptual entities are postulated.

free parameters (3)
  • k (history length) = 25
    Number of past test results encoded in test_result[]; determined empirically on the evaluation data (Section III.B).
  • alpha (cost weighting in CostRank) = 0.9
    Weights execution-cost impact in the pre-submit reward; empirically set to work best in this use case (Section III.C.a).
  • flaky transition threshold = 3 CI cycles
    A transition is labeled flaky if it switches back within three consecutive CI cycles; used to define ground-truth relevant transitions for both training and evaluation (Section IV.C).
assumptions (4)
  • domain assumption Test target names encode affected component and test type due to naming conventions.
    The BoW+PCA feature extracts failure-relevant signal from test names; relied on in Section III.B.
  • domain assumption Historical execution results, durations, and test names are sufficient to predict future failures and outcome transitions.
    The feature set excludes code changes and coverage information; Section III.B. This is central to the 'lightweight' claim.
  • ad hoc to paper A transition that reverses within three CI cycles is flaky and developer-irrelevant.
    Defines training labels and evaluation ground truth; Section IV.C, chosen based on churn rate and Fig. 6 distribution.
  • domain assumption Evaluation on only non-aborted cycles with at least one failing test is unbiased for prioritization quality.
    Cycles without failures are excluded to avoid NAPFD bias, following [34]; Section IV.B. This restricts the scope of the pre-submit claims to failing cycles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Practical Pipeline-Aware Regression Test Optimization for Continuous Integration." pith.science (2026). https://pith.science/paper/3YKIGLH3

@misc{pith2026250111550,
  author       = {Pith},
  title        = {Pith review of: Practical Pipeline-Aware Regression Test Optimization for Continuous Integration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3YKIGLH3}},
  note         = {Machine review of arXiv:2501.11550}
}
read the original abstract

Massive, multi-language, monolithic repositories form the backbone of many modern, complex software systems. To ensure consistent code quality while still allowing fast development cycles, Continuous Integration (CI) is commonly applied. However, operating CI at such scale not only leads to a single point of failure for many developers, but also requires computational resources that may reach feasibility limits and cause long feedback latencies. To address these issues, developers commonly split test executions across multiple pipelines, running small and fast tests in pre-submit stages while executing long-running and flaky tests in post-submit pipelines. Given the long runtimes of many pipelines and the substantial proportion of passing test executions (98% in our pre-submit pipelines), there not only a need but also potential for further improvements by prioritizing and selecting tests. However, many previously proposed regression optimization techniques are unfit for an industrial context, because they (1) rely on complex and difficult-to-obtain features like per-test code coverage that are not feasible in large, multi-language environments, (2) do not automatically adapt to rapidly changing systems where new tests are continuously added or modified, and (3) are not designed to distinguish the different objectives of pre- and post-submit pipelines: While pre-submit testing should prioritize failing tests, post-submit pipelines should prioritize tests that indicate non-flaky changes by transitioning from pass to fail outcomes or vice versa. To overcome these issues, we developed a lightweight and pipeline-aware regression test optimization approach that employs Reinforcement Learning models trained on language-agnostic features. We evaluated our approach on a large industry dataset collected over a span of 20 weeks of CI test executions. When predicting...

Figures

Figures reproduced from arXiv: 2501.11550 by the authors.

Figure 1
Figure 1. Comparison of the test target execution duration [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the executed CI pipelines in the software [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Test execution results over consecutive CI cycles [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Test target prioritization and selection process. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Overview of our regression test prioritization and [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Distribution of the number of test executions between [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Percentage of test cases executed in each CI cycle [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 40 canonical work pages

  1. [35]

    Assessing transition-based test selection algorithms at google,

    C. Leong, A. Singh, M. Papadakis, Y . Le Traon, and J. Micco, “Assessing transition-based test selection algorithms at google,” in International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP), 2019, pp. 101–110

    work page 2019

  2. [1]

    Why google stores billions of lines of code in a single repository,

    J. Levenberg, “Why google stores billions of lines of code in a single repository,” Communications of the ACM , pp. 78–87, 2016

    work page 2016

  3. [2]

    The largest git repo on the planet,

    B. Harry, “The largest git repo on the planet,”

    work page

  4. [3]

    Scaling mercurial at facebook,

    D. Goode, “Scaling mercurial at facebook,” 2014. [On- line]. Available: https://engineering.fb.com/2014/01/07/core-infra/ scaling-mercurial-at-facebook/

    work page 2014

  5. [4]

    Uber technology day: Monorepo to multirepo and back again,

    A. Lucido, “Uber technology day: Monorepo to multirepo and back again,” 2017. [Online]. Available: https://www.youtube.com/watch?v= lV8-1S28ycM

    work page 2017

  6. [5]

    Taming google-scale continuous testing,

    A. Memon, Z. Gao, B. Nguyen, S. Dhanda, E. Nickell, R. Siemborski, and J. Micco, “Taming google-scale continuous testing,” in International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP), 2017, pp. 233–242

    work page 2017

  7. [6]

    Cost-cognizant combinatorial test case prioritization,

    Z. Wang, L. Chen, B. Xu, and Y . Huang, “Cost-cognizant combinatorial test case prioritization,” International Journal of Software Engineering and Knowledge Engineering , pp. 829–854, 2011

    work page 2011

  8. [7]

    Test case prioritization,

    P. R. Srivastava, “Test case prioritization,” Journal of Theoretical & Applied Information Technology, pp. 78–181, 2008

    work page 2008

Show all 41 references
  1. [8]

    Test case selection and prioritization using machine learning: A systematic literature review,

    R. Pan, M. Bagherzadeh, T. A. Ghaleb, and L. Briand, “Test case selection and prioritization using machine learning: A systematic literature review,” Empirical Software Engineering , p. 29, 2022

    2022

  2. [9]

    project contributors

    Z. project contributors. (2023) Zuul - a project gating system. [Online]. Available: https://zuul-ci.org/docs/zuul/

    2023

  3. [10]

    (2023) Using bazel

    Bazel. (2023) Using bazel. [Online]. Available: https://bazel.build/docs

    2023

  4. [11]

    An empirical analysis of flaky tests,

    Q. Luo, F. Hariri, L. Eloussi, and D. Marinov, “An empirical analysis of flaky tests,” in International Symposium on Foundations of Software Engineering (FSE), 2014, pp. 643–653

    2014

  5. [12]

    A survey of flaky tests,

    O. Parry, G. M. Kapfhammer, M. Hilton, and P. McMinn, “A survey of flaky tests,” ACM Transactions on Software Engineering and Methodo- logy, pp. 1–74, 2021

    2021

  6. [13]

    A survey on how test flakiness affects developers and what support they need to address it,

    M. Gruber and G. Fraser, “A survey on how test flakiness affects developers and what support they need to address it,” in International Conference on Software Testing, Verification and Validation (ICST), 2022, pp. 82–92

    2022

  7. [14]

    Cost of flaky tests in continuous integration: An industrial case study,

    F. Leinen, D. Elsner, A. Pretschner, A. Stahlbauer, M. Sailer, and E. J ¨urgens, “Cost of flaky tests in continuous integration: An industrial case study,” in International Conference on Software Testing, Verification and Validation (ICST), 2024, pp. 329–340

    2024

  8. [15]

    Modeling and ranking flaky tests at Apple,

    E. Kowalczyk, K. Nair, Z. Gao, L. Silberstein, T. Long, and A. Memon, “Modeling and ranking flaky tests at Apple,” in International Conference on Software Engineering: Software Engineering in Practice Track (ICSE- SEIP), 2020, pp. 110–119

    2020

  9. [16]

    Presubmit rescue: Automatically ig- noring flaky test executions,

    M. Hoang and A. Berding, “Presubmit rescue: Automatically ig- noring flaky test executions,” in International Flaky Tests Work- shop (FTW@ICSE), 2024, pp. 1–2

    2024

  10. [17]

    Techniques for improving re- gression testing in continuous integration development environments,

    S. Elbaum, G. Rothermel, and J. Penix, “Techniques for improving re- gression testing in continuous integration development environments,” in International Symposium on Foundations of Software Engineering (FSE) , 2014, pp. 235–245

    2014

  11. [18]

    The impact of flaky tests on historical test prioritization on chrome,

    E. Fallahzadeh and P. C. Rigby, “The impact of flaky tests on historical test prioritization on chrome,” in International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP) , 2022, pp. 273–282

    2022

  12. [19]

    Learning to rank for information retrieval,

    T.-Y . Liuet al., “Learning to rank for information retrieval,” Foundations and Trends® in Information Retrieval , pp. 225–331, 2009

    2009

  13. [20]

    Q-learning,

    C. J. Watkins and P. Dayan, “Q-learning,” Machine Learning, pp. 279– 292, 1992

    1992

  14. [21]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” Nature, pp. 529–533, 2015

    2015

  15. [22]

    Deep reinforcement learning: From q- learning to deep q-learning,

    F. Tan, P. Yan, and X. Guan, “Deep reinforcement learning: From q- learning to deep q-learning,” in Neural Information Processing: 24th International Conference (ICONIP) , 2017, pp. 475–483

    2017

  16. [23]

    TCP-Net: Test case prioritization using end-to-end deep neural networks,

    M. Abdelkarim and R. ElAdawi, “TCP-Net: Test case prioritization using end-to-end deep neural networks,” in International Conference on Software Testing, Verification and Validation Workshops (ICST- Workshops), 2022, pp. 122–129

    2022

  17. [24]

    Bag-of-word normalized n-gram models,

    A. Sethy and B. Ramabhadran, “Bag-of-word normalized n-gram models,” in Annual Conference of the International Speech Communication Association (INTERSPEECH), 2008, pp. 1594–1597

    2008

  18. [25]

    Indexing by latent semantic analysis,

    S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, “Indexing by latent semantic analysis,” Journal of the American Society for Information Science , pp. 391–407, 1990

    1990

  19. [26]

    Dimensionality reduction for bag-of-words models: PCA vs LSA,

    B. F. Ljungberg, “Dimensionality reduction for bag-of-words models: PCA vs LSA,” Semanticscholar.org, 2017

    2017

  20. [27]

    A multi-armed bandit approach for test case prioritization in continuous integration environments,

    J. A. P. Lima and S. R. Vergilio, “A multi-armed bandit approach for test case prioritization in continuous integration environments,” IEEE Transactions on Software Engineering , pp. 453–465, 2022

    2022

  21. [28]

    Reinforcement learning for automatic test case prioritization and selection in continuous integration,

    H. Spieker, A. Gotlieb, D. Marijan, and M. Mossige, “Reinforcement learning for automatic test case prioritization and selection in continuous integration,” in International Symposium on Software Testing and Analysis (ISSTA), 2017, pp. 12–22

    2017

  22. [29]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR) , 2015

    2015

  23. [30]

    Revisiting fundamentals of experience replay,

    W. Fedus, P. Ramachandran, R. Agarwal, Y . Bengio, H. Larochelle, M. Rowland, and W. Dabney, “Revisiting fundamentals of experience replay,” in International Conference on Machine Learning , 2020, pp. 3061–3071

    2020

  24. [31]

    Self-improving reactive agents based on reinforcement learning, planning and teaching,

    L.-J. Lin, “Self-improving reactive agents based on reinforcement learning, planning and teaching,” Machine Learning, pp. 293–321, 1992

    1992

  25. [32]

    IOF/ROL - testing results (Travis CI),

    J. Prado Lima, “IOF/ROL - testing results (Travis CI),” 2020. [Online]. Available: doi.org/10.7910/DVN/GIJ5DE

    2020 doi

  26. [33]

    Test case prioritization for continuous regression testing: An industrial case study,

    D. Marijan, A. Gotlieb, and S. Sen, “Test case prioritization for continuous regression testing: An industrial case study,” in International Conference on Software Maintenance (ICSM) , 2013, pp. 540–543

    2013

  27. [34]

    Reinforcement learning for test case prioritization,

    M. Bagherzadeh, N. Kahani, and L. Briand, “Reinforcement learning for test case prioritization,” IEEE Transactions on Software Engineering , pp. 2836–2856, 2021

    2021

  28. [36]

    Combinatorial interaction regression testing: A study of test case generation and prioritization,

    X. Qu, M. B. Cohen, and K. M. Woolf, “Combinatorial interaction regression testing: A study of test case generation and prioritization,” in International Conference on Software Maintenance (ICSM) , 2007, pp. 255–264

    2007

  29. [37]

    Weighted reward for reinforcement learning based test case prioritization in continuous integration testing,

    G. Li, Y . Yang, Z. Wu, T. Cao, Y . Liu, and Z. Li, “Weighted reward for reinforcement learning based test case prioritization in continuous integration testing,” in Computers, Software, and Applications Conference (COMPSAC), 2021, pp. 980–985

    2021

  30. [38]

    Data-driven test selection at scale,

    S. Mehta, F. Farmahinifarahani, R. Bhagwan, S. Guptha, S. Jafari, R. Kumar, V . Saini, and A. Santhiar, “Data-driven test selection at scale,” in Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/FS...

    2021

  31. [39]

    DeepOrder: Deep learning for test case prioritization in continuous integration testing,

    A. Sharif, D. Marijan, and M. Liaaen, “DeepOrder: Deep learning for test case prioritization in continuous integration testing,” in International Conference on Software Maintenance and Evolution (ICSME) , 2021, pp. 525–534

    2021

  32. [40]

    Learning-to-rank vs ranking-to-learn: Strategies for regression testing in continuous integration,

    A. Bertolino, A. Guerriero, B. Miranda, R. Pietrantuono, and S. Russo, “Learning-to-rank vs ranking-to-learn: Strategies for regression testing in continuous integration,” in International Conference on Software Engineering (ICSE), 2020, pp. 1–12. All online resources accessed...

    2020

  33. [2017]

    Available: https://devblogs.microsoft.com/bharry/ the-largest-git-repo-on-the-planet/

    [Online]. Available: https://devblogs.microsoft.com/bharry/ the-largest-git-repo-on-the-planet/

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.