Pith. sign in

REVIEW 4 major objections 5 minor 74 references

Scalability of Reinforcement Learning Methods for Dispatching in Semiconductor Frontend Fabs: A Comparison of Open-Source Models with Real Industry Datasets

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that an evolution-strategy optimizer (CMA-ES) scales to realistic semiconductor fab dispatching far better than policy-gradient reinforcement learning (PPO), cutting tardiness by double digits on open-source benchmark…

desk verdict Useful same-simulator comparison of CMA-ES and PPO on real fab data, but the 'scales much better' headline is not yet earned because the PPO baseline is structurally disadvantaged. read the letter →

arxiv 2505.11135 v1 pith:HR3V7B3P submitted 2025-05-16 cs.AI cs.LGcs.NE

Patrick Stöckermann , Henning Südfeld , Alessandro Immordino , Thomas Altenmüller , Marc Wegmann , Martin Gebser , Konstantin Schekotihin , Georg Seidel
show 2 more authors
Chew Wye Chan Fei Fei Zhang
This is my paper · ORCID
classification cs.AIcs.LGcs.NE
keywords ReinforcementLearningDispatchingSchedulingSemiconductorManufacturingScalabilityEvolutionStrategiesSMT2020Minifab
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper compares two reinforcement-learning optimizers, the policy-gradient method PPO and the evolution-strategy method CMA-ES, on the same lot-dispatching task across three simulation environments of increasing realism: the small Minifab model, the large SMT2020 benchmark, and a real frontend-fab dataset with more than a thousand machines. It tries to establish that CMA-ES scales much better than PPO as the simulation grows: CMA-ES improves tardiness in all three settings, while PPO improves only the tiny Minifab and degrades performance on the larger models. It also claims that controlling more bottleneck tool groups, not just lithography, is decisive, and that training on diverse loading scenarios and failure patterns improves generalization. If true, this gives fab owners a practical path: a derivative-free evolution strategy running many parallel simulator instances can beat hand-tuned dispatching heuristics on realistic industrial data, without the credit-assignment and value-estimation problems that hurt PPO in this setting.

What carries the argument

The load-bearing object is the policy network trained by CMA-ES: a small attention-based network (scaled dot-product attention over lot-wise observations, followed by a feed-forward scoring head) that ranks the lots waiting at a tool. CMA-ES treats the network weights as a black-box parameter vector, samples perturbed candidates in parallel, runs a full fab simulation episode for each candidate, and updates the mean and covariance of the search distribution toward the candidates with the best normalized tardiness-and-throughput cost. The other central piece is the experiment design: every model runs on the same commercial discrete-event simulator with the same observation and action interface, so the only variables changed are the optimizer, the set of controlled tool groups, and the training data mix. The attention mechanism is what lets the policy handle queues of arbitrary length, while the episode-level cost function is what lets the evolution strategy sidestep the credit-assignment problem that the paper identifies as PPO's bottleneck.

What would settle it

Run the same CMA-ES and PPO training pipelines on an independently written discrete-event simulator implementation of the Minifab and SMT2020 models with identical due dates and random seeds: if PPO matches or beats CMA-ES there, or if CMA-ES's advantage disappears, the scalability ordering is an artifact of the particular simulator. A second check: deploy the CMA-ES policy on a different real fab with different tool dedications and load mixes and measure tardiness against that fab's own heuristics; the claim fails if the improvement does not reproduce or turns negative.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the scalability of reinforcement learning for fab dispatching is determined less by the sophistication of the learning algorithm than by how it handles delayed, noisy rewards inside a large simulator. CMA-ES assigns one cost per full simulation episode, samples perturbations of the policy parameters in parallel, and uses the covariance of successful perturbations to guide the next search; that black-box structure fits the sparse, long-delayed reward signal of a wafer fab. PPO, by contrast, needs frequent state-action-reward samples and an accurate value-function estimate, both of which become unreliable when an episode spans thousands of actions and weeks of simulated time. The measured outcome is that CMA-ES produced a 29.4% average tardiness improvement on Minifab when all tool types were controlled, double-digit tardiness gains on SMT2020, and up to 4% tardiness and 1% throughput improvement on the industrial dataset, while PPO improved tardiness only on Minifab and made it worse on SMT2020 and industry. The paper also finds that the share of dispatching decisions placed under the agent's control matters more than which single tool is optimized: batching tools, for example, only pay off when the agent also controls the upstream tools that feed them.

Load-bearing premise

The entire comparison rests on the assumption that the commercial simulator evaluates all three fab models faithfully and that the industry dataset's due dates and loading scenarios are realistic ground truth; if either is distorted, the measured improvements and the claimed ordering of the two methods may not transfer to other fabs.

Editorial extensions

If this is right

  • On fab models of realistic size and complexity, evolution-strategy optimization should be the preferred reinforcement-learning choice over policy-gradient methods, because long-delayed and noisy rewards break value-function learning.
  • The more dispatching decisions the agent controls, especially batching tools together with their upstream feeders, the larger the tardiness improvement, so choosing which bottleneck tool groups to control is part of the optimization itself.
  • Training on multiple loading scenarios and random seeds produces agents that generalize to unseen scenarios and longer time horizons, though with some loss of improvement.
  • Training cost is dominated by simulator runtime and grows with the share of controlled tools, but wall-clock time scales with the number of CPU cores used, so the method is embarrassingly parallel.
  • A policy that beats a fab's current expert-tuned heuristics by 4% tardiness and 1% throughput on industrial data is operationally significant, since fab capacity carries a large weekly cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If this scalability ordering holds beyond the particular simulator, derivative-free evolution-strategy policies could transfer to other sparse-reward production scheduling problems, such as back-end testing or reticle logistics, where the credit-assignment problem is similarly severe.
  • The authors' memory constraint suggests a concrete extension: a PPO variant that compresses or replays episode samples could recover some of the lost performance at industrial scale, since the paper's own data show complete-episode updates are more stable than truncated ones.
  • A cheap practical recipe implied by the tool-combination results is to run the small Minifab model first to select the bottleneck tool set, then train only on those tools for the expensive industrial model, reducing the training hours documented in the paper.
  • Training a CMA-ES agent on a deliberately diverse set of loading scenarios rather than a fixed one should reduce the over-optimization of a single training scenario that the paper observes, since its generalization experiments link diversity to robustness.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper compares two reinforcement learning approaches, CMA-ES (an evolution-strategy method) and PPO (a policy-gradient method), for lot dispatching in semiconductor frontend fabs. Experiments are run on three simulation models—Minifab, SMT2020, and a proprietary large-scale industry dataset—all implemented in the D-SIMCON commercial simulator. The authors report that CMA-ES achieves double-digit tardiness improvements on Minifab and SMT2020 and up to 4% tardiness and 1% throughput improvement on the industry dataset, while PPO improves tardiness only on Minifab and degrades it on the larger models. They conclude that the evolution-strategy approach 'scales much better' than the policy-gradient approach, and they additionally analyze the influence of the set of controlled tools, computational cost, and generalization to unseen loading scenarios.

Significance. If the main claim were established, this would be a useful contribution to a field where comparisons are often made on incompatible testbeds: the paper uses the same simulator engine for all three scenarios, includes an industry-scale dataset with diverse load mixes and tool dedications, and provides generalization experiments for changing loading scenarios. The work also gives concrete quantitative results on computational cost and shows that controlling more tool groups improves CMA-ES performance. However, the headline comparison between CMA-ES and PPO is currently not a controlled algorithm comparison: the two methods differ in action-space scope, reward function, episode-buffer handling, and tuning effort. Because the paper itself concedes (Section 6) that PPO might perform better with more dedicated hardware, the claim that the proposed method 'scales much better' is overstated as presented. The underlying observations are valuable, but the paper needs a substantial revision to support the central scalability statement.

major comments (4)
  1. [3.2 and 4.2] The comparison between CMA-ES and PPO is not matched in action space. Section 3.2 states that 'Batching tools are therefore not considered for the PPO experiments', and Section 4.2 reports PPO results only for the lithography area. In contrast, CMA-ES's best results are obtained when all tool types are controlled (Fig. 8 for Minifab, Fig. B3 for SMT2020, Fig. B4 for industry). The observed gap therefore confounds algorithmic scalability with the scope of dispatching decisions available to each agent. The paper should either run PPO with the same expanded action space (including batching, which would require an appropriate action representation) or explicitly frame the headline result as a comparison of the two methods under different control scopes.
  2. [4.2 and 6] PPO uses truncated episode buffers for SMT2020 and the industry model because full-buffer samples overflow memory, while complete episodes are used only for Minifab. The Limitations section explicitly states that 'with more dedicated hardware resources, the PPO potentially would deliver more promising result for the industrial scale scenario.' This means the reported PPO degradation on larger models may reflect memory and compute constraints rather than algorithmic scaling behavior. To support the 'scales much better' claim, the authors should report matched computational budgets (e.g., CPU-hours and memory usage per method) or, if feasible, run PPO with a larger buffer. At minimum, the conclusion should be rephrased to say that CMA-ES scaled better under the authors' resource constraints, not that it scales better in general.
  3. [3.3 and Appendix A] The two methods are optimized against different objectives. CMA-ES uses the cost functions c_ES or c'_ES, while PPO uses the reward r_PPO,t, which was selected as the best among several candidate rewards tested on the same MiniFab benchmark (Appendix A). Moreover, per-model hyperparameters (rollout fragment length, episode truncation, learning rates) are tuned for PPO, and CMA-ES hyperparameters are also tuned. This asymmetric tuning burden means the comparison reflects one particular configuration of each method, not a general property of the algorithms. The paper should state this limitation prominently and ideally provide a sensitivity analysis for the PPO reward choice on the larger models.
  4. [Figures 5, 7, 8 and Section 4] Results are reported as averages over the training progress without confidence intervals or seed-level variance. Because the simulator uses random seeds for stochastic tool failures and load scenarios, the differences between, e.g., CMA-ES LIT (15.1%) and LIT+DIF (15.1%) in Fig. 8 or the negative PPO values in Fig. 5 might be within run-to-run noise. The paper should report multiple seeds and error bars, or at least provide a statistical measure of significance for the central comparison. This is particularly important for the industry dataset, where improvements are small (up to 4% tardiness, 1% throughput).
minor comments (5)
  1. [2.1] The text contains a typo: 'Eearliest Due Date' should be 'Earliest Due Date' in the list of dispatching heuristics in Section 2.1.
  2. [2.2] The heading 'Semiconductor Frontened Manufacturing Testbeds' contains a typo: 'Frontened' should be 'Frontend'.
  3. [Table 3] Several entries in Table 3 have formatting problems (e.g., '20243 Yedidsion', 'Zhangg', and superscript markers that are not explained consistently). Please clean up the table formatting and define all footnotes.
  4. [4.4] Figure 9 reports computation times in hours/days but does not specify the number of CPU cores used for each experiment. Since the paper claims scalability with CPU cores, the core count should be stated explicitly in the caption or text.
  5. [References] Reference [21] (Karaboga) is cited with a DOI for the Grey Wolf Optimizer paper [38], which appears incorrect. Please verify all DOIs.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: all claimed improvements are empirical comparisons against fixed dispatching baselines.

full rationale

The paper's load-bearing claims are empirical: CMA-ES improves tardiness versus fixed heuristics on Minifab, SMT2020, and an industrial scenario, and these gains are measured against reference runs of SRPT, CR, and expert rules (Tables 4-5; Section 4). The ES cost function and PPO reward are expressed in terms of the same tardiness and throughput KPIs that are later reported, but that is choosing an optimization objective, not deriving a prediction from a fitted input; no equation in Sections 3.3 or 4 makes an output equal to an input by construction. The citation to the authors' prior CMA-ES work ([56]) is not load-bearing: the present paper describes the network architecture, observation/action space, and training procedure, and the CMA-ES algorithm itself is cited to the independent source [19]. The paper's own limitations explicitly concede that PPO was trained with truncated episode buffers on the larger models and never controlled batching tools, so the 'scales much better' headline is weakened by an asymmetric comparison; however, that is a fairness/validity concern, not circularity. Generalization tests on unseen loading scenarios and longer horizons (Figures 10-12) provide held-out checks rather than re-reporting fitted values. No self-definitional, fitted-input-as-prediction, or uniqueness-imported-from-authors pattern is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on simulator fidelity, baseline choice, and algorithm implementations; no new physical entities are introduced. Freely chosen cost and reward forms plus several undisclosed hyperparameters influence the results, so the empirical comparison is not fully pinned down by the paper alone.

free parameters (5)
  • alpha1 = alpha2 = 10 = 10
    Weights in the ES cost function c_ES that penalize throughput and internal-tardiness degradation; chosen by hand in Section 3.3.
  • MiniFab planned flow factor = Uniform(2.1, 2.5)
    Used to generate due dates for MiniFab, which lacks them; sampled from a distribution chosen by the authors in Section 3.1.
  • PPO reward r_D = Selected among r_A, r_B, r_C, r_D
    The reward used for PPO was "empirically found to be the best performing one" in Section 3.3 and Appendix A.
  • Per-model hyperparameters (rollout fragment length, episode truncation, learning rates) = Not reported in full
    Section 4.2 says hyperparameters had to be adapted for each model, but full values are not given.
  • CMA-ES hyperparameters (population size, sigma, N_best) = Not reported
    Algorithm 2 inputs are not specified in the paper, which affects training dynamics and the fairness of the algorithm comparison.
assumptions (4)
  • domain assumption D-SIMCON simulator faithfully represents all three fab models
    Section 3.1 states that D-SIMCON is "highly realistic" and handles all models; if this fails, the comparisons are not meaningful.
  • domain assumption The selected baseline heuristics are appropriate references
    Section 4.1 selects SRPT for Minifab and CR for SMT2020 as baselines; weaker baselines would inflate reported improvements.
  • standard math CMA-ES and PPO implementations correctly instantiate their published algorithms
    Section 2.3 presents both algorithms as standard; implementation bugs would invalidate the scalability comparison.
  • domain assumption The industry data, due dates, and loading scenarios are representative of real operations
    Section 3.1 uses due dates from the industry dataset as ground truth; unrepresentative data would limit transferability of the results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scalability of Reinforcement Learning Methods for Dispatching in Semiconductor Frontend Fabs: A Comparison of Open-Source Models with Real Industry Datasets." pith.science (2026). https://pith.science/paper/HR3V7B3P

@misc{pith2026250511135,
  author       = {Pith},
  title        = {Pith review of: Scalability of Reinforcement Learning Methods for Dispatching in Semiconductor Frontend Fabs: A Comparison of Open-Source Models with Real Industry Datasets},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HR3V7B3P}},
  note         = {Machine review of arXiv:2505.11135}
}
read the original abstract

Benchmark datasets are crucial for evaluating approaches to scheduling or dispatching in the semiconductor industry during the development and deployment phases. However, commonly used benchmark datasets like the Minifab or SMT2020 lack the complex details and constraints found in real-world scenarios. To mitigate this shortcoming, we compare open-source simulation models with a real industry dataset to evaluate how optimization methods scale with different levels of complexity. Specifically, we focus on Reinforcement Learning methods, performing optimization based on policy-gradient and Evolution Strategies. Our research provides insights into the effectiveness of these optimization methods and their applicability to realistic semiconductor frontend fab simulations. We show that our proposed Evolution Strategies-based method scales much better than a comparable policy-gradient-based approach. Moreover, we identify the selection and combination of relevant bottleneck tools to control by the agent as crucial for an efficient optimization. For the generalization across different loading scenarios and stochastic tool failure patterns, we achieve advantages when utilizing a diverse training dataset. While the overall approach is computationally expensive, it manages to scale well with the number of CPU cores used for training. For the real industry dataset, we achieve an improvement of up to 4% regarding tardiness and up to 1% regarding throughput. For the less complex open-source models Minifab and SMT2020, we observe double-digit percentage improvement in tardiness and single digit percentage improvement in throughput by use of Evolution Strategies.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 46 canonical work pages

  1. [1]

    Prod Eng Res Devel 14

    Altenm¨ uller T, St¨ uker T, Waschneck B, et al (2020) Reinforcement learning for an intelligent and autonomous production control of complex job-shops under time constraints. Prod Eng Res Devel 14. https://doi.org/10.1007/s11740-020-00967-8

    work page doi:10.1007/s11740-020-00967-8 2020

  2. [2]

    INFORMS J Comput 3(2):149–156

    Applegate DL, Cook WJ (1991) A computational study of the job-shop scheduling problem. INFORMS J Comput 3(2):149–156. https://doi.org/10.1287/IJOC.3.2. 149

    work page doi:10.1287/ijoc.3.2 1991

  3. [3]

    Springer, Cham, Switzerland, https://doi.org/10.1007/978-3-030-41544-0

    Bangsow S (2016) Tecnomatix Plant Simulation. Springer, Cham, Switzerland, https://doi.org/10.1007/978-3-030-41544-0

    work page doi:10.1007/978-3-030-41544-0 2016

  4. [4]

    Applied Sciences 13(6):3615

    Bauer D, Umgelter D, Schlereth A, et al (2023) Complex job shop simula- tion “cojosim”—a reference model for simulating semiconductor manufacturing. Applied Sciences 13(6):3615. https://doi.org/10.3390/app13063615

    work page doi:10.3390/app13063615 2023

  5. [5]

    Optim Lett 8(4):1417–1431

    Birgin EG, Feofiloff P, Fernandes CG, et al (2014) A MILP model for an extended version of the flexible job shop problem. Optim Lett 8(4):1417–1431. https://doi. org/10.1007/S11590-013-0669-7

    doi:10.1007/s11590-013-0669-7 2014

  6. [6]

    Springer, Berlin and New York, https://doi.org/10.1007/10.1007/978-3-540-69516-5

    Brucker P (2007) Scheduling algorithms, 5th edn. Springer, Berlin and New York, https://doi.org/10.1007/10.1007/978-3-540-69516-5

    work page doi:10.1007/10.1007/978-3-540-69516-5 2007

  7. [7]

    SEMATECH Technical Transfer report 26

    Campbell E, Ammenheuser J (2000) 300mm factory layout and material handling modeling: Phase ii report. SEMATECH Technical Transfer report 26

    work page 2000

  8. [8]

    Comput Ind Eng 162:107782

    Chien C, Lan Y (2021) Agent-based approach integrating deep reinforcement learning and hybrid genetic algorithm for dynamic scheduling for industry 3.5 smart production. Comput Ind Eng 162:107782. https://doi.org/10.1016/J.CIE. 2021.107782

    arXiv 2021

Show all 74 references
  1. [9]

    URL http://www.d-simlab.com/ category/d-simcon/products-d-simcon/forecaster-and-scenario-manager/

    D-SIMLAB Technologies (2023) Forecaster. URL http://www.d-simlab.com/ category/d-simcon/products-d-simcon/forecaster-and-scenario-manager/

    2023

  2. [10]

    Operations Research Perspectives 9:100249

    Da Col G, Teppan EC (2022) Industrial-size job shop scheduling with constraint programming. Operations Research Perspectives 9:100249. https://doi.org/10. 1016/j.orp.2022.100249

    2022

  3. [11]

    Eur J Oper Res 109(1):137–141

    Demirkol E, Mehta S, Uzsoy R (1998) Benchmarks for shop scheduling problems. Eur J Oper Res 109(1):137–141. https://doi.org/10.1016/S0377-2217(97)00019-2

    1998 doi

  4. [13]

    In: Proceedings of 35th IEEE Conference on Decision and Control, vol 2

    El Adl MK, Rodriguez AA, Tsakalis KS (1996) Hierarchical modeling and con- trol of re-entrant semiconductor manufacturing facilities. In: Proceedings of 35th IEEE Conference on Decision and Control, vol 2. IEEE, pp 1736–1742, https://doi.org/10.1109/CDC.1996.572810

    1996

  5. [14]

    In: 2009 International Conference on Computers & Industrial Engineering

    El-Khouly IA, El-Kilany KS, El-Sayed AE (2009) Modelling and simulation of re-entrant flow shop scheduling: An application in semiconductor manufacturing. In: 2009 International Conference on Computers & Industrial Engineering. IEEE, pp 211–216, https://doi.org/10.1109/ICCIE.2...

    2009

  6. [15]

    URL https://www.scopus.com, accessed: 2025-01-09

    Elsevier (2025) Scopus. URL https://www.scopus.com, accessed: 2025-01-09

    2025

  7. [16]

    Fowler JW, Robinson J (1995) Measurement and improvement of manufacturing capacity (mimac) final report

    1995

  8. [17]

    In: Hammer P, Johnson E, Korte B (eds) Discrete Optimization II, Annals of Discrete Mathematics, vol 5

    Graham R, Lawler E, Lenstra J, et al (1979) Optimization and approximation in deterministic sequencing and scheduling: a survey. In: Hammer P, Johnson E, Korte B (eds) Discrete Optimization II, Annals of Discrete Mathematics, vol 5. Elsevier, Amsterdam, New York, Oxford, p 287...

    1979

  9. [18]

    The International Journal of Advanced Manufacturing Technology 27(11–12):1163–1169

    Gupta AK, Sivakumar AI (2004) Job shop scheduling techniques in semicon- ductor manufacturing. The International Journal of Advanced Manufacturing Technology 27(11–12):1163–1169. https://doi.org/10.1007/s00170-004-2296-z

    2004 doi

  10. [19]

    Evol Comput 9(2):159–195

    Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2):159–195. https://doi.org/10.1162/ 106365601750190398 27

    2001

  11. [20]

    In: AAAI

    van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning. In: AAAI. AAAI Press, pp 2094–2100, https://doi.org/10.1609/AAAI. V30I1.10295

    2016 doi

  12. [21]

    https://doi.org/10.1609/AAAI.V30I1.10295

    Karabo˘ ga D (2005) An idea based on honey bee swarm for numerical optimiza- tion. https://doi.org/10.1609/AAAI.V30I1.10295

    2005 doi

  13. [22]

    Production and Inventory Management Journal 38(4):51–57

    Kayton D, Teyner T, Schwartz C, et al (1997) Focusing maintenance improvement efforts in a wafer fabrication facility operating under the theory of constraints. Production and Inventory Management Journal 38(4):51–57

    1997

  14. [23]

    Vieweg+Teubner Verlag, Wiesbaden, Germany, https://doi.org/10.1007/ 978-3-8348-1994-9

    Klemmt A (2012) Ablaufplanung in der Halbleiter- und Elektronikproduk- tion. Vieweg+Teubner Verlag, Wiesbaden, Germany, https://doi.org/10.1007/ 978-3-8348-1994-9

    2012

  15. [24]

    Eur J Oper Res 263(1):50–61

    Knopp S, Dauz` ere-P´ er` es S, Yugma C (2017) A batch-oblivious approach for complex job-shop scheduling problems. Eur J Oper Res 263(1):50–61. https: //doi.org/10.1016/J.EJOR.2017.04.050

    2017 doi

  16. [25]

    IEEE Transactions on Semiconductor Manufacturing 33(4):522–

    Kopp D, Hassoun M, Kalir A, et al (2020) Smt2020—a semiconductor manufac- turing testbed. IEEE Transactions on Semiconductor Manufacturing 33(4):522–

    2020

  17. [26]

    In: ASMC

    Kovacs B, Tassel P, Ali R, et al (2022) A customizable simulator for artificial intelligence research to schedule semiconductor fabs. In: ASMC. IEEE, pp 1–6, https://doi.org/10.1109/ASMC54647.2022.9792520

    2022

  18. [27]

    Constraints 23

    Laborie P, Rogerie J, Shaw P, et al (2018) Ibm ilog cp optimizer for scheduling: 20+ years of scheduling with constraints at ibm/ilog. Constraints 23. https://doi. org/10.1007/S10601-018-9281-X

    2018 doi

  19. [28]

    Lee W, Kim B, Ko K, et al (2019) Simulation based multi-objective fab scheduling by using reinforcement learning. In: WSC. IEEE, pp 2236–2247, https://doi.org/ 10.1109/WSC40007.2019.9004886

    2019

  20. [29]

    PhD thesis, University of California, Berkeley, USA

    Liang E (2021) Scalable reinforcement learning systems and their applications. PhD thesis, University of California, Berkeley, USA

    2021

  21. [30]

    In: ICML, Proceedings of Machine Learning Research, vol 80

    Liang E, Liaw R, Nishihara R, et al (2018) Rllib: Abstractions for distributed reinforcement learning. In: ICML, Proceedings of Machine Learning Research, vol 80. PMLR, pp 3059–3068

    2018

  22. [31]

    In: IEA/AIE (2), Lecture Notes in Computer Science, vol 13926

    Liao Z, Chen J, Zhang Z (2023) Solving job-shop scheduling problem via deep reinforcement learning with attention model. In: IEA/AIE (2), Lecture Notes in Computer Science, vol 13926. Springer, pp 201–212, https://doi.org/10.1007/ 978-3-031-36822-6 18 28

    2023

  23. [32]

    IEEE Trans Autom Sci Eng 19(4):3659–3671

    Lin C, Cao Z, Zhou M (2022) Learning-based grey wolf optimizer for stochastic flexible job shop scheduling. IEEE Trans Autom Sci Eng 19(4):3659–3671. https: //doi.org/10.1109/TASE.2021.3129439

    2022

  24. [33]

    IEEE Trans Cybern 53(10):6663–

    Lin C, Cao Z, Zhou M (2023) Learning-based cuckoo search algorithm to schedule a flexible job shop with sequencing flexibility. IEEE Trans Cybern 53(10):6663–

    2023

  25. [34]

    Complex & Intelligent Systems 8(6):4641–4662

    Liu J, Qiao F, Zou M, et al (2022) Dynamic scheduling for semiconductor man- ufacturing systems with uncertainties using convolutional neural networks and reinforcement learning. Complex & Intelligent Systems 8(6):4641–4662. https: //doi.org/10.1007/s40747-022-00844-0

    2022 doi

  26. [35]

    Eng Appl Artif Intell 133:108487

    Lu S, Wang Y, Kong M, et al (2024) A double deep q-network framework for a flexible job shop scheduling problem with dynamic job arrivals and urgent job insertions. Eng Appl Artif Intell 133:108487. https://doi.org/10.1016/J. ENGAPPAI.2024.108487

    2024

  27. [36]

    Neural Comput Appl 35(30):22281–22296

    Ma Y, Cai J, Li S, et al (2023) Double deep q-network-based self-adaptive schedul- ing approach for smart shop floor. Neural Comput Appl 35(30):22281–22296. https://doi.org/10.1007/S00521-023-08877-3

    2023 doi

  28. [37]

    Journal of Scheduling https://doi.org/10.1002/jos.102

    Mason SJ, Fowler JW, Carlyle WM (2022) A modified shifting bottleneck heuris- tic for minimizing total weighted tardiness in complex job shops. Journal of Scheduling https://doi.org/10.1002/jos.102

    2022 doi

  29. [38]

    Adv Eng Softw 69:46–61

    Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61. https://doi.org/10.1016/J.ADVENGSOFT.2013.12.007

    2014 doi

  30. [39]

    CoRR abs/1312.5602

    Mnih V, Kavukcuoglu K, Silver D, et al (2013) Playing atari with deep reinforcement learning. CoRR abs/1312.5602

    2013 arXiv

  31. [40]

    In: ICML, JMLR Workshop and Conference Proceedings, vol 48

    Mnih V, Badia AP, Mirza M, et al (2016) Asynchronous methods for deep rein- forcement learning. In: ICML, JMLR Workshop and Conference Proceedings, vol 48. JMLR.org, pp 1928–1937

    2016

  32. [41]

    J Sched 14(6):583–599

    M¨ onch L, Fowler JW, Dauz` ere-P´ er` es S, et al (2011) A survey of problems, solu- tion techniques, and future challenges in scheduling semiconductor manufacturing operations. J Sched 14(6):583–599. https://doi.org/10.1007/s10951-010-0222-9

    2011 doi

  33. [42]

    Springer, Cham, Switzerland, https://doi.org/10.1007/978-1-4614-4472-5

    M¨ onch L, Fowler JW, Mason SJ (2013) Production Planning and Control for Semiconductor Wafer Fabrication Facilities - Modeling, Analysis, and Systems, Operations research / computer science interfaces series, vol 52. Springer, Cham, Switzerland, https://doi.org/10.1007/978-1-...

    2013 doi

  34. [43]

    IEEE Trans Autom Sci Eng 29 17(3):1420–1431

    Park I, Huh J, Kim J, et al (2020) A reinforcement learning approach to robust scheduling of semiconductor manufacturing facilities. IEEE Trans Autom Sci Eng 29 17(3):1420–1431. https://doi.org/10.1109/TASE.2019.2956762

    2020

  35. [44]

    URL https://developers.google.com/ optimization/cp/cp solver

    Perron L, Didier F (2023) Cp-sat. URL https://developers.google.com/ optimization/cp/cp solver

    2023

  36. [45]

    Phillips T (1998) AUTOSCHED AP by autosimulations. In: WSC. WSC, pp 219–222, https://doi.org/10.1109/WSC.1998.744926

    1998

  37. [46]

    Springer series in operations research, Springer, New York, NY, USA, https://doi.org/10

    Pinedo M (2005) Planning and scheduling in manufacturing and services. Springer series in operations research, Springer, New York, NY, USA, https://doi.org/10. 1007/978-1-4419-0910-7

    2005

  38. [47]

    Springer US, New York, NY, USA, https://doi.org/10.1007/978-3-031-05921-6

    Pinedo ML (2012) Scheduling: Theory, Algorithms, and Systems. Springer US, New York, NY, USA, https://doi.org/10.1007/978-3-031-05921-6

    2012 doi

  39. [48]

    J Intell Manuf 34(3):1311–1324

    Sakr AH, AboElHassan A, Yacout S, et al (2023) Simulation and deep reinforce- ment learning for adaptive dispatching in semiconductor manufacturing systems. J Intell Manuf 34(3):1311–1324. https://doi.org/10.1007/S10845-021-01851-7

    2023 doi

  40. [49]

    CoRR abs/1703.03864

    Salimans T, Ho J, Chen X, et al (2017) Evolution strategies as a scalable alternative to reinforcement learning. CoRR abs/1703.03864

    2017 arXiv

  41. [50]

    URL https://simpy.readthedocs.io/en/ latest/

    Scherfke S, L¨ unsdorf O (2013) Simpy. URL https://simpy.readthedocs.io/en/ latest/

    2013

  42. [51]

    In: ICML, JMLR Workshop and Conference Proceedings, vol 37

    Schulman J, Levine S, Abbeel P, et al (2015) Trust region policy optimization. In: ICML, JMLR Workshop and Conference Proceedings, vol 37. JMLR.org, pp 1889–1897

    2015

  43. [52]

    CoRR abs/1707.06347

    Schulman J, Wolski F, Dhariwal P, et al (2017) Proximal policy optimization algorithms. CoRR abs/1707.06347

    2017 arXiv

  44. [53]

    In: 2024 5th International Conference on Electronic Communication and Artificial Intelligence (ICECAI), pp 713–716, https://doi.org/10.1109/ICECAI62591.2024.10675000

    Shao H, Ge J, Wang G (2024) Attention assigning: Stacked dual network-based reinforcement learning for solving assignment problems. In: 2024 5th International Conference on Electronic Communication and Artificial Intelligence (ICECAI), pp 713–716, https://doi.org/10.1109/ICECA...

    2024

  45. [54]

    IEEE Access 8:106542– 106553

    Shiue Y, Lee K, Su C (2020) A reinforcement learning approach to dynamic scheduling in a product-mix flexibility environment. IEEE Access 8:106542– 106553. https://doi.org/10.1109/ACCESS.2020.3000781

    2020

  46. [55]

    In: ASMC

    Spier J, Kempf K (1995) Simulation of emergent behavior in manufactur- ing systems. In: ASMC. IEEE, pp 90–94, https://doi.org/10.1109/ASMC.1995. 484347

    1995 doi

  47. [57]

    CIRP Annals 67(1):511 – 514

    Stricker N, Kuhnle A, Sturm R, et al (2018) Reinforcement learning for adaptive order dispatching in the semiconductor industry. CIRP Annals 67(1):511 – 514

    2018

  48. [58]

    Adap- tive Computation and Machine Learning Series, Massachusetts Institute of Technology Press, Cambridge, Massachusetts

    Sutton RS, Barto AG (1998) Reinforcement Learning: An Introduction. Adap- tive Computation and Machine Learning Series, Massachusetts Institute of Technology Press, Cambridge, Massachusetts

    1998

  49. [59]

    Eur J Oper Res 64(2):278–285

    Taillard E (1993) Benchmarks for basic scheduling problems. Eur J Oper Res 64(2):278–285. https://doi.org/10.1016/0377-2217(93)90182-M

    1993 doi

  50. [60]

    CoRR abs/2104.03760

    Tassel P, Gebser M, Schekotihin K (2021) A reinforcement learning environment for job-shop scheduling. CoRR abs/2104.03760

    2021 arXiv

  51. [61]

    Tassel P, Kov´ acs B, Gebser M, et al (2023) Semiconductor fab scheduling with self-supervised and reinforcement learning. In: WSC. IEEE, pp 1924–1935, https: //doi.org/10.1109/WSC60868.2023.10407747

    2023

  52. [62]

    In: NIPS, pp 5998–6008

    Vaswani A, Shazeer N, Parmar N, et al (2017) Attention is all you need. In: NIPS, pp 5998–6008

    2017

  53. [63]

    In: Proceedings of International Conference on Computers and Industrial Engineering, CIE

    Wang J, He J, Zhang J (2018) A reinforcement learning method to optimize the priority of product for scheduling the large-scale complex manufacturing sys- tems. In: Proceedings of International Conference on Computers and Industrial Engineering, CIE

    2018

  54. [64]

    Int J Prod Res 63(8):2871–2888

    Wang M, Zhang J, Zhang P, et al (2025) Cooperative multi-agent reinforce- ment learning for multi-area integrated scheduling in wafer fabs. Int J Prod Res 63(8):2871–2888. https://doi.org/10.1080/00207543.2024.2411615

    2025

  55. [65]

    In: ASMC

    Waschneck B, Reichstaller A, Belzner L, et al (2018) Deep reinforcement learning for semiconductor production scheduling. In: ASMC. IEEE, pp 301–306, https: //doi.org/10.1109/ASMC.2018.8373191

    2018

  56. [66]

    Procedia CIRP 72:1264–

    Waschneck B, Reichstaller A, Belzner L, et al (2018) Optimization of global production scheduling with deep reinforcement learning. Procedia CIRP 72:1264–

    2018

  57. [67]

    MIS Q 26(2)

    Webster J, Watson RT (2002) Analyzing the past to prepare for the future: Writing a literature review. MIS Q 26(2)

    2002

  58. [68]

    Expert Syst Appl 35(1-2):485–496

    Wu J, Chien C (2008) Modeling semiconductor testing job scheduling and dynamic testing machine configuration. Expert Syst Appl 35(1-2):485–496. https: //doi.org/10.1016/j.eswa.2007.07.026

    2008 doi

  59. [69]

    J Intell Manuf 31 23(6):2255–2270

    Wu J, Hao X, Chien C, et al (2012) A novel bi-vector encoding genetic algo- rithm for the simultaneous multiple resources scheduling problem. J Intell Manuf 31 23(6):2255–2270. https://doi.org/10.1007/S10845-011-0570-0

    2012 doi

  60. [70]

    https://doi.org/10.1109/ NABIC.2009.5393690

    Yang X, Deb S (2009) Cuckoo search via l´ evy flights. https://doi.org/10.1109/ NABIC.2009.5393690

    2009

  61. [71]

    Yedidsion H, Dawadi P, Norman D, et al (2022) Deep reinforcement learning for queue-time management in semiconductor manufacturing. In: WSC. IEEE, pp 3275–3284, https://doi.org/10.1109/WSC57314.2022.10015463

    2022

  62. [72]

    In: NeurIPS, pp 11960–11970

    Yun S, Jeong M, Kim R, et al (2019) Graph transformer networks. In: NeurIPS, pp 11960–11970

    2019

  63. [73]

    Comput Ind Eng 193:110259

    Zhang L, Lin Y, Xu C, et al (2024) A new EDA algorithm combined with q- learning for semiconductor final testing scheduling problem. Comput Ind Eng 193:110259. https://doi.org/10.1016/J.CIE.2024.110259 32

    2024

  64. [531]

    https://doi.org/10.1109/TSM.2020.3001933

    2020

  65. [1269]

    https://doi.org/10.1016/j.procir.2018.03.212

    2018 doi

  66. [6675]

    https://doi.org/10.1109/TCYB.2022.3210228

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.