Pith. sign in

REVIEW 5 major objections 5 minor 23 references

Scalable HPC Job Scheduling and Resource Management in SST

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An SST-based simulator accurately reproduces HPC job wait times and node usage, and scales in parallel, the paper claims.

desk verdict A useful SST scheduling element whose accuracy validation may be circular, since the text says trace data determine job start and end times. read the letter →

arxiv 2501.18191 v1 pith:L7B55XRU submitted 2025-01-30 cs.DC

Abubeker Abdurahman , Abrar Hossain , Kevin A Brown , Kazutomo Yoshii , Kishwar Ahmed This is my paper · ORCID
classification cs.DC
keywords HPCjobschedulingdiscrete-eventsimulationSSTworkflowmanagementresourceallocationparallelbackfillingworkloadtraces
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a job scheduling and resource management component built inside the structural simulation toolkit (SST), a parallel discrete-event simulator. It claims that this simulator reproduces real HPC job behavior--specifically job wait times and node occupancy--when replaying a large workload trace, and that it scales well as more MPI ranks are added. The simulator supports five scheduling policies (FCFS, shortest job first, longest job first, FCFS with best fit, and FCFS with backfilling) and adds a workflow layer that models task dependencies as a directed acyclic graph. The significance would be a single simulation framework where HPC architects can test both scheduling policies and workflow dependencies at scale without disturbing a production system.

What carries the argument

The structural simulation toolkit (SST) is a modular, parallel discrete-event simulator that uses conservative synchronization; the paper's component plugs scheduling decisions into this engine as event handlers. Jobs arrive as serialized TaskEvent objects, are placed into a priority queue, and trigger resource allocation and deallocation as simulation time advances. The workflow component stores tasks and their interdependencies as a directed acyclic graph using adjacency lists, so that a task becomes eligible only when all of its dependency tasks have completed. This combination--event-driven resource accounting plus DAG-based dependency tracking--is what the paper relies on to produce accurate wait times, node usage, and workflow behavior.

What would settle it

An immediate check is to feed the same trace to two different scheduling policies in the simulator and see whether the reported wait times change. If the simulator uses the trace's recorded start times as fixed inputs, every policy would produce identical wait times, and the validation against the trace's wait times would be tautological. A reader could compare a subset of individually reported job wait times from the simulator against the trace's wait times for those exact jobs, and verify whether the simulator's wait times equal the trace's wait times to the tick.

Watch

Extended reading notes

Core claim

The central claim is that job scheduling and resource management for HPC systems can be modeled faithfully inside SST's event-driven parallel engine. The authors report that their component's node-occupancy curve and job wait times closely match a reference cluster scheduling simulator when both replay the same real-world workload trace, and that parallel speedup grows with the number of MPI ranks on two large traces (about 1.1 million jobs and about 73,000 jobs). They also report that a workflow management component, which represents dependencies among tasks as a DAG and triggers tasks when their predecessors complete, reproduces the wait-time behavior of a real bioinformatics workflow. The paper frames this as validation of both accuracy and scalability in one framework.

Load-bearing premise

The load-bearing premise is that the simulator's wait-time validation is meaningful, which requires that the scheduling policy, not the trace file, generates job start times; the paper says it uses the trace to determine start and end times, so this premise is not settled.

Editorial extensions

If this is right

  • HPC scheduling policies can be exercised and compared within a parallel discrete-event engine that also models network and memory effects.
  • Million-job workload traces can be replayed at scale, since the simulator's runtime improves as MPI ranks increase.
  • Workflow-driven workloads can be simulated with dependency constraints enforced, allowing evaluation of scientific pipelines before deployment.
  • The five implemented policies provide a basis for controlled head-to-head experiments on identical traces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stronger version of this validation would feed only arrival times and resource requests to the simulator and let each policy compute start times; the current setup's reliance on trace start and end times leaves open whether the wait-time match reflects the scheduler or the trace.
  • The same component could be extended to test policies the trace never used, such as replacing first-come-first-served with backfilling on a repeat of the same workload, which would show whether the simulator can predict the effect of a policy change.
  • The workflow DAG representation could be combined with the network and memory models already in SST to study how data movement between dependent tasks affects overall workflow makespan, a direction the paper does not pursue.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces a job scheduling and resource management component for the Structural Simulation Toolkit (SST), a parallel discrete-event simulator. The component supports five job scheduling algorithms (FCFS, FCFS with Best Fit, FCFS with Backfilling, SJF, LJF) and a workflow management module for DAG-based task dependencies. Validation is reported against CQSim using the GWA-DAS2 trace and against a SIPHT workflow, and MPI scalability results are presented for the DAS-2, SDSC-SP2, and Galactic workflow workloads.

Significance. If its claims were established, the component would be a useful extension of SST, addressing the need for scalable and workflow-capable HPC scheduling simulation. The paper has concrete strengths: a modular design integrated into a known simulator, a clearly described task-event interface (Listing 1), a workflow input format with a JSON example, and an effort to test on real workload traces. However, the current evidence is not sufficient: the accuracy validation is at risk of circularity because Section 4.1 says trace data determine job start and end times, and the reported matches in Figures 3 and 4 are purely visual with no quantitative error measures. The parallel performance claim also lacks concrete speedup or efficiency numbers.

major comments (5)
  1. [Section 4.1] The sentence "We use the trace data to determine the start time and end time of jobs in our simulator" is load-bearing for the accuracy claim. If taken literally, the simulator consumes the very quantities it is supposed to predict: job start and end times come from the DAS-2 trace rather than from the scheduling policy. In that case, the node-occupancy match in Figure 3(a) and the wait-time match in Figure 4(a) are not independent validation but a restatement of the trace. The paper must clarify the intended input semantics: either the scheduler actually generates start times and only execution durations and resource demands are taken from the trace, or the validation is circular. Listing 1's TaskEvent has executionTime but no field for trace-supplied start/end timestamps, so the intended semantics are not documented.
  2. [Section 4.2, Figures 3 and 4] The accuracy claim is supported only by visual curve inspection: the text says the occupancy and wait-time outputs are "similar" and "closely matches" CQSim, but no quantitative error metric (e.g., mean absolute error, RMSE, relative error, correlation) is reported. Please add numerical error measures for wait time and node occupancy, and state which scheduling algorithm (FCFS, backfilling, etc.) was used for the validation runs and all relevant configuration parameters so the comparison is reproducible.
  3. [Section 4.1 vs Section 4.2] There is a direct inconsistency about the workflow experiments. The last paragraph of Section 4.1 says "we plan to extend our testing to encompass real-life workflow simulations using Pegasus," yet Section 4.2 reports results for the Galactic workflow from the Pegasus gallery (Figure 6) and a validation against real-life SIPHT measurements (Figure 7). Please correct the status description. In addition, Figure 7's validation is not described: the ground-truth source, the compared metric (per-task wait time? makespan?), the workflow configuration, and the error measure are all missing, so the workflow validation claim is currently unsupported.
  4. [Section 2.2, Algorithm 1] Algorithm 1 describes a simple immediate-allocation loop that schedules a job only when enough cores are available, which corresponds to FCFS without backfilling or best-fit. The paper claims support for five scheduling algorithms, but the pseudocode does not specify how backfilling, best-fit, SJF, or LJF are implemented inside the event-driven framework, nor how the priority queue is ordered differently per policy. Without this specification, the paper does not establish that the five algorithms are actually simulated. Please provide the per-policy queue ordering and event rules, and indicate which algorithm produced each reported result.
  5. [Section 4.2, Figures 5 and 6] The parallel performance claim is not quantified. Figures 5 and 6 appear to lack labeled axes, and the text gives no speedup, parallel-efficiency, or wall-clock-time values, no hardware/SST configuration (number of ranks, host machine, synchronization settings), and no comparison against ideal scaling or against other simulators. Please report concrete timings and speedups at the rank counts shown, and add axis labels to the figures.
minor comments (5)
  1. [Abstract] The phrase "introduces a workflow management components" should be "introduces a workflow management component."
  2. [Section 4.1] The sentences about the 4seq, 5seq, and 6seq epigenomic datasets mention that these are analyzed, but no result or plot involving them appears anywhere in the paper; either report the corresponding analysis or remove the mention.
  3. [References] The in-text citation "Simulator. 2024" in Section 2.2 does not match the reference list entry "SST Simulator. 2024"; please unify the citation format. Also, the reference for Mitchell et al. (2019) is listed with the title "Alea 2: Job Scheduling Simulator," which appears to be a copy-paste error for a paper about Apache Airflow; please correct it.
  4. [Section 4.2, Figure 4(b)] The wait-time comparison across the five scheduling algorithms would be more informative with quantitative summary statistics (mean and median wait time, or a table), and with a statement of the workload and resource configuration used; currently the figure is described only qualitatively.
  5. [Section 3.1] The task representation section lists fields such as resource_requirements and dependencies, but it does not state whether these fields are stored in the same TaskEvent class as in Listing 1 or in a separate class; please clarify the relationship between the job-level TaskEvent and the workflow-level Task representation.

Circularity Check

1 steps flagged · score 6.0 of 10

Accuracy validation may be circular: trace start/end times are used as simulator inputs, making wait-time and occupancy matches to DAS-2/CQSim predetermined by construction.

  1. self definitional [Section 4.1 (Experiment Setup) and Section 4.2 (Results, Figure 4(a))]
    "We use the trace data to determine the start time and end time of jobs in our simulator."

    If the trace's start and end times are loaded into the simulator as inputs, then the quantities used for validation are defined by those same inputs: job wait time equals trace start time minus submission time, and node occupancy is the binning of trace start/end intervals. The close match reported in Figure 4(a) against DAS-2 and CQSim would therefore follow by construction rather than from the scheduling algorithms. The paper does not clarify an alternative reading, e.g., that only execution durations or submission times are taken from the trace while start times are generated by the simulator, and Listing 1/Algorithm 1 do not resolve the ambiguity. As written, the central accuracy claim reduces to replaying the trace.

full rationale

The paper's central accuracy claim is that the SST-based scheduler produces job wait times and node occupancy that match DAS-2 and CQSim. The quoted sentence in Section 4.1 says trace data determine job start and end times in the simulator. Under a literal reading, the simulator consumes the exact quantities it is supposed to predict, so the wait-time and occupancy matches are tautological. The paper also validates against CQSim, another simulator running the same trace, which tests inter-simulator consistency rather than prediction against ground truth; this is not itself circular but does not resolve the input-semantics problem. The scalability results (Figures 5 and 6) are independent of this issue and appear self-contained, and there are no load-bearing self-citations or imported uniqueness theorems. However, because the primary accuracy validation is compromised by the stated trace-input semantics, the overall circularity score is 6: one or more central predictions reduce by construction unless the authors clarify that start times are actually scheduled rather than read from the trace.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on domain assumptions about workload traces and scheduling policy fidelity, not on free parameters or fitted constants. No unique model constants are tuned; the main risk is that the validation conditions (such as using trace start/end times and comparing against CQsim) do not independently confirm accuracy.

assumptions (4)
  • domain assumption The five standard scheduling policies (FCFS, SJF, LJF, FCFS+best-fit, FCFS+backfilling) faithfully model real HPC scheduler behavior.
    The validation compares against CQsim and traces, but no real HPC scheduler logs are used to verify that these policies produce realistic wait times.
  • domain assumption The DAS-2 and SDSC-SP2 workload traces are representative of modern HPC workloads.
    These historical traces from the early 2000s may not reflect current HPC job mixes, yet the paper treats them as sufficient for validation.
  • domain assumption Simulating a workflow as a DAG with simple dependency checks captures the dynamics of real scientific workflows.
    Only one simplified workflow (SIPHT/Galactic) is tested; complex data and failure behaviors are ignored.
  • domain assumption The comparison against CQsim assumes CQsim itself produces correct scheduling results.
    No independent real-system validation of the scheduling logic is performed in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scalable HPC Job Scheduling and Resource Management in SST." pith.science (2026). https://pith.science/paper/L7B55XRU

@misc{pith2026250118191,
  author       = {Pith},
  title        = {Pith review of: Scalable HPC Job Scheduling and Resource Management in SST},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L7B55XRU}},
  note         = {Machine review of arXiv:2501.18191}
}
read the original abstract

Efficient job scheduling and resource management contribute towards system throughput and efficiency maximization in high-performance computing (HPC) systems. In this paper, we introduce a scalable job scheduling and resource management component within the structural simulation toolkit (SST), a cycle-accurate and parallel discrete-event simulator. Our proposed simulator includes state-of-the-art job scheduling algorithms and resource management techniques. Additionally, it introduces workflow management components that support the simulation of task dependencies and resource allocations, crucial for workflows typical in scientific computing and data-intensive applications. We present the validation and scalability results of our job scheduling simulator. Simulation shows that our simulator achieves good accuracy in various metrics (e.g., job wait times, number of nodes usage) and also achieves good parallel performance.

Figures

Figures reproduced from arXiv: 2501.18191 by the authors.

Figure 1
Figure 1. The simulator components. queueing and scheduling. Serialization within the TaskEvent class ensures accurate transfer of task data across SST components. We show the TaskEvent class below. 1 TaskEvent(const std::string& id, int cores, double time, size_t memory) : 2 Event(), jobID(id), requiredCores(cores), executionTime(time), requiredMemory( memory), state(QUEUED) {} 3 4 void serialize_order(SST::Core::Serializati… view at source ↗
Figure 2
Figure 2. The workflow management components. 1 { 2 "tasks": [ 3 {"id": 1, "execution_time": 100, "resources": {"cpu": 2, "memory": 1024}, "dependencies": []}, 4 {"id": 2, "execution_time": 150, "resources": {"cpu": 1, "memory": 512}, " dependencies": [1]}, 5 {"id": 3, "execution_time": 200, "resources": {"cpu": 1, "memory": 512}, " dependencies": [1]}, 6 {"id": 4, "execution_time": 300, "resources": {"cpu": 2, "memory": 1024… view at source ↗
Figure 3
Figure 3. Comparison of our simulation output with CQSim. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Job wait time validation and comparison of different scheduling algorithms. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Scalability of our proposed simulator [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Scalability of workflow simulation. (Pegasus. 2024). As can be seen in the figures, simulator performance scales well as the number of MPI ranks increases [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Job wait time validation of workflow simulation. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 21 canonical work pages

  1. [1]

    Buyya, R. and M. Murshed. 2002. ``Gridsim: A Toolkit for The Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing''. Concurrency and Computation: Practice and Experience\/ 14(13-15):1175--1220

    work page 2002

  2. [2]

    Deelman, E., K. Vahi, G. Juve, M. Rynge, S. Callaghan, P. J. Maechling et al . 2015. ``Pegasus, A Workflow Management System for Science Automation''. Future Generation Computer Systems\/ 46:17--35

    work page 2015

  3. [3]

    Fan, Y., B. Li, D. Favorite, N. Singh, T. Childers, P. Rich et al . 2022. ``DRAS: Deep Reinforcement Learning for Cluster Scheduling in High Performance Computing''. IEEE Transactions on Parallel and Distributed Systems\/ 33(12):4903--4917

    work page 2022

  4. [4]

    Choudhary, and P

    Gupta, I., A. Choudhary, and P. K. Jana. 2017, 24-26 November. ``Generation and Proliferation of Random Directed Acyclic Graphs for Workflow Scheduling Problem''. In Proceedings of the 7th International Conference on Computer and Communication Technology , 123–127. Allahabad, India

    work page 2017

  5. [5]

    Iosup, A., H. Li, M. Jan, S. Anoep, C. Dumitrescu, L. Wolters et al . 2008. ``The Grid Workloads Archive''. Future Generation Computer Systems\/ 24(7):672--686

    work page 2008

  6. [6]

    Juve, G. 2014. ``Pegasus Workflow Gallery - SIPHT Workflow''. https://pegasus.isi.edu/workflow_gallery/gallery/sipht/index.php. Accessed on: 21 st April 2024

    work page 2014

  7. [7]

    Chervenak, E

    Juve, G., A. Chervenak, E. Deelman, S. Bharathi, G. Mehta and K. Vahi. 2013. ``Characterizing and Profiling Scientific Workflows''. Future Generation Computer Systems\/ 29(3):682--692

    work page 2013

  8. [8]

    Klus\' a c ek, D. and H. Rudov\' a . 2010. ``Alea 2: Job Scheduling Simulator''. In Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques . March 15 th -19 th , Malaga, Spain, 1–10

    work page 2010

Show all 23 references
  1. [9]

    Pottier, S

    Mitchell, R., L. Pottier, S. Jacobs, R. F. da Silva, M. Rynge, K. Vahi et al . 2019. ``Alea 2: Job Scheduling Simulator''. In 2019 IEEE International Conference on Big Data (Big Data) . December 9 th -12 th , Los Angeles, CA, USA, 4537--4544

    2019

  2. [10]

    Pegasus. 2024. ``Workflow Gallery''. https://pegasus.isi.edu/workflow_gallery/. Accessed: 12 th April 2024

    2024

  3. [11]

    Hemmert, B

    Rodrigues, A., K. Hemmert, B. Barrett, C. Kersey, R. Oldfield, M. Weston et al . 2011. ``The Structural Simulation Toolkit''. ACM SIGMETRICS Performance Evaluation Review\/ 38(4):37--42

    2011

  4. [12]

    San Diego Supercomputer Center. 2000a. ``Parallel Workloads Archive''. https://www.cs.huji.ac.il/labs/parallel/workload/. Accessed: 13 th October 2023

    2023

  5. [13]

    San Diego Supercomputer Center. 2000b. `` SDSC SP2 Workload Log ''. https://www.cs.huji.ac.il/labs/parallel/workload/l_sdsc_sp2/. Accessed: 13 th October 2023

    2023

  6. [14]

    Simakov, N. A., M. D. Innus, M. D. Jones, R. L. DeLeon, J. P. White, S. M. Gallo et al . 2017. ``A Slurm Simulator: Implementation and Parametric Analysis''. In 8th International Workshop on Performance Modeling, Benchmarking, and Simulation (PMBS) . November 13 th , Denver, C...

    2017

  7. [15]

    SST Simulator. 2024. ``SST Elements Repository''. https://github.com/sstsimulator/sst-elements.git. Accessed: 7 th February 2023

    2024

  8. [16]

    Smith, J. E. 1978. ``A New Class of Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment''. Journal of the ACM\/ 25(3):455--470 https://doi.org/10.1145/322063.322079

    1978

  9. [17]

    Soysal, M. and A. Streit. 2021. ``Collection of Job Scheduling Prediction Methods''. In 24th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP) . May 21 st , Virtual Event, 35--42

    2021

  10. [18]

    SPEAR Lab. 2023. ``CQSim - A Discrete-Event Driven Scheduling Simulator''. https://github.com/SPEAR-UIC/CQSim. Accessed: 7 th February 2024

    2023

  11. [19]

    Tang, W., Z. Lan, N. Desai, and D. Buettner. 2009. ``Fault-Aware, Utility-Based Job Scheduling on Blue, Gene/p Systems''. In 2009 IEEE International Conference on Cluster Computing and Workshops . August 31 st - September 4 th , New Orleans, LA, USA, 1--10

    2009

  12. [20]

    Yang, X., Z. Zhou, S. Wallace, Z. Lan, W. Tang, S. Coghlan et al . 2013. ``Integrating Dynamic Pricing of Electricity into Energy Aware Scheduling for HPC Systems''. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis ...

    2013

  13. [21]

    Yoo, A. B., M. A. Jette, and M. Grondona. 2003. ``Slurm: Simple Linux Utility for Resource Management''. In 9th International Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP) . June 24 th , Seattle, WA, USA, 44--60

    2003

  14. [22]

    write newline

    " write newline "" before.all 'output.state := FUNCTION article output.bibitem format.authors "author" output.check author format.key output output.year.check new.block format.title "title" output.check new.block crossref missing format.jour.vol output format.article.crossref ...

  15. [23]

    write newline

    " write newline "" before.all 'output.state := FUNCTION article output.bibitem format.authors "author" output.check author format.key output output.year.check new.block format.title "title" output.check new.block crossref missing format.jour.vol output format.article.crossref ...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.