{"id":"8b3349b5-e6ae-4929-943c-56c055040efc","arxiv_id":"1908.06714","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Kernel-ridge models on molecular fingerprints predict quantum-chemistry job runtimes well enough to improve simulated HPC scheduling and reduce CPU overhead by 10 to 90 percent.","lead":"This paper trains machine-learning models to predict how long common quantum-chemistry calculations will take, based only on a molecule's geometry and composition. In simulations, using these predictions to pack jobs onto compute clusters cuts wasted CPU time by 10 to 90 percent.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Scheduling gains rest on an idealized simulator with fixed queue delays and no preemption; if real cluster dynamics differ, the 10–90% overhead reductions may not transfer.","rationale":"The reader identified the scheduling simulator as the weakest assumption, and I agree. The QML learning result itself is supported by out-of-sample learning curves and by the FLOP-count control experiment, which is a genuine independent check. However, the central practical claim of 10–90% overhead reductions depends entirely on the simulator's idealized job model and on excluding I/O-contaminated timings. The manuscript explicitly discloses these assumptions, which is why this is a conditionality concern rather than a soundness flaw. The suggested trace-driven simulation is a concrete, feasible check that would determine whether the scheduling gains survive realistic queue dynamics, preemption, and unfiltered I/O interference. If the check passes, the conditional verdict can be upgraded; if it fails, the scheduling claim must be downgraded to a simulation-specific result. Since the reader already assigned CONDITIONAL, my read does not change that verdict.","tokens_in":22879,"tokens_out":5009,"duration_ms":58934,"concrete_test":"Replace the fixed assumptions in Section 3.2.2 with a trace-driven simulation using actual scheduler logs (queue-delay distributions, node availability, preemption/restart events) from sciCORE and/or CSCS, and do not exclude high wall-time-overhead runs. Recompute the Figure 9 CPU-time overhead for FFD/QML scheduling versus job arrays and job steps. If the median overhead reduction drops below 10% or changes sign for any of the seven tasks, the scheduling claim should be restated as environment-specific rather than general.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline 10–90% CPU-time overhead reductions are not measured on a cluster; they are outputs of the job scheduling simulator described in Section 3.2.2. That simulator assumes idempotent uninterruptible tasks, a fixed 30-second startup cost, and a fixed one-hour queue delay for every job. Additionally, the data pipeline in Section 4.2.4 (and Figure 3) removes runs whose wall-time/CPU-time ratio exceeds task-specific thresholds up to 50%, explicitly to discard timings affected by heavy I/O. Real clusters have stochastic queue delays, preemption or restart events, and I/O contention that cannot be filtered away at scheduling time. If those effects cause idle reserved cores or wasted restarts, the FFD/QML packing advantage can shrink or even reverse. The QML wall-time accuracy results are not at issue; the unsupported part is the translation from normalized MAEs of roughly 2–40% to a demonstrated 10–90% overhead reduction in production. The conclusion's phrase 'demonstrably used compute clusters more efficiently' overstates what a simulator shows, since no real scheduling experiment was run.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces kernel ridge regression (KRR) models of computational cost for quantum chemistry, predicting wall times, CPU times, and FLOP counts for seven tasks spanning single-point, geometry-optimization, and transition-state calculations, plus a 2D toy optimization problem. Molecular inputs are represented with Bag of Bonds and FCHL fingerprints, and the authors report learning curves with normalized mean absolute errors, reaching roughly 1.3–48% MAE at the largest training sizes. The predicted absolute wall times are then fed into a first-fit-decreasing job scheduling simulator, and the paper claims reductions in CPU-time overhead of 10–90% relative to conventional job-array and job-step packing. The manuscript makes code, data, and a GitHub repository available.","tokens_in":23139,"tokens_out":4853,"duration_ms":47783,"significance":"If the scheduling claims survive closer scrutiny, this is a useful and novel application of QML to a practical HPC problem, and one of the first demonstrations that computational cost can be treated as a learnable molecular quasi-property across chemical space. The paper's strengths include systematically decaying learning curves, a broad set of tasks and levels of theory, the use of FLOP counts as a cleaner target, and openly available timings and scripts. The main weakness is that the headline efficiency gains are outputs of an idealized simulator, coupled with data-cleaning choices and test-set model selection that may bias the reported gains. The QML accuracy results themselves are honest and reproducible, but the translation from normalized MAEs to production scheduling savings is not yet fully supported.","major_comments":[{"comment":"The headline 10–90% CPU-time overhead reductions are simulator outputs, not cluster measurements. The simulator assumes idempotent uninterruptible tasks, a fixed 30-second startup cost, and a fixed one-hour queue for every job, and it does not model preemption, stochastic queue delays, or I/O contention. The conclusion that the authors 'demonstrably used compute clusters more efficiently' therefore overstates what the evidence shows. Please validate the scheduling claims on a real scheduler trace, or at minimum add a sensitivity analysis over queue-delay distributions, startup costs, and preemption behavior, and revise the wording to 'simulated' rather than 'demonstrably'.","section":"§3.2.2 and §4.3.1, Figs. 9–10"},{"comment":"Runs with wall-time-to-CPU-time ratios above task-specific thresholds of 3%, 5%, 10%, 30%, and 50% are excluded as 'heavy I/O' before training and evaluation. At scheduling time this ratio is not known before a job executes, so a production scheduler cannot apply the same filter. As a result, the simulator evaluates the QML scheduler on a cleaned distribution that is not available in practice. Please report how many jobs are excluded per task, and show that the learning curves and overhead reductions survive when the excluded jobs are kept or otherwise modeled, since the reported 10–90% range may be optimistically biased.","section":"§4.2.4 and Fig. 3"},{"comment":"The representation used for scheduling is selected as the one with the lowest MAE at the maximum training size, where those MAEs are computed on the held-out test set. This constitutes test-set model selection, which inflates the reported generalization accuracy and, consequently, the downstream scheduling gains. Please replace this procedure with nested cross-validation or a pre-registered representation choice, or at least report the scheduling results for both BoB and FCHL models so that the sensitivity to this selection is visible.","section":"§4.3.1 and Table 2"}],"minor_comments":[{"comment":"The abstract says 'three commonly used optimizer' and the body contains several typos, including 'fucntions' and 'obtianed'; please correct them.","section":"Abstract and text"},{"comment":"The task label 'QM9SP_CC/DT' is a typo for 'QM9SP_CC/TZ'.","section":"§4.3.1"},{"comment":"The caption refers to task 'QM9GO_CASSCF', but the corresponding task in Table 1 is 'QMspinGO_CASSCF'.","section":"Fig. 10 caption"},{"comment":"The reference to the Argonne Leadership Computing Facility is misspelled as 'Argone'; please fix it.","section":"Reference 3"},{"comment":"The lower-panel discussion in Figure 9 contains a repeated sentence; one of the two occurrences should be removed.","section":"§4.3.1"},{"comment":"The figure caption uses 'FLOPS' while the text and Table 2 refer to 'FLOP count'; please use consistent terminology.","section":"Fig. 8"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope and the core QML learning result is solid and worth publishing after revision. The main risk is the scheduling claim, which rests on an idealized simulator and data-cleaning steps that are not transferable to production clusters. If the authors add a real trace validation or, failing that, clearly restrict the claims to the simulated setting, I would support publication. I do not see grounds for rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper: the core learning result is real. The authors treat wall time as a quasi-property, use kernel ridge regression with BoB and FCHL representations on seven distinct quantum-chemistry tasks (single points, geometry optimizations, transition states, at theory levels from B3LYP to MRCI+Q), and show out-of-sample errors that decay systematically with training set size, down to 1.3–40% at the largest N. That is genuinely new in the QML literature, and the data generation alone is substantial: thousands of timings across Molpro and ORCA, plus a FLOP-count control experiment that gives a cleaner target than wall time and confirms the learnability.\n\nWhat the paper does well: it frames computational cost as a molecular quasi-property and shows that established representations capture it. The toy-system study is a useful sanity check; the kernel PCA analysis gives a plausible reason why FCHL beats BoB; and the code and most data are shipped, which helps others build on it.\n\nWhere it gets soft: the scheduling half. The headline 10–90% CPU-time overhead reductions are not measured on a cluster. They are outputs of a simulator that assumes idempotent, uninterruptible tasks, a fixed 30-second startup, and a one-hour queue delay for every job (Section 3.2.2). Real clusters have preemption, variable queue delays, and I/O contention that cannot be filtered away at scheduling time. The authors also drop runs with wall/CPU ratios above task-specific thresholds up to 50% (Section 4.2.4), which makes sense for building a clean learning target but means the scheduling experiments run on a sanitized workload. So the conclusion's 'demonstrably used compute clusters more efficiently' is too strong; what they demonstrated is that a simulator with those assumptions produces better packing.\n\nSmaller issues: no error bars on the learning curves or overhead reductions; the choice of FCHL as the scheduling representation is made after seeing test-set errors (mild selection); and the carbene data set is only 'available upon request,' which is a reproducibility wrinkle. None of these sink the core result.\n\nBottom line: the learnability claim holds up. The scheduling gains are plausible but not yet proven in production. This paper deserves a serious referee who will push for a sensitivity analysis or a real scheduler trace. I would accept it for peer review, and I would cite it for the wall-time learning result, not for the efficiency numbers.","headline":"A genuinely new and mostly solid demonstration that quantum-chemistry wall times are learnable from molecular structure; the scheduling-efficiency headline rests on a simulator with idealized assumptions and is softer than the abstract suggests.","tokens_in":23623,"tokens_out":2624,"would_cite":true,"duration_ms":26435,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Quantum machine learning models can learn how long a quantum chemistry calculation will run from molecular structure alone, and packing jobs with those predictions cuts CPU-time overhead by 10 to 90 percent.","keywords":["quantum machine learning","job scheduling","wall time prediction","kernel ridge regression","computational cost modeling","quantum chemistry","first-fit decreasing","high-performance computing"],"falsifier":"Run the published seven task sets on a production cluster that preempts jobs and has load-dependent queue delays, then compare CPU overhead when jobs are ordered by the learned run-time estimates against the conventional job-array approach; if overhead does not drop, the scheduling claim fails. A second check is whether the FLOP-count model, presented as a noise-free cost measure, transfers to a different CPU architecture within the same error bounds.","tokens_in":22677,"feed_emoji":"⏱️","tokens_out":11588,"duration_ms":99399,"temperature":0.7,"pith_summary":"The paper tries to establish that the computational cost of routine quantum chemistry jobs is a learnable, smoothly varying function of molecular structure. It trains kernel ridge regression models on wall-time labels for single-point, geometry-optimization, and transition-state calculations across thousands of organic molecules, and shows that out-of-sample prediction error falls systematically as the training set grows. If true, this matters because high-performance computing clusters currently treat all jobs of one type as identical, wasting CPU time and energy; accurate per-molecule run-time estimates turn scheduling into a bin-packing problem. The authors report that replacing indiscriminate scheduling with estimates from these models cuts CPU-time overhead by 10 to 90 percent across seven task types after training on only thousands of molecules.","feed_headline":"ML predicts quantum-chemistry run times and cuts job overhead 10-90%","feed_subtitle":"Trained on thousands of molecules, the models learn job costs from structure alone.","key_machinery":"The engine is kernel ridge regression applied to molecular representations: wall time is normalized by electron count, described by either a two-body bond-count representation or a one-, two-, and three-body atomic descriptor, and learned with a Laplacian or Gaussian kernel. The learned absolute run-time estimates are then sorted in decreasing order and packed into compute jobs with a first-fit-decreasing heuristic, so idle time left by long jobs is filled by shorter ones. A secondary mechanism is step splitting for geometry optimizations: because the first optimization step costs about 20 percent more than later steps, modeling it separately improves both the timing model and the scheduler.","core_discovery":"The central claim is that wall time is a molecular quasi-property: from the initial geometry and composition of a molecule, a kernel machine can predict how long a specified electronic-structure calculation will run, accurately enough to improve scheduling. Evidence is given for seven tasks spanning single-point calculations, geometry optimizations, and transition-state searches, with five levels of electronic-structure theory and thousands of closed- and open-shell organic molecules. At the largest training sizes, normalized mean absolute errors range from about 1.3 percent for single-point coupled-cluster timings to roughly 27 to 48 percent for geometry optimizations, and the learned timings, fed into a first-fit-decreasing scheduler, reduce CPU overhead by 10 to 90 percent relative to job arrays or job steps.","pith_inferences":["The same learned-cost view could extend to other expensive workflows, such as periodic solid-state calculations or ab initio molecular dynamics, where runtime also depends on structure and on the path taken.","Predicted run times could be used online to detect abnormal jobs: a large gap between predicted and observed duration would flag I/O contention, convergence failure, or a misconfigured input.","If runtime is truly a molecular quasi-property, cost models could guide generative molecular design by preferring cheaper candidate molecules, not just by scheduling a fixed campaign.","The reported gains depend on a simulator with fixed startup and queue delays; on real clusters with load-dependent queuing the savings could be larger if backfilling becomes easier, or smaller if preemption breaks the packing."],"forward_implications":["Single-point, geometry-optimization, and transition-state wall times are all learnable from structure alone; the optimization tasks are harder but still accurate enough for scheduling.","Training sets of a few thousand molecules suffice to beat the baseline assumption that every calculation of a given type takes the mean run time.","Accurate run-time estimates let clusters pack jobs in decreasing order, roughly halving overhead at large core counts and allowing much larger parallel campaigns at the same overhead.","For geometry optimizations, predicting individual step durations lets a scheduler release resources early instead of starting a step that cannot finish, cutting overhead by up to 90 percent at small wall-time limits."],"supporting_citations":[{"why":"It supplies the QM9 molecular structures and properties that form the basis for several timing data sets.","marker":"41"},{"why":"It establishes kernel ridge regression with the Coulomb matrix, the learning approach adapted here.","marker":"28"},{"why":"It provides the Bag of Bonds representation used as one of the two molecular descriptors.","marker":"45"},{"why":"It provides the FCHL atomic descriptor used as the second, generally better-performing representation.","marker":"50"},{"why":"It defines the local coupled-cluster method whose implementations generate the coupled-cluster timing tasks.","marker":"65"},{"why":"It is the electronic-structure package used to record CPU and wall timings for five of the seven tasks.","marker":"84"},{"why":"It is the electronic-structure package used to obtain wall times for the MP2 geometry-optimization and transition-state tasks.","marker":"85"},{"why":"It provides the learning-curve power law used to analyze how prediction error decays with training-set size.","marker":"90"},{"why":"It is the software toolkit used to build and evaluate the kernel ridge regression models.","marker":"93"}],"fun_headline_variants":["Quantum ML predicts compute time, cuts job overhead up to 90%","Trained on molecules, ML slashes quantum-chemistry job costs","Machine learning forecasts quantum chemistry run times","Quantum chemistry timings predicted by ML, saving CPU hours","ML cuts quantum-job overhead 10-90% by forecasting wall times"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported savings assume that once a calculation starts it runs to completion without interruption, that starting a new job costs 30 seconds, and that every job waits exactly one hour in the queue; clusters with preemption, variable waiting times, or busy input-output systems may not see those savings.","fun_headline_variants_meta":{"raw":{"variants":["Quantum ML predicts compute time, cuts job overhead up to 90%","Trained on molecules, ML slashes quantum-chemistry job costs","Machine learning forecasts quantum chemistry run times","Quantum chemistry timings predicted by ML, saving CPU hours","ML cuts quantum-job overhead 10-90% by forecasting wall times"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000189,"raw_usage":{"total_tokens":1313,"prompt_tokens":899,"completion_tokens":414,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":329}},"tokens_in":515,"tokens_out":414,"duration_ms":3937,"temperature":1.0,"reasoning_tokens":329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:36:14.424987+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the published seven task sets on a production cluster that preempts jobs and has load-dependent queue delays, then compare CPU overhead when jobs are ordered by the learned run-time estimates against the conventional job-array approach; if overhead does not drop, the scheduling claim fails. A second check is whether the FLOP-count model, presented as a noise-free cost measure, transfers to a different CPU architecture within the same error bounds.","supporting_citations":[],"review_version":1}