{"id":"b5511f64-81df-45f1-857c-249ac7460349","arxiv_id":"2604.26576","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A novel workload replay method validates MPI malleability on Marenostrum 5, showing 27% reduction in malleable job time without delaying baseline workload.","lead":"The paper presents a methodology to validate dynamic resource management techniques like MPI malleability by replaying real-world HPC workload logs on actual supercomputer hardware. This approach addresses skepticism about their practicality by testing under realistic conditions rather than simulations.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Static log replay fixes submission times and may miss dynamic user adjustments to malleability-altered runtimes","rationale":"This matches the reader's weakest assumption exactly and is the point at which the validation claim is least secured. The concern is methodological rather than about the malleability mechanism itself. A positive outcome on the proposed check would allow the headline numbers to stand with a clear caveat; a negative outcome would require the paper to qualify its 'real-world conditions' statement. No deeper inconsistency (e.g., in the efficiency-aware policy or measurement) is visible from the given text.","tokens_in":1673,"tokens_out":333,"duration_ms":58638,"concrete_test":"Augment the replay driver with a simple per-user model that advances the next submission timestamp by 0.5× the reduction in the prior job's wall-clock time; re-run both malleable and baseline scenarios on the same 125-node partition and report whether the malleable workload completion time still improves by ≥20% and whether utilization remains within 5% of the static-replay result.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The 27% reduction claim requires that replayed conditions match real HPC behavior. Workload logs record fixed submission times derived from the original non-malleable execution. When malleability shortens some jobs, real users often submit follow-on work sooner; the described methodology adapts the workload to the 125-node partition but retains static log timestamps without modeling such feedback loops. This leaves open whether the observed time savings, queueing delays, and unchanged utilization would persist under actual user responses.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes a methodology for validating dynamic resource management techniques such as MPI malleability by replaying real-world workload logs on actual HPC hardware, with workload adaptation to the target cluster. Evaluated on a 125-node malleability-enabled partition of Marenostrum 5, it claims that parallel efficiency-aware malleability reduces malleable workload completion time by 27% without delaying the baseline workload, while introducing queueing delays for some jobs but preserving overall resource utilization.","tokens_in":1753,"tokens_out":329,"duration_ms":29802,"significance":"If the replay methodology accurately captures real conditions, this provides empirical evidence from production hardware that could help overcome administrator skepticism toward DRM techniques. The use of replayed logs on real infrastructure rather than pure simulation is a strength for credibility and reproducibility.","major_comments":[{"comment":"The central 27% reduction claim (abstract) depends on static log replay with fixed submission times reproducing actual cluster behavior. However, this does not model dynamic user responses where shortened job runtimes could prompt earlier follow-on submissions, potentially changing observed time savings, queueing delays, and utilization.","section":"Evaluation methodology"},{"comment":"Support for the reported outcomes on Marenostrum 5 is limited by insufficient details on experimental setup, controls, statistical significance, and how the workload was adapted to the 125-node partition, as these are load-bearing for validating the malleability benefits.","section":"Abstract and evaluation"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments on our manuscript. We address each major comment below and indicate the revisions we will make to improve clarity and completeness.","responses":[{"response":"Our methodology deliberately replays fixed submission times from production logs to reproduce observed cluster conditions on real hardware without introducing unverified assumptions about user behavior. Modeling dynamic responses (e.g., earlier follow-on submissions) would require additional user studies or behavioral models outside the scope of this hardware-validation-focused work. We will add a brief discussion of this limitation and its implications in the revised manuscript.","revision_made":"partial","referee_comment":"[Evaluation methodology] The central 27% reduction claim (abstract) depends on static log replay with fixed submission times reproducing actual cluster behavior. However, this does not model dynamic user responses where shortened job runtimes could prompt earlier follow-on submissions, potentially changing observed time savings, queueing delays, and utilization."},{"response":"We agree that additional details are required. In the revised manuscript we will expand the experimental setup section with explicit information on controls, the statistical methods used to assess significance of the reported 27% reduction, and the precise adaptation steps applied to the workload logs for the 125-node partition.","revision_made":"yes","referee_comment":"[Abstract and evaluation] Support for the reported outcomes on Marenostrum 5 is limited by insufficient details on experimental setup, controls, statistical significance, and how the workload was adapted to the 125-node partition, as these are load-bearing for validating the malleability benefits."}],"tokens_in":1275,"tokens_out":350,"duration_ms":44579,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core contribution is a methodology that takes real workload logs, adapts them to a specific 125-node partition on Marenostrum 5, and replays them to evaluate MPI malleability in a live setting. This moves past pure simulations and gives administrators something closer to production conditions. They report that parallel efficiency-aware malleability cut the time for the malleable workload by 27% without pushing back the baseline jobs, though individual jobs saw some extra queue time and overall utilization held steady. Running this on real hardware with actual logs is the part that stands out as useful. The main soft spot is exactly what the stress test flags: the logs keep original submission times fixed. When malleability shortens runs, real users might submit their next work earlier, creating a feedback loop the replay does not model. The abstract gives no sign they adjusted for that, so the observed savings and delay patterns might not hold up under actual user responses. Without more on how they handled job dependencies or multiple replay variations, the evidence for the 27% number feels preliminary. This work is aimed at HPC system researchers and cluster operators who need practical checks on dynamic resource techniques. Someone looking for a template to test malleability on their own machine would find the adaptation and replay steps worth reading. The paper shows clear thinking on why simulations fall short and tries to close that gap with hardware runs. It deserves a serious referee to check the replay details and see if the user-behavior concern can be addressed or bounded. I would recommend sending it to peer review.","headline":"Replaying real logs on Marenostrum 5 gives a concrete test of MPI malleability, but fixed timestamps leave the 27% claim open to questions about user behavior.","tokens_in":2233,"tokens_out":390,"would_cite":false,"duration_ms":42142,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Replaying real HPC workload logs on a production supercomputer shows efficiency-aware MPI malleability shortens malleable job times by 27% without delaying baseline workloads.","keywords":["MPI malleability","dynamic resource management","workload replay","HPC clusters","parallel efficiency","resource utilization","job scheduling"],"falsifier":"Running the identical malleability policy on the same replayed logs but measuring no reduction near 27% in malleable workload time or an increase in baseline workload completion time would falsify the reported performance benefit.","tokens_in":2577,"feed_emoji":"🖥️","tokens_out":659,"duration_ms":57818,"temperature":0.7,"pith_summary":"The paper develops a method to validate dynamic resource management techniques by replaying actual workload logs on real HPC hardware, adapting the logs to match the target cluster's configuration. This directly tackles administrator skepticism that malleability and similar approaches only work in simulations. Testing occurred on a 125-node partition where malleable applications could resize their MPI process counts. Results indicate that guiding malleability by parallel efficiency cut the overall time for the malleable portion of the workload by 27 percent, preserved resource utilization, and left the baseline non-malleable workload's completion time unchanged, even though some individual jobs waited longer in the queue.","feed_headline":"Malleability cuts malleable HPC workload time by 27%","feed_subtitle":"Real log replay on 125-node partition keeps baseline jobs on schedule and utilization steady.","key_machinery":"The workload log replay methodology, which adapts historical job and user data to current cluster conditions to enable realistic validation of malleable MPI applications that dynamically adjust their process count based on observed parallel efficiency.","core_discovery":"The authors introduce a replay-based validation method that reproduces real cluster conditions by adapting workload logs to the target HPC infrastructure. When applied to MPI malleability on a 125-node partition, parallel efficiency-aware malleability reduced the malleable workload's execution time by 27% without increasing the completion time of the baseline workload, while maintaining the resource utilization rate despite added queueing delays for certain jobs.","pith_inferences":["The replay technique could be reused to test other dynamic resource management methods such as job migration or power capping.","Production clusters might first run limited log-replay pilots before enabling malleability cluster-wide.","Comparing replay outcomes against pure simulation results would quantify how much additional realism the log replay supplies.","Similar efficiency gains could appear on other systems whose workload traces show comparable job-size distributions."],"forward_implications":["Malleable workloads finish earlier when resizing follows parallel efficiency.","Non-malleable baseline workloads incur no extra delay from the presence of malleable jobs.","Cluster-wide resource utilization remains at the same level as without malleability.","Some jobs experience longer queue waits as a side effect of dynamic process adjustment."],"fun_headline_variants":["Real log replay validates MPI malleability on HPC clusters","27% workload time reduction via efficiency-aware malleability","Malleability maintains utilization in real-world replay tests","Replay method confirms no delays to baseline HPC jobs"],"cache_read_input_tokens":64,"weakest_assumption_plain":"Replaying the workload logs faithfully reproduces the actual job arrival patterns, user behaviors, and cluster conditions on the target system.","fun_headline_variants_meta":{"raw":{"variants":["Real log replay validates MPI malleability on HPC clusters","27% workload time reduction via efficiency-aware malleability","Malleability maintains utilization in real-world replay tests","Replay method confirms no delays to baseline HPC jobs"]},"model":"grok-4.3","cost_usd":0.004074,"raw_usage":{"total_tokens":1968,"prompt_tokens":623,"num_sources_used":0,"completion_tokens":60,"cost_in_usd_ticks":40740500,"prompt_tokens_details":{"text_tokens":623,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1285,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":623,"tokens_out":60,"duration_ms":22460,"temperature":1.0,"reasoning_tokens":1285,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-07T10:56:19.099666+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the identical malleability policy on the same replayed logs but measuring no reduction near 27% in malleable workload time or an increase in baseline workload completion time would falsify the reported performance benefit.","supporting_citations":[],"review_version":1}