{"id":"db74ae6c-5141-46ec-a2f7-46ac84a645a3","arxiv_id":"2506.00929","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":10,"one_line_summary":"WA3C extends A3C with a priority-weighted softmax and a five-term reward, and the paper reports simulated gains in latency, energy, and fairness over six baselines.","lead":"A research group proposes WA3C, a deep reinforcement learning scheduler for cloud jobs that combines priority-weighted action choice with a multi-part reward for latency, energy, fairness, and priority. The paper reports simulation results claiming lower latency, lower energy, and fairer scheduling than standard baselines, but the evidence is weak and internally inconsistent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central 'consistently outperforms' claim is unsupported because the evidence is a set of unreproducible, single-run simulation curves with contradictory notation (β/γ swapped, non-RL baselines assigned reward curves in Fig. 5d) and no statistical confidence intervals.","rationale":"The paper's central claim is that WA3C consistently outperforms both traditional and RL-based baselines on four metrics. For that claim to hold, the experimental comparison must be both internally coherent and sufficiently reliable to support a 'consistent' superiority statement. My review finds that neither condition is satisfied. The notation and algorithm specification are contradictory: β is introduced as a priority-scaling factor in Eq. (10) but also appears inside TD-error and advantage computations in Algorithm 1, and Section IV-C reinterprets β as the discount factor while renaming γ (originally the discount factor) as the learning rate. This prevents an independent re-implementation from being unambiguous. More fundamentally, Fig. 5(d) plots total reward for non-RL schedulers; since those schedulers do not optimize the weighted reward of Eq. (1) and no reward formula is given for them, the comparison is incoherent. The lack of error bars, significance tests, or code release further means the reported single curves cannot substantiate 'consistently' outperforming. I agree with the reader's concern that the energy model is hand-tuned and unvalidated, but I see the broader reproducibility and internal-consistency problem as the most load-bearing issue: even if the energy model were eventually validated, the current evidence for the headline claim would still be insufficient. If the proposed concrete test were run and the authors' results were reproduced with non-overlapping confidence intervals, my assessment would change; absent that, rejection is the appropriate verdict. No independent support is present: there are no machine-checked proofs, no released code, and no parameter-free derivations that could offset the evidentiary gaps.","tokens_in":13579,"tokens_out":4157,"duration_ms":40424,"concrete_test":"Request the authors' simulator and configuration files, or re-implement from Section III; run WA3C and all six baselines for 500 episodes with at least 10 random seeds each, computing mean and 95% bootstrap confidence intervals for latency, energy, dismissal rate, and per-episode reward. Explicitly require the reward formula used to score non-RL baselines in Fig. 5(d). If the non-RL baselines were not scored by Eq. (1), Fig. 5(d) is invalid; if any confidence interval overlaps between WA3C and a baseline, the claim of consistent outperformance is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the claim that WA3C consistently outperforms six baselines, the paper must provide a valid, reproducible comparison. That condition is not met. Section III-C defines β=2.0 as the priority scaling factor in Eq. (10), but Algorithm 1 (lines 16 and 18) uses the same symbol β inside the TD error and advantage estimates, and Section IV-C treats β as the discount factor (varied 0.6–0.9) while γ, defined as the discount factor 0.95 in Section III-A, is called the learning rate in that same section. A reader cannot determine which quantity was actually varied in Figs. 3–4 or whether the reported updates are A3C-style. In Fig. 5(d), 'Total Reward per Episode' is plotted for non-RL heuristics (RR, SJF, LJF, Tetris, Random) that by definition do not maximize the Section III-B reward; no reward formula for those baselines is given, so Fig. 5(d) is semantically incoherent. The evaluation also lacks code release, seeds, error bars, or significance tests; the reported single curves are therefore insufficient to establish consistent outperformance. The energy model (Eqs. 4–6) with hand-set z=0.3, θ_j=0.2, and reward weights w1–w5 is unvalidated against any measured power data, so the energy comparisons could reflect the model's assumptions rather than real datacenter behavior. Collectively, the load-bearing evidence for the headline claim is neither internally consistent nor independently checkable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Weighted A3C (WA3C), an extension of the A3C deep reinforcement learning framework for cloud job scheduling. The contribution is a composite reward function that linearly combines QoS, energy, priority, fairness, and dismissal-penalty terms, together with a priority-weighted softmax action-selection mechanism. The authors evaluate WA3C against non-RL heuristics (RR, SJF, LJF, Tetris, Random) and DRL baselines (A3C, DQL, PG) on synthetic workloads, claiming that WA3C consistently outperforms all baselines in average job latency, energy consumption, job dismissal rate, and total reward. The paper also reports a sensitivity analysis of the discount factor and learning rate.","tokens_in":14006,"tokens_out":6349,"duration_ms":65874,"significance":"If the headline claim were reliably established, WA3C would be a practically relevant contribution to multi-objective cloud scheduling: the idea of injecting job priority and fairness directly into a weighted A3C reward is timely, and the asynchronous worker design is a reasonable starting point. However, the manuscript does not provide reproducible code, data, seeds, error bars, or significance tests, and it contains multiple internal inconsistencies in notation, units, and reported reward values. Because the entire contribution is empirical, these issues prevent the reader from verifying the central claim; the significance of the work cannot currently be assessed.","major_comments":[{"comment":"The symbol β is overloaded with at least three different meanings. Eq. (10) defines β=2.0 as the priority scaling factor in the softmax action selection; Algorithm 1 lines 16 and 18 use β in the advantage and TD-error formulas as a discount-like coefficient; and Section IV-C treats β as the discount factor varied over {0.6, 0.7, 0.8, 0.9} while γ, defined as the discount factor γ=0.95 in Section III-A, is called the learning rate. As a result, the sensitivity analysis in Figures 3 and 4 cannot be mapped to the algorithm as written, and the reported A3C-style updates are not well-defined.","section":"Section III-C, Algorithm 1, Section IV-C"},{"comment":"The 'Total Reward per Episode' panel for non-RL baselines is semantically incoherent. RR, SJF, LJF, Tetris, and Random are heuristics that do not optimize the reward in Eq. (1), and no reward formula is provided for them, so plotting their 'total reward' has no clear meaning. Additionally, the text says the x-axis is average system load (0.4 to 2.8), while the caption and paragraph refer to 'training epochs'; this mismatch makes the panel uninterpretable as evidence.","section":"Section IV-D and Fig. 5(d)"},{"comment":"The load-bearing claim that WA3C 'consistently outperforms' baselines is supported only by single simulation curves. No code, data, random seeds, number of repeated runs, confidence intervals, or significance tests are reported. Because the evaluation is the only evidence for the paper's central claim, the current results are not independently checkable and cannot establish consistent outperformance.","section":"Section IV (entire evaluation)"},{"comment":"The energy model in Eqs. (4)-(6) relies on hand-set constants (z=0.3, θ_j=0.2) and is not validated against any measured power data. The reported units are also inconsistent: Section IV-D reports energy in kWh, whereas Eq. (5) with W in Watts and τ in seconds yields Joules, and Section IV-E mixes 'kW' and 'kWh' within the same sentence. The energy comparisons may therefore reflect modeling assumptions rather than real datacenter behavior.","section":"Section III-B.2, Section IV-D, Section IV-E"},{"comment":"Fairness is presented as a key contribution, but no experimental result reports a fairness metric. Eq. (8) defines the fairness reward via the variance of resource utilization, while Section V-A states that fairness is measured using 'Jain's Index'; these are different quantities. The fairness component is never isolated, reported, or validated, so the 'fair' part of the headline claim is unsupported.","section":"Abstract, Section III-B.4, Section V-A"}],"minor_comments":[{"comment":"The text says real-world Google Cluster Workload traces were 'incorporated,' but the results section reports only synthetic job traces; please clarify which experiments use which dataset.","section":"Section IV-A"},{"comment":"The discussion refers to 'a tunable parameter α' for priority in the softmax policy, while Eq. (10) and Section III-C use β=2.0; the notation should be consistent.","section":"Section V-A"},{"comment":"Line 7 writes the critic gradient as ∇(R_i−V(s_i))^2, but the squared TD error is not defined with the correct sign or factor, and R_i is not defined (immediate reward versus discounted return).","section":"Algorithm 2"},{"comment":"The energy axis mixes units ('12 kW' and '10 kW h'); please standardize to a single unit, e.g., energy per job in kWh or Joules.","section":"Fig. 6(b)"},{"comment":"Reference [21] is cited as the source of the A2C model, but the cited Sutton and Barto book introduces actor-critic methods more generally; the attribution should be corrected.","section":"References"}],"recommendation":"reject","confidential_remarks":"The reader's report is substantially correct. The manuscript's empirical core is not reproducible and contains internal contradictions in the definition of β/γ, in the reward values, and in energy units, so the central claim cannot be evaluated. I would not encourage a standard revision of this text; however, if the authors were to reconstruct the evaluation with consistent notation, a clear reward definition for all baselines, released code and data, and statistical rigor, a substantially rewritten manuscript might be worth reconsidering in the future."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's the short version: this paper describes a legitimate extension of A3C for cloud scheduling—a five-term weighted reward plus a priority-weighted softmax—but the evidence for the headline claim is not there. The combination is new in the sense that I don't find it in the cited papers. Each ingredient is standard, so novelty is a new integration, not a new principle. That is fine if the experiments are solid. They aren't.\n\nWhat the paper does well: the reward decomposition is clearly laid out (QoS, energy, priority, fairness, dismissal), and the authors include a limitation section that honestly lists training cost, reward-tuning difficulty, and integration effort. The related work is relevant and the writing is mostly readable.\n\nThe soft spots are load-bearing. The experiments are single-run curves with no error bars, no seeds, no significance tests, and no code or data release. The notation is self-contradictory: beta is the priority-scaling weight in Eq. (10), but in Algorithm 1 it appears inside the TD error and advantage, and in Section IV-C it is treated as the discount factor. Meanwhile gamma, defined as the discount factor in the MDP, is called the learning rate in the sensitivity study. A reader cannot tell which hyperparameter was varied in Figs. 3–4 or whether the update is actually A3C. Figure 5(d) plots 'Total Reward per Episode' for non-RL heuristics like RR and SJF that have no reward function defined; that panel is semantically undefined. The energy model (Eqs. 4–6) uses hand-set constants and weights, is never validated against measured power data, and the energy results are entirely produced by that model. The abstract mentions 'extensive experiments using synthetic job traces'; the setup says Google Cluster Data were incorporated, but no real-trace results appear. These are not minor polish issues; they undermine the central claim of consistent outperformance.\n\nBottom line: the idea is worth testing, but this paper does not test it in a reproducible way. It could become a real paper if the authors release code, run multiple seeds, fix the notation, and validate the energy model. For now, the claim is unsupported.\n\nI'd send it to peer review only because a competent reviewer could demand those fixes and the algorithm is concrete enough to be checked. My own verdict is reject as presented. Not worth citing until the evidence appears.","headline":"A reasonable weighted-reward A3C extension, but the evaluation is single-run, internally inconsistent, and the headline outperformance claim is not supported.","tokens_in":14471,"tokens_out":3080,"would_cite":false,"duration_ms":29800,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A weighted A3C scheduler beats heuristic and deep-RL baselines on latency, energy, and dismissals in cloud simulations.","keywords":["cloud resource allocation","deep reinforcement learning","A3C","job scheduling","fairness","priority scheduling","energy efficiency","actor-critic"],"falsifier":"Measure actual power draw of a cluster running WA3C and the baselines on the Google cluster-usage trace, then compare measured kWh against the simulated values from Eqs. (4)–(6); if the measured ranking differs from the simulated one, the paper's energy-advantage claim is refuted.","tokens_in":13364,"feed_emoji":"⚡","tokens_out":10612,"duration_ms":90287,"temperature":0.7,"pith_summary":"This paper claims that a deep reinforcement learning scheduler for cloud datacenters can serve high-priority jobs without starving low-priority ones while also cutting latency, energy use, and job rejections. The proposed Weighted A3C (WA3C) model extends the A3C actor-critic algorithm with a five-term reward that combines quality-of-service, energy, priority satisfaction, fairness, and dismissal penalties, plus a priority-weighted softmax that biases action selection toward urgent jobs. In simulations with synthetic workloads and Google cluster-usage traces, WA3C reports lower average latency, lower energy consumption, a lower job dismissal rate, and higher total reward than Round Robin, Shortest Job First, Longest Job First, Tetris, Random, standard A3C, Deep Q-Learning, and Policy Gradient. If correct, this gives cloud operators a scheduler that adapts online to workload shifts without retraining from scratch. The paper's own discussion acknowledges that deep-RL training is costly, reward shaping requires tuning, and integration with existing orchestration frameworks is non-trivial.","feed_headline":"Priority-aware cloud scheduler cuts latency, energy, and job drops","feed_subtitle":"A weighted reward and priority-aware action selection let WA3C beat heuristics and other deep-RL baselines.","key_machinery":"The load-bearing object is the composite reward function of Eq. (1), which forces the agent to optimize latency, energy, priority satisfaction, fairness, and dismissal avoidance simultaneously; without it the scheduler reduces to plain A3C. The second piece is the priority-weighted softmax of Eq. (10), which injects job urgency directly into action selection rather than relying only on learned Q-values. The third is the asynchronous worker architecture, in which each VM runs a worker that accumulates gradients and periodically synchronizes with the global actor and critic, providing scalability and online adaptation. All results are generated from this combination together with the hand-calibrated energy model of Eqs. (5)–(6), which converts CPU utilization, CPI, and MAPI into a per-job energy estimate.","core_discovery":"The central claim is that adding job priority and fairness to the reward signal of an asynchronous actor-critic learner transforms the scheduler: WA3C maximizes $R_t = w_1 R^{QoS}_t + w_2 R^E_t + w_3 R^P_t + w_4 R^F_t + w_5 R^D_t$, with the penalty terms shaping behaviour so that urgent jobs are served promptly, utilization variance is penalized, and dismissals are discouraged. Action selection then uses the priority-weighted softmax $\\pi(a_t=j) \\propto \\exp(Q(s_t,j) + \\beta P_j)$, and $n$ asynchronous workers, one per VM, push gradients to a shared global actor and critic. The authors report that this configuration converges to better cumulative rewards, lower normalized latency, and lower dismissal rates than standard A3C, DQL, and Policy Gradient, and outperforms non-RL schedulers on every metric across all tested system loads. The simulations use both synthetic 1000-job traces and the Google cluster-usage dataset.","pith_inferences":["A reader should double-check the sensitivity analysis: Section IV-C calls the discount factor $\\beta$ and the learning rate $\\gamma$, whereas the model definition in Sections III-A and III-C sets $\\gamma=0.95$ as the discount factor and $\\beta=2.0$ as the priority weight; if the labels were swapped in the experiments, the reported best hyperparameters do not describe the model as specified.","A natural next test is to replace the hand-set energy model with measured power traces, turning the simulated kWh savings into verifiable operational savings.","The fairness term penalizes variance in resource utilization; for workloads with heterogeneous resource demands, a size-aware fairness metric would be needed to avoid penalizing legitimate differences."],"forward_implications":["If the reported results hold, cloud schedulers can lower average latency and energy use while respecting job priorities, without retraining from scratch when workload mix shifts.","Operators can rebalance scheduler behavior by changing the five reward weights, e.g., increasing $w_4$ to strengthen fairness in multi-tenant systems.","The priority-weighted softmax is a generic mechanism for injecting urgency into DRL schedulers beyond clouds, such as edge or serverless platforms.","The worker-per-VM design ties scalability to cluster size, so larger deployments would simply add more asynchronous workers.","The reported curves indicate WA3C's advantage persists as average system load rises from 0.4 to 2.8, implying the policy generalizes to congested conditions."],"supporting_citations":[{"why":"Supplies the base A3C algorithm with asynchronous parallel workers that WA3C extends.","marker":"[4]"},{"why":"Prior A3C-based adaptive resource allocation strategy that WA3C builds on and compares against.","marker":"[5]"},{"why":"DeepRM, the deep-RL resource management work used as a baseline and motivating application.","marker":"[2]"},{"why":"Google cluster-usage trace dataset used to validate WA3C under production-scale workloads.","marker":"[25]"}],"fun_headline_variants":["WA3C: Priority-aware deep RL for fair cloud scheduling","Weighted A3C scheduler balances latency, energy, and fairness","Priority-weighted actor-critic cuts cloud job delays and energy","WA3C cloud scheduler: Better latency, fairness, and efficiency"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The energy and fairness results come from a hand-set simulation model—power drawn as a linear ramp in CPU utilization and per-job constants $z=0.3$ and $\\theta_j=0.2$, plus hand-chosen reward weights $w_1$ through $w_5$—rather than from measured power readings, so the reported energy and fairness advantages may not transfer to real hardware if these constants are wrong.","fun_headline_variants_meta":{"raw":{"variants":["WA3C: Priority-aware deep RL for fair cloud scheduling","Weighted A3C scheduler balances latency, energy, and fairness","Priority-weighted actor-critic cuts cloud job delays and energy","WA3C cloud scheduler: Better latency, fairness, and efficiency"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2628,"prompt_tokens":1022,"completion_tokens":1606,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":638,"completion_tokens_details":{"reasoning_tokens":1534}},"tokens_in":638,"tokens_out":1606,"duration_ms":11436,"temperature":1.0,"reasoning_tokens":1534,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:54:12.035273+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure actual power draw of a cluster running WA3C and the baselines on the Google cluster-usage trace, then compare measured kWh against the simulated values from Eqs. (4)–(6); if the measured ranking differs from the simulated one, the paper's energy-advantage claim is refuted.","supporting_citations":[{"cited_title":"Asynchronous methods for deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the base A3C algorithm with asynchronous parallel workers that WA3C extends."},{"cited_title":"Adaptive and efficient resource allocation in cloud datacenters using actor-critic deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Prior A3C-based adaptive resource allocation strategy that WA3C builds on and compares against."},{"cited_title":"Resource man- agement with deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"DeepRM, the deep-RL resource management work used as a baseline and motivating application."},{"cited_title":"Google cluster-usage traces: format+ schema,","cited_arxiv_id":null,"evidence_quote":"Google cluster-usage trace dataset used to validate WA3C under production-scale workloads."}],"review_version":1}