{"id":"3699bb39-7696-4aca-b4e2-b1f37c3e73b4","arxiv_id":"2506.00822","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A hierarchical O-RAN platform using federated deep reinforcement learning reconfigures robot MCS and transmit power, yielding 12% throughput gain and 28% energy savings in simulation.","lead":"The paper proposes a zero-touch factory network based on O-RAN where robots' transmission settings are chosen by federated deep reinforcement learning agents. In simulation, the approach reports 12% higher throughput and 32% higher energy efficiency than independent DRL baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The FedDRL gains depend on a per-agent state (Eq. 9) that omits inter-transmitter interference; without observing other agents' states/actions the D3QN MDP is not Markov, so the reported 12%/32%/28% gains are not yet established.","rationale":"The reader's weakest assumption is the same one I consider most load-bearing: the local observation in Eq. (9) is not a Markov state in a multiuser interference channel, because transmitter n's future SINR depends on other agents' current powers, channels, and hence their unobserved states. This directly undermines the central claim because every reported percentage is produced by a D3QN trained under exactly that state definition. Secondary issues (no multiple seeds, no code or data, unreported reward weights, only IDRL and RA baselines) would already justify a conditional verdict, but they do not by themselves show the argument is unsound. I keep the verdict at CONDITIONAL rather than REJECT because the non-Markov issue is a correctness risk that could be resolved by a concrete experiment; the architecture description itself is plausible and the platform design is clearly presented. No ad hominem is intended; the concern is about the RL formulation, not the authors' conduct.","tokens_in":11410,"tokens_out":6289,"duration_ms":69013,"concrete_test":"Modify the simulator so that agent n's state also includes, for every interfering transmitter on the same PRBs, the previous action and SINR (or, equivalently, replace per-agent states with a centralized joint state and shared D3QN). Train FedDRL and IDRL under both state definitions for at least 10 random seeds, and report the three headline metrics at N=20 as mean plus/minus 95% confidence intervals. If the FedDRL-vs-IDRL margins shrink or reverse under the augmented state, the original Eq. (9) state was the cause of the reported gains; if the margins persist with non-overlapping intervals, the Markov concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (9) defines agent n's state using only its own previous SINR, throughput, action, action outcome, and received power. In the multiuser OFDMA uplink of Sec. III-A, the SINR at time t+1 depends on the transmit powers and channels of other transmitters sharing the same PRBs, not just agent n's own action and channel. Because those other agents select actions from their own unobserved states, the transition P(s'_n | s_n, a_n) is not well-defined; the environment from agent n's perspective is a POMDP. D3QN's Bellman target in Eq. (8) assumes the tuple (s, a, r, s') is sampled from a Markov chain, but a feedforward network over a one-step local observation cannot represent the missing information, so the learned Q-values are biased and there is no convergence guarantee. The shared global reward in Eq. (11) intensifies the problem: each agent's reward depends on all agents' actions while its state conditions on none of them. Consequently, the headline claims of 12% throughput gain, 32% efficiency gain, and 28% energy reduction may reflect non-stationarity artifacts of the simulator rather than genuine policy improvement. The paper provides no code, data, or multiple-seed statistics, so these numbers cannot be independently checked; Sec. VI also defers real-world validation to future work.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a zero-touch network platform that integrates a hierarchical O-RAN architecture with federated deep reinforcement learning (FedDRL) for joint MCS selection and transmit power control in multirobot systems. The authors model the problem as a Markov decision process solved by D3QN agents with prioritized experience replay and momentum-based gradient descent, and they federate the local models via FedAvg. Simulation results report a 12% throughput increase, 32% improvement in normalized energy efficiency, and 28% reduction in average energy consumption relative to independent DRL baselines. The paper includes an architecture description, a signaling procedure, problem formulation, the FedDRL algorithm, a complexity analysis, and a simulation study.","tokens_in":11709,"tokens_out":3552,"duration_ms":35759,"significance":"If the reported gains are robust, the paper contributes a plausible integration of federated DRL with O-RAN's hierarchical control plane for industrial multirobot scenarios, an area that is timely and practically relevant. The paper's clear architecture description and the concrete mapping of xApps to transmitter reconfiguration are useful strengths. However, the empirical claims rest on a single simulation setup without statistical validation, and the MDP formulation is questionable in a multi-user interference-limited environment. The measured performance improvements are therefore not yet established. The paper would be strengthened by multiple seeds, full hyperparameter disclosure, and a state formulation that accounts for inter-transmitter interference.","major_comments":[{"comment":"The state in Eq. (9) contains only each transmitter's own previous SINR, throughput, action, action outcome, and received power. In the multiuser OFDMA uplink of §III-A, the SINR at time t depends on the transmit powers and channels of other transmitters sharing the same PRBs, not just on agent n's own action and channel. Because the global reward in Eq. (11) depends on all agents' actions while the state includes none of the other agents' information, the transition P(s'_n | s_n, a_n) is not well-defined; from each agent's perspective the environment is a POMDP. The D3QN Bellman target in Eq. (8) assumes the tuple (s, a, r, s') is generated by a Markov chain, so the Q-learning update is not justified and the learned values may be biased. The paper should either augment the state with an interference summary (e.g., measured interference plus other agents' recent actions) or explicitly reformulate the problem as a Dec-POMDP and discuss the implications for convergence.","section":"§IV-A, Eq. (9) and Eq. (11)"},{"comment":"The quantitative claims in the abstract and §V-B are based on a single simulation run without error bars, multiple seeds, or confidence intervals. Furthermore, the reward function in Eq. (12) and the momentum update in Eq. (6) depend on parameters α1, α2, τ1, τ2, τ3, C, and η, and none of these values appear in Tables I or II. Without these parameters and without seed-averaged statistics, the reported 12%/32%/28% improvements cannot be reproduced or statistically distinguished from random variation. The authors should report all hyperparameter values and provide means and confidence intervals over at least several random seeds.","section":"§V-A and §V-B, Tables I and II"},{"comment":"The complexity expression in Eq. (15) is internally inconsistent: the left side uses σ while the right side uses Φ, and σ is then defined as 'the function Φ(·)'. The quantity Λ = R/ρ is not clearly defined because ρ is described as the total number of model parameter aggregations occurring within R, but the relation to the FLOPS of forward/backward passes is never derived. As written, the equation is not a meaningful asymptotic bound. The authors should replace it with a step-by-step derivation of the per-layer and per-update costs, with all symbols explicitly defined.","section":"§IV-B, Eq. (15)"}],"minor_comments":[{"comment":"The phrase 'multirobot system autonomously operates' should be made plural ('multirobot systems autonomously operate') for grammatical consistency, and similar subject-verb agreement issues appear elsewhere.","section":"Abstract"},{"comment":"The expression 'TX_{t=1}' in the maximization statement appears to be a typographical error for the summation symbol; please replace it with the standard summation notation.","section":"§III-C, Eq. (1)"},{"comment":"The caption for Fig. 3(d) says 'for different numbers of transmitters,' but the x-axis is labeled 'Methods' and the text only discusses the case with 20 transmitters; please reconcile the caption with the actual plot.","section":"§V-B, Fig. 3(d)"},{"comment":"The simulation setup states 'Ne = [12, 20]' without explaining whether this is a range, a set, or two separate configurations; please clarify how the number of transmitters is varied.","section":"§V-A"}],"recommendation":"major_revision","confidential_remarks":"The paper's central idea is reasonable and fits the journal's scope, but the empirical evidence is currently too weak to support the headline numbers. The Markov assumption issue in Eq. (9) is a genuine technical concern, not a mere presentation nit, because it directly affects the validity of the D3QN training procedure. If the authors can provide a corrected state/reward formulation and robust statistical evaluation, the paper could become acceptable. The manuscript also contains a notable amount of self-citation in the introduction, though I do not see evidence that the conclusions depend on those citations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Things you should know: this is a straightforward application of well-known FedDRL (D3QN, PER, FedAvg with momentum) to MCS and power selection in a simulated O-RAN smart factory. The new part is the specific hierarchical O-RAN integration and the joint state/action/reward design. It is not a breakthrough, but it is a clean, clearly written architecture plus simulation study.\n\nWhat it does well: the O-RAN signaling procedure is described in detail (which xApp talks to which RIC over which interface), Algorithm 1 is complete, and the simulation setup is mostly specified. The baselines are at least sensible (independent DRL and random). The chosen metrics match the claims.\n\nSoft spots, in order of severity. First, the headline numbers (12% throughput, 32% efficiency, 28% energy) come from a single simulated scenario with no error bars, no multiple seeds, no confidence intervals. The reader cannot tell whether the gains are robust or an artifact of one run. This is the load-bearing issue. Second, several reward- and algorithm-related parameters are not reported (alpha1, alpha2, tau1..tau3, constant C, momentum attenuation eta), so reproduction is impossible even with code, which is not provided. Third, the state in Eq. (9) contains only the agent's own past SINR, throughput, action, outcome, and received power. In a multiuser OFDMA uplink the SINR depends on other agents' current powers and channels, whose evolution depends on their own unobserved states; the environment per agent is therefore a POMDP, not the MDP implied by D3QN's Bellman update. The shared global reward makes this worse. This does not automatically invalidate the simulation results – independent learners with local observations often work in practice in these simulators – but the paper should acknowledge the partial observability and discuss why the learned policy is still expected to behave. Fourth, the complexity analysis in Eq. (15) is muddled (the sigma/Lambda notation is never cleanly defined). Minor.\n\nThe citation pattern is fine; self-citations are relevant, not padding. The paper is honest about deferring real-world validation to future work.\n\nWho it's for: researchers working on O-RAN automation or multi-agent DRL for link adaptation; they will get a useful reference architecture and a plausible baseline point. It deserves a serious referee; I would send it to review, but with the expectation that the authors add repeated-seed statistics, report the missing parameters, and address the POMDP issue – either by expanding the state or by justifying the local-observation approximation. Without those changes the specific numerical gains should not be taken as established.","headline":"A competent FedDRL-plus-O-RAN application paper whose simulation claims lack statistical support and whose per-agent state design raises an unaddressed POMDP issue; deserves a referee but needs revision.","tokens_in":12254,"tokens_out":2162,"would_cite":false,"duration_ms":20644,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated deep reinforcement learning over O-RAN gives simulated factory robots 12% more throughput while cutting transmission energy by 28%.","keywords":["federated deep reinforcement learning","O-RAN","zero-touch networks","multirobot systems","MCS selection","transmission power control","smart factories","D3QN"],"falsifier":"Run the identical 20-transmitter scenario but include in each agent's state the interference or SINR contribution of neighboring transmitters, or their last actions; if FedDRL's 12% throughput and 28% energy gains over independent DRL disappear or reverse, the reported advantage is an artifact of the incomplete state rather than a robust property of federated D3QN.","tokens_in":11237,"feed_emoji":"🤖","tokens_out":3997,"duration_ms":36218,"temperature":0.7,"pith_summary":"This paper proposes a zero-touch factory-network platform in which O-RAN control units run federated deep reinforcement learning (FedDRL) to reconfigure each robot's uplink modulation-and-coding scheme and transmit power. The claim is that federating Dueling Double Deep Q-Network (D3QN) agents through FedAvg-style averaging, plus momentum and prioritized replay, lets robots jointly decide these parameters better than if each agent trains alone. The paper reports, at 20 transmitters, a 12% throughput gain, a 32% gain in normalized transmission energy efficiency, and a 28% reduction in average transmission energy consumption relative to independent DRL. A sympathetic reader would care because plugging learning into O-RAN's hierarchical control loops is a concrete route to the 'zero-touch' automation goal for Industry 4.0.","feed_headline":"FedDRL lifts robot-network throughput 12%, cuts energy 28%","feed_subtitle":"Federating the robots' deep-Q agents through a shared global model beats independent training on MCS and power choice.","key_machinery":"The load-bearing object is the FedDRL framework: a D3QN, a dueling double deep Q-network that splits Q-values into state-value and action-advantage streams and uses a periodic target network, augmented with prioritized experience replay, momentum gradient descent on both local and global parameters, and FedAvg aggregation of the local models at the non-real-time RIC. The framework's designed role is to convert each transmitter's MCS-and-power choice into one distributed reinforcement-learning problem, with the global reward in Eq. (11) as the shared objective the federated model maximizes.","core_discovery":"The paper's central discovery is that federated training of the D3QN agents, rather than independent local training, is what turns transmitter reconfiguration into a shared optimization problem without centralizing experience. Each robot is an xApp agent observing its own previous SINR, throughput, action, action outcome, and received power; all agents receive the same global reward composed of throughput and transmission energy efficiency. Local gradients are updated with momentum, local models are periodically averaged at the non-real-time RIC, and the updated global model is sent back for the next round. Under the simulated OFDMA factory scenario, this procedure yields the reported gains over independent DRL and random action selection.","pith_inferences":["The paper's state space excludes other transmitters' actions and channels; because uplink SINR depends on mutual interference, the MDP may not be Markov, so the reported gains could shrink or vanish in settings where interference dominates. A direct test would augment the state with per-neighbor interference and rerun the comparison.","The single-edge-cloud simulation leaves open whether global aggregation across multiple nRT-RICs would help or hurt when robots hand over between control domains; one can extend the same FedDRL loop to the multi-edge-cloud case.","The global reward is identical for all agents, so the approach implicitly assumes cooperative robots; task diversity or competition could make a shared reward misaligned, suggesting a future extension to individualized rewards under a federated constraint."],"forward_implications":["If FedDRL is correct, factory operators can reconfigure uplink parameters through O-RAN software control loops without human retuning of each robot.","Federating D3QN agents with global momentum should converge faster and more stably than independent DRL, because each agent's loss benefits from shared experience.","The reported 28% energy reduction implies battery-powered robots could sustain longer operations under the same traffic, assuming the simulated gains carry over to hardware.","The 12% throughput gain and 32% energy-efficiency gain quantify the value of distributed collaborative learning over per-robot selfish learning in a multiuser interference scenario."],"supporting_citations":[{"why":"Supplies the smart-factory multirobot scenario, the channel model, and the path-loss parameters used in the simulation.","marker":"[1]"},{"why":"Defines the O-RAN architectural elements (O-CU, O-DU, O-RU, RIC, SMO) that the proposed platform is built on.","marker":"[5]"},{"why":"Supplies FedAvg, the aggregation rule used to combine local D3QN models in Eqs. (13) and (14).","marker":"[9]"},{"why":"Establishes the prior FedDRL-in-O-RAN use case that this paper extends to transmitter parameter reconfiguration.","marker":"[17]"},{"why":"Provides the D3QN multi-agent resource-allocation architecture adopted for MCS and power decisions.","marker":"[18]"},{"why":"Provides the momentum gradient descent update rules used for local and global parameter updates.","marker":"[19]"},{"why":"Provides the prioritized experience replay mechanism used to sample minibatches during local training.","marker":"[20]"},{"why":"Supplies the 29 MCS indices and spectral efficiencies that determine data bits per MCS in the simulations.","marker":"[21]"}],"fun_headline_variants":["Federated robot learning: 12% throughput gain, 28% energy drop","FedDRL in O-RAN: robots share learning for 12% more throughput, 28% less energy","Robot collaboration via federated DRL: 12% throughput, 28% energy savings","O-RAN-fed federated DRL reconfigures robots: 12% throughput, 28% energy cut","Shared robot agents: 12% higher throughput, 28% lower energy consumption"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The learning pipeline assumes each robot's one-step state, its own previous SINR, throughput, action, outcome, and received power, is enough to make the decision problem Markovian, even though interference from other transmitters is not part of the state.","fun_headline_variants_meta":{"raw":{"variants":["Federated robot learning: 12% throughput gain, 28% energy drop","FedDRL in O-RAN: robots share learning for 12% more throughput, 28% less energy","Robot collaboration via federated DRL: 12% throughput, 28% energy savings","O-RAN-fed federated DRL reconfigures robots: 12% throughput, 28% energy cut","Shared robot agents: 12% higher throughput, 28% lower energy consumption"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00028,"raw_usage":{"total_tokens":1628,"prompt_tokens":883,"completion_tokens":745,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":622}},"tokens_in":499,"tokens_out":745,"duration_ms":6640,"temperature":1.0,"reasoning_tokens":622,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:56:52.362749+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical 20-transmitter scenario but include in each agent's state the interference or SINR contribution of neighboring transmitters, or their last actions; if FedDRL's 12% throughput and 28% energy gains over independent DRL disappear or reverse, the reported advantage is an artifact of the incomplete state rather than a robust property of federated D3QN.","supporting_citations":[{"cited_title":"Wireless networked multirobot systems in smart factories,","cited_arxiv_id":null,"evidence_quote":"Supplies the smart-factory multirobot scenario, the channel model, and the path-loss parameters used in the simulation."},{"cited_title":"O-RAN Architecture Description 9.0,","cited_arxiv_id":null,"evidence_quote":"Defines the O-RAN architectural elements (O-CU, O-DU, O-RU, RIC, SMO) that the proposed platform is built on."},{"cited_title":"Communication-Efficient Learning of Deep Networks from Decentral- ized Data,","cited_arxiv_id":null,"evidence_quote":"Supplies FedAvg, the aggregation rule used to combine local D3QN models in Eqs. (13) and (14)."},{"cited_title":"Federated deep reinforce- ment learning for efficient jamming attack mitigation in o-ran,","cited_arxiv_id":null,"evidence_quote":"Establishes the prior FedDRL-in-O-RAN use case that this paper extends to transmitter parameter reconfiguration."},{"cited_title":"Multi-agent reinforcement learning resources allocation method using dueling double deep q-network in vehicular networks,","cited_arxiv_id":null,"evidence_quote":"Provides the D3QN multi-agent resource-allocation architecture adopted for MCS and power decisions."},{"cited_title":"Accelerating federated learn- ing via momentum gradient descent,","cited_arxiv_id":null,"evidence_quote":"Provides the momentum gradient descent update rules used for local and global parameter updates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 29 MCS indices and spectral efficiencies that determine data bits per MCS in the simulations."}],"review_version":1}