{"id":"9132624f-ea75-4610-ba86-9d55b97dcad8","arxiv_id":"2412.18856","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A DRL-based IOS controller that jointly optimizes phase and amplitude, accelerated by a supervised-learning digital twin, outperforms random and bandit baselines in simulated MU-MIMO systems.","lead":"This paper proposes a deep reinforcement learning framework, DeepIOS, to configure intelligent omni-surfaces in multi-user MIMO systems, adding a digital twin trained on historical data to speed up training and enable near-real-time decisions. A generalist reader might care because it represents a practical route to one of the harder control problems in 6G wireless: configuring large metasurfaces without full channel state information.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Digital twin accuracy is never validated: the 71.4% convergence speedup and 0.6 ms real-time claim rest on an unmeasured assumption that the supervised twin generalizes under distribution shift and stays accurate over Γ=1000 autoregressive rollouts.","rationale":"The reader's identification of the digital twin accuracy and generalization as the weakest assumption is correct. The headline numbers in Table III are meaningful only if the twin is a reliable proxy for the physical environment; the paper gives no held-out prediction error, no ablation varying twin accuracy, and no confidence intervals (all simulations are averaged over 20 runs but no variance is shown). The twin's architecture (a GRU plus FC for state prediction, a ResNet block for reward prediction) is reasonable, and the end-to-end result that policies trained in the twin achieve comparable data rates when deployed in the physical simulator is genuine supporting evidence. Credit is also due for the closed-loop calibration mechanism and for the fact that the physical controller is intentionally inference-only, which legitimately yields the 0.6 ms decision time. The concern is load-bearing because the environment used to build the twin (λ=9, random UE positions) differs from the evaluation environments (λ=5-15 with Gauss-Markov mobility), and the autoregressive rollouts of length Γ=1000 can compound small one-step errors into substantial policy divergence. The paper itself describes the digital twin as 'a small generative model' (Section I) but never tests how small or how accurate. A secondary implementation issue is the unusual loss in Eq. (12), which squares the sum of the two branches' TD errors rather than summing squared per-branch errors; this could allow errors to cancel, but because it affects both compared schemes equally, it does not undermine the digital-twin-specific comparison as directly as the unvalidated twin fidelity. The proposed concrete test, measuring one-step and Γ-step prediction error on held-out real data and comparing state distributions, would directly settle whether the speedup and real-time guarantees transfer beyond the paper's specific simulation settings.","tokens_in":26419,"tokens_out":10171,"duration_ms":100105,"concrete_test":"Train the digital twin exactly as in Section IV-B on D=1000 samples, then on a fresh held-out set of 2000 real-environment transitions (λ=10, Gauss-Markov mobility) compute (i) one-step MSE of predicted state and absolute error of predicted sum rate, and (ii) Γ-step error after 1000 autoregressive rollouts using the same action sequence in both the twin and the real simulator, plus the KL divergence between the induced state distributions. If the Γ-step predicted sum rate deviates by more than 10% from the true value or the rollout state MSE grows with Γ, the twin is not faithful and the convergence-speedup claim is not transferable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central speedup claim (Table III: at least 71.4% fewer physical time slots to converge) and the real-time claim (0.6 ms per configuration, Section V-D) both depend on the digital twin module (Section IV-B) being a sufficiently faithful simulator. The paper never reports any accuracy metric for the twin. The twin is trained on D=1000 samples collected with λ=9 and random UE positions, then used at λ∈{5,10,15} under Gauss-Markov mobility (Section V-A), which is a distribution shift. Worse, training in the digital space unrolls the twin autoregressively for Γ=1000 steps (Algorithm 1, lines 16-29), so any one-step prediction error compounds. The predicted state is high-dimensional (2×(NK+M^2+M^2) with M=32, N=5, K=5, i.e., roughly 4146 real values), yet the model uses only 64-neuron layers and calibrates with ND=24 samples every T1=10 slots, making overfitting and drift likely but unquantified. If the twin's predictions are biased, the digital DeepIOS learns a policy for a wrong environment, and because the physical DeepIOS has no target DNN and cannot train online (Section IV-C), the deployed policy cannot correct itself. The paper's end-to-end comparison shows similar final data rates, which is partial evidence, but without error bars, twin prediction-error curves, or a comparison against a perfect simulator, the 71.4% speedup may largely reflect the Γ=1000 gradient updates per physical slot rather than a genuine digital-twin advantage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a deep reinforcement learning (DRL) framework, DeepIOS, for configuring intelligent omni-surface (IOS) phase-shifts and amplitudes in MU-MIMO systems, and augments it with a supervised-learning digital twin module to move trial-and-error training into a virtual space. The central claims are: (i) DeepIOS achieves higher sum data rates than random and multi-armed bandit (MAB) baselines without sub-channel CSI or UE mobility information; (ii) an action-branch architecture reduces the DNN's action-evaluation complexity from (2L1+1)×L2 to 2L1+1+L2; and (iii) the digital twin module improves convergence speed by at least 71.4% and enables a 0.6 ms per-configuration decision time (Sections V-B and V-D). The framework is formulated as three closed loops connecting the physical MU-MIMO system, a digital twin trained on historical/fresh data, and a digital DeepIOS agent that generates experience samples and delivers trained parameters to the physical controller.","tokens_in":26850,"tokens_out":3405,"duration_ms":34121,"significance":"If the claims hold, the paper offers a useful engineering contribution: a model-free, real-time capable IOS controller that avoids the CSI acquisition bottleneck of traditional optimization and the sample-inefficiency of purely online DRL. The action-branch architecture is a sensible adaptation of dueling/branching DQN to coupled IOS sub-actions, and the closed-loop design with periodic digital-twin calibration is a plausible blueprint for practical deployment. The work also provides a concrete, reproducible simulation setup (Rician channels, Gauss-Markov mobility, ES/MS protocols) and a clear algorithmic pseudocode. However, the empirical support for the two headline claims (convergence speedup and real-time guarantee) is incomplete: the digital twin's predictive accuracy is never measured, no error bars are reported despite 20 simulation runs, and the baselines are limited to random and MAB schemes. These gaps weaken the current evidence but are addressable with additional experiments and analysis, so the central idea remains defensible.","major_comments":[{"comment":"The digital twin's predictive accuracy is never evaluated, yet the convergence-speedup claim (Table III) and the 0.6 ms real-time claim (Section V-D) rest entirely on the assumption that Eqs. (15a)-(15b) faithfully predict next states and rewards. The twin is trained on D=1000 samples collected at λ=9 and random UE positions, then used at λ∈{5,10,15} with Gauss-Markov mobility (Section V-A), which is a distribution shift. Furthermore, training inside the digital space unrolls the twin autoregressively for Γ=1000 steps (Algorithm 1, lines 16-29), so any one-step error compounds. Please report quantitative twin-accuracy metrics (e.g., normalized MSE of next-state and reward predictions on held-out data), a comparison of twin rollouts against the actual simulator over the same horizon, and an ablation that uses the actual simulator (or a perfect twin) as the digital environment to isolate the speedup attributable to the twin rather than to the Γ gradient updates per physical slot.","section":"Section V-B and Section V-D"},{"comment":"The reported results lack statistical uncertainty quantification. The paper states that \"all simulations run 20 times to obtain the average result\" (Section V-A, Metric Setup), but Table III gives single-point convergence times, Figs. 6-7 show only mean curves, and the 71.4% speedup and 94.2% run-time savings are given without confidence intervals. Given the large fluctuations in the data-rate curves, the reader cannot assess whether the observed differences are significant. Please add standard deviations or confidence intervals, and state the convergence criterion used to mark the \"convergence lines\" (e.g., a threshold on the short-term average's slope).","section":"Section V-A and Section V-B"},{"comment":"The baseline set is too weak to support the claim that DeepIOS attains \"a higher data rate\" in an absolute sense. MAB is stateless and random is a trivial lower bound; neither reflects the state of the art for IOS/IRS configuration. Without a stronger reference—such as a perfect-CSI optimization upper bound, an alternating-optimization scheme with full CSI, or a state-of-the-art DRL method (e.g., a meta-critic or an actor-critic with continuous actions)—the data-rate gain could be an artifact of the baselines' limitations rather than a genuine advantage of the proposed state and action design. Please add at least one performance upper bound (e.g., exhaustive search or a block-coordinate-ascent solution with perfect CSI) and one additional learning-based baseline.","section":"Section V-B"},{"comment":"Several hyperparameters are selected on the same simulated test scenario, which makes the reported performance optimistic and limits generalization claims. The penalty factor ω is tuned to 20 based on Table IV, the sub-action sets are chosen based on Fig. 8, and Γ is selected from Fig. 9—all under identical simulation conditions. This circularity means the reported convergence times and data rates are conditional on tuning to this specific configuration. Please either use a separate validation phase (e.g., tune on one Rician factor and test on others) or present a systematic sensitivity analysis showing that the conclusions are stable across a range of hyperparameters.","section":"Section V-C, V-D, V-E and Table II"},{"comment":"Section IV-C and Algorithm 1: the physical DeepIOS module has no target DNN and does not train online, so it cannot correct for a biased or drifting digital twin. The calibration loop (lines 11-14) updates only the twin's parameters; the physical policy is overwritten blindly from the digital space via Eq. (18). If the twin's predictions are systematically wrong (e.g., due to distribution shift or accumulated error), the deployed policy has no feedback mechanism to recover. At minimum, this asymmetry should be discussed as a limitation; better, the paper could include an experiment where the twin is deliberately biased (e.g., trained on mismatched λ or mobility parameters) and show how the physical system's performance degrades with and without recalibration.","section":"Section IV-C"}],"minor_comments":[{"comment":"Typo: \"By dong so\" should be \"By doing so\".","section":"Section IV-C"},{"comment":"The loss in Eq. (12) appears to square the sum of two target-value differences from the two action branches, which is not the standard sum of two MSE losses. Please clarify whether the two branches are trained jointly with the square of the sum or with two separate squared terms (the latter is more typical for action-branching DQN).","section":"Section III-B, Eq. (12)"},{"comment":"The computational-complexity expressions contain unbalanced parentheses and nonstandard GRU complexity (e.g., \"3hi−1h2i\" appears to be a typo for the usual 3(hi−1·hi+hi·hi) or similar). Please re-derive and present the per-layer complexities cleanly, and state clearly whether the analysis counts forward-propagation only, back-propagation, or both.","section":"Section IV-E"},{"comment":"The metric \"convergence time\" is defined only as \"the time slots required for the algorithm to converge,\" without a precise operational criterion. Please specify how convergence is detected (e.g., when the short-term average stays within a band for a certain number of slots), as this directly affects the numerical values in Table III and Fig. 9b.","section":"Section V-A"},{"comment":"The phrase \"the convergence time slots remain unchanged, i.e., 0.00%\" is ambiguous; if the time is unchanged, the relative change is 0%, but the wording is confusing. Please report the actual convergence time values for each sub-action set in addition to percentages.","section":"Section V-C"},{"comment":"The description \"all simulations run 20 times to obtain the average result\" does not state what random seeds are used, how the 20 runs differ (e.g., independent channel realizations vs. different UE trajectories), or whether the plotted curves are ensemble averages or single representative runs. Please specify the random-seed protocol and error-bar computation.","section":"Section V-A"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and proposes a plausible combination of DRL and digital twins for IOS control. The main reasons for major revision are the missing digital-twin accuracy validation, the absence of statistical error bars, and the weak baseline set—these are load-bearing for the paper's two headline claims. I believe the authors can address these concerns within a revision: adding twin-prediction MSE curves, confidence intervals, one stronger baseline, and a validation/tuning split would substantially strengthen the evidence. I would not recommend rejection because the algorithmic framework and simulation study are solid enough to be a useful step toward practical IOS configuration, provided the empirical claims are properly qualified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is a legitimate combination of known pieces—DQN with action branching for IOS phase/amplitude control, plus a supervised-learning digital twin to offload training. The new twist is real: prior DRL works on IOS only did phase-shift; this is the first to do phase and amplitude jointly via DRL, and the first to use a learned simulator for IOS. The system-level idea is sensible and the writing is clear.\n\nWhat it does well: the POMDP formulation is careful, the action branch architecture is a natural complexity reduction, and the complexity analysis is honest—they admit the digital space shifts computation rather than removing it. The simulation setup is fairly detailed (Gauss-Markov mobility, Rician channels, ES/MS protocols) and the comparison to MAB and random is adequate for a first cut.\n\nSoft spots, in order of severity. First and most important: the digital twin is the load-bearing module for both headline claims (71.4% faster convergence, 0.6 ms real-time), yet its predictive accuracy is never measured. The twin is trained on D=1000 samples under λ=9 and random UE positions, then used at λ=5/10/15 with Gauss-Markov mobility and unrolled autoregressively for Γ=1000 steps per physical slot. One-step errors compound; the calibration uses only 24 samples per 10 slots. If the twin is biased, the digital policy is trained on a wrong environment, and the physical DeepIOS has no online learning to correct it. The end-to-end data rate comparison is partial evidence, but without twin error curves or a perfect-simulator control, the speedup might just reflect 1000 gradient updates per physical slot.\n\nSecond: no error bars or variance despite 20 runs. For a paper making a 71.4% claim, that's a real gap. Third: key hyperparameters (ω, Rth, A1, A2, Γ) are selected using the same simulated scenario on which performance is reported, so the numbers are optimistic. Fourth: baselines are weak—random and Thompson Sampling MAB, with no upper bound or optimization-based reference.\n\nNone of these are fatal; they are all fixable. The paper is not overclaiming grossly, and the authors do flag that the digital twin adds computational cost. But as it stands, the central quantitative claims are not robust.\n\nWho it's for: researchers working on DRL-based RIS/IOS control or digital twins for wireless. It's a reasonable conference or journal submission after revision. I'd send it to review, and I'd ask for twin accuracy metrics, error bars, and a perfect-simulator ablation.","headline":"A plausible incremental DRL+digital-twin controller for IOS, but the central speedup claim rests on an unvalidated digital twin and weak baselines.","tokens_in":27330,"tokens_out":2331,"would_cite":false,"duration_ms":20626,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A supervised-learning digital twin lets a deep-reinforcement-learning controller configure intelligent omni-surfaces in 0.6 ms and converge 71% faster than physical-space training.","keywords":["intelligent omni-surface","deep reinforcement learning","digital twin","MU-MIMO systems","real-time configuration","action branch architecture","sum data rate","POMDP"],"falsifier":"Measure the twin's holdout prediction error on next-state and reward as the Rician factor or UE mobility pattern shifts; if the error grows with rollout length or jumps when $\\lambda$ changes, the claimed convergence speedup and 0.6 ms decision time would not survive in a deployed environment.","tokens_in":26217,"feed_emoji":"📶","tokens_out":5958,"duration_ms":50894,"temperature":0.7,"pith_summary":"This paper tries to make intelligent omni-surface (IOS) configuration fast enough for real-time use in multi-user MIMO systems, without requiring perfect channel state information or knowledge of user movement. It argues that deep reinforcement learning alone is too slow because the agent must probe the real wireless channel to learn, and that a digital twin, built as a supervised-learning model of the wireless environment, can absorb that trial-and-error in a virtual space. The proposed scheme, DeepIOS with digital twins, is claimed to find a configuration in 0.6 ms and to converge at least 71.4% faster than training directly in the physical system, while delivering a comparable sum data rate. A sympathetic reader would care because it suggests a path from learning-based surface control to deployment in systems where the channel changes every millisecond.","feed_headline":"Digital twin enables 0.6 ms smart-surface configuration","feed_subtitle":"Deep RL trained in a virtual wireless copy matches data rates and converges 71 percent faster.","key_machinery":"The load-bearing mechanism is the digital twin module, a supervised DNN that maps a state-action pair to a next state and a reward, implemented with a GRU-based state predictor and a ResNet-based reward predictor. It is what lets the digital DeepIOS agent generate a large number of experience samples in a fraction of a physical time slot. The supporting architecture is the action branch design, which gives each sub-action (phase-shift increment and reflecting amplitude) its own output branch, dropping the number of Q-values the DNN must evaluate from $(2L_1+1)L_2$ to $2L_1+1+L_2$.","core_discovery":"The paper claims that a closed-loop framework pairing a deep Q-network controller with a supervised-learning digital twin makes intelligent omni-surface (IOS) configuration practical for real-time multi-user MIMO operation. The twin predicts the next estimated channel state and the sum-rate reward for a given IOS phase and amplitude choice, so the digital agent can gather experience without trial-and-error in the live network; the trained Q-network is then copied to the physical controller, which only executes forward passes. The authors report that this arrangement determines an IOS configuration in 0.6 ms, converges in at least 71.4% fewer time slots than DeepIOS trained directly in the physical system, and achieves a comparable sum data rate under both energy-splitting and mode-selection IOS protocols across Rician factors from 5 to 15. A separate claim is that the action branch architecture, which handles phase-shift and amplitude on independent DQN branches, reduces the action count from $(2L_1+1)L_2$ to $2L_1+1+L_2$ and improves data rate and convergence when action sets are large.","pith_inferences":["If the digital twin retains accuracy under environmental drift, the same training scheme could be lifted to intelligent reflecting surfaces, joint base-station and surface beamforming, or other reconfigurable metasurfaces; the paper only evaluates IOS with zero-forcing precoding at the BS.","The reported 0.6 ms decision time excludes the cost of twin construction and recalibration; since the twin is recalibrated every $T_1=10$ time slots, the end-to-end savings depend on calibration cost, which the paper does not quantify.","A direct test of the paper's logic would be to compare a physical DeepIOS agent with one trained in the twin under a distribution shift in the Rician factor; the paper's own simulations vary $\\lambda$ only across different episodes, not within a single rollout."],"forward_implications":["An IOS controller trained this way can adapt its phase and amplitude settings within one millisecond-class transmission interval, which is the paper's justification for claiming real-time decision-making.","Action branching makes the DNN's per-decision cost grow linearly with the number of phase and amplitude options rather than multiplicatively, which matters for surfaces with many elements.","Because the physical controller does no online learning, most of the computational load moves to an edge server or cloud, which could simplify the IOS hardware.","The closed-loop calibration mechanism implies that operators could pre-train a controller in a digital twin before field deployment and then refine it with live data."],"supporting_citations":[{"why":"Supplies the deep Q-network algorithm that DeepIOS is built on, including fixed Q-targets and experience replay.","marker":"[24]"},{"why":"Provides the DRL-based IRS configuration method and the discrete Fourier transform phase-increment action design that DeepIOS adapts to IOS.","marker":"[3]"},{"why":"Introduces the action branching architecture that DeepIOS uses to separate phase-shift and amplitude decisions.","marker":"[36]"},{"why":"Frames the digital twin concept on which the closed-loop training framework is based.","marker":"[25]"},{"why":"Motivates the supervised-learning approach used to construct the digital twin module from historical transmission data.","marker":"[37]"},{"why":"Defines the IOS model with simultaneous reflection and refraction and the amplitude constraints used in the simulation.","marker":"[11]"},{"why":"Provides the Gauss-Markov mobility model that generates UE movement in the performance evaluation.","marker":"[46]"}],"fun_headline_variants":["Digital twin cuts smart-surface config time to 0.6 ms","0.6 ms: Deep RL with a digital twin configures smart surfaces","Deep RL + digital twin: smart surfaces in 0.6 ms, 71% faster convergence","Digital twin lets deep RL train smart surfaces without real-world trials","Real-time smart-surface control via digital twin and deep RL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The digital twin must predict next states and rewards accurately for configurations it has never seen, so that policies learned in the virtual environment remain effective when copied to the physical system.","fun_headline_variants_meta":{"raw":{"variants":["Digital twin cuts smart-surface config time to 0.6 ms","0.6 ms: Deep RL with a digital twin configures smart surfaces","Deep RL + digital twin: smart surfaces in 0.6 ms, 71% faster convergence","Digital twin lets deep RL train smart surfaces without real-world trials","Real-time smart-surface control via digital twin and deep RL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00094,"raw_usage":{"total_tokens":4084,"prompt_tokens":1076,"completion_tokens":3008,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":692,"completion_tokens_details":{"reasoning_tokens":2911}},"tokens_in":692,"tokens_out":3008,"duration_ms":18545,"temperature":1.0,"reasoning_tokens":2911,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:24:24.035488+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the twin's holdout prediction error on next-state and reward as the Rician factor or UE mobility pattern shifts; if the error grows with rollout length or jumps when $\\lambda$ changes, the claimed convergence speedup and 0.6 ms decision time would not survive in a deployed environment.","supporting_citations":[{"cited_title":"A survey o n digital twin: Definitions, characteristics, applications, and design im plications,","cited_arxiv_id":null,"evidence_quote":"Frames the digital twin concept on which the closed-loop training framework is based."},{"cited_title":"Human-level control through deep reinforcement learnin g,","cited_arxiv_id":null,"evidence_quote":"Supplies the deep Q-network algorithm that DeepIOS is built on, including fixed Q-targets and experience replay."},{"cited_title":"Intelligent reflecting surface con figurations for smart radio using deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the DRL-based IRS configuration method and the discrete Fourier transform phase-increment action design that DeepIOS adapts to IOS."},{"cited_title":"Action branch ing architectures for deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Introduces the action branching architecture that DeepIOS uses to separate phase-shift and amplitude decisions."},{"cited_title":"An empirical compa rison of supervised learning algorithms,","cited_arxiv_id":null,"evidence_quote":"Motivates the supervised-learning approach used to construct the digital twin module from historical transmission data."},{"cited_title":"Intelligent omni-surfaces: Simult aneous refraction and reflection for full-dimensional wireless communicatio ns,","cited_arxiv_id":null,"evidence_quote":"Defines the IOS model with simultaneous reflection and refraction and the amplitude constraints used in the simulation."},{"cited_title":"Fundamentals o f mobility- aware performance characterization of cellular networks: A tutorial,","cited_arxiv_id":null,"evidence_quote":"Provides the Gauss-Markov mobility model that generates UE movement in the performance evaluation."}],"review_version":1}