{"id":"a5358485-13e7-4e1f-8eb6-a60f1efd1624","arxiv_id":"2506.02458","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"TD3, a standard reinforcement learning algorithm, achieves higher average reward than DDPG in a simulated three-user mobile edge computing offloading task with moving users.","lead":"This paper replaces the DDPG deep reinforcement learning algorithm with TD3 in a mobile edge computing offloading simulator, where each user decides how much computation to process locally versus upload. Its reported result is that TD3 achieves better average reward than DDPG in a three-user simulation, especially when users move.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"One unseeded run and a Table I that conflicts with several qualitative claims leave the TD3-over-DDPG superiority unestablished; multi-seed replication with explicit per-user or aggregate reporting is needed.","rationale":"The reader's weakest assumption identifies the core issue: a single unseeded run with fixed hyperparameters cannot establish a robust algorithmic superiority claim. I agree with that, and I would add that the paper's own Table I makes the problem more acute: the per-user entries for User 3 power and User 1 delay contradict the unqualified statements in Section IV-B. The aggregate numbers do favor TD3 on all three metrics when averaged over users, so the central reward-based claim may survive a fair test, but the paper currently neither states an aggregate interpretation nor provides variance information. Because the reported reward advantage is large and the primary comparison is consistent with the stated claim, the appropriate disposition remains the reader's CONDITIONAL verdict rather than REJECT: the authors should supply seeded runs, full hyperparameters, a definition of average delay, and corrected per-metric wording.","tokens_in":7980,"tokens_out":5818,"duration_ms":58174,"concrete_test":"Re-run the Section IV-A environment for DDPG and TD3 with at least 10 independent random seeds, identical hyperparameters, and the same mobility process; report per-user mean and standard deviation (or confidence intervals) for reward, power, and delay, and state explicitly whether superiority is claimed per user or only in aggregate. If TD3's reward or power advantage is not statistically significant, or if the per-user contradictions in Table I persist, the headline superiority claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing condition is that the single unseeded comparison in Section IV-B is representative and supports the per-metric superiority statements. That condition is not met. Table I itself contains entries that conflict with the text: for User 3, average power is 0.30 for TD3 versus 0.20 for DDPG, yet Section IV-B states that 'regarding average power per episode, TD3 also achieved better results'; for User 1, average delay is 4.67 for TD3 versus 1.95 for DDPG, while the surrounding text claims better delay performance. If the intended claim is aggregate over users, the paper never says so and provides no error bars or standard deviations. Hyperparameters are also incomplete: Section IV-A says networks use 'almost the same settings as in [7]' but omits learning rates, minibatch size, target-network update delay, and layer sizes, and the 'average delay' metric reported in Table I is never defined (the reward uses queue length, not delay). With one run, fixed w_m=0.8, and a random-walk mobility process, the observed reward gap (mean about -3.6 versus -14.4) cannot be separated from run-to-run variance or from the specific weighting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper applies Twin Delayed DDPG (TD3) to the decentralized multi-user mobile edge computing (MEC) problem studied by [7], with the added feature that users are portable and their distances to the base station change over time. Each user is an independent DRL agent with state comprising queue length, estimated zero-forcing power ratio, and channel vector; the action is the continuous local-execution and offloading power pair; and the reward is a negatively weighted sum of power consumption and queue length. The paper reports a simulation with three users over 2000 episodes, comparing TD3 against DDPG and claiming that TD3 achieves better average reward, average power, and average delay. The contribution is entirely empirical: no theoretical analysis is offered, and the central claim rests on the reported experimental comparison.","tokens_in":8225,"tokens_out":6184,"duration_ms":56947,"significance":"If the reported advantage were actually established, the paper would offer a simple, practical improvement over DDPG for decentralized MEC with moving users, and the system model in Section II is coherent and closely follows a known benchmark. Credit is due for providing explicit pseudocode for both DDPG and TD3 and for specifying the simulation setup in some detail. However, the significance is severely limited by the fact that the only evidence is a single unseeded run, the numerical table contradicts the qualitative text, and key metrics and hyperparameters are left undefined. As it stands, the central claim is not supported to the standard expected for an empirical comparison.","major_comments":[{"comment":"The paper's central claim that TD3 'has much better results than DDPG' (Abstract) and 'exceeded the conventional DDPG-based power control strategy' is based on a single unseeded run per algorithm. There are no standard deviations, random seeds, confidence intervals, or statistical tests. With only three users and one realization of the random channel, task, and mobility processes, the observed reward gap (mean about -3.6 versus -14.4) cannot be separated from run-to-run variance. The authors should report multiple independent seeds with per-user and aggregate means and standard deviations, and ideally a paired statistical test, or temper the superiority claim accordingly.","section":"Section IV-A and IV-B"},{"comment":"Table I contradicts the qualitative statements in Section IV-B. For User 3, the average power for TD3 is 0.30 versus 0.20 for DDPG, meaning DDPG is better on this metric, yet the text states that 'regarding average power per episode, TD3 also achieved better results compared with DDPG.' Similarly, for User 1, the average delay for TD3 is 4.67 versus 1.95 for DDPG, while the surrounding text implies better delay performance. If the intended claim is only about the average over users or only about some users, that is not stated. The table and the qualitative summary must be reconciled.","section":"Table I and Section IV-B"},{"comment":"Table I reports 'Average delay' but no definition of this metric is given in Section IV-A or elsewhere. The reward function (17) uses the queue length B_m(t), not delay; Section III-A mentions Little's theorem, but the exact empirical quantity reported in the table (mean queue length over time, steady-state queue length, or an actual delay) is never specified. Without this definition, the delay column is uninterpretable and cannot be used to support the superiority claim.","section":"Table I and Section IV-A"},{"comment":"The TD3 pseudocode computes the target next action as clip(μ(s'|θ^μ) + clip(ε,-c,c), a_low, a_high), using the current actor parameters θ^μ. In standard TD3, the target action must be computed with the target actor parameters θ^{μ'}, which are defined in line 1 but are never used in line 14. If the implementation follows this pseudocode, the algorithm is not TD3, and the reported improvement over DDPG could be an artifact of a different exploration and update scheme. This must be corrected, and if the experiments were run with the incorrect version, they should be rerun.","section":"Algorithm 2, line 14"},{"comment":"The simulation setup is incomplete with respect to the DRL hyperparameters. The text says the networks use 'almost the same settings as in [7]' and cites [17] for initialization, but it omits the learning rates α_Q and α_μ, the minibatch size, the soft-update rate τ, the policy-delay parameter update_every, the number of gradient updates per step, and the layer sizes, widths, and activations. The mobility model is also only sketched: distances are 'added by a value drawn randomly from the standard normal distribution, such that the absolute value of the cumulative sum of these distances is lower than 10.' These omissions prevent reproduction and make it impossible to judge whether the result is robust to hyperparameter choices, which is critical because the whole claim rests on one comparison.","section":"Section IV-A"}],"minor_comments":[{"comment":"The summation index is j, but the channel, power, and data symbol inside the sum are all indexed by m; in addition, the noise term n(t) appears inside the summation. This should be h_j(t), p_{o,j}(t), and s_j(t), with n(t) outside the sum.","section":"Equation (1)"},{"comment":"The text 'variances 2 = 1' is garbled; it should state, for example, that E[|s_m(t)|^2] = 1.","section":"Section II-A"},{"comment":"The symbol γ is used both for SINR in (4) and for the discount factor in (15) and in the algorithms. These two uses should be distinguished to avoid confusion.","section":"Section III"},{"comment":"The symbol B is used both for the task buffer queue length in (5) and for the experience replay buffer in Algorithms 1 and 2. The notation should be changed.","section":"Algorithms 1 and 2"},{"comment":"The figures are described in terms of orange and black lines, but no legend or axis-label description is provided in the text; including legends and defining what is averaged (over episodes or over users) would improve readability.","section":"Section IV-B and Figures 3-5"}],"recommendation":"major_revision","confidential_remarks":"The empirical core is under-powered and internally inconsistent: the table contradicts the text on two metrics, the delay metric is undefined, and the TD3 pseudocode appears incorrect. I would not consider accepting without a full rerun with multiple seeds, a corrected algorithm definition, and a complete reporting of hyperparameters and metrics. The contribution over [7] is incremental, so the bar for evidence should be higher."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one-sentence take: this is a routine empirical comparison of TD3 against DDPG in the MEC setup of [7], with the only new elements being a random-walk mobility model and a claim that TD3 wins. That claim may be true, but the evidence as reported is not enough to establish it.\n\nWhat the paper does well: the system model and the two algorithms are described clearly, and the match to [7] is explicit, so a reader can reconstruct the setup without going back to the original. The result that TD3 gives a better average reward for all three users is at least directionally consistent with what TD3 is designed to do. The paper also honestly says it is building on prior work; it does not oversell the algorithmic novelty.\n\nThe soft spots are real and load-bearing. Section IV-B states that TD3 achieved better average power and better average delay, but Table I contradicts that for User 3 (power 0.30 vs 0.20, worse for TD3) and for User 1 (delay 4.67 vs 1.95, worse for TD3). If the intended claim is aggregate-over-users, the paper never says so. More importantly, the results are from a single unseeded run with one hand-chosen reward weight (w_m=0.8). No standard deviations, no seeds, no confidence intervals. Hyperparameters are partially taken from [7] but the learning rates, minibatch size, target update delay, and layer sizes are not given. The 'average delay' metric in Table I is never defined; the reward uses queue length, not delay. With one run, the observed reward gap (about -3.6 vs -14.4) cannot be separated from run-to-run variance or from the particular weight choice.\n\nThe novelty is also thin: TD3 is a standard algorithm, the system model is from [7], and the simulation has only three users. So this is not a paper that opens a new direction. It is a small empirical data point.\n\nWho is this for? A reader who wants a quick, informal check of whether TD3 might work in this decentralized offloading setting. That reader should treat the comparison as preliminary.\n\nMy recommendation: engage with it, but require revision. A serious referee should ask for multiple seeds, error bars, a defined delay metric, full hyperparameter disclosure, a corrected Table I or corrected text, and ideally code. With those fixes the paper could be a modest but honest contribution. Send it to peer review; it deserves referee time to force the experimental rigor.","headline":"A routine TD3-vs-DDPG benchmark in a borrowed MEC model, with a plausible reward claim that the reported single unseeded run does not actually support.","tokens_in":8779,"tokens_out":1550,"would_cite":false,"duration_ms":17098,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Twin Delayed DDPG outperforms DDPG as a decentralized offloading policy in multi-user MEC.","keywords":["mobile edge computing","computation offloading","deep reinforcement learning","Twin Delayed DDPG","Deep Deterministic Policy Gradient","decentralized control","power allocation","portable mobile users"],"falsifier":"Run the exact Table I setup (three users, $w_m = 0.8$, 2000 episodes, 200 steps per episode) with ten or more independent random seeds for both algorithms, and compare distributions of per-user average reward, power, and delay. If TD3's per-user reward advantage does not persist with non-overlapping confidence intervals, or if average power is not lower for the majority of users, the paper's superiority claim is falsified.","tokens_in":7768,"feed_emoji":"📡","tokens_out":5893,"duration_ms":51883,"temperature":0.7,"pith_summary":"This paper argues that a specific deep reinforcement learning algorithm, Twin Delayed DDPG (TD3), can replace Deep Deterministic Policy Gradient (DDPG) as the per-user learning engine in a decentralized mobile edge computing (MEC) system. In the setup, each mobile user independently decides how much power to spend on local computation and how much to spend on offloading tasks to a base station-attached MEC server, using only local observations such as buffer queue length, channel vector, and a power-ratio estimate. The paper simulates three portable users whose distances to the base station change randomly, and reports that TD3-learned policies achieve higher average reward for all three users compared with DDPG, with the overall performance read as superior to the DDPG-based strategy. If the comparison holds, the result matters because it suggests that standard improvements to actor-critic reinforcement learning transfer to decentralized offloading, making MEC systems more scalable and energy-efficient without requiring global state sharing.","feed_headline":"Twin Delayed DDPG beats DDPG for decentralized MEC offloading","feed_subtitle":"Simulation with portable users shows TD3 learns higher-reward offloading policies from local observations alone.","key_machinery":"The central object is the Twin Delayed DDPG (TD3) algorithm, an actor-critic deep reinforcement learning method that maintains two Q-value networks and uses the smaller of their target values when computing Bellman errors, updates the policy less frequently than the critics, and adds clipped noise to target actions. It operates within a per-user reward $r_{m,t} = -w_{m,1}(p_{l,m}(t)+p_{o,m}(t)) - w_{m,2}B_m(t)$ and a state $s_{m,t} = [B_m(t), \\phi_m(t-1), h_m(t)]$, where $B_m(t)$ is the task buffer queue length, $\\phi_m(t-1)$ is the estimated power ratio after zero-forcing detection, and $h_m(t)$ is the user's channel vector. This machinery carries the argument by giving each user a more stable critic target, mitigating the overestimation that the paper identifies as DDPG's weakness, and thereby producing the reported reward gains in the portable-user simulations.","core_discovery":"On the paper's own terms, the central discovery is that in the decentralized multi-user MEC framework introduced in [7], replacing the DDPG agent with TD3 yields better learned computation offloading policies, especially when mobile users are portable. Concretely, the reported per-user average rewards improve from -9.52 to -3.09, from -14.08 to -3.71, and from -19.49 to -3.89 while users move; average power decreases for two of the three users and average delay decreases for two of the three users. The paper attributes the improvement to TD3's remedies for DDPG's Q-value overestimation, namely clipped double-Q learning and delayed policy updates, which stabilize training and lead to higher-reward policies.","pith_inferences":["If the reported advantage is robust to random seeds and hyperparameter choices, the same TD3 recipe (clipped double-Q targets and delayed policy updates) could be applied to other decentralized resource-allocation problems in MEC, such as channel assignment or split inference, without changing the local-state architecture.","The reward weight $w_m$ is fixed at 0.8 for all users; varying it would trace the energy-delay Pareto frontier, and TD3's edge over DDPG may depend on the operating point, which the paper does not explore.","A natural extension is to test the same decentralized learning rule with more than three users, since the scalability argument in the paper is not yet backed by simulations with larger $M$."],"forward_implications":["Each portable user can learn an offloading policy independently from its own local state, without knowing the total number of users or the statistics of task arrivals and channels.","Replacing DDPG with TD3 improves the average reward per episode for all three simulated users, and the paper interprets the overall results as showing that TD3 exceeds the DDPG-based power control strategy.","The learned policies operate with finite feedback: each user observes only its buffer queue, the estimated power ratio after zero-forcing detection, and its own channel vector.","Because TD3 uses two Q-networks and delayed policy updates, its runtime stays close to DDPG's while reducing Q-value overestimation, making it a practical drop-in replacement in this simulator."],"supporting_citations":[{"why":"Provides the decentralized DRL offloading framework and the DDPG baseline that the paper extends.","marker":"[7]"},{"why":"Supplies the DDPG algorithm and the neural-network initialization settings used for both agents.","marker":"[17]"},{"why":"Little's theorem justifies using the task buffer queue length as the buffering-delay term in the per-user reward.","marker":"[16]"},{"why":"Motivates zero-forcing detection as a low-complexity MIMO receiver for multi-user uplink transmission.","marker":"[11]"},{"why":"Supports the use of zero-forcing detection in large antenna-array MIMO systems.","marker":"[12]"}],"fun_headline_variants":["TD3 outperforms DDPG for decentralized MEC offloading","Twin Delayed DDPG beats DDPG in MEC offloading","Decentralized offloading improved with TD3 agent","TD3 yields higher rewards in mobile edge offloading"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's conclusion that TD3 is better rests on a single set of simulation runs with one reward weighting, one movement pattern, and unstated random seeds; if the reported gaps shrink or flip under different seeds or hyperparameters, the superiority claim collapses.","fun_headline_variants_meta":{"raw":{"variants":["TD3 outperforms DDPG for decentralized MEC offloading","Twin Delayed DDPG beats DDPG in MEC offloading","Decentralized offloading improved with TD3 agent","TD3 yields higher rewards in mobile edge offloading"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1411,"prompt_tokens":897,"completion_tokens":514,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":442}},"tokens_in":513,"tokens_out":514,"duration_ms":4651,"temperature":1.0,"reasoning_tokens":442,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:22:52.035397+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact Table I setup (three users, $w_m = 0.8$, 2000 episodes, 200 steps per episode) with ten or more independent random seeds for both algorithms, and compare distributions of per-user average reward, power, and delay. If TD3's per-user reward advantage does not persist with non-overlapping confidence intervals, or if average power is not lower for the majority of users, the paper's superiority claim is falsified.","supporting_citations":[{"cited_title":"Decentralized computation offloading for multi-user mobile edge computing: a deep reinforcement learning approach.EURASIP J","cited_arxiv_id":null,"evidence_quote":"Provides the decentralized DRL offloading framework and the DDPG baseline that the paper extends."},{"cited_title":"Prasobh Sankar and Sundeep Prabhakar Chepuri","cited_arxiv_id":null,"evidence_quote":"Motivates zero-forcing detection as a low-complexity MIMO receiver for multi-user uplink transmission."},{"cited_title":"Larsson, and Thomas L","cited_arxiv_id":null,"evidence_quote":"Supports the use of zero-forcing detection in large antenna-array MIMO systems."}],"review_version":1}