Pith. sign in

REVIEW 4 major objections 6 minor 11 references

Maximizing the Promptness of Metaverse Systems using Edge Computing by Deep Reinforcement Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that DQN and DDQN can be trained to choose the offloading amount that keeps total latency within the Metaverse user's requirement, thereby keeping a digital twin prompt in a dynamic environment.

desk verdict A small incremental DRL-offloading simulation whose central claim is unsupported by a circular reward-based evaluation. read the letter →

arxiv 2506.02657 v1 pith:6PRQXOGT submitted 2025-06-03 cs.IT cs.LGmath.IT

Tam Ninh Thi-Thanh , Trinh Van Chien , Hung Tran , Nguyen Hoai Son , Van Nhan Vo This is my paper
classification cs.ITcs.LGmath.IT
keywords Metaversedigitaltwindeepreinforcementlearningtaskoffloadingedgecomputing6GDQNDDQN
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that deep reinforcement learning can decide how much sensing data a Metaverse access point offloads to an edge server, keeping the system's digital twin 'prompt' in a dynamic environment. Promptness here means the total time from sensing to delivery stays within the user's latency requirement $t_{\mathrm{Total}} \le t_{\mathrm{Require}}$. The authors write a closed-form total-latency model, frame the offloading decision as a Markov decision process, and train DQN and DDQN agents with a reward of $+20$ when the latency constraint holds and $-1$ when it does not. Simulations with three user devices show all three learning algorithms converging to near-maximum reward while random offloading stays far below, with DDQN converging fastest. The paper concludes that the DRL approach is suitable for ensuring digital-twin promptness under time-varying channels, CPU speeds, and data rates.

What carries the argument

The load-bearing object is the total-latency expression $t_{\mathrm{Total}} = t_{\mathrm{Sensing-Comm}} + t_{\mathrm{Delivery}} + \max\{t_{\mathrm{Local}}, t_{\mathrm{Offloading-ECS}}\}$, where $t_{\mathrm{Sensing-Comm}}$ is the maximum over user devices of sensing plus communication time, and the maximum captures the fact that the access point must wait for whichever processing path, local or edge, finishes last. The decision machinery is a Markov decision process whose action is the offloading amount chosen from the discrete set $\mathcal{W} = \{0, B_{\mathrm{Total}}/F, \dots, B_{\mathrm{Total}}\}$, with the reward function $r = +20$ if $t_{\mathrm{Total}} \le t_{\mathrm{Require}}$ and $r = -1$ otherwise. DQN and DDQN use experience replay and target networks to estimate Q-values, and DDQN separates action selection from value estimation to reduce overestimation bias.

What would settle it

Re-run the converged DDQN policy and record, over many episodes, the fraction of time slots in which $t_{\mathrm{Total}} > t_{\mathrm{Require}}$; if this constraint-violation rate is not near zero even though the training reward is near maximum, then the reward is not a faithful proxy for promptness. A direct alternative is to compare the learned $B_{\mathrm{Offloading}}$ at each state against the exhaustive best split.

Watch

Extended reading notes

Core claim

The central claim is that a learned offloading policy maximizes the promptness of a Metaverse digital twin: the agent observes a state made of total collected bits, edge-server SINR, current total latency, and the two CPU speeds, then picks an offloading amount $B_{\mathrm{Offloading}}$ from a discrete set; the reward is $+20$ if $t_{\mathrm{Total}} \le t_{\mathrm{Require}}$ and $-1$ otherwise. In the reported experiments, DQN, DDQN, and Q-learning all converge to average total rewards near the maximum, whereas random action selection earns an average reward below 10, and DDQN reaches high reward around episode 450 while DQN takes around 650. The paper also shows that rewards grow as the latency requirement $t_{\mathrm{Require}}$ becomes looser and saturate once $t_{\mathrm{Require}}$ exceeds about 1.8 seconds, because the constraint in Eq. (17) is then almost always satisfied.

Load-bearing premise

The claim stands or falls on the assumption that the $+20/-1$ reward rule in Eq. (17) and the latency expression in Eq. (10) it checks faithfully capture what it means for a digital twin to be prompt.

Editorial extensions

If this is right

  • If the claim is right, a Metaverse access point can maintain prompt service without knowing the future trajectory of channels or CPU speeds, because the trained network maps each observed state directly to an offloading split.
  • DDQN is the more practical choice of the two neural methods: it reaches near-maximum reward at roughly 450 episodes, before DQN's roughly 650 and Q-learning's roughly 950.
  • The offloading decision becomes insensitive to the latency requirement once $t_{\mathrm{Require}}$ is loose enough, above about 1.8 seconds, so tightening the requirement below that value is where the learned policy actually matters.
  • Random offloading earns an average reward below 10 against the learned agents' near-20, so the gap provides a baseline showing that the DRL policies are doing real work rather than succeeding trivially.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the near-maximum reward is earned under the same $+20/-1$ rule the agent was trained to maximize, the paper's evidence would be stronger if the converged policy were tested with an independent metric, such as the fraction of slots in which $t_{\mathrm{Total}} > t_{\mathrm{Require}}$.
  • The same architecture could be pointed at a richer objective by replacing the binary reward with a shaped reward proportional to the slack $t_{\mathrm{Require}} - t_{\mathrm{Total}}$; that would let the agent maximize promptness margin, not just pass/fail.
  • The latency expression uses $\max\{t_{\mathrm{Local}}, t_{\mathrm{Offloading-ECS}}\}$, which assumes local processing and edge processing run in parallel; if a real access point serializes them, the learned offloading policy would be optimistic.
  • The single-access-point formulation invites a multi-agent extension, but simultaneous offloading from several access points would change the interference term $I$ in the SINR, so the current reward and state space would need to be coupled across agents.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a deep reinforcement learning approach for task offloading in a Metaverse digital twin system. A central node (MVAP) collects sensing data from multiple user devices and decides how many bits to offload to an edge computing server; the objective is to keep total latency below a user requirement (promptness). The authors formulate the problem as an MDP with a five-component state, a finite offloading action set, and a threshold reward (+20 if the latency constraint is met, -1 otherwise). They implement DQN, DDQN, and Q-learning baselines plus a random-action baseline, and report convergence curves and reward versus latency-requirement curves. The central claim is that the proposed DRL algorithm is suitable for ensuring digital twin promptness in a dynamic environment.

Significance. If the empirical claim were fully supported, the paper would be a modest application-level contribution: it applies standard DRL algorithms (DQN/DDQN) to an offloading problem in a Metaverse/edge-computing setting, building on the architecture of [9]. The latency model in Eqs. (1)-(11) is transparent and the algorithms are standard, which makes the paper easy to follow. However, the significance is limited by the lack of algorithmic novelty and, more importantly, by the fact that the reported evaluation does not currently establish the central claim. The work would be more significant if it provided an independent, quantitative validation that the learned policy satisfies user latency requirements under held-out dynamic conditions, and if it clarified the observability and reproducibility of the system model.

major comments (4)
  1. [Section IV, Fig. 2, Eq. (17)] The evaluation of the central claim is in-sample and partly circular. The reward in Eq. (17) is defined as +20 when tTotal ≤ tRequire and -1 otherwise, which is exactly the indicator of the promptness constraint in Eq. (12). Since DQN and DDQN are trained to maximize this reward, Fig. 2 mainly shows that the agents learned their training objective; it does not independently demonstrate that the learned offloading policy keeps the digital twin prompt on new realizations. The paper should report held-out constraint satisfaction rates, the distribution of tTotal, and a comparison with the optimal offloading action computed by exhaustive enumeration over the finite action set (or with the baseline in [9]). Without such an external metric, the abstract's claim that the algorithm is 'suitable for offloading tasks to ensure the promptness of DT' is not supported.
  2. [Section III-A.1, Eq. (14)] The state space in Eq. (14) omits tRequire, although the reward in Eq. (17) depends directly on tRequire and Fig. 3 varies this quantity. If tRequire is not part of the observation, the agent cannot condition its offloading decision on the actual latency requirement; at best it can maximize expected reward over the distribution of requirements. If tRequire is assumed known to the MVAP, it should be included in the state S; if it is unknown, the claim that the policy 'ensures promptness' for varying user requirements needs to be relaxed or justified. This is a load-bearing modeling issue because the entire objective is defined through tRequire.
  3. [Section II, Eq. (4), and Table II] Several parameters needed to reproduce the simulation are missing or inconsistent. Eq. (4) uses μLoS, βLoS, ν, and δ, but Table II does not list values for these quantities; Table II instead lists β0 = 10^{-6}, which is not defined in the text. The bandwidths Wn and WMVAP in Eqs. (3) and (7) are also never specified. In addition, fMVAP is listed as '∼ 10.5 GHz' while the text defines it as Gaussian with mean μfMVAP and variance σ^2_fMVAP, and the interference term I is given as 3 without units. These omissions prevent the numerical study from being verified and make the reported convergence and reward values non-reproducible. The dimensional consistency of h_n in Eq. (4) should also be clarified when βLoS differs from 2.
  4. [Section IV, Figs. 2 and 3] The experimental comparison is statistically weak. Fig. 2 and Fig. 3 appear to be single-run curves with no error bars, no multiple random seeds, and no evaluation on test episodes. The baselines are also limited: the random-action baseline gives an expected reward close to zero, and Q-learning with a discretized state space is not a strong competitor for this problem. The paper should report means and standard deviations over multiple seeds, and should compare against a simple threshold-satisfying policy or the optimal offloading action computed by exhaustive search over the 1001 possible actions. Without such comparisons, the claim that DDQN 'converges faster' or is 'more suitable' than DQN is not established beyond the specific training curves shown.
minor comments (6)
  1. [Abstract and Introduction] There are several typographical and grammatical issues, such as 'attraction' for 'attention', 'Metaveser' for 'Metaverse', and inconsistent spacing in 'MV AP'. A careful proofreading pass is needed.
  2. [Section II-E, Eq. (13)] The distribution of tRequire_j is not defined in the model section. Table II later states a uniform distribution, but this should be stated where tRequire is introduced.
  3. [Section IV, Fig. 3] The horizontal axis label in Fig. 3 reads 'requirement of latency of MU' without units; if the quantity is seconds, this should be stated. The text also says tRequire ranges from 1.4 to 2.3 seconds while Table II gives U[1.5, 2.3], and this inconsistency should be resolved.
  4. [Algorithms 1 and 2] The equations referenced in Algorithms 1 and 2 as Eq. (18) and Eq. (19) appear only after the algorithm listings. The equations should be numbered before their first reference, and the pseudo-code should make the terminal-state condition explicit.
  5. [Table I] Table I duplicates state and reward notation (si, st, ri, rt, etc.) and includes both 'at ith step' and 'at ith episode', which is confusing. The table should be simplified to one consistent set of symbols.
  6. [Section III-B] The description of the neural network output layer states that La4 contains F neurons for F = 1000 actions, but if the action set A depends on BTotal, the mapping from network outputs to BOffloading values should be described more precisely, especially because BTotal changes over time.

Circularity Check

1 steps flagged · score 7.0 of 10

The central evaluation is self-definitional: the reward function in Eq. (17) is exactly the promptness constraint in Eq. (12), so the near-maximum rewards in Fig. 2 only demonstrate that DQN/DDQN learned to optimize their own objective, not that digital-twin promptness is ensured.

  1. self definitional [Section II.E Eq. (12); Section III.A.3 Eq. (17); Section IV Fig. 2]
    "The reward structure is designed to ensure compliance with the user latency requirements outlined in (12), which specifies that the total latency should not exceed the defined threshold for Metaverse user devices. ... r = ( Positive− value if tTotal ≤ tRequire Negative− value if tTotal > tRequire. (17) ... Fig. 2 illustrates that the DDQN, DQN, and QL algorithms converge to maximum reward values."

    Eq. (12) defines promptness as tTotal ≤ tRequire. The reward in Eq. (17) is +20 exactly when this inequality holds and -1 otherwise, so the reward is the promptness condition itself. DQN/DDQN are trained to maximize this reward, and the paper's central empirical evidence, Fig. 2, plots the resulting reward. Hence near-maximum rewards merely show the agent optimized its own training objective; they do not independently demonstrate that the digital twin remains prompt. No external metric (e.g., constraint-violation rate, mean/percentile of tTotal, distribution of BOffloading, or exhaustive enumeration of the 1001 actions per state) breaks the self-reference, so the conclusion 'suitable ... to ensure the promptness of DT' is supported only by the definition of the evaluation function.

full rationale

The paper's only quantitative evidence for the headline claim is Fig. 2, which plots the total reward during training. The reward in Eq. (17) is chosen so that the agent receives +20 precisely when tTotal ≤ tRequire (Eq. (12)) and -1 otherwise; this is exactly the promptness criterion defined in Section II.E. DQN and DDQN are trained to maximize this reward, so convergence to reward near 20 is an in-sample check that the Q-network learned its own objective. It is not an independent measurement of promptness: the paper does not report constraint-violation rates, the distribution of tTotal, the chosen BOffloading values, or a comparison with the optimal offloading decision (which is computable for the finite F=1000 action set). Fig. 3 further shows rewards rise with tRequire simply because Eq. (17) is easier to satisfy, confirming that the metric is the threshold itself. The model equations are not circular, and there is no load-bearing self-citation, but the empirical claim reduces to the definition of the reward, so the score is high.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim depends on an MDP formulation, a LoS-only channel model, stochastic CPU distributions, a hand-specified Markov transition matrix, and a hand-tuned binary reward. No new physical entities are introduced. The main free parameters are the reward values and action discretization.

free parameters (4)
  • Positive reward value = 20
    Chosen by hand in Eq. (17) to encourage learning; the paper notes negative=-1 must be far from positive. The simulation results depend on this ratio.
  • Negative reward value = -1
    Chosen by hand in Eq. (17); combined with +20 defines the objective that the algorithms maximize.
  • Task-splitting factor F = 1000
    Action space discretization; F=1000 chosen for the outcome layer in the DQN/DDQN networks (Section III-B).
  • SINR Markov transition matrix = 5x5 matrix (0.60 diagonal, 0.25-0.04 off-diagonal)
    Hand-specified transition probabilities for gamma_ECS (Section IV), not derived from measurements.
assumptions (6)
  • domain assumption The system is modeled as an MDP <S,A,Pr,R> where the state includes BTotal, gamma_ECS, tTotal, f_MVAP, f_ECS.
    Section III-A; the Markov property is assumed for the offloading decision, and tTotal is included in the state even though it is the previous decision's outcome.
  • domain assumption Only LoS links exist between MVD and MVAP; channel gain follows Eq. (4) with Rice fading and undefined parameters mu_LoS, beta_LoS, nu, delta.
    Section II-A; this channel model underpins the communication delay in Eqs. (2)-(3).
  • domain assumption The total latency is the sum of sensing-communication, delivery, and max of local and offloaded processing (Eq. 10).
    Section II-D; adopted from [9], assumes MVAP can process locally and offload in parallel after receiving all data.
  • domain assumption f_MVAP and f_ECS follow Gaussian distributions with variances 0.1 times their means.
    Section III-A; stochastic CPU capacities are claimed to follow normal distributions, with no empirical justification.
  • domain assumption gamma_ECS evolves as a Markov chain with the hand-specified transition matrix Gamma_ECS.
    Section III-A/IV; the transition probabilities are chosen, not measured.
  • ad hoc to paper The reward in Eq. (17) with +20/-1 is a valid proxy for promptness.
    Section III-A3; the paper 'empirically' chooses these values; the entire evaluation rests on this reward encoding.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Maximizing the Promptness of Metaverse Systems using Edge Computing by Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/6PRQXOGT

@misc{pith2026250602657,
  author       = {Pith},
  title        = {Pith review of: Maximizing the Promptness of Metaverse Systems using Edge Computing by Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6PRQXOGT}},
  note         = {Machine review of arXiv:2506.02657}
}
read the original abstract

Metaverse and Digital Twin (DT) have attracted much academic and industrial attraction to approach the future digital world. This paper introduces the advantages of deep reinforcement learning (DRL) in assisting Metaverse system-based Digital Twin. In this system, we assume that it includes several Metaverse User devices collecting data from the real world to transfer it into the virtual world, a Metaverse Virtual Access Point (MVAP) undertaking the processing of data, and an edge computing server that receives the offloading data from the MVAP. The proposed model works under a dynamic environment with various parameters changing over time. The experiment results show that our proposed DRL algorithm is suitable for offloading tasks to ensure the promptness of DT in a dynamic environment.

Figures

Figures reproduced from arXiv: 2506.02657 by the authors.

Figure 1
Figure 1. Model architecture C. Edge Computing Server (ECS) We denote γECS = gpOffloading I+σ as SINR at the ECS, where the offloading transmitting power, channel gain between ECS and MVAP, the interference power caused by the shared channel between other communication systems and MVAP, and additive thermal noise at ECS are p Offloading , g, I, and σ, respectively. We also denote the data rate that MVAP achieved when choosing… view at source ↗
Figure 2
Figure 2. illustrates that the DDQN, DQN, and QL algorithms converge to maximum reward values. The rewards of these algorithms are remarkably more elevated than the reward received by the RM. Additionally, DDQN converges faster than DQN at around 450 and 650 episodes due to decoupling action selection and evaluation tasks. Q-learning converges after around 950 episodes. It is shown that the Q-table of ones is updated extremel… view at source ↗
Figure 3
Figure 3. The requirement of latency of MU scenarios involving multiple MVAPs, where advanced learning algorithms like multi-agent DRL might prove beneficial. REFERENCES [1] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018. [2] R. Ali, I. Ashraf, A. K. Bashir, and Y. B. Zikria, “Reinforcement￾learning-enabled massive internet of things for 6g wireless communi￾cations,” IEEE Communications… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

11 extracted references · 8 canonical work pages

  1. [9]

    Dynamic offloading for edge computing-assisted metaverse systems,

    N. T. Hoa, L. V . Huy, B. D. Son, N. C. Luong, and D. Niyato, “Dynamic offloading for edge computing-assisted metaverse systems,” IEEE Communications Letters , vol. 27, no. 7, pp. 1749–1753, 2023

    work page 2023

  2. [1]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction . MIT press, 2018

    2018

  3. [2]

    Reinforcement- learning-enabled massive internet of things for 6g wireless communi- cations,

    R. Ali, I. Ashraf, A. K. Bashir, and Y . B. Zikria, “Reinforcement- learning-enabled massive internet of things for 6g wireless communi- cations,” IEEE Communications Standards Magazine , vol. 5, no. 2, pp. 126–131, 2021

    work page 2021

  4. [3]

    Q-learning based reinforcement learning approach for lane keeping,

    A. Feher, S. Aradi, and T. Becsi, “Q-learning based reinforcement learning approach for lane keeping,” in 2018 IEEE 18th International Symposium on Computational Intelligence and Informatics (CINTI) . IEEE, 2018, pp. 000 031–000 036

    work page 2018

  5. [4]

    A reinforcement learning-based adaptive path tracking approach for autonomous driving,

    Y . Shan, B. Zheng, L. Chen, L. Chen, and D. Chen, “A reinforcement learning-based adaptive path tracking approach for autonomous driving,” IEEE Transactions on Vehicular Technology, vol. 69, no. 10, pp. 10 581– 10 595, 2020

    work page 2020

  6. [5]

    Digital twin networks: A survey,

    Y . Wu, K. Zhang, and Y . Zhang, “Digital twin networks: A survey,”IEEE Internet of Things Journal , vol. 8, no. 18, pp. 13 789–13 804, 2021

    work page 2021

  7. [6]

    Digital twin in industry: State-of-the-art,

    F. Tao, H. Zhang, A. Liu, and A. Y . Nee, “Digital twin in industry: State-of-the-art,” IEEE Transactions on industrial informatics , vol. 15, no. 4, pp. 2405–2415, 2018

    2018

  8. [7]

    Digital twin in the iot context: A survey on technical features, scenarios, and architectural models,

    R. Minerva, G. M. Lee, and N. Crespi, “Digital twin in the iot context: A survey on technical features, scenarios, and architectural models,” Proceedings of the IEEE , vol. 108, no. 10, pp. 1785–1824, 2020

    work page 2020

Show all 11 references
  1. [8]

    Digital twins from a networking perspective,

    M. Vaezi, K. Noroozi, T. D. Todd, D. Zhao, G. Karakostas, H. Wu, and X. Shen, “Digital twins from a networking perspective,” IEEE Internet of Things Journal , vol. 9, no. 23, pp. 23 525–23 544, 2022

    2022

  2. [10]

    Deep reinforcement learning with double q-learning,

    H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, 2016

    2016

  3. [11]

    Optimizing communication and computation for multi-uav in- formation gathering applications,

    M. Thammawichai, S. P. Baliyarasimhuni, E. C. Kerrigan, and J. B. Sousa, “Optimizing communication and computation for multi-uav in- formation gathering applications,” IEEE Transactions on Aerospace and Electronic Systems, vol. 54, no. 2, pp. 601–615, 2018

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.