Pith. sign in

REVIEW 4 major objections 4 minor 29 references

A Lyapunov Drift-Plus-Penalty Method Tailored for Reinforcement Learning with Queue Stability

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper derives a reward function that makes Lyapunov drift-plus-penalty optimization compatible with reinforcement learning while preserving queue stability.

desk verdict The proposed reward is a plausible heuristic, but the proof that it is the theoretically justified Lyapunov-RL adaptation is invalid, so the paper should not be published as a theory contribution. read the letter →

arxiv 2506.04291 v1 pith:SLSELTFJ submitted 2025-06-04 cs.LG

Wenhan Xu , Jiashuo Jiang , Lei Deng , Danny Hin-Kwok Tsang This is my paper
classification cs.LG
keywords Lyapunovdrift-plus-penaltyreinforcementlearningqueuestabilityrewarddesignmobileedgecomputingbackpressureroutingonlineoptimizationmeanrate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the right way to feed Lyapunov drift-plus-penalty objectives into a reinforcement learning agent is to use the symmetrized reward $R(t)=-\frac{1}{2}\sum_n\big(Q_n^2(t+1)+Q_n^2(t)\big)-Vp(t)$ rather than the usual per-slot drift difference. The authors start from the standard Lyapunov problem, observe that any increasing reshuffling of its objective gives an equivalent optimization, and force compatibility with RL's long-horizon view by demanding that the reshuffling preserve the optimal actions in a greedy special case. This pins the reshuffling down to an additive constant, and a time-symmetry argument fixes the queue term, producing Eq. (38). If the derivation is right, an RL agent can keep queues stable while learning policies for complex, non-convex penalties that greedy Lyapunov methods handle poorly. Simulations in mobile edge computing and network routing report faster convergence and lower queue variance than direct Lyapunov-RL hybrids.

What carries the argument

The load-bearing object is the symmetrized reward form (Eq. (38)), in which the squared queue length enters at both $t$ and $t+1$, rather than only as a difference. Two lemmas carry the argument: Lemma 1 says that Eq. (29), the additivity condition forced by matching the long-horizon greedy case, holds only for affine per-slot transforms $f_{1,t}(x)=x+a_{0,t}$; Lemma 2 says swapping the queue lengths at $t$ and $t+1$ should not change the reward, which pins the additive queue term to $\sum_n Q_n^2(t)+a_0$. Together they convert the unknown monotone transform into an additive constant and then into the explicit squared-queue term, yielding the final reward.

What would settle it

Take the special case used in the derivation, set $V=1$, $p(0)=p(1)=1$, $\tau=2$, and try the non-affine per-slot map $f_{1,t}(x)=x^2$ for both slots. Equation (29) then demands $f_{1,0}(2)=f_{1,0}(1)+f_{1,1}(1)$, i.e. $4=2$, which fails; checking whether such a failure can occur while the two long-horizon problems share the same optimal actions would settle whether the reward formula follows from the stated assumptions.

Watch

Extended reading notes

Core claim

The central claim is that the reward $R(t)=-\frac{1}{2}\sum_n\big(Q_n^2(t+1)+Q_n^2(t)\big)-Vp(t)$ is the theoretically justified adaptation of Lyapunov drift-plus-penalty for reinforcement learning. The derivation starts from the standard per-slot problem $\min \frac{1}{2}\sum_n(Q_n^2(t+1)-Q_n^2(t))+Vp(t)$ and notes that applying any monotonically increasing function to its objective leaves the optimal action unchanged. By considering a special case in which queues stay empty and greedy minimization already optimizes the long-run average, the paper argues that the unknown increasing function must act additively over time, forcing it to be of the form $x+a_{0,t}$, and then uses a swap-invariance argument on the queue lengths to show the additive term must be $\sum_n Q_n^2(t)$. Dropping the irrelevant constant yields Eq. (38). The paper presents this reward as balancing the greedy per-slot logic of Lyapunov optimization with RL's long-term perspective, and reports that the resulting LDPTRLQ algorithm preserves queue stability while outperforming baselines that use the Lyapunov objective directly as a reward.

Load-bearing premise

The derivation depends on assuming that two long-horizon optimization problems with the same best actions must have objective values that can be matched term by term for every horizon, a condition that forces each per-slot reshuffling of the objective to be a straight-line shift.

Editorial extensions

If this is right

  • A single-step greedy choice under the proposed reward reproduces the original Lyapunov drift-plus-penalty action, so the RL policy and the greedy solver agree when the policy is myopic.
  • Queue stability remains governed by the mean-rate-stability condition, because the reward keeps the Lyapunov drift structure while adding a squared current-queue term that sharpens the penalty for large queues.
  • Complex, non-convex penalty functions such as the energy cost in the mobile edge computing model become learnable by a neural policy, which the greedy per-slot optimizer cannot handle directly.
  • In the reported simulations, the proposed reward converges faster and achieves lower average energy and average queue length than the Original and Simplified LDPRLQ baselines, and it keeps queue-length variance below that of a linear-reward baseline.
  • The reward is compatible with standard RL agents, with both PPO and DQN tested, so it can be dropped into existing policy-gradient or value-based pipelines without changing the learning algorithm.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same derivation recipe, requiring long-horizon equivalence in a greedy special case and invariance under time-slot swaps, could be applied to other greedy online algorithms such as drift-plus-penalty variants with virtual queues or finite-buffer constraints to produce analogous RL rewards.
  • Beyond the paper: a direct ablation that removes the $-\frac{1}{2}\sum_n Q_n^2(t)$ term from Eq. (38) would test whether the symmetrized current-queue term is what drives the reported queue stability; the paper does not run this ablation.
  • Beyond the paper: because the additivity identity in Eq. (29) is strong, the reward may survive as a useful heuristic even if the uniqueness part of the derivation fails; the simulations alone would then support heuristic value, not theoretical necessity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes LDPTRLQ, a reinforcement-learning reward design that combines the Lyapunov drift-plus-penalty framework with RL. The central claim is a theoretical derivation, under 'common and reasonable conditions', of the reward R(t)=-(1/2)Σ_n(Q_n^2(t+1)+Q_n^2(t))-V p(t), which the authors argue balances the greedy per-slot Lyapunov optimization with the long-term perspective of RL. The derivation proceeds through a chain of optimization-problem equivalences (P6-P11), a functional identity (Eq. (27)), Lemma 1 characterizing f1,t as affine, Lemma 2 fixing the queue-dependent intercept, and culminates in Eq. (38). The paper also reports simulations in mobile edge computing and network routing comparing LDPTRLQ against Lyapunov-assisted RL baselines and backpressure routing.

Significance. If the derivation were valid, the paper would make a useful contribution by giving a principled way to adapt drift-plus-penalty rewards for RL. The motivating observation—that greedy Lyapunov optimization can conflict with RL's long-horizon objective—is reasonable, and the empirical comparison against existing Lyapunov-RL hybrids is a useful data point. However, the theoretical core is not sound: the equivalence-to-functional-identity step is invalid, Lemma 1 is internally inconsistent for nonzero intercepts, and Lemma 2 encodes the final reward symmmetry as an assumption. The paper therefore does not establish 'theoretical superiority' as claimed. Its empirical results, which lack error bars and reproducibility details, cannot compensate for the collapse of the central derivation. The final reward may still be a useful heuristic, but the manuscript's central claim is not supported.

major comments (4)
  1. [Section III.A.1, Theorem 2 (Eq. (12))] The converse direction of Theorem 2 is unproven and false as stated. Two optimization problems with identical argmin sets need not be related by a monotonically increasing transformation. For example, on A=[-1,1], f(x)=x^2 and g(x)=x^2 for x≥0 and x^4 for x<0 have the same unique minimizer x=0, yet no strictly increasing φ satisfies g=φ∘f. The subsequent P5 in Eq. (19) depends on this characterization to represent every equivalent objective as f1,t of the original drift-plus-penalty objective, so the claimed generality of the derivation is not established.
  2. [Section III.A.2, Eq. (27)] The step from 'P11 is equivalent to P9' to the pointwise functional identity f_{1,t'}(Σ_{t=0}^{τ-1} Vp(t)) = f2(Σ_{t=0}^{τ-1} f_{1,t}(Vp(t))) for all t' is invalid. Equivalence of two minimization problems constrains only their argmin sets, not the values of their objective functions on arbitrary inputs. This identity is the sole route to Eq. (29) and Lemma 1, so the derivation of the affine form f1,t(x)=x+a0,t collapses without it.
  3. [Section III.A.2, Lemma 1 and Eqs. (29)-(31)] Lemma 1 is stated without proof, and its 'if and only if' claim is inconsistent with the affine form f1,t(x)=x+a0,t adopted immediately afterward. Inserting this form into Eq. (29) and taking τ=2 with t'=0 gives a0,0=a0,0+a0,1, hence a0,1=0; the symmetric choice with t'=1 gives a0,0=0. Thus all a0,t must vanish. The later reintroduction a0,t=f3(Q(t)) in Eq. (32) is therefore not licensed by Lemma 1 or Eq. (29).
  4. [Section III.A.2, Lemma 2 and Eqs. (33)-(37)] Lemma 2 is not a theorem derived from the Lyapunov drift-plus-penalty framework but an auxiliary assumption that directly encodes the desired conclusion. Requiring reward equivalence when Q(t) and Q(t+1) are swapped leads to f3(Q)-Σ Q_n^2 being constant, i.e., f3(Q)=Σ Q_n^2+a0 in Eq. (37), and inserting this into Eq. (32) yields the final reward Eq. (38) up to an additive constant. Combined with the zero intercept forced by Eq. (29), the final reward is effectively assumed rather than derived. The 'special case' assumption in P6-P7 that greedy per-slot minimization attains the long-term time average also resembles the very property the paper seeks to justify.
minor comments (4)
  1. [Section IV.A, after Eq. (45)] The text says the reward is 'as derived in Eq. (35)', but the final reward expression is Eq. (38).
  2. [Fig. 8 caption] The caption reads 'The end-to-end latency versus the average arriving rate', but the horizontal axis is labeled 'The weight of the energy term V'.
  3. [Notation throughout] R(t) is used both for the RL reward (e.g., Eq. (20)) and for the transmission data rate (e.g., Eqs. (41) and (51)); this makes some equations ambiguous.
  4. [Section III.A.2, Lemma 1] No proof of Lemma 1 is provided; if the lemma is retained, the proof should be included.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the questionable steps are formal gaps or unstated postulates, not reductions of the result to its inputs.

full rationale

The paper's derivation contains serious logical gaps, but they are not circular in the sense defined here. The jump from 'P11 is equivalent to P9' to Eq. (27), f1,t'(sum V p(t)) = f2(sum f1,t(V p(t))), asserts pointwise equality of objective functions from optimizer equivalence; this is an invalid inference and a correctness flaw, not an identity built into the definitions. Likewise, Eq. (38) is obtained from Eq. (32) together with Lemma 2, which is an explicitly stated invariance postulate about swapping Q(t) and Q(t+1). Deriving a reward formula from a stated postulate may make the postulate as strong as the conclusion, but it is not the same as defining the predicted reward in terms of the conclusion or relabeling a fitted parameter as a prediction. The self-references in the paper, notably [28], are motivational or baseline citations and are not load-bearing in the reward derivation; no uniqueness theorem from the authors is imported, and no established result is merely renamed. Because no core 'prediction' is equivalent to its input by construction, the circularity score is 0. Correctness concerns about Theorem 2 and Eqs. (27)-(29) remain outside this circularity assessment.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The derivation imports the standard Lyapunov theorem, then adds three paper-specific assumptions: the greedy special case, Q(t)-only dependence of the intercept, and the time-swap reward invariance. The time-swap assumption effectively injects the final reward, so the ledger is dominated by ad hoc premises rather than derived content.

free parameters (2)
  • a0 = 0
    Additive constant in the reward; set to zero 'without loss of generality'. Under discounted RL an additive per-step constant is not automatically behavior-neutral, so the WLOG claim is not fully established.
  • a1 = 1
    Scaling factor in f1,t(x)=a1 x + a1 a0,t; fixed to 1 for simplicity. A positive scaling of the reward can be absorbed into value-function scales, so this is a benign normalization.
assumptions (4)
  • domain assumption The Lyapunov drift-plus-penalty theorem: per-slot minimization of P3 provides queue stability and near-optimal average penalty.
    Invoked as Theorem 1 and attributed to Neely; it is background imported from prior literature rather than derived here.
  • ad hoc to paper In the special case considered, greedy per-slot minimization of V p(t) achieves the minimum of its long-term time average.
    Used to equate P11 with P9 in Section III-A2; no general criterion is given for when this greedy property holds.
  • ad hoc to paper The intercept a0,t depends only on the current queue state Q(t).
    Stated after Eq. (31) with the justification that Q(t) is the only common fixed value across slots; other slot-fixed quantities such as arrivals or channel gains are not excluded.
  • ad hoc to paper Swapping queue lengths at t and t+1 while keeping the penalty unchanged leaves the RL reward equivalent.
    Lemma 2 is asserted from 'the specific time at which a particular queue length occurs is of no concern'; this is not a property of per-step Markov rewards and it directly forces the final reward form.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Lyapunov Drift-Plus-Penalty Method Tailored for Reinforcement Learning with Queue Stability." pith.science (2026). https://pith.science/paper/SLSELTFJ

@misc{pith2026250604291,
  author       = {Pith},
  title        = {Pith review of: A Lyapunov Drift-Plus-Penalty Method Tailored for Reinforcement Learning with Queue Stability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SLSELTFJ}},
  note         = {Machine review of arXiv:2506.04291}
}
read the original abstract

With the proliferation of Internet of Things (IoT) devices, the demand for addressing complex optimization challenges has intensified. The Lyapunov Drift-Plus-Penalty algorithm is a widely adopted approach for ensuring queue stability, and some research has preliminarily explored its integration with reinforcement learning (RL). In this paper, we investigate the adaptation of the Lyapunov Drift-Plus-Penalty algorithm for RL applications, deriving an effective method for combining Lyapunov Drift-Plus-Penalty with RL under a set of common and reasonable conditions through rigorous theoretical analysis. Unlike existing approaches that directly merge the two frameworks, our proposed algorithm, termed Lyapunov drift-plus-penalty method tailored for reinforcement learning with queue stability (LDPTRLQ) algorithm, offers theoretical superiority by effectively balancing the greedy optimization of Lyapunov Drift-Plus-Penalty with the long-term perspective of RL. Simulation results for multiple problems demonstrate that LDPTRLQ outperforms the baseline methods using the Lyapunov drift-plus-penalty method and RL, corroborating the validity of our theoretical derivations. The results also demonstrate that our proposed algorithm outperforms other benchmarks in terms of compatibility and stability.

Figures

Figures reproduced from arXiv: 2506.04291 by the authors.

Figure 1
Figure 1. System model of N distinct queues. lengths, we denote the arrival process for the n th queue at time slot t as d arr n (t), and the departure process as d dep n (t). The evolution of the queue lengths can be described by the following queuing model: Qn(t + 1) = max{Qn(t) − d dep n (t), 0} + d arr n (t). (2) Definition 1. [6] A discrete time queue Qn is mean rate stable if limt→∞ E{Qn(t)} t = 0, (3) where E{Qn(t)} is… view at source ↗
Figure 2
Figure 2. System model of mobile edge computing. The evolution of the task queue lengths at the k th mobile user at time slot t can be described by the following queuing model: Qk(t + 1) = max{Qk(t) − d dep k (t), 0} + d arr k (t), ∀k, (39) where d arr k (t) is the total length of the newly generated task demand at time slot t, and d dep k (t) represents the number of tasks departing from the task queue at time slot t, which … view at source ↗
Figure 4
Figure 4. The energy and queue length behavior of our proposed LDPTRLQ [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: The convergence behavior of our proposed LDPTRLQ with bench [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]
Figure 6
Figure 6. Figure 6: System model of network routing. B. Network Routing In our simulation, we developed a method for constructing a randomly generated network topology for analysis as shown in Fig.6. The generation process begins with the creation of 20 nodes, each assigned a random two-d…
Figure 7
Figure 7. Figure 7: The end-to-end latency versus the average arriving rate of our proposed [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: The end-to-end latency versus the average arriving rate of our proposed [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 28 canonical work pages

  1. [1]

    Distributed Lyapunov drift-plus-penalty routing for wifi mesh networks with adaptive penalty weight,

    J. N ´u˜nez-Mart´ınez and J. Mangues-Bafalluy, “Distributed Lyapunov drift-plus-penalty routing for wifi mesh networks with adaptive penalty weight,” in2012 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM). IEEE, 2012, pp. 1–6

    work page 2012

  2. [2]

    Lyapunov optimized resource management for multiuser mobile video streaming,

    N. Li, Y . Hu, Y . Chen, and B. Zeng, “Lyapunov optimized resource management for multiuser mobile video streaming,”IEEE Trans. Cir- cuits Systems Video Technol., vol. 29, no. 6, pp. 1795–1805, 2018. 11

    work page 2018

  3. [3]

    Lyapunov optimization framework for 5G mobile nodes with multi-homing,

    T. Shuminoski and T. Janevski, “Lyapunov optimization framework for 5G mobile nodes with multi-homing,”IEEE Commun. Letters, vol. 20, no. 5, pp. 1026–1029, 2016

    work page 2016

  4. [4]

    On the convergence time of the drift-plus- penalty algorithm for strongly convex programs,

    H. Yu and M. J. Neely, “On the convergence time of the drift-plus- penalty algorithm for strongly convex programs,” in2015 54th IEEE Conference on Decision and Control (CDC). IEEE, 2015, pp. 2673– 2679

    work page 2015

  5. [5]

    Dynamic resource allocation in metro elastic optical networks using Lyapunov drift optimization,

    M. Hadi, M. R. Pakravan, and E. Agrell, “Dynamic resource allocation in metro elastic optical networks using Lyapunov drift optimization,”J. Optical Commun. Network., vol. 11, no. 6, pp. 250–259, 2019

    work page 2019

  6. [6]

    Neely,Stochastic network optimization with application to commu- nication and queueing systems

    M. Neely,Stochastic network optimization with application to commu- nication and queueing systems. Springer Nature, 2022

    work page 2022

  7. [7]

    On the optimality of greedy policies in dynamic matching,

    S. Kerimov, I. Ashlagi, and I. Gurvich, “On the optimality of greedy policies in dynamic matching,”Operations Research, vol. 73, no. 1, pp. 560–582, 2025

    work page 2025

  8. [8]

    Deep learning for channel tracking in IRS-assisted UA V communication systems,

    J. Yu, X. Liu, Y . Gao, C. Zhang, and W. Zhang, “Deep learning for channel tracking in IRS-assisted UA V communication systems,”IEEE Trans. Wireless Commun., 2022

    work page 2022

Show all 29 references
  1. [9]

    Lyapunov-guided deep reinforcement learning for stable online computation offloading in mobile-edge computing networks,

    S. Bi, L. Huang, H. Wang, and Y . J. A. Zhang, “Lyapunov-guided deep reinforcement learning for stable online computation offloading in mobile-edge computing networks,”IEEE Trans. Wireless Commun., vol. 20, no. 11, pp. 7519–7537, 2021

    2021

  2. [10]

    Multi-agent deep reinforcement learning for computation offloading and interference coordination in small cell networks,

    X. Huang, S. Leng, S. Maharjan, and Y . Zhang, “Multi-agent deep reinforcement learning for computation offloading and interference coordination in small cell networks,”IEEE Trans. Vehicular Technol., vol. 70, no. 9, pp. 9282–9293, 2021

    2021

  3. [11]

    Neely,Stochastic network optimization with application to commu- nication and queueing systems

    M. Neely,Stochastic network optimization with application to commu- nication and queueing systems. Morgan & Claypool Publishers, 2010

    2010

  4. [12]

    When Lyapunov drift meets DRL: Energy efficient resource allocation for IoT data collecting,

    D. U. Kim, K. Kim, S. M. Kang, S.-B. Park, and C. S. Hong, “When Lyapunov drift meets DRL: Energy efficient resource allocation for IoT data collecting,” in2023 International Conference on Advanced Technologies for Communications (ATC). IEEE, 2023, pp. 382–387

    2023

  5. [13]

    Lyapunov-guided resource allocation and task scheduling for edge computing cognitive radio networks via deep reinforcement learning,

    C. Xu, P. Zhang, and H. Yu, “Lyapunov-guided resource allocation and task scheduling for edge computing cognitive radio networks via deep reinforcement learning,”IEEE Sensors J., 2025

    2025

  6. [14]

    Lyapunov drift-plus-penalty optimization for queues with finite capacity,

    L. Bracciale and P. Loreti, “Lyapunov drift-plus-penalty optimization for queues with finite capacity,”IEEE Commun. Lett., vol. 24, no. 11, pp. 2555–2558, 2020

    2020

  7. [15]

    Joint task offloading and resource allocation for energy-constrained mobile edge computing,

    H. Jiang, X. Dai, Z. Xiao, and A. Iyengar, “Joint task offloading and resource allocation for energy-constrained mobile edge computing,” IEEE Trans. Mobile Computing, vol. 22, no. 7, pp. 4000–4015, 2022

    2022

  8. [16]

    UA V-assisted task offloading in vehicular edge computing networks,

    X. Dai, Z. Xiao, H. Jiang, and J. C. Lui, “UA V-assisted task offloading in vehicular edge computing networks,”IEEE Trans. Mobile Computing, vol. 23, no. 4, pp. 2520–2534, 2023

    2023

  9. [17]

    Energy-efficient federated edge learning with streaming data: A lyapunov optimization approach,

    C.-H. Hu, Z. Chen, and E. G. Larsson, “Energy-efficient federated edge learning with streaming data: A lyapunov optimization approach,”IEEE Trans. Commun., 2024

    2024

  10. [18]

    Profit maxi- mization of independent task offloading in MEC-enabled 5G internet of vehicles,

    G. Sun, Z. Wang, H. Su, H. Yu, B. Lei, and M. Guizani, “Profit maxi- mization of independent task offloading in MEC-enabled 5G internet of vehicles,”IEEE Trans. Intelligent Transportat. Systems, 2024

    2024

  11. [19]

    An online rein- forcement learning-based energy management strategy for microgrids with centralized control,

    Q. Meng, S. Hussain, F. Luo, Z. Wang, and X. Jin, “An online rein- forcement learning-based energy management strategy for microgrids with centralized control,”IEEE Trans. Industry Applications, 2024

    2024

  12. [20]

    Offline meta- reinforcement learning for active pantograph control in high-speed railways,

    H. Wang, Z. Liu, G. Hu, X. Wang, and Z. Han, “Offline meta- reinforcement learning for active pantograph control in high-speed railways,”IEEE Trans. Industrial Informatics, 2024

    2024

  13. [21]

    Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods,

    Y . Cao, H. Zhao, Y . Cheng, T. Shu, Y . Chen, G. Liu, G. Liang, J. Zhao, J. Yan, and Y . Li, “Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods,”IEEE Trans. Neural Networks Learning Systems, 2024

    2024

  14. [22]

    Developments in image processing using deep learning and reinforcement learning,

    J. Valente, J. Ant ´onio, C. Mora, and S. Jardim, “Developments in image processing using deep learning and reinforcement learning,”Journal of Imaging, vol. 9, no. 10, p. 207, 2023

    2023

  15. [23]

    Maximum entropy reinforcement learn- ing in two-player perfect information games,

    T. Nakayashiki and T. Kaneko, “Maximum entropy reinforcement learn- ing in two-player perfect information games,” in2021 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 2021, pp. 01–08

    2021

  16. [24]

    Deep reinforcement learning for stochastic computation offloading in digital twin networks,

    Y . Dai, K. Zhang, S. Maharjan, and Y . Zhang, “Deep reinforcement learning for stochastic computation offloading in digital twin networks,” IEEE Trans. Industrial Informatics, vol. 17, no. 7, pp. 4968–4977, 2020

    2020

  17. [25]

    Learning-based context-aware resource allocation for edge-computing-empowered industrial IoT,

    H. Liao, Z. Zhou, X. Zhao, L. Zhang, S. Mumtaz, A. Jolfaei, S. H. Ahmed, and A. K. Bashir, “Learning-based context-aware resource allocation for edge-computing-empowered industrial IoT,”IEEE Internet Things J., vol. 7, no. 5, pp. 4260–4277, 2019

    2019

  18. [26]

    Predictable wireless networked scheduling for bridging hybrid time-sensitive and real-time services,

    Y . Zhou, L. Feng, X. Jiang, W. Li, and F. Zhou, “Predictable wireless networked scheduling for bridging hybrid time-sensitive and real-time services,”IEEE Trans. Commun., vol. 72, no. 6, pp. 3664–3680, 2024

    2024

  19. [27]

    Lyapunov optimization based mobile edge computing for internet of vehicles systems,

    Y . Jia, C. Zhang, Y . Huang, and W. Zhang, “Lyapunov optimization based mobile edge computing for internet of vehicles systems,”IEEE Trans. commun., vol. 70, no. 11, pp. 7418–7433, 2022

    2022

  20. [28]

    Enhancing fog computing through intelligent reflecting surface assistance: A Lyapunov driven reinforcement learning approach,

    W. Xu, Y . Yuan, and D. H. K. Tsang, “Enhancing fog computing through intelligent reflecting surface assistance: A Lyapunov driven reinforcement learning approach,” in2024 IEEE 10th World Forum on Internet of Things (WF-IoT). IEEE, 2024, pp. 759–764

    2024

  21. [29]

    Energy-latency aware intelligent reflecting surface aided multi-cell mobile edge computing,

    W. Xu, J. Yu, Y . Wu, and D. H. K. Tsang, “Energy-latency aware intelligent reflecting surface aided multi-cell mobile edge computing,” IEEE Trans. Green Commun. Network., vol. 8, no. 1, pp. 362–374, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.