Pith. sign in

REVIEW 3 major objections 5 minor 31 references

Goal-oriented Transmission Scheduling: Structure-guided DRL with a Unified Dual On-policy and Off-policy Approach

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The optimal transmission-scheduling value function is asymptotically convex in data age, and a structure-guided hybrid DRL exploits this for up to 45% lower cost.

desk verdict Genuinely useful algorithm and a correct small-case theory, but the general asymptotic convexity theorem is unproven due to a wrong-direction inequality, so the paper's central claim overreaches. read the letter →

arxiv 2501.11921 v1 pith:XATYX2DM submitted 2025-01-21 cs.IT cs.AIcs.LGcs.SYeess.SPeess.SYmath.IT

Jiazheng Chen , Wanchun Liu This is my paper · ORCID
classification cs.ITcs.AIcs.LGcs.SYeess.SPeess.SYmath.IT
keywords goal-orientedcommunicationstransmissionschedulingageofinformationdeepreinforcementlearninghybridon-policyoff-policyvaluefunctionconvexitymonotonicityremotestateestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets multi-device, multi-channel wireless scheduling where freshness of information is measured by age of information (AoI). It claims to prove that the optimal value function—the minimum expected discounted long-term cost—is monotone in channel quality and asymptotically convex in AoI, and that the optimal policy is monotone in channel quality. These structural results are then built into SUDO-DRL, a deep reinforcement learning algorithm that mixes on-policy and off-policy training, penalizing critic outputs that violate the proven structure and storing only trajectories that satisfy it. If the claims are right, structure-guided training reaches better schedules in large systems, with reported gains of 25–45% over state-of-the-art baselines and roughly 40% faster convergence.

What carries the argument

The central object is the optimal state-value function $\upsilon^*(\delta, G)$ defined by the Bellman optimality equation, with discrete convexity in AoI defined by the interpolation inequality $\alpha \upsilon(\delta''_n,\cdot)+(1-\alpha)\upsilon(\delta'_n,\cdot) \geq \upsilon(\delta_n,\cdot)$. The argument's machinery is a preservation proof: value iteration carries convexity (and the related monotonicity) from an initial function to the optimum, using an asymptotic dominance lemma stating that states with much larger AoI have much larger values. On the algorithmic side, the mechanism is a structural-property evaluation framework that computes critic-monotonicity, critic-convexity, and actor-monotonicity scores over sampled states, folds violations into the on-policy loss, and uses the scores to admit trajectories into the off-policy replay buffer.

What would settle it

Run exact value iteration on a small multi-device multi-channel instance (e.g., 3 devices and 2 channels) with a convex cost, and test the discrete convexity inequality $\alpha\upsilon(\delta''_n,\cdot)+(1-\alpha)\upsilon(\delta'_n,\cdot) \geq \upsilon(\delta_n,\cdot)$ for every state triple with $\delta'_n \geq \delta_n \gg \delta''_n$. One violation — or a direct verification of the action-comparison inequality in Appendix C case (b) — would show the asymptotic convexity theorem and the penalty derived from it are not guaranteed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the optimal state-value function of a goal-oriented scheduling MDP carries more mathematical structure than previously known: it is non-decreasing in both AoI and channel states, asymptotically convex in AoI when the per-device cost is convex, and the optimal policy is monotone in channel states; for co-located devices the optimal policy is asymptotically greedy in the sense that all devices in a mandatory scheduling set are scheduled. The asymptotic convexity is proved for the general multi-device multi-channel case by showing that one step of the Bellman iteration preserves convexity when the higher AoI states are sufficiently large, and for the two-device-one-channel case it is proved without the asymptotic condition. These properties are not just descriptive: SUDO-DRL converts them into penalty terms in the critic loss and into trajectory-selection criteria for the replay buffer, and the numerical section reports that this structure-guided hybrid outperforms both on-policy and off-policy baselines.

Load-bearing premise

The load-bearing premise is that when one device's data age is far larger than another's, its value term dominates all other terms in the cost-update equation, and this dominance is asserted with the symbol 'much greater than' but never quantified by error bounds or a threshold; if that dominance is not true at the states where the algorithm applies the penalty, the asymptotic convexity and greedy-structure results are not guaranteed.

Editorial extensions

If this is right

  • If Theorem 3 holds, then a critic network trained with the convexity penalty is being pushed toward a true property of the optimal value function, not an arbitrary regularization.
  • The hybrid loss unifies on-policy and off-policy updates, so the method inherits stability from PPO-style clipping and sample reuse from the priority replay buffer; at scale (40 devices, 20 channels) off-policy baselines fail while SUDO-DRL converges.
  • The greedy pre-training stage, based on the mandatory scheduling set, provides a better initialization and is reported to cut convergence time by about 40%.
  • Reported empirical average cost is 25–45% lower than PPO and comparable to state-of-the-art off-policy methods in small systems, with the gap widening as system scale grows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extrapolating beyond the paper, the same structure-guided penalty idea could be applied to other MDPs where value-function convexity or monotonicity is provable, such as inventory or queueing control, not just AoI scheduling.
  • The paper uses the asymptotic convexity result to penalize violations at all states; if the asymptotic threshold is large, early-training states far from the asymptotic regime could be penalized incorrectly, so a testable refinement is to apply the penalty only when the minimum AoI in the state exceeds a threshold.
  • The reported gains depend on the specific cost function being asymptotically convex; for non-convex goal-oriented metrics such as value-of-information with saturating importance, the structural guidance may require a different property.
  • A theoretical tightening would be to replace the informal 'much greater than' dominance with explicit thresholds or error bounds, which would turn the asymptotic theorems into quantitative ones and would tell practitioners when the structure penalties are guaranteed to be valid.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies goal-oriented transmission scheduling with N devices and M channels, modeling AoI and fading channel states as an infinite-horizon discounted MDP. It claims four theoretical contributions: monotonicity of the optimal value function with respect to channel states (Theorem 1), convexity of the optimal value function for two-device-one-channel systems (Theorem 2), asymptotic convexity of the optimal value function for general multi-device-multi-channel systems (Theorem 3), and an asymptotic greedy structure of the optimal policy for co-located devices (Theorem 5). It then proposes SUDO-DRL, a hybrid on-policy/off-policy algorithm that penalizes violations of these structural properties in the critic loss, uses structural scores to decide which trajectories enter the replay buffer, and uses the greedy structure to pre-train the policy. Numerical experiments report 25--45% cost improvements over PPO and scalability to 40-device/20-channel systems.

Significance. The engineering problem is timely and the proposed algorithm is clearly specified, with a thoughtful combination of PPO-style on-policy updates and SAC-style off-policy replay. The numerical study is reasonably extensive and shows consistent gains over PPO at larger scales, which is a useful empirical result. However, the advertised theoretical novelty is not established. The proof of Theorem 3, the paper's central structural result and the stated justification for the convexity penalty in SUDO-DRL, contains a wrong-direction inequality in Appendix C, case (b). The paper also relies throughout on an undefined and unquantified asymptotic dominance relation '≫'. Because the algorithm's structure-guided component is presented as being derived from these theorems, the failure of the proof removes the theoretical basis for the convexity regularization; the empirical method could still be viewed as a heuristic, but that is not the paper's claim. The manuscript does not ship machine-checked proofs, reproducible code, or parameter-free derivations, so the strengths are the empirical comparison and the algorithm design rather than the theoretical statements.

major comments (3)
  1. [Appendix C, proof of Theorem 3, case (b)] The displayed chain after the first inequality replaces the optimal action ˇa1 at state δ'' by the action ˆa1 in the first expectation of the α-term. This step requires A(ˇa1) ≥ A(ˆa1), where A(a) = Σ_{δ''+} P(δ''+|δ'',G,a)υ0(δ''+). But ˇa1 is the minimizer of Z(δ'',·;υ0), so A(ˇa1) ≤ A(ˆa1). The inequality therefore has the wrong direction. The subsequent appeal to Lemma 4 and δ'_i ≫ δ''_i compares states δ' and δ'' rather than actions at the same state δ'', and it does not repair the invalid replacement. Since Theorem 3 is the stated justification for the convexity penalty (26), the CC score (29), and the trajectory-storage criterion (34), the theoretical support for the convexity regularization in SUDO-DRL is missing.
  2. [Section IV-B, Lemma 4, Appendices C, D, F, G] The notation '≫' is used with no quantitative meaning and no error bounds. Lemma 4, Theorem 3, Proposition 1, Theorem 5, and the proofs in Appendices C, D, F, and G all manipulate '≫' as if it were a transitive order that is preserved under summation, expectation, minimization, and multiplication by probabilities. For example, in Appendix C, case (b), the inequality Σ_{δ'+}P(δ'+|δ',G,ˆa1)υ0(δ'+) ≫ Σ_{δ''+}P(δ''+|δ'',G,ˇa1)υ0(δ''+) is used to discard the αA(ˇa1) term, but no threshold is given that relates δ'−δ'', the discount factor γ, the transition probabilities, and the magnitude of υ0. Without such bounds, the 'asymptotic' theorems cannot be checked as mathematical claims, and Theorem 5 inherits the same problem through Lemma 6.
  3. [Appendix F, proof of Lemma 6] The proof states, without derivation, that 'in the 1st iteration, we can prove that the optimal action of the device i w.r.t. the state s° should be scheduled', and then analyzes only cases with a1_i ≠ 0. This omitted step is exactly the scheduling property that Lemma 6 is used to prove in Theorem 5, so the argument is circular at that point. The surrounding dominance inequalities do not fill the gap because the threshold ¯δ in Definition 2 is not tied to any quantitative condition needed by Lemma 4.
minor comments (5)
  1. [Appendix B, Eq. (44)] The definition of π_k(s) uses 'arg max' and places c(s)+γ outside the arg max; given Eq. (43) and the inequality (46), the intended operation is 'arg min', and the definition should be corrected.
  2. [Section V-B, Eq. (36)] The off-policy critic loss is written as (1/B2) Σ TDb, which is a sum of signed TD errors; it should presumably be (1/B2) Σ TD_b^2, consistent with the on-policy loss.
  3. [Appendix A, proof of Lemma 2] The final step 'y(d+1)−y(d)>0 for d≫1' does not follow merely from the non-negativity of y(d); the proof needs to show that the leading polynomial coefficient is positive, and the notation ¯r is never defined.
  4. [Section III-A] The heading contains the typo 'MDP Formulartion'.
  5. [Section VI-A] The packet drop rates are listed as 0.2, 0.15, 0.1, 0.05, 0.01 for quantization levels 1 through 5, while Section II-A states 'higher channel states correspond to higher packet drop rates'; the ordering should be reconciled.

Circularity Check

2 steps flagged · score 4.0 of 10

SUDO-DRL's structural-score validation is partly circular (penalties are the metric), and Theorem 3's asymptotic-convexity proof leans on an unproved self-cited dominance lemma.

  1. fitted input called prediction [Section V-B2, eqs. (29) and (31); Section VI-B, Figs. 5-7]
    "The loss function for the critic NN in the on-policy component is defined as: LOn(ν) = 1/B1 Σ TD2_l + 1/(KΞ) Σ( ´VAoI,k,ξ + ´VCh,k,ξ + ˘VAoI,k,ξ ), (31) ... For the critic NN convexity, SUDO-DRL guarantees a full score after 200 episodes, whereas PPO remains below 80 until the end of training."

    The CM and CC scores (28)-(29) are defined as the fraction of samples for which the monotonicity/convexity slack variables in (24)-(26) are zero. Those same slack variables are added directly to the critic's on-policy loss (31), so minimizing (31) drives them to zero by construction. Reporting CC=100 and CM=100 as evidence that SUDO-DRL exploits the theoretical structure, and as the reason it outperforms PPO, is circular: the evaluation metric is the training objective. The external cost-performance comparisons are independent, but the structural-score validation is not.

  2. self citation load bearing [Appendix C (Proof of Theorem 3), Lemma 4 and case (b)]
    "Lemma 4 (Asymptotic monotonicity of the optimal V function w.r.t. AoI state [21]): For states s = (δ, G) and s′AoI = (δ′(i), G), where δ′i ≫ δi, the optimal V function holds the inequality: υ∗(s′AoI) ≫ υ∗(s). ... the following inequality ... achieved by Lemma 4 and δ′i ≫ δ′′i."

    Theorem 3 is the advertised first asymptotic-convexity result and the stated justification for the convexity penalty (26). In case (b), the action-replacement inequality that carries the proof is justified only as 'achieved by Lemma 4', a lemma imported verbatim from the authors' own [21] and not restated or proved here. Since '≫' is never quantified, the proof reduces the central structural claim to an unverified asymptotic-dominance assertion from the authors' prior work. Independently, the substitution ˇa1 → ˆa1 also contradicts Bellman optimality (optimality of ˇa1 gives the opposite inequality), so the self-citation is the only stated support for the step.

full rationale

The derivation is not circular in the basic sense of assuming the target theorem: it starts from the Bellman equation and uses value-iteration preservation arguments, and the reported cost improvements (25-45%) are measured against external PPO/DDPG-style benchmarks. However, two load-bearing elements are circular or self-referential. First, the CM/CC scores used to validate SUDO-DRL's structure guidance are the same penalty terms that appear in the critic loss (31), so high scores are a direct consequence of the training objective rather than independent evidence. Second, Theorem 3's proof depends on Lemma 4 of [21] for an asymptotic '≫' dominance that is neither proved nor bounded, and the case (b) action replacement has a wrong-direction inequality; as written, the advertised convexity theorem reduces to that unverified self-citation. These issues do not invalidate the empirical comparisons, but they do mean the structural-score evaluation and the theoretical grounding of the convexity penalty are partly circular. Overall circularity score: 4.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The central claim rests on the standard MDP framework, the i.i.d. fading channel model, cost convexity, and the informal asymptotic-dominance machinery that powers the general theorems. The unquantified '≫' relations and the uncomputed thresholds δ̄ are the weakest free parameters.

free parameters (3)
  • Asymptotic threshold for '≫ 1' = not specified
    The proofs of Theorems 3, 5, and Proposition 1 require AoI states to be '≫ 1' and the dominance inequalities to hold with sufficient margin. The threshold at which asymptotic convexity and greedy structure kick in is never quantified, so the theorem's applicability to the simulated AoI values is an unmeasured free parameter.
  • Mandatory scheduling threshold δ̄ = not specified
    Definition 2 assumes existence of a threshold δ̄ such that the cost functions have a fixed ordering for all δ ≥ δ̄. This threshold is used in Theorem 5 but is never computed or estimated from the system parameters.
  • Algorithm hyperparameters β1, β2, ϱ, ω, λ, ϵ, I1 = 0.9, 0.9, 0.95, 0.01, 0.99, 0.2, 10N
    These are hand-chosen values listed in Table I with no sensitivity analysis. The performance claims depend on these choices, and they are not derived from the theory.
assumptions (5)
  • standard math Value iteration converges to the unique optimal value function for any initial v0 (Puterman).
    Invoked in Appendices B-D and G (Lemma 3) to reduce preservation of a property to a single Bellman iteration.
  • domain assumption Channel states are i.i.d. block fading with known distribution and monotone packet drop rates in the channel state.
    Stated in Section II-A and used throughout, especially in Theorem 1 and the value function proofs to average over channel states.
  • domain assumption The cost function is non-decreasing in AoI and convex (or asymptotically convex) in AoI.
    Assumed in Definition 1 and Lemma 2; the remote state estimation example justifies it asymptotically, but the general proof of Theorem 3 takes convexity of c(·) as an input.
  • ad hoc to paper The asymptotic dominance relations denoted by '≫' behave like a transitive partial order and survive summation, expectation, and minimization in the Bellman update.
    Appendices C-F repeatedly use statements such as v0(δ'_i + 1, ·) ≫ v0(δ_i + 1, ·) without epsilon-delta bounds. This is an ad hoc assumption specific to this paper's proof style, and it is where the flawed inequality in Appendix C, case (b), enters.
  • ad hoc to paper The structural properties proven for special cases (two-device-one-channel, co-located devices, asymptotic regimes) extend heuristically to the general systems used in the SUDO-DRL experiments.
    Explicitly acknowledged in the footnote after (23): although Theorem 3 is asymptotic, the algorithm enforces convexity at all states. This is a design heuristic, not a theorem.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Goal-oriented Transmission Scheduling: Structure-guided DRL with a Unified Dual On-policy and Off-policy Approach." pith.science (2026). https://pith.science/paper/XATYX2DM

@misc{pith2026250111921,
  author       = {Pith},
  title        = {Pith review of: Goal-oriented Transmission Scheduling: Structure-guided DRL with a Unified Dual On-policy and Off-policy Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XATYX2DM}},
  note         = {Machine review of arXiv:2501.11921}
}
read the original abstract

Goal-oriented communications prioritize application-driven objectives over data accuracy, enabling intelligent next-generation wireless systems. Efficient scheduling in multi-device, multi-channel systems poses significant challenges due to high-dimensional state and action spaces. We address these challenges by deriving key structural properties of the optimal solution to the goal-oriented scheduling problem, incorporating Age of Information (AoI) and channel states. Specifically, we establish the monotonicity of the optimal state value function (a measure of long-term system performance) w.r.t. channel states and prove its asymptotic convexity w.r.t. AoI states. Additionally, we derive the monotonicity of the optimal policy w.r.t. channel states, advancing the theoretical framework for optimal scheduling. Leveraging these insights, we propose the structure-guided unified dual on-off policy DRL (SUDO-DRL), a hybrid algorithm that combines the stability of on-policy training with the sample efficiency of off-policy methods. Through a novel structural property evaluation framework, SUDO-DRL enables effective and scalable training, addressing the complexities of large-scale systems. Numerical results show SUDO-DRL improves system performance by up to 45% and reduces convergence time by 40% compared to state-of-the-art methods. It also effectively handles scheduling in much larger systems, where off-policy DRL fails and on-policy benchmarks exhibit significant performance loss, demonstrating its scalability and efficacy in goal-oriented communications.

Figures

Figures reproduced from arXiv: 2501.11921 by the authors.

Figure 1
Figure 1. Goal-oriented communication system with N edge devices, M channels, and a remote destination sample efficiency of off-policy methods through a uni￾fied loss function. A structural property evaluation framework is introduced to derive critic-monotonicity, critic-convexity, and actor-monotonicity scores, which are incorporated into the on-policy loss function. For the off-policy component, the structural scores guide … view at source ↗
Figure 2
Figure 2. Critic and Actor NNs’ structural property evaluation framework. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. SUDO-DRL Architecture. TOn for use during the pre-training stage. The architecture and details of the SUDO-DRL algorithm are shown in [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: illustrates the average sum MSE cost during the training of the proposed SUDO-DRL algorithm, both with and without the pre-training stage, and compares it with TABLE I SUMMARY OF TRAINING HYPERPARAMETERS Hyperparameters of SUDO-DRL and benchmarks Value Critic NN learni…
Figure 5
Figure 5. Figure 5: Critic monotonicity (CM) score during training with [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Critic convexity (CC) score during training with [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Actor monotonicity (AM) score during training with [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 28 canonical work pages

  1. [1]

    Beyond transmitting bits: Context, seman- tics, and task-oriented communications,

    D. G ¨und¨uz, Z. Qin, I. E. Aguerri, H. S. Dhillon, Z. Yang, A. Yener, K. K. Wong, and C.-B. Chae, “Beyond transmitting bits: Context, seman- tics, and task-oriented communications,” IEEE J. Sel. Areas Commun. , vol. 41, no. 1, pp. 5–41, Jan. 2023

    work page 2023

  2. [2]

    Age of incorrect information in semantic communications for NOMA aided XR applications,

    J. Chen, J. Wang, C. Jiang, and J. Wang, “Age of incorrect information in semantic communications for NOMA aided XR applications,” IEEE J. Sel. Top. Signal Process. , early access, Jun. 2023

    work page 2023

  3. [3]

    Goal-oriented semantic commu- nications for avatar-centric augmented reality,

    Z. Wang, Y . Deng, and A. H. Aghvami, “Goal-oriented semantic commu- nications for avatar-centric augmented reality,” IEEE Trans. Commun. , early access, Jun. 2024

    work page 2024

  4. [4]

    Optimal downlink- uplink scheduling of wireless networked control for industrial IoT,

    K. Huang, W. Liu, Y . Li, B. Vucetic, and A. Savkin, “Optimal downlink- uplink scheduling of wireless networked control for industrial IoT,” IEEE Internet Things J. , vol. 7, no. 3, pp. 1756–1772, Mar. 2020

    work page 2020

  5. [5]

    AoI-centric task scheduling for autonomous driving systems,

    C. Xu, Q. Xu, J. Wang, K. Wu, K. Lu, and C. Qiao, “AoI-centric task scheduling for autonomous driving systems,” in Proc. IEEE INFOCOM. IEEE, Jun. 2022, pp. 1019–1028

    work page 2022

  6. [6]

    Towards goal-oriented semantic communications: New metrics, open challenges, and future research directions,

    A. Li, S. Wu, S. Meng, and Q. Zhang, “Towards goal-oriented semantic communications: New metrics, open challenges, and future research directions,” arXiv preprint , Apr. 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2304.00848

    work page doi:10.48550/arxiv.2304.00848 2024

  7. [7]

    Status updates through queues,

    S. K. Kaul, R. D. Yates, and M. Gruteser, “Status updates through queues,” in 2012 46th Annual Conference on Information Sciences and Systems (CISS). IEEE, 2012, pp. 1–6

    work page 2012

  8. [8]

    Goal- oriented transmission scheduling for energy-efficient wireless networked control in SAGIN: An AoI-thresholding mechanism,

    Y . Wang, S. Wu, Y . Wang, X. Zhang, J. Jiao, and Q. Zhang, “Goal- oriented transmission scheduling for energy-efficient wireless networked control in SAGIN: An AoI-thresholding mechanism,” IEEE Trans. Veh. Technol., early access, Mar. 2024

    work page 2024

Show all 31 references
  1. [9]

    Value of information analysis in feedback control,

    T. Soleymani, “Value of information analysis in feedback control,” Ph.D. dissertation, Technische Universit ¨at M ¨unchen, 2019

    2019

  2. [10]

    Optimal scheduling of multiple sensors over shared channels with packet transmission constraint,

    S. Wu, X. Ren, S. Dey, and L. Shi, “Optimal scheduling of multiple sensors over shared channels with packet transmission constraint,” Automatica, vol. 96, pp. 22–31, Oct. 2018

    2018

  3. [11]

    Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems,

    A. S. Leong, A. Ramaswamy, D. E. Quevedo, H. Karl, and L. Shi, “Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems,” Automatica, vol. 113, pp. 1–8, Mar. 2020. Art. no. 108759

    2020

  4. [12]

    Goal-oriented scheduling in sensor networks with appli- cation timing awareness,

    J. Holm, F. Chiariotti, A. E. Kalør, B. Soret, T. B. Pedersen, and P. Popovski, “Goal-oriented scheduling in sensor networks with appli- cation timing awareness,” IEEE Trans. Commun. , vol. 71, no. 8, pp. 4513–4527, Aug. 2023

    2023

  5. [13]

    AoI-aware joint scheduling and power allocation in intelligent transportation system: A deep reinforce- ment learning approach,

    G. Bai, L. Qu, J. Liu, and D. Sun, “AoI-aware joint scheduling and power allocation in intelligent transportation system: A deep reinforce- ment learning approach,” IEEE Trans. Veh. Technol., vol. 73, no. 4, pp. 5781–5795, Apr, 2024

    2024

  6. [14]

    Scheduling for minimizing the age of information in multisensor multiserver industrial internet of things systems,

    X. Xie, H. Wang, and X. Liu, “Scheduling for minimizing the age of information in multisensor multiserver industrial internet of things systems,” Trans Ind. Informat. , vol. 20, no. 1, pp. 573–582, Jan. 2024

    2024

  7. [15]

    Age-based scheduling for mobile edge computing: A deep reinforcement learning approach,

    X. He, C. You, and T. Q. Quek, “Age-based scheduling for mobile edge computing: A deep reinforcement learning approach,” IEEE Trans. Mob. Comput., early access, Feb. 2024

    2024

  8. [16]

    Aoi-aware joint spectrum and power allocation for internet of vehicles: A trust region policy optimization-based approach,

    N. Peng, Y . Lin, Y . Zhang, and J. Li, “Aoi-aware joint spectrum and power allocation for internet of vehicles: A trust region policy optimization-based approach,” IEEE Internet Things J. , vol. 9, no. 20, pp. 19 916–19 927, Oct. 2022

    2022

  9. [17]

    DRL-based resource allocation in remote state estimation,

    G. Pang, W. Liu, Y . Li, and B. Vucetic, “DRL-based resource allocation in remote state estimation,” IEEE Trans. Wirel. Commun., vol. 22, no. 7, pp. 4434–4448, Jul. 2022

    2022

  10. [18]

    Bias in natural actor-critic algorithms,

    P. Thomas, “Bias in natural actor-critic algorithms,” in Proc. 31nd Int. Conf. Int. Conf. Machine Learning, vol. 32. PMLR, 2014, pp. 441–448

    2014

  11. [19]

    Learning optimal scheduling policy for remote state estimation under uncertain channel condition,

    S. Wu, X. Ren, Q.-S. Jia, K. H. Johansson, and L. Shi, “Learning optimal scheduling policy for remote state estimation under uncertain channel condition,” IEEE Trans. Control. Netw. Syst., vol. 7, no. 2, pp. 579–591, Jun. 2020

    2020

  12. [20]

    Optimal scheduling of multiple sensors over lossy and bandwidth limited channels,

    S. Wu, K. Ding, P. Cheng, and L. Shi, “Optimal scheduling of multiple sensors over lossy and bandwidth limited channels,” IEEE Trans. Netw. Syst., vol. 7, no. 3, pp. 1188–1200, Jan. 2020

    2020

  13. [21]

    Structure-enhanced drl for optimal transmission scheduling,

    J. Chen, W. Liu, D. E. Quevedo, S. R. Khosravirad, Y . Li, and B. Vucetic, “Structure-enhanced drl for optimal transmission scheduling,” IEEE Trans. Wirel. Commun., vol. 23, no. 1, pp. 379–393, Jan. 2024

    2024

  14. [22]

    Semantic-aware transmission scheduling: A monotonicity-driven deep reinforcement learning approach,

    J. Chen, W. Liu, D. E. Quevedo, Y . Li, and B. Vucetic, “Semantic-aware transmission scheduling: A monotonicity-driven deep reinforcement learning approach,” IEEE Commun. Lett. , Dec. 2023

    2023

  15. [23]

    Channel estimation techniques based on pilot arrangement in OFDM systems,

    S. Coleri, M. Ergen, A. Puri, and A. Bahai, “Channel estimation techniques based on pilot arrangement in OFDM systems,” IEEE Trans. Broadcast., vol. 48, no. 3, pp. 223–229, Sep. 2002

    2002

  16. [24]

    Remote state estimation with smart sensors over Markov fading channels,

    W. Liu, D. E. Quevedo, Y . Li, K. H. Johansson, and B. Vucetic, “Remote state estimation with smart sensors over Markov fading channels,” IEEE Trans. Autom. Control, vol. 67, no. 6, pp. 2743–2757, Jun. 2022

    2022

  17. [25]

    Age-of-Information oriented scheduling for multichannel IoT systems with correlated sources,

    J. Tong, L. Fu, and Z. Han, “Age-of-Information oriented scheduling for multichannel IoT systems with correlated sources,” IEEE Trans. Wirel. Commun., vol. 21, no. 11, pp. 9775–9790, Nov. 2022

    2022

  18. [26]

    Stability conditions for remote state estimation of multiple systems over multiple markov fading channels,

    W. Liu, D. E. Quevedo, K. H. Johansson, B. Vucetic, and Y . Li, “Stability conditions for remote state estimation of multiple systems over multiple markov fading channels,” IEEE Trans. Autom. Control , early access, Aug. 2022

    2022

  19. [27]

    M. L. Puterman, Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014

    2014

  20. [28]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint, Sep. 2015. [Online]. Available: https://doi.org/10.48550/arXiv.1509.02971

    2015 doi arXiv

  21. [29]

    Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks,

    L. Huang, S. Bi, and Y .-J. A. Zhang, “Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks,” IEEE Trans. Mob. Comput. , vol. 19, no. 11, pp. 2581–2593, Jul. 2019

    2019

  22. [30]

    Scheduling two gauss–markov systems: An optimal solution for remote state estimation under bandwidth constraint,

    L. Shi and H. Zhang, “Scheduling two gauss–markov systems: An optimal solution for remote state estimation under bandwidth constraint,” IEEE Trans. Signal Process., vol. 60, no. 4, pp. 2038–2042, Apr. 2012

    2012

  23. [31]

    Markov decision processes,

    M. L. Puterman, “Markov decision processes,” Handbooks in operations research and management science , vol. 2, pp. 331–434, 1990

    1990

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.