Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Offline Diffusion Policy for Multi-User Delay-Constrained Scheduling

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A diffusion policy plus critic guidance and an offline dual update can learn delay-constrained, resource-bounded schedulers purely from logged data.

desk verdict A plausible combination of diffusion policies and offline Lagrange optimization for delay-constrained scheduling, with the main weaknesses in experimental rigor rather than in the core idea. read the letter →

arxiv 2501.12942 v2 pith:4ZOJCYPM submitted 2025-01-22 cs.AI

Zhuoran Li , Ruishuo Chen , Hai Zhong , Longbo Huang This is my paper · ORCID
classification cs.AI
keywords offlinereinforcementlearningdelay-constrainedschedulingdiffusionpolicyLagrangiandualcriticguidanceresourceconstraintsmulti-userconstraint-aware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SOCD is an offline reinforcement learning algorithm that claims schedulers for multi-user delay-constrained systems can be trained entirely from pre-collected data, with no interaction with the live system during training. It combines a diffusion model that clones the behavior policy with a critic network that scores and selects actions, and it uses an offline Lagrange multiplier update to keep average resource use within budget. The authors argue this matters because real schedulers for messaging, streaming, and data centers often cannot tolerate the service degradation or cost of online exploration. Experiments across single-hop, multi-hop, partially observable, real-data, and 100-user settings report that SOCD outperforms the prior SOLAR algorithm and traditional policies. If true, SOCD provides a practical route to constraint-aware scheduling where online RL is infeasible.

What carries the argument

The load-bearing mechanism is the coupling of three components: a score-based diffusion model $\mu_\theta(\cdot\mid s)$ used as the behavior-cloning policy, a sampling-free double-Q critic $Q_\phi$ trained against trajectory returns, and an offline Lagrange multiplier update. The diffusion model, trained once with the score-matching objective, gives a high-fidelity multi-modal clone of the behavior policy; the critic then guides action selection by importance weighting or argmax over $K$ sampled actions, implementing the optimal-policy form $\pi^*(a\mid s) \propto \mu(a\mid s)\exp(\alpha Q(s,a))$. The multiplier $\lambda$ is updated by $\lambda \leftarrow \lambda - \alpha(E_0 - \hat{E}_{\pi^*(\lambda)})$ using a dataset-state estimate of average resource consumption, which closes the loop that keeps average resource use under the budget. User-level decomposition splits the MDP by user index so state and action dimensions stay constant as the number of users grows.

What would settle it

Deploy SOCD on a live system whose arrival or channel statistics differ from the dataset, and measure the online average resource consumption against the budget; if it exceeds the budget while the offline estimate said it would not, the claim that the dual update keeps constraints in the offline-only setting is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that one diffusion-based policy trained once on an offline dataset, plus a sampling-free critic and an offline dual update, can solve the multi-user delay-constrained scheduling problem without online interaction. Formally, the scheduler maximizes weighted average timely throughput subject to an average resource budget; SOCD writes this as a Lagrangian, learns the dual function by cloning the unknown behavior policy with a score-based diffusion model, trains a double-Q critic on whole-trajectory discounted returns so no new actions need be sampled, and at action time draws K diffusion samples and selects or weights them by critic values. The Lagrange multiplier is then updated from an offline estimate of the learned policy's average resource consumption computed by feeding dataset states through the policy. The paper reports that the resulting policy stays within the resource budget while delivering higher throughput than SOLAR, behavior cloning, uniform allocation, EDF, and the behavior policy itself.

Load-bearing premise

The load-bearing assumption is that the states recorded in the offline dataset fairly represent the states the learned scheduler will meet when deployed, because the Lagrange update estimates the policy's true average resource use by feeding those dataset states through the policy.

Editorial extensions

If this is right

  • Schedulers for delay-constrained systems can be trained before deployment from logged trajectories, so online exploration costs and service disruptions are removed.
  • The diffusion policy needs only one training pass, so the same behavior model can be reused across different resource budgets as the Lagrange multiplier is adjusted.
  • The critic-guidance procedure yields throughput above the behavior policy while keeping average resource consumption at or below the constraint, across synthetic and real-data environments.
  • The method scales to 100 users, multi-hop paths, and partially observable channels without needing channel state information.
  • Compared with the earlier SOLAR actor-critic offline approach, the diffusion-based SOCD reports consistently higher throughput and better resource-usage efficiency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the offline dual update would fail cleanly under strong state-distribution shift, since the resource estimate averages over dataset states; a deployment-time correction or online fine-tuning of only the multiplier would be a natural extension.
  • Beyond the paper: the same critic-guided generative policy recipe could be applied to other constrained sequential decision problems, such as energy-aware computation offloading or bandwidth allocation, wherever logged trajectories from a suboptimal policy exist.
  • Beyond the paper: because the behavior-cloning model is trained once and is independent of the multiplier, the practical cost of serving multiple resource budgets is dominated by critic training and sampling, suggesting a testable claim that per-budget retraining is unnecessary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SOCD, an offline reinforcement learning algorithm for multi-user delay-constrained scheduling. SOCD combines a diffusion-based behavior-cloning policy, a critic that is trained by regressing to Monte Carlo returns from offline trajectories, and an offline Lagrange multiplier update that aims to enforce average resource constraints without online interaction. The method is evaluated in simulated single-hop, multi-hop, real-data, partially observable, and 100-user environments, and the paper claims it outperforms SOLAR, BC, Uniform, EDF, and the behavior policy. The central claim is that SOCD learns high-quality, constraint-aware scheduling policies purely from pre-collected data.

Significance. If validated, SOCD would be a practically valuable contribution: it replaces online interaction with offline training for a scheduling problem with both delay and resource constraints, and the diffusion policy is trained only once rather than per Lagrange multiplier value. The experimental suite is broad, covering multi-hop topologies, real-world traffic/channel records, partial observability, and scalability to 100 users. The main load-bearing limitation is that the offline estimate of the learned policy's resource consumption, used in the Lagrange update, is acknowledged to be approximate but is never validated against actual deployment behavior; this directly affects the advertised constraint-awareness guarantee. The paper also lacks comparisons with modern offline RL baselines, which weakens the generality of the performance claims.

major comments (3)
  1. [Section 4.2, Eq. (20)] The Lagrange multiplier update in Eq. (19) relies on the estimate \hat{E}_{\pi^*(\lambda)} from Eq. (20), which averages resource consumption obtained by feeding dataset states to the trained policy. This estimator is unbiased only if the deployed policy induces the same state distribution as the dataset; offline policies generally shift the state distribution through their action choices, so a biased estimate can lead to a systematically incorrect multiplier and hence a violated resource constraint or sacrificed throughput. The paper acknowledges this ('in the offline phase, it is impossible to acquire the accurate value') but provides no empirical comparison between the offline estimate and the actual resource consumption of the deployed policy, and no conservative correction. Because constraint satisfaction is a central advertised advantage over non-Lagrangian baselines, the paper should report this comparison for each environment and constraint level, or provide a principled bound on the estimation error.
  2. [Section 4.1.2, Eq. (15)] The critic is trained by regressing Q_\phi(s_t,a_t) to the Monte Carlo return \sum_{i\ge t}\gamma^{i-t}r_i of the behavior trajectory. This target estimates the value under the behavior policy, not under the guided policy that SOCD deploys; using this critic in the action-selection rules (16) and (17) therefore lacks a clear optimality justification. In addition, the min over two critics does not mitigate bootstrap overestimation in the usual double-Q sense, because the target contains no bootstrap term, and the return-based target may have high variance for T=100 with \gamma=0.8. Please provide a variance or bias analysis, or an ablation showing that the critic target is robust to these concerns.
  3. [Sections 5.3-5.4] The experimental comparison is restricted to BC, SOLAR, Uniform, EDF, and the behavior policy. SOLAR is the authors' prior method, and none of the modern offline RL algorithms that the diffusion component is designed to improve upon (e.g., Diffusion-QL, IQL, IDQL) are included. Without such comparisons, the claim of 'superior performance compared to existing methods' is supported only against a narrow set of baselines. Please add representative offline RL baselines or qualify the claim accordingly.
minor comments (5)
  1. [Algorithm 2, line 9] The reward-update notation 'r = E - \lambda D' is inconsistent with Eq. (7), where r = D - \lambda E; please correct the sign and naming to avoid confusion.
  2. [Notation throughout] The symbol T is used for the episode length (Section 5.1), for the diffusion time horizon in Section 4.1.1, and for the number of critic training steps in Algorithm 2 (line 8); please disambiguate these uses.
  3. [Section 5.3, Eq. (22)] The BC baseline objective in Eq. (22) minimizes an L2 action error, but the text states that the diffusion-based BC model within SOCD is used as the BC baseline; please clarify how the L2 objective applies to a diffusion policy.
  4. [Section 5.2, Table 4] Key hyperparameters K=1024, \alpha=100, and the DPM-solver step count are given without sensitivity analysis; since \alpha is described as governing the trade-off between behavior following and Q-guidance, a brief sensitivity study would help.
  5. [Figure 9] The caption states that all data are averaged across 10 time points, but the x-axis shows 1000 time units; please specify the averaging window and how the curves are smoothed.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central offline-RL derivation is self-contained, with only minor non-load-bearing self-citations to prior scheduling work.

full rationale

The claimed derivation chain is not circular by construction. SOCD's policy is a diffusion behavior-cloning model trained with the score-matching objective (Eq. 12), a critic trained against Monte Carlo returns (Eq. 15), and an action-selection rule that reweights or selects among sampled actions using critic values (Eqs. 16-17). None of these steps fits a parameter to the throughput or resource-constraint numbers that SOCD is later claimed to predict; the reported performance comes from environment rollouts against the baselines, not from the training objective itself. The most likely candidate for circularity is the Lagrange-multiplier update: Eq. (19) uses g'(λ)=E0−Eπ*(λ), and Eq. (20) estimates Eπ*(λ) by feeding dataset states into the learned policy and averaging resource use. This is an acknowledged approximation rather than an identity; the paper states that 'in the offline phase, it is impossible to acquire the accurate value.' The estimate could be biased under state-distribution shift, which is a real correctness risk, but Eq. (20) is not defined in terms of the reported outcome and does not force the deployed resource consumption to equal the dataset estimate. The paper does cite the same group's earlier work: the scheduling formulation follows [72, 22] and SOLAR [43] is a self-cited baseline. These self-citations are not load-bearing in the sense required for circularity: the Lagrangian derivative and dual optimality are standard results also attributed to the external reference [72], and SOLAR is used as a baseline comparison rather than as a justification of SOCD's correctness. The experiments also include non-self-cited baselines (Uniform, EDF, behavior cloning) and real-data environments. Thus the central claim is independently grounded, and only minor self-citation is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. The free parameters are standard RL hyperparameters, though the lack of sensitivity analysis weakens confidence. The axiomatic load rests on prior theoretical results and domain models inherited from the authors' earlier work and cited literature.

free parameters (4)
  • temperature coefficient α = 100
    Governs the trade-off between behavior cloning and Q-guided action selection in Eq (16). Set by hand, no sensitivity analysis is reported.
  • number of sampled behavior actions K = 1024
    Inference-time search budget for the critic-guided selection. Larger K improves the approximation of the target policy but is chosen without analysis of the cost-benefit trade-off.
  • discount factor γ = 0.8
    Used in the critic's Monte Carlo return target in Eq (15). A relatively short-horizon discount, but no sensitivity study is provided.
  • DPM-solver sampling steps = 10
    Number of ODE steps for the reverse diffusion sampling. Affects policy fidelity; no ablation is reported.
assumptions (4)
  • domain assumption The dual function equality T* = min_λ g(λ) and the gradient formula g'(λ) = E0 - E_{π*(λ)} (Remark 1) hold for the scheduling problem.
    Inherited from [72,22], this is the basis for the Lagrange multiplier update in Eq (19). It is a standard result for constrained average-reward MDPs under the assumptions of those papers.
  • standard math The KL-regularized policy objective has the closed-form solution π*(a|s) ∝ μ(a|s) exp(αQ(s,a)) as given in Eq (10).
    Standard result from maximum-entropy RL / AWR, cited from [62,55]. Used to justify the critic-guided action selection.
  • domain assumption The service success probability model P_i(v,c) = 2/(1+exp(-2v/(l_i^3 c))) - 1 (Eq 21) accurately models the wireless downlink.
    All experiments use this model from [8,22]. If real channels differ, the policy's generalization may not transfer.
  • domain assumption The offline dataset generated by RSD4 from [22] is a representative, medium-quality behavior policy dataset for the scheduling problem.
    The method's performance depends on dataset diversity and coverage. No formal characterization is provided, and the data collection pipeline is not independently reproducible from the paper alone.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Offline Diffusion Policy for Multi-User Delay-Constrained Scheduling." pith.science (2026). https://pith.science/paper/4ZOJCYPM

@misc{pith2026250112942,
  author       = {Pith},
  title        = {Pith review of: Offline Diffusion Policy for Multi-User Delay-Constrained Scheduling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4ZOJCYPM}},
  note         = {Machine review of arXiv:2501.12942}
}
read the original abstract

Effective multi-user delay-constrained scheduling is crucial in various real-world applications, including embodied AI, instant messaging, live streaming, and data center management, where efficient resource allocation is required among users with diverse delay sensitivities. In these scenarios, schedulers must make real-time decisions to satisfy both delay and resource constraints without prior knowledge of system dynamics, which are often time-varying and challenging to estimate. {Current learning-based methods typically require online interactions with actual systems during the training stage. Therefore, these approaches are often difficult or impractical, as they can significantly degrade system performance and incur substantial service costs.} To address these challenges, we propose a novel offline reinforcement learning-based algorithm, named \underline{S}cheduling By \underline{O}ffline Learning with \underline{C}ritic Guidance and \underline{D}iffusion Model (SOCD), to learn efficient scheduling policies purely from pre-collected \emph{offline data}. SOCD innovatively employs a diffusion policy, complemented by a sampling-free critic network for policy guidance. By integrating the Lagrangian multiplier optimization into the offline reinforcement learning, SOCD efficiently trains high-quality constraint-aware policies exclusively from available datasets, eliminating the need for online interactions with the system. Experimental results demonstrate that SOCD is resilient to various system dynamics, including partially observable and large-scale environments, and delivers superior performance compared to existing methods.

Figures

Figures reproduced from arXiv: 2501.12942 by the authors.

Figure 1
Figure 1. A four-user delay-constrained single-hop network. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. A four-user delay-constrained multi-hop network: (a) The multi-hop network with user [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The SOCD algorithm: Operating solely in the offline phase, it does not require online [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparison of different algorithms in the Poisson-1hop environment, where arrivals and [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]
Figure 5
Figure 5. Figure 5: Comparison of different algorithms in a 2-hop environment, Poisson-2hop, with arrivals [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 6
Figure 6. Figure 6: Comparison of different algorithms in two environments that utilize real-world data to [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Comparison of different algorithms in a partially observable environment, where the [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]
Figure 8
Figure 8. Figure 8: Comparison of different algorithms as the number of hops increases to 3. The left plot [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: Comparison of various algorithms under an resource constraint of 350 as the number of [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Censored Sampling for Topology Design: Guiding Diffusion with Human Preferences

    cs.LG 2025-08 unverdicted novelty 4.0 of 10

    Guiding a pretrained topology-diffusion generator with human-preference reward classifiers is claimed to suppress floating-material and boundary-violation failure modes without retraining the generator.

Reference graph

Works this paper leans on

87 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [1]

    Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657, 2022

    Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657, 2022. xxvii

    arXiv 2022

  2. [2]

    Deterministic Uncertainty Propagation for Improved Model-Based Offline Reinforcement Learning

    Abdullah Akg¨ ul, Manuel Haußmann, and Melih Kandemir. Deterministic uncertainty propaga- tion for improved model-based offline reinforcement learning.arXiv preprint arXiv:2406.04088, 2024

    work page Pith review arXiv 2024

  3. [3]

    Lyapunov- based optimization of edge resources for energy-efficient adaptive federated learning

    Claudio Battiloro, Paolo Di Lorenzo, Mattia Merluzzi, and Sergio Barbarossa. Lyapunov- based optimization of edge resources for energy-efficient adaptive federated learning. IEEE Transactions on Green Communications and Networking , 7(1):265–280, 2022

    2022

  4. [4]

    Rajarshi Bhattacharyya, Archana Bura, Desik Rengarajan, Mason Rumuly, Bainan Xia, Srini- vas Shakkottai, Dileep Kalathil, Ricky K. P. Mok, and Amogh Dhamdhere. Qflow: A learning approach to high qoe video streaming at the wireless edge. IEEE/ACM Transactions on Net- working, 30(1):32–46, 2022

    2022

  5. [5]

    Score regularized policy optimization through diffusion behavior

    Huayu Chen, Cheng Lu, Zhengyi Wang, Hang Su, and Jun Zhu. Score regularized policy optimization through diffusion behavior. arXiv preprint arXiv:2310.07297 , 2023

    arXiv 2023

  6. [7]

    Offline reinforcement learn- ing via high-fidelity generative behavior modeling

    Huayu Chen, Cheng Lu, Chengyang Ying, Hang Su, and Jun Zhu. Offline reinforcement learn- ing via high-fidelity generative behavior modeling. ArXiv, abs/2209.14548, 2022

    arXiv 2022

  7. [8]

    Timely-throughput optimal scheduling with prediction

    Kun Chen and Longbo Huang. Timely-throughput optimal scheduling with prediction. IEEE/ACM Transactions on Networking , 26(6):2457–2470, 2018

    2018

  8. [9]

    Score approximation, esti- mation and distribution recovery of diffusion models on low-dimensional data

    Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, esti- mation and distribution recovery of diffusion models on low-dimensional data. arXiv preprint arXiv:2302.07194, 2023

    arXiv 2023

Show all 87 references
  1. [10]

    Diffusion policies creating a trust region for offline reinforcement learning

    Tianyu Chen, Zhendong Wang, and Mingyuan Zhou. Diffusion policies creating a trust region for offline reinforcement learning. arXiv preprint arXiv:2405.19690 , 2024

    2024 arXiv

  2. [11]

    The roles of carbon capture, utilization and storage in the transition to a low-carbon energy system using a stochastic optimal scheduling approach

    Xianhao Chen and Xiao Wu. The roles of carbon capture, utilization and storage in the transition to a low-carbon energy system using a stochastic optimal scheduling approach. Journal of Cleaner Production , 366:132860, 2022

    2022

  3. [12]

    Channel estimation for extremely large-scale mimo: Far-field or near-field? IEEE Transactions on Communications , 70(4):2663–2677, 2022

    Mingyao Cui and Linglong Dai. Channel estimation for extremely large-scale mimo: Far-field or near-field? IEEE Transactions on Communications , 70(4):2663–2677, 2022

    2022

  4. [13]

    Data center energy consumption modeling: A survey

    Miyuru Dayarathna, Yonggang Wen, and Rui Fan. Data center energy consumption modeling: A survey. IEEE Communications Surveys & Tutorials , 18(1):732–794, 2016

    2016

  5. [14]

    Channel-aware earliest deadline due fair schedul- ing for wireless multimedia networks.Wireless Personal Communications, 38(2):233–252, 2006

    Khaled MF Elsayed and Ahmed KF Khattab. Channel-aware earliest deadline due fair schedul- ing for wireless multimedia networks.Wireless Personal Communications, 38(2):233–252, 2006

    2006

  6. [15]

    One step diffusion via shortcut models

    Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. arXiv preprint arXiv:2410.12557 , 2024. xxviii

    2024 arXiv

  7. [16]

    A minimalist approach to offline reinforcement learn- ing

    Scott Fujimoto and Shixiang Shane Gu. A minimalist approach to offline reinforcement learn- ing. Advances in neural information processing systems , 34:20132–20145, 2021

    2021

  8. [17]

    Off-policy deep reinforcement learning with- out exploration

    Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning with- out exploration. In International conference on machine learning , pages 2052–2062. PMLR, 2019

    2019

  9. [18]

    Channel estimation for extremely large-scale massive mimo systems

    Yu Han, Shi Jin, Chao-Kai Wen, and Xiaoli Ma. Channel estimation for extremely large-scale massive mimo systems. IEEE Wireless Communications Letters , 9(5):633–637, 2020

    2020

  10. [19]

    Idql: Implicit q-learning as an actor-critic method with diffusion policies

    Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, and Sergey Levine. Idql: Implicit q-learning as an actor-critic method with diffusion policies. arXiv preprint arXiv:2304.10573, 2023

    2023 arXiv

  11. [20]

    Double q-learning

    Hado Hasselt. Double q-learning. Advances in neural information processing systems, 23, 2010

    2010

  12. [21]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems , 33:6840–6851, 2020

    2020

  13. [22]

    Multi-user delay- constrained scheduling with deep recurrent reinforcement learning

    Pihe Hu, Yu Chen, Ling Pan, Zhixuan Fang, Fu Xiao, and Longbo Huang. Multi-user delay- constrained scheduling with deep recurrent reinforcement learning. IEEE/ACM Transactions on Networking , pages 1–16, 2024

    2024

  14. [23]

    When backpressure meets predictive scheduling

    Longbo Huang, Shaoquan Zhang, Minghua Chen, and Xin Liu. When backpressure meets predictive scheduling. IEEE/ACM Transactions on Networking , 24(4):2237–2250, 2015

    2015

  15. [24]

    Planning with diffusion for flexible behavior synthesis

    Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991 , 2022

    2022 arXiv

  16. [25]

    A review of power consumption models of servers in data centers

    Chaoqiang Jin, Xuelian Bai, Chao Yang, Wangxin Mao, and Xin Xu. A review of power consumption models of servers in data centers. Applied Energy, 265:114806, 2020

    2020

  17. [26]

    Joint message-passing and convex optimization framework for energy-efficient surveillance uav scheduling

    Soyi Jung, Joongheon Kim, and Jae-Hyun Kim. Joint message-passing and convex optimization framework for energy-efficient surveillance uav scheduling. Electronics, 9(9):1475, 2020

    2020

  18. [27]

    Factors influencing user satisfaction with information systems: A systematic review

    Leila Kalankesh, Zahra Nasiry, Rebecca Fein, and Shahla Damanabi. Factors influencing user satisfaction with information systems: A systematic review. Galen Medical Journal , 9:1686, 06 2020

    2020

  19. [28]

    Efficient diffusion policies for offline reinforcement learning

    Bingyi Kang, Xiao Ma, Chao Du, Tianyu Pang, and Shuicheng Yan. Efficient diffusion policies for offline reinforcement learning. Advances in Neural Information Processing Systems , 36, 2024

    2024

  20. [29]

    Understanding diffusion objectives as the elbo with simple data augmentation

    Diederik Kingma and Ruiqi Gao. Understanding diffusion objectives as the elbo with simple data augmentation. Advances in Neural Information Processing Systems , 36, 2024. xxix

    2024

  21. [30]

    Variational diffusion models

    Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural information processing systems , 34:21696–21707, 2021

    2021

  22. [31]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  23. [32]

    Optimization energy consumption with multiple mobile sinks using fuzzy logic in wireless sensor networks

    Kambiz Koosheshi and Saeed Ebadi. Optimization energy consumption with multiple mobile sinks using fuzzy logic in wireless sensor networks. Wireless Networks, 25:1215–1234, 2019

    2019

  24. [33]

    Offline reinforcement learn- ing with fisher divergence critic regularization

    Ilya Kostrikov, Rob Fergus, Jonathan Tompson, and Ofir Nachum. Offline reinforcement learn- ing with fisher divergence critic regularization. In International Conference on Machine Learn- ing, pages 5774–5783. PMLR, 2021

    2021

  25. [34]

    Offline reinforcement learning with implicit q-learning

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169 , 2021

    2021 arXiv

  26. [35]

    Stabilizing off- policy q-learning via bootstrapping error reduction.Advances in Neural Information Processing Systems, 32, 2019

    Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. Stabilizing off- policy q-learning via bootstrapping error reduction.Advances in Neural Information Processing Systems, 32, 2019

    2019

  27. [36]

    Conservative q-learning for offline reinforcement learning

    Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in Neural Information Processing Systems , 33:1179– 1191, 2020

    2020

  28. [37]

    Batch reinforcement learning

    Sascha Lange, Thomas Gabel, and Martin Riedmiller. Batch reinforcement learning. In Rein- forcement learning, pages 45–73. Springer, 2012

    2012

  29. [38]

    Graph colouring meets deep learning: Effective graph neural network models for combinatorial problems

    Henrique Lemos, Marcelo Prates, Pedro Avelar, and Luis Lamb. Graph colouring meets deep learning: Effective graph neural network models for combinatorial problems. In 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI) , pages 879–885. IEEE, 2019

    2019

  30. [39]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 , 2020

    2005 arXiv

  31. [40]

    Low-carbon optimal learning scheduling of the power system based on carbon capture system and carbon emission flow theory

    Jifeng Li, Xingtang He, Weidong Li, Mingze Zhang, and Jun Wu. Low-carbon optimal learning scheduling of the power system based on carbon capture system and carbon emission flow theory. Electric Power Systems Research , 218:109215, 2023

    2023

  32. [41]

    Delay-aware vnf scheduling: A rein- forcement learning approach with variable action set

    Junling Li, Weisen Shi, Ning Zhang, and Xuemin Shen. Delay-aware vnf scheduling: A rein- forcement learning approach with variable action set. IEEE Transactions on Cognitive Com- munications and Networking , 7(1):304–318, 2021

    2021

  33. [42]

    To- wards an information theoretic framework of context-based offline meta-reinforcement learning

    Lanqing Li, Hai Zhang, Xinyu Zhang, Shatong Zhu, Junqiao Zhao, and Pheng-Ann Heng. To- wards an information theoretic framework of context-based offline meta-reinforcement learning. arXiv preprint arXiv:2402.02429 , 2024. xxx

    2024 arXiv

  34. [43]

    Offline learning-based multi-user delay-constrained scheduling

    Zhuoran Li, Pihe Hu, and Longbo Huang. Offline learning-based multi-user delay-constrained scheduling. In 2024 IEEE 21st International Conference on Mobile Ad-Hoc and Smart Systems (MASS), pages 92–99. IEEE, 2024

    2024

  35. [44]

    Learning to schedule tasks with deadline and throughput constraints

    Qingsong Liu and Zhixuan Fang. Learning to schedule tasks with deadline and throughput constraints. In IEEE INFOCOM 2023 - IEEE Conference on Computer Communications , pages 1–10, 2023

    2023

  36. [45]

    Off-policy policy gradient with state distribution correction

    Yao Liu, Adith Swaminathan, Alekh Agarwal, and Emma Brunskill. Off-policy policy gradient with state distribution correction. arXiv preprint arXiv:1904.08473 , 2019

    1904 arXiv

  37. [46]

    N. Loi. Predict traffic of lte network. https://www.kaggle.com/naebolo/ predicttraffic-of-lte-network , 2018. Accessed: Jul. 2021

    2018

  38. [47]

    Contrastive energy prediction for exact energy-guided diffusion sampling in offline reinforcement learning

    Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, and Jun Zhu. Contrastive energy prediction for exact energy-guided diffusion sampling in offline reinforcement learning. arXiv preprint arXiv:2304.12824 , 2023

    2023 arXiv

  39. [48]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems , 35:5775–5787, 2022

    2022

  40. [49]

    Contract and lyapunov optimization-based load scheduling and energy management for uav charging stations

    Lingling Lv, Chan Zheng, Lei Zhang, Chun Shan, Zhihong Tian, Xiaojiang Du, and Mohsen Guizani. Contract and lyapunov optimization-based load scheduling and energy management for uav charging stations. IEEE Transactions on Green Communications and Networking , 5(3):1381–1394, 2021

    2021

  41. [50]

    Iteratively refined behavior regularization for offline reinforcement learning

    Yi Ma, HAO Jianye, Xiaohan Hu, Yan Zheng, and Chenjun Xiao. Iteratively refined behavior regularization for offline reinforcement learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2023

    2023

  42. [51]

    Neural adaptive video streaming with pensieve

    Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. Neural adaptive video streaming with pensieve. In ACM SIGCOMM, pages 197–210, 2017

    2017

  43. [52]

    A queuing theory model for fog computing

    Llu ´ ıs Mas, Jordi Vilaplana, Jordi Mateo, and Francesc Solsona. A queuing theory model for fog computing. The Journal of Supercomputing , 78(8):11138–11155, 2022

    2022

  44. [53]

    Power allocation in multi-user cellular networks: Deep reinforcement learning approaches

    Fan Meng, Peng Chen, Lenan Wu, and Julian Cheng. Power allocation in multi-user cellular networks: Deep reinforcement learning approaches. IEEE Transactions on Wireless Commu- nications, 19(10):6255–6267, 2020

    2020

  45. [54]

    Al- gaedice: Policy gradient from arbitrary experience

    Ofir Nachum, Bo Dai, Ilya Kostrikov, Yinlam Chow, Lihong Li, and Dale Schuurmans. Al- gaedice: Policy gradient from arbitrary experience. arXiv preprint arXiv:1912.02074 , 2019

    1912 arXiv

  46. [55]

    Awac: Accelerating online reinforcement learning with offline datasets

    Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. Awac: Accelerating online reinforcement learning with offline datasets. arXiv preprint arXiv:2006.09359 , 2020. xxxi

    2006 arXiv

  47. [56]

    Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks

    Yasar Sinan Nasir and Dongning Guo. Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks. IEEE Journal on Selected Areas in Communications , 37(10):2239–2250, 2019

    2019

  48. [57]

    Plan better amid conservatism: Offline multi-agent reinforcement learning with actor rectification

    Ling Pan, Longbo Huang, Tengyu Ma, and Huazhe Xu. Plan better amid conservatism: Offline multi-agent reinforcement learning with actor rectification. In International Conference on Machine Learning, pages 17221–17237. PMLR, 2022

    2022

  49. [58]

    Energy-optimal scheduling of mobile cloud computing based on a modified lyapunov optimization method

    Su Pan and Yuqing Chen. Energy-optimal scheduling of mobile cloud computing based on a modified lyapunov optimization method. In GLOBECOM 2017 - 2017 IEEE Global Commu- nications Conference, pages 1–6, 2017

    2017

  50. [59]

    Energy-optimal scheduling of mobile cloud computing based on a modified lyapunov optimization method

    Su Pan and Yuqing Chen. Energy-optimal scheduling of mobile cloud computing based on a modified lyapunov optimization method. IEEE Transactions on Green Communications and Networking, 3(1):227–235, 2018

    2018

  51. [60]

    Is value learning really the main bottleneck in offline rl? arXiv preprint arXiv:2406.09329 , 2024

    Seohong Park, Kevin Frans, Sergey Levine, and Aviral Kumar. Is value learning really the main bottleneck in offline rl? arXiv preprint arXiv:2406.09329 , 2024

    2024 arXiv

  52. [61]

    Online convex optimization for caching networks

    Georgios S Paschos, Apostolos Destounis, and George Iosifidis. Online convex optimization for caching networks. IEEE/ACM Transactions on Networking , 28(2):625–638, 2020

    2020

  53. [62]

    Advantage-weighted regres- sion: Simple and scalable off-policy reinforcement learning

    Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine. Advantage-weighted regres- sion: Simple and scalable off-policy reinforcement learning. arXiv preprint arXiv:1910.00177 , 2019

    1910 arXiv

  54. [63]

    Youtube live and twitch: a tour of user-generated live streaming systems

    Karine Pires and Gwendal Simon. Youtube live and twitch: a tour of user-generated live streaming systems. In Proceedings of the 6th ACM Multimedia Systems Conference , MMSys ’15, page 225–230, New York, NY, USA, 2015. Association for Computing Machinery

    2015

  55. [64]

    Learn- ing to solve np-complete problems: A graph neural network for decision tsp

    Marcelo Prates, Pedro HC Avelar, Henrique Lemos, Luis C Lamb, and Moshe Y Vardi. Learn- ing to solve np-complete problems: A graph neural network for decision tsp. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 4731–4738, 2019

    2019

  56. [65]

    Class-balancing diffusion models

    Yiming Qin, Huangjie Zheng, Jiangchao Yao, Mingyuan Zhou, and Ya Zhang. Class-balancing diffusion models. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18434–18443, 2023

    2023

  57. [66]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in neural information processing systems , 20, 2007

    2007

  58. [67]

    Ramachandran, B

    P. Ramachandran, B. Zoph, and Q. V. Le. Swish: A self-gated activation function. arXiv preprint arXiv:1710.05941, 2017

    2017 arXiv

  59. [68]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 , 2022. xxxii

    2022 arXiv

  60. [69]

    Offline reinforcement learning as anti-exploration

    Shideh Rezaeifar, Robert Dadashi, Nino Vieillard, L´ eonard Hussenot, Olivier Bachem, Olivier Pietquin, and Matthieu Geist. Offline reinforcement learning as anti-exploration. InProceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 8106–8114, 2022

    2022

  61. [70]

    Simple near-optimal scheduling for the m/g/1

    Ziv Scully, Mor Harchol-Balter, and Alan Scheller-Wolf. Simple near-optimal scheduling for the m/g/1. ACM SIGMETRICS Performance Evaluation Review , 47(2):24–26, 2019

    2019

  62. [71]

    Keep doing what worked: Behavioral modelling priors for offline reinforcement learning

    Noah Y Siegel, Jost Tobias Springenberg, Felix Berkenkamp, Abbas Abdolmaleki, Michael Neunert, Thomas Lampe, Roland Hafner, Nicolas Heess, and Martin Riedmiller. Keep doing what worked: Behavioral modelling priors for offline reinforcement learning. arXiv preprint arXiv:2002.0...

    2002 arXiv

  63. [72]

    Rahul Singh and P. R. Kumar. Throughput optimal decentralized scheduling of multihop net- works with end-to-end deadline constraints: Unreliable links. IEEE Transactions on Automatic Control, 64(1):127–142, 2019

    2019

  64. [73]

    Deep unsuper- vised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265. PMLR, 2015

    2015

  65. [74]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  66. [75]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems , 32, 2019

    2019

  67. [76]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  68. [77]

    Batch learning from logged bandit feed- back through counterfactual risk minimization

    Adith Swaminathan and Thorsten Joachims. Batch learning from logged bandit feed- back through counterfactual risk minimization. The Journal of Machine Learning Research , 16(1):1731–1755, 2015

    2015

  69. [78]

    Minimizing age of information with power constraints: Multi-user opportunistic scheduling in multi-state time-varying channels

    Haoyue Tang, Jintao Wang, Linqi Song, and Jian Song. Minimizing age of information with power constraints: Multi-user opportunistic scheduling in multi-state time-varying channels. IEEE Journal on Selected Areas in Communications , 38(5):854–868, 2020

    2020

  70. [79]

    Taotao, X

    W. Taotao, X. Jiantao, X. Wensen, C. Yucheng, and Z. Shengli. Wireless signal strength on 2.4 ghz (wss24) dataset. https://github.com/postman511/Wireless-Signal-Strength-on-2. 4GHz-WSS24-dataset, 2021. Accessed: Feb. 17, 2022

    2021

  71. [80]

    Learning combinatorial optimization on graphs: A survey with applications to networking.IEEE Access, 8:120388–120416, 2020

    Natalia Vesselinova, Rebecca Steinert, Daniel F Perez-Ramirez, and Magnus Boman. Learning combinatorial optimization on graphs: A survey with applications to networking.IEEE Access, 8:120388–120416, 2020. xxxiii

    2020

  72. [81]

    Scheduling real-time wireless traffic: A network-aided offline reinforcement learning approach

    Jialin Wan, Sen Lin, Zhaofeng Zhang, Junshan Zhang, and Tao Zhang. Scheduling real-time wireless traffic: A network-aided offline reinforcement learning approach. IEEE Internet of Things Journal , 2023

    2023

  73. [82]

    Logistics-involved task scheduling in cloud manufacturing with offline deep reinforcement learning

    Xiaohan Wang, Lin Zhang, Yongkui Liu, and Chun Zhao. Logistics-involved task scheduling in cloud manufacturing with offline deep reinforcement learning. Journal of Industrial Infor- mation Integration, 34:100471, 2023

    2023

  74. [83]

    Diffusion policies as an expressive policy class for offline reinforcement learning

    Zhendong Wang, Jonathan J Hunt, and Mingyuan Zhou. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.06193 , 2022

    2022 arXiv

  75. [84]

    Pro- lificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems , 36, 2024

    2024

  76. [85]

    Behavior regularized offline reinforcement learn- ing

    Yifan Wu, George Tucker, and Ofir Nachum. Behavior regularized offline reinforcement learn- ing. arXiv preprint arXiv:1911.11361 , 2019

    1911 arXiv

  77. [86]

    Understanding instant messaging traffic characteristics

    Zhen Xiao, Lei Guo, and John Tracey. Understanding instant messaging traffic characteristics. In 27th International Conference on Distributed Computing Systems (ICDCS ’07) , pages 51– 51, 2007

    2007

  78. [87]

    Offline reinforcement learning for wireless network optimization with mixture datasets

    Kun Yang, Chengshuai Shi, Cong Shen, Jing Yang, Shu-ping Yeh, and Jaroslaw J Sydir. Offline reinforcement learning for wireless network optimization with mixture datasets. IEEE Transactions on Wireless Communications, 2024

    2024

  79. [88]

    Reles: A neural adaptive multipath scheduler based on deep reinforcement learning

    Han Zhang, Wenzhong Li, Shaohua Gao, Xiaoliang Wang, and Baoliu Ye. Reles: A neural adaptive multipath scheduler based on deep reinforcement learning. In IEEE INFOCOM 2019 - IEEE Conference on Computer Communications , pages 1648–1656, 2019. xxxiv

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.