A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report

Brooke Shrader; Eytan Modiano; Jerrod Wigmore

REVIEW 5 major objections 6 minor 1 cited by

A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A policy network that forces switch-type structure—more queue length means no lower chance of serving that queue—learns faster and generalizes far better than an MLP on queueing-network control.

desk verdict A clean architectural idea—enforcing switch-type structure via monotonic per-component scoring—with suggestive but statistically underpowered empirical support. read the letter →

arxiv 2501.11136 v1 pith:23CHUSJC submitted 2025-01-19 cs.LG cs.SYeess.SY

Jerrod Wigmore , Brooke Shrader , Eytan Modiano This is my paper · ORCID

classification cs.LGcs.SYeess.SY MSC 68T0790B2290C40

keywords switch-typepolicyqueueingnetworkcontrolresourceallocationdeepreinforcementlearningPPOzero-shotgeneralizationmonotonicneuralsampleefficiency

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a policy network architecture for learned queueing-network control that restricts the learned policy to be switch-type: if the state of one component improves, the probability of choosing that component cannot decrease. The authors argue that this built-in structure, inspired by MaxWeight and similar policies, makes training with Proximal Policy Optimization more sample-efficient and prevents the policy from overfitting to the training environments. They report that the switch-type network matches a standard MLP policy on environments seen during training and clearly outperforms it on unseen environments, with a normalized average cost of 0.870 versus 1.20 on unseen single-hop scheduling environments after removing MLP runs that failed to stabilize. The sympathetic reading is that restricting the policy class to switch-type behavior gives better generalization because the true optimal policies for these problems are themselves switch-type.

What carries the argument

The architecture replaces the early layers of the policy network with monotonic hidden layers: each layer computes $\sigma_{RN}(\exp(W^{(l)}) z^{(l-1)} + b^{(l)})$, where exponentiated weights keep the map componentwise non-decreasing and the bounded ReLU-N activation allows stacked layers to represent any monotone function. The network is applied separately to each component state $s_k$ (queue length, capacity, and environment parameters), and a softmax converts the per-component scores into action probabilities. Lemma 1 guarantees the result is a stochastic switch-type policy: increasing any element of $s_k$ can only increase the probability of choosing action $k$. This is the mechanism that shrinks the policy class and blocks overfitting.

What would settle it

Take the environments that the sampling procedure discarded because their baseline average cost exceeded 200, train both an STN and an MLP policy on them with the same PPO setup, and compare stabilized costs; if the MLP stabilizes them or achieves lower cost, the switch-type bias is not the reason for the reported generalization gains. A second check is to test on a problem with a provably non-switch-type optimal policy, where the STN should underperform if its inductive bias is doing the work.

Watch

Extended reading notes

Core claim

The central claim is that a policy network whose architecture enforces switch-type structure is better for learning queueing-network control policies than a standard MLP. In single-environment training, the switch-type network reaches a cost below the MaxWeight or Shortest-Queue baseline with fewer environment samples, and after training on five environments it matches the MLP on those environments while clearly outperforming it on ninety-five unseen ones. For single-hop scheduling, the normalized average cost on unseen environments is 0.870 for the switch-type network versus 1.20 for the MLP after dropping the MLP runs that did not stabilize; with those runs included, the gap is much larger. The authors read the near-equal training-time performance as evidence that the optimal policies for these two problem classes are switch-type.

Load-bearing premise

Every training and test environment is kept only if a simple switch-type policy (MaxWeight or Shortest Queue) achieves an estimated average cost below 200, so the comparison may silently exclude environments where non-switch-type policies would be competitive or necessary.

Editorial extensions

If this is right

An STN policy trained with PPO reaches a cost below the MaxWeight or Shortest-Queue baseline in fewer training steps than an MLP policy on the same environment.
On environments seen during training, the STN matches the MLP, so narrowing the policy class to switch-type policies does not cost performance.
On unseen single-hop scheduling and multi-path routing environments, the STN maintains low normalized costs while the MLP frequently fails to stabilize the network.
The authors interpret the training-time parity as evidence that the optimal policies for these resource-allocation problems are switch-type.
The same architecture can replace an MLP policy network in a PPO pipeline for queueing-network control without changing the learning algorithm.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same inductive bias may transfer to other queueing-control objectives where index or threshold policies are natural, such as age-of-information minimization or restless-bandit index policies, but the paper does not test those settings.
Because every environment in the study passed a switch-type baseline's stability filter (estimated average cost below 200), the comparison likely under-represents settings where a non-switch-type policy is needed; environments rejected by that filter are exactly where an MLP could plausibly win.
A natural extension is to test the STN on problems with provably non-switch-type optimal policies; the architecture would then be expected to underperform an unrestricted network, which would bound the scope of the claim.
The monotone construction could also be applied to the value network in addition to the policy network, potentially stabilizing PPO training further, though the paper does not report that experiment.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: this paper combines monotonic neural networks with per-component scoring and a softmax to make a stochastic switch-type policy for queueing control. That is a genuinely new and simple design, and Lemma 1 is correct. If the empirical claims hold up, it gives DRL practitioners a cheap architectural change that improves sample efficiency and zero-shot generalization. The paper deserves a serious referee, but the evidence as reported is not yet conclusive.

What the paper does well: the architecture is clean, the connection to MaxWeight and Whittle-index policies is sensible, and the zero-shot results are striking—the STN maintains normalized cost around 0.87 on unseen single-hop environments while the MLP averages 1.20 even after rejecting 19 of 95 outliers. The STN also never fails to stabilize, whereas the MLP sometimes blows up. That pattern is consistent across two problem classes, which is more than a single anecdote. The authors also compare against MaxWeight/Shortest-Queue baselines and show the STN beats them, which gives the reader a useful reference point.

Where the soft spots are, in proportion: the biggest issue is statistical. All training curves and final comparisons come from a single run per architecture. PPO in queueing control can be high-variance, especially near instability, and the paper itself reports different learning rates for STN and MLP (3e-3 vs 3e-4), so part of the sample-efficiency gap could be hyperparameter choice rather than architecture. The post-hoc outlier rejection is also worrying: omitting exactly the contexts where the MLP fails, and not applying the same rule to the STN, flatters the comparison. The observation encoding was selected on the training environments, which is a form of peeking. And the environment sampling procedure filters out environments that a switch-type baseline cannot stabilize under cost 200, which may bias the test set toward problems friendly to switch-type policies—though this applies equally to both architectures, so it weakens the claim about absolute performance more than the relative STN-vs-MLP comparison.

No code or data is provided, which makes the empirical claims hard to verify. Even with those caveats, the core idea is sound, the writing is honest about what was done, and the limitation statements in the text (single run, learning-rate sweep, outlier rejection) are not hidden. This is a good technical report that would benefit from multiple seeds, error bars, and code release.

Who is this for: people working on DRL for queueing networks or on inductive biases in policy architectures. I would bring it to a reading group, and I would cite the STN architecture if I worked in this area. Send it to peer review—but require the authors to run multiple seeds and release code before acceptance.

Referee Report

5 major / 6 minor

Summary. The paper proposes a switch-type neural network (STN) policy architecture for DRL-based control of queueing and resource-allocation systems. The STN applies a monotonic neural network to each component of the state and then a softmax over the resulting scores, so the induced stochastic policy is switch-type: increasing a component's state cannot decrease the probability of choosing that component. The authors compare STN with a conventional MLP policy under PPO on single-hop scheduling and multi-path routing. They report (i) higher sample efficiency during training, (ii) comparable final performance on training environments, and (iii) substantially better zero-shot generalization on 95 unseen environments per problem class. The claimed advantage is attributed to the inductive bias of switch-type structure.

Significance. The proposed architecture is simple and plausibly useful: if the empirical claims hold, it gives the DRL community a cheap structural prior for queueing controls that is supported by known switch-type optimality results, and the paper's comparison across two problem classes is a reasonable first test. The paper is also transparent in reporting outlier-rejected means and normalized costs relative to MaxWeight/Shortest Queue. However, the central evidence is currently under-powered: single training runs, no error bars over training seeds, an environment filter that selects for switch-type-stabilizable systems, and an observation encoding chosen after experiments on the training environments. These issues are fixable but currently limit the force of the generalization and sample-efficiency claims.

major comments (5)

[Section II-A, Eq. (1); Section IV-A1] The objective in Eq. (1) and all quantitative results depend on a cost function c(s), but the paper never defines it. The environment-sampling filter in Section IV-A1 uses an empirical average cost threshold of 200, and the normalized cost in Eq. (11) is computed with the same undefined c(s). Without a definition of c(s) (for example, whether it is the total queue length, a weighted sum, or something else), the reader cannot interpret the tables, the stability filter, or the statements about outperforming by factors of 8 and 38.7, and the experiments cannot be reproduced. Please state c(s) explicitly and justify the threshold 200 relative to that cost.
[Section IV-A1; Section IV-A2; Section V] The environment-generation procedure filters out any environment whose MaxWeight (single-hop) or Shortest-Queue (routing) baseline does not achieve an estimated average cost below 200, and the same procedure is used to build the 100-environment set for zero-shot evaluation. Because MaxWeight and Shortest-Queue are switch-type policies, this filtering selects for systems that are already stabilizable by a switch-type baseline and may exclude systems in which a non-switch-type policy would be competitive. This creates selection pressure in favor of the STN architecture and weakens the fairness of the zero-shot comparison. Please report the number of environments rejected by the filter and provide results on a broader or unfiltered environment distribution, or at least vary the threshold to show the conclusions are not an artifact of this choice.
[Sections IV-B, IV-C, V-A, V-C; Tables II-III; Figs. 4, 5, 7] Each experimental condition is a single PPO training run per architecture, and the learning curves and final-cost comparisons have no error bars or confidence intervals over training seeds. The word 'significantly' in Section V-C is therefore not supported by the reported statistics. In addition, the outlier-rejected means remove 1 of 5 training contexts and 19 of 95 test contexts, all for the MLP, and those are precisely the environments where the MLP failed to stabilize; the factor-of-38.7 claim in Section V-C is based on the un-rejected means. Please repeat training with multiple seeds, report paired comparisons, and include a metric that accounts for all contexts (for example, median cost, fraction of stabilized environments, or a truncated mean applied symmetrically to both architectures).
[Section V-B] The signs of the observation encoding were not fixed a priori: they were selected by training STN policies with different encodings on five environments and keeping the one that gave the best result. If those five environments are the same training environments used in the zero-shot study, then the encoding is part of the model selection and contributes to the reported generalization gain. Please state explicitly whether the encoding search was limited to the training set, justify the chosen encoding from domain principles, or report the performance of all encodings considered so the reader can judge the sensitivity.
[Section IV-B; Table I] The paper says the architectures use the same hyperparameters except learning rate, but it never specifies the number of layers, hidden widths, activation details, or initialization for either the STN or the MLP, which prevents reproduction. The two architectures also use different learning rates (3e-3 for STN and 3e-4 for MLP) selected separately; this means the sample-efficiency comparison is not purely an architectural comparison. Please provide complete architecture specifications and, if possible, a matched-learning-rate or multi-learning-rate sensitivity analysis.

minor comments (6)

[Abstract and throughout] There are repeated typos and grammatical errors, e.g., 'Our works presents', 'Massachusets', 'the used of', 'an point', 'Its apparent', and 'Meaning at 20% of the contexts were ignored'. A careful proofread is needed.
[Section II-B] The sentence containing 'πsw(s) = 1 . ).' has a stray punctuation artifact that should be cleaned up.
[Eq. (8)] The definition of ReLU-N uses limit notation and arrows in a way that is difficult to parse; it would be clearer as a standard piecewise function with clipping to [0,N].
[Figure 4] The captions list '(b) Environment MP2' and '(d) Environment MP2'; one of these should refer to MP1.
[Figures 6 and 7] The axis labels and legends render poorly, and Figure 7 has no error bars; at a minimum, clarify the axis meaning and units.
[Section VI] The conclusion's phrase 'suggesting that switch-type policies are optimal' is stronger than the evidence supports and should be qualified as an empirical hypothesis, especially since the paper defers optimality proofs to future work.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the STN construction is a self-contained architectural design, and the empirical comparisons, though subject to benchmark-filtering caveats, are not forced by definition or by self-citation.

full rationale

The paper's central derivation is the construction of a stochastic switch-type policy from componentwise monotone scores: Definition 3.1 defines the target property, Lemma 1 proves that SoftMax of a decomposable componentwise monotone function satisfies it, and the monotonic hidden layers (exponentiated weights plus ReLU-N) are cited to external prior work [38], [39] for their monotonicity and approximation guarantees. That chain is not circular: the property is proved from the construction, not assumed by it. The empirical claims are benchmark comparisons, not identities. The environment-sampling filter, which rejects environments where a switch-type baseline has estimated average cost above 200 (Sections IV-A1 and IV-A2), restricts the evaluation distribution to environments stabilizable by a switch-type policy; this is a potential fairness or generalization concern, but it does not make the STN's measured normalized costs equal to the filter by construction, and the MLP could in principle have achieved lower costs on those environments. The observation-encoding search in Section V-B and the per-architecture learning-rate sweep in Section IV-B are model-selection choices, not fitted parameters subsequently renamed as predictions. The optimality language in Section VI is explicitly hedged with 'We believe' and 'Future research will focus on demonstrating,' so it is an interpretive extrapolation rather than a derived theorem. The self-citations [5], [7], [10], and [11] appear as background examples and are not load-bearing for the architecture's validity, which rests on Lemma 1 and on external monotonic-network results. Concerns about single training runs and post-hoc outlier rejection are statistical robustness issues, not circularity. No circular step meeting the quote-and-reduction standard was found.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central empirical claims rest on a small number of hand-chosen values: the threshold used to filter environments, the sign encoding of observations, and per-architecture learning rates. The theoretical justification borrows the monotonic network approximation results from prior work, and the optimality of switch-type policies is a motivating conjecture rather than a proven premise.

free parameters (3)

Environment stabilizability threshold for filtering = 200
Environments are resampled when the baseline MaxWeight or Shortest-Queue policy's estimated average cost exceeds 200. This hand-chosen threshold changes the distribution of both training and test environments (Section IV-A1, IV-A2).
Observation encoding signs for zero-shot experiments = ok=(qk, yk, λk, -μk) for single-hop; ok=(-qk, yk, μk) for multi-path
Different sign encodings were tried by training STN on five environments, and the best-performing encoding was selected (Section V-B). This is a fitted modeling choice for the zero-shot evaluation.
Learning rates per architecture = STN 3e-3, MLP 3e-4
Learning rates were selected by sweeping 10^-2 to 10^-5 to minimize evaluation cost for each architecture (Section IV-B, Table I). This is hyperparameter tuning, but it is an architecture-specific choice that affects the comparison.

assumptions (4)

domain assumption The queueing network is an MDP with independent component transitions and the objective is long-run average cost, as in Eq. (1).
The entire formulation (Section II-A) relies on this Markovian, independence, and average-cost structure; real networks may have correlated arrivals or time-varying statistics.
standard math The ReLU-N monotonic network class from Kim and Lee [38] can represent the needed monotonic functions.
The paper relies on MonotonicNets' universal approximation guarantees for monotonic functions to justify using the STN architecture (Section III-A1).
ad hoc to paper The optimal policy for the studied scheduling and routing problems is switch-type.
This is stated as an interpretation of the experiments in Sections IV-C and VI, not proven. The paper's significance claim depends on it, and the authors defer a proof to future work.
ad hoc to paper A concrete cost function c(s) exists and is used in all experiments, but it is never explicitly defined.
Equation (1) defines the objective with c(s_k,t), and experiments compute J(pi, tau) from c(s_t), but the text never states what c(.) is, e.g., whether it is the sum of queue lengths (Section II-A, IV-C).

invented entities (1)

Switch-type neural network (STN) architecture independent evidence
purpose: Restricts learned policies to the switch-type class by applying a shared monotone scoring function to each queue state and feeding scores through softmax.
The architecture is implemented and evaluated in simulation in this paper; it is a design artifact, not a natural entity, but it has empirical support from the reported experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report." pith.science (2026). https://pith.science/paper/23CHUSJC

@misc{pith2026250111136,
  author       = {Pith},
  title        = {Pith review of: A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/23CHUSJC}},
  note         = {Machine review of arXiv:2501.11136}
}

read the original abstract

Deep Reinforcement Learning (DRL) has become a powerful tool for developing control policies in queueing networks, but the common use of Multi-layer Perceptron (MLP) neural networks in these applications has significant drawbacks. MLP architectures, while versatile, often suffer from poor sample efficiency and a tendency to overfit training environments, leading to suboptimal performance on new, unseen networks. In response to these issues, we introduce a switch-type neural network (STN) architecture designed to improve the efficiency and generalization of DRL policies in queueing networks. The STN leverages structural patterns from traditional non-learning policies, ensuring consistent action choices across similar states. This design not only streamlines the learning process but also fosters better generalization by reducing the tendency to overfit. Our works presents three key contributions: first, the development of the STN as a more effective alternative to MLPs; second, empirical evidence showing that STNs achieve superior sample efficiency in various training scenarios; and third, experimental results demonstrating that STNs match MLP performance in familiar environments and significantly outperform them in new settings. By embedding domain-specific knowledge, the STN enhances the Proximal Policy Optimization (PPO) algorithm's effectiveness without compromising performance, suggesting its suitability for a wide range of queueing network control problems.

Figures

Figures reproduced from arXiv: 2501.11136 by the authors.

**Figure 3.** Decision regions for the policy πP I . Blue denotes πP I (s) = 1 and red denotes πP I (s) = 2. The x-axis corresponds to q1 and the y-axis corresponds to q2. Each plot corresponds to a different set of service states y = (y1, y2). Thus the set of plots represents the decision region over the truncated state-space of (q1, q2) ∈ [0, 20] ∩ (y1, y2) ∈ [1, 2] any point (q1, q2) on any scatter plot. If the point is blue, … view at source ↗

**Figure 4.** Moving average cost of the training policy versus [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: Average cost of the trained policies in the same [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Histogram of Jˆ 0(πθ, E (j) ) for the test environments E (j) ∈ Etest for the single-hop scheduling environments (top) and multi-path routing environments (bottom) . Note the “break” on the x-axis halfway through the plot, where the scaling changes, where the right hal…

**Figure 7.** Figure 7: Moving average cost of the training policy averaged [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Multi-Robot Multi-Queue Control via Exhaustive Assignment Actor-Critic Learning
eess.SY 2026-04 conditional novelty 5.0 of 10

An exhaustive-assignment actor-critic that only learns idle-robot reassignment outperforms ESL under asymmetric multi-robot multi-queue arrivals and remains near-optimal when benchmarks exist.

Reference graph

Works this paper leans on

40 extracted references · 37 canonical work pages · cited by 1 Pith paper

[1]

Queueing Network Controls via Deep Reinforcement Learning,

J. G. Dai and M. Gluzman, “Queueing Network Controls via Deep Reinforcement Learning,” Stochastic Systems , vol. 12, no. 1, pp. 30– 67, mar 2022, publisher: INFORMS

work page 2022
[2]

Reinforcement learning in queues,

U. Ayesta, “Reinforcement learning in queues,” Queueing Systems, vol. 100, no. 3, pp. 497–499, apr 2022

work page 2022
[3]

Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service,

M. Raeis, A. Tizghadam, and A. Leon-Garcia, “Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service,” Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 1, pp. 461–468, may 2021, number: 1

work page 2021
[4]

Deep Reinforcement Learning for Smart Queue Management,

H. Fawaz, D. Zeghlache, T. A. Pham, J. Leguay, and P. Medagliani, “Deep Reinforcement Learning for Smart Queue Management,” Elec- tronic Communications of the EASST , vol. 80, sep 2021

work page 2021
[5]

Intervention-Assisted Policy Gradient Methods for Online Stochastic Queuing Network Optimization: Technical Report

J. Wigmore, B. Shrader, and E. Modiano, “Intervention-Assisted Policy Gradient Methods for Online Stochastic Queuing Network Optimization: Technical Report,” apr 2024, arXiv:2404.04106 [cs]

work page Pith review arXiv 2024
[6]

Approximation theory of the MLP model in neural net- works,

A. Pinkus, “Approximation theory of the MLP model in neural net- works,” Acta Numerica, vol. 8, pp. 143–195, jan 1999

work page 1999
[7]

Scheduling Algorithms for Minimizing Age of Information in Wireless Broadcast Networks with Random Arrivals: The No-Buffer Case

Y .-P. Hsu, E. Modiano, and L. Duan, “Scheduling Algorithms for Minimizing Age of Information in Wireless Broadcast Networks with Random Arrivals: The No-Buffer Case,” sep 2018, arXiv:1712.07419 [cs, math]

work page Pith review arXiv 2018
[8]

Dynamic server allocation to parallel queues with randomly varying connectivity,

L. Tassiulas and A. Ephremides, “Dynamic server allocation to parallel queues with randomly varying connectivity,” IEEE Transactions on Information Theory, vol. 39, no. 2, pp. 466–478, mar 1993, conference Name: IEEE Transactions on Information Theory

work page 1993

Show all 40 references

[9]

Stability and Asymptotic Optimality of Generalized MaxWeight Policies,

S. Meyn, “Stability and Asymptotic Optimality of Generalized MaxWeight Policies,” SIAM Journal on Control and Optimization , vol. 47, no. 6, pp. 3259–3294, jan 2009, publisher: Society for Industrial and Applied Mathematics

2009
[10]

Minimizing the Age of Information in Wireless Networks with Stochastic Arrivals,

I. Kadota and E. Modiano, “Minimizing the Age of Information in Wireless Networks with Stochastic Arrivals,” IEEE Transactions on Mobile Computing, vol. 20, no. 3, pp. 1173–1185, mar 2021, conference Name: IEEE Transactions on Mobile Computing

2021
[11]

Tracking MaxWeight: Optimal Control for Partially Observable and Controllable Networks,

B. Liu, Q. Liang, and E. Modiano, “Tracking MaxWeight: Optimal Control for Partially Observable and Controllable Networks,”IEEE/ACM Transactions on Networking , vol. PP, no. 99, pp. 1–13, 2022

2022
[12]

MaxWeight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic,

A. L. Stolyar, “MaxWeight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic,” The Annals of Applied Probability , vol. 14, no. 1, feb 2004

2004
[13]

An analysis of the join the shortest queue (JSQ) policy,

H.-C. Lin and C. Raghavendra, “An analysis of the join the shortest queue (JSQ) policy,” in [1992] Proceedings of the 12th International Conference on Distributed Computing Systems , jun 1992, pp. 362–366

1992
[14]

Stochastic Network Optimization with Application to Com- munication and Queueing Systems,

M. Neely, “Stochastic Network Optimization with Application to Com- munication and Queueing Systems,” Synthesis Lectures on Communica- tion Networks, vol. 3, no. 1, pp. 1–211, 2010

2010
[15]

Restless bandits: activity allocation in a changing world,

P. Whittle, “Restless bandits: activity allocation in a changing world,” Journal of Applied Probability , vol. 25, no. A, pp. 287–298, jan 1988

1988
[16]

Dynamic priority allocation via restless bandit marginal productivity indices,

J. Ni ˜no-Mora, “Dynamic priority allocation via restless bandit marginal productivity indices,” TOP, vol. 15, no. 2, pp. 161–198, oct 2007

2007
[17]

Whit- tle’s index policy for a multi-class queueing system with convex holding costs,

M. O’Keeffe, P. S. Ansell, K. D. Glazebrook, and J. Nio-Mora, “Whit- tle’s index policy for a multi-class queueing system with convex holding costs,” Mathematical Methods of Operations Research (ZOR) , vol. 57, no. 1, pp. 21–39, apr 2003

2003
[18]

Congestion control of TCP flows in Internet routers by means of index policy,

K. Avrachenkov, U. Ayesta, J. Doncel, and P. Jacko, “Congestion control of TCP flows in Internet routers by means of index policy,” Computer Networks, vol. 57, no. 17, pp. 3463–3478, dec 2013

2013
[19]

Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access,

K. Liu and Q. Zhao, “Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access,” IEEE Transactions on Information Theory , vol. 56, no. 11, pp. 5547–5567, nov 2010, conference Name: IEEE Transactions on Information Theory

2010
[20]

A unifying computations of Whittle’s Index for Markovian bandits,

U. Ayesta, M. K. Gupta, and I. M. Verloop, “A unifying computations of Whittle’s Index for Markovian bandits,” arXiv: Optimization and Control, jun 2019

2019
[21]

Optimal control of a queueing system with two heterogeneous servers,

W. Lin and P. Kumar, “Optimal control of a queueing system with two heterogeneous servers,” IEEE Transactions on Automatic Control , vol. 29, no. 8, pp. 696–703, aug 1984, conference Name: IEEE Trans- actions on Automatic Control

1984
[22]

A simple proof of the optimality of a threshold policy in a two-server queueing system,

G. Koole, “A simple proof of the optimality of a threshold policy in a two-server queueing system,” Systems & Control Letters , vol. 26, no. 5, pp. 301–303, dec 1995

1995
[23]

Extension of the optimality of the threshold policy in heterogeneous multiserver queueing systems,

I. Viniotis and A. Ephremides, “Extension of the optimality of the threshold policy in heterogeneous multiserver queueing systems,” IEEE Transactions on Automatic Control , vol. 33, no. 1, pp. 104–109, jan 1988, conference Name: IEEE Transactions on Automatic Control

1988
[24]

Monotone Control of Queueing Systems with Heterogeneous Servers,

V . Rykov, “Monotone Control of Queueing Systems with Heterogeneous Servers,” Queueing Systems, vol. 37, no. 4, pp. 391–403, mar 2001

2001
[25]

D. P. Bertsekas, Dynamic Programming and Optimal Control, Vol 1
[26]

Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs,

L. I. Sennott, “Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs,” Operations Research, vol. 37, no. 4, pp. 626–633, 1989, publisher: INFORMS

1989
[27]

R. S. Sutton and A. G. Barto, Reinforcement learning: an introduction , second edition ed., ser. Adaptive computation and machine learning series. Cambridge, Massachusetts: The MIT Press, 2018

2018
[28]

Proxi- mal Policy Optimization Algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proxi- mal Policy Optimization Algorithms,” aug 2017, arXiv:1707.06347 [cs]

2017 arXiv
[29]

A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning,

A. Zhang, N. Ballas, and J. Pineau, “A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning,”ArXiv, jun 2018

2018
[30]

A Survey of Zero-shot Generalisation in Deep Reinforcement Learning,

R. Kirk, A. Zhang, E. Grefenstette, and T. Rockt ¨aschel, “A Survey of Zero-shot Generalisation in Deep Reinforcement Learning,” Journal of Artificial Intelligence Research , vol. 76, pp. 201–264, jan 2023, arXiv:2111.09794 [cs]

2023 arXiv
[31]

Quantifying Generalization in Reinforcement Learning,

K. Cobbe, O. Klimov, C. Hesse, T. Kim, and J. Schulman, “Quantifying Generalization in Reinforcement Learning,” dec 2018

2018
[32]

Neuroevolution of self-interpretable agents,

Y . Tang, D. Nguyen, and D. Ha, “Neuroevolution of self-interpretable agents,” in Proceedings of the 2020 Genetic and Evolutionary Compu- tation Conference, ser. GECCO ’20. New York, NY , USA: Association for Computing Machinery, jun 2020, pp. 414–424

2020
[33]

The Sensory Neuron as a Transformer: Permutation- Invariant Neural Networks for Reinforcement Learning,

Y . Tang and D. Ha, “The Sensory Neuron as a Transformer: Permutation- Invariant Neural Networks for Reinforcement Learning,” sep 2021, arXiv:2109.02869 [cs]

2021 arXiv
[34]

Unsupervised Visual Attention and Invariance for Reinforcement Learning,

X. Wang, L. Lian, and S. X. Yu, “Unsupervised Visual Attention and Invariance for Reinforcement Learning,” 2021, pp. 6677–6687

2021
[35]

Deep reinforcement learning with relational inductive biases,

V . Zambaldi, D. Raposo, A. Santoro, V . Bapst, Y . Li, I. Babuschkin, K. Tuyls, D. Reichert, T. Lillicrap, E. Lockhart, M. Shanahan, V . Langston, R. Pascanu, M. Botvinick, O. Vinyals, and P. Battaglia, “Deep reinforcement learning with relational inductive biases,” sep 2018

2018
[36]

DEEP REINFORCEMENT LEARNING WITH RELATIONAL INDUCTIVE BIASES,

——, “DEEP REINFORCEMENT LEARNING WITH RELATIONAL INDUCTIVE BIASES,” 2019

2019
[37]

Neuro-algorithmic Policies Enable Fast Combinatorial Generalization,

M. Vlastelica, M. Rolinek, and G. Martius, “Neuro-algorithmic Policies Enable Fast Combinatorial Generalization,” in Proceedings of the 38th International Conference on Machine Learning . PMLR, jul 2021, pp. 10 575–10 585, iSSN: 2640-3498

2021
[38]

Scalable Monotonic Neural Networks,

H. Kim and J.-S. Lee, “Scalable Monotonic Neural Networks,” oct 2023

2023
[39]

Bounded activation functions for enhanced training stability of deep neural networks on visual pattern recognition problems,

S. S. Liew, M. Khalil-Hani, and R. Bakhteri, “Bounded activation functions for enhanced training stability of deep neural networks on visual pattern recognition problems,” Neurocomputing, vol. 216, pp. 718–734, dec 2016

2016
[40]

Certified Monotonic Neural Networks,

X. Liu, X. Han, N. Zhang, and Q. Liu, “Certified Monotonic Neural Networks,” dec 2022, arXiv:2011.10219 [cs]

2022 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Queueing Network Controls via Deep Reinforcement Learning,

J. G. Dai and M. Gluzman, “Queueing Network Controls via Deep Reinforcement Learning,” Stochastic Systems , vol. 12, no. 1, pp. 30– 67, mar 2022, publisher: INFORMS

work page 2022

[2] [2]

Reinforcement learning in queues,

U. Ayesta, “Reinforcement learning in queues,” Queueing Systems, vol. 100, no. 3, pp. 497–499, apr 2022

work page 2022

[3] [3]

Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service,

M. Raeis, A. Tizghadam, and A. Leon-Garcia, “Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service,” Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 1, pp. 461–468, may 2021, number: 1

work page 2021

[4] [4]

Deep Reinforcement Learning for Smart Queue Management,

H. Fawaz, D. Zeghlache, T. A. Pham, J. Leguay, and P. Medagliani, “Deep Reinforcement Learning for Smart Queue Management,” Elec- tronic Communications of the EASST , vol. 80, sep 2021

work page 2021

[5] [5]

Intervention-Assisted Policy Gradient Methods for Online Stochastic Queuing Network Optimization: Technical Report

J. Wigmore, B. Shrader, and E. Modiano, “Intervention-Assisted Policy Gradient Methods for Online Stochastic Queuing Network Optimization: Technical Report,” apr 2024, arXiv:2404.04106 [cs]

work page Pith review arXiv 2024

[6] [6]

Approximation theory of the MLP model in neural net- works,

A. Pinkus, “Approximation theory of the MLP model in neural net- works,” Acta Numerica, vol. 8, pp. 143–195, jan 1999

work page 1999

[7] [7]

Scheduling Algorithms for Minimizing Age of Information in Wireless Broadcast Networks with Random Arrivals: The No-Buffer Case

Y .-P. Hsu, E. Modiano, and L. Duan, “Scheduling Algorithms for Minimizing Age of Information in Wireless Broadcast Networks with Random Arrivals: The No-Buffer Case,” sep 2018, arXiv:1712.07419 [cs, math]

work page Pith review arXiv 2018

[8] [8]

Dynamic server allocation to parallel queues with randomly varying connectivity,

L. Tassiulas and A. Ephremides, “Dynamic server allocation to parallel queues with randomly varying connectivity,” IEEE Transactions on Information Theory, vol. 39, no. 2, pp. 466–478, mar 1993, conference Name: IEEE Transactions on Information Theory

work page 1993