Pith. sign in

REVIEW 3 major objections 5 minor 57 references

Learning Not to Optimize: Physics-Informed Action-Space Reshaping for Intent-Based Network Control

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that cheap structural signals—law residuals, state–intent symmetries, and dominance—can safely remove most candidate network deployments before value-based ranking, and it proves conditions under which this removal loses no

desk verdict Solid, honest integration of known reduction ideas with explicit error accounting; the gap between exact theory and the approximate pipeline is real but clearly disclosed. read the letter →

arxiv 2608.00908 v1 pith:K3YHYPHF submitted 2026-08-02 cs.NI cs.LG

Zuyuan Zhang , Vaneet Aggarwal , Tian Lan This is my paper · ORCID
classification cs.NIcs.LG
keywords intent-basednetworkingconstrainedMarkovdecisionprocessaction-spacereductionquotientactionspaceresidualscreeningdominancepruningnetworkpolicycontrollearningnottooptimize
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that before a value function is accurate enough to pick the best network deployment, cheaper intermediate signals can already rule out many candidates: actions that violate hard network laws, actions equivalent under relabeling, and actions that leave a uniformly worse future state. It proposes LNO-QRD, a shadow process that screens, quotients, and dominance-prunes the candidate set before the primal optimizer ranks it. This turns the optimization problem into a smaller one, with lossless reduction guarantees under equivariance and monotonicity, bounded frontier size, and an explicit performance-gap bound. If right, intent-based network controllers can run faster and more safely without needing a perfect value function.

What carries the argument

The central object is the reduced frontier $B_\lambda(x) = \mathrm{Top}^{\psi_\lambda}_{K_{\mathrm{fr}}}(\mathrm{ND}_{\lambda,I}(F^I_\varepsilon(\mathcal{C}_N(x))/\Gamma_x))$: a pipeline that screens candidates by law residual, quotients by the state–intent stabilizer $\Gamma_x$, removes dominated quotient classes, and keeps the top-K by a diagnostic priority score. It is carried by two structural conditions: equivariance of the deployment interface under the relabeling group $\Gamma$ (Assumption 1) and monotone fixed-$\lambda$ value under the state preorder (Assumption 2). These make quotienting and dominance lossless, so the expensive primal value ranking only needs to see a smaller set.

What would settle it

Run exhaustive search on a small 8- to 10-node integer placement instance and check whether a candidate that dominates another by the paper's badness vector is ever the unique optimal action under the true value function; one such instance would falsify the lossless dominance claim and show that the monotonicity assumption fails under integrality.

Watch

Extended reading notes

Core claim

The central claim is that candidate reduction itself can be lossless or nearly lossless when it is guided by structural certificates rather than learned scores alone. Theorems 1 and 2, together with Corollary 1, show that under an equivariant deployment interface and monotone fixed-λ values, keeping one representative per state–intent orbit and removing dominated candidates preserves at least one optimal generated executable deployment. LNO-QRD operationalizes this with residual screening against executable network laws, exact or signature quotienting, dominance pruning via a normalized badness vector, and a budgeted composite-priority frontier. Theorem 6 bounds the value gap of the resultin

Load-bearing premise

The lossless dominance result stands on the assumption that a network state with more leftover capacity, lower queue pressure, and better reliability can never lead to a worse optimal future value; real integrality and coupling constraints can break this monotonicity, which the paper acknowledges by excluding non-monotone coordinates.

Editorial extensions

If this is right

  • Under exact screening, quotienting, and dominance, at least one optimal generated executable deployment is preserved, making the reduction lossless on the generated candidate set.
  • The retained frontier size is bounded by the law-survival fraction, non-dominated fraction, stabilizer size, and original candidate count divided by orbit size; ranking cost shrinks whenever reduction overhead is smaller than the saved scoring cost.
  • Approximation loss decomposes by stage—generation, residual tolerance, quotienting, dominance, and top-K truncation—and is bounded by stage slack plus twice the critic error, so failures are attributable to a named stage.
  • The reduced Boltzmann policy's value gap is bounded by $(\delta+2\epsilon+\tau\log N_{\max})/(1-\gamma)$, making frontier coverage, energy error, temperature, frontier size, and discounting explicit trade-offs.
  • In the paper's experiments, the full pipeline cuts candidate counts by 75.9% on small instances and by 73.0% on average in large instances while improving utility, violation, and post-generation latency over all compared baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to apply the same not-to-optimize pattern to other combinatorial decision pipelines—virtual network embedding, job scheduling, chip placement—where constraints are cheaply checkable and equivalence classes exist; the paper only demonstrates network deployment.
  • The 73–76% reduction suggests that much of the compute in RL-based network control is spent ranking provably irrelevant options; an explicit-certificate pre-filter could complement any policy optimizer, not just the PPO-style learner used here.
  • A testable extension would replace the monotone fixed-λ assumption with a learned per-coordinate monotonicity mask or with stochastic-dominance checks on short rollouts, potentially widening lossless dominance to regimes where Assumption 2 fails.
  • The near-oracle coverage on small instances hints that the core bottleneck in intent-based control may be candidate generation and pre-ranking filtering rather than policy expressiveness; a generator with better coverage would make the reduction even closer to lossless.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces LNO-QRD, a pre-ranking reduction layer for intent-conditioned network deployment. It combines law-residual screening, quotienting under state–intent automorphisms, dominance pruning, and a budgeted top-K frontier to shrink the candidate set before expensive primal ranking. The theoretical core consists of: exact quotienting under an equivariant deployment interface (Theorem 1), lossless dominance pruning under a monotone fixed-λ value assumption (Theorem 2), preservation of one optimal generated executable deployment under exact versions of all stages (Corollary 1), a frontier-size and ranking-cost bound (Theorem 3), stagewise frontier-loss accounting (Theorem 4), an error bound for approximate intermediate signals (Theorem 5), and a fixed-λ value gap for the resulting Boltzmann policy (Theorem 6). Experiments on small and large placement-control benchmarks, using a shared candidate generator, report 75.9% candidate reduction with 90.8% near-oracle coverage on small instances, and best utility, IntentSat, violation, reduction, and latency on large instances, with ablations attributing the gains to the individual reduction stages.

Significance. The core idea—excluding candidates on the basis of cheap structural or law-based signals before expensive value-based ranking—is timely and potentially useful for constrained network control and for constrained RL more broadly. The paper's explicit assumptions, stagewise loss accounting, and ablation study are commendable: the experiments use a shared generator across candidate-based baselines, and the ablation isolates the contribution of each certificate. The theoretical statements are standard and the proof sketches are plausible, provided the stated assumptions hold. However, the lossless dominance guarantee is load-bearing for the main claim, and it rests on an assumption about the finite-candidate optimal value that is neither proved nor empirically checked, while the implemented dominance certificate is acknowledged to be only conditionally exact. Because dominance contributes the largest share of the reported compression, the paper's headline lossless claims are not yet established for the system actually evaluated.

major comments (3)
  1. [§III-C, Assumption 2, Eq. (28), Theorem 2] Theorem 2's conclusion Q*_λ(x,a) ≥ Q*_λ(x,b) relies on Assumption 2, which states that V*_λ is increasing under the state preorder. But V*_λ(x) is defined in Eq. (28) as the maximum over the finite executable generated set \tilde E_N(x), which depends on the generator C_N(x). Even if the underlying network dynamics are monotone, a state with more slack can have a generator that returns fewer or worse candidates, and the fallback action ⊥ can also have different values across states. Thus the proof step E[V*_λ(x')|x,a] ≥ E[V*_λ(x')|x,b] is not justified. This is not a minor technicality: Corollary 1's “preserves at least one optimal generated executable deployment” is the central lossless claim, and Table IV shows that removing dominance cuts reduction from 0.731 to 0.402, i.e., dominance is the largest single compression source. The paper should either prove a generator-side condition th
  2. [§III-C, Definition 2 vs. practical dominance certificate] Definition 2 requires stochastic dominance of the transition kernels. The online certificate instead checks coordinate-wise inequality of the normalized badness vector b(x,a). The paper states that this certificate is exact only when the badness-vector order implies Definition 2, but no theorem or experiment establishes that implication for the benchmark dynamics. Consequently, the reduction numbers in Tables II–IV and Figures 1 and 5, which are produced with this certificate, are not covered by the lossless Theorem 2; their losses are merely booked to Δ_dom. Since the reported compression is the paper's main experimental selling point, the authors should either (i) prove or empirically test that the badness-vector order implies Definition 2 in these deployment models, or (ii) present the experimental reductions as approximate and provide the corresponding η_i and ϵ_Q bounds from Theorem
  3. [§IV, Theorem 5 and reporting of approximation terms] The error accounting framework in Theorem 5 is appropriate, but the paper does not report the estimated slacks η_i(x) or the critic error ϵ_Q for the approximate stages used in the experiments. Without these quantities, the reader cannot tell how far the practical system is from the exact-lossless regime. In particular, the approximate dominance certificate and signature-based quotienting are explicitly approximate, so the corresponding η_i terms are essential for interpreting the 75.9% reduction / 90.8% coverage result as a manifestation of the theoretical bounds rather than as an unexplained empirical outcome. Reporting these terms would strengthen the claimed connection between Theorems 4–5 and the experimental results.
minor comments (5)
  1. [Abstract] The acronym appears as both LNO-QRD and LNOQRD; please use a single consistent form.
  2. [§V, Metrics] The metrics “Coverage” and “OracleGap” are not fully specified. What is the threshold for “near-oracle”, and how is the preserved fraction computed over episodes? This matters for interpreting the 90.8% coverage number.
  3. [§III-D, Eq. (35)] The notation bUub(x,a) is introduced without a clear definition. Please define this learned upper utility score and how it is obtained/trained.
  4. [Algorithm 1] Line 7 says “using exact or certified approximate dominance”; the term “certified” is not defined. Please clarify what makes an approximate dominance certificate certified.
  5. [References] References [30] and [32] give the conference name but no year or pages; please complete the bibliographic entries.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; theoretical claims are conditional on explicit assumptions and experiments are empirical, though self-citations are numerous.

full rationale

The derivation chain is self-contained. Theorem 1 is proved from Assumption 1 (equivariant deployment interface) via the Bellman operator's unique fixed point; Theorem 2 is proved from Assumption 2 (monotone fixed-lambda value) via stochastic dominance. Neither conclusion is defined in terms of the assumption's conclusion. Assumption 2 is the main fragility: V*_lambda in (28) is defined over the finite generated candidate set eEN(x), so monotonicity depends on the candidate generator and can fail even when network dynamics are monotone; the paper explicitly excludes non-monotone coordinates and charges approximate-certificate error to the dominance term in Theorem 5. This is an unverified assumption and a correctness risk, not a circular reduction. Theorem 3's bound is a definitional bookkeeping identity (orbit-stabilizer and fractions alpha_law, alpha_nd), not a fitted prediction; Theorem 4's error accounting is a telescoping identity by construction, used only to localize losses. Experimental results are measured directly against a shared generator and external baselines; no fitted parameter is relabeled as a prediction. The manuscript contains many self-citations ([10], [21], [25]-[26], [30]-[32], [36]-[38], [46]), but none is load-bearing: the core theorems rely on standard references [11], [22]-[24], [27]-[29] and on proofs included in the text. There is no imported uniqueness theorem and no ansatz smuggled in via citation. Thus the correct finding is no significant circularity; score of 2 reflects only the density of non-load-bearing self-citation.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities or postulates. The analysis relies on standard MDP assumptions plus three domain-specific assumptions (equivariance, monotonicity, uniform coverage/energy accuracy) that the authors explicitly state. The free parameters are mostly hyperparameters of the learning and screening pipeline; none are fitted constants inserted into the theorems to force a result.

free parameters (7)
  • Screening tolerance epsilon
    Threshold for residual screening (Res_law <= epsilon). Controls how many candidates survive the first stage; not derived from first principles.
  • Frontier budget K_fr = 16 small, 24 large
    Maximum number of candidates kept after top-K selection; set by the experimenter based on computational budget.
  • Multiplier lambda (CMDP)
    Lagrange multiplier for the constrained objective. Fixed in theory, learned via primal-dual updates in practice.
  • Policy temperature tau
    Temperature in the Boltzmann policy (35). Affects exploration and the entropy term in Theorem 6.
  • Diagnostic priority weights w_phi, w_d, w_c, w_r, w_s, w_p, w_u
    Weights in the composite priority psi_lambda used for top-K selection. Chosen by hand or tuned, not derived.
  • Learning objective weights c_v, beta_g, beta_ord, beta_inv, beta_H
    Hyperparameters balancing PPO loss, value fitting, order loss, invariance loss, and entropy. Tuned for experiments.
  • Normalization scales sigma_nu
    Per-coordinate normalizers in the law residual (15). Must be chosen to make different units comparable.
assumptions (6)
  • domain assumption Assumption 1: Equivariant deployment interface (e_EN(gamma x) = gamma e_EN(x), r_lambda invariance, P_X equivariance)
    Used to prove Theorem 1 (lossless quotienting). Requires the network dynamics and rewards to respect all relabelings in the symmetry group Gamma. Real networks may not be fully equivariant under all attributes.
  • domain assumption Assumption 2: Monotone fixed-lambda value (x+ ⪰_X x- implies V*_lambda(x+) >= V*_lambda(x-))
    Used to prove Theorem 2 (lossless dominance). This is the weakest point: residual capacity monotonicity is not guaranteed in combinatorial network optimization.
  • domain assumption Assumption 3: Uniform reduced-frontier coverage (delta bound on value loss from restricting to B_lambda(x))
    Used in Theorem 6 to bound the suboptimality of the reduced policy. It is a restatement that the reduction does not lose too much, rather than a free condition.
  • domain assumption Assumption 4: Uniform energy approximation (|-E_theta(x,a) - Q*_lambda(x,a)| <= epsilon on B_lambda(x))
    Used in Theorem 6. Requires the learned energy to approximate the true fixed-lambda Q-value uniformly on the reduced frontier.
  • domain assumption Finite candidate set C_N(x) generated by a fixed generator G
    Everything operates on a generated finite set. If the generator misses all near-optimal actions, the reduction cannot recover them. This is shared with all candidate-based baselines but is still an unstated premise for the practical value of LNO-QRD.
  • standard math Bellman optimality and contraction for the fixed-lambda CMDP
    Used implicitly in the proof sketches of Theorems 1, 2, and 6 (unique fixed point, performance-difference lemma). Standard in MDP theory.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Not to Optimize: Physics-Informed Action-Space Reshaping for Intent-Based Network Control." pith.science (2026). https://pith.science/paper/K3YHYPHF

@misc{pith2026260800908,
  author       = {Pith},
  title        = {Pith review of: Learning Not to Optimize: Physics-Informed Action-Space Reshaping for Intent-Based Network Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3YHYPHF}},
  note         = {Machine review of arXiv:2608.00908}
}
abstract

Modern network policy control maps intent to sequential placement-control decisions. Bellman-style policy optimization primarily asks which action to optimize, while constraints are commonly handled through penalty, barrier, or Lagrangian mechanisms. We observe that before a value function can certify the best deployment, intermediate signals may already identify many candidates that should be excluded from further optimization. This motivates a complementary direction: \emph{Learning Not to Optimize}. Before a value function is accurate enough to select the best placement-control decision, intermediate signals may already show that candidates are equivalent under state--intent relabeling (quotienting), lead to a uniformly worse future state (dominance), or violate executable network laws (residual screening). \LNOQRD{} uses these computed or learned signals as a shadow process to reshape the domain on which primal policy optimization is performed, thereby reducing the action space. We prove lossless quotienting and dominance under explicit equivariance and monotonicity conditions, bound frontier size and ranking cost, and quantify losses from approximate certificates and primal estimates. Experiments show that \LNOQRD{} reduces small-instance candidates by $75.9\%$ while retaining $90.8\%$ near-oracle coverage and, on large instances, achieves the highest utility and intent satisfaction, the lowest hard-law violation and post-generation latency, and a $73.0\%$ average reduction among candidate-based baselines.

Figures

Figures reproduced from arXiv: 2608.00908 by the authors.

Figure 1
Figure 1. Small-instance frontier diagnostics. LNO-QRD progressively shrinks [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Budget-induced reduction–gap tradeoff. A larger frontier budget [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. F. Scalability and Dual-Learning Overhead Scalability and overhead results test the speedup condition in Theorem 3 and (33). As scale grows, the retained-frontier ratio |Bλ(x)|/|CN (x)| remains substantially below one, so the primal scorer is applied to a compact subset rather than the full candidate pool. The overhead decomposition further shows that residual evaluation, quotienting, dominance testing, and [PITH_F… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Scalability and overhead. Panel (a) shows the retained-frontier ratio [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 4
Figure 4. Figure 4: Feasibility diagnostics. LNO-QRD reduces multiple residual coordi [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 6
Figure 6. Figure 6: Ablation summary. Residual screening mainly improves feasibility, [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 34 canonical work pages

  1. [1]

    Intent-based networking-concepts and definitions,

    A. Clemm, L. Ciavaglia, L. Z. Granville, and J. Tantsura, “Intent-based networking-concepts and definitions,” 2022

    work page 2022

  2. [2]

    Intent classification,

    C. Li, O. Havel, A. Olariu, P. Martinez-Julia, J. C. Nobre, and D. Lopez, “Intent classification,”RFC 9316, 2022

    work page 2022

  3. [3]

    A survey on intent-based networking,

    A. Leivadeas and M. Falkner, “A survey on intent-based networking,” IEEE Communications Surveys & Tutorials, vol. 25, no. 1, pp. 625–655, 2022

    2022

  4. [4]

    Service function chaining (sfc) architec- ture,

    J. Halpern and C. Pignataro, “Service function chaining (sfc) architec- ture,” Tech. Rep., 2015

    work page 2015

  5. [5]

    Network function virtualization: State-of-the-art and re- search challenges,

    R. Mijumbi, J. Serrat, J.-L. Gorricho, N. Bouten, F. De Turck, and R. Boutaba, “Network function virtualization: State-of-the-art and re- search challenges,”IEEE Communications surveys & tutorials, vol. 18, no. 1, pp. 236–262, 2015

    work page 2015

  6. [6]

    A survey on service function chaining,

    D. Bhamare, R. Jain, M. Samaka, and A. Erbad, “A survey on service function chaining,”Journal of Network and Computer Applications, vol. 75, pp. 138–155, 2016

    work page 2016

  7. [7]

    Network slicing in 5g: Survey and challenges,

    X. Foukas, G. Patounas, A. Elmokashfi, and M. K. Marina, “Network slicing in 5g: Survey and challenges,”IEEE communications magazine, vol. 55, no. 5, pp. 94–100, 2017

    work page 2017

  8. [8]

    Network slicing for 5g with sdn/nfv: Con- cepts, architectures, and challenges,

    J. Ordonez-Lucena, P. Ameigeiras, D. Lopez, J. J. Ramos-Munoz, J. Lorca, and J. Folgueira, “Network slicing for 5g with sdn/nfv: Con- cepts, architectures, and challenges,”IEEE Communications Magazine, vol. 55, no. 5, pp. 80–87, 2017

    work page 2017

Show all 57 references
  1. [9]

    A survey on the placement of virtual network functions,

    J. Sun, Y . Zhang, F. Liu, H. Wang, X. Xu, and Y . Li, “A survey on the placement of virtual network functions,”Journal of Network and Computer Applications, vol. 202, p. 103361, 2022

    2022

  2. [10]

    Lisfc-search: Lifelong search for network sfc optimization under non-stationary drifts,

    Z. Zhang, V . Aggarwal, and T. Lan, “Lisfc-search: Lifelong search for network sfc optimization under non-stationary drifts,” inIEEE INFO- COM 2026-IEEE Conference on Computer Communications. IEEE, 2026, pp. 1–6

    2026

  3. [11]

    M. L. Puterman,Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014

    2014

  4. [12]

    Resource man- agement with deep reinforcement learning,

    H. Mao, M. Alizadeh, I. Menache, and S. Kandula, “Resource man- agement with deep reinforcement learning,” inProceedings of the 15th ACM workshop on hot topics in networks, 2016, pp. 50–56

    2016

  5. [13]

    Learning scheduling algorithms for data processing clusters,

    H. Mao, M. Schwarzkopf, S. B. Venkatakrishnan, Z. Meng, and M. Al- izadeh, “Learning scheduling algorithms for data processing clusters,” in Proceedings of the ACM special interest group on data communication, 2019, pp. 270–288

    2019

  6. [14]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,”arXiv preprint arXiv:1609.02907, 2016

    2016 arXiv

  7. [15]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y . Ben- gio, “Graph attention networks,”arXiv preprint arXiv:1710.10903, 2017

    2017 arXiv

  8. [16]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  9. [17]

    Set trans- former: A framework for attention-based permutation-invariant neural networks,

    J. Lee, Y . Lee, J. Kim, A. Kosiorek, S. Choi, and Y . W. Teh, “Set trans- former: A framework for attention-based permutation-invariant neural networks,” inInternational conference on machine learning. PMLR, 2019, pp. 3744–3753

    2019

  10. [18]

    Constrained policy optimization,

    J. Achiam, D. Held, A. Tamar, and P. Abbeel, “Constrained policy optimization,” inInternational conference on machine learning. Pmlr, 2017, pp. 22–31

    2017

  11. [19]

    Distributed age-of-information scheduling with noma via deep reinforcement learning,

    C. Zhang, Y . Zou, Z. Zhang, D. Yu, J. T. G ´omez, T. Lan, F. Dressler, and X. Cheng, “Distributed age-of-information scheduling with noma via deep reinforcement learning,”IEEE Transactions on Mobile Computing, vol. 24, no. 1, pp. 30–44, 2024

    2024

  12. [20]

    A distributed abstract mac layer for cooperative learning on internet of vehicles,

    Y . Zou, Z. Zhang, C. Zhang, Y . Zheng, D. Yu, and J. Yu, “A distributed abstract mac layer for cooperative learning on internet of vehicles,”IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 8, pp. 8972–8983, 2024

    2024

  13. [21]

    Network diffuser for placing- scheduling service function chains with inverse demonstration,

    Z. Zhang, V . Aggarwal, and T. Lan, “Network diffuser for placing- scheduling service function chains with inverse demonstration,” in IEEE INFOCOM 2025-IEEE Conference on Computer Communications. IEEE, 2025, pp. 1–10

    2025

  14. [22]

    Equivalence notions and model minimization in markov decision processes,

    R. Givan, T. Dean, and M. Greig, “Equivalence notions and model minimization in markov decision processes,”Artificial intelligence, vol. 147, no. 1-2, pp. 163–223, 2003

    2003

  15. [23]

    Ravindran,An algebraic approach to abstraction in reinforcement learning

    B. Ravindran,An algebraic approach to abstraction in reinforcement learning. University of Massachusetts Amherst, 2004

    2004

  16. [24]

    Mdp homomorphic networks: Group symmetries in reinforcement learning,

    E. Van der Pol, D. Worrall, H. van Hoof, F. Oliehoek, and M. Welling, “Mdp homomorphic networks: Group symmetries in reinforcement learning,”Advances in Neural Information Processing Systems, vol. 33, pp. 4199–4210, 2020

    2020

  17. [25]

    Operator-guided invariance learning for continuous reinforcement learning,

    Z. Zhang, F. X. Yu, and T. Lan, “Operator-guided invariance learning for continuous reinforcement learning,”arXiv preprint arXiv:2605.06500, 2026

    2026 arXiv

  18. [26]

    Look-ahead robust network optimization with generative state predictions,

    F. X. Yu, Z. Zhang, E. Grob, G. Adam, S. Coffey, N. D. Bastian, and T. Lan, “Look-ahead robust network optimization with generative state predictions,” inAAAI 2025 Workshop on Artificial Intelligence for Wireless Communications and Networking (AI4WCN), 2024

    2025

  19. [27]

    D. M. Topkis,Supermodularity and complementarity. Princeton university press, 1998

    1998

  20. [28]

    Structural properties of stochastic dynamic programs,

    J. E. Smith and K. F. McCardle, “Structural properties of stochastic dynamic programs,”Operations Research, vol. 50, no. 5, pp. 796–809, 2002

    2002

  21. [29]

    An approximate dynamic programming algorithm for monotone value functions,

    D. R. Jiang and W. B. Powell, “An approximate dynamic programming algorithm for monotone value functions,”Operations research, vol. 63, no. 6, pp. 1489–1511, 2015

    2015

  22. [30]

    Geometric coherence learning for structuring value functions in plain mdps,

    Z. Zhang, Z. Fang, and T. Lan, “Geometric coherence learning for structuring value functions in plain mdps,” inForty-third International Conference on Machine Learning

  23. [31]

    Geometry of drifting mdps with path- integral stability certificates,

    Z. Zhang, M. Imani, and T. Lan, “Geometry of drifting mdps with path- integral stability certificates,”arXiv preprint arXiv:2601.21991, 2026

    2026

  24. [32]

    Hodgeflow policy search by topolog- ically dissecting temporal-difference signals in non-markovian environ- ments,

    Z. Zhang, S. Tang, and T. Lan, “Hodgeflow policy search by topolog- ically dissecting temporal-difference signals in non-markovian environ- ments,” inForty-third International Conference on Machine Learning

  25. [33]

    Altman,Constrained Markov decision processes

    E. Altman,Constrained Markov decision processes. Routledge, 2021

    2021

  26. [34]

    Reward constrained policy optimization,

    C. Tessler, D. J. Mankowitz, and S. Mannor, “Reward constrained policy optimization,”arXiv preprint arXiv:1805.11074, 2018

    2018 arXiv

  27. [35]

    Ipo: Interior-point policy optimization under constraints,

    Y . Liu, J. Ding, and X. Liu, “Ipo: Interior-point policy optimization under constraints,” inProceedings of the AAAI conference on artificial intelligence, vol. 34, no. 04, 2020, pp. 4940–4947

    2020

  28. [36]

    Br-defedrl: Byzantine-robust decentralized federated reinforce- ment learning with fast convergence and communication efficiency,

    J. Qiao, Z. Zhang, S. Yue, Y . Yuan, Z. Cai, X. Zhang, J. Ren, and D. Yu, “Br-defedrl: Byzantine-robust decentralized federated reinforce- ment learning with fast convergence and communication efficiency,” in Ieee infocom 2024-ieee conference on computer communications. IEEE, ...

    2024

  29. [37]

    Nonzero: Interaction-guided exploration for multi-agent monte carlo tree search,

    S. Tang, Z. Zhang, M. Imani, and T. Lan, “Nonzero: Interaction-guided exploration for multi-agent monte carlo tree search,”arXiv preprint arXiv:2605.00751, 2026

    2026 arXiv

  30. [38]

    Counterfactual regret minimization-mixing for noncooperative stochastic spectrum games with imperfect information,

    Z. Zhang, L. Liu, N. D. Bastian, and T. Lan, “Counterfactual regret minimization-mixing for noncooperative stochastic spectrum games with imperfect information,”IEEE Transactions on Networking, 2026

    2026

  31. [39]

    Use cases and practices for intent-based networking,

    K. Yao, D. Chen, J. Jeong, Q. Wu, C. Yang, L. Contreras, and G. Fioc- cola, “Use cases and practices for intent-based networking,”Internet Engineering Task Force, Internet-Draft draft-irtf-nmrg-ibnusecases-00, 2025

    2025

  32. [40]

    Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks,

    L. Tassiulas and A. Ephremides, “Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks,” in29th IEEE Conference on Decision and Control. IEEE, 1990, pp. 2130–2132

    1990

  33. [41]

    Georgiadis, M

    L. Georgiadis, M. J. Neely, and L. Tassiulas,Resource allocation and cross-layer control in wireless networks. Now Publishers Inc, 2006

    2006

  34. [42]

    Boyd and L

    S. Boyd and L. Vandenberghe,Convex optimization. Cambridge university press, 2004

    2004

  35. [43]

    C. H. Papadimitriou and K. Steiglitz,Combinatorial optimization: algorithms and complexity. Courier Corporation, 1998

    1998

  36. [44]

    Deep sets,

    M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola, “Deep sets,”Advances in neural information processing systems, vol. 30, 2017

    2017

  37. [45]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  38. [46]

    Metric-gradient projection for stable multi-agent policy learning,

    Z. Zhang, S. Tang, M. Imani, and T. Lan, “Metric-gradient projection for stable multi-agent policy learning,”arXiv preprint arXiv:2605.18809, 2026

    2026 arXiv

  39. [47]

    Intent-driven autonomous network and service management in future cellular networks: A struc- tured literature review,

    K. Mehmood, K. Kralevska, and D. Palma, “Intent-driven autonomous network and service management in future cellular networks: A struc- tured literature review,”Computer Networks, vol. 220, p. 109477, 2023

    2023

  40. [48]

    Applications of deep reinforcement learning in communications and networking: A survey,

    N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang, Y .-C. Liang, and D. I. Kim, “Applications of deep reinforcement learning in communications and networking: A survey,”IEEE communications surveys & tutorials, vol. 21, no. 4, pp. 3133–3174, 2019

    2019

  41. [49]

    Leveraging deep reinforcement learning for traffic engineering: A survey,

    Y . Xiao, J. Liu, J. Wu, and N. Ansari, “Leveraging deep reinforcement learning for traffic engineering: A survey,”IEEE Communications Sur- veys & Tutorials, vol. 23, no. 4, pp. 2064–2097, 2021

  42. [50]

    Bounding performance loss in approximate mdp homomorphisms,

    J. Taylor, D. Precup, and P. Panagaden, “Bounding performance loss in approximate mdp homomorphisms,”Advances in Neural Information Processing Systems, vol. 21, 2008

    2008

  43. [51]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,

    M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational physics, vol. 378, pp. 686–707, 2019

    2019

  44. [52]

    Network flows: Theory, algorithms, and applications,

    G. R. Waissi, “Network flows: Theory, algorithms, and applications,” 1994

    1994

  45. [53]

    T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein,Introduction to algorithms. MIT press, 2022

    2022

  46. [54]

    Learning to rank using gradient descent,

    C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender, “Learning to rank using gradient descent,” inProceed- ings of the 22nd international conference on Machine learning, 2005, pp. 89–96

    2005

  47. [55]

    A survey on mobile edge computing: The communication perspective,

    Y . Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,”IEEE communications surveys & tutorials, vol. 19, no. 4, pp. 2322–2358, 2017

    2017

  48. [56]

    Software-defined networking: A comprehensive survey,

    D. Kreutz, F. M. Ramos, P. E. Verissimo, C. E. Rothenberg, S. Azodol- molky, and S. Uhlig, “Software-defined networking: A comprehensive survey,”Proceedings of the IEEE, vol. 103, no. 1, pp. 14–76, 2014

    2014

  49. [57]

    Data center network virtualization: A survey,

    M. F. Bari, R. Boutaba, R. Esteves, L. Z. Granville, M. Podlesny, M. G. Rabbani, Q. Zhang, and M. F. Zhani, “Data center network virtualization: A survey,”IEEE communications surveys & tutorials, vol. 15, no. 2, pp. 909–928, 2012

    2012

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.