Pith. sign in

REVIEW 5 major objections 5 minor 52 references

Structured Reinforcement Learning for Bayesian Persuasion : Application to Intelligent Interactive Driving

T0 review · 5 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that in dynamic Bayesian persuasion with a monotone agent, the principal's signaling problem can be solved online and efficiently by exploiting the supermodular structure of the principal's Q-function, and it reports a 30%

desk verdict A promising structural synthesis for dynamic Bayesian persuasion, but the core theorems rely on a self-citation and a circular proof, so the paper needs major revision before the claims can be trusted. read the letter →

arxiv 2607.13576 v1 pith:GLWEMT36 submitted 2026-07-15 cs.LG eess.SP

Merlin Paul , Anup Aprem This is my paper · ORCID
classification cs.LGeess.SP MSC 91A2668T0590C40
keywords Bayesianpersuasionsupermodularitymonotonepoliciesreinforcementlearningstructuredsignalingstrategyintelligentdrivinglaneselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that Bayesian persuasion in sequential decision-making—where a lead vehicle guides a connected vehicle by selectively revealing traffic information—can be solved online, even when both vehicles must learn their environments. The key claim is that if the agent's policy is monotone and the rewards satisfy supermodularity, then the principal's value function is also supermodular, so a softmax-derived signaling strategy is persuasive and the NP-hard design problem becomes computationally efficient. The proposed algorithms (MAPL and SQP) exploit these structures, and a lane-selection case study reports 30% cost efficiency over existing signaling strategies. If true, this would make real-time persuasive routing feasible for far-sighted drivers without needing model knowledge.

What carries the argument

The central objects are the supermodular Q-function of the principal (Theorem 1) and the monotone best response of the agent (Theorem 3). Under assumptions (A1)-(A6) on rewards, transitions, prior beliefs, and the agent's policy, the principal's Q-function has increasing differences; under (B1), (B2), (A3), and (A4) the agent's Q-function is supermodular and its optimal policy is monotone. This monotonicity makes the softmax policy MLR-ordered (Lemma 1), so higher signals induce higher posterior beliefs and monotone actions, ensuring the agent rationally follows the recommendation. SQP imposes the supermodularity constraint as a linear inequality in Q-learning, and MAPL restricts the agent's

What would settle it

In the paper's two-lane numerical setup, evaluate the learned agent policy and check whether Q_A(s,g,a) has increasing differences in (s,a) and (g,a) for every state, signal, and action; a single violation of this monotonicity would falsify Theorem 3 and the incentive-compatibility claim.

Watch

Extended reading notes

Core claim

The paper establishes that in dynamic Bayesian persuasion with a monotone agent, the principal's Q-function is supermodular in the state/external state and signal under sufficient conditions (Theorem 1), and that an MLR-ordered signaling strategy is incentive-compatible for a monotone agent whose best response is itself monotone (Theorems 2 and 3). This turns the NP-hard nested planning problem into a structured reinforcement learning problem: MAPL learns the agent's threshold policy efficiently, and SQP learns the principal's signaling strategy through constrained supermodular Q-learning. The numerical lane-selection case study reports a 30% cost saving for both lead and connected vehicle r

Load-bearing premise

The persuasion guarantee collapses if the agent's learned policy is not actually tail-sum supermodular (assumption A6), a structure the paper assumes rather than verifies, and Theorem 3's proof is deferred to an earlier preprint rather than given here.

Editorial extensions

If this is right

  • The lead vehicle can guide a far-sighted connected vehicle to choose system-beneficial lanes without knowing reward or transition dynamics in advance.
  • The computational cost of the agent's policy learning no longer grows with the state-space dimension, enabling online use in large traffic networks.
  • The same structural recipe applies to other domains with monotone threshold policies: credit rating, queueing, sensor scheduling, and transmission control.
  • Since the signaling strategy is persuasive, the connected vehicle voluntarily follows recommendations, avoiding the congestion that full information revelation can cause.
  • The reported 30% cost efficiency for both vehicles suggests concrete travel-time and fuel savings if the structural conditions hold.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper proves persuasiveness, not optimality; a natural open question is whether supermodular structure also bounds the gap between the learned signaling strategy and the optimal one.
  • Assumption (A6) is essentially what the algorithm tries to learn, so a practitioner would need to validate monotonicity of the agent's behavior from data before trusting SQP's guarantees.
  • If the structural results extend to non-homogeneous agent populations, the same framework could coordinate fleets of mixed risk-averse and risk-neutral drivers, an extension the conclusion flags.
  • The 30% figure comes from a single simulated two-lane scenario; its magnitude likely depends on how strongly monotonicity holds in the real system.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper studies dynamic Bayesian persuasion between a lead vehicle (principal) and a connected vehicle (agent), where the agent is farsighted and both players learn online. The authors propose: (i) MAPL, a structured policy-gradient algorithm that exploits an assumed monotone threshold structure of the agent's policy; (ii) sufficient conditions for the principal's Q-function to be supermodular (Theorem 1); (iii) structural conditions under which an MLR-ordered softmax signaling strategy is persuasive/incentive-compatible (Theorems 2 and 3); and (iv) SQP, a constrained Q-learning algorithm for the principal, combined in Algorithm 3 for repeated learning. A numerical case study on lane selection claims a 30% cost efficiency improvement over existing signaling-strategy design methods.

Significance. The topic is timely and the model formulation—combining average-reward MDPs with Bayesian persuasion—is a natural and potentially useful framework for intelligent traffic routing. If the structural claims were fully established, the paper would offer a meaningful new avenue for computationally tractable signaling in dynamic, model-free settings. The paper has some self-contained components: Lemma 1's proof of softmax MLR preservation is correct as stated, and Lemma 2's policy-gradient derivation is standard. However, the central theoretical guarantee rests on Theorem 3, whose proof is deferred to the authors' own TechRxiv preprint, and the proof of Theorem 1 contains a circular step. The numerical evidence is too thin to validate the empirical claims. Thus the significance is conditional on substantial revision.

major comments (5)
  1. [Appendix C, Step 3 and Theorem 3] The incentive-compatibility guarantee depends on Theorem 3, specifically the claim that Q_A(s,g,a) is supermodular in (s,a) and (g,a) under (B1),(B2),(A3),(A4). The Appendix does not prove this; it says only 'The proof is direct from [7]', where [7] is the authors' own TechRxiv preprint. This is a load-bearing, nontrivial statement—supermodularity must propagate through the Bellman operator, not merely the immediate reward. Without an in-manuscript proof or a precise statement of the theorem from [7] with all model assumptions, the persuasion guarantee is unsupported.
  2. [Appendix D, Step 1] The proof of Theorem 1 assumes 'π_k(g|s,θ) is MLR with respect to s and θ' in order to prove V_P is increasing. But π_k is the softmax over Q_P, and by Lemma 1 this MLR property follows only if Q_P is already supermodular—the very property the induction is meant to establish. No simultaneous induction or base case is given. As written, the proof is circular. The argument can likely be repaired (e.g., start with an increasing V_P^0 and prove supermodularity and MLR jointly), but the present text does not do this.
  3. [Section IV-B, Assumption (A6)] Assumption (A6) postulates that the learning agent's stochastic policy ~ρ(a|s,g) is stochastically increasing and tail-sum supermodular in (s,g). This is essentially the monotone structure that MAPL is supposed to learn and use. The manuscript does not prove that the sigmoid parameterization (16) or the MAPL update preserves (A6), and the numerical experiments do not verify it. Since (A6) is used directly in the proof of Theorem 1, the theory is conditional on the very structure that the algorithms claim to exploit. Please state whether (A6) is a maintained assumption, an enforced constraint, or a property to be verified; if the latter, provide a proof or experimental validation.
  4. [Section V-E and Algorithm 3] Algorithm 3 is presented as the online framework for repeated principal-agent interaction, and the paper claims it yields a persuasive signaling strategy. However, no convergence or stability theorem is given. The sentence defining dynamic stability ('∇π R_P(π,ρ)=0 as π_k→π*') is only a definition, not a proof. Given that the paper's central algorithmic claim is that SQP synthesizes a computationally efficient, persuasive strategy, this absence is significant. Please add a convergence result, or state explicitly that Algorithm 3 is a heuristic with only empirical support.
  5. [Section VI, Tables I and II] The numerical results are reported as single numbers with no error bars, no number of random seeds, no standard deviations, and no code or reproducibility statement. For example, Table II reports connected-vehicle average rewards 230, 218, 251, 242, 247, 257 for different signaling strategies, but without variance or multiple runs the '30% cost efficient' claim in the abstract is not statistically supported. The figures show trajectories but no confidence intervals. The authors should provide multi-seed results and a clear statement of the experimental setup.
minor comments (5)
  1. [Equation (9)] There is an inconsistency in the notation for the agent's action-value function: the text defines Q_A(s,θ,g_a) while Eq. (9) uses Q_A(s,g_a,a). Please use one consistent convention.
  2. [Equation (10)] The principal's optimization is written as 'min_{V_P} ∑ β(s,θ) V_P(s,θ)' with constraints, but the average reward constant R_P is not clearly introduced in this linear program. As written, the constraints do not appear to be the standard average-reward Bellman inequalities. Please clarify the derivation or cite the exact transformation.
  3. [Section VII, Assumption (A3)] The wording 'prior belief of the external states of higher states MLR dominates the lower' is ambiguous. It should read: for s_i ≤ s_j, μ_{s_i} ≤_r μ_{s_j} in the MLR order on Θ.
  4. [Appendix C and Theorem 2] The proof of Theorem 2 is not clearly separated within Appendix C; Steps 1–5 are presented as a general discussion. Reorganize so that the theorem statements, lemmas, and proofs are explicit.
  5. [Table I] The first column of Table I is labeled 'Number' and contains values 3,4,5,6 while the first two rows have the same value 3. This appears to be a formatting error; the column should be removed or made meaningful.

Circularity Check

2 steps flagged · score 7.0 of 10

Persuasion guarantee rests on Theorem 3, whose key proof step is deferred to the authors' own preprint; Theorem 1's proof also assumes the MLR signaling property it is meant to establish.

  1. self citation load bearing [Appendix C, Step 3 (proof of Theorem 3)]
    "Under (B1) and (B2), (A3) and (A4), we have Q_A(s, g, a) is supermodular in both (s, a) and (g, a). The proof is direct from [7] with the corresponding change while considering supermodularity in reward structures."

    Theorem 3 is the central persuasion guarantee: it is used to conclude that the agent's Q-function is supermodular, that a monotone best response exists, and hence that the MLR signaling strategy is incentive compatible. Yet the key proof step is not present in this manuscript; it is deferred to [7], the authors' own TechRxiv preprint. No machine-checked proof, code reproduction, or independent verification is supplied for that imported result. Steps 4 and 5 of the same appendix then rely directly on this unproved supermodularity to assert monotonicity and incentive compatibility, making the self-citation load-bearing for the paper's main claim.

  2. other [Appendix D, Step 1 (proof of Theorem 1)]
    "V_P^{k+1}(s, θ) = Σ_{g∈G} Q_P^k(s, θ, g)·π_k(g|s, θ). Assuming π_k(g|s, θ) is MLR with respect to s and θ, V_P^{k+1}(s, θ) is also increasing in s and θ."

    In the induction proving monotonicity of the principal's value function, the proof assumes that the signaling strategy π_k is MLR in s and θ. But π_k is defined in the paper as the softmax over Q_P^k (Eq. 12 and Algorithm 2), and Lemma 1 shows this softmax is MLR only when Q_P^k is supermodular — which is precisely the property the induction in Step 2 is meant to establish from V_P^k increasing. Since V_P0 is allowed to be arbitrary and no base case or explicit simultaneous induction is written, the proof assumes the MLR property whose only justification in the paper goes through the supermodularity being proved. The two structural pillars are therefore mutually dependent, and neither is independently demonstrated here.

full rationale

The paper's central claim — that SQP synthesizes a persuasive signaling strategy for a far-sighted agent — depends on the chain: Theorem 1 (supermodular principal Q), Lemma 1 (softmax over supermodular Q gives MLR signaling), and Theorem 3 (monotone, incentive-compatible agent best response). The weakest link is Theorem 3: Appendix C, Step 3, states only 'The proof is direct from [7]', where [7] is the authors' own TechRxiv preprint. This is not an independently established, machine-checked, or externally verified result; it is a load-bearing self-citation. The numerical experiments do not repair this gap because they instantiate the assumptions (e.g., A6) rather than independently verifying the agent's monotone best response or the persuasiveness guarantee. In addition, the proof of Theorem 1 in Appendix D assumes π_k is MLR in order to prove monotonicity of the value function, while MLR of the softmax signaling strategy follows from Lemma 1 only if Q_P^k is already supermodular — the property the induction is trying to establish. Thus the two structural pillars of the framework are mutually dependent, and the persuasion guarantee is not fully established in this manuscript. The algorithmic contributions and empirical comparisons against baselines are independent content, so the score is not higher, but the load-bearing self-citation plus the circular induction step make the central structural claim substantially unsupported as presented.

Assumptions & free parameters 3 free parameters · 11 assumptions · 0 invented entities

The central claim rests on a chain of structural assumptions (monotonicity, supermodularity, MLR) that are either assumed (A1-A6, B1-B2) or imported from the authors' own prior preprint. The only hand-tuned numbers are the softmax temperature and the sigmoid parameters; the case-study reward/transition models are specifically chosen to satisfy the assumptions. No new physical entities are introduced.

free parameters (3)
  • softmax temperature τ = not specified
    Controls exploration vs exploitation in the principal's signaling policy (eq. 12). Chosen by hand; directly shapes the MLR ordering and the persuasiveness guarantee.
  • sigmoid policy parameters σ = {φ1,φ2,φ3,φ4} = learned via MAPL
    Parameterize the agent's threshold policy (eq. 16). The sigmoidal form is an ad hoc approximation of the true threshold; φ1,φ2 are floor/ceiling, φ3,φ4 location/shape. The central numerical results depend on this parameterization.
  • learning rates α', η and forgetting factor β = not specified
    Step sizes for SQP and MAPL updates. No values reported; affect convergence and final performance.
assumptions (11)
  • standard math Bayes' rule for posterior updating (eq. 1)
    Used to compute agent's belief over θ from signals.
  • standard math Average reward MDP theory (Sennott conditions C1-C4, Lemma 3/4)
    Justifies existence of optimal average-reward policies and Bellman optimality for the agent.
  • standard math Policy gradient theorem for Markov reward processes (Lemma 2)
    Derives the gradient of the agent's average reward w.r.t. policy parameters.
  • standard math Q-learning convergence for average-reward MDPs and primal-dual stochastic approximation
    Basis for SQP update and convergence claims (citing [24],[31],[34]).
  • standard math Supermodularity/MLR properties (Theorems 4,5, Lemma 5-7)
    Used in the proofs of Theorem 1 and 3.
  • domain assumption Monotone agent assumption (Section IV-A)
    The agent's optimal policy is assumed monotone in state and signal; this is what enables MAPL and the structural results.
  • domain assumption Assumptions A1-A6 (Section IV-B)
    Sufficient conditions for the principal's Q supermodularity; A6 in particular assumes the learning agent's policy is tail-sum supermodular, which is the structure the algorithm tries to learn.
  • domain assumption Assumptions B1,B2,A3,A4 (Theorem 3)
    Sufficient conditions for the agent's Q supermodularity and monotone best response, imported from authors' prior work [7].
  • domain assumption Agent's prior belief uniform (Section VI-A)
    Satisfies (A3) and is used in the numerical model.
  • domain assumption Ergodicity of agent's policy (C1)
    Ensures average reward independent of initial state.
  • ad hoc to paper Sigmoid threshold policy parameterization (16)
    Assumed functional form of the agent's policy to enable policy-gradient updates; not derived from the MDP.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Structured Reinforcement Learning for Bayesian Persuasion : Application to Intelligent Interactive Driving." pith.science (2026). https://pith.science/paper/GLWEMT36

@misc{pith2026260713576,
  author       = {Pith},
  title        = {Pith review of: Structured Reinforcement Learning for Bayesian Persuasion : Application to Intelligent Interactive Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GLWEMT36}},
  note         = {Machine review of arXiv:2607.13576}
}
read the original abstract

Interactive driving, wherein an intelligent lead vehicle equipped with real-time traffic data coordinates route choices of connected vehicles, offers a promising approach to dynamic traffic management. To address the challenge of harmonising decisions, this paper considers the strategic information revealing framework of Bayesian persuasion. Here, the principal (lead vehicle) aims to guide the agent's (connected vehicle) partially observable sequential decision making towards its own objectives by selectively revealing information, such as real-time traffic ahead, using signals. However, the agent's farsighted response to maximize its long-term reward, renders the principal's signaling strategy design computationally challenging. We propose an online structured reinforcement learning framework to synthesize computationally efficient signaling strategy which is persuasive for a far-sighted agent. The main contributions of the paper are as follows: (i) For a monotonic agent with approximate best response, we propose MAPL, a structured policy learning algorithm for faster online learning, (ii) Identification of sufficient conditions for the supermodular structure of the Q function of the principal for a monotonic agent, (iii) Identification of sufficient conditions to ensure the persuasiveness of the principal's signaling strategy, (iv) Supermodular Q learning for Principal (SQP), which leverages the supermodular structure of principal's action value to synthesize computationally efficient signaling strategy that is persuasive for a monotonic learning agent, (v) Numerical analysis considering a real-time application of Bayesian persuasive driving for lane selection demonstrates that the proposed method is 30% cost efficient for optimising travelling rewards of both the lead and connected vehicle compared to the existing methodologies for signaling strategy design.

Figures

Figures reproduced from arXiv: 2607.13576 by the authors.

Figure 1
Figure 1. The schematic representation of intelligent interactive driving based [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Dynamic framework of Bayesian persuasion in which the principal [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The figure illustrates an online learning framework in which the [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The figure gives an illustration of Bayesian persuasive driving. Here, [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: State transition diagram for the two-lane system. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Average reward accumulated by the connected vehicle [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Average reward accumulated by the lead vehicle [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Comparison of computational time of existing and structured learning [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 1 linked inside Pith

  1. [7]

    Monotonic agent policies in dynamic bayesian persuasion: Application to intelligent traffic routing.TechRxiv, 2024

    Merlin Paul and Anup Aprem. Monotonic agent policies in dynamic bayesian persuasion: Application to intelligent traffic routing.TechRxiv, 2024

    2024

  2. [1]

    Co- operative platoon forming strategy for connected autonomous vehicles in mixed traffic flow.Physica A: Statistical Mechanics and its Applications, 623:128828, 2023

    Yan-Tao Zhang, Mao-Bin Hu, Yu-Zhang Chen, and Cong-Ling Shi. Co- operative platoon forming strategy for connected autonomous vehicles in mixed traffic flow.Physica A: Statistical Mechanics and its Applications, 623:128828, 2023

    2023

  3. [2]

    Bayesian persuasion in sequential decision-making

    Jiarui Gan, Rupak Majumdar, Goran Radanovic, and Adish Singla. Bayesian persuasion in sequential decision-making. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, 2022

    2022

  4. [3]

    Bayesian exploration: Incentivizing exploration in Bayesian games.Operations Research, 70(2):1105–1127, 2022

    Yishay Mansour, Aleksandrs Slivkins, Vasilis Syrgkanis, and Zhi- wei Steven Wu. Bayesian exploration: Incentivizing exploration in Bayesian games.Operations Research, 70(2):1105–1127, 2022

    2022

  5. [4]

    Grading standards and education quality.American Economic Journal: Microeconomics, 7(2):248–279, 2015

    Raphael Boleslavsky and Christopher Cotton. Grading standards and education quality.American Economic Journal: Microeconomics, 7(2):248–279, 2015

    2015

  6. [5]

    Optimal signaling mecha- nisms in unobservable queues.Operations research, 67(5):1397–1416, 2019

    David Lingenbrink and Krishnamurthy Iyer. Optimal signaling mecha- nisms in unobservable queues.Operations research, 67(5):1397–1416, 2019. 16

    2019

  7. [6]

    When the line says more: Optimal signaling in queues with congestion-driven expectations.International Applied Mathematics, Modelling and Simulation Conference, 2025

    Aibai Ma and Li Xiao. When the line says more: Optimal signaling in queues with congestion-driven expectations.International Applied Mathematics, Modelling and Simulation Conference, 2025

    2025

  8. [8]

    Stress tests and information disclosure

    Itay Goldstein and Yaron Leitner. Stress tests and information disclosure. Journal of Economic Theory, 177:34–69, 2018

    2018

Show all 52 references
  1. [9]

    Sequential information design: Markov persuasion process and its efficient reinforcement learning

    Jibang Wu, Zixuan Zhang, Zhe Feng, Zhaoran Wang, Zhuoran Yang, Michael I Jordan, and Haifeng Xu. Sequential information design: Markov persuasion process and its efficient reinforcement learning. arXiv:2202.10678, 2022

    2022 arXiv

  2. [10]

    Sequential information design: Learning to persuade in the dark.Advances in Neural Information Processing Systems, 35:15917–15928, 2022

    Martino Bernasconi, Matteo Castiglioni, Alberto Marchesi, Nicola Gatti, and Francesco Trov `o. Sequential information design: Learning to persuade in the dark.Advances in Neural Information Processing Systems, 35:15917–15928, 2022

    2022

  3. [11]

    Bayesian persuasive driving

    Cheng Peng and Masayoshi Tomizuka. Bayesian persuasive driving. In 2019 American Control Conference (ACC). IEEE, 2019

    2019

  4. [12]

    Examining lane change gap acceptance, duration and impact using naturalistic driving data.Transportation research part C: emerging technologies, 104:317–331, 2019

    Minming Yang, Xuesong Wang, and Mohammed Quddus. Examining lane change gap acceptance, duration and impact using naturalistic driving data.Transportation research part C: emerging technologies, 104:317–331, 2019

    2019

  5. [13]

    A survey of autonomous driving from a deep learning perspective.ACM Computing Surveys, 57(10):1–60, 2025

    Jingyuan Zhao, Yuyan Wu, Rui Deng, Susu Xu, Jinpeng Gao, and Andrew Burke. A survey of autonomous driving from a deep learning perspective.ACM Computing Surveys, 57(10):1–60, 2025

    2025

  6. [14]

    Driver lane-changing behavior prediction based on deep learning.Journal of advanced transportation, (1):6676092, 2021

    Cheng Wei, Fei Hui, and Asad J Khattak. Driver lane-changing behavior prediction based on deep learning.Journal of advanced transportation, (1):6676092, 2021

    2021

  7. [15]

    Automated lane change decision making using deep reinforcement learning in dynamic and uncertain highway environment

    Ali Alizadeh, Majid Moghadam, Yunus Bicer, Nazim Kemal Ure, Ugur Yavas, and Can Kurtulus. Automated lane change decision making using deep reinforcement learning in dynamic and uncertain highway environment. InProc. Intelligent Transportation Systems Conference (ITSC), 1399–14...

    2019

  8. [16]

    Efficient real-time routing for autonomous vehicles through Bayes correlated equilibrium: An information design framework.Information Economics and Policy, 47:14–26, 2019

    Yixuan Liu and Andrew B Whinston. Efficient real-time routing for autonomous vehicles through Bayes correlated equilibrium: An information design framework.Information Economics and Policy, 47:14–26, 2019

    2019

  9. [17]

    Beeps.American Economic Review, 107(1):31–53, 2017

    Jeffrey C Ely. Beeps.American Economic Review, 107(1):31–53, 2017

    2017

  10. [18]

    Persuading farsighted receivers in MDPs: the power of honesty

    Martino Bernasconi, Matteo Castiglioni, Alberto Marchesi, and Mirco Mutti. Persuading farsighted receivers in MDPs: the power of honesty. Advances in Neural Information Processing Systems, 36:14987–15014, 2023

    2023

  11. [19]

    Online learning in sequential bayesian persuasion: Handling unknown priors.Artificial Intelligence, 338:104245, 2025

    Martino Bernasconi, Matteo Castiglioni, Alberto Marchesi, Nicola Gatti, and Francesco Trov`o. Online learning in sequential bayesian persuasion: Handling unknown priors.Artificial Intelligence, 338:104245, 2025

    2025

  12. [20]

    Multiple stopping time POMDPs: Structural results & application in interactive advertising on social media.Automatica, 95:385–398, 2018

    Vikram Krishnamurthy, Anup Aprem, and Sujay Bhatt. Multiple stopping time POMDPs: Structural results & application in interactive advertising on social media.Automatica, 95:385–398, 2018

    2018

  13. [21]

    Optimal scheduling of multiple sensors over lossy and bandwidth limited channels.IEEE Transactions on Control of Network Systems, 7(3):1188–1200, 2020

    Shuang Wu, Kemi Ding, Peng Cheng, and Ling Shi. Optimal scheduling of multiple sensors over lossy and bandwidth limited channels.IEEE Transactions on Control of Network Systems, 7(3):1188–1200, 2020

    2020

  14. [22]

    Transmit power control policies for energy harvesting sensors with retransmissions.IEEE Journal of Selected Topics in Signal Processing, 7(5):895–906, 2013

    Anup Aprem, Chandra R Murthy, and Neelesh B Mehta. Transmit power control policies for energy harvesting sensors with retransmissions.IEEE Journal of Selected Topics in Signal Processing, 7(5):895–906, 2013

    2013

  15. [23]

    Optimality of threshold policies for transmission scheduling in correlated fading channels.IEEE Transactions on Communications, 57(8):2474–2483, 2009

    Minh Hanh Ngo and Vikram Krishnamurthy. Optimality of threshold policies for transmission scheduling in correlated fading channels.IEEE Transactions on Communications, 57(8):2474–2483, 2009

    2009

  16. [24]

    Dejan V Djonin and Vikram Krishnamurthy. Q-learning algorithms for constrained Markov decision processes with randomized monotone policies: Application to MIMO transmission control.IEEE Transactions on Signal Processing, 55(5):2170–2181, 2007

    2007

  17. [25]

    Algorithmic Bayesian persuasion

    Shaddin Dughmi and Haifeng Xu. Algorithmic Bayesian persuasion. In Proceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing, 412–425, 2016

    2016

  18. [26]

    Farsightedness in games: Stabilizing cooperation in international conflict.Available at SSRN 3313025, 2019

    Steven J Brams and Mehmet Ismail. Farsightedness in games: Stabilizing cooperation in international conflict.Available at SSRN 3313025, 2019

    2019

  19. [27]

    Applying Markov decision process to understand driving decisions using basic safety messages data.Trans- portation Research Part C: Emerging Technologies, 115:102642, 2020

    Mohsen Kamrani, Aravinda Ramakrishnan Srinivasan, Subhadeep Chakraborty, and Asad J Khattak. Applying Markov decision process to understand driving decisions using basic safety messages data.Trans- portation Research Part C: Emerging Technologies, 115:102642, 2020

    2020

  20. [28]

    Markov Decision Processes.Handbooks in operations research and management science, 2:331–434, 1990

    Martin L Puterman. Markov Decision Processes.Handbooks in operations research and management science, 2:331–434, 1990

    1990

  21. [29]

    Information design in multi-agent reinforcement learning.Advances in Neural Information Processing Systems, 36:25584–25597, 2023

    Yue Lin, Wenhao Li, Hongyuan Zha, and Baoxiang Wang. Information design in multi-agent reinforcement learning.Advances in Neural Information Processing Systems, 36:25584–25597, 2023

    2023

  22. [30]

    Persuading a behavioral agent: Approximately best responding and learning.arXiv preprint, 2023

    Yiling Chen and Tao Lin. Persuading a behavioral agent: Approximately best responding and learning.arXiv preprint, 2023

    2023

  23. [31]

    Cambridge university press, 2016

    Vikram Krishnamurthy.Partially observed Markov decision processes. Cambridge university press, 2016

    2016

  24. [32]

    Simulation-based optimization of Markov reward processes.IEEE Transactions on Automatic Control, 46(2):191–209, 2001

    Peter Marbach and John N Tsitsiklis. Simulation-based optimization of Markov reward processes.IEEE Transactions on Automatic Control, 46(2):191–209, 2001

    2001

  25. [33]

    Simulation- based approximate policy iteration with generalized logistic functions

    Antoine Saur ´e, Jonathan Patrick, and Martin L Puterman. Simulation- based approximate policy iteration with generalized logistic functions. INFORMS Journal on Computing, 27(3):579–595, 2015

    2015

  26. [34]

    Relative Q-learning for average-reward Markov decision processes with continuous states.IEEE Transactions on Automatic Control, 2024

    Xiangyu Yang, Jiaqiao Hu, and Jian-Qiang Hu. Relative Q-learning for average-reward Markov decision processes with continuous states.IEEE Transactions on Automatic Control, 2024

    2024

  27. [35]

    Reducing congestion through information design

    Sanmay Das, Emir Kamenica, and Renee Mirka. Reducing congestion through information design. InProc. 55th Annual Allerton Conference on Communication, Control and Computing (Allerton), 1279–1284. IEEE, 2017

    2017

  28. [36]

    Model predictive control-based multivariable controller for traffic flows in automated freeway systems.IEEE Intelligent Transportation Systems Magazine, 15(2):149–161, 2022

    Hongguang Pan, Xinyu Yu, Lei Gao, Yongfu Li, and Wei Hua. Model predictive control-based multivariable controller for traffic flows in automated freeway systems.IEEE Intelligent Transportation Systems Magazine, 15(2):149–161, 2022

    2022

  29. [37]

    Congestion avoidance routing based on large-scale social signals.IEEE Transactions on Intelligent Transportation Systems, 17(9):2613–2626, 2015

    Kun He, Zhongzhi Xu, Pu Wang, Lianbo Deng, and Lai Tu. Congestion avoidance routing based on large-scale social signals.IEEE Transactions on Intelligent Transportation Systems, 17(9):2613–2626, 2015

    2015

  30. [38]

    On the estimation of arterial route travel time distribution with Markov chains.Transportation Research Part B: Methodological, 46(10):1576–1590, 2012

    Mohsen Ramezani and Nikolas Geroliminis. On the estimation of arterial route travel time distribution with Markov chains.Transportation Research Part B: Methodological, 46(10):1576–1590, 2012

    2012

  31. [39]

    A novel method for lane- change maneuver in urban driving using predictive Markov decision process.arXiv preprint, 2022

    Avinash Prabu, Niranjan Ravi, and Lingxi Li. A novel method for lane- change maneuver in urban driving using predictive Markov decision process.arXiv preprint, 2022

    2022

  32. [40]

    Learning-based model predictive control: Toward safe learning in control.Annual Review of Control, Robotics, and Autonomous Systems, 3(1):269–296, 2020

    Lukas Hewing, Kim P Wabersich, Marcel Menner, and Melanie N Zeilinger. Learning-based model predictive control: Toward safe learning in control.Annual Review of Control, Robotics, and Autonomous Systems, 3(1):269–296, 2020

    2020

  33. [41]

    Lane-change intention estimation for car-following control in autonomous driving.IEEE Transactions on Intelligent Vehicles, 3(3):276–286, 2018

    Yihuan Zhang, Qin Lin, Jun Wang, Sicco Verwer, and John M Dolan. Lane-change intention estimation for car-following control in autonomous driving.IEEE Transactions on Intelligent Vehicles, 3(3):276–286, 2018

    2018

  34. [42]

    A framework for efficient dynamic routing under stochastically varying conditions.Transportation research part B: methodological, 160:97–124, 2022

    Nikki Levering, Marko Boon, Michel Mandjes, and Rudesindo N ´u˜nez- Queija. A framework for efficient dynamic routing under stochastically varying conditions.Transportation research part B: methodological, 160:97–124, 2022

    2022

  35. [43]

    Ulmer, Dirk C

    Marlin W. Ulmer, Dirk C. Mattfeld, and Felix K ¨oster. Budgeting time for dynamic vehicle routing with stochastic customer requests. Transportation Science, 52(1):20–37, 2018

    2018

  36. [44]

    Optimal signaling mecha- nisms in unobservable queues.Available at SSRN 2964093, 2018

    David Lingenbrink and Krishnamurthy Iyer. Optimal signaling mecha- nisms in unobservable queues.Available at SSRN 2964093, 2018

    2018

  37. [45]

    Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs.Operations Research, 37(4):626–633, 1989

    Linn I Sennott. Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs.Operations Research, 37(4):626–633, 1989

    1989

  38. [46]

    Monotonicity of con- strained optimal transmission policies in correlated fading channels with ARQ.IEEE Transactions on Signal Processing, 58(1):438–451, 2009

    Minh Hanh Ngo and Vikram Krishnamurthy. Monotonicity of con- strained optimal transmission policies in correlated fading channels with ARQ.IEEE Transactions on Signal Processing, 58(1):438–451, 2009

    2009

  39. [47]

    Approximation algorithms for inventory problems with submodular or routing costs.Mathematical Programming, 160:225–244, 2016

    Viswanath Nagarajan and Cong Shi. Approximation algorithms for inventory problems with submodular or routing costs.Mathematical Programming, 160:225–244, 2016

    2016

  40. [48]

    Optimal paths in graphs with stochastic or multidimensional weights.Communications of the ACM, 26(9):670– 676, 1983

    Ronald Prescott Loui. Optimal paths in graphs with stochastic or multidimensional weights.Communications of the ACM, 26(9):670– 676, 1983

    1983

  41. [49]

    Decision-oriented learning with differentiable submodular maximization for vehicle routing problem

    Guangyao Shi and Pratap Tokekar. Decision-oriented learning with differentiable submodular maximization for vehicle routing problem. In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023

    2023

  42. [50]

    Adaptive submodular ranking and routing.Operations Research, 68(3):856–877, 2020

    Fatemeh Navidi, Prabhanjan Kambadur, and Viswanath Nagarajan. Adaptive submodular ranking and routing.Operations Research, 68(3):856–877, 2020

    2020

  43. [51]

    Efficient stochastic routing in path- centric uncertain road networks.Proceedings of the VLDB Endowment, 17(11):2893–2905, 2024

    Chenjuan Guo, Ronghui Xu, Bin Yang, Ye Yuan, Tung Kieu, Yan Zhao, and Christian S Jensen. Efficient stochastic routing in path- centric uncertain road networks.Proceedings of the VLDB Endowment, 17(11):2893–2905, 2024

    2024

  44. [52]

    Vehicle velocity prediction and energy management strategy part 1: Deterministic and stochastic vehicle velocity prediction using machine learning

    Kuan Liu, Zachary Asher, Xun Gong, Mike Huang, and Ilya Kol- manovsky. Vehicle velocity prediction and energy management strategy part 1: Deterministic and stochastic vehicle velocity prediction using machine learning. Technical report, SAE Technical Paper, 2019

    2019

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.