Pith. sign in

REVIEW 1 major objections 6 minor 300 references

Finite-Time Analysis of Discounted Exponential-Utility Reinforcement Learning

T0 review · 1 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Two model-free algorithms for risk-sensitive discounted exponential-utility RL are shown to converge at the optimal Õ(1/√n) rate under Markovian sampling, with stepsizes that do not depend on unknown MDP parameters.

desk verdict Solid finite-time analysis for two discounted exponential-utility RL algorithms; the main rates are real, and the unique-greedy restriction on the averaging theorem is explicit, not a hidden flaw. read the letter →

arxiv 2608.01917 v1 pith:RIEK45G4 submitted 2026-08-03 cs.LG

Ankur Naskar , Vivek T A , Aditya Kumar , Gugan Thoppe , Prashanth L. A This is my paper
classification cs.LG MSC 68Q3290C4062L20
keywords risk-sensitivereinforcementlearningexponentialutilityfinite-timeanalysisPolyak–RuppertaveragingMarkoviansamplingstochasticapproximationThompsonmetricQ-learning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to prove that two recently proposed model-free algorithms for discounted exponential-utility reinforcement learning—a risk-sensitive criterion—converge at the optimal rate up to logarithms, and that this can be achieved without knowing the MDP's transition dynamics. It establishes that, under asynchronous sampling from a geometrically ergodic behavior policy, the one-timescale algorithm's Polyak–Ruppert averaged iterates satisfy an Õ(1/√n) bound on the relative error, and the two-timescale algorithm's Q iterates satisfy an Õ(1/√n) bound, both with parameter-free stepsizes. The one-timescale proof is the hard part: the operator is only contractive in a logarithmic metric, so the paper builds a local pseudo-contraction for the relative-error dynamics and smooths it with a Moreau-envelope Lyapunov function. If the bounds are right, risk-sensitive policies learned from samples come with the same statistical guarantees as ordinary Q-learning, closing the asymptotic-only gap in the previous work.

What carries the argument

The Bellman-compatible surrogate operator F(x)(s,a) = exp(−θr(s,a)/γ) Σ_{s'} P(s'|s,a) [min_{a'} x(s',a')]^γ, which is a γ-contraction in the Thompson metric d(x,y)=||ln x − ln y||_∞. The proof transfers this contraction to the relative error x⊘x*−1 via a one-point (local pseudo-) contraction, with factor γ_eff in the synchronous case and ν=1−δ(1−γ_eff) for the Markovian mean drift. A Moreau-envelope Lyapunov function V_λ(r)=min_y {½||y||_∞² + (1/2λ)||r−y||₂²} smooths the sup norm and converts the local contraction into a one-step recursion, while for Polyak–Ruppert averaging the nonlinearity is split into a linear part with contraction matrix B* (||B*||_∞ ≤ 1−δ(1−γ)) and quadratic remainder

What would settle it

Construct a two-state MDP with symmetric rewards and transitions engineered so that x* has an exact tie (two greedy actions) at some state, run the one-timescale algorithm with α_n = (n+1)^{-0.8} and Polyak–Ruppert averaging, and record E||x̄_N ⊘ x* − 1||_∞ as N grows: if the decay is slower than ~1/√N (or stalls), the uniqueness assumption is doing the work; a direct check that c* from Eq. (79) is zero predicts the bound's failure.

Watch

Extended reading notes

Core claim

The central discovery is that the additive update of the one-timescale algorithm, although mismatched with the power-law operator's logarithmic contraction geometry, can be analyzed through the relative error r_n = x_n ⊘ x* − 1. The operator F and its Markovian mean drift F̄ satisfy a one-point contraction: ||F̄(x)⊘x* − 1||_∞ ≤ ν ||x⊘x* − 1||_∞ for x in the invariant box K, with ν = 1 − δ(1−γ_eff). Combined with the Moreau envelope V_λ, this yields a one-step drift inequality, and Polyak–Ruppert averaging turns the resulting bound into E||x̄_N ⊘ x* − 1||_∞ ≤ C ln N / √N. For the two-timescale method, the paper controls the fast recursion's tracking error of the moving target e^{−θT(Q_n)/γ} a

Load-bearing premise

The load-bearing premise is that the greedy policy with respect to the fixed point x* picks a unique action at every state; without that margin, the quadratic bound on the policy-switching perturbation collapses and the optimal averaging rate is not established.

Editorial extensions

If this is right

  • Both algorithms now come with Õ(1/√n) convergence under Markovian sampling, matching the optimal stochastic-approximation rate up to logarithmic factors.
  • The stepsize choices are universal: they are computable from γ, x₀, and the reward bound, not from unknown mixing or visitation parameters.
  • Risk-sensitive policies learned from sampled transitions are now guaranteed to stabilize in finite time, which supports practical deployment of exponential-utility RL.
  • The local pseudo-contraction plus Moreau-envelope technique may apply to other fixed-point iterations whose operators are contractive only in a nonlinear metric.
  • The two-timescale analysis provides a template for recursions with a fast tracker and a slow policy iterate, extending beyond the specific exponential-utility setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The uniqueness-of-greedy-policy assumption is the practical bottleneck: if ties occur at x*, the margin c* is zero, and the PR-averaging rate argument would need a tie-breaking or perturbation device to be restored; one testable extension is whether uniform random tie-breaking re-establishes the rate.
  • The parameter-free rates suggest a practical recipe: run the one-timescale algorithm with α_n=(n+1)^{-α} for α∈(1/2,1) and average iterates, which may be more robust than tuning stepsizes to unknown spectral or mixing constants.
  • Since the achieved rate matches risk-neutral Q-learning, adopting this risk-sensitive surrogate may not cost convergence speed, making it an attractive drop-in for applications that want conservative policies.
  • The extra ln^{3/2} factor in the two-timescale bound is likely an artifact of the tracking-error control; a sharper nonasymptotic tracking inequality might remove that logarithmic excess.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 6 minor

Summary. The paper analyzes two model-free stochastic approximation algorithms for discounted exponential-utility RL, introduced by Thoppe et al. (2026). The one-timescale method is studied in synchronous and asynchronous (Markovian) sampling; the two-timescale method is studied under asynchronous sampling. The main results are finite-time rates: O(1/sqrt(N)) for synchronous one-timescale; O(ln N / N^{alpha/2}) and O(ln N / sqrt(N)) for asynchronous one-timescale, with Polyak--Ruppert averaging giving O(ln N / sqrt(N)) under a unique-greedy-policy assumption; and O(ln^{3/2} N / sqrt(N)) for two-timescale. Stepsizes are claimed parameter-free. The proof uses monotonicity, degree-gamma homogeneity, a Moreau-envelope Lyapunov function, blocking at the mixing time, and a perturbed-linear-recursion analysis of averaging. A cliff-walk experiment illustrates risk-averse behavior.

Significance. If correct, these are the first finite-time guarantees for model-free discounted exponential-utility RL. The proofs are detailed and mostly self-contained, with explicit constants in Tables 1--2 and transparent use of prior operator properties. The parameter-free step-size choices and the explicit treatment of Markovian noise are concrete strengths. The main caveat is the uniqueness of the greedy policy required for the optimal averaging rate; this restriction is not advertised in the abstract or introduction.

major comments (1)
  1. [Theorem 4.7 / Appendix D, Eq. (79)--(80)] The O~(1/sqrt(N)) averaging rate for the one-timescale algorithm relies critically on the unique-greedy-policy assumption. The bound (28)/(80) uses c* > 0, the radius of the ball around x* inside the greedy-policy cone. If x* has a tie at any state, x* lies on a cone boundary, c* = 0, and the quadratic bound becomes vacuous; the nonlinear perturbation xi_n can be O(1) for iterates arbitrarily close to x*, so Lemma 5.5 and hence Theorem 4.7 are not established for tied MDPs. The assumption is stated in Theorem 4.7 but is absent from the abstract and Section 1, where the result is advertised as the optimal parameter-free rate. The authors should either extend the argument (e.g., via a tie-breaking or margin-free analysis) or prominently state and discuss this restriction in the abstract, introduction, and Remark 4.8.
minor comments (6)
  1. [Section 4.1, after Eq. (13)] The claim that gamma_eff is easily seen to lie in (0,1) deserves a one-line proof or a reference; this quantity is central to the subsequent contraction arguments.
  2. [Appendix E, Eq. (99)] The displayed bound for ||epsilon_n||_infty appears to drop a factor of theta arising from the Lipschitz constant of u -> exp(-theta u/gamma). The bound should be 2 theta C_max exp(theta C_max/gamma) beta_n. This affects constants only, not rates.
  3. [Appendix E, inequality before Eq. (118)] The displayed recursion should read (1 - beta_n) and (1 - (1-gamma) beta_n), not (1 - alpha_n) and (1 - (1-gamma) alpha_n). As written, the alpha_n in the display is inconsistent with the subsequent use of beta_n.
  4. [Appendix D, after Lemma D.1] Typo: 'delta = min + D.F.' should be 'delta = min_i D_mu(i,i)'.
  5. [Figure 1 caption] The caption refers to the update rules (10) and (11) before they are defined in Section 3. Consider moving the figure or adjusting the caption.
  6. [Throughout] Minor typos: 'positve' in Section 2; 'compontentwise' in Section 4.1.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: finite-time rates are derived from the operator's contraction geometry and explicit Lyapunov recursions, not from fitted parameters or self-referential definitions.

full rationale

The paper's central claims (Theorems 4.2, 4.6, 4.7, 4.9) are proved by constructing explicit recursions for the relative error, bounding the Markov noise, and unrolling contractive one-step Lyapunov inequalities. The one-point contraction (Prop 4.1) and its mean-drift analogue (Cor 4.5) are proved in the appendix from the monotonicity, homogeneity, and fixed-point property of F, not assumed. The Moreau-envelope and Polyak-Ruppert averaging are standard analysis tools applied to this recursion. Self-citations to Thoppe et al. (2026) and Naskar et al. (2026) supply auxiliary boundedness facts and a general analysis template, but the load-bearing rate arguments—the pseudo-contraction lemma (Lemma 5.1), the perturbed-linear decomposition (Lemma D.1), and the four term bounds (Lemmas 5.2-5.5)—are carried out in this paper. The unique-greedy-policy assumption in Theorem 4.7 is a genuine non-degeneracy condition (c* > 0), not a way of assuming the conclusion; it is explicitly stated and its necessity for the quadratic bound (28) is transparent. No fitted parameter is relabeled as a prediction, and the stepsizes are universal. The rates are therefore not circular.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The only assumptions are standard in the RL theory literature: geometric ergodicity, positivity and invariance of the iterate box, the known Thompson-metric contraction of the operator, and a unique-greedy-action non-degeneracy for the averaging result. No new physical or algorithmic entities are postulated.

assumptions (4)
  • domain assumption The sampling process (s_n, a_n) is geometrically ergodic with a unique stationary distribution eta having positive mass on every state-action pair (Assumption 1).
    Used in all asynchronous results to control the mixing time tau_n = O(ln n) and to bound Markovian noise.
  • domain assumption The Bellman operator F defined in (5) is a gamma-contraction in the Thompson metric (taken from Thoppe et al. 2026).
    Foundation for Proposition 4.1 and the one-point contraction of the relative error; cited from the companion paper.
  • domain assumption The greedy policy w.r.t. x* assigns a unique action to every state (Theorem 4.7).
    Guarantees a positive margin c* to tie boundaries, enabling the quadratic bound on the nonlinear perturbation in the PR-averaging proof.
  • domain assumption Iterates stay inside the box K=[C_l,C_u]^(SA) for the chosen initialization and reward bounds (Thoppe et al. 2026, Lemma C.1).
    Used to bound all noise terms and to apply the local contraction; follows from the definitions of C_l and C_u in (12).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Finite-Time Analysis of Discounted Exponential-Utility Reinforcement Learning." pith.science (2026). https://pith.science/paper/RIEK45G4

@misc{pith2026260801917,
  author       = {Pith},
  title        = {Pith review of: Finite-Time Analysis of Discounted Exponential-Utility Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RIEK45G4}},
  note         = {Machine review of arXiv:2608.01917}
}
abstract

Discounted exponential utility provides a principled criterion for risk-sensitive sequential decision-making, but its nonlinear structure complicates reinforcement learning. A recent work \citep{thoppe2026reinforcement} addressed this difficulty by introducing a Bellman-compatible surrogate and two model-free fixed-point algorithms for optimizing it over stationary policies. However, their main convergence results are asymptotic. In this work, we establish finite-time rates of $\tilde{O} (1/\sqrt{n})$ for the aforementioned two algorithms under asynchronous Markovian sampling, where $n$ is the iteration index and $\tilde{O}$ hides logarithmic expressions. Importantly, we employ parameter-free choices for the stepsize parameter to derive these rate results. For the algorithmically simpler one-timescale method, the main challenge is that its update equation is not directly aligned with the contraction geometry of its underlying power-law operator. We overcome this mismatch by exploiting the boundedness, monotonicity, and homogeneity of the operator to obtain a local pseudo-contraction property for the relative-error dynamics. We then use a Moreau-envelope-based Lyapunov function and Polyak--Ruppert averaging to obtain the stated convergence rate with parameter-free stepsizes. For the two-timescale method, the main challenge is to control a tracking error on the faster timescale. These results provide the first finite-time guarantees for model-free discounted exponential-utility reinforcement learning.

Figures

Figures reproduced from arXiv: 2608.01917 by the authors.

Figure 1
Figure 1. Visual representation of the Learned policies for risk-neutral Q-learning and the one- and two-timescale algorithms, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

300 extracted references · 1 canonical work pages

  1. [1]

    and Bhatnagar, Shalabh , journal=

    Guin, Soumyajit and Borkar, Vivek S. and Bhatnagar, Shalabh , journal=. An Actor–Critic Algorithm With Function Approximation for Risk Sensitive Cost Markov Decision Processes , year=

  2. [2]

    Machine Learning , author =

    Risk-. Machine Learning , author =. 2002 , pages =. doi:10.1023/A:1017940631555 , number =

    doi:10.1023/a:1017940631555 2002

  3. [3]

    Mathematical Methods of Operations Research , author =

    Markov decision processes with risk-sensitive criteria: an overview , volume =. Mathematical Methods of Operations Research , author =. 2024 , keywords =. doi:10.1007/s00186-024-00857-0 , abstract =

    doi:10.1007/s00186-024-00857-0 2024

  4. [4]

    Annual Reviews in Control , author =

    Ergodic risk-sensitive control—. Annual Reviews in Control , author =. 2023 , keywords =. doi:10.1016/j.arcontrol.2023.03.001 , abstract =

    work page doi:10.1016/j.arcontrol.2023.03.001 2023

  5. [5]

    IEEE Transactions on Automatic Control , volume=

    Stochastic approximation for risk-aware Markov decision processes , author=. IEEE Transactions on Automatic Control , volume=. 2020 , publisher=

    2020

  6. [6]

    , booktitle =

    Thoppe, Gugan and A, Prashanth L and Bhat, Sanjay P. , booktitle =. Risk Estimation in a. 2024 , volume =

    2024

  7. [7]

    Proceedings of the 41st International Conference on Machine Learning (ICML) , pages =

    Policy Evaluation for Variance in Average Reward Reinforcement Learning , author =. Proceedings of the 41st International Conference on Machine Learning (ICML) , pages =. 2024 , volume =

    2024

  8. [8]

    Journal of machine learning research , volume=

    Risk-averse learning by temporal difference methods with Markov risk measures , author=. Journal of machine learning research , volume=

Show all 300 references
  1. [9]

    Conference on learning theory , pages=

    Finite-time error bounds for linear stochastic approximation andtd learning , author=. Conference on learning theory , pages=. 2019 , organization=

    2019

  2. [10]

    European Journal of Operational Research , volume=

    Approximate solutions to constrained risk-sensitive. European Journal of Operational Research , volume=. 2023 , author =

    2023

  3. [11]

    Finite Sample Analysis of Average-Reward

    Zhang, Sheng and Zhang, Zhe and Maguluri, Siva Theja , journal=. Finite Sample Analysis of Average-Reward

  4. [12]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Parameter-free optimal rates for nonlinear semi-norm contractions with applications to q-learning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  5. [13]

    Proceedings of The 5th Annual Learning for Dynamics and Control Conference , pages =

    Modified Policy Iteration for Exponential Cost Risk Sensitive MDPs , author =. Proceedings of The 5th Annual Learning for Dynamics and Control Conference , pages =. 2023 , editor =

    2023

  6. [14]

    arXiv preprint arXiv:2109.04976 , year=

    Optimal bounds for bit-sizes of stationary distributions in finite Markov chains , author=. arXiv preprint arXiv:2109.04976 , year=

    arXiv

  7. [15]

    Proceedings of the AAAI Conference on Artificial Intelligence , author=

    A Risk-Sensitive Approach to Policy Optimization , volume=. Proceedings of the AAAI Conference on Artificial Intelligence , author=. 2023 , month=

    2023

  8. [16]

    IEEE transactions on automatic control , volume=

    Sequential decision making with coherent risk , author=. IEEE transactions on automatic control , volume=. 2016 , publisher=

    2016

  9. [17]

    Machine Learning , volume=

    Variance-constrained actor-critic algorithms for discounted and average reward MDPs , author=. Machine Learning , volume=. 2016 , publisher=

    2016

  10. [18]

    2022 , volume =

    Foundations and Trends® in Machine Learning , title =. 2022 , volume =

    2022

  11. [19]

    Mathematical Methods of Operations Research , volume=

    Markov decision processes with risk-sensitive criteria: an overview , author=. Mathematical Methods of Operations Research , volume=. 2024 , publisher=

    2024

  12. [20]

    Advances in neural information processing systems , volume=

    Exponential bellman equation and improved regret bounds for risk-sensitive reinforcement learning , author=. Advances in neural information processing systems , volume=

  13. [21]

    and Baras, John S

    Noorani, Erfaun and Mavridis, Christos N. and Baras, John S. , journal=. Risk-Sensitive Reinforcement Learning With Exponential Criteria , year=

  14. [22]

    Risk-sensitive

    Jiang, Yuhua and Huang, Jiawei and Yuan, Yufeng and Mao, Xin and Yue, Yu and Zhao, Qianchuan and Yan, Lin , journal=. Risk-sensitive

  15. [23]

    2011 , publisher=

    Viability theory: new directions , author=. 2011 , publisher=

    2011

  16. [24]

    arXiv preprint arXiv:2402.09992 , year=

    Risk-sensitive soft actor-critic for robust deep reinforcement learning under distribution shifts , author=. arXiv preprint arXiv:2402.09992 , year=

    arXiv

  17. [25]

    A utility criterion for

    Jaquette, Stratton C , journal=. A utility criterion for. 1976 , publisher=

    1976

  18. [26]

    Uncertainty in economics , pages=

    Risk aversion in the small and in the large , author=. Uncertainty in economics , pages=. 1978 , publisher=

    1978

  19. [27]

    2002 , publisher=

    Nonlinear systems , author=. 2002 , publisher=

    2002

  20. [28]

    2025 , school=

    Contraction Theory in Control, Learning, and Optimization , author=. 2025 , school=

    2025

  21. [29]

    SIAM Journal on Applied Mathematics , volume=

    The theory of max-min, with applications , author=. SIAM Journal on Applied Mathematics , volume=. 1966 , publisher=

    1966

  22. [30]

    L. A. Prashanth and N. Korda and R. Munos , title =. Mach. Learn. , volume =

  23. [31]

    Conference On Learning Theory , pages=

    A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation , author=. Conference On Learning Theory , pages=

  24. [32]

    Dalal, G. and Sz. Thirty-Second AAAI Conference on Artificial Intelligence , year=

  25. [33]

    Lakshminarayanan and C

    C. Lakshminarayanan and C. Szepesvari , booktitle =. 2018 , volume =

    2018

  26. [34]

    and Ying, L

    Srikant, R. and Ying, L. , booktitle =. 2019 , volume =

    2019

  27. [35]

    The Annals of Applied Probability , volume=

    Convergence rate of linear two-time-scale stochastic approximation , author=. The Annals of Applied Probability , volume=. 2004 , publisher=

    2004

  28. [36]

    The Annals of Applied Probability , volume=

    Convergence rate and averaging of nonlinear two-time-scale stochastic approximation algorithms , author=. The Annals of Applied Probability , volume=. 2006 , publisher=

    2006

  29. [37]

    Conference On Learning Theory , pages=

    Finite sample analysis of two-timescale stochastic approximation with applications to reinforcement learning , author=. Conference On Learning Theory , pages=. 2018 , organization=

    2018

  30. [38]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    A tale of two-timescale reinforcement learning with the tightest finite-time bound , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  31. [39]

    Automatica , volume=

    Average cost temporal-difference learning , author=. Automatica , volume=. 1999 , publisher=

    1999

  32. [40]

    SIAM Journal on Control and Optimization , volume=

    Stochastic approximation for nonexpansive maps: Application to Q-learning algorithms , author=. SIAM Journal on Control and Optimization , volume=. 2002 , publisher=

    2002

  33. [41]

    and Bertsekas, D

    Abounadi, J. and Bertsekas, D. P. and Borkar, V. S. , journal=. Learning algorithms for. 2001 , publisher=

    2001

  34. [42]

    Operations Research Letters , volume=

    Concentration bounds for empirical conditional value-at-risk: The unbounded case , author=. Operations Research Letters , volume=. 2019 , publisher=

    2019

  35. [43]

    2009 , publisher=

    Approximation theorems of mathematical statistics , author=. 2009 , publisher=

    2009

  36. [44]

    Prashanth, L. A. and K. Jagannathan and R. K. Kolla , pages =. 2020 , Booktitle =

    2020

  37. [45]

    Advances in Neural Information Processing Systems , pages=

  38. [46]

    Operations Research Letters , volume=

    Deviation inequalities for an estimator of the conditional value-at-risk , author=. Operations Research Letters , volume=. 2010 , publisher=

    2010

  39. [47]

    Operations Research Letters , volume=

    Large deviations bounds for estimating conditional value-at-risk , author=. Operations Research Letters , volume=. 2007 , publisher=

    2007

  40. [48]

    2020 , eprint=

    Concentration of risk measures: A Wasserstein distance approach , author=. 2020 , eprint=

    2020

  41. [49]

    Proceedings of the 31st Conference On Learning Theory , pages =

    A general approach to multi-armed bandits under risk criteria , author=. Proceedings of the 31st Conference On Learning Theory , pages =

  42. [50]

    International Conference on Machine Learning , pages=

    Concentration Inequalities for Conditional Value at Risk , author=. International Conference on Machine Learning , pages=

  43. [51]

    Moharrami and Y

    M. Moharrami and Y. Murthy and A. Roy and R. Srikant , year=. A Policy Gradient Algorithm for the Risk-Sensitive Exponential Cost. 2202.04157 , archivePrefix=

    arXiv

  44. [52]

    Proceedings of the IEEE , volume=

    Variance-reduced methods for machine learning , author=. Proceedings of the IEEE , volume=. 2020 , publisher=

    2020

  45. [53]

    Methodology and Computing in Applied Probability , volume=

    Properties of distortion risk measures , author=. Methodology and Computing in Applied Probability , volume=. 2009 , publisher=

    2009

  46. [54]

    arXiv preprint arXiv:2102.01567 , year=

    A Lyapunov theory for finite-sample guarantees of asynchronous Q-learning and TD-learning variants , author=. arXiv preprint arXiv:2102.01567 , year=

    arXiv

  47. [55]

    Journal of Banking & Finance , volume=

    Spectral measures of risk: A coherent representation of subjective risk aversion , author=. Journal of Banking & Finance , volume=. 2002 , publisher=

    2002

  48. [56]

    Journal of Banking & Finance , volume=

    On the coherence of expected shortfall , author=. Journal of Banking & Finance , volume=. 2002 , publisher=

    2002

  49. [57]

    and Blake, D

    Dowd, K. and Blake, D. , journal=. 2006 , publisher=

    2006

  50. [58]

    Siam Review , volume=

    Optimization methods for large-scale machine learning , author=. Siam Review , volume=. 2018 , publisher=

    2018

  51. [59]

    Finance and stochastics , volume=

    Convex measures of risk and trading constraints , author=. Finance and stochastics , volume=. 2002 , publisher=

    2002

  52. [60]

    SIAM Journal on Control and Optimization , volume=

    Acceleration of stochastic approximation by averaging , author=. SIAM Journal on Control and Optimization , volume=. 1992 , publisher=

    1992

  53. [61]

    Handbook of Sequential Analysis , pages=

    Stochastic approximation , author=. Handbook of Sequential Analysis , pages=

  54. [62]

    2016 , publisher=

    Stochastic finance , author=. 2016 , publisher=

    2016

  55. [63]

    Electronic Journal of Statistics , volume=

    Stochastic optimization with momentum: Convergence, fluctuations, and traps avoidance , author=. Electronic Journal of Statistics , volume=. 2021 , publisher=

    2021

  56. [64]

    Mathematics of operations research , volume=

    Q-learning for risk-sensitive control , author=. Mathematics of operations research , volume=. 2002 , publisher=

    2002

  57. [65]

    Konda, V. R. and Borkar, V. S. , journal=. Actor-Critic--Type Learning Algorithms for. 1999 , publisher=

    1999

  58. [66]

    Bhatnagar and V

    S. Bhatnagar and V. S. Borkar and M. Akarapu , title =. Journal of Machine Learning Research , year =

  59. [67]

    Systems & Control Letters , volume=

    A sensitivity formula for risk-sensitive cost and the actor--critic algorithm , author=. Systems & Control Letters , volume=. 2001 , publisher=

    2001

  60. [68]

    SIAM Journal on Control and Optimization , volume=

    The ODE method for convergence of stochastic approximation and reinforcement learning , author=. SIAM Journal on Control and Optimization , volume=. 2000 , publisher=

    2000

  61. [69]

    Zhang and A

    K. Zhang and A. Koppel and H. Zhu and T. Basar , title =

  62. [70]

    International Conference on Machine Learning , pages =

    Stochastic variance-reduced policy gradient , author =. International Conference on Machine Learning , pages =. 2018 , volume =

    2018

  63. [71]

    International Conference on Machine Learning , pages=

    Hessian aided policy gradient , author=. International Conference on Machine Learning , pages=. 2019 , organization=

    2019

  64. [72]

    Gelfand, S. B. and Mitter, S. K. , journal=. Recursive stochastic algorithms for global optimization in R\^. 1991 , publisher=

    1991

  65. [73]

    Annales de l'IHP Probabilit

    Les algorithmes stochastiques contournent-ils les pieges? , author=. Annales de l'IHP Probabilit

  66. [74]

    Conference on Learning Theory , pages=

    Escaping from saddle points—online stochastic gradient for tensor decomposition , author=. Conference on Learning Theory , pages=. 2015 , organization=

    2015

  67. [75]

    International Conference on Machine Learning , pages=

    How to escape saddle points efficiently , author=. International Conference on Machine Learning , pages=. 2017 , organization=

    2017

  68. [76]

    The Annals of Probability , volume=

    Nonconvergence to unstable points in urn models and stochastic approximations , author=. The Annals of Probability , volume=. 1990 , publisher=

    1990

  69. [77]

    and Glynn, P

    Asmussen, S. and Glynn, P. W. , volume=. 2007 , publisher=

    2007

  70. [78]

    2014 , publisher=

    Lectures on Stochastic Programming: Modeling and Theory , author=. 2014 , publisher=

    2014

  71. [79]

    Aleksandrov and V.I

    V.M. Aleksandrov and V.I. Sysoyev and V.V. Shemeneva , Journal =. Stochastic optimization , Volume =

  72. [80]

    , author=

    Policy gradient methods for reinforcement learning with function approximation. , author=. NIPS , volume=

  73. [81]

    Mathematics of Operations Research , volume=

    Risk-averse approximate dynamic programming with quantile-based risk measures , author=. Mathematics of Operations Research , volume=. 2017 , publisher=

    2017

  74. [82]

    More risk-sensitive

    B\". More risk-sensitive. Mathematics of Operations Research , Number =

  75. [83]

    and Ruszczynski, A

    Dentcheva, D. and Ruszczynski, A. , Journal =. Optimization with stochastic dominance constraints , Volume =

  76. [84]

    arXiv , Author =:1502.03919 , Journal =

    Policy gradient for coherent risk measures , Volume =. arXiv , Author =:1502.03919 , Journal =

    arXiv

  77. [85]

    Tsitsiklis, J. N. and Van Roy, B. , Journal =. Average cost temporal-difference learning , Volume =

  78. [86]

    Wiley Encyclopedia of Operations Research and Management Science , Title =

    Szepesv. Wiley Encyclopedia of Operations Research and Management Science , Title =

  79. [87]

    Tsitsiklis, J. N. and Van Roy, B. , Journal =. An analysis of temporal-difference learning with function approximation , Volume =

  80. [88]

    Bhatnagar and R

    S. Bhatnagar and R. Sutton and M. Ghavamzadeh and M. Lee , Journal =. Natural actor-critic algorithms , Volume =

  81. [89]

    Fleming, W. H. and McEneaney, W. M. , Journal =. Risk-sensitive control on an infinite time horizon , Volume =

  82. [90]

    , Isbn =

    Whittle, P. , Isbn =

  83. [91]

    Borkar, V. S. and Meyn, S. P. , Journal =

  84. [92]

    Barto and R

    A. Barto and R. S. Sutton and C. Anderson , Journal =. Neuron-like elements that can solve difficult learning control problems , Volume =

  85. [93]

    R. S. Sutton , School =

  86. [94]

    Shen and W

    Y. Shen and W. Stannat and K. Obermayer , Journal =. Risk-sensitive

  87. [95]

    V. S. Borkar , Journal =. A sensitivity formula for the risk-sensitive cost and the actor-critic algorithm , Volume =

  88. [96]

    V. S. Borkar , Journal =. Q-learning for risk-sensitive control , Volume =

  89. [97]

    Basu and T

    A. Basu and T. Bhattacharyya and V. S. Borkar , Journal =. A learning algorithm for risk-sensitive cost , Volume =

  90. [98]

    Mathematics of Operations Research , volume =

    Browne, Sid , title =. Mathematics of Operations Research , volume =. 1995 , doi =. https://doi.org/10.1287/moor.20.4.937 , abstract =

    1995 doi

  91. [99]

    Mihatsch and R

    O. Mihatsch and R. Neuneier , Journal =. Risk-sensitive reinforcement learning , Volume =

  92. [100]

    Filar and D

    J. Filar and D. Krass and K. Ross , Journal =. Percentile performance criteria for limiting average

  93. [101]

    A. Ruszczy. Risk-averse dynamic programming for. Mathematical Programming , Pages =

  94. [102]

    Prashanth, L. A. , Booktitle =

  95. [103]

    and Chow, Y

    Tamar, A. and Chow, Y. and Ghavamzadeh, M. and Mannor, S. , booktitle =. Policy gradient for coherent risk measures , volume =

  96. [104]

    SIAM Journal on Optimization , volume=

    Stochastic first-and zeroth-order methods for nonconvex stochastic programming , author=. SIAM Journal on Optimization , volume=. 2013 , publisher=

    2013

  97. [105]

    The Journal of Machine Learning Research , volume=

    Risk-constrained reinforcement learning with percentile risk criteria , author=. The Journal of Machine Learning Research , volume=. 2017 , publisher=

    2017

  98. [106]

    Prashanth, L. A. and Ghavamzadeh, M. , Journal =. Variance-constrained actor-critic algorithms for discounted and average reward

  99. [107]

    Assouad,

    Yu, Bin , Booktitle =. Assouad,

  100. [108]

    Learning when to stop thinking and do something! , Year =

    P. Learning when to stop thinking and do something! , Year =. International Conference on Machine Learning , Pages =

  101. [109]

    and Bartlett, P

    Baxter, J. and Bartlett, P. L. , Journal =. Infinite-horizon policy-gradient estimation , Volume =

  102. [110]

    and Prashanth, L

    Gopalan, A. and Prashanth, L. A. and Fu, M. C. and Marcus, S. I. , Booktitle =. Weighted bandits or: How bandits learn distorted values that are not expected , Year =

  103. [111]

    Puterman , Date-Modified =

    M. Puterman , Date-Modified =

  104. [112]

    D. P. Bertsekas and J. N. Tsitsiklis , Publisher =. Neuro-Dynamic Programming , Year =

  105. [113]

    and Frikha, N

    Bardou, O. and Frikha, N. and Pages, G. , Journal =

  106. [114]

    Arrow, K. J. , Publisher =. Essays in the Theory of Risk Bearing , Year =

  107. [115]

    , Journal =

    Markowitz, H. , Journal =. Portfolio selection , Volume =

  108. [116]

    and Vreeken, J

    Wiering, M. and Vreeken, J. and van Veenen, J. and Koopman, A. , Booktitle =. Simulation and optimization of traffic in a city , Year =

  109. [117]

    and Li, Y

    Yu, Y.-L. and Li, Y. and Schuurmans, D. and. A General Projection Property for Distribution Families , Year =. NIPS , Date-Added =

  110. [118]

    Prashanth, L. A. and Bhatnagar, S. , Date-Modified =. Reinforcement learning With function approximation for traffic signal control , Volume =

  111. [119]

    Jie and L

    C. Jie and L. A. Prashanth and M. C. Fu and S. I. Marcus and C. Szepesv. Stochastic optimization in a cumulative prospect theory framework , Volume =. IEEE Transactions on Automatic Control , Number =

  112. [120]

    L. A. Prashanth and S. Bhatnagar and M. C. Fu and S. I. Marcus , Journal =. Adaptive system optimization using random directions stochastic approximation , Volume =

  113. [121]

    Prashanth, L. A. and Bhatnagar, S. , Date-Modified =. Threshold tuning using stochastic optimization for graded signal control , Volume =. IEEE Transactions on Vehicular Technology , Number =

  114. [122]

    Barberis, N. C. , Journal =. Thirty years of prospect theory in economics: A review and assessment , Year =

  115. [123]

    Theories of decision-making in economics and behavioral science , Volume =

    Simon, Herbert Alexander , Date-Added =. Theories of decision-making in economics and behavioral science , Volume =. The American Economic Review , Pages =

  116. [124]

    and Morgenstern, O

    Von Neumann, J. and Morgenstern, O. , Booktitle =. Theory of Games and Economic Behavior , Year =

  117. [125]

    Risk, ambiguity and the

    Ellsberg, D , Date-Added =. Risk, ambiguity and the. The Quarterly Journal of Economics , Number =

  118. [126]

    , Date-Added =

    Allais, M. , Date-Added =. Le comportement de l'homme rationel devant le risque: Critique des postulats et axioms de l'ecole americaine , Volume =. Econometrica , Pages =

  119. [127]

    , Journal =

    Starmer, C. , Journal =. Developments in non-expected utility theory: The hunt for a descriptive theory of choice under risk , Year =

  120. [128]

    , Publisher =

    Fishburn, P.C. , Publisher =

  121. [129]

    Quiggin, John , Publisher =

  122. [130]

    The probability weighting function , Year =

    Prelec, Drazen , Journal =. The probability weighting function , Year =

  123. [131]

    Curvature of the probability weighting function , Volume =

    Wu, George and Gonzalez, Richard , Journal =. Curvature of the probability weighting function , Volume =

  124. [132]

    Three variants on the

    Conlisk, John , Journal =. Three variants on the

  125. [133]

    An experimental test of several generalized utility theories , Volume =

    Camerer, Colin F , Journal =. An experimental test of several generalized utility theories , Volume =

  126. [134]

    Recent tests of generalizations of expected utility theory , Year =

    Camerer, Colin F , Booktitle =. Recent tests of generalizations of expected utility theory , Year =

  127. [135]

    Predictions about indifference curves inside the unit triangle: A test of variants of expected utility theory , Volume =

    Harless, David W , Journal =. Predictions about indifference curves inside the unit triangle: A test of variants of expected utility theory , Volume =

  128. [136]

    A test of generalized expected utility theory , Volume =

    Sopher, Barry and Gigliotti, Gary , Journal =. A test of generalized expected utility theory , Volume =

  129. [137]

    Violations of the betweenness axiom and nonlinearity in probability , Volume =

    Camerer, Colin F and Ho, Teck-Hua , Journal =. Violations of the betweenness axiom and nonlinearity in probability , Volume =

  130. [138]

    On the shape of the probability weighting function , Volume =

    Gonzalez, Richard and Wu, George , Journal =. On the shape of the probability weighting function , Volume =

  131. [139]

    Parameter-free elicitation of utility and probability weighting functions , Volume =

    Abdellaoui, Mohammed , Journal =. Parameter-free elicitation of utility and probability weighting functions , Volume =

  132. [140]

    Are groups more (or less) consistent than individuals? , Volume =

    Bone, John and Hey, John and Suckling, John , Journal =. Are groups more (or less) consistent than individuals? , Volume =

  133. [141]

    Can agents learn to rational expectations? Some results on convergence and stability of learning in the UK stock market , Year =

    Timmerman, Allan , Journal =. Can agents learn to rational expectations? Some results on convergence and stability of learning in the UK stock market , Year =

  134. [142]

    Prospect theory and asset prices , Year =

    Barberis, Nicholas and Huang, Ming and Santos, Tano , Institution =. Prospect theory and asset prices , Year =

  135. [143]

    Associations between health status and utilities implications for policy , Volume =

    Lenert, Leslie A and Treadwell, Jonathan R and Schwartz, Carolyn E , Journal =. Associations between health status and utilities implications for policy , Volume =

  136. [144]

    Application of cumulative prospect theory: Implied seismic design preference , Volume =

    Goda, K and Hong, HP , Journal =. Application of cumulative prospect theory: Implied seismic design preference , Volume =

  137. [145]

    Adaptive route choices in risky traffic networks: A prospect theory approach , Volume =

    Gao, Song and Frejinger, Emma and Ben-Akiva, Moshe , Journal =. Adaptive route choices in risky traffic networks: A prospect theory approach , Volume =

  138. [146]

    Drivers' mental representation of travel time and departure time choice in uncertain traffic network conditions , Volume =

    Fujii, Satoshi and Kitamura, Ryuichi , Journal =. Drivers' mental representation of travel time and departure time choice in uncertain traffic network conditions , Volume =

  139. [147]

    Network knowledge and route choice , Year =

    Ramming, Michael Scott , School =. Network knowledge and route choice , Year =

  140. [148]

    Exploring the

    Weinberg, Bruce D and Davis, Lenita , Journal =. Exploring the

  141. [149]

    Non-expected utility and the robustness of the classical insurance paradigm , Volume =

    Machina, Mark J , Journal =. Non-expected utility and the robustness of the classical insurance paradigm , Volume =

  142. [150]

    Substitution, risk aversion, and the temporal behavior of consumption and asset returns: A theoretical framework , Year =

    Epstein, Larry G and Zin, Stanley E , Journal =. Substitution, risk aversion, and the temporal behavior of consumption and asset returns: A theoretical framework , Year =

  143. [151]

    Substitution, risk aversion, and the temporal behavior of consumption and asset returns: An empirical analysis , Year =

    Epstein, Larry G and Zin, Stanley E , Journal =. Substitution, risk aversion, and the temporal behavior of consumption and asset returns: An empirical analysis , Year =

  144. [152]

    and Kallenberg, L

    Filar, J. and Kallenberg, L. and Lee, H. , Journal =. Variance-penalized

  145. [153]

    , Journal =

    Sobel, M. , Journal =. The variance of discounted

  146. [154]

    and Sobel, M

    Chung, K. and Sobel, M. , journal=. Discounted. 1987 , publisher=

    1987

  147. [155]

    and Tsitsiklis, J

    Mannor, S. and Tsitsiklis, J. N. , Journal =

  148. [156]

    and Glassner, Y

    Tamar, A. and Glassner, Y. and Mannor, S. , Journal =

  149. [157]

    Chow, Yinlam and Ghavamzadeh, Mohammad , Booktitle =

  150. [158]

    and Kumar, S

    Bhatnagar, S. and Kumar, S. , Journal =. A simultaneous perturbation stochastic approximation-based actor-critic algorithm for Markov decision processes , Volume =

  151. [159]

    Van der Vaart, A. W. , Publisher =. Asymptotic statistics , Volume =

  152. [160]

    Athreya, K. B. and Lahiri, S. N. , Publisher =

  153. [161]

    Laha, R. G. and Rohatgi, V. K. , Publisher =. Probability Theory , Year =

  154. [162]

    Chang, H. S. and Fu, M. C. and Hu, J. and Marcus, S. I. , Publisher =

  155. [163]

    Chang, H. S. and Hu, J. and Fu, M. C. and Marcus, S. I. , Publisher =

  156. [164]

    Handbook of Simulation Optimization , Year =

  157. [165]

    M. C. Fu and J. Q. Hu , Publisher =. Conditional

  158. [166]

    Wasserman, L. A. , Pages =. All of Nonparametric Statistics , Year =

  159. [167]

    and Murray, W

    Gill, P.E. and Murray, W. and Wright, M.H. , Publisher =

  160. [168]

    Bhatnagar , Journal =

    S. Bhatnagar , Journal =. An actor--critic algorithm with function approximation for discounted cost constrained

  161. [169]

    and Prashanth, L

    Bhatnagar, S. and Prashanth, L. A. , Date-Modified =. Simultaneous perturbation Newton algorithms for simulation optimization , Volume =. Journal of Optimization Theory and Applications , Number =

  162. [170]

    Borkar, V. S. , Booktitle =. Learning algorithms for risk-sensitive control , Volume =

  163. [171]

    Spall, J. C. , Journal =. Adaptive stochastic approximation by the simultaneous perturbation method , Volume =

  164. [172]

    and Monro, S

    Robbins, H. and Monro, S. , Journal =. A stochastic approximation method , Year =

  165. [173]

    Random gradient-free minimization of convex functions , Year =

    Nesterov, Yurii , Institution =. Random gradient-free minimization of convex functions , Year =

  166. [174]

    Flaxman, A. D. and Kalai, A. T. and McMahan, H. B. , Booktitle =. Online convex optimization in the bandit setting: gradient descent without a gradient , Year =

  167. [175]

    Optimal Algorithms for Online Convex Optimization with Multi-Point Bandit Feedback , Year =

    Agarwal, Alekh and Dekel, Ofer and Xiao, Lin , Booktitle =. Optimal Algorithms for Online Convex Optimization with Multi-Point Bandit Feedback , Year =

  168. [176]

    Duchi, J. C. and Jordan, M. I. and Wainwright, M. J. and Wibisono, A. , Journal =. Optimal rates for zero-order convex optimization: the power of two function evaluations , Year =

  169. [177]

    and Frikha, N

    Fathi, M. and Frikha, N. , Journal =. Transport-entropy inequalities and deviation estimates for stochastic approximation schemes , Volume =

  170. [178]

    Spall, J. C. , Publisher =

  171. [179]

    Bertsekas, D. P. , Publisher =. Abstract Dynamic Programming , Year =

  172. [180]

    and Prasad, H

    Bhatnagar, S. and Prasad, H. L. and Prashanth,. Stochastic Recursive Algorithms for Optimization , Volume =

  173. [181]

    Sutton, R. S. , Journal =. Learning to predict by the methods of temporal differences , Volume =

  174. [182]

    and Wakker, P

    Fennema, H. and Wakker, P. , Journal =. Original and cumulative prospect theory: A discussion of empirical differences , Volume =

  175. [183]

    and Tversky, A

    Kahneman, D. and Tversky, A. , Journal =. Prospect theory: An analysis of decision under risk , Year =

  176. [184]

    and Kahneman, D

    Tversky, A. and Kahneman, D. , Journal =. Advances in prospect theory: Cumulative representation of uncertainty , Volume =

  177. [185]

    , Date-Modified =

    Lin, K. , Date-Modified =. Stochastic Systems with Cumulative Prospect Theory , Year =

  178. [186]

    2018 , publisher=

    Reinforcement Learning: An Introduction , author=. 2018 , publisher=

    2018

  179. [187]

    Bertsekas, D. P. , Keywords =. Dynamic Programming and Optimal Control, Vol. II, 4th edition , Year =

  180. [188]

    , Publisher =

    Altman, E. , Publisher =

  181. [189]

    , Journal =

    Sion, M. , Journal =. On general minimax theorems , Volume =

  182. [190]

    Bertsekas , Publisher =

    D. Bertsekas , Publisher =. Nonlinear programming , Year =

  183. [191]

    Bartlett, Peter L and Baxter, Jonathan , Journal =

  184. [192]

    Mannor, Shie and Tsitsiklis, John , Journal =

  185. [193]

    Atchade, Yves F and Fort, Gersende and Moulines, Eric , Journal =

  186. [194]

    and Delbaen, F

    Artzner, P. and Delbaen, F. and Eber, J. and Heath, D. , Journal =

  187. [195]

    Borkar, V. S. , Publisher =

  188. [196]

    Tamar and D

    A. Tamar and D. Temporal difference methods for the variance of the reward to go , Year =. International Conference on Machine Learning , Pages =

  189. [197]

    and Castro, D

    Tamar, A. and Castro, D. D. and Mannor, S. , Booktitle =

  190. [198]

    and Mannor, S

    Tamar, A. and Mannor, S. , Journal =

  191. [199]

    Borkar, V. S. and Jain, R. , Booktitle =

  192. [200]

    Lemaire, Vincent and Pages, Gilles , Journal =

  193. [201]

    Glynn, Peter W , Booktitle =

  194. [202]

    Borkar, V. S. , Journal =

  195. [203]

    and Whinston, M

    Mas-Colell, A. and Whinston, M. and Green, J. , Publisher =

  196. [204]

    and Clark, D

    Kushner, H. and Clark, D. , Pages =

  197. [205]

    Rockafellar, R. T. and Uryasev, S. , Journal =

  198. [206]

    Prashanth, L. A. and Jie, Cheng and Fu, M. C. and Marcus, S. I. and Szepesv. arXiv preprint arXiv:1506.02632v3 , Title =

    arXiv

  199. [207]

    , Publisher =

    Powell, W.B. , Publisher =

  200. [208]

    P. T. A Tutorial on the Cross-Entropy Method. , Volume =. Annals of Operation Research , Pages =

  201. [209]

    S. \'. Metaheuristics , Year =. Handbooks in Operations Research and Management Science: Simulation , Chapter =

  202. [210]

    Grama, A

    A. Grama, A. Gupta, G. Karypis, and V. Kumar , Edition =. Introduction to Parallel Computing , Url =. 2003 , Bdsk-Url-1 =

    2003

  203. [211]

    Abate and W

    J. Abate and W. Whitt , Journal =. Numerical Inversion of

  204. [212]

    and Broadie, M

    Acworth, P. and Broadie, M. and Glasserman, P. , Booktitle =. A Comparison of Some

  205. [213]

    Agrawal , Journal =

    R. Agrawal , Journal =. Sample mean based index policies with O( n) regret for the multi-armed bandit problem , Volume =

  206. [214]

    Agrawal and D

    R. Agrawal and D. Teneketzis and V. Anantharam , Journal =. Asymptotically efficient adaptive allocation schemes for controlled

  207. [215]

    and Thompson, H

    Ahn, C. and Thompson, H. , Journal =. Jump-diffusion processes and the term structure of interest rates , Volume =

  208. [216]

    Alba , Publisher =

    E. Alba , Publisher =. Parallel Metaheuristics: A New Class of Algorithms , Year =

  209. [217]

    F. B. Alt and M. C. Fu and B. L. Golden , Booktitle =

  210. [218]

    Altman and G

    E. Altman and G. Koole , Journal =. On submodular value functions and complex dynamic programming , Volume =

  211. [219]

    Amin, K. I. and Bodurtha, J. N. , Journal =. Discrete-time valuation of

  212. [220]

    Andrad\'ottir , Booktitle =

    S. Andrad\'ottir , Booktitle =. Simulation optimization , Year =

  213. [221]

    Andrad\'ottir , Booktitle =

    S. Andrad\'ottir , Booktitle =. An overview of simulation optimization with random search , Year =

  214. [222]

    A method for discrete stochastic optimization , Volume =

    Andrad\'. A method for discrete stochastic optimization , Volume =. Management Science , Pages =

  215. [223]

    On the choice of alternative measures in importance sampling with

    Andrad\'. On the choice of alternative measures in importance sampling with. Operations Research , Number =

  216. [224]

    Arapostathis and V

    A. Arapostathis and V. S. Borkar and E. Fern. Discrete-time controlled

  217. [225]

    Arulampalam and S

    S. Arulampalam and S. Maskell and N. J. Gordon and T. Clapp , Journal =. A tutorial on particle filters for on-line non-linear/non-

  218. [226]

    Asanovic and R

    K. Asanovic and R. Bodik and B. C. Catanzaro and J. J. Gebis and P. Husbands and K. Keutzer and D. A. Patterson and W. L. Plishker and J. Shalf and S. W. Willams and K. A. Yelick , Institution =. The Landscape of Parallel Computing Research: A View from. 2006 , Bdsk-Url-1 =

    2006

  219. [227]

    and Glynn, P

    Asmussen, S. and Glynn, P. , Publisher =. Stochastic Simulation: Algorithms and Analysis , Year =

  220. [228]

    and Glynn, P

    Asmussen, S. and Glynn, P. and Pitman, J. , Journal =. Discretization error in simulation of one dimensional reflecting

  221. [229]

    Auer and N

    P. Auer and N. Cesa-Bianchi and P. Fisher , Journal =. Finite-time analysis of the multiarmed bandit problem , Volume =

  222. [230]

    Auer and N

    P. Auer and N. Cesa-Bianchi and Y. Freund and R. E. Schapire , Journal =. The nonstochastic multiarmed bandit problem , Volume =

  223. [231]

    , School =

    Averbukh, V. , School =. Pricing

  224. [232]

    Avramidis, A. N. , Booktitle =. Efficient pricing of barrier options with the variance gamma model , Year =

  225. [233]

    Avramidis, A. N. and L'Ecuyer, P. , Journal =. Efficient

  226. [234]

    A. N. Avramidis and P. L'Ecuyer , Booktitle =. Modeling and simulation of call centers , Year =

  227. [235]

    Avramidis, A. N. and L'Ecuyer, P. and Tremblay, P.-A. , Booktitle =. Efficient simulation of gamma and variance-gamma processes , Year =

  228. [236]

    Azimi-Sadjadi and P

    B. Azimi-Sadjadi and P. S. Krishnaprasad , Journal =. Approximate nonlinear filtering and its application in navigation , Volume =

  229. [237]

    , Journal =

    Bachelier, L. , Journal =. Th\'

  230. [238]

    Baglietto and T

    M. Baglietto and T. Parisini and R. Zoppoli , Booktitle =. Neural approximators and team theory for dynamic routing: a receding horizon approach , Year =

  231. [239]

    and Cao, C

    Bakshi, G. and Cao, C. and Chen, Z. , Journal =. Empirical performance of alternative option pricing models , Volume =

  232. [240]

    and Madan, D

    Bakshi, G. and Madan, D. B. , Journal =. Spanning and derivative security valuation , Year =

  233. [241]

    Balakrishnan and A

    V. Balakrishnan and A. L. Tits , Booktitle =. Numerical optimization-based design , Year =

  234. [242]

    , Publisher =

    Banks, J. , Publisher =. Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice , Year =

  235. [243]

    and Carson, J

    Banks, J. and Carson, J. S. and Nelson, B. L. , Edition =. Discrete Event Systems Simulation , Year =

  236. [244]

    Carson and Barry L

    Jerry Banks and John S. Carson and Barry L. Nelson and David M. Nicol , Edition =. Discrete-Event System Simulation , Year =

  237. [245]

    Barash , Booktitle =

    D. Barash , Booktitle =. A genetic search in policy space for solving

  238. [246]

    Barndorff-Nielsen, O. E. , Journal =. Processes of normal inverse

  239. [247]

    and Martineau, D

    Barraquand, J. and Martineau, D. , Journal =. Numerical valuation of high dimensional multivariate

  240. [248]

    Barton and J.S

    R.R. Barton and J.S. Ivey , Journal =

  241. [249]

    R. R. Barton and M. Meckesheimer , Booktitle =. Metamodel-based simulation optimization , Year =

  242. [250]

    Bashyam and M

    S. Bashyam and M. C. Fu , Journal =. Optimization of (s,

  243. [251]

    Bashyam and M

    S. Bashyam and M. C. Fu , Journal =. Application of Perturbation Analysis to a Class of Periodic Review (s,

  244. [252]

    Bashyam and M

    S. Bashyam and M. C. Fu and B. K. Kaku , Journal =. Application of Perturbation Analysis to Multiproduct Capacitated Production-Inventory Control , Volume =

  245. [253]

    Bashyam and M

    S. Bashyam and M. C. Fu and B. K. Kaku , Booktitle =. Application of Perturbation Analysis to Multiproduct Capacitated Production-Inventory Control , Year =

  246. [254]

    and Rennie, A

    Baxter, M. and Rennie, A. , Publisher =. Financial Calculus: An Introduction to Derivative Pricing , Year =

  247. [255]

    Bean and W

    J. Bean and W. Hopp and I. Duenyas , Journal =. A Stopping Rule for Forecast Horizon in Nonhomogeneous

  248. [256]

    Beasley, J. D. and Springer, S. G. , Journal =. The Percentage Points of the Normal Distribution , Volume =

  249. [257]

    R. E. Bechhofer and T. J. Santner and D. M. Goldsman , Publisher =. Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons , Year =

  250. [258]

    R. J. Beckman and R. D. Cook , Journal =. Outliers , Volume =

  251. [259]

    Bellman and Stuart E

    Richard E. Bellman and Stuart E. Dreyfus , Journal =. Functional approximations and dynamic programming , Volume =

  252. [260]

    Bellman , Publisher =

    Richard E. Bellman , Publisher =. Dynamic Programming , Year =

  253. [261]

    Finite State

    Cyrus Derman , Publisher =. Finite State

  254. [262]

    R. L. Bennett and M. C. Fu and R. Jarrow and D. A. Nuxoll and H. Zhang , Booktitle =. A Loss Default Simulation Model of the

  255. [263]

    Bentley , Journal =

    J. Bentley , Journal =. Multidimensional binary search trees in database applications , Volume =

  256. [264]

    Benveniste and M

    A. Benveniste and M. Metivier and P. Priouret , Publisher =. Adaptive Algorithms and Stochastic Approximations , Year =

  257. [265]

    , Journal =

    Berman, L. , Journal =. Accelerating

  258. [266]

    D. A. Berry and B. Fristedt , Publisher =. Bandit Problems: Sequential Allocation of Experiments , Year =

  259. [267]

    J. M. Berry , Journal =. Divided on Derivatives , Volume =

  260. [268]

    D. P. Bertsekas , Edition =. Dynamic Programming and Optimal Control, Vols. 1 & 2 , Year =

  261. [269]

    D. P. Bertsekas , Journal =. Dynamic programming and suboptimal control: A survey from

  262. [270]

    D. P. Bertsekas , Booktitle =. Differential training of rollout policies , Year =

  263. [271]

    D. P. Bertsekas and D. A. Castanon , Journal =. Rollout algorithms for stochastic scheduling problems , Volume =

  264. [272]

    D. P. Bertsekas and D. A. Castanon , Journal =. Adaptive aggregation methods for infinite horizon dynamic programming , Volume =

  265. [273]

    D. P. Bertsekas and S. E. Shreve , Publisher =. Stochastic Control: The Discrete Time Case , Year =

  266. [274]

    D. P. Bertsekas and J. N. Tsitsiklis , Publisher =. Parallel and Distributed Computation: Numerical Methods , Year =

  267. [275]

    Bes and J

    C. Bes and J. B. Lasserre , Journal =. An on-line procedure in discounted infinite-horizon stochastic optimal control , Volume =

  268. [276]

    Bhatnagar and E

    S. Bhatnagar and E. Fern. Proceedings of the 38th

  269. [277]

    Bhatnagar and M

    S. Bhatnagar and M. C. Fu and S. I. Marcus , Booktitle =. Rate-Based

  270. [278]

    Systems & Control Letters , volume=

    Stochastic approximation with two time scales , author=. Systems & Control Letters , volume=. 1997 , publisher=

    1997

  271. [279]

    Bhatnagar and M

    S. Bhatnagar and M. C. Fu and S. I. Marcus , Booktitle =. Two Timescale

  272. [280]

    Bhatnagar and M

    S. Bhatnagar and M. C. Fu and S. I. Marcus and S. Bhatnagar , Journal =. Two Timescale Algorithms for Simulation Optimization of Hidden

  273. [281]

    Bhatnagar and M

    S. Bhatnagar and M. C. Fu and S. I. Marcus and P. J. M. Fard , Journal =. An Optimal Structured Feedback Policy for

  274. [282]

    Bhatnagar and M

    S. Bhatnagar and M. C. Fu and S. I. Marcus and Y. He , Booktitle =

  275. [283]

    Bhatnagar and M

    S. Bhatnagar and M. C. Fu and S. I. Marcus and I. J. Wang , Journal =. Two-Timescale Simultaneous Perturbation Stochastic Approximation Using Deterministic Perturbation Sequences , Volume =

  276. [284]

    and Constantinides, G

    Bhattacharya, S. and Constantinides, G. , Publisher =. Theory of Valuation: Frontiers of Modern Financial Theory , Year =

  277. [285]

    Bhulai and G

    S. Bhulai and G. Koole , Institution =. On the structure of value functions for threshold policies in queueing models , Year =

  278. [286]

    Bieniawski and D

    S. Bieniawski and D. H. Wolpert and I. Kroo , Booktitle =. Discrete, Continuous, and Constrained Optimization Using Collectives , Year =

  279. [287]

    Bingham, N. H. and Kiesel, R. , Publisher =. Risk-Neutral Valuation: Pricing and Hedging of Financial Derivatives , Year =

  280. [288]

    Birge, J. R. , Journal =. Stochastic Programming Computation and Applications , Volume =

  281. [289]

    Birge, J. R. and Louveaux, F. , Edition =. Introduction to Stochastic Programming , Year =

  282. [290]

    and Scholes, M

    Black, F. and Scholes, M. , Journal =. The Pricing of Options and Corporate Liabilities , Volume =

  283. [291]

    V. D. Blondel and J. N. Tsitsiklis , Journal =. A survey of computational complexity results in systems and control , Volume =

  284. [292]

    Bonet , Booktitle =

    B. Bonet , Booktitle =. An -optimal grid-based algorithm for partially observable

  285. [293]

    V. S. Borkar , Booktitle =. Convex analytic methods in

  286. [294]

    V. S. Borkar , Journal =. White-noise representations in stochastic realization theory , Volume =

  287. [295]

    , Institution =

    Bossaerts, P. , Institution =. Simulation Estimators of Optimal Early Exercise , Year =

  288. [296]

    Botev and D

    Z. Botev and D. P. Kroese , Booktitle =. Global Likelihood Optimization via the Cross-Entropy Method with an Application to Mixture Models , Year =

  289. [297]

    Bourbeau and B

    B. Bourbeau and B. Gendron and T. G. Crainic , Journal =. Branch-and-Bound Parallelization Strategies Applied to a Depot Location and Container Fleet Management Problem , Volume =

  290. [298]

    R. A. Bowman , Journal =. Stochastic gradient-based time-cost tradeoffs in

  291. [299]

    George E. P. Box and Norman R. Draper , Month =. Response surfaces, mixtures, and ridge analyses , Year =

  292. [300]

    Boyle, P. P. , Journal =. Options: A

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.