Pith. sign in

REVIEW 2 major objections 4 minor 156 references

Fitted Occupancy-Ratio Evaluation without Bellman Completeness

T0 review · 2 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that offline policy evaluation can be based on fitting the discounted occupancy ratio alone, with no Bellman-completeness condition when the ratio is realizable.

desk verdict FORE's core claim — ratio realizability suffices, no Bellman completeness — is correct for convex log-ratio classes; the neural/GBM implementations advertised in Section 3.3 sit outside the proven guarantees, a real scoping gap but not a fatal one. read the letter →

arxiv 2607.05375 v2 pith:O4CKTELL submitted 2026-07-06 stat.ML cs.LG

Lars van der Laan , Nathan Kallus This is my paper · ORCID
classification stat.MLcs.LG
keywords off-policyevaluationdiscountedoccupancyratioadjointBellmanequationKLprojectioncompletenessfittediterationofflinereinforcementlearningcoveragediagnosis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces FORE, a fitted iteration that estimates the discounted occupancy ratio by repeatedly applying a KL projection to adjoint Bellman updates. Its central claim is that this projected recursion contracts in relative entropy toward the true ratio at the discount factor, so the only substantive approximation requirement is that the true discounted occupancy ratio be realizable (or well approximated) by the chosen log-ratio class. No Bellman completeness of a value class, adjoint Bellman completeness of a ratio class, or dual completeness of a critic class is needed. The finite-sample bound separates a geometrically decaying initialization term, a KL approximation error, and a statistical error controlled by the complexity of the ratio class. A coverage-stopped variant targets the occupancy accumulated before the first uncovered state-action pair, giving a conservative value bound when full coverage fails.

What carries the argument

The central object is the adjoint Bellman operator combined with KL projection onto a normalized log-ratio class. The argument's load-bearing identity is the contraction of the adjoint Bellman map in relative entropy, which follows from joint convexity of KL divergence and the data-processing inequality for Markov kernels. A key lemma converts the KL-projected update into a single-level density-ratio objective evaluated on initial-state moments and one-step target-policy transitions. This aligns the geometry of the Bellman update with the geometry of the projection, which is what lets ratio realizability replace Bellman completeness.

What would settle it

In a finite MDP with a one-dimensional feature where the true discounted occupancy ratio is provably outside the fitted log-linear class, run the population projected recursion; the theorem predicts KL error plateauing at a positive multiple of the KL approximation error rather than decaying to zero. If the error instead continues to zero, the claimed control by approximation error alone is false. Alternatively, a target policy that assigns zero probability to an offline-supported action should make the full-ratio bound fail and the method should only recover the coverage-stopped value.

Watch

Extended reading notes

Core claim

The discounted occupancy ratio is characterized as the unique fixed point of the adjoint Bellman operator. FORE replaces each exact adjoint Bellman image with its KL projection onto a normalized exponential-family log-ratio class. Because the unprojected adjoint map contracts relative entropy by the discount factor and the projection is taken in the same KL geometry, the population projected recursion satisfies a contraction up to a KL approximation error of the true ratio. The projection can be computed from one-step transition data through a single-level convex loss. This removes the need for any completeness assumption: the error is governed by how well the class approximates the fixed po

Load-bearing premise

The guarantee collapses if the target occupancy ratio is zero on a positive-mass offline region or the one-step target transitions are not covered by the data; then FORE no longer estimates the full target-policy value, only the coverage-stopped one.

Editorial extensions

If this is right

  • Whenever the discounted occupancy ratio is realizable by the log-ratio class, FORE converges to the true ratio at a geometric rate without value-, adjoint-, or critic-completeness assumptions.
  • Each FORE iteration is a single-level convex (or stochastically optimized) density-ratio objective; no separate critic class has to be tuned.
  • The fitted ratio plugs into reward reweighting, doubly robust estimation, and occupancy-weighted FQE, with value-error bounds that factor the ratio error against a Bellman residual or value-approximation error rather than a completeness condition.
  • In datasets with partial coverage, coverage-stopped FORE estimates the occupancy accumulated before the first uncovered state-action pair, yielding a conservative lower bound on target value for nonnegative rewards and a diagnostic occupancy mass.
  • The approximation term in the value bound scales like the square root of the KL approximation error divided by one minus the discount, while the statistical term carries the usual horizon factor, so long-horizon behavior remains controlled.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The theorem's convexity assumption is a real restriction: the proven guarantees cover closed convex log-ratio classes, so the advertised neural-network and gradient-boosted implementations are an extrapolation until nonconvex analogues of the projection inequality are established.
  • The method's reverse-KL projection requires the true ratio to be strictly positive on the offline support; near-deterministic target policies that put zero occupancy on some offline region fall outside the full-ratio guarantees, and only the coverage-stopped estimand is recovered.
  • A natural stress test is to find an MDP where ratio realizability holds but every standard critic-based estimator fails; if FORE remains stable there, it isolates ratio realizability as the essential ingredient rather than a side effect of minimax balance.
  • The same KL-contraction mechanism may extend to stationary-distribution correction and undiscounted settings under a one-step strong data-processing inequality, as the appendix sketches; verifying this in nonconvex classes would broaden the result.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces FORE, a fitted iterative method for estimating the discounted occupancy ratio ωπ,γ in offline policy evaluation. At each iteration, FORE replaces the exact adjoint Bellman image Bπγω by its KL projection onto a normalized exponential-family ratio class W = {ωh : h ∈ H}, using a single-level loss that depends only on initial-state moments and one-step target-policy transitions. The main theoretical contribution is a population contraction result (Theorem 4.1) showing that, when H is convex, closed, and bounded (A2, A4), the projected recursion contracts in KL divergence up to an approximation error εKL = inf_{v∈W} Dν(v∥ωπ,γ). This removes the need for Bellman completeness, adjoint Bellman completeness, or critic dual-completeness: realizability of the ratio itself suffices. The finite-sample result (Theorem 4.2) provides a high-probability bound on the generalized KL error of the empirical recursion, decomposing into an exponentially decaying iteration term, an εKL term, and a local-Rademacher statistical term. The paper also gives a doubly robust value estimator (Theorem 5.2), an occupancy-weighted FQE procedure (Theorem 5.3), and a coverage-stopped variant that targets the subprobability occupancy accumulated before the first uncovered state–action pair (Section 6). Numerical experiments on a Baird-style MRP and a linear-Gaussian MDP illustrate that FORE is stable where linear FQE diverges.

Significance. If the central claim holds, the paper makes an important contribution to offline policy evaluation: it identifies discounted occupancy-ratio realizability as a sufficient condition for fitted ratio evaluation without any completeness-type closure assumptions. The key insight — that the adjoint Bellman operator is a KL contraction and that KL projection aligns with this contraction — is elegant and, as far as the core derivations go, correctly worked out. The finite-sample analysis appears technically sound, and the doubly robust and coverage-stopped extensions are valuable in themselves. I verified several central pieces: Lemma 3.1 (joint convexity of KL plus data processing), the moment identity (3), the realizable-case Pythagorean argument behind Theorem 4.1, and the doubly robust identity in Theorem 5.2. The paper is also careful to state its assumptions explicitly and to acknowledge the positivity/coverage limitations. However, as detailed below, the advertised implementation scope (neural networks, gradient-boosted trees) is not covered by the proven guarantees, which require convexity of the log-ratio class. This is a substantial scoping gap that the authors should address, e

major comments (2)
  1. [§3.3, §4.1, Assumption A2, Lemma B.2] The paper advertises neural networks and gradient-boosted trees as implementation classes (Intro, §3.3, and §6.3), and §3.3 explicitly provides batched stochastic gradients for nonlinear classes. However, the central guarantees — Theorem 4.1, Theorem 4.2, and Theorem 6.3 — require Assumption A2 that the log-ratio class H be convex, closed, and totally bounded. This convexity is not incidental: Lemma B.2 (the Pythagorean inequality for KL projections onto the normalized exponential family) and the star-shapedness of H∆ and G× used in Lemma C.2 both rely on convexity of H. For a nonconvex two-point class W={v0,v1}, one can choose an adjoint Bellman image u that is closer in reverse KL to v1 than to v0 while v0 is the true ratio; then the KL projection maps u to v1 and Dν(Π_W u∥v0) can exceed Dν(u∥v0), breaking the one-step contraction that underlies Theorem 4.1. Thus, as stated, the theore
  2. [Theorem 4.2 / Algorithm 1] Theorem 4.2 analyzes the exact-ERM version of Algorithm 1, requiring ĥ_{k+1} ∈ arg min_{h∈H} L̂(h;ω̂(k)). For the nonlinear classes advertised in §3.3, exact ERM is computationally intractable, and the batched stochastic gradient variant described in §3.3 is not analyzed: the theorem's uniform statistical bounds are for the exact minimizer, not for one or a few gradient steps. The numerical experiments use log-linear classes, so they are consistent with the theorem. But the practical algorithm proposed for nonlinear classes lacks the advertised finite-sample guarantee. This reinforces the scoping gap in the first major comment.
minor comments (4)
  1. [Abstract / Introduction] The abstract states “each iteration can be implemented with standard supervised learners, such as gradient-boosted trees or neural networks.” Given Assumption A2, this sentence is misleading. Please qualify it to “convex supervised learners” or move the nonlinear potential to a heuristic remark.
  2. [§4.1, Theorem 4.1] The proof sketch for the realizable case is clear. However, the non-realizable case uses Lemma B.3, whose proof relies on Lemma B.2 (convex projection inequality). The dependence of the constant C_app = e^{4R} on A4 is fine, but the convexity dependence should be stated explicitly in the theorem statement, not only in A2.
  3. [§6.3, Algorithm 2] The sentence “In practice, one may use flexible classifiers, such as neural networks or gradient-boosted trees” in §6.3 is again outside the theorem scope: the ratio class H must still be convex (A2) for Theorem 6.3. The classifier class C need not be convex, but the ratio class does.
  4. [References] The reference list is unusually heavy on the authors' own work (van der Laan, Kallus, et al.) and includes several items dated 2025/2026 with IDs that may not be final. This makes it hard for a reader to separate established results from new claims; consider trimming self-citations and using stable versions where possible.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: the FORE convergence theorem is derived from explicitly stated realizability and contraction assumptions, with proofs contained in the appendix; self-citations are contextual and not load-bearing.

full rationale

The derivation chain is self-contained. Theorem 4.1 is proved in the paper from Lemma 3.1 (joint convexity of KL and the data-processing inequality) and Lemma B.2 (the Pythagorean-type projection inequality for normalized exponential families under convexity A2), both of which are standard and are proved in the appendix rather than imported from the authors' prior work. The approximation error epsilon_KL = inf_{v in W} D_nu(v || omega_{pi,gamma}) is an explicit oracle term: if the class realizes the target ratio, epsilon_KL = 0 and the bound reduces to geometric contraction; this is a stated hypothesis, not a fitted parameter later relabeled as a prediction. Theorem 4.2's finite-sample bound is derived in Appendix C using local Rademacher complexity, Bousquet's inequality, and Bernstein concentration; the constants depend only on structural quantities (R, K0, K+, A, alpha) and the critical radius r_{n,fit} is defined from the function classes, not fitted to the data. No fitted value is renamed as a prediction, and no 'uniqueness theorem' is invoked as an external fact to force a choice. Self-citations (van der Laan and Kallus 2025a,b; van der Laan 2026; etc.) appear for context, related-work framing, and proof technique, but the load-bearing fixed-point and projection arguments are reproduced in the manuscript rather than justified only by citation. The Limitations section explicitly flags scope: full-ratio recovery requires positivity and coverage, and the coverage-stopped extension 'changes the estimand' to V_{pi,cov}(r). This is an honest scoping limitation, not circularity. The advertised neural-network and gradient-boosted-tree implementations are outside the convex-class condition A2, but that is a correctness/scoping gap, not a circular derivation. Overall, the central claim is logically derived from explicit assumptions and does not reduce to its own inputs.

Assumptions & free parameters 3 free parameters · 9 assumptions · 1 invented entities

The central claim rests on: (i) coverage (A1) plus positivity and margin of the true ratio (A3, A6); (ii) exponential-family geometry with convex H (A2, A4) for the Pythagorean step that removes completeness; (iii) subexponential tails (A5) for empirical-process transfer. No free parameters are fitted inside the proofs; experimental hyperparameters (clipping constants, K, feature dimension 128) are implementation choices. The linear-Gaussian experiment includes the true log-ratio h* in the feature vector for all compared methods, which instantiates the realizability assumption rather than testing approximation-error robustness.

free parameters (3)
  • Clipping schedule constant A (τu,n = 1 ∨ A log(en)) = A > Kcov/2 (chosen by analyst)
    Coverage-stopped theory: the upper clipping level must grow logarithmically; the theory only constrains A relative to the subexponential norm Kcov.
  • Iteration counts K, Kω, KQ = chosen; K ≈ log n / log(2/(1+γ)) kills the iteration term
    User-chosen hyperparameters; the paper gives guidance that moderate K makes the fixed-point term O(1/n).
  • Lower/upper envelopes τℓ, τu in coverage-stopped FORE = 0 < τℓ ≤ 1 ≤ τu < ∞
    Hyperparameters of the clipped recursion; enter the constants (τu/τℓ) in Theorem 6.1.
assumptions (9)
  • domain assumption Discounted MDP with known target policy π and offline samples from ν with one-step successors
    Section 2.1: defines the estimand dπ,γ and the data model (no behavior policy needed).
  • domain assumption A1: one-step target coverage: d0 ≪ ν and ν+π ≪ ν
    Section 2.1; without it the full occupancy ratio is not identified.
  • domain assumption A2: H convex, closed, totally bounded in L2(ν)
    Section 4.1; powers the exponential-family KL projection inequality (Lemma B.2) and compactness.
  • domain assumption A3: ωπ,γ > 0 ν-a.e. with finite entropy terms
    Section 4.1; needed for KL divergences to the target to be finite and for log-ratio positivity.
  • domain assumption A4: uniformly bounded centered log class (range R)
    Section 4.1; supplies curvature (Lemma C.3), envelope control, and the e^{4R} approximation constant.
  • domain assumption A5: subexponential ψ1 bounds on ω0 and transition densities
    Section 4.2; transfers loss control from ν to d0 and ν+π.
  • domain assumption A6 + B1: polynomial lower-tail margin; hard margin for applications
    Sections 4.2 and 5.1; converts KL-type ratio error into rates and chi-square bounds.
  • domain assumption C1–C4 for coverage-stopped FORE (subexponential ωcov, lower-tail margin, threshold margin, compactness)
    Section 6.4; controls classifier estimation and clipping.
  • standard math KL facts: joint convexity, data processing inequality, Pythagorean inequality for information projections onto exponential families
    Lemma 3.1 and Lemma B.2 (Csiszár 1975; Banerjee et al. 2005).
invented entities (1)
  • Coverage-stopped discounted occupancy ratio ωcov (and clipped fixed point ωτu) independent evidence
    purpose: Estimand for settings where full coverage fails; yields conservative lower bounds on target value and a coverage diagnostic 1 − E[γ^T]
    Not an unexplained construct: Lemma E.8 proves ωcovν equals the discounted occupancy accumulated before the first uncovered state–action pair, so it is directly simulable and falsifiable against data-generating processes; the paper verifies it in Section 7.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fitted Occupancy-Ratio Evaluation without Bellman Completeness." pith.science (2026). https://pith.science/paper/O4CKTELL

@misc{pith2026260705375,
  author       = {Pith},
  title        = {Pith review of: Fitted Occupancy-Ratio Evaluation without Bellman Completeness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O4CKTELL}},
  note         = {Machine review of arXiv:2607.05375}
}
read the original abstract

Occupancy ratios correct distribution shift in offline reinforcement learning and are central to off-policy evaluation. Existing primal-dual and minimax methods typically estimate these ratios by enforcing occupancy-balance moments over a critic class. We propose fitted occupancy-ratio evaluation (FORE), a fitted fixed-point method that characterizes the discounted occupancy ratio through an adjoint Bellman recursion. At each iteration, FORE solves a single-level density-ratio objective on one-step-transition data, thereby projecting the adjoint Bellman image onto a log-ratio class in Kullback-Leibler (KL) divergence. Unlike analyses of fitted Q-evaluation, which typically require value-function realizability together with Bellman completeness or projected-operator stability, our central approximation condition is just realizability of the discounted occupancy ratio itself. Under this condition, the population KL-projected recursion contracts in relative entropy toward the true ratio by virtue of the adjoint Bellman operator being a KL-contraction. For the empirical recursion, we establish finite-sample regret bounds that yield convergence in KL up to approximation error and a statistical error governed by the complexity of the ratio hypothesis class. When full coverage fails, we introduce coverage-stopped FORE, which targets the discounted occupancy accumulated before the first uncovered state-action pair and yields a conservative lower bound on target-policy value for nonnegative rewards. The fitted ratio supports direct value estimation by reward reweighting, occupancy-weighted fitted Q-evaluation, and doubly robust estimation that combines the fitted ratio with a fitted Q-function. Together, these results identify discounted occupancy-ratio realizability as a sufficient condition for offline policy evaluation without any completeness assumptions.

Figures

Figures reproduced from arXiv: 2607.05375 by the authors.

Figure 1
Figure 1. illustrates the population recursions. The population FORE KL recursion converges to the true ratio. In contrast, under the offline data distribution, the projected linear FQE recursion has scalar multiplier 2.103, so coefficient errors are amplified across iterations. Using the FORE ratio as the FQE projection weight changes this multiplier to 0.801. Tabular FQE is included as a Bellman-complete benchmark, for whic… view at source ↗
Figure 1
Figure 1. illustrates the population recursions. The population FORE KL recursion converges to the true ratio. In contrast, under the offline data distribution, the projected linear FQE recursion has scalar multiplier 2.103, so coefficient errors are amplified across iterations. Using the FORE ratio as the FQE projection weight changes this multiplier to 0.801. Tabular FQE is included as a Bellman-complete benchmark, for whic… view at source ↗
Figure 2
Figure 2. Linear-Gaussian population recursions. The left panel shows policy-value error for linear FQE, direct FORE, and FORE-reweighted FQE. The right panel shows FORE occupancy error. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figures from the paper (8 more)
Figure 2
Figure 2. Figure 2: Linear-Gaussian population recursions. The left panel shows policy-value error for linear FQE, direct FORE, and FORE-reweighted FQE. The right panel shows FORE occupancy error. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_2.png]
Figure 3
Figure 3. Figure 3: Linear-Gaussian finite-sample value error. Curves report value RMSE over 300 repetitions; vertical bars indicate Monte Carlo uncertainty. 500 1000 2000 5000 10000 Offline transitions 0.3 0.4 0.5 0.75 1 1.25 1.5 2 6 × 10 Value RMSE 1 Direct value error FORI RFF-RBF MWL …
Figure 3
Figure 3. Figure 3: Linear-Gaussian finite-sample value error. Curves report value RMSE over 300 repetitions; vertical bars indicate Monte Carlo uncertainty. 500 1000 2000 5000 10000 Offline transitions 0.3 0.4 0.5 0.75 1 1.25 1.5 2 6 × 10 Value RMSE 1 Direct value error FORI RFF-RBF MWL …
Figure 4
Figure 4. Figure 4: Direct ratio estimators in the linear-Gaussian experiment. The left panel reports value RMSE from direct reward reweighting; the right panel reports empirical L 2 (ν) error of the fitted density ratio [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 4
Figure 4. Figure 4: Direct ratio estimators in the linear-Gaussian experiment. The left panel reports value RMSE from direct reward reweighting; the right panel reports empirical L 2 (ν) error of the fitted density ratio [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]
Figure 5
Figure 5. Figure 5: Linear-Gaussian value error as the discount varies. Curves report value RMSE over 500 repetitions at n = 5000, with the horizontal axis scaled by the effective horizon (1 − γ) −1 . Linear FQE is evaluated after the same fixed number of fitted updates at every discount.…
Figure 5
Figure 5. Figure 5: Linear-Gaussian value error as the discount varies. Curves report value RMSE over 500 repetitions at n = 5000, with the horizontal axis scaled by the effective horizon (1 − γ) −1 . Linear FQE is evaluated after the same fixed number of fitted updates at every discount.…
Figure 6
Figure 6. Figure 6: Coverage-stopped occupancy under insufficient data coverage. Columns show the L 1 (ν) error of the coverage-stopped occupancy ratio and the mean absolute coverage-stopped value error; rows correspond to the training sample size. At each covered-context fraction, points…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

156 extracted references · 25 linked inside Pith

  1. [1]

    Proceedings of The 33rd International Conference on Machine Learning , pages =

    Doubly Robust Off-policy Value Evaluation for Reinforcement Learning , author =. Proceedings of The 33rd International Conference on Machine Learning , pages =. 2016 , editor =

    2016

  2. [2]

    Proceedings of The 33rd International Conference on Machine Learning , pages =

    Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning , author =. Proceedings of The 33rd International Conference on Machine Learning , pages =. 2016 , editor =

    2016

  3. [3]

    Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling , url =

    Xie, Tengyang and Ma, Yifei and Wang, Yu-Xiang , booktitle =. Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling , url =

  4. [4]

    arXiv preprint arXiv:2602.21501 , year=

    A Researcher's Guide to Empirical Risk Minimization , author=. arXiv preprint arXiv:2602.21501 , year=

    arXiv

  5. [5]

    arXiv preprint arXiv:2607.05375 , year=

    Fitted Occupancy-Ratio Evaluation without Bellman Completeness , author=. arXiv preprint arXiv:2607.05375 , year=

    arXiv

  6. [6]

    International conference on machine learning , pages=

    Information-theoretic considerations in batch reinforcement learning , author=. International conference on machine learning , pages=. 2019 , organization=

    2019

  7. [7]

    Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , pages =

    Asymptotically Efficient Off-Policy Evaluation for Tabular Reinforcement Learning , author =. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , pages =. 2020 , editor =

    2020

  8. [8]

    Journal of Machine Learning Research , year =

    Nathan Kallus and Masatoshi Uehara , title =. Journal of Machine Learning Research , year =

Show all 156 references
  1. [9]

    Proceedings of the 37th International Conference on Machine Learning , pages =

    Double Reinforcement Learning for Efficient and Robust Off-Policy Evaluation , author =. Proceedings of the 37th International Conference on Machine Learning , pages =. 2020 , editor =

    2020

  2. [10]

    Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning , url =

    Kallus, Nathan and Uehara, Masatoshi , booktitle =. Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning , url =

  3. [11]

    Proceedings of the 37th International Conference on Machine Learning , pages =

    Statistically Efficient Off-Policy Policy Gradients , author =. Proceedings of the 37th International Conference on Machine Learning , pages =. 2020 , editor =

    2020

  4. [12]

    Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies , url =

    Kallus, Nathan and Uehara, Masatoshi , booktitle =. Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies , url =

  5. [13]

    Nathan Kallus and Masatoshi Uehara , title =. Oper. Res. , volume =. 2022 , url =. doi:10.1287/OPRE.2021.2249 , timestamp =

    2022

  6. [14]

    Biometrika , volume =

    Kallus, Nathan and Uehara, Masatoshi , title =. Biometrika , volume =. 2024 , doi =

    2024

  7. [15]

    CoRR , volume =

    Masatoshi Uehara and Chengchun Shi and Nathan Kallus , title =. CoRR , volume =. 2022 , url =. doi:10.48550/ARXIV.2212.06355 , eprinttype =. 2212.06355 , timestamp =

    2022 doi arXiv

  8. [16]

    arXiv preprint arXiv:2501.06926 , year=

    Semiparametric double reinforcement learning with applications to long-term causal inference , author=. arXiv preprint arXiv:2501.06926 , year=

  9. [17]

    European conference on machine learning , pages=

    Neural fitted Q iteration--first experiences with a data efficient neural reinforcement learning method , author=. European conference on machine learning , pages=. 2005 , organization=

    2005

  10. [18]

    International Conference on Machine Learning , pages=

    Boosted fitted q-iteration , author=. International Conference on Machine Learning , pages=. 2017 , organization=

    2017

  11. [19]

    Proceedings of the 34th International Conference on Machine Learning , pages =

    Reinforcement Learning with Deep Energy-Based Policies , author =. Proceedings of the 34th International Conference on Machine Learning , pages =. 2017 , editor =

    2017

  12. [20]

    Proceedings of the 35th International Conference on Machine Learning , pages =

    Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , author =. Proceedings of the 35th International Conference on Machine Learning , pages =. 2018 , editor =

    2018

  13. [21]

    A Theory of Regularized

    Geist, Matthieu and Scherrer, Bruno and Pietquin, Olivier , booktitle =. A Theory of Regularized. 2019 , editor =

    2019

  14. [22]

    Inverse Reinforcement Learning Using Just Classification and a Few Regressions , journal =

    Lars van der Laan and Nathan Kallus and Aur. Inverse Reinforcement Learning Using Just Classification and a Few Regressions , journal =. 2025 , url =. doi:10.48550/ARXIV.2509.21172 , eprinttype =. 2509.21172 , timestamp =

    2025 doi arXiv

  15. [23]

    Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models , journal =

    Lars van der Laan and Aur. Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models , journal =. 2025 , url =. doi:10.48550/ARXIV.2512.24407 , eprinttype =. 2512.24407 , timestamp =

    2025 doi

  16. [24]

    Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation , url =

    Liu, Qiang and Li, Lihong and Tang, Ziyang and Zhou, Dengyong , booktitle =. Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation , url =

  17. [25]

    Proceedings of the 34th International Conference on Machine Learning , pages =

    Consistent On-Line Off-Policy Evaluation , author =. Proceedings of the 34th International Conference on Machine Learning , pages =. 2017 , editor =

    2017

  18. [26]

    Sutton and A

    Richard S. Sutton and A. Rupam Mahmood and Martha White , title =. Journal of Machine Learning Research , year =

  19. [27]

    Bellemare , title =

    Carles Gelada and Marc G. Bellemare , title =. The Thirty-Third. 2019 , url =. doi:10.1609/AAAI.V33I01.33013647 , timestamp =

    2019 doi

  20. [28]

    DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections , url =

    Nachum, Ofir and Chow, Yinlam and Dai, Bo and Li, Lihong , booktitle =. DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections , url =

  21. [29]

    8th International Conference on Learning Representations,

    Ruiyi Zhang and Bo Dai and Lihong Li and Dale Schuurmans , title =. 8th International Conference on Learning Representations,. 2020 , url =

    2020

  22. [30]

    2020 , editor =

    Zhang, Shangtong and Liu, Bo and Whiteson, Shimon , booktitle =. 2020 , editor =

    2020

  23. [31]

    Proceedings of the 37th International Conference on Machine Learning , pages =

    Minimax Weight and Q-Function Learning for Off-Policy Evaluation , author =. Proceedings of the 37th International Conference on Machine Learning , pages =. 2020 , editor =

    2020

  24. [32]

    CoRR , volume =

    Masatoshi Uehara and Masaaki Imaizumi and Nan Jiang and Nathan Kallus and Wen Sun and Tengyang Xie , title =. CoRR , volume =. 2021 , url =. 2102.02981 , timestamp =

    2021 arXiv

  25. [33]

    Off-Policy Evaluation via the Regularized Lagrangian , url =

    Yang, Mengjiao and Nachum, Ofir and Dai, Bo and Li, Lihong and Schuurmans, Dale , booktitle =. Off-Policy Evaluation via the Regularized Lagrangian , url =

  26. [34]

    CoinDICE: Off-Policy Confidence Interval Estimation , url =

    Dai, Bo and Nachum, Ofir and Chow, Yinlam and Li, Lihong and Szepesvari, Csaba and Schuurmans, Dale , booktitle =. CoinDICE: Off-Policy Confidence Interval Estimation , url =

  27. [35]

    Rupam , journal=

    Che, Fengdi and Chan, Bryan and Ma, Chen and Mahmood, A. Rupam , journal=

  28. [36]

    CoRR , volume =

    Ofir Nachum and Bo Dai and Ilya Kostrikov and Yinlam Chow and Lihong Li and Dale Schuurmans , title =. CoRR , volume =. 2019 , url =. 1912.02074 , timestamp =

    2019 arXiv

  29. [37]

    8th International Conference on Learning Representations,

    Ilya Kostrikov and Ofir Nachum and Jonathan Tompson , title =. 8th International Conference on Learning Representations,. 2020 , url =

    2020

  30. [38]

    Proceedings of the 38th International Conference on Machine Learning , pages =

    OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation , author =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , editor =

    2021

  31. [39]

    Mankowitz and Nicolas Heess and Doina Precup and Kee

    Jongmin Lee and Cosmin Paduraru and Daniel J. Mankowitz and Nicolas Heess and Doina Precup and Kee. COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation , booktitle =. 2022 , url =

    2022

  32. [40]

    Proceedings of the 39th International Conference on Machine Learning , pages =

    Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching , author =. Proceedings of the 39th International Conference on Machine Learning , pages =. 2022 , editor =

    2022

  33. [41]

    LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimation , journal =

    Geon. LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimation , journal =. 2022 , url =. 2202.13536 , timestamp =

    2022 arXiv

  34. [42]

    Lagoudakis and Ronald Parr , title =

    Michail G. Lagoudakis and Ronald Parr , title =. J. Mach. Learn. Res. , volume =. 2003 , url =

    2003

  35. [43]

    Journal of Machine Learning Research , year =

    Damien Ernst and Pierre Geurts and Louis Wehenkel , title =. Journal of Machine Learning Research , year =

  36. [44]

    arXiv preprint arXiv:2402.08201 , year=

    Off-policy evaluation in markov decision processes under weak distributional overlap , author=. arXiv preprint arXiv:2402.08201 , year=

  37. [45]

    International Statistical Review , volume=

    Stable discovery of interpretable subgroups via calibration in causal studies , author=. International Statistical Review , volume=. 2020 , publisher=

    2020

  38. [46]

    Bernoulli , volume=

    Robust importance sampling with adaptive winsorization , author=. Bernoulli , volume=. 2022 , publisher=

    2022

  39. [47]

    Fitted Q-iteration in continuous action-space MDPs , url =

    Antos, Andr\'. Fitted Q-iteration in continuous action-space MDPs , url =. Advances in Neural Information Processing Systems , editor =

  40. [48]

    2007 , publisher=

    Measure Theory , author=. 2007 , publisher=. doi:10.1007/978-3-540-34514-5 , url=

    2007 doi

  41. [49]

    arXiv preprint arXiv:2302.02252 , year=

    Reinforcement Learning in Low-Rank MDPs with Density Features , author=. arXiv preprint arXiv:2302.02252 , year=

    arXiv

  42. [50]

    Advances in Neural Information Processing Systems , year=

    Occupancy-based Policy Gradient: Estimation, Convergence, and Optimality , author=. Advances in Neural Information Processing Systems , year=

  43. [51]

    arXiv preprint arXiv:2512.24407 , year=

    Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models , author=. arXiv preprint arXiv:2512.24407 , year=

  44. [52]

    Bernoulli , volume=

    Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency , author=. Bernoulli , volume=. 2009 , doi=

    2009

  45. [53]

    Electronic Journal of Probability , keywords =

    Piet Groeneboom , doi =. Electronic Journal of Probability , keywords =. 2011 , Bdsk-Url-1 =

    2011

  46. [54]

    The Annals of Statistics , keywords =

    Hang Deng and Qiyang Han and Cun-Hui Zhang , doi =. The Annals of Statistics , keywords =. 2021 , Bdsk-Url-1 =

    2021

  47. [55]

    Statistics & probability letters , pages=

    An estimator for isotonic regression with boundary consistency , author=. Statistics & probability letters , pages=. 2025 , publisher=

    2025

  48. [56]

    Electronic journal of statistics , volume=

    The bias of isotonic regression , author=. Electronic journal of statistics , volume=

  49. [57]

    arXiv preprint arXiv:2502.05676 , year=

    Generalized venn and venn-abers calibration with applications in conformal prediction , author=. arXiv preprint arXiv:2502.05676 , year=

    arXiv

  50. [58]

    Proceedings of the Eighteenth International Conference on Machine Learning , pages=

    Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers , author=. Proceedings of the Eighteenth International Conference on Machine Learning , pages=. 2001 , publisher=

    2001

  51. [59]

    International conference on machine learning , pages=

    Accurate uncertainties for deep learning using calibrated regression , author=. International conference on machine learning , pages=. 2018 , organization=

    2018

  52. [60]

    The Annals of Statistics , volume=

    Projected state-action balancing weights for offline reinforcement learning , author=. The Annals of Statistics , volume=. 2023 , publisher=

    2023

  53. [61]

    Finite-Time Bounds for Fitted Value Iteration , journal =

    R. Finite-Time Bounds for Fitted Value Iteration , journal =. 2008 , volume =

    2008

  54. [62]

    Proceedings of the 36th International Conference on Machine Learning , pages =

    Batch Policy Learning under Constraints , author =. Proceedings of the 36th International Conference on Machine Learning , pages =. 2019 , editor =

    2019

  55. [63]

    CoRR , volume =

    Lars van der Laan and Nathan Kallus , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2512.23805 , eprinttype =. 2512.23805 , timestamp =

    2025 doi arXiv

  56. [64]

    CoRR , volume =

    Lars van der Laan and Nathan Kallus , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2512.23927 , eprinttype =. 2512.23927 , timestamp =

    2025 doi arXiv

  57. [65]

    Journal of Machine Learning Research , year =

    Andrew Patterson and Adam White and Martha White , title =. Journal of Machine Learning Research , year =

  58. [66]

    Proceedings of the 2008

    Tsuboi, Yuta and Kashima, Hisashi and Hido, Shohei and Bickel, Steffen and Sugiyama, Masashi , title =. Proceedings of the 2008. 2008 , doi =

    2008

  59. [67]

    Correcting Sample Selection Bias by Unlabeled Data , url =

    Huang, Jiayuan and Gretton, Arthur and Borgwardt, Karsten and Sch\". Correcting Sample Selection Bias by Unlabeled Data , url =. Advances in Neural Information Processing Systems , editor =

  60. [68]

    Journal of Machine Learning Research , year =

    Takafumi Kanamori and Shohei Hido and Masashi Sugiyama , title =. Journal of Machine Learning Research , year =

  61. [69]

    Wainwright and Michael I

    XuanLong Nguyen and Martin J. Wainwright and Michael I. Jordan , title =. 2010 , url =. doi:10.1109/TIT.2010.2068870 , timestamp =

    2010

  62. [70]

    Proceedings of The 33rd International Conference on Machine Learning , pages =

    Linking losses for density ratio and class-probability estimation , author =. Proceedings of The 33rd International Conference on Machine Learning , pages =. 2016 , editor =

    2016

  63. [71]

    2012 , isbn =

    Sugiyama, Masashi and Suzuki, Taiji and Kanamori, Takafumi , title =. 2012 , isbn =

    2012

  64. [72]

    Puterman, Martin L. , year=. Markov Decision Processes: Discrete Stochastic Dynamic Programming , ISBN=. doi:10.1002/9780470316887 , journal=

    doi

  65. [73]

    and Tweedie, Richard L

    Meyn, Sean P. and Tweedie, Richard L. , year=. Markov Chains and Stochastic Stability , ISBN=. doi:10.1017/cbo9780511626630 , publisher=

    doi

  66. [74]

    and Meyn, Sean P

    Glynn, Peter W. and Meyn, Sean P. , year=. A Liapounov bound for solutions of the Poisson equation , volume=. The Annals of Probability , publisher=. doi:10.1214/aop/1039639370 , number=

  67. [75]

    CoRR , volume =

    Cameron Voloshin and Hoang Minh Le and Nan Jiang and Yisong Yue , title =. CoRR , volume =. 2019 , url =. 1911.06854 , timestamp =

    2019 arXiv

  68. [76]

    CoRR , volume =

    Justin Fu and Aviral Kumar and Ofir Nachum and George Tucker and Sergey Levine , title =. CoRR , volume =. 2020 , url =. 2004.07219 , timestamp =

    2020 arXiv

  69. [77]

    arXiv preprint arXiv:2510.16127 , year=

    Learning density ratios in causal inference using Bregman-Riesz regression , author=. arXiv preprint arXiv:2510.16127 , year=

    arXiv

  70. [78]

    and Bousquet, Olivier and Mendelson, Shahar , title =

    Bartlett, Peter L. and Bousquet, Olivier and Mendelson, Shahar , title =. The Annals of Statistics , volume =. 2005 , doi =

    2005

  71. [79]

    Comptes Rendus Mathematique , volume =

    A Bennett concentration inequality and its application to suprema of empirical processes , author =. Comptes Rendus Mathematique , volume =. 2002 , publisher =

    2002

  72. [80]

    and Wellner, Jon A

    van der Vaart, Aad W. and Wellner, Jon A. , title =. Electronic Journal of Statistics , volume =

  73. [81]

    Bracketing Metric Entropy Rates and Empirical Central Limit Theorems for Function Classes of

    Nickl, Richard and P. Bracketing Metric Entropy Rates and Empirical Central Limit Theorems for Function Classes of. Journal of Theoretical Probability , volume =

  74. [82]

    Interpolation between

    Molchanova, Anastasia and Roskovec, Tom. Interpolation between. Journal of Mathematical Analysis and Applications , volume =. 2018 , doi =

    2018

  75. [83]

    Leoni, Giovanni , title =

  76. [84]

    , title =

    Wainwright, Martin J. , title =. 2019 , doi =

    2019

  77. [85]

    and Thomas, Joy A

    Cover, Thomas M. and Thomas, Joy A. , title =. 2006 , isbn =

    2006

  78. [86]

    2011 , doi =

    Brezis, Haim , title =. 2011 , doi =

    2011

  79. [87]

    CoRR , volume =

    Raginsky, Maxim , title =. CoRR , volume =. 2014 , url =

    2014

  80. [88]

    Bregman, L. M. , title =. USSR Computational Mathematics and Mathematical Physics , volume =. 1967 , doi =

    1967

  81. [89]

    and Hirayama, Jun-ichiro , title =

    Gutmann, Michael U. and Hirayama, Jun-ichiro , title =. Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence , pages =. 2011 , publisher =

    2011

  82. [90]

    Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , booktitle =

    Sugiyama, Masashi and Nakajima, Shinichi and Kashima, Hisashi and von B. Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , booktitle =. 2007 , publisher =

    2007

  83. [91]

    Annals of the Institute of Statistical Mathematics , volume =

    Sugiyama, Masashi and Suzuki, Taiji and Kanamori, Takafumi , title =. Annals of the Institute of Statistical Mathematics , volume =. 2012 , doi =

    2012

  84. [92]

    Advances in Neural Information Processing Systems , volume=

    Minimax estimation of conditional moment models , author=. Advances in Neural Information Processing Systems , volume=

  85. [93]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , pages=

    Inference on strongly identified functionals of weakly identified functions , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , pages=. 2025 , publisher=

    2025

  86. [94]

    arXiv preprint arXiv:2307.13793 , year=

    Source condition double robust inference on functionals of inverse problems , author=. arXiv preprint arXiv:2307.13793 , year=

    arXiv

  87. [95]

    Proceedings of the 38th International Conference on Machine Learning , pages =

    A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representation , author =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , editor =

    2021

  88. [96]

    Proceedings of the 37th AAAI Conference on Artificial Intelligence , year =

    Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstraction , author =. Proceedings of the 37th AAAI Conference on Artificial Intelligence , year =

  89. [97]

    Biometrika , volume =

    Qin, Jing , title =. Biometrika , volume =. 1998 , doi =

    1998

  90. [98]

    Cheng, K. F. and Chu, C. K. , title =. Bernoulli , volume =. 2004 , doi =

    2004

  91. [99]

    Epidemiology , volume=

    Marginal structural models and causal inference in epidemiology , author=. Epidemiology , volume=. 2000 , publisher=

    2000

  92. [100]

    2018 , volume=

    Espeholt, Lasse and Soyer, Hubert and Munos, Remi and Simonyan, Karen and Mnih, Vlad and Ward, Tom and Doron, Yotam and Firoiu, Vlad and Harley, Tim and Dunning, Iain and Legg, Shane and Kavukcuoglu, Koray , booktitle=. 2018 , volume=

    2018

  93. [101]

    Journal of Computational and Graphical Statistics , volume=

    Truncated importance sampling , author=. Journal of Computational and Graphical Statistics , volume=. 2008 , publisher=

    2008

  94. [102]

    Advances in neural information processing systems , volume=

    Safe and efficient off-policy reinforcement learning , author=. Advances in neural information processing systems , volume=

  95. [103]

    Biometrika , volume=

    Dealing with limited overlap in estimation of average treatment effects , author=. Biometrika , volume=. 2009 , publisher=

    2009

  96. [104]

    Statistical methods in medical research , volume=

    Propensity score weighting under limited overlap and model misspecification , author=. Statistical methods in medical research , volume=. 2020 , publisher=

    2020

  97. [105]

    Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques , pages=

    Calibration of machine learning models , author=. Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques , pages=. 2010 , publisher=

    2010

  98. [106]

    International conference on machine learning , pages=

    On calibration of modern neural networks , author=. International conference on machine learning , pages=. 2017 , organization=

    2017

  99. [107]

    Advances in Large Margin Classifiers , pages=

    Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods , author=. Advances in Large Margin Classifiers , pages=. 1999 , publisher=

    1999

  100. [108]

    International Conference on Machine Learning , pages=

    Doubly robust off-policy evaluation with shrinkage , author=. International Conference on Machine Learning , pages=. 2020 , organization=

    2020

  101. [109]

    arXiv preprint arXiv:2302.08507 , year=

    The scope of multicalibration: Characterizing multicalibration via property elicitation , author=. arXiv preprint arXiv:2302.08507 , year=

    arXiv

  102. [110]

    Calibration of probabilities: The state of the art , author=. Decision Making and Change in Human Affairs: Proceedings of the Fifth Research Conference on Subjective Probability, Utility, and Decision Making, Darmstadt, 1--4 September, 1975 , pages=. 1977 , organization=

    1975

  103. [111]

    2025 , eprint=

    Calibration Strategies for Robust Causal Estimation: Theoretical and Empirical Insights on Propensity Score-Based Estimators , author=. 2025 , eprint=

    2025

  104. [112]

    International Conference on Machine Learning , pages=

    Distribution-free calibration guarantees for histogram binning without sample splitting , author=. International Conference on Machine Learning , pages=. 2021 , organization=

    2021

  105. [113]

    arXiv preprint arXiv:2306.00382 , year=

    Calibrated and Conformal Propensity Scores for Causal Effect Estimation , author=. arXiv preprint arXiv:2306.00382 , year=

    arXiv

  106. [114]

    arXiv preprint arXiv:2211.01221 , year=

    Propensity score models are better when post-calibrated , author=. arXiv preprint arXiv:2211.01221 , year=

    arXiv

  107. [115]

    2024 , eprint=

    Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibration , author=. 2024 , eprint=

    2024

  108. [116]

    PloS one , volume=

    Weight trimming and propensity score weighting , author=. PloS one , volume=. 2011 , publisher=

    2011

  109. [117]

    Journal of Machine Learning Research , volume=

    Truncated emphatic temporal difference methods for prediction and control , author=. Journal of Machine Learning Research , volume=

  110. [118]

    arXiv preprint arXiv:2512.23694 , year=

    Bellman Calibration for V-Learning in Offline Reinforcement Learning , author=. arXiv preprint arXiv:2512.23694 , year=

    arXiv

  111. [119]

    Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining , pages=

    Transforming classifier scores into accurate multiclass probability estimates , author=. Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining , pages=

  112. [120]

    arXiv preprint arXiv:2411.02771 , year=

    Doubly robust inference via calibration , author=. arXiv preprint arXiv:2411.02771 , year=

    arXiv

  113. [121]

    Proceedings of the 22nd international conference on Machine learning , pages=

    Predicting good probabilities with supervised learning , author=. Proceedings of the 22nd international conference on Machine learning , pages=

  114. [122]

    2014 , publisher=

    Nonparametric Estimation under Shape Constraints: Estimators, Algorithms and Asymptotics , author=. 2014 , publisher=. doi:10.1017/CBO9781139020893 , url=

    2014 doi

  115. [123]

    The Annals of Statistics , volume=

    On risk bounds in isotonic and other shape restricted regression problems , author=. The Annals of Statistics , volume=. 2015 , doi=

    2015

  116. [124]

    Calibeating

    “Calibeating”: Beating forecasters at their own game , author=. Theoretical Economics , volume=. 2023 , publisher=

    2023

  117. [125]

    arXiv preprint arXiv:2603.22167 , year=

    Calibeating Made Simple , author=. arXiv preprint arXiv:2603.22167 , year=

  118. [126]

    arXiv preprint arXiv:2604.21260 , year=

    Calibeating prediction-powered inference , author=. arXiv preprint arXiv:2604.21260 , year=

    arXiv

  119. [127]

    Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography , volume=

    Reliability, sufficiency, and the decomposition of proper scores , author=. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography , volume=. 2009 , publisher=

    2009

  120. [128]

    Journal of the Royal Statistical Society: Series D (The Statistician) , volume=

    The comparison and evaluation of forecasters , author=. Journal of the Royal Statistical Society: Series D (The Statistician) , volume=. 1983 , publisher=

    1983

  121. [129]

    Journal of Applied Meteorology and Climatology , volume=

    A new vector partition of the probability score , author=. Journal of Applied Meteorology and Climatology , volume=

  122. [130]

    arXiv preprint arXiv:2406.01933 , year=

    Orthogonal causal calibration , author=. arXiv preprint arXiv:2406.01933 , year=

    arXiv

  123. [131]

    Advances in Neural Information Processing Systems , volume=

    Calibrated structured prediction , author=. Advances in Neural Information Processing Systems , volume=

  124. [132]

    Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining , pages=

    Xgboost: A scalable tree boosting system , author=. Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining , pages=

  125. [133]

    Targeted learning: causal inference for observational and experimental data , pages=

    Cross-validated targeted minimum-loss-based estimation , author=. Targeted learning: causal inference for observational and experimental data , pages=. 2011 , publisher=

    2011

  126. [134]

    The Econometrics Journal , volume=

    Double/debiased machine learning for treatment and structural parameters , author=. The Econometrics Journal , volume=. 2018 , doi=

    2018

  127. [135]

    Proceedings of the Fourth Conference on Causal Learning and Reasoning , volume=

    Stabilized inverse probability weighting via isotonic calibration , author=. Proceedings of the Fourth Conference on Causal Learning and Reasoning , volume=. 2025 , publisher=

    2025

  128. [136]

    International Conference on Machine Learning , pages=

    Causal isotonic calibration for heterogeneous treatment effects , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  129. [137]

    Journal of statistical software , volume=

    Isotone optimization in R: pool-adjacent-violators algorithm (PAVA) and active set methods , author=. Journal of statistical software , volume=

  130. [138]

    SIAM Journal on Optimization , volume=

    Minimizing separable convex functions subject to simple chain constraints , author=. SIAM Journal on Optimization , volume=. 2000 , publisher=

    2000

  131. [139]

    Annals of statistics , volume=

    Batch policy learning in average reward markov decision processes , author=. Annals of statistics , volume=

  132. [140]

    Annals of the Institute of Statistical Mathematics , volume=

    Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals , author=. Annals of the Institute of Statistical Mathematics , volume=. 2022 , publisher=

    2022

  133. [141]

    Journal of the American Statistical Association , volume=

    The isotonic regression problem and its dual , author=. Journal of the American Statistical Association , volume=. 1972 , publisher=

    1972

  134. [142]

    Journal of machine learning research , volume=

    Clustering with Bregman divergences , author=. Journal of machine learning research , volume=

  135. [143]

    The annals of probability , pages=

    I-divergence geometry of probability distributions and minimization problems , author=. The annals of probability , pages=. 1975 , publisher=

    1975

  136. [144]

    IEEE Transactions on Information Theory , volume=

    Strong data processing inequalities and Phi -Sobolev inequalities for discrete channels , author=. IEEE Transactions on Information Theory , volume=. 2016 , publisher=

    2016

  137. [145]

    Proceedings of the 25th International Conference on Machine Learning , pages =

    Bickel, Steffen and Bogojeska, Jasmina and Lengauer, Thomas and Scheffer, Tobias , title =. Proceedings of the 25th International Conference on Machine Learning , pages =. 2008 , publisher =

    2008

  138. [146]

    Proceedings of the Twelfth International Conference on Machine Learning , pages =

    Baird, Leemon , title =. Proceedings of the Twelfth International Conference on Machine Learning , pages =. 1995 , publisher =

    1995

  139. [147]

    arXiv preprint arXiv:2111.10919 , year=

    Offline reinforcement learning: Fundamental barriers for value function approximation , author=. arXiv preprint arXiv:2111.10919 , year=

    arXiv

  140. [148]

    arXiv preprint arXiv:2210.04157 , year=

    The role of coverage in online reinforcement learning , author=. arXiv preprint arXiv:2210.04157 , year=

    arXiv

  141. [149]

    arXiv preprint arXiv:2011.01075 , year=

    A variant of the wang-foster-kakade lower bound for the discounted setting , author=. arXiv preprint arXiv:2011.01075 , year=

    2011 arXiv

  142. [150]

    Advances in Neural Information Processing Systems , volume=

    An exponential lower bound for linearly realizable mdp with constant suboptimality gap , author=. Advances in Neural Information Processing Systems , volume=

  143. [151]

    International Conference on Machine Learning , pages=

    Learning bellman complete representations for offline policy evaluation , author=. International Conference on Machine Learning , pages=. 2022 , organization=

    2022

  144. [152]

    International Conference on Learning Representations , year=

    What are the Statistical Limits of Offline RL with Linear Function Approximation? , author=. International Conference on Learning Representations , year=

  145. [153]

    IEEE Transactions on Automatic Control , volume=

    An Analysis of Temporal-Difference Learning with Function Approximation , author=. IEEE Transactions on Automatic Control , volume=. 1997 , doi=

    1997

  146. [154]

    Mathematics of Operations Research , volume=

    Fast rates for the regret of offline reinforcement learning , author=. Mathematics of Operations Research , volume=. 2025 , publisher=

    2025

  147. [155]

    arXiv preprint arXiv:1312.5602 , year=

    Playing atari with deep reinforcement learning , author=. arXiv preprint arXiv:1312.5602 , year=

    arXiv

  148. [156]

    Weak Convergence and Empirical Processes: With Applications to Statistics , author=

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.