Pith. sign in

REVIEW 3 major objections 5 minor 35 references

Predicting Long Term Sequential Policy Value Using Softer Surrogates

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Under a relaxed "soft surrogacy" assumption, the long-term value of a new sequential policy can be estimated from short-horizon on-policy data combined with long-horizon historical data, with a doubly robust estimator that carries…

desk verdict Solid OPE paper with a real theory contribution and a load-bearing statistical flaw in the significance claims; worth reviewing, but the p-values should not be taken at face value. read the letter →

arxiv 2412.20638 v2 pith:7XAUEHYU submitted 2024-12-30 cs.AI cs.LG

Hyunji Nam , Allen Nie , Ge Gao , Vasilis Syrgkanis , Emma Brunskill This is my paper · ORCID
classification cs.AIcs.LG
keywords off-policypolicyevaluationlong-termvaluesoftsurrogacydoublyrobustestimationfinite-sampleguaranteesnovelactionssequentialdecisionmakingclinicalsimulators
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Off-policy policy evaluation normally breaks when the new policy introduces actions absent from historical data, because importance weights cannot be formed. This paper argues that under a "soft surrogacy" condition—conditioned on the short-horizon trajectory, the expected future returns of the new and behavioral policies coincide—the long-term value of the new policy can be estimated from short on-policy data plus full-horizon historical records. It develops a regression-based soft surrogate estimator and a doubly robust variant with finite-sample error bounds, and shows in HIV and sepsis simulators that accurate value estimates (and p<$10^{{-6}}$ significance tests against the behavioral policy) are achievable after observing only 10% of the full horizon.

What carries the argument

The object that carries the argument is the soft surrogacy identity (Equation 1), which equates the expected future returns of the target policy $\pi_e$ and the behavioral policy $\pi_b$ given the same short-horizon trajectory $\tau_{0:h}$. This identity converts the long-term policy value into a linear functional of the historical regression $f(\tau_{0:h}) := E_{\pi_b}[G \mid \tau_{0:h}]$, averaged over short on-policy trajectories. On top of this, the paper builds a doubly robust estimator with K-fold cross-fitting that adds a density-ratio-weighted residual term; its theoretical engine is the representation of the estimand as a linear functional of a regression under covariate shift, whose Riesz representer is the density ratio, so the bias factorizes as the product of the two nuisance errors and can be controlled by Bernstein bounds.

What would settle it

Run a simulator with known ground truth in which the short-horizon trajectory distributions of the two policies match but their future returns diverge because of a later novel action, and check whether the doubly robust estimator's bias grows with that divergence even when the regression and density-ratio models are well-specified.

Watch

Extended reading notes

Core claim

The paper's central claim is that evaluating a sequential policy with novel actions becomes tractable once the standard surrogacy assumption is relaxed to a policy-pair "soft surrogacy" condition (Equation 1). This identity makes the target policy's value equal to the expectation, over short trajectories drawn from the new policy, of the regression $f(\tau_{0:h}) = E_{\pi_b}[G \mid \tau_{0:h}]$ learned from behavioral data. The paper proves that a doubly robust, cross-fitted estimator of this quantity has bias equal to the average product of the regression error and the relative density-ratio error across folds, is consistent whenever both nuisance estimates are consistent at any rate, and admits high-probability finite-sample bounds with exponential tails. Empirically, the estimators predict the long-term value of new treatment policies in HIV (H=200, h=10) and sepsis (H=20, h=2) simulators with lower MSE than baselines and significant p-values, even though the soft surrogacy assumption is expected to be violated in these simulators.

Load-bearing premise

The load-bearing premise is that, conditioned on the short-horizon trajectory, the new policy and the historical behavioral policy have identical expected future returns; if that equality fails, the regression target is biased and the error propagates to every estimator.

Editorial extensions

If this is right

  • Clinical trials of novel treatments could be shortened from a full multi-year horizon to a short observation window: in the sepsis simulator, 2 of 20 steps suffice to predict the value of a policy that adds vasopressors, and in HIV, 10 of 200 steps suffice to rank the new policy against the behavioral one.
  • The doubly robust estimator is consistent at any nuisance convergence rate, and unbiased if either the regression or the density ratio is exactly correct, so it inherits the usual double-robustness guarantees in a covariate-shift sequential setting.
  • The finite-sample bound has exponential tails, improving on prior finite-sample DML results that did not cover this covariate-shift regime, and it shows the variance terms from true nuisances dominate when the estimates are consistent.
  • The same machinery transfers, in principle, to education and e-commerce, where short-term activity logs are soft surrogates for end-of-year scores or customer churn, provided the soft surrogacy and coverage assumptions hold.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the theory only requires soft surrogacy for the specific pair $(\pi_b, \pi_e)$, a practical diagnostic for deploying the method would be to compare $f$'s predictions on overlapping trajectories under the two policies; a systematic gap would warn that the assumption is violated.
  • The bound's explicit dependence on $|D_e|$ suggests an experiment-design rule: when soft surrogacy holds approximately, collecting more diverse short-horizon trajectories under the new policy may reduce error faster than extending the observation horizon, as long as coverage is maintained.
  • The fact that estimators perform well in simulators where the assumption is likely violated hints that the assumption is sufficient but not necessary; a sensitivity analysis characterizing the maximum tolerable violation would broaden the method's applicability.
  • A natural validation strategy before trusting the method on a genuinely novel action is to apply it to past policies (whose full-horizon outcomes are already known) and measure the calibration of its h-step predictions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses the problem of estimating the long-term value of a sequential policy when the target policy takes novel actions not covered by the behavioral policy's action space, so standard coverage-based off-policy evaluation cannot be applied. The authors propose combining a short-horizon rollout under the target policy with long-horizon historical data collected under a behavioral policy, using a 'soft surrogacy' assumption (Assumption 2) and a coverage condition (Assumption 3). They define unweighted and weighted regression estimators and a doubly robust estimator (Section 5), prove a finite-sample high-probability bound for the doubly robust estimator (Theorem 4) and a product-form bias decomposition (Theorem 2), and report experiments on a toy problem and two clinical simulators (HIV and sepsis) showing that the estimators achieve low MSE after observing only 10% of the full horizon, along with significance tests claiming p-values below 10^-6 for detecting policy improvement.

Significance. If the claims hold, the paper addresses an important and timely problem: predicting long-term outcomes of policies with novel actions using short-horizon data, which is highly relevant in healthcare and other domains. The theoretical contribution is solid: the doubly robust estimator and its finite-sample, exponential-tail bound are useful extensions of the ADML framework of Chernozhukov et al. (2023), and the paper provides detailed proofs and code. However, the empirical significance claims are weakened by the invalid p-value procedure in Appendix 12.4.1, the absence of variability estimates in the HIV experiments, and the gap between the theoretical assumptions and the experimental settings where those assumptions are acknowledged to be violated. The MSE results and the theoretical machinery remain valuable, but the paper as written overstates the strength of the empirical evidence.

major comments (3)
  1. [Table 2] The reported p-values less than 1e-6 are not statistically valid. The t-tests compare 500 per-trajectory estimates \hat f(\tau_{0:h}) or their doubly robust analogues against observed long-term returns, treating the predictions as i.i.d. observations of the long-term outcome. This is incorrect for three reasons: (i) the predictions are conditional expectations, so their variance is strictly smaller than the variance of actual returns (the term E[Var(G | \tau)] is missing), which deflates standard errors and inflates the t-statistic; (ii) all 500 predictions are produced from a single fitted regression or density-ratio model, so the predictions are correlated across trajectories, violating the t-test's independence assumption; and (iii) the estimation error in \hat f and \hat h is ignored. Consequently, the p-values in Tables 6 and 7, and the claims in the Introduction and Section 7.4 that the estimators 'can reject the null with p-values less than 1e-6', have no valid frequentist interpretation. The MSE results in Table 2 are not invalidated by this issue, but the statistical-significance claim is. The authors should replace this analysis with valid confidence intervals or tests, for example using the asymptotic normality of the cross-fitted DR estimator or a bootstrap that resamples both datasets and refits the nuisance functions, and report those intervals.
  2. [Section 7.4 and Appendix 9.1] The HIV columns in Table 2 report only point estimates of MSE, without any measure of variability across seeds or bootstrap resamples, in contrast to the Sepsis columns which report mean ± std across 5 bootstrap seeds. Given that the on-policy sample size is only 500 and that hyperparameters are selected by cross-validation, the apparent advantage of the proposed estimators over LOPE at h=10 (e.g., 68.15 vs. 69.95 for the soft surrogate estimator) could be within the noise of the experiment. The authors should report the same bootstrap or seed-based variability for the HIV domain as they do for Sepsis, so that the empirical claims are comparable and reproducible.
  3. [Section 7.4 and Appendix 9.1] The theory in Section 6 (Theorems 4–6) establishes consistency and double robustness under Assumption 2, but the authors themselves state in Section 7.4 that Assumption 2 'likely does not hold' in the HIV and sepsis simulators. Appendix 9.1 notes that without Assumption 2 the estimator targets V^{π_e,π_b}, not V^{π_e}, and the experiments compare against the ground-truth V^{π_e}. This creates a load-bearing gap: the empirical demonstrations are not covered by the theoretical results. To support the claim that the method predicts the target policy's long-term value, the authors should either (a) compute V^{π_e,π_b} in the simulators and quantify the bias introduced by the assumption violation, or (b) provide a sensitivity analysis that explicitly varies the degree of violation and shows the estimators remain close to V^{π_e}. Without one of these, the connection between the theoretical guarantees and the main empirical setting is not established.
minor comments (5)
  1. [Section 5.2 and Section 6] The displayed bound in Theorem 4 ends with an incomplete fragment 'or 2K log(4K/δ)/N' that should be removed or completed; as written it is unclear whether the last term is of Bernstein type or a product-error bound.
  2. [Table 9 and Section 12.4.2] The symbol h is used both for the short-horizon length and for the density-ratio estimator (e.g., \hat h^{(k)}(\tau) in Equations (7) and (11)). This is confusing, especially in the proof of Theorem 4 where both meanings appear close together; consider using a different symbol such as a(\tau) for the density ratio.
  3. [Table 8] In Section 12.4.2 the ground-truth value of the sixth target policy is listed as -0.022 in the text, but Table 9 lists -0.002; the discrepancy should be corrected both places.
  4. [Appendix 12.4.1] The heading 'Weighed soft surrogate estimator' in Table 8 contains a typo; it should be 'Weighted soft surrogate estimator'.
  5. [Appendix 12.4.1] The sentence 'We included the columns for h = 20 and h = 2 in the main text' is confusing because the main text does not include the hypothesis-testing tables; please clarify which results appear in the main text and which are in the appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation chain is anchored in an explicit soft-surrogacy assumption, and the doubly robust theory is proved in the paper rather than imported by self-citation.

full rationale

I walked the paper's derivation chain and found no step that reduces, by the paper's own equations, to its own inputs. The identification step (Assumption 2, Eq. 1) states a conditional-expectation equality between the target and behavioral policies; this is an explicit, substantive identifying assumption, not a definitional restatement of the estimand. Under it, E_{pi_e}[f_0(tau_{0:h})] equals V^{pi_e} by the tower rule plus the assumption, which is a standard identification argument. The regression f_soft is fit on historical data to predict long-term returns from short-horizon trajectories and is then averaged over on-policy short-horizon data; this is the proposed estimator, not a fitted parameter renamed as a prediction. The doubly robust estimator is derived as a linear functional of a regression under covariate shift, and the paper reproduces the doubly robust bias decomposition (Theorem 3) and the finite-sample proof (Theorem 4) rather than merely citing it. The citations to Battocchi et al. (2021) and Chernozhukov et al. (2023) include a coauthor of this paper, but the cited results are either restated with proofs in the present appendix or are standard ADML facts, so they are not load-bearing self-citations that force the conclusion. The paper's own discussion acknowledges that Assumption 2 may be violated in the simulators and evaluates against ground-truth policy values, which is an external check. The t-test p-value concern raised in the review is a statistical validity issue about treating predictions as observations, not a circularity of the derivation chain, and therefore does not change the circularity score.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The central identifying assumption is soft surrogacy (Assumption 2); coverage (Assumption 3) and reward-state dependence (Assumption 1) are also imported. The theory adds boundedness and consistency conditions on nuisance estimators. No physically new entities are introduced; the only hand-chosen design quantity is the short horizon h.

free parameters (1)
  • Short-horizon length h = 10, 20, 50 for HIV; 2, 4 for Sepsis
    The empirical claim of accurate prediction at 10% of the full horizon depends on the choice of h. The paper evaluates several h values and notes that selecting h is left for future work, so h is a design choice, not derived.
assumptions (6)
  • domain assumption Assumption 2 (Soft surrogacy): E_{τ_{h:H}~π_e}[Σ_{t=h}^H r_t | τ_{0:h}] = E_{τ_{h:H}~π_b}[Σ_{t=h}^H r_t | τ_{0:h}].
    The central identifying condition. It is untestable from short-horizon data alone and is likely violated in the simulation domains, as the paper notes in Section 7.4.
  • domain assumption Assumption 3 (Coverage): p(τ_{0:h}|π_e) > 0 implies p(τ_{0:h}|π_b) > 0.
    Needed for the density ratio h=p(τ|π_e)/p(τ|π_b) to be finite and for the regression model trained on π_b data to be evaluated on π_e support. In novel-action settings, the support condition may fail even if the trajectory density is similar.
  • domain assumption Assumption 1 (Reward depends only on states or history of states).
    Ensures the trajectory τ0:h contains enough information for the conditional expectation of future rewards; used to define G as a function of states.
  • domain assumption Boundedness: |G|, |f|, |f0| ≤ C1 H and density ratios ≤ C2 almost surely.
    Technical conditions for the Bernstein inequalities in Theorem 4. These are standard but not guaranteed in practice, especially for unbounded rewards.
  • domain assumption The regression model f̂ and density ratio ĥ are asymptotically consistent at any rate.
    Required for Theorem 5 (consistency of the DR estimator). The paper does not provide rate conditions for the ML estimators, only assumes consistency.
  • standard math Standard probability and DML theory (Chernozhukov et al. 2023): Riesz representation, cross-fitting, Bernstein and Cauchy-Schwarz inequalities.
    Invoked in the proofs of Theorems 3 and 4 (Section 9.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Predicting Long Term Sequential Policy Value Using Softer Surrogates." pith.science (2026). https://pith.science/paper/7XAUEHYU

@misc{pith2026241220638,
  author       = {Pith},
  title        = {Pith review of: Predicting Long Term Sequential Policy Value Using Softer Surrogates},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7XAUEHYU}},
  note         = {Machine review of arXiv:2412.20638}
}
read the original abstract

Off-policy policy evaluation (OPE) estimates the outcome of a new policy using historical data collected from a different policy. However, existing OPE methods cannot handle cases when the new policy introduces novel actions. This issue commonly occurs in real-world domains, like healthcare, as new drugs and treatments are continuously developed. Novel actions necessitate on-policy data collection, which can be burdensome and expensive if the outcome of interest takes a substantial amount of time to observe--for example, in multi-year clinical trials. This raises a key question of how to predict the long-term outcome of a policy after only observing its short-term effects? Though in general this problem is intractable, under some surrogacy conditions, the short-term on-policy data can be combined with the long-term historical data to make accurate predictions about the new policy's long-term value. In two simulated healthcare examples--HIV and sepsis management--we show that our estimators can provide accurate predictions about the policy value only after observing 10\% of the full horizon data. We also provide finite sample analysis of our doubly robust estimators.

Figures

Figures reproduced from arXiv: 2412.20638 by the authors.

Figure 1
Figure 1. We are interested in the task of predict [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 32 canonical work pages

  1. [1]

    Jacobs, J., and K

    Abdelshiheed, M., K. Jacobs, J., and K. D’Mello, S. (2024). Aligning tutor discourse supporting rigorous thinking with tutee content mastery for predicting math achievement. InInternational Conference on Artificial Intelligence in Education , pages 150–164. Springer

    work page 2024

  2. [2]

    M., Banks, H

    Adams, B. M., Banks, H. T., Kwon, H.-D., and Tran, H. T. (2004). Dynamic multidrug therapies for HIV: Optimal and STI control approaches. Mathematical Biosciences & Engineering , 1(2):223–241

    work page 2004

  3. [3]

    Athey, S., Chetty, R., Imbens, G., and Kang, H. (2024). Estimating treatment effects using multiple surrogates: The role of the surrogate score and the surrogate index

    work page 2024

  4. [4]

    W., and Kang, H

    Athey, S., Chetty, R., Imbens, G. W., and Kang, H. (2019). The surrogate index: Combining short-term proxies to estimate long-term treatment effects more rapidly and precisely. Working Paper 26463, National Bureau of Economic Research

    work page 2019

  5. [5]

    M., Grover, A., Jin, N., Severson, K

    Attia, P. M., Grover, A., Jin, N., Severson, K. A., Markov, T. M., Liao, Y.-H., Chen, M. H., Cheong, B., Perkins, N., Yang, Z., and et al. (2020). Closed-loop optimization of fast-charging protocols for batteries with machine learning. Nature

    work page 2020

  6. [6]

    Battocchi, K., Dillon, E., Hei, M., Lewis, G., Oprescu, M., and Syrgkanis, V. (2021). Estimating the long-term effects of novel treatments. Advances in Neural Information Processing Systems , 34:2925–2935

    work page 2021

  7. [7]

    Cheng, L., Guo, R., and Liu, H. (2021). Long-term effect estimation with surrogate representation. InProceedings of the 14th ACM International Conference on Web Search and Data Mining , pages 274–282

    work page 2021

  8. [8]

    Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018). Dou- ble/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1– C68

    work page 2018

Show all 35 references
  1. [9]

    K., Singh, R., and Srygkanis, V

    Chernozhukov, V., Newey, M., Newey, W. K., Singh, R., and Srygkanis, V. (2023). Automatic debiased machine learning for covariate shifts. arXiv preprint arXiv:2307.04527

    2023

  2. [10]

    K., and Singh, R

    Chernozhukov, V., Newey, W. K., and Singh, R. (2022). A simple and general debiased machine learning theorem with finite sample guarantees. Dud ´ ık, M., Langford, J., and Li, L. (2011). Doubly robust policy evaluation and learning. In International Conference on Machine Learn...

    2022

  3. [11]

    Ernst, D., Geurts, P., and Wehenkel, L. (2005). Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6:503–556

    2005

  4. [12]

    Ernst, D., Stan, G.-B., Goncalves, J., and Wehenkel, L. (2006). Clinical data based optimal sti strategies for hiv: a reinforcement learning approach. In Proceedings of the 45th IEEE Conference on Decision and Control , pages 667–672

    2006

  5. [13]

    Gao, G., Gao, Q., Yang, X., Ju, S., Pajic, M., and Chi, M. (2024). On trajectory augmentations for off-policy evaluation. In The Twelfth International Conference on Learning Representations

    2024

  6. [14]

    Grover, A., Markov, T., Attia, P., Jin, N., Perkins, N., Cheong, B., Chen, M., Yang, Z., Harris, S., Chueh, W., et al. (2018). Best arm identification in multi-armed bandits with delayed feedback. In International conference on artificial intelligence and statistics , pages 83...

    2018

  7. [15]

    and Mao, X

    Kallus, N. and Mao, X. (2024). On the role of surrogates in the efficient estimation of treatment effects with limited outcome data

    2024

  8. [16]

    Levine, S., Kumar, A., Tucker, G., and Fu, J. (2020). Offline reinforcement learning: Tutorial, review, and perspectives on open problems

    2020

  9. [17]

    J., Greene, J

    Liu, V., Escobar, G. J., Greene, J. D., Soule, J., Whippy, A., Angus, D. C., and Iwashyna, T. J. (2014). Hospital deaths in patients with sepsis from 2 independent cohorts. Jama, 312(1):90–92

    2014

  10. [18]

    Liu, Y., Flet-Berliac, Y., and Brunskill, E. (2022). Offline policy optimization with eligible actions. 23

    2022

  11. [19]

    M., Maystre, L., Lalmas, M., Russo, D., and Ciosek, K

    McDonald, T. M., Maystre, L., Lalmas, M., Russo, D., and Ciosek, K. (2023). Impatient bandits: Optimizing recommendations for the long-term without delay. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 1687–1697

    2023

  12. [20]

    Morgan, M. A. and Rubin, S. C. (1998). Long-term complications of chemotherapy. Primary Care Update for OB/GYNS, 5(2):65–68

    1998

  13. [21]

    Mu, T., Jetten, A., and Brunskill, E. (2020). Towards suggesting actionable interventions for wheel-spinning students

    2020

  14. [22]

    Namkoong, H., Keramati, R., Yadlowsky, S., and Brunskill, E. (2020). Off-policy policy evaluation for sequential decisions under unobserved confounding. Advances in Neural Information Processing Systems, 33:18819–18831

    2020

  15. [23]

    and Sontag, D

    Oberst, M. and Sontag, D. (2019). Counterfactual off-policy evaluation with gumbel-max structural causal models. In International Conference on Machine Learning , pages 4881–4890. PMLR

    2019

  16. [24]

    Prentice, R. L. (1989). Surrogate endpoints in clinical trials: definition and operational criteria. Statistics in medicine, 8(4):431–440

    1989

  17. [25]

    Saito, Y., Abdollahpouri, H., Anderton, J., Carterette, B., and Lalmas, M. (2024). Long-term off-policy evalua- tion and learning. In Proceedings of the ACM on Web Conference 2024 , pages 3432–3443

    2024

  18. [26]

    Z., Harris, S

    Fraggedakis, D., Bazant, M. Z., Harris, S. J., Chueh, W. C., and Braatz, R. D. (2019). Data-driven prediction of battery cycle life before capacity degradation. Nature Energy, 4(5):383–391

    2019

  19. [27]

    Shi, C., Wang, X., Luo, S., Zhu, H., Ye, J., and Song, R. (2023). Dynamic causal effects evaluation in a/b testing with a reinforcement learning framework. Journal of the American Statistical Association, 118(543):2059–2071

    2023

  20. [28]

    Sugiyama, M., Krauledat, M., and M¨ uller, K.-R. (2007). Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research

    2007

  21. [29]

    Sugiyama, M., Suzuki, T., and Kanamori, T. (2011). Density-ratio matching under the bregman divergence: A unified framework of density-ratio estimation - annals of the institute of statistical mathematics

    2011

  22. [30]

    Tang, Z., Duan, Y., Zhang, S., and Li, L. (2022). A reinforcement learning approach to estimating long-term treatment effects. arXiv preprint arXiv:2210.07536

    2022 arXiv

  23. [31]

    R., Lin, J., Gatz, E., Gurung, A., Gupta, S., Norberg, K., Fancsali, S

    Thomas, D. R., Lin, J., Gatz, E., Gurung, A., Gupta, S., Norberg, K., Fancsali, S. E., Aleven, V., Branstetter, L., Brunskill, E., et al. (2024). Improving student learning with hybrid human-ai tutoring: A three-study quasi-experimental investigation. In Proceedings of the 14t...

    2024

  24. [32]

    and Brunskill, E

    Thomas, P. and Brunskill, E. (2016). Data-efficient off-policy policy evaluation for reinforcement learning. In International Conference on Machine Learning , pages 2139–2148. PMLR

    2016

  25. [33]

    Tran, A., Bibaut, A., and Kallus, N. (2024). Inferring the long-term causal effects of long-term treatments from short-term experiments. In Forty-first International Conference on Machine Learning

    2024

  26. [34]

    Zhang, V., Zhao, M., Le, A., and Kallus, N. (2023). Evaluating the surrogate index as a decision-making tool using 200 a/b tests at netflix. arXiv preprint arXiv:2311.11922

    2023 arXiv

  27. [35]

    E., Ritter, S., and Berman, S

    Zheng, G., Fancsali, S. E., Ritter, S., and Berman, S. (2019). Using instruction-embedded formative assessment to predict state summative test scores and achievement levels in mathematics. Journal of Learning Analytics , 6(2):153–174. 24

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.