Estimating Causal Effects from Data Generated by Stochastic Algorithms

Guido Imbens; Susan Athey; Zoe Ji

REVIEW 2 major objections 6 minor 17 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Two extra log fields turn AI randomness into causal evidence

2026-07-08 23:48 UTC pith:OUNETAIE

load-bearing objection New identification result for causal effects from stochastic algorithm outputs; clean theory, no empirical validation, but the core insight is genuinely novel and the proofs hold up. the 2 major comments →

arxiv 2607.05792 v1 pith:OUNETAIE submitted 2026-07-07 stat.ME econ.EM

Estimating Causal Effects from Data Generated by Stochastic Algorithms

Susan Athey , Guido Imbens , Zoe Ji This is my paper

classification stat.ME econ.EM

keywords causal inferencestochastic algorithmscounterfactual exposurestreatment effectsrecommendation systemslarge language modelsidentificationmicro experiments

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proves that the causal effect of content features (like the tone of an LLM response or the price of a recommended product) can be recovered from routine user interactions with stochastic algorithms, without running purpose-built experiments and without observing all confounders, provided that two additional pieces of information are logged per interaction: the identity of at least one item the algorithm could have shown but did not, and an estimate of the relative probability of the shown versus unshown item. The central object is Logged Counterfactual Exposures (LCE) data, and the central mechanism is that the stochastic algorithm's internal randomization over which item to display, conditional on having selected a pair, acts as a valid experiment of size one (a micro experiment) even when the selection of the pair itself is confounded by unobserved user traits. The estimand, called the Conditional Treatment-Averaged Causal Effect (C-TACE), averages unit-level causal contrasts over both users and the pairs of items the algorithm naturally produces. Theorem 1 establishes that this estimand is identified as a simple difference in mean outcomes between two subpopulations: those who received the higher-feature item from their pair and those who received the lower-feature item, among pairs where the feature differs. The paper further shows that when exact assignment probabilities are not logged, they can be estimated by replaying the same stochastic algorithm (e.g., re-querying an LLM at the same prompt and temperature), and an inverse-sampling design yields exactly unbiased weights with modest replay costs. The practical consequence is that platforms already running LLMs or recommendation systems can, at the cost of a second API call or logging a candidate set, generate causal evidence about content features from data they already collect in the ordinary course of business.

Core claim

The paper's core discovery is that the randomization a stochastic algorithm uses to decide which item to display, given a set of candidates, is sufficient to identify causal effects of content features even in the presence of unobserved confounders that affect both which candidates the algorithm considers and how users respond. The identification does not require a model of the assignment mechanism, observation of all content features, or adjustment for user characteristics. It requires only that, for each user, the features of one unexposed alternative and the relative display probability of the pair are recorded. The C-TACE estimand averages over the pairs the algorithm actually produces,禀

What carries the argument

The treatment characterizing function (TCF) h(v, v') is an anti-symmetric function that maps the features of the exposed and unexposed items to a scalar contrast. It defines subpopulations of comparable units (those with h = δ versus h = -δ). The pseudo-experiment construction transforms the observed quadruple (Y_i, V_i, V_i^N, X_i) into data resembling a standard randomized experiment by assigning a three-valued pseudo-treatment based on the sign of the TCF. Under Assumption 1 (randomized within-pair assignment), the pseudo-treatment is independent of potential outcomes, and the C-TACE reduces to a difference in conditional expectations of the observed outcome.

Load-bearing premise

The load-bearing premise is that, once the algorithm has selected its candidate pair for a given user, the final choice of which item to display is a fair coin flip (or a flip with known probability) that is independent of how the user would respond to either item. In production systems, the final ranking step often incorporates additional real-time signals or context that could correlate with user outcomes, which would break this assumption. A second premise, relevant when概率

What would settle it

If the within-pair display decision is correlated with the potential outcomes conditional on the pair—for example, because a final ranking step uses real-time engagement predictions that also drive the outcome—then the two subpopulations (those who received the higher-feature item and those who received the lower-feature item from a discordant pair) are no longer comparable, and the difference in their mean outcomes does not have a causal interpretation. The placebo test in Section 10.8 can detect this: a nonzero coefficient on the non-shown item's features after partialling out the shown item

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Platforms deploying LLMs can estimate the causal effect of interpretable content features (e.g., friendliness, concreteness) on user engagement by making a second API call per interaction to log the unexposed response and its relative probability, converting routine traffic into causal evidence.
The placebo test (Section 10.8) provides a practical diagnostic: if the non-shown item's features predict the outcome after controlling for the shown item's features, the analyst has evidence that selection bias is present and the identification assumptions may be violated.
The inverse-sampling design (Section 9.3) means that even when exact probabilities are unavailable, a platform can replay its algorithm a small number of times per interaction (r between 5 and 10) and obtain unbiased causal estimates with only a few percent variance inflation, making the method feasible for production-scale systems.
The framework extends naturally to comparing LLMs: by randomly forwarding one of two models' responses to each user and logging features of both, the relative quality of models can be assessed from single-response ratings rather than head-to-head rankings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The C-TACE's interpretation as a bundled effect (averaging over correlated features that co-vary with the target feature in the algorithm's output) means it answers a policy question that standard factorial experiments cannot: not 'what is the effect of changing this feature in isolation' but 'what happens if we steer the algorithm toward this feature as it currently operates.' This makes it direc
The framework could be extended to sequential or multi-turn interactions, where the 'content feature' of interest is a property of the conversation trajectory rather than a single response. The micro-experiment logic would still apply if the algorithm's stochastic branching points are logged, though the pair structure would need to generalize to paths.
The reproducibility requirement (Assumption 2) creates a natural tension with the pace of model deployment: as LLM providers deprecate older model versions, the window for valid replays closes. This suggests a practical imperative to log counterfactual exposures at the time of original interaction rather than relying on retroactive replay, even though the paper shows both regimes are valid in prin

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

New identification result for causal effects from stochastic algorithm outputs; clean theory, no empirical validation, but the core insight is genuinely novel and the proofs hold up.

read the letter

The main thing to know: this paper introduces a genuinely new identification strategy for causal effects of content features using data from stochastic algorithms. The core insight is that if you log the identity of one unexposed item alongside the exposed one, plus the relative probability of the pair, you can identify a causal effect even with unobserved confounders driving which items the algorithm considered. The identification (Theorem 1) is clean and I checked the key step: for discordant pairs, within-pair randomization ensures the distribution of pairs is identical in the h=δ and h=-δ groups, so the difference in means identifies the within-pair causal effect. No circularity. The estimand (C-TACE) is a local contrast averaged over the pairs the algorithm actually produces, which the paper is transparent about. The linear example in Section 10 is well done — it makes the estimand concrete, shows what the naive regression and unconfoundedness approaches recover, and demonstrates the placebo test cleanly. The practical discussion of how to obtain LCE data from LLM APIs (token log-probabilities, replays) is timely and concrete. The inverse-sampling design (Section 9.3) using negative binomial stopping is a nice touch — exact unbiasedness for the weight at any r≥1. The related literature is thorough and the positioning relative to off-policy evaluation, negative sampling, and compound treatment literatures is accurate. The distinction from embedding-level propensity methods (Saito-Joachims) is clearly drawn. The soft spots are real but proportionate. First, no empirical evaluation on real data. The linear example is illustrative, not validating. For a paper claiming practical feasibility at scale, this is a gap. Second, the stress-test note raises a point I agree with: in the LLM case, the analyst creates the pair by making two API calls and randomly selecting, so Assumption 1 holds by construction — but then the method is essentially a randomized experiment, and the contribution is the estimand and data structure, not the identification. In recommender systems, within-pair selection may incorporate real-time signals that correlate with outcomes, violating Assumption 1, and the paper provides a diagnostic for Assumption 2 (reproducible randomization) but not for Assumption 1. Third, the C-TACE is local to the discordant subpopulation, which is algorithm-selected and potentially confounded. The paper acknowledges this but the external validity limitation is sharper than the tone suggests. These are the kind of gaps a good referee process should push on, not reasons to dismiss the paper. The identification is correct, the estimand is well-motivated, and the practical relevance to LLMs and recommender systems is clear. This is a theory paper from authors who know what they are doing, and the core contribution — using relative pair probabilities rather than marginal propensities for identification — is new. It deserves a serious referee.

Referee Report

2 major / 6 minor

Summary. This paper proposes a new method for estimating causal effects of content features using observational data generated by stochastic algorithms. The key insight is that when a stochastic algorithm selects content for a user, the randomization within the unit-specific treatment set can be exploited for causal identification, provided two additional data elements are logged: the identity of at least one unexposed item and the relative probability of the exposed versus unexposed item. The authors introduce the Logged Counterfactual Exposures (LCE) data structure and the Conditional Treatment-Averaged Causal Effect (C-TACE) estimand, which averages unit-level treatment effects over both users and the pairs of items the algorithm generates. The paper proves identification (Theorems 1-4) under a randomized assignment assumption (Assumption 1), extends the framework to unequal assignment probabilities and estimated probabilities via replays (Section 9), and provides a linear data-generating example (Section 10) that concretely illustrates the estimand's properties relative to naive regression, unconfoundedness adjustment, and negative-sample approaches.

Significance. The paper makes a substantive contribution to causal inference for stochastic algorithm outputs. The identification results (Theorems 1-4) are clean and the derivations are verifiable. The linear example (Section 10) with replication code in the supplement is a strength, as it makes the subtle interpretation of the C-TACE estimand concrete and demonstrates the properties of the estimator relative to alternatives. The inverse sampling design for estimated assignment probabilities (Section 9.3, Lemma 5, Theorem 4) is an elegant contribution that provides exact unbiasedness of the weight. The practical discussion of how LCE data can be obtained from LLM log-probabilities, replayed mechanisms, and recommendation system candidate sets adds real-world relevance. The core claim—that causal effects of content features are identified even with unobserved confounders in pair selection, given within-pair randomization and the logged relative probability—is well-supported by the proofs.

major comments (2)

Section 9.2, trimming population definition: The trimmed population P(κ) is defined in terms of q_i, the probability that the display mechanism produces the high-feature side. However, q_i is a function of the unit-specific treatment set A_i and potential outcomes, which are not fully observed. The paper does not clearly specify how q_i is to be evaluated or estimated for the purpose of defining the trimmed population in practice. The estimator q̂_i from replays is introduced, but the relationship between trimming on the true q_i (which defines the estimand) and trimming on the estimated q̂_i (which is what would be feasible) is not fully addressed. This is load-bearing for the estimated-probability regime (Theorem 4), where the feasible estimator's properties are established on P(κ). Clarifying whether trimming is on q_i or q̂_i, and what the consequences are for the former case, would
Section 10: The linear example is valuable for illustrating the estimand's properties, but it is purely a simulation/theoretical exercise. The paper lacks any empirical evaluation on real data, even from a small-scale or illustrative setting. Given that the paper's motivation heavily emphasizes practical applicability to LLMs and recommendation systems, and that the authors claim LCE data is 'practically obtainable at scale' (Section 13), at least one demonstration on real or semi-synthetic data would substantially strengthen the paper's contribution and address concerns about feasibility. This is particularly relevant for the replay-based estimation (Section 9), where practical issues like API costs, model version drift, and the accuracy of the diagnostic (Section 9.4) are discussed but not empirically tested.

minor comments (6)

Assumption 1 (Section 3) states that the assignment probability is 1/2 conditional on the full set of variables including potential outcomes. The text later refers to 'random assignment' and 'randomized assignment' in ways that could be read as unconditional. Making the conditional nature of the randomization more explicit in the prose surrounding Assumption 1 would improve clarity.
Section 5, Definition of pseudo potential outcomes: The definitions of V^h_i(-1), V^h_i(1), Y^h_i(-1), Y^h_i(1) involve case distinctions based on the sign of h(V_i(0), V_i(1)). The notation is somewhat dense and the connection to the observed data (Y_i, V_i, V^N_i) could be made more explicit. A brief remark explaining how these pseudo potential outcomes relate to the observed quantities for a unit with D^h_i != 0 would help readers.
Section 7 (Multiple TCFs): The discussion of what can and cannot be learned with multiple treatment-characterizing functions is insightful but somewhat informal. A more formal statement of the identification results (or lack thereof) for the multi-TCF case, perhaps as a theorem or proposition, would improve clarity.
Section 12.1: The expansion of the unconditional difference in means Y_{V_i=1} - Y_{V_i=0} into three terms is algebraically detailed but could benefit from a more concise summary of the key takeaway. The main point—that the causal contrast is contaminated by imbalance and incomparability bias from tied pairs—could be stated more prominently before the derivation.
Typo in Section 10.5: The unconfoundedness regression coefficient is denoted β̂_unc in some places and the text refers to 'the unconfoundedness analysis of Section 12' before Section 12 is formally introduced. Consider forward-referencing more carefully.
Figure captions: Figures 1-6 are referenced in the text but the captions themselves are somewhat terse. Adding more detail to the figure captions, particularly regarding what the axes and different lines represent, would improve readability for readers who skim the figures.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for a careful and constructive report. The referee raises two major comments: (1) clarification of the trimmed population definition in Section 9.2, specifically whether trimming is on the true q_i or the estimated q̂_i, and (2) the absence of an empirical evaluation on real data. We agree that both points warrant attention and will revise the manuscript accordingly.

read point-by-point responses

Referee: Section 9.2, trimming population definition: The trimmed population P(κ) is defined in terms of q_i, the probability that the display mechanism produces the high-feature side. However, q_i is a function of the unit-specific treatment set A_i and potential outcomes, which are not fully observed. The paper does not clearly specify how q_i is to be evaluated or estimated for the purpose of defining the trimmed population in practice. The estimator q̂_i from replays is introduced, but the relationship between trimming on the true q_i (which defines the estimand) and trimming on the estimated q̂_i (which is what would be feasible) is not fully addressed. This is load-bearing for the estimated-probability regime (Theorem 4), where the feasible estimator's properties are established on P(κ). Clarifying whether trimming is on q_i or q̂_i, and what the consequences are for the former case, would.

Authors: The referee correctly identifies an ambiguity in Section 9.2 that we will resolve in the revision. To clarify the intended logic: the estimand is defined on the population P(κ) = {i : κ ≤ q_i ≤ 1−κ}, where q_i is the true probability that the display mechanism produces the high-feature side for unit i's configuration. This is the population-level quantity that defines the target of inference. In practice, q_i is not directly observed, but it can be estimated via replays (Regime III) or from logged counterfactual exposures (Regime II). The feasible estimator trims on the estimated q̂_i rather than the true q_i, and the referee is right that the manuscript does not currently make this distinction explicit or discuss the consequences. We will revise Section 9.2 to address this in three ways. First, we will state clearly that the estimand is defined by trimming on the true q_i, while the feasible estimator trims on q̂_i. Second, we will note that when q̂_i is obtained from a fixed number M of replays, misclassification of the trimming boundary can occur for units whose true q_i is near κ or 1−κ. This introduces a boundary effect: the feasible trimmed population differs from the target population by units whose q_i is within the estimation error of the threshold. The resulting estimand is defined on a slightly different population, but the difference vanishes as M grows (or as r grows in the sequential design of Section 9.3), because q̂_i converges to q_i. Third, and most importantly, we will clarify that the inverse sampling design of Section 9.3 sidesteps this issue entirely: under the sequential design, the weight T_i/r is exactly unbiased for 1/P_i at every r ≥ 1 (Lemma 5), and Theorem 4 establishes that the feasible estimator identifies the same estimand on P(κ) without revision: no
Referee: Section 10: The linear example is valuable for illustrating the estimand's properties, but it is purely a simulation/theoretical exercise. The paper lacks any empirical evaluation on real data, even from a small-scale or illustrative setting. Given that the paper's motivation heavily emphasizes practical applicability to LLMs and recommendation systems, and that the authors claim LCE data is 'practically obtainable at scale' (Section 13), at least one demonstration on real or semi-synthetic data would substantially strengthen the paper's contribution and address concerns about feasibility. This is particularly relevant for the replay-based estimation (Section 9), where practical issues like API costs, model version drift, and the accuracy of the diagnostic (Section 9.4) are discussed but not empirically tested.

Authors: We agree with the referee that an empirical demonstration would strengthen the paper and address practical feasibility concerns. We will add a real-data illustration in the revision. Specifically, we plan to implement the LCE framework using an open-weight LLM (e.g., Llama or Mistral) in a setting where we generate pitches for a lending scenario similar to the running example in the paper. Because we control the generation pipeline, we can log the exposed response, sample counterfactual responses via replays at the same prompt and temperature, extract features (e.g., concreteness, tone), and compute relative probabilities from token-level log-probabilities. This setup will allow us to demonstrate the full pipeline: constructing LCE data, estimating the C-TACE, implementing the replay-based estimation of Section 9.3, and applying the diagnostic of Section 9.4. We will report API/compute costs, the number of replays used, and the behavior of the diagnostic, directly addressing the practical concerns the referee raises. We note that a fully observational dataset from a production system is not available to us, which is why we focus on a setting where we control the stochastic algorithm. However, we believe this semi-synthetic demonstration, where the data-generating mechanism is a real LLM rather than a parametric simulation, will substantively address the feasibility question. We will also add discussion of model version drift: since we use an open-weight model hosted locally, the algorithm remains fixed across replays, which is the ideal case for Assumption 2. We will discuss how the diagnostic of Section 9.4 can detect violations when this condition is not met, as would be the case with a hosted API whose model version may change. We believe this addition will address revision: no

Circularity Check

0 steps flagged

No significant circularity found; identification is self-contained

full rationale

The paper's central identification result (Theorem 1) follows directly from Assumption 1 (randomized within-pair assignment) and the definition of the pseudo-treatment D_i^h. The estimand C-TACE is defined in terms of observable quantities (Y_i, V_i, V_i^N) and does not reduce to a fitted parameter or a self-citation chain. The proof logic is clean: for discordant pairs (h ≠ 0), Assumption 1 ensures P(h=δ|pair) = P(h=-δ|pair) = 1/2, so the distribution of pairs—and hence potential outcomes—is identical in both groups regardless of confounding in pair selection. This makes E[Y_i|h=δ] - E[Y_i|h=-δ] a valid causal contrast. The self-citations (Athey et al. 2018, 2025) are used for context on pseudo-experiments and surrogacy, not as load-bearing axioms. The 2018 citation is invoked conceptually ('echoes the discussion in Athey et al. (2018)') regarding the pseudo-experiment idea, and the 2025 citation is an optional surrogacy assumption mentioned in passing in Section 12.2. Neither is used to forbid alternatives or to smuggle in an ansatz. The estimated-probability regime (Section 9) uses Assumption 2 (Reproducible Randomization) and inverse binomial sampling (Lemma 5, citing Haldane 1945 and Girshick et al. 1946—external sources), which is a standard statistical result, not a self-citation. The linear example (Section 10) is self-contained with replication code. No step in the derivation chain reduces to its inputs by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 3 invented entities

The axiom ledger is minimal. The core result relies on Assumption 1 (randomized assignment within the pair), which is a domain assumption about the data-generating process. The free parameters (κ, r, h) are design choices for the analyst, not fitted constants. No new physical entities or forces are invented; the 'invented entities' are formal statistical constructs (LCE, C-TACE) with stated identification conditions.

free parameters (3)

Trimming parameter κ
Chosen by the analyst to exclude units with extreme assignment probabilities (stochastic ties) in Section 9.2.
Stopping parameter r
Integer chosen for the inverse sampling design in Section 9.3, controlling the variance of the replay-based weight estimator.
Treatment characterizing function h(v, v')
A substantive choice by the researcher defining the direction of the treatment contrast (Section 4).

axioms (3)

domain assumption Assumption 1 (Randomized Assignment): pr(W_i = 1 | A_i(0), A_i(1), Y_i(A_i(0)), Y_i(A_i(1)), X_i) = 1/2.
Invoked in Section 3. The load-bearing premise that the final display is randomized within the unit-specific treatment set.
domain assumption Assumption 2 (Reproducible Randomization): Conditional on the configuration, the feature indicators of the realized and counterfactual exposures are independent Bernoulli draws with common success probability q_i.
Invoked in Section 9.1. Required for the replay-based estimation of assignment probabilities.
standard math Stable Unit Treatment Value Assumption (SUTVA) is relaxed.
Section 3 explicitly allows for versions of the treatment, relaxing the no-multiple-versions condition from Rubin (1980).

invented entities (3)

Logged Counterfactual Exposures (LCE) independent evidence
purpose: A data structure augmenting standard interaction logs with unexposed items and their relative probabilities.
The paper argues LCE data is practically obtainable from LLM log-probabilities and recommender candidate sets.
Conditional Treatment-Averaged Causal Effect (C-TACE) independent evidence
purpose: An estimand averaging unit-level causal effects over both users and pairs of items.
The estimand is defined non-parametrically and its identification is proven under stated assumptions.
Pseudo experiment independent evidence
purpose: A conceptual transformation of the observed quadruple into data resembling a standard randomized experiment.
Constructed via the treatment characterizing function in Section 5.

pith-pipeline@v1.1.0-glm · 35308 in / 2768 out tokens · 550584 ms · 2026-07-08T23:48:38.759468+00:00 · methodology

0 comments

read the original abstract

Recommendation systems and chatbots present content to users, typically using stochastic algorithms that select the content based on user characteristics or context. Examples of content include chat responses, videos, or items available for purchase. Scientists and application developers are often interested in whether characteristics of content increase outcomes such as user engagement. Estimates of such causal effects may guide content providers to generate content that emphasize desirable features. However, in settings with a large content library or where content is generated uniquely for a given user, it can be difficult to use observational data to learn the causal effect of content features, because the content a user sees is tailored to that user, and because content varies in many dimensions. This paper proposes a new method for estimating the impact of content features using observational data, when the algorithm that determines user exposure incorporates some randomization, and when two additional data elements are logged for each user: $(i)$ the identity of at least one item that could have been exposed to the user, but was not (the unexposed item); $(ii)$ an estimate of the ratio of the probability that the unexposed item would have been shown to the probability that the exposed item was shown. We show that causal effects of features are identified in this setting, even in the presence of unobserved confounders that affect both user preferences and the identity of the considered pair of items (exposed and unexposed). Our estimator differs from prior approaches in terms of what data is used and how the estimator is constructed.

discussion (0)

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages · 5 internal anchors

[1]

Policy evaluation with latent confounders via optimal balance

Andrew Bennett and Nathan Kallus. Policy evaluation with latent confounders via optimal balance. InAdvances in Neural Information Processing Systems, volume 32, 2019 . Léon Bottou, Jonas Peters, Joaquin Quiñonero-Candela, Denis X. Charles, D. Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Simard, and Ed Snelson. Counterfactual reasoning and learnin...

work page 2019
[2]

Semi-parametric effi- cient policy learning with continuous actions

Victor Chernozhukov, Mert Demirer, Greg Lewis, and Vasilis Syrgkanis. Semi-parametric effi- cient policy learning with continuous actions. InAdvances in Neural Information Processing Systems, volume 32, pages 15039–15049 , 2019 . Tirthankar Dasgupta, Natesh S Pillai, and Donald B Rubin. Causal inference from 2k factorial designs by using potential outcome...

work page 2019
[3]

Estimation Considerations in Contextual Bandits

Maria Dimakopoulou, Zhengyuan Zhou, Susan Athey, and Guido Imbens. Estimation considera- tions in contextual bandits. Technical report, arXiv preprint arXiv:1711.07077 ,

work page internal anchor Pith review Pith/arXiv arXiv
[4]

Balanced linear contextual bandits

Maria Dimakopoulou, Zhengyuan Zhou, Susan Athey, and Guido Imbens. Balanced linear contextual bandits. InProceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3445–3453, 2019 . Jingtao Ding, Guanghui Yu, Xiangnan He, Yuhan Quan, Yong Li, Tat-Seng Chua, Depeng Jin, and Jiajie Yu. Improving implicit recommender systems with view da...

work page 2019
[5]

Reinforced negative sam- pling for recommendation with exposure data

Jingtao Ding, Yuhan Quan, Xiangnan He, Yong Li, and Depeng Jin. Reinforced negative sam- pling for recommendation with exposure data. InProceedings of the 28th International Joint Conference on Artificial Intelligence, pages 2230–2236, 2019 . Miroslav Dudík, John Langford, and Lihong Li. Doubly robust policy evaluation and learning. InProceedings of the 2...

work page 2019
[6]

Top challenges from the first practical online controlled experiments summit.ACM SIGKDD Explorations Newsletter, 21(1):20–35, 2019

Somit Gupta, Ronny Kohavi, Diane Tang, Ya Xu, Reid Andersen, Eytan Bakshy, Niall Cardin, Sumita Chandran, Nanyu Chen, Dominic Coey, et al. Top challenges from the first practical online controlled experiments summit.ACM SIGKDD Explorations Newsletter, 21(1):20–35, 2019 . Vitor Hadad, David A. Hirshberg, Ruohan Zhan, Stefan Wager, and Susan Athey. Confiden...

work page 2019
[7]

Ruining He and Julian McAuley

PMLR, 2017 . Ruining He and Julian McAuley. Vbpr: visual bayesian personalized ranking from implicit feedback. InProceedings of the AAAI conference on artificial intelligence, volume 30,

work page 2017
[8]

Unbiased learning-to-rank with biased feedback

Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. Unbiased learning-to-rank with biased feedback. InProceedings of the tenth ACM international conference on web search and data mining, pages 781–789 , 2017 . Nathan Kallus. Balanced policy evaluation and learning. InAdvances in Neural Information 45 Processing Systems, volume 31, pages 8895–8906,

work page 2017
[9]

Large-scale Validation of Counterfactual Learning Methods: A Test-Bed

Damien Lefortier, Adith Swaminathan, Xiaotao Gu, Thorsten Joachims, and Maarten de Rijke. Large-scale validation of counterfactual learning methods: A test-bed. Technical report, arXiv preprint arXiv:1612.00367 ,

work page internal anchor Pith review Pith/arXiv arXiv
[10]

Algorithm as Experiment: Machine Learning, Market Design, and Policy Eligibility Rules

Yusuke Narita and Kohei Yata. Algorithm as experiment: Machine learning, market design, and policy eligibility rules. Technical report, arXiv preprint arXiv:2104.12909 , revised 2023,

work page internal anchor Pith review Pith/arXiv arXiv 2023
[11]

Efficient counterfactual learning from bandit feedback

Yusuke Narita, Shota Yasui, and Kohei Yata. Efficient counterfactual learning from bandit feedback. InProceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4634–4641, 2019 . Donald B Rubin. Randomization analysis of experimental data: The fisher randomization test comment.Journal of the American Statistical Association, 75(371):591–593,

work page 2019
[12]

Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation

Yuta Saito, Shunsuke Aihara, Megumi Matsutani, and Yusuke Narita. Open bandit dataset and pipeline: Towards realistic and reproducible off-policy evaluation. Technical report, arXiv preprint arXiv:2008.07146,

work page internal anchor Pith review Pith/arXiv arXiv 2008
[13]

Unbiased compar- ative evaluation of ranking functions

Tobias Schnabel, Adith Swaminathan, Peter I Frazier, and Thorsten Joachims. Unbiased compar- ative evaluation of ranking functions. InProceedings of the 2016 ACM International Conference on the Theory of Information Retrieval, pages 109–118,

work page 2016
[14]

Batch learning from logged bandit feedback through counterfactual risk minimization.Journal of Machine Learning Research, 16(52):1731–1755, 2015a

Adith Swaminathan and Thorsten Joachims. Batch learning from logged bandit feedback through counterfactual risk minimization.Journal of Machine Learning Research, 16(52):1731–1755, 2015a. Adith Swaminathan and Thorsten Joachims. Counterfactual risk minimization: Learning from logged bandit feedback. InProceedings of the 32nd International Conference on Ma...

work page arXiv
[15]

Lost in aggrega- tion: The causal interpretation of the iv estimand.arXiv preprint arXiv:2601.12120,

Danielle Tsao, Krikamol Muandet, Frederick Eberhardt, and Emilija Perković. Lost in aggrega- tion: The causal interpretation of the iv estimand.arXiv preprint arXiv:2601.12120,

work page arXiv
[16]

A Review of Off-Policy Evaluation in Reinforcement Learning

Masatoshi Uehara, Chengchun Shi, and Nathan Kallus. A review of off-policy evaluation in reinforcement learning.arXiv preprint arXiv:2212.06355,

work page internal anchor Pith review Pith/arXiv arXiv
[17]

Optimal and adaptive off-policy evaluation in contextual bandits

Yu-Xiang Wang, Alekh Agarwal, and Miroslav Dudík. Optimal and adaptive off-policy evaluation in contextual bandits. InProceedings of the 34th International Conference on Machine Learning, volume 70 ofProceedings of Machine Learning Research, pages 3589–3597 , 2017 . Yueqing Xuan, Kacper Sokol, Mark Sanderson, and Jeffrey Chan. Evaluating and addressing fa...

work page 2017

[1] [1]

Policy evaluation with latent confounders via optimal balance

Andrew Bennett and Nathan Kallus. Policy evaluation with latent confounders via optimal balance. InAdvances in Neural Information Processing Systems, volume 32, 2019 . Léon Bottou, Jonas Peters, Joaquin Quiñonero-Candela, Denis X. Charles, D. Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Simard, and Ed Snelson. Counterfactual reasoning and learnin...

work page 2019

[2] [2]

Semi-parametric effi- cient policy learning with continuous actions

Victor Chernozhukov, Mert Demirer, Greg Lewis, and Vasilis Syrgkanis. Semi-parametric effi- cient policy learning with continuous actions. InAdvances in Neural Information Processing Systems, volume 32, pages 15039–15049 , 2019 . Tirthankar Dasgupta, Natesh S Pillai, and Donald B Rubin. Causal inference from 2k factorial designs by using potential outcome...

work page 2019

[3] [3]

Estimation Considerations in Contextual Bandits

Maria Dimakopoulou, Zhengyuan Zhou, Susan Athey, and Guido Imbens. Estimation considera- tions in contextual bandits. Technical report, arXiv preprint arXiv:1711.07077 ,

work page internal anchor Pith review Pith/arXiv arXiv

[4] [4]

Balanced linear contextual bandits

Maria Dimakopoulou, Zhengyuan Zhou, Susan Athey, and Guido Imbens. Balanced linear contextual bandits. InProceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3445–3453, 2019 . Jingtao Ding, Guanghui Yu, Xiangnan He, Yuhan Quan, Yong Li, Tat-Seng Chua, Depeng Jin, and Jiajie Yu. Improving implicit recommender systems with view da...

work page 2019

[5] [5]

Reinforced negative sam- pling for recommendation with exposure data

Jingtao Ding, Yuhan Quan, Xiangnan He, Yong Li, and Depeng Jin. Reinforced negative sam- pling for recommendation with exposure data. InProceedings of the 28th International Joint Conference on Artificial Intelligence, pages 2230–2236, 2019 . Miroslav Dudík, John Langford, and Lihong Li. Doubly robust policy evaluation and learning. InProceedings of the 2...

work page 2019

[6] [6]

Top challenges from the first practical online controlled experiments summit.ACM SIGKDD Explorations Newsletter, 21(1):20–35, 2019

Somit Gupta, Ronny Kohavi, Diane Tang, Ya Xu, Reid Andersen, Eytan Bakshy, Niall Cardin, Sumita Chandran, Nanyu Chen, Dominic Coey, et al. Top challenges from the first practical online controlled experiments summit.ACM SIGKDD Explorations Newsletter, 21(1):20–35, 2019 . Vitor Hadad, David A. Hirshberg, Ruohan Zhan, Stefan Wager, and Susan Athey. Confiden...

work page 2019

[7] [7]

Ruining He and Julian McAuley

PMLR, 2017 . Ruining He and Julian McAuley. Vbpr: visual bayesian personalized ranking from implicit feedback. InProceedings of the AAAI conference on artificial intelligence, volume 30,

work page 2017

[8] [8]

Unbiased learning-to-rank with biased feedback

Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. Unbiased learning-to-rank with biased feedback. InProceedings of the tenth ACM international conference on web search and data mining, pages 781–789 , 2017 . Nathan Kallus. Balanced policy evaluation and learning. InAdvances in Neural Information 45 Processing Systems, volume 31, pages 8895–8906,

work page 2017

[9] [9]

Large-scale Validation of Counterfactual Learning Methods: A Test-Bed

Damien Lefortier, Adith Swaminathan, Xiaotao Gu, Thorsten Joachims, and Maarten de Rijke. Large-scale validation of counterfactual learning methods: A test-bed. Technical report, arXiv preprint arXiv:1612.00367 ,

work page internal anchor Pith review Pith/arXiv arXiv

[10] [10]

Algorithm as Experiment: Machine Learning, Market Design, and Policy Eligibility Rules

Yusuke Narita and Kohei Yata. Algorithm as experiment: Machine learning, market design, and policy eligibility rules. Technical report, arXiv preprint arXiv:2104.12909 , revised 2023,

work page internal anchor Pith review Pith/arXiv arXiv 2023

[11] [11]

Efficient counterfactual learning from bandit feedback

Yusuke Narita, Shota Yasui, and Kohei Yata. Efficient counterfactual learning from bandit feedback. InProceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4634–4641, 2019 . Donald B Rubin. Randomization analysis of experimental data: The fisher randomization test comment.Journal of the American Statistical Association, 75(371):591–593,

work page 2019

[12] [12]

Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation

Yuta Saito, Shunsuke Aihara, Megumi Matsutani, and Yusuke Narita. Open bandit dataset and pipeline: Towards realistic and reproducible off-policy evaluation. Technical report, arXiv preprint arXiv:2008.07146,

work page internal anchor Pith review Pith/arXiv arXiv 2008

[13] [13]

Unbiased compar- ative evaluation of ranking functions

Tobias Schnabel, Adith Swaminathan, Peter I Frazier, and Thorsten Joachims. Unbiased compar- ative evaluation of ranking functions. InProceedings of the 2016 ACM International Conference on the Theory of Information Retrieval, pages 109–118,

work page 2016

[14] [14]

Batch learning from logged bandit feedback through counterfactual risk minimization.Journal of Machine Learning Research, 16(52):1731–1755, 2015a

Adith Swaminathan and Thorsten Joachims. Batch learning from logged bandit feedback through counterfactual risk minimization.Journal of Machine Learning Research, 16(52):1731–1755, 2015a. Adith Swaminathan and Thorsten Joachims. Counterfactual risk minimization: Learning from logged bandit feedback. InProceedings of the 32nd International Conference on Ma...

work page arXiv

[15] [15]

Lost in aggrega- tion: The causal interpretation of the iv estimand.arXiv preprint arXiv:2601.12120,

Danielle Tsao, Krikamol Muandet, Frederick Eberhardt, and Emilija Perković. Lost in aggrega- tion: The causal interpretation of the iv estimand.arXiv preprint arXiv:2601.12120,

work page arXiv

[16] [16]

A Review of Off-Policy Evaluation in Reinforcement Learning

Masatoshi Uehara, Chengchun Shi, and Nathan Kallus. A review of off-policy evaluation in reinforcement learning.arXiv preprint arXiv:2212.06355,

work page internal anchor Pith review Pith/arXiv arXiv

[17] [17]

Optimal and adaptive off-policy evaluation in contextual bandits

Yu-Xiang Wang, Alekh Agarwal, and Miroslav Dudík. Optimal and adaptive off-policy evaluation in contextual bandits. InProceedings of the 34th International Conference on Machine Learning, volume 70 ofProceedings of Machine Learning Research, pages 3589–3597 , 2017 . Yueqing Xuan, Kacper Sokol, Mark Sanderson, and Jeffrey Chan. Evaluating and addressing fa...

work page 2017