Using causal abstractions to accelerate decision-making in complex bandit problems

Anisoara Calinescu; Fabio Massimo Zennaro; Joel Dyer; Michael Wooldridge; Nicholas Bishop

REVIEW 3 major objections 4 minor 27 references

Using causal abstractions to accelerate decision-making in complex bandit problems

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes AT-UCB, which uses cheap abstract causal models to filter arms before UCB, and proves a regret bound that beats standard UCB when the surviving action set is small.

desk verdict A clear, honest paper extending CAMABs to the inverse abstraction direction; the regret bound is a real contribution, but the proof of Proposition 1 quietly assumes the abstract optimal arm has a base preimage—an easy fix that nevertheless needs to be stated. read the letter →

arxiv 2509.04296 v1 pith:3KKXJWNO submitted 2025-09-04 cs.LG

Joel Dyer , Nicholas Bishop , Anisoara Calinescu , Michael Wooldridge , Fabio Massimo Zennaro This is my paper · ORCID

classification cs.LG

keywords causalmulti-armedbanditsabstractionregretboundarmfilteringUCBalgorithmerrorepidemicsimulationthresholding

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that when a decision problem has both a cheap, coarse causal model and an expensive, fine-grained one, the cheap model can be used before learning to eliminate bad actions, so that an upper-confidence-bound learner only pays regret over a small surviving set. It proposes AT-UCB and gives a regret bound stating that total expected regret is roughly the abstract exploration cost plus UCB regret over the surviving base actions, which beats classic UCB when the surviving set is much smaller than the full action set. The practical upshot for simulators in epidemiology and similar fields is that a small budget spent on a fast approximate model can shrink the search space before spending expensive simulation budget on plausible interventions.

What carries the argument

The threshold epsilon(alpha) = 2(s(alpha)+e(alpha)), built from the abstraction's interventional consistency error and reward discrepancy error, is the central object. It converts two distribution-level measures of how faithful the coarse model is to the fine model into a single action-filtering threshold: abstract actions whose estimated means fall more than epsilon below the abstract best are discarded, and only base actions mapping to the remaining abstract actions enter the UCB phase.

What would settle it

Construct a two-level bandit with abstract arms a'_1 and a'_2 whose means differ by more than epsilon, where a'_1 has no base preimage and every base arm maps to a'_2. Let AT-UCB explore the abstract model long enough to satisfy its sample-size assumption; if it then classifies a'_2 as sub-threshold and never selects the true base optimal arm, the Proposition 1 bound is violated and the predicted regret reduction does not occur.

Watch

Extended reading notes

Core claim

The central claim is that causal abstraction gives a principled, quantitative way to transfer knowledge from a cheap approximate model to an expensive base model, with guarantees. For an abstraction with interventional consistency error e(alpha) and reward discrepancy error s(alpha), Proposition 1 bounds the abstract mean of the image of the true base optimal arm: it lies within epsilon(alpha) = 2(s(alpha)+e(alpha)) of the abstract optimal mean. This justifies defining a candidate set of abstract actions whose means are within epsilon of the abstract best, then restricting the base search to base actions mapping into that set. AT-UCB estimates abstract means by uniform exploration, removes a

Load-bearing premise

The proof depends on the best simplified action being the image of at least one fine-grained action; if the best simplified action has no fine-grained counterpart, the algorithm can eliminate the true best fine-grained action and the regret guarantee no longer holds.

Editorial extensions

If this is right

Decision-makers with multi-fidelity simulators can allocate a small exploration budget to the cheap model and then restrict expensive simulation to a small candidate action set, reducing cumulative regret when the abstraction is accurate.
The regret bound scales with the number of surviving base actions rather than the total number of base actions, so the gain grows as the abstraction becomes more faithful and the surviving set shrinks.
Even approximate abstractions with nonzero consistency and reward-distortion errors yield a working algorithm, provided the errors are known well enough to set epsilon.
The bound reduces to the standard UCB bound as a special case when the surviving set is the full action set, so AT-UCB is never worse in the bound when abstraction quality is poor.
The same filter-then-UCB pattern applies to any problem with a coarse simulator related to a fine simulator by an approximate causal abstraction, such as agent-based epidemic models paired with differential-equation models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the paper never states that the intervention map omega is surjective, a natural strengthening is to define the candidate set over abstract actions that have at least one base preimage; this would preserve the regret bound even when the abstract optimum has no fine-grained counterpart.
The regret guarantee depends on knowing epsilon exactly; a testable extension is to estimate epsilon online during the abstract exploration phase, since in practice the abstraction errors are often unavailable in advance.
AT-UCB ignores the causal graph structure inside the base and abstract models; combining its arm-filtering step with causal-bandit algorithms that exploit d-separators could reduce the abstract exploration cost further.
The thresholding mechanism is generic: any bandit setting with a relative threshold derived from an error bound could reuse AT-UCB's structure, so the result may extend beyond causally abstracted multi-armed bandits.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new thing here is the direction: instead of transferring information from base to abstract model, AT-UCB uses a cheap abstract bandit to filter actions before running UCB on the expensive base model. That inversion is not in the Zennaro et al. CAMAB paper, and the regret bound in Proposition 2 is a legitimate extension. The algorithm itself is simple—uniform exploration, thresholding, UCB on surviving preimages—but that simplicity is a virtue for a first result in this setting. The writing is clear, and the authors are candid about the practical limitations in Appendix D: they know that n' requires unknown gaps, that epsilon(alpha) is usually unknown, and that they don't use the causal graph for exploration. That honesty counts for something.

The soft spots are real but mostly proportionate to the paper's scope. First, the stress-test note is correct and matters: the proof of Proposition 1 applies Lemma 2 to an arm a_k in omega^{-1}(a'_1) without ever stating that this set is nonempty. If the abstract optimal arm has no base preimage, the bound can fail badly, and the filter can delete the true optimal base arm. The counterexample with both base arms mapping to the suboptimal abstract action is a valid illustration. This is load-bearing in the sense that the regret guarantee rests on it, but it's a one-line repair: add surjectivity of omega, or at minimum a'_1 in omega(A), to the hypotheses. The authors should flag it as a modeling assumption.

Second, the empirical work is thin. One SIRS model, 10 repeats, no released code. For a theory-first paper that's acceptable, but it limits what we can conclude about how often the speedup actually materializes. The authors don't overclaim—they say the advantage depends on the abstraction being accurate and epsilon being known.

Third, the self-citation is fine. Lemma 1 comes from their prior work, but the inverse-direction algorithm and its regret bound are not a dressed-up restatement.

Who is this for? People working on causal abstraction, multi-fidelity optimization with simulators, or warm-started bandits. It's a reasonable, honest contribution that deserves referee time. I'd like to see the surjectivity assumption added, code released, and ideally a second experiment that varies epsilon to show sensitivity. As is, a conditional accept is fair.

Referee Report

3 major / 4 minor

Summary. The paper proposes AT-UCB, an algorithm for causally abstracted multi-armed bandits (CAMABs) that first uniformly explores a cheap, coarse-grained abstract bandit, uses the resulting reward estimates to discard abstract actions that appear suboptimal relative to the best observed mean minus a threshold ε, and then runs UCB on the base actions whose abstract image survives the filter. The main theoretical contribution is Proposition 2, an expected regret bound of the form n′C + 3Σ_{a∈ω^{-1}(D^c)} Δ_a + Σ_{a∈ω^{-1}(D^c), Δ_a>0} 16 log(n)/Δ_a + 2 max_a Δ_a, claimed to improve on standard UCB when the abstraction is accurate enough that |ω^{-1}(D^c)| ≪ k. The paper also reports a small experiment on SIR epidemic simulators with k=10 base regions and k′=4 abstract regions, showing lower empirical regret for small ε. The theoretical analysis is conditional on Proposition 1 and Assumption 1, which relate the abstraction errors to the abstract optimality gaps.

Significance. If the theory is repaired, this is a useful contribution to the CAMAB literature: it addresses the inverse direction (abstract-to-base transfer), which is less studied than the base-to-abstract direction in prior work by Zennaro et al., and it gives a clean conditional regret bound rather than only heuristics. The paper is transparent about several limitations, especially in Appendix D, and the proof strategy is largely standard. The central claim, however, currently rests on a missing attainability/surjectivity condition and on assumptions that are not well-defined in edge cases; these issues are load-bearing rather than cosmetic. With explicit separation and attainability conditions, and with a clearer statement about the oracle knowledge needed to set ε and n′, the result would be a solid, publishable contribution.

major comments (3)

[§3, Proposition 1 (Eq. 3) and Appendix B] The proof of the left inequality applies Lemma 2 to 'a_k ∈ ω^{-1}(a'_1)', but Definition 4 only gives ω: A → A′ and never asserts that a′_1 is in the image of ω. If a′_1 has no base preimage, the proof is invalid and the statement can be false. For example, take an exact abstraction (s(α)=e(α)=0) with two base arms both of mean 0, abstract arms of means 100 and 0, and ω mapping both base arms to the abstract arm of mean 0. All abstraction-error definitions are satisfied, yet Eq. (3) would require 0 ≥ 100. This is not cosmetic: under Algorithm 1 with ε=ε(α), the set ω^{-1}(D^c) can be empty or can exclude the true optimal base arm, so the regret bound in Proposition 2 cannot hold. Please add an explicit attainability/surjectivity assumption (e.g., a′_1 ∈ ω(A), or ω surjective) to the hypotheses of Proposition 1 and consequently to Assumption 1/Algorithm 1.
[§3.1, Assumption 1 (Eq. 6) and Lemma 3] Assumption 1 is not well-defined when D = ∅, since min_{a′∈D} Δ′_{a′} is taken over an empty set; if interpreted as +∞, no finite n′ can satisfy the displayed lower bound. In addition, the right-hand side contains (ε(α)-Δ′_{ω(a1)})^{-2}; Proposition 1 only guarantees Δ′_{ω(a1)} ≤ ε(α). If equality holds, the denominator is zero and no finite n′ exists, so the phrase 'n′ sufficiently large' is not enough. The proof of Lemma 3 requires this quantity to be strictly positive. Please state the required separation conditions explicitly and handle the D=∅ case separately; in that degenerate case AT-UCB reduces to UCB on the full base action set and the second sum in Lemma 3 is empty.
[§3, Proposition 2; §5 and Appendix D] The central regret guarantee is conditional on ε = ε(α) and on an n′ satisfying Assumption 1, both of which depend on quantities (ε(α), abstract optimality gaps) that the learner is assumed to know. The paper candidly acknowledges this in the Conclusion and Appendix D, but the limitation is load-bearing: the algorithm as stated takes a user-supplied ε, Figure 1 shows that performance degrades as ε increases, and no guidance is given for setting n′ without oracle knowledge. As written, the practical speed-up claim holds only under oracle tuning. Please state the result explicitly as a separation-type guarantee that includes the oracle requirement, and either discuss constructive choices for ε and n′ or clearly identify adaptive/parameter-free variants as open problems.

minor comments (4)

[Appendix B, Proposition 1 proof] In the restated Proposition 1 in Appendix B, the text says 'with IC error s(α) and RD error e(α)', which swaps the names from Definition 5 (where e is IC error and s is RD error). The definition of ε(α) is unaffected, but the naming should be consistent.
[§3, text after Eq. (7)] The comparison with the standard UCB bound is informal. Constants (16 vs. the usual 8) and the presence of surviving near-duplicate arms with very small gaps can affect the comparison. Please either make the comparison precise or label it as a heuristic motivating the algorithm.
[§4, Experiments] The experimental evaluation is limited to one configuration (k=10, k′=4, n=100, n′=20, 10 repeats) and a single plot. A sensitivity analysis over n′, over the abstraction errors, and across different reward distributions would substantially strengthen the empirical claim. The captions could also define the plotted quantities more explicitly.
[§4, Code availability] The paper states that code will be released upon acceptance. Providing a public repository at submission time, or at least pseudocode with exact parameter choices, would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the regret bound follows from the abstraction-error definitions and standard UCB analysis; the self-cited Lemma 1 is parameter-free and does not encode the target result.

full rationale

The derivation chain is self-contained once the CAMAB definitions are fixed. Lemma 1 (quoted from Zennaro et al. 2024, Prop. 4.1) is a parameter-free inequality following immediately from the IC and RD error definitions via the Wasserstein triangle inequality; it is stated for an arbitrary action a and does not assume or encode Proposition 1 or the AT-UCB regret bound. Proposition 1 is then proved from Lemma 2 (proved in Appendix B from Lemma 1) and the definition of the abstract optimal arm. Proposition 2 is a standard UCB regret bound applied to the restricted action set ω^{-1}(D^c), plus a union-bound/Subgaussian analysis of the filtering event G; no fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. The only substantive issue is a missing attainability/surjectivity assumption in Proposition 1 (the proof selects a_k ∈ ω^{-1}(a'_1) without proving existence); that is a correctness gap, not a circular reduction. Self-citations to the CAMAB paper and to Dyer et al. are used for background and a supporting lemma, but the new inverse-direction algorithm and its bound have independent mathematical content. Hence no circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard bandit assumptions, a horizon assumption that is hard to verify, a prior lemma from the authors' own framework, and an unstated surjectivity condition on the intervention map. No new physical or mathematical entities are introduced.

free parameters (3)

threshold hyperparameter epsilon = swept over [0.1, 0.5] in Figure 1; theory requires epsilon = epsilon(alpha), which is not computed
Algorithm 1 input; the regret bound is only proven for epsilon = epsilon(alpha), and the experiment's improvement depends on choosing small epsilon.
abstract horizon n' = 20 in experiments
Forced to satisfy Assumption 1 in theory; no practical guidance for choosing it without knowing abstract gaps, as acknowledged in Appendix D.
UCB parameter delta = 0.1 in experiments
Standard tuning parameter for UCB; not central to the theoretical claim.

assumptions (4)

domain assumption Reward distributions are 1-subgaussian
Used throughout the regret proof in Section 3.1 to justify Chernoff and subgaussian tail bounds.
domain assumption Assumption 1: abstract horizon n' is large enough relative to unknown gaps and epsilon(alpha)
The regret bound only holds if the abstract exploration horizon is long enough; the lower bound depends on unknown optimality gaps and epsilon(alpha), acknowledged as difficult to verify in advance in Appendix D.
ad hoc to paper Surjectivity of omega or nonempty preimage of the abstract optimal arm
Proof of Proposition 1 in Appendix B selects a_k in omega^{-1}(a'_1); the paper never states that omega is surjective or that the abstract optimal arm has a base preimage, and the filtering guarantee fails otherwise.
domain assumption Lemma 1 from Zennaro et al. 2024: |mu_a - mu_{omega(a)}| <= s(alpha) + e(alpha)
Used as the core bound relating base and abstract rewards; it is cited rather than proved here and comes from the authors' earlier work.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Using causal abstractions to accelerate decision-making in complex bandit problems." pith.science (2026). https://pith.science/paper/3KKXJWNO

@misc{pith2026250904296,
  author       = {Pith},
  title        = {Pith review of: Using causal abstractions to accelerate decision-making in complex bandit problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3KKXJWNO}},
  note         = {Machine review of arXiv:2509.04296}
}

read the original abstract

Although real-world decision-making problems can often be encoded as causal multi-armed bandits (CMABs) at different levels of abstraction, a general methodology exploiting the information and computational advantages of each abstraction level is missing. In this paper, we propose AT-UCB, an algorithm which efficiently exploits shared information between CMAB problem instances defined at different levels of abstraction. More specifically, AT-UCB leverages causal abstraction (CA) theory to explore within a cheap-to-simulate and coarse-grained CMAB instance, before employing the traditional upper confidence bound (UCB) algorithm on a restricted set of potentially optimal actions in the CMAB of interest, leading to significant reductions in cumulative regret when compared to the classical UCB algorithm. We illustrate the advantages of AT-UCB theoretically, through a novel upper bound on the cumulative regret, and empirically, by applying AT-UCB to epidemiological simulators with varying resolution and computational cost.

Figures

Figures reproduced from arXiv: 2509.04296 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 22 canonical work pages

[1]

Bandits with unobserved confounders: A causal approach

Elias Bareinboim, Andrew Forney, and Judea Pearl. Bandits with unobserved confounders: A causal approach. Advances in Neural Information Processing Systems, 28: 0 1342--1350, 2015

work page 2015
[2]

Abstracting causal models

Sander Beckers and Joseph Y Halpern. Abstracting causal models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 2678--2685, 2019

work page 2019
[3]

Adaptively Exploiting d-Separators with Causal Bandits

Blair Bilodeau, Linbo Wang, and Daniel M Roy. Adaptively exploiting d-separators with causal bandits. arXiv preprint arXiv:2202.05100, 2022

work page Pith review arXiv 2022
[4]

Leveraging side observations in stochastic bandits

St \'e phane Caron, Branislav Kveton, Marc Lelarge, and Smriti Bhagat. Leveraging side observations in stochastic bandits. arXiv preprint arXiv:1210.4839, 2012

arXiv 2012
[5]

Leveraging (biased) information: Multi-armed bandits with offline data

Wang Chi Cheung and Lixing Lyu. Leveraging (biased) information: Multi-armed bandits with offline data. arXiv preprint arXiv:2405.02594, 2024

arXiv 2024
[6]

Interventionally consistent surrogates for complex simulation models

Joel Dyer, Nicholas Bishop, Yorgos Felekis, Fabio Massimo Zennaro, Anisoara Calinescu, Theodoros Damoulas, and Michael Wooldridge. Interventionally consistent surrogates for complex simulation models. Advances in Neural Information Processing Systems, 37: 0 21814--21841, 2024

work page 2024
[7]

Causal bandits: Learning good interventions via causal inference

Finnian Lattimore, Tor Lattimore, and Mark D Reid. Causal bandits: Learning good interventions via causal inference. In Advances in Neural Information Processing Systems, pages 1181--1189, 2016

work page 2016
[8]

Bandit algorithms

Tor Lattimore and Csaba Szepesv \'a ri. Bandit algorithms. Cambridge University Press, 2020

2020

Show all 27 references

[9]

Structural causal bandits with non-manipulable variables

Sanghack Lee and Elias Bareinboim. Structural causal bandits with non-manipulable variables. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4164--4172, 2019

2019
[10]

An optimal algorithm for the thresholding bandit problem

Andrea Locatelli, Maurilio Gutzeit, and Alexandra Carpentier. An optimal algorithm for the thresholding bandit problem. In International Conference on Machine Learning, pages 1690--1698. PMLR, 2016

2016
[11]

Regret analysis of bandit problems with causal background knowledge

Yangyi Lu, Amirhossein Meisami, Ambuj Tewari, and William Yan. Regret analysis of bandit problems with causal background knowledge. In Conference on Uncertainty in Artificial Intelligence, pages 141--150. PMLR, 2020

2020
[12]

From bandits to experts: On the value of side-observations

Shie Mannor and Ohad Shamir. From bandits to experts: On the value of side-observations. Advances in neural information processing systems, 24, 2011

2011
[13]

Finding all epsilon-good arms in stochastic bandits

Blake Mason, Lalit Jain, Ardhendu Tripathy, and Robert Nowak. Finding all epsilon-good arms in stochastic bandits. In Advances in Neural Information Processing Systems, volume 33, pages 20707--20718, 2020

2020
[14]

Causality

Judea Pearl. Causality. Cambridge University Press, 2009

2009
[15]

Stochastic bandits with groups of similar arms

Fabien Pesquerel, Hassan Saber, and Odalric-Ambrym Maillard. Stochastic bandits with groups of similar arms. Advances in Neural Information Processing Systems, 34: 0 19461--19472, 2021

2021
[16]

Elements of causal inference: F oundations and learning algorithms

Jonas Peters, Dominik Janzing, and Bernhard Sch \"o lkopf. Elements of causal inference: F oundations and learning algorithms . MIT Press, 2017

2017
[17]

Graph feedback bandits with similar arms

Han Qi, Guo Fei, and Li Zhu. Graph feedback bandits with similar arms. arXiv preprint arXiv:2405.11171, 2024

2024 arXiv
[18]

The category theory of causal models

Eigil Fjeldgren Rischel. The category theory of causal models. Master's thesis, University of Copenhagen, 2020

2020
[19]

Causal consistency of structural equation models

Paul K Rubenstein, Sebastian Weichwald, Stephan Bongers, Joris M Mooij, Dominik Janzing, Moritz Grosse-Wentrup, and Bernhard Sch \"o lkopf. Causal consistency of structural equation models. In 33rd Conference on Uncertainty in Artificial Intelligence (UAI 2017), pages 808--817...

2017
[20]

Aligning graphical and functional causal abstractions

Willem Schooltink and Fabio Massimo Zennaro. Aligning graphical and functional causal abstractions. arXiv preprint arXiv:2412.17080, 2024

2024 arXiv
[21]

Warm starting bandits with side information from confounded data

Nihal Sharma, Soumya Basu, Karthikeyan Shanmugam, and Sanjay Shakkottai. Warm starting bandits with side information from confounded data. arXiv preprint arXiv:2002.08405, 2020

2002 arXiv
[22]

Multi-armed bandits with dependent arms

Rahul Singh, Fang Liu, Yin Sun, and Ness Shroff. Multi-armed bandits with dependent arms. Machine Learning, 113 0 (1): 0 45--71, 2024

2024
[23]

Regional multi-armed bandits

Zhiyang Wang, Ruida Zhou, and Cong Shen. Regional multi-armed bandits. In International Conference on Artificial Intelligence and Statistics, pages 510--518. PMLR, 2018

2018
[24]

Best arm identification with possibly biased offline data

Le Yang, Vincent YF Tan, and Wang Chi Cheung. Best arm identification with possibly biased offline data. arXiv preprint arXiv:2505.23165, 2025

2025 arXiv
[25]

Quantifying consistency and information loss for causal abstraction learning

Fabio Massimo Zennaro, Paolo Turrini, and Theodoros Damoulas. Quantifying consistency and information loss for causal abstraction learning. In Proceedings of the Thrity-Second International Conference on International Joint Conferences on Artificial Intelligence, 2023

2023
[26]

Causally abstracted multi-armed bandits

Fabio Massimo Zennaro, Nicholas Bishop, Joel Dyer, Yorgos Felekis, Anisoara Calinescu, Michael Wooldridge, and Theodoros Damoulas. Causally abstracted multi-armed bandits. In Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, pages 4109--4139, 2024

2024
[27]

Transfer learning in multi-armed bandit: a causal approach

Junzhe Zhang and Elias Bareinboim. Transfer learning in multi-armed bandit: a causal approach. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, pages 1778--1780, 2017

2017

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Bandits with unobserved confounders: A causal approach

Elias Bareinboim, Andrew Forney, and Judea Pearl. Bandits with unobserved confounders: A causal approach. Advances in Neural Information Processing Systems, 28: 0 1342--1350, 2015

work page 2015

[2] [2]

Abstracting causal models

Sander Beckers and Joseph Y Halpern. Abstracting causal models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 2678--2685, 2019

work page 2019

[3] [3]

Adaptively Exploiting d-Separators with Causal Bandits

Blair Bilodeau, Linbo Wang, and Daniel M Roy. Adaptively exploiting d-separators with causal bandits. arXiv preprint arXiv:2202.05100, 2022

work page Pith review arXiv 2022

[4] [4]

Leveraging side observations in stochastic bandits

St \'e phane Caron, Branislav Kveton, Marc Lelarge, and Smriti Bhagat. Leveraging side observations in stochastic bandits. arXiv preprint arXiv:1210.4839, 2012

arXiv 2012

[5] [5]

Leveraging (biased) information: Multi-armed bandits with offline data

Wang Chi Cheung and Lixing Lyu. Leveraging (biased) information: Multi-armed bandits with offline data. arXiv preprint arXiv:2405.02594, 2024

arXiv 2024

[6] [6]

Interventionally consistent surrogates for complex simulation models

Joel Dyer, Nicholas Bishop, Yorgos Felekis, Fabio Massimo Zennaro, Anisoara Calinescu, Theodoros Damoulas, and Michael Wooldridge. Interventionally consistent surrogates for complex simulation models. Advances in Neural Information Processing Systems, 37: 0 21814--21841, 2024

work page 2024

[7] [7]

Causal bandits: Learning good interventions via causal inference

Finnian Lattimore, Tor Lattimore, and Mark D Reid. Causal bandits: Learning good interventions via causal inference. In Advances in Neural Information Processing Systems, pages 1181--1189, 2016

work page 2016

[8] [8]

Bandit algorithms

Tor Lattimore and Csaba Szepesv \'a ri. Bandit algorithms. Cambridge University Press, 2020

2020