Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

Confidence intervals for causal effects in sequential decision making

T0 review · 2 major / 2 minor · reviewed 2026-06-29 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Causal effects identified by the back-door criterion admit valid confidence intervals and sequences even when interventions depend on past data.

desk verdict The paper gives back-door causal effect CIs that widen with LIL terms under adaptive interventions and unknown horizons, and the derivation looks standard but correctly applied. read the letter →

arxiv 2605.25687 v2 pith:AQ5XXXJ4 submitted 2026-05-25 math.ST stat.TH

Vladimir Vovk , Ruodu Wang This is my paper · ORCID
classification math.STstat.TH
keywords causaleffectsconfidenceintervalsback-doorcriterionsequentialdecisionmakinglawoftheiteratedlogarithmsequencesadaptiveinterventionsIIDobservations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper derives confidence intervals for causal effects that can be identified from observational data using the back-door criterion. These intervals are tightest when observations are independent and identically distributed from a system with a known causal diagram. When interventions depend on previous observations, the intervals widen to include a term from the law of the iterated logarithm. In sequential settings where the total number of observations is not fixed in advance, the intervals form confidence sequences that incorporate additional iterated logarithm terms. A reader would care because this supplies rigorous coverage guarantees for causal quantities in adaptive, online data collection scenarios.

What carries the argument

Back-door criterion on a given causal diagram, combined with law-of-the-iterated-logarithm bounds to produce valid intervals under adaptive sampling.

What would settle it

Generate data from a known causal model, apply the adaptive sampling rule, construct the proposed intervals, and check whether the true causal effect lies outside them at a rate exceeding the nominal coverage probability.

Watch

Extended reading notes

Core claim

We derive confidence intervals and confidence sequences for causal effects in situations where the back-door criterion is applicable. Our tightest confidence intervals hold in the standard setting where the training data consists of IID observations over a system described by a given causal diagram. When interventions are allowed to depend on the past data, our confidence intervals become wider and involve a term coming from the law of the iterated logarithm, even where the number of observations is known in advance. In the sequential setting where the number of observations is not given, our confidence intervals, arranged into a confidence sequence for causal effects, involve more iterated

Load-bearing premise

The back-door criterion applies to the given causal diagram so the causal effect can be identified from the observable data distribution.

Editorial extensions

If this is right

  • Standard tight intervals apply directly when observations are IID.
  • Adaptive dependence on past data requires wider intervals containing an iterated-logarithm term.
  • Unknown sample size leads to confidence sequences with extra iterated-logarithm terms.
  • The constructions remain valid for causal effects under sequential decision processes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same widening pattern could guide the construction of intervals for other quantities identified from observational data under adaptivity.
  • Real-time monitoring of causal effects in decision systems becomes feasible without committing to a fixed sample size in advance.
  • The logarithmic penalty quantifies the statistical price of allowing interventions to react to accumulating evidence.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper derives confidence intervals and confidence sequences for causal effects identifiable via the back-door criterion. For IID observational data from a known causal diagram, it provides tight intervals based on standard identification and concentration inequalities. When interventions may depend on past observations, the intervals widen to incorporate a law-of-the-iterated-logarithm term even when the total sample size is known in advance. For the fully sequential case with unknown horizon, the construction yields a confidence sequence containing additional iterated-logarithm factors.

Significance. If the derivations hold, the results supply valid, non-asymptotic inference for back-door identifiable effects under adaptive sampling, extending classical concentration tools to causal estimators in sequential decision-making. This is relevant for applications such as adaptive experimentation and online causal inference where standard IID assumptions fail. The explicit separation of identification from concentration, together with the LIL-based widening, is a clear technical contribution when the proofs are complete.

major comments (2)
  1. [§3] §3 (IID case): the claimed tightness of the intervals rests on applying a specific concentration inequality directly to the identified functional; the manuscript must verify that the variance proxy used in the bound is estimable from the same data without inflating the coverage error beyond the stated level.
  2. [§4] §4 (adaptive interventions): the LIL term is introduced to handle dependence on past data, but the argument requires showing that the martingale difference sequence induced by the adaptive policy still satisfies the conditions of the LIL; a counter-example or explicit verification under the back-door identification is needed.
minor comments (2)
  1. [§2] Notation for the causal diagram and the identified functional should be introduced once in §2 and used consistently; several later equations reuse symbols without redefinition.
  2. The abstract states that intervals 'become even wider' in the unknown-horizon case, but the precise additional log-log factor is not quantified; a short comparison table of the three regimes would improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments. The two major points can be addressed by adding explicit verifications; we outline the responses below and will revise accordingly.

read point-by-point responses

  1. Referee: [§3] §3 (IID case): the claimed tightness of the intervals rests on applying a specific concentration inequality directly to the identified functional; the manuscript must verify that the variance proxy used in the bound is estimable from the same data without inflating the coverage error beyond the stated level.

    Authors: We agree that an explicit check is required. The variance proxy is the sample variance of the back-door identified functional (a bounded function of the observed variables). In the revision we will insert a short lemma applying a union bound over the concentration inequality and a separate concentration inequality for the variance estimator itself; the resulting additive term is of lower order and does not alter the claimed tightness for any fixed coverage level. revision: yes

  2. Referee: [§4] §4 (adaptive interventions): the LIL term is introduced to handle dependence on past data, but the argument requires showing that the martingale difference sequence induced by the adaptive policy still satisfies the conditions of the LIL; a counter-example or explicit verification under the back-door identification is needed.

    Authors: Under the back-door criterion the causal functional is an expectation of a fixed (data-independent) function of the observed variables. Any policy whose decisions depend only on past observations therefore yields a martingale-difference sequence with respect to the natural filtration. Boundedness of the functional (assumed throughout the paper) supplies the moment conditions required by the martingale LIL. We will add this one-paragraph verification to §4. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity in derivation chain

full rationale

The paper derives confidence intervals and sequences for back-door identifiable causal effects, first under IID sampling and then under adaptive interventions using the law of the iterated logarithm. The argument applies standard causal identification (explicitly conditioned on the back-door criterion and given diagram) plus off-the-shelf concentration inequalities to the resulting estimators. No equation reduces a claimed prediction to a fitted parameter by construction, no load-bearing premise rests solely on a self-citation chain, and no ansatz or uniqueness result is smuggled in via prior work by the same authors. The derivation is therefore self-contained against external benchmarks.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central claim rests on the applicability of the back-door criterion and the IID assumption for the standard case, both standard domain assumptions in causal inference rather than new postulates.

assumptions (2)
  • domain assumption The back-door criterion is applicable
    Explicitly stated as the setting in which the derivations hold.
  • domain assumption Training data consists of IID observations over a given causal diagram
    Described as the standard setting for the tightest intervals.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Confidence intervals for causal effects in sequential decision making." pith.science (2026). https://pith.science/paper/AQ5XXXJ4

@misc{pith2026260525687,
  author       = {Pith},
  title        = {Pith review of: Confidence intervals for causal effects in sequential decision making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AQ5XXXJ4}},
  note         = {Machine review of arXiv:2605.25687}
}
read the original abstract

We derive confidence intervals and confidence sequences for causal effects in situations where the back-door criterion is applicable. Our tightest confidence intervals hold in the standard setting where the training data consists of IID observations over a system described by a given causal diagram. When interventions are allowed to depend on the past data, our confidence intervals become wider and involve a term coming from the law of the iterated logarithm, even where the number of observations is known in advance. In the sequential setting where the number of observations is not given, our confidence intervals, arranged into a confidence sequence for causal effects, involve more iterated logarithm terms and become even wider.

Figures

Figures reproduced from arXiv: 2605.25687 by the authors.

Figure 1
Figure 1. The basic causal graph of this paper This paper was motivated by the difficulty of applying the methods devel￾oped in [15] to the most natural setting of sequential causal inference (which we called the “strong interpretation” of causal diagrams). The methods of this paper are completely different, and we are targeting confidence intervals rather than prediction sets, which were targeted in [15]. In Sect. 6 we brief… view at source ↗
Figure 2
Figure 2. The repeated causal graph 3 The IID setting We start from the most standard setting where the observations before inter￾vention are IID; see, e.g., [7, the beginning of Sect. 2.2]. Informally, Nature possesses stable causal mechanisms that are organized in the form of a graphi￾cal structure. In causal calculus, the structure is a known dag, and the stable causal mechanisms are unknown probability distributions of th… view at source ↗
Figure 1
Figure 1. In this section we are interested in Figure 2 with an arbitrary dag as [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: The napkin graph Theorems 3–6 may be considered to be finite-sample analogues of Theorem 1 in [14]. Their characteristic feature is the presence of iterated logarithm terms, which are unavoidable (details omitted) and are especially prominent under the strong interpret…
Figure 4
Figure 4. Figure 4: Illustration of an inequality. Next we need a simple result from interval arithmetic. We are only interested in subintervals of [0, 1]. Let c ± ∆c, where c ∈ R and ∆c ≥ 0, stand for the interval c ± ∆c := [c − ∆c, c + ∆c] ∩ [0, 1]. For a binary operation ∗ on the reals…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Local conformal prediction for individual causal effects

    stat.ME 2026-08 reject novelty 4.0 of 10

    A local conformal prediction framework that aims to provide finite-sample valid intervals for individual causal effects by calibrating on causally similar units.

Reference graph

Works this paper leans on

15 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    On testability of the front-door model via Verma constraints

    Rohit Bhattacharya and Razieh Nabi. On testability of the front-door model via Verma constraints. InProceedings of the 38th Conference on Uncertainty in Artificial Intelligence (UAI 2022), volume 180 ofProceedings of Machine Learning Research, pages 202–212, 2022

    2022

  2. [2]

    Greedy algorithms for optimizing multivariate Horner schemes.ACM SIGSAM Bulletin, 38:8–15, 2004

    Martine Ceberio and Vladik Kreinovich. Greedy algorithms for optimizing multivariate Horner schemes.ACM SIGSAM Bulletin, 38:8–15, 2004

    2004

  3. [3]

    Estimation of causal effects with small data in the presence of trapdoor variables.Journal of the Royal Statistical Society A, 184:1030–1051, 2021

    Jouni Helske, Santtu Tikka, and Juha Karvanen. Estimation of causal effects with small data in the presence of trapdoor variables.Journal of the Royal Statistical Society A, 184:1030–1051, 2021

    2021

  4. [4]

    Kolmogorov.Grundbegriffe der Wahrscheinlichkeitsrechnung

    Andrei N. Kolmogorov.Grundbegriffe der Wahrscheinlichkeitsrechnung. Springer, Berlin, 1933. English translation:Foundations of the Theory of Probability. Chelsea, New York, 1950

    1933

  5. [5]

    Kuipers, A

    J. Kuipers, A. Plaat, J. A. M. Vermaseren, and H. J. van den Herik. Improv- ing multivariate Horner schemes with Monte Carlo tree search.Computer Physics Communications, 184:2391–2395, 2013

    2013

  6. [6]

    Some inequalities relating to the partial sum of binomial probabilities.Annals of the Institute of Statistical Mathematics, 10:29–35, 1959

    Masashi Okamoto. Some inequalities relating to the partial sum of binomial probabilities.Annals of the Institute of Statistical Mathematics, 10:29–35, 1959

    1959

  7. [7]

    Cambridge, Cambridge University Press, second edition, 2009

    Judea Pearl.Causality: Models, Reasoning, and Inference. Cambridge, Cambridge University Press, second edition, 2009

    2009

  8. [8]

    Wiley, Hoboken, NJ, 2019

    Glenn Shafer and Vladimir Vovk.Game-Theoretic Foundations for Proba- bility and Finance. Wiley, Hoboken, NJ, 2019

    2019

Show all 15 references
  1. [9]

    Shiryaev.Probability-1

    Albert N. Shiryaev.Probability-1. Springer, New York, third edition, 2016

    2016

  2. [10]

    Shiryaev.Probability-2

    Albert N. Shiryaev.Probability-2. Springer, New York, third edition, 2019

    2019

  3. [11]

    Vapnik.Statistical Learning Theory

    Vladimir N. Vapnik.Statistical Learning Theory. Wiley, New York, 1998

    1998

  4. [12]

    Gauthier-Villars, Paris, 1939

    Jean Ville.Etude critique de la notion de collectif. Gauthier-Villars, Paris, 1939

    1939

  5. [13]

    Treatment of statistical estimation problems in random- ized smoothing for adversarial robustness

    V´ aclav Vor´ aˇ cek. Treatment of statistical estimation problems in random- ized smoothing for adversarial robustness. InNeurIPS 2024, 2024

    2024

  6. [14]

    Another semantics for Pearl’s action calculus

    Vladimir Vovk. Another semantics for Pearl’s action calculus. In Alex Gammerman, editor,Computational Learning and Probabilistic Reasoning, chapter 7, pages 127–146. Wiley, New York, 1996

    1996

  7. [15]

    +”, “−”, and “×

    Vladimir Vovk and Ruodu Wang. Conformal e-prediction in the presence of confounding. Technical Report arXiv:2603.11134 [math.ST], arXiv.org e-Print archive, March 2026. 10 A Proofs In the following two propositions we consider an IID Bernoulli sequence ξ1, ξ2, . . .with probab...

    2026

Pith tools

Reviewed June 29, 2026 · model on record in the stance chip above.