The Promises of Multiple Experiments: Identifying Joint Distribution of Potential Outcomes

Peng Wu; Xiaojie Mao

arxiv: 2504.20470 · v2 · submitted 2025-04-29 · 📊 stat.ME

The Promises of Multiple Experiments: Identifying Joint Distribution of Potential Outcomes

Peng Wu , Xiaojie Mao This is my paper

Pith reviewed 2026-05-22 18:58 UTC · model grok-4.3

classification 📊 stat.ME

keywords causal inferencejoint distributionpotential outcomesmultiple experimentstransportabilityprincipal causal effectsleast squares estimator

0 comments

The pith

Multiple experiments identify the joint distribution of potential outcomes under transportability of state transitions

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a framework to identify and estimate the joint distribution of potential outcomes from multiple experimental datasets rather than relying on marginal distributions alone. It introduces the assumption that state transition probabilities for potential outcomes are transportable across datasets, together with a full-column rank condition, to recover how outcomes would co-occur under different treatments. This matters for applications such as surrogate endpoint evaluation where joint behaviors matter more than averages. The authors propose a least-squares estimator that is shown to be consistent and asymptotically normal and extend the approach to principal causal effects, with the dataset indicator playing a role analogous to an instrument.

Core claim

Under the assumption of transportability of state transition probabilities for potential outcomes across datasets and a regular full-column rank condition, the joint distribution of potential outcomes is identified; a least-squares-based estimator is consistent and asymptotically normal. The key identification assumptions are testable in an overidentified setting and are analogous to those in the context of instrumental variables, with the dataset indicator serving as an instrument. The framework further extends to identify and estimate principal causal effects.

What carries the argument

Transportability of state transition probabilities for potential outcomes across datasets, which connects separate experiments to recover the full joint distribution.

If this is right

The joint distribution of potential outcomes can be recovered from multiple experiments.
A simple least-squares estimator for the joint distribution is consistent and asymptotically normal.
Principal causal effects become identifiable and estimable within the same framework.
The identifying assumptions can be tested directly in overidentified settings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The approach may extend to settings where experiments share similar transition structures even if populations differ slightly.
It offers a way to pool information across trials for estimands that require joint rather than separate outcome information.

Load-bearing premise

State transition probabilities for potential outcomes are the same across the different experimental datasets.

What would settle it

If combining the datasets under the transportability assumption produces inconsistent estimates of the joint distribution when checked against held-out data or when the observed matrix fails the full-column rank condition, the identification would not hold.

read the original abstract

Typical causal effects are defined based on the marginal distribution of potential outcomes. However, many real-world applications require causal estimands involving the joint distribution of potential outcomes to enable more nuanced treatment evaluation and selection. In this article, we propose a novel framework for identifying and estimating the joint distribution of potential outcomes using multiple experimental datasets. We introduce the assumption of transportability of state transition probabilities for potential outcomes across datasets and establish the identification of the joint distribution under this assumption, along with a regular full-column rank condition. The key identification assumptions are testable in an overidentified setting and are analogous to those in the context of instrumental variables, with the dataset indicator serving as "instrument". Moreover, we propose an easy-to-use least-squares-based estimator for the joint distribution of potential outcomes in each dataset, proving its consistency and asymptotic normality. We further extend the proposed framework to identify and estimate principal causal effects. We empirically demonstrate the proposed framework by conducting extensive simulations and applying it to evaluate the surrogate endpoint in a real-world application.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper shows how to identify the joint distribution of potential outcomes from multiple experiments under a transportability assumption on state transitions plus a rank condition, with a simple least-squares estimator.

read the letter

The core contribution is an identification strategy for the full joint distribution of potential outcomes rather than just marginals. They combine data from several experiments by assuming that the state transition probabilities for the potential outcomes transport across datasets, then use the dataset indicator as an instrument-like variable together with a full-column-rank condition on the resulting design matrix. This lets them recover the joint via a least-squares fit that they prove is consistent and asymptotically normal. They also carry the same logic over to principal causal effects and show simulations plus one real-data surrogate endpoint example. The IV-style overidentification and testability angle is a clean way to frame the assumptions. The transportability assumption on the transitions is the main load-bearing piece and will need careful justification in applications, but the stress-test confirms the derivation from that assumption to the estimator has no internal gaps or circularity. The math appears to hold up on its own terms. This is aimed at causal inference researchers who already work with multiple studies or need joint distributions for things like surrogate analysis. A reader who cares about identification under transportability will get something concrete to build on. It deserves a serious referee because the identification result is new on its face and the estimator is practical, even if the assumption will draw the usual scrutiny in review.

Referee Report

2 major / 2 minor

Summary. The paper claims to identify the joint distribution of potential outcomes using multiple experimental datasets under the transportability of state transition probabilities across datasets and a regular full-column rank condition on the design matrix (with the dataset indicator serving as an instrument). It establishes that the key assumptions are testable in an overidentified setting, proposes a least-squares-based estimator that is consistent and asymptotically normal, extends the framework to principal causal effects, and validates the approach via simulations and a real-world surrogate endpoint application.

Significance. If the identification result holds, the work would advance causal inference by enabling estimation of joint distributions of potential outcomes, which support more nuanced treatment evaluation and selection than marginal effects alone. The IV-style analogy with testable assumptions and the provision of an easy-to-use consistent estimator with asymptotic normality represent practical strengths. Credit is due for the explicit consistency and asymptotic normality results for the least-squares estimator as well as the empirical demonstrations through extensive simulations and a real-data application.

major comments (2)

[§3] §3 (Identification result): the claim that the joint distribution is identified under transportability of state transition probabilities plus the full-column rank condition requires an explicit step-by-step derivation showing how the observed conditional distributions map to the target joint probabilities; without this, it is difficult to verify that the rank condition is sufficient and non-circular.
[§5] §5 (Estimator and asymptotics): the consistency and asymptotic normality of the least-squares estimator are central to the practical contribution, yet the manuscript should state the precise form of the estimator (e.g., the design matrix construction) and the regularity conditions invoked for the asymptotic normality result.

minor comments (2)

[Introduction] The notation for potential outcomes and state transitions would benefit from a small illustrative example early in the introduction to improve accessibility.
[Simulations] Simulation tables should report standard errors or coverage probabilities alongside point estimates to allow readers to assess finite-sample performance more fully.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive comments. We address each major point below and will incorporate clarifications to improve the manuscript's clarity and rigor.

read point-by-point responses

Referee: §3 (Identification result): the claim that the joint distribution is identified under transportability of state transition probabilities plus the full-column rank condition requires an explicit step-by-step derivation showing how the observed conditional distributions map to the target joint probabilities; without this, it is difficult to verify that the rank condition is sufficient and non-circular.

Authors: We agree that an explicit step-by-step derivation would strengthen the presentation. In the revised manuscript, we will expand Section 3 with a detailed derivation that starts from the observed conditional distributions, applies the transportability assumption on state transition probabilities, and shows how the full-column rank condition on the design matrix (with dataset indicator as instrument) yields unique identification of the joint distribution of potential outcomes. Intermediate algebraic steps will be included to confirm that the rank condition is sufficient and non-circular. revision: yes
Referee: §5 (Estimator and asymptotics): the consistency and asymptotic normality of the least-squares estimator are central to the practical contribution, yet the manuscript should state the precise form of the estimator (e.g., the design matrix construction) and the regularity conditions invoked for the asymptotic normality result.

Authors: We appreciate this recommendation for greater precision. In the revised Section 5, we will explicitly define the least-squares estimator, including the construction of the design matrix that incorporates the dataset indicators. We will also state the regularity conditions (e.g., bounded moments, positive definiteness of the limiting matrix, and the maintained rank condition) under which consistency and asymptotic normality hold, with a brief reference to standard results for linear estimators. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper derives identification of the joint distribution of potential outcomes directly from the explicitly stated transportability assumption on state-transition probabilities across datasets together with the full-column-rank condition on the matrix of dataset indicators. The least-squares estimator is then constructed from this identified expression and its consistency and asymptotic normality are shown via standard arguments for linear estimators under the maintained assumptions; neither step reduces to a fitted parameter being relabeled as a prediction nor relies on a self-citation chain for its justification. The overidentification testability claim follows from the IV-style analogy with the dataset indicator as instrument and does not presuppose the target result. The derivation chain is therefore self-contained against the paper's own stated assumptions and external statistical theory.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the transportability assumption for state transition probabilities and the full-column rank condition; no free parameters or new entities are introduced in the abstract.

axioms (1)

domain assumption Transportability of state transition probabilities for potential outcomes across datasets
This assumption links the multiple experimental datasets to enable identification of the joint distribution.

pith-pipeline@v0.9.0 · 5700 in / 1195 out tokens · 49962 ms · 2026-05-22T18:58:02.653481+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Improving Treatment Effect Estimation in Trials through Adaptive Borrowing of External Controls
stat.ME 2026-04 unverdicted novelty 6.0

An adaptive influence-function framework selects optimal external control subsets to minimize MSE of the ATE estimator in RCTs and adds outcome calibration for better data use.