On the Impossibility of Specification Testing of Interference Models Based on Exposure Mappings

Chao Gao; Christopher Harshaw; Fredrik S\"avje; Yitan Wang

On the Impossibility of Specification Testing of Interference Models Based on Exposure Mappings

T0 review · 0 major / 2 minor · reviewed 2026-06-30 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Specification tests for exposure mapping models of interference cannot be uniformly consistent.

desk verdict The paper proves no test of exposure mapping models can beat random guessing in the worst case over separated alternatives, but shows consistency is possible once alternatives are narrowed. read the letter →

arxiv 2605.09726 v2 pith:SJZMNSRN submitted 2026-05-10 math.ST stat.MEstat.TH

Chao Gao , Christopher Harshaw , Fredrik Sävje , Yitan Wang This is my paper · ORCID

classification math.STstat.MEstat.TH

keywords specificationtestingexposuremappingsinterferencecausalinferencehypothesisuniformconsistencyerrorrates

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proves that any test of an interference model based on exposure mappings has worst-case Type I and Type II error rates that sum to one. This bound is the same as that of a test which ignores the data entirely and rejects the null at random. The result holds for every exposure mapping, every sample size, bounded outcomes, and alternatives that are as far from the null as possible. A sympathetic reader would conclude that useful specification tests must impose further restrictions on the possible alternatives beyond those in the exposure mapping itself.

What carries the argument

The worst-case error rate sum of one for specification tests under exposure mappings, which prevents uniform consistency.

What would settle it

Existence of a specification test where the sum of worst-case Type I error and worst-case Type II error is strictly less than one under the stated conditions of bounded outcomes and maximally separated alternatives.

Watch

Extended reading notes

Core claim

The central claim is that the worst-case Type I and Type II error rates must sum to one for any specification test of exposure mapping models. This rules out the existence of a uniformly consistent test. The result applies to all exposure mappings, all sample sizes, uniformly bounded outcomes, and alternatives maximally separated from the null.

Load-bearing premise

Alternatives to the null model are maximally separated from it.

Editorial extensions

If this is right

Existing specification tests suffer from poor power against some model violations.
Tests can only attain power by restricting the set of alternatives considered.
The paper provides an example of a consistent test when restricting to no-interference versus a network linear-in-means model.
Any test will leave some relevant departures from the model undetectable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Researchers need to specify concrete alternative models to make testing informative.
This impossibility may suggest similar limits in other causal models defined by summary statistics or mappings.
Future work could explore the minimal restrictions needed on alternatives to allow consistent testing.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The main takeaway is that specification testing for exposure mapping models is impossible in a strong sense: for any test, the worst-case Type I and Type II errors sum to one across all sample sizes and all exposure mappings when outcomes are bounded and alternatives are maximally separated from the null. This matches the error rate of just ignoring the data and flipping a coin, so no uniformly consistent test exists without further restrictions.

What is new is the clean minimax argument establishing this bound for the entire class of exposure mappings. The paper also supplies a positive counterpart by constructing a uniformly consistent test that separates a no-interference null from a network linear-in-means alternative. That example is useful because it demonstrates exactly where the impossibility can be escaped.

The work is clear on why existing tests have low power and why that is not an accident of poor design. The decision-theoretic framing keeps the argument direct and avoids extra assumptions.

The main soft spot is the reliance on maximally separated alternatives. If actual model violations in data are not that far from the null, the bound may not constrain practical tests as tightly, and some power could remain. The result is also stated for fixed exposure mappings, which leaves open whether data-dependent mappings change the picture. These conditions are stated explicitly, so they are not hidden, but they do limit how broadly the impossibility applies.

This paper is for researchers working on causal inference under interference, particularly those designing or evaluating specification tests in network experiments. A reader who needs to know the fundamental limits on testing exposure mappings will find it directly useful. It deserves serious peer review because the central negative result is sharp, the positive illustration balances it, and the topic matters for applied work in the area. I would send it out.

Referee Report

0 major / 2 minor

Summary. The paper claims that specification testing for interference models based on exposure mappings is impossible in a minimax sense: for any test, the worst-case Type I error rate plus the worst-case Type II error rate equals 1 for all exposure mappings, all sample sizes, uniformly bounded outcomes, and alternatives maximally separated from the null. This rules out uniformly consistent tests and is achieved by a naive test that discards data and rejects at random. The authors also construct a uniformly consistent test under the restricted alternative of distinguishing no-interference from a network-linear-in-means model.

Significance. If the result holds, it is significant because it provides a decision-theoretic explanation for the poor power of existing tests and shows that informative specification testing requires additional restrictions on the alternative beyond the exposure mapping itself. The generality across all exposure mappings and sample sizes, together with the sharpness of the bound (achieved by a data-ignoring procedure), strengthens the negative result. The positive illustration under a restricted alternative is a constructive strength that offers practical guidance for when consistent testing is feasible.

minor comments (2)

[Abstract] The abstract would benefit from a brief parenthetical clarification of how 'maximally separated' is formalized (e.g., via total variation or outcome distribution distance) to help readers immediately gauge the scope of the impossibility.
Consider adding one sentence in the introduction contrasting the general impossibility with the restricted positive result to improve readability for readers focused on applications.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary of the manuscript, recognition of its significance, and recommendation for minor revision. No specific major comments were provided in the report, so we have no points requiring point-by-point rebuttal or revision at this stage.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity

full rationale

The central claim is a decision-theoretic minimax result: for any test, worst-case Type I + Type II error equals 1 under maximally separated alternatives (with bounded outcomes and fixed exposure mappings). This follows directly from standard arguments on error rates and does not reduce to any fitted parameter, self-definition, or self-citation chain. The negative result is self-contained and externally falsifiable via the stated separation condition; no load-bearing step collapses to its own inputs.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The impossibility result rests on standard assumptions from causal inference and statistical decision theory; no free parameters, new entities, or ad-hoc axioms beyond bounded outcomes and fixed mappings are introduced.

assumptions (2)

domain assumption Outcomes are uniformly bounded
Explicitly stated as holding for the negative result.
domain assumption Randomized experiment with known exposure mappings
Core setup of the models under test.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Impossibility of Specification Testing of Interference Models Based on Exposure Mappings." pith.science (2026). https://pith.science/paper/SJZMNSRN

@misc{pith2026260509726,
  author       = {Pith},
  title        = {Pith review of: On the Impossibility of Specification Testing of Interference Models Based on Exposure Mappings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SJZMNSRN}},
  note         = {Machine review of arXiv:2605.09726}
}

read the original abstract

Researchers use interference models based on exposure mappings to facilitate estimation of causal effects in randomized experiments with interference. To test the veracity of such models, researchers can use specification tests that aim to detect departures from the stipulated model. However, existing tests suffer from poor power and are often unable to detect important model violations. The main result in this paper is to show that the specification testing problem for exposure mapping models is inherently difficult, and the poor power of existing tests is inescapable. In particular, the worst-case Type I and Type II error rates must sum to one for any specification test of such models, ruling out the existence of a uniformly consistent test. This is the worst-case overall error rate achieved by a naive test that discards all data and arbitrarily rejects the null at random; the testing problem is in this sense impossible. This negative result holds true for all exposure mappings, all sample sizes, for uniformly bounded outcomes, and for alternatives that are maximally separated from the null. While some tests can detect some type of departures from the null model, there will always be relevant departures from the null that are undetectable. Informative specification tests must therefore restrict the alternative model against which they seek to attain power for, beyond the restrictions imposed by the exposure mappings alone. We illustrate this by providing a uniformly consistent test for differentiating no-interference from a network-linear-in-means model.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

A General Exposure-Mapping-Agnostic Framework for Causal Inference under Interference
stat.ME 2026-07 accept novelty 7.5 of 10

A new class of linear weighting estimators provides unbiased, asymptotically normal inference for causal effects in two-stage cluster randomized experiments with cross-cluster interference.
A Design-Based Minimax Theory for Network Experiments
math.ST 2026-08 conditional novelty 7.0 of 10

The minimax risk of any network experiment under arbitrary neighborhood interference is a function of the conflict graph of observable exposures, with rates bounded by the graph's independence number, critical degree,...

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages · cited by 2 Pith papers

[1]

Aronow, P. M. (2012). A general method for detecting interference between units in randomized experiments. Sociological Methods & Research,41(1), 3–16. Aronow, P. M., & Samii, C. (2017). Estimating average causal effects under general interference.Annals of Applied Statistics,11(4), 1912–1947. Athey, S., Eckles, D., & Imbens, G. W. (2018). Exact p-values ...

work page arXiv 2012
[2]

Recall that the separation functional is defined as g(y) = 1 n n∑ i=1 β2 i,3

The restriction 16 di≥1is merely a technicality, because the null and alternative models trivially coincide for units withd i = 0, so the null is known to be true a priori for units with no neighbors. Recall that the separation functional is defined as g(y) = 1 n n∑ i=1 β2 i,3 . Our goal is to construct a consistent estimator of the separation valueg(y)fo...

work page 2022

Pith tools

Reviewed June 30, 2026 · model on record in the stance chip above.

[1] [1]

Aronow, P. M. (2012). A general method for detecting interference between units in randomized experiments. Sociological Methods & Research,41(1), 3–16. Aronow, P. M., & Samii, C. (2017). Estimating average causal effects under general interference.Annals of Applied Statistics,11(4), 1912–1947. Athey, S., Eckles, D., & Imbens, G. W. (2018). Exact p-values ...

work page arXiv 2012

[2] [2]

Recall that the separation functional is defined as g(y) = 1 n n∑ i=1 β2 i,3

The restriction 16 di≥1is merely a technicality, because the null and alternative models trivially coincide for units withd i = 0, so the null is known to be true a priori for units with no neighbors. Recall that the separation functional is defined as g(y) = 1 n n∑ i=1 β2 i,3 . Our goal is to construct a consistent estimator of the separation valueg(y)fo...

work page 2022