A Practical Introduction to Regression-based Causal Inference in Meteorology (I): All confounders measured

Caren Marzban; Michael Richman; Nicholas Bond; Yikun Zhang

REVIEW 4 major objections 5 minor 2 references

A Practical Introduction to Regression-based Causal Inference in Meteorology (I): All confounders measured

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Propensity-score matching yields a positive causal effect of shortwave radiation on surface potential temperature, while regression adjustment gives a negative one, a reversal the paper reads as Simpson's paradox.

desk verdict A solid teaching paper with a real but fixable blind spot about spatial interference; worth refereeing. read the letter →

arxiv 2506.18808 v3 pith:HDOU477F submitted 2025-06-23 stat.AP

Caren Marzban , Yikun Zhang , Nicholas Bond , Michael Richman This is my paper · ORCID

classification stat.AP MSC 62D2062P12

keywords causalinferencematchingpropensityscorepotentialoutcomesgriddeddatameteorologySimpson'sparadoxNorthAmericanRegionalReanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to bring causal inference into meteorology by showing that propensity-score matching, built on regression, can estimate an average treatment effect from gridded observational data without temporal information. Using North American Regional Reanalysis fields for two January months, it treats downward shortwave radiation flux, dichotomized at its median, as the treatment; surface potential temperature as the outcome; and geopotential height at 875, 900, and 925 hPa as confounders. The central result is that matching yields a positive causal effect of radiation on temperature, whereas a standard regression that adjusts for the same confounders gives a negative effect. The paper interprets this reversal as an instance of Simpson's paradox. The broader point is that regression-based causal inference applied to gridded data is one of the novel features of this paper and its companion, and the accompanying R code is meant to be a reusable template for meteorologists.

What carries the argument

The central mechanism is propensity-score matching within the potential-outcomes framework. The propensity score is the probability of receiving the treatment given the confounders, here estimated by a probit treatment model; matching on this scalar is intended to balance the covariate distributions between treatment and control groups, and the resulting weights enter a weighted-least-squares outcome model that includes treatment–confounder interactions. The average treatment effect is then obtained by g-computation. The machinery also includes balance checks, specifically standardized mean differences and two-sample Q-Q plots, that justify the pseudo-population.

What would settle it

Re-run the matching analysis with additional plausibly confounding fields such as cloud fraction, specific humidity, albedo, or land–sea mask, or with the geopotential levels varied; if the matching average treatment effect ceases to be positive, or if a natural experiment that manipulates shortwave radiation while holding height fields fixed fails to raise potential temperature, the paper's central causal claim would be contradicted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is an empirical demonstration together with a methodological framing: when the treatment is dichotomized at its median and the confounders are geopotential heights at the 875, 900, and 925 hPa levels, matching on the propensity score balances the treatment and control groups, and the resulting average treatment effect estimates are positive across ten random subsamples of 10,000 grid points. The simple difference in means is also positive but larger; the confounder-adjusted regression estimates are negative in every trial. The paper concludes that downward shortwave radiation flux has a direct causal effect on surface potential temperature once geopotential height is accounted for, and that the adjusted estimate reverses sign because of Simpson's paradox.

Load-bearing premise

The estimate is causal only if geopotential height at 875, 900, and 925 hPa captures every confounder of the radiation–temperature relationship, an assumption the paper openly states is untestable.

Editorial extensions

If this is right

If the claim is right, regression-based causal inference can be applied to gridded fields without time-series information, giving meteorologists a way to estimate causal effects from reanalysis and model output.
The sign reversal implies that a conventional adjusted regression in such settings can be actively misleading, and matching or an equivalent balancing step is a necessary check before drawing physical conclusions.
The paper's template, comprising a treatment model, matching, an outcome model with interactions, and g-computation, can be rerun on any gridded variable pair using public data and the provided R code.
The demonstration suggests that categorical treatment definitions, such as the median split used here, are workable for spatial fields, though the threshold choice itself remains arbitrary.
The paper provides a concrete path for the meteorology community to shift from purely associative regression to causal-effect estimation without needing graphical models or time-series methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Implicit in the paper but not developed: the analysis treats grid points as independent units; if spatial interference is strong, the matching estimate could absorb those spillovers, so extending the method to account for spatial structure would test how much of the positive effect survives.
A testable extension would be to vary the dichotomization threshold of the treatment; if the positive causal effect flips sign for a threshold other than the median, the result is partly an artifact of that arbitrary choice.
The paper defers the choice of the three geopotential levels to the companion article; a direct sensitivity analysis adding other pressure levels or other plausible confounders such as cloud fraction would show whether the positive estimate is robust.
The Simpson's-paradox framing suggests that similar sign reversals may occur in other meteorological regression studies where the treatment is spatially correlated with a neglected confounder, and matching provides a way to diagnose them.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this is a teaching paper, not a research contribution, and it is mostly a good one. The novel bit is packaging regression-based causal inference (matching, propensity scores, g-computation) for meteorologists using NARR gridded fields, with public data and clear code references. The Appendix B derivations are standard and correctly done, and the Simpson's paradox example is genuinely useful.

Soft spots, in order of importance. The spatial-interference problem is real and underplayed. The units are grid points of a reanalysis field; shortwave radiation at one cell plausibly affects potential temperature at nearby cells through advection, clouds, boundary-layer processes. The random subsampling of 10,000 grid points addresses spatial correlation for sampling the estimate, but it does not remove interference. So the matched contrast may not identify the stated unit-level ATE. This is a checkable assumption and the authors don't discuss it in the main text. For a pedagogical paper, this matters because students will internalize the workflow; a one-paragraph caveat pointing to the spatial causal inference literature (which they do cite at the end) would do. The treatment model is described as least-squares-estimated probit; standard probit is MLE. That's a minor description error but worth fixing. The confounder choice is deferred to the companion paper, so the NUCA assumption rests on three geopotential heights selected elsewhere; the authors are transparent about this, and it's fine for a tutorial, but it means the substantive result is illustrative. The R code is promised but not visible in the arXiv text—make sure the supplement actually ships.

Citation pattern looks fine. The self-citation to the companion paper is appropriate. The claim that applying regression-based causal inference to gridded data is 'one of the novel features' is a bit strong given the existing spatial causal inference literature, but they do acknowledge that literature in the discussion.

Who is this for: meteorologists and graduate students wanting an approachable entry into causal inference. It deserves a serious referee; the issues are addressable in revision. I'd accept it for review.

Referee Report

4 major / 5 minor

Summary. This paper is a pedagogical introduction to regression-based causal inference for meteorologists, using matching on the propensity score to estimate the average treatment effect (ATE) of downward shortwave radiation flux on surface potential temperature, with geopotential heights at three pressure levels as confounders. The data are NARR reanalysis grid points from two January months. The authors present the potential-outcomes framework, define the propensity score, matching, balance diagnostics, and g-computation in simple terms, and illustrate the workflow in R with the MatchIt package. The main substantive result, reported in Section 4, is that the matched ATE is positive and differs from both the simple difference in means and the negative confounder-adjusted estimate, which they interpret as evidence for a direct causal effect after accounting for confounding.

Significance. If the methodological template is sound, the paper fills a real gap: it offers meteorologists an accessible, reproducible entry point into non-temporal causal inference, with public data, R code, and standard balance diagnostics. The Appendix B derivations of the decomposition in Eq. (3) and the balancing-score properties of the propensity score are standard and appear correctly carried out. The explicit use of g-computation and treatment-confounder interactions in the outcome model is a strength, as is the visual balance assessment via Q-Q plots. However, the causal interpretation of the headline estimate rests on assumptions that are not adequately checked or even stated in the main analysis, most importantly the no-interference (SUTVA) component and the unmeasured-confounding assumption. The paper is therefore useful as a tutorial but the substantive causal claim is not yet established without additional discussion and diagnostics.

major comments (4)

[Sections 3 and 4] The causal interpretation of the matched ATE requires the Stable Unit Treatment Value Assumption, but the paper never states or defends it. The units are contiguous grid points of a reanalysis field; Section 3 says 'all correlations between variables are strictly spatial in nature,' and Section 4 claims that random subsampling 'minimiz[es] the effect of spatial dependence.' Random subsampling does not remove interference: if downward shortwave radiation at one grid point affects potential temperature at a neighboring grid point through advection, boundary-layer processes, or cloud-radiation feedbacks, then the matched contrast is not E[Y(1)-Y(0)] for a well-defined unit-level intervention. This is a checkable, plausibly violated assumption, and the manuscript should either discuss it explicitly, restrict the inferential claim, or employ spatial causal-inference tools such as those cited in the Conclusions.
[Section 2d, Eq. (8)] The treatment model is specified as a probit model, but the text says the parameters are 'estimated via the least-squares criterion.' Probit regression is normally estimated by maximum likelihood, not by least squares; if the authors instead mean a nonlinear least-squares implementation, that is nonstandard and needs justification. This matters because the propensity score is the central quantity for matching, and an incorrect treatment-model estimator can affect the matching weights and the resulting ATE.
[Section 3, Confounding variables] The choice of exactly three confounders (geopotential heights at 875, 900, and 925 hPa) is deferred entirely to the companion paper: 'The reason for selecting only these three confounders is explained in the sister article.' Since the No Unmeasured Confounding Assumption of Eq. (2) requires that all confounders be measured, the identifying assumption is not justified within this manuscript. The tutorial value is preserved if the paper explicitly states that the demonstration is conditional on the companion paper's variable selection, but the current wording asks the reader to accept a load-bearing assumption on faith.
[Section 4, Figures 4-6] The ATE estimate relies on adequate overlap between the treatment and control propensity-score distributions, but no overlap or positivity diagnostics are reported. The Q-Q plots in Figure 5 show that balance is achieved only in limited ranges of the confounders, and the SMD values for individual confounders remain nonzero even after matching. Without an examination of common support, the g-computation estimate may extrapolate into regions where few or no matched control units exist, and the positive matched ATE in Figure 6 may reflect such extrapolation rather than a causal effect.

minor comments (5)

[Section 4, Figure 7] The text describing the Simpson's-paradox figure appears to swap the variable names: it refers to 'the outcome (downward shortwave radiation flux) versus the treatment (potential temperature),' but earlier the treatment is downward shortwave radiation and the outcome is potential temperature. The axes and caption should be checked and corrected.
[Appendix B] The appendix labels equations as (B1), (B2), etc., but the text refers to '(A5)', '(A6)', '(A7)', and '(A12)'; these cross-references should be updated to the B-series labels.
[Section 3] The statement that 'all correlations between variables are strictly spatial' is slightly misleading because data from two separate January months are pooled, so correlations also reflect between-month differences; this is likely a minor wording issue.
[Section 2a] The sentence 'the ATE cannot be estimated because only one of the two potential outcomes is observed' is too strong in context, since the rest of the section shows that ATE can be estimated under consistency, ignorability, and positivity; rewording to 'cannot be estimated without further assumptions' would be more precise.
[References] There are several typographical errors in the reference list, including 'SBN 978-0-300-25168-5' for ISBN, and malformed URLs for Deng (2021), Marzban et al. (2025), and Mesinger et al. (2006); these should be corrected for a journal submission.

Circularity Check

1 steps flagged · score 2.0 of 10

No meaningful circularity: the ATE is estimated from external NARR data by standard matching/g-computation; the only self-referential element is that the choice of the three pressure-level confounders is deferred to the authors' own companion paper, which is a support gap rather than a circular reinsertion of the target effect.

self citation load bearing [Section 3 (Data), confounder selection paragraph]
"The reason for selecting only these three confounders is explained in the sister article by the authors (Marzban et al. 2025). Suffice it to say that the method employed in that article requires only these three variables, and the present article uses the same confounders so that the results of the two papers/methods can be compared."

The paper's causal claim depends on the No Unmeasured Confounding Assumption (Eq. 2) holding for X = geopotential height at 875, 900, and 925 hPa. The selection of exactly those three levels is the substantive content of X, and that selection is not derived or independently justified here; it is delegated to the authors' own companion paper. This makes a load-bearing premise of the causal analysis rest on a self-citation rather than on an external, code-verified, or independently falsifiable result. However, this does not make the ATE arithmetic circular: the treatment model uses only confounders, the outcome model is fit to the observed treatment/outcome data, and the target effect is not fed back as an input.

full rationale

The derivation chain in Sections 2 and 4 is self-contained with respect to standard causal-inference machinery. Equations (1) through (7) are textbook potential-outcome identities; the propensity score is estimated from confounders alone; matching weights are constructed without using the outcome; and the outcome model is a weighted regression whose g-computed average contrast estimates ATE. The conclusion that downward shortwave radiation has a positive direct effect on surface potential temperature is an empirical estimate from NARR data, not a quantity inserted by definition. The paper explicitly acknowledges that the unconfoundedness assumption is untestable, and it flags the limitation that only three pressure levels are used. The only mildly self-referential load-bearing element is the deferral of the three-level confounder choice to the authors' own companion paper; this is a real support gap but it does not reduce the central estimate to its inputs. Spatial interference and SUTVA concerns are validity threats, not circularity: random subsampling of grid points does not remove interference, but that is an assumption-violation critique rather than a demonstration that the paper's equations are equivalent to its conclusions. Accordingly, the circularity burden is low, with one minor self-citation that slightly raises the score above zero.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The paper's causal estimate rests on standard potential-outcomes axioms rather than new theory. The main free choices are data-driven: the median treatment split, the square-root transform, and the specific pressure levels borrowed from a companion paper. No invented entities appear.

free parameters (5)

Median dichotomization threshold for treatment = Median of square-root transformed downward shortwave radiation flux across the spatial domain and two dates (not…
Treatment is defined as above vs below this sample median; Section 3 states the choice is 'arbitrary and not based on a physical criterion,' so the ATE depends on it.
Confounder set: geopotential heights at 875, 900, 925 hPa = NARR levels 24, 25, 26
Section 3 says the selection is explained in the companion paper and is used here for comparability; no meteorological justification is provided in this paper.
Square-root transform of treatment = sqrt(flux)
Applied so the treatment histogram is 'more bell-shaped,' a data-dependent preprocessing choice that changes the scale of the median threshold.
Treatment model (probit) coefficients = Not reported
The propensity score depends on coefficients fitted from the treatment and confounders; any misspecification affects balance and ATE.
Outcome model coefficients = Not reported
ATE via g-computation depends on weighted least-squares coefficients and treatment-confounder interactions in Eq. (9).

assumptions (6)

domain assumption Consistency assumption: observed outcome equals the potential outcome under the assigned treatment (Eq. 1).
Required to link potential outcomes to observed data; standard but untestable.
domain assumption No Unmeasured Confounding Assumption: (Y(1),Y(0)) independent of A given X (Eq. 2).
The causal interpretation stands or falls on this untestable premise; the paper acknowledges it and notes geopotential height must account for all confounding.
domain assumption Positivity/overlap: every unit has positive probability of both treatment levels given X.
Needed for matching and g-computation; not explicitly discussed in the paper.
domain assumption Stable unit treatment value / no interference between grid points.
Spatial units are treated independently although meteorological variables are spatially correlated; this is not addressed.
domain assumption Causal structure (radiation causes potential temperature; geopotential height is a confounder) is assumed a priori.
Section 3 provides a physical narrative but no test; the methods are not designed to discover causal structure.
standard math Standard probability laws (iterated expectation, conditional independence manipulations) in Appendix B.
Used to prove the propensity score is a balancing score and that conditioning on PS preserves conditional independence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Practical Introduction to Regression-based Causal Inference in Meteorology (I): All confounders measured." pith.science (2026). https://pith.science/paper/HDOU477F

@misc{pith2026250618808,
  author       = {Pith},
  title        = {Pith review of: A Practical Introduction to Regression-based Causal Inference in Meteorology (I): All confounders measured},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HDOU477F}},
  note         = {Machine review of arXiv:2506.18808}
}

read the original abstract

Whether a variable is the cause of another, or simply associated with it, is often an important scientific question. Causal Inference is the name associated with the body of techniques for addressing that question in a statistical setting. Although assessing causality is relatively straightforward in the presence of temporal information, outside of that setting - the situation considered here - it is more difficult to assess causal effects. The development of the field of causal inference has involved concepts from a wide range of topics, thereby limiting its adoption across some fields, including meteorology. However, at its core, the requisite knowledge for causal inference involves little more than basic probability theory and regression, topics familiar to most meteorologists. By focusing on these core areas, this and a companion article provide a steppingstone for the meteorology community into the field of (non-temporal) causal inference. Although some theoretical foundations are presented, the main goal is the application of a specific method, called matching, to a problem in meteorology. The data for the application are in public domain, and R code is provided as well, forming an easy path for meteorology students and researchers to enter the field.

Figures

Figures reproduced from arXiv: 2506.18808 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 1 canonical work pages

[1]

Abadie, A., and G. W. Imbens, 2016: Matching on the estimated propensity score. Econo- metrica, 84.2, 781-807. Athey, S., and G. W. Imbens, 2017: The Econometrics of Randomized Experiments, in Handbook of Economic Field Experiments , 1, 73-140. Camps-Valls, G., A. Gerhardus, U. Ninad, G. Varando, G. Martius, E. Balaguer-Ballester, R. Vinuesa, E. Diaz, L. ...

arXiv 2016
[2015]

Imbens, G. W. and Y. Xu, 2024: LaLonde (1986) after Nearly Four Decades: Lessons Learned. Available at SSRN: https://ssrn.com/abstract=4849285 or http://dx.doi.org/10.2139/ssrn.4849285 Kretschmer, M., S. V. Adams, A. Arribas, R. Prudden, N. Robinson, E. Saggioro, and T. G. Shepherd, 2021: Quantifying Causal Pathways of Teleconnections, BAMS, E2247–E2263. ...

work page Pith review arXiv 1986

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Abadie, A., and G. W. Imbens, 2016: Matching on the estimated propensity score. Econo- metrica, 84.2, 781-807. Athey, S., and G. W. Imbens, 2017: The Econometrics of Randomized Experiments, in Handbook of Economic Field Experiments , 1, 73-140. Camps-Valls, G., A. Gerhardus, U. Ninad, G. Varando, G. Martius, E. Balaguer-Ballester, R. Vinuesa, E. Diaz, L. ...

arXiv 2016

[2] [2015]

Imbens, G. W. and Y. Xu, 2024: LaLonde (1986) after Nearly Four Decades: Lessons Learned. Available at SSRN: https://ssrn.com/abstract=4849285 or http://dx.doi.org/10.2139/ssrn.4849285 Kretschmer, M., S. V. Adams, A. Arribas, R. Prudden, N. Robinson, E. Saggioro, and T. G. Shepherd, 2021: Quantifying Causal Pathways of Teleconnections, BAMS, E2247–E2263. ...

work page Pith review arXiv 1986