REVIEW 3 major objections 4 minor 55 references
Estimation with missing not at random binary outcomes via exponential tilts
T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Exponential tilt models identify missing-not-at-random binary outcomes from the observed marginal.
desk verdict A genuinely new identifiability condition and a clean DR theorem, but the Waterbirds real-data claim rests on an unverified assumption that the paper waves through. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the exponential tilt model (1.2), a two-component exponential-family shift $\omega(x,y)=\exp(\alpha_y+\beta_y^\top T(x))$ between the $R=1$ and $R=0$ subpopulations. This single specification carries the whole argument: it turns the observed covariate distribution into a two-component tilted mixture (eq. (2.2)), which is matched to the data by minimizing Kullback-Leibler divergence under a normalization constraint (eqs. (3.1)-(3.2)); it delivers the propensity score formula in Lemma 3.1; and its log-odds difference $\log\omega(x,1)-\log\omega(x,0)=(\alpha_1-\alpha_0)+(\beta_1-\beta_0)^\top T(x)$ provides the outcome regression in Lemma 3.2. The double-robustness theorem then rests on those two building blocks.
What would settle it
Construct data in which the $R=1$ log odds has exactly the forbidden form, for example by taking two Gaussian components whose means are swapped as in the paper's Section 2 counterexample. Then run Algorithm 1 from multiple starting points: the objective (3.2) has multiple optima, so the recovered importance weights will differ across runs, directly contradicting the claim of unique identification.
Extended reading notes
Core claim
Under the exponential tilt model (1.2), the joint law of $(X,Y)$ in the missing domain is obtained from the observed domain by multiplying each outcome-specific density by $\exp(\alpha_y+\beta_y^\top T(x))$; therefore the observed covariate marginal satisfies eq. (2.2). The paper shows that Assumption 2.1 selects a unique set of tilt parameters from that marginal, making the importance weights, and hence all mean functionals, identifiable. The mechanism is complete: the missingness propensity score is $\mathrm{P}(R=1\mid X,Y)=\mathrm{P}(R=1)/(\mathrm{P}(R=1)+\omega(X,Y)\mathrm{P}(R=0))$, the conditional outcome probability under missingness is obtained from a log odds ratio $(\alpha_1-\alpha_0)+(\beta_1-\beta_0)^\top T(x)$, and the doubly robust estimators are asymptotically unbiased if either the tilt model is correct or the outcome regression plus log-odds-ratio model is correct (Theorem 3.4). The paper additionally reports that the fitted linear combination $(\beta_0^\top T(X),\beta_1^\top T(X))$ acts as an estimated missingness controlling variable, and that in the Waterbirds study this suffices for unsupervised transfer learning at accuracy comparable to a labeled target oracle.
Load-bearing premise
The entire identification rests on Assumption 2.1: the true log odds $\log\{\mathrm{P}(Y=1\mid X=x,R=1)/\mathrm{P}(Y=0\mid X=x,R=1)\}$ must not be representable by the forbidden log-ratio-of-exponentials expression, otherwise two different tilt parameter values explain the observed covariate marginal equally well.
Editorial extensions
If this is right
- Mean functionals $\mathrm{E}[\tau(X,Y)]$ and $\mathrm{E}[\tau(X,Y)\mid R=0]$ become estimable from observed data alone whenever the tilt parameters are identified.
- A user no longer needs to pre-specify a nonresponse instrument or shadow variable; the procedure returns an estimated missingness controlling variable $(\beta_0^\top T(X),\beta_1^\top T(X))$ as a byproduct.
- Unsupervised transfer learning when the outcome-given-covariate relationship differs between source and target can be carried out without labeled target samples, as long as the source-population log odds satisfy Assumption 2.1.
- The doubly robust estimator keeps asymptotic unbiasedness under misspecification of the tilt model provided the log-odds-ratio and outcome regression are correct, a robustness property shown in Theorem 3.4.
- A computationally stable exponentiated-gradient algorithm solves the constrained KL matching problem, outperforming a profile empirical likelihood alternative in the paper's simulations.
Reading between the lines
- The paper asserts Assumption 2.1 is in principle verifiable but gives no practical test; fitting the observed log odds nonparametrically and checking whether it ever enters the forbidden log-ratio family would be a natural way to make the method safer in practice.
- Because the estimated missingness controlling variable is only identified up to the choice of summary $T(x)$, the surrogate may recover a transformation of the true controlling variable rather than the variable itself; this is enough for prediction but not automatically for causal interpretation.
- The Waterbirds experiment suggests the DR estimator can become unstable when the product of estimated residuals and covariate directions is unbounded, so a clipped or regularized variant of the DR loss is a natural extension.
- The same tilt machinery should extend to multi-category outcomes or to $T(x)$ chosen as a neural embedding, as in the Waterbirds experiment, but the identifiability condition would need to be re-derived for each new family.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies estimation of mean functionals when binary outcomes are missing not at random. The central assumption is that the importance weight mapping the observed (R=1) joint distribution to the missing (R=0) joint distribution has the exponential-tilt form exp(alpha_y + beta_y' T(x)). Under a new sufficient identifiability condition (Assumption 2.1), the tilt parameters are claimed identifiable from the marginal distribution of X conditional on R=0 and R=1. The paper proposes a constrained KL-matching estimator solved by an exponentiated-gradient algorithm, constructs importance-weighted and doubly robust estimators for mean functionals, and proves a double-robustness property. Synthetic normal-mixture experiments support the theory, and a Waterbirds transfer-learning experiment is used to argue that the method mitigates spurious correlation without observing the missingness-controlling variable.
Significance. If the claims hold, the paper offers a genuinely useful direction: it weakens the usual requirements of nonresponse instruments or shadow variables for MNAR binary outcomes, and it provides a double-robustness result that can tolerate certain misspecifications of the tilt model. The internal theoretical core is largely consistent: Lemma C.4, Lemma 3.3, and Theorem 3.4 are coherent, the bias decompositions are explicit, and the synthetic experiments match the claimed behavior. The paper also ships code and evaluates the transfer-learning claim on a held-out target-test split with oracle benchmarks. However, the practical usefulness of the method rests on Assumption 2.1, and the only real-data demonstration does not verify that assumption. In addition, the estimation algorithm lacks convergence guarantees, so the statistical claims are not fully supported.
major comments (3)
- [Section 5, Assumption 2.1] The sentence 'The functional structure of single hidden layer neural network satisfies the Assumption 2.1' does not establish the identifiability condition. The forbidden family in Assumption 2.1 contains every affine function, since taking mu0=mu1 and delta0=delta1=0 gives log odds equal to mu2 + delta2' t. A single-hidden-layer neural network is a universal approximator on compact covariate support, so the network class is not structurally disjoint from the forbidden family. What would need to be checked is whether the fitted log-odds logit(eta1(x)) learned from the Waterbirds source data is not close to the forbidden family on the support of X. No such check is reported. Because identification of theta through equation (2.2) is entirely conditional on Assumption 2.1, the reported IW and OR accuracy gains in Figure 3 are not established as identifiable quantities; they could reflect an arbitrary selection among near-equivalent theta values.
- [Section 3.1, Algorithm 1] No convergence, stationarity, or local-minimum guarantee is given for the exponentiated-gradient algorithm. The objective in (3.2) is nonconvex in theta because of the exponential terms and the estimated nuisance function eta1, and the Lagrangian update in Algorithm 1 is heuristic. Lemma 3.3 and Theorem 3.4 are asymptotic bias statements that assume thetahat -> theta* and xihat -> xi*, but no theorem establishes these limits for the estimator produced by Algorithm 1. Please either provide conditions under which Algorithm 1 converges to a consistent estimator, or explicitly state the consistency claim as an assumption/conjecture and adjust the wording of Theorem 3.4 accordingly.
- [Section 2, Assumption 2.1] The paper states that Assumption 2.1 is 'in principle verifiable from the observed dataset', but it offers no practical verification procedure. The quadratic-log-odds example is only a sufficient instance, not a test of non-membership in the forbidden family. Since this assumption is the sole route to identification of the tilt parameters, the manuscript should either provide a falsifiable diagnostic for the assumption (for example, a goodness-of-fit test against the forbidden family on the R=1 sample) or clearly frame the real-data analysis as illustrative under an unverified condition. Without this, the identification claim in the Waterbirds study is not supported.
minor comments (4)
- [Section 5, Eq. (5.1)] In the definition of the outcome-regression loss, the second term should be (1-p) times the loss at Y=0; as written, ell(X,p;beta) = p ell(X,1;beta) + (1-p) ell(X,1;beta) is not the expected loss under a probability p.
- [Algorithm 1, line 9] The indicator condition for updating eta2 has a missing closing parenthesis: 'I{gn(theta(t) < -epsilon}' should read 'I{gn(theta(t)) < -epsilon}'.
- [Section 4, paragraph after Eq. (4.1)] The text identifies the target as mu0 = pi_{1|r} = 0.6; this should be pi_{1|0} = 0.6, since mu0 = E[Y | R=0].
- [Appendix C.2.4, proof of Theorem 3.4] The proof contains a duplicated, incomplete sentence: 'leading to bias(hat(mu)) = 0. This establishes (i). To establish (ii)' appears before the decomposition in (C.8), and should be removed.
Circularity Check
No significant circularity: the tilt parameters are estimated from observed R=0 marginals and R=1 labels, and the target-test evaluation is held out; the main theorems are proved in-appendix from stated assumptions.
full rationale
Walking the derivation chain: identifiability (Assumption 2.1 plus Lemma C.4) is a sufficient condition proved from the model equations, not imported from prior work. Estimation in (3.2) minimizes KL(p(x|R=0), sum_y exp(alpha_y+beta_y^T t)p(x,y|R=1)) using only observed X in R=0 and the fitted source classifier eta_1; no missing Y values from R=0 enter the fitting objective. The IW and DR estimators are plug-in constructions, and Theorem 3.4's bias expressions are derived in Lemmas 3.3 and C.7 rather than assumed. In the Waterbirds experiment, target-train is used only to estimate the tilt parameters and target-test is used only for evaluation, with oracle benchmarks; no fitted quantity is presented as a prediction of itself. The one potentially circular-looking sentence, 'The functional structure of single hidden layer neural network satisfies the Assumption 2.1,' is not a construction-level circularity but an unsupported identifiability claim: Assumption 2.1 is a non-membership condition on the true source log-odds, and the neural-network functional class is not structurally disjoint from the forbidden family (constant functions are in the family). That is a correctness or verification risk, not a reduction of the result to its inputs. Self-citations to Maity et al. (2023, 2024) supply terminology and background; they do not carry the derivation. The paper is self-contained with proofs in the appendix, so no circular step is exhibited.
Assumptions & free parameters
free parameters (4)
- tilt parameters θ = (α0, α1, β0, β1) =
estimated by Algorithm 1; β in R^2 (synthetic) and R^512 (Waterbirds)
- nuisance classifier parameters for η1 (random forest / single hidden layer NN) =
fitted on R=1 data; values not reported
- sufficient statistic T(x) =
T(x)=x in both experiments
- Algorithm 1 hyperparameters (B, ρ1, ρ2, ε, λ, tol, max_iter) =
B=5, eps=1e-3, tol=2e-3, lr=4e-3, max_iter=4000, reg=1e-5 (from code signature)
assumptions (6)
- domain assumption Exponential tilt model (2.1): p(x,y|R=0) = exp(α_y + β_y^T t) p(x,y|R=1)
- ad hoc to paper Assumption 2.1 (sufficient identifiability condition on the log odds)
- standard math Existence of densities for P(X|Y=y,R=r)
- domain assumption Correct specification of the nuisance model η1 for IW consistency and DR condition (ii)
- domain assumption Convergence of Algorithm 1 to the global minimizer of (3.2)
- domain assumption Subpopulation shift in Waterbirds: X ⊥ R | Y, A with A the background, and linear log-odds approximation by T(x)=x
Cite this review
Pith. "Pith review of Estimation with missing not at random binary outcomes via exponential tilts." pith.science (2026). https://pith.science/paper/36HWDM5O
@misc{pith2026250206046,
author = {Pith},
title = {Pith review of: Estimation with missing not at random binary outcomes via exponential tilts},
year = {2026},
howpublished = {\url{https://pith.science/paper/36HWDM5O}},
note = {Machine review of arXiv:2502.06046}
}
read the original abstract
We study the problem of missing not at random (MNAR) datasets with binary outcomes. We propose an exponential tilt based approach that bypasses any knowledge on 'nonresponse instruments' or 'shadow variables' that are usually required for statistical estimation. We establish a sufficient condition for identifiability of tilt parameters and propose an algorithm to estimate them. Based on these tilt parameter estimates, we propose importance weighted and doubly robust estimators for any mean functions of interest, and validate their performances in a synthetic dataset. In an experiment with the Waterbirds dataset, we utilize our tilt framework to perform unsupervised transfer learning, when the responses are missing from a target domain of interest, and achieve a prediction performance that is comparable to a gold standard.
Figures
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Bang and J
H. Bang and J. M. Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61 0 (4): 0 962--973, 2005
2005
- [3]
-
[4]
T. T. Cai and H. Wei. Transfer learning for nonparametric classification: Minimax rate and adaptive classifier. The Annals of Statistics, 49 0 (1): 0 100--128, 2021
work page 2021
-
[5]
P. E. Cheng. Nonparametric estimation of mean functionals with data missing at random. Journal of the American statistical association, 89 0 (425): 0 81--87, 1994
work page 1994
-
[6]
M. Das, W. K. Newey, and F. Vella. Nonparametric estimation of sample selection models. The Review of Economic Studies, 70 0 (1): 0 33--58, 2003
work page 2003
-
[7]
A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological), 39 0 (1): 0 1--22, 1977
work page 1977
-
[8]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248--255. Ieee, 2009
2009
Show all 55 references
-
[9]
Efromovich
S. Efromovich. Nonparametric regression with predictors missing at random. Journal of the American Statistical Association, 106 0 (493): 0 306--319, 2011
2011
-
[10]
R. E. Fay. Causal models for patterns of nonresponse. Journal of the American Statistical Association, 81 0 (394): 0 354--365, 1986
1986
-
[11]
S. Garg, Y. Wu, S. Balakrishnan, and Z. Lipton. A unified view of label shift estimation. Advances in Neural Information Processing Systems, 33: 0 3290--3300, 2020
2020
-
[12]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016
2016
-
[13]
J. J. Heckman. Sample selection bias as a specification error. Econometrica: Journal of the econometric society, pages 153--161, 1979
1979
-
[14]
D. G. Horvitz and D. J. Thompson. A generalization of sampling without replacement from a finite universe. Journal of the American statistical Association, 47 0 (260): 0 663--685, 1952
1952
-
[15]
D. Hu, T. Yu, and P. Li. Receiver operating characteristic curve analysis with non-ignorable missing disease status. arXiv preprint arXiv:2411.17402, 2024
2024 arXiv
-
[16]
H. Jin, Y. Ma, and F. Jiang. Matrix completion with covariate information and informative missingness. Journal of Machine Learning Research, 23 0 (180): 0 1--62, 2022
2022
-
[17]
M. G. Kenward and G. Molenberghs. Likelihood based frequentist inference when data are missing at random. Statistical Science, pages 236--247, 1998
1998
-
[18]
J. K. Kim and C. L. Yu. A semiparametric estimation of mean functionals with nonignorable missing data. Journal of the American Statistical Association, 106 0 (493): 0 157--165, 2011
2011
-
[19]
Kivinen and M
J. Kivinen and M. K. Warmuth. Exponentiated gradient versus gradient descent for linear predictors. information and computation, 132 0 (1): 0 1--63, 1997
1997
-
[20]
M. J. Laan and J. M. Robins. Unified methods for censored longitudinal data and causality. Springer, 2003
2003
-
[21]
P. Li, J. Qin, and Y. Liu. Instability of inverse probability weighting methods and a remedy for nonignorable missing data. Biometrics, 79 0 (4): 0 3215--3226, 2023
2023
-
[22]
Lipton, Y.-X
Z. Lipton, Y.-X. Wang, and A. Smola. Detecting and correcting for label shift with black box predictors. In International conference on machine learning, pages 3122--3130. PMLR, 2018
2018
-
[23]
R. J. Little. Regression with missing x's: a review. Journal of the American statistical association, 87 0 (420): 0 1227--1237, 1992
1992
-
[24]
R. J. Little. Pattern-mixture models for multivariate incomplete data. Journal of the American Statistical Association, 88 0 (421): 0 125--134, 1993
1993
-
[25]
R. J. Little. A class of pattern-mixture models for normal incomplete data. Biometrika, 81 0 (3): 0 471--483, 1994
1994
-
[26]
R. J. Little and D. B. Rubin. Statistical analysis with missing data, volume 793. John Wiley & Sons, 2019
2019
-
[27]
L. Liu, W. Miao, B. Sun, J. Robins, and E. T. Tchetgen. Identification and inference for marginal average treatment effect on the treated with an instrumental variable. Statistica sinica, 30 0 (3): 0 1517, 2020
2020
-
[28]
Y. Liu, P. Li, and J. Qin. Full-semiparametric-likelihood-based inference for non-ignorable missing data. Statistica Sinica, 32 0 (1): 0 271--292, 2022
2022
-
[29]
W.-Q. Ma, Z. Geng, and Y.-H. Hu. Identification of graphical models for nonignorable nonresponse of binary outcomes in longitudinal studies. Journal of multivariate analysis, 87 0 (1): 0 24--45, 2003
2003
-
[30]
Maity, Y
S. Maity, Y. Sun, and M. Banerjee. Minimax optimal approaches to the label shift problem in non-parametric settings. Journal of Machine Learning Research, 23 0 (346): 0 1--45, 2022
2022
-
[31]
Maity, M
S. Maity, M. Yurochkin, M. Banerjee, and Y. Sun. Understanding new tasks through the lens of training data via exponential tilting. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=DBMttEEoLbw
2023
-
[32]
Maity, D
S. Maity, D. Dutta, J. Terhorst, Y. Sun, and M. Banerjee. A linear adjustment-based approach to posterior drift in transfer learning. Biometrika, 111 0 (1): 0 31--50, 2024
2024
-
[33]
X. Mao, S. X. Chen, and R. K. Wong. Matrix completion with covariate information. Journal of the American Statistical Association, 114 0 (525): 0 198--210, 2019
2019
-
[34]
Miao and E
W. Miao and E. J. Tchetgen Tchetgen. On varieties of doubly robust estimators under missingness not at random with a shadow variable. Biometrika, 103 0 (2): 0 475--482, 2016
2016
-
[35]
W. Miao, P. Ding, and Z. Geng. Identifiability of normal and normal mixture models with nonignorable missing data. Journal of the American Statistical Association, 111 0 (516): 0 1673--1683, 2016
2016
-
[36]
W. Miao, L. Liu, Y. Li, E. J. Tchetgen Tchetgen, and Z. Geng. Identification and semiparametric efficiency theory of nonignorable missing data with a shadow variable. ACM/JMS Journal of Data Science, 1 0 (2): 0 1--23, 2024
2024
-
[37]
N. G. Nair, P. Satpathy, J. Christopher, et al. Covariate shift: A review and analysis on classifiers. In 2019 Global Conference for Advancement in Technology (GCAT), pages 1--6. IEEE, 2019
2019
-
[38]
J. M. Robins, A. Rotnitzky, and D. O. Scharfstein. Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In Statistical models in epidemiology, the environment, and clinical trials, pages 1--94. Springer, 2000
2000
-
[39]
Rotnitzky, J
A. Rotnitzky, J. M. Robins, and D. O. Scharfstein. Semiparametric regression for repeated outcomes with nonignorable nonresponse. Journal of the american statistical association, 93 0 (444): 0 1321--1339, 1998
1998
-
[40]
D. B. Rubin. Inference and missing data. Biometrika, 63 0 (3): 0 581--592, 1976
1976
-
[41]
D. B. Rubin. Multiple imputation for nonresponse in surveys, volume 81. John Wiley & Sons, 2004
2004
-
[42]
Sagawa, P
S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=ryxGuJrFvS
2020
-
[43]
Schenker and A
N. Schenker and A. H. Welsh. Asymptotic results for multiple imputation. The Annals of Statistics, 16 0 (4): 0 1550--1566, 1988
1988
-
[44]
C. Scott. A generalized neyman-pearson criterion for optimal domain adaptation. In Algorithmic Learning Theory, pages 738--761. PMLR, 2019
2019
-
[45]
Sugiyama, M
M. Sugiyama, M. Krauledat, and K.-R. M \"u ller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8 0 (5), 2007
2007
-
[46]
B. Sun, L. Liu, W. Miao, K. Wirth, J. Robins, and E. J. T. Tchetgen. Semiparametric estimation with data missing not at random using an instrumental variable. Statistica Sinica, 28 0 (4): 0 1965, 2018
1965
-
[47]
Tang and Y
N. Tang and Y. Ju. Statistical inference for nonignorable missing-data problems: a selective review. Statistical Theory and Related Fields, 2 0 (2): 0 105--133, 2018
2018
-
[48]
E. J. Tchetgen Tchetgen and K. E. Wirth. A general instrumental variable framework for regression analysis with outcome missing not at random. Biometrics, 73 0 (4): 0 1123--1131, 2017
2017
-
[49]
A. A. Tsiatis. Semiparametric theory and missing data, volume 4. Springer, 2006
2006
-
[50]
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-ucsd birds-200-2011 dataset. 2011
2011
-
[51]
L. Wang, J. Shao, and F. Fang. Propensity model selection with nonignorable nonresponse and instrument variable. Statistica Sinica, 31 0 (2): 0 647--672, 2021
2021
-
[52]
S. Wang, J. Shao, and J. K. Kim. An instrumental variable approach for identification and estimation with nonignorable nonresponse. Statistica Sinica, pages 1097--1116, 2014
2014
-
[53]
W. Yu, J. K. Kim, and T. Park. Estimation of area under the roc curve under nonignorable verification bias. Statistica Sinica, 28 0 (4): 0 2149, 2018
2018
-
[54]
B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40 0 (6): 0 1452--1464, 2017
2017
-
[55]
Z. Zhu, T. Wang, and R. J. Samworth. High-dimensional principal component analysis with heterogeneous missingness. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84 0 (5): 0 2000--2031, 2022
2000
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.