{"id":"f4707601-995d-4236-8952-20d87ca5b1b4","arxiv_id":"2502.06046","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Binary-outcome MNAR estimation is made identifiable through an exponential tilt model with a sufficient identifiability condition, estimated by KL matching without instruments or shadow variables.","lead":"This paper introduces an exponential tilt method to estimate averages when binary outcomes are missing not at random, without needing instruments or shadow variables. It demonstrates improved transfer learning accuracy on the Waterbirds dataset, comparable to an oracle that sees true labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Waterbirds identifiability premise is unsupported: Section 5 asserts a neural network's 'functional structure' satisfies Assumption 2.1 without testing the fitted log-odds against the forbidden family.","rationale":"The paper's theoretical structure is coherent: Lemma C.4 genuinely proves that Assumption 2.1 suffices for uniqueness of the tilt parameters, the KL matching objective is well motivated, and the synthetic experiments implement the method under a correctly specified tilt model. The DR theorem is honestly scoped, and the paper explicitly reports that the DR estimator fails in Waterbirds, which is a point in its favor. The load-bearing soft spot is exactly the bridge from 'sufficient condition' to 'applicable without instruments': the condition has to hold in the data, and the only real-data argument offered is an unexamined assertion about a neural network's functional form. Since the neural network class is flexible enough to approximate the forbidden family, the class-level assertion cannot establish the pointwise non-membership required by Assumption 2.1. This does not invalidate the conditional theoretical claim, but it means the empirical demonstration does not actually verify the identification premise. My concern therefore agrees with the reader's weakest_assumption and leaves the verdict at CONDITIONAL; it sharpens the reason: not merely 'unverified', but 'the provided verification argument is not valid as stated'.","tokens_in":25551,"tokens_out":21770,"duration_ms":206050,"concrete_test":"Directly test Assumption 2.1 on the Waterbirds source data: (1) compute the fitted log-odds L(x) from the same single hidden-layer network used in Section 5, or from a more flexible estimator; (2) fit the forbidden family F(x;mu,delta) to L(x) by nonlinear least squares over the target-train covariate distribution, using multiple random starts; (3) compare the best F approximation's residual norm to the network's held-out log-likelihood or classification error. If the residual is within the network's estimation noise, e.g. the forbidden fit is not rejected by a likelihood-ratio or goodness-of-fit test, then Assumption 2.1 is not credible and the Waterbirds results need a sensitivity analysis over the unidentified directions. If the forbidden family is clearly rejected, the identifiability premise is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central identification claim is conditional on Assumption 2.1, i.e. the source log-odds L(x)=logit P(Y=1|X=x,R=1) must not belong to the family F(x)=mu2+delta2^T t+log((e^{mu1+delta1^T t}-1)/(e^{mu0+delta0^T t}-1)). In the only real-data demonstration (Section 5), this is waived by the sentence 'The functional structure of single hidden layer neural network satisfies the Assumption 2.1.' That sentence does not do the required work: F contains every affine function (take delta1=delta0=0, mu1=mu0), and a single hidden-layer network can approximate any function in F arbitrarily well on the covariate support, so the network class is not structurally disjoint from F. What matters is the fitted log-odds function actually learned from Waterbirds, and the paper never checks whether it is close to F. If the true L(x) is approximately affine or otherwise near F in the region where X has mass, Assumption 2.1 fails and the tilt parameters are not identified from the observed marginal; the reported IW and OR accuracy gains could then be artifacts of the optimizer's arbitrary choice among near-equivalent theta values. 'In principle verifiable' is not the same as verified; no test of non-membership in F is provided or run.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies estimation of mean functionals when binary outcomes are missing not at random. The central assumption is that the importance weight mapping the observed (R=1) joint distribution to the missing (R=0) joint distribution has the exponential-tilt form exp(alpha_y + beta_y' T(x)). Under a new sufficient identifiability condition (Assumption 2.1), the tilt parameters are claimed identifiable from the marginal distribution of X conditional on R=0 and R=1. The paper proposes a constrained KL-matching estimator solved by an exponentiated-gradient algorithm, constructs importance-weighted and doubly robust estimators for mean functionals, and proves a double-robustness property. Synthetic normal-mixture experiments support the theory, and a Waterbirds transfer-learning experiment is used to argue that the method mitigates spurious correlation without observing the missingness-controlling variable.","tokens_in":25895,"tokens_out":7853,"duration_ms":73207,"significance":"If the claims hold, the paper offers a genuinely useful direction: it weakens the usual requirements of nonresponse instruments or shadow variables for MNAR binary outcomes, and it provides a double-robustness result that can tolerate certain misspecifications of the tilt model. The internal theoretical core is largely consistent: Lemma C.4, Lemma 3.3, and Theorem 3.4 are coherent, the bias decompositions are explicit, and the synthetic experiments match the claimed behavior. The paper also ships code and evaluates the transfer-learning claim on a held-out target-test split with oracle benchmarks. However, the practical usefulness of the method rests on Assumption 2.1, and the only real-data demonstration does not verify that assumption. In addition, the estimation algorithm lacks convergence guarantees, so the statistical claims are not fully supported.","major_comments":[{"comment":"The sentence 'The functional structure of single hidden layer neural network satisfies the Assumption 2.1' does not establish the identifiability condition. The forbidden family in Assumption 2.1 contains every affine function, since taking mu0=mu1 and delta0=delta1=0 gives log odds equal to mu2 + delta2' t. A single-hidden-layer neural network is a universal approximator on compact covariate support, so the network class is not structurally disjoint from the forbidden family. What would need to be checked is whether the fitted log-odds logit(eta1(x)) learned from the Waterbirds source data is not close to the forbidden family on the support of X. No such check is reported. Because identification of theta through equation (2.2) is entirely conditional on Assumption 2.1, the reported IW and OR accuracy gains in Figure 3 are not established as identifiable quantities; they could reflect an arbitrary selection among near-equivalent theta values.","section":"Section 5, Assumption 2.1"},{"comment":"No convergence, stationarity, or local-minimum guarantee is given for the exponentiated-gradient algorithm. The objective in (3.2) is nonconvex in theta because of the exponential terms and the estimated nuisance function eta1, and the Lagrangian update in Algorithm 1 is heuristic. Lemma 3.3 and Theorem 3.4 are asymptotic bias statements that assume thetahat -> theta* and xihat -> xi*, but no theorem establishes these limits for the estimator produced by Algorithm 1. Please either provide conditions under which Algorithm 1 converges to a consistent estimator, or explicitly state the consistency claim as an assumption/conjecture and adjust the wording of Theorem 3.4 accordingly.","section":"Section 3.1, Algorithm 1"},{"comment":"The paper states that Assumption 2.1 is 'in principle verifiable from the observed dataset', but it offers no practical verification procedure. The quadratic-log-odds example is only a sufficient instance, not a test of non-membership in the forbidden family. Since this assumption is the sole route to identification of the tilt parameters, the manuscript should either provide a falsifiable diagnostic for the assumption (for example, a goodness-of-fit test against the forbidden family on the R=1 sample) or clearly frame the real-data analysis as illustrative under an unverified condition. Without this, the identification claim in the Waterbirds study is not supported.","section":"Section 2, Assumption 2.1"}],"minor_comments":[{"comment":"In the definition of the outcome-regression loss, the second term should be (1-p) times the loss at Y=0; as written, ell(X,p;beta) = p ell(X,1;beta) + (1-p) ell(X,1;beta) is not the expected loss under a probability p.","section":"Section 5, Eq. (5.1)"},{"comment":"The indicator condition for updating eta2 has a missing closing parenthesis: 'I{gn(theta(t) < -epsilon}' should read 'I{gn(theta(t)) < -epsilon}'.","section":"Algorithm 1, line 9"},{"comment":"The text identifies the target as mu0 = pi_{1|r} = 0.6; this should be pi_{1|0} = 0.6, since mu0 = E[Y | R=0].","section":"Section 4, paragraph after Eq. (4.1)"},{"comment":"The proof contains a duplicated, incomplete sentence: 'leading to bias(hat(mu)) = 0. This establishes (i). To establish (ii)' appears before the decomposition in (C.8), and should be removed.","section":"Appendix C.2.4, proof of Theorem 3.4"}],"recommendation":"major_revision","confidential_remarks":"The theoretical skeleton is sound and the synthetic experiments are useful, but the Waterbirds section makes an identifiability claim that is not justified and is likely false as stated. If the authors cannot verify Assumption 2.1 for the fitted model, the real-data contribution should be substantially downweighted or reframed. The missing convergence theory for Algorithm 1 is also a serious gap for a statistics paper, since the consistency of the proposed estimators is a premise of the main theorems."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read it. The paper has a genuine new idea: Assumption 2.1 gives a sufficient identifiability condition that lets you estimate exponential tilt weights for MNAR binary outcomes without instruments or shadow variables, and the KL-matching estimator is implementable. The double-robustness theorem (Thm 3.4) is clean; the bias decomposition in Lemma 3.3 is honest, and the synthetic experiments line up with the theory. The author also deserves credit for reporting that the DR estimator fails on Waterbirds rather than hiding it.\n\nThe main soft spot is load-bearing: Assumption 2.1 is never actually verified. The paper calls it 'in principle verifiable' but supplies no test. In the Waterbirds section it is dealt with by the sentence 'The functional structure of single hidden layer neural network satisfies the Assumption 2.1.' That does not work. The forbidden family F contains every affine function, and a single hidden layer network can approximate any continuous function on the covariate support, so the class is not structurally disjoint from F. What matters is the fitted log-odds actually learned from Waterbirds; the paper never checks whether it is close to F. Without that, the tilt parameters are not guaranteed identified from the observed marginal, and the reported IW/OR gains could be an artifact of arbitrary θ among near-equivalent values.\n\nThis is not a fatal flaw for the method: when Assumption 2.1 does hold, the approach is sound, and the synthetic design explicitly makes it hold. But it means the Waterbirds experiment does not validate the method in the way the abstract implies. Other gaps are minor by comparison: Algorithm 1 has no convergence guarantees (acceptable for a first cut, but it should be acknowledged), and the DR estimator—the theoretical centerpiece—fails in the only real-data experiment, leaving IW and OR to carry the empirical case. The author gives a plausible instability explanation, but the practical value of the DR estimator remains open.\n\nCitation pattern is fine: Kim and Yu and the author's own earlier tilting papers are properly credited. The combination of Assumption 2.1, KL-matching without instruments, and the DR theorem appears to be genuinely new.\n\nWho gets value: statisticians working on MNAR or transfer learning will find the identifiability lemma and the estimation framework worth engaging. It deserves a serious referee, not a desk reject. I would accept it and ask for a practical test of Assumption 2.1, or a more careful real-data claim that does not lean on an unverified functional-structure assertion.","headline":"A genuinely new identifiability condition and a clean DR theorem, but the Waterbirds real-data claim rests on an unverified assumption that the paper waves through.","tokens_in":26415,"tokens_out":4115,"would_cite":true,"duration_ms":37113,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62D10","62F12"],"pacs":[],"model":"deepseek-v4-flash","headline":"Exponential tilt models identify missing-not-at-random binary outcomes from the observed marginal.","keywords":["missing not at random","binary outcomes","exponential tilt","importance weighting","doubly robust estimation","identifiability","missingness controlling variable","unsupervised transfer learning"],"falsifier":"Construct data in which the $R=1$ log odds has exactly the forbidden form, for example by taking two Gaussian components whose means are swapped as in the paper's Section 2 counterexample. Then run Algorithm 1 from multiple starting points: the objective (3.2) has multiple optima, so the recovered importance weights will differ across runs, directly contradicting the claim of unique identification.","tokens_in":25335,"feed_emoji":"📊","tokens_out":10609,"duration_ms":94232,"temperature":0.7,"pith_summary":"This paper claims that, for binary outcomes missing not at random (MNAR), the missingness problem is identifiable when the importance weight between the responder and non-responder subpopulations has an exponential tilt form, $\\omega(x,y)=\\exp(\\alpha_y+\\beta_y^\\top T(x))$. Under one sufficient condition on the observed-population log odds (Assumption 2.1), the tilt parameters are uniquely determined by the observed marginal distribution of $X$ alone, so the method needs neither nonresponse instruments nor shadow variables. The paper estimates the tilt parameters by a constrained Kullback-Leibler matching problem and feeds them into importance-weighted and doubly robust estimators for any mean functional. If the central claim is right, routine observed data with binary outcomes can support transfer learning and missing-data adjustment in settings where previous MNAR methods required external identifying variables.","feed_headline":"Tilt model estimates missing binary outcomes without shadow variables","feed_subtitle":"It reaches gold-standard transfer accuracy without target labels.","key_machinery":"The load-bearing object is the exponential tilt model (1.2), a two-component exponential-family shift $\\omega(x,y)=\\exp(\\alpha_y+\\beta_y^\\top T(x))$ between the $R=1$ and $R=0$ subpopulations. This single specification carries the whole argument: it turns the observed covariate distribution into a two-component tilted mixture (eq. (2.2)), which is matched to the data by minimizing Kullback-Leibler divergence under a normalization constraint (eqs. (3.1)-(3.2)); it delivers the propensity score formula in Lemma 3.1; and its log-odds difference $\\log\\omega(x,1)-\\log\\omega(x,0)=(\\alpha_1-\\alpha_0)+(\\beta_1-\\beta_0)^\\top T(x)$ provides the outcome regression in Lemma 3.2. The double-robustness theorem then rests on those two building blocks.","core_discovery":"Under the exponential tilt model (1.2), the joint law of $(X,Y)$ in the missing domain is obtained from the observed domain by multiplying each outcome-specific density by $\\exp(\\alpha_y+\\beta_y^\\top T(x))$; therefore the observed covariate marginal satisfies eq. (2.2). The paper shows that Assumption 2.1 selects a unique set of tilt parameters from that marginal, making the importance weights, and hence all mean functionals, identifiable. The mechanism is complete: the missingness propensity score is $\\mathrm{P}(R=1\\mid X,Y)=\\mathrm{P}(R=1)/(\\mathrm{P}(R=1)+\\omega(X,Y)\\mathrm{P}(R=0))$, the conditional outcome probability under missingness is obtained from a log odds ratio $(\\alpha_1-\\alpha_0)+(\\beta_1-\\beta_0)^\\top T(x)$, and the doubly robust estimators are asymptotically unbiased if either the tilt model is correct or the outcome regression plus log-odds-ratio model is correct (Theorem 3.4). The paper additionally reports that the fitted linear combination $(\\beta_0^\\top T(X),\\beta_1^\\top T(X))$ acts as an estimated missingness controlling variable, and that in the Waterbirds study this suffices for unsupervised transfer learning at accuracy comparable to a labeled target oracle.","pith_inferences":["The paper asserts Assumption 2.1 is in principle verifiable but gives no practical test; fitting the observed log odds nonparametrically and checking whether it ever enters the forbidden log-ratio family would be a natural way to make the method safer in practice.","Because the estimated missingness controlling variable is only identified up to the choice of summary $T(x)$, the surrogate may recover a transformation of the true controlling variable rather than the variable itself; this is enough for prediction but not automatically for causal interpretation.","The Waterbirds experiment suggests the DR estimator can become unstable when the product of estimated residuals and covariate directions is unbounded, so a clipped or regularized variant of the DR loss is a natural extension.","The same tilt machinery should extend to multi-category outcomes or to $T(x)$ chosen as a neural embedding, as in the Waterbirds experiment, but the identifiability condition would need to be re-derived for each new family."],"forward_implications":["Mean functionals $\\mathrm{E}[\\tau(X,Y)]$ and $\\mathrm{E}[\\tau(X,Y)\\mid R=0]$ become estimable from observed data alone whenever the tilt parameters are identified.","A user no longer needs to pre-specify a nonresponse instrument or shadow variable; the procedure returns an estimated missingness controlling variable $(\\beta_0^\\top T(X),\\beta_1^\\top T(X))$ as a byproduct.","Unsupervised transfer learning when the outcome-given-covariate relationship differs between source and target can be carried out without labeled target samples, as long as the source-population log odds satisfy Assumption 2.1.","The doubly robust estimator keeps asymptotic unbiasedness under misspecification of the tilt model provided the log-odds-ratio and outcome regression are correct, a robustness property shown in Theorem 3.4.","A computationally stable exponentiated-gradient algorithm solves the constrained KL matching problem, outperforming a profile empirical likelihood alternative in the paper's simulations."],"supporting_citations":[{"why":"Defines the nonresponse-instrument framework this method bypasses, and shows the need for extra identifiability assumptions.","marker":"Wang et al. (2014)"},{"why":"Introduces the shadow-variable identification strategy, the main alternative avoided here.","marker":"Miao and Tchetgen Tchetgen (2016)"},{"why":"Earlier exponential tilt model with a constant tilt parameter that the paper generalizes to outcome- and covariate-dependent tilts.","marker":"Kim and Yu (2011)"},{"why":"Source of the exponential tilt model terminology and training-data tilting idea extended to missing-data estimation.","marker":"Maity et al. (2023)"},{"why":"Provides the exponentiated gradient optimization used to enforce the normalization constraint in Algorithm 1.","marker":"Kivinen and Warmuth (1997)"},{"why":"Documents inverse probability weighting instability that motivates the doubly robust estimators.","marker":"Li et al. (2023)"},{"why":"Supplies the Waterbirds benchmark and its spurious-correlation setting for the transfer-learning experiment.","marker":"Sagawa et al. (2020)"},{"why":"Establishes that even fully parametric MNAR models can be nonidentified from observed data, framing the role of Assumption 2.1.","marker":"Miao et al. (2016)"}],"fun_headline_variants":["Skip shadow variables: exponential tilt estimates MNAR binary outcomes","No shadow variables needed: tilt model estimates missing binary outcomes","Tilt model matches gold standard without labels or shadow variables","Exponential tilt recovers MNAR binary outcomes, no shadow variables"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire identification rests on Assumption 2.1: the true log odds $\\log\\{\\mathrm{P}(Y=1\\mid X=x,R=1)/\\mathrm{P}(Y=0\\mid X=x,R=1)\\}$ must not be representable by the forbidden log-ratio-of-exponentials expression, otherwise two different tilt parameter values explain the observed covariate marginal equally well.","fun_headline_variants_meta":{"raw":{"variants":["Skip shadow variables: exponential tilt estimates MNAR binary outcomes","No shadow variables needed: tilt model estimates missing binary outcomes","Tilt model matches gold standard without labels or shadow variables","Exponential tilt recovers MNAR binary outcomes, no shadow variables"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001716,"raw_usage":{"total_tokens":6778,"prompt_tokens":924,"completion_tokens":5854,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":5785}},"tokens_in":540,"tokens_out":5854,"duration_ms":36569,"temperature":1.0,"reasoning_tokens":5785,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T16:56:19.587365+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct data in which the $R=1$ log odds has exactly the forbidden form, for example by taking two Gaussian components whose means are swapped as in the paper's Section 2 counterexample. Then run Algorithm 1 from multiple starting points: the objective (3.2) has multiple optima, so the recovered importance weights will differ across runs, directly contradicting the claim of unique identification.","supporting_citations":[],"review_version":1}