{"id":"8eb3b3af-5741-49c3-a8b3-043c716bc8cb","arxiv_id":"2411.13829","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"New variants of multiple imputation, NAR-SMCFCS and NAR-SMC-stack, keep imputation compatible with a causal analysis model when the outcome causes its own missingness, and reduce bias in estimated effects.","lead":"What if the very thing you cannot measure, the outcome, is the reason some data are missing? This paper develops two statistical methods for that situation, specifically for estimating cause and effect from observational data. Such methods matter because ignoring this type of missingness can distort conclusions about whether an exposure like cannabis use harms mental health.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NAR-SMCFCS imputes non-outcome variables from a target that marginalizes over MY rather than conditioning on it; compatibility with the delta-adjusted outcome model is therefore unproven, and the simulation design may not expose the discrepancy.","rationale":"The paper's central claim is that NAR-SMCFCS and NAR-SMC-stack give approximately unbiased ACE estimates when the outcome causes its own missingness and the substantive model includes exposure-confounder interactions. The simulation evidence for this claim is reasonably strong, especially for NAR-SMC-stack, whose missingness-pattern weights are carefully derived. For NAR-SMCFCS, however, the non-outcome imputation step uses the substantive outcome model without conditioning on MY, even though the outcome imputation model conditions on MY and is delta-shifted. This is precisely the point the reader identified as the weakest assumption. The concern is load-bearing because unbiasedness of the full FCS procedure requires that each conditional imputation model be compatible with the target joint distribution; if the Vj conditionals are drawn from the wrong target, bias can enter even when δ is correctly specified. The paper's simulation does not directly refute this worry: data are generated from a selection model, the outcome imputation model is acknowledged to be misspecified, and δ values are calibrated by fitting that same model to complete data, so any incompatibility in Algorithm 1 is confounded with other misspecifications. A targeted pattern-mixture simulation would settle whether the marginalization matters. I do not think this concern warrants rejection: the method may still perform well in practice, and the paper is transparent about its limitations, including the variance shortfall in the stacked approach. But the compatibility claim for NAR-SMCFCS should be either proven or explicitly qualified before full acceptance. The reader's conditional verdict already captures this, so no change to the verdict is needed.","tokens_in":19066,"tokens_out":8187,"duration_ms":81170,"concrete_test":"Generate data directly from the pattern-mixture model that motivates delta-adjustment: fix MY by design, draw V|S,MY from distributions that differ by MY (e.g., exposure prevalence 0.4 for MY=0 and 0.6 for MY=1), and draw Y|X,C,MY from model (5) with the true δ used in the analysis. Apply NAR-SMCFCS with the correct δ and compare bias and coverage with NAR-SMC-stack. Also run a variant of Algorithm 1 in which step 5 fits f(Y|X,Z1,Z2,MY,θ′,δ) conditional on MY and step 7 uses this conditional model in the target; if the variant removes bias that the published algorithm shows, the marginalization over MY in step 7 is the cause.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 step 7 imputes each non-outcome variable Vj from the target f(Y|X,Z1,Z2,θ) f(Vj|V−j,S,λj), where θ is fitted in step 5 to the current imputed dataset. Because Y values for records with MY=1 were drawn in step 8 from the shifted model f(Y|X,Z1,Z2,MY=1,θ′,δ), the fitted θ describes f(Y|V,S) marginalized over MY, not f(Y|V,S,MY). The compatible conditional for Vj under the pattern-mixture model implicit in delta-adjustment is f(Vj|V−j,S,Y,MY) ∝ f(Y|V,S,MY,θ′,δ) f(Vj|V−j,S,MY,λ), which conditions on MY. The step-7 target is instead proportional to f(Vj|V−j,S,Y), averaging over MY. Unless f(Y|V,MY=0)=f(Y|V,MY=1) after integrating out the missingness model—not generally true when δ≠0—NAR-SMCFCS is not exactly compatible with its own outcome imputation model. The paper provides no argument that the bias from this mismatch is negligible, and the simulation, generated under a selection model with the outcome imputation model itself misspecified, may not isolate this discrepancy. This is the weakest load-bearing point because the unbiasedness claim for NAR-SMCFCS depends on it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses sensitivity analysis for outcome missingness in causal inference settings where the outcome causes its own missingness and the target estimand (the average causal effect, ACE) is non-recoverable. It extends two multiple-imputation approaches, SMCFCS and SMC-stack, by incorporating delta-adjustment for the outcome, yielding NAR-SMCFCS and NAR-SMC-stack. The proposed methods are intended to maintain compatibility between the imputation models and a substantive analysis that includes exposure-confounder interactions, as in g-computation. The paper evaluates the methods in a simulation study motivated by the Victorian Adolescent Health Cohort Study, across continuous and binary outcomes, simple and complex outcome-missingness mechanisms, and null/weak/strong exposure-confounder interactions. It reports that a naive NARFCS implementation is biased in the interaction scenarios, while the proposed approaches produce approximately unbiased ACE estimates, with NAR-SMC-stack showing some under-coverage. The methods are also applied to the case study.","tokens_in":19388,"tokens_out":6219,"duration_ms":58835,"significance":"If the compatibility properties claimed for the new approaches can be established or empirically bounded, the paper would fill a practical gap: it offers a way to perform delta-adjustment sensitivity analysis for g-computation with multivariable missingness, where standard NARFCS is incompatible when exposure-confounder interactions are in the substantive model. The simulation study is carefully tied to a real cohort, the results are reported transparently, and the variance shortfall of NAR-SMC-stack is explicitly acknowledged rather than hidden. The paper also builds on existing published approaches with available implementations, which supports reproducibility. However, the central unbiasedness claim rests on a compatibility step in NAR-SMCFCS that is not formally justified, and the simulation's 'true' sensitivity parameters are acknowledged approximations obtained from a misspecified model; these issues weaken the strength of the conclusions as currently stated.","major_comments":[{"comment":"The compatibility claim for NAR-SMCFCS is not justified. In step 7, each non-outcome variable Vj is imputed from the target distribution f(Y|X,Z1,Z2,theta)f(Vj|V-j,S,lambda_j), with theta estimated in step 5 from the current imputed dataset. But in step 8 the missing outcomes are imputed from the delta-shifted model f(Y|X,Z1,Z2,MY,theta',delta). The theta fitted in step 5 therefore describes a distribution of Y averaged over MY, not conditional on MY. The pattern-mixture conditional that is compatible with the outcome imputation model is f(Vj|...,Y,MY) proportional to f(Y|...,MY,theta',delta)f(Vj|...,MY,lambda), which conditions on MY. Unless f(Y|V,MY=0)=f(Y|V,MY=1) after integrating out the missingness model, which is not generally true when delta is nonzero, the step-7 target is not the compatible conditional for records with MY=1. The paper provides no argument that the bias from this mismatch is negligible, and the simulation, generated under a selection model with a misspecified outcome imputation model, does not isolate this discrepancy. Since the approximate unbiasedness claim for NAR-SMCFCS depends on this step, this is a load-bearing gap.","section":"Section 5.3 and Section 7"},{"comment":"The simulation's 'true' sensitivity parameter values are not true data-generating deltas. They are obtained by fitting the outcome imputation model (5), which the paper acknowledges may be misspecified, to a large generated complete dataset from the selection model. The Discussion correctly states that these values represent only an approximation to the closest possible pattern-mixture representation. Consequently, evaluating the methods at these values and at multiples of them does not establish approximate unbiasedness with respect to the actual missingness mechanism; it establishes behavior along a projection of that mechanism. I would like to see either a pattern-mixture data-generating process in which delta is exactly the shift in the outcome model, or an explicit assessment of how large the projection error is. Without this, the headline claim that the proposed methods are 'approximately unbiased' at the true sensitivity parameters is overstated.","section":"Section 5.4, Table 2"},{"comment":"The variance estimator for NAR-SMC-stack produces systematically below-nominal coverage, with values around 90-93% across scenarios and model-based standard errors that are substantially smaller than those from NAR-SMCFCS (e.g., 0.009-0.103 versus 0.108-0.112 for continuous outcomes). The paper notes this issue and cites importance-sampling literature, but it still concludes in Section 7 that both proposed approaches perform well. Given that coverage is a central operating characteristic for a sensitivity analysis method, and given that one of the two proposed approaches is the subject of the paper's recommendation, this needs more than a remark: either a corrected variance estimator, a clear statement that NAR-SMC-stack should be used only after further development of its variance estimator, or a substantive justification that the under-coverage is acceptable in this setting.","section":"Section 7"}],"minor_comments":[{"comment":"The text says each method was implemented by setting delta_0 to the true value, twice the true value, and zero, but the complex missingness scenario has two sensitivity parameters, delta_0 and delta_1. Please clarify whether delta_1 was fixed at its true value while delta_0 was varied, and how this is reflected in Figure 2.","section":"Section 5.3"},{"comment":"The notation in steps 5 and 8 makes the distinction between theta and theta' difficult to follow. A sentence clarifying that theta in step 5 is the substantive-model parameter fitted to the current imputed dataset (including delta-shifted imputed outcomes) and that theta' in step 8 is the identifiable part of the delta-adjusted outcome model would help the reader.","section":"Section 4.1, Algorithm 1"},{"comment":"The weight formula is clear in substance, but the denominator uses a sum over m without making the index on the numerator explicit. Please write the numerator as f(Y_i | X_i^m, Z_{1i}, Z_{2i}^m, M_{Yi}=0, theta') to avoid ambiguity about which imputation m is being weighted.","section":"Section 4.2, equation (3)"},{"comment":"The abbreviations MAR-SMCFCS and MAR-SMC-stack are introduced in Section 3.4 after SMCFCS and SMC-stack have already been used; defining both variants once in Section 1 or at first use would improve readability.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a good fit for the journal and the simulation work is substantial. The main risk is the unproven compatibility of NAR-SMCFCS in Algorithm 1, which is the theoretical basis for the unbiasedness claim. A formal argument or a targeted simulation under a pattern-mixture mechanism with a known delta would substantially strengthen the paper. The acknowledged approximation in the 'true' delta values also needs to be addressed rather than merely noted. I do not see grounds for rejection, but the load-bearing points need work before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a genuinely useful methods paper. The gap it fills is real: sensitivity analysis for outcome-caused missingness has not been reconciled with substantive-model-compatible imputation when the analysis model includes exposure-confounder interactions. The proposed NAR-SMCFCS and NAR-SMC-stack are natural extensions of existing machinery, and the simulation evidence that naive NARFCS is biased while the proposed approaches are approximately unbiased is convincing for the scenarios considered. The importance-ratio construction for missingness pattern III in NAR-SMC-stack is a nice piece of original detail.\n\nThe paper is also honest in the right places. The variance shortfall of NAR-SMC-stack is reported head-on, as is the fact that the 'true' delta values are fitted to a misspecified pattern-mixture representation of the selection-model data generation. The discussion of auxiliary variables and the limits of the approach is sensible and balanced.\n\nThe main soft spot is the compatibility claim for NAR-SMCFCS, and this is where the stress-test note lands. In Algorithm 1, step 5 fits the substantive outcome model to the current imputed dataset, which includes delta-shifted outcome values for records with MY=1. Step 7 then imputes non-outcome variables from a target distribution involving f(Y|X,Z1,Z2,theta) but not conditioning on MY. That is not obviously the compatible conditional under the implicit pattern-mixture model; the compatible target should condition on MY. The paper gives no argument that the mismatch is negligible, and the simulation, where the outcome imputation model itself is misspecified, may not isolate this discrepancy. This is not necessarily a fatal flaw, but it is a load-bearing step for the unbiasedness claim, and the authors should provide a derivation or a more explicit justification, or at least a targeted simulation that teases it apart.\n\nA second, more minor concern: no code or simulation scripts are shipped. For a methods paper in this area, that is now the norm and should be expected. The NAR-SMC-stack variance undercoverage is also unresolved, though the authors openly flag it and point to the importance-sampling literature.\n\nOverall, the central message--that compatibility matters in sensitivity analysis and that these two approaches reduce bias from naive NARFCS--holds up. The paper deserves a serious referee, with requests to address the compatibility step, ship the code, and discuss the variance issue further.","headline":"A solid, useful extension of compatible MI to delta-adjustment sensitivity analysis, with an honest simulation study and one unproven compatibility step that deserves a careful response.","tokens_in":19910,"tokens_out":1656,"would_cite":true,"duration_ms":16456,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"When the outcome causes its own missingness, imputation compatibility still matters—two delta-adjusted approaches avoid the bias of naive NARFCS.","keywords":["multiple imputation","sensitivity analysis","missing not at random","delta-adjustment","substantive model compatibility","causal inference","g-computation","NARFCS"],"falsifier":"Generate data under the paper's missingness DAG but add a direct arrow from the outcome missingness indicator $M_Y$ to a non-outcome variable $V$, so that $f(V|Y, M_Y=1)$ differs from $f(V|Y)$, while keeping strong exposure-confounder interactions in the outcome model; if NAR-SMCFCS or NAR-SMC-stack then shows average causal effect relative bias above 10%, the core compatibility claim would be falsified.","tokens_in":18866,"feed_emoji":"📊","tokens_out":8600,"duration_ms":98728,"temperature":0.7,"pith_summary":"Multiple imputation is only trustworthy when the imputation model is compatible with the analysis model, and this paper extends that requirement to sensitivity analysis for missing-not-at-random outcomes. The paper proposes two methods, NAR-SMCFCS and NAR-SMC-stack, that bring delta-adjustment for outcome missingness into two existing substantive-model-compatible imputation frameworks. In simulations based on a real cohort study, the proposed methods estimate the average causal effect with relative bias below 6% even when the substantive model includes strong exposure-confounder interactions, whereas a naive implementation of NARFCS shows relative bias as large as -31.8%. The central claim is that ignoring compatibility in sensitivity analysis can make the results of the sensitivity analysis itself unreliable.","feed_headline":"Compatible imputation removes bias when the outcome causes missingness","feed_subtitle":"Naive NARFCS loses up to 32% of the true causal effect; the proposed approaches keep relative bias below 6%.","key_machinery":"The central object is the compatible target distribution for imputing each incomplete non-outcome variable $V_j$, proportional to $f(Y|X,Z_1,Z_2,\\theta)\\,f(V_j|V_{-j},S,\\lambda_j)$, where $f(Y|X,Z_1,Z_2,\\theta)$ is the substantive outcome model. NAR-SMCFCS samples from this target within a chained-equations algorithm; NAR-SMC-stack imputes $V_j$ from the proposal $f(V_j|V_{-j},S,\\lambda_j)$ and then assigns each stacked record an importance weight proportional to $f(Y|X,Z_1,Z_2,M_Y=0,\\theta')$. The outcome itself is imputed from the delta-adjusted model $f(Y|X,Z_1,Z_2,M_Y,\\theta',\\delta)$, whose fixed sensitivity parameters $\\delta$ shift the imputed values for records with $M_Y=1$. This division of labour is what keeps the non-outcome imputations compatible with an interaction-containing substantive model while allowing the outcome's own missingness to be reflected in the imputation.","core_discovery":"Under the missingness mechanism where the outcome causes its own missingness, the average causal effect is not recoverable, so analysts must assess how conclusions change under alternative assumptions. The paper argues that the standard sensitivity-analysis tool, NARFCS, is misspecified when the substantive analysis uses g-computation with exposure-confounder interactions, because its univariate imputation models omit those interactions. The proposed NAR-SMCFCS and NAR-SMC-stack methods impute the outcome from a delta-adjusted model, $f(Y|X,Z_1,Z_2,M_Y,\\theta',\\delta)$, while keeping the substantive outcome model inside the target distribution used to impute the non-outcome variables. In the simulation scenario with strong exposure-confounder interactions, naive NARFCS gave mean ACE estimates of 0.21 and 0.20 (relative bias -30.6% and -31.8%), whereas the proposed approaches kept relative bias below 6% and achieved near-nominal coverage for NAR-SMCFCS. The same pattern appeared for binary outcomes, and a case study illustrated that naive NARFCS can make an ACE estimate look more sensitive to missingness assumptions than it really is.","pith_inferences":["A direct extension would replace exposure-confounder interactions with quadratic terms, since a quadratic term is an interaction of a variable with itself; the same target-distribution construction should remove the bias that a main-effects NARFCS would introduce there.","In a real application with modest interaction strength, the difference between naive and compatible NARFCS may be small, but with a strong interaction whose delta effect has the opposite sign, naive NARFCS could reverse the direction of the sensitivity conclusion.","Because NAR-SMC-stack undercovers consistently across all scenarios, practitioners should prefer NAR-SMCFCS for reporting confidence intervals until the stacked variance estimator is improved.","The simulation generates missingness from a selection model while delta-adjustment is a pattern-mixture assumption, so the 'true' delta values are only approximate; a simulation that generates data directly from the pattern-mixture model would provide a cleaner test of the methods."],"forward_implications":["In sensitivity analyses for outcome missingness, using NAR-SMCFCS or NAR-SMC-stack removes imputation incompatibility as a source of bias, so the sensitivity curve reflects the assumed delta values rather than misspecification of the imputation model.","With strong exposure-confounder interactions, naive NARFCS can misstate the average causal effect by roughly a third even when the true sensitivity parameter is used, which means conclusions drawn from such sensitivity analyses may be wrong in either direction.","NAR-SMCFCS achieves approximately nominal 95% coverage when the true delta is used, while NAR-SMC-stack slightly undercovers (around 90-93%), indicating a lingering variance-estimation issue for the stacked approach.","The proposed methods are not limited to g-computation; because compatibility is built into the target distributions, they apply to any substantive analysis based on a regression model."],"supporting_citations":[{"why":"Defines SMCFCS, the substantive-model-compatible FCS procedure that NAR-SMCFCS extends with delta-adjustment.","marker":"[9]"},{"why":"Introduces the stacked-imputation approach with importance weights that NAR-SMC-stack builds on.","marker":"[10]"},{"why":"Provides the NARFCS procedure and its delta-adjustment implementation, which is the benchmark and the sensitivity-analysis starting point.","marker":"[15]"},{"why":"Establishes that the average causal effect is not recoverable when the outcome causes its own missingness, motivating the sensitivity analysis.","marker":"[3]"},{"why":"Supplies the stacked-imputation variance rule and a not-at-random stacked approach that NAR-SMC-stack uses for inference.","marker":"[27]"},{"why":"Supports pooling g-computation estimates from multiply imputed datasets with bootstrap variances and Rubin's rules.","marker":"[28]"},{"why":"Shows that outcome-covariate interactions must appear in imputation models for other variables, which motivates the compatible target distribution.","marker":"[25]"}],"fun_headline_variants":["Outcome-driven missingness: compatible imputation cuts bias to under 6%","New sensitivity methods keep causal estimates unbiased when outcome causes missing data","Naive NARFCS biased: NAR-SMCFCS and NAR-SMC-stack fix it","Compatible delta-adjustment for outcome missingness in causal inference","When outcome causes missingness, compatible imputation saves the effect estimate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the delta-adjusted outcome model fitted to observed outcomes, together with the unadjusted substantive model used in the target distribution for non-outcome variables, remains a compatible description for records with missing outcomes; the paper does not prove this and its simulation may not exercise violations of it.","fun_headline_variants_meta":{"raw":{"variants":["Outcome-driven missingness: compatible imputation cuts bias to under 6%","New sensitivity methods keep causal estimates unbiased when outcome causes missing data","Naive NARFCS biased: NAR-SMCFCS and NAR-SMC-stack fix it","Compatible delta-adjustment for outcome missingness in causal inference","When outcome causes missingness, compatible imputation saves the effect estimate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000717,"raw_usage":{"total_tokens":3314,"prompt_tokens":1133,"completion_tokens":2181,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":749,"completion_tokens_details":{"reasoning_tokens":2078}},"tokens_in":749,"tokens_out":2181,"duration_ms":15882,"temperature":1.0,"reasoning_tokens":2078,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:50:20.362889+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate data under the paper's missingness DAG but add a direct arrow from the outcome missingness indicator $M_Y$ to a non-outcome variable $V$, so that $f(V|Y, M_Y=1)$ differs from $f(V|Y)$, while keeping strong exposure-confounder interactions in the outcome model; if NAR-SMCFCS or NAR-SMC-stack then shows average causal effect relative bias above 10%, the core compatibility claim would be falsified.","supporting_citations":[{"cited_title":"Multiple imputation of covariates by fully conditional specification: accommodating the substantive model","cited_arxiv_id":null,"evidence_quote":"Defines SMCFCS, the substantive-model-compatible FCS procedure that NAR-SMCFCS extends with delta-adjustment."},{"cited_title":"A stacked approach for chained equations multiple imputation incorporating the substantive model","cited_arxiv_id":null,"evidence_quote":"Introduces the stacked-imputation approach with importance weights that NAR-SMC-stack builds on."},{"cited_title":"On the use of the not-at-random fully conditional specification (narfcs) procedure in practice","cited_arxiv_id":null,"evidence_quote":"Provides the NARFCS procedure and its delta-adjustment implementation, which is the benchmark and the sensitivity-analysis starting point."},{"cited_title":"Accounting for not-at-random missingness through imputation stacking","cited_arxiv_id":null,"evidence_quote":"Supplies the stacked-imputation variance rule and a not-at-random stacked approach that NAR-SMC-stack uses for inference."},{"cited_title":"Bootstrap inference for multiple im- putation under uncongeniality and misspecification","cited_arxiv_id":null,"evidence_quote":"Supports pooling g-computation estimates from multiply imputed datasets with bootstrap variances and Rubin's rules."},{"cited_title":"Appropriate inclusion of interactions was needed to avoid bias in multiple imputation","cited_arxiv_id":null,"evidence_quote":"Shows that outcome-covariate interactions must appear in imputation models for other variables, which motivates the compatible target distribution."}],"review_version":1}