{"id":"c82554d8-afbc-44ad-a608-818565fd1c64","arxiv_id":"2505.10624","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A one-step TMLE for the variance of the influence function of the log causal risk ratio yields better coverage and Type I error in simulations, especially under small samples and near-positivity violations.","lead":"This paper introduces a new way to estimate the uncertainty around a causal risk ratio, using a one-step targeted machine learning estimator of the variance of the influence function. Its simulations suggest the resulting confidence intervals stay closer to the nominal 95% in small samples and when positivity is violated.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The described algorithm does not meet the estimating-equation condition used in Theorem 2, so the claimed one-step TMLE efficiency is unsupported; the EIF typos add uncertainty but the core formulas appear recoverable.","rationale":"The paper proposes a genuinely useful target: estimating the sampling variance of the log(CRR) TMLE by targeting the variance of its influence function. The derivation of the variance formula in Eq. 4 is clear, and an independent check of the EIF in Eq. 10 suggests the main Q and g components are structurally correct, with typos rather than fundamental errors. The reader's weakest assumption—that the EIF is correct—is plausible but not the most damaging issue, because the formulas can be repaired. The more load-bearing problem is that the algorithm as described is internally inconsistent with the theoretical framework used to justify it. A one-step TMLE should solve the EIF estimating equation in one fluctuation; the described loop with a fixed tiny step and a stopping threshold that is many orders of magnitude too large does not do so. Consequently, Theorem 2's conclusion does not follow for the implemented procedure, and the simulation evidence, while suggestive, is not linked to the stated asymptotic guarantee. Since the code is public, this is directly checkable. If the code actually performs a proper one-step update, the concern would be resolved; if it follows the text literally, the paper needs either a corrected algorithm or a revised theoretical claim. This warrants a conditional verdict rather than rejection, because the finite-sample coverage improvement may still hold. I agree with the reader that the paper is not ready in its current form, but I would focus the required revision on the algorithm/Theorem 2 mismatch and on correcting the EIF typos, not on rejecting the EIF derivation wholesale.","tokens_in":19298,"tokens_out":15784,"duration_ms":163345,"concrete_test":"Instrument the released R code from the GitHub repository to record the number of fluctuation steps and the final value of PnD*_{Σ2} with its standard error; verify whether the stopping rule actually drives PnD* below o_p(n^{-1/2}) scale, e.g., |PnD*| ≤ σ_n log(n)/sqrt(n). Then re-run the coverage simulations replacing the fixed dε=0.001 loop with a one-step fluctuation that solves for the step length (or uses a line search) so that PnD* is numerically zero, and compare the resulting coverage and Type I error tables; if they change materially, the published algorithm is not the estimator being claimed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central theoretical claim is that the proposed one-step TMLE for the variance is asymptotically efficient (Theorem 2). The proof in Appendix C requires the targeting step to make PnD*_{Σ2,Pn} = o_p(n^{-1/2}). As written, the algorithm in Section 3.2.4 cannot be expected to do this. Step 5 stops when |PnD*| is no larger than σ_n sqrt(n log n). But PnD* is a sample mean of a mean-zero function, so its typical scale is σ_n / sqrt(n); the threshold is therefore about n log n times too large. The loop effectively stops after one or very few tiny (dε = 0.001) updates, leaving PnD* at O_p(n^{-1/2}), not o_p(n^{-1/2}). Thus the implemented estimator is not a one-step TMLE solving the EIF estimating equation, and Theorem 2 does not apply to it. The simulation results may still describe a useful finite-sample estimator, but they do not substantiate the stated asymptotic efficiency claim. Two additional issues compound this: Corollary 1 (Eq. 14) writes P0 in the clever covariates where the current distribution P is required, which breaks the universal least favorable submodel property in Eq. 19 if implemented literally; and Eq. 10 contains an apparent typo (Q0/ψ1 instead of Q0/ψ0) in the QW component. These are correctable, but they must be checked against the released code before the targeting step can be trusted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a one-step targeted maximum likelihood estimator (TMLE) for the variance of the efficient influence function of the log causal risk ratio (log(CRR)). Using a one-dimensional universally least favorable submodel, the authors update initial estimates of the outcome regression and propensity score, then plug the updated estimates into an analytic variance formula. The manuscript derives the efficient influence function of this variance parameter, states an asymptotic efficiency theorem, and compares the proposed estimator with the empirical influence-function variance, a simple substitution estimator, and an iterative TMLE in simulations across sample sizes, effect sizes, and positivity violations. The reported simulations show that confidence intervals using the one-step TMLE achieve coverage closer to the nominal 0.95 and lower Type-I error than the empirical influence-function variance estimator, especially in small samples and under near-positivity violations. A real-data-based simulation using the WASH Benefits Bangladesh study illustrates the method.","tokens_in":19682,"tokens_out":5126,"duration_ms":49819,"significance":"If the theoretical claims were fully established, the paper would provide a principled way to improve inference for causal risk ratios in finite samples and near-positivity settings, where the standard influence-function variance is known to be anti-conservative. The numerical results are plausible and the code is released, which is a strength. However, the central theoretical contribution is an asymptotic efficiency claim for the proposed one-step TMLE, and that claim is not currently supported: the algorithm as described does not solve the EIF estimating equation, the printed EIF formulas contain apparent typos, and the conditions of the efficiency theorem are not met by the simulation implementation. The finite-sample coverage improvements may well be real, but the manuscript does not yet substantiate the stated efficiency result, and the practical value depends on correcting the targeting step and verifying the formulas against the code.","major_comments":[{"comment":"The stopping criterion in Step 5, |P_n D^{(i)}_{Σ2,P_n}| > σ̂_n √n log n, is dimensionally inconsistent with the requirement P_n D*_{Σ2,P_n} = o_p(n^{-1/2}) that is used in the von Mises expansion in Appendix C. For a mean-zero function, the sample mean P_n D* has typical scale σ̂_n/√n, so the threshold is larger than the typical scale by a factor of order n log n. As a result, the loop will stop after one or very few dε = 0.001 updates, leaving P_n D* at O_p(n^{-1/2}) rather than o_p(n^{-1/2}). The implemented estimator therefore does not solve the EIF estimating equation, and Theorem 2 (Eq. 15) does not apply to the procedure as described. The authors should replace the stopping rule with one that targets |P_n D*| ≤ tol·n^{-1/2} and re-run the simulations and code checks.","section":"Section 3.2.4, Step 5; Theorem 2; Appendix C"},{"comment":"The EIF formulas as printed cannot serve as the validated basis for the targeting step. In Eq. (10), the last term of D*_{Σ2,QW,P} contains (Q̄_1/ψ_1 − Q̄_0/ψ_1)², where the second denominator should be ψ_0. More importantly, the clever covariates H_{1,Q̄,P} and H_{0,Q̄,P} in Eq. (14) are written with P_0 expectations rather than expectations under the current distribution P_ε; the universal least favorable submodel property in Eq. (19) requires the current distribution, so an implementation that literally uses P_0 would break the identity in Eq. (19). These appear to be correctable typos, but they must be fixed and verified against the released code before the targeting update can be trusted. The appendix also defers the component EIFs to a book chapter references, so the manuscript alone does not allow an independent check of the derivation.","section":"Eq. (10) and Corollary 1 (Eqs. 12-14)"},{"comment":"The remainder analysis in Appendix C.1 concludes that asymptotic efficiency requires both Q̄ and g to converge at an n^{-1/4} L2 rate, achievable by HAL, and the paper explicitly notes the estimator is not doubly robust. However, the simulation implementation uses SuperLearner libraries (GLM, elastic net, BART, GAM) rather than HAL, and no rate condition is verified for these libraries in the simulation settings. Consequently, the coverage and Type-I error results in Section 4.2 are finite-sample empirical findings, not consequences of Theorem 2; the manuscript should state this distinction clearly and either add HAL-based simulations or temper the asymptotic efficiency claim.","section":"Appendix C and Section 4.1"}],"minor_comments":[{"comment":"The second term in Eq. (6) and the second term in Eq. (7) are missing the square on ψ_0 in the denominator; compare with Eq. (4) where the factor is 1/(ψ_0)². In Eq. (7), the notation g*_n(0|W) should be g^{(**)}_n(0|W) for consistency.","section":"Eq. (6) and Eq. (7)"},{"comment":"The step size dε = 0.001 is introduced without any sensitivity analysis or guidance on choosing it; the manuscript should report whether results are sensitive to this tuning choice.","section":"Section 3.2.4"},{"comment":"The introduction attributes the iterative TMLE for the variance of the treatment-specific mean to Coyle et al. and cites reference [9], but [9] is the van der Laan and Rose book; the specific chapter or paper by Coyle and colleagues should be cited directly.","section":"References and citations"},{"comment":"The derivation says the component EIFs were obtained using the method in 'A.3 in the targeted learning book [10]', but reference [10] is the van der Laan and Gruber paper; the intended book reference should be corrected.","section":"Appendix B"},{"comment":"Table 1 reports the proportion of observations with propensity scores truncated at 0.025, but the text in Section 4.1 says truncation is within [0.025, 0.975]; the table caption should refer to the estimated propensity scores rather than g_0 and should clarify the truncation interval.","section":"Table 1 and Section 4.1"},{"comment":"The real-data simulation says 10,000 repeated samples of size 500 are drawn, but the analysis is then described as 'based on 140 data samples'; the relationship between these numbers should be explained.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the simulations suggest a useful finite-sample estimator, but the asymptotic efficiency claim is currently unsupported due to the stopping-rule issue and the EIF typos. The authors should be asked to correct the stopping criterion, fix the formulas, and either verify that the code implements the corrected procedure or clearly present the paper as an empirical finite-sample study without the Theorem 2 claim. I would also recommend checking the released code for consistency with the corrected formulas."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a plausible extension of targeted variance estimation to the log causal risk ratio. The derivation of the EIF for the variance of the log(CRR) influence function appears new, and the simulations are designed and reported in a way that gives real, if modest, evidence that an analytic targeted variance estimator beats the naive empirical-IF estimator in small samples and near-positivity. But the paper overclaims: the efficiency theorem does not apply to the estimator the authors actually run.\n\nThe main gap is the stopping rule in Section 3.2.4. Step 5 continues while |PnD| > sigma_hat sqrt(n) log n. For a mean-zero function, PnD has typical scale sigma/sqrt(n), so the threshold is about n log n times too large. The loop stops after one or two tiny updates, long before it solves the EIF estimating equation. So Theorem 2's targeting condition is not met by the implemented procedure. Calling the estimator a one-step TMLE is also misleading: the algorithm as written is an iterative loop that just happens to stop quickly.\n\nThere are also apparent typos in the EIF formulas: Eq. 10 has psi_1 instead of psi_0 inside the QW component, and Corollary 1 uses P0 in the clever covariates where the current distribution P is required, which would break the universal least favorable submodel property in Eq. 19. These are correctable, but they need to be checked against the released code before the targeting step can be trusted. The appendix also says the remainder is o_p(n^{-1/2}) only if Q and g converge faster than n^{-1/4}, typically via HAL; the simulations use SuperLearner with GLM/BART/boosting, which carry no such rate guarantee. So the asymptotic efficiency claim is doubly unsupported.\n\nWhat the paper does well: the analytic form of the target parameter in Eq. 4 is useful, the simulation study covers effect sizes, sample sizes, and positivity levels systematically, the real-data simulation is sensible, and the code is posted. The coverage results are not circular: they compare against Monte Carlo truth and, as empirical findings, they are plausible. But the claimed asymptotic theory does not back the implemented estimator.\n\nI would send this to referees rather than desk reject: the idea is worth engaging and the errors are fixable in principle. But the referees should be asked to verify the EIF and the stopping rule against the code, and the authors should either implement a true one-step solver or scale back the efficiency claim. I would not cite it in its current form.","headline":"Plausible new EIF-based variance estimator for log(CRR), but the paper overclaims asymptotic efficiency for the algorithm it actually implements, and the formulas need a close check before the method can be trusted.","tokens_in":20157,"tokens_out":2327,"would_cite":false,"duration_ms":26268,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G05","62G20","62P10"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a one-step targeted variance estimator whose confidence intervals for the log causal risk ratio reach nominal 0.95 coverage where the standard influence-function variance underestimates.","keywords":["targeted maximum likelihood estimation","influence function","variance estimation","causal risk ratio","positivity violations","one-step TMLE","universal least favorable submodel","confidence interval coverage"],"falsifier":"Numerically verify Theorem 1 by finite-difference Gateaux differentiation of $\\Sigma^2(P)$ along one-parameter submodels for the outcome regression and propensity score on a small simulated dataset; if the numerical derivative disagrees with Eq. 9, especially in the $Q_W$ component, the targeting step is misaligned. Independently run the paper's $n=100$, $\\beta_p=1$, $\\beta_{\\psi_0}=0$ simulation 1000 times: if Wald intervals using the one-step variance estimator cover the true log(CRR) less than 90% of the time, the claimed restoration of nominal coverage fails.","tokens_in":19116,"feed_emoji":"🎯","tokens_out":6708,"duration_ms":66027,"temperature":0.7,"pith_summary":"Standard errors for causal-effect estimates are usually obtained as the sample variance of the estimated influence function, but these standard errors can be too small in small samples or when some treatment probabilities are near zero, producing confidence intervals with coverage far below the claimed 0.95. This paper treats the variance of the influence function itself as the target parameter and derives its efficient influence function, then estimates that variance with a one-step targeted maximum likelihood estimator (TMLE), a plug-in method that makes a small likelihood-based update to remove bias for a chosen target. The result is a plug-in variance estimator whose confidence intervals for the log causal risk ratio hold nominal coverage in simulations, and whose Type I error stays near 0.05 where the usual estimator's error inflates under positivity violations. A reader should care because reliable standard errors are what make causal conclusions trustworthy in the finite samples typical of observational studies.","feed_headline":"One-step variance fix restores 95% coverage for log risk ratios","feed_subtitle":"A targeted variance estimator beats the standard error when samples are small and positivity is near-violated.","key_machinery":"The load-bearing object is the efficient influence function of the variance parameter $\\Sigma^2(P) = P\\{D^*_{\\Psi,P}\\}^2$, decomposed orthogonally into components in the tangent spaces of the covariate distribution, the outcome regression, and the propensity score (Theorem 1, Eq. 9). It is used twice: its clever covariates $H_{\\bar{Q}}$ and $H_g$ define a one-dimensional universal least favorable parametric submodel along which both the outcome regression and propensity score are fluctuated in small logistic steps, and solving its estimating equation is what makes the resulting variance plug-in estimator asymptotically efficient. The one-step TMLE is the engine that carries the argument: one targeted update, rather than iterative refitting, corrects bias for the variance without the additional variance inflation that the paper attributes to iterative TMLE.","core_discovery":"The paper claims that the anti-conservatism of the standard influence-function variance estimator is not an inevitable small-sample artifact but a correctable bias: by defining the variance of the efficient influence function of log(CRR) as a statistical parameter and estimating it with a one-step TMLE, the variance estimator tracks the true sampling variance even when propensity scores approach 0 or 1. The central object is the analytic variance formula, which averages terms with 1/g(1|W) and 1/g(0|W) in the denominators, so rare treatment and outcome patterns are explicitly integrated rather than being missed through indicator functions. The one-step TMLE, built on the derived EIF, targets this variance directly using the universal least favorable submodel, avoiding the overfitting the authors attribute to iterative TMLE. Simulations across effect sizes 0, 0.5, and 2, sample sizes 100 to 1000, and six positivity levels show coverage near 0.95 and Type I error near 0.05, with the empirical estimator dropping below 0.85 coverage and inflating Type I error in extreme cases.","pith_inferences":["Extension: the gap between the empirical influence-function variance and the targeted variance could serve as a data-driven positivity diagnostic—a large gap flags samples where standard inference is unreliable.","Extension: in rare-outcome longitudinal analyses where nonparametric bootstrap has been observed to beat the empirical influence-function variance, the one-step targeted variance may provide a computationally cheaper alternative without bootstrap resampling.","Extension: simultaneously targeting the causal estimand and its variance in a joint TMLE is a direct next step; if realized, it would give jointly efficient estimation and inference.","Extension: the ratio of the one-step TMLE variance to the empirical IF variance could be used to calibrate existing software outputs, giving applied users a finite-sample correction without changing their estimator."],"forward_implications":["Point estimates of log(CRR) from TMLE will come with standard errors that are not systematically too small when treatment probabilities approach zero.","Type I error rates for testing no causal effect will stay near 0.05 in small samples, instead of rising as positivity degrades.","The same derivation template—derive the EIF of the variance of the influence function and target it with a universal least favorable submodel—applies to other causal parameters such as treatment-specific means, risk differences, and odds ratios.","Because the estimator is a plug-in with an analytic form, it can be computed without iteratively refitting the data, avoiding the variance inflation associated with iterative TMLE.","The derived influence function of the variance also opens the possibility of constructing confidence intervals for the variance itself, which could be used in adaptive experiments to bound Type I error."],"supporting_citations":[{"why":"Supplies the Highly Adaptive Lasso rate condition needed to control the second-order remainder in Theorem 2.","marker":"[1]"},{"why":"Supports the claim that one-step TMLE improves finite-sample robustness over iterative updating.","marker":"[3]"},{"why":"Provides the R implementation used to compute TMLE point estimates in the simulations.","marker":"[7]"},{"why":"Supplies the one-dimensional universal least favorable submodel that the proposed one-step TMLE relies on.","marker":"[8]"},{"why":"Provides the iterative TMLE for the variance of the EIF of the treatment-specific mean that the paper follows and extends.","marker":"[9]"},{"why":"Supplies the method for deriving component efficient influence functions used in the appendix.","marker":"[10]"},{"why":"Provides the ensemble learning algorithm used to obtain initial estimates of the outcome regression and propensity score.","marker":"[11]"},{"why":"Documents the anti-conservative behavior of variance estimators under positivity violations, motivating the need for a targeted variance estimator.","marker":"[19]"}],"fun_headline_variants":["Targeted variance estimator boosts coverage for log risk ratios","One-step TMLE fixes variance inflation in small samples","Improved inference: targeted variance for causal risk ratio","Variance estimator hits 95% coverage despite positivity issues","Targeted variance lowers Type I error in causal inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The derived efficient influence function in Theorem 1 must be correct; the appendix obtains component influence functions from a book chapter and the printed equation contains apparent typographical slips (for example, a $\\psi_1$ where $\\psi_0$ is expected), so an error here would move the targeting update in the wrong direction and the claimed efficiency would collapse.","fun_headline_variants_meta":{"raw":{"variants":["Targeted variance estimator boosts coverage for log risk ratios","One-step TMLE fixes variance inflation in small samples","Improved inference: targeted variance for causal risk ratio","Variance estimator hits 95% coverage despite positivity issues","Targeted variance lowers Type I error in causal inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00113,"raw_usage":{"total_tokens":4755,"prompt_tokens":1060,"completion_tokens":3695,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":676,"completion_tokens_details":{"reasoning_tokens":3619}},"tokens_in":676,"tokens_out":3695,"duration_ms":23958,"temperature":1.0,"reasoning_tokens":3619,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:06:32.004676+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Numerically verify Theorem 1 by finite-difference Gateaux differentiation of $\\Sigma^2(P)$ along one-parameter submodels for the outcome regression and propensity score on a small simulated dataset; if the numerical derivative disagrees with Eq. 9, especially in the $Q_W$ component, the targeting step is misaligned. Independently run the paper's $n=100$, $\\beta_p=1$, $\\beta_{\\psi_0}=0$ simulation 1000 times: if Wald intervals using the one-step variance estimator cover the true log(CRR) less than 90% of the time, the claimed restoration of nominal coverage fails.","supporting_citations":[{"cited_title":"The Highly Adaptive Lasso Estimator","cited_arxiv_id":null,"evidence_quote":"Supplies the Highly Adaptive Lasso rate condition needed to control the second-order remainder in Theorem 2."},{"cited_title":"One-Step Targeted Minimum Loss-based Esti- mation Based on Universal Least Favorable One-Dimensional Submodels","cited_arxiv_id":null,"evidence_quote":"Supplies the one-dimensional universal least favorable submodel that the proposed one-step TMLE relies on."},{"cited_title":"Targeted Learning in Data Science: Causal Inference for Complex Longitudinal Studies","cited_arxiv_id":null,"evidence_quote":"Provides the iterative TMLE for the variance of the EIF of the treatment-specific mean that the paper follows and extends."},{"cited_title":"Targeted Minimum Loss Based Estimation of an Intervention Specific Mean Outcome","cited_arxiv_id":null,"evidence_quote":"Supplies the method for deriving component efficient influence functions used in the appendix."}],"review_version":1}