REVIEW 3 major objections 4 minor 3 cited by
Regularized Targeted Maximum Likelihood Estimation in Highly Adaptive Lasso Implied Working Models
T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Two regularized updates let HAL-based TMLE remove first-order bias while staying inside the working model, even when p exceeds n.
desk verdict Projection-based regHAL-TMLE is a real algorithmic improvement with strong simulation support, but the paper's central guarantee of first-order bias removal rests on an unproved sup-norm rate. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the working-model efficient influence curve, $D^*_{M_n}(P) = \gamma^T S_\beta$, where $S_\beta$ is the vector of HAL score functions and $\gamma$ solves $\gamma^T I_P(\beta) = \partial_\beta \Psi(Q_\beta)$. The delta-method variant estimates $\gamma$ by replacing the inverse Fisher information with a ridge-regularized inverse, $(I_{P_n} + \eta I)^{-1}$, while the projection variant estimates $\gamma$ by a lasso regression of an initial influence function on the score columns, avoiding the inverse entirely. The identity that carries the argument is Theorem 5.2, the coincidence of the projected parameter's influence curve with the working model's parametric influence curve inside $M_n$. The algorithm then updates the HAL coefficients in the direction of the approximated influence curve until the empirical mean of the EIC falls below a tolerance set by its standard error, turning targeting into a sequence of coefficient updates.
What would settle it
Construct a data-generating process where the true regression function has a sharp discontinuity that the HAL sieve approximates only slowly; for increasing n, measure the sup-norm distance between the best approximation in the active HAL basis and the truth. If that distance does not decay faster than $n^{-1/2}$, the remainder term used to prove Theorem 5.5 is not controlled, and confidence intervals built from the projected EIC should fail to reach nominal coverage.
Extended reading notes
Core claim
On its own terms, the paper establishes that the finite-dimensional working model selected by HAL is enough for valid TMLE, provided that the target parameter's influence curve is approximated inside that model with regularization. Theorem 5.2 shows that inside the working model the parametric efficient influence curve coincides with the full-model efficient influence curve, and Theorem 5.5 gives the asymptotic linearity statement $\Psi_{M_n}(P^*_{M_n}) - \Psi(P_0) = P_n D^*_{M_0}(P_0) + o_p(n^{-1/2})$ for the original target parameter. The lasso-projected influence curve is the engine: projecting an initial gradient onto the HAL score space with an $\ell^1$ penalty produces a stable update direction that removes first-order bias without requiring a matrix inverse. In the paper's simulations, the projection update reduces absolute bias by 50-85 percent for average treatment effects and by roughly 98 percent for survival curves compared with relaxed HAL, while keeping confidence interval coverage near nominal. The ridge-regularized delta-method variant is presented as the alternative when projection is not practical.
Load-bearing premise
The proof relies on the HAL-selected working model being so close to the true regression function that the largest pointwise error shrinks faster than one over the square root of the sample size, a rate the paper invokes from the HAL sieve but does not derive from its own stated assumptions.
Editorial extensions
If this is right
- Projection-based regHAL-TMLE is reported to cut absolute bias by 50-85 percent for average treatment effect estimation compared with relaxed HAL, and by roughly 98 percent for survival curves, while keeping coverage near nominal.
- Because each update modifies only the coefficient vector inside the working model, survival-curve estimation no longer requires recursive clever-covariate integrations, and the per-iteration cost is $O(p)$.
- For distributions inside the working model, targeting the projected parameter with the within-model EIC is equivalent to targeting the original parameter with the full-model EIC.
- The plateau-selector extension provides a data-driven way to grow the working model, selecting simpler models under positivity violations and more complex models as the sample size grows.
Reading between the lines
- The same regularized projection could treat arbitrary initial estimators, such as random forests or neural networks, as high-dimensional parametric fits in their implicit feature spaces; the paper sketches this as a general TMLE blueprint but does not develop it.
- Viewing one-step debiased lasso as an unbounded linear special case suggests that regularized targeting could replace debiasing corrections in collinear designs, trading a little bias for more stable confidence intervals; the paper identifies this connection but does not test it.
- A testable extension is to tune the projection penalty by the stability of the resulting EIC rather than fixing a tiny lambda, since the paper acknowledges that data-driven penalty selection lacks theoretical guarantees.
- The idea of targeting in one working model and inferring in a richer one could improve coverage under misspecification; the paper lists this as future work rather than a demonstrated result.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two regularized Targeted Maximum Likelihood Estimation procedures that operate inside the finite-dimensional working model implied by a Highly Adaptive Lasso fit: a delta-method regHAL-TMLE that uses a ridge-stabilized inverse of the empirical information matrix, and a projection-based regHAL-TMLE that uses a lasso-regularized projection of an influence function onto the HAL score space. The main theoretical claim, Theorem 5.5, asserts that the working-model TMLE is asymptotically linear for the original target parameter with influence curve equal to the EIC in a limiting oracle model. The paper also presents an adaptive extension with a plateau selector, and supports the methods with simulations for average treatment effects, survival curves, and adaptive model selection. The empirical section is extensive and consistently favors the projection-based estimator over relaxed HAL and delta-method alternatives.
Significance. If the theoretical guarantee were fully established, this would be a practically valuable contribution: it offers a computationally stable way to perform TMLE inside HAL-induced working models, avoiding ill-conditioned matrix inverses and expensive clever-covariate constructions, and the simulation evidence is strong and reproducible from the provided GitHub repository. The paper's central advertised property, however, is not currently backed by a complete proof. The gap in condition (C.3) and the mismatch between the EIC solved by the algorithm and the EIC used in the theorems are load-bearing, so the paper cannot yet be accepted as providing the claimed first-order bias-removal guarantee. The extensive simulations and reproducible code are genuine strengths and give reason to believe the practical claims, but the theoretical section needs substantial repair.
major comments (3)
- [Section 5.4, condition (C.3), and Appendix D.4] The verification of condition (C.3) in Appendix D.4 uses the assertion that ||Q_{0,M_n} - Q_0||_infty = o(n^{-1/2}) 'by the HAL-sieve rate', and the same sup-norm rate is invoked in Appendix D.3 for the remainder R(P_{0,M_n}, P_0, M_0). However, Assumption 5.6 only states sup-norm convergence o_P(1) and L2 convergence O_P(n^{-r}) with r > 1/4. A sup-norm rate of order o_P(n^{-1/2}) is not a consequence of the stated assumption and is not proved anywhere in the manuscript or in the cited results. Because (C.3) is exactly the term P_0 D*_{M_n}(P_{0,M_n}) used to show that the working-model target equals the original target up to o_p(n^{-1/2}), Theorem 5.5 does not establish the advertised first-order bias removal for the original target in the stated generality. The assumption r > 1/4 only yields O_P(n^{-1/2}) for the product term in (C.2), and the additional sup-norm rate in (C.3) needs to be either proved or added as an explicit assumption.
- [Section 5.2 and Algorithm 1] Theorem 5.4 assumes that the regHAL-TMLE P*_{M_n} solves P_n D*_{M_n}(P*_{M_n}) = o_p(n^{-1/2}) for the working-model EIC D*_{M_n}. Algorithm 1, however, computes an approximated or regularized EIC D*_{n,beta,Q} and stops when |P_n D*_{n,beta,Q}| < se(D*_{n,beta,Q})/(sqrt(n) log n). No theorem or proof shows that the regularized/projected D*_{n,beta,Q} is close enough to the true working-model EIC D*_{M_n} that the empirical-mean equation assumed in Theorem 5.4 is satisfied, nor is there a bound on the difference between the two influence curves. In particular, the lasso projection with penalty lambda and the ridge parameter eta are algorithmic choices, yet the asymptotic results are stated for the exact parametric EIC. The first-order bias-removal guarantee therefore remains conditional on an unverified relationship between the algorithm's stopping criterion and the theorem's EIC equation.
- [Section 5.4, conditions (A.2) and (B.2)] The proof of conditions (A.2) and (B.2) asserts that the map P -> D*_{M_n}(P) is Lipschitz in sup-norm because Psi_{M_n} is pathwise differentiable on the finite-dimensional model M_n. This Lipschitz property is not demonstrated. The working-model EIC has the form gamma(P)^T S_beta, where gamma involves the inverse Fisher information, so a uniform Lipschitz bound in sup-norm is not automatic and may fail as the information matrix becomes nearly singular. Since (A.2) and (B.2) are needed for the empirical-process equicontinuity steps in Theorems 5.4 and 5.5, this is another load-bearing point in the theoretical verification that should either be proved under explicit conditions on the HAL basis and target parameter or replaced by a weaker sufficient condition.
minor comments (4)
- [Appendix D.4] The section header 'D.4 Proof for theorem 5.4' is a duplicate; the content proves the verification of the regularity conditions in Section 5.4 and should be labeled accordingly, e.g., 'Proof of Theorem 5.7'.
- [Section 1, bullet (a)] The introduction states that both updates 'guarantee first-order bias removal', but in light of the gaps in Section 5 this is stronger than what the theorems currently establish; the wording should be softened or conditioned on the additional sup-norm rate and EIC-equation assumptions.
- [Section 7.2 and Section 10.1] There are minor typos: 'statstical' should be 'statistical' in Section 7.2, and 'collniearity' should be 'collinearity' in Section 10.1.
- [Section 8.2] The plateau selector in Algorithm 2 is described only by an intuitive rule and no formal guarantee is given for its selected model complexity; this is acceptable as a heuristic, but the text should clearly separate this algorithmic heuristic from the formal asymptotic claims in Section 5.
Circularity Check
Theorem 5.5's guarantee is imported from same-group reference [10], and its condition check assumes a sup-norm rate not stated in Assumption 5.6; the algorithmic proposal is supported independently by simulations.
-
self citation load bearing
[Section 5 opening; Theorem 5.5; Appendix D.4 (verification of (C.3))]
"In this section, we closely followed the theorems and the proofs in [10]. ... (C.3). Finally P0D∗Mn(P0,Mn) = (P0−P0,Mn)D∗Mn(P0,Mn) ≤ ∥Q0,Mn−Q0∥∞∥D∗Mn(P0,Mn)∥∞ = oP(n−1/2), since ∥Q0,Mn−Q0∥∞ = o(n−1/2) by the HAL-sieve rate."
The paper's headline guarantee ('guarantee first-order bias removal', Section 1) is formalized by Theorem 5.5. Rather than proving this theorem from the paper's own assumptions, Section 5 imports it from reference [10], which shares senior authorship with the present paper and is the same-group source for both the theorem statements and their proofs. The Appendix D verification of the needed condition (C.3) further asserts a sup-norm rate o(n^{-1/2}) 'by the HAL-sieve rate' that is stronger than Assumption 5.6's stated o_P(1) sup-norm and L2 O_P(n^{-r}) with r>1/4. Thus the advertised first-order bias-removal guarantee is not independently derived in this manuscript; it is inherited from a same-group citation plus an unstated rate assumption.
full rationale
The only genuine circularity concern is the theoretical guarantee. Section 5 explicitly says the theorems and proofs are 'closely followed' from [10], and Theorem 5.5 is the formal statement of the paper's central claim that the updates guarantee first-order bias removal for the original target. Because [10] is a same-group preprint and the proof adaptation depends on [12] (also same senior author) for the HAL-sieve rate, the asymptotic result is not self-contained here. Moreover, the verification of (C.3) in Appendix D.4 uses a sup-norm rate o(n^{-1/2}) that is not stated in Assumption 5.6, so the theorem is not even connected to the paper's stated assumptions; this is a proof gap adjacent to the circularity concern. I do not count the algorithmic steps as circular: the lasso/ridge EIC approximations in Section 4 are new proposals, the stopping rule is a tolerance rather than a fitted parameter renamed as a prediction, and the simulations compare estimators on independent DGPs. The plateau selector and the debiased-lasso connection are either attributed to [10] or presented as observations, not derived from the target result. Hence a moderate score of 4 reflects the imported, load-bearing same-group theory while recognizing the independent algorithmic and empirical content.
Assumptions & free parameters
free parameters (5)
- ridge_eta_delta_method =
1e-6
- lasso_lambda_projection =
1e-5
- gradient_step_size =
1e-4 (ATE), 1e-3 (survival)
- stopping_threshold =
se(D*)/(sqrt(n) log n)
- undersmoothing_factor_survival =
1.61
assumptions (4)
- domain assumption Assumption 5.1: projection differentiability of Pi_{M_n} and smoothness of Psi on M_n.
- domain assumption Assumption 5.6: Q_0 and all nuisances are cadlag with bounded sectional variation; HAL-MLE has sup-norm o_P(1) and L2 rate O_P(n^{-r}) with r > 1/4.
- standard math Donsker class properties: finite-dimensional spans of Donsker classes are Donsker, and the cadlag bounded-variation class is Donsker.
- ad hoc to paper The HAL sieve achieves sup-norm approximation error o_P(n^{-1/2}) in the proof of condition (C.3).
Cite this review
Pith. "Pith review of Regularized Targeted Maximum Likelihood Estimation in Highly Adaptive Lasso Implied Working Models." pith.science (2026). https://pith.science/paper/4H2D3FPN
@misc{pith2026250617214,
author = {Pith},
title = {Pith review of: Regularized Targeted Maximum Likelihood Estimation in Highly Adaptive Lasso Implied Working Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/4H2D3FPN}},
note = {Machine review of arXiv:2506.17214}
}
read the original abstract
We address the challenge of performing Targeted Maximum Likelihood Estimation (TMLE) after an initial Highly Adaptive Lasso (HAL) fit. Existing approaches that utilize the data-adaptive working model selected by HAL-such as the relaxed HAL update-can be simple and versatile but may become computationally unstable when the HAL basis expansions introduce collinearity. Undersmoothed HAL may fail to solve the efficient influence curve (EIC) at the desired level without overfitting, particularly in complex settings like survival-curve estimation. A full HAL-TMLE, which treats HAL as the initial estimator and then targets in the nonparametric or semiparametric model, typically demands costly iterative clever-covariate calculations in complex set-ups like survival analysis and longitudinal mediation analysis. To overcome these limitations, we propose two new HAL-TMLEs that operate within the finite-dimensional working model implied by HAL: Delta-method regHAL-TMLE and Projection-based regHAL-TMLE. We conduct extensive simulations to demonstrate the performance of our proposed methods.
Forward citations
Cited by 3 Pith papers
-
Targeted Highly Adaptive Lasso Minimum Loss Estimation of Target Functions
Targeted HAL-MLE projects a non-pathwise-differentiable target onto a HAL spline working model, applies LASSO-targeted TMLE, and attains dimension-free pointwise asymptotic normality up to log n factors.
-
When Does Trial-Real-World Data Fusion Improve Precision? Model Auditing and Selection-Aware Inference for Adaptive-TMLE
Under A-TMLE, RCT+RWD efficiency gain is driven by bias magnitude not complexity, crosses break-even near one residual SD of bias, erodes with sample size, and only a block jackknife gives honest intervals for the gain.
-
Targeted Deep Architectures: A TMLE-Based Framework for Robust Causal Inference in Neural Networks
TDA updates a small subset of a neural network's weights along a projected influence-function gradient, debiasing plug-in causal estimates and giving valid confidence intervals.
Reference graph
Works this paper leans on
-
[10]
Lars van der Laan, Marco Carone, Alex Luedtke, and Mark van der Laan. Adaptive debiased machine learning using data-driven model selection techniques.arXiv preprint arXiv:2307.12544, 2023
arXiv 2023
-
[1]
The highly adaptive lasso estimator
David Benkeser and Mark Van Der Laan. The highly adaptive lasso estimator. In2016 IEEE interna- tional conference on data science and advanced analytics (DSAA), pages 689–696. IEEE, 2016
work page 2016
-
[2]
Aur´ elien F Bibaut and Mark J van der Laan. Fast rates for empirical risk minimization over c\adl\ag functions with bounded sectional variation norm.arXiv preprint arXiv:1907.09244, 2019. 15
arXiv 1907
-
[3]
The sandwich (robust covariance matrix) estimator.Unpublished manuscript, 1998
RJ Carroll, Suojin Wang, DG Simpson, AJ Stromberg, and D Ruppert. The sandwich (robust covariance matrix) estimator.Unpublished manuscript, 1998
work page 1998
-
[4]
High-dimensional simultaneous inference with the bootstrap.Test, 26:685–719, 2017
Ruben Dezeure, Peter B¨ uhlmann, and Cun-Hui Zhang. High-dimensional simultaneous inference with the bootstrap.Test, 26:685–719, 2017
work page 2017
-
[5]
Cheng Ju, Susan Gruber, Samuel D Lendle, Antoine Chambaz, Jessica M Franklin, Richard Wyss, Sebastian Schneeweiss, and Mark J van Der Laan. Scalable collaborative targeted learning for high- dimensional data.Statistical methods in medical research, 28(2):532–554, 2019
work page 2019
-
[6]
Springer International Publishing,, 2018
Sherri Rose and Mark J van der Laan.Targeted learning in data science: causal inference for complex longitudinal studies. Springer International Publishing,, 2018
work page 2018
-
[7]
One-step TMLE for targeting cause-specific absolute risks and survival curves
Helene CW Rytgaard and Mark J van der Laan. One-step tmle for targeting cause-specific absolute risks and survival curves.arXiv preprint arXiv:2107.01537, 2021
work page Pith review arXiv 2021
Show all 23 references
-
[8]
Regression shrinkage and selection via the lasso.Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267–288, 1996
Robert Tibshirani. Regression shrinkage and selection via the lasso.Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267–288, 1996
1996
-
[9]
Springer, 2006
Anastasios A Tsiatis.Semiparametric theory and missing data, volume 4. Springer, 2006
2006
-
[11]
A generally efficient targeted minimum loss based estimator based on the highly adaptive lasso.The international journal of biostatistics, 13(2), 2017
Mark van der Laan. A generally efficient targeted minimum loss based estimator based on the highly adaptive lasso.The international journal of biostatistics, 13(2), 2017
2017
-
[12]
Higher order spline highly adaptive lasso estimators of functional parameters: Pointwise asymptotic normality and uniform convergence rates.arXiv preprint arXiv:2301.13354, 2023
Mark van der Laan. Higher order spline highly adaptive lasso estimators of functional parameters: Pointwise asymptotic normality and uniform convergence rates.arXiv preprint arXiv:2301.13354, 2023
2023 arXiv
-
[13]
Springer, 2011
Mark J Van der Laan, Sherri Rose, et al.Targeted learning: causal inference for observational and experimental data, volume 4. Springer, 2011
2011
-
[14]
A generally efficient hal-tmle.Targeted Learning in Data Science: Causal Inference for Complex Longitudinal Studies, pages 95–102, 2018
Mark J van der Laan, Sherri Rose, and Mark J van der Laan. A generally efficient hal-tmle.Targeted Learning in Data Science: Causal Inference for Complex Longitudinal Studies, pages 95–102, 2018
2018
-
[15]
Springer, 1996
Aad W Van Der Vaart, Jon A Wellner, Aad W van der Vaart, and Jon A Wellner.Weak convergence. Springer, 1996
1996
-
[16]
Targeted maximum likelihood based estimation for longitudinal mediation analysis.Journal of Causal Inference, 13(1):20230013, 2025
Zeyi Wang, Lars van der Laan, Maya Petersen, Thomas Gerds, Kajsa Kvist, and Mark van der Laan. Targeted maximum likelihood based estimation for longitudinal mediation analysis.Journal of Causal Inference, 13(1):20230013, 2025
2025
-
[17]
Cun-Hui Zhang and Stephanie S Zhang. Confidence intervals for low dimensional parameters in high dimensional linear models.Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(1):217–242, 2014
2014
-
[18]
Regularization and variable selection via the elastic net.Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(2):301–320, 2005
Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net.Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(2):301–320, 2005. 16 A regHAL-TMLE Algorithm for Multi Dimensional Target Pa- rameter Algorithm 3HAL Updating Alg...
2005
-
[19]
Fit auxiliary nuisances.Estimate the propensity scoreg(or any other nuisance quantities that enter the clever covariate) using HAL or any compatible learner; denote the resulting estimate by ˆg
-
[20]
Compute clever covariate.For each observationO i = (W i,Ai,Yi) set ˆHi = ˆH(Ai,Wi) with ˆH(a,w) =a/ˆg(w)−(1−a)/(1−ˆg(w))
-
[21]
Target within the HAL basis.Re-express ˆHin the span of the outcome HAL basis by solving ˆα= argmin α∈Rp 1 n nX i=1 ˆHi−α⊤ϕ(Ai,Wi) 2 +λ∥α∥ 1. Theℓ 1 penaltyregularises the clever covariate itselfrather than the score projection used in Sections 4.2, which is computationally ch...
-
[22]
Choose the (possibly line-searched or run linear regression) step size ˆεthat maximize the log likelihood (or minimize the loss) and set ˆβ=β (0) + ˆεˆα
One-step coefficient update.Define the fluctuation pathQ fluct β(0)(ε) =Q β(0) +ε ˆH≈β (0)⊤ϕ+εˆα⊤ϕ. Choose the (possibly line-searched or run linear regression) step size ˆεthat maximize the log likelihood (or minimize the loss) and set ˆβ=β (0) + ˆεˆα. The updated fitQ ˆβ is ...
-
[23]
finer-grained
Variance and inference.The estimated EIC is now ˆD∗ i = ˆα⊤ϕ(Ai,Wi) Yi−Q ˆβ(Ai,Wi) + Qˆβ(1,Wi)−Q ˆβ(0,Wi) −Ψ(Q ˆβ),and (n−2P i ˆD∗2 i )1/2 gives the usual Wald standard error. 26 E.2 Why does it work? Because the ULFM for the ATE islinearinH, any ˆHthat admits a (possibly pena...
2000
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.