REVIEW 2 major objections 4 minor 56 references
For semiparametric estimation with black-box nuisance learners, TAME attains the unimprovable error rate n^{-1/2} + δ_appr_μ·δ_appr_π + (δ_stoc_n)^2, removing the mixed term that double machine learning pays.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 07:08 UTC pith:GP2TEX65
load-bearing objection The upper bound is a real advance on DML rates, but the matching lower bound is only asserted by analogy, so the 'unimprovable' claim is not yet supported. the 2 major comments →
Optimal use of a black-box learner in semiparametric estimation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that the structure-agnostic optimal rate for estimating the linear coefficient is n^{-1/2}+δ_appr_μδ_appr_π+(δ_stoc_n)^2, attainable for any black-box class with the stated error budgets and provably unimprovable even when the class is a sparse linear model with an orthonormal dictionary. The estimator achieves this by selecting one of two branches: when the outcome function is easier, it fine-tunes the treatment-side debiasing weights on the inference sample; when the treatment function is easier, it fine-tunes the outcome-side weights using a cross-sample objective. In both cases the weights are kept close to the black-box-induced weights while approximately sa
What carries the argument
TAME-1 (and its generic version TAME) is a minimax weight-editing program: on the inference sample it solves argmin_a λ·(1/n)Σ(a_i−ê_a_i)^2 + sup over the class's difference set of an adversarial moment functional, where ê_a is the debiasing weight vector that double machine learning would use. The first term (local editing) preserves the beneficial product structure of the approximation errors; the second term (adversarial moment calibration) forces the higher-order stochastic error to appear only squared. The outcome-harder branch uses a cross-sample construction so that the edited weights stay independent of the treatment residuals, preserving conditional mean-zero structure.
Load-bearing premise
The procedure requires knowing the learner's error budgets (the two approximation errors and the stochastic error) and the ordering between the approximation errors; if these are unknown or mis-specified, the branch choice and λ are not guaranteed to yield the advertised rate.
What would settle it
Find a hypothesis class with known budgets for which, with non-negligible probability, the oracle weights do not satisfy the adversarial moment constraint (i.e., the feasible set is empty), and show the error then contains a leftover mixed term such as max(δ_appr_μ,δ_appr_π)·δ_stoc_n. Conversely, the paper's lower bound predicts that in the sparse-linear class with the same budgets no estimator can beat n^{-1/2}+δ_appr_μδ_appr_π+(δ_stoc_n)^2; a counterexample would refute the no-gap claim.
If this is right
- Asymptotic normality of the coefficient estimate is possible even if one nuisance function is only consistently estimable, provided the other is well estimated and the easier learner is under-smoothed.
- The lower bound shows that being agnostic to the nuisance structure costs nothing relative to the worst-case structure-aware model with the same error budgets.
- TAME can be combined with any initial black-box estimates; when nuisance difficulties are imbalanced it strictly improves on double machine learning's guarantee.
- A two-learner variant (TAME-2) improves further, and the model-selection rule that emerges is to under-smooth the easier nuisance and balance the harder one.
- The O-branch extends to general linear functionals and the R-branch to average treatment effect estimation, with analogous oracle-type inequalities.
Where Pith is reading between the lines
- A practical extension the paper does not develop: an adaptive version that estimates the error budgets from the data; if the budget estimates are biased the rate may degrade, so a sensitivity analysis of the branch choice would be valuable.
- The transductive weight-editing idea could be applied to other debiased estimators beyond the ones listed, e.g., quantile or survival functionals, where the same mixed-error term appears.
- The under-smoothing principle for TAME-2 suggests a concrete testable rule in neural-network benchmarks: choose the easier nuisance's network complexity above its bias-variance balance point and the harder one at the balance point.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies structure-agnostic estimation of the linear coefficient in the partial linear model when the nuisance functions are estimated by a generic black-box hypothesis class. It characterizes the class by three budgets — approximation errors for the outcome and treatment regressions and a stochastic-error radius — and proposes TAME-1, a debiased estimator claimed to attain |θ̂−θ0| ≲ n^{-1/2} + δ_appr_μ δ_appr_π + (δ_stoc_n)^2, removing the DML mixed term (δ_appr_μ∨δ_appr_π)δ_stoc_n. The paper also develops a two-learner version TAME-2, neural-network rate comparisons, and extensions to general linear functionals and ATE estimation. The upper-bound proof of Theorem 2.1 is detailed; the matching lower bound, however, is stated without proof.
Significance. If the matching lower bound can be established, the paper is a substantial contribution: it would show that the DML first-order stochastic nuisance errors are removable in a fully structure-agnostic setting, recover known sparse-linear rates as special cases, and provide a no-gap result between structure-agnostic and worst-case structure-aware estimation. The Section 5 proof of the upper bound is careful and appears internally consistent, and the extensions to two learners and to ATE estimation are nontrivial. The main weakness is that the paper's headline optimality claim currently rests on an unproved proposition; once that proof is supplied, the contribution is likely to be of high interest to the semiparametric and machine-learning communities.
major comments (2)
- [Section 2.5 (Proposition 2.3)] Proposition 2.3 is the sole basis for the 'matching lower bound' and the no-gap claim, but no proof is given anywhere in the manuscript. Section 5 proves only Theorem 2.1, and the text states the result is 'analogous to Proposition 3.1 in Gu et al. (2026)' with the two lower-bound components following 'similarly' from Balakrishnan et al. (2023) and Gu et al. (2026). This is not a routine combination: the product misspecification bound and the sparse-linear slog(p)/n bound must be realized simultaneously in a single experiment under the same orthonormal dictionary, boundedness constraints, and constant conditional variance E[u^2|X]≡v. If the two constructions only yield a maximum rather than a sum, the advertised additive rate is not established. Because 'unimprovable' is the headline contribution, this missing proof is load-bearing and must be supplied or replaced by a precise reference
- [Section 2.2 (Eq. (2.3), Corollary 2.2)] The estimator is not adaptive: it requires knowledge of the ordering between δ_appr_μ and δ_appr_π, and λ must be tuned from the full error budgets. Corollary 2.2's choice λ=(δ_stoc_n/(δ_max+δ_stoc_n))^2 depends on these budgets, and the branch choice in (2.3) depends on the ordering. The manuscript is transparent about this in Section 2.2, but the abstract's claim that the improvement over DML is obtained 'at no extra cost or assumption' overstates the situation, since DML does not require the analyst to know these budgets or the ordering in order to implement the estimator. No adaptive procedure or sensitivity analysis is provided. The authors should either develop a data-driven choice or explicitly qualify the practical scope of the advertised rate.
minor comments (4)
- [Abstract and Section 2] Notation is inconsistent: the abstract uses δ_s and δ_a, while the body uses δ_stoc_n and δ_appr_μ, δ_appr_π. Please unify the notation throughout.
- [Section 5, Proposition 5.1] Proposition 5.1 is a restatement of Proposition S2.3 of Gu et al. (2026), and the proof is deferred to that paper. For a standalone journal submission, either include the proof or state the precise lemma with a full reference so the reader can verify the initial-estimate bounds without consulting a separate preprint.
- [Theorem 2.1 and Corollary 2.2] The probability bounds differ: Theorem 2.1 states 1−2e^{−t}−2e^{−n(δ_stoc_n)^2}, while Corollary 2.2 states 1−3e^{−t}−2e^{−n(δ_stoc_n)^2}. Please reconcile the constants.
- [Corollary 3.2, Eq. (3.7)] The expression (δ_appr_μ δ_π + δ_μ^2) ∧ (δ_appr_π δ_μ + δ_π^2) is potentially ambiguous. Add explicit parentheses: (δ_appr_μ δ_π + δ_μ^2) ∧ (δ_appr_π δ_μ + δ_π^2).
Circularity Check
No circular reduction found; the central upper bound is self-contained, but the matching lower bound is stated without proof via analogy to prior work (rigor gap, not circularity).
full rationale
I walked the claimed derivation chain. The main upper bound (Corollary 2.2) follows from Theorem 2.1, whose proof in Section 5 is essentially self-contained; the only imported technical piece is Proposition 5.1, explicitly a restatement of Proposition S2.3 in the authors' own Gu et al. (2026). That initial-estimate bound is a standard empirical-process result and does not already contain the target rate, so citing it is not a circular reduction. The matching lower bound, Proposition 2.3, is asserted with the sentence 'The following proposition is analogous to Proposition 3.1 in Gu et al. (2026)' and is not proved in the manuscript. This is a serious omitted-proof/rigor issue for the headline optimality claim, but it is not circularity: there is no displayed derivation in which the lower bound reduces to its inputs by construction, nor is a fitted parameter renamed as a prediction. Similarly, the TAME-2 comparison in Corollary 3.2 is a direct application of the oracle-type inequality rather than a renaming of the DML result. The requirement in Section 2.2 that the estimator know the ordering of the error budgets and tune lambda accordingly is an explicit oracle assumption, not a hidden fit of the target quantity. Because the central derivation does not assume the result it claims to establish, the score reflects only minor self-citation in technical lemmas and the lower-bound analogy, not load-bearing circularity.
Axiom & Free-Parameter Ledger
free parameters (1)
- λ (tuning parameter) =
λ = (δ_stoc_n / (δ_appr_max + δ_stoc_n))^2 in Corollary 2.2; oracle choices in Theorems 3.1 and 4.1
axioms (6)
- domain assumption Boundedness and nondegeneracy of the model (Condition 2.1): |β0|, ||μ0||∞, ||π0||∞, |ε|, |u| ≤ c, and E[u^2] ≥ 1/c.
- domain assumption The black-box class G has stochastic error δ_stoc_n defined via localized Rademacher complexity (Condition 2.2).
- ad hoc to paper The error budgets (δ_appr_μ, δ_appr_π, δ_stoc_n) and the ordering of δ_appr_μ vs δ_appr_π are known to the analyst.
- domain assumption For the generic TAME procedure, the reference class G0 must contain the estimated easier nuisance: bμ ∈ G0 (O-branch) or bπ ∈ G0 (R-branch), as in Condition 3.1.
- domain assumption Initial estimates are independent of the inference split D1.
- standard math Standard empirical process inequalities and concentration bounds for local Rademacher complexities.
read the original abstract
Consider the partial linear model $Y = \mu_0(X) + \beta_0 \cdot T + \varepsilon$ and $T = \pi_0(X) + u$ in the structure-agnostic setting, where we are blind to the structure $\mu_0$ and $\pi_0$ and estimate the nuisances by a black-box hypothesis class. The learnability of the class is characterized by the estimation error $\delta_s$ in the absence of model misspecification and its $L_2$ mis-specification error $\delta_{a, \mu}$ and $\delta_{a, \pi}$ for $\mu_0$ and $\pi_0$, respectively. We propose a novel estimator of the target linear coefficient $\theta_0 = \beta_0$ with error rate \[ \frac{1}{\sqrt{n}} + \delta_{a, \mu} \cdot \delta_{a, \pi} + [\delta_s]^2. \] A matching lower bound is also established, implying that this rate is unimprovable. Compared with the product rate yielded by double machine learning (DML), our estimator removes the suboptimal term $\max(\delta_{a, \mu}, \delta_{a, \pi})\cdot \delta_s$ at no extra cost or assumption. Building on the underlying insights, which are neither tailored to the one-learner setting nor the partial linear model, we propose Transductive Adversarial Moment-calibrated Editing (TAME), which locally edits debiasing weights induced by black-box regression estimates on the inference sample through adversarial conditional moment calibration. TAME can be combined with any initial black-box estimates and can strictly improve on DML guarantees when the nuisance difficulties are imbalanced. We discuss how to fully exploit the advantages introduced by TAME, including the gains from using two learners, the resulting under-smoothing principle for model selection, and extensions to other linear functional estimation problems.
Reference graph
Works this paper leans on
-
[1]
& Chen, X
Ai, C. & Chen, X. (2003). Efficient estimation of models with conditional moment restrictions containing unknown functions. Econometrica , 71(6), 1795--1843
2003
-
[2]
W., & Wager, S
Athey, S., Imbens, G. W., & Wager, S. (2018). Approximate residual balancing: debiased inference of average treatment effects in high dimensions. Journal of the Royal Statistical Society Series B: Statistical Methodology , 80(4), 597--623
2018
-
[3]
Balakrishnan, S., Kennedy, E. H., & Wasserman, L. (2023). The fundamental limits of structure-agnostic functional estimation. arXiv preprint arXiv:2305.04116
Pith/arXiv arXiv 2023
-
[4]
Bartlett, P., Bousquet, O., & Mendelson, S. (2005). Local rademacher complexities. Annals of Statistics , 33(4), 1497--1537
2005
-
[5]
& Kohler, M
Bauer, B. & Kohler, M. (2019). On deep learning as a remedy for the curse of dimensionality in nonparametric regression. The Annals of Statistics , 47(4), 2261--2285
2019
-
[6]
J., Ritov, Y., & Tsybakov, A
Bickel, P. J., Ritov, Y., & Tsybakov, A. B. (2009). Simultaneous analysis of Lasso and Dantzig selector . The Annals of Statistics , 37(4), 1705 -- 1732
2009
-
[7]
H., Dukes, O., & Balakrishnan, S
Bonvini, M., Kennedy, E. H., Dukes, O., & Balakrishnan, S. (2024). Doubly-robust inference and optimality in structure-agnostic models with smoothness. arXiv preprint arXiv:2405.08525
Pith/arXiv arXiv 2024
-
[8]
Bradic, J., Chernozhukov, V., Newey, W. K., & Zhu, Y. (2019). Minimax semiparametric learning with approximate sparsity. arXiv preprint arXiv:1912.12213
Pith/arXiv arXiv 2019
-
[9]
Bradic, J., Fan, J., & Zhu, Y. (2022). Testability of high-dimensional linear models with nonsparse structures. Annals of statistics , 50(2), 615
2022
-
[10]
Bruns-Smith, D., Dukes, O., Feller, A., & Ogburn, E. L. (2025). Augmented balancing weights as linear regression. Journal of the Royal Statistical Society Series B: Statistical Methodology , (pp.\ qkaf019)
2025
-
[11]
Cai, T. T. & Guo, Z. (2017). Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity . The Annals of Statistics , 45(2), 615 -- 646
2017
-
[12]
& Tao, T
Candes, E. & Tao, T. (2007). The dantzig selector: Statistical estimation when p is much larger than n. The Annals of Statistics , (pp.\ 2313--2351)
2007
-
[13]
J., Fan, J., Gijbels, I., & Wand, M
Carroll, R. J., Fan, J., Gijbels, I., & Wand, M. P. (1997). Generalized partially linear single-index models. Journal of the American Statistical Association , 92(438), 477--489
1997
-
[14]
Celentano, M. & Wainwright, M. J. (2023). Challenges of the inconsistency regime: Novel debiasing methods for missing data models. arXiv preprint arXiv:2309.01362
Pith/arXiv arXiv 2023
-
[15]
Chen, X. (2007). Large sample sieve estimation of semi-nonparametric models. Handbook of econometrics , 6, 5549--5632
2007
-
[16]
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., & Newey, W. (2017). Double/debiased/neyman machine learning of treatment effects. American Economic Review , 107(5), 261--265
2017
-
[17]
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal , 21(1), C1--C68
2018
-
[18]
K., & Singh, R
Chernozhukov, V., Newey, W. K., & Singh, R. (2022a). Automatic debiased machine learning of causal and structural effects. Econometrica , 90(3), 967--1027
-
[19]
K., & Singh, R
Chernozhukov, V., Newey, W. K., & Singh, R. (2022b). Debiased machine learning of global and local parameters using regularized riesz representers. The Econometrics Journal , 25(3), 576--601
-
[20]
K., Singh, R., & Syrgkanis, V
Chernozhukov, V., Newey, W. K., Singh, R., & Syrgkanis, V. (2026). Adversarial estimation of riesz representers. Journal of the American Statistical Association , (pp.\ 1--12)
2026
-
[21]
Donald, S. G. & Newey, W. K. (1994). Series estimation of semilinear models. Journal of Multivariate Analysis , 50(1), 30--40
1994
-
[22]
F., Granger, C
Engle, R. F., Granger, C. W., Rice, J., & Weiss, A. (1986). Semiparametric estimates of the relation between weather and electricity sales. Journal of the American statistical Association , 81(394), 310--320
1986
-
[23]
Fan, J. & Gu, Y. (2024). Factor augmented sparse throughput deep relu neural networks for high dimensional regression. Journal of the American Statistical Association , 119(548), 2680--2694
2024
-
[24]
Fan, J., Gu, Y., & Zhou, W.-X. (2024). How do noise tails impact on deep ReLU networks? The Annals of Statistics , 52(4), 1845 -- 1871
2024
-
[25]
& Huang, T
Fan, J. & Huang, T. (2005). Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli , 11(6), 1031--1057
2005
-
[26]
Fan, J., Imai, K., Lee, I., Liu, H., Ning, Y., & Yang, X. (2022). Optimal covariate balancing conditions in propensity score estimation. Journal of Business & Economic Statistics , 41(1), 97--110
2022
-
[27]
Foster, D. J. & Syrgkanis, V. (2023). Orthogonal statistical learning. The Annals of Statistics , 51(3), 879--908
2023
-
[28]
Gu, Y., Fang, C., B \"u hlmann, P., & Fan, J. (2025). Causality pursuit from heterogeneous environments via neural adversarial invariance learning . The Annals of Statistics , 53(5), 2230 -- 2257
2025
-
[29]
Gu, Y., Yin, Q., Cai, T., & Fan, J. (2026). Optimally taming biases in black-box models for efficient semiparametric estimation. arXiv preprint arXiv:2606.06368
Pith/arXiv arXiv 2026
-
[30]
Hellerstein, J. K. & Imbens, G. W. (1999). Imposing moment restrictions from auxiliary data by weighting. Review of Economics and Statistics , 81(1), 1--14
1999
-
[31]
Hirshberg, D. A. & Wager, S. (2021). Augmented minimax linear estimation. The Annals of Statistics , 49(6), 3206--3227
2021
-
[32]
& Ratkovic, M
Imai, K. & Ratkovic, M. (2014). Covariate balancing propensity score. Journal of the Royal Statistical Society Series B: Statistical Methodology , 76(1), 243--263
2014
-
[33]
& Montanari, A
Javanmard, A. & Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. Journal of Machine Learning Research , 15(82), 2869--2909
2014
-
[34]
& Montanari, A
Javanmard, A. & Montanari, A. (2018). Debiasing the lasso: Optimal sample size for Gaussian designs . The Annals of Statistics , 46(6A), 2593 -- 2622
2018
-
[35]
Jin, J., Mackey, L., & Syrgkanis, V. (2025). It's hard to be normal: The impact of noise on structure-agnostic estimation. arXiv preprint arXiv:2507.02275
arXiv 2025
-
[36]
& Langer, S
Kohler, M. & Langer, S. (2021). On the rate of convergence of fully connected deep neural network regression estimates. The Annals of Statistics , 49(4), 2231--2249
2021
-
[37]
Koltchinskii, V. (2006). Local Rademacher complexities and oracle inequalities in risk minimization . The Annals of Statistics , 34(6), 2593 -- 2656
2006
-
[38]
Kong, J. (2025). On the asymptotics of the minimax linear estimator. arXiv preprint arXiv:2510.16661
arXiv 2025
-
[39]
Liu, L., Mukherjee, R., Newey, W. K., & Robins, J. M. (2017). Semiparametric efficient empirical higher order influence function estimators. arXiv preprint arXiv:1705.07577
arXiv 2017
-
[40]
& N \'e d \'e lec, \'E
Massart, P. & N \'e d \'e lec, \'E . (2006). Risk bounds for statistical learning. Ann. Statist. , 34(1), 2326--2366
2006
-
[41]
Ning, Y., Sida, P., & Imai, K. (2020). Robust estimation of causal effects via a high-dimensional covariate balancing propensity score. Biometrika , 107(3), 533--554
2020
-
[42]
Rice, J. (1986). Convergence rates for partially splined models. Statistics & probability letters , 4(4), 203--208
1986
-
[43]
Robins, J., Li, L., Tchetgen, E., van der Vaart, A., et al. (2008). Higher order influence functions and minimax estimation of nonlinear functionals. In Probability and statistics: essays in honor of David A. Freedman , volume 2 (pp.\ 335--422). Institute of Mathematical Statistics
2008
-
[44]
Robins, J., Li, L., Tchetgen, E. T., & van der Vaart, A. (2016). Technical report: Higher order influence functions and minimax estimation of nonlinear functionals. arXiv preprint arXiv:1601.05820
Pith/arXiv arXiv 2016
-
[45]
M., Rotnitzky, A., & Zhao, L
Robins, J. M., Rotnitzky, A., & Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association , 89(427), 846--866
1994
-
[46]
Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with relu activation function (with discussion). The Annals of Statistics , 48(4), 1875--1921
2020
-
[47]
Tan, Z. (2020). Model-assisted inference for treatment effects using regularized calibrated estimation with high-dimensional data. Annals of Statistics , 48(2), 811--837
2020
-
[48]
Van de Geer, S., B \"u hlmann, P., Ritov, Y., & Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. The Annals of Statistics , 42(3), 1166--1202
2014
-
[49]
van der Laan, L., Luedtke, A., & Carone, M. (2024). Doubly robust inference via calibration. arXiv preprint arXiv:2411.02771
Pith/arXiv arXiv 2024
-
[50]
Van der Laan, M. J. (2014). Targeted estimation of nuisance parameters to obtain valid statistical inference. The international journal of biostatistics , 10(1), 29--57
2014
-
[51]
& Chervonenkis, A
Vapnik, V. & Chervonenkis, A. Y. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability & Its Applications , 16(2), 264--280
1971
-
[52]
Wainwright, M. J. (2019). High-dimensional statistics: A non-asymptotic viewpoint , volume 48. Cambridge University Press
2019
-
[53]
& Shah, R
Wang, Y. & Shah, R. D. (2024). Debiased inverse propensity score weighting for estimation of average treatment effects with high-dimensional confounders . The Annals of Statistics , 52(5), 1978 -- 2003
2024
-
[54]
& Zhang, S
Zhang, C.-H. & Zhang, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 76(1), 217--242
2014
-
[55]
Zhao, Q. (2019). Covariate balancing propensity score by tailored loss functions . The Annals of Statistics , 47(2), 965 -- 993
2019
-
[56]
Zubizarreta, J. R. (2015). Stable weights that balance covariates for estimation with incomplete outcome data. Journal of the American Statistical Association , 110(511), 910--922
2015
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.