Pith. sign in

REVIEW 3 major objections 5 minor 46 references

The paper claims that making spectral features outcome-aware — by adding a column δr₀ to the treatment–instrument operator — turns the worst spectral misalignment case into a one-feature recovery problem.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Augmented Spectral Feature Learning biases the learned treatment–instrument features toward the outcome, recovering NPIV structural functions that spectrum-only SpecIV features miss.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection New outcome-augmented spectral loss is a good idea and the experiments are encouraging, but the main 'one feature suffices' theorem rests on a false inequality and needs repair before the central claim can be trusted. the 3 major comments →

arxiv 2512.00919 v2 pith:44Y4OTZJ submitted 2025-11-30 stat.ML cs.LG

Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression

Dimitri Meunier , Jakub Wornbard , Vladimir R. Kostic , Antoine Moulin , Alek Fr\"ohlich , Karim Lounici , Massimiliano Pontil , Arthur Gretton This is my paper
classification stat.ML cs.LG MSC 62G0562G08
keywords nonparametric instrumental variable regressionspectral feature learningoutcome-aware featuresaugmented operatorcontrastive lossspectral misalignmenttwo-stage least squaresoff-policy evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper takes on a known failure of spectral feature learning in nonparametric instrumental variable (IV) regression: features are learned from the operator linking treatments to instruments, so they can be useless when the true causal function is concentrated in the tail of that operator's spectrum. Its central move is to augment the operator with a column aligned with the outcome, defining an augmented operator Tδ and a new contrastive loss whose global minimiser is exactly the rank-d truncated SVD of Tδ. The paper proves a non-asymptotic, high-probability error bound for the resulting two-stage least squares estimator, and in the 'bad' scenario — where the structural function is aligned with a deep singular function — shows that one outcome-aware feature can recover the structural function where the old method would need many features. Empirically, a small positive augmentation strength helps even in well-aligned settings, and on a deliberately misaligned image-based benchmark the outcome-aware version matches or beats a competitive deep-feature IV baseline. A reader should care because spectral methods for IV are useful but fragile, and this is a direct fix with a proof of the mechanism.

Core claim

On its own terms, the paper's claim is that the outcome-agnostic failure of spectral IV regression is not intrinsic to spectral features, but only to the operator being decomposed. If the operator is changed from T to Tδ = [T | δr₀], then minimising the augmented loss Lδ(θ,ω) = ‖Tδ − Ψθ[Φθ* | ω]‖_HS² − ‖Tδ‖_HS² is equivalent to computing the rank-d truncated SVD of Tδ. When the structural function h₀ concentrates on a singular function deep in the spectrum of T, the paper proves that for large enough δ the top singular subspace of Tδ aligns with that signal, and the estimation error becomes approximately (1/λ_k²)(‖q₁‖/‖s₁‖) plus a representation-learning error; thus whenever the signal-to-no

What carries the argument

The load-bearing object is the augmented operator Tδ = [T | δr₀], which maps (h, a) to Th + aδr₀; adding the outcome column δr₀ bends the top singular subspaces toward components of the structural function that the original operator T ignores. The paper proves that the new contrastive loss Lδ(θ,ω) equals ‖Tδ − Ψθ[Φθ* | ω]‖_HS² − ‖Tδ‖_HS², so minimising it is exactly truncated SVD of Tδ. A singular-value gap γδ controls the perturbation between the subspaces of T and Tδ, and that gap is what makes one-feature recovery possible in the misaligned regime.

Load-bearing premise

The misaligned-regime recovery guarantee assumes the user already knows the spectral index where the signal sits and sets the augmentation strength from the true signal norm, and that the feature networks reach a small optimality gap in training; neither is guaranteed by the theory.

What would settle it

Take a finite-rank IV problem where the structural function equals one deep singular function exactly, compute the top singular subspace of Tδ with δ given by the paper's formula, and check whether the singular-value gap is positive and the top left singular vector aligns with the intended direction; then train the one-feature model and compare the L2 error to the bound. If the error does not fall as the signal-to-noise ratio grows with the singular value fixed, the mechanism fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Spectral IV estimators no longer need to learn many features when the signal sits deep in the spectrum; one outcome-aware feature can be enough, reducing feature dimension and statistical cost in misaligned regimes.
  • Because the loss identity holds for any parameterisation, existing contrastive feature-learning pipelines can be made outcome-aware by adding the regularisation term, without changing the downstream 2SLS step.
  • The augmented loss gives a principled way to measure how much of the structural function lies in the learned feature span, which can be used to compare feature sets and to guide the augmentation strength.
  • In off-policy evaluation, where the outcome changes at every iteration of the value iteration, an outcome-agnostic spectral method can lose the relevant direction; the augmented method tracks the shifting signal and remains competitive.
  • The generalisation bound is high-probability rather than in expectation, so practitioners get a finite-sample guarantee that degrades gracefully with the ill-posedness of the learned representation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The rank-one augmentation is a template beyond IV: any spectral decomposition of a conditional operator can be steered by appending a rank-one column proportional to a downstream target, provided the contrastive loss admits the same Hilbert–Schmidt identity. This suggests analogues in state-space model learning and in solving conditional moment restrictions more broadly.
  • The bad-scenario guarantee is oracle-dependent: choosing the right spectral index and augmentation strength needs the true signal norm. A data-driven rule that estimates these from the fitted operator would turn the theory into a practical procedure; the paper does not provide one.
  • The higher-rank extension with moments such as E[Y^k|Z] is only sketched, but if the noise is heteroscedastic or the conditional distribution is asymmetric, those extra moments carry information the first moment misses. A full perturbation analysis for rank-K augmentation could make the method robust beyond mean-zero confounders.
  • The consistent benefit of small positive augmentation even in well-aligned settings hints that the augmentation acts as a regulariser, biasing features toward predictive directions at negligible cost to the operator approximation; testing this interpretation across a wider range of signal-to-noise ratios would be a cheap, direct extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Augmented Spectral Feature Learning, an outcome-aware variant of SpecIV for nonparametric instrumental variable regression. The method replaces the target-agnostic spectral loss of SpecIV with a contrastive loss derived from an augmented operator T_δ = [T | δ r_0]. Proposition 1 establishes that minimizing this loss is equivalent to best rank-d approximation of T_δ. Theorem 1 gives a high-probability 2SLS excess-risk bound under sub-Gaussian assumptions, and Section 4/Appendix B.3 develop perturbation bounds intended to show that in a 'bad scenario' of spectral misalignment the method recovers the structural function with a single feature whenever the signal-to-noise ratio dominates the spectral decay. Experiments on synthetic data, dSprites benchmarks, and off-policy evaluation show gains over standard SpecIV for suitable δ.

Significance. The core idea is well motivated: making spectral feature learning outcome-aware directly targets the known failure mode of SpecIV when h_0 is misaligned with the top singular functions of T. The paper has several strengths: the data-splitting design (independent feature-learning and 2SLS datasets) avoids circularity in the statistical analysis; Proposition 1 is clean; Theorem 1 is a useful high-probability 2SLS bound; and the experimental section is substantial, including a genuinely harder dSprites task and an OPE setting. The authors are also candid about limitations, explicitly flagging the unquantified optimality gap E_d and the lack of theoretical justification for stage-2-loss model selection. However, the central 'bad scenario' guarantee — the paper's main advertised advantage over SpecIV — is not established by the current proof because of an invalid inequality in Appendix B.3. The idea may be repairable, but the theoretical claim as written is unsupported.

major comments (3)
  1. [Appendix B.3, 'Bad scenario' paragraph; Section 4, Eq. (18)] The one-feature recovery guarantee rests on an invalid inequality. From Eq. (8) with N={k}, Λ_d=λ_k, the correct gap is γ1(δ)=λ_k(√(1+δ²‖s1‖²)−1), not λ_k√(1+δ²‖s1‖²)−1 as written. With the text's formula, the claimed inequality γ1(δ)>λ_kδ‖s1‖−(1−λ_k) is equivalent, after dividing by λ_k, to √(1+x)>√x+1 for x=(δ‖s1‖)², which is false for every x>0. With the correct Eq. (8) formula, the claimed lower bound γ1>6(1−λ_k) requires conditions on λ_k that are not stated or proved. Therefore Eq. (18), the condition 7‖q1‖/‖s1‖+3E_1λ_k≤κλ_k², and the headline conclusion 'one feature suffices whenever ‖s1‖/‖q1‖≫λ_k⁻²' do not follow from the given derivation. The choice of δ (or the statement of the required spectral gap) must be corrected.
  2. [Section 4, bad-scenario paragraph] The advertised recovery guarantee is oracle-dependent in a way that is not acknowledged. The proof assumes the practitioner knows the singular index k on which h_0 concentrates and sets δ=7(1−λ_k)/(λ_k‖s1‖) using the true signal norm ‖s1‖. No data-driven procedure for identifying k is given, and Section 6.4's δ-selection rules are heuristic; Section D.8 explicitly concedes that stage-2-loss model selection has no theoretical justification. As a result, the theorem is a conditional oracle bound rather than a guarantee for the proposed algorithm. The paper should either provide a practical selection rule or clearly state that the theoretical result assumes oracle knowledge.
  3. [Section 4, Eq. (7) and Theorem 4] The end-to-end guarantee is conditional on the optimality gap E_d(θ,ω,δ)=‖T_δ^(d)−Ψ_θ[Φ_θ^*|ω]‖, which the paper explicitly leaves unquantified ('requires architecture-specific generalization bounds... an open problem'). Consequently the corollaries in the good and bad scenarios contain an additive term that could be arbitrarily large, and the results do not establish that training the neural networks with the contrastive loss attains the advertised rates. This is a known and acknowledged limitation, but it should be more prominently qualified in the abstract and conclusions, since the stated 'full generalization error bound' overstates what is proved.
minor comments (5)
  1. [Section 2, Eq. (2)] The notation 'Z⊥⊥/X' is a typo; it should express conditional independence, e.g., Z⊥⊥X with a slash through the independence symbol.
  2. [Proposition 1] The 'if and only if' statement assumes uniqueness of the best rank-d approximation of T_δ. If the singular values of T_δ have ties, the minimizer is not unique; the statement should say 'one of the minimizers' or add an explicit spectral-gap assumption for T_δ.
  3. [Eq. (18)] The displayed coefficient (5‖h0‖+λ_k²‖s1‖)/(λ_k²‖s1‖) appears to be a typo; presumably it should be 5‖h0‖/(λ_k²‖s1‖)+1. The condition '7‖q1‖/‖s1‖+3E_1λ_k≤κλ_k²' also does not match the substitution into Theorem 4's condition, even before accounting for the invalid γ1 bound.
  4. [Section 6.4 / Section D.8] The stage-2-loss model-selection strategy is used in the OPE experiments, despite the paper's own Section D.8 explaining that its theoretical justification 'remains elusive.' The main text should mark this strategy as heuristic, since a reader may otherwise take it as validated.
  5. [Section 6.3 / Section D.7] The sentence 'Since δ was automatically tuned ... and took the values of 1/10−3/10−2 for Cartpole/Mountain Car/Catch' is hard to parse; it should read something like 'took the values 1, 10⁻³, and 10⁻² for Cartpole, Mountain Car, and Catch, respectively.'

Circularity Check

0 steps flagged

No circularity: the augmented-loss/operator equivalence is an algebraic identity, the theoretical guarantees are oracle bounds not fitted predictions, and self-citations are contextual.

full rationale

Walking the derivation chain: the central equivalence (Prop. 1) is not a prediction but a proved identity—Appendix B.1 shows Lδ(θ,ω)=∥Tδ−Tθ,ω∥²_HS−∥Tδ∥²_HS, and Eckart–Young then makes 'minimizing Lδ equals best rank-d approximation of Tδ' true by construction. This is not an input being renamed as an output; the loss was deliberately constructed to match the operator, and the proof is self-contained. The final estimator is evaluated on data independent of the feature-learning sample, and δ is a user-tuned hyperparameter; the theory's δ=7(1−λ_k)/(λ_k∥s1∥) and conditions involving ∥s1∥, ∥q1∥, and k are oracle-style assumptions in the theorem statement, not fitted values later reported as predictions. Self-citations to [27] and [21] supply the SpecIV baseline loss, the good/bad-scenario vocabulary, and the optimality-gap measure; the new augmented operator and perturbation analysis are proved in the paper from Wedin, Weyl, and Eckart–Young, and the h_new benchmark is independently constructed. The paper explicitly acknowledges open items (architecture-specific bound for E_d; no theoretical backing for 2SLS-loss selection in D.8). The alleged false inequality in Eq. (18) is a proof-correctness concern, not a circularity: even if the displayed lower bound is wrong, that is an arithmetic error in the derivation, not an equivalence of the conclusion to the assumptions. Hence no circular step.

Axiom & Free-Parameter Ledger

3 free parameters · 7 axioms · 1 invented entities

The central claim rests on standard compact-operator and sub-Gaussian assumptions, plus two load-bearing premises that are only partially supported: (1) an unquantified optimization gap E_d for DNN training with the new contrastive loss, and (2) oracle knowledge of the signal singular index k and true ‖s1‖ to set δ in the bad-scenario guarantee. The method's practical benefit depends on the free hyperparameter δ and feature dimension d, both tuned by heuristics or validation.

free parameters (3)
  • δ (augmentation strength) = 0.1–5.0 synthetic; 0.1–1.0 dSprites; 10^-3–1 OPE
    Controls the weight of the outcome term in the augmented loss; is the object of the proposed δ-selection heuristics and directly determines the augmented operator Tδ.
  • d (number of spectral features) = 11 synthetic; 16/32 dSprites; 1 in bad-scenario theory
    Model capacity chosen by hand; the theory's one-feature recovery relies on d=1 with a known signal index k.
  • Network hyperparameters (architectures, learning rates, batch sizes, regularization) = Various grids in Appendix D, Tables 1–5
    Selected by validation; the paper does not report all settings for every experiment (e.g., dSprites learning rates are omitted).
axioms (7)
  • domain assumption T: L2(X) → L2(Z) is a compact operator (Assumption 1)
    Enables countable SVD; standard in spectral NPIV literature [11,27].
  • domain assumption r0 lies in the range of T (solution existence)
    Assumed at the end of Section 2; if violated the target is the minimum-norm solution, which the paper also handles.
  • domain assumption Whitened features are uniformly bounded (Assumption 2)
    Controlling feature norms is needed for matrix/vector concentration in Theorem 1.
  • domain assumption Measure of ill-posedness c > 0 (Assumption 3)
    Guarantees invertibility of the cross-covariance and a finite 2SLS bound.
  • standard math Noise U and approximation error are sub-Gaussian (Assumption 4)
    Used for the high-probability concentration lemma (Lemma 2).
  • ad hoc to paper The optimality gap E_d(θ,ω,δ) is small (unquantified in Sec. 4)
    All approximation-error bounds are stated in terms of this gap; no DNN generalization bound is provided for the contrastive loss, as the paper itself states.
  • ad hoc to paper Bad-scenario oracle: h0 is concentrated on a known singular index k and δ is set from ‖s1‖ (Eq. 18)
    The one-feature recovery guarantee is conditional on knowing which singular direction carries the signal; the paper gives no procedure to find it.
invented entities (1)
  • Augmented operator Tδ = [T | δ r0] no independent evidence
    purpose: Makes spectral feature learning outcome-aware; its top singular subspaces define the learned features.
    A mathematical construction internal to the paper, not an empirical entity; its usefulness is tested indirectly through IV regression experiments.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression." pith.science (2026). https://pith.science/paper/44Y4OTZJ

@misc{pith2026251200919,
  author       = {Pith},
  title        = {Pith review of: Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/44Y4OTZJ}},
  note         = {Machine review of arXiv:2512.00919}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We address the problem of causal effect estimation in the presence of hidden confounders using nonparametric instrumental variable (IV) regression. An established approach is to use estimators based on learned spectral features, that is, features spanning the top singular subspaces of the operator linking treatments to instruments. While powerful, such features are agnostic to the outcome variable. Consequently, the method can fail when the true causal function is poorly represented by these dominant singular functions. To mitigate, we introduce Augmented Spectral Feature Learning, a framework that makes the feature learning process outcome-aware. Our method learns features by minimizing a novel contrastive loss derived from an augmented operator that incorporates information from the outcome. By learning these task-specific features, our approach remains effective even under spectral misalignment. We provide a theoretical analysis of this framework and validate our approach on challenging benchmarks.

Figures

Figures reproduced from arXiv: 2512.00919 by Alek Fr\"ohlich, Antoine Moulin, Arthur Gretton, Dimitri Meunier, Jakub Wornbard, Karim Lounici, Massimiliano Pontil, Vladimir R. Kostic.

Figure 1
Figure 1. Figure 1: In case of severe misalignment of h0 and T (left), an ideal solution would aim to find another operator T˜ whose top singular functions v˜i capture the signal in h0 (right). learned features to retain predictive information about additional aspects of the outcome by extending the augmentation to multiple functions of Y , such as higher conditional moments E[Y k | Z]. We discuss this higher rank extension i… view at source ↗
Figure 2
Figure 2. Figure 2: Distributions of relative IV regression MSEs ( [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Distribution of ∥bhθ −h0∥ 2 on hold (left) and hnew (right) evaluated for a range of δ values, compared to those attained by DFIV and KIV [34]. Our method is evaluated on 9 independently fitted models with identical hyperparameters for each x-axis value. 16 DFIV models were fitted in both settings. Experiment results. We compare to DFIV [40], which is currently the most competitive method to bench￾mark aga… view at source ↗
Figure 4
Figure 4. Figure 4: Left: Evolution of L (d) 0 and R (d) δ for models learning hnew. Models with non-zero δ and a small L (d) 0 (close to the value attained at δ = 0) demonstrate the best results. Each bar’s mean value is noted above it. Right: Estimation of ∥Π φ (d) ⋆ hnew∥ 2 for a range of δ values. Estimating alignment with h0. By Proposition 1, our strategy targets the features φ (d) ⋆ from the SVD of Tδ as a basis in whi… view at source ↗
Figure 5
Figure 5. Figure 5: Distributions of cumulative alignment estimates and true values [PITH_FULL_IMAGE:figures/full_fig_p027_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of how the distributions of hold and hnew vary with each component of the instrument Z (scale, orientation or x position). The values of each component of Z in dSprites are quantized. The x￾axis positions in the figure correspond to those values. For each value of a component of Z, we display the distribution of the values of h0 = hold, hnew evaluated on images where the component takes that val… view at source ↗
Figure 7
Figure 7. Figure 7: Examples of sprite images on which hold (left) and hnew (right) are evaluated. D.3 Synthetic data models We train our models using the same architecture as [27], which is described in [PITH_FULL_IMAGE:figures/full_fig_p030_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Distributions of relative IV regression MSEs for the synthetic example with [PITH_FULL_IMAGE:figures/full_fig_p031_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Scatter plot of estimated policy values vs. ground truth. [PITH_FULL_IMAGE:figures/full_fig_p033_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Box plot of mean absolute error (MAE) between policy value and ground-truth. [PITH_FULL_IMAGE:figures/full_fig_p033_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Distribution of MAE across NPIV methods. [PITH_FULL_IMAGE:figures/full_fig_p034_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Depiction of the considered environments. [PITH_FULL_IMAGE:figures/full_fig_p034_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Comparison of the estimators of ∥Π φ (d) ∗ h0∥ 2 proposed in Section 6.4 and negative stage-2 errors, for a range of δ values on the dSprites benchmark with h0 = hnew. Despite the lack of theoretical backing for the latter, both methods attain their maxima near the same values of δ. 39 [PITH_FULL_IMAGE:figures/full_fig_p039_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: IV MSE for rank two perturbations with fk = E[Y k |Z], k = 1, 2, on dSprites with h0 = hnew. For example, δ = (0.2, 0.0) reduces to the rank one case. 40 [PITH_FULL_IMAGE:figures/full_fig_p040_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 1 canonical work pages

  1. [1]

    Strong converse for identification via quantum channels

    Rudolf Ahlswede and Andreas Winter. Strong converse for identification via quantum channels. InIEEE Transactions on Information Theory, volume 48-3, pages 569–579, 2002. doi: 10.1109/TIT.2002.9 98035

    work page doi:10.1109/tit.2002.9 2002

  2. [2]

    Barto, Richard S

    Andrew G. Barto, Richard S. Sutton, and Charles W. Anderson. Neuronlike adaptive elements that can solve difficult learning control problems.IEEE Transactions on Systems, Man, and Cybernetics, SMC-13(5):834–846, 1983. doi: 10.1109/TSMC.1983.6313077

    arXiv 1983

  3. [3]

    Minimax instrumental variable regression andl_2convergence guarantees without identification or closedness

    Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, and Masatoshi Ue- hara. Minimax instrumental variable regression andl_2convergence guarantees without identification or closedness. InThe Thirty Sixth Annual Conference on Learning Theory, pages 2291–2318. PMLR, 2023

    2023

  4. [4]

    Semi-nonparametric iv estimation of shape- invariant engel curves.Econometrica, 75(6):1613–1669, 2007

    Richard Blundell, Xiaohong Chen, and Dennis Kristensen. Semi-nonparametric iv estimation of shape- invariant engel curves.Econometrica, 75(6):1613–1669, 2007

    2007

  5. [5]

    Bradtke and Andrew G

    Steven J. Bradtke and Andrew G. Barto. Linear least-squares algorithms for temporal difference learning. Machine Learning, 22(1–3):33–57, 1996. ISSN 1573-0565. doi: 10.1007/bf00114723. URLhttp: //dx.doi.org/10.1007/BF00114723

    doi:10.1007/bf00114723 1996

  6. [6]

    Optimal rates for the regularized least-squares algorithm

    Andrea Caponnetto and Ernesto De Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7(3):331–368, 2007

    2007

  7. [7]

    Using geographic variation in college proximity to estimate the return to schooling, 1993

    David Card. Using geographic variation in college proximity to estimate the return to schooling, 1993

    1993

  8. [8]

    Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric iv regression.Quantitative Economics, 9(1):39–84, 2018

    Xiaohong Chen and Timothy M Christensen. Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric iv regression.Quantitative Economics, 9(1):39–84, 2018

    2018

  9. [9]

    Estimation of nonparametric conditional moment models with pos- sibly nonsmooth generalized residuals.Econometrica, 80(1):277–321, 2012

    Xiaohong Chen and Demian Pouzo. Estimation of nonparametric conditional moment models with pos- sibly nonsmooth generalized residuals.Econometrica, 80(1):277–321, 2012

    2012

  10. [10]

    On instrumental variable regression for deep offline policy evaluation.Journal of Machine Learning Research, 23(302):1–40, 2022

    Yutian Chen, Liyuan Xu, Caglar Gulcehre, Tom Le Paine, Arthur Gretton, Nando de Freitas, and Arnaud Doucet. On instrumental variable regression for deep offline policy evaluation.Journal of Machine Learning Research, 23(302):1–40, 2022. 12

    2022

  11. [11]

    Nonparametric instrumental regres- sion.Econometrica, 79(5):1541–1565, 2011

    Serge Darolles, Yanqin Fan, Jean-Pierre Florens, and Eric Renault. Nonparametric instrumental regres- sion.Econometrica, 79(5):1541–1565, 2011

    2011

  12. [12]

    Minimax estimation of conditional moment models.Advances in Neural Information Processing Systems, 33, 2020

    Nishanth Dikkala, Greg Lewis, Lester Mackey, and Vasilis Syrgkanis. Minimax estimation of conditional moment models.Advances in Neural Information Processing Systems, 33, 2020

    2020

  13. [13]

    Identification and estimation by penal- ization in nonparametric instrumental regression.Econometric Theory, 27(3):472–496, 2011

    Jean-Pierre Florens, Jan Johannes, and Sébastien Van Bellegem. Identification and estimation by penal- ization in nonparametric instrumental regression.Econometric Theory, 27(3):472–496, 2011

    2011

  14. [14]

    Lehman, Matthieu Komorowski, Matthieu Komorowski, Aldo Faisal, Leo Anthony Celi, David Sontag, and Finale Doshi-Velez

    Omer Gottesman, Fredrik Johansson, Joshua Meier, Jack Dent, Donghun Lee, Srivatsan Srinivasan, Linying Zhang, Yi Ding, David Wihl, Xuefeng Peng, Jiayu Yao, Isaac Lage, Christopher Mosch, Li wei H. Lehman, Matthieu Komorowski, Matthieu Komorowski, Aldo Faisal, Leo Anthony Celi, David Sontag, and Finale Doshi-Velez. Evaluating reinforcement learning algorit...

  15. [15]

    Nonparametric methods for inference in the presence of instrumental variables.Annals of Statistics, 33(6):2904–2929, 2005

    Peter Hall and Joel L Horowitz. Nonparametric methods for inference in the presence of instrumental variables.Annals of Statistics, 33(6):2904–2929, 2005

    2005

  16. [16]

    Deep iv: A flexible approach for counterfactual prediction

    Jason Hartford, Greg Lewis, Kevin Leyton-Brown, and Matt Taddy. Deep iv: A flexible approach for counterfactual prediction. InInternational Conference on Machine Learning, pages 1414–1423. PMLR, 2017

    2017

  17. [17]

    Primal-dual spectral representation for off-policy evaluation, 2024

    Yang Hu, Tianyi Chen, Na Li, Kai Wang, and Bo Dai. Primal-dual spectral representation for off-policy evaluation, 2024. URLhttps://arxiv.org/abs/2410.17538

    Pith/arXiv arXiv 2024

  18. [18]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis Bach and David Blei, editors,Proceedings of the 32nd Inter- national Conference on Machine Learning, volume 37 ofProceedings of Machine Learning Research, pages 448–456, Lille, France, 07–09 Jul 2015. PMLR. URLhttps:...

    2015

  19. [19]

    Offline reinforcement learning in large state spaces: Algorithms and guarantees.arXiv preprint arXiv:2510.04088, 2025

    Nan Jiang and Tengyang Xie. Offline reinforcement learning in large state spaces: Algorithms and guarantees.arXiv preprint arXiv:2510.04088, 2025

    arXiv 2025

  20. [20]

    Optimality and adaptivity of deep neural features for instrumental variable regression

    Juno Kim, Dimitri Meunier, Arthur Gretton, Taiji Suzuki, and Zhu Li. Optimality and adaptivity of deep neural features for instrumental variable regression. InInternational Conference on Learning Representations, 2025

    2025

  21. [21]

    Neuralconditionalprobabilityforuncertaintyquantification.Advances in Neural Information Processing Systems, 37, 2024

    Vladimir Kostic, Grégoire Pacreau, Giacomo Turri, Pietro Novelli, Karim Lounici, and Massimiliano Pontil. Neuralconditionalprobabilityforuncertaintyquantification.Advances in Neural Information Processing Systems, 37, 2024

    2024

  22. [22]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems, 2020

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems, 2020. URLhttps://arxiv.org/abs/2005.01643

    Pith/arXiv arXiv 2020

  23. [23]

    Adversarial generalized method of moments.arXiv preprint arXiv:1803.07164, 2018

    Greg Lewis and Vasilis Syrgkanis. Adversarial generalized method of moments.arXiv preprint arXiv:1803.07164, 2018

    Pith/arXiv arXiv 2018

  24. [24]

    Provably efficient neural estimation of structural equation models: An adversarial approach.Advances in Neural Information Processing Systems, 33, 2020

    Luofeng Liao, You-Lin Chen, Zhuoran Yang, Bo Dai, Mladen Kolar, and Zhaoran Wang. Provably efficient neural estimation of structural equation models: An adversarial approach.Advances in Neural Information Processing Systems, 33, 2020

    2020

  25. [25]

    dsprites: Disentanglement testing sprites dataset

    Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement testing sprites dataset. https://github.com/deepmind/dsprites-dataset/, 2017

    2017

  26. [26]

    Nonparametric instrumental regression via kernel methods is minimax optimal.arXiv preprint arXiv:2411.19653, 2024

    Dimitri Meunier, Zhu Li, Tim Christensen, and Arthur Gretton. Nonparametric instrumental regression via kernel methods is minimax optimal.arXiv preprint arXiv:2411.19653, 2024. 13

    Pith/arXiv arXiv 2024

  27. [27]

    Kostic, and Arthur Gretton

    Dimitri Meunier, Antoine Moulin, Jakub Wornbard, Vladimir R. Kostic, and Arthur Gretton. Demysti- fying spectral feature learning for instrumental variable regression, 2025. URLhttps://arxiv.org/ab s/2506.10899

    arXiv 2025

  28. [28]

    Spectral normalization for generative adversarial networks

    Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. InInternational Conference on Learning Representations, 2018. URLhttps://openreview.net/forum?id=B1QRgziT-

    2018

  29. [29]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement l...

    doi:10.1038/nature14236 2015

  30. [30]

    Phd thesis, Uni- versity of Cambridge, 1990

    Andrew William Moore.Efficient Memory-Based Learning for Robot Control. Phd thesis, Uni- versity of Cambridge, 1990

    1990

  31. [31]

    Instrumental variable estimation of nonparametric models

    Whitney K Newey and James L Powell. Instrumental variable estimation of nonparametric models. Econometrica, 71(5):1565–1578, 2003

    2003

  32. [32]

    Functional bilevel optimization for machine learning

    Ieva Petrulionyte, Julien Mairal, and Michael Arbel. Functional bilevel optimization for machine learning. Advances in Neural Information Processing Systems, 37, 2024

    2024

  33. [33]

    Nonparametric instrumental variable regression with observed covariates.arXiv preprint arXiv:2511.19404, 2025

    Zikai Shen, Zonghao Chen, Dimitri Meunier, Ingo Steinwart, Arthur Gretton, and Zhu Li. Nonparametric instrumental variable regression with observed covariates.arXiv preprint arXiv:2511.19404, 2025

    arXiv 2025

  34. [34]

    Kernel instrumental variable regression.Advances in Neural Information Processing Systems, 32, 2019

    Rahul Singh, Maneesh Sahani, and Arthur Gretton. Kernel instrumental variable regression.Advances in Neural Information Processing Systems, 32, 2019

    2019

  35. [35]

    Spectral representation for causal estimation with hidden confounders

    Haotian Sun, Antoine Moulin, Tongzheng Ren, Arthur Gretton, and Bo Dai. Spectral representation for causal estimation with hidden confounders. InThe 28th International Conference on Artificial Intelligence and Statistics, pages 2719–2727. PMLR, 2025

    2025

  36. [36]

    An introduction to matrix concentration inequalities.Foundations and Trends®in Machine Learning, 8(1-2):1–230, 2015

    Joel A Tropp. An introduction to matrix concentration inequalities.Foundations and Trends®in Machine Learning, 8(1-2):1–230, 2015

    2015

  37. [37]

    Self-supervised evolution operator learning for high-dimensional dynamical systems.arXiv preprint arXiv:2505.18671, 2025

    Giacomo Turri, Luigi Bonati, Kai Zhu, Massimiliano Pontil, and Pietro Novelli. Self-supervised evolution operator learning for high-dimensional dynamical systems.arXiv preprint arXiv:2505.18671, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Fast instrument learning with faster rates.Advances in Neural Information Processing Systems, 35:16596–16611, 2022

    Ziyu Wang, Yuhao Zhou, and Jun Zhu. Fast instrument learning with faster rates.Advances in Neural Information Processing Systems, 35:16596–16611, 2022

    2022

  39. [39]

    Perturbation bounds in connection with singular value decomposition.BIT Numerical Mathematics, 12(1):99–111, 1972

    Per-Åke Wedin. Perturbation bounds in connection with singular value decomposition.BIT Numerical Mathematics, 12(1):99–111, 1972

    1972

  40. [40]

    Learning deep features in instrumental variable regression

    Liyuan Xu, Yutian Chen, Siddarth Srinivasan, Nando de Freitas, Arnaud Doucet, and Arthur Gretton. Learning deep features in instrumental variable regression. InInternational Conference on Learning Representations, 2021

    2021

  41. [41]

    Deep proxy causal learning and its application to confounded bandit policy evaluation, 2024

    Liyuan Xu, Heishiro Kanagawa, and Arthur Gretton. Deep proxy causal learning and its application to confounded bandit policy evaluation, 2024. URLhttps://arxiv.org/abs/2106.03907

    Pith/arXiv arXiv 2024

  42. [42]

    measures

    Liu Ziyin, Tilman Hartwig, and Masahito Ueda. Neural networks fail to learn periodic functions and how to fix it.Advances in Neural Information Processing Systems, 33:1583–1594, 2020. 14 A Technical Tools Notation.Throughout the appendix, for a compact operatorA,σ d(A)denotes thed-th largest singular value ofAand ifAis self-adjoint,λ d(A)denotes thed-th l...

    2020

  43. [44]

    Start with an initial guess,Q0 (e.g.,Q 0 = 0)

  44. [45]

    At iterationk+ 1, we solve the NPIV problem defined byQk: •We construct the target outcomeYk using our previous estimateQk and the observed rewardr(S, A) from the data: Yk(S, A) =−1 γ (r(S, A)−Qk(S, A)) •We solve the spectral NPIV problemE[Yk |Z] = ˜TQ k+1 to find the new estimateQk+1

  45. [46]

    outcome-aware

    Repeat until convergence. This iterative process introduces a potential for dynamic spectral misalignment. The targetYk changes at every iterationk. If the spectral features required to estimateYk (the “outcome-aware” direction) are not the same as the dominant spectral features of˜T, or if this direction shifts asQk converges, an outcome-agnostic method ...

  46. [2018]

    URLhttps://arxiv.org/abs/1805.12298

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.