Pith. sign in

REVIEW 4 major objections 7 minor 35 references

Learning Treatment Representations for Downstream Instrumental Variable Regression

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read The paper establishes that instrumental-variable regression on representations learned with the instruments in the loop identifies directions of treatment that improve the outcome.

desk verdict Real idea and a correct k=r linear result, but the identification theorems silently require dim(D_tilde)=r, and the experiments run outside that regime. read the letter →

arxiv 2506.02200 v2 pith:NTMH54GU submitted 2025-06-02 cs.LG stat.ME

classification cs.LGstat.ME
keywords instrumentalvariablestreatmentrepresentationlearningomittedvariablebiascausalinferenceautoencodersidentificationintervention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the common practice of compressing a high-dimensional treatment with PCA, autoencoders, or VAEs before running instrumental variable (IV) regression is fundamentally flawed: the compression step, blind to the instruments, can discard exactly the variation in the treatment that the instruments drive, creating omitted-variable bias that breaks the exclusion restriction. The proposed fix is to build the representation with the instruments inside the learning loop, so that the learned features equal, up to an invertible linear transformation, the instrument-driven latent component of the treatment. Once that holds, IV regression on the representation provably recovers the causal directions of the outcome, and intervening along the estimated direction improves the expected outcome for small step sizes. The significance is practical as well as theoretical: it supplies a way to run causal analyses on treatments that are as unstructured as clinical text, images, or genomic profiles while keeping the number of instruments small, and it does not require identifying the true latent factors.

What carries the argument

The load-bearing object is the matrix linking the instrument-driven latent component $D$ to the learned representation, and the argument that this matrix is square and invertible. In the linear setting the machinery is the thin SVD of the first-stage regression coefficient $C = BA$: full row rank of $A$ makes the column space of $C$ equal the column space of the treatment-loading matrix $B$, so the left singular vectors $\hat{B}$ lie in the same $r$-dimensional subspace and $P = \hat{B}^\top B$ is invertible. In the non-linear setting the machinery is a differentiation-and-completeness argument: differentiating the conditional expectation $E[\tilde{D} \mid Z = z] = \tilde{A}z$ and using $Z \perp\!\!\!\perp U \perp\!\!\!\perp V$ yields the conditional identity $E[\tilde{q}_1^{(1)}(D) \mid Z] = \tilde{A}A^+$, which bounded completeness upgrades to the pointwise identity $\tilde{q}_1(d) = \tilde{A}A^+ d$; a further exchange argument shows any optimal solution must realize this linear map exactly, with $P = \tilde{A}A^+$ invertible by a row-span lemma. The theorems turn on this invertibility: it is what lets the estimated direction $u$ in representation space be pulled back to a direction in the true latent space.

What would settle it

Simulate the linear SEM with, for example, $r = 4$ latent decisions and $k = 6$ instruments (more instruments than latent decisions, as the paper says Assumption 2.2 allows), run the LIRR algorithm, and test whether $\hat{B}^\top B$ is invertible: it is a $6 \times 4$ matrix, so $\tilde{D} = P D$ with invertible $P$ is impossible and the claimed unique IV solution $\tilde{\theta} = (P^{-1})^\top \theta$ is undefined. A complementary check is to train IRAE with a bottleneck of 10 on data with known $r = 4$ and verify whether the learned representation lies on a 4-dimensional subspace that is an invertible linear image of the true $D$; the paper reports outcome improvement in this configuration but does not verify the representation identity its theorems require.

Watch

Extended reading notes

Core claim

The paper's central claim is that fitting IV regression on instrument-guided representations identifies the directions of treatment that optimize outcome prediction, and that this holds without identifying the true latent factors. In the linear structural model $D = AZ + U$, $X = BD + B_\perp V$, $Y = \theta^\top D + \eta$, regressing $X$ on $Z$ yields the coefficient matrix $C = BA$; because $A$ has full row rank, the column space of $C$ equals the column space of $B$, so the left singular vectors $\hat{B}$ from a thin SVD of $C$ yield a representation $\tilde{D} = \hat{B}^\top X = P D$ with $P = \hat{B}^\top B$ invertible. IV regression of $Y$ on $\tilde{D}$ with instrument $Z$ then has the unique solution $\tilde{\theta} = (P^{-1})^\top \theta$, and the intervention $X + \alpha \hat{B} u$ improves the outcome by $\alpha \lVert (P^{-1})^\top \theta \rVert$ (Theorem 3.1). In the non-linear setting, under joint independence $Z \perp\!\!\!\perp U \perp\!\!\!\perp V$, differentiability of the decoding function, and bounded completeness of $D$ for $Z$, any optimal encoder-decoder pair minimizing $E[\lVert \tilde{e}_D(X) - \tilde{A}Z \rVert^2]$ with exact reconstruction and a residual independent of $Z$ satisfies $\tilde{D} = P D$ and $\tilde{U} = P U$ with $P = \tilde{A}A^+$ invertible (Theorem 4.5). The downstream IV moment condition $E[Y - \tilde{h}(\tilde{D}) \mid Z] = 0$ then forces $\tilde{h}(P \cdot) = h(\cdot)$, so the average derivative $\tilde{\theta} = (P^{-1})^\top E[\nabla_D h(D)]$ gives a small-step outcome improvement (Theorem 4.8).

Load-bearing premise

The invertibility of $P = \hat{B}^\top B$ and $P = \tilde{A}A^+$ requires the learned representation to have exactly the same dimension as the true latent vector $D$; this dimension-matching condition is never stated as an assumption, and it fails in the paper's own experiments where the autoencoder bottleneck is 10 while the true latent dimension is 4.

Editorial extensions

If this is right

  • Two-stage pipelines that compress the treatment without consulting the instruments (PCA, vanilla autoencoders, VAEs) can violate the exclusion restriction; instrument-guided representation learning removes that source of bias by construction.
  • IV regression becomes usable for treatments with far more coordinates than instruments, because the learned representation is sized by the instrument-driven latent structure rather than by the raw treatment dimension.
  • Even when the true latent decisions are not identifiable, the estimated intervention direction in representation space maps to an outcome-improving intervention in treatment space, with a formal improvement guarantee for small step sizes.
  • In the linear setting LIRR is a closed-form, provable algorithm: regress $X$ on $Z$, take the thin SVD of the coefficient matrix, run two-stage least squares on the projected representation, and intervene along the estimated direction.
  • In the non-linear setting the IRAE loss with HSIC and d-HSIC independence penalties implements the theory, and the paper's experiments show it outperforming PCA, vanilla AE, VAE, and iVAE on quadratic synthetic data and on an MNIST color-brightness task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My reading is that the identification guarantee requires the learned representation dimension to equal the true latent dimension $r$: $P$ is square only then, a premise the paper never states and that its own IRAE[2]/IRAE runs (bottleneck 10, $r = 4$) and MNIST DGP (two instruments, three color latents) violate; a natural extension is to characterize which directions remain identified in the over-
  • I infer the estimated direction is instrument-relative: because $\tilde{D}$ is a linear transform of the $Z$-driven part of the treatment, the intervention direction lives in that subspace, so weak or incomplete instruments (violating bounded completeness) would silently omit outcome-relevant directions; sensitivity to instrument strength deserves explicit analysis.
  • The theory's constraints are exact — perfect reconstruction and exact independence of residuals — while IRAE enforces them as soft penalties with hyperparameters; I would expect finite-sample violations to translate directly into bias in the improvement guarantee, and one testable consequence is that outcome improvement degrades monotonically as the HSIC penalty weights are lowered.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes instrument-guided representation learning for instrumental variable (IV) regression with high-dimensional, possibly unstructured treatments. In a linear setting, it gives the LIRR algorithm, which uses the regression of X on Z to construct a representation 𝐃̃ and then runs IV on that representation. In a nonlinear setting, it proposes the IRAE autoencoder objective that regularizes a reconstruction loss with linear prediction of the representation by Z and with independence penalties. The main theoretical claims are Theorem 3.1 (LIRR yields 𝐃̃ = P D with P invertible and guarantees outcome improvement) and Theorems 4.5 and 4.8 (under the SEM in Equation (1) and assumptions 2.1, 2.2, 4.1–4.4, any optimal encoder–decoder of the constrained objective in Equation (10) satisfies 𝐃̃ = P D with P invertible, so downstream IV on 𝐃̃ identifies outcome-improving directions). The paper reports experiments on linear DGPs, quadratic DGPs, and MNIST colorization tasks, claiming that the proposed methods outperform PCA and unsupervised autoencoders.

Significance. The problem addressed is genuine and important: applying unsupervised dimension reduction before IV regression can discard instrument-driven variation and create omitted-variable bias, and the idea of folding the instruments into the representation learner is a sensible and novel response. The theoretical development is nontrivial and, if the stated theorems held under the stated assumptions, would provide a useful framework for high-dimensional endogenous treatments. The paper also includes extensive experiments with multiple ablations and detailed appendices. However, the central identification claims depend on a dimension-matching premise that is never stated and is violated by the experiments as implemented; as written, the paper does not establish its headline claim that fitting IV on the learned representations “ensures identification”. This is fixable in principle, but it requires either restricting the theory to matched dimensions (and adjusting the algorithms and experiments accordingly) or extending the analysis to overcomplete representations, and so the current version is not yet publishable in its present form.

major comments (4)
  1. [Section 3, Theorem 3.1 and Algorithm 2] The claim that P = \widehat{B}^T B is invertible is only valid when k = r. Algorithm 2 keeps the top k singular values of C, so \widehat{B} is m × k while B is m × r, making P a k × r matrix that is not square when k > r. The proof states “\widehat{B}^T \widehat{B} = I_r”, but orthonormality gives \widehat{B}^T \widehat{B} = I_k; the equality of column spaces in Lemma B.1 implies B = \widehat{B} Q for a k × r matrix Q of full column rank, not an invertible P. The text explicitly allows “more instruments Z than latent decisions D” (k > r), so the theorem’s conclusion fails under the paper’s own assumptions unless k = r, which is never stated. This invalidates the identification of \widetilde{θ} and the improvement guarantee in the stated generality.
  2. [Appendix C, proof of Theorem 4.5] The conclusion that P = \widetilde{A} A^+ is invertible presupposes that \widetilde{A} and A have the same number of rows, i.e., \dim(\widetilde{D}) = \dim(D) = r. This is never stated as an assumption. Lemma C.1 is proved only for two r × k full-row-rank matrices with the same row count; when \widetilde{r} ≠ r the lemma does not apply. If \widetilde{r} > r, the identity \widetilde{A} = \widetilde{A} A^+ A derived in the proof forces rowspace(\widetilde{A}) ⊆ rowspace(A), which is impossible for a full-row-rank \widetilde{A} with more than r rows. If \widetilde{r} < r, P is rectangular and non-invertible, and the subsequent argument that \h(P D) = h(D) cannot identify outcome directions in the missing subspace. The nonlinear identification result and the improvement guarantee of Theorem 4.8 therefore depend on an unstated dimension-matching premise.
  3. [Section 5, Tables 2 and 3; Remark D.4] The experimental implementation does not satisfy the dimension-matching condition required by the theory. In the quadratic experiments, r = k = 4 but IRAE[2] and IRAE use a bottleneck of 10; in the MNIST Case 1 experiments, k = 2 instruments while the natural color representation has r = 3 dimensions, and IRAE[2] and IRAE use a bottleneck of 10. The paper’s own Remark D.4 concedes that the MNIST Case 1 example does not align with Equation (1), and that the mapping from Z to D is “not immediately invertible as assumed in the theory.” Consequently, the positive improvements reported for the full IRAE method are outside the scope of Theorems 4.5 and 4.8, and the abstract’s claim that fitting IV models on these representations “ensures identification” is not supported for the method as implemented.
  4. [Section 4, Equation (IRAE)] The theorems require exact conditions—\widetilde{A} full row rank, \widetilde{U} ⊥ Z, \widetilde{D} ⊥ \widetilde{V} in Lemma 4.6, and joint independence—but the IRAE loss only penalizes these properties via weighted HSIC regularizers. No argument is given that a minimizer of the penalized objective satisfies any of these conditions exactly, or that finite-sample minimization yields representations close enough to the exact constraints for the identification conclusions to apply. The text acknowledges that “we do not explicitly enforce \widetilde{A} to be full row rank,” and instead expresses an expectation that it will be satisfied; this leaves a gap between the objective actually optimized in the experiments and the premises of the identification theorems.
minor comments (7)
  1. [Section 3, text before Algorithm 2] The phrase “we have more instruments Z than latent decisions D” should be “at least as many instruments as latent decisions,” since Assumption 2.2 only requires A to have full row rank, which allows k ≥ r and not strictly k > r.
  2. [Algorithm 2, step 3] The instruction “keeping only the top k singular values” should be “top r singular values” when the latent dimension r is known; otherwise the left singular vectors corresponding to zero singular values introduce spurious directions in \widehat{B} and \widetilde{D}.
  3. [Appendix B, proof of Theorem 3.1] The identity “\widehat{B}^T \widehat{B} = I_r” should be “I_k,” and the statement that there exists an invertible change-of-basis matrix P with B = \widehat{B} P is only correct when k = r.
  4. [Theorem 4.8 and Appendix C] The symbol D is used both for the random vector and for its support (e.g., “the variable D has full support in R^r, i.e. D = R^r” in Theorem 4.8, and “∀d ∈ D” in the proof of Theorem 4.5). The support should be denoted by a separate symbol, such as 𝒟, to avoid confusion.
  5. [Section 4, Equation (IRAE)] The notation R(A, B) and R(A, B, C) is informal; the text should state explicitly that these are sample-based dependence measures and specify the kernel choices in the main text or by direct reference to the relevant equations in the appendix.
  6. [Section 5, Table 2] LIRR, a linear method, achieves substantial improvements on the quadratic DGPs; a brief comment explaining why a linear representation suffices in these particular quadratic settings would help the reader interpret the comparison.
  7. [Section 3, paragraph after Equation (7)] There is a typo: “internventional outcome” should be “interventional outcome.”

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; central claims are derived rather than assumed, with only minor non-load-bearing self-citations and a rigor gap around dimension matching.

full rationale

The paper's central identification results are derived rather than assumed. Theorem 3.1 and Theorem 4.5 start from the SEM in Eq. (5)/(1) and show, via SVD column-space arguments and a completeness-based exchange argument, that the learned representation D_tilde equals P D with P invertible. The IV moment condition then algebraically gives theta_tilde = (P^{-1})^T theta; this is a mathematical derivation, not a restatement of the objective. The representation objective (IRAE) does not use Y, so the outcome-improvement guarantee cannot be baked into the fitted loss. The only self-citations are refs [3,4], cited as optional finite-sample doubly robust estimators for nonparametric IV average derivatives; they are not used in the identification proofs or in the main experiments (which use 2SLS), so they are not load-bearing. A real rigor issue, but not circularity, is the unstated dimension-match premise: P = A_tilde A^+ is square/invertible only when dim(D_tilde) = rank(A_tilde) = r, and Lemma C.1 is stated only for two r-by-k matrices with equal row count; the paper's own experiments use bottleneck 10 with r = k = 4, and Remark D.4 concedes MNIST has k = 2 < r = 3 and is outside the theory. That gap undermines the guarantee for the implemented method, but it does not make any proved statement equivalent to its inputs.

Assumptions & free parameters 3 free parameters · 7 assumptions · 0 invented entities

The central claim rests on standard structural assumptions (invertible encoding, full-rank instruments, joint independence, completeness) plus an unstated dimension-matching premise. The practical method introduces hyperparameters (bottleneck size, regularization weights, intervention magnitude) that are chosen by hand and not covered by the theory. No new physical entities are postulated.

free parameters (3)
  • bottleneck dimension = 4 or 10 in experiments
    The dimension of the learned representation is a hyperparameter; Theorems 3.1 and 4.5 require it to equal the true latent dimension r, which is unknown and not fitted. Experiments use bottleneck = k (4) or 10 for IRAE[2] and IRAE.
  • regularization weights lambda, mu1, mu2, mu3 = 0, 1, or 10
    The penalized IRAE loss approximates the constrained objective of Theorem 4.5; the theory does not specify how large these weights must be for the guarantee to hold, and the paper uses finite values.
  • intervention magnitude alpha = 0.2, 1.0
    The improvement guarantee holds for small enough alpha; experiments pick alpha arbitrarily without verifying the small-alpha regime.
assumptions (7)
  • domain assumption Invertible encoding e(X) = f^{-1}(X) = (D, V)
    Assumption 2.1; needed for a one-to-one correspondence between X and (D,V).
  • domain assumption A full row rank and E[ZZ^T] > 0
    Assumption 2.2; ensures instruments vary all latent directions and are not collinear, but implicitly requires dimension of D to equal the number of instruments for P to be square.
  • domain assumption Joint independence Z perpendicular U perpendicular V
    Assumption 4.1; used in the proof of Theorem 4.5 to condition on Z and U.
  • domain assumption f differentiable with uniformly bounded derivatives
    Assumption 4.2; allows exchanging differentiation and expectation in the proof of Theorem 4.5.
  • domain assumption Support of Z is an open subset of R^k and E[Z] = 0
    Assumption 4.3; used to differentiate the conditional expectation identity with respect to z.
  • domain assumption Bounded completeness of D for Z
    Assumption 4.4; a strong nonparametric IV completeness condition, with sufficient conditions relegated to Lemma C.3.
  • ad hoc to paper Learned representation dimension equals true latent dimension r
    Required for P = B_hat^T B and P = A_tilde A^+ to be square and invertible; nowhere stated as an assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Treatment Representations for Downstream Instrumental Variable Regression." pith.science (2026). https://pith.science/paper/NTMH54GU

@misc{pith2026250602200,
  author       = {Pith},
  title        = {Pith review of: Learning Treatment Representations for Downstream Instrumental Variable Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NTMH54GU}},
  note         = {Machine review of arXiv:2506.02200}
}
read the original abstract

Traditional instrumental variable (IV) estimators face a fundamental constraint: they can only accommodate as many endogenous treatment variables as available instruments. This limitation becomes particularly challenging in settings where the treatment is presented in a high-dimensional and unstructured manner (e.g. descriptions of patient treatment pathways in a hospital). In such settings, researchers typically resort to applying unsupervised dimension reduction techniques to learn a low-dimensional treatment representation prior to implementing IV regression analysis. We show that such methods can suffer from substantial omitted variable bias due to implicit regularization in the representation learning step. We propose a novel approach to construct treatment representations by explicitly incorporating instrumental variables during the representation learning process. Our approach provides a framework for handling high-dimensional endogenous variables with limited instruments. We demonstrate both theoretically and empirically that fitting IV models on these instrument-informed representations ensures identification of directions that optimize outcome prediction. Our experiments show that our proposed methodology improves upon the conventional two-stage approaches that perform dimension reduction without incorporating instrument information.

Figures

Figures reproduced from arXiv: 2506.02200 by the authors.

Figure 1
Figure 1. Omitted variable bias in instrumental variable analysis with learned treatment representa [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Intervention on learned representation. 4 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparing LIRR with a PCA followed by IV approach to constructing improving interven [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (25 more)
Figure 4
Figure 4. Figure 4: Alignment of recovered latent variables with instrument, true representation [R,G,B], [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Alignment of recovered latent variables with instrument, true representation [R,G,B], [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Original gray, original color, reconstructed, treated( [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Original gray, original color, reconstructed, treated( [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Original gray, original color, reconstructed, treated( [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Distribution of Average Improvement for Linear Experiment [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: Distribution of Average Improvement for Quadratic Experiment [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 11
Figure 11. Figure 11: Original gray, original color, reconstructed, treated( [PITH_FULL_IMAGE:figures/full_fig_p030_11.png]
Figure 12
Figure 12. Figure 12: Original gray, original color, reconstructed, treated( [PITH_FULL_IMAGE:figures/full_fig_p030_12.png]
Figure 13
Figure 13. Figure 13: Original gray, original color, reconstructed, treated( [PITH_FULL_IMAGE:figures/full_fig_p030_13.png]
Figure 14
Figure 14. Figure 14: IRAE[2] on Case 1 DGP for one random seed (random seed 22), with a Conv AutoEncoder, linear HSIC as independence criterion, latent dimension 10, regularization weights λ = µ1 = µ2 = 10 and training for 50 epochs with early stopping (patience 5 epochs) warm start from …
Figure 15
Figure 15. Figure 15: IRAE[1] on Case 1 DGP for one random seed (random seed 22), with a Conv AutoEncoder, linear HSIC as independence criterion, latent dimension 2, regularization weights λ = µ1 = 10 and training for 50 epochs with early stopping (patience 5 epochs) from scratch [PITH_FU…
Figure 16
Figure 16. Figure 16: IRAE[0] on Case 1 DGP for one random seed (random seed 22), with a Conv AutoEncoder, linear HSIC as independence criterion, latent dimension 2, regularization weights λ = µ1 = 10 and training for 50 epochs with early stopping (patience 5 epochs) from scratch Our findi…
Figure 17
Figure 17. Figure 17: Experimental results for the Case 2 data generating process. Mean improvement and stan￾dard deviation of improvement is reported. reconstructed refers to the mean outcome improvement of the reconstructed image from the autoencoder with no intervention in the latents, …
Figure 18
Figure 18. Figure 18: IRAE on Case 2 DGP for one random seed (random seed 22), with a Dense AutoEncoder, linear HSIC as independence criterion, latent dimension 32, regularization weights λ = µ1 = µ2 = µ3 = 1 and training for 50 epochs with early stopping (patience 5 epochs) from scratch (…
Figure 19
Figure 19. Figure 19: IRAE on Case 2 DGP for one random seed (random seed 22), with a Dense AutoEncoder, linear HSIC as independence criterion, latent dimension 10, regularization weights λ = µ1 = µ2 = µ3 = 1 and training for 50 epochs with early stopping (patience 5 epochs) from scratch (…
Figure 20
Figure 20. Figure 20: IRAE[2] on Case 2 DGP for one random seed (random seed 22), with a Dense Au￾toEncoder, linear HSIC as independence criterion, latent dimension 32, regularization weights λ = µ1 = µ2 = 1 and training for 50 epochs with early stopping (patience 5 epochs) from scratch (n…
Figure 21
Figure 21. Figure 21: IRAE[2] on Case 2 DGP for one random seed (random seed 22), with a Dense Au￾toEncoder, linear HSIC as independence criterion, latent dimension 10, regularization weights λ = µ1 = µ2 = 1 and training for 50 epochs with early stopping (patience 5 epochs) from scratch (n…
Figure 22
Figure 22. Figure 22: IRAE[1] on Case 2 DGP for one random seed (random seed 22), with a Dense Au￾toEncoder, linear HSIC as independence criterion, latent dimension 3 = number of instruments, regularization weights λ = µ1 = 1 and µ2 = µ3 = 0 and training for 50 epochs with early stopping (…
Figure 23
Figure 23. Figure 23: IRAE[0] on Case 2 DGP for one random seed (random seed 22), with a Dense Au￾toEncoder, linear HSIC as independence criterion, latent dimension 3 = number of instruments, regularization weights λ = µ1 = 1 and µ2 = µ3 = 0 and training for 50 epochs with early stopping (…
Figure 24
Figure 24. Figure 24: Vanilla AE on Case 2 DGP for one random seed (random seed 22), with a Dense AutoEncoder, linear HSIC as independence criterion, latent dimension 3 = number of instruments, regularization weights λ = µ1 = µ2 = µ3 = 0 and training for 50 epochs with early stopping (pati…
Figure 25
Figure 25. Figure 25: Experimental results for the Case 3 data generating process. Mean improvement and standard deviation of improvement is reported. D.6 Case 4: Confounded DGP with One Outcome Relevant Dimension We examine the following confounded outcome generating process, where the in…
Figure 26
Figure 26. Figure 26: Experimental results for the Case 4 data generating process. Mean improvement and standard deviation of improvement is reported [PITH_FULL_IMAGE:figures/full_fig_p038_26.png]
Figure 27
Figure 27. Figure 27: IRAE on Case 4 DGP for one random seed, with a Dense AutoEncoder, linear HSIC as independence criterion, latent dimension 32, regularization weights λ = µ1 = µ2 = µ3 = 1 and training for 50 epochs with early stopping (patience 5 epochs) from scratch (no warm start fro…
Figure 28
Figure 28. Figure 28: IRAE on Case 4 DGP for one random seed, with a Dense AutoEncoder, linear HSIC as independence criterion, latent dimension 32, regularization weights λ = µ1 = µ2 = µ3 = 1 and training for 50 epochs with early stopping (patience 5 epochs) from scratch (no warm start fro…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 26 canonical work pages

  1. [2]

    Contrastive representations of high-dimensional, structured treatments

    Oriol Corcoll Andreu, Athanasios Vlontzos, Michael O’Riordan, and Ciaran M Gilligan- Lee. Contrastive representations of high-dimensional, structured treatments. arXiv preprint arXiv:2411.19245, 2024

  2. [1]

    Towards efficient representation identification in supervised learning

    Kartik Ahuja, Divyat Mahajan, Vasilis Syrgkanis, and Ioannis Mitliagkas. Towards efficient representation identification in supervised learning. In Conference on Causal Learning and Reasoning, pages 19–43. PMLR, 2022

  3. [3]

    Inference on strongly identified functionals of weakly identified func- tions

    Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, and Masatoshi Uehara. Inference on strongly identified functionals of weakly identified func- tions. arXiv preprint arXiv:2208.08291, 2022. 12

  4. [4]

    Source condition double robust inference on functionals of inverse problems

    Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, and Masatoshi Uehara. Source condition double robust inference on functionals of inverse problems. arXiv preprint arXiv:2307.13793, 2023

  5. [5]

    Semiparametric proximal causal inference

    Yifan Cui, Hongming Pu, Xu Shi, Wang Miao, and Eric Tchetgen Tchetgen. Semiparametric proximal causal inference. Journal of the American Statistical Association, 119(546):1348– 1359, 2024

  6. [6]

    Capacity management in networks: A structural estimation approach for hospital inpatient wards

    Jing Dong, Pengyi Shi, Fanyin Zheng, and Xin Jin. Capacity management in networks: A structural estimation approach for hospital inpatient wards

  7. [7]

    Off-service placement in inpatient ward network: Resource pooling versus service slowdown

    Jing Dong, Pengyi Shi, Fanyin Zheng, and Xin Jin. Off-service placement in inpatient ward network: Resource pooling versus service slowdown. Columbia Business School Research Paper Forthcoming, 2019

  8. [8]

    Labor-llm: Language-based occupational representations with large language models

    Tianyu Du, Ayush Kanodia, Herman Brunborg, Keyon Vafa, and Susan Athey. Labor-llm: Language-based occupational representations with large language models. arXiv preprint arXiv:2406.17972, 2024

Show all 35 references
  1. [9]

    On the completeness condition in nonparametric instrumental problems

    Xavier D’Haultfoeuille. On the completeness condition in nonparametric instrumental problems. Econometric Theory, 27(3):460–471, 2011

  2. [10]

    A kernel statistical test of independence

    Arthur Gretton, Kenji Fukumizu, Choon Teo, Le Song, Bernhard Schölkopf, and Alex Smola. A kernel statistical test of independence. Advances in neural information processing systems, 20, 2007

  3. [11]

    Hidden markov nonlinear ica: Unsupervised learning from nonstationary time series

    Hermanni Hälvä and Aapo Hyvarinen. Hidden markov nonlinear ica: Unsupervised learning from nonstationary time series. In Conference on Uncertainty in Artificial Intelligence, pages 939–948. PMLR, 2020

  4. [12]

    Identifiable feature learning for spatial data with nonlinear ica

    Hermanni Hälvä, Jonathan So, Richard E Turner, and Aapo Hyvärinen. Identifiable feature learning for spatial data with nonlinear ica. InInternational Conference on Artificial Intelligence and Statistics, pages 3331–3339. PMLR, 2024

  5. [13]

    Graphite: Estimating individual effects of graph- structured treatments

    Shonosuke Harada and Hisashi Kashima. Graphite: Estimating individual effects of graph- structured treatments. In Proceedings of the 30th ACM international conference on information & knowledge management, pages 659–668, 2021

  6. [14]

    Independent component analysis: recent advances.Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371(1984):20110534, 2013

    Aapo Hyvärinen. Independent component analysis: recent advances.Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371(1984):20110534, 2013

  7. [15]

    Independent component analysis: algorithms and applications

    Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):411–430, 2000

  8. [16]

    Nonlinear independent component analysis for principled disentanglement in unsupervised deep learning

    Aapo Hyvärinen, Ilyes Khemakhem, and Hiroshi Morioka. Nonlinear independent component analysis for principled disentanglement in unsupervised deep learning. Patterns, 4(10), 2023

  9. [17]

    Identifiability of latent-variable and structural-equation models: from linear to nonlinear

    Aapo Hyvärinen, Ilyes Khemakhem, and Ricardo Monti. Identifiability of latent-variable and structural-equation models: from linear to nonlinear. Annals of the Institute of Statistical Mathematics, 76(1):1–33, 2024

  10. [18]

    Learning causal representations from general environments: Iden- tifiability and intrinsic ambiguity

    Jikai Jin and Vasilis Syrgkanis. Learning causal representations from general environments: Iden- tifiability and intrinsic ambiguity. arXiv preprint arXiv:2311.12267, (to appear at NeurIPS24), 2023

  11. [19]

    Instrumented principal component analysis.Available at SSRN 2983919, 2020

    Bryan T Kelly, Seth Pruitt, and Yinan Su. Instrumented principal component analysis.Available at SSRN 2983919, 2020

  12. [20]

    Variational au- toencoders and nonlinear ica: A unifying framework

    Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational au- toencoders and nonlinear ica: A unifying framework. In International conference on artificial intelligence and statistics, pages 2207–2217. PMLR, 2020. 13

  13. [21]

    Cost-effective incentive allocation via structured counterfactual inference

    Romain Lopez, Chenchen Li, Xiang Yan, Junwu Xiong, Michael Jordan, Yuan Qi, and Le Song. Cost-effective incentive allocation via structured counterfactual inference. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 4997–5004, 2020

  14. [22]

    Invariant causal representation learning for out-of-distribution generalization

    Chaochao Lu, Yuhuai Wu, José Miguel Hernández-Lobato, and Bernhard Schölkopf. Invariant causal representation learning for out-of-distribution generalization. InInternational Conference on Learning Representations, 2021

  15. [23]

    Causal discovery with general non-linear relationships using non-linear ica

    Ricardo Pio Monti, Kun Zhang, and Aapo Hyvärinen. Causal discovery with general non-linear relationships using non-linear ica. In Uncertainty in artificial intelligence , pages 186–195. PMLR, 2020

  16. [24]

    Kernel conditional moment test via maximum moment restriction

    Krikamol Muandet, Wittawat Jitkrittum, and Jonas Kübler. Kernel conditional moment test via maximum moment restriction. In Conference on Uncertainty in Artificial Intelligence, pages 41–50. PMLR, 2020

  17. [25]

    Semiparametric causal sufficient dimension reduction of multidimensional treatments

    Razieh Nabi, Todd McNutt, and Ilya Shpitser. Semiparametric causal sufficient dimension reduction of multidimensional treatments. In Uncertainty in Artificial Intelligence , pages 1445–1455. PMLR, 2022

  18. [26]

    Kernel-based tests for joint independence

    Niklas Pfister, Peter Bühlmann, Bernhard Schölkopf, and Jonas Peters. Kernel-based tests for joint independence. Journal of the Royal Statistical Society Series B: Statistical Methodology, 80(1):5–31, 2018

  19. [27]

    Waiting online versus in-person: An empirical study on outpatient clinic visit incompletion

    Jimmy Qin, Carri W Chan, Jing Dong, Shunichi Homma, and Siqin Ye. Waiting online versus in-person: An empirical study on outpatient clinic visit incompletion. 2023

  20. [28]

    The use and interpretation of principal component analysis in applied research

    C Radhakrishna Rao. The use and interpretation of principal component analysis in applied research. Sankhy¯a: The Indian Journal of Statistics, Series A, pages 329–358, 1964

  21. [29]

    Principal component analysis with instrumental variables as a tool for modelling composition data

    Robert Sabatier, Jean-Dominique Lebreton, and D Chessel. Principal component analysis with instrumental variables as a tool for modelling composition data. Multiway data analysis, pages 341–352, 1989

  22. [30]

    Identifying representations for intervention extrapolation

    Sorawit Saengkyongam, Elan Rosenfeld, Pradeep Ravikumar, Niklas Pfister, and Jonas Peters. Identifying representations for intervention extrapolation. arXiv preprint arXiv:2310.04295, 2023

  23. [32]

    Toward causal representation learning

    Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021

  24. [33]

    Estimating wage disparities using foundation models

    Keyon Vafa, Susan Athey, and David M Blei. Estimating wage disparities using foundation models. arXiv preprint arXiv:2409.09894, 2024

  25. [34]

    Counterfactual and synthetic control method: Causal inference with instrumented principal component analysis

    Cong Wang. Counterfactual and synthetic control method: Causal inference with instrumented principal component analysis. arXiv preprint arXiv:2408.09271, 2024

  26. [35]

    Constrained principal component analysis: A comprehensive theory

    M Hunter Y Takane. Constrained principal component analysis: A comprehensive theory. 2001. URL https://doi.org/10.1007/s002000100081. 14 A Further Related Work In this section we provide a more discussion on related work that is not covered in the main text. Identifying Repres...

  27. [36]

    All models are trained with 50 epochs after initialization with early stopping of patience 5. 31 Table 6: Summary of parameters explored in MNIST Experiment 2 Setting Category Options Description Data Generating Process DGP2 Three Instruments Autoencoder Architecture Dense Enc...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.