Pith. sign in

REVIEW 2 major objections 5 minor 22 references

Two outcomes that push on each other within the same measurement window usually defeat causal analysis without instruments; this paper proves that feature-dependent variation in the two structural shock variances is enough to separate the t

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-01 05:08 UTC pith:VWE64HRQ

load-bearing objection Solid population identification theory and an honest Monte Carlo, but the neural-transfer assumption (N2) is effectively the conclusion of the theorem, and the finite-sample bias for gamma1 is large; deserves review but not unqualified acceptance. the 2 major comments →

arxiv 2607.22313 v1 pith:VWE64HRQ submitted 2026-07-24 stat.ML cs.LGstat.ME

Learning Bidirectional Causal Interactions with Heteroscedastic Neural Networks

classification stat.ML cs.LGstat.ME MSC 62F1262M45
keywords bidirectional causal inferencesimultaneous equationsheteroscedasticityneural networksconditional covariance diagonalizationidentification without instrumentsquasi-likelihoodstructural interaction coefficients
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper targets a two-equation feedback system: y1 = γ1 y2 + f1(x) + ε1 and y2 = γ2 y1 + f2(x) + ε2, where the pair (γ1, γ2) records how strongly each outcome directly moves the other within the same time window. Because each outcome is endogenous to the other, ordinary regression — even with powerful neural networks — can only learn reduced-form predictive association, not these structural effects. The paper's central claim is that the two directions can be separated from observational data alone when the shock variances g10(x) and g20(x) vary non-proportionally across the features x: under conditional shock orthogonality, the true coefficient pair is the only stable one that diagonalizes the conditional covariance of the transformed residuals, so no external instruments are needed. The author proves that the profiled likelihood has positive-definite curvature at the truth, with the curvature matrix built from the variance ratio, and that a neural implementation inherits that curvature despite non-unique network weights. If correct, this gives a practical route to bidirectional causal estimation for price–sales systems, platform interactions, and coupled biological processes where instruments are unavailable but heteroscedastic variation is real.

Core claim

In the paper's own terms, the discovery is an identification result with an explicit curvature computation. For any candidate γ, the mean-profiled residuals satisfy r_γ = Γ(γ)Γ0⁻¹ ε, so a misspecified γ mixes the two structural shocks and induces conditional correlation between the residual components. The unrestricted profiled quasi-likelihood Q*(γ) is shown to expand as Q*(γ0 + h) − Q*(γ0) = −(1/2) hᵀ I*_γ h + o(‖h‖²) with I*_γ = (1/d0²)[[E{ρ0(x)⁻¹}, 1], [1, E{ρ0(x)}]], where ρ0(x) = g10(x)/g20(x) is the variance ratio. Since E{ρ}E{ρ⁻¹} > 1 exactly when ρ0 is not almost surely constant, I*_γ is positive definite, making γ0 a strict local maximizer; a companion lemma shows γ0 is the unique

What carries the argument

The carrying object is the diagonalization identity at the center of the argument: the residual covariance under a candidate interaction vector is Vγ(x) = Γ(γ)Γ0⁻¹ diag(g10(x), g20(x)) (Γ(γ)Γ0⁻¹)ᵀ, whose off-diagonal term is nonzero for any γ ≠ γ0 unless the variance ratio ρ0(x) is constant. Around this sit the components that make the argument work: the diagonal Gaussian quasi-likelihood with the simultaneous-system Jacobian log|det Γ(γ)|; the profiled curvature matrix I*_γ whose off-diagonal entries are identically 1; the Cauchy–Schwarz step E{ρ}E{ρ⁻¹} ≥ 1 with equality iff ρ0 is constant; and the quotient-slice construction that treats network weights inducing the same functions as equiva

Load-bearing premise

Everything falls apart unless the variance ratio ρ0(x) = g10(x)/g20(x) genuinely varies over the feature space and the neural networks can locally reproduce the profiled conditional mean and variance functions to second order — the load-bearing step is the neural-profile compatibility condition (Assumption 3(N2)), which the paper does not verify constructively.

What would settle it

Generate data from the paper's own design but replace g2(x) with an exact copy of g1(x), making the variance ratio constant while keeping everything else identical; the theory predicts a flat direction in the profile, so estimates of γ1 and γ2 cannot both converge. The direct check is numeric: compute the sample analogue of I*_γ and look at det(Ĥρ) = mean(ρ) mean(ρ⁻¹) − 1, which should be near zero in that design and clearly positive in designs where identification is claimed to hold.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Two-way contemporaneous effects are identifiable from conditional second moments alone; external instruments are not required when the shock variances vary non-proportionally.
  • The variance networks in SEM-DNN do identification work, not just uncertainty calibration; the fitted variance-ratio matrix tells the analyst whether identification is strong or near-empty, with determinant zero exactly when the fitted variance ratio is constant.
  • Separate flexible regressions systematically confuse structural feedback with reduced-form dependence, and the simulation evidence indicates that bias does not vanish with sample size; only a system-level likelihood with the Jacobian distinguishes the two directions.
  • The causal reading of the recovered coefficients holds exactly when the equations are interpreted as autonomous, intervention-invariant mechanisms; statistical identification alone does not supply causality.
  • Held-out diagnostics — residual-diagonalization moments, variance calibration, and gradient leverage — give concrete checks on whether the identifying assumptions are plausible for a given dataset.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the identification result is right, the same curvature formula suggests an immediate extension to feedback systems with more than two outcomes: the scalar ratio ρ0 would be replaced by the ratio matrix of shock variances, and identification would require the joint distribution of those ratios to be non-degenerate — a testable simulation design.
  • A practical consequence of the theorem is that every fitted model should report the sample analogue of I*_γ; when the empirical distribution of the fitted variance ratio is tight, the profiled criterion is nearly flat, and seed-to-seed dispersion in the interaction estimates becomes a warning sign of weak identification rather than a numerical artifact.
  • The paper's empirical application leaves the causal reading deliberately qualified; a natural follow-up is to add historical instruments or exclusion restrictions and test whether the heteroscedasticity-based estimates move, which would probe the conditional-shock-orthogonality assumption that diagnostics can only partially check.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes SEM-DNN, a heteroscedastic neural estimator for a two-equation simultaneous system y1 = γ1 y2 + f1(x) + ε1, y2 = γ2 y1 + f2(x) + ε2, where f1, f2, and the conditional variances g1, g2 are unknown nonlinear functions represented by neural networks. Identification of the interaction coefficients (γ1, γ2) is based on conditional covariance diagonalization: under zero conditional mean, conditional uncorrelatedness of the shocks given x, and a nonconstant conditional variance ratio ρ0(x) = g10(x)/g20(x), only the true γ makes the transformed residuals conditionally uncorrelated across the feature space. The paper derives a profiled population criterion Q*(γ), proves a local second-order curvature result (Proposition 1), a global diagonalization uniqueness result (Lemma 1), and a local neural identification theorem (Theorem 1) under an explicit neural-profile compatibility condition. A stabilized training objective with β-NLL weighting and bounded γ-parameterization is introduced, followed by Monte Carlo experiments and a Dominick's scanner-data application with extensive diagnostics. The paper is careful to separate causal-interpretation assumptions from statistical identification assumptions and to present the empirical illustration as a workflow demonstration rather than a definitive estimate.

Significance. If the identification result is accepted, the paper makes a useful contribution: it shows that conditional heteroscedasticity can identify two contemporaneous structural interaction coefficients without external instruments, while allowing flexible neural nuisance functions. Proposition 1 and Lemma 1 are cleanly derived: the covariance expansion is algebraically correct, and the Cauchy-Schwarz argument for positive definiteness of I*γ is sound. The paper also ships an unusually transparent empirical protocol, including sample-construction timing rules, held-out diagnostics, seedwise reporting, and detailed Monte Carlo and robustness descriptions. The main risk is that Theorem 1's transfer from the population profiled criterion to the neural-profile criterion is largely assumed rather than derived; the strength of the paper's central claim therefore rests on an assumption that is not constructively verified.

major comments (2)
  1. [Assumption 3(N2), §2.2] The central transfer from the unrestricted profiled criterion to the neural profile is assumed, not derived. Assumption 3(N2) requires, for every h near zero, a path a*(h) such that the neural mean and variance networks represent the unrestricted profiled functions m*γ,k and v*γ,k up to o(||h||^2) error, equivalently Qδ,η0(γ0+h) = Q*(γ0+h) + o(||h||^2). This is precisely the inheritance property needed for Theorem 1. No primitive conditions on network width/depth, activation class, or DGP smoothness are supplied from which such a path follows; universal approximation of a fixed function does not imply second-order approximation of a γ-indexed family of conditional expectations. If N2 fails, the implemented criterion's curvature is the Schur complement Jγγ − Jγη Jηη+ Jηγ, which need not equal I*γ, and local identification could be weak or absent. The paper states N2 as an assumption, so T
  2. [Table D.1 and Section D.1] Section D.1 states that c=0 recovers the baseline variance design. For n=10,000 and R=4, Table D.1 reports mean bias −0.478 and RMSE 0.679 for γ1. If c=0 is indeed the baseline, this is a very large bias relative to the true γ1=0.5 and appears inconsistent with the visual impression of the main Monte Carlo figures; if the D.1 protocol differs from the main design, the text should say so explicitly. More generally, at n=10,000 the estimator is far from unbiased, and without a consistency theorem the claim that SEM-DNN 'recovers structural effects more reliably as information increases' rests heavily on the n=20,000 results. The authors should reconcile this table with the main Monte Carlo presentation and either provide the n=20,000 analogue for the D.1 design or qualify the empirical claim.
minor comments (5)
  1. [General notation] Several key equations, including the definition of I*γ in §2.3, are not numbered; numbering would make the paper easier to referee and reuse.
  2. [§3.2] The β-NLL weight uses si(α1,α2) and ¯sB(α1,α2); it would help to state explicitly that the stop-gradient operator applies only to the weight, not to qi(θ), since the text is otherwise easy to misread.
  3. [§5.2] The seedwise interquartile ranges are described as not confidence intervals, which is appropriate. The representative-seed choice ('lower median rank') is explained in the appendix; a one-sentence reminder in the main text would improve readability.
  4. [Appendix D.1] The column label 'Param' in Table D.1 is cryptic; using 'Coefficient' or 'Parameter' would be clearer.
  5. [Appendix E] The sample-construction protocol is very detailed and transparent; consider adding a short statement on code/data availability for the replication pipeline, since the text repeatedly refers to 'replication output' without giving a location.

Circularity Check

1 steps flagged

The neural curvature-transfer theorem restates Assumption 3(N2): the conclusion that the neural profile inherits I*γ is exactly the compatibility assumption plus Proposition 1.

specific steps
  1. self definitional [Section 2.2, Assumption 3(N2); Section 2.3, Theorem 1; Appendix A.3]
    "Assumption 3 (N2): ... Equivalently, Qδ,η0(γ0 + h) = Q∗(γ0 + h) + o(∥h∥2), h → 0. ... Theorem 1: Qδ,η0(γ0 +h)−Qδ,η0(γ0) = − 1 2 h⊤I∗γ h+o(∥h∥2). Proof: 'By Assumption 3 (N2), the local neural profiled criterion satisfies Qδ,η0(γ0 +h) = Q∗(γ0 +h) + o(∥h∥2).'"

    Theorem 1's displayed expansion is obtained by substituting Proposition 1 into Assumption 3(N2). N2 already asserts that the neural profiled criterion equals the unrestricted profile to o(||h||^2); Proposition 1 then supplies the quadratic term -1/2 h^T I*γ h. Therefore the central claim that the implemented neural criterion inherits the population curvature is not derived from primitive network or DGP conditions—it is assumed verbatim as the compatibility condition. Appendix A.3's proof says 'By Assumption 3 (N2)...' and then combines it with Proposition 1, so the neural-curvature result is the input itself. No width/depth/activation/smoothness argument establishes the existence of a*(h).

full rationale

The population-level identification argument is self-contained: Proposition 1 and Lemma 1 derive the profiled curvature and unique diagonalization directly from Assumption 2 via a second-order expansion of the unrestricted profiled quasi-likelihood, without using fitted neural outputs as targets. The one self-citation ([16]) is an extension of the same heteroscedasticity approach and is not load-bearing. However, the neural-transfer step is circular in a narrow but important sense: Assumption 3(N2) is defined as the statement that the local neural profile matches Q* to o(||h||^2), and Theorem 1's conclusion (curvature I*γ and local identification on the quotient slice) is exactly that assumption combined with Proposition 1. The paper is transparent that Assumption 3 is a compatibility condition rather than a primitive source of identification, so the Q* identification retains independent content; but the headline claim that the implemented criterion 'inherits' the curvature is, by the paper's own equations, an assumption rather than a derivation. Finite-sample bias in the Monte Carlo baseline is a correctness risk, not additional circularity. Overall: partial circularity in the neural inheritance theorem, independent population identification.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The central identification result rests on the conditional moment assumptions (S2–S3) and on the technical neural-profile compatibility conditions (N1–N3). The causal interpretation adds Assumption 1. Tuning constants and architecture are finite-sample stabilizers and are not part of the population identification argument.

free parameters (5)
  • g_min = 0.01
    Lower bound for fitted conditional variances to stabilize quasi-likelihood; chosen by hand (Section 2.1). Ablation D.8 shows g_min=0.05 performs better, so the baseline value is not innocuous.
  • β (β-NLL stabilization) = 0.5
    Variance-weighting exponent taken from Seitzer et al. (2022); fixed before validation. Ablation D.4 shows no monotonic effect across β, so the choice is a training stabilizer rather than part of identification.
  • ¯γ (stability bound) = 0.99
    Upper bound on |γk| to keep the simultaneous system nonsingular; fixed in implementation. Ablation D.9 shows mild sensitivity and no theoretical role beyond the stable-region assumption.
  • Regularization vector φ = Selected by Sobol/validation over log10 penalties in [-8,-1]
    L2 penalties on mean and variance networks tuned per dataset. These affect finite-sample mean-variance allocation and therefore the estimates, but are not part of the population identification argument.
  • Network architecture = 2 hidden layers × 256 units, GELU, per network
    Chosen capacity for mean and variance networks. Ablation D.5 shows larger architectures improve γ1 recovery, so capacity is an implementation choice that matters in practice.
axioms (6)
  • domain assumption Assumption 2(S2): E(ε|x)=0 and Var(ε|x)=diag(g10(x),g20(x))
    Conditional shock orthogonality is the core identifying restriction; it cannot be verified from observational data and fails with unobserved common causes, selection, or misspecified covariates.
  • domain assumption Assumption 2(S3): 0<cρ≤ρ0(x)=g10/g20≤Cρ<∞ and ρ0 is not almost surely constant
    Nonproportional heteroscedasticity is the source of identification. If ρ0 is constant, the profiled curvature matrix I*γ is singular and γ is not locally identified.
  • domain assumption Assumption 1: structural equations are autonomous and invariant under the relevant interventions
    This is required for the causal interpretation of γ1 and γ2 as direct causal effects; it is conceptually distinct from statistical identification.
  • ad hoc to paper Assumption 3(N1–N3): neural-profile compatibility, local paths representing unrestricted profiled functions, and Hessian kernel inclusion ker(Jηη)⊆ker(Jγη)
    Technical conditions that allow the neural profiled criterion to inherit the population curvature despite nonunique network weights. They are strong and not constructively checkable in practice.
  • domain assumption Stable parameter region |γk|≤¯γ<1
    Needed in Lemma 1 to rule out reciprocal algebraic solutions and to keep Γ(γ) nonsingular; also imposed in the implementation via tanh reparameterization.
  • standard math Standard analytic tools: dominated convergence, Cauchy-Schwarz, Schur complement
    Used in the proofs of Proposition 1, Lemma 1, and Theorem 1; the bounded-ratio condition makes the dominating integrand integrable.

pith-pipeline@v1.3.0-alltime-deepseek · 54427 in / 11512 out tokens · 103825 ms · 2026-08-01T05:08:02.524564+00:00 · methodology

0 comments
read the original abstract

Estimating contemporaneous bidirectional interactions from observational data is difficult because each outcome is endogenous to the other, while flexible regressions may capture only reduced-form dependence. This paper proposes SEM-DNN, a heteroscedastic neural simultaneous-equation estimator that learns reciprocal structural interactions without external instruments. Identification exploits conditional covariance diagonalization: when structural shocks have zero conditional means, are conditionally uncorrelated given predetermined covariates, and exhibit nonproportional conditional variances, only the true interaction coefficients diagonalize the conditional residual covariance across the feature space. The method jointly approximates nonlinear structural mean functions and feature-dependent variances using a diagonal Gaussian quasi-likelihood that incorporates the simultaneous-system Jacobian. We establish unique identification and positive-definite local curvature of the profiled population criterion and show that, under neural-profile compatibility conditions, the implemented neural criterion inherits this curvature despite nonunique network parameterizations. The coefficients admit a causal interpretation when the structural equations represent autonomous mechanisms that remain invariant under the relevant interventions. Monte Carlo experiments with nonlinear, high-dimensional nuisance functions and non-Gaussian shocks show that SEM-DNN recovers structural effects more reliably than parametric, kernel-based, and separate-equation neural alternatives as information increases, although at greater computational cost. An application to ready-to-eat cereal scanner data illustrates how the method can study contemporaneous price-sales feedback and assess identification strength, residual diagonalization, variance calibration, and optimization sensitivity.

Figures

Figures reproduced from arXiv: 2607.22313 by Masahiro Tanaka.

Figure 1
Figure 1. Figure 1: Monte Carlo mean bias of the structural-interaction estimates across 100 replications. [PITH_FULL_IMAGE:figures/full_fig_p020_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Monte Carlo root mean squared error (RMSE) of the structural-interaction estimates [PITH_FULL_IMAGE:figures/full_fig_p021_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Monte Carlo standard deviation of the structural-interaction estimates across 100 [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Mean validation negative quasi-log-likelihood (VNQLL) across 100 Monte Carlo [PITH_FULL_IMAGE:figures/full_fig_p023_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Mean runtime in seconds across 100 Monte Carlo replications for signal-to-noise [PITH_FULL_IMAGE:figures/full_fig_p024_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Fitted variance-ratio diagnostics for the representative empirical run. Panel (a) shows [PITH_FULL_IMAGE:figures/full_fig_p026_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 1 linked inside Pith

  1. [1]

    Causal machine learning for predicting treatment outcomes , year =

    Feuerriegel, Stefan and Frauen, Dennis and Melnychuk, Valentyn and Schweisthal, Jonas and Hess, Konstantin and Curth, Alicia and Bauer, Stefan and Kilbertus, Niki and Kohane, Isaac S and van der Schaar, Mihaela , journal =. Causal machine learning for predicting treatment outcomes , year =

  2. [2]

    and Silva, Ricardo , journal =

    Kaddour, Jean and Lynch, Aengus and Liu, Qi and Kusner, Matt J. and Silva, Ricardo , journal =. Causal Machine Learning: A Survey and Open Problems , year =

  3. [3]

    Hartford, Jason and Lewis, Greg and Leyton-Brown, Kevin and Taddy, Matt , booktitle =. Deep. 2017 , pages =

  4. [4]

    Shimizu, Shohei and Hoyer, Patrik O. and Hyv. J. Mach. Learn. Res. , title =. 2006 , number =

  5. [5]

    , booktitle =

    Lacerda, Gustavo and Spirtes, Peter and Ramsey, Joseph and Hoyer, Patrik O. , booktitle =. Discovering Cyclic Causal Models by Independent Components Analysis , year =

  6. [6]

    Bongers, Stephan and Forr. Ann. Stat. , title =. 2021 , number =

  7. [7]

    Immer, Alexander and Schultheiss, Christoph and Vogt, Julia E and Sch. Proc. 40th Int. Conf. Mach. Learn. , title =. 2023 , pages =

  8. [8]

    Effective

    Immer, Alexander and Palumbo, Emanuele and Marx, Alexander and Vogt, Julia , booktitle =. Effective. 2023 , pages =

  9. [9]

    The Causal-Neural Connection: Expressiveness, Learnability, and Inference , year =

    Xia, Kevin and Lee, Kai-Zhan and Bengio, Yoshua and Bareinboim, Elias , booktitle =. The Causal-Neural Connection: Expressiveness, Learnability, and Inference , year =

  10. [10]

    Identification Through Heteroskedasticity , year =

    Rigobon, Roberto , journal =. Identification Through Heteroskedasticity , year =

  11. [11]

    Simultaneous Equation Systems with Heteroscedasticity: Identification, Estimation, and Stock Price Elasticities , year =

    Milunovich, George and Yang, Minxian , journal =. Simultaneous Equation Systems with Heteroscedasticity: Identification, Estimation, and Stock Price Elasticities , year =

  12. [12]

    Estimating Bidirectional Causal Effects with Large Scale Online Kernel Learning , year =

    Tanaka, Masahiro , booktitle =. Estimating Bidirectional Causal Effects with Large Scale Online Kernel Learning , year =

  13. [13]

    Detlefsen, Nicki S. and J. Proc. Adv. Neural Inf. Process. Syst. , title =. 2019 , pages =

  14. [14]

    On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks , year =

    Seitzer, Maximilian and Tavakoli, Arash and Antic, Dimitrije and Martius, Georg , booktitle =. On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks , year =

  15. [15]

    Faithful Heteroscedastic Regression with Neural Networks , year =

    Stirn, Andrew and Wessels, Harm and Schertzer, Megan and Pereira, Laura and Sanjana, Neville and Knowles, David , booktitle =. Faithful Heteroscedastic Regression with Neural Networks , year =

  16. [16]

    Understanding Pathologies of Deep Heteroskedastic Regression , year =

    Wong-Toi, Eliot and Boyd, Alex and Fortuin, Vincent and Mandt, Stephan , booktitle =. Understanding Pathologies of Deep Heteroskedastic Regression , year =

  17. [17]

    On Deep Learning As a Remedy for the Curse of Dimensionality in Nonparametric Regression , year =

    Bauer, Benedikt and Kohler, Michael , journal =. On Deep Learning As a Remedy for the Curse of Dimensionality in Nonparametric Regression , year =

  18. [18]

    Random Features for Large-Scale Kernel Machines , year =

    Rahimi, Ali and Recht, Benjamin , booktitle =. Random Features for Large-Scale Kernel Machines , year =

  19. [19]

    High-Performance Large-Scale Image Recognition without Normalization , year =

    Brock, Andy and De, Soham and Smith, Samuel L and Simonyan, Karen , booktitle =. High-Performance Large-Scale Image Recognition without Normalization , year =

  20. [20]

    and Filippi, Sarah , booktitle =

    Flaxman, Seth and Sejdinovic, Dino and Cunningham, John P. and Filippi, Sarah , booktitle =. Bayesian Learning of Kernel Embeddings , year =

  21. [21]

    arXiv preprint arXiv:1707.07269 , year =

    Garreau, Damien and Jitkrittum, Wittawat and Kanagawa, Motonobu , title =. arXiv preprint arXiv:1707.07269 , year =

  22. [22]

    and Ba, Jimmy Lei , booktitle =

    Kingma, Diederik P. and Ba, Jimmy Lei , booktitle =. Adam: A Method for Stochastic Optimization , year =