DISCO: Mitigating Bias in Deep Learning with Conditional Distance Correlation

Christian Wachinger; Emre Kavak; Tom Nuno Wolf

REVIEW 2 major objections 5 minor 9 references

DISCO: Mitigating Bias in Deep Learning with Conditional Distance Correlation

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read If a model's predictions are independent of bias given the label, every spurious and indirect counterfactual effect vanishes — and a new $O(n^2)$ estimator makes that condition enforceable in deep learning.

desk verdict Useful regularizer and honest empirical work, but the sDISCO exactness claim is undercut by a definitional mismatch between population dCor2 and the sample estimator. read the letter →

arxiv 2506.11653 v3 pith:BGRJJKQW submitted 2025-06-13 cs.CV cs.AIcs.LG

Emre Kavak , Tom Nuno Wolf , Christian Wachinger This is my paper · ORCID

classification cs.CVcs.AIcs.LG

keywords conditionaldistancecorrelationbiasmitigationcausalstabilityanti-causalpredictionspuriouscorrelationsindependenceshortcutlearningdeepregularization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that one simple observational condition — a model's predictions being independent of the bias attributes given the true target, $\hat{Y}\perp B\mid Y$ — is enough to make a deep learning model causally stable, in the sense that counterfactual changes to bias variables do not alter its outputs. It grounds this in the Standard Anti-Causal Model (SAM), a structural causal model for the anti-causal setting $Y\to X$, and proves that the criterion kills both mediated and spurious counterfactual effects regardless of whether the bias is a confounder, a collider, or a mediator. To make the criterion enforceable in gradient-based training, the paper introduces two differentiable estimators of conditional distance correlation; the main one, sDISCO, computes the exact sample value in $O(n^2)$ memory instead of the naive $O(n^3)$ tensor. If right, the paper gives practitioners a single regularization recipe — no per-bias-type methods, few hyperparameters — that works for classification and regression, for continuous and categorical bias attributes, and for several biases at once. Six datasets in vision and NLP support the claim that the approach matches or beats existing bias mitigation baselines.

What carries the argument

The theoretical machinery is the Standard Anti-Causal Model (SAM), a structural causal model specifying target $Y$, input $X$, backdoor variables $Z$, and mediator variables $W$, together with the path-specific counterfactual effects ctf-stable, ctf-IE, and ctf-SE that decompose the total variation of the prediction (Proposition 2.1). The observational criterion $\hat{Y}\perp B\mid Y$ then acts as the single sufficient condition for causal stability. The computational machinery is the V-statistic for conditional distance correlation: sDISCO rewrites the local double-centered inner products through the diagonal-extraction identity $\operatorname{diag}(W S W^\top) = (W\circ (W S))\mathbf{1}$, computing all local covariances at once with dense $(n\times n)$ matrix multiplications and Hadamard products, so the $O(n^3)$ tensor is never materialized.

What would settle it

Take a small batch, compute the global conditional distance correlation by the naive $O(n^3)$ double-centered expansion of the V-statistic, and compare it numerically to sDISCO's closed-form output; Proposition 3.6 predicts exact equality for every entry, so any mismatch falsifies the exactness claim — a check made pointed by the paper's own disclosure of a prior implementation error.

Watch

Extended reading notes

Core claim

Under the Standard Anti-Causal Model (SAM) — the anti-causal setting where the target $Y$ generates the input $X$, and the bias set $B = W\cup Z$ collects mediator variables $W$ together with backdoor variables $Z$ — the paper proves (Theorem 2.3) that any predictor whose output satisfies $\hat{Y}\perp B\mid Y$ is causally stable: the counterfactual indirect effect and the counterfactual spurious effect both vanish, leaving only the counterfactually stable effect along $Y\to X\to \hat{Y}$. Theorem 2.4 adds that a maximum-likelihood predictor satisfying this constraint also maximizes the stable effect. Corollary 2.5 collapses the distinction between mediated and spurious bias, so one penalty handles confounder, collider, and mediator biases uniformly. When unobserved backdoor paths exist and no proxies for the hidden confounders appear in $X$, the same criterion is the best achievable approximation of causal stability. On the computational side, sDISCO evaluates the exact sample conditional distance correlation of the classical V-statistic in $O(n^2)$ memory through an algebraic factorization, making the penalty tractable for deep learning, and the controlled counterfactual experiments on dSprites show that the resulting models are in fact insensitive to interventions on the bias variables.

Load-bearing premise

The guarantee that enforcing $\hat{Y}\perp B\mid Y$ removes all bias holds only when every spurious path between the input $X$ and the target $Y$ runs through the observed bias variables $B$; if a hidden confounder exists, and especially if its proxy appears in $X$, the criterion is only the best achievable approximation rather than a full guarantee.

Editorial extensions

If this is right

Bias type is irrelevant for mitigation: because mediated and spurious effects collapse into a single set $B=W\cup Z$, the one penalty $\hat{Y}\perp B\mid Y$ uniformly handles confounder, collider, and mediator biases.
The methods extend debiasing to regression tasks and to continuous, categorical, or mixed bias attributes, including several biases at once — settings where group-based baselines such as GDRO cannot be applied.
With unobserved confounders and no proxies in $X$, enforcing $\hat{Y}\perp B\mid Y$ is the best achievable approximation of causal stability; the dSprites counterfactual study shows residual sensitivity that matches this prediction.
Because sDISCO is a single-shot exact computation over the full batch, it scales to large batch sizes at roughly the speed of the cheapest baselines, where the naive tensorized estimator and the sampled DISCO$m$ exhaust GPU memory.
Under a controlled data-generating process, SAM enables counterfactual path analysis that separates whether a model's failure comes from bias sensitivity or from losing task-relevant signal — a diagnostic that a balanced test metric alone cannot provide.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same diagonal-extraction trick that factorizes the conditional distance correlation V-statistic plausibly applies to other kernel-based conditional dependence penalties, such as conditional HSIC or conditional MMD, giving them the same $O(n^2)$ memory profile; the paper does not claim this.
The paper's collapse of the mediated/spurious distinction suggests a testable unification: any debiasing method that enforces $\hat{Y}\perp B\mid Y$, whatever its name or mechanism, should inherit the same counterfactual guarantees under SAM, making the criterion a natural benchmark for comparing shortcut-removal methods.
The paper's own disclosure that an earlier sDISCO implementation deviated from the proven factorization (Appendix B) implies that the advertised gains depend on faithful implementation; a replication should include the numerical equality check against the naive expansion.
Because the theory requires positivity ($P(B=b\mid Y=y)>0$), datasets where any bias attribute is a deterministic function of the label mark a hard boundary: no observational debiasing, including this one, can recover the stable effect, so checking overlap is a practical precondition to test before training.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things worth knowing. First, the paper gives a clean causal packaging: under a standard anti-causal model, confounder, collider, and mediator biases all reduce to one criterion, Ŷ⊥B|Y. This is not deeply original—Makar & D'Amour and Veitch et al. have overlapping conclusions—but SAM is a useful unifying story and the path-specific analysis adds something. Second, the sDISCO estimator is a genuine computational contribution: it factorizes the conditional distance correlation V-statistic into O(n²) memory via Hadamard products and dense matrix multiplies. The algebra in Appendix G is correct. The experiments are broad—six datasets, regression and classification, seven baselines—and the results are competitive or better. The authors also disclose that the sDISCO implementation was corrected after the initial submission, and they report both old and new results, which is more transparent than most.

The main soft spot is a definitional mismatch. Definition 3.4 defines dCor²(X,Y|Z) as E[dCov²]/sqrt(E[dVar²(X)]E[dVar²(Y)]). The estimators, DISCO_m and sDISCO, instead average per-point local correlations V_XY_i / sqrt(V_XX_i V_YY_i). Those are different quantities, and Appendix F.3 Eq. 38 defines the target as the expectation of local ratios, directly contradicting Definition 3.4. So Proposition 3.6's exactness claim does not follow from the factorization in Appendix G. This is fixable—redefine dCor² or change the estimator—but as written, the central algorithmic claim is unsupported. The causal proof also drops X from the graph, which may be harmless but is not justified. The consistency proof for DISCO_m is a sketch that relies on Slutsky without treating the m-sampling dependence or ratio bias.

None of this destroys the empirical contribution; the regularizer clearly works in practice, the comparison is fair, and the limitations (observed bias, no unobserved backdoors) are stated plainly. I'd send this to peer review, but the reviewer should be asked to resolve the dCor² definition mismatch and tighten the proof before acceptance. This is a paper for people who want a practical conditional-independence regularizer with a causal justification.

Referee Report

2 major / 5 minor

Summary. The paper introduces the Standard Anti-Causal Model (SAM), a causal graphical framework for analyzing bias in anti-causal prediction tasks, and derives the conditional independence criterion Ŷ ⊥ B | Y as a sufficient condition for causal stability (Theorem 2.3). It then proposes two differentiable estimators of conditional distance correlation, DISCO_m and sDISCO, and uses them as regularization penalties to enforce the criterion. The authors report experiments on six datasets (dSprites, Blob, YaleB, FairFace, Waterbirds, MNLI) showing competitive or superior performance against seven bias-mitigation baselines, and they present counterfactual sensitivity analyses on dSprites to illustrate the causal stability of their method.

Significance. If the theoretical and algorithmic claims are correct, the paper makes a useful contribution: it unifies confounder, collider, and mediator biases under one graphical model, gives an observational criterion for counterfactual stability, and provides a memory-efficient estimator that makes conditional distance correlation regularization practical for deep learning. The empirical evaluation is broad and includes regression, multi-class classification, and NLP tasks, with comparison to a reasonable set of baselines. The paper is also transparent about a previously flawed implementation of sDISCO and reports corrected results (Appendix B). However, the central algorithmic claim—that sDISCO exactly computes the conditional distance correlation defined in the paper—is undermined by a mismatch between the definition of dCor² and the quantity sDISCO actually computes. This issue is fixable but affects the theoretical grounding of the main method.

major comments (2)

[Section 3, Definitions 3.2–3.4 and Section 3.2 'Method 2: sDISCO'] The definition of conditional distance correlation is not consistent with the estimator. Definition 3.4 sets dCor²(X,Y|Z) = dCov²(X,Y|Z) / sqrt(dVar²(X|Z) dVar²(Y|Z)), where each term is an expectation over Z (e.g., dCov²(X,Y|Z) = E_Z[dCov²(X,Y|Z=z)]). The natural sample analogue is a ratio of summed local statistics. However, the sDISCO estimator described after Eq. (18) and in Proposition 3.6 is the average of per-point local ratios, V_XY_i / sqrt(V_XX_i V_YY_i). These two quantities differ because E[A/sqrt(BC)] ≠ E[A]/sqrt(E[B]E[C]) in general. Appendix F.3, Eq. (38), explicitly adopts the per-point-average definition, contradicting Definition 3.4. The factorization in Appendix G proves only that sDISCO evaluates the local V-statistics and averages their ratios; it does not prove equality to the dCor² of Definition 3.4. Since Theorem 3.3 and Proposition 3.5 are stated for Definition 3.2/3.4 quantity, Proposition 3.6 as written does not follow. This is load-bearing: either Definition 3.4 must be changed to the expectation of local correlations, or the estimator must be changed to the ratio of summed local statistics, or Proposition 3.6 must be restated as an estimator of a different quantity. I recommend clarifying this before publication.
[Appendix F.3, Proposition F.8 (consistency of DISCO_m)] The consistency proof is only a sketch and contains a gap that interacts with the definitional issue. Step 4 invokes Slutsky's theorem / continuous mapping on each local ratio and then the WLLN on the empirical average, but it does not address the possibility of zero or near-zero denominators, nor does it state conditions under which the ratio of converging estimators converges uniformly enough to allow the expectation to pass through. More importantly, the m reference points are sampled from the same empirical distribution as the n data points, so the sampled indices are not independent of the data; the WLLN step treats them as if they were. Finally, Eq. (38) defines the target as E_Z[local correlation], which is a different estimand from Definition 3.4. Consequently, Proposition 3.5 establishes consistency for a modified target, not the dCor² defined in the main text. The proof needs to be made rigorous and aligned with the chosen definition.

minor comments (5)

[General / notation] The notation 'DISCOm' and 'DISCO_m' is used interchangeably; please standardize (e.g., DISCOₘ or DISCOm throughout).
[Appendix F.3] The condition 'nh^dZ → ∞' is missing the superscript on h; it should read n h^{d_Z} → ∞.
[Appendix B] The paper includes a 'Camera-Ready Changes' section that explicitly references a reviewer ('Reviewer Uimw') and the review process. For an archival publication, this should be removed or rewritten to describe the changes without reviewer-specific details.
[Section 3.2, Eq. (15)] The text says DISCO_m 'limits the reference points to m examples' but the mathematical definition in Eq. (15) averages over a uniformly sampled subset; please clarify how the subset is chosen and whether the same subset is used for all local terms.
[Section 4.2] The paper says 'seven representative baselines' but Table 1 lists eight baseline names (including c-MMD, CIRCE, HSCIC, etc.). Please reconcile the count.

Circularity Check

1 steps flagged · score 6.0 of 10

sDISCO's 'exact dCor2' claim holds only under an appendix redefinition of dCor2 as the estimator's own average-of-local-ratios, making Proposition 3.6 true by construction rather than by derivation from Definition 3.4.

self definitional [Section 3.2 ('Method 2: sDISCO'), Proposition 3.6; Definition 3.4; Appendix F.3, Eq. (38)]
"The global sDISCO estimator is the average of the exact local correlations. ... The theoretical global conditional distance correlation is defined as the expected value of the local correlations over the marginal distribution of Z: dCor2(X, Y|Z) = E_Z[ dCov2(X, Y|Z=z)/sqrt(dVar2(X|Z=z) dVar2(Y|Z=z)) ]."

Definition 3.4 defines dCor2(X,Y|Z) as dCov2(X,Y|Z)/sqrt(dVar2(X|Z)dVar2(Y|Z)), where dCov2 is E_Z[dCov2(X,Y|Z=z)]; this is a ratio of global expectations. The estimators DISCO_m and sDISCO instead compute (1/n) sum_i V_XY^(i)/sqrt(V_XX^(i) V_YY^(i)), an average of per-point local correlations. Proposition 3.6 asserts that sDISCO 'calculates the exact sample conditional distance correlation dCor2', but the proof in Appendix G only establishes equality to the per-point-average functional. Appendix F.3 Eq. (38) makes that functional true by definition, replacing the main-text dCor2 with E_Z[local ratio] — exactly the estimator's output. The sample analogue of Definition 3.4 would be a ratio of summed local statistics, not an average of ratios.

full rationale

The SAM-based causal theory is not circular: Theorem 2.3 is derived from explicit SCM and counterfactual assumptions, with proofs using external machinery such as twin graphs and counterfactual unnesting, and it is not justified by a self-citation chain. The empirical evaluation is self-contained against external benchmarks (Waterbirds, FairFace, MNLI, dSprites, YaleB, Blob) and is not a fitted-input-called-prediction setup: hyperparameters are selected by standard validation. However, the central algorithmic claim of exact conditional distance correlation does contain a definitional circularity. Definition 3.4 defines dCor2 as a ratio of expectations, while DISCO_m and sDISCO average local ratios, and Appendix F.3 Eq. (38) silently redefines the population dCor2 as the expected local ratio, which is precisely the estimator's functional. Consequently, Proposition 3.6's exactness is true by construction only relative to the appendix definition, not by derivation from Definition 3.4. This is a partial circularity confined to one central claim, so the score is 6 rather than 0-2 or 8-10.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or ontological entities. Its contributions are a causal graph formalism (SAM), two estimators, and a conditional independence criterion. The key free parameters are the kernel bandwidth and the regularization strength, both tuned on validation data. The load-bearing assumptions are the anti-causal graph structure and the absence of unobserved confounding or proxies.

free parameters (3)

kernel bandwidth sigma_Y = grid search over 6 values (Table 5)
Bandwidth of the RBF kernel on the conditioning variable Y in DISCO_m and sDISCO; the estimator's behavior depends strongly on it.
regularization strength lambda = grid search over 6 values (Table 5)
Trade-off between prediction loss and conditional independence penalty in Eq. 19.
DISCO_m reference subset size m = 0.2 * batch size
Number of conditioning points sampled for DISCO_m; fixed to 20 percent of batch size rather than tuned.

assumptions (5)

domain assumption The data follow the Standard Anti-Causal Model: Y causes X, with bias variables partitioned into backdoor Z and mediators W.
Invoked in Section 2.1 and Figure 2; the entire derivation of the conditional independence criterion depends on this graph.
domain assumption No unobserved backdoor paths between X and Y, and X contains no valid proxies for hidden confounders.
Stated in Section 1 and Section 2.2; Theorem 2.3's sufficiency guarantee relies on it.
domain assumption Positivity: P(B=b|Y=y) is greater than 0 for all b,y.
Assumed in Section 2.2; needed to condition on Y and to allow debiasing from observational data.
standard math Euclidean metric spaces are of strong negative type.
Used in Theorem 3.3 for the equivalence of zero dCov and conditional independence; cited to Lyons 2013 and Sejdinovic et al.
standard math Kernel regression consistency conditions, including bounded continuous kernel, h tends to 0 and n h^d tends to infinity.
Used in Appendix F.3 for the consistency proof of DISCO_m.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DISCO: Mitigating Bias in Deep Learning with Conditional Distance Correlation." pith.science (2026). https://pith.science/paper/BGRJJKQW

@misc{pith2026250611653,
  author       = {Pith},
  title        = {Pith review of: DISCO: Mitigating Bias in Deep Learning with Conditional Distance Correlation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BGRJJKQW}},
  note         = {Machine review of arXiv:2506.11653}
}

abstract

Dataset bias often leads deep learning models to exploit spurious correlations instead of task-relevant signals. We introduce the Standard Anti-Causal Model (SAM), a unifying causal framework that characterizes bias mechanisms and yields a conditional independence criterion for causal stability. Building on this theory, we propose DISCO$_m$ and sDISCO, efficient and scalable estimators of conditional distance correlation that enable independence regularization in gradient-based models. Across six diverse datasets, our methods consistently outperform or are competitive in existing observed bias mitigation approaches, while requiring fewer hyperparameters and scaling seamlessly to multi-bias scenarios. This work bridges causal theory and practical deep learning, providing both a principled foundation and effective tools for robust prediction. Source Code: https://github.com/yakamoz5/DISCO.

Figures

Figures reproduced from arXiv: 2506.11653 by the authors.

**Figure 1.** Canonical causal structures that induce dataset bias. Grey nodes indicate latent conditioning. formalize how deep learning models can isolate these ctfstable effects, we introduce the Standard Anti-Causal Model (SAM), a unifying framework that characterizes bias mechanisms in prediction tasks. Specifically, it builds on the anti-causal prediction setup (Schölkopf et al., 2012), where the target variable is assumed… view at source ↗

**Figure 2.** SAM graph. Y is the target, Z are variables on active, non-directed paths between X and Y , and W are mediator variables assumed to be unwanted shortcuts. Yˆ denotes the prediction from a fixed prediction model. See appendix Fig. D.1 for more details on permitted relationships. stable components. From this decomposition, we derive the observational conditional independence criterion required to achieve causal stabi… view at source ↗

**Figure 3.** illustrates the causal structures underlying our datasets. Across all datasets, except MNLI, we follow the standard evaluation protocol in domain generalization and bias mitigation (Sagawa et al., 2019; Arjovsky et al., 2019; Ganin et al., 2016): we train on a biased dataset, select models on an unbiased validation set, and report final results on an unseen unbiased test set. For these datasets, we additionally add… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Multi Bias Scenario dSprites. (Quinzan et al., 2022), CIRCE (Pogodin et al., 2022), and cMMD (Kaur et al., 2022; Makar & D’Amour, 2023; Veitch et al., 2021). We further adapt two models from the domain generalization field, namely Fishr (Rame et al., 2022) and IRM (Ar…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

9 extracted references · 9 canonical work pages

[1]

Modeling:We propose the Standard Anti-Causal Model (SAM) to formally characterize the data-generating process and its biasesB

work page
[2]

Criterion:From SAM, we derive a formal observational criterion,Causal Stability( ˆY⊥B|Y ), which we prove is sufficient to ensure that counterfactual indirect and spurious effects vanish, isolating the stable direct effect

work page
[3]

A key distinction of our approach is the assumption that the bias B is observed

Estimation:We develop DISCO as an efficient regularization technique to optimize deep models toward this causal criterion. A key distinction of our approach is the assumption that the bias B is observed. While some CRL methods operate under latent confounding, the assumption of known attributes is foundational to the specific subfield of bias mitigation a...

work page 2022
[4]

Generative Targets:DISCO is designed for a discriminative task

Discriminative vs. Generative Targets:DISCO is designed for a discriminative task. We apply the independence constraint to thefinal prediction ˆY , not to an intermediate representation r(x). We allow the model’s internal layers to utilize bias information if necessary, provided the final output is purged of spurious signal

work page
[5]

Information:In the example provided by Puli et al

Stability vs. Information:In the example provided by Puli et al. (2021), the "optimal" representation depends on the nuisance. Consequently, a predictor built on such a representation would also depend on the nuisance. By definition, this results in a predictor that is not causally stable against shifts in that nuisance. Our objective (maximizing the stab...

work page 2021
[6]

For a single reference point i, D(i) 1 =w (i)Sw (i)⊤

The Joint Distance Term (T1): Let S=A◦B . For a single reference point i, D(i) 1 =w (i)Sw (i)⊤. Using the identity from Eq. 47, we compute this for all npoints simultaneously: T1 = (W◦(W(A◦B)))1.(48)

work page
[7]

The grand mean for X at reference point i is exactly w(i)Aw(i)⊤

The Grand Mean Term (T2): The term D(i) 2 is the product of the weighted grand means of X and Y . The grand mean for X at reference point i is exactly w(i)Aw(i)⊤. Applying Eq. 47, the vector of all grand means for X is gX = (W◦(W A))1. Thus, the vectorized component T2 is simply the Hadamard product of the grand mean vectors: T2 =g X ◦g Y .(49) 22 DISCO: ...

work page
[8]

race” attribute, specifically the “Black

The Cross Term (T3): To compute D(i) 3 , we first factor the summation to isolate the local row means. Let M X =W Aand M Y =W Bbe the matrices of local row means, such thatM X ik = Pn l=1 WilAkl (sinceAis symmetric). We can rewriteD (i) 3 as: D(i) 3 = nX k=1 Wik nX l=1 WilAkl ! nX m=1 WimBkm ! = nX k=1 WikM X ik M Y ik.(50) Vectorizing this row-wise dot p...

work page 2017

Show all 9 references

[2011]

Original

on land or water backgrounds (Zhou et al., 2017). 25 DISCO: Mitigating Bias in Deep Learning with Conditional Distance Correlation Figure H.7.Waterbirds examples. For Waterbirds, we have again access to the data generating process. The recipe is to select the backgrounds for e...

2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Modeling:We propose the Standard Anti-Causal Model (SAM) to formally characterize the data-generating process and its biasesB

work page

[2] [2]

Criterion:From SAM, we derive a formal observational criterion,Causal Stability( ˆY⊥B|Y ), which we prove is sufficient to ensure that counterfactual indirect and spurious effects vanish, isolating the stable direct effect

work page

[3] [3]

A key distinction of our approach is the assumption that the bias B is observed

Estimation:We develop DISCO as an efficient regularization technique to optimize deep models toward this causal criterion. A key distinction of our approach is the assumption that the bias B is observed. While some CRL methods operate under latent confounding, the assumption of known attributes is foundational to the specific subfield of bias mitigation a...

work page 2022

[4] [4]

Generative Targets:DISCO is designed for a discriminative task

Discriminative vs. Generative Targets:DISCO is designed for a discriminative task. We apply the independence constraint to thefinal prediction ˆY , not to an intermediate representation r(x). We allow the model’s internal layers to utilize bias information if necessary, provided the final output is purged of spurious signal

work page

[5] [5]

Information:In the example provided by Puli et al

Stability vs. Information:In the example provided by Puli et al. (2021), the "optimal" representation depends on the nuisance. Consequently, a predictor built on such a representation would also depend on the nuisance. By definition, this results in a predictor that is not causally stable against shifts in that nuisance. Our objective (maximizing the stab...

work page 2021

[6] [6]

For a single reference point i, D(i) 1 =w (i)Sw (i)⊤

The Joint Distance Term (T1): Let S=A◦B . For a single reference point i, D(i) 1 =w (i)Sw (i)⊤. Using the identity from Eq. 47, we compute this for all npoints simultaneously: T1 = (W◦(W(A◦B)))1.(48)

work page

[7] [7]

The grand mean for X at reference point i is exactly w(i)Aw(i)⊤

The Grand Mean Term (T2): The term D(i) 2 is the product of the weighted grand means of X and Y . The grand mean for X at reference point i is exactly w(i)Aw(i)⊤. Applying Eq. 47, the vector of all grand means for X is gX = (W◦(W A))1. Thus, the vectorized component T2 is simply the Hadamard product of the grand mean vectors: T2 =g X ◦g Y .(49) 22 DISCO: ...

work page

[8] [8]

race” attribute, specifically the “Black

The Cross Term (T3): To compute D(i) 3 , we first factor the summation to isolate the local row means. Let M X =W Aand M Y =W Bbe the matrices of local row means, such thatM X ik = Pn l=1 WilAkl (sinceAis symmetric). We can rewriteD (i) 3 as: D(i) 3 = nX k=1 Wik nX l=1 WilAkl ! nX m=1 WimBkm ! = nX k=1 WikM X ik M Y ik.(50) Vectorizing this row-wise dot p...

work page 2017