Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

CLEAR: Unlearning Spurious Style-Content Associations with Contrastive LEarning with Anti-contrastive Regularization

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A weakly supervised VAE can separate content from style using only content labels, via a pair-switched anti-contrastive loss that cuts style-label mutual information.

desk verdict Pair-switching is a genuinely simple idea and the OOD evaluation is useful, but the paper's central MI-minimization proof does not apply to the implemented cosine-similarity loss. read the letter →

arxiv 2507.18794 v1 pith:QJXUJHRZ submitted 2025-07-24 cs.LG

Minghui Sun , Benjamin A. Goldstein , Matthew M. Engelhard This is my paper · ORCID
classification cs.LG
keywords representationdisentanglementcontrastivelearningmutualinformationminimizationvariationalautoencoderout-of-distributiongeneralizationweaklysuperviseddomainunlearningspuriouscorrelations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CLEAR, a weakly supervised method that separates task-relevant content from task-irrelevant style in a variational autoencoder using only content-label supervision. Its anti-contrastive Pair-Switching (PS) term reverses the roles of positive and negative pairs in the supervised contrastive loss, and the authors prove that minimizing this term minimizes an upper bound on the mutual information $I(y; z^{(s)})$ between style features and content labels. In experiments on styled and colored MNIST, PACS, CelebA, and Camelyon17, CLEAR-VAE swaps and interpolates content and style between samples and improves downstream classification on content-style combinations not seen during training. If correct, a single labeling signal suffices to regularize both latent subspaces, which matters for equitable and generalizable prediction when demographic or hospital-specific style shifts at test time.

What carries the argument

The load-bearing object is the Pair-Switching (PS) loss, an anti-contrastive regularization term built from the same soft-nearest-neighbour contrast used for content. For style vectors it is $L_{\mathrm{PS-SNN}} = \frac{1}{N}\sum_i -\log \frac{\mathrm{neg}_i^{(s)}}{\mathrm{pos}_i^{(s)}+\mathrm{neg}_i^{(s)}}$, where same-label style vectors act as positives and different-label style vectors as negatives, so the objective rewards a style vector that looks unrelated to its own label. The proof that this loss upper-bounds $I(y; z^{(s)}) - \log N$ is what turns a simple pair-flip into a principled unlearning mechanism, and the same contrastive machinery simultaneously maximizes $I(y; z^{(c)})$ for the content subspace.

What would settle it

Construct a synthetic dataset in which disease status changes both the content pattern and the style distribution, so no style subspace is truly independent of the label. If CLEAR-VAE can still drive an estimate of $I(y; z^{(s)})$ to zero while preserving reconstruction, the independence assumption is unnecessary; if it cannot, the method's success is limited to separable content-style data.

Watch

Extended reading notes

Core claim

The central claim is that a pair-switched contrastive loss on the style subspace drives the style representation to carry no information about the content label. Starting from the graphical model in which $z^{(c)}$ is determined by $y$, $z^{(s)}$ is independent of $y$, and $x$ is generated from both, the authors derive an ELBO whose last term is equivalent to a supervised InfoNCE/soft-nearest-neighbour objective on content. Flipping positive and negative pairs in that objective gives $L_{\mathrm{PS-SNN}}$, and the paper proves $L_{\mathrm{PS-SNN}} > I(y; z^{(s)}) - \log N$, so minimizing the loss pushes the mutual information down. Empirically, the resulting CLEAR-VAE achieves higher group Mutual Information Gap scores and better out-of-distribution classification than VAEs without this term and than variants using other mutual-information minimizers.

Load-bearing premise

The load-bearing premise is that content and style are independent in the data-generating process, with only content determining the label; if style and content are causally entangled in real data, the ELBO decomposition and the unlearning target become ill-defined.

Editorial extensions

If this is right

  • Using only content labels, a VAE can learn content and style subspaces that support semantically correct swapping and interpolation between any pair of samples.
  • Classifiers that read only the content subspace retain accuracy on test data whose content-style combinations were never seen during training, as shown on Camelyon17 tumor detection and other datasets.
  • The pair-switching loss is non-negative and requires no auxiliary discriminator or alternating adversarial training, so it is a simpler drop-in replacement for other mutual-information minimizers.
  • Because the loss is built from the same pairs as the contrastive term, adding it to a VAE objective introduces only one extra hyperparameter, with the paper setting the two contrastive weights equal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the independence assumption holds only approximately, unlearning style could also remove predictive signal when a protected attribute genuinely causes the outcome; a fairness-oriented variant might control the amount of style information removed rather than minimize it unconditionally.
  • The bound depends on batch size through $\log N$, so the degree of unlearning should improve with larger batches or memory banks; varying batch size while holding the model fixed would test this directly.
  • The ELBO derivation is VAE-specific, but the pair-switching loss is defined on any paired latent representation, so the same idea could be grafted onto other generative or contrastive models, though the MI upper-bound proof would need to be re-established.
  • On clinical data, the learned style subspace could be checked against known demographic or hospital labels to see whether CLEAR removes demographic signal only when it is spurious and preserves it when it is clinically meaningful.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The manuscript proposes CLEAR, a weakly supervised framework for learning content and style representations in a VAE when only content labels are available. The method augments a β-VAE objective with a supervised contrastive loss on the content latent subspace and a 'pair-switched' anti-contrastive loss on the style latent subspace, claiming that the pair-switched loss minimizes the mutual information between style latents and content labels. The authors instantiate the framework as CLEAR-VAE and evaluate it on Styled/Colored-MNIST, PACS, CelebA, and Camelyon17-WILDS, reporting qualitative content-style swaps and interpolations, ablations, and improved downstream classification on out-of-distribution style-content combinations.

Significance. If the theoretical claim held, the paper would contribute a simple and practically attractive way to unlearn spurious style-content associations using only content labels. The empirical component is substantial: multiple datasets, comparisons with several MI-minimization baselines, an OOD evaluation protocol, and qualitative swap/interpolation results. However, the proof that the implemented PS loss upper-bounds I(y; z(s)) − log N contains a load-bearing gap, and the manuscript itself concedes in Sec. 5 that the SNN-to-MI connection 'has not been rigorously established.' The practical value of the method may still stand on the OOD classification evidence, but the theoretical contribution, as stated in the abstract and Sec. 3.3, is not supported by the current derivation.

major comments (4)
  1. [Appx. A.4, Eq. (A.19)] The proof that L_PS-SNN upper-bounds I(y; z(s)) − log N replaces the denominator Σ_j h(y_j, z_i^{(s)}) with (N − 1) E_y[h(y, z_i^{(s)})] and marks this expectation as equal to 1. That normalization holds only if h(y,z) is a valid density-ratio estimate h(y,z) = f(y|z)/p(y). In the implemented loss, Eq. (7) defines h through cosine similarity on normalized latents, h(z_i,z_j) = exp(sim(z_i,z_j)/τ), for which E_y[exp(cos_sim(y,z)/τ)] has no reason to equal 1. Therefore the inequality L_PS-SNN ≥ I(y; z(s)) − log N is not established for the objective actually optimized, and the theorem claimed in Sec. 3.3 does not apply to the implemented PS-SNN loss.
  2. [Appx. A.4, Eqs. (A.20)-(A.21)] Even if h were a valid density ratio, the step from the batch average (1/N) Σ_i log h(y_i, z_i^{(s)}) to the population mutual information I(y; z(s)) requires that h is exactly f(y|z)/p(y) and that the samples are drawn i.i.d. from the joint distribution; no such convergence argument is provided. The PS-SNN loss is a ratio of sums over batch negatives, and its expectation is not obviously the log-ratio whose expectation defines MI. The paper's own Sec. 5 states that the SNN/InfoNCE-to-MI connection 'has not been rigorously established,' which directly undermines the proof's final step.
  3. [Sec. 5] The manuscript acknowledges, as a limitation, that supervised contrastive losses such as SupCon and the modified SNN do not have a rigorously established connection to MI, and that 'a closed-form relationship still remains unclear.' Since the PS upper-bound proof in Appx. A.4 depends exactly on this connection, the abstract's claim that PS 'minimizes the Mutual Information between the style attributes and content labels' is stronger than what the manuscript itself supports. The claims should be revised to describe PS as an empirical/approximate regularizer, or the proof must be repaired by using a properly normalized density-ratio critic.
  4. [Appx. A.1 and Fig. 2(a)] The ELBO decomposition and the content/style objective assume that z(c) and z(s) are independent a priori and a posteriori and that style is independent of the content label y. In real medical data, demographic attributes can be causally associated with pathology, so this separation may fail and the unlearning target becomes ill-defined; optimizing the proposed objective could then remove content-relevant information. The authors should explicitly scope the method's validity to settings where the independence assumption is plausible and discuss how to detect or handle violations.
minor comments (3)
  1. [Appx. E, Eq. (E.35)] The gMIG metric measures exactly the quantities that L(c)_SNN maximizes and L(s)_PS-SNN minimizes, namely the MI between z(c) and y and between z(s) and y. High gMIG therefore partly reflects alignment with the training objective rather than an independent measure of disentanglement; it should be presented as a diagnostic, with the OOD classification results serving as the independent evidence.
  2. [Appx. A.3.2, Eqs. (A.13)-(A.14)] The derivation switches from an inner product with a class-embedding vector e_k to the sample-sample cosine similarity used in Eq. (7). This transition should be made explicit, including whether e_k is a batch-dependent average and how the final loss uses pairwise sample similarities rather than class embeddings.
  3. [Throughout] There are several presentation issues: inconsistent spacing in 'V AE', inconsistent dataset naming ('Camelyon17-WILD' vs 'Camelyon17-WILDS'), and an apparent typo in Eq. (A.19) where h(y_j, z_i) should likely be h(y_i, z_i).

Circularity Check

1 steps flagged · score 2.0 of 10

Minor metric overlap, no load-bearing circularity: gMIG scores the same content-label signal used to train CLEAR, but the central OOD generalization claim is tested on held-out style-content combinations and is not forced by construction.

  1. other [Sec. 4.1 and Appx. E (Eq. E.35) vs. Sec. 3.3 (Eq. 11) and Appx. A.4]
    "We design a metric called Group Mutual Information Gap (gMIG) introduced in Appx. E to gauge the degree of the disentanglement. ... gMIG(y) = 1/H(y) ( (1/dc) Σ_{j=1}^{dc} I(z(c)_j; y) − (1/ds) Σ_{j=1}^{ds} I(z(s)_j; y) )"

    The second term of gMIG is the average mutual information between the style partition z(s) and the content label y, which is exactly the quantity that the pair-switching loss L(s)_PS-SNN is constructed to minimize (Sec. 3.3, Eq. 11; Appx. A.4). Optimizing CLEAR therefore moves the representation along the same axis that gMIG scores, so reporting gMIG as evidence of disentanglement is partly a restatement of the training objective rather than an independent measurement. This overlap is not load-bearing for the paper's main generalization claim, which is evaluated on unseen style-content combinations in Sec. 4.4, but it does make gMIG a partially self-confirming metric.

full rationale

The central empirical claim of the paper is that CLEAR-VAE improves downstream classification when style-content combinations are unseen at training time (Sec. 4.4, Appx. D). That evidence is independent of the training signal: test styles are held out and the classifier is evaluated on frozen VAE representations, so the result is not forced by construction. The gMIG metric does overlap with the training objective, but it serves as a disentanglement diagnostic rather than the main generalization claim, so it contributes only minor circularity. The Appx. A.4 proof that L(s)_PS-SNN upper-bounds I(y; z(s)) − log(N) relies at line A.19 on the replacement E_y[h(y,z)] = 1, which holds for a normalized density-ratio h but not for the implemented cosine-similarity form exp(sim/τ); this is a genuine proof gap and a correctness risk, not a circular reduction, and the paper itself concedes in Sec. 5 that the connection between SNN-style losses and MI 'has not been rigorously established.' No load-bearing self-citation chain appears: the content/style separation is an explicit modeling assumption from Fig. 2 and Appx. A.1, and comparisons against other MI-minimization methods (CLEAR-TC, CLEAR-L1OutUB, CLEAR-CLUB-S) provide external empirical anchors. Overall, the derivation is not equivalent to its inputs; the main OOD result stands on independent held-out evidence.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a hand-specified decomposition of latent space into content and style, the assumed independence of these subspaces, and the density-ratio normalization of h. The main free parameters are contrastive weights, beta, temperature, and latent dimensionality, all tuned per dataset.

free parameters (4)
  • alpha (alpha1 = alpha2) contrastive weight = e.g., 100 for Styled-MNIST, Colored-MNIST, CelebA, and PACS in Table F.4
    Weights of L(c)_SNN and L(s)_PS-SNN in Eq. 3; tuned per dataset.
  • beta (beta-VAE KL weight) = e.g., 1/8, 1/4, 1/16 across datasets in Appendix F
    Controls the VAE KL penalty; authors show gMIG depends strongly on it.
  • tau (temperature) = e.g., 0.3 for cosine metric configurations in Table F.4
    Temperature in the contrastive cosine similarity; changed per dataset and per similarity metric.
  • latent dimension dz (content + style) = e.g., 16 (Styled-MNIST), 8 (Colored-MNIST), 32 (CelebA), 64 (PACS) in Table F.4
    Dimensionality of the latent space, a design choice affecting disentanglement.
assumptions (5)
  • domain assumption z(c) and z(s) are independent a priori and a posteriori
    Invoked at the start of Appx A.1 and in the graphical model Fig. 2(a). If false, the ELBO decomposition into separate KL terms and the disentanglement objective do not follow.
  • domain assumption Content z(c) is determined by y and style z(s) is independent of y
    Fig. 2(a); the definition of content and style and the supervised contrastive objective rely on this causal structure.
  • ad hoc to paper h(y,z) is the density ratio f(y|z)/p(y), so E_y[h(y,z)] = 1
    Used in Appx A.4 Eq. A.19-A.21 to prove L_PS-SNN upper-bounds I(y; z(s)) - log N, but the implemented loss uses unnormalized cosine similarity, so the normalization condition is not enforced.
  • standard math InfoNCE loss minimization maximizes mutual information
    Borrowed from Oord et al. [36]; the paper relies on this to justify L(c)_InfoNCE, though it later admits the modified multi-positive SNN version has no rigorous MI link.
  • standard math Variational autoencoder ELBO and reparameterization are valid
    Used throughout Sec. 3.1 and Appx A to derive Eq. 2 and the training loss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CLEAR: Unlearning Spurious Style-Content Associations with Contrastive LEarning with Anti-contrastive Regularization." pith.science (2026). https://pith.science/paper/QJXUJHRZ

@misc{pith2026250718794,
  author       = {Pith},
  title        = {Pith review of: CLEAR: Unlearning Spurious Style-Content Associations with Contrastive LEarning with Anti-contrastive Regularization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QJXUJHRZ}},
  note         = {Machine review of arXiv:2507.18794}
}
read the original abstract

Learning representations unaffected by superficial characteristics is important to ensure that shifts in these characteristics at test time do not compromise downstream prediction performance. For instance, in healthcare applications, we might like to learn features that contain information about pathology yet are unaffected by race, sex, and other sources of physiologic variability, thereby ensuring predictions are equitable and generalizable across all demographics. Here we propose Contrastive LEarning with Anti-contrastive Regularization (CLEAR), an intuitive and easy-to-implement framework that effectively separates essential (i.e., task-relevant) characteristics from superficial (i.e., task-irrelevant) characteristics during training, leading to better performance when superficial characteristics shift at test time. We begin by supposing that data representations can be semantically separated into task-relevant content features, which contain information relevant to downstream tasks, and task-irrelevant style features, which encompass superficial attributes that are irrelevant to these tasks, yet may degrade performance due to associations with content present in training data that do not generalize. We then prove that our anti-contrastive penalty, which we call Pair-Switching (PS), minimizes the Mutual Information between the style attributes and content labels. Finally, we instantiate CLEAR in the latent space of a Variational Auto-Encoder (VAE), then perform experiments to quantitatively and qualitatively evaluate the resulting CLEAR-VAE over several image datasets. Our results show that CLEAR-VAE allows us to: (a) swap and interpolate content and style between any pair of samples, and (b) improve downstream classification performance in the presence of previously unseen combinations of content and style. Our code will be made publicly available.

Figures

Figures reproduced from arXiv: 2507.18794 by the authors.

Figure 1
Figure 1. CLEAR-based disentanglement aids classification of unseen style-content combinations [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Graphical representation and schematic illustration of CLEAR-VAE [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. MI maximization and minimization on simulated data from Gaussian distributions. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Swapping and interpolation experiments. In (a), each image is generated using the [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Downstream classifications on OOD samples. CNN baselines achieve descent performance [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Association Restoration Test: Revealing Restorable Shortcuts after Unlearning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    ART shows that many shortcut-mitigation and association-unlearning methods leave label-attribute associations that can be reactivated by amplifying residual feature directions under the original classifier head.

Reference graph

Works this paper leans on

43 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    Contrastive variational autoencoder enhances salient features

    Abubakar Abid and James Zou. Contrastive variational autoencoder enhances salient features. arXiv preprint arXiv:1902.04601, 2019

    arXiv 1902

  2. [2]

    Fixing a broken elbo

    Alexander Alemi, Ben Poole, Ian Fischer, Joshua Dillon, Rif A Saurous, and Kevin Murphy. Fixing a broken elbo. In International conference on machine learning, pages 159–168. PMLR, 2018

    work page 2018

  3. [3]

    Deep variational information bottleneck

    Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410, 2016

    arXiv 2016

  4. [4]

    From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge

    Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, et al. From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE transactions on medical imaging, 38(2):550–560, 2018

    work page 2018

  5. [5]

    Multi-level variational autoencoder: Learning disentangled representations from grouped observations

    Diane Bouchacourt, Ryota Tomioka, and Sebastian Nowozin. Multi-level variational autoencoder: Learning disentangled representations from grouped observations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018

    work page 2018

  6. [6]

    Understanding disentangling in β-vae

    Christopher P Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner. Understanding disentangling in β-vae. arXiv preprint arXiv:1804.03599, 2018

    arXiv 2018

  7. [7]

    Isolating sources of disentanglement in variational autoencoders

    Ricky TQ Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disentanglement in variational autoencoders. Advances in neural information processing systems, 31, 2018

    2018

  8. [8]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020

    2020

Show all 43 references
  1. [9]

    Club: A contrastive log-ratio upper bound of mutual information

    Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. Club: A contrastive log-ratio upper bound of mutual information. In International conference on machine learning, pages 1779–1788. PMLR, 2020

    2020

  2. [10]

    Learning a similarity metric discriminatively, with application to face verification

    Sumit Chopra, Raia Hadsell, and Yann LeCun. Learning a similarity metric discriminatively, with application to face verification. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), volume 1, pages 539–546. IEEE, 2005

    2005

  3. [11]

    Elements of information theory

    Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999

    1999

  4. [12]

    From t-sne to umap with contrastive learning

    Sebastian Damrich, Jan Niklas Böhm, Fred A Hamprecht, and Dmitry Kobak. From t-sne to umap with contrastive learning. arXiv preprint arXiv:2206.01816, 2022

    2022 arXiv

  5. [13]

    Analyzing and improving representations with the soft nearest neighbor loss

    Nicholas Frosst, Nicolas Papernot, and Geoffrey Hinton. Analyzing and improving representations with the soft nearest neighbor loss. In International conference on machine learning, pages 2012–2020. PMLR, 2019

    2012

  6. [14]

    Consistency regularization for domain generalization with logit attribution matching

    Han Gao, Kaican Li, Weiyan Xie, Zhi Lin, Yongxiang Huang, Luning Wang, Caleb Cao, and Nevin Zhang. Consistency regularization for domain generalization with logit attribution matching. In Negar Kiyavash and Joris M. Mooij, editors, Proceedings of the Fortieth Conference on Unc...

    2024

  7. [15]

    Rare event detection using disentangled representation learning

    Ryuhei Hamaguchi, Ken Sakurada, and Ryosuke Nakamura. Rare event detection using disentangled representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9327–9335, 2019

    2019

  8. [16]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015

    2015

  9. [17]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738, 2020

    2020

  10. [18]

    beta-vae: Learning basic visual concepts with a constrained variational framework

    Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. ICLR (Poster), 3, 2017. 11

    2017

  11. [19]

    Group-based learning of disentangled representations with generalizability for novel contents

    Haruo Hosoya. Group-based learning of disentangled representations with generalizability for novel contents. arXiv preprint arXiv:1809.02383, 2018

    2018 arXiv

  12. [20]

    An invariant form for the prior probability in estimation problems

    Harold Jeffreys. An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, 186(1007):453–461, 1946

    1946

  13. [21]

    Supervised contrastive learning

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673, 2020

    2020

  14. [22]

    Disentangling by factorising

    Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International conference on machine learning, pages 2649–2658. PMLR, 2018

    2018

  15. [23]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  16. [24]

    Bridging the data processing inequality and function-space variational inference

    Andreas Kirsch. Bridging the data processing inequality and function-space variational inference. In ICLR Blogposts 2024, 2024. URL https://iclr-blogposts.github.io/2024/blog/dpi-fsvi/ . https://iclr-blogposts.github.io/2024/blog/dpi-fsvi/

    2024

  17. [25]

    Siamese neural networks for one-shot image recognition

    Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2, pages 1–30. Lille, 2015

    2015

  18. [26]

    Wilds: A benchmark of in-the-wild distribution shifts

    Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International conference on machine learning, p...

    2021

  19. [27]

    Gradient-based learning applied to document recognition

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998

    1998

  20. [28]

    Hospedales

    Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Deeper, broader and artier domain generalization, 2017. URL https://arxiv.org/abs/1710.03077

    2017 arXiv

  21. [29]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015

    2015

  22. [30]

    Challenging common assumptions in the unsupervised learning of disentangled representations

    Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning, pages 4114–4124....

    2019

  23. [31]

    A commentary on the unsupervised learning of disentangled representations

    Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. A commentary on the unsupervised learning of disentangled representations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1...

    2020

  24. [32]

    Sepvae: a contrastive vae to separate pathological patterns from healthy ones

    Robin Louiset, Edouard Duchesnay, Antoine Grigis, Benoit Dufumier, and Pietro Gori. Sepvae: a contrastive vae to separate pathological patterns from healthy ones. arXiv preprint arXiv:2307.06206, 2023

    2023 arXiv

  25. [33]

    On the generalized distance in statistics

    Prasanta Chandra Mahalanobis. On the generalized distance in statistics. Sankhy¯a: The Indian Journal of Statistics, Series A (2008-), 80:S1–S7, 2018

    2008

  26. [34]

    Disentangling factors of variation in deep representation using adversarial training

    Michael F Mathieu, Junbo Jake Zhao, Junbo Zhao, Aditya Ramesh, Pablo Sprechmann, and Yann LeCun. Disentangling factors of variation in deep representation using adversarial training. Advances in neural information processing systems, 29, 2016

    2016

  27. [35]

    Mnist-c: A robustness benchmark for computer vision

    Norman Mu and Justin Gilmer. Mnist-c: A robustness benchmark for computer vision. arXiv preprint arXiv:1906.02337, 2019

    1906 arXiv

  28. [36]

    Representation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  29. [37]

    On variational bounds of mutual information

    Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. In International Conference on Machine Learning, pages 5171–5180. PMLR, 2019

    2019

  30. [38]

    Mutual information between discrete and continuous data sets

    Brian C Ross. Mutual information between discrete and continuous data sets. PloS one, 9(2):e87357, 2014

    2014

  31. [39]

    Density-ratio matching under the bregman diver- gence: a unified framework of density-ratio estimation

    Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the bregman diver- gence: a unified framework of density-ratio estimation. Annals of the Institute of Statistical Mathematics, 64:1009–1044, 2012. 12

    2012

  32. [40]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008

    2008

  33. [41]

    swapping

    Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International conference on machine learning, pages 9929–9939. PMLR, 2020. 13 A ELBO A.1 ELBO Decomposition Assume z(c) and z(s) are indep...

    2020

  34. [42]

    (C × 28 × 28): (a) Conv2D: [C, 32, 3, 2, 1], [32, 64, 3, 2, 1], [64, 128, 3, 2, 1] (b) ConvTranspose2D: [128, 64, 3, 2, 1], [64, 32, 3, 2, 1, 1], [32, C, 3, 2, 1, 1]

  35. [43]

    The input dimension should match z(c)’s dimension

    (C × 64 × 64): (a) Conv2D: [C, 32, 4, 2, 1], [32, 64, 4, 2, 1], [64, 128, 4, 2, 1], [128, 256, 4, 2, 1], [256, 512, 4, 2, 1] (b) ConvTranspose2D: [512, 256, 4, 2, 1], [256, 128, 4, 2, 1], [128, 64, 4, 2, 1], [64, 32, 4, 2, 1], [32, C, 4, 2, 1] In CLEAR-L1OutUB and CLEAR-CLUB-S...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.