Pith. sign in

REVIEW 5 major objections 5 minor 30 references

Rethinking Self-Supervised Learning Within the Framework of Partial Information Decomposition

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding a progressive pseudo-label loss to any SSL baseline improves its representations by capturing the unique-information component of partial information decomposition.

desk verdict Pseudo-label supervision gives real but modest gains; the PID 'unique information' claim is asserted, not derived, and points at the wrong component. read the letter →

arxiv 2412.02121 v1 pith:UXGDYNVO submitted 2024-12-03 cs.CV

Salman Mohamadi , Gianfranco Doretto , Donald A. Adjeroh This is my paper · ORCID
classification cs.CV MSC 68T0794A1562H30
keywords self-supervisedlearningpartialinformationdecompositionjointmutualuniquepseudo-labelingprogressivesupervisionrepresentationclustering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the long-standing dispute over whether self-supervised learning should increase or decrease mutual information between augmented views is a false dichotomy. The authors recast SSL as a three-variable information system and apply partial information decomposition (PID), which splits joint mutual information into redundant, synergistic, and unique components. They claim that ordinary SSL objectives only exploit the redundant and synergistic parts, leaving the unique information of each individual view unused. To fix this, they add a progressively re-weighted pseudo-label classification loss, generated by k-means++ clustering, to any existing SSL baseline. Experimental results across four baselines and four datasets show consistent accuracy gains, which the paper takes as evidence that the unique-information component is being captured.

What carries the argument

The load-bearing identity is the Williams-Beer decomposition of joint mutual information, $I(S_1,S_2;T) = R + Sy + U(S_1) + U(S_2)$, which separates information about a target $T$ into redundant, synergistic, and unique parts. In this paper the two augmented views are the sources and the true data distribution is the target. The mechanism claimed to realize the unique components $U(S_1)$ and $U(S_2)$ is the pseudo-label cross-entropy loss $L_{PS}$ applied to each view's projector output, balanced against the original SSL loss by a progressively growing coefficient $\alpha$. The progressive re-clustering with k-means++ is what turns the pseudo-labels from a static signal into a co-evolving supervisory signal.

What would settle it

Estimate the three PID components directly from the learned representations (for example, using a deterministic redundancy measure such as the Williams-Beer I-min formulation) for a baseline and for the upgraded model. If the upgraded model's accuracy gain is not accompanied by a measurable increase in the unique information attributed to each view, the central claim is falsified. A simpler control: train the pipeline with randomly shuffled pseudo-labels; if the accuracy improvement persists, it cannot be attributed to unique-information extraction from real cluster structure.

Watch

Extended reading notes

Core claim

The paper's central claim is that the correct objective for SSL is not maximization or minimization of pairwise mutual information, but the joint utilization of all three PID components of the joint mutual information between two augmented views and the target representation. Existing SSL methods learn from two views jointly, thereby touching only synergy and redundancy; the unique information each view carries about the target is never supervised. The proposed pipeline adds a classification loss on each view's projection against pseudo-labels obtained by k-means++ on the current features, with a weight that grows during training and with pseudo-labels refreshed every 100 epochs. According to the paper, this loss lets each view be supervised individually and independently, which instantiates the missing unique component, and the progressive refinement turns sample-level supervision (local clustering) into cluster-level supervision (global clustering). The consistent improvements across SimCLR, BYOL, Whitening-MSE, and Barlow Twins on CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet are presented as empirical support for this interpretation.

Load-bearing premise

The paper assumes, without a derivation or direct estimation, that the per-view pseudo-label classification loss $L_{PS}$ equals the unique-information component $U$ of the Williams-Beer PID; if that mapping fails, the theoretical resolution of the mutual-information debate does not follow and the method remains an empirical pseudo-labeling extension.

Editorial extensions

If this is right

  • Applying the progressive pseudo-label pipeline to SimCLR, BYOL, Whitening-MSE, and Barlow Twins improves both linear and k-NN top-1 accuracy on CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet.
  • The gains are larger under k-NN evaluation than under linear evaluation, which the paper reads as evidence that the learned features are more task-relevant rather than merely better aligned with a linear head.
  • Progressive re-labeling matters: holding the pseudo-labels fixed for the entire second phase cuts the average improvement roughly in half, from 1.59% to 0.98% on CIFAR-100.
  • The number of clusters affects the gain: using the true class count works best, while overshooting the cluster count reduces the improvement more than undershooting it.
  • The total loss is argued to align with the information bottleneck principle at two levels: the SSL term balances augmentation removal against target information, and the pseudo-label term re-introduces global association.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An immediate testable extension is to compute the Williams-Beer PID quantities on the learned features of a baseline and of the upgraded model; the paper's interpretation predicts a measurable increase in unique information $U(S_1)+U(S_2)$ rather than only in redundancy or synergy.
  • If the PID mapping is the true source of the gains, then the same progressive pseudo-labeling recipe should transfer to non-visual SSL domains, such as speech or text, where views and clusters can be defined by augmentation and clustering respectively.
  • The paper's framing suggests a design principle for future SSL losses: any objective that supervises each view independently against a co-evolving target should inject unique information, which may explain why semi-supervised hybrids already showed such gains.
  • A null variant that shuffles pseudo-labels across samples would isolate whether the improvement comes from cluster structure or merely from an extra classification head; the paper does not report this control.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper argues that the debate about whether self-supervised learning (SSL) should increase or decrease mutual information between augmented views can be resolved by reformulating SSL within the framework of partial information decomposition (PID), using joint mutual information I(S1,S2;T) instead of pairwise mutual information. The authors propose a general pipeline that adds a progressively weighted pseudo-label classification loss L_PS to any existing SSL loss L_SSL, claiming that L_PS extracts the unique information component of PID while L_SSL covers redundant and synergistic components. They report improved linear and k-NN evaluation accuracy over four SSL baselines (SimCLR, BYOL, WMSE, Barlow Twins) on CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet, plus transfer-learning results.

Significance. If the core PID claim were correct, the paper would offer a principled resolution to a long-standing debate in SSL and a generic plug-in upgrade for existing methods. The experimental effort is substantial: four baselines, four datasets, linear and k-NN evaluation, transfer learning, and an ablation study. The paper also publicly builds on the Solo-Learn library, which aids reproducibility. However, the central theoretical mapping from the proposed loss to unique information is asserted rather than derived, and the specific loss appears to instantiate the opposite PID component (redundancy). The empirical gains are modest and are reported without error bars or multiple seeds. Because the theoretical contribution is load-bearing and unsupported, the paper as it stands does not establish its advertised claims.

major comments (5)
  1. [Section 3.4, Eq. (2)] The central claim that L_PS extracts the unique information component of PID is unsupported and appears inconsistent with the Williams-Beer PID definition in Eq. (1). In Eq. (2), the same pseudo-label ŷ is used for both views, so the loss trains both z1 and z2 to predict the same cluster. In PID, information about the target T that is available from both sources S1 and S2 is precisely the redundant component R(T;S1,S2), not the sum of unique components U(T;S1)+U(T;S2). The fact that z1 and z2 are classified separately does not make the information unique, because uniqueness is defined relative to the other source. No derivation, no specific PID measure (e.g., I_min or I_broja), and no estimator is provided to connect Eq. (2) to unique information, and no PID quantity is computed from the learned representations in the experiments. This is load-bearing: the paper's advertised contribution is the extraction of unique information, and the proposed mechanism appears to target redundant information instead.
  2. [Section 3.5, Eq. (3)] The intra-alignment of any valid SSL baseline with the information bottleneck principle is asserted without proof or a supporting citation: the text states 'since the first term of our loss function, L_SSL is already taken from a valid SSL framework that follows the IB principle (e.g., Barlow-Twins, SimCLR, ...)'. This premise is not demonstrated, and it is not obvious that arbitrary SSL losses (including contrastive, non-contrastive, and whitening losses) each correspond to the same IB objective with a fixed trade-off parameter β. Because the IB-alignment argument underlies the claimed theoretical resolution, this assertion needs to be made precise or replaced with a citation to a formal equivalence result.
  3. [Appendix B] The specification of redundancy as feature correlation and synergy as 'pairwise complementariness' of learned features is ad hoc and is never connected to any formal PID measure. The paper does not estimate R, Sy, or U from the learned representations of any trained model, nor does it compare the proposed method against these estimates. Consequently, the central claim that existing SSL frameworks only use redundant and synergistic information while missing unique information is not empirically verified; it remains an unquantified metaphor rather than an information-theoretic analysis.
  4. [Tables 1-3] The experimental results are reported for single runs without error bars, multiple seeds, or significance tests. The reported improvements range from about 0.74% to 3.3% in top-1 accuracy, and run-to-run variance in SSL pretraining over 1000 epochs can easily be on this scale. The claim of generality across four baselines and four datasets is therefore not statistically established. In addition, key hyperparameters (the α schedule, the number of clusters K, and the re-clustering period) are chosen per-dataset, and the sensitivity analysis in Section 5 only varies K on CIFAR-100; the other choices are not ablated.
  5. [Section 3 and Fig. 1 caption] The proposed method is conceptually close to clustering-based SSL methods such as DeepCluster and SwAV, which also assign pseudo-labels from clustered features and train to predict those labels, yet the paper does not compare against any such method. The caption of Fig. 1 asserts that the framework 'differentiates itself from paradigms including clustering in SSL by joint invariance enforcement to augmented representations', but no experiment or analysis supports that distinction. Without a comparison to clustering-based baselines, the empirical contribution over the existing pseudo-labeling family is not established.
minor comments (5)
  1. [Sections 3 and 4.1] Figures are referenced as 'Fig. ??' in several places, indicating missing figure references that should be fixed.
  2. [Section 4.1] The sentence 'Following original implementation of SimCLR [5], we set τ = 0.1' is ambiguous: τ is used earlier for augmentation operations, and here it presumably refers to the SimCLR temperature parameter; please clarify the notation.
  3. [Section 3.5] There are typos such as 'prin ciple' and 'LP Senables' that should be corrected.
  4. [Section 5B] The ablation text says improvements 'drop to 01.18%' for K=50; the leading zero is a formatting error.
  5. [Section 3.3 and 5B] The number of clusters is set to the number of ground-truth classes in the dataset, which uses label information and should be discussed as a limitation; the sensitivity analysis only examines K on CIFAR-100, not on the other datasets.

Circularity Check

2 steps flagged · score 6.0 of 10

The claimed PID unique-information term is a shared-label classification objective whose label information is redundant under the paper's own Eq. (1), and the pseudo-labels are generated by the model itself; the theoretical contribution reduces to re-labeling self-training as unique information.

  1. self definitional [Section 3.4, Eq. (2) and surrounding text]
    "Specifically, LP Senables the model to learn representation via extracting the unique information of each view (unique component of PID) as it learns from the classification of views individually and independently."

    In Eq. (1), U(T;S1) is information about target T available from source S1 but not from S2. Eq. (2) trains both z1 and z2 to predict the same pseudo-label yhat for both views. Information about a shared label is, by the PID definition in Eq. (1), redundant information R(T;S1,S2), not unique information. No PID measure, estimator, or derivation connects the cross-entropy loss to U; the identification is asserted. Performing two separate forward passes does not make the label information unique, because uniqueness is defined relative to the other source, not by the number of classifiers. Thus the central theoretical claim reduces to renaming a cluster-assignment objective as the unique PID component.

  2. fitted input called prediction [Section 3.3 and Section 3.5]
    "We perform a clustering technique, k-means++ algorithm [1] on the output of the projector of the given framework, fed by the original samples to leverage the learned features to cluster them into a certain number of clusters depending on the prior knowledge of number of classes of the dataset. These labeled (hard pseudo-labeled) samples will be used for next phase of training. ... given that the pseudo-labels are accurate, the second term in the loss, need not proof of alignment with IB as it is in fact supervised learning."

    The 'higher-level supervision' is generated by running k-means on the output of the same projector that is being trained; the model is then supervised to predict those self-generated labels. The paper's IB-alignment argument explicitly conditions on the pseudo-labels being accurate, but accuracy is never established during training, and the labels are a function of the current model. Thus the claimed 'task-related feature learning' and 'unique information' extraction presuppose that the model's own clusters correspond to ground-truth classes, which is exactly the property the method is supposed to learn. The held-out evaluation provides independent empirical grounding, but the theoretical derivation of unique information from pseudo-labels is circular.

full rationale

The paper's empirical contribution is not circular in the narrow predictive sense: final accuracies are measured on held-out labels, so the reported improvements are externally grounded. However, the central theoretical claim—that Eq. (2)'s pseudo-label classification loss instantiates the unique information component of Williams-Beer PID—is asserted without any derivation, estimator, or chosen PID measure. Under the paper's own Eq. (1), the same pseudo-label used for both views is shared label information, which is redundant information, not unique information. The 'individual and independent' classification of the two views does not change this, because uniqueness is defined relative to the other source. Additionally, the pseudo-labels are produced by k-means on the model's own projector outputs, and the IB-alignment argument in Section 3.5 explicitly assumes they are accurate; this makes the 'task-related' supervision a self-referential loop. Self-citations in the paper ([22], [23]) are contextual rather than load-bearing, and no uniqueness theorem is imported from the authors' prior work, so the circularity is concentrated in the unique-information identification and the self-generated pseudo-label supervision. Because the final evaluation is independent, the paper retains partial empirical content, but the theoretical resolution of the mutual-information debate reduces to renaming a self-training objective as a PID component.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a set of unvalidated assumptions: the PID mapping to SSL, the identification of L_PS with unique information, and the reliability of self-generated pseudo-labels. The free parameters are largely hand-tuned, and the number of clusters is set to the true class count, which is a form of label leakage.

free parameters (4)
  • alpha schedule = 10^-5, 10^-4, 10^-3, 10^-2, 10^-1 for epochs 101-200, 201-400, 401-600, 601-800, 801-1000
    Hand-chosen weighting schedule for the pseudo-label loss; grows over training with no sensitivity analysis.
  • number of clusters K = true number of classes (10, 100, 200, or 1000)
    The number of k-means++ clusters is set to the dataset's true class count, leaking label information into pre-training and acting as a tuned hyperparameter.
  • initial training epochs = 100
    Chosen to learn features before pseudo-labeling; no ablation on this value is reported.
  • re-clustering period = 100 epochs
    Pseudo-labels are updated every 100 epochs; fixed without sensitivity analysis.
assumptions (4)
  • domain assumption SSL can be modeled as a three-variable PID system with two view representations as sources and the data distribution as target.
    The paper postulates this mapping in Section 3.1 without validating that the target T is identifiable or that PID quantities are estimable for deep representations.
  • ad hoc to paper The SSL loss L_SSL captures redundant and synergistic information, while the pseudo-label classification loss L_PS captures unique information.
    Core assertion in Section 3.4; no formal derivation connects the losses to PID components.
  • domain assumption Pseudo-labels from k-means++ are accurate enough to provide useful supervision and improve over time.
    Self-training relies on the model's own predictions; the paper does not measure pseudo-label accuracy or address confirmation bias.
  • ad hoc to paper Valid SSL baselines satisfy the information bottleneck principle, so their intra-alignment with IB is assumed.
    Section 3.5 asserts this without proof, using it to claim the combined loss is IB-aligned.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking Self-Supervised Learning Within the Framework of Partial Information Decomposition." pith.science (2026). https://pith.science/paper/UXGDYNVO

@misc{pith2026241202121,
  author       = {Pith},
  title        = {Pith review of: Rethinking Self-Supervised Learning Within the Framework of Partial Information Decomposition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UXGDYNVO}},
  note         = {Machine review of arXiv:2412.02121}
}
read the original abstract

Self Supervised learning (SSL) has demonstrated its effectiveness in feature learning from unlabeled data. Regarding this success, there have been some arguments on the role that mutual information plays within the SSL framework. Some works argued for increasing mutual information between representation of augmented views. Others suggest decreasing mutual information between them, while increasing task-relevant information. We ponder upon this debate and propose to revisit the core idea of SSL within the framework of partial information decomposition (PID). Thus, with SSL under PID we propose to replace traditional mutual information with the more general concept of joint mutual information to resolve the argument. Our investigation on instantiation of SSL within the PID framework leads to upgrading the existing pipelines by considering the components of the PID in the SSL models for improved representation learning. Accordingly we propose a general pipeline that can be applied to improve existing baselines. Our pipeline focuses on extracting the unique information component under the PID to build upon lower level supervision for generic feature learning and on developing higher-level supervisory signals for task-related feature learning. In essence, this could be interpreted as a joint utilization of local and global clustering. Experiments on four baselines and four datasets show the effectiveness and generality of our approach in improving existing SSL frameworks.

Figures

Figures reproduced from arXiv: 2412.02121 by the authors.

Figure 1
Figure 1. A given SSL framework undergoes an initial training, and clustering-based labeling, before the main phase of training, iterative refinement. The three-variable information system under PID framework is instantiated to have all three components of PID, namely unique, redundant and synergistic components. The detailed derivation supporting this design is depicted in Supplementary. Our framework differentiates itself f… view at source ↗
Figure 2
Figure 2. PID in case of three variables, PID presents the structure of multivariate information consisting of two source variables S1 and S2 as well as a target variable T . where R, Sy, U are redundant, synergistic and unique com￾ponents of information respectively. Williams et al. [29] present such decomposition in a way that all three compo￾nents are non-negative. There are different ways in which one can perform the deco… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 24 canonical work pages

  1. [1]

    K-means++ the ad- vantages of careful seeding

    David Arthur and Sergei Vassilvitskii. K-means++ the ad- vantages of careful seeding. In Proceedings of The Eigh- teenth Annual ACM-SIAM Symposium on Discrete Algo- rithms, pages 1027–1035, 2007. 5

    work page 2007

  2. [2]

    Direc- tional self-supervised learning for heavy image augmenta- tions

    Yalong Bai, Yifan Yang, Wei Zhang, and Tao Mei. Direc- tional self-supervised learning for heavy image augmenta- tions. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 16692–16701,

    work page

  3. [3]

    Deep clustering for unsupervised learning of visual features

    Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pages 132–149, 2018. 1

    work page 2018

  4. [4]

    Unsupervised learning of visual features by contrasting cluster assignments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Pi- otr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Ad- vances in Neural Information Processing Systems, 33:9912– 9924, 2020. 1

    work page 2020

  5. [5]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 1, 2, 6, 7

    work page 2020

  6. [6]

    Exploring simple siamese rep- resentation learning

    Xinlei Chen and Kaiming He. Exploring simple siamese rep- resentation learning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 15750–15758, 2021. 1

    work page 2021

  7. [7]

    Cover and Joy A

    Thomas M. Cover and Joy A. Thomas. Elements of informa- tion theory (2. ed.). Wiley, 2006. 1, 3

    work page 2006

  8. [8]

    solo-learn: A library of self- supervised methods for visual representation learning

    Victor Guilherme Turrisi da Costa, Enrico Fini, Moin Nabi, Nicu Sebe, and Elisa Ricci. solo-learn: A library of self- supervised methods for visual representation learning. J. Mach. Learn. Res., 23:56–1, 2022. 6

    work page 2022

Show all 30 references
  1. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. Ieee, 2009. 6

    2009

  2. [10]

    Whitening for self-supervised representation learning

    Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening for self-supervised representation learning. In International Conference on Machine Learning, pages 3015–3024. PMLR, 2021. 1, 2, 3, 4, 6, 7, 12

    2021

  3. [11]

    Scaling and benchmarking self-supervised visual rep- resentation learning

    Priya Goyal, Dhruv Mahajan, Abhinav Gupta, and Ishan Misra. Scaling and benchmarking self-supervised visual rep- resentation learning. In Proceedings of the ieee/cvf Inter- national Conference on Computer Vision, pages 6391–6400,

  4. [12]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Ghesh- laghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neur...

    2020

  5. [13]

    Bits and pieces: Understanding information decomposition from part-whole relationships and formal logic

    Aaron J Gutknecht, Michael Wibral, and Abdullah Makkeh. Bits and pieces: Understanding information decomposition from part-whole relationships and formal logic. Proceedings of the Royal Society A, 477(2251):20210110, 2021. 3

    2021

  6. [14]

    On feature decorrelation in self- supervised learning

    Tianyu Hua, Wenxiao Wang, Zihui Xue, Sucheng Ren, Yue Wang, and Hang Zhao. On feature decorrelation in self- supervised learning. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 9598–9608,

  7. [15]

    Learning where to learn in cross-view self-supervised learning

    Lang Huang, Shan You, Mingkai Zheng, Fei Wang, Chen Qian, and Toshihiko Yamasaki. Learning where to learn in cross-view self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14451–14460, 2022. 2

    2022

  8. [16]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

    arXiv

  9. [17]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 6

    2009

  10. [18]

    Tiny imagenet visual recognition challenge

    Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015. 6

    2015

  11. [19]

    Prototypical contrastive learning of unsupervised representa- tions

    Junnan Li, Pan Zhou, Caiming Xiong, and Steven CH Hoi. Prototypical contrastive learning of unsupervised representa- tions. arXiv preprint arXiv:2005.04966, 2020. 1

    2005 arXiv

  12. [20]

    Active uncertainty representation learn- ing: Toward more label efficiency in deep learning

    Salman Mohamadi. Active uncertainty representation learn- ing: Toward more label efficiency in deep learning. 2024. 2

    2024

  13. [21]

    Deep active ensemble sampling for image classifica- tion

    Salman Mohamadi, Gianfranco Doretto, and Donald A Ad- jeroh. Deep active ensemble sampling for image classifica- tion. arXiv preprint arXiv:2210.05770, 2022. 1

    2022 arXiv

  14. [22]

    Fussl: Fuzzy uncertain self supervised learning

    Salman Mohamadi, Gianfranco Doretto, and Donald A Ad- jeroh. Fussl: Fuzzy uncertain self supervised learning. In Proceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision, pages 2799–2808, 2023. 2

    2023

  15. [23]

    More synergy, less redundancy: Exploiting joint mu- tual information for self-supervised learning

    Salman Mohamadi, Gianfranco Doretto, and Donald A Ad- jeroh. More synergy, less redundancy: Exploiting joint mu- tual information for self-supervised learning. In 2023 IEEE International Conference on Image Processing (ICIP), pages 1390–1394. IEEE, 2023. 2

    2023

  16. [24]

    Tighter variational bounds are not necessarily better

    Tom Rainforth, Adam Kosiorek, Tuan Anh Le, Chris Mad- dison, Maximilian Igl, Frank Wood, and Yee Whye Teh. Tighter variational bounds are not necessarily better. In In- ternational Conference on Machine Learning , pages 4277–

  17. [25]

    Decomposed mutual information estimation for contrastive representation learning

    Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, and Remi Tachet Des Combes. Decomposed mutual information estimation for contrastive representation learning. In International Conference on Ma- chine Learning, pages 9859–9869. PMLR, 2021. 1, 3

    2021

  18. [26]

    What makes for good 9 views for contrastive learning? Advances in Neural Infor- mation Processing Systems, 33:6827–6839, 2020

    Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good 9 views for contrastive learning? Advances in Neural Infor- mation Processing Systems, 33:6827–6839, 2020. 1, 12, 13

    2020

  19. [27]

    Synergy, redundancy, and multivariate infor- mation measures: an experimentalist’s perspective

    Nicholas Timme, Wesley Alford, Benjamin Flecker, and John M Beggs. Synergy, redundancy, and multivariate infor- mation measures: an experimentalist’s perspective. Journal of computational neuroscience, 36(2):119–140, 2014. 3

    2014

  20. [28]

    On mutual information maximization for representation learning

    Michael Tschannen, Josip Djolonga, Paul K Rubenstein, Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. arXiv preprint arXiv:1907.13625, 2019. 1

    1907 arXiv

  21. [29]

    Nonnegative de- composition of multivariate information

    Paul L Williams and Randall D Beer. Nonnegative de- composition of multivariate information. arXiv preprint arXiv:1004.2515, 2010. 2, 3, 11

    2010 arXiv

  22. [30]

    Barlow twins: Self-supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St´ephane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Ma- chine Learning, pages 12310–12320. PMLR, 2021. 1, 2, 3, 4, 6, 7, 12 10 A. More on PID Mutual information is a...

    2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.