REVIEW 4 major objections 6 minor 38 references
$\texttt{InfoHier}$: Hierarchical Information Extraction via Encoding and Embedding
T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read InfoHier claims that jointly training a contrastive self-supervised loss with a continuous hierarchical-clustering loss makes learned representations reflect the data's underlying hierarchy, recovering CIFAR100 superclasses from unlabeled…
desk verdict InfoHier is an honest, clearly written vision for combining contrastive SSL with hyperbolic hierarchical clustering, but it contains no evidence that the combination works, so it reads as a project proposal rather than a paper with a supported claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the joint loss $\mathcal{L} = \lambda_{\text{ct}} \mathcal{L}_{\text{ct}} + \lambda_{\text{hc}} \mathcal{L}_{\text{hc}}$, where $\mathcal{L}_{\text{hc}}$ is a continuous Dasgupta hierarchical-clustering loss and $\mathcal{L}_{\text{ct}}$ is a unified contrastive loss. The HC term is written as a sum over triples $(i,j,k)$ of pairwise similarities minus a triplet similarity $w_{ijk}(\mathcal{E}_X; w)$ that is determined by the decoded tree, equivalently by the embedding in two-dimensional hyperbolic space. This continuous relaxation is what makes the tree learnable by gradient descent, and the contrastive term is what steers the encoder toward representations that respect the hierarchy. The two gradient flows refine each other: hierarchical clustering gives self-supervised learning a structural target, and self-supervised learning gives clustering adaptive similarity measures.
What would settle it
Train InfoHier on a dataset with a known ground-truth hierarchy and compare the decoded tree against that hierarchy with a tree-agreement measure while monitoring the HC loss; if the loss keeps decreasing but agreement with the ground truth does not improve, the surrogate loss is unfaithful. A simpler check is to disable the hierarchical term entirely and show that the decoded tree is no better than chance at recovering the known superclasses.
Extended reading notes
Core claim
The central discovery claim is that self-supervised learning and hierarchical clustering can be co-trained through one differentiable objective, producing latent representations that are hierarchy-aware. The objective pairs a contrastive loss, expressed in a unified form that covers multiple SSL methods, with an HC loss obtained by reformulating the Dasgupta cost as a triplet sum over a two-dimensional hyperbolic embedding. The embedding decodes into a rooted binary tree whose leaves are data points, and the two loss terms are balanced by hyperparameters. On a 64-sample subset of CIFAR100 covering 16 classes and four superclasses, InfoHier reports that most samples of the same superclass form distinct clusters with no label information, and argues that the recovered structure can expose intrinsic hierarchies beyond the human-defined categories.
Load-bearing premise
The load-bearing premise is that the smooth hierarchical-clustering loss really captures the discrete Dasgupta cost, so that minimizing it yields a tree matching the true data hierarchy; the paper does not prove this equivalence and says the proof is planned for future work.
Editorial extensions
If this is right
- Unlabeled datasets can be organized into trees at customizable levels of granularity, enabling category-specific retrieval.
- The recovered hierarchy supports balanced sampling, giving a structured way to handle imbalanced datasets.
- Vector databases can use the hierarchy for hierarchical indexing, improving retrieval efficiency and relevance.
- Multi-tiered storage systems could use the structure to identify high-priority data and reduce latency.
- Nearest-neighbor search can be narrowed by first descending the hierarchy, benefiting recommendation and search systems.
Reading between the lines
- If the claim holds, the Dasgupta cost of the tree induced by a learned representation becomes a label-free proxy for representation quality, giving SSL an evaluation signal that does not need downstream classifiers.
- A testable extension: run InfoHier on ImageNet's full class hierarchy and measure whether the decoded tree agrees with the known superclass structure at several depths, not just one level.
- The same loop could be inverted: hierarchy-aware clustering could choose hard negatives for contrastive learning, making the structural signal shape the representation instead of only checking it afterward.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes InfoHier, a framework that jointly trains a self-supervised contrastive encoder with a hierarchical-clustering loss based on a continuous relaxation of the Dasgupta cost in hyperbolic space. The stated goal is to learn representations that reflect the underlying hierarchy of unlabeled data, improving both clustering and representation learning. The manuscript describes the joint loss, presents a schematic architecture, and reports a single qualitative visualization on 64 CIFAR100 images grouped into four superclasses. The authors explicitly frame the work as preliminary and defer comprehensive experiments and theoretical proofs to future publication.
Significance. The problem addressed is real: SSL methods typically produce flat representations, while classical hierarchical clustering is sensitive to fixed similarity metrics, and combining the two in an end-to-end trainable way is a plausible direction. The paper is clearly written and cites relevant prior work on hyperbolic hierarchical clustering and contrastive SSL. However, as submitted, the manuscript is a position/proposal rather than a validated research contribution: the only experimental evidence is a 64-sample visualization, the central loss is under-specified, and no quantitative evaluation or comparison is provided. If the framework were validated with controlled experiments and a rigorous derivation of the HC loss, it could be of interest to the hierarchical representation learning and information retrieval communities, but that validation is not present here.
major comments (4)
- [Section 4, Figure 4] The only experimental evidence for the central claim is a qualitative visualization on 64 CIFAR100 samples. No quantitative metrics (e.g., clustering accuracy, dendrogram purity, Dasgupta cost, or linear-probe accuracy), no baselines (e.g., SimCLR-only or HC-only), and no error bars or ablations are reported. Consequently, the claim in Section 1 that joint training 'enhances both clustering and representation learning' is not supported by the presented evidence.
- [Section 3, Eq. (1)] The HC loss is not fully specified: the triplet similarity wijk(EX; w) is described only as being determined by the tree structure or equivalently by the embedding EX, but no explicit formula, decoding procedure from the hyperbolic embedding to a tree, or derivation of the equivalence to the discrete Dasgupta cost is given. Since Section 5 explicitly defers 'essential theoretical proofs' to future publication, the core objective of the method cannot be reproduced or verified from the manuscript as it stands.
- [Section 4, Figure 4] The visualization does not demonstrate that the HC loss contributes beyond the pretrained SimCLR encoder. Because no comparison to a model trained with the contrastive loss alone is provided, the figure is compatible with the possibility that the pretrained encoder already separates the four CIFAR100 superclasses in the sampled 64 images, making the joint-training benefit unsubstantiated.
- [Section 3, Eq. (3)] The joint objective is a fixed weighted sum with no stated values, tuning procedure, or sensitivity analysis for the hyperparameters lambda_ct and lambda_hc. Given that Section 5 acknowledges different convergence rates of the encoder and embedding networks, the reported behavior cannot be attributed to the proposed joint objective rather than to an arbitrary hyperparameter choice.
minor comments (6)
- [Section 3, Eq. (2)] The notation in Eq. (2) is imprecise: theta, z_i, z'_i, and z_j are not defined before the equation; please specify that theta denotes the trainable encoder parameters and clearly define the positive and negative pairs used in the contrastive loss.
- [Section 3, paragraph 2] The text states that the raw input is encoded by a 'pre-trained encoder' and later says the contrastive loss 'further fine-tune[s] the encoder,' but the training protocol is not specified; clarify whether the encoder is initialized from a SimCLR checkpoint and whether both the encoder and the hyperbolic embedding map are fine-tuned together.
- [References, [32]] Reference [32] appears garbled as 'YM., A., C., R., A., V.'; it should be formatted as Asano, Y.M., Rupprecht, C., Vedaldi, A., with the correct title and venue.
- [Section 5, first paragraph] The phrase 'demonstrated on a large-scale problem' conflicts with the actual experiment, which uses only 64 samples; please rephrase to describe the result as a small-scale illustrative study.
- [Figure 2 and surrounding text] The caption and the inline description of Figure 2 are hard to follow, particularly the 'GFP #Distance' table; please clarify the distance values and the intended comparison between images D02, C05, and B04.
- [Section 5, second paragraph] The claim that hyperbolic embeddings give InfoHier 'greater scalability over traditional methods' is not supported by any experiment or citation in the manuscript; either justify this claim or remove it.
Circularity Check
No significant circularity: the joint loss is a weighted combination of borrowed, independently-motivated HC and SSL losses; the visualization and deferred proofs are evidence gaps, not definitional circularity.
full rationale
The paper's derivation chain begins with two existing loss families. Equation (1) is a continuous surrogate of the Dasgupta hierarchical-clustering cost, attributed to prior gradient-based HC work (refs 6, 9, 26); equation (2) is a generic unification of contrastive losses taken from ref 29; equation (3) is simply the weighted sum of the two. No parameter is fitted on a subset and then presented as a prediction of that same subset, and the HC objective is not defined in terms of the reported CIFAR100 superclasses. The only empirical evidence is the 64-sample visualization of Figure 4, and Section 5 explicitly acknowledges that essential theoretical proofs and comprehensive experiments are planned for future publication. That is an admission of missing support, which the reviewer should weigh, but it is not circularity. The self-citations (refs 34-36) appear only in the speculative applications paragraph about storage management and do not support the central representation-learning claim. No uniqueness theorem, ansatz smuggled by citation, or renaming of a known result is used to force the outcome. The central claim therefore has independent content even though it is not yet convincingly demonstrated.
Assumptions & free parameters
free parameters (3)
- lambda_ct
- lambda_hc
- hyperbolic_embedding_dimension =
2
assumptions (4)
- domain assumption The continuous Dasgupta loss (Eq. 1) is a valid differentiable surrogate for the discrete tree cost.
- standard math The unified contrastive loss form (Eq. 2) covers the NT-Xent loss used in the experiments.
- domain assumption A 2D hyperbolic space can embed the CIFAR100 hierarchy with sufficient fidelity.
- domain assumption A pre-trained ResNet-18 provides a suitable starting latent space.
Cite this review
Pith. "Pith review of $\texttt{InfoHier}$: Hierarchical Information Extraction via Encoding and Embedding." pith.science (2026). https://pith.science/paper/EDYHPYIC
@misc{pith2026250108717,
author = {Pith},
title = {Pith review of: $\textttInfoHier$: Hierarchical Information Extraction via Encoding and Embedding},
year = {2026},
howpublished = {\url{https://pith.science/paper/EDYHPYIC}},
note = {Machine review of arXiv:2501.08717}
}
abstract
Analyzing large-scale datasets, especially involving complex and high-dimensional data like images, is particularly challenging. While self-supervised learning (SSL) has proven effective for learning representations from unlabelled data, it typically focuses on flat, non-hierarchical structures, missing the multi-level relationships present in many real-world datasets. Hierarchical clustering (HC) can uncover these relationships by organizing data into a tree-like structure, but it often relies on rigid similarity metrics that struggle to capture the complexity of diverse data types. To address these we envision $\texttt{InfoHier}$, a framework that combines SSL with HC to jointly learn robust latent representations and hierarchical structures. This approach leverages SSL to provide adaptive representations, enhancing HC's ability to capture complex patterns. Simultaneously, it integrates HC loss to refine SSL training, resulting in representations that are more attuned to the underlying information hierarchy. $\texttt{InfoHier}$ has the potential to improve the expressiveness and performance of both clustering and representation learning, offering significant benefits for data analysis, management, and information retrieval.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Balestriero, R., Ibrahim, M., Sobal, V., Morcos, A., Shekhar, S., Goldstein, T., Bor- des, F., Bardes, A., Mialon, G., Tian, Y., Schwarzschild, A., Wilson, A.G., Geiping, J., Garrido, Q., Fernandez, P., Bar, A., Pirsiavash, H., LeCun, Y., Goldblum, M.: A cookbook of self-supervised learning (2023),https://arxiv.org/abs/2304.12210
arXiv 2023
-
[2]
arXiv preprint arXiv:2105.04906 (2021) 8 T
Bardes, A., Ponce, J., LeCun, Y.: Vicreg: Variance-invariance-covariance regular- ization for self-supervised learning. arXiv preprint arXiv:2105.04906 (2021) 8 T. Zhang et al
arXiv 2021
-
[3]
In: Proceedings of the European conference on computer vision (ECCV)
Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features. In: Proceedings of the European conference on computer vision (ECCV). pp. 132–149 (2018)
2018
-
[4]
Advances in neural information processing systems33, 9912–9924 (2020)
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems33, 9912–9924 (2020)
2020
-
[5]
In: Proceedings of the IEEE/CVF international conference on computer vision
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9650–9660 (2021)
2021
-
[6]
In: Proceedings of the 34th Interna- tional Conference on Neural Information Processing Systems
Chami, I., Gu, A., Chatziafratis, V., Ré, C.: From trees to continuous embeddings and back: hyperbolic hierarchical clustering. In: Proceedings of the 34th Interna- tional Conference on Neural Information Processing Systems. NIPS ’20, Curran Associates Inc., Red Hook, NY, USA (2020)
work page 2020
-
[7]
In: International conference on machine learning
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International conference on machine learning. pp. 1597–1607. PMLR (2020)
2020
-
[8]
Chen,X.,He,K.:Exploringsimplesiameserepresentationlearning.In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 15750–15758 (2021)
2021
Show all 38 references
-
[9]
Advances in neural information processing systems32 (2019)
Chierchia, G., Perret, B.: Ultrametric fitting by gradient descent. Advances in neural information processing systems32 (2019)
2019
-
[10]
In: Pro- ceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing
Dasgupta, S.: A cost function for similarity-based hierarchical clustering. In: Pro- ceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing. p. 118–127. STOC ’16, Association for Computing Machinery, New York, NY, USA (2016), https://doi.org/10.1145/2897518.2897527
2016
-
[11]
In: CVPR 2011
Deng, J., Berg, A.C., Fei-Fei, L.: Hierarchical semantic indexing for large scale image retrieval. In: CVPR 2011. pp. 785–792 (2011).https://doi.org/10.1109/ CVPR.2011.5995516
2011
-
[12]
In: 2009 IEEE Conference on Computer Vision and Pattern Recognition
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255 (2009).https://doi.org/10.1109/CVPR.2009. 5206848
2009 doi
-
[13]
In: The World Wide Web Conference
Epasto, A., Perozzi, B.: Is a single embedding enough? learning node representa- tions that capture multiple social contexts. In: The World Wide Web Conference. p. 394–404. WWW ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3308558.3313660
2019
-
[14]
Ermolov, A., Siarohin, A., Sangineto, E., Sebe, N.: Whitening for self-supervised representationlearning.In:Internationalconferenceonmachinelearning.pp.3015–
-
[15]
National Science Review 1(2), 293–314 (02 2014).https://doi.org/10.1093/nsr/nwt032
Fan, J., Han, F., Liu, H.: Challenges of Big Data analysis. National Science Review 1(2), 293–314 (02 2014).https://doi.org/10.1093/nsr/nwt032
2014 doi
-
[16]
Advances in neural information processing systems 33, 21271–21284 (2020)
Grill, J.B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Do- ersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al.: Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems 33, 212...
2020
-
[17]
arXiv preprint arXiv:2310.11703 (2023)
Han, Y., Liu, C., Wang, P.: A comprehensive survey on vector database: Storage and retrieval technique, challenge. arXiv preprint arXiv:2310.11703 (2023)
2023
-
[18]
Advances in Neural Information Pro- cessing Systems 34, 5000–5011 (2021) InfoHier: Hierarchical Information Extraction via Encoding and Embedding 9
HaoChen,J.Z.,Wei,C.,Gaidon,A.,Ma,T.:Provableguaranteesforself-supervised deep learning with spectral contrastive loss. Advances in Neural Information Pro- cessing Systems 34, 5000–5011 (2021) InfoHier: Hierarchical Information Extraction via Encoding and Embedding 9
2021
-
[19]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16000–16009 (2022)
2022
-
[20]
Jagadish, H.V., Gehrke, J., Labrinidis, A., Papakonstantinou, Y., Patel, J.M., Ra- makrishnan, R., Shahabi, C.: Big data and its technical challenges. Commun. ACM 57(7), 86–94 (Jul 2014).https://doi.org/10.1145/2611567
2014 doi
-
[21]
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
2009
-
[22]
IEEE Transactions on Knowledge and Data Engineering 35(1), 857–876 (2023)
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., Tang, J.: Self- supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering 35(1), 857–876 (2023). https://doi.org/10.1109/TKDE. 2021.3090866
2023
-
[23]
Nature methods9(7), 637–637 (2012)
Ljosa, V., Sokolnicki, K.L., Carpenter, A.E.: Annotated high-throughput mi- croscopy image sets for validation. Nature methods9(7), 637–637 (2012)
2012
-
[24]
IEEE Transactions on Pattern Analysis and Machine Intelligence 42(4), 824–836 (2020)
Malkov, Y.A., Yashunin, D.A.: Efficient and robust approximate nearest neigh- bor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(4), 824–836 (2020). https: //doi.org/10.1109/TPAMI.2018.2889473
2020
-
[25]
Nature 498(7453), 255–260 (2013)
Marx, V.: The big challenges of big data. Nature 498(7453), 255–260 (2013). https://doi.org/10.1038/498255a
2013 doi
-
[26]
In: Proceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining
Monath, N., Zaheer, M., Silva, D., McCallum, A., Ahmed, A.: Gradient-based hi- erarchical clustering using continuous representations of trees in hyperbolic space. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining. p. 714–72...
2019 doi
-
[27]
arXiv preprint arXiv:1807.03748 (2018)
Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748 (2018)
2018 arXiv
-
[28]
In: The Twelfth International Conference on Learning Repre- sentations (2024), https://openreview.net/forum?id=hLZQTFGToA
Tan, Z., Zhang, Y., Yang, J., Yuan, Y.: Contrastive learning is spectral clustering on similarity graph. In: The Twelfth International Conference on Learning Repre- sentations (2024), https://openreview.net/forum?id=hLZQTFGToA
2024
-
[29]
Advances in Neural Information Processing Systems35, 19511–19522 (2022)
Tian, Y.: Understanding deep contrastive learning via coordinate-wise optimiza- tion. Advances in Neural Information Processing Systems35, 19511–19522 (2022)
2022
-
[30]
In: Proceedings of the 25th Interna- tional Conference on Machine Learning
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th Interna- tional Conference on Machine Learning. p. 1096–1103. ICML ’08, Association for Computing Machinery, New York, NY,...
2008
-
[31]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Yang, J., Parikh, D., Batra, D.: Joint unsupervised learning of deep representations and image clusters. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 5147–5156 (2016)
2016
-
[32]
In: International Conference on Learning Representations (2020), https://openreview.net/forum?id=Hyx-jyBFPr
YM., A., C., R., A., V.: Self-labelling via simultaneous clustering and represen- tation learning. In: International Conference on Learning Representations (2020), https://openreview.net/forum?id=Hyx-jyBFPr
2020
-
[33]
In: International conference on machine learn- ing
Zbontar, J., Jing, L., Misra, I., LeCun, Y., Deny, S.: Barlow twins: Self-supervised learning via redundancy reduction. In: International conference on machine learn- ing. pp. 12310–12320. PMLR (2021)
2021
-
[34]
Proceedings of the VLDB Endowment
Zhang, T.: Autonomous hierarchical storage management via reinforcement learn- ing. Proceedings of the VLDB Endowment. ISSN2150, 8097 (2024) 10 T. Zhang et al
2024
-
[35]
Expert Systems with Applications 237, 121443 (2023), https: //doi.org/10.1016/j.eswa.2023.121443
Zhang, T., Gupta, A., Rodríguez, M.A.F., Spjuth, O., Hellander, A., Toor, S.: Data managementofscientificapplicationsinareinforcementlearning-basedhierarchical storage system. Expert Systems with Applications 237, 121443 (2023), https: //doi.org/10.1016/j.eswa.2023.121443
2023
-
[36]
IEEE Transactions on Knowledge and Data Engineering 35(6), 5780–5793 (2022)
Zhang, T., Hellander, A., Toor, S.: Efficient hierarchical storage management empowered by reinforcement learning. IEEE Transactions on Knowledge and Data Engineering 35(6), 5780–5793 (2022). https://doi.org/10.1109/TKDE. 2022.3176753
2022
-
[37]
Proceedings of the AAAI Conference on Artifi- cial Intelligence 34(03), 3065–3072 (Apr 2020).https://doi.org/10.1609/aaai
Zhang, Z., Cai, J., Zhang, Y., Wang, J.: Learning hierarchy-aware knowledge graph embeddings for link prediction. Proceedings of the AAAI Conference on Artifi- cial Intelligence 34(03), 3065–3072 (Apr 2020).https://doi.org/10.1609/aaai. v34i03.5701
2020 doi
-
[38]
In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J
Zhang, Z., Zhuang, F., Qu, M., Lin, F., He, Q.: Knowledge graph embedding with hierarchical relation structure. In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pp. 3198–3207....
2018 doi
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.