{"id":"fc96f863-9385-4c51-a853-12c9db72b468","arxiv_id":"2505.04484","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A tutorial argues that mutual information has been the guiding objective of discriminative clustering across its evolution, and reframes contrastive clustering as implicit variational inference over an energy-based model.","lead":"This tutorial traces how discriminative clustering evolved from linear decision boundaries to contrastive learning, with mutual information as the recurring objective. It also introduces GemClus, a Python package for the discussed methods, and argues that model selection should match the clustering hypotheses.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The contrastive-learning reinterpretation hinges on an energy-based variational family whose optimality is asserted, not derived; Sec. 5.3.2 equates a lower-bound objective with recovery of p(x|aug(x)) without a tightness or expressivity check.","rationale":"The reader's weakest_assumption correctly identifies the capacity/tightness issue at Eq. (53), and I agree that the paper provides no formal guarantee that finite critics make q_theta approach p(x|z). That said, I do not think this concern moves the verdict. As a tutorial, the interpretive claim is explicitly framed as a proposal rather than a theorem, and the paper's own language ('we show', 'we highlight') introduces the energy-based reading as a variational perspective backed by the standard InfoNCE derivation in Eq. (59). The paper is conditional because of concrete algebraic errors (Eq. (8) sign error in the sigmoid, Eq. (43) sign error in the MI decomposition), which are unambiguous and fixable, not because of the capacity concern, which is a genuine open matter but does not invalidate the tutorial's message. The paper self-reports its Sec. 7.6 example is rigged, which is a strength rather than an objection because it is a warning. The load-bearing concern I identified is the tightness claim underlying the central reinterpretation; I recommend no verdict change because the paper's claim is a tutorial-level interpretive synthesis with the required standard derivation, and the remaining algebraic errors are what justify the conditional verdict.","tokens_in":31942,"tokens_out":2417,"duration_ms":21956,"concrete_test":"On the two-circle example from Sec. 7.6, fix the learned critic E_theta and estimate the normalized density ratio q_theta(x|Aug(x)) = p_data(x) exp(E_theta(x,Aug(x))) / E_{p_data}[exp(E_theta(.,Aug(x)))] via a large held-out sample. Compare the conditional distributions q_theta(x|z) against kernel density estimates of p(x|z) for several augmentations z (e.g., rotation angles in [0,2pi]). If the KL divergence between q_theta and the estimated p(x|z) does not decrease as the critic/backbone capacity increases (wider MLP or more epochs), the tightness/expressivity assumption underlying the paper's Sec. 5.3.2 and Sec. 8 conclusion is unsupported. If the divergence does decrease toward zero, the paper's energy-based reinterpretation is empirically validated on its own example.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Secs. 5.3.2 and 8) is that contrastive clustering does not maximize a learned MI but instead trains an energy-based model q_theta(x|z) proportional to p_data(x) exp(E_theta(x,z)) to approximate the true posterior p(x|z) with z=Aug(x). The argument in Sec. 5.3.1 uses the Barber-Agakov lower bound with variational family q(x|z)=p_data(x)e^{E(x,z)}/Z(z), Eq. (53), and notes the optimal critic is E*=log p(z|x), Eq. (60), which would make q equal p(x|z). However, no tightness or capacity statement is given for finite critic families, and Eq. (62) is only a lower bound on the constant I(x;Aug(x)). The paper then asserts 'contrastive learning is about learning the distribution p(x|aug(x)) using the approximate q(x|aug(x))' and that the 'true underlying model is an energy-based model' (Sec. 8). The load-bearing assumption is that the energy-based variational family is sufficiently rich so that optimizing the InfoNCE lower bound drives q_theta toward p(x|z); otherwise, training maximizes a lower bound on a constant, and the conclusion reduces to a heuristic interpretation. The paper also acknowledges in Sec. 7.6 that its worked example deliberately rigs augmentations, which supports the interpretation but does not validate the capacity/tightness claim. A separate algebraic error in Eq. (43) (sign-flipped MI decomposition) is real but does not affect the main conceptual hinge.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This tutorial reviews discriminative clustering and the role of mutual information as a training objective, from early work by Bridle et al. and Krause et al. through deep clustering and contrastive learning. It introduces generative versus discriminative modelling, explains mutual-information estimation and its limitations, discusses internal model-selection metrics, and presents reproducible experiments with the GemClus package. The paper's central interpretive claim, developed in Sections 5.3.2 and 8, is that contrastive clustering does not maximize a learned mutual information: because the augmentation strategy is fixed, I(x;Aug(x)) is constant, and the actual optimized object is an energy-based variational model q_theta(x|z) proportional to p_data(x) exp(E(x,z)), so that \"contrastive learning is about learning the distribution p(x|aug(x)) using the approximate q(x|aug(x))\".","tokens_in":32284,"tokens_out":14195,"duration_ms":127554,"significance":"If the central claim is adequately supported, this tutorial fills a genuine gap: it connects historical MI-based discriminative clustering to modern contrastive methods, and it offers a provocative reinterpretation of contrastive losses as energy-based posterior estimation rather than MI maximization. The paper is generally clearly written, and it ships reproducible code (GemClus and a companion notebook) and an honest admission that the contrastive example in Section 7.6 is deliberately rigged. However, the interpretive claim is currently asserted from an optimal-critic argument without a tightness or expressivity analysis, and several display equations in the derivation are incorrect or inconsistent with each other. The correctable nature of these issues makes major revision appropriate.","major_comments":[{"comment":"Equation (62) is introduced as \"rewriting Eq. (59)\" for contrastive learning, but the denominator has the indices reversed. Eq. (59) has denominator sum_j exp(E(xxx_j, z_i)); with z_i = Aug(xxx_i) this is sum_j exp(E(xxx_j, Aug(xxx_i))). Eq. (62) instead writes sum_j exp(E(xxx_i, Aug(xxx_j))). These are different lower bounds: the first corresponds to the variational family q(x|z) used in Eq. (53), while the second corresponds to a family q(z|x). Only the first orientation supports the paper's conclusion that contrastive learning learns q(x|Aug(x)). The code in Listing 5 uses softmax(similarities, dim=0), which implements the first orientation, so Eq. (62) should be corrected to match either the derivation or the code.","section":"5.3.2, Eq. (62)"},{"comment":"The multi-sample derivation leading to the InfoNCE lower bound is not correct as written. Eq. (56) is true only if the variable on the right is the representation zzz_1 of the first sample and the other samples are independent of (xxx_1, zzz_1). Eq. (57) then changes the object: I(xxx_1,...,xxx_n; zzz) = (1/n) sum_i I(xxx_i; zzz_i) is not a standard identity unless zzz denotes the collection (zzz_1,...,zzz_n), in which case the sum has no 1/n factor (equivalently, by i.i.d. each term equals I(xxx_1; zzz_1)). The final lower bound Eq. (59) is correct, but the intermediate steps should be rewritten with explicit random variables so that readers can follow the derivation.","section":"5.3.1, Eqs. (56)-(58)"},{"comment":"The central claim that contrastive learning \"is about learning the distribution p(x|aug(x)) using the approximate q(x|aug(x))\" relies on the optimal-critic identity E(xxx,zzz) = log p(zzz|xxx) in Eq. (60). This identity yields q = p(x|z) only when the critic family contains that log-density and the energy-based family in Eq. (53) is rich enough to represent the posterior. For a fixed finite-capacity critic, the objective is only a lower bound on the constant I(xxx;Aug(xxx)); no tightness, expressivity, or consistency argument is given. As a result, Section 8's statement that \"the true underlying model is an energy-based model\" overstates what the derivation shows. The paper's admission in Section 7.6 that the example is rigged is commendable, but it also means that the experiment provides no independent support for the variational claim. Please add an explicit discussion of the idealized conditions under which the interpretation is exact and soften the conclusions accordingly.","section":"5.3.2 and Section 8"},{"comment":"Equation (43) has a sign error: the mutual-information decomposition is I(xxx;y) = H(y) - H(y|xxx), not H(y|xxx) - H(y). The surrounding sentence that a good clustering model is \"fair but firm\" corresponds to the correct sign, since fairness is H(y) and firmness is -H(y|xxx). This is a tutorial, so the equation should be fixed to avoid propagating the error.","section":"5.1, Eq. (43)"}],"minor_comments":[{"comment":"The sigmoid is written as 1/(1 - e^{-a}), which is undefined at a = 0 and is not a sigmoid; it should be 1/(1 + e^{-a}).","section":"2.2.1, Eq. (8)"},{"comment":"The expression after \"we ignore the constant\" is the negative conditional entropy -E[sum_k p_theta(y=k|x) log p_theta(y=k|x)], not the cross-entropy; the factor multiplying log p_theta(y=k|x) should be p_data(y=k|x) (or the empirical indicator) to match Eq. (28).","section":"3.2, Eq. (26)"},{"comment":"The KL notation has mismatched parentheses: it should read D_KL(q1(z) || q2(z)) without the extra parenthesis after q1(z).","section":"2.1, Eq. (21)"},{"comment":"Several typos should be cleaned up: \"Econstrastive\" in Eqs. (62)-(63), \"we we do not have access\" in Section 3.1, \"the case the case\" in the Introduction, and \"ernegy-based\" in Section 5.4.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"Most of the worked examples and the package are tied to the authors' own GEMINI/GemClus line of work, and the paper repeatedly cites their own prior papers. This is acceptable for a tutorial, but the example menu is narrow; including at least one independently implemented baseline would broaden the tutorial's utility. The central interpretive claim is interesting and publishable after the derivation errors are fixed and the optimality caveats are stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is the first tutorial focused on discriminative clustering as a tradition, and the historical line it draws — from Bridle's 'fair but firm' MI objective through RIM, deep clustering, and into contrastive learning — is genuinely useful. The strongest part is Section 5.3.2: once you fix an augmentation strategy, I(x; Aug(x)) is a constant, so contrastive training is better understood as fitting an energy-based variational model q(x|aug(x)) to p(x|aug(x)) than as maximizing a learned MI. That reframing is the real contribution, and it is worth discussing even if the formal claim only holds exactly at the optimal critic. The worked example with GemClus and the honest note that the augmentations were rigged to match known clusters are both good practice; the code makes the tutorial reproducible. I also appreciate the explicit discussion of model selection, a topic most deep-clustering surveys skip. Now the soft spots. Eq. (8) defines the sigmoid as 1/(1 - exp(-a)), which is not the logistic sigmoid and is undefined at a = 0; it should be 1/(1 + exp(-a)). Eq. (43) writes I(x;y) = H(y|x) - H(y), with the sign flipped relative to the surrounding text and to the standard identity I = H(y) - H(y|x). For a tutorial meant to be a reference, these are not cosmetic; both need to be corrected before publication. A third, softer issue is the tightness claim in Sections 5.3.2 and 8. The paper says the 'true underlying model' is the energy-based q(x|aug(x)). Strictly, optimizing the InfoNCE lower bound minimizes the KL between p(x|z) and q_theta only when the critic family is rich enough to reach E* = log p(z|x). With a finite neural critic, you get a variational approximation, not the posterior itself. The paper says 'approximate' earlier, but the Section 8 phrasing goes beyond what is shown. That caveat should be stated explicitly. The self-citation is heavy but mostly legitimate; the authors' own GEMINI and GemClus work is what the tutorial actually demonstrates, so it earns its place. Who should read this? Students and researchers coming into deep clustering who want the intellectual history and a clean conceptual frame. It will not change practice, but it will improve how people think about contrastive clustering. I would bring it to a reading group and cite it in a survey. Ask the authors for a revision fixing Eqs. (8) and (43) and qualifying the tightness claim; then it deserves to be published as a reference tutorial.","headline":"A useful historical tutorial on discriminative clustering with a genuinely clarifying reinterpretation of contrastive learning, currently held back by two easily fixable algebraic errors and an overstrong optimality claim.","tokens_in":762,"tokens_out":825,"would_cite":true,"duration_ms":48906,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Contrastive clustering does not maximize a learned mutual information: with the augmentation distribution fixed, $I(x;\\mathrm{Aug}(x))$ is constant, and the object being trained is an energy-based variational model of…","keywords":["discriminative clustering","mutual information","contrastive learning","energy-based variational models","InfoNCE","deep clustering","model selection","data augmentation"],"falsifier":"Train a contrastive model with a fixed augmentation on data whose true conditional $p(x|\\mathrm{Aug}(x))$ is known analytically, then estimate the normalized $q_\\theta(x|\\mathrm{Aug}(x))$ implied by the trained critic through Eq. (53) and compare it with the true conditional; a systematic mismatch at large critic capacity would break the claim that contrastive learning is density estimation of $p(x|\\mathrm{Aug}(x))$.","tokens_in":31750,"feed_emoji":"🔁","tokens_out":13010,"duration_ms":115844,"temperature":0.7,"pith_summary":"This tutorial traces the evolution of discriminative clustering from linear decision boundaries to invariance critics and argues that mutual information has been the thread connecting each stage. Its central claim is that modern contrastive clustering no longer maximizes a meaningful mutual information: once the augmentation distribution is fixed, $I(x;\\mathrm{Aug}(x))$ is a constant, so the InfoNCE-style losses used in practice are not tightening a lower bound on a learned mutual information. Instead, the objective implicitly trains an energy-based variational distribution $q_\\theta(x|\\mathrm{Aug}(x)) \\propto p_{\\mathrm{data}}(x) e^{E_\\theta(x,\\mathrm{Aug}(x))}$, an approximation to the true conditional $p(x|\\mathrm{Aug}(x))$. The paper also argues that the successes of deep clustering should be credited to augmentation design and regularizations rather than the mutual information objective itself, and that model selection metrics must be chosen to match the clustering hypotheses encoded in the model. A reader should care because, if this interpretation is right, common descriptions of contrastive clustering as mutual information maximization are misleading and should be replaced by density-estimation language.","feed_headline":"Contrastive clustering optimizes an energy model, not mutual information","feed_subtitle":"With augmentations fixed, I(x;Aug(x)) is constant; training learns q(x|Aug(x)) from an energy-based family.","key_machinery":"The load-bearing object is the energy-based variational conditional $q(x|z) = p_{\\mathrm{data}}(x) e^{E(x,z)}/Z(z)$, with $Z(z) = \\mathbb{E}_{p_{\\mathrm{data}}(x)}[e^{E(x,z)}]$, inserted into the standard variational lower bound on mutual information. It turns the intractable conditional $p(x|z)$ into a tractable approximation whose normalizer is estimated by Monte Carlo; the resulting InfoNCE bound is the objective that contrastive losses implement. The second key move is observing that $z = \\mathrm{Aug}(x)$ with a fixed augmentation distribution makes $I(x;\\mathrm{Aug}(x))$ constant, leaving the critic $E_\\theta$ as the only trainable quantity, so the model's actual task is density estimation of $p(x|\\mathrm{Aug}(x))$.","core_discovery":"The paper's central discovery is an identity about what contrastive clustering actually optimizes. Using the energy-based variational family $q(x|z) = p_{\\mathrm{data}}(x) e^{E(x,z)}/Z(z)$ with $Z(z) = \\mathbb{E}_{p_{\\mathrm{data}}}[e^{E(x,z)}]$, the InfoNCE lower bound becomes a Monte Carlo estimate of $\\log(e^{E}/\\sum e^{E})$ plus $\\log n$, and the entropy term cancels. The authors observe that in contrastive settings $z = \\mathrm{Aug}(x)$, so the mutual information $I(x;\\mathrm{Aug}(x))$ depends only on the fixed augmentation distribution and is constant with respect to the model parameters. Consequently, maximizing the lower bound cannot be maximizing mutual information; it drives $q(x|\\mathrm{Aug}(x))$ toward $p(x|\\mathrm{Aug}(x))$. The paper states this as 'contrastive learning is about learning the distribution $p(x|\\mathrm{aug}(x))$ using the approximate $q(x|\\mathrm{aug}(x))$,' and it draws the historical line from early mutual-information clustering through regularized information maximization, the information bottleneck, deep clustering, and contrastive methods.","pith_inferences":["A testable extension the authors leave implicit: if the trained object really is an energy-based $q(x|\\mathrm{Aug}(x))$, then sampling from this model (e.g. by Markov-chain or score-based methods) should produce plausible data conditioned on the augmentation; this would separate the density-estimation interpretation from a purely discriminative one.","The same constant-mutual-information argument applies to self-supervised contrastive learning generally, not just clustering: any temperature-scaled contrastive loss with fixed view generation is better analyzed as learning a conditional view distribution than as maximizing information between views.","The reinterpretation suggests a concrete design direction: replace the Monte Carlo estimate of the normalizer $Z(z)$ with a learned or analytic partition function, potentially improving contrastive learning beyond what InfoNCE-style normalization allows."],"forward_implications":["Contrastive clustering should be described as density estimation of $p(x|\\mathrm{Aug}(x))$ via an energy-based model, not as mutual information maximization; claims of maximizing $I(x;\\mathrm{Aug}(x))$ refer to a constant.","The clustering hypothesis in modern methods lives in the choice of augmentation distribution and of the critic family, so those choices carry the inductive bias that decision boundaries carried in earlier methods.","Performance differences between deep clustering methods are more plausibly attributed to architecture, regularizations, and augmentation design than to the mutual information objective itself.","Model selection for contrastive clustering should use internal scores based on neighbourhood and invariance structure, rather than generic distance-based metrics such as silhouette or Davies-Bouldin, which encode a different clustering hypothesis."],"supporting_citations":[{"why":"Derives the variational lower bound with a proposal $q(x|z)$ that the paper uses to rewrite mutual information before substituting the energy-based family.","marker":"Barber and Agakov, 2003"},{"why":"Introduces the energy-based variational family and the InfoNCE bound that the paper identifies as the actual contrastive objective.","marker":"Poole et al., 2019"},{"why":"Defines InfoNCE and contrastive predictive coding, the estimator whose form the paper analyzes.","marker":"Van den Oord et al., 2018"},{"why":"Provides the temperature-scaled cross-entropy contrastive loss that Eq. (59) links to practical contrastive learning.","marker":"Chen et al., 2020"},{"why":"Proposes a contrastive clustering model with two critics summing representation and clustering mutual information, the concrete instance in Eq. (66).","marker":"Do et al., 2021"},{"why":"Extends contrastive clustering to representation-level and cluster-level mutual information, a principal modern deep clustering method discussed in Section 5.3.2.","marker":"Li et al., 2021"},{"why":"Maximizes mutual information between cluster assignment distributions before and after augmentation, providing the cluster-distribution critic notation in Section 5.3.2.","marker":"Huang et al., 2020"},{"why":"Proves a formal limitation on measuring mutual information from finite samples, supporting the paper's conclusion that contrastive objectives are not effectively maximizing MI.","marker":"McAllester and Stratos, 2020"},{"why":"Reports that mutual information values are hardly predictive of downstream task performance, supporting the paper's dissonance argument.","marker":"Tschannen et al., 2020"}],"fun_headline_variants":["Contrastive clustering optimizes an energy model, not MI","Under fixed augmentations, MI is constant: contrastive learning targets q(x|aug)","InfoNCE lower bound: it's really about estimating p(x|aug(x)), not MI","Contrastive clustering: the MI term is constant, so it's not MI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the network used to score similarities is flexible enough to represent the true conditional distribution of an original sample given its augmented version; the paper offers no capacity or tightness analysis, so the identification holds exactly only for an optimal critic.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive clustering optimizes an energy model, not MI","Under fixed augmentations, MI is constant: contrastive learning targets q(x|aug)","InfoNCE lower bound: it's really about estimating p(x|aug(x)), not MI","Contrastive clustering: the MI term is constant, so it's not MI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001008,"raw_usage":{"total_tokens":4296,"prompt_tokens":1016,"completion_tokens":3280,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":3202}},"tokens_in":632,"tokens_out":3280,"duration_ms":25711,"temperature":1.0,"reasoning_tokens":3202,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:27:55.610486+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a contrastive model with a fixed augmentation on data whose true conditional $p(x|\\mathrm{Aug}(x))$ is known analytically, then estimate the normalized $q_\\theta(x|\\mathrm{Aug}(x))$ implied by the trained critic through Eq. (53) and compare it with the true conditional; a systematic mismatch at large critic capacity would break the claim that contrastive learning is density estimation of $p(x|\\mathrm{Aug}(x))$.","supporting_citations":[{"cited_title":"On Variational Bounds of Mutual Information","cited_arxiv_id":null,"evidence_quote":"Introduces the energy-based variational family and the InfoNCE bound that the paper identifies as the actual contrastive objective."},{"cited_title":"Representation Learning with Contrastive Predictive Coding","cited_arxiv_id":null,"evidence_quote":"Defines InfoNCE and contrastive predictive coding, the estimator whose form the paper analyzes."},{"cited_title":"Deep Semantic Clustering by Partition Confidence Maximisation","cited_arxiv_id":null,"evidence_quote":"Maximizes mutual information between cluster assignment distributions before and after augmentation, providing the cluster-distribution critic notation in Section 5.3.2."},{"cited_title":"Formal Limitations on the Measurement of Mutual Information","cited_arxiv_id":null,"evidence_quote":"Proves a formal limitation on measuring mutual information from finite samples, supporting the paper's conclusion that contrastive objectives are not effectively maximizing MI."},{"cited_title":"Rubenstein, Sylvain Gelly, and Mario Lucic","cited_arxiv_id":null,"evidence_quote":"Reports that mutual information values are hardly predictive of downstream task performance, supporting the paper's dissonance argument."}],"review_version":1}