{"id":"143c7e91-d52d-440c-829c-256067318ca3","arxiv_id":"2507.13779","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SuperCM, a clustering-based regularizer with class-conditional centroids, improves SSL and UDA accuracy, especially with very few labels.","lead":"This paper adds a differentiable clustering module, called SuperCM, to semi-supervised and domain-adaptation models, using labeled data to estimate cluster centers. It reports accuracy gains in low-label image classification and in domain adaptation benchmarks such as Office-31.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) decays unobserved class centroids toward zero, so the label-driven clustering mechanism may not be as claimed; a code-level check is needed.","rationale":"The paper's central claim is that a differentiable clustering regularizer with class-conditional centroids improves SSL and UDA, especially with little supervision. The reader's weakest assumption concerns the reliability of those centroids when each class has only a handful of labels. My stress test targets the same component, Eq. (3), but identifies a more concrete failure mode: the update formula as written is not a moving average when classes are missing from a mini-batch, because absent classes are still multiplied by (τ−1)/τ and therefore decay toward zero. This is load-bearing because the centroids are the only label-driven part of the CM loss; if they collapse, the reported improvements could arise from the reconstruction term or from accidental regularization rather than from the proposed mechanism. I do not claim the results are fabricated or that the method is useless; the code is available and the issue is checkable. Other concerns—the inconsistent two-cluster derivation in Eq. (2), the overstatement of 'consistently outperforms,' and the use of older baselines—are real but secondary: Eq. (2) is not used in the final loss, and the headline 'up to 15 points' remains true even if some cells regress. The centroid-update ambiguity, by contrast, affects every table and the interpretation of every improvement. I therefore retain the reader's CONDITIONAL verdict: the paper needs a code-level clarification or correction, but the empirical claim is plausible and can be verified. My substantive concern reinforces the conditional status rather than moving it to accept or reject.","tokens_in":18087,"tokens_out":10752,"duration_ms":127442,"concrete_test":"Run the released repository with a debug hook that logs, for each class k, ||μ_k|| and whether class k appears in the labeled batch, over the headline settings (MNIST-100 labels, SVHN-250 labels, Office-31 A→W). If ||μ_k|| decreases by the factor (τ−1)/τ on absent iterations, Eq. (3) is implemented as written. Then replace Eq. (3) with a count-based update (maintain per-class running sums S_k and counts C_k, set μ_k = S_k/C_k, updating only when the class is present) and re-run the same three settings. Compare the resulting accuracies to Tables 2 and 3 and to the 'learned vs GS' ablation in §9.2; if the corrected update changes the gains or the ablation, the central mechanism depends on a mis-specified centroid update.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's central novelty is that class-conditional centroids from Eq. (3) inject label information into the CM loss. As written, however, Eq. (3) is not a valid moving average when a class is absent from the current labeled mini-batch. At each global iteration τ, the update applies the multiplicative factor (τ−1)/τ to every class, but the data term 1/τ * (class mean) is zero for absent classes. Thus μ_k is shrunk on every iteration in which class k does not appear. Over many iterations these missing updates telescope (the product of (τ−1)/τ over an interval starting at τ0 is τ0/T), so the norms of centroids for classes that are not present in most batches collapse toward zero. This is precisely the low-supervision and many-class regime where the paper claims its largest gains: with 100 MNIST labels there are ~10 examples per class, and in UDA source batches with 31 or 65 classes, most classes are missing from any given batch. If the released code implements Eq. (3) literally, the E2/E3 clustering terms vanish for those classes and the CM loss degenerates to the reconstruction term, so the reported improvements would not be evidence for the 'class-conditional centroid' mechanism claimed in the paper. The text does not state that τ is a per-class update counter or that updates are skipped for absent classes, so this is currently an unresolved, load-bearing ambiguity.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SuperCM, a regularization method for semi-supervised learning (SSL) and unsupervised domain adaptation (UDA) that appends a differentiable clustering module (CM) to a feature extractor. The module's centroids are updated as class-wise moving averages of labeled features rather than learned by gradient descent, and the CM loss is combined with a standard cross-entropy loss and optionally with an existing SSL/UDA objective. The method is evaluated as a standalone SSL objective and as a regularizer for five SSL baselines and for DANN, on MNIST, SVHN, CIFAR-10/100, STL-10, Office-31, Office-Home, Office-Caltech, and ImageClef. The authors report gains of up to 15 percentage points in SSL accuracy and up to 10 percentage points in UDA performance, particularly in low-supervision settings, and provide an ablation of centroid estimation strategies.","tokens_in":18377,"tokens_out":8452,"duration_ms":93802,"significance":"If the proposed mechanism is sound, the paper offers a simple, plug-in clustering regularizer with substantial empirical gains in low-label and domain-shift regimes, supported by extensive benchmarks and released code. The main conceptual contribution is the injection of label information into a differentiable clustering objective through class-conditional centroids, which is a plausible and potentially useful idea. However, the load-bearing derivations and the centroid update rule contain unresolved technical issues that must be addressed before the reported gains can be confidently attributed to the claimed mechanism.","major_comments":[{"comment":"The claimed simplification of E2+E3 is algebraically incorrect. For two clusters, writing γ_i1=γ and γ_i2=1−γ, Eq. (1) gives E2 = −γ(1−γ)(‖μ1‖²+‖μ2‖²) and E3 = γ²‖μ1‖² + (1−γ)²‖μ2‖² + 2γ(1−γ)μ1ᵀμ2, whose sum is not γ(1−γ)‖μ1−μ2‖². A concrete counterexample is γ=0.5 with orthogonal unit centroids, for which Eq. (1) yields 0 while Eq. (2) yields 0.5. Because Eq. (2) is the paper's only stated justification for the 'sparsity and cluster merging' effect of E2+E3, the explanation of the clustering mechanism is unsupported as printed and should be corrected or replaced with the actual expression.","section":"§2.3.1, Eq. (2)"},{"comment":"The centroid update rule is either underspecified or incompatible with the claimed mechanism. If τ is the global iteration counter, then for a class absent from the labeled mini-batch the data term is zero and the centroid is multiplied by (τ−1)/τ on every such iteration, so the norms of rarely seen classes collapse toward zero over training. The paper does not state that τ is a per-class counter or that the update is skipped when class k has no labeled sample in the batch. Additionally, the data term is normalized by the total labeled batch size n_B^(l) rather than by the number of class-k samples, which further dilutes updates for rare classes. In the low-supervision and many-class settings where the largest gains are claimed (100 MNIST labels, Office-31/Office-Home with 31/65 classes), most classes are absent from most batches, so the literal Eq. (3) would destroy the label-injected centroids and the E2/E3 terms would not receive the class-conditional signal that is the paper's novelty. The authors must specify the update precisely and, ideally, verify against the released code; otherwise the reported results cannot be attributed to the described mechanism.","section":"§3.2, Eq. (3)"}],"minor_comments":[{"comment":"The sentence 'This work was supported by by the Research Council of Norway' contains a duplicated 'by' that should be removed.","section":"Footnote on page 2"},{"comment":"The column headings '10δ' and '10β' are ambiguous: it is unclear whether the entries are δ and β themselves or ten times those values, and if a scaling is intended it should be stated explicitly in the caption or text.","section":"Table 1"},{"comment":"The baseline column label 'DANN [?]' contains an unresolved citation placeholder that should be replaced with the proper reference.","section":"Table 4"},{"comment":"The symbol γ is used both for the ramp-up steepness in Eq. (5) and for the soft assignment responsibilities γ_ik in Eq. (1); these should be distinguished to avoid confusion.","section":"§7.3.3, Eq. (5)"},{"comment":"The text states that experiments use '400 and 4000 labeled data' for STL-10, but Table 2 and the surrounding discussion report '600 labels' and '4000 labels'; please correct the inconsistency.","section":"§5, STL-10 paragraph"}],"recommendation":"major_revision","confidential_remarks":"The central claim of the paper hinges on the correctness of Eq. (3). I strongly recommend that the editor request the authors to clarify the update rule and, if possible, check the released code against the printed formula. The ambiguity is not a minor presentational issue: if the literal formula is implemented, the reported gains would be difficult to explain through the claimed label-driven clustering mechanism. The reuse of the CM module from the authors' earlier paper [7] and the ICASSP version [8] is disclosed, and the UDA extension is new, but the novelty relative to [8] is incremental."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read on arXiv:2507.13779 (SuperCM). The paper extends the authors' earlier clustering module to SSL and UDA by computing class-conditional centroids from labeled data via a moving average, then using the CM loss as a regularizer. The headline results are real in the tables: e.g., MNIST with 100 labels jumps from 82.3 to 97.5, and DANN on Office-31 A to W from 80.8 to 91.0. Those are the kind of gains people in label-scarce domains would care about, and the code is available.\n\nWhat's genuinely new here is the UDA application and the broader SSL benchmark coverage (CIFAR-100, STL-10, etc.). The implementation is simple, the overhead is negligible, and the ablations on centroid strategies and hyperparameter sensitivity are useful. The t-SNE and Proxy-A distance analysis give some supporting evidence for the alignment story.\n\nNow the soft spots.\n\nFirst, Eq. (2) claims a simplification of E2+E3 that doesn't follow from Eq. (1) with the printed signs. I worked through the two-cluster case; the expression in Eq. (2) is not what you get from the loss as written. This is probably a typo or an unstated rearrangement, but it makes the derivation hard to trust.\n\nSecond, and more seriously, Eq. (3) as written is ambiguous. The update uses a global iteration counter tau, and for classes absent from the current labeled batch the data term is zero, so the centroid is multiplied by (tau-1)/tau and shrinks. Over hundreds of thousands of iterations, classes that appear rarely get their norms pulled toward zero. In the low-supervision and many-class settings where the paper claims its largest gains, absent classes are common. If the code implements Eq. (3) literally, the clustering terms for those classes vanish and the mechanism is not what the text claims. If the implementation instead uses per-class update counters or skips absent classes, the equation needs to say so. This is easily checked against the released code, but it's currently a load-bearing ambiguity.\n\nThird, the text overclaims. \"Consistently outperforms\" is contradicted by some of its own tables: Pseudo-Label on SVHN 250 drops from 81.2 to 79.9, and on STL-10 600 from 61.7 to 55.2. Several improvements are within standard deviation, and the significance tests are only mentioned for bold numbers, not systematically reported.\n\nThe baselines are also dated (mostly pre-2020), but that's somewhat acceptable for a regularizer study.\n\nOverall: the core idea is plausible, the empirical evidence is substantial, and the code availability is a plus. The paper deserves a serious referee, but the referee should ask for a code check of the centroid update, a corrected Eq. (2), and a more careful wording of the claims. If the centroid update is indeed as ambiguous as it looks, the method's mechanism needs to be re-stated before the results can be taken at face value. I'd still cite it as an example of differentiable clustering regularization, but not before the ambiguity is resolved.","headline":"A useful plug-in clustering regularizer with real low-supervision gains, but the centroid update equation needs a careful second look before the mechanism claims can be trusted.","tokens_in":18958,"tokens_out":5642,"would_cite":true,"duration_ms":61164,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SuperCM shows that a differentiable clustering loss with class-conditional centroids estimated from labeled data raises semi-supervised accuracy by up to 15 percentage points and DANN domain-adaptation accuracy by up to 10 percentage…","keywords":["semi-supervised learning","unsupervised domain adaptation","differentiable clustering","Gaussian mixture model","clustering regularizer","class-conditional centroids","DANN","low-supervision regimes"],"falsifier":"Train SuperCM on a benchmark where classes are deliberately multimodal, such as a modified CIFAR-10 where each object class is split into two visually distinct subgroups, using 100 to 250 labels per dataset, and compare against the plain cross-entropy baseline; if SuperCM fails to beat or matches that baseline, the single-cluster-per-class assumption is falsified.","tokens_in":17874,"feed_emoji":"📈","tokens_out":7706,"duration_ms":73729,"temperature":0.7,"pith_summary":"SuperCM is a training strategy that makes the clustering assumption explicit instead of relying on implicit consistency or entropy losses. It attaches a differentiable clustering module to a classifier's feature extractor and computes the module's cluster centers as moving averages of labeled examples per class. The paper claims this raises SSL accuracy by up to 15 percentage points and DANN's UDA accuracy by up to 10 points, with the largest gains in low-supervision regimes. The same plug-in term can regularize several established SSL methods and DANN-based UDA variants.","feed_headline":"Clustering regularizer lifts SSL by 15 points, UDA by 10","feed_subtitle":"SuperCM adds class-guided clustering to existing models; biggest gains appear when labels are scarce.","key_machinery":"The central object is the Clustering Module (CM), a one-layer auto-encoder that outputs soft cluster assignments and is trained with a GMM-derived loss containing reconstruction, sparsity/cluster-merging, and Dirichlet-prior terms. SuperCM sets K equal to the number of classes, computes centroids as exponentially moving averages of labeled-source features according to Eq. (3), and sets the Dirichlet coefficient alpha = 1 to disable the prior. The total training loss is cross-entropy on labeled class responsibilities plus beta times the CM loss on labeled and unlabeled data plus delta times the base SSL/UDA loss, so the clustering term acts as a differentiable regularizer in an end-to-end pipeline.","core_discovery":"The paper's central claim is that a GMM-inspired, end-to-end differentiable clustering loss becomes a broadly useful regularizer once its centroids are anchored to supervised information: per-class moving averages of labeled features replace freely learned cluster centers, preventing collapsed or misaligned clusters. On MNIST with 100 labels, SuperCM alone goes from 82.28% to 97.45% accuracy over the cross-entropy baseline; on Office-31 A to W it raises DANN from 80.80% to 91.03%. Similar improvements appear across CIFAR-10, SVHN, CIFAR-100, STL-10 and four UDA benchmarks, and the learned features become visibly more compact and separated.","pith_inferences":["The single-centroid-per-class assumption suggests the method is most suited to balanced, unimodal class structure; on long-tail or intra-class-multimodal data, the same recipe may need multiple centroids per class.","Because the centroids are moving averages over labeled examples, their reliability should depend on feature quality early in training, so warm-starting with self-supervised pretraining might amplify the gains.","An ablation separating the reconstruction term from the pure clustering terms would test whether part of the benefit comes from representation smoothness rather than clustering per se.","If the mechanism transfers beyond images, the same labeled-centroid clustering regularizer could be applied to medical or textual data with few annotations, where class structure is less clean."],"forward_implications":["In low-label SSL settings, e.g. MNIST with 100 labels, SuperCM brings standalone accuracy from 82.28% to 97.45%.","DANN's accuracy improves by up to 10 points on Office-31, and average accuracy increases on Office-31, Office-Home, Office-Caltech, and ImageClef.","The regularizer plugs into five existing SSL methods (Pi-model, Mean Teacher, VAT, Pseudo-labels, ICT) and improves most of their tested configurations.","Combining SuperCM with MCC or BNM regularizers further improves several Office-31 and Office-Home transfers.","Training with SuperCM reduces Proxy-A distance between source and target features, indicating better domain alignment."],"supporting_citations":[{"why":"Defines the Clustering Module and its GMM-based loss that SuperCM extends.","marker":"[7]"},{"why":"Supplies the DANN baseline and adversarial domain-alignment training that SuperCM regularizes.","marker":"[30]"},{"why":"Provides the SSL data split, augmentation, and training protocol for the SSL experiments.","marker":"[37]"},{"why":"Supplies the VAT baseline used as a base SSL model in Table 2.","marker":"[12]"},{"why":"Supplies the Pi-model baseline used as a base SSL model in Table 2.","marker":"[10]"},{"why":"Supplies the Mean Teacher baseline used as a base SSL model in Table 2.","marker":"[11]"},{"why":"Supplies the Pseudo-label baseline used as a base SSL model in Table 2.","marker":"[14]"},{"why":"Supplies the ICT baseline used in CIFAR-100 SSL experiments.","marker":"[20]"},{"why":"Provides MCC, one UDA regularizer combined with SuperCM in complementary experiments.","marker":"[28]"},{"why":"Provides BNM, the other UDA regularizer combined with SuperCM in complementary experiments.","marker":"[29]"}],"fun_headline_variants":["SuperCM's labeled centroids push MNIST SSL from 82% to 97% with 100 labels","Class-anchored clusters: +15 pts SSL, +10 pts UDA from SuperCM","SuperCM anchors clusters to labels, lifting DANN from 81% to 91% on A→W","Label-anchored clustering: a universal boost for SSL and UDA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each class is well described by a single compact cluster in the learned feature space, and that the centroids estimated from the few labeled examples are reliable enough to anchor that cluster.","fun_headline_variants_meta":{"raw":{"variants":["SuperCM's labeled centroids push MNIST SSL from 82% to 97% with 100 labels","Class-anchored clusters: +15 pts SSL, +10 pts UDA from SuperCM","SuperCM anchors clusters to labels, lifting DANN from 81% to 91% on A→W","Label-anchored clustering: a universal boost for SSL and UDA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001717,"raw_usage":{"total_tokens":6731,"prompt_tokens":822,"completion_tokens":5909,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":438,"completion_tokens_details":{"reasoning_tokens":5809}},"tokens_in":438,"tokens_out":5909,"duration_ms":51219,"temperature":1.0,"reasoning_tokens":5809,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:17:00.741048+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SuperCM on a benchmark where classes are deliberately multimodal, such as a modified CIFAR-10 where each object class is split into two visually distinct subgroups, using 100 to 250 labels per dataset, and compare against the plain cross-entropy baseline; if SuperCM fails to beat or matches that baseline, the single-cluster-per-class assumption is falsified.","supporting_citations":[{"cited_title":"Boubekki, M","cited_arxiv_id":null,"evidence_quote":"Defines the Clustering Module and its GMM-based loss that SuperCM extends."},{"cited_title":"Ganin, E","cited_arxiv_id":null,"evidence_quote":"Supplies the DANN baseline and adversarial domain-alignment training that SuperCM regularizes."},{"cited_title":"Oliver, A","cited_arxiv_id":null,"evidence_quote":"Provides the SSL data split, augmentation, and training protocol for the SSL experiments."},{"cited_title":"Miyato, S","cited_arxiv_id":null,"evidence_quote":"Supplies the VAT baseline used as a base SSL model in Table 2."},{"cited_title":"Laine, T","cited_arxiv_id":null,"evidence_quote":"Supplies the Pi-model baseline used as a base SSL model in Table 2."},{"cited_title":"Tarvainen, H","cited_arxiv_id":null,"evidence_quote":"Supplies the Mean Teacher baseline used as a base SSL model in Table 2."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Pseudo-label baseline used as a base SSL model in Table 2."},{"cited_title":"Verma, A","cited_arxiv_id":null,"evidence_quote":"Supplies the ICT baseline used in CIFAR-100 SSL experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides MCC, one UDA regularizer combined with SuperCM in complementary experiments."}],"review_version":1}