{"id":"5f8c8427-51e9-417a-a0cd-b232b24dbcd5","arxiv_id":"2412.12501","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SDC reuses the biased outputs of a pre-trained model to adjust logits and generate better pseudo-labels, improving novel category discovery in text classification.","lead":"This paper introduces a method called Self-Debiasing Calibration (SDC) for discovering new categories in text data. Instead of treating a model's bias toward known categories as a problem, SDC uses that bias to create better labels for unknown categories, and the authors report improved accuracy on three intent-detection benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Entropy gate in Eq. 8 suppresses the very low-entropy novel samples that the transfer mechanism is designed to exploit, so the claimed bias-leverage mechanism is not established.","rationale":"The paper's stated novelty is that the frozen biased model's confident predictions can transfer knowledge from known to novel categories via Eqs. 4-5. That claim only operates when the biased model commits to a known category; a high-entropy prediction carries little category-specific transfer information. But Eq. 8 scales the transfer by sigmoid(E_i - E_max), which is near zero exactly for confident, low-entropy predictions. So for the motivating 'dog→cat' example, the transfer weight is smallest. The reader's weakest assumption identifies the same failure mode; I agree and sharpen it by connecting it to the paper's own motivating example. This is an internal consistency problem, not a dispute with consensus. The empirical headline could still survive: the method may simply improve pseudo-labels for already-uncertain novel samples, and the CBM/CCM terms might still help via the Sinkhorn balancing and other components. Thus a CONDITIONAL verdict remains appropriate, pending the instrumented check. The separate issue of per-dataset β selection is real but secondary; it would matter for the quantitative claim even if the mechanism check passes.","tokens_in":12787,"tokens_out":8338,"duration_ms":81901,"concrete_test":"Instrument a training run on HWU64: for each novel unlabeled sample, record E_i from the frozen biased logits and whether its pseudo-label is corrected after adding the CCM term (Eq. 5). Then run a variant where α_i in Eq. 8 is replaced by a floor, e.g., max(α_i, β/4), for low-entropy samples. If the correction rate among low-E_i novel samples is near zero in the original run but rises under the floor, the entropy gate is suppressing the transfer mechanism the paper motivates. If the correction rate is already substantial at low E_i, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. 8 defines α_i = β·sigmoid(E_i − E_max), so α_i is near zero for any sample whose biased logits have low entropy. The motivating example for CCM is precisely a novel 'cat' that the biased model confidently predicts as known 'dog'; such a sample has low E_i, so α_i ≈ 0 and the transfer term α_i · T^T L_bias in Eq. 5 is suppressed. Thus the method's knowledge-transfer mechanism is weakest when the biased model has the most category-specific signal to transfer. The batch-relative normalization compounds this: if a minibatch contains only known samples, even a known sample near the batch entropy maximum receives a non-negligible α. The ablation 'w/o CCM' (Table 2) shows CCM contributes roughly 1.4 H-score points on HWU64, but it does not show whether that gain comes from high-entropy novel samples or from confident misclassified ones. Unless Eq. 8 is replaced by a per-sample weighting that does not collapse for confident errors, the paper has not shown that bias-based transfer helps Novel–Known errors; it may only relabel already-uncertain novel samples. This is an internal tension between the stated motivation and the implemented gate, not merely a tuning issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Self-Debiasing Calibration (SDC), a method for Generalized Category Discovery (GCD) on text. The core idea is to use a frozen pre-trained model biased toward known categories to adjust the logits of a trainable model: category bias mitigation subtracts the biased logits from known-category logits (Eq. 3), and category confusion mitigation adds a transfer-matrix-weighted version of the biased logits to novel-category logits (Eq. 5). An entropy-based weight (Eq. 8) modulates these adjustments per sample, and pseudo-labels are produced from the calibrated logits via Sinkhorn-Knopp optimal transport. The model is trained with a supervised loss on labeled data, a pseudo-label loss on unlabeled data, and a contrastive loss. Experiments on BANKING, HWU64, and CLINC report state-of-the-art results, with particular gains in novel-category accuracy.","tokens_in":13021,"tokens_out":5807,"duration_ms":51089,"significance":"If the reported findings hold, the paper offers a conceptually interesting reframing: instead of merely suppressing model bias toward known categories, the biased model's outputs can be reused as a source of knowledge for novel categories. The method is clearly specified and the open-source code is a reproducible artifact. However, the empirical evidence is narrow (three text intent datasets only), and the absence of standard deviations, significance tests, and hyperparameter sensitivity analysis substantially tempers the strength of the claims. The central mechanism also has a potential internal tension between the motivating example and the entropy gate (Eq. 8), which the authors would need to resolve for the conceptual claim to be fully convincing.","major_comments":[{"comment":"The entropy-based weighting in Eq. (8) undermines the stated mechanism of Category Confusion Mitigation (CCM). The motivating example in the introduction is a novel 'cat' confidently misclassified as known 'dog'; such a sample has low entropy on the biased model, so alpha_i approaches zero and the transfer term alpha_i * T^T L_bias in Eq. (5) is suppressed. The batch-relative sigmoid further weakens the transfer for any sample whose entropy is below the batch maximum. The ablation in Table 2 shows that removing CCM lowers H-score by 1.4 points on HWU64, but it does not tell us whether that gain comes from confident misclassified novel samples or only from already-uncertain novel samples. Unless the authors provide evidence that CCM helps specifically for low-entropy novel samples (e.g., by binning pseudo-label accuracy by entropy of the biased model), the claim that bias transfer resolves Novel-Known errors is not established. This is an internal tension between the stated motivation and the implemented gate, not a tuning issue.","section":"Eq. (8) and Category Confusion Mitigation"},{"comment":"All numbers in Table 1 are averages over 3 runs without standard deviations, confidence intervals, or significance tests. The headline improvements (2.64% average on novel accuracy, 2.00% on H-score) are modest relative to typical run-to-run variability in deep clustering methods on these benchmarks. Without a measure of dispersion, the reader cannot judge whether the reported superiority over KTN, TAN, and DPN is systematic or within noise. I request per-run results or standard deviations for the main table and the ablation table, and, where feasible, a paired test over runs.","section":"Table 1 and Experimental Setup"},{"comment":"The key hyperparameter beta is set to 0.03, 0.05, and 0.42 for BANKING, HWU64, and CLINC, respectively, which is a large variation and suggests strong per-dataset tuning. No sensitivity analysis is provided, so it is unclear whether the reported gains persist across reasonable choices of beta. Since beta scales the entire logit adjustment in Eqs. (3), (5), and (8), the authors should provide a plot or table of H-score and known/novel accuracy versus beta on at least one dataset, and ideally a principled selection rule. As it stands, the method may be over-fit to these three benchmarks.","section":"Implementation Details"},{"comment":"The evaluation is limited to three text intent detection datasets (BANKING, HWU64, CLINC). Generalized Category Discovery is commonly evaluated on image benchmarks such as CIFAR-100, ImageNet-100, and the Semantic Shift Benchmark; by restricting the experiments to text, the paper does not demonstrate that the proposed mechanism generalizes beyond this narrow setting. Given that the abstract frames the method in terms of general 'category bias' without domain-specific assumptions, this is a significant gap. At minimum, the authors should either add one image-domain GCD experiment (even a representative one) or explicitly state and justify the text-only scope.","section":"Datasets (Section: Experiments)"}],"minor_comments":[{"comment":"The notation uses the hat symbol both for pseudo-labels and for the model's predictions (\\hat{Y}^l and \\hat{Y}^u), which is confusing. Please rename the pseudo-label variable (e.g., \\tilde{Y}) or use distinct notation.","section":"Model Training (Eq. 10)"},{"comment":"The paper states that testing is inductive, but Table 1 reports clustering accuracy, which typically relies on transductive clustering over the test set. The Real-world Applications section later says 'We report experimental results using clustering on the testing sets offline'. Please reconcile the terminology and clarify whether the main results come from a transductive clustering step or from the learned classifier.","section":"Evaluation Protocol"},{"comment":"The manuscript says that K estimation uses the dropout algorithm (Zhang et al., 2021a) but provides no description or algorithmic details. Since Table 2 reports results 'w/o Ground-truth K', the estimation method should be specified enough to be reproducible.","section":"K Estimation"},{"comment":"The GPT-3.5 and GPT-4 baselines are stated to be based on the turbo API, but the prompt template is not given. Including the exact prompt would make the zero-shot comparison reproducible.","section":"Comparison with GPT models"}],"recommendation":"major_revision","confidential_remarks":"The manuscript shows a high concentration of self-citations (the authors' own prior work TAN, KTN, DPN, and related), and the evaluation uses the same three text benchmarks as those prior papers. This is not a scientific flaw by itself, but it may be worth the editor's attention regarding how strongly the comparisons are positioned. The paper appears formatted as a conference submission (AAAI) rather than a journal article; the level of detail in the experiments and the scope of benchmarks is more typical of a conference paper. If the journal's scope is broad, the authors should be pushed to broaden the evaluation or narrow the claims accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"New here is the reframing: instead of treating the known-class bias of a pretrained model as noise to remove, SDC uses it as a signal. The two logit adjustments—subtract the biased logits from known-class logits, add prototype-similarity-weighted biased logits to novel-class logits—are close to the authors' earlier TAN and KTN work, but the combination with entropy-based instance weighting is new. The reported gains are consistent: about 2 points average H-score over the strongest prior method on BANKING, HWU64, and CLINC, with most of the gain on novel classes. The method is clearly specified, the ablation is informative, and code/data are promised. That part is solid.\n\nThe soft spot is the entropy gate, and it is not a tuning detail. Eq. 8 sets α_i = β·sigmoid(E_i − E_max). For a novel sample the biased model confidently calls 'dog' (the motivating cat example), E_i is low, so α_i is near zero and both CBM and CCM leave that sample's logits untouched. The method therefore does least adjusting on the exact cases the paper says bias should help. The ablation shows CCM contributes about 1.4 H-score on HWU64, but it does not show that the gain comes from confident misclassifications rather than from already-uncertain novel samples. That is an internal tension between the story and the math, not a fatal flaw—the method can still help the high-entropy novel majority—but it means the paper has not actually demonstrated the mechanism it claims.\n\nTwo other things stand out. Results are averages over three runs with no standard deviations or significance tests, and the critical β is tuned per dataset (0.03, 0.05, 0.42) with no sensitivity analysis or validation protocol. Given the small performance gaps, that matters. Also, the ablation is only on HWU64, and the paper does not say whether the transfer matrix T is recomputed during training. Those are fixable.\n\nBottom line: this is a plausible, well-specified heuristic that moves the SOTA on a narrow subfield by a small but consistent margin. It deserves a serious referee, but the referee should push on the entropy-gate tension and the missing error bars. I would not cite it yet; I would wait to see the revision.","headline":"Reasonable incremental GCD work with a real mechanism gap: the entropy gate suppresses the confident errors the transfer step is meant to exploit.","tokens_in":13611,"tokens_out":6180,"would_cite":false,"duration_ms":49368,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A frozen biased model can be reused, not just fought, to discover novel categories.","keywords":["Generalized Category Discovery","logit adjustment","model bias","pseudo-labeling","novel category discovery","entropy weighting","text intent detection","Sinkhorn-Knopp"],"falsifier":"Construct a probe set where a novel class is semantically near a known class, so the biased model confidently predicts the known class (low entropy); if SDC's gains on novel accuracy come from the entropy gate, such a probe should show smaller or negative gains compared to a dataset with well-separated novel classes.","tokens_in":12560,"feed_emoji":"🎯","tokens_out":5969,"duration_ms":46867,"temperature":0.7,"pith_summary":"Generalized Category Discovery asks a model to recognize both known and previously unseen categories from unlabeled data, but models pre-trained only on known categories are biased toward them. This paper argues that this bias is not merely an obstacle: a frozen biased model's predictions encode exactly how much the model over-favors each known class, and they also encode similarity hints that can separate novel classes from each other. The proposed Self-Debiasing Calibration (SDC) subtracts the biased logits from known-category logits and adds transferred logits to novel-category logits, with a per-sample entropy weight $\\alpha$ that decides how strongly to intervene. On three text benchmarks, the framework lifts novel-category accuracy by about 2.6 points on average over prior methods while keeping known-category accuracy roughly intact. If the paper is right, the standard practice of treating model bias as pure noise is leaving useful signal on the table.","feed_headline":"Reuse the biased model: +2.6% novel categories","feed_subtitle":"SDC subtracts and transfers the frozen model's logits per sample, lifting novel-class accuracy on three text benchmarks.","key_machinery":"The central mechanism is a pair of logit-adjustment identities built from the frozen biased model. Category Bias Mitigation computes $L_{\\mathrm{CBM}} = L_\\theta[:M] - \\alpha \\cdot L_{\\mathrm{bias}}$, subtracting the biased logits from the known-category slice of the current logits; Category Confusion Mitigation computes $L_{\\mathrm{CCM}} = L_\\theta[M:] + \\alpha \\cdot T^\\top \\times L_{\\mathrm{bias}}$, where $T$ is a prototype-similarity transfer matrix from known to novel categories. The two outputs are concatenated into $L_c$, and the entropy-weighted $\\alpha = \\beta \\cdot \\mathrm{Sigmoid}(E - E_{\\max})$ decides per-sample how much debiasing and transfer to apply. These identities turn the frozen model's predictions into a structured prior rather than a nuisance, and they are what the ablation study shows the framework depends on.","core_discovery":"The paper's central claim is that the output logits of a frozen model biased toward known categories can be repurposed as a calibration signal for the current training model. Two logit-level operations carry the argument: for known categories, subtracting the biased logits (scaled by $\\alpha$) from the current model's logits removes the over-confidence the biased prior injects; for novel categories, adding the transpose of a prototype-similarity transfer matrix times the biased logits pushes the current logits along the directions the biased model already associates with each novel cluster. The per-sample weight $\\alpha = \\beta \\cdot \\mathrm{Sigmoid}(E - E_{\\max})$ comes from the entropy of the biased softmax, which is low for known samples and high for novel samples. The calibrated logits feed the Sinkhorn-Knopp pseudo-labeling scheme, and the resulting pseudo-labels improve especially on novel categories. The paper reports that this yields an average improvement of 2.64% in novel-category accuracy over state-of-the-art baselines on BANKING, HWU64, and CLINC, while known-category accuracy improves by 0.28% on average.","pith_inferences":["The logit-adjustment recipe should transfer to vision GCD if the frozen model's class-similarity structure is replaced by a vision-language model's text-conditional logits; this is a direct testable extension.","The method's boundary case is a novel sample that the biased model confidently places in a known class - such samples get little $\\alpha$ and may not be recovered, so gains should shrink on datasets with high known-novel similarity.","One could close the loop by periodically fine-tuning the 'biased' model on the current pseudo-labels, letting the reference distribution co-evolve with the discovered novel classes rather than staying fixed."],"forward_implications":["Novel-category accuracy on text GCD benchmarks rises by about 2.6 points on average, and known-category accuracy is not sacrificed, so the debiasing and transfer steps are complementary.","The classifier-based design allows online inference that is about 5.6 times faster than clustering-based inference, which matters for streaming intent data.","Estimating the category count $K$ with the dropout algorithm gives results close to using the true $K$, so the framework's benefit does not hinge on knowing $K$ in advance.","Ablations on HWU64 show that removing CBM, CCM, or the entropy weighting each drops the H-score, so all three components are load-bearing."],"supporting_citations":[{"why":"Provides the prior knowledge-transfer and entropy-based known/novel separation ideas that SDC extends with its dual logit adjustment.","marker":"Shi et al. 2024"},{"why":"The transfer-and-alignment baseline shows bias towards known categories and measures category similarity via prototypes, which SDC's transfer matrix builds on.","marker":"An et al. 2024b"},{"why":"Supplies the prototype-initialization and index-alignment scheme used to initialize the trainable classifier for K total categories.","marker":"An et al. 2023c"},{"why":"Introduces the self-labeling via optimal transport that SDC uses to generate pseudo-labels from calibrated logits.","marker":"Asano, Rupprecht, and Vedaldi 2019"},{"why":"Provides the Sinkhorn-Knopp algorithm that solves the optimal transport problem in pseudo-label generation.","marker":"Cuturi 2013"},{"why":"Gives the contrastive pseudo-label training and S-K implementation settings that SDC follows in its contrastive loss and label assignment.","marker":"Caron et al. 2020"},{"why":"Defines the Generalized Category Discovery task and its evaluation of known and novel category accuracy.","marker":"Vaze et al. 2022"}],"fun_headline_variants":["Turn model bias into a novel-category boost","Calibrate known bias, gain novel classes: +2.6%","Frozen biased logits: the secret to finding new categories","Subtract known bias, add novel clusters: +2.6%","Biased model output as a debiasing signal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on entropy of the biased model's softmax cleanly separating known from novel samples, so a single scalar alpha per sample can steer both the debiasing and the transfer; if a novel sample is confidently mispredicted as a known class, or a known sample has high entropy, the correction goes the wrong way.","fun_headline_variants_meta":{"raw":{"variants":["Turn model bias into a novel-category boost","Calibrate known bias, gain novel classes: +2.6%","Frozen biased logits: the secret to finding new categories","Subtract known bias, add novel clusters: +2.6%","Biased model output as a debiasing signal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000686,"raw_usage":{"total_tokens":3175,"prompt_tokens":1074,"completion_tokens":2101,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":690,"completion_tokens_details":{"reasoning_tokens":2015}},"tokens_in":690,"tokens_out":2101,"duration_ms":13194,"temperature":1.0,"reasoning_tokens":2015,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:01:15.873054+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a probe set where a novel class is semantically near a known class, so the biased model confidently predicts the known class (low entropy); if SDC's gains on novel accuracy come from the entropy gate, such a probe should show smaller or negative gains compared to a dataset with well-separated novel classes.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the prior knowledge-transfer and entropy-based known/novel separation ideas that SDC extends with its dual logit adjustment."}],"review_version":1}