{"id":"573278f7-e9d0-4d88-b8d1-33720bfc4d5b","arxiv_id":"2411.10019","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Mid-level (near-zero) logit activations contain information about spurious correlations and mislabels that maximal activations hide, enabling a no-group-label retraining method, MID, that improves worst-group accuracy on Waterbirds and CelebA.","lead":"The authors show that examples where a classifier's output logit is near zero, not maximal, reveal hidden spurious patterns such as biased correlations between hair color and gender. They use this to build MID, a method that selects low-confidence samples, clusters them, and retrains the classification layer to improve worst-group accuracy on biased benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central mechanism—that near-zero class logits are enriched for counter-spurious and low-spuriosity samples—is supported only by a small qualitative synthetic observation and is never directly measured on Waterbirds or CelebA; if that enrichment is absent, the reported WGA gains do not…","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing point: MID's usefulness depends on mid-level logits being enriched for samples that break the spurious trend, and this is demonstrated only in a single synthetic setting before being transferred to CelebA and Waterbirds. I agree with that assessment. The synthetic evidence is anecdotal (six images around the logit sign change), and the real-dataset pipeline inserts human judgment at both the choice of k and the selection of which cluster to use for retraining, making it hard to separate the effect of the filtering step from the effect of last-layer retraining on a deliberately chosen subset. The concrete test above would directly quantify the enrichment using the true group labels (for evaluation only), providing the missing evidence for or against the core assumption. Since the paper currently lacks this measurement, the conditional verdict is appropriate: the method is promising and the reported numbers are strong, but the central interpretability claim is not yet fully supported. No verdict change is needed; the reader's recommendation to require more evidence stands.","tokens_in":20913,"tokens_out":5197,"duration_ms":58313,"concrete_test":"Using the exact ERM checkpoints from Sec. 6.3, order the full Waterbirds and CelebA training sets by per-class logit and reproduce the mid-range selection from Step 2 (e.g., the 2,000 points nearest zero per class, as in Appendix A.5). Using only the held-out true group labels for evaluation, compute the fraction of the selected set belonging to the minority groups (land-bird-on-water and water-bird-on-land for Waterbirds; blonde-male for CelebA) and compare it with the base rate over the full training set. Report the enrichment ratio per dataset. If the ratio is near 1, Step 2 does not specifically enrich for counter-spurious or low-spuriosity data, and the WGA gains in Table 1 cannot be attributed to the mid-level activation mechanism; if the ratio is substantially above 1, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"MID's Step 2 assumes that samples with class logits near zero are preferentially drawn from low-spuriosity in-class examples and counter-spurious out-of-class examples, so that k-means clustering of their penultimate embeddings can isolate the spurious attribute. The only evidence offered for this assumption is a qualitative inspection of six images in DSpritesUnfair (bias 0.4, Sec. 4.2, Fig. 3); no quantitative enrichment statistic is reported for either real benchmark. Appendix A.5 shows how many mid-range points were selected and how many were removed by BLIP, but it never measures the group composition of the mid-range set before clustering. Instead, the paper shows the composition of the final selected cluster (Fig. 10), which is already the product of human selection of k and of the 'spurious' cluster. Consequently, the Table 1 improvements could in principle arise from last-layer retraining on a balanced, hard-example subset—a mechanism known from DFR [39]—rather than from any property specific to mid-level activations. The human-in-the-loop choice of cluster also risks implicitly using prior knowledge of each benchmark's known spurious attribute (gender in CelebA, background in Waterbirds), which would undercut the claim of operating without group information. Thus the central interpretability conclusion rests on an unmeasured enrichment assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that interpretability analyses based on maximally activating examples miss information encoded at non-maximal activation levels, and proposes a practical method, MID (Modifying with logit Intercept Data). MID first trains a standard ERM classifier, then selects samples whose class logits are near zero (the logit 'intercept'), filters suspected mislabels with BLIP, clusters the penultimate-layer embeddings of the remaining samples with k-means, manually selects a cluster that exhibits a harmful spurious pattern, and retrains the classification layer using DFR-style logistic regression. On Waterbirds and CelebA, MID reports worst-group accuracies of 87.8 and 85.5, respectively, compared with ERM baselines of 68.4 and 47.8, and is competitive with methods that use group labels while requiring none.","tokens_in":21228,"tokens_out":4391,"duration_ms":48706,"significance":"If the mechanism claimed by the paper holds, this is a valuable bridge between interpretability and robustness: it would show that non-maximal activations can expose confounds without group labels, and the proposed last-layer retraining is simple and cheap. The paper is honest about the human-in-the-loop nature of the method and provides ablations suggesting that both the mid-logit filter and the choice of cluster matter. The reported WGA gains are large, and the use of DFR-style retraining is well motivated by prior work. However, the central enrichment assumption -- that near-zero logit samples are enriched for low-spuriosity and counter-spurious examples -- is only qualitatively demonstrated on a synthetic dataset and never directly measured on the real benchmarks, so the significance of the empirical results is conditional on that assumption being validated.","major_comments":[{"comment":"The core claim that near-zero class logits are enriched for low-spuriosity and counter-spurious examples is supported only by a qualitative inspection of six images on DSpritesUnfair (bias 0.4). No quantitative enrichment statistic is reported for Waterbirds or CelebA, and Fig. 10 shows the group composition only of the final cluster, which is already the product of human selection of k and of the 'spurious' cluster. Because the Table 1 gains could in principle arise from DFR-style retraining on any hard or balanced subset rather than from a property specific to mid-level activations, the paper should measure the group composition of the mid-logit selection before clustering (e.g., the fraction of minority-group examples among selected samples versus random samples matched by logit rank) and report this for both benchmarks.","section":"Sec. 4.2, Fig. 3 and Sec. 5.1"},{"comment":"The cluster selection step is not prespecified: k is increased 'until a pattern emerges', and the cluster used for retraining is chosen by manual inspection of cluster contents. For CelebA this selects the cluster corresponding to the known spurious attribute (blonde with masculine-associated features). This introduces human selection based on prior knowledge of each benchmark's known bias and undercuts the claim of operating without group information. Please specify a preselectable or automated rule for choosing k and the retraining cluster, and report sensitivity to both choices (e.g., WGA for every cluster and every k tried).","section":"Sec. 5.1 and Appendix A.6"},{"comment":"The ablation of the mid-logit filter is qualitative only. The paper states that clustering the full CelebA embeddings did not reveal a spurious pattern and that retraining on one ambiguous cluster does not help WGA, but no quantitative worst-group accuracies are given for these ablations. A quantitative comparison of MID against (a) retraining on a random balanced subset, (b) retraining on BLIP-filtered mid-logit data without clustering, and (c) retraining on each individual k-means cluster would isolate the contribution of the mid-logit selection and the cluster choice. This is needed to support the claim that the mid-level activation lens, rather than the last-layer retraining mechanism, is responsible for the improvements.","section":"Appendix A.7"},{"comment":"The mid-range selection rule is underspecified. For CelebA the paper reports 2,000 logit-intercept points per class, but for Waterbirds no analogous count or threshold is given, and the exact definition of 'near-zero logit' is not operationalized (e.g., a window around zero, a fixed number per class, or a sign-change region). Without a precise selection rule the method cannot be reproduced or compared across datasets; please specify the selection criterion and report the number of selected points for both benchmarks.","section":"Sec. 5.1 and Appendix A.5"}],"minor_comments":[{"comment":"The entry 'DFRTrTr' is not defined in the table or caption; the text in Sec. 6.3 mentions 'DFR T r T r' but it should be spelled out as a specific DFR variant.","section":"Table 1"},{"comment":"No variance or number of seeds is reported for the Waterbirds and CelebA ERM and MID runs, despite the DSprites pilot reporting means over five seeds. Since Table 1 is the main quantitative claim, please include standard deviations or error bars over multiple initializations.","section":"Sec. 6.3 and Table 1"},{"comment":"The sentence 'we train on the available validation data with additional randomly selected training set data' is ambiguous and appears to contradict Sec. 6.4's statement that group information is used only for evaluation. Please clarify whether unlabeled validation data are used during retraining and how this affects the comparison with baselines that use group-balanced validation sets.","section":"Appendix A.6"},{"comment":"The claim that points 'right above' the zero logit are more likely to be low-spuriosity memorized points while points 'right below' are more likely to be counter-spurious examples is presented as a finding but is not measured anywhere in the paper; it should be framed as a hypothesis or supported with data.","section":"Sec. 7"},{"comment":"There is a LaTeX artifact in Table 2 ('L n(L)'), and several reference formatting issues such as 'Hermann et Lampinen' and 'TCA V' with inconsistent spacing; these should be cleaned up.","section":"Table 2 and References"},{"comment":"The example images in Figs. 3 and 6 are very small, making the qualitative claims about position and shape difficult to verify; larger crops or additional annotations would help.","section":"Figs. 3 and 6"}],"recommendation":"major_revision","confidential_remarks":"The paper's novelty relative to GEORGE rests primarily on the mid-logit prefilter before clustering. Given that the cluster selection is human-guided and the enrichment assumption is unmeasured on real data, the central empirical claim is not yet fully supported. The requested measurements -- group composition of the mid-logit set, quantitative ablations, and a prespecified cluster-selection rule -- are within the scope of a revision and would determine whether the paper meets the bar for publication. The related work coverage is adequate and there are no obvious novelty disclosure concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper earns a real read. MID is a no-group-label method that filters training examples by near-zero logits, clusters their penultimate embeddings, picks a problematic cluster, and retrains only the last layer. The reported worst-group accuracies — 87.8 on Waterbirds, 85.5 on CelebA, up from ERM's 68.4/47.8 — are large gains, competitive with methods that require group labels. The core observation, that non-maximal activations reveal spurious associations that maxima hide, is a genuine extension of prior work on low-confidence OOD detection and neuron-level interpretability, and the paper gives credit where it's due (DFR, GEORGE, Hendrycks & Gimpel, La Rosa).\n\nThe soft spots are real but not fatal. The central mechanism is that near-zero logits are enriched for counter-spurious and low-spuriosity samples. That is only demonstrated qualitatively on the synthetic DSpritesUnfair (six images), never quantified on Waterbirds or CelebA. Since the group labels exist for evaluation, the authors could measure the composition of the mid-logit set and settle it. Without that, the gains could partly come from retraining on a hard, balanced subset — the DFR mechanism — rather than from a property specific to mid-level activations. The cluster-selection step is human-in-the-loop: k is tried from 2 upward, and the 'spurious' cluster is chosen after inspecting images. On CelebA the human sees blonde males and recognizes the benchmark's known gender bias; that's fine as a tool, but it weakens the 'no group information' framing. The ablations do address the obvious confound — filtering matters, random clusters don't help — which is more than many method papers do.\n\nOther smaller issues: two datasets is thin, code isn't available yet, and the regularizer is tuned on a split of the selected data (reasonable, but another knob). The citations are appropriate and the limitations are stated in the paper itself.\n\nWho's this for: anyone working on spurious correlations without group labels, shortcut learning, or activation-level interpretability. It deserves a serious referee. The referee should ask for enrichment statistics on the real benchmarks, a blinded or automated cluster-selection step, and at least one more dataset. The method is promising and the paper is honest; it just needs a second round of evidence.","headline":"A promising no-group-label method for finding spurious correlations via mid-level logits, with strong WGA gains but an unmeasured core mechanism and a human-in-the-loop cluster selection that deserves scrutiny.","tokens_in":21747,"tokens_out":3599,"would_cite":true,"duration_ms":37660,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T45"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the samples a classifier is least confident about—those with near-zero logits—concentrate the examples that break spurious correlations, and that retraining the classifier's last layer on a clustered subset of those…","keywords":["mid-level activations","spurious correlations","worst-group accuracy","interpretability","last-layer retraining","mid-range logits","concept entanglement","logit intercept data"],"falsifier":"Measure, on CelebA or Waterbirds, the density of minority-group samples (blonde males, or water birds on land backgrounds) inside the near-zero-logit band selected by MID; if it is no higher than the base rate in the training set, the mid-range selection does not target spurious correlations and the method's mechanism is wrong. A second check: retrain the last layer on a randomly chosen cluster instead of the cluster judged to encode the spurious pattern—the paper's own ablation shows random-cluster retraining does not reliably improve worst-group accuracy, so a reader could verify this holds across seeds and datasets.","tokens_in":20732,"feed_emoji":"🔍","tokens_out":6376,"duration_ms":60554,"temperature":0.7,"pith_summary":"Interpretability research has mostly asked which inputs maximally activate a neuron or concept direction, implicitly treating activation as binary. This paper argues that the unexamined middle of the activation range carries the signal that matters: samples with near-zero logits for a class are disproportionately either in-class examples that lack the spurious attribute the model relies on, or out-of-class examples that contain it. The authors turn this observation into a practical method, MID (Modifying with logit Intercept Data): select mid-range logit samples, filter mislabels, cluster their frozen penultimate embeddings, identify the cluster whose poor classification is inappropriate (a spurious pattern), and retrain only the classification layer on that cluster. On the Waterbirds and CelebA benchmarks this lifts worst-group accuracy from 68.4 and 47.8 under standard training to 87.8 and 85.5, matching or beating methods that require group labels. The paper's broader claim is that non-extremal activations are a general resource for locating entangled and confounding concepts in representations.","feed_headline":"Mid-range logits reveal the spurious cues classifiers hide","feed_subtitle":"Retraining on uncertain samples lifts worst-group accuracy to 87.8 on Waterbirds and 85.5 on CelebA, with no group labels","key_machinery":"The load-bearing object is the 'intercept data': the set of training samples whose logit for a chosen class lies near zero, where the model's prediction confidence is lowest. The paper argues that this band is enriched for two kinds of examples that break the spurious trend—in-class samples lacking the spurious attribute and out-of-class samples carrying it—and that k-means clustering of their penultimate-layer embeddings separates a coherent group whose poor accuracy is caused by the spurious association rather than by benign ambiguity. Carrying the argument further, the last layer of the ERM model is replaced and retrained by $\\ell^1$-regularised logistic regression on this selected cluster, exploiting the finding that ERM still learns core features and only the final mapping needs correction.","core_discovery":"The paper's central claim is that intermediate representations encode information at every level of stimulation, not just at the extremes, and that the mid-range of output-neuron activations is where a model's learned associations between class labels and confounding attributes become visible. Concretely, when images are ranked by their logit for a class, the maximally activating examples are typical of the majority spurious trend, while examples near the zero-logit intercept include shapes in a spurious location for the wrong class, low-spuriosity instances of the class, and mislabels—associations that maximum-activation analysis hides. MID operationalises this: for each class it keeps the near-zero logit samples, removes disputed labels using a vision-language model, clusters the encoder embeddings with k-means, and selects the cluster whose poor performance cannot be excused by image quality or labelling ambiguity (for CelebA, blonde faces with masculine features). Retraining the final layer on that cluster with $\\ell^1$-regularised logistic regression, on top of the frozen feature extractor, improves worst-group accuracy to 87.8 on Waterbirds and 85.5 on CelebA without access to spurious-attribute labels for training or validation. The same logic, the paper argues, extends beyond output neurons to any concept direction in latent space found by methods such as TCAV or SVD-based decomposition.","pith_inferences":["A natural extension the paper does not test: iterate MID—retrain, re-select the mid-range band, and re-cluster—to see whether repeated rounds keep removing spurious reliance or converge to a stable selection.","The mid-logit band is a free, unsupervised proxy for the samples a model has memorised against its own trend; the same filter could be reused to flag out-of-distribution inputs or label noise in other benchmarks, since the paper shows it captures 55.1% of the ERM model's errors on CelebA.","Because the cluster-interpretation step is the only human-in-the-loop part, automating it with a vision-language model (which the paper mentions but does not evaluate) is a direct testable extension.","The near-zero-logit heuristic suggests a broader principle: for any neuron or direction, the decision boundary in activation space may be where entangled concepts live, connecting this work to geometric accounts of polysemanticity."],"forward_implications":["Maximal-activation interpretability is incomplete: analyses that only inspect the highest-activating examples will systematically miss the associations and confounds a model encodes at mid-range stimulation.","Last-layer retraining on automatically selected mid-logit clusters can match or approach the worst-group accuracy of methods that require group labels, while needing none for training or validation.","The pipeline is cheap: it reuses the frozen penultimate features of a standard ERM model and only re-fits the final layer with logistic regression.","The same mid-range inspection applies to any concept direction in latent space, not just output neurons, so confounding concepts found by concept-vector methods could be audited and mitigated the same way."],"supporting_citations":[{"why":"Establishes that last-layer retraining on group-balanced data is sufficient to remove spurious reliance; MID adopts this retraining recipe.","marker":"[39]"},{"why":"Provides evidence that ERM models learn core features despite relying on spurious ones, justifying the decision to retrain only the classification layer.","marker":"[36]"},{"why":"GEORGE is the closest no-group-label baseline; its failure to find spurious clusters on unfiltered embeddings motivates MID's filtering step, and MID outperforms it.","marker":"[80]"},{"why":"JTT is the standard group-robustness baseline that needs validation group labels and serves as a comparison point for MID's no-label performance.","marker":"[43]"},{"why":"The DSpritesUnfair construction generates the synthetic correlated-feature setting used for the motivating demonstrations of mid-range activations.","marker":"[12]"},{"why":"BLIP is the vision-language model used in a VQA setting to filter mislabelled and ambiguous images out of the mid-logit data.","marker":"[42]"},{"why":"CelebA is one of the two benchmark datasets whose hair-colour/gender spurious correlation provides the harder test case for MID.","marker":"[44]"}],"fun_headline_variants":["Mid-level activations expose spurious correlations","Beyond max activations: mid-logits reveal hidden bias","Retrain on uncertain samples to beat spurious cues","Mid-range logits uncover confounds that max activations miss","Non-maximal activations: key to deciphering representations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that near-zero-logit samples are enriched for examples that break the spurious trend rather than dominated by mislabels, label noise, or low-quality images, and that this enrichment, demonstrated on synthetic DSpritesUnfair data, transfers to CelebA and Waterbirds.","fun_headline_variants_meta":{"raw":{"variants":["Mid-level activations expose spurious correlations","Beyond max activations: mid-logits reveal hidden bias","Retrain on uncertain samples to beat spurious cues","Mid-range logits uncover confounds that max activations miss","Non-maximal activations: key to deciphering representations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000458,"raw_usage":{"total_tokens":2353,"prompt_tokens":1057,"completion_tokens":1296,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":673,"completion_tokens_details":{"reasoning_tokens":1218}},"tokens_in":673,"tokens_out":1296,"duration_ms":9766,"temperature":1.0,"reasoning_tokens":1218,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:02:58.481094+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, on CelebA or Waterbirds, the density of minority-group samples (blonde males, or water birds on land backgrounds) inside the near-zero-logit band selected by MID; if it is no higher than the base rate in the training set, the mid-range selection does not target spurious correlations and the method's mechanism is wrong. A second check: retrain the last layer on a randomly chosen cluster instead of the cluster judged to encode the spurious pattern—the paper's own ablation shows random-cluster retraining does not reliably improve worst-group accuracy, so a reader could verify this holds across seeds and datasets.","supporting_citations":[{"cited_title":"Last layer re-training is sufficient for robustness to spu- rious correlations","cited_arxiv_id":null,"evidence_quote":"Establishes that last-layer retraining on group-balanced data is sufficient to remove spurious reliance; MID adopts this retraining recipe."},{"cited_title":"On feature learning in the presence of spuri- ous correlations","cited_arxiv_id":null,"evidence_quote":"Provides evidence that ERM models learn core features despite relying on spurious ones, justifying the decision to retrain only the classification layer."},{"cited_title":"No subclass left behind: Fine- grained robustness in coarse-grained classification prob- lems","cited_arxiv_id":null,"evidence_quote":"GEORGE is the closest no-group-label baseline; its failure to find spurious clusters on unfiltered embeddings motivates MID's filtering step, and MID outperforms it."},{"cited_title":"Just train twice: Improving group robustness without training group information","cited_arxiv_id":null,"evidence_quote":"JTT is the standard group-robustness baseline that needs validation group labels and serves as a comparison point for MID's no-label performance."},{"cited_title":"Deep learning face attributes in the wild","cited_arxiv_id":null,"evidence_quote":"CelebA is one of the two benchmark datasets whose hair-colour/gender spurious correlation provides the harder test case for MID."}],"review_version":1}