{"id":"79a49394-3851-4912-ab25-3055e052e963","arxiv_id":"2605.28780","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A label-free method identifies spurious concepts in vision models by ranking NMF-derived concept vectors using gradient interactions on misclassified examples, enabling suppression that improves worst-group accuracy on Waterbirds and CelebA.","lead":"The paper introduces a post-hoc method to identify spurious biases in frozen vision models using only standard class labels, by decomposing activations into concepts via NMF and ranking them with a gradient-based bias estimator on misclassifications. A smart generalist might read it because it offers a practical way to audit and debias deployed AI systems where bias attributes are unknown or unlabeled.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Gradient probe on misclassified examples may fail to isolate spurious directions when only class labels are available","rationale":"The identified concern is identical to the reader's weakest assumption; the abstract supplies empirical outcomes but supplies no independent check on the gradient-ranking step, so the low-confidence UNVERDICTED verdict is appropriate until that link is tested.","tokens_in":1772,"tokens_out":276,"duration_ms":30145,"concrete_test":"On Waterbirds, extract the top-5 ranked concepts using only class labels for misclassification detection, then compute their average activation correlation with the known background attribute (via held-out group labels); if this correlation is not statistically higher than for the bottom-5 concepts, the estimator does not isolate the spurious direction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the bias estimator—ranking NMF concepts by their gradient interactions on false-negative vs. false-positive corrections—reliably surfaces decision-relevant spurious concepts from a class-label-only audit set. This step is least secure because misclassifications can arise from multiple sources, and nothing in the procedure enforces that the highest-ranked concepts correspond to the spurious cue rather than core features or noise; the reported alignment on Waterbirds and CelebA therefore rests on an unverified mapping between gradient sign patterns and bias.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes a post-hoc, label-free method to identify spurious concepts in frozen vision classifiers using only class labels from a held-out audit set. For each target class, patches from predicted inputs undergo NMF on intermediate activations to yield concept vectors; these are ranked by a bias estimator based on gradient interactions with misclassified examples (activated on false-negative corrections, suppressed on false-positive corrections). The method recovers known spurious cues on Colored MNIST and Waterbirds, surfaces partially gender-aligned directions on CelebA, and shows that suppressing the top-ranked concepts at inference improves worst-group accuracy by up to 17.9pp on Waterbirds and 10.4pp on CelebA without retraining or parameter updates. Code is provided.","tokens_in":1911,"tokens_out":447,"duration_ms":23826,"significance":"If the gradient-probe ranking on misclassifications reliably isolates decision-relevant spurious directions (rather than core features or noise) from class-label-only data, the approach would offer a practical auditing and debiasing tool for deployed models where bias or group labels are unavailable. The reported gains without retraining are notable for real-world applicability, and the public code is a clear strength supporting reproducibility.","major_comments":[{"comment":"Method section (bias estimator): the central claim that ranking NMF concepts by gradient sign patterns on false-negative vs. false-positive corrections reliably surfaces spurious cues (rather than core features) rests on an unverified mapping; no controls or ablations are described that test whether highest-ranked concepts correspond to the known bias cue versus other sources of misclassification.","section":"Method section"},{"comment":"Experiments (Waterbirds and CelebA results): the reported worst-group accuracy gains of 17.9pp and 10.4pp after suppressing top-ranked concepts are load-bearing for the actionable debiasing claim, yet the manuscript provides no baseline comparison (e.g., random concept suppression or suppression of bottom-ranked concepts) to establish that the gradient-based ranking step is necessary or effective.","section":"Experiments section"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address each major comment below with clarifications based on the existing experiments and indicate where revisions will be made.","responses":[{"response":"The Colored MNIST experiment functions as a direct control for the claimed mapping: the spurious cue (background color) is known a priori, and the bias estimator ranks the corresponding NMF concept highest, as reported in Section 4.1. This shows that gradient sign patterns on misclassifications prioritize the bias cue over core features. We agree that additional ablations would further strengthen the claim and will add a comparison against random concept selection in the revision.","revision_made":"partial","referee_comment":"[Method section] Method section (bias estimator): the central claim that ranking NMF concepts by gradient sign patterns on false-negative vs. false-positive corrections reliably surfaces spurious cues (rather than core features) rests on an unverified mapping; no controls or ablations are described that test whether highest-ranked concepts correspond to the known bias cue versus other sources of misclassification."},{"response":"The gains are obtained specifically from the top-ranked concepts identified by the gradient estimator, and the alignment with known spurious cues on Waterbirds provides supporting evidence for the ranking's relevance. We agree that explicit baselines would better isolate the contribution of the ranking step and will add comparisons to random concept suppression and bottom-ranked concepts in the revised experiments.","revision_made":"yes","referee_comment":"[Experiments section] Experiments (Waterbirds and CelebA results): the reported worst-group accuracy gains of 17.9pp and 10.4pp after suppressing top-ranked concepts are load-bearing for the actionable debiasing claim, yet the manuscript provides no baseline comparison (e.g., random concept suppression or suppression of bottom-ranked concepts) to establish that the gradient-based ranking step is necessary or effective."}],"tokens_in":1499,"tokens_out":409,"duration_ms":118437,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is a label-free audit for spurious concepts in already-deployed vision classifiers. It runs NMF on patches from predicted-class activations to build a concept bank, then ranks concepts by their gradient behavior on misclassified audit examples—higher activation when fixing false negatives, lower when fixing false positives. Suppressing the top-ranked concepts at test time lifts worst-group accuracy by 17.9 points on Waterbirds and 10.4 on CelebA with no retraining.\n\nWhat is new is the specific combination of NMF decomposition with this gradient-based ranking that uses only standard class labels. Prior work often needs bias annotations or group labels; this one does not. The recovery of known cues on Colored MNIST and Waterbirds is a clear positive, and the fact that code is released makes the claims testable.\n\nThe soft spot sits in the bias estimator. Misclassifications can come from noise, core feature weaknesses, or other factors, so nothing in the procedure forces the top-ranked concepts to be the spurious ones rather than something else. The abstract reports alignment on the benchmark cases but gives limited detail on the exact ranking formula, threshold choices, or controls for K in NMF. On CelebA the surfaced directions only partly overlap the annotated gender attribute, which is consistent with the claim that bias need not match annotations but also makes external verification harder.\n\nThis paper is for people who audit production vision models where extra labels or retraining are off the table. A reader working on post-hoc interpretability or trustworthy ML will find the empirical results useful even if they want tighter validation of the ranking step.\n\nIt deserves a serious referee. The practical framing and the held-out gains are enough to justify review time, though reviewers will likely press on the reliability of the gradient-to-bias mapping.","headline":"The paper gives a workable post-hoc way to rank bias concepts in frozen vision models from class labels alone via NMF plus gradient probes on mistakes, with measurable worst-group gains, but the mapping from those probes to actual spurious cues rests on an assumption that needs more checks.","tokens_in":2384,"tokens_out":464,"would_cite":false,"duration_ms":25332,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A post-hoc method identifies spurious concepts in frozen vision models by ranking NMF decompositions via gradient interactions on misclassified examples using only class labels.","keywords":["bias identification","spurious correlations","concept decomposition","gradient probing","label-free auditing","vision classifiers","distribution shift","worst-group accuracy"],"falsifier":"An experiment on Waterbirds or CelebA in which the top-ranked concepts are suppressed yet worst-group accuracy does not rise or falls compared with the unedited model.","tokens_in":2678,"feed_emoji":"🔍","tokens_out":614,"duration_ms":16499,"temperature":0.7,"pith_summary":"Vision classifiers often exploit spurious correlations that break under distribution shift. The paper presents a label-free approach that decomposes intermediate activations from an audit set into concepts via non-negative matrix factorization. It then ranks those concepts according to how their activations align with backpropagated gradients on false positives and false negatives. Suppressing the highest-ranked concepts at inference time raises worst-group accuracy without any model updates or extra labels.","feed_headline":"Gradient probes rank spurious concepts without bias labels","feed_subtitle":"NMF decompositions plus gradient signs on misclassified examples let suppression raise worst-group accuracy up to 17.9 points on standard be","key_machinery":"The bias estimator that ranks NMF concept vectors by the sign and magnitude of their interaction with gradients on misclassified examples from a class-label-only audit set.","core_discovery":"Bias concepts leave a detectable gradient trail: they tend to activate when correcting false negatives and to be suppressed when correcting false positives; a bias estimator built from these interactions surfaces decision-relevant spurious directions that need not match any annotated attribute, enabling both auditing and direct mitigation by concept suppression.","pith_inferences":["The same gradient-probe logic could be tested on non-vision modalities if activations admit stable non-negative decompositions.","If the audit set is drawn from the training distribution rather than a held-out shift, the recovered directions may capture training-time shortcuts instead of shift-sensitive biases.","Iteratively re-ranking after each suppression round might expose chained or higher-order spurious directions the single-pass estimator misses."],"forward_implications":["Suppressing the top-ranked concepts raises worst-group accuracy by up to 17.9 points on Waterbirds and 10.4 points on CelebA with no retraining.","The ranked concepts recover the known spurious cue on Colored MNIST and Waterbirds.","On CelebA the surfaced directions only partially overlap the annotated gender attribute.","The same procedure supplies both an interpretable audit and an actionable debiasing step for already-deployed models."],"fun_headline_variants":["Gradient trails reveal bias without labels","Spurious concepts spotted via gradient probes","Label-free bias ID from concept gradients","Bias directions emerge from gradient interactions","Concept suppression guided by gradient trails"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Gradient interactions of the decomposed concepts on misclassified audit examples reliably mark spurious decision directions even without group or attribute labels.","fun_headline_variants_meta":{"raw":{"variants":["Gradient trails reveal bias without labels","Spurious concepts spotted via gradient probes","Label-free bias ID from concept gradients","Bias directions emerge from gradient interactions","Concept suppression guided by gradient trails","Gradient signs rank spurious concepts label-free","Label-free bias auditing via gradient probes on decomps"]},"model":"grok-4.3","cost_usd":0.006657,"raw_usage":{"total_tokens":3042,"prompt_tokens":705,"num_sources_used":0,"completion_tokens":69,"cost_in_usd_ticks":66565500,"prompt_tokens_details":{"text_tokens":705,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2268,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":705,"tokens_out":69,"duration_ms":26263,"temperature":1.0,"reasoning_tokens":2268,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T12:45:59.522123+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment on Waterbirds or CelebA in which the top-ranked concepts are suppressed yet worst-group accuracy does not rise or falls compared with the unedited model.","supporting_citations":[],"review_version":1}