{"id":"4f7f5e94-7d18-46a0-a4a7-14663a436243","arxiv_id":"2512.05038","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Reliable concept presence in transformers is concentrated in the extreme high-activation tail of in-concept tokens; thresholding that tail improves concept detection and localization.","lead":"The paper shows that in transformer models, the only reliable concept signal is in the extreme high-activation tail of concept-positive tokens, and that thresholding on that tail beats standard aggregation and prompting baselines by up to 0.14 F1. A cheap, broadly tested upgrade for concept-based interpretability — provided the claimed proof of the underlying mechanism is actually supplied and the effect transfers beyond synthetic text labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Text-domain generality rests on GPT-4o-generated labels; tail signal may partly reflect GPT style rather than concept semantics.","rationale":"The reader's weakest_assumption correctly identifies the GPT-4o-generated token labels as the most fragile link in the generality claim. The central detection result is a cross-modal empirical pattern; if the text leg is an artifact, the 'text modality' half of the claim collapses, leaving only the vision-side evidence. The image-based results are independently convincing (human annotations, multiple models, fixed-N variant), so the paper should remain CONDITIONAL pending a human-labeled text validation rather than being rejected outright. The abstract's missing proof of attention-head amplification is a serious overclaim and should also be corrected, but it is not load-bearing for the empirical detection F1 numbers. The proposed N@Tail test on original iSarcasm is feasible because the original dataset provides human-authored sarcastic/sincere pairs with sample-level labels; it directly isolates whether the tail signal exists without GPT-generated context. This is a concrete, cheap check that would settle whether the concern lands.","tokens_in":53093,"tokens_out":5944,"duration_ms":64310,"concrete_test":"Run the Appendix L N@Tail procedure on the original iSarcasm dataset without GPT augmentation, using only the human-authored sarcastic tweets and their non-sarcastic rewrites and sample-level labels (select top 2% tokens per sample, learn one threshold per concept). If N@Tail no longer outperforms CLS/mean baselines, the text-domain results are substantially an artifact of GPT-context contrast. A complementary check: repeat the Sarcasm experiment using a human-labeled sarcasm corpus (e.g., SemEval-2018 Task 3 with human labels) with token-level or comment-level labels; if SuperActivator's F1 advantage disappears, the fully synthetic dataset is not evidence of text-domain generality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central detection claim is supported by strong human-labeled image results, but the claimed cross-modality generality to text depends on token-level labels sourced from GPT-4o. For the fully synthetic Sarcasm dataset, both positive (sarcastic) and negative (neutral) paragraphs are generated by GPT-4o, so the SuperActivator threshold τ = Q_{1−δ}(S+_val) could be isolating GPT-4o's stylistic markers of sarcasm rather than a general concept signal. For iSarcasm and GoEmotions, the augmented GPT-written neutral context forms the out-of-concept token set; the separation between the tail of Din_c and Dout_c may reflect human-vs-GPT stylometric differences rather than concept presence per se. Appendix B itself concedes that for text, not all in-concept samples contain a tail token, so coverage is imperfect and the mechanism is more diffuse. If the text results are partly an artifact of GPT-context contrast, the paper's headline claim of a fundamental, modality-independent transformer property is not established. The image results (COCO, CLEVR, Pascal, OpenSurfaces) with human segmentations remain strong and would support a more qualified claim about visual concepts. Additionally, the abstract asserts a proof of attention-head amplification that does not appear anywhere in the body; this is an omitted proof and an overclaim, but it is less directly load-bearing for the empirical detection result than the label-dependence issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the 'SuperActivator Mechanism,' the observation that for transformer concept vectors, the extreme high-activation tail of the in-concept token distribution provides a sparse but reliable signal for concept presence. The authors define a SuperActivator threshold as the (1−δ) quantile of validation-set in-concept activations, aggregate token activations by max-pooling, and threshold at this quantile for concept detection. They report consistent F1 improvements over standard aggregators and prompting baselines across four image datasets and three text datasets, several models (CLIP, LLaMA, Gemma, Qwen), multiple concept-vector types (mean prototypes, linear separators, k-means, k-means separators, SAEs), and multiple layers. They also propose using the average embedding of local SuperActivators as an attribution objective, reporting improved alignment with ground-truth masks and better insertion/deletion scores. The Appendix contains extensive ablations on layer depth, sparsity, token position, and a fixed-sparsity variant (N@Tail) that requires only sample-level labels and nearly matches the tuned method.","tokens_in":53401,"tokens_out":4464,"duration_ms":49528,"significance":"If the empirical claims hold, the paper identifies a practically useful and broadly applicable property of transformer concept representations: instead of pooling all token activations, focusing on the extreme tail yields better concept detection and attribution. The strongest strengths are the breadth of the evaluation (7 datasets, 5 concept-vector types, 4-5 model families, layer and sparsity ablations), the inclusion of error bars, the released code, and especially the N@Tail variant in Appendix L, which shows that the main detection benefit survives without per-concept tuning of the sparsity parameter. These are substantial. However, the cross-modality generality claim is weakened by the reliance on GPT-4o-generated token-level labels for the text datasets, and the abstract asserts a proof that does not appear in the manuscript. The attribution results are also partly circular because the SuperActivator objective is derived from the same ground-truth labels used for evaluation. With appropriate qualification and additional controls, the image-domain results and the detection methodology would be a solid contribution.","major_comments":[{"comment":"The abstract claims: 'we prove that concept-aligned attention heads multiplicatively amplify pairwise activation gaps, with already-extreme activations growing fastest.' No theorem, proof, or proof sketch appears anywhere in the main text or appendices. This is an unsupported central claim as stated. Either provide the proof (or a precise theorem with assumptions) or remove/soften the claim to an empirical observation.","section":"Abstract and §1"},{"comment":"The text-domain evaluation is confounded by the use of GPT-4o for token-level supervision. In the fully synthetic Sarcasm dataset, both sarcastic and neutral sentences are GPT-generated, so the SuperActivator threshold τ = Q_{1−δ}(S+_val) may be isolating GPT-4o's stylistic markers rather than a general concept signal. For iSarcasm and GoEmotions, the out-of-concept tokens come from GPT-written neutral context, so the separation between the tail of Din_c and Dout_c could reflect human-vs-GPT stylometric differences. This directly undermines the headline claim of a modality-independent transformer property. Appendix B itself concedes that for text not all in-concept samples contain a tail token. Please add a control using human-written neutral context or a human-annotated token-level text dataset, or explicitly qualify the text results as potentially dataset-specific.","section":"§4.1, Appendix C.4"},{"comment":"The attribution evaluation is partly circular. SuperActivators are defined as the top δ of validation in-concept activations, where δ is calibrated per concept to maximize detection F1, and at test time they are tokens exceeding that label-derived threshold. The attribution objective is then the mean embedding of these SuperActivators, and the resulting maps are evaluated against the same ground-truth masks used to define in-concept tokens. It is not surprising that they align better than a global concept vector. The paper needs a control that uses an equally label-informed objective without the tail selection — e.g., the mean embedding of all positive tokens, or a random subset of positive tokens — to show that the improvement is due to the tail property rather than to label leakage into the attribution objective.","section":"§4.3, Appendix N"},{"comment":"The claim that detection 'peaks when using only 5–10% of in-concept token activations' is in part definitional, because δ directly controls the quantile of the in-concept distribution used as the threshold. Figure 5 and the Appendix H/I curves are calibration curves for a family of quantile thresholds, not independent evidence of a mechanism. The N@Tail variant in Appendix L mitigates the tuning concern, and I would encourage the authors to present that as the primary practical evidence for the tail being sufficient. The text should be carefully worded to avoid presenting the optimal-δ analysis as an unanticipated discovery.","section":"§3.2 and Appendix H/I"}],"minor_comments":[{"comment":"The sentence 'there appears to be activations in the tail of Din_c that are well-separated from Din_c' should read 'well-separated from Dout_c'; as written it is self-referential and likely a typo.","section":"Appendix B, last paragraph"},{"comment":"The citation for MFABA is inconsistent: the main Table 2 cites [67] for MFABA, but [67] is FullGrad, while Appendix N.1 correctly cites [89] for MFABA. Please correct the reference.","section":"Table 2 and Appendix N.1"},{"comment":"The COCO SAE SuperAct row reports F1 = 0.582 ± 0.000, which is suspiciously precise; if this is due to a degenerate single concept or rounding, please clarify.","section":"Appendix O, Table 12"},{"comment":"The augmentation prompts include some duplicated phrases (e.g., 'remaining 100%') and inconsistent punctuation. A careful copyedit would improve readability.","section":"Appendix C.4"},{"comment":"The caption says 'SuperActivators' after all three concept activation rows, but only one concept's tail is marked. Clarify which concept the green boxes refer to.","section":"Figure 13 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong empirical study for image-domain concepts, and the N@Tail experiment is a good robustness check. My main concerns are the unsubstantiated proof claim in the abstract, the GPT-4o confound in the text experiments, and the circularity in the attribution evaluation. These are fixable with additional experiments or careful qualification, but the current version overstates its generality. I would look favorably on a revised version that addresses these points, even if the text-domain claims are narrowed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth reading. The core finding: for transformer hidden states, the extreme tail of the in-concept activation distribution carries most of the reliable concept signal, and a global quantile threshold tau = Q_{1-delta}(S+_val) beats standard aggregation (CLS, mean, max, prompt) across a wide sweep of image datasets, models, layers, and concept vector types. The sparsity analysis is thorough: 2-10% tail works, and the N@Tail variant with fixed 10% nearly matches the tuned version, which shows the result is not just a calibration artifact. Credit where due: this is a simple, reproducible improvement for concept detection and localization, with code released.\n\nSoft spots, in order. First, the abstract promises a proof that attention heads multiplicatively amplify activation gaps. No such proof is in the body - no theorem, no appendix. Either supply it or delete the sentence. The paper calls it a 'mechanism' and a 'fundamental property,' but the evidence is correlational distribution analysis. That framing should be softened.\n\nSecond, the text-domain generality rests on GPT-4o-generated token labels. Sarcasm is fully synthetic; iSarcasm and GoEmotions are original content wrapped in GPT-written neutral context. The tail threshold is a quantile over activations of GPT-labeled positive tokens, so the separation between in- and out-of-concept tokens may partially reflect GPT style rather than concept presence. The image results (COCO, Pascal, OpenSurfaces, CLEVR) use human segmentations and stand on their own. The text results alone would not establish cross-modality generality; they need validation on human-labeled spans, or at least a stated limitation.\n\nThird, the circularity concern is real but minor. The threshold is the top-delta quantile of the validation in-concept distribution, and delta is tuned per concept to maximize the same F1 used for evaluation. That is standard calibration, not cheating, and the fixed-sparsity N@Tail experiment mitigates it. Still, the 'tail is the mechanism' language partly follows from how the threshold is defined.\n\nOverall: the empirical program is extensive and well-controlled. The paper is a useful contribution to concept-based interpretability for visual concepts. A serious referee should engage; I would send it out. I'd ask for the proof claim to be fixed, the text labels to be addressed, and the framing to be pulled back to what the data shows.","headline":"Strong empirical case that the tail of in-concept activations is a reliable detection signal for visual concepts; text modality rests on synthetic labels and the abstract overclaims a proof that never appears.","tokens_in":53939,"tokens_out":2770,"would_cite":true,"duration_ms":29193,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Concept signals in transformers live in a sparse high-activation tail — the rest is noise.","keywords":["concept vectors","concept detection","SuperActivator mechanism","token activation distributions","transformer interpretability","sparse concept signals","max pooling","concept localization"],"falsifier":"Run the identical SuperActivator detector on a human-annotated, token-level text corpus for a concept like sarcasm or emotion, with no LLM-generated context. If the top 5–10% in-concept activation tail does not separate from out-of-concept activations, or F1 falls to baseline levels, the mechanism's text-domain claim collapses. A simpler check: train on LLM-labeled positive tokens and test on concept-positive sentences with LLM-stylistic cues such as hashtags, emoji, or tag-wrapped spans removed; a sharp performance drop would indicate the tail encodes label-generation style rather than the co","tokens_in":52946,"feed_emoji":"🎯","tokens_out":5254,"duration_ms":50515,"temperature":0.7,"pith_summary":"The paper argues that although concept vectors produce noisy, overlapping activations for in-concept and out-of-concept tokens, transformers systematically push the most reliable evidence for a concept into a tiny set of extreme token activations, which it calls SuperActivators. It proves that concept-aligned attention heads amplify activation gaps multiplicatively, so the strongest activations grow fastest, and shows empirically across vision and language transformers that thresholding the global in-concept tail — typically the top 5–10% of in-concept activations — detects concept presence more accurately than standard pooling, [CLS]-score, or prompting baselines, by up to 0.14 F1. The same tail signal improves concept localization: attributions computed against the average embedding of local SuperActivators align better with ground-truth masks and are more faithful to model behavior. A reader should care because it turns a known nuisance, noisy concept activations, into a simple detection rule that needs only sample-level labels and a per-concept quantile threshold.","feed_headline":"Threshold the tail: a 5-10% slice of activations beats all baselines","feed_subtitle":"A per-concept quantile on in-concept tokens turns noisy transformer activations into a reliable detector, across images and text.","key_machinery":"SuperActivator threshold — τ^{super}_{c,δ}=Q_{1−δ}(S^+_{val,c}), the (1−δ) quantile of in-concept validation activations; tokens scoring above it are SuperActivators. The detection rule is max-pooling over token activations with this threshold, which is what turns the tail into a concept detector. The theoretical engine is a proof that concept-aligned attention heads amplify pairwise activation gaps multiplicatively, making extreme activations grow faster than typical ones; this is offered as the reason the tail is sparse, consistent across depth, and present in both modalities.","core_discovery":"The central claim is the SuperActivator Mechanism: for a concept vector v_c, the token activation scores s_c(z)=<z, v_c> mostly overlap between in-concept and out-of-concept tokens, but the in-concept distribution D^in_c develops a positive tail that separates from D^out_c with depth; the tokens in that tail, called SuperActivators, appear in most concept-positive samples and are therefore reliable indicators of presence. Detection reduces to max-pooling token activations and thresholding at the global in-concept quantile τ = Q_{1−δ}(S^+_{val,c}) with small δ. The paper further proves that concept-aligned attention heads multiplicatively amplify pairwise activation gaps, so already-extreme a","pith_inferences":["Because the three text datasets define token-level concepts with LLM-generated labels — one fully synthetic, two augmented with machine-written neutral context — the tail signal for sarcasm and emotion may be partly a stylistic signature of the label generator rather than the concept itself; human-labeled span data would settle this.","If the mechanism is as general as claimed, the tail threshold should also work for detecting concepts discovered entirely without labels on new domains and non-transformer architectures; the paper tests transformers only.","Extreme-value reasoning suggests the SuperActivator threshold is an estimate of a high quantile of a heavy-tailed distribution; this connects to outlier and novelty detection and could be formalized to set sparsity δ without validation.","The coverage result implies the detector inherits label noise: positive spans that lack any extreme token will be systematically missed, which may explain the weaker tail coverage the paper admits for text."],"forward_implications":["Concept detection can be done with almost no tuning: fixing the tail at 10% of in-concept activations and learning a single threshold from sample-level labels nearly matches the fully tuned SuperActivator detector and beats all baselines.","Concept localization improves by using the mean embedding of local SuperActivators as the attribution objective, yielding higher alignment with human masks and better insertion/deletion faithfulness than the global concept vector.","The tail signal persists across supervised and unsupervised concept extraction, including mean prototypes, linear separators, k-means, and sparse autoencoders, so the mechanism is tied to how transformers encode semantics rather than to one concept-vector construction.","SuperActivators are not position-dependent, so the signal is semantic rather than an artifact of where tokens appear in the input.","The amplification theorem gives a testable reason concept separability grows with depth and plateaus in middle layers before final-layer compression."],"fun_headline_variants":["SuperActivators: how transformers hide reliable concepts in a few tokens","The 5% tail of activations that reliably signals concept presence","New proof: concept signals live in the extreme tail of activations","Quantile thresholding on in-concept tokens boosts concept detection by 0.14 F1","Sparse SuperActivators: less than 10% of tokens carry the signal"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The token-level in/out concept partition is trusted as ground truth — and for the text datasets that partition comes from an LLM's generated or wrapped spans, so if embeddings encode the generator's style rather than the concept, the tail signal is partly a dataset artifact.","fun_headline_variants_meta":{"raw":{"variants":["SuperActivators: how transformers hide reliable concepts in a few tokens","The 5% tail of activations that reliably signals concept presence","New proof: concept signals live in the extreme tail of activations","Quantile thresholding on in-concept tokens boosts concept detection by 0.14 F1","Sparse SuperActivators: less than 10% of tokens carry the signal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00084,"raw_usage":{"total_tokens":3521,"prompt_tokens":793,"completion_tokens":2728,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":2629}},"tokens_in":537,"tokens_out":2728,"duration_ms":18485,"temperature":1.0,"reasoning_tokens":2629,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T18:27:45.379312+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical SuperActivator detector on a human-annotated, token-level text corpus for a concept like sarcasm or emotion, with no LLM-generated context. If the top 5–10% in-concept activation tail does not separate from out-of-concept activations, or F1 falls to baseline levels, the mechanism's text-domain claim collapses. A simpler check: train on LLM-labeled positive tokens and test on concept-positive sentences with LLM-stylistic cues such as hashtags, emoji, or tag-wrapped spans removed; a sharp performance drop would indicate the tail encodes label-generation style rather than the co","supporting_citations":[],"review_version":1}