{"id":"bce088a1-515a-4053-9961-4d915efc119f","arxiv_id":"2501.04784","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Using the mean register-token embedding together with the CLS token in frozen DINOv2 backbones improves ImageNet out-of-distribution accuracy by about 2-4% and anomaly rejection FPR by about 2-3% over CLS plus mean-patch baselines.","lead":"Vision transformers typically use a special CLS token for classification. This paper finds that averaging the model's embedded register tokens and attaching them to the CLS token improves out-of-distribution accuracy and anomaly rejection in frozen DINOv2 models, without extra compute.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central comparison is confounded: mu_R averages M=4 register tokens while mu_P averages hundreds of patch tokens, so OOD gains may reflect token-count statistics rather than register-specific global information.","rationale":"We focused on the token-count confound because it targets the central causal claim (registers are uniquely useful), not just the magnitude of the effect. If the control experiment shows any M-token subset mean performs like mu_R, the method's premise collapses, even if the specific comparison in Table I is reproducible. The paper's own protocol (Section III) uses the same linear layer size for [CLS; mu_P] and [CLS; mu_R], so the only difference is the choice of auxiliary tokens. The paper invokes Darcet et al. to argue registers carry global information, but Darcet's observation does not establish that the average over all M registers is the right way to extract it, nor that the gain is register-specific. We agree with the reader that missing error bars and ID drops matter, but those concern statistical confidence and reporting; the confound, if real, changes the interpretation. A simple controlled comparison would settle it. We therefore keep the conditional verdict but add this specific requirement.","tokens_in":8048,"tokens_out":9237,"duration_ms":94413,"concrete_test":"On the same DINOv2 ViT-B/L/G with-register checkpoints and the same linear-probe protocol, compute OOD accuracy for: (i) [CLS; mu_R]; (ii) [CLS; mu_P]; (iii) [CLS; mean of M randomly sampled patch tokens], repeated for at least 5 different random subsets; and (iv) [CLS; concatenation of all M register tokens] instead of the mean. If (iii) matches or exceeds (i) on ImageNet-A/R/S, the register-specific explanation is unsupported. If (i) consistently beats (iii), the confound is resolved. Comparing (i) vs (iv) also tests whether averaging destroys per-register information.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's key empirical result is that [CLS; mu_R] beats [CLS; mu_P] for OOD generalization and anomaly rejection (Table I). But this comparison varies two things at once: token type (register vs patch) and the number of tokens averaged (M=4 registers vs L patch tokens, typically 196-256 for DINOv2 at 224px). A mean over 4 tokens has much higher variance and preserves local/outlier structure that a mean over hundreds of tokens smooths away. The paper never runs the control [CLS; mean of M randomly chosen patch tokens], so the observed gains could be caused by the aggregation statistics, not by any 'global image-level information' peculiar to registers. The premise that registers carry such information is imported from Darcet et al. [6] and assumed to survive averaging; this is exactly the load-bearing assumption, and it is untested. If the control matches the method, the title and hypothesis ('leveraging registers') are not supported, although the recipe might still work for incidental reasons.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether register tokens in DINOv2 Vision Transformers are useful beyond their original role in suppressing high-norm patch tokens. It proposes to concatenate the [CLS] token with the mean of the register embeddings, µR, and train a linear classifier on the resulting representation. This is compared with the standard [CLS; µP] representation, where µP is the mean of patch embeddings. Experiments with ViT-Base, ViT-Large, and ViT-Giant, evaluated on ImageNet-A/R/S for OOD generalization and on six datasets for anomaly rejection, report 2–4% top-1 OOD accuracy gains and 2–3% false-positive-rate reductions while claiming that in-distribution accuracy is maintained, all with no additional inference cost. The main evidence is in Table I.","tokens_in":8208,"tokens_out":5061,"duration_ms":51578,"significance":"If the central empirical claim is correct, the paper provides a zero-cost, post-hoc feature choice that improves OOD generalization and anomaly rejection for frozen DINOv2 backbones, which would be a practically useful finding. The strengths of the paper are that it uses standard linear-probing protocols, open-source checkpoints, three backbone scales, and it does not tune the method on the OOD test sets. However, the result is purely empirical, and its load-bearing comparison is confounded by the different numbers of tokens being averaged; the paper also reports single-run numbers without uncertainty estimates. The stated quantitative gains in the abstract are not fully consistent with Table I. The direction of the result is plausible, but the central attribution to 'registers' is not yet established.","major_comments":[{"comment":"The central comparison [CLS; µR] versus [CLS; µP] varies two factors simultaneously: the token type (register vs patch) and the number of tokens averaged (M=4 registers vs roughly 196–256 patch tokens). A mean over four vectors has very different statistical properties from a mean over hundreds, so the OOD gains could be caused by the aggregation statistics rather than by any 'global image-level information' specific to registers. The paper does not run the control [CLS; mean of M randomly chosen patch tokens] (or an equivalent control that matches the number of averaged tokens), so the attribution of the gain to registers is not supported. Please add this control, or reframe the claim to a recipe-level statement with the confound acknowledged.","section":"§IV, Table I"},{"comment":"All numbers appear to be single-run estimates with no standard deviations or seeds. Several of the reported margins are small: for example, the ViT-G energy-score mean FPR is 21.25 for [CLS; µP] versus 20.86 for [CLS; µR], and the ID accuracy differences are below one point for all three backbones. Without variance or multiple runs, the claim that the improvements are consistent across backbones is not fully supported. In addition, the claim that ID accuracy is 'maintained' is contradicted by the printed numbers: ViT-G drops from 87.1 to 86.57, ViT-Large from 86.7 to 85.82, and ViT-Base from 84.21 to 83.84. Please report error bars or multiple runs, and explicitly acknowledge this small ID tradeoff.","section":"§IV, Table I"},{"comment":"The abstract states a '2-3% reduction in false positive rates,' but Table I does not consistently show this. Mean FPR reductions relative to the stronger [CLS; µP] baseline are 3.91/4.65/3.58 points for MSP and 0.39/3.22/5.63 points for Energy across the three backbones; the ViT-G Energy improvement is only 0.39 points. The quantitative claim in the abstract needs to be restated precisely, specifying which baseline and scoring function it refers to.","section":"Abstract and §IV-B"}],"minor_comments":[{"comment":"The sentence 'for ViT-Large, our method reduces the mean FPR by 22.18 and 3.22 percentage points compared to the baselines respectively' is unclear and appears to hold only for the Energy score compared with the no-register baseline on one architecture; please rephrase and specify which baselines and which scoring functions are being compared.","section":"§IV-B"},{"comment":"The abstract and conclusion say the method works on backbones 'trained with and without registers,' but the proposed [CLS; µR] representation requires a model that has register tokens. Please clarify that the no-register models are used only as baselines.","section":"Abstract and Conclusion"},{"comment":"The last sentence of Section IV-A is garbled: 'the combination of by viewing register tokens as auxiliary features and combining it with CLS embeddings yield substantially superior capture more features in terms of generalization and robustness.' This needs to be rewritten.","section":"§IV-A"},{"comment":"The caption says 'Each color indicates the token embeddings chosen,' but the figure legend is not described in the caption and the axes are not labeled. Please make the figure self-contained.","section":"Fig. 1"},{"comment":"There are minor language and formatting issues, including 'auxilliary' in the conclusion and inconsistent capitalization of 'DINOv2' (Dino-v2, Dinov2, DINO-v2).","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The basic experimental protocol is sound and the paper is readable in structure, but the missing token-count control and the absence of error bars are load-bearing issues for the central claim. The abstract's FPR range also does not match the table. This is suitable for major revision rather than rejection, because the missing control is a feasible experiment within the scope of the manuscript. If the authors add the control and rescale their claims, the paper could become a solid empirical contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful empirical recipe, but the central comparison has a confound that needs a control before I'd trust the mechanistic story.\n\nThe new thing here is narrow and legitimate: no one has systematically reported that concatenating the CLS token with the mean of register tokens improves OOD accuracy and anomaly rejection for frozen DINOv2. The paper shows this across three backbones with a fair baseline (CLS + mean patch). The gains are a few points, the protocol is simple, and there's no inference overhead. That's worth knowing for practitioners.\n\nWhere it gets soft:\n\n1. The stress-test note is right. Comparing [CLS; µ_R] to [CLS; µ_P] changes token type and the number of averaged tokens at once (4 registers vs ~256 patches). The paper never runs the control [CLS; mean of 4 random patches]. If that control matches µ_R, then the effect is about aggregation statistics, not register-specific global information. This is the load-bearing claim, and it's untested. This is not a rejection-level flaw if you treat the paper as a recipe, but it is a hole in the interpretation.\n\n2. All results are single-run with no error bars or seeds. ID accuracy drops 0.4-0.9 points on the register backbones, which is small but contradicts the \"maintaining ID performance\" wording.\n\n3. The abstract says 2-4% OOD improvement, but Table I shows several improvements around 1.5 points (e.g., ViT-G In-R +1.46, In-S +1.53). The FPR reductions are mostly larger than 2-3%. So the headline numbers don't match the tables carefully.\n\n4. There's a garbled sentence in Section IV.A that should have been caught.\n\nNone of these are fatal to the core observation. The direction is consistent across architectures and the method is zero-cost. But the mechanistic story is overinterpreted until the random-patch control is run. The authors should also report multiple seeds and fix the abstract.\n\nThis paper is for practitioners who want a quick feature change on frozen DINOv2; it's not a theoretical contribution. It deserves a proper peer review with a request for the control, seeds, and error bars. I'd engage with it.","headline":"A simple and plausibly useful feature-concatenation recipe for frozen DINOv2, but the missing random-patch control and lack of error bars keep it from being fully convincing.","tokens_in":8797,"tokens_out":3107,"would_cite":true,"duration_ms":28288,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Concatenating CLS with register token means improves OOD accuracy and anomaly rejection across frozen DINOv2 ViTs.","keywords":["vision transformers","register tokens","out-of-distribution generalization","anomaly detection","linear probing","DINOv2","robust adaptation","feature concatenation"],"falsifier":"Train the same linear probes on $[c; \\mu_R]$ and $[c; \\mu_P]$ after randomly shuffling register embeddings across images; if the OOD accuracy gain persists, the effect is not due to register-specific global information. Alternatively, measure whether $\\mu_R$ is linearly separable for ImageNet classes with a rank or mutual-information analysis; if it carries no more class-relevant variance than $\\mu_P$, the claimed mechanism is unsupported.","tokens_in":7843,"feed_emoji":"🛡️","tokens_out":5521,"duration_ms":43300,"temperature":0.7,"pith_summary":"This paper tries to establish that the register tokens introduced to clean up high-norm artifacts in vision transformers are not just a training crutch: their embeddings carry global image information that survives averaging and can be reused at test time. Concretely, it claims that training a linear classifier on the concatenation of the [CLS] token and the mean register embedding yields 2–4% higher top-1 accuracy on ImageNet-A, ImageNet-R and ImageNet-Sketch, and a 2–3% lower false-positive rate in anomaly rejection, across DINOv2 ViT-B/L/G, while in-distribution accuracy stays essentially flat. The attraction is that this is a post-hoc feature choice: the backbone stays frozen, and only the input to the final linear layer changes, so the gains come with no added inference cost.","feed_headline":"Register tokens lift OOD accuracy by 2-4 percent","feed_subtitle":"A frozen-backbone feature swap boosts robustness and anomaly rejection at zero extra inference cost.","key_machinery":"The central object is the average-pooled register embedding $\\mu_R = \\frac{1}{M}\\sum_{k=1}^{M} r_k$, concatenated with the [CLS] token $c$ to form the feature $[c; \\mu_R]$ that feeds a linear classifier. Registers are extra tokens appended to the input sequence during pretraining; prior work showed they absorb high-norm patch tokens and also capture global image-level information. The argument works by exploiting that auxiliary information: replacing the mean patch tokens in the standard $[c; \\mu_P]$ protocol with the mean register tokens gives the linear probe access to a complementary global signal, improving robustness at no extra compute because the backbone is frozen.","core_discovery":"The central discovery is that register token embeddings, which earlier work discarded after pretraining, contain linearly accessible auxiliary information that complements the [CLS] token. On DINOv2 ViT-Base, ViT-Large and ViT-Giant, the paper shows that a linear probe trained on $f_i = [c_i; \\mu_R^i]$ — the CLS token concatenated with the mean of the register tokens — outperforms the standard concatenation $[c_i; \\mu_P^i]$ with the mean patch token on out-of-distribution generalization and anomaly rejection. The comparison holds against both the no-register backbone using $[c; \\mu_P]$ and the register backbone using $[c; \\mu_P]$, while in-distribution accuracy remains comparable. Register tokens alone are not sufficient: a probe trained on $\\mu_R$ alone degrades OOD accuracy, so the benefit is specifically the combination.","pith_inferences":["Inference: if the register mean carries genuinely complementary global information, then learning per-register weights or a small pooling network could beat plain averaging; the paper does not test this.","Inference: the same concatenation trick may transfer to other architectures that append register-like tokens, such as Mamba-R, or to dense tasks where global context matters.","Inference: a causal test of the mechanism would be to remove register tokens from the pretrained backbone and see if the OOD gain vanishes; the paper only compares backbones trained with and without registers, and the without-register backbone cannot produce $\\mu_R$."],"forward_implications":["Linear-probe robustness of frozen vision transformer backbones can be improved by a simple feature change, with no fine-tuning or extra inference cost.","The improvement is consistent across ViT-Base, ViT-Large and ViT-Giant, suggesting it is not an artifact of one architecture.","Anomaly rejection improves under both MSP and energy scoring, so open-set and safety-critical applications inherit the benefit.","Since the linear layer input dimension is the same as for the $[CLS; \\mu_P]$ baseline, deployment memory and compute are unchanged."],"supporting_citations":[{"why":"Introduces register tokens and shows they contain global image-level information, providing the core premise the paper builds on.","marker":"[6]"},{"why":"Shows that combining representations that are redundant for in-distribution data can help under distribution shift, motivating the concatenation strategy.","marker":"[7]"},{"why":"Supplies the frozen DINOv2 backbones and the standard [CLS; mean-patch] linear-probe protocol used as the baseline.","marker":"[8]"},{"why":"Provides the energy-based scoring function used to evaluate anomaly rejection.","marker":"[23]"},{"why":"Provides the maximum softmax probability baseline and the FPR/ROC evaluation framework for anomaly rejection.","marker":"[24]"}],"fun_headline_variants":["CLS + registers lift OOD accuracy by 2-4%","Frozen backbone: combine CLS and registers for OOD gains","Registers add hidden OOD signal, no extra compute","Simple CLS-register fusion boosts anomaly rejection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the average of all register embeddings from a frozen, register-trained checkpoint still contains global image information that is distinct from and complementary to the [CLS] token; that property is imported from earlier work on registers rather than established here.","fun_headline_variants_meta":{"raw":{"variants":["CLS + registers lift OOD accuracy by 2-4%","Frozen backbone: combine CLS and registers for OOD gains","Registers add hidden OOD signal, no extra compute","Simple CLS-register fusion boosts anomaly rejection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000758,"raw_usage":{"total_tokens":3372,"prompt_tokens":951,"completion_tokens":2421,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":2363}},"tokens_in":567,"tokens_out":2421,"duration_ms":16725,"temperature":1.0,"reasoning_tokens":2363,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:25:24.550295+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same linear probes on $[c; \\mu_R]$ and $[c; \\mu_P]$ after randomly shuffling register embeddings across images; if the OOD accuracy gain persists, the effect is not due to register-specific global information. Alternatively, measure whether $\\mu_R$ is linearly separable for ImageNet classes with a rank or mutual-information analysis; if it carries no more class-relevant variance than $\\mu_P$, the claimed mechanism is unsupported.","supporting_citations":[{"cited_title":"Vision transformers need registers","cited_arxiv_id":null,"evidence_quote":"Introduces register tokens and shows they contain global image-level information, providing the core premise the paper builds on."},{"cited_title":"Learning useful representations for shifting tasks and distributions","cited_arxiv_id":null,"evidence_quote":"Shows that combining representations that are redundant for in-distribution data can help under distribution shift, motivating the concatenation strategy."},{"cited_title":"Energy- based out-of-distribution detection","cited_arxiv_id":null,"evidence_quote":"Provides the energy-based scoring function used to evaluate anomaly rejection."}],"review_version":1}