{"id":"a4a9d56d-64a9-4d27-bc6d-9af229bd068c","arxiv_id":"2501.00243","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Adding cross-layer aggregation and a cross-layer cache to token-reduced vision transformers improves accuracy at 10% keep rates, enabling cheaper ultra-fine-grained image recognition.","lead":"This paper adds two small modules to vision transformers: a head that combines features from several network layers, and a cache that restores dropped image patches after token pruning. The result is a large accuracy gain at very low token budgets, which could make fine-grained recognition cheaper in low-compute settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Cross-Layer Cache mechanism is never ablated separately, so the claimed accuracy recovery at low keep rates could be driven entirely by the CLA head rather than by cache reinjection.","rationale":"The reader correctly identified the cached-feature premise as a weak spot, but the more precise and more damaging gap is that the cache's causal contribution is never separately measured. The paper's strongest claim names the Cross-Layer Cache mechanism as the method that counteracts token-reduction information loss; without an isolated ablation, the observed improvements are consistent with the CLA head alone providing deep supervision over intermediate CLS tokens, which is a well-established effect in fine-grained recognition. I therefore route the concern through correctness risk rather than through internal inconsistency or disagreement with consensus. The paper does provide broad experimental coverage and the aggregate consistency across backbones and token-reduction schemes is genuine evidence that the combined CLCA helps, but aggregate improvement does not identify which component matters. The proposed four-way ablation would settle the question: if the CLC-only increment is negligible, the abstract and Section V-C overstate the role of the cache and the method's novelty should be repositioned; if the full-model increment over CLA-head-only is real and consistent, the mechanism-specific claim is supported. Until that check is run, a conditional verdict is appropriate because the central mechanism claim is not yet established, even though the combined method appears empirically useful.","tokens_in":7920,"tokens_out":6583,"duration_ms":70924,"concrete_test":"Run a four-way ablation on one representative setting, e.g., DeiT3-B/16 on SoyGene with EViT token reduction at KR=10% and KR=25%, using the same 50-epoch training budget and three seeds: (1) EViT baseline, (2) EViT plus CLA head only, (3) EViT plus CLC only (cached GAP and CLR tokens appended after reduction, with the original classification head), and (4) EViT plus full CLCA. Report per-seed top-1 accuracies and FLOPs. If accuracy(4) - accuracy(2) is within run-to-run noise while accuracy(2) - accuracy(1) is large, the CLC mechanism is not responsible for the claimed recovery and the central claim should be revised.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central mechanism-specific claim is that the Cross-Layer Cache (CLC) recovers information lost under token reduction and thereby enables 10% keep rates. However, every reported comparison introduces the CLA head and the CLC together, along with the extra parameters of both modules, so the individual contribution of the CLC is never isolated. Section III-C motivates the cache by appeal to FFVT, but Section V reports no accuracy ablation with CLC alone or CLA head alone; Fig. 4 only shows a gradient trace, which does not establish that the cache, rather than deep supervision through the CLA head, is responsible for the accuracy recovery. The CLA head alone is essentially a cross-layer aggregation of intermediate CLS tokens, which is known to help fine-grained recognition via deep supervision and feature fusion. It is therefore possible that at KR=10% all of the recovered accuracy comes from that head, while the cached GAP/CLR reinjection adds nothing or even introduces noise. If so, the novel 'cache recovery' component of CLCA is not validated, and the method reduces to a classification-head change with a different explanation. This is load-bearing because the cache is the part of the method that is claimed to counteract token-reduction information loss specifically.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Cross-Layer Cache Aggregation (CLCA), two add-on modules for Vision Transformers under token reduction for ultra-fine-grained image recognition: a Cross-Layer Aggregation (CLA) head that fuses CLS-token outputs from encoder groups via batch norm, depth-wise convolution, and point-wise convolution, and a Cross-Layer Cache (CLC) that stores global-average-pooled local features and a learnable Cross-Layer Register token after each encoder block and re-injects them after token reduction. The authors evaluate CLCA across 5 UFGIR leaf datasets, 9 pretrained backbones, 7 token-reduction methods, 5 keep rates, and 2 resolutions, reporting that it consistently improves accuracy and allows a 10% token keep rate with competitive accuracy at much lower FLOPs.","tokens_in":8186,"tokens_out":4622,"duration_ms":44579,"significance":"If the claims hold, CLCA is a practically useful, plug-and-play recipe for making aggressive token reduction viable in fine-grained recognition, with unusually broad empirical coverage (2000+ runs, multiple TR paradigms, code released). The main strengths are the breadth of settings and the reproducible code link. However, the paper does not isolate the contribution of the Cross-Layer Cache from the Cross-Layer Aggregation head, and the main state-of-the-art table lacks error bars and contains mixed results, so the central mechanism-specific claim is not yet established.","major_comments":[{"comment":"Table I reports CLCA(10%) on Cotton as 55.6, below CSDNet's 57.9, and CLCA(10%) on SoyGlobal as 61.1, below SIM-Tr's 70.7; because the table has no error bars or standard deviations despite Section IV stating that results are averaged over three seeds, the reader cannot judge whether these gaps are significant or whether the 'competitive accuracy' claim holds at 10% KR across datasets. Please report per-seed results or standard deviations and state precisely on which datasets the 10% accuracy is competitive.","section":"Section V-B, Table I"},{"comment":"No component-level ablation is reported: every CLCA configuration combines the CLA head and the CLC, so the accuracy recovery at low keep rates cannot be attributed to the cache. Fig. 4 is a gradient-magnitude trace, not an ablation, and it does not test the CLC mechanism. Please add experiments with (i) base TR model, (ii) base + CLA head only, (iii) base + CLC only, and (iv) base + both, at least for EViT on SoyLocal/SoyGene at KR=10%, 25%, and 50%; this is load-bearing because Section III-C and the abstract credit the CLC specifically with counteracting token-reduction information loss.","section":"Section III-C, Fig. 4"},{"comment":"The explanation of why CLCA works is qualitative: the claim that gradients are 'significantly unstable' before adding the modules is not quantified, and the analogy to skip connections and deep supervision does not distinguish the CLC from the CLA head. Please either support the gradient claim with a quantitative comparison (e.g., gradient norm statistics with and without each module) or restrict the discussion to what the experiments actually show.","section":"Section III-C"}],"minor_comments":[{"comment":"The phrase 'We refer to our full method method' contains a duplicated word; please correct it.","section":"Section I"},{"comment":"The word 'specially' is used where 'especially' is intended in several places; please correct them.","section":"Throughout"},{"comment":"The caption calls the gradient trace an 'Ablation'; rename it to 'Gradient analysis' to avoid confusion, since it does not ablate any component.","section":"Fig. 4 caption"},{"comment":"The sentence 'inspired by previous which suggests' is missing the noun 'work'; please revise it.","section":"Section III-C"},{"comment":"The CLC description does not state whether cached tokens receive positional embeddings or how their order is fixed when appended to the reduced sequence; please clarify this implementation detail.","section":"Section III-C"},{"comment":"The paper claims 'more than 2000 runs' but does not give an exact count or a per-condition breakdown; please add a run count/breakdown or remove the number.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"This is an empirical systems paper with broad coverage and released code, which fits an applied computer vision venue. The main load-bearing gap is the missing component ablation separating the Cross-Layer Cache from the CLA head; if the authors can show that CLC alone (without the CLA head) improves accuracy at low keep rates, I would be satisfied. I would also ask the editor to verify the '2000 runs' claim in the final version, since the manuscript does not provide a count or per-condition breakdown. No other scope or attribution issues arose beyond those noted in the minor comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a decent, well-scoped efficiency paper. The combination of a cross-layer aggregation head and a cross-layer cache is new, and the empirical sweep is genuinely broad: 5 datasets, 9 backbones, 7 token-reduction methods, 5 keep rates. The headline result—accuracy largely held at 10-25% keep rates with CLCA—is supported by the aggregated distributions in Fig 3, and it matters for mobile/agricultural settings. I believe the central claim: the CLCA combo (both modules together) is a reliable plug-and-play improvement.\n\nThe soft spots are real but not fatal. The paper never ablates the cache separately. Every comparison introduces the CLA head and the CLC together, so you cannot attribute the recovered accuracy to the cache. The stress-test note is right: the CLA head alone is basically deep supervision over intermediate CLS tokens, which is known to help fine-grained tasks. The gradient trace in Fig 4 is suggestive but does not separate the two modules. That means the paper's more specific claim—that the cache recovers information lost during token reduction—is currently under-supported. This is fixable with one ablation table (head alone, cache alone, both) with error bars.\n\nTwo smaller issues: Table I reports means over three seeds but no variance, and the per-dataset results are mixed (Cotton at 10% is close to a plain DeiT; SoyGlobal is below SIM-Tr). The 'why it works' section leans on skip-connection intuitions; that is fine, but it does not add quantitative evidence. The citation pattern is fine; prior work (FFVT, EViT, registers) is properly credited, and the components are honestly presented as inspired by existing ideas.\n\nWho benefits: anyone combining ViTs with token reduction on fine-grained or ultra-fine-grained tasks. It is not a field-reorganizing paper, but it is a useful, reproducible engineering contribution. I would send it to peer review; it deserves a serious referee. The review should ask for the component ablation and error bars. If the authors provide them, this becomes a solid accept at a mid-tier venue; without the ablation, the mechanism claim should be softened.","headline":"Solid, well-scoped efficiency paper for UFGIR; the CLCA combo clearly helps, but the paper never isolates the cache, so the headline 'cache recovers lost information' is under-supported.","tokens_in":8692,"tokens_out":2627,"would_cite":false,"duration_ms":26768,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that caching pre-reduction features and feeding intermediate-layer CLS tokens to the classifier lets token-reduced vision transformers keep competitive ultra-fine-grained accuracy at keep rates as low as 10%.","keywords":["ultra-fine-grained image recognition","token reduction","vision transformer","cross-layer cache","feature aggregation","efficient inference","fine-grained visual categorization","register tokens"],"falsifier":"Run EViT at 10% keep rate on SoyGene with full CLCA, with the cache disabled, and with the cache populated by post-reduction features instead of pre-reduction features; if accuracies are nearly equal across all three, the claim that the cache recovers lost information is refuted.","tokens_in":7748,"feed_emoji":"⚡","tokens_out":6115,"duration_ms":57005,"temperature":0.7,"pith_summary":"Ultra-fine-grained image recognition needs high-resolution inputs and attention to tiny visual differences, which makes the quadratic cost of vision transformer attention painful. Token reduction cuts that cost by dropping or merging image-patch tokens, but it also throws away exactly the small details these tasks depend on. This paper tries to recover that lost information with two plug-and-play modules: a Cross-Layer Cache that stores pooled local features and a learned register token before reduction and re-inserts them afterwards, and a Cross-Layer Aggregation classification head that routes intermediate CLS features straight into the classifier. Across more than 2000 runs, the combined method, CLCA, consistently improves accuracy over the token-reduced baseline and lets the keep rate fall to 10% while staying competitive with state-of-the-art ultra-fine-grained models. If this works, it provides a cheap way to make efficient vision transformers accurate on detail-critical tasks.","feed_headline":"Cached features rescue token-reduced vision transformers at 10% keep rate","feed_subtitle":"Two plug-in modules keep ultra-fine-grained accuracy while slashing compute to a tenth of the tokens.","key_machinery":"The load-bearing mechanism is the pair of cross-layer connections. The Cross-Layer Cache (CLC) stores, after each transformer encoder block, the global average pooling (GAP) of local patch tokens and the output of a learnable Cross-Layer Register (CLR) token appended at the end of the sequence; after a reduction point these stored tokens are appended to the reduced sequence, then the cache is emptied to avoid repetition. The Cross-Layer Aggregation (CLA) Head takes the CLS tokens from the end of each encoder group, applies batch normalization, a depth-wise convolution that mixes channels across layers, a nonlinearity, and a point-wise convolution that outputs class predictions. Together they reintroduce information that token reduction discarded and expose intermediate features to the classifier with negligible extra computation.","core_discovery":"The central claim is that the information lost when a vision transformer drops tokens can be recovered by preserving compact summaries of pre-reduction features and by letting the classifier see intermediate layers directly. Concretely, after every encoder block the model caches a global-average-pooled vector of local patch tokens plus a learnable Cross-Layer Register token appended to the sequence; after token reduction at layers 4, 7, and 10, and before the final layer, these cached tokens are appended back into the stream. In parallel, the Cross-Layer Aggregation head collects the CLS token from each encoder group, normalizes them, mixes them channel-wise with a depth-wise convolution, and projects the result to class scores. The authors argue these connections act as skip connections that smooth optimization and give the classifier access to low-, middle-, and high-level features, which is exactly what fine-grained discrimination needs. The experimental claim is that CLCA raises the accuracy floor of token-reduced models across five datasets, nine backbones, and seven token-reduction schemes, making 10% keep rates viable.","pith_inferences":["A direct extension would test the same cache-and-reinject recipe on other detail-critical tasks, such as medical image classification or document understanding, where aggressive token pruning also risks discarding the decisive pixels.","The design leaves open which cache entry matters more: a run that keeps only the GAP features, or only the CLR token, would show whether the learned register is essential or whether pooled local statistics alone can carry the lost information.","If 10% keep rates become reliable, the practical cost bottleneck for ultra-fine-grained recognition may move from transformer FLOPs to data-loading and memory bandwidth, so system-level co-design becomes the next lever.","The gradient-instability discussion suggests CLCA may also stabilize training of token-reduced transformers from scratch, which could be tested by comparing convergence curves with and without the modules under identical random seeds."],"forward_implications":["Attaching CLCA to a token-reduced ViT consistently improves accuracy across seven token-reduction schemes, so the recovery mechanism does not depend on how tokens are selected for removal.","With CLCA, the token keep rate can be dropped to 10%, cutting FLOPs sharply while keeping accuracy competitive with much costlier state-of-the-art methods (for example, CLCA at 10% keep rate uses 25.2 GFLOPs on Cotton versus 447.9 GFLOPs for TransFG).","The gains hold across nine pretrained backbones, including self-supervised and weakly supervised ones, indicating the modules are a general fix rather than a re-tuning of one architecture.","Because the added modules cost little extra compute, CLCA shifts the accuracy-versus-cost frontier of ultra-fine-grained recognition toward much cheaper high-resolution inference."],"supporting_citations":[{"why":"Supplies the five leaf datasets and the ultra-fine-grained benchmark that every accuracy number is measured on.","marker":"[2]"},{"why":"Defines the vision transformer architecture whose token sequence CLCA modifies.","marker":"[4]"},{"why":"Motivates aggregating low- and middle-level features for fine-grained classification and directly inspires the Cross-Layer Cache.","marker":"[6]"},{"why":"EViT is the primary token-reduction method with which CLCA is tested most extensively.","marker":"[15]"},{"why":"DynamicViT provides the static-pruning token-reduction scheme used to show CLCA is method-agnostic.","marker":"[10]"},{"why":"ATS provides the dynamic-pruning scheme used to show CLCA works across reduction paradigms.","marker":"[11]"},{"why":"ToMe provides a hard-merging token-reduction baseline in the cross-scheme comparison.","marker":"[13]"},{"why":"Motivates the learnable Cross-Layer Register token that the cache stores and re-inserts.","marker":"[20]"},{"why":"DeiT III is the backbone used in the headline accuracy-versus-FLOPs plots on the SoyGene dataset.","marker":"[17]"}],"fun_headline_variants":["Cache and reuse: token reduction without fine-grained loss","Plug-in cache keeps 10% token rate for fine-grained vision","Cross-layer cache rescues token-reduced image recognition","Token-saving cache preserves ultra-fine-grained accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pre-reduction GAP-pooled local features and the learned register token still contain the fine details the reduced sequence has lost, and that appending them after reduction neither misaligns the sequence nor injects noise.","fun_headline_variants_meta":{"raw":{"variants":["Cache and reuse: token reduction without fine-grained loss","Plug-in cache keeps 10% token rate for fine-grained vision","Cross-layer cache rescues token-reduced image recognition","Token-saving cache preserves ultra-fine-grained accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1419,"prompt_tokens":1033,"completion_tokens":386,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":321}},"tokens_in":649,"tokens_out":386,"duration_ms":4536,"temperature":1.0,"reasoning_tokens":321,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:54:42.391207+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run EViT at 10% keep rate on SoyGene with full CLCA, with the cache disabled, and with the cache populated by post-reduction features instead of pre-reduction features; if accuracies are nearly equal across all three, the claim that the cache recovers lost information is refuted.","supporting_citations":[{"cited_title":"Benchmark Platform for Ultra-Fine-Grained Visual Categorization Beyond Human Perfor- mance,","cited_arxiv_id":null,"evidence_quote":"Supplies the five leaf datasets and the ultra-fine-grained benchmark that every accuracy number is measured on."},{"cited_title":"Feature Fusion Vision Transformer for Fine-Grained Visual Categorization","cited_arxiv_id":"2107.02341","evidence_quote":"Motivates aggregating low- and middle-level features for fine-grained classification and directly inspires the Cross-Layer Cache."},{"cited_title":"EViT: Expediting Vision Transformers via Token Reorganizations,","cited_arxiv_id":null,"evidence_quote":"EViT is the primary token-reduction method with which CLCA is tested most extensively."},{"cited_title":"DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification,","cited_arxiv_id":null,"evidence_quote":"DynamicViT provides the static-pruning token-reduction scheme used to show CLCA is method-agnostic."},{"cited_title":"Vision Transformers Need Registers,","cited_arxiv_id":null,"evidence_quote":"Motivates the learnable Cross-Layer Register token that the cache stores and re-inserts."}],"review_version":1}