{"id":"87fe5fa9-4019-456d-b52b-d6ccbfffdbab","arxiv_id":"2504.16691","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"EET speeds up vision transformers for fine-grained image retrieval by pruning background tokens and using teacher-student distillation to preserve accuracy, cutting latency by 42.7% with little or no drop in retrieval quality.","lead":"This paper presents EET, a vision transformer for image search that cuts out unimportant image patches during processing, reducing computing time by about 43 percent while keeping search accuracy. It adds a training trick where a slower teacher model teaches the faster model where to look, which is useful for systems that search huge photo collections.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hash-learning update equations in Section IV-D are algebraically wrong as written, so the described method cannot reproduce the reported retrieval numbers without correction.","rationale":"The reader's weakest assumption is that the CTP importance score may prune informative tokens. I agree this is a real risk, and the paper's own ablation (Table V) and Figure 8 show CTP alone degrades accuracy, especially on Stanford Cars. However, the final EET framework with DKT and DRG outperforms the ViT-Small baseline on every dataset in Table V (e.g., NABirds 16-bit 58.30 to 63.45), so the central efficiency-accuracy claim is not falsified by the pruning heuristic. The more load-bearing issue is formal: the hash optimization updates in Section IV-D are written incorrectly. Eq. (14) uses the wrong inverse, and Eq. (16) has the wrong sign for the B-subproblem. These equations define how the target hash codes B are produced; if a reader implements the text literally, the training procedure is not the one that yields the reported mAP. The availability of code mitigates this, and the empirical claims may well be correct, but the paper as written does not support reproduction. This does not change the conditional verdict; it sharpens the condition: correct the algebra or point explicitly to the code as the authoritative implementation.","tokens_in":20485,"tokens_out":13082,"duration_ms":119256,"concrete_test":"Independently re-derive the P and B updates from Eq. (13) using matrix calculus. If the correct P-update is P = YV^T(VV^T)^{-1} and the correct B-subproblem is max_B Tr(B^T R V), then Eqs. (14) and (16) are wrong. Then run the released training code with Eq. (14) literally implemented (YV^T(YV^T)^{-1}) on CUB-200-2011 16-bit; if the run diverges or the code path differs from the paper, the reported 79.85 mAP cannot be attributed to the method as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that EET is both faster and more accurate than ViT-Small. The accuracy side depends on the hash-code learning procedure in Section IV-D. Eq. (14) states P = YV^T(YV^T)^{-1} when minimizing ||Y - PV||_F^2; the normal equations give P = YV^T(VV^T)^{-1}. As written, the formula is dimensionally invalid unless C = k and is wrong in general. Eq. (16) then minimizes Tr(B^T(RV)), but minimizing ||B - RV||_F^2 is equivalent to maximizing Tr(B^T(RV)), so the sign is flipped. These are not cosmetic typos: they are the exact updates used to generate the hash codes B that EET is trained to match, and they make the method non-reproducible from the text. The released code may implement the correct algebra, but the paper's formal description does not support the reported mAP numbers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EET, an efficient ViT-based hashing framework for large-scale fine-grained image retrieval. EET consists of three components: content-based token pruning (CTP), which progressively removes low-discriminative tokens at fixed transformer layers using attention weighted by per-head output-content norms; discriminative knowledge transfer (DKT), which aligns the student hash codes with a heavier teacher ViT; and discriminative region guidance (DRG), which masks salient regions during training to force attention to subtler cues. The hash codes are learned through an alternating proxy-based optimization following FISH, and only the pruned student is used at inference. Experiments on CUB-200-2011, Stanford Cars, NABirds, VegFru, Food101, and iNat2017 report mAP, PR curves, GFLOPs, and latency, with the headline result that EET reduces ViT-Small latency by 42.7% and improves 16-bit NABirds mAP from 58.30 to 63.45.","tokens_in":20715,"tokens_out":5329,"duration_ms":52105,"significance":"If the reported results hold, EET is a practically useful combination of token pruning and distillation for fine-grained hashing: it is the first ViT-based method in this comparison to address both the quadratic attention cost and the fine-grained discriminative bottleneck, and the paper includes publicly available code, ablation studies, latency measurements, and experiments on six datasets. The central efficiency claim is supported by Table V, where CTP cuts latency from 0.82 ms to 0.47 ms. However, the significance is currently reduced by two technical problems in the hash-code optimization (Eqs. (14) and (16)), by the omission of the strongest competitor CMBH from the main comparison tables, and by the absence of variance reporting for hyperparameters tuned on a single dataset. With corrections and a more transparent comparison, this would be a solid contribution to the fine-grained retrieval and efficient-ViT literature.","major_comments":[{"comment":"The closed-form update for P is algebraically incorrect. Minimizing ||Y - PV||_F^2 with fixed V gives P = YV^T(VV^T)^{-1}, not P = YV^T(YV^T)^{-1}. As written, the formula is dimensionally invalid unless C = k and is wrong in general. Since the optimized P and the subsequent alternating updates produce the hash-code target B used in Eq. (17), this error makes the described method non-reproducible from the text; please correct the equation and verify the derivation.","section":"Section IV-D, Eq. (14)"},{"comment":"Minimizing ||B - RV||_F^2 over B is equivalent to maximizing Tr(B^T(RV)), not minimizing it. The sign in Eq. (16) is therefore flipped; B = sign(RV) is the argmax, not the argmin. This is not cosmetic: the B matrix is the training target for L_hash in Eq. (17), so the displayed optimization would generate inverted hash codes. Please fix the sign or clarify that Eq. (16) is intended as a maximization.","section":"Section IV-D, Eq. (16)"},{"comment":"CMBH is omitted from the main comparisons (Tables I–III) and appears only in Table IV, where it outperforms EET on nearly every entry (e.g., CUB 16-bit 84.07 vs. 75.02; VegFru 24-bit 88.63 vs. 86.94; Food101 32-bit 89.28 vs. 88.31). The statement in Section V-C3 that EET 'slightly lags behind CMBH' understates these gaps. The paper should either include CMBH in the headline tables or explicitly qualify the state-of-the-art claim.","section":"Tables I–IV and Section V-C3"},{"comment":"The hyperparameters beta, sigma, and the pruning ratios are tuned on CUB-200-2011 and then applied to all other datasets, but the paper reports no standard deviations or repeated runs. Since the main cross-dataset claims (especially NABirds and the large-scale benchmarks) depend on these choices, please report variance across seeds and show at least one sensitivity analysis on a second dataset to support the claim that the settings transfer.","section":"Section V-B and Figures 4–5"}],"minor_comments":[{"comment":"There is a typo in the text: 'As aforementioned, the the FGIR task' should read 'the FGIR task'.","section":"Section IV-C2"},{"comment":"The heading 'Ablatuion Studies' should be spelled 'Ablation Studies'.","section":"Section V-C4 heading"},{"comment":"The sentence 'The learning process of the hash code is divided into two steps' is repeated verbatim twice in consecutive lines; please remove the duplicate.","section":"Section IV-D"},{"comment":"The notation for pruning ratios is inconsistent: Section IV-B uses N_omega, Section V-B uses gamma_j, and Table VIII uses (N1, N2, N3). Please unify the notation.","section":"Section IV-B and Table VIII"},{"comment":"The identity hamm(b_i,b_j) = (k/2)(1 - cos(b_i,b_j)) is exact only for binary vectors b_i, b_j; the approximation with continuous hash codes h_i, h_j should be stated more carefully, since the cosine distance between continuous vectors does not exactly equal the Hamming distance after sign binarization.","section":"Section IV-C1, Eq. (9)"},{"comment":"The rotation matrix R is declared as R in R^{r x r}, but it multiplies V in R^{k x n}; please define r = k or replace r with k throughout the optimization.","section":"Section IV-D, Eq. (13)"},{"comment":"The order of the heads in b_q = sign(FChash(FCcls(E_q_class))) appears reversed relative to the figure and the text, which describe the hash head as a projection of the final class token; please confirm whether FC_cls is applied before FC_hash or whether this is a typo.","section":"Section IV-F, Eq. (20)"}],"recommendation":"major_revision","confidential_remarks":"The hash-learning equations are central to the paper's reproducibility, and the CMBH comparison affects the strength of the SOTA claim; both need to be addressed before publication. The publicly available code is a positive signal, and the latency/accuracy trade-off is interesting, so I do not see a reason to reject outright. However, I would ask the editor to verify that the authors actually correct the equations in the text rather than only pointing to the code."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful engineering contribution—pruned ViT plus distillation and region masking gives a real speed-accuracy improvement over the ViT-Small baseline—but the hash-learning update equations in Section IV-D are algebraically wrong as written, so the method as described cannot be reproduced from the text. That's a load-bearing flaw, not a cosmetic typo.\n\nWhat's actually new: the CTP weighting in Eqs. (6)-(8), where per-head attention output norms are used to weight the class-attention scores, is a modest variation of EViT/Evo-ViT pruning, but the extension to fine-grained retrieval with the DKT and DRG transfer objectives is new. The ablations in Table V are the strongest part: they show CTP alone drops Stanford Cars mAP, and DKT plus DRG recovers and exceeds the baseline. The 42.7% latency reduction is believable from the table.\n\nNow the soft spots. Eq. (14) states P = YV^T(YV^T)^{-1} when minimizing ||Y - PV||_F^2; the correct closed form is YV^T(VV^T)^{-1}. Eq. (16) minimizes Tr(B^T(RV)), but the equivalent of minimizing ||B - RV||_F^2 is maximizing that trace, so the sign is flipped. Both are in the hash-code learning section, which is the part that generates the binary codes EET is trained to match. If the released code does the right thing, the paper's numbers may be fine, but the formal description is not reproducible. I'd call this a major revision issue.\n\nAlso, CMBH appears in Table IV and clearly beats EET on CUB, VegFru, and Food101 at comparable GFLOPs, yet it is absent from Tables I-III. The paper says it is \"competitive\" with CMBH, which is fair, but leaving it out of the main comparison buries that result. No error bars are reported, and beta/sigma are tuned on CUB and applied to all datasets; their own sensitivity plots are reassuring but not a substitute.\n\nBottom line: the central efficiency claim holds up, and the accuracy claim is plausible but overstated without CMBH in the main tables and with the algebra errors. The paper deserves peer review—the idea is solid enough—but it needs a corrected Section IV-D and an honest presentation of CMBH before it can be considered reliable.","headline":"Solid efficiency story for fine-grained hashing, but Section IV-D's hash update equations are wrong as written and the CMBH comparison is buried.","tokens_in":21217,"tokens_out":3583,"would_cite":true,"duration_ms":32346,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EET shows a pruned ViT-Small can cut fine-grained retrieval latency by 42.7% while improving 16-bit hash mAP on NABirds from 58.30 to 63.45.","keywords":["fine-grained image retrieval","vision transformer","token pruning","learning to hash","knowledge distillation","content-based token importance","efficient inference","binary hash codes"],"falsifier":"Take a fine-grained dataset with pixel-level annotations of discriminative regions (or use the paper's own failure images) and compute recall@kept: the fraction of annotated discriminative patches among the tokens EET retains at each pruning stage. If recall is low yet an oracle pruner that keeps the annotated patches scores higher mAP at equal latency, the CTP importance score is not the cause of the accuracy gain and the method's central mechanism is falsified.","tokens_in":20307,"feed_emoji":"⚡","tokens_out":7475,"duration_ms":65889,"temperature":0.7,"pith_summary":"The paper claims that the main barrier to deploying Vision Transformers in large-scale fine-grained image retrieval is not the architecture itself but the computation wasted on background and low-discriminative tokens. It proposes EET, a framework that progressively prunes those tokens and, during training only, transfers discriminative knowledge from a full ViT teacher into the pruned student, so inference stays fast. On six fine-grained benchmarks, EET reports ViT-Small inference latency cut by 42.7% (from 0.82 ms to 0.47 ms) while improving 16-bit hash-code mAP on NABirds from 58.30 to 63.45, surpassing the unpruned baseline and several dedicated hashing methods. If correct, the claim means efficiency and accuracy do not have to be traded off in fine-grained retrieval hashing.","feed_headline":"Pruning ViT tokens speeds fine-grained retrieval by 42.7%","feed_subtitle":"A token-pruning plus distillation model lifts NABirds 16-bit hash mAP from 58.30 to 63.45 at lower latency.","key_machinery":"The engine of the method is the content-weighted token-importance score $M^l = \\sum_{h=1}^{H} W^{h,l} \\cdot A^{h,l}$, where $A^{h,l}$ is the class-token attention to each patch in layer $l$ and $W^{h,l}$ is the L2 norm of that head's output content normalized across heads. CTP keeps the top $N_\\omega \\cdot \\mathrm{len}(M^l)$ tokens at layers 4, 8, and 10, progressively discarding background and low-discriminative patches. Two loss terms repair what pruning removes: DKT minimizes $1 - \\cos(h_e, h_d)$ between the pruned student's hash codes and the teacher's, and DRG masks the most salient image region so the student is forced to learn secondary discriminative cues. The hash codes themselves come from a proxy-based two-step optimization borrowed from FISH, so the paper's contribution is the efficiency-and-distillation loop around a standard hash learner.","core_discovery":"On its own terms, the paper's central discovery is that a ViT's redundant tokens can be productively removed rather than tolerated: ranking tokens by an attention score weighted by the L2 norm of each head's output content identifies discriminative patches, and a training-time distillation pair—cosine-distance hash-code transfer from the teacher plus a masked-image region-guidance loss—restores and even exceeds the accuracy lost to pruning. The result is a hashing model (EET) that, at 16-bit code length, raises NABirds mAP from 58.30 to 63.45 and cuts latency from 0.82 ms to 0.47 ms relative to ViT-Small, and that matches or beats the unpruned baseline across CUB-200-2011, Stanford Cars, VegFru, Food101, and iNat2017 at multiple code lengths.","pith_inferences":["The CTP ranking heuristic has not been shown to generalize to objects whose discriminative parts are subtle and low-response; the paper's own Figure 8 shows it failing on car fronts, headlights, and logos, and its ablations show CTP alone dropping Stanford Cars 16-bit mAP from 82.85 to 79.96. A natural next test is replacing the L2-content weight with a learned saliency predictor and measuring whe","DKT aligns continuous hash embeddings, not logits or intermediate features, so the teacher-student pair is architecture-asymmetric; this suggests the same transfer could run from a CNN or a larger ViT teacher into any efficient student, which the paper does not test.","DRG masks the globally most salient region rather than the per-class discriminative region; making the mask class-aware or adaptively sized could reduce the background-confusion failures the paper reports and is a direct, cheap ablation to try.","The paper frames progressive pruning as mimicking human global-to-local attention, but the mechanism is simpler: conditional computation plus distillation. If the distillation terms were removed, the model would be just a faster, weaker ViT, which is exactly what the ablation table shows."],"forward_implications":["A pruned ViT-Small can replace the full ViT-Small in fine-grained hashing pipelines: latency drops from 0.82 ms to 0.47 ms with retrieval mAP equal or better on the reported datasets.","The speedup is not paid for in hash-code quality: EET's gains over the baseline appear at 16-, 32-, 48-, and 64-bit codes, and the largest relative gain is at 16 bits on NABirds (+5.15 mAP).","Pruning positions and ratios matter hierarchically: inserting CTP at layers 4, 8, and 10 with retention ratios 1/2, 1/2, and 1/4 gives the best accuracy-efficiency balance, while pruning only at layer 10 is nearly as accurate but slower.","Content-weighted class attention is a better token-ranking signal than raw class attention for fine-grained images; replacing it with the raw score lowers mAP on all three ablation datasets.","Because DKT and DRG are training-only, the discriminative transfer is cost-free at inference, so the efficiency gain of pruning is kept end-to-end."],"supporting_citations":[{"why":"Defines the Vision Transformer backbone and patch-token representation that EET prunes and distills.","marker":"[14]"},{"why":"Provides the ViT-Small baseline model whose latency and retrieval performance EET is compared against.","marker":"[15]"},{"why":"Introduces DynamicViT, the token-sparsification paradigm that CTP extends from coarse-grained to fine-grained images.","marker":"[23]"},{"why":"Supplies the cosine-similarity-to-Hamming-distance relation used to define the DKT distillation loss.","marker":"[8]"},{"why":"FISH contributes the proxy-based two-step hash-code learning objective that EET reuses for hash learning.","marker":"[42]"},{"why":"DVF is the strongest ViT-based fine-grained retrieval baseline; EET is benchmarked against it as the main accuracy competitor.","marker":"[5]"},{"why":"Provides the knowledge-distillation paradigm on which the discriminative knowledge transfer is built.","marker":"[63]"},{"why":"EViT uses class-attention scores for token pruning, the approach CTP modifies by weighting attention with content magnitude.","marker":"[57]"}],"fun_headline_variants":["Token pruning cuts ViT latency 42.7% for fine-grained search","Prune ViT tokens: 42.7% faster, 5.15% better hash retrieval","Efficient ViT: prune tokens, distill knowledge, improve fine-grained retrieval","42.7% latency cut for ViT fine-grained retrieval with token pruning","Prune tokens, not accuracy: ViT for fast fine-grained search"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that ranking image patches by the L2 magnitude of each attention head's output content, folded into class-attention scores, reliably picks out the subtle discriminative regions of fine-grained objects; if that ranking prunes informative tokens, retrieval accuracy degrades, and the paper's own ablations and Figure 8 show exactly such failures on Stanford Cars and background-cluttered images.","fun_headline_variants_meta":{"raw":{"variants":["Token pruning cuts ViT latency 42.7% for fine-grained search","Prune ViT tokens: 42.7% faster, 5.15% better hash retrieval","Efficient ViT: prune tokens, distill knowledge, improve fine-grained retrieval","42.7% latency cut for ViT fine-grained retrieval with token pruning","Prune tokens, not accuracy: ViT for fast fine-grained search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001443,"raw_usage":{"total_tokens":5863,"prompt_tokens":1044,"completion_tokens":4819,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":4713}},"tokens_in":660,"tokens_out":4819,"duration_ms":32831,"temperature":1.0,"reasoning_tokens":4713,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:57:48.021624+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fine-grained dataset with pixel-level annotations of discriminative regions (or use the paper's own failure images) and compute recall@kept: the fraction of annotated discriminative patches among the tokens EET retains at each pruning stage. If recall is low yet an oracle pruner that keeps the annotated patches scores higher mAP at equal latency, the CTP importance score is not the cause of the accuracy gain and the method's central mechanism is falsified.","supporting_citations":[{"cited_title":"Training data-efficient image transformers & distillation through attention,","cited_arxiv_id":null,"evidence_quote":"Provides the ViT-Small baseline model whose latency and retrieval performance EET is compared against."},{"cited_title":"Dynamicvit: Efficient vision transformers with dynamic token sparsification,","cited_arxiv_id":null,"evidence_quote":"Introduces DynamicViT, the token-sparsification paradigm that CTP extends from coarse-grained to fine-grained images."},{"cited_title":"One loss for all: Deep hashing with a single cosine similarity based learning objective,","cited_arxiv_id":null,"evidence_quote":"Supplies the cosine-similarity-to-Hamming-distance relation used to define the DKT distillation loss."},{"cited_title":"Fine-grained hashing with double filtering,","cited_arxiv_id":null,"evidence_quote":"FISH contributes the proxy-based two-step hash-code learning objective that EET reuses for hash learning."},{"cited_title":"Dvf: Advancing robust and accurate fine-grained image retrieval with retrieval guidelines,","cited_arxiv_id":null,"evidence_quote":"DVF is the strongest ViT-based fine-grained retrieval baseline; EET is benchmarked against it as the main accuracy competitor."}],"review_version":1}