{"id":"eb8cfb1e-405f-4b37-a2c1-f94f86088fcb","arxiv_id":"2412.02344","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"UniForm shares one attention matrix across all attention heads and adds multi-scale depthwise convolutions to value projections, claiming faster edge inference at comparable accuracy.","lead":"A proposed \"Reuse Attention\" mechanism computes one shared attention map for all heads in a vision transformer to cut memory traffic on edge devices. The paper claims state-of-the-art ImageNet accuracy and large speedups, but internal tables contradict parts of the claim and no code is provided.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 5x edge speedup is not attributable to Reuse Attention: no controlled latency ablation or code isolates the mechanism from other architecture changes, and the reported timing is internally inconsistent.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the measured speed advantage is attributed to Reuse Attention without a fair, implementation-matched comparison. My analysis confirms this is the decisive soft spot. The architecture changes other than attention are substantial enough that the 5x speedup could plausibly arise from them; without code or a controlled ablation, the paper's flagship result cannot be verified. The internal timing inconsistency between the abstract and Table 6 is a concrete red flag that the measurements were not carefully checked. The paper does contain an interesting idea and some independent support in the form of memory-traffic reasoning and consistent accuracy numbers, but those do not establish the central causal claim. Given the absence of reproducible artifacts and the presence of internal contradictions, rejection is appropriate. If the authors later supply code and a clean A/B latency study, the verdict could be revisited.","tokens_in":16454,"tokens_out":4444,"duration_ms":50168,"concrete_test":"Release code and run a controlled A/B benchmark: take the exact UniForm-t architecture and replace Reuse Attention with standard multi-head attention of the same width and depth, keeping patch embedding, DWConv+FFN placement, training recipe, and inference harness identical. Measure Top-1 accuracy and end-to-end latency on Jetson Nano, Raspberry Pi 4B, and Jetson AGX Orin under the same batch size and thread settings. If the Reuse Attention variant is not consistently and substantially faster, or if the numbers do not reproduce Table 6, the central speedup claim is not attributable to the proposed mechanism. Also verify whether UniForm-l's AGX Orin latency is 2.4ms or 21.8ms.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central empirical claim is that Reuse Attention is what makes UniForm up to 5x faster than EfficientViT on edge devices (Section 4.2.2, Table 6). This attribution is unsupported. UniForm differs from baselines in multiple architectural dimensions beyond the attention mechanism: overlapping patch embedding, DWConv+FFN placement, multi-scale depthwise value convolutions (Eq. 3), and stage channels/depths (Table 4). No code or measurement harness is provided, and the ablation section (4.4) shows only Grad-CAM visualizations and an unquantified assertion of 'significantly improving inference time'; it never reports latency for UniForm with standard attention versus Reuse Attention under identical settings. Moreover, the headline number is internally inconsistent: the abstract says UniForm-l achieves 21.8ms on 'edge devices like the Jetson AGX Orin,' but Table 6 reports 2.4ms on AGX Orin and 21.8ms on Raspberry Pi 5. Without a controlled A/B experiment isolating the attention mechanism, the observed speedup could be due to any other component, the inference framework, or measurement protocol, leaving the strongest claim untested. The abstract also claims reduced computational complexity, although Section 3.4 concedes matrix multiplication complexity remains O(N^2D); that contradiction further weakens the contribution as stated, but the empirical attribution is the load-bearing issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Reuse Attention, a mechanism that computes a single shared attention matrix per transformer layer and reuses it across all heads, with multi-scale depthwise convolutions on value projections to preserve representational diversity. The authors build a hierarchical vision backbone, UniForm, in four sizes, and report ImageNet-1K accuracy, FLOPs, latency on GPUs and edge devices, plus downstream classification and instance segmentation results. The central empirical claim is that Reuse Attention provides up to a 5x speedup over efficient baselines such as EfficientViT on edge devices, while the abstract also claims reduced computational complexity and memory overhead.","tokens_in":16748,"tokens_out":3470,"duration_ms":38654,"significance":"If the central claims were well supported, sharing an attention matrix across heads would be a useful memory-traffic optimization for edge inference, and the proposed multi-scale value processing is a plausible way to retain accuracy. The paper does include a memory-movement accounting (Tables 1 and 2) that makes the intended I/O-reduction argument explicit, and it evaluates across a broad set of edge platforms. However, the key empirical attribution and several headline claims are contradicted by the paper's own content, so the contribution as presented is not yet established. No code or measurement harness is provided, which further weakens the reproducibility of the latency results.","major_comments":[{"comment":"The abstract states that UniForm-l achieves 21.8ms inference time on \"edge devices like the Jetson AGX Orin,\" but Table 6 reports 2.4ms on the Jetson AGX Orin and 21.8ms on the Raspberry Pi 5. This is an internal inconsistency in the paper's most visible claim. The abstract either conflates two different devices or misreports the hardware, and the \"up to 5x speedup\" claim must be tied to a specific comparison on a specific device.","section":"Abstract vs. Table 6"},{"comment":"The abstract and introduction claim that Reuse Attention \"significantly reduc[es] ... computational complexity,\" but Section 3.4 explicitly concedes that \"the matrix multiplication complexity remains O(N^2D).\" The proposed mechanism reduces attention-matrix memory traffic, not the asymptotic FLOP count. The wording of the central contribution is therefore misleading and should be corrected to claim reduced memory I/O or memory overhead rather than reduced computational complexity.","section":"Abstract and §3.4"},{"comment":"The ablation section is the only place where Reuse Attention is compared against a standard-attention variant of UniForm, but it reports only Grad-CAM visualizations. The text asserts that Reuse Attention \"significantly improv[es] inference time\" without providing any latency or throughput numbers for the controlled comparison. Since UniForm differs from baselines such as EfficientViT in patch embedding, DWConv+FFN placement, value convolutions, and stage widths/depths, the 5x speedup attributed to Reuse Attention in Table 6 is not isolated by any experiment in the paper.","section":"§4.4, Ablation study"},{"comment":"The edge-device latency methodology is under-specified. The paper does not report the inference precision (FP32/FP16/INT8), number of warmup iterations, number of repeated trials, or whether the same inference framework and kernel implementations were used across all compared models. Without this information, the cross-model latency numbers in Table 6 cannot be treated as controlled measurements, and the speedup ratios derived from them are not reliable.","section":"§4.2.2 and Table 6"}],"minor_comments":[{"comment":"The sentence \"In this section, we ablate\" is incomplete; it should introduce the ablation experiments and their settings.","section":"§4.4"},{"comment":"The caption contains the typo \"prodyced\" and claims that UniForm with Reuse Attention \"can more precisely locate the objects of interest than other methods,\" which is not supported by the qualitative Grad-CAM figures alone.","section":"Figure 7 caption"},{"comment":"The phrase \"Similar to previous Like previous hierarchical backbones\" is a duplicated-word error and should be rewritten.","section":"§3.5"},{"comment":"There are typos such as \"buit\" for \"built\" and \"plateforms\" for \"platforms,\" and the section would benefit from a careful proofread.","section":"§4.1"},{"comment":"The star-rating comparison of attention mechanisms is subjective and lacks a defined rubric; consider replacing it with quantitative metrics or removing it.","section":"Table 3"},{"comment":"The downstream classification paragraph reports qualitative trends but does not describe the fine-tuning protocol for CIFAR, Flowers-102, and Oxford-IIIT Pet, making the results hard to reproduce.","section":"§4.3.1"}],"recommendation":"reject","confidential_remarks":"The manuscript has several internal contradictions and incomplete sections, which suggests it was not ready for peer review. The novelty relative to prior attention-map reuse work (e.g., Shim et al., 2023) is not clearly delineated, and the lack of code or a reproducible measurement harness makes the empirical claims difficult to verify. If the authors address the attribution and measurement issues in a thorough revision, a resubmission could be considered, but in its current form the central claims are not supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the paper's central empirical claim — that Reuse Attention is what makes UniForm up to 5x faster than EfficientViT on edge hardware — is not supported by its own experiments. The abstract is also internally inconsistent: it says \"reduced computational complexity\" while Section 3.4 concedes matrix multiplication stays O(N^2D), and it attributes 21.8ms to Jetson AGX Orin while Table 6 reports 2.4ms on AGX Orin and 21.8ms on Raspberry Pi 5.\n\nWhat is actually new is limited. Sharing one attention matrix across heads is already present in multi-query attention, cascaded group attention, and the attention-map reuse work of Shim et al., all cited in the paper. The useful addition is the multi-scale depthwise value processing (Eq. 3), a straightforward but sensible way to give heads diverse receptive fields while keeping a single attention map. The paper also deserves credit for the unusually broad edge benchmark: ten devices from Raspberry Pi 2B to Jetson AGX Orin, with batch size and threading specified. That table is genuinely useful to practitioners if the numbers are measured carefully.\n\nThe problems run deeper than the abstract. The ablation (Section 4.4) shows only Grad-CAM visuals and an unquantified claim of improved inference time. There is no A/B test that swaps Reuse Attention for standard attention under identical architecture and reports latency. Instead, Table 6 compares entire models that differ in patch embedding, DWConv+FFN placement, value convolutions, and stage widths/depths, so the 5x speedup cannot be attributed to the mechanism. Also, the paper claims state-of-the-art accuracy, but its own Table 7 lists EfficientViT-M5 at 77.1% versus UniForm-l at 76.7%. No code is provided, which makes these measurements even harder to trust.\n\nI would not call the paper incoherent: the body is more honest than the abstract, and the idea is a reasonable incremental variant. But the load-bearing claim is untested, and the internal contradictions need fixing.\n\nFor a referee: I'd send it out, because the edge benchmark and the multi-scale value trick are worth examining, and a thoughtful referee could get the authors to run the missing controlled latency experiment and correct the abstract. As submitted, the central claim should not be accepted.\n\nHope that helps.","headline":"The shared-attention idea is not new and the headline speedup is not isolated by the experiments, but the broad edge-device benchmark and the honest body text make it a borderline case that needs a controlled ablation and an abstract rewrite.","tokens_in":17239,"tokens_out":2724,"would_cite":false,"duration_ms":28107,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shared attention matrix, reused across heads, is enough to run vision transformers on edge devices at up to 5x the speed of comparable baselines, without giving up accuracy.","keywords":["reuse attention","vision transformer","edge inference","memory efficiency","ImageNet classification","multi-scale value processing","on-device speedup","efficient attention"],"falsifier":"Take the exact UniForm-l configuration and replace only the shared attention matrix with standard per-head attention, leaving every other component (patch embedding, DWConv+FFN, value convolutions) unchanged; measure inference time on the same edge devices and batch sizes. If the time on Jetson AGX Orin rises to the same level as EfficientViT-M4 or above, the causal claim that reuse produces the speedup is falsified.","tokens_in":16242,"feed_emoji":"⚡","tokens_out":5856,"duration_ms":57149,"temperature":0.7,"pith_summary":"The paper argues that the main obstacle to running vision transformers on edge devices is the memory traffic caused by multi-head attention, and that this obstacle can be removed by computing a single shared attention matrix per layer rather than one matrix per head. It introduces Reuse Attention, a mechanism that reuses that one matrix across all heads and compensates for the lost per-head diversity with multi-scale depthwise convolutions on the value projections. The authors build UniForm, a family of hierarchical vision transformers around this mechanism, and report ImageNet-1K top-1 accuracy of up to 76.7% with inference times as low as 2.4ms on Jetson AGX Orin and 21.8ms on Raspberry Pi 5, claiming up to 5x speedups over baselines such as EfficientViT-M0 while also beating them in accuracy. If correct, the mechanism makes real-time transformer-based vision practical on memory-constrained hardware without sacrificing accuracy.","feed_headline":"Shared attention matrix speeds edge vision up to 5x","feed_subtitle":"UniForm computes one attention map per layer, cutting memory traffic while keeping ImageNet accuracy at 76.7% on edge devices.","key_machinery":"The load-bearing object is the shared attention matrix $A = \\mathrm{softmax}(QK^\\top/\\sqrt{D})$, computed once per layer from a single query/key projection and reused by every head; the paper's argument is that this cuts memory traffic for the attention scores from $N^2 \\times h$ (per-head) to a single $N^2$, which matters because edge devices are memory-bandwidth-bound. The complementary mechanism is multi-scale value processing: each head applies a depthwise convolution of a different kernel size to its value projection before multiplying by $A$, which the authors claim preserves the representational diversity that multi-head attention would otherwise provide.","core_discovery":"The central claim is that per-head attention matrices are redundant computation, so a single shared attention matrix per layer preserves model performance while drastically cutting memory load and store. Given input $X$, UniForm computes one query/key projection, forms $A = \\mathrm{softmax}(QK^\\top/\\sqrt{D})$, and applies this same $A$ to every head's value output $V_h$, where $V_h$ is produced by a depthwise convolution with a distinct kernel size. The authors state this replaces $h$ separate $N \\times N$ attention matrices with one, reducing memory movement (e.g., from 141.73 GB to 8.59 GB for a Llama-scale model in their extrapolation) and enabling faster inference on GPUs, CPUs, and edge devices. They support this with ImageNet-1K classification, downstream classification, and COCO instance segmentation results showing accuracy on par with or better than comparably sized efficient models.","pith_inferences":["If the shared-matrix mechanism is what drives the speedup, the same design could be applied to language-model inference, where repeated attention-matrix reads dominate memory traffic; the paper's Table 2 extrapolates this to LLMs and VLMs, but reports no language-model measurements, so that extension needs a direct test.","The paper changes several architectural components at once (patch embedding, DWConv+FFN placement, multi-scale value convolutions), so the 5x speedup may not be solely attributable to the shared attention matrix; a controlled ablation that swaps only the attention module would isolate the mechanism's true contribution.","The memory-traffic reductions for models like Llama 2 and GPT-3 are computed from the reuse formula rather than measured, so the 93-94% reduction figures should be read as upper bounds until benchmarked on those architectures.","Since the shared matrix is computed from a single pair of query/key projections, the mechanism is also a parameter reduction of the attention module; this may reduce overfitting in small-data fine-tuning regimes, which the paper's Flowers-102 results hint at but do not analyze."],"forward_implications":["UniForm-l reaches 76.7% ImageNet-1K top-1 accuracy with 2.4ms inference on Jetson AGX Orin, showing transformer-based classification can run in real time on edge hardware.","Reuse Attention reduces attention memory movement from $N^2 \\times h$ to $N^2$ per layer, so the benefit grows with token count and head count, making it attractive for high-resolution inputs and larger models.","Smaller UniForm variants (UniForm-t, 66.0% top-1) beat EfficientViT-M0 (63.2%) in accuracy and run up to ~5x faster on Jetson-Nano (11.9ms vs 56.8ms), implying the efficiency gain is not bought by accuracy loss.","The same backbone transfers to COCO instance segmentation with Mask R-CNN, where UniForm-l achieves 33.2 box AP and 31.5 mask AP, outperforming comparable efficient backbones listed in the paper.","On CPU and GPU throughput, UniForm-s reaches 50,582 images/s on A100 and 231 images/s on CPU, exceeding MobileNetV3-small and EfficientViT-M1, so the mechanism is not edge-specific."],"supporting_citations":[{"why":"Supplies the memory-I/O argument that minimizing transfers between memory and compute improves efficiency, which Reuse Attention positions itself against.","marker":"[7]"},{"why":"Provides the primary baseline (EfficientViT) and the observation that attention maps across heads are similar, motivating reuse.","marker":"[21]"},{"why":"Shows many heads contribute little, supporting the claim that computing separate attention per head is redundant.","marker":"[27]"},{"why":"Demonstrates that Value projections carry more critical information than Query/Key, motivating retention of full-dimensional values while sharing the attention matrix.","marker":"[39]"},{"why":"Shows synthetic or fixed attention patterns can maintain performance, supporting the idea of not computing fresh attention per head.","marker":"[26]"},{"why":"Inspires the multi-scale depthwise convolution kernels used on value projections.","marker":"[36]"}],"fun_headline_variants":["One attention matrix per layer cuts edge vision memory and speeds it 5x","UniForm's reuse attention hits 76.7% ImageNet accuracy at 21.8ms on edge","Shared attention: up to 5x faster vision transformers on edge devices","Reuse attention slashes memory traffic, boosting edge inference 5x","UniForm: one attention map per layer, 5x speedup on edge hardware"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed speedups assume the comparisons are fair and implementation-matched, because UniForm also changes patch embedding, DWConv+FFN placement, and value convolutions; if these other changes carry most of the speedup, the reuse mechanism itself may contribute far less than the 5x figure suggests.","fun_headline_variants_meta":{"raw":{"variants":["One attention matrix per layer cuts edge vision memory and speeds it 5x","UniForm's reuse attention hits 76.7% ImageNet accuracy at 21.8ms on edge","Shared attention: up to 5x faster vision transformers on edge devices","Reuse attention slashes memory traffic, boosting edge inference 5x","UniForm: one attention map per layer, 5x speedup on edge hardware"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000352,"raw_usage":{"total_tokens":1924,"prompt_tokens":957,"completion_tokens":967,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":857}},"tokens_in":573,"tokens_out":967,"duration_ms":8799,"temperature":1.0,"reasoning_tokens":857,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:33:41.318536+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the exact UniForm-l configuration and replace only the shared attention matrix with standard per-head attention, leaving every other component (patch embedding, DWConv+FFN, value convolutions) unchanged; measure inference time on the same edge devices and batch sizes. If the time on Jetson AGX Orin rises to the same level as EfficientViT-M4 or above, the causal claim that reuse produces the speedup is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows many heads contribute little, supporting the claim that computing separate attention per head is redundant."},{"cited_title":"Fu, Stefano Ermon, Atri Rudra, and Christopher R ´e","cited_arxiv_id":null,"evidence_quote":"Supplies the memory-I/O argument that minimizing transfers between memory and compute improves efficiency, which Reuse Attention positions itself against."},{"cited_title":"Efficientvit: Memory effi- cient vision transformer with cascaded group attention","cited_arxiv_id":null,"evidence_quote":"Provides the primary baseline (EfficientViT) and the observation that attention maps across heads are similar, motivating reuse."},{"cited_title":"Global vision trans- former pruning with hessian-aware saliency","cited_arxiv_id":null,"evidence_quote":"Demonstrates that Value projections carry more critical information than Query/Key, motivating retention of full-dimensional values while sharing the attention matrix."},{"cited_title":"Delight: Deep and light-weight transformer","cited_arxiv_id":null,"evidence_quote":"Shows synthetic or fixed attention patterns can maintain performance, supporting the idea of not computing fresh attention per head."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Inspires the multi-scale depthwise convolution kernels used on value projections."}],"review_version":1}