{"id":"d456b2a8-9d92-4b98-ba77-ce047b2fcdfc","arxiv_id":"2607.13983","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A vision transformer with thresholded, non-normalized patch relevance (Screening) outperforms a same-scale ViT on ImageNet-1k and CIFAR-100.","lead":"This paper replaces the attention layer in a vision transformer with Screening, a mechanism that scores each image patch independently and discards low-relevance patches. On ImageNet-1k and CIFAR-100, the modified tiny model beats the standard ViT baseline by 4.4 and 2.1 percentage points with fewer parameters.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VisionScreen's accuracy gain over ViT may stem from 2D RoPE and smaller head dim, not Screening—no ablation isolates these changes.","rationale":"The reader's weakest_assumption correctly identifies the absence of an ablation isolating Screening from the other architectural changes. This is the single most load-bearing gap because the paper's thesis is specifically about Screening's efficacy, not about the full VisionScreen model. The reader's CONDITIONAL verdict already reflects this uncertainty, so my stress-test does not suggest a different verdict. I agree with the reader's assessment and can only add precision about the concrete alternative explanation (RoPE is a known performance-booster in ViTs) and the fact that the gate ablation leaves the primary gain unexplained. The recommendation remains CONDITIONAL: the empirical numbers, if reproducible, support that VisionScreen works, but the paper must demonstrate that Screening, not incidental components, drives the improvement.","tokens_in":7427,"tokens_out":4803,"duration_ms":47229,"concrete_test":"Train a 'RoPE-ViT' baseline: ViT-Tiny/16 with the same axial 2D RoPE (§3.2.1) and q/k head dimension 16, but retaining standard softmax attention (no Screening). Use the exact training setup described in §4.1 on ImageNet-1k. If this baseline reaches approximately 72.5% top-1 accuracy (VisionScreen's reported value), then the gains are attributable to RoPE/head-dim rather than Screening. If it stays near the original ViT's 68.1%, the Screening mechanism is more likely responsible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that Screening is effective for visual recognition—rests on Table 1, where VisionScreen outperforms ViT-Tiny/16 on ImageNet-1k and CIFAR-100. However, VisionScreen differs from the baseline in three simultaneous ways: (1) Screening replaces softmax attention, (2) axial 2D RoPE (§3.2.1) replaces learned absolute position embeddings, and (3) the query/key head dimension is reduced from 64 to 16 (§4.2). The only ablation (§4.5, Table 2) removes the gate, which accounts for only 0.7 points of the 4.4-point gain, leaving the dominant source unexplained. Since 2D RoPE is known to improve ViT performance (Heo et al., ECCV 2024), it is entirely plausible that the observed gains arise from RoPE or the head-dimension reduction rather than from Screening's thresholding mechanism. Without a controlled experiment that isolates Screening while holding the other changes fixed, the evidence does not substantiate the qualitative claim that 'Screening can be effective for visual recognition' (§Abstract, §1). This is a load-bearing concern because the paper's purpose is to advocate Screening as an alternative to softmax attention, yet the experimental design fails to separate the contribution of Screening itself.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper adapts the Screening mechanism from language modeling to visual recognition. VisionScreen replaces the softmax attention module in a ViT-Tiny/16 backbone with unit-normalized query-key similarity, a Trim transform, axial 2D RoPE, and a distance-based spatial softmask, with gated aggregation following the original Screening design. Experiments on ImageNet-1k and CIFAR-100 report top-1 accuracies of 72.5% versus 68.1% and 52.4% versus 50.3% respectively, with slightly fewer parameters. The paper also presents interaction visualizations, learned screening-window analyses, and a gate ablation. The central claim is that Screening is an effective alternative to softmax-based relative aggregation for visual recognition.","tokens_in":7777,"tokens_out":5797,"duration_ms":57230,"significance":"If the reported gains were robust and attributable to the Screening mechanism, the result would be a useful addition to the design space of token mixers for vision, showing that non-normalized, thresholded relevance aggregation can be competitive with softmax attention. The paper is clearly written and the training protocol is deliberately simple, which is a sensible way to compare architectures. However, the headline comparison is a systems-level comparison rather than an isolation of Screening, and the quantitative evidence is based on single runs without variance estimates. The manuscript currently provides a plausible demonstration of a competitive architecture, but not a controlled demonstration that Screening causes the observed gains. Reproducibility is also limited by the absence of code and by reliance on an unpublished preprint for core operations.","major_comments":[{"comment":"The central empirical claim is not isolated. VisionScreen differs from the ViT-Tiny/16 baseline in three simultaneous ways: (1) Screening replaces softmax attention, (2) axial 2D RoPE replaces learned absolute position embeddings, and (3) the query/key head dimension is reduced from 64 to 16. The 4.4-point ImageNet gain and 2.1-point CIFAR gain could therefore arise from any of these changes or their interaction. The only ablation, in §4.5, removes the gate and accounts for 0.7 points, leaving the dominant source of the improvement unexplained. Since 2D RoPE alone is known to improve ViT performance (ref. [4]), this confound is load-bearing. Please add controlled experiments: (a) ViT-Tiny/16 with axial 2D RoPE and q/k dimension 16 while keeping softmax attention, to quantify the non-Screening changes; (b) the same configuration with Screening replacing softmax, so the only difference is","section":"§4.2, Table 1; §3.2.1; §4.5"},{"comment":"All quantitative results are single runs with no error bars, no multiple seeds, and no statistical significance testing. The CIFAR-100 gain is 2.1 points and the gate ablation is 0.7 points; with run-to-run variance, these differences may not be meaningful. Because the paper's conclusion is a comparative empirical claim, please report mean and standard deviation over at least three seeds for both datasets and for the ablation. Providing code and trained model checkpoints would also substantially strengthen reproducibility, especially since the paper defines Trim, TanhNorm, and the gate only by reference to the unpublished preprint [9].","section":"§4.2, Table 1; §4.5, Table 2"},{"comment":"The visualizations are used to support the mechanistic narrative that VisionScreen 'explicitly rejects low-relevance patches' and 'is less prone to relying on spurious correlations,' but they show a single Tench image and report no quantitative measure of selectivity. This is not by itself fatal, but it means the paper's claim that improved accuracy is 'accompanied by more selective and semantically meaningful patch aggregation' is only illustrative. A quantitative comparison, such as average attention entropy, overlap of selected patches with a foreground/segmentation mask, or a controlled patch-occlusion experiment, would provide much stronger evidence for the mechanism.","section":"§4.3, Figs. 3–4"}],"minor_comments":[{"comment":"The two-dimensional cosine softmask is introduced without comparison to alternative spatial weighting schemes (e.g., Gaussian decay, hard window, or no spatial mask). If a subsequent ablation isolates Screening, an additional ablation of the spatial softmask would clarify what the 2D extension contributes.","section":"§3.2.2, Eq. (7)"},{"comment":"Core operations from the original Screening mechanism — Trim, TanhNorm, and the gate — are named but not defined. The reader must consult the unpublished reference [9]. Please define these operations explicitly or include a short appendix so the paper is self-contained.","section":"§3.1"},{"comment":"The caption says the selected patch is 'marked with ×' while the text says it is 'marked with a light-blue cross.' Please make the notation consistent.","section":"§4.3, Fig. 4 caption"},{"comment":"The screening window values are reported in coordinate units, but the reader is not told the grid size (14×14 for 224×224 input at patch size 16). Stating the grid dimensions would help interpret values such as 'approximately above 1000.'","section":"§4.4, Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is not ready for acceptance in its current form. The confound in the headline comparison is serious but fixable: with targeted ablations and multi-seed results, the paper could become a solid empirical contribution. I would not reject it outright because the proposed architecture is concrete, the writing is clear, and the basic evaluation protocol is reasonable. The reliance on an unpublished preprint for core definitions also needs to be resolved, either by including the definitions or by making the reference publicly accessible in a citable form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper does something genuinely new: it takes Nakanishi's Screening mechanism, designed for causal language modeling, and adapts it to non-causal 2D image grids. The specific novelties—axial 2D RoPE plus a distance-based spatial softmask, applied to normalized query–key similarity—are a reasonable and honest extension. The authors also run the right kind of basic experiments: ViT-Tiny/16 at the same scale, simple training setup, two benchmarks, and they report loss curves and interaction maps that make the mechanism's behavior visible. The window-size analysis across heads is a nice touch. For a first vision application of Screening, this is a solid piece of work.\n\nThe problem is that the central claim—\"Screening is effective for visual recognition\"—rests on a comparison that changes three things at once. VisionScreen differs from the ViT baseline by (1) replacing softmax with Screening, (2) switching absolute learned position embeddings to axial 2D RoPE, and (3) dropping the q/k head dimension from 64 to 16. The only ablation, in Table 2, removes the gate and accounts for just 0.7 points of the 4.4-point ImageNet gain. That leaves most of the improvement unexplained. It is entirely plausible, as the stress-test note says, that 2D RoPE or the smaller head dimension is doing the heavy lifting. The paper acknowledges these design choices but never isolates Screening itself. Without that controlled comparison, the evidence does not yet support the abstract's conclusion.\n\nThere are also smaller issues: no error bars, no multiple seeds, no code, and no comparison to other attention-replacement models like PoolFormer. The visualizations are suggestive rather than quantitative. None of these are fatal by themselves, but they add to the sense that the paper is under-analyzed.\n\nThat said, the underlying idea is worthwhile and the empirical direction is encouraging. The authors seem to understand the mechanism and are not hiding the confound—they just haven't run the experiment that would clear it up. This is exactly the kind of paper that should go to peer review: a serious referee can push for the missing ablation, multiple seeds, and code release. If the Screening contribution survives an isolated test, this becomes a useful result. If not, the paper still has value as a demonstration that RoPE plus lower-dimensional keys can help ViT. Either way, it deserves reviewer time, not a desk reject.\n\nMy recommendation: send it to review, but make the controlled-ablation requirement explicit.","headline":"First 2D adaptation of Screening is promising, but the empirical claim isn't isolated from RoPE and head-dimension changes.","tokens_in":8214,"tokens_out":1622,"would_cite":false,"duration_ms":18193,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A vision transformer that replaces softmax attention with Screening—independent relevance scoring with threshold rejection—outperforms the softmax baseline on ImageNet-1k and CIFAR-100 while using fewer parameters.","keywords":["Vision Transformer","Screening","absolute relevance","softmax attention","patch selection","image classification","rotary position embedding","spatial softmask"],"falsifier":"Train VisionScreen against ViT-Tiny/16 while changing only the attention operation: keep learned absolute position embeddings and head dimension 64, and replace only softmax with Screening. If the accuracy gain over ViT disappears or reverses, the central claim that Screening itself improves visual recognition is refuted. Alternatively, ablate each of the three differences independently; if removing the RoPE and head-dimension change makes VisionScreen match ViT, Screening alone is not the cause.","tokens_in":7333,"feed_emoji":"🎯","tokens_out":4346,"duration_ms":41736,"temperature":0.7,"pith_summary":"This paper tries to establish that Screening, a token-selection mechanism from language modeling, can replace softmax attention in vision transformers. Screening assigns each query–key pair an absolute relevance score and explicitly discards low-relevance patches via thresholding, rather than forcing a normalized distribution over all patches. The proposed VisionScreen applies this idea to a two-dimensional image grid and reports higher top-1 accuracy than ViT-Tiny/16 on ImageNet-1k (72.5% vs 68.1%) and CIFAR-100 (52.4% vs 50.3%) with fewer parameters. If correct, this would mean softmax's competitive normalization is not necessary for strong visual recognition; independent, thresholded relevance can work better by rejecting background and redundant patches.","feed_headline":"Screening beats ViT on ImageNet and CIFAR-100 with fewer parameters","feed_subtitle":"Absolute relevance thresholds let vision models reject background patches and learn cleaner features.","key_machinery":"The central mechanism is the Screening module: unit-normalized queries and keys yield bounded absolute similarities, a Trim transform and thresholding explicitly zero out low-relevance pairs, and a learnable cosine spatial softmask over Euclidean patch distances further restricts aggregation. Axial 2D rotary position embeddings inject grid coordinates into the similarity computation, and a gating projection modulates the aggregated features. This set replaces softmax's forced competition over all keys with independent per-pair relevance selection, allowing each patch to aggregate only content-wise and spatially relevant patches.","core_discovery":"The paper claims that replacing softmax-normalized self-attention in a ViT with Screening produces more selective patch aggregation and better visual recognition. VisionScreen computes bounded cosine similarities between unit-normalized queries and keys, applies a Trim transform, thresholds low-relevance pairs to zero, and gates the result through a two-dimensional spatial softmask based on Euclidean patch distance, with axial rotary position embeddings providing grid awareness. On ImageNet-1k and CIFAR-100, VisionScreen outperforms ViT-Tiny/16 while using 5.4M parameters versus 5.7M, and the authors present visualization evidence that its interaction maps concentrate on foreground objects r","pith_inferences":["Editorial: The headline comparison changes three components at once—Screening replaces softmax, axial 2D RoPE replaces learned absolute position embeddings, and the query/key head dimension drops from 64 to 16. The 4.4-point ImageNet gain may be partly or wholly due to position encoding or lower-dimensional q/k rather than Screening itself; a per-change ablation would settle this.","Editorial: If Screening's benefit comes from rejecting background patches, the accuracy gap versus ViT should widen on cluttered images and shrink on centered-object images; this is testable on existing benchmarks without new training pipelines.","Editorial: The ability to assign zero relevance to all keys could make representations more robust for inputs where the target object is absent or ambiguous, but the paper does not test this setting.","Editorial: The spatial softmask uses a fixed cosine window per head; a content-dependent or learned window shape might extend the gains, but this is an open design choice beyond the paper."],"forward_implications":["Screening-based attention could make patch interactions more interpretable, since relevance maps visibly concentrate on class-relevant regions rather than spreading weight across the whole image.","The same architecture may transfer to dense prediction tasks such as segmentation and detection, where local and global heads naturally coexist in the learned screening windows.","Because Screening can assign zero weight to all keys, it can represent the absence of relevant information, a capability softmax attention lacks.","Lower-dimensional query and key vectors (head dimension 16) appear sufficient when relevance is based on bounded cosine similarity, reducing parameter count without hurting accuracy.","The learned mixture of local and global screening windows suggests that explicit hierarchical designs may not be necessary to obtain multi-scale receptive fields."],"fun_headline_variants":["VisionScreen: Selective patch attention tops ViT on ImageNet","Screening mechanism boosts ViT on classification benchmarks","Rejecting irrelevant patches: VisionScreen outperforms ViT","ViT upgrade: Absolute relevance thresholds for cleaner features","VisionScreen beats ViT-Tiny on ImageNet, CIFAR-100"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper attributes the accuracy gain to Screening, but VisionScreen differs from the ViT baseline in three simultaneous ways—Screening replaces softmax, axial 2D RoPE replaces learned position embeddings, and the q/k head dimension drops from 64 to 16—so the load-bearing assumption is that none of these other changes, or their interaction, causes the improvement; no ablation isolates Screening alone.","fun_headline_variants_meta":{"raw":{"variants":["VisionScreen: Selective patch attention tops ViT on ImageNet","Screening mechanism boosts ViT on classification benchmarks","Rejecting irrelevant patches: VisionScreen outperforms ViT","ViT upgrade: Absolute relevance thresholds for cleaner features","VisionScreen beats ViT-Tiny on ImageNet, CIFAR-100"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1062,"prompt_tokens":724,"completion_tokens":338,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":253}},"tokens_in":468,"tokens_out":338,"duration_ms":3821,"temperature":1.0,"reasoning_tokens":253,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T03:05:37.736580+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train VisionScreen against ViT-Tiny/16 while changing only the attention operation: keep learned absolute position embeddings and head dimension 64, and replace only softmax with Screening. If the accuracy gain over ViT disappears or reverses, the central claim that Screening itself improves visual recognition is refuted. Alternatively, ablate each of the three differences independently; if removing the RoPE and head-dimension change makes VisionScreen match ViT, Screening alone is not the cause.","supporting_citations":[],"review_version":1}