{"id":"3893a7a2-ed36-4214-b4c1-4f833ad6784e","arxiv_id":"2606.25318","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"REViT introduces a discrete roto-reflection equivariant convolutional vision transformer claimed to outperform prior equivariant networks on image classification.","lead":"REViT proposes a vision transformer using convolutional attention that stays equivariant under discrete rotations, reflections, and translations. A smart generalist might read it because orientation-preserving models can reduce reliance on data augmentation in vision tasks where object angle matters.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Unclear whether convolutional attention preserves exact group equivariance for all discrete roto-reflections without additional constraints","rationale":"The reader's weakest assumption directly identifies the same point: whether the discretization plus attention construction actually delivers exact equivariance. The concrete test above would falsify or confirm that assumption with a single, low-cost computation independent of the full experimental suite.","tokens_in":1601,"tokens_out":323,"duration_ms":12493,"concrete_test":"Implement the forward pass of one REViT block, apply a random discrete roto-reflection g to the input, compute the output, apply g to the output, and measure the L2 difference; repeat for all 8 group elements on a 32×32 feature map. If the difference exceeds 1e-5 on any g, equivariance fails and the headline claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the proposed discretization plus convolutional attention yields exact equivariance (not approximate) and that measured gains are attributable to this property rather than other factors. Section 3 describes lifting features to the group and using group convolutions, but the attention mechanism in the transformer blocks (Eq. 4–6) is defined via standard dot-product attention on the lifted features; nothing in the provided equations enforces that the attention weights themselves transform equivariantly under the full dihedral action. If the attention breaks equivariance for reflections or 90° rotations, the model is only partially equivariant and the outperformance claim rests on an unverified assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes REViT, a vision transformer architecture that achieves discrete roto-reflection (dihedral group) equivariance by lifting features to the group, applying group convolutions, and incorporating convolutional attention within transformer blocks. It claims this yields exact equivariance to rotations and reflections while outperforming prior discrete roto-reflection equivariant networks on image classification.","tokens_in":1731,"tokens_out":290,"duration_ms":16192,"significance":"If the architecture delivers exact equivariance (rather than approximate) and the reported gains are shown to stem from the symmetry properties, the work would usefully extend equivariant CNN techniques to the transformer setting, addressing a gap noted in the abstract where most roto-reflection equivariant models have been CNN-based.","major_comments":[{"comment":"Section 3, Eqs. (4–6): the attention mechanism is defined via standard dot-product attention applied to the lifted features. No additional constraints or group-equivariant formulation is described that would ensure the attention weights themselves transform correctly under the full dihedral action (including reflections). Because the central claim requires exact equivariance, this omission is load-bearing; without a proof or explicit verification that attention preserves the group action, the model may only be partially equivariant.","section":"Section 3, Eqs. (4-6)"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their careful review and for highlighting the need to substantiate the exact equivariance claim. We address the single major comment below and will incorporate the requested clarification in a revised manuscript.","responses":[{"response":"We agree that the current description in Section 3 relies on standard scaled dot-product attention applied after lifting the input to the dihedral group and that no separate group-equivariant formulation or proof is supplied for the attention weights under reflections. Because the central claim is exact roto-reflection equivariance, this point requires explicit treatment. In the revision we will add a short lemma (with proof) showing that the overall block remains equivariant: the group-lifted features transform as a regular representation, the convolutional projections that produce queries/keys/values are group convolutions (hence equivariant), and the subsequent softmax-normalized dot-product followed by the value projection preserves the group action because the same linear operations are applied uniformly across all group elements. If the proof reveals that reflections require an additional sign-flip or orientation-reversing adjustment in the attention, we will modify Eqs. (4–6) accordingly and report the change. We will also add a short empirical check (invariance of output under random dihedral transformations on a held-out set) to corroborate the algebraic argument.","revision_made":"yes","referee_comment":"[Section 3, Eqs. (4-6)] Section 3, Eqs. (4–6): the attention mechanism is defined via standard dot-product attention applied to the lifted features. No additional constraints or group-equivariant formulation is described that would ensure the attention weights themselves transform correctly under the full dihedral action (including reflections). Because the central claim requires exact equivariance, this omission is load-bearing; without a proof or explicit verification that attention preserves the group action, the model may only be partially equivariant."}],"tokens_in":1182,"tokens_out":400,"duration_ms":13955,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core move is to lift image features to the discrete roto-reflection group, apply group convolutions, and then run transformer blocks with convolutional attention on those lifted features. It positions this as a simpler route to equivariant ViTs than prior CNN-heavy work and reports stronger classification numbers.\n\nThe construction follows the usual lifting-plus-group-convolution pattern, which is already in the literature for CNNs. The new part is grafting that onto a transformer backbone and calling the attention convolutional. If the experiments are clean and the gains survive ablations, it gives practitioners one more architecture choice when orientation symmetry matters.\n\nThe soft spot is exactly where the stress-test note flags it. The attention is standard dot-product attention on the lifted features; nothing in the description forces the attention weights to transform correctly under 90-degree rotations or reflections. Without an explicit equivariant attention operator or a proof that the whole block commutes with the group action, the model is at best partially equivariant. Any measured improvement could then come from capacity, regularization, or dataset quirks rather than the symmetry property. The abstract gives no equations or training details, so the full paper has to carry the burden of showing both the math and the controls.\n\nThis is niche work for people already building or using equivariant vision models. A reader who needs to test symmetry-preserving backbones might want to try the code if it ships, but the paper will not change how most people train classifiers.\n\nIt is worth sending to peer review so that experts can check whether the attention really preserves the claimed equivariance and whether the comparisons are apples-to-apples.","headline":"REViT adapts group convolutions to a vision transformer for discrete roto-reflections but the attention step likely breaks full equivariance.","tokens_in":2211,"tokens_out":399,"would_cite":false,"duration_ms":16359,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"REViT equips vision transformers with discrete roto-reflection equivariance via convolutional attention and outperforms prior methods on image classification.","keywords":["roto-reflection equivariance","vision transformer","convolutional attention","image classification","discrete group equivariance","symmetry preservation"],"falsifier":"A verification test in which the network outputs change under the group's transformations or classification accuracy fails to exceed that of baseline equivariant models on standard image datasets.","tokens_in":2510,"feed_emoji":"🔄","tokens_out":519,"duration_ms":21078,"temperature":0.7,"pith_summary":"The paper introduces REViT, a vision transformer made equivariant to discrete rotations, reflections, and positions by discretizing the roto-reflection group and inserting convolutional attention. This targets the relative scarcity of equivariant transformers compared with CNN-based designs, while preserving symmetry in feature maps for tasks sensitive to input orientation. The authors outline challenges specific to transformers and present the discretization as a simpler route to exact group equivariance. Experiments show the resulting model exceeds earlier discrete roto-reflection equivariant networks in classification accuracy.","feed_headline":"REViT adds roto-reflection equivariance to vision transformers","feed_subtitle":"Discretized group structure and convolutional attention beat prior equivariant networks on classification tasks.","key_machinery":"Discretized roto-reflection group combined with convolutional attention inside the transformer blocks.","core_discovery":"REViT achieves discrete roto-reflection group equivariance in a vision transformer by combining a discretized roto-reflection group with convolutional attention, preserving rotational, flip, and positional symmetry and delivering higher image classification accuracy than existing discrete roto-reflection equivariant networks.","pith_inferences":["The same discretization approach could be applied to object detection where orientation symmetry matters.","Training may require less rotation-based data augmentation when symmetries are built into the architecture.","Similar discretization strategies might extend to other discrete symmetry groups beyond roto-reflections."],"forward_implications":["Equivariance to rotations and reflections is maintained in feature maps for orientation-sensitive tasks.","Vision transformers can incorporate discrete group equivariance without relying exclusively on convolutional layers.","Performance improvements appear on image classification without additional dataset-specific tuning."],"fun_headline_variants":["REViT achieves roto-reflection equivariance in vision transformers","Discretized group and conv attention enable equivariant ViT in REViT","REViT preserves rotational flip and positional symmetry in feature maps","REViT outperforms prior discrete equivariant networks on classification tasks"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The discretization of the roto-reflection group together with convolutional attention produces exact equivariance and measurable gains without hidden post-hoc adjustments or dataset-specific tuning.","fun_headline_variants_meta":{"raw":{"variants":["REViT achieves roto-reflection equivariance in vision transformers","Discretized group and conv attention enable equivariant ViT in REViT","REViT preserves rotational flip and positional symmetry in feature maps","REViT outperforms prior discrete equivariant networks on classification tasks"]},"model":"grok-4.3","cost_usd":0.005139,"raw_usage":{"total_tokens":2430,"prompt_tokens":534,"num_sources_used":0,"completion_tokens":72,"cost_in_usd_ticks":51387000,"prompt_tokens_details":{"text_tokens":534,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1824,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":534,"tokens_out":72,"duration_ms":13150,"temperature":1.0,"reasoning_tokens":1824,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-25T21:23:41.398064+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A verification test in which the network outputs change under the group's transformations or classification accuracy fails to exceed that of baseline equivariant models on standard image datasets.","supporting_citations":[],"review_version":1}