Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Vision encoders should be image size agnostic and task driven

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vision encoders should spend compute on the task, not on the image's pixel count.

desk verdict An honest, well-scoped position paper with a concrete first step; the 'size-agnostic' claim is about compute, not invariance, and the experiments only show feasibility at one resolution. read the letter →

arxiv 2508.16317 v1 pith:GOSRKXW5 submitted 2025-08-22 cs.CV

Nedyalko Prisadnikov , Danda Pani Paudel , Yuqian Fu , Luc Van Gool This is my paper
classification cs.CV
keywords visionencodersimagesizeagnostictask-drivenfoveatedmulti-zoompatchesiterativetransformerreinforcementlearningpolicyGRPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that the next generation of vision encoders should be image size agnostic and task driven: computing cost should grow with task difficulty, not with image resolution. To show this direction is feasible, the authors build a proof-of-concept image classifier that looks at an image through a small number of fovea-like glimpses. A transformer evolves a learned internal state, a policy decides where to look next, and patches are extracted top-down at resolutions relative to the image, so the number of tokens per step is fixed regardless of image size. On ImageNet the learned gaze policy reaches 65% top-1 accuracy after eight steps, versus 60% for the random-policy pretrained baseline, which the authors read as evidence that the approach works and warrants further research.

What carries the argument

The load-bearing mechanism is the multi-zoom patch: given a gaze center (x,y) in normalized coordinates, the extractor takes M square crops of size min(H,W)/2^z for a geometric series of zoom levels z, resizes each to 16x16, and feeds them as tokens together with N learned state vectors into a ViT-with-registers-style transformer. The state vectors carry memory across iterations while gradients are stopped between steps. A policy network maps the current state to the next (x,y), trained with GRPO in a second stage after the transformer is pretrained with random gaze points.

What would settle it

Take the trained model and run it on the same scene rendered at 224, 1024, and 4096 pixels with the same eight gaze steps; record top-1 accuracy and total FLOPs. If accuracy falls sharply with resolution or compute grows with pixels, the image-size-agnostic property is not supported.

Watch

Extended reading notes

Core claim

The central claim is that biological vision's efficiency comes from selective, task-dependent attention, and that vision encoders should copy this behavior instead of processing every pixel uniformly. The paper proposes a transformer that iteratively receives small sets of multi-zoom patches centered at a gaze point, evolves an internal state across steps, and lets a GRPO-trained policy choose the next gaze point. Patches are defined relative to image dimensions and resized to a fixed token size, so tokens per step stay constant and the encoder is nominally image size agnostic. The ImageNet classification proof-of-concept is offered as evidence that multi-zoom patches work with ViT tokenizer

Load-bearing premise

The whole 'image size agnostic' claim rests on the assumption that a fixed number of foveal glimpses can extract enough information for a task at any image resolution; the paper tests this only at one training resolution, so no experiment actually varies image size.

Editorial extensions

If this is right

  • If image size agnosticism holds, computational cost per image becomes a function of the number of glimpses, not pixel count, so megapixel inputs can be processed at roughly the same cost as small thumbnails.
  • A task-driven encoder could condition its initial state on a text prompt or question, making later glimpses answer-specific rather than fixed features.
  • The iterative state gives a natural memory: the model can accumulate evidence about an image over steps, which suits multi-stage or fine-grained recognition tasks.
  • The proof-of-concept numbers (65% vs 60% baseline at step 8 on ImageNet) suggest even a simple learned gaze policy helps, and that better policies or end-to-end training could close more of the gap to full ViTs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The most direct test the paper leaves open is to measure accuracy and FLOPs on the same image at several resolutions; the authors' position predicts flat cost and stable accuracy, while current ViTs predict both degrade.
  • The task-driven argument transfers most naturally to vision-language models: the text prompt could serve as the task prompt and directly guide the gaze policy, which image classification's single label cannot exercise.
  • Classification may actually understate the value of learned gaze; tasks like finding a small object, reading text, or detecting camouflaged objects should reward selective high-acuity looks much more than class recognition does.
  • If the policy could be made differentiable (e.g., through neural implicit image representations), end-to-end training of transformer and policy might replace the two-stage scheme and unlock large-scale self-supervised pretraining.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This position paper argues that future vision encoders should be image-size agnostic and task-driven, drawing motivation from the efficiency of biological vision. The authors propose a proof-of-concept encoder for ImageNet classification: a transformer is applied iteratively to small contexts, evolving an internal state without backpropagation through time, while a policy selects successive gaze centers. Image content is provided as 'multi-zoom patches' extracted in a top-down manner around each gaze location, with a GRPO-trained gaze policy. Results are reported at 256×256: an iterative transformer over shuffled ViT patches reaches 72% top-1 accuracy versus 78% for ViT-Base, and the full multi-zoom system with the learned policy reaches 65% versus 60% for a random-policy baseline. The paper explicitly frames itself as a position paper and acknowledges that classification is not representative and that policy training remains open.

Significance. If the central claims hold, the paper offers a useful programmatic contribution: a concrete, code-level top-down patch extraction routine (Algorithm 1), a simple no-BPTT iterative transformer recipe, and a GRPO-based gaze-policy training pipeline, all clearly described in the appendix. The authors also state their open questions honestly. However, the empirical support is modest and based on single runs, and the headline 'image size agnostic' property is not directly tested. As it stands, the significance is mostly programmatic; the paper would be substantially stronger if the size-agnostic claim were verified or carefully scoped.

major comments (4)
  1. [§2, §6, Table 2] The central property 'image size agnostic' is never tested as a function of image size. All experiments use ImageNet at 256×256; the fixed compute (8 steps × 16 multi-zoom patches) is by construction, but there is no evidence that the same fixed gaze budget preserves accuracy at other resolutions. §6.1 explains that small images were skipped because they make size-agnostic testing hard, but no larger-resolution experiment is supplied either. Because the paper calls size agnosticism a major contribution, this missing experiment is load-bearing. Please add a resolution sweep (accuracy and compute vs. resolution) or explicitly narrow the claim to 'fixed budget at training resolution'.
  2. [Algorithm 1 / §5.1] The implemented top-down patch extraction is not resolution-invariant. For a patch at center (x,y) and zoom z, the crop size is min(H,W)/2^z; if the input image is scaled by s, the crop size scales by s, so after resizing to 16×16 the token contains different source content. This is a resolution-dependent input distribution shift for the tokenizer and transformer, exactly the kind of bottom-up shift the paper argues against in §2. The statement in §2 that 'if the resolution of the image is higher, then it only gives us an ability to zoom' is not literally true for the implemented system. Please analyze this shift quantitatively or modify the extraction so that the token is truly resolution-normalized.
  3. [Table 2 / §6.2] The reported gain of the learned policy over the random baseline (65% vs. 60% at step 8) is based on a single run. No error bars, seeds, or significance measures are reported, and the random-policy baseline itself likely has high variance. The conclusion that 'the policy does help us to perform well' and the overall feasibility claim rest on this difference. Please report multiple seeds with mean±std, and ideally include a compute-matched comparison to a fixed-glimpse baseline or a standard ViT.
  4. [§6.1 / Table 1] The iterative transformer comparison to ViT (0.72 vs. 0.78) is informative, but the setup is not controlled for compute or parameters: the iterative model processes the same patches in groups with extra state tokens, and the comparison does not match MACs, token counts, or run time. Moreover, the text's claim that validation accuracy strictly increases with step index is unsupported because only final accuracies without variance are shown. Please clarify the compute comparison and add variance statistics.
minor comments (5)
  1. [Abstract/§4] Typo: 'fovean' should likely be 'foveal' in 'benefits of fovean vision' and related phrases. Also, the comma style in 'We, also, provide' is awkward.
  2. [Algorithm 1 / Appendix A.3] Several hyperparameters are not stated: maximum zoom level Zmax, number of Gaussian components K in the policy, the policy's fixed standard deviation, and how zoom levels are scaled for positional embeddings. Please report these to make the experiments reproducible.
  3. [Figure 5 caption] The caption contains grammatical and spacing errors: 'updates patches than are discarded' should be 'updated patches that are discarded', and 'New StateDiscard' should be split into separate words.
  4. [§6.1] The claim that 'in all epochs during training the validation accuracy at step i is always higher than at step j when i > j' needs a precise definition: does this hold for every epoch, or only for the final model? The table alone does not support the temporal claim.
  5. [§5.2] The relationship to ViT with registers and DETR queries should be clarified: state vectors are described as 'added to each layer of the transformer as skip connections', but the textual comparison to DETR is not precise enough to see how the queries interact with the patch tokens across layers.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the proof-of-concept is empirical and the size-agnostic property is a design goal, not a fitted prediction.

full rationale

The paper is a position/proof-of-concept submission; it does not claim to derive a predictive result from a fitted model. The main 'image size agnostic' property is introduced as a design objective (Sec. 2) and realized by fixing the number of iterative steps and multi-zoom patches (Secs. 5.1, 6.2), so constant compute follows from construction rather than from any fitted constant. The reported accuracies (Table 2) compare a GRPO-trained gaze policy against a random-policy pretrained baseline on the same ImageNet benchmark; this is standard empirical validation, not a fitted parameter renamed as a prediction. The paper explicitly acknowledges prior foveated-attention works ([29], [34], [1]) and positions its contribution as a top-down variant, so it does not rename an existing result as new. The only self-citation is [28] (Ma, Paudel, Van Gool) mentioned in Open Questions as a possible future direction; it is not load-bearing for any claim. The skeptical concern about untested resolution invariance is a correctness/empirical limitation, not circularity. No circular steps identified.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The proof-of-concept depends on several hand-chosen hyperparameters (state size, patch count, episode length) and on a set of normative or biological premises that are not independently established. The invented entities are architectural constructs, not physical postulates, so they carry a lighter burden but still lack external evidence.

free parameters (6)
  • N (number of state vectors) = 32 in Sec 6.1, 16 in Sec 6.2
    Chosen by hand; controls memory capacity of the iterative transformer.
  • M (number of multi-zoom patches per gaze) = 16 in Sec 6.2
    Chosen by hand; trade-off between context length and per-step compute.
  • Episode length (iterations) = 4 in Sec 6.1, 8 in Sec 6.2
    Chosen by hand; determines total number of glimpses per image.
  • max_z / Zmax (maximum zoom level) = not reported
    Controls how small the highest-zoom patches are; affects the range of scales seen by the tokenizer.
  • Policy standard deviation = fixed during training
    Set by hand in the Gaussian mixture action distribution (Appendix A.3.3).
  • K (number of Gaussian components in policy) = not reported
    Chosen by hand; the paper only states the policy uses a mixture of K Gaussians.
assumptions (5)
  • domain assumption The goal of vision is not to process and understand every detail, but to extract biologically relevant information.
    Sec 1: this is the philosophical basis for efficiency-driven design, not empirically established.
  • ad hoc to paper It is self-evident that compute should depend on task difficulty rather than context length.
    Sec 2 and Sec 9 call this 'self-evidently desirable'; it is a normative premise, not a theorem.
  • domain assumption Our visual system is inherently selective and requires shifting gaze to build a detailed representation.
    Sec 2: biological analogy used as inspiration. The transfer of this trait to artificial encoders is not proven.
  • ad hoc to paper A transformer can act as a stateful iterative module without backpropagation through time and still accumulate task-relevant information.
    Sec 5.2: the core architectural assumption, tested only indirectly through the ImageNet experiments.
  • ad hoc to paper Task-driven encoders will eventually be better and more efficient than task-agnostic ones.
    Sec 9: the authors admit this is their view, not a demonstrated result.
invented entities (2)
  • Internal state vectors (memory tokens)
    purpose: Maintain an evolving representation of the image across iterative glimpses, used by the task head and policy.
    Sec 5.2: a learned latent construct. It only exists inside the proposed architecture and is evaluated solely through the paper's own ImageNet experiments.
  • (x,y,z) multi-zoom patch coordinate
    purpose: Parametrizes arbitrary square crops relative to image size, enabling top-down, resolution-relative patch extraction.
    Sec 5.1: a new data representation. No external benchmark or independent measurement validates it outside the paper's setup.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vision encoders should be image size agnostic and task driven." pith.science (2026). https://pith.science/paper/GOSRKXW5

@misc{pith2026250816317,
  author       = {Pith},
  title        = {Pith review of: Vision encoders should be image size agnostic and task driven},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GOSRKXW5}},
  note         = {Machine review of arXiv:2508.16317}
}
read the original abstract

This position paper argues that the next generation of vision encoders should be image size agnostic and task driven. The source of our inspiration is biological. Not a structural aspect of biological vision, but a behavioral trait -- efficiency. We focus on a couple of ways in which vision in nature is efficient, but modern vision encoders not. We -- humans and animals -- deal with vast quantities of visual data, and need to be smart where we focus our limited energy -- it depends on the task. It is our belief that vision encoders should be dynamic and the computational complexity should depend on the task at hand rather than the size of the image. We, also, provide concrete first steps towards our vision -- a proof-of-concept solution for image classification. Despite classification being not very representative for what we are trying to achieve, it shows that our approach is feasible and promising.

Figures

Figures reproduced from arXiv: 2508.16317 by the authors.

Figure 1
Figure 1. Overall system architecture. We evolve an initial state (prompt) through iteratively calling [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Example multi-zoom patches over￾layed over each other after being resized to their original size. The center here is sharp from the patches with significant zoom. The top level patch covers the whole image, but it is of very low acuity. use the policy on s2 to get new patches p2. The transformer takes the new patches and updates the state to s3. This process repeats and can continue as long as the task requires. 5 I… view at source ↗
Figure 4
Figure 4. Example of distribution shift with ViT patches when changing the image size. The original size of this image is 2880×2880. On the right we see two patches (of size 32) extracted from a resized image (288 × 288) and from the original image. You can see how much types of details visible change due to resizing. Image Patches State Transformer Stop Gradient Discard New State [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Self-supervised pretraining for an iterative image size agnostic vision transformer

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    A sequential-to-global SSL method based on DINO pretrains iterative foveal-inspired vision transformers to achieve competitive ImageNet-1K performance with constant compute regardless of input resolution.

Reference graph

Works this paper leans on

49 extracted references · 14 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Multiple object recognition with visual attention

    Jimmy Ba, V olodymyr Mnih, and Koray Kavukcuoglu. Multiple object recognition with visual attention. arXiv preprint arXiv:1412.7755, 2014

    arXiv 2014

  2. [2]

    Recurrent memory transformer.Advances in Neural Information Processing Systems, 35:11079–11091, 2022

    Aydar Bulatov, Yury Kuratov, and Mikhail Burtsev. Recurrent memory transformer.Advances in Neural Information Processing Systems, 35:11079–11091, 2022

    2022

  3. [3]

    Unsupervised foveal vision neural networks with top-down attention

    Ryan Burt, Nina N Thigpen, Andreas Keil, and Jose C Principe. Unsupervised foveal vision neural networks with top-down attention. arXiv preprint arXiv:2010.09103, 2020

    arXiv 2010

  4. [4]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213–229. Springer, 2020

    2020

  5. [5]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021

    2021

  6. [6]

    Crossvit: Cross-attention multi- scale vision transformer for image classification

    Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi- scale vision transformer for image classification. In Proceedings of the IEEE/CVF international conference on computer vision, pages 357–366, 2021

    2021

  7. [7]

    Twins: Revisiting the design of spatial attention in vision transformers

    Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, and Chunhua Shen. Twins: Revisiting the design of spatial attention in vision transformers. Advances in neural information processing systems, 34:9355–9366, 2021

    2021

  8. [8]

    Transformer-xl: Attentive language models beyond a fixed-length context

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860, 2019

    arXiv 1901

Show all 49 references
  1. [9]

    Vision transformers need registers

    Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309.16588, 2023

    2023 arXiv

  2. [10]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  3. [11]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human langu...

    2019

  4. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  5. [13]

    Multiscale vision transformers

    Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichtenhofer. Multiscale vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6824–6835, 2021

    2021

  6. [14]

    Levit: a vision transformer in convnet’s clothing for faster inference

    Benjamin Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Hervé Jégou, and Matthijs Douze. Levit: a vision transformer in convnet’s clothing for faster inference. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12259– 12269, 2021

    2021

  7. [15]

    Gmat: Global memory augmentation for transformers

    Ankit Gupta and Jonathan Berant. Gmat: Global memory augmentation for transformers. arXiv preprint arXiv:2006.03274, 2020

    2006 arXiv

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 10

    2016

  9. [17]

    Rethinking spatial dimensions of vision transformers

    Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, and Seong Joon Oh. Rethinking spatial dimensions of vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11936–11945, 2021

    2021

  10. [18]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997

    1997

  11. [19]

    Foveater: Foveated transformer for image classification

    Aditya Jonnalagadda, William Yang Wang, BS Manjunath, and Miguel P Eckstein. Foveater: Foveated transformer for image classification. arXiv preprint arXiv:2105.14173, 2021

    2021 arXiv

  12. [20]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  13. [21]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012

    2012

  14. [22]

    The shape of ai to come! Talk presented at the AI Action Summit 2025, February

    Yann LeCun. The shape of ai to come! Talk presented at the AI Action Summit 2025, February

    2025

  15. [23]

    Gradient-based learning applied to document recognition

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998

    1998

  16. [24]

    Swin transformer v2: Scaling up capacity and resolution

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12009–12019, 2022

    2022

  17. [25]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  18. [26]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019

    2019

  19. [27]

    Biologically inspired deep learning model for efficient foveal-peripheral vision

    Hristofor Lukanov, Peter König, and Gordon Pipa. Biologically inspired deep learning model for efficient foveal-peripheral vision. Frontiers in Computational Neuroscience, 15:746204, 2021

    2021

  20. [28]

    Implicit-zoo: A large-scale dataset of neural implicit functions for 2d images and 3d scenes, 2024

    Qi Ma, Danda Pani Paudel, Ender Konukoglu, and Luc Van Gool. Implicit-zoo: A large-scale dataset of neural implicit functions for 2d images and 3d scenes, 2024

    2024

  21. [29]

    Recurrent models of visual attention

    V olodymyr Mnih, Nicolas Heess, Alex Graves, and Koray Kavukcuoglu. Recurrent models of visual attention. Advances in neural information processing systems, 27, 2014

    2014

  22. [30]

    A focused backpropagation algorithm for temporal pattern recognition

    Michael C Mozer. A focused backpropagation algorithm for temporal pattern recognition. In Backpropagation, pages 137–169. Psychology Press, 2013

    2013

  23. [31]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  24. [32]

    Compressive transformers for long-range sequence modelling

    Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, and Timothy P Lillicrap. Compressive transformers for long-range sequence modelling. arXiv preprint arXiv:1911.05507, 2019

    1911 arXiv

  25. [33]

    The utility driven dynamic error propagation network, volume 11

    Anthony J Robinson and Frank Fallside. The utility driven dynamic error propagation network, volume 11. University of Cambridge Department of Engineering Cambridge, 1987

    1987

  26. [34]

    Learning to generate artificial fovea trajectories for target detection

    Juergen Schmidhuber and Rudolf Huber. Learning to generate artificial fovea trajectories for target detection. International Journal of Neural Systems, 2(01n02):125–134, 1991

    1991

  27. [35]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. 11

    2017 arXiv

  28. [36]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  29. [37]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  30. [38]

    Going deeper with convolutions

    Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015

    2015

  31. [39]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pages 10347–10357. PMLR, 2021

    2021

  32. [40]

    Fixing the train-test resolution discrepancy

    Hugo Touvron, Andrea Vedaldi, Matthijs Douze, and Hervé Jégou. Fixing the train-test resolution discrepancy. Advances in neural information processing systems, 32, 2019

    2019

  33. [41]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  34. [42]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference on computer vision, pa...

    2021

  35. [43]

    Generalization of backpropagation with application to a recurrent gas market model

    Paul J Werbos. Generalization of backpropagation with application to a recurrent gas market model. Neural networks, 1(4):339–356, 1988

    1988

  36. [44]

    Simple statistical gradient-following algorithms for connectionist reinforce- ment learning

    Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8:229–256, 1992

    1992

  37. [45]

    Memformer: A memory-augmented transformer for sequence modeling

    Qingyang Wu, Zhenzhong Lan, Kun Qian, Jing Gu, Alborz Geramifard, and Zhou Yu. Memformer: A memory-augmented transformer for sequence modeling. arXiv preprint arXiv:2010.06891, 2020

    2010 arXiv

  38. [46]

    Neural fields in visual computing and beyond

    Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural fields in visual computing and beyond. In Computer Graphics Forum, volume 41, pages 641–676. Wiley Online Library, 2022

    2022

  39. [47]

    Focal self-attention for local-global interactions in vision transformers

    Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, and Jianfeng Gao. Focal self-attention for local-global interactions in vision transformers. arXiv preprint arXiv:2107.00641, 2021

    2021 arXiv

  40. [48]

    ""Extract for a given zoom level

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017. 12 A Technical Appendices and Supplementary Material The supplementary material is organized in the followin way. Appendix A.1 ...

    2017 arXiv

  41. [2025]

    Retrieved from https://www.youtube.com/watch?v=xnFmnU0Pp-8

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.