Pith. sign in

REVIEW 4 major objections 5 minor 30 references

Neuroevolution of Self-Attention Over Proto-Objects

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read The paper claims that replacing rectangular image patches with segmentation-derived proto-objects as attention tokens lets an evolved controller match or beat patch-based agents with 62% fewer parameters and 2.6 times less training time.

desk verdict A genuinely new combination of proto-object tokens and evolved hard attention, but the headline savings are measured against a halved-budget rerun of the baseline, not the published SOTA. read the letter →

arxiv 2505.00186 v1 pith:6HKUSX7H submitted 2025-04-30 cs.NE cs.AIcs.CV

classification cs.NEcs.AIcs.CV
keywords proto-objectshardattentionself-attentionneuroevolutionCMA-ESvisualreinforcementlearningimagesegmentationinterpretableagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that visual attention in reinforcement learning works better when the attention module operates on proto-objects—coherent image regions produced by segmentation—rather than on fixed rectangular patches. It reports that a self-attention module that hard-selects the top-k proto-objects, with the whole pipeline evolved by CMA-ES, matches or exceeds a patch-based baseline on Car Racing and Doom Take Cover while using 62% fewer learnable parameters and training about 2.6 times faster. If correct, the finding means an information bottleneck built from segmentation can carry the task-relevant visual content with far fewer tokens than a patch grid, and the controller can then run on a single attended location per frame. This would make evolved visual attention cheaper and more interpretable without sacrificing performance.

What carries the argument

The load-bearing object is the proto-object token: a color-connected region of the quantized frame, encoded as an 11-dimensional descriptor (RGB color, center of mass, pixel area, bounding-box width, height and area, aspect ratio, and extent). A single $1\times1$ convolution followed by 1-bit-per-channel quantization makes the grouping adaptive, while a residual connection lets evolution start from the trivial segmentation by original color. The attention layer applies PReLU-augmented linear maps to produce queries and keys, builds the standard softmax attention matrix, sums its rows into an importance score, and hard-selects the top-$k$ proto-objects—$k=1$ in most runs. Because segmentation, selection, and coordinate transfer are nondifferentiable, the whole pipeline is optimized jointly by CMA-ES, which evolves the convolution, attention, and LSTM controller together.

What would settle it

Rerun the patch-based model of [22] under its original settings (2000 generations, population 256, 16 seeds) on Car Racing and Doom Take Cover; if its best scores exceed the proto-object scores reported here (910.39 and 1192.82) with statistical significance, the claim of matching or exceeding the published patch-based state of the art fails.

Watch

Extended reading notes

Core claim

The paper claims that the representation of what is attended to is the key efficiency lever in hard-attention agents. Instead of splitting a frame into a fixed $7\times7$ grid of image patches and selecting the top ten, the agent decomposes the quantized image into color-connected regions—proto-objects—and describes each by 11 features such as color, center of mass, area, bounding box, and aspect ratio. That reduces the typical token count from 529 per frame to about 12 in these environments and the token dimension from 147 to 11, which lets the top-k selection run with $k=1$ and an LSTM controller receive only the selected region's $x,y$ coordinates. On Car Racing the proto-object agent scores 910.39 versus 888.69 for the patch baseline ($p = 1.1e-22$); on Doom Take Cover $k=1$ matches the baseline ($p = 0.414$) while $k=10$ exceeds it (1192.82 versus 959.27, $p = 2.8e-5$). The total parameter count drops from 3667 to 1406, and training is 2.6 times faster even though the proto-object system runs on CPU against the patch system's GPU use.

Load-bearing premise

As Section 5 notes, the original patch-based experiments ran with 2000 generations, population 256, and 16 seeds, while the comparison here reruns that baseline with half of each; the central 'matches or exceeds' claim assumes this reduced-budget rerun is a faithful representation of the patch-based method.

Editorial extensions

If this is right

  • A segmentation bottleneck can replace the patch grid: the same task is solved with about twelve tokens per frame instead of 529, and the attention module no longer needs a value matrix $V$ because it only selects tokens rather than mixing them.
  • The controller can operate on a single attended location per frame ($k=1$), with the LSTM supplying memory; this is enough to match or beat the patch-based agent on both tested environments.
  • Because the proto-object system trains 2.6 times faster while running on CPU against the baseline's GPU, the computational savings come from the representation itself and not from faster hardware.
  • Evolved attention over proto-objects produces directly inspectable strategies—such as focusing on the grass region beside the track or on the rightmost monster—so the architecture's interpretability claim from the patch-based line of work carries over.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the token-count collapse is the real driver, this suggests a scaling law for hard attention: any pre-processing that yields semantically compact superpixel tokens should reduce controller size and training time, not just this particular color-connected segmentation.
  • The paper's $k=1$ result implies a minimal-information controller is viable; an obvious extension the authors do not pursue is feeding a short history or a small set of attended coordinates into the LSTM to handle scenes with many relevant entities, where $k=1$ showed a performance dip.
  • The authors note CMA-ES may be too greedy for the coupled attention/controller architecture; comparing against differential evolution under the same budget would test whether escaping local optima of attention strategies is the main obstacle.
  • If the approach transfers to real-world images, the key risk is segmentation quality; testing with adaptive learned quantization or a trainable region proposal stage would show whether the bottleneck's value survives when segments are less clean than in game frames.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes replacing fixed grid patches with proto-objects, defined as color-connected regions obtained after convolution, quantization, and segmentation, as tokens for a hard self-attention module in a visual reinforcement learning agent. The attention module performs top-k selection over proto-objects, whose center-of-mass coordinates are fed to an LSTM controller, and the whole network is evolved with CMA-ES. Section 5 reports experiments on CarRacing and Doom Take Cover: on CarRacing the proto-object agent reaches 910.39 versus 888.69 for a rerun of the patch-based baseline (p=1.1e-22), and on Doom Take Cover it reaches 930.68 with k=1 (p=0.414) or 1192.82 with k=10 (p=2.8e-5) versus 959.27. The abstract concludes that the approach matches or exceeds the state of the art with 62% fewer parameters and 2.6 times less training time.

Significance. The core idea is plausible and the empirical package is above average for a neuroevolution paper: performance is measured in standard environments, Mann-Whitney U tests are reported, and the paper documents how the evolved segmentation changes over generations, which is a genuine interpretability strength. If the comparison with Tang et al. [22] is fair, the results support the claim that segmentation-derived proto-object tokens are a valid substitute for patches in evolved hard-attention agents, and the reduction in token count and parameters is remarkable. The current claims, however, overreach because the patch baseline is a rerun at half the original evolutionary budget and because the headline efficiency figures apply only to the k=1 configuration, while the Doom result that actually exceeds the baseline is the k=10 configuration.

major comments (4)
  1. [Section 5, Table 2] The central claim that the proto-object method 'matches or exceeds the state-of-the-art performance of patch-based implementations' is established only against the authors' own rerun of [22] at half the evolutionary budget (1000 generations, population 128, 8 seeds, versus 2000/256/16 in the original work), as stated in Section 5. CMA-ES solutions are budget-dependent, so it is not demonstrated that the rerun scores of 888.69 and 959.27 are representative of the published [22] state of the art. The manuscript should report the original [22] scores, show that the rerun protocol reproduces them, and provide the rerun configuration including environment versions and hyperparameters; without this evidence, 'matches or exceeds state of the art' is stronger than the data support.
  2. [Abstract, Section 4.2, Section 5.2] The abstract's efficiency headline ('62% less parameters and 2.6 times less training time') is tied to the k=1 configuration of Table 1, whereas the Doom result that actually exceeds the patch baseline is the k=10 variant described in Section 5.2, which uses 2671 parameters, 55 hours, and additional design changes (d_q=4 and 3x3 convolutions). The k=1 Doom result is statistically tied (p=0.414). The abstract and conclusion should separate the two configurations, or state precisely which configuration supports each claim.
  3. [Section 5.2, Figure 7, Table 2] The Doom comparison is ambiguous about which solutions are being compared. Table 2 labels its rows 'Best Score and 95% CI After 1000 Iterations (n=400)', but Figure 7's caption says the patch model peaked at 700 generations and reports p=0.414 'between best solutions'; for the k=10 result it reports 1193 at 700 generations. Please state whether the p-values compare final scores at a fixed generation or each method's individual peak, and report the corresponding scores; the current presentation allows the reader to infer either comparison.
  4. [Section 4.1.3] The hand-designed 11-dimensional segment feature set is a central component of the method, but its choice is justified only by 'thorough experimentation' with no ablation. Without varying the feature set, k, d_q, and PReLU on/off, it is unclear which components drive the reported gains; a small ablation study would make the claims about 'richer semantic information' testable and would strengthen the causal interpretation.
minor comments (5)
  1. [Section 4.1.4] The claim that 'a single PReLU neuron was shown to solve the XOR problem' cites a self-archived paper [17] and does not explain why this property transfers to top-k attention selection; either replace this with an analysis of the actual attention behavior or remove the XOR motivation.
  2. [Figure 3] The flowchart variables M, K and the notation (1, MK) are not defined in the caption or in Section 4.2; please specify all tensor dimensions explicitly.
  3. [Section 5] The statement that seeds are based on generation and repetition numbers is not a reproducible seed protocol; please specify the seed derivation and provide the hyperparameters of [22] that were retained in the rerun.
  4. [Section 5.1] The assertion that CarRacing V0 and V2 have 'no significant differences' is unsupported; if the authors optimized V0 for speed, the 2.6x training-time comparison should disclose the exact environment package and version.
  5. [General] The paper would benefit from a statement on code and data availability; several implementation details (connected-components library, quantization thresholds, test-seed generation) are needed to reproduce the results.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claim is an empirical comparison plus plain arithmetic; the only self-citation is peripheral and non-load-bearing.

full rationale

No circular step can be exhibited. The central quantities are all measured in the environment (scores), counted from architectures (parameters), or clocked (training time); none is obtained by plugging the target result into its own definition. The proto-object features are hand-selected descriptors computed from pixel regions; the attention selects top-k by a softmax row-sum; the controller is an LSTM; CMA-ES maximizes environment reward. There is no equation of the form 'predicted X = fitted X'. The self-citation to [17] (PReLU XOR) is peripheral: it motivates the PReLU nonlinearity in the attention layer, but the paper's headline result does not reduce to that citation, and the PReLU property is independently checkable. The explicit limitation in Section 5 — that Tang et al. ran 2000 generations/256 population/16 seeds and the paper halved these 'due to hardware limitations' — is a real fairness risk in the empirical comparison, but it is not a circularity: the rerun is an external baseline, not an input of the proto-object model. Likewise, 'after thorough experimentation' choosing the 11 segment features is a feature-selection risk, not a fitted parameter renamed as a prediction. Since no load-bearing step reduces to its inputs, the circularity score is minimal; I assign 1.0 only to acknowledge the non-load-bearing self-citation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on several hand-chosen hyperparameters (feature descriptor set, top-k=1, embedding dimension 2, 1-bit quantization, LSTM size 16) and on assumptions that CMA-ES finds good solutions, that color segmentation yields useful proto-objects, and that the rerun baseline is faithful. No new physical or architectural entities are postulated. The feature set and k were tuned on the same tasks, which is a fitting concern but not a circular derivation.

free parameters (6)
  • Segment feature set and dimension (d_in=11) = 11 features: color (R,G,B), center (X,Y), area, bounding box width/height/area, aspect ratio, extent
    Hand-selected after experimentation in Sec 4.1.3; every token in the attention module is this vector.
  • Top-k selection count (k) = 1 for default; 10 for Doom variant
    Set to 1 'to the extreme' in Sec 4.1.4; k=10 used to match the patch setup and get the best Doom result.
  • Attention embedding dimension (d_q) = 2 for default; 4 for Doom k=10 variant
    Hand-set hyperparameter; controls expressiveness of the Q/K projections.
  • Quantization bits per channel = 1 bit per channel (8 colors)
    Fixed in Sec 4.1.2; determines the number and shape of segmentation regions.
  • LSTM hidden size = 16
    Adopted from the patch-based model in Table 1; controls controller capacity.
  • CMA-ES budget = population 128, 1000 generations, 8 evaluation seeds
    Chosen due to hardware limits in Sec 5; the comparison is against a baseline run with the same halved budget.
assumptions (5)
  • domain assumption CMA-ES can reliably optimize the joint attention/control pipeline.
    All results depend on evolutionary search over non-differentiable operations; no convergence guarantees are given (Sec 5).
  • domain assumption Color-connected regions after 1-bit quantization are behaviorally meaningful proto-objects.
    The whole method relies on segmentation producing regions that correspond to track, grass, monsters, etc. (Sec 4.1.3, Figures 5 and 8).
  • domain assumption The 11 hand-picked descriptor features contain enough information for control.
    The controller sees only coordinates of selected proto-objects in the default setup, so the descriptors must be sufficient for attention to rank correctly (Sec 4.1.3 and 4.1.5).
  • ad hoc to paper The rerun patch-based baseline faithfully represents [22] under halved budget.
    Stated in Sec 5; if false, the comparison is unfair. Added by the authors for this paper.
  • standard math Mann-Whitney U test on 400 test seeds is a valid way to compare agent scores.
    Used in Sec 5; appropriate for non-normal, bounded video game returns assuming independent runs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neuroevolution of Self-Attention Over Proto-Objects." pith.science (2026). https://pith.science/paper/6HKUSX7H

@misc{pith2026250500186,
  author       = {Pith},
  title        = {Pith review of: Neuroevolution of Self-Attention Over Proto-Objects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6HKUSX7H}},
  note         = {Machine review of arXiv:2505.00186}
}
read the original abstract

Proto-objects - image regions that share common visual properties - offer a promising alternative to traditional attention mechanisms based on rectangular-shaped image patches in neural networks. Although previous work demonstrated that evolving a patch-based hard-attention module alongside a controller network could achieve state-of-the-art performance in visual reinforcement learning tasks, our approach leverages image segmentation to work with higher-level features. By operating on proto-objects rather than fixed patches, we significantly reduce the representational complexity: each image decomposes into fewer proto-objects than regular patches, and each proto-object can be efficiently encoded as a compact feature vector. This enables a substantially smaller self-attention module that processes richer semantic information. Our experiments demonstrate that this proto-object-based approach matches or exceeds the state-of-the-art performance of patch-based implementations with 62% less parameters and 2.6 times less training time.

Figures

Figures reproduced from arXiv: 2505.00186 by the authors.

Figure 1
Figure 1. Our attentional agent is capable of focusing on [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The agent extracts patches from the current frame [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Flowchart of our complete process. In our exper [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Evolution of segmentation at 5 relevant points in time. The basic attention strategy (focus on the smaller grass region, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The Doom Take Cover environment. The agent has [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Learning curve comparison in the Doom Take Cover [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 17 canonical work pages

  1. [22]

    Yujin Tang, Duong Nguyen, and David Ha. 2020. Neuroevolution of self- interpretable agents. In Proceedings of the 2020 Genetic and Evolutionary Com- putation Conference (Cancún, Mexico) (GECCO ’20). Association for Computing Machinery, New York, NY, USA, 414–424. doi:10.1145/3377930.3389847

  2. [1]

    William Agnew and Pedro Domingos. 2018. Unsupervised Object-Level Deep Reinforcement Learning. Deep Reinforcement Learning Workshop (NIPS 2018)

  3. [2]

    Ali Borji and Laurent Itti. 2012. State-of-the-art in visual attention modeling.IEEE transactions on pattern analysis and machine intelligence 35, 1 (2012), 185–207

  4. [3]

    Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. arXiv:1606.01540 [cs.LG] Figure 8: Processing stages in the Doom Take Cover environ- ment. Top-left: raw image resized to 96x96. Top-right: 1x1 convolution + residual. Bottom-left: quantization. Bottom- right: attention. Note t...

  5. [4]

    Marisa Carrasco. 2011. Visual attention: The past 25 years. Vision Research 51, 13 (2011), 1484–1525. doi:10.1016/j.visres.2011.04.012 Vision Research 50th Anniversary Issue: Part 2

  6. [5]

    Zhe Chen. 2012. Object-based attention: A tutorial review. Attention, Perception, & Psychophysics 74 (2012), 784 – 802. doi:10.3758/s13414-012-0322-z

  7. [6]

    Finkel and Paul Sajda

    Leif H. Finkel and Paul Sajda. 1992. Proto-objects: an intermediate-level visual representation, In Optical Society of America Annual Meeting. Optical Society of America Annual Meeting -, -, FO1. doi:10.1364/OAM.1992.FO1

  8. [7]

    Christophe Fiorio and Jens Gustedt. 1996. Two linear time union-find strategies for image processing. Theoretical Computer Science 154, 2 (1996), 165–181

Show all 30 references
  1. [8]

    Nikolaus Hansen. 2006. The CMA Evolution Strategy: A Comparing Review . Springer Berlin Heidelberg, Berlin, Heidelberg, 75–102. doi:10.1007/3-540-32494- 1_4

  2. [9]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition . In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE Computer Society, Los Alamitos, CA, USA, 770–778. doi:10.1109/CVPR.2016.90

  3. [10]

    Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory.Neural computation 9, 8 (1997), 1735–1780

  4. [11]

    Freed, Michael J

    Kristin Koch, Judith McLean, Ronen Segev, Michael A. Freed, Michael J. Berry, Vijay Balasubramanian, and Peter Sterling. 2006. How Much the Eye Tells the Brain. Current Biology 16, 14 (2006), 1428–1434. doi:10.1016/j.cub.2006.05.056

  5. [12]

    Machado, Erik Talvitie, and Michael Bowling

    Yitao Liang, Marlos C. Machado, Erik Talvitie, and Michael Bowling. 2016. State of the Art Control of Atari Games Using Shallow Reinforcement Learning. In Proceedings of the 2016 International Conference on Autonomous Agents & Multia- gent Systems (Singapore, Singapore) (AAMAS...

  6. [13]

    H. B. Mann and D. R. Whitney. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics 18, 1 (1947), 50 – 60. doi:10.1214/aoms/1177730491

  7. [14]

    Volodymyr Mnih, Nicolas Heess, Alex Graves, and Koray Kavukcuoglu. 2014. Recurrent models of visual attention. In Proceedings of the 28th International Con- ference on Neural Information Processing Systems - Volume 2 (Montreal, Canada) (NIPS’14). MIT Press, Cambridge, MA, USA,...

  8. [15]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602 [cs.LG]

  9. [16]

    Francesco Orabona, Giorgio Metta, and Giulio Sandini. 2007. A Proto-object Based Visual Attention Model. In Attention in Cognitive Systems. Theories and Neuroevolution of Self-Attention Over Proto-Objects Systems from an Interdisciplinary Viewpoint , Lucas Paletta and Erich Ro...

  10. [17]

    Pinto and Anderson R

    Rafael C. Pinto and Anderson R. Tavares. 2024. PReLU: Yet Another Single-Layer Solution to the XOR Problem. arXiv:2409.10821 [cs.NE]

  11. [18]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv:1908.10084 [cs.CL]

  12. [19]

    Ronald A Rensink. 2000. The dynamic representation of scenes. Visual cognition 7, 1-3 (2000), 17–42

  13. [20]

    William Silversmith. 2025. connected-components-3d: Connected Components on Discrete and Continuous Multilabel 3D & 2D Images . GitHub. https://github.com/ seung-lab/connected-components-3d

  14. [21]

    Rainer Storn and Kenneth V. Price. 1997. Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces. Journal of Global Optimization 11 (1997), 341–359. https://api.semanticscholar.org/ CorpusID:5297867

  15. [23]

    Pereira, and William Bialek

    Naftali Tishby, Fernando C. Pereira, and William Bialek. 2000. The information bottleneck method. arXiv:physics/0004057 [physics.data-an]

  16. [24]

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, and Omar G. Younis. 2024. Gymnasium: A S...

  17. [25]

    Martínez Trujillo

    Stefan Treue and Julio C. Martínez Trujillo. 1999. Feature-based attention in- fluences motion processing gain in macaque visual cortex. Nature 399 (1999), 575–579. https://api.semanticscholar.org/CorpusID:4424973

  18. [26]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, ...

  19. [27]

    Vecera and Martha J

    Shaun P. Vecera and Martha J. Farah. 1994. Does visual attention select objects or locations? Journal of experimental psychology. General 123 2 (1994), 146–60. doi:10.1037//0096-3445.123.2.146

  20. [28]

    Dirk Walther and Christof Koch. 2006. Modeling attention to salient proto-objects. Neural Networks 19, 9 (2006), 1395–1407. doi:10.1016/j.neunet.2006.10.001 Brain and Attention

  21. [29]

    Jeremy M Wolfe. 1994. Guided search 2.0 a revised model of visual search. Psychonomic bulletin & review 1 (1994), 202–238

  22. [30]

    William Woof and Ke Chen. 2018. Learning to Play General Video-Games via an Object Embedding Network. In 2018 IEEE Conference on Computational Intelligence and Games (CIG) . IEEE, Maastricht, Netherlands, 1–8. doi:10.1109/ CIG.2018.8490438

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.