REVIEW 4 major objections 5 minor 30 references
Neuroevolution of Self-Attention Over Proto-Objects
T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read The paper claims that replacing rectangular image patches with segmentation-derived proto-objects as attention tokens lets an evolved controller match or beat patch-based agents with 62% fewer parameters and 2.6 times less training time.
desk verdict A genuinely new combination of proto-object tokens and evolved hard attention, but the headline savings are measured against a halved-budget rerun of the baseline, not the published SOTA. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the proto-object token: a color-connected region of the quantized frame, encoded as an 11-dimensional descriptor (RGB color, center of mass, pixel area, bounding-box width, height and area, aspect ratio, and extent). A single $1\times1$ convolution followed by 1-bit-per-channel quantization makes the grouping adaptive, while a residual connection lets evolution start from the trivial segmentation by original color. The attention layer applies PReLU-augmented linear maps to produce queries and keys, builds the standard softmax attention matrix, sums its rows into an importance score, and hard-selects the top-$k$ proto-objects—$k=1$ in most runs. Because segmentation, selection, and coordinate transfer are nondifferentiable, the whole pipeline is optimized jointly by CMA-ES, which evolves the convolution, attention, and LSTM controller together.
What would settle it
Rerun the patch-based model of [22] under its original settings (2000 generations, population 256, 16 seeds) on Car Racing and Doom Take Cover; if its best scores exceed the proto-object scores reported here (910.39 and 1192.82) with statistical significance, the claim of matching or exceeding the published patch-based state of the art fails.
Extended reading notes
Core claim
The paper claims that the representation of what is attended to is the key efficiency lever in hard-attention agents. Instead of splitting a frame into a fixed $7\times7$ grid of image patches and selecting the top ten, the agent decomposes the quantized image into color-connected regions—proto-objects—and describes each by 11 features such as color, center of mass, area, bounding box, and aspect ratio. That reduces the typical token count from 529 per frame to about 12 in these environments and the token dimension from 147 to 11, which lets the top-k selection run with $k=1$ and an LSTM controller receive only the selected region's $x,y$ coordinates. On Car Racing the proto-object agent scores 910.39 versus 888.69 for the patch baseline ($p = 1.1e-22$); on Doom Take Cover $k=1$ matches the baseline ($p = 0.414$) while $k=10$ exceeds it (1192.82 versus 959.27, $p = 2.8e-5$). The total parameter count drops from 3667 to 1406, and training is 2.6 times faster even though the proto-object system runs on CPU against the patch system's GPU use.
Load-bearing premise
As Section 5 notes, the original patch-based experiments ran with 2000 generations, population 256, and 16 seeds, while the comparison here reruns that baseline with half of each; the central 'matches or exceeds' claim assumes this reduced-budget rerun is a faithful representation of the patch-based method.
Editorial extensions
If this is right
- A segmentation bottleneck can replace the patch grid: the same task is solved with about twelve tokens per frame instead of 529, and the attention module no longer needs a value matrix $V$ because it only selects tokens rather than mixing them.
- The controller can operate on a single attended location per frame ($k=1$), with the LSTM supplying memory; this is enough to match or beat the patch-based agent on both tested environments.
- Because the proto-object system trains 2.6 times faster while running on CPU against the baseline's GPU, the computational savings come from the representation itself and not from faster hardware.
- Evolved attention over proto-objects produces directly inspectable strategies—such as focusing on the grass region beside the track or on the rightmost monster—so the architecture's interpretability claim from the patch-based line of work carries over.
Reading between the lines
- If the token-count collapse is the real driver, this suggests a scaling law for hard attention: any pre-processing that yields semantically compact superpixel tokens should reduce controller size and training time, not just this particular color-connected segmentation.
- The paper's $k=1$ result implies a minimal-information controller is viable; an obvious extension the authors do not pursue is feeding a short history or a small set of attended coordinates into the LSTM to handle scenes with many relevant entities, where $k=1$ showed a performance dip.
- The authors note CMA-ES may be too greedy for the coupled attention/controller architecture; comparing against differential evolution under the same budget would test whether escaping local optima of attention strategies is the main obstacle.
- If the approach transfers to real-world images, the key risk is segmentation quality; testing with adaptive learned quantization or a trainable region proposal stage would show whether the bottleneck's value survives when segments are less clean than in game frames.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes replacing fixed grid patches with proto-objects, defined as color-connected regions obtained after convolution, quantization, and segmentation, as tokens for a hard self-attention module in a visual reinforcement learning agent. The attention module performs top-k selection over proto-objects, whose center-of-mass coordinates are fed to an LSTM controller, and the whole network is evolved with CMA-ES. Section 5 reports experiments on CarRacing and Doom Take Cover: on CarRacing the proto-object agent reaches 910.39 versus 888.69 for a rerun of the patch-based baseline (p=1.1e-22), and on Doom Take Cover it reaches 930.68 with k=1 (p=0.414) or 1192.82 with k=10 (p=2.8e-5) versus 959.27. The abstract concludes that the approach matches or exceeds the state of the art with 62% fewer parameters and 2.6 times less training time.
Significance. The core idea is plausible and the empirical package is above average for a neuroevolution paper: performance is measured in standard environments, Mann-Whitney U tests are reported, and the paper documents how the evolved segmentation changes over generations, which is a genuine interpretability strength. If the comparison with Tang et al. [22] is fair, the results support the claim that segmentation-derived proto-object tokens are a valid substitute for patches in evolved hard-attention agents, and the reduction in token count and parameters is remarkable. The current claims, however, overreach because the patch baseline is a rerun at half the original evolutionary budget and because the headline efficiency figures apply only to the k=1 configuration, while the Doom result that actually exceeds the baseline is the k=10 configuration.
major comments (4)
- [Section 5, Table 2] The central claim that the proto-object method 'matches or exceeds the state-of-the-art performance of patch-based implementations' is established only against the authors' own rerun of [22] at half the evolutionary budget (1000 generations, population 128, 8 seeds, versus 2000/256/16 in the original work), as stated in Section 5. CMA-ES solutions are budget-dependent, so it is not demonstrated that the rerun scores of 888.69 and 959.27 are representative of the published [22] state of the art. The manuscript should report the original [22] scores, show that the rerun protocol reproduces them, and provide the rerun configuration including environment versions and hyperparameters; without this evidence, 'matches or exceeds state of the art' is stronger than the data support.
- [Abstract, Section 4.2, Section 5.2] The abstract's efficiency headline ('62% less parameters and 2.6 times less training time') is tied to the k=1 configuration of Table 1, whereas the Doom result that actually exceeds the patch baseline is the k=10 variant described in Section 5.2, which uses 2671 parameters, 55 hours, and additional design changes (d_q=4 and 3x3 convolutions). The k=1 Doom result is statistically tied (p=0.414). The abstract and conclusion should separate the two configurations, or state precisely which configuration supports each claim.
- [Section 5.2, Figure 7, Table 2] The Doom comparison is ambiguous about which solutions are being compared. Table 2 labels its rows 'Best Score and 95% CI After 1000 Iterations (n=400)', but Figure 7's caption says the patch model peaked at 700 generations and reports p=0.414 'between best solutions'; for the k=10 result it reports 1193 at 700 generations. Please state whether the p-values compare final scores at a fixed generation or each method's individual peak, and report the corresponding scores; the current presentation allows the reader to infer either comparison.
- [Section 4.1.3] The hand-designed 11-dimensional segment feature set is a central component of the method, but its choice is justified only by 'thorough experimentation' with no ablation. Without varying the feature set, k, d_q, and PReLU on/off, it is unclear which components drive the reported gains; a small ablation study would make the claims about 'richer semantic information' testable and would strengthen the causal interpretation.
minor comments (5)
- [Section 4.1.4] The claim that 'a single PReLU neuron was shown to solve the XOR problem' cites a self-archived paper [17] and does not explain why this property transfers to top-k attention selection; either replace this with an analysis of the actual attention behavior or remove the XOR motivation.
- [Figure 3] The flowchart variables M, K and the notation (1, MK) are not defined in the caption or in Section 4.2; please specify all tensor dimensions explicitly.
- [Section 5] The statement that seeds are based on generation and repetition numbers is not a reproducible seed protocol; please specify the seed derivation and provide the hyperparameters of [22] that were retained in the rerun.
- [Section 5.1] The assertion that CarRacing V0 and V2 have 'no significant differences' is unsupported; if the authors optimized V0 for speed, the 2.6x training-time comparison should disclose the exact environment package and version.
- [General] The paper would benefit from a statement on code and data availability; several implementation details (connected-components library, quantization thresholds, test-seed generation) are needed to reproduce the results.
Circularity Check
No significant circularity: the central claim is an empirical comparison plus plain arithmetic; the only self-citation is peripheral and non-load-bearing.
full rationale
No circular step can be exhibited. The central quantities are all measured in the environment (scores), counted from architectures (parameters), or clocked (training time); none is obtained by plugging the target result into its own definition. The proto-object features are hand-selected descriptors computed from pixel regions; the attention selects top-k by a softmax row-sum; the controller is an LSTM; CMA-ES maximizes environment reward. There is no equation of the form 'predicted X = fitted X'. The self-citation to [17] (PReLU XOR) is peripheral: it motivates the PReLU nonlinearity in the attention layer, but the paper's headline result does not reduce to that citation, and the PReLU property is independently checkable. The explicit limitation in Section 5 — that Tang et al. ran 2000 generations/256 population/16 seeds and the paper halved these 'due to hardware limitations' — is a real fairness risk in the empirical comparison, but it is not a circularity: the rerun is an external baseline, not an input of the proto-object model. Likewise, 'after thorough experimentation' choosing the 11 segment features is a feature-selection risk, not a fitted parameter renamed as a prediction. Since no load-bearing step reduces to its inputs, the circularity score is minimal; I assign 1.0 only to acknowledge the non-load-bearing self-citation.
Assumptions & free parameters
free parameters (6)
- Segment feature set and dimension (d_in=11) =
11 features: color (R,G,B), center (X,Y), area, bounding box width/height/area, aspect ratio, extent
- Top-k selection count (k) =
1 for default; 10 for Doom variant
- Attention embedding dimension (d_q) =
2 for default; 4 for Doom k=10 variant
- Quantization bits per channel =
1 bit per channel (8 colors)
- LSTM hidden size =
16
- CMA-ES budget =
population 128, 1000 generations, 8 evaluation seeds
assumptions (5)
- domain assumption CMA-ES can reliably optimize the joint attention/control pipeline.
- domain assumption Color-connected regions after 1-bit quantization are behaviorally meaningful proto-objects.
- domain assumption The 11 hand-picked descriptor features contain enough information for control.
- ad hoc to paper The rerun patch-based baseline faithfully represents [22] under halved budget.
- standard math Mann-Whitney U test on 400 test seeds is a valid way to compare agent scores.
Cite this review
Pith. "Pith review of Neuroevolution of Self-Attention Over Proto-Objects." pith.science (2026). https://pith.science/paper/6HKUSX7H
@misc{pith2026250500186,
author = {Pith},
title = {Pith review of: Neuroevolution of Self-Attention Over Proto-Objects},
year = {2026},
howpublished = {\url{https://pith.science/paper/6HKUSX7H}},
note = {Machine review of arXiv:2505.00186}
}
read the original abstract
Proto-objects - image regions that share common visual properties - offer a promising alternative to traditional attention mechanisms based on rectangular-shaped image patches in neural networks. Although previous work demonstrated that evolving a patch-based hard-attention module alongside a controller network could achieve state-of-the-art performance in visual reinforcement learning tasks, our approach leverages image segmentation to work with higher-level features. By operating on proto-objects rather than fixed patches, we significantly reduce the representational complexity: each image decomposes into fewer proto-objects than regular patches, and each proto-object can be efficiently encoded as a compact feature vector. This enables a substantially smaller self-attention module that processes richer semantic information. Our experiments demonstrate that this proto-object-based approach matches or exceeds the state-of-the-art performance of patch-based implementations with 62% less parameters and 2.6 times less training time.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[22]
Yujin Tang, Duong Nguyen, and David Ha. 2020. Neuroevolution of self- interpretable agents. In Proceedings of the 2020 Genetic and Evolutionary Com- putation Conference (Cancún, Mexico) (GECCO ’20). Association for Computing Machinery, New York, NY, USA, 414–424. doi:10.1145/3377930.3389847
arXiv 2020
-
[1]
William Agnew and Pedro Domingos. 2018. Unsupervised Object-Level Deep Reinforcement Learning. Deep Reinforcement Learning Workshop (NIPS 2018)
work page 2018
-
[2]
Ali Borji and Laurent Itti. 2012. State-of-the-art in visual attention modeling.IEEE transactions on pattern analysis and machine intelligence 35, 1 (2012), 185–207
2012
-
[3]
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. arXiv:1606.01540 [cs.LG] Figure 8: Processing stages in the Doom Take Cover environ- ment. Top-left: raw image resized to 96x96. Top-right: 1x1 convolution + residual. Bottom-left: quantization. Bottom- right: attention. Note t...
arXiv 2016
-
[4]
Marisa Carrasco. 2011. Visual attention: The past 25 years. Vision Research 51, 13 (2011), 1484–1525. doi:10.1016/j.visres.2011.04.012 Vision Research 50th Anniversary Issue: Part 2
-
[5]
Zhe Chen. 2012. Object-based attention: A tutorial review. Attention, Perception, & Psychophysics 74 (2012), 784 – 802. doi:10.3758/s13414-012-0322-z
-
[6]
Leif H. Finkel and Paul Sajda. 1992. Proto-objects: an intermediate-level visual representation, In Optical Society of America Annual Meeting. Optical Society of America Annual Meeting -, -, FO1. doi:10.1364/OAM.1992.FO1
-
[7]
Christophe Fiorio and Jens Gustedt. 1996. Two linear time union-find strategies for image processing. Theoretical Computer Science 154, 2 (1996), 165–181
work page 1996
Show all 30 references
-
[8]
Nikolaus Hansen. 2006. The CMA Evolution Strategy: A Comparing Review . Springer Berlin Heidelberg, Berlin, Heidelberg, 75–102. doi:10.1007/3-540-32494- 1_4
2006 doi
-
[9]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition . In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE Computer Society, Los Alamitos, CA, USA, 770–778. doi:10.1109/CVPR.2016.90
2016 doi
-
[10]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory.Neural computation 9, 8 (1997), 1735–1780
1997
-
[11]
Freed, Michael J
Kristin Koch, Judith McLean, Ronen Segev, Michael A. Freed, Michael J. Berry, Vijay Balasubramanian, and Peter Sterling. 2006. How Much the Eye Tells the Brain. Current Biology 16, 14 (2006), 1428–1434. doi:10.1016/j.cub.2006.05.056
2006 doi
-
[12]
Machado, Erik Talvitie, and Michael Bowling
Yitao Liang, Marlos C. Machado, Erik Talvitie, and Michael Bowling. 2016. State of the Art Control of Atari Games Using Shallow Reinforcement Learning. In Proceedings of the 2016 International Conference on Autonomous Agents & Multia- gent Systems (Singapore, Singapore) (AAMAS...
2016
-
[13]
H. B. Mann and D. R. Whitney. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics 18, 1 (1947), 50 – 60. doi:10.1214/aoms/1177730491
1947
-
[14]
Volodymyr Mnih, Nicolas Heess, Alex Graves, and Koray Kavukcuoglu. 2014. Recurrent models of visual attention. In Proceedings of the 28th International Con- ference on Neural Information Processing Systems - Volume 2 (Montreal, Canada) (NIPS’14). MIT Press, Cambridge, MA, USA,...
2014
-
[15]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602 [cs.LG]
2013 arXiv
-
[16]
Francesco Orabona, Giorgio Metta, and Giulio Sandini. 2007. A Proto-object Based Visual Attention Model. In Attention in Cognitive Systems. Theories and Neuroevolution of Self-Attention Over Proto-Objects Systems from an Interdisciplinary Viewpoint , Lucas Paletta and Erich Ro...
2007
-
[17]
Pinto and Anderson R
Rafael C. Pinto and Anderson R. Tavares. 2024. PReLU: Yet Another Single-Layer Solution to the XOR Problem. arXiv:2409.10821 [cs.NE]
2024 arXiv
-
[18]
Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv:1908.10084 [cs.CL]
2019 arXiv
-
[19]
Ronald A Rensink. 2000. The dynamic representation of scenes. Visual cognition 7, 1-3 (2000), 17–42
2000
-
[20]
William Silversmith. 2025. connected-components-3d: Connected Components on Discrete and Continuous Multilabel 3D & 2D Images . GitHub. https://github.com/ seung-lab/connected-components-3d
2025
-
[21]
Rainer Storn and Kenneth V. Price. 1997. Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces. Journal of Global Optimization 11 (1997), 341–359. https://api.semanticscholar.org/ CorpusID:5297867
1997
-
[23]
Pereira, and William Bialek
Naftali Tishby, Fernando C. Pereira, and William Bialek. 2000. The information bottleneck method. arXiv:physics/0004057 [physics.data-an]
2000 arXiv
-
[24]
Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, and Omar G. Younis. 2024. Gymnasium: A S...
2024 arXiv
-
[25]
Martínez Trujillo
Stefan Treue and Julio C. Martínez Trujillo. 1999. Feature-based attention in- fluences motion processing gain in macaque visual cortex. Nature 399 (1999), 575–579. https://api.semanticscholar.org/CorpusID:4424973
1999
-
[26]
Gomez, Łukasz Kaiser, and Illia Polosukhin
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, ...
2017
-
[27]
Vecera and Martha J
Shaun P. Vecera and Martha J. Farah. 1994. Does visual attention select objects or locations? Journal of experimental psychology. General 123 2 (1994), 146–60. doi:10.1037//0096-3445.123.2.146
1994 doi
-
[28]
Dirk Walther and Christof Koch. 2006. Modeling attention to salient proto-objects. Neural Networks 19, 9 (2006), 1395–1407. doi:10.1016/j.neunet.2006.10.001 Brain and Attention
2006 doi
-
[29]
Jeremy M Wolfe. 1994. Guided search 2.0 a revised model of visual search. Psychonomic bulletin & review 1 (1994), 202–238
1994
-
[30]
William Woof and Ke Chen. 2018. Learning to Play General Video-Games via an Object Embedding Network. In 2018 IEEE Conference on Computational Intelligence and Games (CIG) . IEEE, Maastricht, Netherlands, 1–8. doi:10.1109/ CIG.2018.8490438
2018
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.