{"id":"217b7004-3c85-4a68-898c-d5961ab9b777","arxiv_id":"2505.00186","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An agent that selects proto-objects (segmented color regions) instead of fixed patches matches or beats a patch-based baseline on CarRacing and Doom Take Cover while using 62% fewer parameters in its smallest configuration.","lead":"An evolved reinforcement learning agent looks at whole color regions, called proto-objects, instead of fixed square patches, and it matches or beats the patch-based version on two game tasks with a smaller controller and faster training. The result is a practical recipe for building compact, interpretable attention in visual reinforcement learning.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim depends on a rerun of the patch-based baseline at halved evolutionary budget (1000 gen, pop 128, 8 seeds vs. 2000/256/16, per §5); if that rerun underperforms Tang et al.'s published results, 'matches or exceeds SOTA' is not established.","rationale":"The reader's weakest assumption identifies the same load-bearing issue: the comparative baseline is a reduced-budget rerun, not the published SOTA. I do not find a more fundamental flaw that would justify a different verdict. A secondary statistical weakness is the Doom k=1 'match': a non-significant Mann-Whitney p=0.414 is used to claim equivalence, though the point estimate is lower (930.68 vs 959.27); an equivalence test would make the 'matches' language more defensible. However, CarRacing and Doom k=10 provide significant evidence for 'exceeds' once the baseline is accepted. The parameter and time figures are honestly disclosed in the tables, even though the abstract is broad. Thus the decisive, testable condition is baseline fidelity. If the rerun reproduces Tang et al.'s published results, the central claim is supported; if not, it is not. CONDITIONAL remains the correct verdict, pending the concrete test described above.","tokens_in":10936,"tokens_out":7550,"duration_ms":78452,"concrete_test":"Obtain the CarRacing and Doom Take Cover final scores and learning curves from Tang et al. [22]. Rerun the patch-based implementation with the original budget (2000 generations, population 256, 16 seeds) using the same environment versions (CarRacing V0, Doom Take Cover) and the same evaluation protocol (e.g., 400 test seeds). If the full-budget patch scores are statistically higher than the reported rerun scores (888.69 CarRacing, 959.27 Doom), then the comparison in Table 2 is against a weakened baseline and the central claim fails; if the scores match within confidence intervals, the halved budget did not bias the comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 states that the original Tang et al. experiments \"are not directly comparable\" and that the authors \"halved each of those hyperparameters due to hardware limitations, and performed the original experiments again in this new setup.\" The patch baseline in Table 2 (CarRacing 888.69, Doom 959.27) is therefore not the published state-of-the-art; it is a lower-budget rerun. CMA-ES is budget-dependent, so halving generations, population, and seeds can reduce the final score of the patch agent. The central claim that the proto-object approach \"matches or exceeds the state-of-the-art performance of patch-based implementations\" holds only if the reduced-budget patch score equals the published Tang et al. score on the same environments. The manuscript does not report Tang et al.'s original scores, does not provide code, and does not show that the rerun reproduces the original learning curves. If the rerun is weaker, the proto-object advantage is at least partly an artifact of the weakened baseline. The 62% parameter reduction and 2.6x time figures, while applying only to the k=1 configuration, are secondary to the validity of the comparative baseline.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes replacing fixed grid patches with proto-objects, defined as color-connected regions obtained after convolution, quantization, and segmentation, as tokens for a hard self-attention module in a visual reinforcement learning agent. The attention module performs top-k selection over proto-objects, whose center-of-mass coordinates are fed to an LSTM controller, and the whole network is evolved with CMA-ES. Section 5 reports experiments on CarRacing and Doom Take Cover: on CarRacing the proto-object agent reaches 910.39 versus 888.69 for a rerun of the patch-based baseline (p=1.1e-22), and on Doom Take Cover it reaches 930.68 with k=1 (p=0.414) or 1192.82 with k=10 (p=2.8e-5) versus 959.27. The abstract concludes that the approach matches or exceeds the state of the art with 62% fewer parameters and 2.6 times less training time.","tokens_in":11148,"tokens_out":6176,"duration_ms":57108,"significance":"The core idea is plausible and the empirical package is above average for a neuroevolution paper: performance is measured in standard environments, Mann-Whitney U tests are reported, and the paper documents how the evolved segmentation changes over generations, which is a genuine interpretability strength. If the comparison with Tang et al. [22] is fair, the results support the claim that segmentation-derived proto-object tokens are a valid substitute for patches in evolved hard-attention agents, and the reduction in token count and parameters is remarkable. The current claims, however, overreach because the patch baseline is a rerun at half the original evolutionary budget and because the headline efficiency figures apply only to the k=1 configuration, while the Doom result that actually exceeds the baseline is the k=10 configuration.","major_comments":[{"comment":"The central claim that the proto-object method 'matches or exceeds the state-of-the-art performance of patch-based implementations' is established only against the authors' own rerun of [22] at half the evolutionary budget (1000 generations, population 128, 8 seeds, versus 2000/256/16 in the original work), as stated in Section 5. CMA-ES solutions are budget-dependent, so it is not demonstrated that the rerun scores of 888.69 and 959.27 are representative of the published [22] state of the art. The manuscript should report the original [22] scores, show that the rerun protocol reproduces them, and provide the rerun configuration including environment versions and hyperparameters; without this evidence, 'matches or exceeds state of the art' is stronger than the data support.","section":"Section 5, Table 2"},{"comment":"The abstract's efficiency headline ('62% less parameters and 2.6 times less training time') is tied to the k=1 configuration of Table 1, whereas the Doom result that actually exceeds the patch baseline is the k=10 variant described in Section 5.2, which uses 2671 parameters, 55 hours, and additional design changes (d_q=4 and 3x3 convolutions). The k=1 Doom result is statistically tied (p=0.414). The abstract and conclusion should separate the two configurations, or state precisely which configuration supports each claim.","section":"Abstract, Section 4.2, Section 5.2"},{"comment":"The Doom comparison is ambiguous about which solutions are being compared. Table 2 labels its rows 'Best Score and 95% CI After 1000 Iterations (n=400)', but Figure 7's caption says the patch model peaked at 700 generations and reports p=0.414 'between best solutions'; for the k=10 result it reports 1193 at 700 generations. Please state whether the p-values compare final scores at a fixed generation or each method's individual peak, and report the corresponding scores; the current presentation allows the reader to infer either comparison.","section":"Section 5.2, Figure 7, Table 2"},{"comment":"The hand-designed 11-dimensional segment feature set is a central component of the method, but its choice is justified only by 'thorough experimentation' with no ablation. Without varying the feature set, k, d_q, and PReLU on/off, it is unclear which components drive the reported gains; a small ablation study would make the claims about 'richer semantic information' testable and would strengthen the causal interpretation.","section":"Section 4.1.3"}],"minor_comments":[{"comment":"The claim that 'a single PReLU neuron was shown to solve the XOR problem' cites a self-archived paper [17] and does not explain why this property transfers to top-k attention selection; either replace this with an analysis of the actual attention behavior or remove the XOR motivation.","section":"Section 4.1.4"},{"comment":"The flowchart variables M, K and the notation (1, MK) are not defined in the caption or in Section 4.2; please specify all tensor dimensions explicitly.","section":"Figure 3"},{"comment":"The statement that seeds are based on generation and repetition numbers is not a reproducible seed protocol; please specify the seed derivation and provide the hyperparameters of [22] that were retained in the rerun.","section":"Section 5"},{"comment":"The assertion that CarRacing V0 and V2 have 'no significant differences' is unsupported; if the authors optimized V0 for speed, the 2.6x training-time comparison should disclose the exact environment package and version.","section":"Section 5.1"},{"comment":"The paper would benefit from a statement on code and data availability; several implementation details (connected-components library, quantization thresholds, test-seed generation) are needed to reproduce the results.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the halved-budget baseline. If the authors cannot obtain or reproduce Tang et al.'s original numbers, the paper should be reframed as a comparison against a reduced-budget rerun rather than as 'matches or exceeds state of the art.' I would ask the editor to require the original comparison numbers and a reproducibility statement before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: this is a real new combination, but the headline number is set against a weaker baseline than the abstract lets on. The authors replace rectangular patches with segmentation-based proto-object tokens for hard attention in RL, evolve everything with CMA-ES, and report big parameter and time savings. That part is real and interesting.\n\nWhat's new: the specific pipeline—connected-component segmentation from quantized 1x1-convolved frames, eleven hand-picked features per region, PReLU-boosted Q/K attention, top-k selection, LSTM control, all evolved. The components are known, but the packaging is not, and the paper does a good job of explaining design choices and showing how the evolved segmentation changes over time. The CarRacing win (910 vs 889, p=1.1e-22) is solid on its own terms.\n\nThe soft spots are where the reader's concern lands. Section 5 plainly says the Tang et al. baseline was rerun with half the generations, half the population, and half the seeds because of hardware limits. That means “matches or exceeds state-of-the-art” is really “matches or exceeds a lower-budget rerun of the state-of-the-art.” The paper is transparent about this, which is good, but it does not report the original published scores or show that the rerun reproduces them, and no code is provided. So the central comparative claim is not auditable as-is. It is a fixable problem, not a fatal one.\n\nAlso, the abstract says “62% less parameters and 2.6 times less training time” without noting those numbers apply only to the k=1 model. The best Doom result uses k=10 with 2671 parameters and 55h, which is 1.6x faster, not 2.6x. The body is honest about this; the abstract overreaches.\n\nThe math is straightforward, the statistics are appropriate, and the citation pattern is fine. The self-citation for PReLU is peripheral and not a real issue. The main weaknesses are empirical reporting choices.\n\nWho should read it: people working on compact visual RL controllers or neuroevolution of attention. It deserves a serious referee, but a referee should push for a baseline comparison against the original published numbers, or at least a matched-budget reproduction, plus code release and a clearer abstract.\n\nMy take: conditional accept path, major revision on the baseline question.","headline":"A genuinely new combination of proto-object tokens and evolved hard attention, but the headline savings are measured against a halved-budget rerun of the baseline, not the published SOTA.","tokens_in":11791,"tokens_out":2163,"would_cite":false,"duration_ms":21439,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that replacing rectangular image patches with segmentation-derived proto-objects as attention tokens lets an evolved controller match or beat patch-based agents with 62% fewer parameters and 2.6 times less training time.","keywords":["proto-objects","hard attention","self-attention","neuroevolution","CMA-ES","visual reinforcement learning","image segmentation","interpretable agents"],"falsifier":"Rerun the patch-based model of [22] under its original settings (2000 generations, population 256, 16 seeds) on Car Racing and Doom Take Cover; if its best scores exceed the proto-object scores reported here (910.39 and 1192.82) with statistical significance, the claim of matching or exceeding the published patch-based state of the art fails.","tokens_in":10605,"feed_emoji":"👁️","tokens_out":9281,"duration_ms":84678,"temperature":0.7,"pith_summary":"This paper tries to establish that visual attention in reinforcement learning works better when the attention module operates on proto-objects—coherent image regions produced by segmentation—rather than on fixed rectangular patches. It reports that a self-attention module that hard-selects the top-k proto-objects, with the whole pipeline evolved by CMA-ES, matches or exceeds a patch-based baseline on Car Racing and Doom Take Cover while using 62% fewer learnable parameters and training about 2.6 times faster. If correct, the finding means an information bottleneck built from segmentation can carry the task-relevant visual content with far fewer tokens than a patch grid, and the controller can then run on a single attended location per frame. This would make evolved visual attention cheaper and more interpretable without sacrificing performance.","feed_headline":"Proto-object attention beats patches with 62% fewer parameters","feed_subtitle":"Proto-object tokens cut parameters 62% and training time 2.6x while matching or beating patch agents in visual RL.","key_machinery":"The load-bearing object is the proto-object token: a color-connected region of the quantized frame, encoded as an 11-dimensional descriptor (RGB color, center of mass, pixel area, bounding-box width, height and area, aspect ratio, and extent). A single $1\\times1$ convolution followed by 1-bit-per-channel quantization makes the grouping adaptive, while a residual connection lets evolution start from the trivial segmentation by original color. The attention layer applies PReLU-augmented linear maps to produce queries and keys, builds the standard softmax attention matrix, sums its rows into an importance score, and hard-selects the top-$k$ proto-objects—$k=1$ in most runs. Because segmentation, selection, and coordinate transfer are nondifferentiable, the whole pipeline is optimized jointly by CMA-ES, which evolves the convolution, attention, and LSTM controller together.","core_discovery":"The paper claims that the representation of what is attended to is the key efficiency lever in hard-attention agents. Instead of splitting a frame into a fixed $7\\times7$ grid of image patches and selecting the top ten, the agent decomposes the quantized image into color-connected regions—proto-objects—and describes each by 11 features such as color, center of mass, area, bounding box, and aspect ratio. That reduces the typical token count from 529 per frame to about 12 in these environments and the token dimension from 147 to 11, which lets the top-k selection run with $k=1$ and an LSTM controller receive only the selected region's $x,y$ coordinates. On Car Racing the proto-object agent scores 910.39 versus 888.69 for the patch baseline ($p = 1.1e-22$); on Doom Take Cover $k=1$ matches the baseline ($p = 0.414$) while $k=10$ exceeds it (1192.82 versus 959.27, $p = 2.8e-5$). The total parameter count drops from 3667 to 1406, and training is 2.6 times faster even though the proto-object system runs on CPU against the patch system's GPU use.","pith_inferences":["If the token-count collapse is the real driver, this suggests a scaling law for hard attention: any pre-processing that yields semantically compact superpixel tokens should reduce controller size and training time, not just this particular color-connected segmentation.","The paper's $k=1$ result implies a minimal-information controller is viable; an obvious extension the authors do not pursue is feeding a short history or a small set of attended coordinates into the LSTM to handle scenes with many relevant entities, where $k=1$ showed a performance dip.","The authors note CMA-ES may be too greedy for the coupled attention/controller architecture; comparing against differential evolution under the same budget would test whether escaping local optima of attention strategies is the main obstacle.","If the approach transfers to real-world images, the key risk is segmentation quality; testing with adaptive learned quantization or a trainable region proposal stage would show whether the bottleneck's value survives when segments are less clean than in game frames."],"forward_implications":["A segmentation bottleneck can replace the patch grid: the same task is solved with about twelve tokens per frame instead of 529, and the attention module no longer needs a value matrix $V$ because it only selects tokens rather than mixing them.","The controller can operate on a single attended location per frame ($k=1$), with the LSTM supplying memory; this is enough to match or beat the patch-based agent on both tested environments.","Because the proto-object system trains 2.6 times faster while running on CPU against the baseline's GPU, the computational savings come from the representation itself and not from faster hardware.","Evolved attention over proto-objects produces directly inspectable strategies—such as focusing on the grass region beside the track or on the rightmost monster—so the architecture's interpretability claim from the patch-based line of work carries over."],"supporting_citations":[{"why":"Supplies the patch-based hard-attention baseline, its architecture, environments, and the published results that the proto-object system must match or beat.","marker":"[22]"},{"why":"CMA-ES is the derivative-free optimizer that jointly evolves the convolution, attention, and controller, making the non-differentiable pipeline learnable.","marker":"[8]"},{"why":"Provides the self-attention formulation (queries, keys, softmax attention matrix) that the paper adapts into top-k token selection.","marker":"[26]"},{"why":"The union-find connected-components labeling method is what extracts the color-connected regions used as proto-object tokens.","marker":"[7]"},{"why":"OpenAI Gym supplies the Car Racing and Doom Take Cover environments used for the experiments.","marker":"[3]"},{"why":"The LSTM controller receives the selected proto-object coordinates and produces actions, and its size is a major part of the parameter reduction.","marker":"[10]"}],"fun_headline_variants":["Proto-objects slash attention params 62%, train 2.6x faster","Attention over proto-objects beats patches with 62% fewer params","Proto-object tokens: 62% less params, 2.6x faster training","Evolving attention on proto-objects: 62% fewer params, 2.6x speed","Proto-objects cut attention cost 62% and train 2.6x faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"As Section 5 notes, the original patch-based experiments ran with 2000 generations, population 256, and 16 seeds, while the comparison here reruns that baseline with half of each; the central 'matches or exceeds' claim assumes this reduced-budget rerun is a faithful representation of the patch-based method.","fun_headline_variants_meta":{"raw":{"variants":["Proto-objects slash attention params 62%, train 2.6x faster","Attention over proto-objects beats patches with 62% fewer params","Proto-object tokens: 62% less params, 2.6x faster training","Evolving attention on proto-objects: 62% fewer params, 2.6x speed","Proto-objects cut attention cost 62% and train 2.6x faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001279,"raw_usage":{"total_tokens":5232,"prompt_tokens":953,"completion_tokens":4279,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":4184}},"tokens_in":569,"tokens_out":4279,"duration_ms":27412,"temperature":1.0,"reasoning_tokens":4184,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:49:25.467486+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the patch-based model of [22] under its original settings (2000 generations, population 256, 16 seeds) on Car Racing and Doom Take Cover; if its best scores exceed the proto-object scores reported here (910.39 and 1192.82) with statistical significance, the claim of matching or exceeding the published patch-based state of the art fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CMA-ES is the derivative-free optimizer that jointly evolves the convolution, attention, and controller, making the non-differentiable pipeline learnable."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The union-find connected-components labeling method is what extracts the color-connected regions used as proto-object tokens."}],"review_version":1}