{"id":"c5a6d01c-b080-43ee-8eca-ddb47c91f579","arxiv_id":"2602.08566","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"In a 2D Q-learning homing model, mean homing time is reported to be non-monotonic in rotational diffusion with a crossover at D_r≈12, and the learned policy is claimed to beat a stochastic-resetting ABP baseline.","lead":"A Q-learning model for self-propelled agents in a 2D arena is presented; homing times are reported to vary non-monotonically with rotational noise, with a claimed optimal noise level. The underlying simulations are simple, but the key comparisons omit crucial baselines and the 'optimal noise' label is misleading.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central result is not evidence of learning: Eq. (7)'s cost makes the align action strictly dominant in both states, so Q-learning collapses to an ε-greedy always-align policy; the Dr-dependent homing curve is thus an artifact of the noise in action 2, not of learned navigation.","rationale":"I read the paper in good faith as a simulation study claiming that Q-learning yields a Dr-dependent optimal homing policy and that this learned policy outperforms an ABP baseline. The most load-bearing problem is not the specific form of φ(r), which the Reader flagged, but the fact that the cost function makes the optimal policy trivially 'always align'. This follows directly from the stated equations: action 1 gives a deterministic negative cost −v0Δt, while action 2 has expected cost strictly larger (less negative) for any nonzero noise. Therefore the Q-learning machinery is unnecessary; the reported non-monotonicity and 'policy crossover' are generic properties of an ε-greedy policy that mostly resets to home, with occasional noisy reorientations. The paper's own Fig. 4f showing a strong Dr-dependence of action counts is actually inconsistent with a converged always-align policy and indicates either a transient learning artifact or an undocumented dependence on the threshold φ. I therefore agree with the Reader's REJECT verdict, but I locate the decisive weakness in the circularity of the cost-policy design rather than in the uncalibrated φ(r). The proposed no-learning control is a single, concrete, computational check that would settle whether the central claim survives; based on the equations as written, I expect it to reproduce the reported curves. If it did not, the paper could be reconsidered as a modest empirical study, but the current text does not support the strong claims about adaptive homing.","tokens_in":16352,"tokens_out":10631,"duration_ms":128554,"concrete_test":"Run a no-learning control: at every step, choose action 1 with probability 0.85 and action 2 with probability 0.15 (equivalently, fix Q[s,1]=−1, Q[s,2]=0 for both states so the greedy policy is always action 1 with ε=0.3), and measure ⟨T_home⟩(Dr), reset counts, and action frequencies under identical initial conditions and noise. If this control reproduces the non-monotonic curve in Fig. 3 and the action-selection trend in Fig. 4f, then Q-learning is not load-bearing and the RL-vs-ABP comparison is an artifact. A second check: compare the same no-learning control against the ABP baseline of Eq. (9); if it matches or beats the reported RL curve, the claimed superiority over ABP is entirely due to the hand-crafted resetting rule.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The main claim—that Q-learning produces a Dr-dependent optimal homing strategy—fails at the level of the MDP. With C(t+Δt)=|r(t+Δt)|−|r(t)| (Eq. 7), the alignment action a=1 (Eq. 5) sets θ(t+Δt)=0, so the next position is reached by moving exactly toward home and the cost is deterministically −v0Δt, independent of Dr, state, or φ(r). For a=2 the heading after update is θ+η, so the expected one-step radial displacement is v0Δt E[cos(θ+η)] < v0Δt for any θ≠0 and any nonzero symmetric noise. Hence the greedy action is a=1 in both states for every Dr; the Q-table's argmin is predetermined by the cost function. The non-monotonic ⟨T_home⟩(Dr) and the reported 'crossover' in action selection (Fig. 4f) must therefore be produced by the ε=0.3 random choices plus the specific noise term in Eq. (5), not by learned state-dependent control. The paper offers no control run of a non-learning agent that simply chooses a=1 with probability 1−ε/2 and a=2 with probability ε/2 at every step. If Q-values are not fully converged during the 1500 realizations, then the reported curves are a transient of the learning algorithm, not a converged policy—another form of the same problem. The ABP baseline (Eqs. 9–10) is also constructed with the hand-set φ(r) and a reset probability derived from it, so it is not 'identical conditions' and cannot validate the RL advantage. Additionally, the claimed 'optimal noise' D_r* is a maximum of ⟨T_home⟩, not a minimum, so the label is misleading. Taken together, the central result does not demonstrate homing through reinforcement learning.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Q-learning framework for homing in a continuous 2D domain. A single self-propelled agent has a binary state based on an angular threshold φ(r) and two actions: align to home (a=1) or undergo stochastic reorientation (a=2). The cost is the change in radial distance to home. The authors report a non-monotonic mean homing time as a function of rotational diffusion strength Dr, with a claimed optimal noise level Dr*≈12, and state that the RL agent outperforms an Active Brownian Particle (ABP) baseline. They extend the model to two and many agents with soft repulsion, claiming that one agent becomes faster and that the fastest agent improves with group size. The central conclusion is that cost-driven learning, stochastic reorientation, and interactions enable adaptive navigation.","tokens_in":16867,"tokens_out":4406,"duration_ms":49198,"significance":"If the claims were correct, the paper would offer a simple RL description of homing that links individual and collective navigation, with testable predictions about resetting statistics and noise-dependent efficiency. The authors provide extensive simulation data and a clear model definition. However, the learning component is effectively trivialized by the cost function: action 1 yields the minimum possible cost in every state, so Q-learning converges to an always-align policy independent of Dr. The non-monotonicity and action-selection crossover then arise from the fixed ε-greedy exploration and the state-independent stochastic action, not from learned adaptive behavior. The ABP comparison is not made under identical conditions, and the multi-agent claims lack a no-interaction control. Because the central mechanism is not supported, the significance of the reported phenomena is substantially diminished.","major_comments":[{"comment":"The cost function C(t+Δt)=|r(t+Δt)|−|r(t)| makes action 1 strictly dominant in both states for every Dr. For a=1, θ(t+Δt)=0 and |r(t+Δt)|=r(t)−v0Δt (except when overshooting near home), giving the minimum possible cost −v0Δt. For a=2, the expected radial displacement is v0Δt E[cos(θ+η)] < v0Δt, so the expected cost is larger (less negative) for any state and any symmetric noise. Hence argmin Q is action 1 in both states; Q-learning reduces to an ε-greedy always-align policy with probability 1−ε/2 on action 1. The reported crossover in action selection (Fig. 4f) and the non-monotonic ⟨T_home⟩(Dr) (Fig. 3) are therefore not evidence of learned state-dependent control; they reflect the fixed exploration probability and the noise in action 2. A control run of a non-learning agent that chooses action 1 with probability 1−ε/2 and action 2 with probability ε/2 at every step is missing.","section":"Eq. (7) with Eqs. (4)-(6)"},{"comment":"The paper repeatedly calls Dr*≈12 an 'optimal noise level,' but Fig. 3 shows that ⟨T_home⟩ has a maximum at this value, not a minimum. If homing efficiency is measured by speed of arrival, this is the worst noise level, not an optimal one. The labeling appears to confuse a maximum of ⟨T_home⟩ with an optimum of the explored parameter. This mislabel affects the abstract and the interpretation of the scaling regimes in Fig. 4(e).","section":"Fig. 3 and Sec. III A"},{"comment":"The ABP baseline is not obtained under 'identical conditions' as claimed. The RL agent's stochastic action uses uniformly distributed noise ζ∈[−π/2,π/2] (Eq. 5), whereas the ABP uses Gaussian white noise η with δ-correlations (Eq. 11). Moreover, the ABP's resetting probability p(t) is a deterministic function of |θ| and φ(r), while the RL agent's effective policy is ε-greedy with fixed random exploration. Any improvement of RL over ABP could be due to these differences in noise type and reset rule rather than to learning. A fair baseline would use the same noise distribution and the same effective action probabilities as the converged RL policy.","section":"Sec. III D, Eqs. (9)-(11)"},{"comment":"The claim that repulsive interactions make one agent consistently faster and that the fastest agent improves with group size lacks a no-repulsion control. In any ensemble of N independent agents, the minimum homing time among N iid draws decreases with N by order statistics alone. To attribute the speed-up to repulsion or coordination, the authors must compare against N non-interacting agents with the same initial conditions and noise. Without this control, the multi-agent findings are not distinguishable from a trivial extremal-statistics effect.","section":"Secs. III B and III C"},{"comment":"The angular threshold φ(r) is a hand-posed function with no calibration or sensitivity analysis. Every reported quantity—the binary state, the resetting statistics, and the ABP reset probability—depends on this functional form. Because the RL policy is effectively fixed, the non-monotonicity likely arises from the interplay between φ(r) and the random action. The authors should show that their qualitative results are robust to alternative choices of φ(r) (e.g., constant, different radial dependence). Without this, the 'optimal' Dr and the scaling exponents are not established as robust phenomena.","section":"Eq. (2) and robustness"}],"minor_comments":[{"comment":"The paper contains several typos and awkward phrases: e.g., 'the cost also acts as an effective signal' in the abstract, 'in multi-agents system' and 'the fastest agent becomes progressively faster' (grammar). The distinction between 'optimal' and 'maximum' should be corrected throughout.","section":"General"},{"comment":"The power-law exponents (β,α) = (1.0,0.5) and (2.0,1.0) are stated without reporting the fitting procedure, confidence intervals, or the range over which the fits are performed. The inset collapse should specify the normalization of f(x).","section":"Fig. 4(e)"},{"comment":"The noise term √(2DrΔt) ζ with ζ uniform on [−π/2,π/2] has variance (π²/12)·2DrΔt. The choice of uniform rather than Gaussian noise is not justified; the ABP comparison is further complicated by this difference.","section":"Eq. (5)"},{"comment":"The reflecting boundary condition at R0=35 is mentioned but not described in detail. How is reflection implemented? This could affect the statistics for large Dr where trajectories may hit the boundary.","section":"Sec. II"},{"comment":"The reference list includes several arXiv preprints and some incomplete citations. For example, Ref. [4] (Paramanick et al.) is cited but not discussed in the text beyond the introduction; the authors should clarify how their work builds on it.","section":"References"}],"recommendation":"reject","confidential_remarks":"The core problem is that the cost function makes the learning component vacuous: the optimal policy is trivially 'always align,' so the Q-learning framework does not provide adaptive decision-making as claimed. The non-monotonic homing time and the RL-vs-ABP comparison are therefore not supportive of the paper's central thesis. The multi-agent results also lack necessary controls. I see no way to fix these issues within the scope of a revision without fundamentally redesigning the cost function and the baselines."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: the RL claim in this paper does not survive contact with its own equations. The cost function C(t+Δt)=|r(t+Δt)|−|r(t)| makes action 1 (align to home) strictly dominant in both states: it yields a deterministic −v0Δt, while action 2's expected cost is always less favorable. So Q-learning collapses to an ε-greedy always-align policy. The non-monotonic ⟨T_home⟩(D_r) curve and the action-selection 'crossover' in Fig. 4(f) are properties of the stochastic reset process driven by the exploration probability ε, not of learned navigation. The authors also mislabel D_r*≈12 as 'optimal noise' when it is actually the maximum of the homing time—the worst noise level.\n\nCredit where it's due: the model is clearly described, the simulations are extensive (1500 realizations, error bars, parameter sweeps in the SM), and the movies help. The power-law scaling of reset intervals is a reasonable fit. There is genuine care in showing error bars don't shrink with averaging.\n\nThe problems are load-bearing. There is no control run of a non-learning agent that simply takes action 1 with probability 1−ε/2, so the claimed RL advantage over the ABP baseline is not a test of learning. The ABP baseline differs in noise distribution (Gaussian vs uniform) and reset rule (Eq. 9 using the hand-crafted φ(r)), so 'identical conditions' is false. The multi-agent 'fastest becomes faster' result is likely an extreme-value artifact, with no no-repulsion control. The φ(r) function is uncalibrated, and all statistics depend on its specific form. No code is provided, so results are not independently reproducible without reimplementation.\n\nIn short, the central argument is circular: the cost function already encodes that moving toward home is good, so the learned policy is predetermined. A revised version with proper baselines, a convergence check, and corrected terminology could be a modest simulation study of stochastic resetting in active particles. As it stands, I would not send this to peer review; a desk reject with a clear explanation is more appropriate.\n\nSkip it in your reading group, and I won't cite it.\n\nBest.","headline":"The cost function makes the align action strictly dominant, so the RL is ε-greedy always-align; the central 'learning' claim is circular.","tokens_in":17363,"tokens_out":7858,"would_cite":false,"duration_ms":79900,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-03T03:13:56.969542+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":1}