Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents

Florian Engert; Kanaka Rajan; Nathan Wu; Raaghav Malik; Roy Harpaz; Satpreet H. Singh; Sonja Johnson-Yu

REVIEW 5 major objections 5 minor 1 cited by

Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents

T0 review · 5 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A minimal recurrent agent trained with deep reinforcement learning reproduces larval zebrafish hunting, and the paper argues that binocular-only distance, coupled bout kinematics, and energetic costs make the stereotyped pursuit optimal.

desk verdict A solid virtual-lab paper whose headline sufficiency claim is only half-tested: the strike reward and binocular-only distance sensing already encode the behaviors the authors present as emergent, but the measured rollouts and honest limitations make it worth referee time. read the letter →

arxiv 2510.03699 v1 pith:OAD4KUVS submitted 2025-10-04 q-bio.NC cs.AIcs.LGcs.NEcs.SYeess.SY

Raaghav Malik , Satpreet H. Singh , Sonja Johnson-Yu , Nathan Wu , Roy Harpaz , Florian Engert , Kanaka Rajan This is my paper

classification q-bio.NCcs.AIcs.LGcs.NEcs.SYeess.SY

keywords larvalzebrafishhuntingbehaviordeepreinforcementlearningrecurrentneuralnetworkvergencepreycapturenormativemodelbout-basedcontrol

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the stereotyped hunting behavior of larval zebrafish—eye convergence during pursuit, speed modulation, and the roughly 50% reduction in prey angle per bout—is not a hardwired species-specific program but the outcome any agent would reach when optimized under a small set of constraints. The authors train a recurrent neural network with deep reinforcement learning in a bout-based simulator where the agent sees through two eyes with distance information only in the binocular overlap, moves with coupled forward/turn bouts, and pays modest energetic costs for fast locomotion, sharp turns, and eye convergence. Without imitating real fish data or modeling body physics, the trained agent develops the same pursuit statistics as larvae, including prey-angle halving and vergence that rises at hunt onset and falls after capture. The paper concludes that binocular sensing, coupled bout kinematics, and energetic costs are sufficient to make zebrafish-like hunting the optimal strategy, and offers this as a normative account of why the motifs exist.

What carries the argument

The central mechanism is the task itself: a bout-based closed loop in which a recurrent actor-critic policy chooses forward speed, turn speed, and vergence angle; the environment returns per-sector object/distance readings with distance available only in the binocular field and lower noise there; and rewards consist of capture (scaled by a modified Laplace function of alignment error, cut off at 80 degrees) minus linear penalties for fast moves, large turns, and eye convergence. That reward/sensory structure is what channels the policy into the empirical hallmarks—angle-halving pursuit and hunting-mode vergence—so the machinery is as much the constraint layout as the network.

What would settle it

Train the same agent in the same simulator but with monocular sectors also reporting distance; if hunting-mode vergence and prey-angle halving persist unchanged, the paper's claim that binocular-only range drives the behaviors is weakened. Alternatively, measure capture probability as a function of alignment error in real larvae; if it is much flatter than the modified Laplace with decay 5 and an 80-degree cutoff, the angle-halving account loses its motive force.

Watch

Extended reading notes

Core claim

The central claim is that a compact set of constraints is sufficient for zebrafish-like hunting to emerge in a minimal agent. The agent's pursuit bouts reduce the angle to prey by a factor of about two per bout (and distance by about 15%), matching measurements from real larvae; eye vergence rises when prey is detected, peaks before strike, and relaxes afterward; and successful hunts are shorter than failed hunts. The same constraints—binocular sensing with distance only in the shared field, coupled forward-speed and turn bout kinematics, and linear costs on speed, turns, and vergence—generate these behaviors under reinforcement learning, so the empirically observed stereotypy is presented a

Load-bearing premise

The load-bearing premise is that the reward and sensory structure already encodes the two headline behaviors: capture probability is a steep function of alignment error (modified Laplace, cutoff 80 degrees, fit to empirical strike data), and distance is supplied only in the binocular field; if either were relaxed—if monocular sectors carried range, or strikes were nearly angle-independent—the reported match (angle halving and vergence-linked pursuit) would likely disappear.

Editorial extensions

If this is right

If the account is right, the striking stereotypy of larval hunting—the roughly 50% per-bout angle reduction and the temporary eye convergence—is an optimal response to binocular sensing, coupled bout mechanics, and energy costs, not a species-specific hardwired program.
The virtual experiments predict concrete, testable relationships: faster prey should increase abort rates, higher prey density should shorten hunts, and constrained vergence limits should weaken hunting-mode convergence and slightly reduce eating success.
The parameter sweeps imply that the reward costs act as knobs: too little cost yields extravagant, less naturalistic policies; too much suppresses pursuit structure, so measured metabolic costs in vivo should sit in the range that preserves the motifs.
Because the trained agents reproduce behavioral variation across seeds and over many arenas, the framework predicts that individual differences in hunting style correspond to different points on the same cost-benefit frontier, comparable to the spread across animals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The 4-fold inflation of eye separation (from 0.5 mm to 2 mm, Appendix 6.2) is a calibration choice needed to reproduce the 1.37 mm blind spot in front of the strike radius; if that gap is what drives the need for vergence, a direct experimental test would be whether larvae with a surgically or genetically altered blind spot show weaker hunting-mode convergence.
The paper's account suggests a testable extension: training agents with distance information also available in monocular sectors should abolish or weaken the vergence-linked pursuit, since the eye would no longer be the only source of range.
A further extension: if the angle-halving law is normative, a simple reactive controller that only halves the prey angle each bout should approach the same capture efficiency as the full RNN policy, providing a stripped-down hypothesis for what the neural circuit needs to compute.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a genuinely useful virtual-lab paper, but the banner claim—that a compact set of constraints is sufficient for zebrafish-like hunting to emerge—is only partly tested. The behaviors themselves are real model outputs, not direct fits to the headline statistics: the per-bout angle halving and the 14.5 deg/s vergence rise are measured from rolled-out trajectories, not tuned numbers. That part is legitimate and worth credit.

What the paper does well: the training and measurement pipeline is competently executed, and the hyperparameter tables in Appendix 6.4 are unusually complete. The bout clustering (Fig. 2) and the hunt-versus-abort analyses (Fig. 3) are clean. The parameter sweeps in Fig. 6 are a nice touch, and the limitations paragraph is refreshingly explicit about missing biomechanics, circuit detail, and seed variability.

The soft spots are real but addressable. First, the strike reward is a modified Laplace in angular error, fit to empirical strike-angle data (Table 1). That creates strong selective pressure to minimize |θ|, so the angle-halving behavior is largely a consequence of optimizing that reward under bout kinematics, not an independent emergent match. The paper never ablates the reward shape. Second, distance information is available only in the binocular field, and eye separation is inflated to force a blind spot; vergence then becomes necessary for distance estimation. That makes the vergence-linked pursuit partly a designed consequence of the sensor layout. The sweeps vary costs and vergence limits but never give monocular sectors distance or relax the angle-dependent strike probability. Until those ablations are run, the sufficiency claim is weaker than stated.

Also, detailed behavioral analyses use only the best of 10 seeds, and Fig. 6d shows naturalism correlates with performance, so seed selection likely inflates the apparent emergence. The 'closely matching' overlays against Bolton et al. lack test statistics. Code is promised but not yet released.

None of this is fatal. The paper is a solid contribution to the normative-modeling literature, and the virtual-lab framing has real utility for designing experiments. But it needs major revisions: release code, analyze all seeds or justify selection, report statistical comparisons to biological data, and—critically—run ablations that vary the reward function and sensory modality. If the authors do that, the sufficiency claim becomes testable rather than asserted. I'd send it to serious peer review, expecting a revise-and-resubmit.

Referee Report

5 major / 5 minor

Summary. The authors train recurrent-network agents with PPO in a bout-based 2-D simulator of larval zebrafish hunting. The agent perceives prey through multi-sector monocular fields and a binocular region, moves in a coupled forward-speed/turn action space, pays energetic costs, and receives a capture reward whose angle dependence is fit to empirical strike data. The paper reports that the trained agent reproduces qualitative and quantitative hallmarks of real hunting—stereotyped bout structure, per-bout prey-angle halving, roughly 15% distance reduction, vergence increase during pursuit (14.5°/s), and abort after long tracking—and that parameter sweeps over costs, prey statistics, and vergence limits yield a compact constraint set (binocular sensing, bout coupling, energetic costs) that is claimed to be sufficient for these behaviors. The central normative claim is that these behaviors are optimal balances of sensory benefit and energetic cost rather than consequences of imitation or detailed biomechanics.

Significance. This is a serious attempt to turn a DRL-trained RNN into a normative model of a well-characterized ethological behavior. Its strengths are the unusually complete parameter reporting (Tables 1–4), the multi-seed training design, the systematic sweeps in Figs. 5–6, and the explicit falsifiable predictions (e.g., vergence-limit manipulations should reduce hunting success). If the sufficiency claim is established, the framework would be a valuable virtual lab and a useful template for NeuroAI. I do not regard the absence of detailed biomechanics or circuit realism as a defect, since the paper is explicit that its contribution is normative. However, the current evidence does not yet establish the claimed sufficiency of the compact constraint set: the sensor layout and reward shape directly encode the headline behaviors, and the headline quantitative matches are computed from a best-of-10 seed. These concerns are addressable with targeted ablations and multi-seed reporting, so they do not in themselves invalidate the project.

major comments (5)

[§2.3, §3.2–3.3, Fig. 6d] Best-of-10 seed selection biases the headline behavioral matches. The manuscript states that only the best-performing seed was carried forward for the detailed analyses in Sections 3.1–3.4, and Fig. 6d shows that eating rate and naturalistic vergence (Δμ) are positively correlated across seeds. Therefore the angle-halving ratio and the 14.5°/s vergence rate are estimates from the most naturalistic policy, not from the training distribution. Please report these headline statistics across all 10 (or 21) seeds with distributions, or explicitly qualify the results as best-policy demonstrations. Without this, the claim that the agent class exhibits these behaviors is over-stated.
[§2.1, Appendix 6.2] Binocular-only distance sensing pre-specifies the vergence behavior. Distance is provided only in the binocular field; monocular sectors give only binary detection. The eye separation is inflated from 0.5 mm to 2 mm specifically to create a 1.37 mm blind spot. As a result, vergence is the only mechanism by which the agent can estimate prey distance, so the emergence of convergence during hunts is a designed consequence of the sensor layout rather than an independently optimized active-sensing strategy. An ablation that supplies distance in monocular sectors (or uses the biological 0.5 mm separation) is needed to test whether vergence-linked pursuit is a genuine emergent consequence of binocular sensing or simply an affordance of the chosen sensor model.
[§2.1, Table 1, Fig. 1e] The capture probability is a modified Laplace in alignment error |θ| with decay 5 and cutoff 80°, fit to empirical strike-angle data. This creates a direct, strong selective pressure to minimize |θ| before striking. Under bounded per-bout turn rates and movement costs, a roughly fifty-percent per-bout reduction in prey angle is the expected consequence of optimizing this reward; it is not an independent behavioral prediction. To support the claim that angle-halving emerges from the compact constraint set, please include control runs with an angle-independent or much weaker angle-dependent capture probability, and show whether the per-bout halving ratio and the strike-distance geometry change. Without these ablations, the 'closely matching' angle-halving is not strong evidence for the normative account.
[Abstract, §3.5, Fig. 6] The claim that coupled forward-speed/turn bout kinematics are part of the sufficient constraint set is not tested. The abstract lists 'coupled vs. uncoupled turns and forward motion' among the manipulations, but the main text contains no comparison of the coupled action space with an uncoupled one. Figure 6 sweeps only αeye, βspeed, and βturn. Either provide the coupled/uncoupled ablation experiment, or remove that element from the claimed sufficient constraint set and from the abstract.
[§2.2, Table 3] The 'weak distance-based shaping reward' R_shape is never defined. It appears only as a value '~0.001' in Table 3. If R_shape rewards decreasing distance to prey, it adds a pursuit gradient on top of the capture reward and may itself be responsible for the approach trajectory and bout structure. The exact equation, input variables, and scale must be specified. Ideally, retrain with R_shape = 0 to demonstrate that capture reward alone suffices; otherwise the contribution of this term to the reported behaviors is unclear.

minor comments (5)

[Appendix 6.3] There is an internal inconsistency in the cluster count: the main text and Fig. 2 describe 'five groups' or 'five clusters', while Appendix 6.3 states that the maximum silhouette score occurs at k = 4. Please reconcile.
[Fig. 4c] The caption says 'all tracking sequences in between normalized to equal duration.' This time-warping could create artificial temporal dynamics. Please state explicitly whether warping was applied and how it affects the reported vergence time course.
[§2.1, Table 1] The exact functional form of the 'modified Laplace' strike probability is not written out. Please provide the equation (including the cutoff behavior) so the reader can connect Table 1 to Fig. 1e.
[Appendix 6.4, Table 4] The training curriculum is mentioned in §2.3 but not specified in the Appendix. Since curriculum details can substantially affect the resulting policy, please report the schedule or cite the exact implementation.
[General] Typos and awkward phrasing should be corrected, e.g., 'inevitible' in the Reproducibility Statement and 'explore an hunt' in §3.1.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the predicted behaviors are emergent rollout statistics, and the empirical inputs are stated modeling assumptions rather than hidden restatements of the reported matches.

full rationale

The paper's derivation chain is not circular. The strike reward is a modified Laplace in alignment error with parameters fit to empirical strike-angle data (Table 1), and distance information is provided only in the binocular field (§2.1, Appendix 6.2). These are explicit, biologically motivated inputs. The reported matches—per-bout prey-angle halving (§3.2, Fig. 3f-g) and vergence dynamics including the 14.5°/s rate (§3.3, Fig. 4d)—are measured from rolled-out trajectories of the trained policy; neither quantity appears as a term in the reward or as a fitted parameter. The binocular-only distance sensing makes vergence beneficial by construction, but the paper claims only that this constraint is sufficient for the behavior to emerge, which is a valid normative/sufficiency claim rather than a re-description of the input. The eye-separation inflation in Appendix 6.2 is an openly stated modeling choice used to create the blind-spot geometry; it is not hidden and does not by itself force the quantitative vergence time course or strike outcomes. Self-citations (e.g., Rajan et al. 2016 for network architecture, Johnson-Yu et al. 2024 for the DRL approach) support methodological choices but are not load-bearing for the central behavioral predictions. The absence of a direct ablation of the binocular-only distance modality is a completeness limitation, not circularity.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The model is calibrated with empirical measurements at nearly every stage (visual field, vergence limits, bout length, strike fit, speed/turn maxima), plus hand-tuned reward coefficients and an ad hoc eye-separation inflation. The ledger shows that the headline 'emergent' behaviors are produced by a simulation whose incentive structure is substantially built from the same empirical phenomena the paper claims to reproduce; the genuinely free emergent quantities are the per-bout halving rate, the distance reduction per bout, the vergence time course and rates, and the abort statistics.

free parameters (9)

eating_distribution_decay = 5
Strike probability vs. alignment error is a modified Laplace with decay 5, fit to empirical strike-angle data (Johnson et al. 2020). Directly sets the reward gradient that produces per-bout prey-angle reduction; a data-fitted parameter in the loop of the paper's headline match.
eating_angle (strike cutoff half-angle) = 80 deg
Cutoff half-angle for strike probability, fit to empirical strike data (Johnson et al. 2020); defines the alignment window the agent learns to enter.
vergence cost scale alpha_eye = 0.006
Hand-tuned reward coefficient for eye convergence; controls whether hunting-mode vergence is worth the cost, a key 'normative' output.
speed penalty beta_speed + threshold v_th = 0.01; 1.5 mm/s
Hand-tuned; shapes the fast-strike / slow-adjust bout split and the exploration speed distribution.
turn penalty beta_turn = 0.01
Hand-tuned; shapes turn usage during pursuit and interacts with vergence strategy (Fig. 6c).
capture reward R_capture and shaping R_shape = 10; ~0.001
Hand-tuned reward magnitudes; baseline identified through exploratory tuning (§2.3).
eye_separation = 2 mm (biological 0.5 mm)
Ad hoc inflation (Appendix 6.2) to force the 1.37 mm blind spot with the model's 1 mm strike radius; alters the binocular geometry the vergence claims rest on.
prey statistics (food_speed, food_turn_std, food_density) = 1 mm/s; 10 deg/s; 0.003 /mm^2
Chosen to approximate Paramecium swimming; not tied to a specific measurement, and prey statistics directly drive capture rates in the virtual experiments.
training curriculum schedule = linear: density decrease, motion-variability increase, strike-tolerance narrowing over 4M steps
Hand-designed curriculum (§2.3); affects which policies are learnable and is not swept for robustness.

assumptions (6)

domain assumption Discrete-bout abstraction: behavior decomposes into 125 ms bouts with coupled forward/turn speed (triangular action space)
Grounded in empirical bout structure (Johnson et al. 2020), but the specific triangular coupling (Fig. 1d) is a modeling choice that is load-bearing for the emergence claim.
domain assumption Paramecium-like prey are memoryless stochastic walkers with fixed speed and uniform turn noise
Used in §2.1; deliberately non-adversarial. If real prey move with different statistics, the optimal policy and the quantitative matches could change.
domain assumption Visual encoding: 163 deg monocular FOV, 10 angular sectors, distance only in the binocular field, binary detection in monocular sectors
The binocular-only-distance asymmetry (§2.1) is what makes vergence useful; biologically motivated by binocular disparity but the sharp 'no distance in monocular field' choice manufactures the vergence benefit.
ad hoc to paper Capture probability is a modified Laplace in alignment error |theta|, fit to empirical strike data
Introduced in §2.2/Table 1 to match strike-angle statistics; this reward term is the main driver of the angle-halving behavior the paper reports as emergent.
domain assumption Reward maximization under the specified costs defines 'optimal' (the normative criterion)
The 'normative account' (§5) assumes that maximizing capture reward minus hand-set energetic costs is the right optimization target; there is no evidence real larvae minimize these exact cost functions.
standard math PPO-trained recurrent policies approximate optimal behavior
Standard RL machinery (§2.3); the trained policy is a local optimum, not a proven global optimum, so 'optimal balance' claims inherit this caveat.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents." pith.science (2026). https://pith.science/paper/OAD4KUVS

@misc{pith2026251003699,
  author       = {Pith},
  title        = {Pith review of: Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OAD4KUVS}},
  note         = {Machine review of arXiv:2510.03699}
}

read the original abstract

Larval zebrafish hunting provides a tractable setting to study how ecological and energetic constraints shape adaptive behavior in both biological brains and artificial agents. Here we develop a minimal agent-based model, training recurrent policies with deep reinforcement learning in a bout-based zebrafish simulator. Despite its simplicity, the model reproduces hallmark hunting behaviors -- including eye vergence-linked pursuit, speed modulation, and stereotyped approach trajectories -- that closely match real larval zebrafish. Quantitative trajectory analyses show that pursuit bouts systematically reduce prey angle by roughly half before strike, consistent with measurements. Virtual experiments and parameter sweeps vary ecological and energetic constraints, bout kinematics (coupled vs. uncoupled turns and forward motion), and environmental factors such as food density, food speed, and vergence limits. These manipulations reveal how constraints and environments shape pursuit dynamics, strike success, and abort rates, yielding falsifiable predictions for neuroscience experiments. These sweeps identify a compact set of constraints -- binocular sensing, the coupling of forward speed and turning in bout kinematics, and modest energetic costs on locomotion and vergence -- that are sufficient for zebrafish-like hunting to emerge. Strikingly, these behaviors arise in minimal agents without detailed biomechanics, fluid dynamics, circuit realism, or imitation learning from real zebrafish data. Taken together, this work provides a normative account of zebrafish hunting as the optimal balance between energetic cost and sensory benefit, highlighting the trade-offs that structure vergence and trajectory dynamics. We establish a virtual lab that narrows the experimental search space and generates falsifiable predictions about behavior and neural coding.

Figures

Figures reproduced from arXiv: 2510.03699 by the authors.

**Figure 1.** A biologically inspired DRL framework grounds recurrent agent perception, actions, and rewards in zebrafish hunting. (a) Closed-loop setup: an actor–critic RNN policy selects actions (forward speed, turn speed, vergence angle) that update the environment, which in turn provides new observations and rewards. (b) Zebrafish visual model: each eye has a 163◦ monocular field of view, subdivided into 10 angular sectors th… view at source ↗

**Figure 2.** Agent movement statistics reveal distinct bout types resembling real zebrafish strikes and adjustments. (a) Agent action selection: density plot and histograms of forward and turn speeds chosen across evaluation (left) along with spatial displacement after one bout (right). (b) Forward and turn speeds as a function of distance to prey: two distinct clusters appear when prey are nearby—fast, straight bouts (strikes) … view at source ↗

**Figure 3.** Successful and failed hunts diverge in bout sequences, durations, and pursuit corrections. (a) Example trajectory ending in a successful strike. (b) Example trajectory ending in an abort. (c) Distribution of forward and turn speeds during hunting versus exploration, showing larger turn speeds and more precise forward bouts during hunts. (d) Hunt durations separated by successful versus failed hunts: successful hunts… view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Vergence dynamics reveal costly but advantageous binocular sensing during hunts. (a) Distribution of vergence angles during successful hunts versus non-hunting periods: hunts show consistently higher vergence. (b) Distribution of vergence angles as a function of distan…

**Figure 5.** Figure 5: Virtual experiments reveal how ecological and sensory constraints shape hunting performance. (a) Increasing prey speed reduces the number of successful strikes and increases abort rates, demonstrating sensitivity to prey motion statistics. (b) Increasing prey density l…

**Figure 6.** Figure 6: Parameter sweeps reveal how energetic and sensory constraints shape hunting performance. We train multiple seeds for each parameter configuration, noting how parameters shape solution degeneracy and naturalistic behaviors. (a) Effect of vergence cost αeye on the change…

**Figure 7.** Figure 7: Screenshot of Simulated Environment: (Left) Arena snapshot showing the agent (blue), prey items (green dots), and the agent’s monocular sensing fields (grey) and binocular sensing field (shaded grey wedge). (Right) Time series of behavioral variables over the last 50 t…

**Figure 8.** Figure 8: Window Length for Trajectory Analysis: Window length was determined by obtaining a characteristic timescale for movements by plotting mean squared error of next-time-step movement prediction from the previous n steps. 8 time steps was selected for the time scale as con…

**Figure 9.** Figure 9: Silhouette Scores for K-Means Clustering: Silhouette scores were computed to select the optimal number of clusters for behavioral segmentation. A maximum silhouette score occurs when k = 4. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Active Electrosensing and Communication in MARL-trained Weakly Electric Fish Collectives
cs.NE 2025-11 conditional novelty 6.0 of 10

Multi-agent reinforcement learning agents with bio-inspired electrosensing reproduce heavy-tailed electric-organ-discharge statistics and emergent social behaviors such as freeloading and dominance-dependent foraging.

Reference graph

Works this paper leans on

21 extracted references · 5 canonical work pages · cited by 1 Pith paper

[4]

6), allowing assessment of robustness

While variability in reinforcement learning studies is inevitible, we trained multiple random seeds for all parameter configurations and reported variability across seeds (e.g., Fig. 6), allowing assessment of robustness. 6.1 DETAILEDRELATEDWORKS Biological foundations of zebrafish hunting.Larval zebrafish hunting has been extensively characterized at bot...

2011
[5]

Understanding biological active sensing behaviors by interpreting learned artificial agent policies

Sonja Johnson-Yu, Satpreet Harcharan Singh, Federico Pedraja, Denis Turcu, Pratyusha Sharma, Naomi Saphra, Nathaniel Sawtell, and Kanaka Rajan. Understanding biological active sensing behaviors by interpreting learned artificial agent policies. InWorkshop on Interpretable Policies in Reinforcement Learning@ RLC-2024,

2024
[10]

Tianwei Ni, Benjamin Eysenbach, and Ruslan Salakhutdinov

doi: 10.1016/j.cub.2019.11.022. Tianwei Ni, Benjamin Eysenbach, and Ruslan Salakhutdinov. Recurrent model-free RL can be a strong baseline for many POMDPs

work page doi:10.1016/j.cub.2019.11.022 2019
[11]

Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

arXiv
[14]

Cees J V oesenek, Remco P M Pieters, Florian T Muijres, and Johan L van Leeuwen

doi: 10.1016/j.cub.2025.05.063. Cees J V oesenek, Remco P M Pieters, Florian T Muijres, and Johan L van Leeuwen. Reorientation and propulsion in fast-starting zebrafish larvae: an inverse dynamics analysis.J. Exp. Biol., 222(Pt 14):jeb203091, July

work page doi:10.1016/j.cub.2025.05.063 2025
[15]

Multi-agent reinforcement learning: A selective overview of theories and algorithms

12 Kaiqing Zhang, Zhuoran Yang, and Tamer Ba¸ sar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. arXiv preprint arXiv:1911.10635,

arXiv 1911
[16]

PMID: 39719697

doi: 10.1016/j.cub.2024.11.057. PMID: 39719697. 13 6 APPENDIX LLM USESTATEMENT We made limited use of Large Language Models (LLMs) to support literature exploration and the drafting of routine code such as plotting scripts. Any outputs from these tools were subsequently checked and validated by at least one author, ensuring that all code and text ultimate...

work page doi:10.1016/j.cub.2024.11.057 2024
[18]

movement trajectory

and electrosensory navigation in weakly electric fish (Johnson-Yu et al., 2024). More broadly, training recurrent agents with reinforcement learning has been used to study emergent planning, memory, and structured representations across tasks in neuroscience and AI (Singh, 2021; Huang et al.; 2025; Simmons-Edler et al., 2025; Singh et al., 2023; Keller et...

2024

Show all 21 references

[19]

16 6.4 (HYPER)PARAMETERSUMMARY Parameter Value Unit Description / Notes max_speed 5 mm/s Max speed of larval zebrafish when foraging (approx from (Fiaz et al., 2012)) max_turn_speed 7 rad/s Max turning speed of larval zebrafish when foraging (ap- prox from (V oesenek et al., 2...

2012
[20]

bout_length 0.125 s Duration of a bout (Bolton et al., 2019; Johnson et al.,

2019
[21]

eye_separation 2 mm Distance between eyes (approx from (Bianco et al., 2011), see 6.2) eye_forward_offset 0.5 mm Forward offset of the eyes from the center of the agent (approx from (Bianco et al., 2011)) detection_range 10 mm Max food/wall detection range (at noisy, lowest re...

2011
[2006]

doi: 10.1038/nrn1949

ISSN 1471-003X. doi: 10.1038/nrn1949. João C. Marques, Simone Lackner, Rita Félix, and Michael B. Orger. Structure of the Zebrafish Locomotor Repertoire Revealed with Unsupervised Behavioral Clustering.Current Biology, 28 (2):181–195.e5, January

doi
[2011]

doi: 10.3389/fnsys.2011.00101

ISSN 1662-5137. doi: 10.3389/fnsys.2011.00101. URL https://www.frontiersin.org/ journals/systems-neuroscience/articles/10.3389/fnsys.2011.00101. Andrew D Bolton, Martin Haesemeyer, Josua Jordi, Ulrich Schaechtle, Feras A Saad, Vikash K Mansinghka, Joshua B Tenenbaum, and Flori...

2011 arXiv
[2014]

Riley Simmons-Edler, Ryan P Badman, Felix Baastad Berg, Raymond Chua, John J Vastola, Joshua Lunger, William Qian, and Kanaka Rajan

doi: 10.7554/eLife.04878. Riley Simmons-Edler, Ryan P Badman, Felix Baastad Berg, Raymond Chua, John J Vastola, Joshua Lunger, William Qian, and Kanaka Rajan. Deep RL needs deep behavior analysis: Exploring implicit planning by model-free agents in open-ended environments

doi
[2018]

doi: 10.1016/j.cub.2017.12.002

ISSN 0960-9822. doi: 10.1016/j.cub.2017.12.002. URL https: //www.sciencedirect.com/science/article/pii/S0960982217316044. Duncan S. Mearns, Joseph C. Donovan, António M. Fernandes, Julia L. Semmelhack, and Herwig Baier. Deconstructing hunting behavior reveals a tightly coupled...

2017 doi
[2019]

doi: 10.7554/eLife.51975

ISSN 2050-084X. doi: 10.7554/eLife.51975. URLhttps://doi.org/10.7554/eLife.51975. Ansa W Fiaz, Karen M Léon-Kloosterziel, Gerrit Gort, Stefan Schulte-Merker, Johan L van Leeuwen, and Sander Kranenbarg. Swim-training changes the spatio-temporal dynamics of skeletogenesis in zeb...

doi
[2020]

Bianco, Adam R

Isaac H. Bianco, Adam R. Kampff, and Florian Engert. Prey capture behavior evoked by simple visual stimuli in larval zebrafish.Frontiers in Systems Neuroscience, volume 5 - 2011,

2011
[2021]

Learning dynamics and the geometry of neural dynamics in recurrent neural controllers

Ann Huang, Satpreet Harcharan Singh, and Kanaka Rajan. Learning dynamics and the geometry of neural dynamics in recurrent neural controllers. InWorkshop on Interpretable Policies in Reinforcement Learning@ RLC-2024. Ann Huang, Satpreet H Singh, Flavio Martinelli, and Kanaka Ra...

2024
[2023]

Neuroprospecting with DeepRL agents

Satpreet Harcharan Singh. Neuroprospecting with DeepRL agents. NeurIPS 2021 Workshop on AI for Science,

2021
[2024]

12.19.629427

doi: 10.1101/2024. 12.19.629427. URL https://www.biorxiv.org/content/10.1101/2024.12.19. 629427v1. Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. InAdvances in Neural Informati...

2024 doi
[2025]

Biswadeep Khan, On-Mongkol Jaesiri, Ivan P Lazarte, Yang Li, Guangnan Tian, Peixiong Zhao, Yicheng Zhao, Viet Duc Ho, and Julie L Semmelhack

URLhttps://arxiv.org/abs/2506.00138. Biswadeep Khan, On-Mongkol Jaesiri, Ivan P Lazarte, Yang Li, Guangnan Tian, Peixiong Zhao, Yicheng Zhao, Viet Duc Ho, and Julie L Semmelhack. Zebrafish larvae use stimulus intensity and contrast to estimate distance to prey.Curr . Biol., 33...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [4]

6), allowing assessment of robustness

While variability in reinforcement learning studies is inevitible, we trained multiple random seeds for all parameter configurations and reported variability across seeds (e.g., Fig. 6), allowing assessment of robustness. 6.1 DETAILEDRELATEDWORKS Biological foundations of zebrafish hunting.Larval zebrafish hunting has been extensively characterized at bot...

2011

[2] [5]

Understanding biological active sensing behaviors by interpreting learned artificial agent policies

Sonja Johnson-Yu, Satpreet Harcharan Singh, Federico Pedraja, Denis Turcu, Pratyusha Sharma, Naomi Saphra, Nathaniel Sawtell, and Kanaka Rajan. Understanding biological active sensing behaviors by interpreting learned artificial agent policies. InWorkshop on Interpretable Policies in Reinforcement Learning@ RLC-2024,

2024

[3] [10]

Tianwei Ni, Benjamin Eysenbach, and Ruslan Salakhutdinov

doi: 10.1016/j.cub.2019.11.022. Tianwei Ni, Benjamin Eysenbach, and Ruslan Salakhutdinov. Recurrent model-free RL can be a strong baseline for many POMDPs

work page doi:10.1016/j.cub.2019.11.022 2019

[4] [11]

Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

arXiv

[5] [14]

Cees J V oesenek, Remco P M Pieters, Florian T Muijres, and Johan L van Leeuwen

doi: 10.1016/j.cub.2025.05.063. Cees J V oesenek, Remco P M Pieters, Florian T Muijres, and Johan L van Leeuwen. Reorientation and propulsion in fast-starting zebrafish larvae: an inverse dynamics analysis.J. Exp. Biol., 222(Pt 14):jeb203091, July

work page doi:10.1016/j.cub.2025.05.063 2025

[6] [15]

Multi-agent reinforcement learning: A selective overview of theories and algorithms

12 Kaiqing Zhang, Zhuoran Yang, and Tamer Ba¸ sar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. arXiv preprint arXiv:1911.10635,

arXiv 1911

[7] [16]

PMID: 39719697

doi: 10.1016/j.cub.2024.11.057. PMID: 39719697. 13 6 APPENDIX LLM USESTATEMENT We made limited use of Large Language Models (LLMs) to support literature exploration and the drafting of routine code such as plotting scripts. Any outputs from these tools were subsequently checked and validated by at least one author, ensuring that all code and text ultimate...

work page doi:10.1016/j.cub.2024.11.057 2024

[8] [18]

movement trajectory

and electrosensory navigation in weakly electric fish (Johnson-Yu et al., 2024). More broadly, training recurrent agents with reinforcement learning has been used to study emergent planning, memory, and structured representations across tasks in neuroscience and AI (Singh, 2021; Huang et al.; 2025; Simmons-Edler et al., 2025; Singh et al., 2023; Keller et...

2024