{"id":"1e1b8aff-f29e-44b4-930f-c9ecc5b3e0b7","arxiv_id":"2507.18849","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"In a simulated low-Reynolds-number swimmer with 2 to 4 rigid paddle pairs, reinforcement learning recovers the biologically common back-to-front metachronal wave as the most efficient stroke, while front-to-back or paired strokes can be faster at wide spacings.","lead":"This paper uses reinforcement learning to discover how a small swimmer with rigid paddles should move through a viscous fluid. It finds that a back-to-front wave, like the one many real microorganisms use, is the most efficient stroke, but at wide paddle spacings a front-to-back stroke can be faster.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Efficiency ranking depends on a drag coefficient measured only for the perpendicular-paddle towing configuration, while the learned strokes differ systematically in paddle tilt; a stroke-specific drag estimate could reorder the efficiency comparison.","rationale":"The paper's strongest claim is explicitly about efficiency, and the efficiency metric is built on a drag coefficient that is measured in a single, fixed configuration. The learned strokes have mean paddle orientations that differ substantially from that configuration, and the different stroke families differ from each other in mean tilt. Because Stokes-flow drag depends on appendage orientation, this is the point where the central comparison is least secure. The proposed check is inexpensive using the authors' own solver and would directly settle whether the efficiency ordering is an artifact of the chosen reference drag. I agree with the reader that this is the weakest assumption; my read does not change the CONDITIONAL verdict. The RL convergence variability in Section 3.5 is a secondary robustness concern, not the main threat to the central claim, since the efficiency claim is about the strokes that were actually learned.","tokens_in":16092,"tokens_out":5559,"duration_ms":69700,"concrete_test":"For each learned stroke in Figure 8, compute a stroke-specific drag coefficient using the existing regularized-Stokeslet solver: hold the paddles fixed at each configuration visited during the stroke, tow the swimmer at unit speed, average the resulting tow force over the stroke cycle to obtain ζ_eff, and replace the Appendix B value in η = ζU²/P. Then check whether the back-to-front stroke still has the highest efficiency for each number of paddles. If any front-to-back or 3-paddle non-wave stroke becomes the most efficient, the abstract's efficiency claim fails; if the ranking is unchanged, the perpendicular-paddle ζ is adequate for the comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim rests on the metric η = ζU²/P in Section 3.4, where ζ is taken from Appendix B. That table is computed by fixing all paddles perpendicular to the body (θ = 0) and towing the swimmer at unit speed. The same ζ per (n, spacing) is then used for every learned stroke. The learned strokes are not small perturbations of this reference configuration: Table C2 shows back-to-front strokes with mean paddle states near ±2 (about 18° tilt) and front-to-back strokes with mean states up to ±3.72 (about 33° tilt). In Stokes flow, the drag coefficient of a flat appendage depends strongly on orientation, so the perpendicular-paddle ζ is not necessarily representative of either stroke, and it is least representative of the more tilted front-to-back strokes. Because the paper's headline claim is a comparison of efficiencies across strokes with systematically different mean paddle orientations, a configuration-fixed ζ can bias the ordering. The paper does not test or discuss this assumption in Appendix B or Section 3.4. The RL parameter study in Section 3.5 is useful, but it does not address this modeling assumption. If the true stroke-averaged ζ differs from the perpendicular value in a stroke-dependent way, the reported efficiency ranking, and with it the claim that the back-to-front stroke is always most efficient, could change.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper uses tabular Q-learning to discover optimal paddling gaits for a two-dimensional swimmer at zero Reynolds number. The swimmer consists of an elongated body with two, three, or four pairs of rigid paddles, and the paddle angles are discretized into eleven states. The reward is the net forward displacement per action, so the learning agent optimizes swimming speed. For tight paddle spacings, the agent learns a back-to-front (antiplectic) metachronal stroke, while for wide spacings it learns front-to-back strokes in which paddles tilt inward. The speed-optimal stroke depends on paddle number, but the authors report that the back-to-front stroke is the most efficient across two, three, and four paddle pairs. The hydrodynamic problem is solved with regularized Stokeslets, and efficiency is computed with a Lighthill-type metric using a drag coefficient obtained by towing the swimmer with perpendicular paddles. A parameter study in Section 3.5 examines sensitivity to training length, discount factor, exploration rate, and learning rate.","tokens_in":16323,"tokens_out":9404,"duration_ms":106532,"significance":"If the results are robust, the paper provides a useful demonstration that model-free reinforcement learning can recover biologically observed antiplectic metachrony in a minimal rigid-paddle model, and it identifies a clear separation between speed-optimal and efficiency-optimal coordination. The qualitative comparison with the biological literature on ciliates and other metachronal swimmers is a strength, and the parameter study in Section 3.5 is a genuine robustness check. The hydrodynamic model and Q-learning setup are standard, and the finding that efficiency, rather than speed, selects the antiplectic wave across different paddle numbers is potentially valuable for designing low-Reynolds-number robotic swimmers.","major_comments":[{"comment":"The efficiency metric η = ζU²/P uses a single drag coefficient ζ per paddle number and spacing, obtained by towing the swimmer with all paddles fixed perpendicular to the body (θ=0). The learned strokes are not small perturbations of this reference: Table C2 reports mean paddle states of about ±2 (≈18°) for the back-to-front strokes and values up to ±3.72 (≈33°) for the front-to-back strokes. Because the drag of an inclined flat appendage in Stokes flow depends on orientation, the perpendicular-paddle ζ may not be representative of either stroke. The bias likely runs in the direction of overestimating the efficiency of the more tilted front-to-back strokes, which would make the main conclusion conservative, but the paper does not state this or test it. I recommend adding a sensitivity analysis that recomputes η using ζ estimated from a stroke-specific reference configuration (for example, the time-averaged or mean-tilt configuration) and reporting whether the efficiency ordering changes.","section":"Section 3.4, Appendix B (Table B1)"},{"comment":"The parameter study reveals a strong sensitivity to the discount factor. At paddle spacing 2 with the longest training loop, γ=0.99 converges to the front-to-back stroke in every trial, but γ=0.98 converges to a suboptimal back-to-front stroke in every trial, and γ=0.97 produces ineffective swimming in all trials. This indicates that the learned strokes are not robustly optimal with respect to small hyperparameter changes. Since the abstract and discussion repeatedly refer to 'optimal' strokes, the paper should either provide a formal optimality check (for example, policy iteration on the converged Q-table, multiple random seeds with the final chosen parameters, or a systematic grid over γ) or consistently soften the terminology to 'learned' or 'RL-selected' strokes. As written, the claim of optimality is stronger than the evidence.","section":"Section 3.5.1, Figure 9"},{"comment":"The headline efficiency claim rests on comparisons of point estimates from only five runs per spacing. The text states that for the two- and four-paddle swimmers the maximum front-to-back efficiency is 'comparable' to the minimum back-to-front efficiency, which suggests a small margin that could be comparable to run-to-run variation. No confidence intervals, standard deviations, or effect sizes are reported for η, and Figure 8 does not include error bars. Given that the central conclusion is an efficiency ranking, the authors should report the quantitative margins (for example, the maximum and minimum η values for each stroke type) and assess whether the ranking is robust to the observed variation across the five runs.","section":"Section 3.4, Figure 8"}],"minor_comments":[{"comment":"The phrase '968 states' should be '968 state-action pairs': the calculation is 121 paddle configurations times 8 actions, which is the number of state-action pairs, not the number of states.","section":"Section 2.2"},{"comment":"The word 'mantitude' should be 'magnitude' in the caption of Table C2.","section":"Appendix C, Table C2 caption"},{"comment":"The caption does not clearly explain the row and column layout of the histograms or what the numbers above the bars represent beyond 'mean swimming speed'; adding a sentence that defines the training-loop combinations for each panel would improve readability.","section":"Figure 9"},{"comment":"The term 'suboptimal' is used without defining the comparison baseline; since the strokes are outputs of the same learning algorithm, the authors should specify that 'suboptimal' means slower than the best stroke found with other hyperparameter choices.","section":"Section 3.5.1"},{"comment":"The code for the Q-learning algorithm and the regularized Stokeslet solver is not provided, which makes it difficult to reproduce the stochastic RL results; making the code and random seeds available would strengthen reproducibility.","section":"Data Availability"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid numerical study, but the novelty relative to existing RL-for-microswimmer papers (References 16, 17, 24) is incremental; the main new content is the efficiency ordering across paddle numbers. The central efficiency claim needs the additional sensitivity and statistical support described in the major comments. If the authors can provide the drag-coefficient sensitivity analysis and quantitative efficiency margins, the paper would be suitable for publication in a fluids journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my take on the Bailey–Guy paper. The headline result is worth knowing: tabular Q-learning on a rigid-paddle swimmer at zero Re rediscovers the antiplectic back-to-front metachronal wave at tight spacings, and the same algorithm finds faster front-to-back strokes at wide spacings. The speed result is robust and new. The efficiency claim—that back-to-front is always the most efficient stroke regardless of paddle number—is the part I would push on.\n\nWhat the paper does well: the model is minimal but appropriate, the RL setup is standard and clearly described, and the authors actually test the learning parameters. Section 3.5 shows that their conclusions depend on the discount factor, and they report that sensitivity honestly. The stroke metrics (phase lag, amplitude, mean tilt) are sensible, and the Appendix C table gives enough detail to see what the learned strokes look like. Citations are fair.\n\nThe soft spot is the drag coefficient used in the efficiency metric. In Section 3.4 and Appendix B, ζ is computed by towing the swimmer with all paddles pinned perpendicular to the body, and that single value is used for every learned stroke at that spacing. But the strokes themselves have systematically different paddle orientations: Table C2 lists mean paddle states up to ±3.72 (about 33° tilt) for the front-to-back strokes. In Stokes flow the drag of a flat appendage depends strongly on orientation, so the perpendicular-paddle ζ is not obviously representative. If the time-averaged drag of the front-to-back strokes is larger than the perpendicular value—which seems likely, since the paddles are tilted into the flow—their efficiency would be underestimated. That could reorder the efficiency comparison, which is the paper's central claim. The speed results do not depend on this assumption, so they stand.\n\nSecondary issues: no code or data posted, just 'available upon reasonable request,' and the main plots show scatter from five runs but no error bars. Those are minor next to the drag issue.\n\nThis paper deserves a real referee. The speed finding is solid and the efficiency question is exactly the right one to raise. A referee should ask the authors to check the drag approximation, either by computing a stroke-averaged ζ or by reporting efficiency over a range of plausible values. If the efficiency ranking survives that check, the paper is a nice contribution. I would bring it up at our group meeting, but I would not cite it until the drag issue is resolved.","headline":"A clean RL study that recovers antiplectic metachrony as efficient, but the efficiency claim rests on a drag coefficient measured for one paddle posture and used for all strokes.","tokens_in":16919,"tokens_out":3640,"would_cite":false,"duration_ms":38461,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that for a rigid-paddle swimmer at zero Reynolds number, the back-to-front (antiplectic) wave is the most efficient learned stroke for two, three, and four paddle pairs, even though the fastest stroke changes with paddle…","keywords":["metachronal paddling","reinforcement learning","low Reynolds number","Stokes flow","antiplectic wave","Q-learning","swimming efficiency","rigid paddles"],"falsifier":"Take two learned strokes at the same spacing and paddle count, measure the time-averaged drag force during each stroke by integrating the computed forces over one cycle, and recompute each efficiency with the measured drag value. If the front-to-back stroke's efficiency then reaches or exceeds the back-to-front stroke's, the paper's central conclusion fails for that configuration.","tokens_in":15827,"feed_emoji":"🦠","tokens_out":9128,"duration_ms":90925,"temperature":0.7,"pith_summary":"The paper asks whether a low-Reynolds-number swimmer made of rigid, unjointed paddles will rediscover metachronal paddling—the wave-like, time-staggered limb rhythm used by ciliated organisms, krill, and mantis shrimp—when left to learn on its own. A tabular Q-learning agent optimizes forward displacement for swimmers with two, three, and four pairs of straight paddles at several fixed spacings. It learns two main gaits: a back-to-front (antiplectic) wave at tight spacings and a front-to-back (symplectic) stroke at wide spacings, with additional non-wave strokes appearing for three paddles at the largest spacings. The paper's central claim is that the fastest stroke depends on the number of paddles, but the most efficient stroke is the back-to-front wave-like stroke for every paddle count considered. If correct, this points to efficiency, not speed, as a likely selection pressure behind the antiplectic metachronal rhythm observed across biological swimmers.","feed_headline":"Back-to-front paddle wave wins on efficiency at low Reynolds number","feed_subtitle":"Reinforcement learning finds the natural back-to-front rhythm is most efficient for 2, 3, and 4 paddle pairs, though not always fastest.","key_machinery":"The central object is a two-dimensional 'paddler': a long body with pairs of straight, inflexible paddles, each with one angular degree of freedom discretized into 11 states and a stepwise action moving it left, right, or not at all. The learning loop is tabular Q-learning with the update rule $Q(s,a)\\leftarrow (1-\\alpha)Q(s,a)+\\alpha[r+\\gamma\\max_{a'}Q(s',a')]$, and the reward for each step is the swimmer's net forward displacement over that time interval. Hydrodynamic forces and swimming speed are computed by solving the Stokes equations with the method of regularized Stokeslets, subject to zero net force on the swimmer. Stroke performance is then compared with the efficiency ratio $\\eta = \\zeta U^2/P$, using a drag coefficient measured by towing the swimmer with paddles held perpendicular to the body. Since the limbs cannot bend, any swimming motion must come from rhythm alone, which is what lets the paper attribute the learned gaits purely to coordination timing.","core_discovery":"Across the strokes the agent learns, the back-to-front wave is the most efficient coordination pattern for two, three, and four paddle pairs, while the fastest stroke is not universal: with two and four paddles a front-to-back, inward-tilted stroke is faster at wide spacings, and with three paddles the back-to-front stroke is fastest. The gaits are learned by Q-learning maximizing each move's net forward displacement, so efficiency is not part of the reward; it is measured afterward using the standard ratio $\\eta = \\zeta U^2/P$, where $U$ is the average swimming speed, $P$ the average mechanical power, and $\\zeta$ the drag coefficient. Because the paddles are rigid and each has a single angular degree of freedom, the only time asymmetry available to the agent is the ordering and timing of paddle motions. The paper takes this result as evidence that the antiplectic rhythm seen in nature can be an efficiency-optimal solution even when the swimmer is free to choose any coordination.","pith_inferences":["The efficiency comparison could be sharpened by measuring each stroke's own time-averaged drag instead of using the single towed value; this is a direct, testable check of whether the efficiency ranking survives the paddles' actual tilted orientations.","An agent trained to maximize efficiency directly, rather than speed, might settle on a different gait; the paper notes this requires changing the action space or reward, so an actor-critic or deep-Q variant is the natural next experiment.","Because Stokes flow has no intrinsic length scale, the spacing threshold near 2 units should scale with paddle length and body geometry; varying those dimensions in the same model would test whether the threshold is a ratio rather than an absolute distance.","The fact that speed and efficiency diverge most at wide spacings suggests that organisms with widely spaced limbs may face a sharper speed-efficiency trade-off than those with tight arrays; comparing learned gaits across more paddle counts would map this trade-off."],"forward_implications":["In this model, an antiplectic back-to-front wave is the efficiency-optimal coordination for two, three, and four paddle pairs, so the biologically common rhythm can arise from a pure efficiency objective without any prescribed wave rule.","Because the speed-optimal and efficiency-optimal strokes differ in most configurations, a microrobot designer must choose explicitly between maximizing distance per time and maximizing distance per energy; the same gait will not do both.","The learned gait switches from back-to-front to front-to-back near a paddle spacing of about 2 body units for all paddle counts, identifying appendage spacing as a control parameter for gait selection.","The back-to-front stroke is both fastest and most efficient at tight spacings, which is consistent with closely packed cilia arrays in biology and suggests a simple design rule: pack appendages tightly for efficient antiplectic swimming."],"supporting_citations":[{"why":"Establishes that reciprocal motion produces no net displacement at zero Reynolds number, which is why the learned strokes must rely on timing asymmetries.","marker":"[1]"},{"why":"Demonstrates that reinforcement learning can discover swimming gaits for simplified low-Reynolds-number swimmers, providing the methodological precedent for this study.","marker":"[16]"},{"why":"Supplies the Q-learning framework and update rule used to train the paddler agent.","marker":"[19]"},{"why":"Provides the regularized Stokeslet numerical method used to solve the Stokes flow equations and compute swimming velocity and forces.","marker":"[22]"},{"why":"Introduces the efficiency ratio used to compare the learned strokes across paddle spacings and limb numbers.","marker":"[23]"}],"fun_headline_variants":["Back-to-front paddle wave emerges as most efficient in RL swimmer","RL finds back-to-front wave is efficiency winner at low Re","Efficiency favors back-to-front paddle rhythm in RL swimmer","Back-to-front wave: most efficient paddle gait learned by AI","RL swimmer: back-to-front wave beats other gaits on efficiency"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the water resistance measured when the paddles are held straight out is close to the resistance the swimmer actually feels during each learned stroke; if that is wrong, the efficiency ranking could change.","fun_headline_variants_meta":{"raw":{"variants":["Back-to-front paddle wave emerges as most efficient in RL swimmer","RL finds back-to-front wave is efficiency winner at low Re","Efficiency favors back-to-front paddle rhythm in RL swimmer","Back-to-front wave: most efficient paddle gait learned by AI","RL swimmer: back-to-front wave beats other gaits on efficiency"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000355,"raw_usage":{"total_tokens":1939,"prompt_tokens":968,"completion_tokens":971,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":882}},"tokens_in":584,"tokens_out":971,"duration_ms":7166,"temperature":1.0,"reasoning_tokens":882,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:07:17.083985+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two learned strokes at the same spacing and paddle count, measure the time-averaged drag force during each stroke by integrating the computed forces over one cycle, and recompute each efficiency with the measured drag value. If the front-to-back stroke's efficiency then reaches or exceeds the back-to-front stroke's, the paper's central conclusion fails for that configuration.","supporting_citations":[{"cited_title":"In: Physics and Our World: Reissue of the Proceedings of a Symposium in Honor of Vic- tor F Weisskopf, pp","cited_arxiv_id":null,"evidence_quote":"Establishes that reciprocal motion produces no net displacement at zero Reynolds number, which is why the learned strokes must rely on timing asymmetries."},{"cited_title":"Physical Review Fluids 5(7), 074101 (2020)","cited_arxiv_id":null,"evidence_quote":"Demonstrates that reinforcement learning can discover swimming gaits for simplified low-Reynolds-number swimmers, providing the methodological precedent for this study."},{"cited_title":"SIAM Journal on Scientific Com- puting 23(4), 1204–1225 (2001)","cited_arxiv_id":null,"evidence_quote":"Provides the regularized Stokeslet numerical method used to solve the Stokes flow equations and compute swimming velocity and forces."},{"cited_title":"SIAM, Philadelphia (1975)","cited_arxiv_id":null,"evidence_quote":"Introduces the efficiency ratio used to compare the learned strokes across paddle spacings and limb numbers."}],"review_version":2}