{"paper":{"title":"Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery","license":"http://creativecommons.org/licenses/by-nc-nd/4.0/","headline":"A running clock since the last odor whiff lets a Q-learning agent learn surging, casting and downwind return to recover plumes in turbulence.","cross_cats":["cs.LG"],"primary_cat":"physics.bio-ph","authors_text":"Agnese Seminara, Marco Rando, Robin A. Heinonen, Yujia Qi","submitted_at":"2026-05-15T13:19:26Z","abstract_excerpt":"Finding an odor source in a turbulent flow requires effectively leveraging the history of olfactory observations into a robust navigation strategy. In this work, we use tabular Q-learning to train an olfactory search agent with a minimal memory of past observations: only a running clock since the last whiff. This agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"The agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That a single running clock since the last whiff supplies enough state information for tabular Q-learning to converge on a robust recovery policy across varying turbulence intermittency levels.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"A tabular Q-learning agent with clock-state memory learns surging, casting, and downwind return to recover odor plumes in turbulent flows from direct numerical simulations.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"A running clock since the last odor whiff lets a Q-learning agent learn surging, casting and downwind return to recover plumes in turbulence.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"d06873dce44d5c6dd6c8ee82540c8be486836bb8b37b253b6982c6ab2983a20c"},"source":{"id":"2605.15938","kind":"arxiv","version":1},"verdict":{"id":"e726d64a-9753-48a2-84b2-f0a64a8e4495","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-19T17:30:50.526306Z","strongest_claim":"The agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.","one_line_summary":"A tabular Q-learning agent with clock-state memory learns surging, casting, and downwind return to recover odor plumes in turbulent flows from direct numerical simulations.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That a single running clock since the last whiff supplies enough state information for tabular Q-learning to converge on a robust recovery policy across varying turbulence intermittency levels.","pith_extraction_headline":"A running clock since the last odor whiff lets a Q-learning agent learn surging, casting and downwind return to recover plumes in turbulence."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2605.15938/integrity.json","findings":[],"available":true,"detectors_run":[{"name":"doi_title_agreement","ran_at":"2026-05-19T18:01:18.546166Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"doi_compliance","ran_at":"2026-05-19T17:36:25.318165Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"ai_meta_artifact","ran_at":"2026-05-19T17:33:46.535698Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"claim_evidence","ran_at":"2026-05-19T17:01:55.729910Z","status":"completed","version":"1.0.0","findings_count":0}],"snapshot_sha256":"f2865797bfd1b74c9ed1edd3c5e2968743a00d2094d1ce9cf191ae4a83de2e7b"},"references":{"count":46,"sample":[{"doi":"","year":2018,"title":"Elementary sensory-motor transfor- mations underlying olfactory navigation in walking fruit- flies.Elife, 7:e37815, 2018","work_id":"c253cc45-23dc-450a-a57d-fd98320105f7","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":1990,"title":"T. C. Baker. Upwind flight and casting flight: com- plementary and tonic systems used for location of sex pheromone sources by male moths.Proc. 10 th Intl Sym- posium on Olfaction and Taste, 13:18, 19","work_id":"eaad9319-4733-4e39-b5fe-24a127f2d19c","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":1987,"title":"Manoeuvres used by flying male oriental fruit moths to relocate a sex pheromone plume in an experimentally shifted wind-field","work_id":"45ac64d6-f1aa-4aa9-a5de-6cdcb7e3b0a0","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2002,"title":"Olfactory search at high Reynolds number.Proceedings of the na- tional academy of sciences, 99(20):12589–12593, 2002","work_id":"79c1978c-57c3-4b8a-8e31-021445a2e6d7","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":1979,"title":"W. J. Bell and E. Kramer. Search and anemotaxis in insects.J. Insect Physiol, 25:631–640, 1979","work_id":"49638bd7-2d3b-442c-afef-2b87ad020902","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":46,"snapshot_sha256":"ede11630c2f807c9704fd9a236107e8b9ee810f8b699839dea5bc85cd5da3de3","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}