Pith. sign in

REVIEW 5 major objections 5 minor 22 references

A meta-bandit model reproduces individual bees' decisions, with an optimal memory window around seven trials.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

MAYA reproduces individual bee left/right choices by matching regret trajectories to four bandit policies with a memory window fixed at tau=7, but the tau value and best metric are selected on the same data used for evaluation.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection MAYA is a plausible descriptive model, but the memory and forecasting claims are not supported by the experiments; the paper needs a proper out-of-sample evaluation before those claims can stand. the 5 major comments →

arxiv 2510.16462 v3 pith:XNEL2A2N submitted 2025-10-18 cs.LG stat.ML

Buzz, Choose, Forget: A Meta-Bandit Framework for Bee-Like Decision Making

Emmanuelle Claeys , Elena Kerjean , Jean-Michel Loubes This is my paper
classification cs.LG stat.ML
keywords imitation learningmulti-armed banditsbee cognitionmemory windowY-mazeWasserstein distancetrajectory predictionindividual differences
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces MAYA, an imitation-learning model that treats each bee's sequence of left/right choices in a Y-maze as a mixture of four simple bandit strategies: epsilon-greedy, UCB, LinUCB, and uniform random. It claims that a sliding memory window of about seven past trials best explains bee behavior, with a slight dependence on weather, and that MAYA predicts individual bee trajectories more accurately than standard imitation-learning and statistical baselines. The work matters because it offers an interpretable, per-bee description of learning strategies — distinguishing fast learners who use contextual cues from slow learners who rely on reward history — and provides a generative tool for simulating realistic bee foraging in ecological applications.

Core claim

MAYA works by comparing the bee's cumulative regret trajectory, computed within a sliding window of size tau, to the regret trajectories of candidate bandit policies, and at each trial switching to the policy whose windowed regret is most similar under a chosen distance (Wasserstein, KL, or DTW). Across five bee datasets and a complementary mouse dataset, the paper finds that tau around 7 trials minimizes imitation error, and that MAYA with the Wasserstein distance consistently achieves the best performance. It also reports that alignment rates with LinUCB explain the choices of low-regret (fast-learning) bees, while epsilon-greedy alignment dominates for high-regret (slow-learning) bees. Th

What carries the argument

The central mechanism is windowed regret-trajectory matching. A sliding window tau truncates each policy's history, and a similarity metric — 1-Wasserstein distance, KL divergence, or dynamic time warping — compares the bee's cumulative regret sequence to those of four candidate bandit policies. MAYA chooses the candidate with the smallest distance, then mimics its action, enabling online adaptation to policy shifts and providing per-trial explanations in terms of which bandit the bee's behavior currently resembles.

Load-bearing premise

The model assumes that a bee's next choice depends only on the current stimulus context and the last tau outcomes, and that this behavior is expressible as a mixture of the four specific bandit policies.

What would settle it

Fit MAYA to data from a Y-maze experiment where the reward pattern is systematically linked to a side bias (e.g., left wins but is rewarded only after a delay) or where the correct side depends on a stimulus shown more than tau trials earlier; if MAYA with tau=7 fails to predict choices and a larger window improves accuracy, the seven-trial memory claim is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If tau around 7 is biologically real, it gives a concrete, testable bound on the memory horizon of bees in visual quantity-discrimination tasks, equivalent to roughly 15–30 minutes of the protocol.
  • MAYA's mixture-of-bandits representation can generate synthetic bee trajectories that cluster with real ones, supporting ecological simulations of pollinator behavior under different weather or resource scenarios.
  • Alignment rates provide a quantitative phenotype for individual bees, potentially allowing researchers to classify fast versus slow learners without hand-crafted metrics.
  • The transfer to mouse data suggests the method could be applied to other species' decision-making tasks where behavior is heterogeneous and non-optimal.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The tau ~7 value may be partly an artifact of the four-policy library: if the true decision process involves strategies outside the library (e.g., side-alternation or inter-trial timing), the fitted window could simply be the one that best mimics the library's limitations rather than a direct measure of memory.
  • A testable extension would manipulate the inter-trial interval or insert distractor trials; if tau remains constant, it suggests a trial-count-based memory, while a shift would indicate time-based forgetting.
  • The weather-dependent shift in optimal tau might reflect lower attention or motivation, but it could also stem from changes in the bees' visiting rhythm, which the current model does not explicitly represent.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces MAYA, a meta-bandit imitation algorithm for modeling individual bee decisions in a two-alternative Y-maze. MAYA compares the bee's cumulative regret trajectory against four candidate bandit policies (Epsilon-Greedy, UCB, LinUCB, Uniform), selects the closest policy at each trial under a windowed distance (KL, Wasserstein, or DTW), and reports MSE/MAE of the reproduced actions. The paper claims that MAYA can predict individual bee choices, that the optimal memory window is approximately τ=7 with a mild weather dependence, that MAYA outperforms established imitation-learning and statistical baselines on five bee datasets and a mice dataset, and it presents worst-case regret bounds in Appendix 14.

Significance. If the claims were supported, MAYA would be a valuable interpretable tool for behavioral ecology, and the open dataset of 80 tracked bees across five weather/location conditions is a genuine asset. The in-sample imitation comparisons are internally consistent, and the mice extension is a useful robustness probe. However, the paper's headline contributions — forecasting individual decisions and identifying a biologically meaningful memory horizon near seven trials — are not established by the current experimental protocol. The evaluation is in-sample, τ is a fitted hyperparameter, and the mathematical appendix rests on unproved assumptions.

major comments (5)
  1. [§4.1, Algorithm 1, Tables 4–8] There is no train/validation split. In Algorithm 1, at trial t the method observes R(π_bee,1,t−1), selects ξ_t by minimizing the regret distance, and only then samples an action at t. The MSE/MAE curves in Figure 2 and Tables 4–8 therefore measure one-step in-sample imitation on the same trajectories used to select τ and the distance metric. The abstract's claim to 'predict individual bees' decisions' and the headline 'optimal τ around 7' are not supported; τ=7 is the best-fitting window on the training data, not a validated forecasting horizon. A held-out suffix or a forward-simulation protocol with τ fixed on a training set is required.
  2. [§2, Algorithm 1, §5] The paper presents τ as 'the memory of the animal' that 'truncates the observation history,' but the candidate policies (UCB, LinUCB, Epsilon-Greedy) maintain full-history estimates of Q_t(a), N_t(a), or Θ̂. τ only truncates the interval used to compute the regret-matching distance; it does not truncate the fitted policy's memory. Therefore the biological reading of τ≈7 as a bee's memory span (abstract, §5) is not justified by the mechanism; at most τ is the length of recent regret history used for policy selection.
  3. [Appendix 14, Eqs. (1), (10), (12)] The theoretical bounds rely on the unproved assertion P[ε_t = ε*_t] = 0.5 in the stationary worst case and on the claim that distances are 'well defined' in cyclic regimes. Neither statement is derived; the 0.5 probability in particular appears to be an assumption about tie-breaking that is not established. Moreover the bounds concern regret gaps under an idealized switching process and do not imply forecast accuracy or validate the τ≈7 recommendation. Eq. (12) also appears to contain an algebraic typo ('−2ST' should likely be '−2S').
  4. [§4.2, Table 1] AIRL is reported with MSE=0±0 and MAE=0±0 on every dataset. The authors interpret this as 'full copy-paste' with no generalization, but reporting a memorizing baseline with zero error in a table headlined as a comparative performance evaluation is misleading, especially without a held-out test set. Either remove AIRL from the comparison or report its out-of-sample generalization error; as presented, the zero-error entries obscure the comparison.
  5. [§12] The mice experiment is described as containing '19 rats' and then 'we reduce the number of individual at 100.' This is internally inconsistent and suggests a data-processing error or a missing explanation. Also, the abstract mentions experiments on 'simulated' datasets, but no externally generated simulated benchmark is described; the simulated trajectories in §4.4 are MAYA's own outputs used for clustering, not a ground-truth simulated dataset.
minor comments (5)
  1. [Algorithm 1, §2] The notation R(π, τ, 1, t−1) is inconsistent with the definition R(π, τ, 1, T) = Σ_{t=τ}^T Δ_{π,t}. In the windowed phase the algorithm intends the sliding-window sum over [t−τ, t−1], not a sum starting at index τ. Please redefine the notation.
  2. [§4.1] The statement that 'the optimal window length decreases under adverse weather conditions' is imprecise: the reported ranges are cold τ∈[5,7], moderate τ∈[6,8], hot τ∈[7,10]. Please clarify what 'adverse' means and whether the trend is supported by significance testing or only by point estimates.
  3. [§12] The section is titled 'Mice Dataset' but describes rats from Ashwood et al. (2020). Please correct the terminology throughout.
  4. [Throughout] The algorithm name is typeset inconsistently as 'MAYA', 'MAY A', and 'MAY A-Wass'. Please standardize.
  5. [§8] The GLM baseline description says it models the regret trajectory R(π,1,T) as a function of time, but the evaluation in Table 1 compares MSE/MAE of reproduced actions. Clarify whether the GLM is fit to actions or to regret, and how its predictions are converted to actions.

Circularity Check

3 steps flagged

The reported τ≈7 memory horizon and action-level accuracy are in-sample fits: τ is selected by minimizing the same MSE/MAE used for headline performance, and Algorithm 1 evaluates one-step imitation on logged regret, so the 'prediction' and 'memory' claims reduce to the fitting protocol.

specific steps
  1. fitted input called prediction [Section 4.1 'Best window size and distance metrics']
    "Across all datasets, the results confirm the trend that for τ∈[5,10] the losses decrease. ... According to this observation, we suggest fixing τ=7 to handle multiple weather conditions. Then, we fix τ=7 for the rest of the paper. With this window, MAY A–Wass provides the best results across all datasets."

    The abstract's claim that 'the optimal value [of τ] is around 7 trials' is obtained by minimizing the same MSE/MAE loss on the same logged bee trajectories later used to report performance. There is no held-out split, no forward-simulation checkpoint, and no independent behavioral measure of memory. The 'optimal τ' is the argmin of the reported in-sample loss, so the headline memory horizon is a fitted hyperparameter restated as a finding, not an independently predicted property.

  2. fitted input called prediction [Algorithm 1, App. 11, and Section 4 'Experimental Evaluation']
    "for t∈ {2, . . . , τ−1} do ... Observe R(π bee,1, t−1) ... ξ t = argmin π∈P δ(πbee, π, t) ... Select A t ∼π θ(at|st−1)"

    At every trial the algorithm reads the bee's regret up to t−1, selects the candidate policy whose regret trajectory is closest to the bee's on that logged prefix, and then samples the action used in the MSE/MAE evaluation. The evaluation compares this sampled action with the bee's action at the same trial on the same dataset, with no train/test suffix and no future simulation. The reported 'prediction' of individual bees' decisions is therefore a one-step in-sample imitation score, not an out-of-sample forecast.

  3. self definitional [Section 2 'Preliminaries' and Section 5 'Discussion']
    "we define here the memory of the animal by a parameter, τ, that truncates the observation history to the τ most recent observations. This parameter needs to be optimized in the imitation learning. ... a memory window of (τ=7) consistently yielded the best fit; this choice is corroborated by complementary experiments and corresponds roughly to 15–30 minutes in our protocol"

    The paper defines 'memory' as the window τ, optimizes τ against the behavioral data, and then presents the optimized value as a biologically meaningful result ('bees' limited memory'). In Algorithm 1, τ does not truncate the candidate policies' internal memory: UCB, LinUCB, and Epsilon-Greedy keep full-history reward counts and estimates, while τ only truncates the regret interval used for policy selection. The biological 'memory horizon' is thus the fitted alignment window by definition, not an independently measured cognitive limit.

full rationale

The derivation chain is not externally circular via self-citation: the self-citation to Besse et al. (2015) is only a literature pointer, and no uniqueness theorem is imported from the authors' prior work. The central circularity is internal and methodological. The paper's two headline outcomes—'τ≈7 trials' and 'MAYA predicts individual bees' decisions'—are both generated by the same fitting loop. Section 4.1 chooses τ by minimizing MSE/MAE on the five datasets, then reports that value as the optimal memory window and fixes it for all subsequent comparisons; the same data then appear in Tables 1 and 2 as evidence of predictive quality. Algorithm 1 confirms the in-sample nature of the fit: each trial uses the bee's logged regret up to t−1 to select the closest policy from a four-policy library, and the reported error is computed by comparing the next sampled action with the bee's actual action on that same logged trial. There is no held-out suffix, no temporal forecast, and no simulation forward in time, so the abstract's 'predict individual bees' decisions' claim is not tested. Finally, the interpretation of τ as a memory parameter is self-definitional: the paper defines animal memory as τ, fits τ, and then reports the fitted value as a discovered biological constraint, even though the candidate policies themselves retain full-history estimates. These issues make the central memory claim and the action-prediction claim partially circular, warranting a score of 6 rather than a higher score, because the model and comparisons to baselines still contain substantial independent algorithmic content.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The model's inferences rest on a small candidate-policy library, a regret-similarity proxy, and a hyperparameter tau chosen by in-sample fit. No new physical entities are introduced; the main modeling construct is the sliding memory window, which is a fitted modeling choice rather than an independently evidenced biological mechanism.

free parameters (3)
  • memory window tau = 7 (per-dataset best in [3,30])
    Selected by minimizing MSE/MAE on the same five datasets (Sec 4.1, Tables 4-8) and then fixed as a biological finding; no held-out validation.
  • similarity metric = Wasserstein
    Chosen because it achieved lowest MSE on the same logged trajectories; the choice is part of the model-selection loop, not an independent prediction.
  • MAB hyperparameters = not reported (epsilon for epsilon-greedy, lambda/ridge for LinUCB)
    The candidate policies need hyperparameters; the paper does not state how they were set, so they are implicit free parameters.
axioms (6)
  • domain assumption Reward at trial t does not depend on state s_{t-1}; the task is a 2-armed bandit, not a general MDP
    Section 2 states this explicitly to justify the bandit simplification; if bees integrate longer history or side-alternation structure, the model is misspecified.
  • domain assumption Bee learning strategies are covered by Epsilon-Greedy, UCB, LinUCB, and Uniform bandits
    Section 2 lists these four policies as the only candidate behaviors; MAYA cannot express any strategy outside this library, so the fitted policy mixture is constrained by this choice.
  • domain assumption Regret-trajectory distance is a valid proxy for behavioral similarity
    Section 3 defines imitations as minimizing d between regret trajectories; the paper does not establish that regret closeness implies action/cognitive closeness.
  • standard math Ross et al. bound applies: if the learned policy is epsilon-close to the expert, expected cost is at most epsilon*T
    Invoked in Section 2 to justify using action-mismatch cost as the success metric; assumes the stated epsilon condition holds.
  • ad hoc to paper In the stationary worst case, P[epsilon_t = epsilon*_t] = 0.5 because no best agent is better than the other
    App 14 asserts this equality to derive the upper bound in Eq. (1); it is not proven and is the central step of the bound.
  • ad hoc to paper In cyclic cases, the distance is 'well defined' in the sense that MAYA selects the closest agent except during explicitly modeled lag phases
    App 14 repeatedly conditions bounds on 'the distances are well defined' (e.g., before Eq. 3 and Eq. 5) without defining what well-defined means or proving when it holds.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Buzz, Choose, Forget: A Meta-Bandit Framework for Bee-Like Decision Making." pith.science (2026). https://pith.science/paper/XNEL2A2N

@misc{pith2026251016462,
  author       = {Pith},
  title        = {Pith review of: Buzz, Choose, Forget: A Meta-Bandit Framework for Bee-Like Decision Making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XNEL2A2N}},
  note         = {Machine review of arXiv:2510.16462}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This work introduces MAYA, a sequential imitation learning model based on multi-armed bandits, designed to reproduce and predict individual bees' decisions in contextualized foraging tasks. The model accounts for bees' limited memory through a temporal window $\tau$, whose optimal value is around 7 trials, with a slight dependence on weather conditions. Experimental results on real, simulated, and complementary (mice) datasets show that MAYA (particularly with the Wasserstein distance) outperforms imitation baselines and classical statistical models, while providing interpretability of individual learning strategies and enabling the inference of realistic trajectories for prospective ecological applications.

Figures

Figures reproduced from arXiv: 2510.16462 by Elena Kerjean, Emmanuelle Claeys, Jean-Michel Loubes.

Figure 1
Figure 1. Figure 1: MAYA (Multi-Agent Y-maze Allocation) is an imitation learning framework for policy [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparative study of the best window size [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: For bee 1 (fast learner, low regret) and bee 15 (slow learner, high regret) from dataset 2. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: MAYA-Wass [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 9
Figure 9. Figure 9: Proportion of cumulative regret for the five datasets, per bees [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: MAYA-KL [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 15
Figure 15. Figure 15: MAYA-Wass [PITH_FULL_IMAGE:figures/full_fig_p015_15.png] view at source ↗
Figure 18
Figure 18. Figure 18: MAYA-KL [PITH_FULL_IMAGE:figures/full_fig_p016_18.png] view at source ↗
Figure 22
Figure 22. Figure 22: MAYA-KL [PITH_FULL_IMAGE:figures/full_fig_p016_22.png] view at source ↗
Figure 27
Figure 27. Figure 27: Bee 15 [PITH_FULL_IMAGE:figures/full_fig_p016_27.png] view at source ↗
Figure 29
Figure 29. Figure 29: Bees trajectories [PITH_FULL_IMAGE:figures/full_fig_p020_29.png] view at source ↗
Figure 35
Figure 35. Figure 35: MAYA-Wass [PITH_FULL_IMAGE:figures/full_fig_p020_35.png] view at source ↗
Figure 39
Figure 39. Figure 39: Cluster 0 (I) [PITH_FULL_IMAGE:figures/full_fig_p020_39.png] view at source ↗
Figure 45
Figure 45. Figure 45: Cluster 1 (I) [PITH_FULL_IMAGE:figures/full_fig_p021_45.png] view at source ↗
Figure 50
Figure 50. Figure 50: Cluster 1 (I) [PITH_FULL_IMAGE:figures/full_fig_p021_50.png] view at source ↗
Figure 54
Figure 54. Figure 54: Left : experimental description of the Mice Dataset. Right : Comparative study of the [PITH_FULL_IMAGE:figures/full_fig_p023_54.png] view at source ↗
Figure 55
Figure 55. Figure 55: Proportion of cumulative regret for the Mice dataset, per mice [PITH_FULL_IMAGE:figures/full_fig_p024_55.png] view at source ↗
Figure 56
Figure 56. Figure 56: MAYA-KL [PITH_FULL_IMAGE:figures/full_fig_p024_56.png] view at source ↗
Figure 60
Figure 60. Figure 60: MAYA-KL [PITH_FULL_IMAGE:figures/full_fig_p024_60.png] view at source ↗
Figure 65
Figure 65. Figure 65: Mouse 2 [PITH_FULL_IMAGE:figures/full_fig_p025_65.png] view at source ↗
Figure 67
Figure 67. Figure 67: Mouse’ trajectories [PITH_FULL_IMAGE:figures/full_fig_p025_67.png] view at source ↗
Figure 70
Figure 70. Figure 70: Cluster 0 [PITH_FULL_IMAGE:figures/full_fig_p025_70.png] view at source ↗
Figure 73
Figure 73. Figure 73: Y-maze for bees experiments 26 [PITH_FULL_IMAGE:figures/full_fig_p026_73.png] view at source ↗
Figure 74
Figure 74. Figure 74: Maximal cumulative gap between MAYA-regret and Bee-regret in [PITH_FULL_IMAGE:figures/full_fig_p028_74.png] view at source ↗
Figure 75
Figure 75. Figure 75: Maximal cumulative gap between MAYA regret and bee regret in a non-stationary case, [PITH_FULL_IMAGE:figures/full_fig_p030_75.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 3 canonical work pages · 1 internal anchor

  1. [1]

    Ashwood, Nicholas A

    Zoe C. Ashwood, Nicholas A. Roy, Ji Hyun Bak, The International Brain Laboratory, and Jonathan W. Pillow. Inferring learning rules from animal decision-making. InProceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY , USA, 2020a. Curran Associates Inc. ISBN 9781713829546. Zoe C. Ashwood, Nicholas...

    arXiv

  2. [7]

    Red range correspond to±σ(standard deviation)

    predictions and real trajectories (R(πMAYA,1, t)−R(π mice1, t)) (z-axis) for Euclidean (I) Clustering according 0 and 1 Cluster. Red range correspond to±σ(standard deviation). 25 13 COMPLEMENTARY INFORMATION ABOUT THE BIOLOGY INTEREST We share with other vertebrates a basic ability for abstract number representation, thenumber senseDehaene (2011). As earl...

    2011

  3. [8]

    Marco Fiandri, Alberto Maria Metelli, and Francesco Trov `o

    URL https://arxiv.org/abs/2311.14115. Marco Fiandri, Alberto Maria Metelli, and Francesco Trov `o. Sliding-window thompson sampling for non-stationary settings.CoRR, abs/2409.05181,

    Pith/arXiv arXiv

  4. [9]

    Sliding-Window Thompson Sampling for Non-Stationary Settings

    doi: 10.48550/ARXIV .2409.05181. URLhttps://doi.org/10.48550/arXiv.2409.05181. Dylan J. Foster, Adam Block, and Dipendra Misra. Is behavior cloning all you need? understanding horizon in imitation learning,

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2409.05181

  5. [10]

    Justin Fu, Katie Luo, and Sergey Levine

    URLhttps://arxiv.org/abs/2407.15007. Justin Fu, Katie Luo, and Sergey Levine. Learning robust rewards with adversarial inverse rein- forcement learning,

    Pith/arXiv arXiv

  6. [15]

    Scarlett R Howard, Aurore Avargu `es-Weber, Jair E Garcia, Andrew D Greentree, and Adrian G Dyer

    Publisher: American Association for the Advancement of Science. Scarlett R Howard, Aurore Avargu `es-Weber, Jair E Garcia, Andrew D Greentree, and Adrian G Dyer. Symbolic representation of numerosity by honeybees (Apis mellifera): matching characters to small quantities.Proceedings of the Royal Society B, 286(1904):20190238,

    1904

  7. [17]

    11 Peter McCullagh and John Ashworth Nelder.Generalized Linear Models

    URLhttps://arxiv.org/abs/2406.01501. 11 Peter McCullagh and John Ashworth Nelder.Generalized Linear Models. Chapman and Hall/CRC, 2nd edition,

    Pith/arXiv arXiv

  8. [23]

    12 6 APPENDIX 6.1 DATASET DESCRIPTION In this dataset, bees are confronted to a numerical discrimination task

    doi: 10.1109/SSCI47803.2020.9308468. 12 6 APPENDIX 6.1 DATASET DESCRIPTION In this dataset, bees are confronted to a numerical discrimination task. Bees first enter the maze in an entrance chamber before flying in a hole and facing two images located at the end of each arm. The image has different number of dots : for example in dataset 1 and 2, one of th...

    arXiv 2020

  9. [24]

    This encourages the policy to imitate the demonstrations, and to simultaneously avoid states not seen in the demonstrations Reddy et al. (2020). • GLM : A Generalized Linear Model (GLM) is a statistical framework that extends linear regression to response variables with non-Gaussian distributions. In our setting, the regret trajectoryR(π,1, T)is modeled a...

    2020

  10. [1960]

    Wenshuai Zhao, Jorge Pe ˜na Queralta, and Tomi Westerlund

    doi: 10.1080/17470216008416717. Wenshuai Zhao, Jorge Pe ˜na Queralta, and Tomi Westerlund. Sim-to-real transfer in deep rein- forcement learning for robotics: a survey. In2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 737–744,

    doi:10.1080/17470216008416717

  11. [1972]

    Andreas Nieder

    doi: 10.2307/2344614. Andreas Nieder. The neuronal code for number.Nature Reviews Neuroscience, 17(6):366–382,

    doi:10.2307/2344614

  12. [2002]

    Philippe Besse, Brendan Guillouet, Jean-Michel Loubes, and Royer Franc ¸ois

    doi: 10.1023/A:1013689704352. Philippe Besse, Brendan Guillouet, Jean-Michel Loubes, and Royer Franc ¸ois. Review and perspec- tive for distance based trajectory clustering,

    doi:10.1023/a:1013689704352

  13. [2006]

    2006.01719.x

    doi: 10.1111/j.1467-9280. 2006.01719.x. PMID: 16683927. Nuno Capela, Xiaodong Duan, El ˙zbieta M. Zi ´ołkowska, and Christopher John Topping. Mod- elling foraging strategies of honey bees as agents in a dynamic landscape representation.F ood and Ecological Systems Modelling Journal, 5:e99103,

    Pith/arXiv arXiv 2006

  14. [2009]

    URLhttp://www.jstor.org/stable/ 10.1086/603629

    ISSN 00030147, 15375323. URLhttp://www.jstor.org/stable/ 10.1086/603629. Jessica F. Cantlon and Elizabeth M. Brannon. Shared system for ordering small and large numbers in monkeys and humans.Psychological Science, 17(5):401–406,

    work page doi:10.1086/603629

  15. [2010]

    Stephan Lochner, Daniel Honerkamp, Abhinav Valada, and Andrew D

    doi: 10.1145/1772690.1772758. Stephan Lochner, Daniel Honerkamp, Abhinav Valada, and Andrew D. Straw. Reinforcement learn- ing as a robotics-inspired framework for insect navigation: From spatial representations to neural implementation,

    arXiv

  16. [2011]

    Richard S

    URLhttps://arxiv.org/abs/ 1011.0686. Richard S. Sutton and Andrew G. Barto.Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA,

    Pith/arXiv arXiv

  17. [2015]

    doi: 10.1038/nature14422

    ISSN 1476-4687. doi: 10.1038/nature14422. URLhttp://dx.doi.org/10.1038/nature14422. Marie Dacke and Mandyam V Srinivasan. Evidence for counting in insects.Animal cognition, 11: 683–689,

    doi:10.1038/nature14422

  18. [2016]

    Scarlett R Howard, Aurore Avargu `es-Weber, Jair E Garcia, Andrew D Greentree, and Adrian G Dyer

    URLhttps: //arxiv.org/abs/1606.03476. Scarlett R Howard, Aurore Avargu `es-Weber, Jair E Garcia, Andrew D Greentree, and Adrian G Dyer. Numerical ordering of zero in honey bees.Science, 360(6393):1124–1126,

    Pith/arXiv arXiv

  19. [2018]

    Maxence G ´erard, Anahit Amiri, B ´er´enice Cariou, and Emily Baird

    URLhttps://arxiv.org/abs/1710.11248. Maxence G ´erard, Anahit Amiri, B ´er´enice Cariou, and Emily Baird. Short-term exposure to heatwave-like temperatures affects learning and memory in bumblebees.Global Change Biol- ogy, 28(14):4251–4259,

    Pith/arXiv arXiv

  20. [2022]

    Hans J Gross, Mario Pahl, Aung Si, Hong Zhu, J ¨urgen Tautz, and Shaowu Zhang

    URLhttps://arxiv.org/ abs/2211.11972. Hans J Gross, Mario Pahl, Aung Si, Hong Zhu, J ¨urgen Tautz, and Shaowu Zhang. Number-based visual generalisation in the honeybee.PloS one, 4(1):e4263,

    Pith/arXiv arXiv

  21. [2023]

    Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret

    URLhttps://arxiv.org/abs/ 1706.03741. Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. Robots that can adapt like animals.Nature, 521(7553):503–507, May

    Pith/arXiv arXiv

  22. [2024]

    URL https://doi.org/10.3897/fmj.5.99103

    doi: 10.3897/fmj.5.99103. URL https://doi.org/10.3897/fmj.5.99103. Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences,

    work page doi:10.3897/fmj.5.99103

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.