Pith. sign in

REVIEW 3 major objections 5 minor 96 references

A structured-text plus code-agent pipeline retrieves camera-trap events from natural-language queries better than zero-shot video-language models, and shows its work.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 14:52 UTC pith:FKZBXYDH

load-bearing objection First real camera-trap TVR benchmark plus an honest, interpretable pipeline that beats VLMs on the domain; absolute F1 is still modest because perception, not language, is the bottleneck. the 3 major comments →

arxiv 2607.09876 v1 pith:FKZBXYDH submitted 2026-07-10 cs.CV cs.AIq-bio.NCq-bio.QM

Prompting-MammAlps: Fine-Grained Text-to-Video Retrieval for Camera-Trap Data

Valentin Gabeff , Baptiste Maquignaz , Jennifer Shan , Sepideh Mamooler , Gencer Sumbul , Blair Costelloe , Devis Tuia , Alexander Mathis This is my paper
classification cs.CV cs.AIq-bio.NCq-bio.QM
keywords text-to-video retrievalcamera trapswildlife monitoringspatiotemporal action localizationLLM coding agentsinterpretable retrievalecological video benchmarks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Wildlife camera traps generate vast video archives that still require manual sifting when researchers want specific behaviours, ages, sexes, or multi-animal interactions. Existing video-language models do poorly on this domain and give no explanation of why a clip was returned. This paper first builds Prompting-MammAlps—the first text-to-video retrieval benchmark for camera-trap data—pairing 135 ethology-inspired queries with 2 865 densely annotated Alpine mammal videos. It then proposes a two-stage method: a vision transformer (SALMA) that converts each video into a structured JSON of tracks, actions, activities and attributes, and an LLM coding agent that turns the user query into a Boolean predicate using a fixed library of parsing functions. On the held-out test set the pipeline reaches 34 % set-based F1, roughly double the best zero-shot baseline, while remaining fully inspectable. The authors argue that the remaining gap is almost entirely perceptual, not linguistic: when the same agent is given perfect annotations it scores 87 %.

Core claim

On the Prompting-MammAlps test set of 135 ecologically relevant queries and 775 candidate videos, converting camera-trap footage into structured JSON via spatiotemporal action localization and then retrieving with an LLM-generated predicate over a custom parsing library yields a set-based F1 of 0.34—outperforming the best zero-shot video-language model (0.18) and a fine-tuned CLIP4Clip baseline (0.26)—while producing intermediate detections and executable code that a user can inspect and correct.

What carries the argument

SALMA-plus-agent pipeline: a curriculum-trained vision transformer that emits per-track multi-attribute JSON, followed by an LLM coding agent constrained to compose predicates from a library of 23 trusted parsing functions rather than free-form reasoning over raw text.

Load-bearing premise

That the vision model’s frame-level detections of species, age, sex, actions and activities are accurate and temporally consistent enough for simple Boolean predicates over the resulting JSON to recover the true video sets for complex multi-animal queries.

What would settle it

Replace SALMA’s predicted JSON with the ground-truth annotations already released for the same test videos and re-run the identical agent; if the F1 does not jump from 0.34 to near the reported 0.87 oracle figure, the claim that perception (not language) is the dominant bottleneck collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Prompting-MammAlps, the first text-to-video retrieval (TVR) benchmark for camera-trap data: 135 ecologically motivated open-vocabulary queries exhaustively matched to 2865 densely annotated videos from the extended MammAlps-S2 corpus (train 2090 / test 775, day-level split). It also proposes an interpretable retrieval pipeline that (i) runs a curriculum-trained transformer (SALMA) for multi-object tracking and multi-attribute recognition, serializing predictions as structured JSON, and (ii) uses an LLM coding agent constrained to a library of 23 parsing primitives to emit a Boolean predicate per query. On the test set, Agent+SALMA reaches a set-based F1 of 0.34, above the best zero-shot VLM (InternVideo2.0, 0.18) and a fine-tuned CLIP4Clip baseline (0.26); an Agent+Oracle upper bound of 0.87 isolates perception as the dominant bottleneck. Ablations cover LLM choice, parsing-library components, and SALMA curriculum stages.

Significance. The work fills a genuine gap: existing TVR benchmarks are YouTube-centric and poorly matched to ecological use cases, while camera-trap pipelines still rely on manual filtering for fine-grained behavior. The benchmark design (empty queries retained, multi-individual and video-comparison queries, ethology-informed labels) and the explicit Oracle vs. SALMA decomposition are particularly valuable. The constrained code-agent design is a concrete, inspectable alternative to latent similarity retrieval and reduces hallucination risk relative to free-form LLM video description. Even with modest absolute F1, the paper provides a reusable evaluation substrate and a clear diagnosis that domain-specific perception, not language reasoning, is currently the limiting factor—useful for both ecology and domain-adapted VLM research.

major comments (3)
  1. [Table 1, §6.1] Table 1 / §6.1: The reported gain of Agent+SALMA (F1 0.34) over fine-tuned CLIP4Clip (0.26) and zero-shot VLMs is only partially controlled. The agent receives ethogram-aligned structured attributes plus a hand-designed library of 23 primitives, whereas the VLM baselines operate on raw frames (or, for GRAM, frames+audio). The paper notes the asymmetry but does not close it (e.g., by feeding SALMA JSON or free-form captions into a text-only LLM retriever, or by giving a VLM the same attribute vocabulary). Without that control, the gap cannot be cleanly attributed to the retrieval strategy rather than to the intermediate representation.
  2. [Table 3b, §6.2, Supp. Table S2] Table 3b and Supp. Table S2: Action macro-F1 is 0.29, with many query-relevant rare classes at or near 0 (nursing, suckling, playing, escaping, browsing, etc.). The 0.53 drop from Agent+Oracle (0.87) to Agent+SALMA (0.34) is therefore expected to concentrate on rare-attribute and multi-individual queries, but the main text only reports coarse category averages. A per-query or rare-vs-common breakdown of that drop is needed to substantiate the claim that perception—not parsing—is the bottleneck for the ecological questions the benchmark targets.
  3. [§5, Table 2, Supp. Table S3] §5 / Table 2: LLM agent results appear to be single-run (one generation trajectory per query, up to 10 retries). Given that code generation is stochastic and that several Oracle F1s in Supp. Table S3 are already <1 (logic errors), variance across seeds or temperature settings should be reported, at least for Qwen3-8B, so that the 0.87 Oracle and 0.34 end-to-end numbers can be interpreted as stable rather than lucky draws.
minor comments (5)
  1. [§5] §5: The set-F1 protocol for empty queries (invert GT and prediction sets) is reasonable but only fully spelled out in the supplement; a one-sentence statement in the main evaluation section would help readers reproduce the aggregate scores.
  2. [Fig. 1, Fig. 4] Fig. 1 and Fig. 4: The illustrative retrieval walkthrough is helpful; adding the corresponding ground-truth match counts next to each returned/missed panel (as partially done) more consistently would make error modes easier to read.
  3. [§4.1] §4.1: The track activation threshold σ_s is introduced but its chosen value and sensitivity are not stated in the main text (only that a fixed threshold is used). A brief note or pointer to the supplement would improve reproducibility.
  4. [§2] Related work: Inquire [76] is correctly positioned as the closest image-side ecological retrieval benchmark; a short explicit contrast on query construction (expert vs. semi-automatic) and on the presence of empty queries would sharpen the novelty claim.
  5. [Abstract, §1] Minor typography: occasional missing spaces after punctuation in the abstract/intro (e.g., “potentialtoretrieve”, “methodsoften”) look like PDF encoding artifacts; worth a pass before camera-ready.

Circularity Check

0 steps flagged

No significant circularity; Agent+Oracle is an explicit diagnostic upper bound using the same GT that defines the benchmark labels, not a hidden circular claim.

full rationale

The paper's central claim is an empirical comparison of set-based F1 on a held-out test split (775 videos, 135 queries) of a newly constructed benchmark. SALMA is trained only on the train split of MammAlps-S2; test videos and the query-video associations are held out. Baselines (CLIP4Clip, SigLIP4Clip, GRAM, InternVideo2.0) are external models evaluated zero-shot or fine-tuned on the same train associations, with decision thresholds calibrated on train only. The Agent+Oracle row (F1 0.87) deliberately applies the LLM-generated predicates to the ground-truth JSON that was used to define the benchmark matches; the paper presents this as an upper-bound diagnostic of parsing quality, not as a claimed prediction. The 0.53 drop to Agent+SALMA (0.34) is therefore a genuine measure of perception error, not a tautology. No equation, free parameter, or uniqueness theorem reduces a reported F1 to a fitted input by construction, and self-citations (prior MammAlps dataset) supply data rather than load-bearing uniqueness results. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 3 invented entities

The central performance claim rests on standard transformer training practice, an ethogram-derived label space, a hand-designed library of 23 parsing primitives, and several inference thresholds; no new physical entities are postulated. The free parameters and domain assumptions listed below are the quantities a re-implementer must accept or re-tune.

free parameters (3)
  • track activation threshold σ_s
    Fixed threshold deciding whether an object query is an active animal track; chosen by hand and used at both training and inference.
  • temporal stride and dual-crop inference settings
    Inference uses stride 4 with forward-fill and two overlapping square crops reconciled by Hungarian matching; these engineering choices affect the JSON content that the agent later parses.
  • LLM agent trial limit and in-context examples
    Agent is allowed up to 10 repair trials and is given three hand-written in-context examples; both are free design choices that materially affect the 0.87 Oracle F1.
axioms (3)
  • domain assumption The 27-action / 13-activity ethogram distilled from Cap et al. and Prikhod’ko & Zvychainaya is an adequate discrete vocabulary for the behaviors that matter to the 135 queries.
    Invoked when constructing MammAlps-S2 labels and the parsing library; if important continuous or multi-scale behaviors are missing, both the benchmark and the method are incomplete.
  • ad hoc to paper A Boolean predicate over frame-level attribute lists is a sufficient retrieval model for the ecological questions of interest.
    The method never produces a graded relevance score; set F1 is the only metric. Continuous notions such as “following” or “proximity” are outside the current formalism.
  • domain assumption Standard DETR/MOTR-style bipartite matching plus InfoNCE temporal consistency yields usable multi-object tracks on alpine camera-trap video.
    Underlying training objective of SALMA; supported by HOTA/IDF1 numbers but still an empirical modeling choice.
invented entities (3)
  • SALMA no independent evidence
    purpose: End-to-end spatiotemporal action localization model that emits the structured JSON used by the retrieval agent.
    New architecture assembled from VideoMAE encoder + MOTR-style decoder + multi-head attribute MLPs; no independent evidence outside this paper’s tracking and attribute tables.
  • Prompting-MammAlps benchmark (135 queries + exhaustive video matches) no independent evidence
    purpose: Provide the first public evaluation set for camera-trap TVR.
    Constructed by the authors with one behavioral ecologist; the query set and matching rules are original to this work.
  • Custom library of 23 parsing primitives no independent evidence
    purpose: Constrain the LLM so that generated code is inspectable and hallucination-resistant.
    Hand-authored API whose coverage determines which queries can be expressed; not derived from an external standard.

pith-pipeline@v1.1.0-grok45 · 37960 in / 3329 out tokens · 34703 ms · 2026-07-14T14:52:56.831543+00:00 · methodology

0 comments
read the original abstract

Automatically retrieving videos from large camera-trap datasets remains challenging. Text-to-Video retrieval (TVR) methods based on large video-language models (VLMs) have potential to retrieve events of interest by describing them with simple text queries. However, current methods often lack spatiotemporal understanding and do not generalize well to ecological data. In this work, we introduce Prompting-MammAlps, the first camera-trap TVR benchmark, and propose a fine-grained and interpretable TVR method. Specifically, we trained a vision transformer to perform spatiotemporal action localization, and convert its output to structured text, describing each video. Independently, ethology-inspired queries are processed by a Large-Language Model (LLM) based coding agent to parse the structured text per video and retrieve videos accordingly. We harnessed the LLM to use functions from a custom parsing library to minimize the risk of LLM hallucinations and to improve method interpretability. This retrieval approach applied on the Prompting-MammAlps benchmark achieved a set-based F1-score of 34\% on a test set of 135 ecologically-relevant queries and 775 candidate videos. In comparison the best zero-shot VLM achieved a F1-score of 18\%, while also lacking interpretability. Project page: https://cnai.epfl.ch/prompting-mammalps

Figures

Figures reproduced from arXiv: 2607.09876 by Alexander Mathis, Baptiste Maquignaz, Blair Costelloe, Devis Tuia, Gencer Sumbul, Jennifer Shan, Sepideh Mamooler, Valentin Gabeff.

Figure 1
Figure 1. Figure 1: Overview of Prompting-MammAlps and our TVR method. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Prompting-MammAlps Benchmark. (a) The 135 queries are categorized in five ecological and five reasoning categories. One query can belong to multiple categories. (b-c) Number of matching videos per query and inversely (excluding zero entries). To create the TVR benchmark, we identified 135 text queries in collaboration with a behavioral ecologist (B.C.). These queries span four ecological categories: courts… view at source ↗
Figure 3
Figure 3. Figure 3: From raw videos to their structured text representations. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Retrieval process decomposition. For illustrative purposes, we chose the "no label constrain" version of Qwen3-8B (third row from Tab. 4) which produced a concise code but with a logic error that we discuss. Example frames are shown; numbers in parentheses indicate video counts. 8 Conclusion We propose Prompting-MammAlps, a TVR benchmark containing 18h of high-resolution wildlife camera￾trap recordings den… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

96 extracted references · 16 linked inside Pith

  1. [1]

    AI@Meta: Llama 3 model card (2024),https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  2. [2]

    Neuron84(1), 18–31 (2014)

    Anderson, D.J., Perona, P.: Toward a science of computational ethology. Neuron84(1), 18–31 (2014)

    2014

  3. [3]

    In: Proceedings of the IEEE international conference on computer vision

    Anne Hendricks, L., Wang, O., Shechtman, E., Sivic, J., Darrell, T., Russell, B.: Localizing moments in video with natural language. In: Proceedings of the IEEE international conference on computer vision. pp. 5803–5812 (2017)

    2017

  4. [4]

    arXiv preprint arXiv:2404.08471 (2024)

    Bardes, A., Garrido, Q., Ponce, J., Chen, X., Rabbat, M., LeCun, Y., Assran, M., Ballas, N.: Revisiting feature prediction for learning visual representations from video. arXiv preprint arXiv:2404.08471 (2024)

    Pith/arXiv arXiv 2024

  5. [5]

    arXiv preprint arXiv:1907.06772 (2019)

    Beery, S., Morris, D., Yang, S.: Efficient pipeline for camera trap image review. arXiv preprint arXiv:1907.06772 (2019)

    Pith/arXiv arXiv 1907

  6. [6]

    In: Proceedings ofthe European conference on computer vision (ECCV)

    Beery,S., Van Horn, G., Perona, P.: Recognitionin terra incognita. In: Proceedings ofthe European conference on computer vision (ECCV). pp. 456–473 (2018)

    2018

  7. [7]

    Behavioral Ecology22(2), 236–239 (2011)

    Berger-Tal, O., Polak, T., Oron, A., Lubin, Y., Kotler, B.P., Saltz, D.: Integrating animal behavior and conservation biology: a conceptual framework. Behavioral Ecology22(2), 236–239 (2011)

    2011

  8. [8]

    Current Opinion in Neurobiology98, 103201 (2026)

    Bonnetto, A., Mamooler, S., Li, C., Mathis, A.: The behavior biopsy: Interpreting animal behavior as em- bodied, situated, and hierarchical. Current Opinion in Neurobiology98, 103201 (2026)

    2026

  9. [9]

    arXiv preprint arXiv:2404.08937 (2024)

    Brookes, O., Mirmehdi, M., Kuhl, H., Burghardt, T.: Chimpvlm: Ethogram-enhanced chimpanzee behaviour recognition. arXiv preprint arXiv:2404.08937 (2024)

    Pith/arXiv arXiv 2024

  10. [10]

    International Journal of Computer Vision pp

    Brookes, O., Mirmehdi, M., Stephens, C., Angedakin, S., Corogenes, K., Dowd, D., Dieguez, P., Hicks, T.C., Jones, S., Lee, K., et al.: Panaf20k: a large video dataset for wild ape detection and behaviour recognition. International Journal of Computer Vision pp. 1–17 (2024)

    2024

  11. [11]

    Journal of applied ecology52(3), 675–685 (2015)

    Burton, A.C., Neilson, E., Moreira, D., Ladle, A., Steenweg, R., Fisher, J.T., Bayne, E., Boutin, S.: Wildlife camera trapping: a review and recommendations for linking surveys to ecological processes. Journal of applied ecology52(3), 675–685 (2015)

    2015

  12. [12]

    Ethology Ecology & Evolution14(3), 199–216 (2002)

    Cap, H., Aulagnier, S., Deleporte, P.: The phylogeny and behaviour of cervidae (ruminantia pecora). Ethology Ecology & Evolution14(3), 199–216 (2002)

    2002

  13. [13]

    Remote Sensing in Ecology and Conservation3(3), 109–122 (2017)

    Caravaggi, A., Banks, P.B., Burton, A.C., Finlay, C.M., Haswell, P.M., Hayward, M.W., Rowcliffe, M.J., Wood, M.D.: A review of camera trapping for conservation behaviour research. Remote Sensing in Ecology and Conservation3(3), 109–122 (2017)

    2017

  14. [14]

    arXiv preprint arXiv:2511.16719 (2025)

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala, K.V., Khedr, H., Huang, A., et al.: Sam 3: Segment anything with concepts. arXiv preprint arXiv:2511.16719 (2025)

    Pith/arXiv arXiv 2025

  15. [15]

    In: European conference on computer vision

    Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European conference on computer vision. pp. 213–229. Springer (2020)

    2020

  16. [16]

    In: proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition

    Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. In: proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 6299–6308 (2017)

    2017

  17. [17]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Chen, J., Hu, M., Coker, D.J., Berumen, M.L., Costelloe, B., Beery, S., Rohrbach, A., Elhoseiny, M.: Mam- malnet: A large-scale video benchmark for mammal recognition and behavior understanding. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 13052–13061 (2023)

    2023

  18. [18]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al.: Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 24185–24198 (2024)

    2024

  19. [19]

    arXiv preprint arXiv:2412.11959 (2024)

    Cicchetti, G., Grassucci, E., Sigillo, L., Comminiello, D.: Gramian multimodal representation learning and alignment. arXiv preprint arXiv:2412.11959 (2024)

    Pith/arXiv arXiv 2024

  20. [20]

    Trends in cognitive sciences13(1), 36–43 (2009) 12 V

    Couzin, I.D.: Collective cognition in animal groups. Trends in cognitive sciences13(1), 36–43 (2009) 12 V. Gabeff et al

    2009

  21. [21]

    Trends in Ecology & Evolution38(4), 346–354 (2023)

    Couzin, I.D., Heins, C.: Emerging technologies for behavioral research in changing environments. Trends in Ecology & Evolution38(4), 346–354 (2023)

    2023

  22. [22]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Croitoru, I., Bogolin, S.V., Leordeanu, M., Jin, H., Zisserman, A., Albanie, S., Liu, Y.: Teachtext: Crossmodal generalized distillation for text-video retrieval. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 11583–11593 (2021)

    2021

  23. [23]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Dong, J., Li, X., Xu, C., Ji, S., He, Y., Yang, G., Wang, X.: Dual encoding for zero-example video retrieval. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9346–9355 (2019)

    2019

  24. [24]

    IEEE transactions on circuits and systems for video technology32(8), 5680–5694 (2022)

    Dong,J.,Wang,Y.,Chen,X.,Qu,X.,Li,X.,He,Y.,Wang,X.:Reading-strategyinspiredvisualrepresentation learning for text-to-video retrieval. IEEE transactions on circuits and systems for video technology32(8), 5680–5694 (2022)

    2022

  25. [25]

    arXiv preprint arXiv:2405.17698 (2024)

    Duporge, I., Kholiavchenko, M., Harel, R., Rubenstein, D., Crofoot, M., Berger-Wolf, T., Lee, S., Wolf, S., Barreau, J., Kline, J., et al.: Baboonland dataset: Tracking primates in the wild and automating behaviour recognition from drone videos. arXiv preprint arXiv:2405.17698 (2024)

    Pith/arXiv arXiv 2024

  26. [26]

    bioRxiv pp

    Dussert, G., Miele, V., Van Reeth, C., Delestrade, A., Dray, S., Chamaille-Jammes, S.: Zero-shot animal behavior classification with image-text foundation models. bioRxiv pp. 2024–04 (2024)

    2024

  27. [27]

    arXiv preprint arXiv:2505.23952 (2025)

    Fragomeni, A., Damen, D., Wray, M.: Leveraging auxiliary information in text-to-video retrieval: A review. arXiv preprint arXiv:2505.23952 (2025)

    Pith/arXiv arXiv 2025

  28. [28]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Gabeff, V., Qi, H., Flaherty, B., Sumbul, G., Mathis, A., Tuia, D.: Mammalps: A multi-view video behavior monitoring dataset of wild mammals in the swiss alps. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 13854–13864 (2025)

    2025

  29. [29]

    International Journal of Computer Vision pp

    Gabeff, V., Rußwurm, M., Tuia, D., Mathis, A.: Wildclip: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models. International Journal of Computer Vision pp. 1–17 (2024)

    2024

  30. [30]

    In: European Conference on Computer Vision

    Gabeur, V., Sun, C., Alahari, K., Schmid, C.: Multi-modal transformer for video retrieval. In: European Conference on Computer Vision. pp. 214–229. Springer (2020)

    2020

  31. [31]

    something something

    Goyal, R., Ebrahimi Kahou, S., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al.: The" something something" video database for learning and evaluating visual common sense. In: Proceedings of the IEEE international conference on computer vision. pp. 5842–5850 (2017)

    2017

  32. [32]

    Hernandez, A., Miao, Z., Vargas, L., Dodhia, R., Lavista, J.: Pytorch-wildlife: A collaborative deep learning framework for conservation (2024)

    2024

  33. [33]

    arXiv preprint arXiv:2509.14233 (2025)

    Hernández-Cano, A., Hägele, A., Huang, A.H., Romanou, A., Solergibert, A.J., Pasztor, B., Messmer, B., Garbaya, D., Ďurech, E.F., Hakimi, I., Giraldo, J.G., Ismayilzada, M., , et al.: Apertus: Democratizing Open and Compliant LLMs for Global Language Environments. arXiv preprint arXiv:2509.14233 (2025)

    arXiv 2025

  34. [34]

    Remote Sensing in Ecology and Conservation6(2), 129–140 (2020)

    Hofmeester, T.R., Young, S., Juthberg, S., Singh, N.J., Widemo, F., Andrén, H., Linnell, J.D., Cromsigt, J.P.: Using by-catch data from wildlife surveys to quantify climatic parameters and timing of phenology for plants and animals using camera traps. Remote Sensing in Ecology and Conservation6(2), 129–140 (2020)

    2020

  35. [35]

    iNaturalist: inaturalist.https://www.inaturalist.org(2026), accessed: 2026-02-17

    2026

  36. [36]

    arXiv preprint arXiv:2310.068253(2023)

    Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.D.L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al.: Mistral-7b. arXiv preprint arXiv:2310.068253(2023)

    Pith/arXiv arXiv 2023

  37. [37]

    IEEE Access (2022)

    Jiang, J., Min, S., Kong, W., Wang, H., Li, Z., Liu, W.: Tencent text-video retrieval: hierarchical cross-modal interactions with multi-level representations. IEEE Access (2022)

    2022

  38. [38]

    arXiv preprint arXiv:1705.06950 (2017)

    Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al.: The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017)

    Pith/arXiv arXiv 2017

  39. [39]

    Science348(6240), aaa2478 (2015)

    Kays, R., Crofoot, M.C., Jetz, W., Wikelski, M.: Terrestrial animal tracking as an eye on life and planet. Science348(6240), aaa2478 (2015)

    2015

  40. [40]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Kholiavchenko, M., Kline, J., Ramirez, M., Stevens, S., Sheets, A., Babu, R., Banerji, N., Campolongo, E., Thompson, M., Van Tiel, N., et al.: Kabr: In-situ dataset for kenyan animal behavior recognition from drone videos. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 31–40 (2024)

    2024

  41. [41]

    arXiv preprint arXiv:1412.6980 (2014)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

    Pith/arXiv arXiv 2014

  42. [42]

    Journal of Animal Ecology92(7), 1357–1371 (2023)

    Koger, B., Deshpande, A., Kerby, J.T., Graving, J.M., Costelloe, B.R., Couzin, I.D.: Quantifying the move- ment, behaviour and environmental context of group-living animals using drones and computer vision. Journal of Animal Ecology92(7), 1357–1371 (2023)

    2023

  43. [43]

    In: Proceed- ings of the IEEE international conference on computer vision

    Krishna, R., Hata, K., Ren, F., Fei-Fei, L., Carlos Niebles, J.: Dense-captioning events in videos. In: Proceed- ings of the IEEE international conference on computer vision. pp. 706–715 (2017)

    2017

  44. [44]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Kriz, R., Sanders, K., Etter, D., Murray, K., Carpenter, C., Recknor, H., Guallar-Blasco, J., Martin, A., Yang, E., Van Durme, B.: Multivent 2.0: A massive multilingual benchmark for event-centric video retrieval. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 24149–24158 (2025)

    2025

  45. [45]

    Naval research logistics quarterly2(1-2), 83–97 (1955)

    Kuhn, H.W.: The hungarian method for the assignment problem. Naval research logistics quarterly2(1-2), 83–97 (1955)

    1955

  46. [46]

    In: European Conference on Computer Vision

    Lei, J., Yu, L., Berg, T.L., Bansal, M.: Tvr: A large-scale dataset for video-subtitle moment retrieval. In: European Conference on Computer Vision. pp. 447–463. Springer (2020) Prompting-MammAlps 13

    2020

  47. [47]

    In: Proceedings of the 2024 conference on empirical methods in natural language processing

    Lin, B., Ye, Y., Zhu, B., Cui, J., Ning, M., Jin, P., Yuan, L.: Video-llava: Learning united visual representation by alignment before projection. In: Proceedings of the 2024 conference on empirical methods in natural language processing. pp. 5971–5984 (2024)

    2024

  48. [48]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Liu, D., Hou, J., Huang, S., Liu, J., He, Y., Zheng, B., Ning, J., Zhang, J.: Lote-animal: A long time- span dataset for endangered animal behavior understanding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 20064–20075 (2023)

    2023

  49. [49]

    Ecological Informatics83, 102797 (2024)

    Liu, J., Hou, J., Liu, D., Zhao, Q., Chen, R., Chen, X., Hull, V., Zhang, J., Ning, J.: A joint time and spatial attention-based transformer approach for recognizing the behaviors of wild giant pandas. Ecological Informatics83, 102797 (2024)

    2024

  50. [50]

    arXiv preprint arXiv:2104.08860 (2021)

    Luo, H., Ji, L., Zhong, M., Chen, Y., Lei, W., Duan, N., Li, T.: Clip4clip: An empirical study of clip for end to end video clip retrieval. arXiv preprint arXiv:2104.08860 (2021)

    Pith/arXiv arXiv 2021

  51. [51]

    arXiv preprint arXiv:2505.03173 (2025)

    Malik, S., Yamada, M., Singh, A., Aggarwal, D.: Ravu: Retrieval augmented video understanding with com- positional reasoning over graph. arXiv preprint arXiv:2505.03173 (2025)

    Pith/arXiv arXiv 2025

  52. [52]

    In: LLM for Scientific Discovery: Reasoning, Assistance, and Collaboration (2025)

    Mamooler, S., Qi, H., Gabeff, V., Montariol, S., Bosselut, A., Mathis, A.: Fine-tuning vision-language models for animal behavior analysis. In: LLM for Scientific Discovery: Reasoning, Assistance, and Collaboration (2025)

    2025

  53. [53]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Manasyan, A., Seitzer, M., Radovic, F., Martius, G., Zadaianchuk, A.: Temporally consistent object-centric learning by contrasting slots. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 5401–5411 (2025)

    2025

  54. [54]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Miech, A., Zhukov, D., Alayrac, J.B., Tapaswi, M., Laptev, I., Sivic, J.: Howto100m: Learning a text-video embedding by watching hundred million narrated video clips. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 2630–2640 (2019)

    2019

  55. [55]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Ng, X.L., Ong, K.E., Zheng, Q., Ni, Y., Yeo, S.Y., Liu, J.: Animal kingdom: A large and diverse dataset for animal behavior understanding. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 19023–19034 (2022)

    2022

  56. [56]

    Proceedings of the National Academy of Sciences115(25), E5716–E5725 (2018)

    Norouzzadeh,M.S.,Nguyen,A.,Kosmala,M.,Swanson,A.,Palmer,M.S.,Packer,C.,Clune,J.:Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. Proceedings of the National Academy of Sciences115(25), E5716–E5725 (2018)

    2018

  57. [57]

    O’Connell, A.F., Nichols, J.D., Karanth, K.U.: Camera traps in animal ecology: methods and analyses, vol. 271. Springer (2011)

    2011

  58. [58]

    Biology Bulletin Reviews1(4), 345–357 (2011)

    Prikhod’ko, V., Zvychainaya, E.Y.: Behavior and phylogenetic relations among artiodactyla families (artio- dactyla, mammalia). Biology Bulletin Reviews1(4), 345–357 (2011)

    2011

  59. [59]

    Pulakurthi, P.R., Wang, J., Rabbani, M., Dianat, S., Rao, R., Tao, Z.: X-cot: Explainable text-to-video re- trievalviallm-basedchain-of-thoughtreasoning.In:Proceedingsofthe2025ConferenceonEmpiricalMethods in Natural Language Processing. pp. 31172–31183 (2025)

    2025

  60. [60]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

    2021

  61. [61]

    bioRxiv pp

    Rigoudy, N., Dussert, G., Benyoub, A., Besnard, A., Birck, C., Boyer, J., Bollet, Y., Bunz, Y., Caussimont, G., Chetouane, E., et al.: The deepfaune initiative: a collaborative effort towards the automatic identification of the french fauna in camera-trap images. bioRxiv pp. 2022–03 (2022)

    2022

  62. [62]

    Scientific Data12(1), 1200 (2025)

    Rodriguez-Juan, J., Ortiz-Perez, D., Benavent-Lledo, M., Mulero-Pérez, D., Ruiz-Ponce, P., Orihuela-Torres, A., Garcia-Rodriguez, J., Sebastián-González, E.: Visual wetlandbirds dataset: Bird species identification and behavior recognition in videos. Scientific Data12(1), 1200 (2025)

    2025

  63. [63]

    arXiv preprint arXiv:2306.11326 (2023)

    Rogers, M., Gendron, G., Valdez, D.A.S., Azhar, M., Chen, Y., Heidari, S., Perelini, C., O’Leary, P., Knowles, K., Tait, I., et al.: Meerkat behaviour recognition dataset. arXiv preprint arXiv:2306.11326 (2023)

    Pith/arXiv arXiv 2023

  64. [64]

    In: International Conference on Multimedia Modeling

    Rossetto, L., Schuldt, H., Awad, G., Butt, A.A.: V3c–a research video collection. In: International Conference on Multimedia Modeling. pp. 349–360. Springer (2018)

    2018

  65. [65]

    Roucher, A., del Moral, A.V., Wolf, T., von Werra, L., Kaunismäki, E.: ‘smolagents‘: a smol library to build great agentic systems.https://github.com/huggingface/smolagents(2025)

    2025

  66. [66]

    Conservation technology79(2021)

    Rovero, F., Kays, R.: Camera trapping for conservation. Conservation technology79(2021)

    2021

  67. [67]

    Methods in ecology and evolution5(11), 1170–1179 (2014)

    Rowcliffe, J.M., Kays, R., Kranstauber, B., Carbone, C., Jansen, P.A.: Quantifying levels of animal activity using camera trap data. Methods in ecology and evolution5(11), 1170–1179 (2014)

    2014

  68. [68]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Song, E., Chai, W., Wang, G., Zhang, Y., Zhou, H., Wu, F., Chi, H., Guo, X., Ye, T., Zhang, Y., et al.: Moviechat: From dense token to sparse memory for long video understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18221–18232 (2024)

    2024

  69. [69]

    Frontiers in Ecology and the Environment15(1), 26–34 (2017)

    Steenweg, R., Hebblewhite, M., Kays, R., Ahumada, J., Fisher, J.T., Burton, C., Townsend, S.E., Carbone, C., Rowcliffe, J.M., Whittington, J., et al.: Scaling-up camera traps: Monitoring the planet’s biodiversity with networks of remote sensors. Frontiers in Ecology and the Environment15(1), 26–34 (2017)

    2017

  70. [70]

    In: European Conference on Computer Vision

    Stoffl, L., Bonnetto, A., d’Ascoli, S., Mathis, A.: Elucidating the hierarchical nature of behavior with masked autoencoders. In: European Conference on Computer Vision. pp. 106–125. Springer (2025)

    2025

  71. [71]

    Philosophical Transactions of the Royal Society B374(1781), 20190012 (2019)

    Tobias, J.A., Pigot, A.L.: Integrating behaviour and ecology into global biodiversity conservation strategies. Philosophical Transactions of the Royal Society B374(1781), 20190012 (2019)

    2019

  72. [72]

    Advances in neural information processing systems35, 10078–10093 (2022) 14 V

    Tong, Z., Song, Y., Wang, J., Wang, L.: Videomae: Masked autoencoders are data-efficient learners for self- supervised video pre-training. Advances in neural information processing systems35, 10078–10093 (2022) 14 V. Gabeff et al

    2022

  73. [73]

    Nature communications13(1), 792 (2022)

    Tuia, D., Kellenberger, B., Beery, S., Costelloe, B.R., Zuffi, S., Risse, B., Mathis, A., Mathis, M.W., van Langevelde, F., Burghardt, T., et al.: Perspectives in machine learning for wildlife conservation. Nature communications13(1), 792 (2022)

    2022

  74. [74]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., Belongie, S.: The inaturalist species classification and detection dataset. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 8769–8778 (2018)

    2018

  75. [75]

    arXiv preprint arXiv:2511.15656 (2025)

    Vendrow, E., Chae, J., Kurinchi-Vendhan, R., Eckert, I., Hall, J., Jarzyna, M., Miyajima, R., Oliver, R., Pollock, L., Shrack, L., et al.: Inquire-search: A framework for interactive discovery in large-scale biodiversity databases. arXiv preprint arXiv:2511.15656 (2025)

    arXiv 2025

  76. [76]

    Advances in Neural Information Processing Systems37, 126500–126514 (2024)

    Vendrow, E., Pantazis, O., Shepard, A., Brostow, G., Jones, K.E., Mac Aodha, O., Beery, S., Van Horn, G.: Inquire: A natural world text-to-image retrieval benchmark. Advances in Neural Information Processing Systems37, 126500–126514 (2024)

    2024

  77. [77]

    Nature Methods pp

    Vogg, R., Lüddecke, T., Henrich, J., Dey, S., Nuske, M., Hassler, V., Murphy, D., Fischer, J., Ostner, J., Schülke, O., et al.: Computer vision for primate behavior analysis in the wild. Nature Methods pp. 1–13 (2025)

    2025

  78. [78]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Wang, J., Sun, G., Wang, P., Liu, D., Dianat, S., Rabbani, M., Rao, R., Tao, Z.: Text is mass: Modeling as stochastic embedding for text-video retrieval. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16551–16560 (2024)

    2024

  79. [79]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Wang, X., Wu, J., Chen, J., Li, L., Wang, Y.F., Wang, W.Y.: Vatex: A large-scale, high-quality multilin- gual dataset for video-and-language research. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4581–4591 (2019)

    2019

  80. [80]

    arXiv preprint arXiv:2212.03191 (2022)

    Wang, Y., Li, K., Li, Y., He, Y., Huang, B., Zhao, Z., Zhang, H., Xu, J., Liu, Y., Wang, Z., et al.: Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.03191 (2022)

    Pith/arXiv arXiv 2022

Showing first 80 references.