Pith. sign in

REVIEW 4 major objections 6 minor 38 references

Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning a video-text model on automatically cleaned clip-transcript pairs extracted from raw footage produces large retrieval gains on capuchin behavior data, lifting Hits@5 by 167% (16-frame) and 114% (8-frame) over raw pre-trained…

desk verdict Plausible and useful proof-of-concept, but the advertised Hits@5 gains are inflated because LoRA was selected on the same test set used to measure performance. read the letter →

arxiv 2505.05681 v1 pith:TK6U3342 submitted 2025-05-08 cs.CV

Giulio Cesare Mastrocinque Santo , Patrícia Izar , Irene Delval , Victor de Napole Gregolin , Nina S. T. Hirata This is my paper · ORCID
classification cs.CV
keywords video-textretrievalcontrastivelearningLoRAfine-tuningX-CLIPcapuchinmonkeybehaviorethogramnoisydatacleaningmultimodalLLMagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a video-text contrastive model can be adapted to a highly specialized domain—wild capuchin monkey behavior—using only raw, unlabeled videos and the imperfect audio descriptions recorded by field collaborators. The authors build an automatic pipeline that transcribes the audio, uses a large language model to filter and translate the transcripts, and uses an image-text model to discard clips whose visual content does not match the text. The surviving clip-text pairs are used to fine-tune X-CLIP with LoRA. On a manually labeled test set, the fine-tuned model raises Hits@5 from 0.06 to 0.16 (16-frame) and from 0.07 to 0.15 (8-frame), while raw pre-trained models score near random. If correct, this provides a path to retrieval tools for animal behavior research without expensive manual annotation.

What carries the argument

The load-bearing mechanism is a two-stage pipeline. First, an agentic data treatment step uses Whisper to transcribe audio, LLaMA 3.2 to score, translate, and behavior-classify transcripts against an ethogram, and BLIP-2 to compute a max-over-frames cosine similarity between clip frames and transcript, keeping only pairs above a 0.32 threshold. Second, X-CLIP is fine-tuned with LoRA on the surviving pairs, with a modified loss that adds a contrastive term on the text embedding computed without the Prompt Generator, so the text and video embeddings can be computed independently at retrieval time.

What would settle it

Retrain the same LoRA setup on the unfiltered noisy clip-transcript pairs (or with the BLIP-2 threshold removed) and compare Hits@5 on the same test set; if retrieval gains persist, the cleaning pipeline is not the cause of the improvement. A second check would be to replace LLaMA's behavior labels with random labels during cleaning and see whether NDCG@5 collapses.

Watch

Extended reading notes

Core claim

The central discovery is that low-rank fine-tuning of a pre-trained video-text model on automatically cleaned, weakly supervised clip-text pairs transfers the model to a new domain where the raw model fails completely. Concretely, the fine-tuned X-CLIP reaches NDCG@5 above 0.4 for most of the tested ethogram behaviors, whereas raw X-CLIP variants are mostly at zero; Hits@5 improves by 167% for the 16-frame model and 114% for the 8-frame model. The paper further shows that the gains appear in retrieval, and to a lesser degree in zero-shot classification, and that qualitative retrievals capture behaviors such as nursing, threatening, eating, and swinging on a vine.

Load-bearing premise

The whole training signal rests on the assumption that the automatic cleaning steps—LLaMA's transcript decisions and BLIP-2's 0.32 similarity threshold chosen by visual inspection—separate genuinely aligned clip-text pairs from the many noisy pairs; if they do not, the model learns from mislabeled pairs.

Editorial extensions

If this is right

  • Researchers can build behavior retrieval systems from raw field footage without manual labels: audio notes from collaborators, when cleaned, provide the training signal.
  • The method scales to the larger 13,060-video, 284-hour collection: more raw footage should yield more clean pairs and better fine-tuning.
  • Behavior ranking becomes usable: most ethogram behaviors reach NDCG@5 above 0.4, so a researcher can query text and retrieve relevant clips.
  • Zero-shot classification also improves, so the fine-tuned model can serve as a domain-specific backbone for downstream classifiers.
  • Rare behaviors such as scratch and sexual remain hard, indicating where more data or different prompts are needed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same agentic cleaning recipe—transcribe, LLM-filter with an ethogram, image-text similarity filter—could transfer to other species or field sites that have audio notes, provided an ethogram exists.
  • The 0.32 BLIP-2 threshold was picked by visual inspection on this dataset; on a new domain it would need recalibration, since it controls the precision/recall trade-off of the training pairs.
  • A possible blind spot is that cleaning via LLaMA and BLIP-2 may teach the model to recognize transcript style rather than behavior content; a stronger test would measure generalization to novel ethogram descriptions not seen in the pipeline.
  • If the method holds, it offers a cheaper route to domain-specific retrieval than full annotation, with the caveat that final validation still requires a small manually labeled set.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper tackles video-text retrieval of capuchin monkey behaviors from raw field footage. It constructs training pairs by transcribing audio with Whisper, filtering transcripts with LLaMA 3.2 (quality score, ethogram-based behavior detection, translation, name removal), and retaining clip-text pairs whose BLIP-2 max-frame cosine similarity exceeds 0.32. It then fine-tunes X-CLIP with LoRA, adding a contrastive loss term that uses a text embedding computed without the Prompt Generator so that retrieval embeddings are independently computable. On a manually annotated 177-pair test set, the fine-tuned 16-frame and 8-frame models achieve Hits@5 of 0.16 and 0.15, respectively, versus at most 0.06 and 0.07 for raw X-CLIP baselines, and per-behavior NDCG@5 for several behaviors is substantially higher than for raw models; zero-shot classification also improves. The authors claim this is the first demonstration of fine-tuning a video-text contrastive model on automatically cleaned, unlabeled primate footage.

Significance. If the evaluation were unbiased, this would be a useful applied contribution: it shows a practical route to turning noisy field video and weak audio descriptions into training signal for an ethology-specific retrieval model, and it provides a benchmark with manual labels for future work. The design choice of filtering with BLIP-2 rather than X-CLIP reduces same-model circularity, and the comparison against several raw X-CLIP variants is appropriate. The core limitation is the evaluation protocol: LoRA hyperparameters and possibly the BLIP-2 threshold are selected on or with reference to the test data, so the reported gains are optimistic upper bounds. With a proper validation split, the qualitative conclusion that fine-tuning helps over raw models would likely survive, but the reported magnitudes would need revision.

major comments (4)
  1. [Section 4.2, Table 2, Appendix A] The LoRA rank and layer placement are selected on the same 177-pair test set used for the headline results in Tables 3 and 4. Section 4.2 defines the best model as the one maximizing avg(Top-1, Top-2, Top-3) on the test set, and no validation split or holdout is described; with 3 layer placements and 4 ranks there are 12 candidate configurations per frame count. The paper's own observation that "the optimal model varies by Top-K" indicates that the ranking is noisy, so selecting on the test set and then reporting Hits@5, Hits@10, and NDCG@5 from that same set can inflate the reported uplift materially; at n=177, one additional hit changes Hits@5 by about 0.006. Because the raw X-CLIP baselines are not subjected to the same selection procedure, the comparison is unfair and the headline 167%/114% Hits@5 improvements are upper-bound estimates rather than unbiased estimates. The authors should either introduce a validation split for model selection or report the full set of configurations with appropriate multiple-comparison awareness.
  2. [Section 3.1.5] The BLIP-2 filtering threshold of 0.32 is described as "obtained through visual inspection: we tried several threshold values and checked the retrieval results." The manuscript does not specify which data were used for this retrieval check. If the threshold was tuned on the same 177-pair test set, then the pipeline itself involves selection on the evaluation data, further biasing the reported gains; if a separate development set was used, this should be stated explicitly. This step is load-bearing because the threshold determines the training pairs, so its selection protocol must be transparent.
  3. [Section 4.1] The manual test set was constructed by using the pipeline's own LLaMA 3.2 behavior labels to select instances per behavior type. This stratified selection may bias the test distribution toward clips whose transcripts are easy for the pipeline to classify, making the test set non-representative of the raw video population. The authors should either sample the test set independently of the pipeline, or provide an analysis of how the LLaMA-based stratification affects the reported metrics.
  4. [Tables 3 and 4] The central comparisons are reported without any measure of uncertainty. At n=177, the difference between Hits@5 0.16 and 0.06 corresponds to roughly 18 additional hits, which may be real, but the absence of confidence intervals or significance tests makes it difficult to assess how much of the gap is attributable to the fine-tuning versus the test-set selection described above. Please report bootstrap confidence intervals or otherwise characterize the variability of the retrieval metrics.
minor comments (6)
  1. [Section 3.2, Eq. (6)] The two contrastive losses are both written as CL(c,t) in the equation and the surrounding text, even though one uses the prompt-generator-bypassing text embedding and the other uses the projector output; please introduce distinct notation (e.g., CL_ret and CL_zs).
  2. [Section 3.1] The pipeline is called "agentic" and "an agent," but it is a fixed sequence of LLM/VLM calls without reasoning, tool use, or memory; consider using a less loaded term such as "LLM-based filtering pipeline."
  3. [Section 1, contributions list] The phrase "several versions os raw X-CLIP pretrained models" contains a typo ("os" should be "of").
  4. [Figure 6] The caption does not state whether the displayed clips are the top-1 retrieval outputs or hand-picked examples; please clarify the selection protocol for the qualitative results.
  5. [Section 4.3.2] The zero-shot classification improvements are reported as percentages relative to the best raw model, but the absolute Top-1 accuracies are only 0.12-0.14; please frame these small absolute gains in the discussion and avoid overclaiming.
  6. [Section 5] The phrase "as proven by the computed metrics" overstates what a single test set can establish; suggest "supported by" instead.

Circularity Check

1 steps flagged · score 4.0 of 10

Headline Hits@5 uplift is selected on the same 177-pair test set used for the final comparison, so the reported gain is an upper bound rather than an unbiased prediction.

  1. fitted input called prediction [Section 4.2 (model selection) and Section 4.3.1 (retrieval metrics)]
    "we define the best model as the one achieving the highest average across Top-1, Top-2, and Top-3 accuracies, i.e., avg(Top-1, Top-2, Top-3). The best performing models are shown in Table 2. ... Evaluation metrics are all computed on the test set."

    The 12 candidate LoRA settings (3 layer placements × 4 ranks) are scored on the same 177-pair test set to pick the 'best' models (Table 2), and the very same test set then produces the headline Hits@5/NDCG@5 numbers (Tables 3-4). Thus the reported 167%/114% Hits@5 uplift is the outcome of a model-selection loop over the gold labels, not an unbiased out-of-sample prediction: the selection criterion avg(Top-1, Top-2, Top-3) is itself fitted on the evaluation set, and at n=177 even a few extra hits materially change Hits@5. Raw X-CLIP baselines are not subjected to this selection, so the comparison is biased upward.

full rationale

The rest of the pipeline is self-contained: Whisper provides transcripts, LLaMA-3.2 filters and classifies them, BLIP-2 filters clip-text similarity (and using BLIP-2 rather than X-CLIP avoids same-model circularity in data cleaning), and the resulting pairs are used to LoRA fine-tune X-CLIP. The test set is manually annotated, drawn from videos excluded from training, and the raw pre-trained X-CLIP baselines provide independent comparison points. The central learning effect is therefore not definitionally identical to its inputs. However, the paper's headline retrieval numbers are partially circular because the LoRA rank/layer configuration is selected on the 177-pair test set and the final metrics are then computed on that same set. The passage in Section 3.1.5 saying the BLIP-2 threshold was chosen 'through visual inspection' after 'checked the retrieval results' is underspecified; if that check used the test set it would add another selection loop, but the text alone does not establish this, so it is not scored as a separate circular step. Overall, the evidence for real adaptation is credible, but the precise uplift magnitude is an upper-bound, test-selected estimate.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The paper's central claim rests on a multi-stage data-cleaning process whose components are external models rather than derived facts. The only free parameters that shape the training set are the BLIP-2 similarity threshold, chosen by visual inspection and retrieval checks, and the LoRA configuration, chosen on the test set. No new entities are introduced. The axioms are domain assumptions about transcription quality, LLM judgment, BLIP-2 transfer, ethogram coverage, and manual label accuracy; none are verified with external benchmarks in this preprint.

free parameters (2)
  • BLIP-2 clip-text cosine similarity threshold = 0.32
    Set in Section 3.1.5 after trying several values and checking retrieval results; it controls which of the 123,871 raw pairs are reduced to 7,862 training pairs.
  • LoRA rank and layer configuration = rank 4 Vertical for 16-frame retrieval; rank 8 Bottom for 16-frame zero-shot; rank 8 Vertical for 8-frame retrieval…
    Selected in Section 4.2 as the best avg(Top-1, Top-2, Top-3) on the test set; no held-out validation is reported.
assumptions (6)
  • domain assumption Whisper Large-V3-Turbo transcribes Brazilian Portuguese field audio well enough to preserve behavior descriptions
    Invoked in Section 3.1.1 to generate the 123,871 raw clip-text pairs; no manual quality check of transcripts is reported.
  • domain assumption LLaMA 3.2 quality scoring and behavior classification correctly identify transcripts relevant to the 34-item ethogram
    Invoked in Sections 3.1.2 and 3.1.3; these binary decisions remove the majority of candidate pairs without human verification.
  • domain assumption BLIP-2 fine-tuned on COCO provides a valid proxy for clip-text alignment for capuchin monkey clips
    Invoked in Section 3.1.5, where the maximum over frames of BLIP-2 cosine similarity is used to accept or reject pairs; the 0.32 threshold is chosen by inspection.
  • domain assumption The adapted ethogram in Table 1 covers the behaviors that matter for retrieval
    Used both in pipeline classification and in test-set labels; behaviors outside the table are ignored.
  • domain assumption The 177-pair test set is manually annotated correctly and its composition is representative
    Invoked in Section 4.1; selection is balanced according to LLaMA pipeline labels, which may bias the distribution.
  • domain assumption X-CLIP's pre-trained Kinetics-600 representations transfer to primate behavior with LoRA updates
    Invoked in Section 3.2; the fine-tuning assumes the existing feature space is a useful starting point for capuchin actions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos." pith.science (2026). https://pith.science/paper/TK6U3342

@misc{pith2026250505681,
  author       = {Pith},
  title        = {Pith review of: Fine-Tuning Video-Text Contrastive Model for Primate Behavior Retrieval from Unlabeled Raw Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TK6U3342}},
  note         = {Machine review of arXiv:2505.05681}
}
abstract

Video recordings of nonhuman primates in their natural habitat are a common source for studying their behavior in the wild. We fine-tune pre-trained video-text foundational models for the specific domain of capuchin monkeys, with the goal of developing useful computational models to help researchers to retrieve useful clips from videos. We focus on the challenging problem of training a model based solely on raw, unlabeled video footage, using weak audio descriptions sometimes provided by field collaborators. We leverage recent advances in Multimodal Large Language Models (MLLMs) and Vision-Language Models (VLMs) to address the extremely noisy nature of both video and audio content. Specifically, we propose a two-folded approach: an agentic data treatment pipeline and a fine-tuning process. The data processing pipeline automatically extracts clean and semantically aligned video-text pairs from the raw videos, which are subsequently used to fine-tune a pre-trained Microsoft's X-CLIP model through Low-Rank Adaptation (LoRA). We obtained an uplift in $Hits@5$ of $167\%$ for the 16 frames model and an uplift of $114\%$ for the 8 frame model on our domain data. Moreover, based on $NDCG@K$ results, our model is able to rank well most of the considered behaviors, while the tested raw pre-trained models are not able to rank them at all. The code will be made available upon acceptance.

Figures

Figures reproduced from arXiv: 2505.05681 by the authors.

Figure 1
Figure 1. Video-Text Dual Encoder Architecture. the distance between disjoint pairs. Because the encoders are independent, those mod￾els can naturally be used in retrieval tasks, which makes them really powerful. For ex￾ample, we can pre-compute video embeddings in a large set of videos and index them in a vector database. Then, given a textual input, we can obtain the corresponding text embedding and, because text and video … view at source ↗
Figure 2
Figure 2. The figure presents two examples of video clips. In (a), we illustrate a commonly encountered video [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Data Generation Pipeline. In (a) one can see that OpenAI’s Whisper is used to extract raw transcripts, which are then treated by a data processing Agent. The clean (clip, transcript) pairs are then submitted into BLIP-2 model (b) and only the pairs with cosine similarity greater than a predefined threshold are maintained, reducing the amount of noisy pairs. The diagram in (c) shows the actual graph produced with Lan… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: X-CLIP Fine-Tuning Architecture. The X-CLIP architecture is the one proposed in [10] and here simplified. It consists of a text and a vision transformer, a Multi-Frame Integration Transformer (MIT), a Prompt Generator and projections to map both modalities into the sam…
Figure 5
Figure 5. Figure 5: Test dataset label distribution: number of instances per behavior. [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: Resulting videos of Sapajus xanthosternos in Una Biological reserve for di [PITH_FULL_IMAGE:figures/full_fig_p026_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 23 canonical work pages

  1. [1]

    D. M. Fragaszy, Y . Eshchar, E. Visalberghi, B. Resende, K. Laity, P. Izar, Syn- chronized practice helps bearded capuchin monkeys learn to extend attention while learning a tradition, Proceedings of the National Academy of Sciences 114 (30) (2017) 7798–7805. doi:10.1073/pnas.1621071114

    work page doi:10.1073/pnas.1621071114 2017

  2. [2]

    Resende, A

    B. Resende, A. Ballesteros-Ardilla, D. Fragaszy, E. Visalberghi, P. Izar, Revisit- ing the fourth dimension of tool use: how objects become tools for capuchin mon- keys, Evolutionary Human Sciences 3 (2021) e18.doi:10.1017/ehs.2021.16

    work page doi:10.1017/ehs.2021.16 2021

  3. [3]

    Araujo, V

    G. Araujo, V . Truppa, P. Izar, Early development of object manipulation in ca- puchin monkeys: A naturalistic approach, Developmental Psychobiology 66 (2) (2024) e22458. doi:https://doi.org/10.1002/dev.22458

    work page doi:10.1002/dev.22458 2024

  4. [4]

    Delval, M

    I. Delval, M. Fernández-Bolaños, P. Izar, A longitudinal assessment of behavioral development in wild capuchins: Personality is not established in the first 3 years, American Journal of Primatology 82 (11) (2020) e23116. doi:https://doi. org/10.1002/ajp.23116

    work page doi:10.1002/ajp.23116 2020

  5. [5]

    Delval, M

    I. Delval, M. Fernández-Bolaños, P. Izar, J.-B. Leca, Carrying the dead: behavior 29 of a primiparous capuchin monkey mother and other individuals towards a dead infant, Primates 66 (3) (2025) 241–247. doi:10.1007/s10329-025-01187-3

    work page doi:10.1007/s10329-025-01187-3 2025

  6. [6]

    Altmann, Observational study of behavior: Sampling methods, Behaviour 49 (3/4) (1974) 227–267

    J. Altmann, Observational study of behavior: Sampling methods, Behaviour 49 (3/4) (1974) 227–267

    work page 1974

  7. [7]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: International Conference on Machine Learning, 2021

    2021

  8. [8]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Roz- ière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, G. Lample, Llama: Open and e fficient foundation language models, CoRR abs/2302.13971 (2023). arXiv:2302.13971, doi:10.48550/ARXIV.2302. 13971

    arXiv doi:10.48550/arxiv.2302.13971 2023

Show all 38 references
  1. [9]

    E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen, LoRA: Low-rank adaptation of large language models, in: International Confer- ence on Learning Representations, 2022

    2022

  2. [10]

    B. Ni, H. Peng, M. Chen, S. Zhang, G. Meng, J. Fu, S. Xiang, H. Ling, Expanding language-image pretrained models for general video recognition, in: S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, T. Hassner (Eds.), Computer Vision – ECCV 2022, Springer Nature Switzerland, Ch...

    2022

  3. [11]

    J. Yu, Z. Wang, V . Vasudevan, L. Yeung, M. Seyedhosseini, Y . Wu, Coca: Con- trastive captioners are image-text foundation models, Transactions on Machine Learning Research (2022)

    2022

  4. [13]

    Tschannen, A

    M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdulmohsin, N. Parthasarathy, T. Evans, L. Beyer, Y . Xia, B. Mustafa, O. Hénaff, J. Harmsen, A. Steiner, X. Zhai, Siglip 2: Multilingual vision-language encoders with im- proved semantic understanding, localization, and ...

    2025 arXiv

  5. [14]

    H. Luo, L. Ji, M. Zhong, Y . Chen, W. Lei, N. Duan, T. Li, Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning, Neurocomput. 508 (C) (2022) 293–304. doi:10.1016/j.neucom.2022.07.028

    2022 doi

  6. [15]

    H. Xu, G. Ghosh, P.-Y . Huang, D. Okhonko, A. Aghajanyan, F. Metze, L. Zettle- moyer, C. Feichtenhofer, VideoCLIP: Contrastive pre-training for zero-shot video-text understanding, in: M.-F. Moens, X. Huang, L. Specia, S. W.-t. Yih (Eds.), Proceedings of the 2021 Conference on ...

    2021 doi

  7. [16]

    Croitoru, S.-V

    I. Croitoru, S.-V . Bogolin, M. Leordeanu, H. Jin, A. Zisserman, Y . Liu, S. Al- banie, Teachtext: Crossmodal text-video retrieval through generalized distillation, Artificial Intelligence 338 (2025) 104235. doi:https://doi.org/10.1016/ j.artint.2024.104235

    2025

  8. [17]

    Zhang, P

    H. Zhang, P. Zeng, L. Gao, J. Song, H. T. Shen, MPT: Multi-grained prompt tuning for text-video retrieval, in: ACM Multimedia 2024, 2024

    2024

  9. [18]

    M. Cao, H. Tang, J. Huang, P. Jin, C. Zhang, R. Liu, L. Chen, X. Liang, L. Yuan, G. Li, RAP: E fficient text-video retrieval with sparse-and-correlated adapter, in: L.-W. Ku, A. Martins, V . Srikumar (Eds.), Findings of the Association for Computational Linguistics: ACL 2024, ...

    2024 doi

  10. [19]

    Zanella, I

    M. Zanella, I. B. Ayed, Low-Rank Few-Shot Adaptation of Vision-Language 31 Models , in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition Workshops (CVPRW), IEEE Computer Society, Los Alamitos, CA, USA, 2024, pp. 1593–1603. doi:10.1109/CVPRW63382.2024.00166

    2024

  11. [20]

    S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, E. Chen, A survey on multimodal large language models, National Science Review 11 (12) (2024) nwae403. doi: 10.1093/nsr/nwae403

    2024 doi

  12. [21]

    J. Li, D. Li, C. Xiong, S. C. H. Hoi, BLIP: bootstrapping language-image pre- training for unified vision-language understanding and generation, in: K. Chaud- huri, S. Jegelka, L. Song, C. Szepesvári, G. Niu, S. Sabato (Eds.), International Conference on Machine Learning, ICML...

    2022

  13. [22]

    J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: bootstrapping language-image pre- training with frozen image encoders and large language models, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org, 2023

    2023

  14. [23]

    L. Xue, M. Shu, A. Awadalla, J. Wang, A. Yan, S. Purushwalkam, H. Zhou, V . Prabhu, Y . Dai, M. S. Ryoo, S. Kendre, J. Zhang, C. Qin, S. Zhang, C. Chen, N. Yu, J. Tan, T. M. Awalgaonkar, S. Heinecke, H. Wang, Y . Choi, L. Schmidt, Z. Chen, S. Savarese, J. C. Niebles, C. Xiong,...

    2024 doi

  15. [24]

    L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y . Lin, W. X. Zhao, Z. Wei, J. Wen, A survey on large language model based autonomous agents, Frontiers Comput. Sci. 18 (6) (2024) 186345. doi:10.1007/S11704-024-40231-1

    2024 doi

  16. [25]

    Z. Bai, T. Xiao, T. He, P. W ANG, Z. Zhang, T. Brox, M. Z. Shou, Bridging information asymmetry in text-video retrieval: A data-centric approach, in: The Thirteenth International Conference on Learning Representations, 2025. 32

    2025

  17. [26]

    Shvetsova, A

    N. Shvetsova, A. Kukleva, B. Schiele, H. Kuehne, In-Style: Bridging Text and Uncurated Videos with Style Transfer for Text-Video Retrieval , in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE Com- puter Society, Los Alamitos, CA, USA, 2023, pp. 21924–21...

    2023

  18. [27]

    J. Wang, P. Wang, G. Sun, D. Liu, S. Dianat, R. Rao, M. Rabbani, Z. Tao, Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrieval , in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, Los Alamitos, CA, USA, 2024...

    2024

  19. [28]

    Zhang, Y

    H. Zhang, Y . Yang, F. Qi, S. Qian, C. Xu, Robust video-text retrieval via noisy pair calibration, IEEE Transactions on Multimedia 25 (2023) 8632–8645. doi: 10.1109/TMM.2023.3239183

    2023

  20. [29]

    Brookes, M

    O. Brookes, M. Mirmehdi, H. Kuhl, T. Burghardt, Chimpvlm: Ethogram- enhanced chimpanzee behaviour recognition (2024). arXiv:2404.08937

    2024 arXiv

  21. [30]

    Brookes, M

    O. Brookes, M. Mirmehdi, C. Stephens, S. Angedakin, K. Corogenes, D. Dowd, P. Dieguez, T. C. Hicks, S. Jones, K. Lee, V . Leinert, J. Lapuente, M. S. Mc- Carthy, A. Meier, M. Murai, E. Normand, V . Vergnes, E. G. Wessling, R. M. Wittig, K. Langergraber, N. Maldonado, X. Yang, ...

    2024 doi

  22. [31]

    J. P. Bohnslav, N. K. Wimalasena, K. J. Clausing, Y . Y . Dai, D. A. Yarmolinsky, T. Cruz, A. D. Kashlan, M. E. Chiappe, L. L. Orefice, C. J. Woolf, C. D. Harvey, Deepethogram, a machine learning pipeline for supervised behavior classification from raw pixels, eLife 10 (2021) ...

    2021 doi

  23. [32]

    M. Bain, A. Nagrani, D. Schofield, S. Berdugo, J. Bessa, J. Owen, K. J. Hockings, T. Matsuzawa, M. Hayashi, D. Biro, S. Carvalho, A. Zisserman, Automated au- 33 diovisual behavior recognition in wild primates, Science Advances 7 (46) (2021) eabi4883. doi:10.1126/sciadv.abi4883

    2021 doi

  24. [33]

    Y . Jing, R. Zhang, K. Liang, Y . Li, Z. He, Z. Ma, J. Guo, Animal-bench: Bench- marking multimodal video models for animal-centric video understanding, in: A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. M. Tomczak, C. Zhang (Eds.), Advances in Neural Information...

    2024

  25. [34]

    J. Chen, M. Hu, D. J. Coker, M. L. Berumen, B. R. Costelloe, S. Beery, A. Rohrbach, M. Elhoseiny, Mammalnet: A large-scale video benchmark for mammal recognition and behavior understanding, in: IEEE /CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouv...

    2023

  26. [35]

    Dussert, V

    G. Dussert, V . Miele, C. Van Reeth, A. Delestrade, S. Dray, S. Chamaillé-Jammes, Zero-shot animal behavior classification with vision-language foundation models, bioRxiv (2024). doi:10.1101/2024.04.05.588078

    2024 doi

  27. [36]

    J. J. Sun, H. Zhou, L. Zhao, L. Yuan, B. Seybold, D. Hendon, F. Schro ff, D. A. Ross, H. Adam, B. Hu, T. Liu, Video foundation models for animal behavior analysis, bioRxiv (2024). doi:10.1101/2024.07.30.605655

    2024 doi

  28. [37]

    Radford, J

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, I. Sutskever, Robust speech recognition via large-scale weak supervision, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org, 2023

    2023

  29. [38]

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. L. Zitnick, Microsoft coco: Common objects in context, in: D. Fleet, T. Pa- jdla, B. Schiele, T. Tuytelaars (Eds.), Computer Vision – ECCV 2014, Springer International Publishing, Cham, 2014, pp. ...

    2014

  30. [39]

    I. Delval, O desenvolvimento da personalidade em macacos-prego: unindo psi- cologia e ecologia comportamental, Doctoral thesis, Instituto de Psicologia, Uni- versity of São Paulo, accessed: 2025-04-21 (2019). doi:10.11606/T.47. 2019.tde-08112019-172134. 35

    2019 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.