Pith. sign in

REVIEW 3 major objections 6 minor 46 references

Domain Adaptation of VLM for Soccer Video Understanding

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A three-stage curriculum adapts a general-purpose video language model to soccer, lifting action classification from 11.8% to 63.5%.

desk verdict A practical, compute-efficient recipe for video VLM domain adaptation with a credible hard-label action classification gain, but the headline VQA/caption numbers are weakened by a Claude-to-Claude evaluation loop and one internal inconsistency in the reported accuracy. read the letter →

arxiv 2505.13860 v2 pith:76ZQXBRQ submitted 2025-05-20 cs.CV cs.AI

Tiancheng Jiang , Henry Wang , Md Sirajus Salekin , Parmida Atighehchian , Shinan Zhang This is my paper · ORCID
classification cs.CVcs.AI
keywords videovisionlanguagemodeldomainadaptationcurriculumlearningsoccerunderstandingvisualquestionansweringactionclassificationsyntheticinstructiondatalow-rankfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a general-purpose video vision-language model can be adapted to a specialized domain without a large annotation budget, and it answers yes for soccer. The claim is that a three-stage curriculum—concept alignment, instruction tuning, then downstream-task fine-tuning—on roughly 20,000 two-second event clips raises visual question-answering by 37.5% relative and soccer action classification from 11.8% to 63.5% accuracy. The recipe uses synthetic instruction data distilled from eight frames plus the event label, and trains only low-rank adapters on four 24 GB GPUs, so the adaptation is cheap enough to repeat. If correct, this is a practical template for moving off-the-shelf video models into specialized domains.

What carries the argument

The load-bearing mechanism is the curriculum itself, applied through a uniform instruction-following format. Clips are cut to two seconds and sampled at eight frames so that each clip centers on one labeled event, which keeps the visual context focused and reduces hallucination when the teacher LLM writes captions. The first two training stages freeze the visual encoder and train low-rank adapters on the language model and the projection layer that connects vision features to text; the third stage trains only the language-side adapters. The progression teaches the model soccer concepts before question-answering behavior and task formatting, which the ablation shows is the order that extracts the largest gains.

What would settle it

Take a set of test clips, replace their frames with frames from a different event class while keeping the original event label in the synthetic caption, and measure whether the adapted model's captions and answers follow the label or the frames; if they follow the label, the curriculum has taught label-to-text imitation rather than visual understanding.

Watch

Extended reading notes

Core claim

The paper's central claim is that a general video VLM can be specialized to a new domain by fine-tuning it in a fixed order on synthetic instruction data built from short, event-aligned clips. Stage one teaches concept alignment by captioning two-second clips from captions written from eight frames plus the ground-truth event label. Stage two turns those captions into five kinds of question-answer pairs and instruction-tunes the model on them. Stage three fine-tunes the model on a downstream task's output format, here 13-class soccer action classification. The full pipeline reaches 63.5% accuracy versus 11.8% for the base model on that task and a 37.5% relative improvement on VQA; ablations show that direct fine-tuning, or skipping either of the first two stages, gives materially worse results.

Load-bearing premise

The synthetic captions, question-answer pairs, and the VQA judge all come from the same commercial LLM; if that LLM writes mostly from the event label and its own priors rather than from the eight frames, the reported VQA gains could be prose mimicry rather than visual understanding.

Editorial extensions

If this is right

  • If the recipe transfers, a general video VLM can be specialized to a dense-action domain with roughly 20k labeled event clips and four consumer-scale GPUs, instead of requiring large domain-specific pretraining.
  • The curriculum ordering is load-bearing: concept alignment before instruction tuning raises the relative VQA score from 79.1 to 85.8, and the full three-stage sequence raises action classification accuracy to 63.5% versus 16% for direct fine-tuning.
  • The model's temporal robustness survives the short training window: classifying 5-second clips reaches a macro F1 of 0.61 versus 0.63 on 2-second clips.
  • Including a second, broader event source in the 20k training set sharply reduces the cross-domain gap on VQA, indicating that data diversity matters as much as volume.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the hard-label action classification task is the most trustworthy evidence of real soccer understanding, because the caption and VQA scores are produced by the same teacher LLM that wrote the synthetic data, so those gains could partly reflect style imitation.
  • Beyond the paper: a sharper test of visual grounding would be to corrupt or swap the frames while keeping the event label and ask whether the adapted model's answers follow the label or the pixels.
  • Beyond the paper: the same event-aligned two-second window recipe is a natural fit for other event-dense video domains with timestamped labels, such as other sports, surveillance, or procedure videos.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper adapts LLaVA-NeXT-Video to soccer video understanding using a three-stage curriculum: Stage 1, Soccer Concept Alignment, trains the model on synthetic captions generated by Claude 3.5 Sonnet from 8 frames plus the ground-truth event label; Stage 2, Soccer VQA Instruction Tuning, trains on question-answer pairs generated by Claude from those captions; Stage 3 fine-tunes on a downstream soccer action classification task. The authors report that the final 20K model improves VQA relative score from 60.20 to 82.79 and action classification accuracy from 11.8% to 63.5%, and they provide ablations on dataset size, training stage order, LoRA rank, projector training, and image-plus-video training. The central claim is that this multi-stage, compute-efficient recipe is an effective way to adapt a general-purpose video VLM to a specialized domain.

Significance. If the results hold, this is a practically useful contribution: the recipe is simple, uses only four 24G A10 GPUs, and the action classification anchor (11.8% to 63.5%) is an independent, hard-label signal that does not depend on the LLM judge. The ablations are reported honestly and cover the key design decisions (training order, dataset scale, LoRA rank, projector choice, clip length). The main weakness is that the caption and VQA headline numbers are produced inside a partially self-referential evaluation loop: Claude generates the training captions, the QA pairs, and the reference answers, and Claude also judges the outputs. The hard-label action classification task is therefore the most important evidence, but the reported accuracy for that task is internally inconsistent between Section 5.2, Figure 4, Table 7, and Table 14.

major comments (3)
  1. [5.1, 5.2, Tables 1-2, Figures 8-9] The headline VQA and caption improvements rest on a partially self-referential evaluation loop. Stage 1 captions are generated by Claude 3.5 Sonnet from 8 frames plus the ground-truth event label (Figure 8); Stage 2 QA pairs are generated from those captions by the same model (Figure 9); the VQA reference answers are written by Claude; and the judge for both caption correctness/detailness and VQA relative score is Claude (Section 5.1). The adapted model is therefore trained to imitate Claude's output style and then scored by a Claude judge, so the 37.5% VQA gain (Base 60.20 to 20K 82.79 in Table 2) and the caption-score gains in Table 1 may partly reflect stylistic mimicry rather than soccer understanding. Section 5.4 concedes that synthetic data 'may introduce stylistic biases.' To make the VQA and caption claims load-bearing, the paper should add an independent evaluation, such as human ratings or a judge from a different model family, and should report variance across runs; without this, the VQA/caption improvements are not separable from style alignment.
  2. [5.2, Figure 4, Table 7, Table 14, Abstract] The action classification headline is internally inconsistent. Section 5.2 reports the 20K model's accuracy as 'increasing from 11.8% to 57.8%,' while Figure 4's caption reports 0.635 and Table 7 reports 20K accuracy 0.635; Table 14 reports 63.5% for the full CA-to-IT-to-AC sequence. The text does not reconcile 57.8% with 63.5%. If one value is for the 1,300-sample test set and the other for the 100-sample test set, that must be stated explicitly. The abstract's '63.5%' claim is not reproducible from the paper as written. In addition, the paper removes four rare SoccerNet classes (Red card, Yellow-to-Red card, Penalty, Kick off) post hoc, so the reported accuracy applies only to a 13-class balanced subset; this caveat should appear wherever the headline number is quoted.
  3. [5.1, Table 2] The VQA relative-score metric is not adequately defined or interpreted. Section 5.1 says the predicted response is scored against a 'reference upper bound response' and a relative score is computed by normalizing with that reference, but Table 2 reports scores exceeding 100 (e.g., LLaMA 3.2 reaches 126.77 on Prediction questions), which contradicts the notion of an upper bound. The paper should report the raw predicted and reference scores, define the normalization formula precisely, and explain how scores above 100 are possible. The current presentation makes the 60.20-to-82.79 improvement difficult to interpret, especially without error bars or multiple runs.
minor comments (6)
  1. [5.1, Task 3] The sentence 'a smaller test set if 100 samples' contains a typo ('if' should be 'of'), and the paper should clarify which test set (1,300 or 100 samples) is used for the numbers in Table 7.
  2. [Table 6 and Section 5.2] The metric 'ROGUE' should be 'ROUGE' in both the table and the text.
  3. [Figures 8 and 10] The caption-generation prompt in Figure 8 instructs the model to anonymize team/player names and not to mention jersey numbers, but the QA-generation few-shot examples in Figure 10 include a player 'wearing the number 11 jersey' and specific player references; this inconsistency should be resolved or explained.
  4. [Table 1] The 3K and 10K rows of Table 1 are formatted as two-line entries without a clear separator, making it hard to tell which numbers belong to Correctness versus Detailness; please reformat for clarity.
  5. [References] References [6] and [7] are the same SoccerNet-V2 paper and should be merged to avoid duplicate entries.
  6. [4.2, Experiments] The paper does not report training hyperparameters (epochs, learning rate, LoRA alpha, batch size, optimizer, warmup). Since the paper claims a repeatable recipe, these values should be added.

Circularity Check

2 steps flagged · score 5.0 of 10

VQA and caption gains are partly Claude-style alignment: training targets, reference answers, and judge are all Claude 3.5 Sonnet, so the reported synthetic-text improvements are partially self-referential; hard-label action classification is the only fully independent anchor, and its reported accuracy is internally inconsistent (57.8% vs 63.5%).

  1. self definitional [Section 4.2 (Stage 2) and Section 5.1 (Task 2)]
    "We construct this stage’s instruction-following dataset using the synthetically generated captions from Stage 1, prompting Claude 3.5 Sonnet V1 to generate five distinct question-answer pairs per caption. … For evaluation, we used Claude 3.5 Sonnet V1 to score both the predicted response and a reference upper bound response on helpfulness, relevance, accuracy, and level of details from 1 to 10 (10 being the highest score). We compute a relative score by normalizing the predicted response score with the upper bound response score."

    The Stage 2 training targets are Claude-generated question-answer pairs derived from Claude-generated Stage 1 captions, and the VQA metric is Claude's judgment of the model's response relative to a Claude-written reference answer. The reported 37.5% VQA improvement (60.20 to 82.79 in Table 2) therefore measures, in part, how closely the adapted model reproduces the same answer style and content distribution that produced its training data; it is not an independent measurement of soccer understanding. The hard-label action-classification task is the only fully external anchor for this claim.

  2. self definitional [Section 5.1 (Task 1)]
    "Unlike the training data in Stages 1 and 2, where the 3k and 10k datasets only contain SoccerNet clips, both test sets here contain an equal split between SoccerNet and WyScout samples. Captions were generated using the same approach as in Stage 1. … The first was LLM-as-a-Judge metrics, where we asked Claude 3.5 Sonnet V1 to rate the predicted captions against the label captions, and two scores from 1 to 5 (5 being the best) were provided for Correctness and Detailness."

    The test captions used as references are generated by the same Claude 3.5 Sonnet procedure as the Stage 1 training captions, and the judge is also Claude 3.5 Sonnet. Consequently the Correctness and Detailness scores reward agreement with Claude's caption style and content, and the caption-score gains in Table 1 may partly reflect imitation of the data generator rather than verified soccer grounding. The BLEU/ROUGE metrics in Table 6 share the same Claude-generated reference corpus, so they do not remove the self-referential element.

full rationale

The central recipe claim — that the CA→IT→AC curriculum improves an open video VLM on soccer — is not shown to be fully circular: the action-classification task uses external SoccerNet event labels, and the ablation in Table 14 (Base→AC 16% vs. CA→AC 52% vs. CA→IT→AC 63.5%) is an internally valid comparison of training strategies on that same external task. However, the headline VQA and caption improvements are partially self-referential by construction. Stage 1 and Stage 2 training targets are Claude 3.5 Sonnet captions and QA pairs generated from those captions, and Task 1/Task 2 evaluation uses Claude 3.5 Sonnet as judge against Claude-generated reference texts, with the relative VQA score defined as alignment with that reference. A model can therefore raise these scores by imitating Claude's output distribution rather than by acquiring ground-truth soccer understanding; the paper's Limitations section concedes that synthetic data 'may introduce stylistic biases.' The fully independent hard-label anchor is weakened by an internal inconsistency in the reported 20k action accuracy (57.8% in Section 5.2 vs. 63.5% in Figure 4/Table 7) and by the post-hoc removal of four rare classes, but those are correctness risks, not circular reductions. On balance, the synthetic-text evaluations are partially self-referential, while the curriculum claim retains independent content, so the score is 5 rather than higher.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central results rest on several hand-chosen design choices (clip duration, removal of rare event classes, LoRA rank, dataset composition) and on assumptions that the Claude-generated synthetic text is visually grounded and that Claude-as-a-judge is a fair evaluator. No new entities are postulated.

free parameters (5)
  • Video clip duration = 2 seconds (0.5 s before, 1.5 s after event)
    Chosen by hand in Sec 3.2; Sec 5.3 argues shorter clips reduce noise and hallucination with a qualitative figure but no formal selection criterion.
  • SoccerNet event classes used = 13 of 17 (removed Red card, Yellow to Red card, Penalty, Kick off)
    Four rare classes removed in Sec 3.2; this changes the downstream classification task and the reported 11.8% baseline.
  • LoRA rank = 64
    Appendix 4.2 ablation showed little difference between rank 32 and 64; the authors chose 64 for the 20k model.
  • Training dataset sizes = 3k, 10k, 20k for stages 1-2; 3.3k for stage 3
    Scaling study in Sec 5.2; the 20k set includes WyScout data, which drives the generalization gains.
  • Free-form to multiple-choice ratio = 4:1
    Fixed in Sec 5.1 and the prompt in Fig 9; affects the composition and difficulty of the VQA evaluation.
assumptions (5)
  • domain assumption A 2-second clip sampled at 8 frames around an event timestamp contains enough visual information to determine the soccer event.
    Sec 3.2; central to both synthetic caption generation and classification; the paper gives only a qualitative figure as support.
  • domain assumption Claude 3.5 Sonnet, given 8 frames and the ground-truth event label, produces captions and QA pairs that are accurate, diverse, and grounded in the frames rather than just the label.
    Sec 4.2 Stage 1; the entire training signal; the paper acknowledges hallucination risk for longer clips in Sec 5.4.
  • domain assumption LLM-as-a-judge scores from Claude 3.5 Sonnet are a valid measure of caption and VQA quality, and the reference responses are a sensible upper bound.
    Sec 5.1 Tasks 1-2; the judge and reference are generated by the same model that produced the training data.
  • domain assumption Event labels and train/test splits from SoccerNet-V2 and WyScout are reliable and preserved correctly in clip extraction.
    Sec 3.1; WyScout labels are proprietary and not auditable from the paper.
  • standard math Autoregressive next-token prediction on synthetic instruction data is the correct objective for domain adaptation.
    Eq. 1 in Sec 4.1; standard VLM training objective.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Domain Adaptation of VLM for Soccer Video Understanding." pith.science (2026). https://pith.science/paper/76ZQXBRQ

@misc{pith2026250513860,
  author       = {Pith},
  title        = {Pith review of: Domain Adaptation of VLM for Soccer Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/76ZQXBRQ}},
  note         = {Machine review of arXiv:2505.13860}
}
read the original abstract

Vision Language Models (VLMs) have demonstrated strong performance in multi-modal tasks by effectively aligning visual and textual representations. However, most video understanding VLM research has been domain-agnostic, leaving the understanding of their transfer learning capability to specialized domains under-explored. In this work, we address this by exploring the adaptability of open-source VLMs to specific domains, and focusing on soccer as an initial case study. Our approach uses large-scale soccer datasets and LLM to create instruction-following data, and use them to iteratively fine-tune the general-domain VLM in a curriculum learning fashion (first teaching the model key soccer concepts to then question answering tasks). The final adapted model, trained using a curated dataset of 20k video clips, exhibits significant improvement in soccer-specific tasks compared to the base model, with a 37.5% relative improvement for the visual question-answering task and an accuracy improvement from 11.8% to 63.5% for the downstream soccer action classification task.

Figures

Figures reproduced from arXiv: 2505.13860 by the authors.

Figure 1
Figure 1. Overall workflow of our proposed multi-stage domain adaptation process. The LLaVA-Next-Video model is iteratively [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of the VQA task abilities measured by [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. A qualitative example from the VQA task test set showing the ground-truth answer and responses from the different [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Confusion Matrices from the Action Classification task for the Base model (LLaVA-NeXT-Video), and our adapted 20k [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Demonstration of short clip vs. long clip on information continuity. The above 2s clip is much more continuous hence [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: The same instruction is applied to all samples in the trainin [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: We used the above to prompt Claude 3.5 Sonnet to generate synth [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: We used the above to prompt Claude 3.5 Sonnet to generate synth [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Definition of the five question types and some few-shot example [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: A multiple choice example from the VQA task test set showin [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: A free response example from the VQA task test set showing t [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 37 canonical work pages

  1. [1]

    Claude 3.5 sonnet: Our best model yet,

    Anthropic. Claude 3.5 sonnet: Our best model yet,

    work page

  2. [2]

    Minigpt4-video: Advanc- ing multimodal llms for video understanding with interleaved visual-textual tokens

    Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrah- man, Essam Sleiman, Deyao Zhu, Jian Ding, and Mohamed Elhoseiny. Minigpt4-video: Advanc- ing multimodal llms for video understanding with interleaved visual-textual tokens. arXiv preprint arXiv:2404.03413, 2024. 2

    arXiv 2024

  3. [3]

    Florian Bordes, Richard Y uanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Ma ˜nas, Zhiqiu Lin, Anas Mah- moud, Bargav Jayaraman, Mark Ibrahim, Melissa Hall, Y unyang Xiong, Jonathan Lebensold, Can- dace Ross, Srihari Jayakumar, Chuan Guo, Diane Bouchacourt, Haider Al-Tahan, Karthik Padthe, V asu Sharma, Huijuan Xu, Xiaoqing...

    arXiv 2024

  4. [4]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carba- jal, Y evgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakr- ishnan, Kehang Han, Karol Hausman, Alex Her- zog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashn...

    arXiv 2023

  5. [5]

    Soccernet-tracking: Multiple object tracking dataset and benchmark in soccer videos

    Anthony Cioppa, Silvio Giancola, Adrien Deliege, Le Kang, Xin Zhou, Zhiyu Cheng, Bernard Ghanem, and Marc V an Droogenbroeck. Soccernet-tracking: Multiple object tracking dataset and benchmark in soccer videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops , pages 3491–3502, 2022. 2

    work page 2022

  6. [6]

    Soccernet- v2: A dataset and benchmarks for holistic under- standing of broadcast soccer videos

    Adrien Deliege, Anthony Cioppa, Silvio Giancola, Meisam J Seikavandi, Jacob V Dueholm, Kamal Nasrollahi, Bernard Ghanem, Thomas B Moes- lund, and Marc V an Droogenbroeck. Soccernet- v2: A dataset and benchmarks for holistic under- standing of broadcast soccer videos. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition Worksh...

    work page 2021

  7. [7]

    Soccernet- v2: A dataset and benchmarks for holistic under- standing of broadcast soccer videos

    Adrien Deliege, Anthony Cioppa, Silvio Giancola, Meisam J Seikavandi, Jacob V Dueholm, Kamal Nasrollahi, Bernard Ghanem, Thomas B Moes- lund, and Marc V an Droogenbroeck. Soccernet- v2: A dataset and benchmarks for holistic under- standing of broadcast soccer videos. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition Worksh...

    work page 2021

  8. [8]

    Mmbench-video: A long-form multi-shot bench- mark for holistic video understanding

    Xinyu Fang, Kangrui Mao, Haodong Duan, Xi- angyu Zhao, Yining Li, Dahua Lin, and Kai Chen. Mmbench-video: A long-form multi-shot bench- mark for holistic video understanding. Advances in Neural Information Processing Systems , 37: 89098–89124, 2024. 1

    work page 2024

Show all 46 references
  1. [9]

    Soccernet- echoes: A soccer game audio commentary dataset

    Sushant Gautam, Mehdi Houshmand Sarkhoosh, Jan Held, Cise Midoglu, Anthony Cioppa, Silvio Giancola, V ajira Thambawita, Michael A Riegler, Pal Halvorsen, and Mubarak Shah. Soccernet- echoes: A soccer game audio commentary dataset. In 2024 International Symposium on Multimedia ...

    2024

  2. [10]

    Soccernet: A scal- able dataset for action spotting in soccer videos

    Silvio Giancola, Mohieddine Amine, Tarek Dghaily, and Bernard Ghanem. Soccernet: A scal- able dataset for action spotting in soccer videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops , pages 1711–1721, 2018. 1, 2

    2018

  3. [11]

    Fine- grained action recognition on a novel basketball dataset

    Xiaofan Gu, Xinwei Xue, and Feng Wang. Fine- grained action recognition on a novel basketball dataset. In International Conference on Acoustics, Speech, and Signal Processing , pages 2563–2567. IEEE, 2020. 2

    2020

  4. [12]

    V ars: Video assistant referee sys- tem for automated soccer decision making from multiple views

    Jan Held, Anthony Cioppa, Silvio Giancola, Ab- dullah Hamdi, Bernard Ghanem, and Marc V an Droogenbroeck. V ars: Video assistant referee sys- tem for automated soccer decision making from multiple views. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Re...

    2023

  5. [13]

    X-vars: Introducing explainability in foot- ball refereeing with multi-modal large language 9 models

    Jan Held, Hani Itani, Anthony Cioppa, Silvio Gi- ancola, Bernard Ghanem, and Marc V an Droogen- broeck. X-vars: Introducing explainability in foot- ball refereeing with multi-modal large language 9 models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Patt...

    2024

  6. [14]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Y elong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Y uanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Confer- ence on Learning Representations , 2022. 4

    2022

  7. [15]

    Vtimellm: Empower llm to grasp video moments

    Bin Huang, Xin Wang, Hong Chen, Zihan Song, and Wenwu Zhu. Vtimellm: Empower llm to grasp video moments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14271–14280, 2024. 1

    2024

  8. [16]

    Wyscout: Football data and analytics plat- form, 2024

    Hudl. Wyscout: Football data and analytics plat- form, 2024. Accessed: March 5, 2025. 3

    2024

  9. [17]

    Chen, and An- drew Y

    Yixing Jiang, Jeremy Irvin, Ji Hun Wang, Muham- mad Ahmed Chaudhry, Jonathan H. Chen, and An- drew Y . Ng. How well does gpt-4v(ision) adapt to distribution shifts? a preliminary investigation. arXiv preprint arXiv:2401.06794, 2024. 1

    2024 arXiv

  10. [18]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Y ang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems , 36: 28541...

    2023

  11. [19]

    Sports-qa: A large-scale video ques- tion answering benchmark for complex and pro- fessional sports

    Haopeng Li, Andong Deng, Qiuhong Ke, Jun Liu, Hossein Rahmani, Y ulan Guo, Bernt Schiele, and Chen Chen. Sports-qa: A large-scale video ques- tion answering benchmark for complex and pro- fessional sports. arXiv preprint arXiv:2401.01505, 2024. 2

    2024

  12. [20]

    BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large lan- guage models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large lan- guage models. In Proceedings of the 40th Inter- national Conference on Machine Learning , pages 19730–19742. PMLR, 2023. 2

    2023

  13. [21]

    Visualbert: A simple and performant baseline for vision and language

    Liunian Harold Li, Mark Y atskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019. 2

    1908 arXiv

  14. [22]

    Chain-of-region: Visual language models need details for diagram analysis

    Xue Li, Yiyou Sun, Wei Cheng, Yinglun Zhu, and Haifeng Chen. Chain-of-region: Visual language models need details for diagram analysis. In The Thirteenth International Conference on Learning Representations, 2025. 2

    2025

  15. [23]

    Video-llava: Learning united vi- sual representation by alignment before projection

    Bin Lin, Bin Zhu, Y ang Y e, Munan Ning, Peng Jin, and Li Y uan. Video-llava: Learning united vi- sual representation by alignment before projection. arXiv preprint arXiv:2311.10122, 2023. 2, 3

    2023 arXiv

  16. [24]

    Vila: On pre- training for visual language models

    Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre- training for visual language models. In Proceed- ings of the IEEE/CVF conference on computer vi- sion and pattern recognition , pages 26689–26699, 2024. 2

    2024

  17. [25]

    Llava-next: Improved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Y uheng Li, Bo Li, Y uanhan Zhang, Sheng Shen, and Y ong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024. 2, 3

    2024

  18. [26]

    Vilbert: Pretraining task-agnostic visiolin- guistic representations for vision-and-language tasks

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolin- guistic representations for vision-and-language tasks. In Advances in Neural Information Process- ing Systems. Curran Associates, Inc., 2019. 2

    2019

  19. [27]

    Video-chatgpt: Towards detailed video understanding via large vi- sion and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vi- sion and language models. In Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (ACL 2024) , 2024. 2, 3

    2024

  20. [28]

    Soccernet-caption: Dense video caption- ing for soccer broadcasts commentaries

    Hassan Mkhallati, Anthony Cioppa, Silvio Gian- cola, Bernard Ghanem, and Marc V an Droogen- broeck. Soccernet-caption: Dense video caption- ing for soccer broadcasts commentaries. In Pro- ceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition Workshops , ...

    2023

  21. [29]

    Med-flamingo: a multimodal medical few-shot learner

    Michael Moor, Qian Huang, Shirley Wu, Michi- hiro Y asunaga, Y ash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Ra- jpurkar. Med-flamingo: a multimodal medical few-shot learner. In Machine Learning for Health (ML4H), pages 353–367. PMLR, 2023. 2

    2023

  22. [30]

    Sports video captioning via attentive mo- tion representation and group relationship model- ing

    Mengshi Qi, Y unhong Wang, Annan Li, and Jiebo Luo. Sports video captioning via attentive mo- tion representation and group relationship model- ing. IEEE Transactions on Circuits and Systems for Video Technology, 30(8):2617–2633, 2019. 2

    2019

  23. [31]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  24. [32]

    Matchtime: Towards au- tomatic soccer game commentary generation

    Jiayuan Rao, Haoning Wu, Chang Liu, Y anfeng Wang, and Weidi Xie. Matchtime: Towards au- tomatic soccer game commentary generation. In Proceedings of the Conference on Empirical Meth- ods in Natural Language Processing , 2024. 2

    2024

  25. [33]

    Multilingual vision-language pre-training for the remote sensing domain

    Jo ˜ao Daniel Silva, Jo ˜ao Magalh ˜aes, Devis Tuia, and Bruno Martins. Multilingual vision-language pre-training for the remote sensing domain. In Pro- ceedings of the 32nd ACM International Confer- 10 ence on Advances in Geographic Information Sys- tems, pages 220–232, 2024. 2

    2024

  26. [34]

    Computer vi- sion for sports: Current applications and research topics

    Graham Thomas, Rikke Gade, Thomas B Moes- lund, Peter Carr, and Adrian Hilton. Computer vi- sion for sports: Current applications and research topics. Computer Vision and Image Understand- ing, 159:3–18, 2017. 2

    2017

  27. [35]

    Semi-supervised training to improve player and ball detection in soccer

    Renaud V andeghen, Anthony Cioppa, and Marc V an Droogenbroeck. Semi-supervised training to improve player and ball detection in soccer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 3481–3490, 2022. 2

    2022

  28. [36]

    Knowledge graph supported benchmark and video captioning for basketball

    Zeyu Xi, Ge Shi, Lifang Wu, Xuefen Li, Junchi Y an, Liang Wang, and Zilin Liu. Knowledge graph supported benchmark and video captioning for basketball. arXiv preprint arXiv:2401.13888 , 2024. 2

    2024 arXiv

  29. [37]

    Sportqa: A benchmark for sports under- standing in large language models

    Haotian Xia, Zhengbang Y ang, Y uqing Wang, Rhys Tracy, Y un Zhao, Dongdong Huang, Zezhi Chen, Y an Zhu, Y uanfang Wang, and Weining Shen. Sportqa: A benchmark for sports under- standing in large language models. In Proceedings of the Conference of the North American Chapter o...

    2024

  30. [38]

    Fine- grained video captioning for sports narrative

    Huanyu Y u, Shuo Cheng, Bingbing Ni, Minsi Wang, Jian Zhang, and Xiaokang Y ang. Fine- grained video captioning for sports narrative. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6006–6015, 2018. 2

    2018

  31. [39]

    Merlot: Multimodal neural script knowledge models

    Rowan Zellers, Ximing Lu, Jack Hessel, Y oungjae Y u, Jae Sung Park, Jize Cao, Ali Farhadi, and Y ejin Choi. Merlot: Multimodal neural script knowledge models. In Advances in Neural Information Pro- cessing Systems , pages 23634–23651. Curran As- sociates, Inc., 2021. 2

    2021

  32. [40]

    Video- llama: An instruction-tuned audio-visual language model for video understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video- llama: An instruction-tuned audio-visual language model for video understanding. In Proceedings of the 2023 Conference on Empirical Methods in Nat- ural Language Processing: System Demonstra- tions, pages 543–553, Singapore, 2023. As...

    2023

  33. [41]

    Llava-next: A strong zero-shot video understanding model, 2024

    Y uanhan Zhang, Bo Li, haotian Liu, Y ong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, 2024. 1, 3

    2024

  34. [42]

    Please provide a description of what happened in the soccer match vi deo?

    Xin Zhou, Le Kang, Zhiyu Cheng, Bo He, and Jingyu Xin. Feature combination meets attention: Baidu soccer embeddings and trans- former based temporal detection. arXiv preprint arXiv:2106.14447, 2021. 2 11 Appendix 1 Data Instructions for video clip caption. The list of instruct...

    2021 arXiv

  35. [44]

    Your output should be formed in a JSON file

  36. [45]

    Dimension

    Only provide the Python dictionary string. Your response should look like and no other text: [{"Dimension": <dimension-1>, "Question": <question-1>, "Answer":<answer-1>, "Is_multiple_choices": <True_or_False>}, {"Dimension": <dimension-2>, "Question": <question-2>, "Answer":<a...

  37. [64]

    Given this, we only opted for Rank 64 when scaling up trainin g to the 20k datasets, as it provides greater capacity for handling complex patterns

    The results show very little differences and mixed results be tween the Caption Generation and VQA tasks. Given this, we only opted for Rank 64 when scaling up trainin g to the 20k datasets, as it provides greater capacity for handling complex patterns. Correctness Scores Detai...

  38. [2024]

    Accessed: 2025-03-11. 4

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.