Multi-Agent System for Comprehensive Soccer Understanding

Haoning Wu; Jiayuan Rao; Weidi Xie; Yanfeng Wang; Ya Zhang; Zifeng Li

REVIEW 5 major objections 7 minor 93 references

Multi-Agent System for Comprehensive Soccer Understanding

T0 review · 5 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a 10,000-question benchmark and an 18-tool multi-agent system bring comprehensive, knowledge-driven soccer understanding within reach.

desk verdict A useful, well-scoped soccer QA benchmark and agent toolkit whose central comparison is only as trustworthy as its QA labels, and those labels are currently unvalidated. read the letter →

arxiv 2505.03735 v2 pith:IWLW4T6T submitted 2025-05-06 cs.CV

Jiayuan Rao , Zifeng Li , Haoning Wu , Ya Zhang , Yanfeng Wang , Weidi Xie This is my paper · ORCID

classification cs.CV

keywords soccerunderstandingmultimodalquestionansweringmulti-agentsystemknowledgebasebenchmarkvideotoolusesportsanalytics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that holistic soccer understanding can be turned into a standard, knowledge-driven benchmark and solved by an agentic system rather than a single monolithic model. To do so it assembles SoccerWiki, a multimodal knowledge base of players, teams, referees, venues, and match records; builds SoccerBench, roughly 10,000 multiple-choice questions across 13 text, image, and video tasks; and introduces SoccerAgent, a multi-agent system that plans an ordered chain of specialized tools and executes them to answer questions. The paper's core empirical claim is that SoccerAgent outperforms representative commercial and open-source vision-language models on SoccerBench, with the largest gains on knowledge-intensive tasks. A sympathetic reader would care because a unified benchmark plus a reusable agent baseline could move sports AI from isolated perception tasks toward open-ended, knowledge-grounded reasoning about matches.

What carries the argument

The load-bearing mechanism is the plan-then-execute agent loop. Given a question, a planning module first decomposes it into an ordered tool chain, chosen from 18 tools that include face recognition, match search, action classification, camera detection, jersey number recognition, score/time recognition, replay grounding, foul recognition, and general-purpose operations such as frame selection and textual retrieval. An execution module then calls each tool in sequence, feeding each call a structured instruction with tool name, query, file paths, and purpose, while accumulating an execution history that later steps can consult. The chain ends with an answer-selection step that maps the gathered evidence to a multiple-choice option. SoccerBench itself is the other half of the machinery: for each task, questions are produced either by templates or by prompting a large language model, distractors are sampled or generated, and roughly 10,000 of 100,000 synthesized pairs are kept after manual filtering. The benchmark supplies the measurement that makes the agent's superiority visible.

What would settle it

Take a random sample of 200 to 300 SoccerBench items, have independent soccer-knowledgeable annotators answer them without seeing the published options' origin, and compare their answers and confidence to the dataset's labels. If a material fraction of items has a wrong key or a distractor that is obviously false to an informed reader, then the benchmark rankings and SoccerAgent's margin rest on shaky ground. A complementary check is to feed the image and video questions to a strong text-only model; any substantial accuracy above chance would mean the visual material is not load-bearing for those items.

Watch

Extended reading notes

Core claim

The central discovery the paper argues for is that SoccerBench, with about 10,000 multimodal multiple-choice QA pairs spanning 13 tasks, is the largest and most comprehensive soccer understanding benchmark currently available, and that SoccerAgent, a multi-agent system combining a planning module, an execution module, and 18 specialized tools over SoccerWiki, achieves robust performance on it. The authors report that SoccerAgent's best variant reaches 85.0 percent on TextQA, 73.3 percent on ImageQA, and 60.9 percent on VideoQA, the top scores among the models compared, while the strongest baseline commercial model trails on most knowledge-heavy tasks. They further argue that the system works in an open-ended mode, generating answers without seeing the candidate options, and that swapping in a stronger vision tool improves results, evidence that the architecture is scalable. On the paper's own account, the benchmark differentiates models, with accuracy ranges wide enough to separate generic perception from soccer-specific knowledge.

Load-bearing premise

The load-bearing premise is that the roughly 10,000 multiple-choice answers in SoccerBench are correct, unambiguous, and paired with three genuinely plausible wrong answers, even though the filtering that produced them was manual and no inter-annotator agreement or external validation is reported.

Editorial extensions

If this is right

SoccerBench gives the field a single standardized yardstick for comparing models across 13 soccer tasks, from background knowledge to multi-view foul recognition, replacing fragmented specialist evaluations.
SoccerAgent's open-ended variant answers without seeing the candidate options and remains competitive, so the benchmark can measure genuine reasoning rather than option-matching behavior.
Because SoccerAgent's components are mostly open-source and interchangeable, replacing one vision tool with a stronger model raises overall accuracy, indicating the framework can absorb future advances without redesign.
The paper's failure analysis attributes most errors to specific tools such as match search, action classification, and entity recognition, making bottlenecks explicit and giving future work a clear target.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The curation pipeline of templates plus language-model generation with manual filtering is not soccer-specific, so a similar benchmark could be built for other sports or for other knowledge-heavy video domains without changing the core machinery.
Because SoccerWiki can refresh from live match data, SoccerBench could evolve into a living benchmark that tracks how quickly models and agents absorb new seasons; the paper hints at updatability but does not measure it.
A text-only probe of the image and video questions would quantify how much of SoccerBench actually requires pixels; the authors manually filter obvious cases, but an automatic leakage statistic would harden the benchmark.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look, and worth a serious referee, but the headline claim needs one more round of validation before I would trust any ranking in Table 2.

What is actually new: SoccerBench is the first unified multimodal QA set over 13 soccer tasks, and SoccerWiki is a genuinely useful knowledge resource. The paper assembles these from existing datasets plus a lot of new curation, and the multi-agent system is a reasonable recipe for tool-use QA in a specialized domain. The transparency about tool bottlenecks (Appendix B.3) is a real credit: they say where the failures come from, and they quantify that Entity Recognition at 63.4% player ID accuracy directly caps Q4 at 65.3%. That is the kind of honest reporting that makes a systems paper useful.

The stress-test concern lands. The whole benchmark evaluation is built on 10K QA pairs synthesized by DeepSeek-v3 and manually filtered, with no inter-annotator agreement, no per-task error rate, no distractor plausibility measurement, and no external audit. Appendix C.3 mentions filtering out image/video questions answerable without visual context but gives no count and no reliability check. If even a few percent of labels are wrong or distractors are trivially disambiguable, the accuracy numbers in Table 2 shift, and the claimed superiority of SoccerAgent over chat-only MLLMs could change. That is a missing-validation problem, not a fraud allegation, but it is load-bearing.

There are softer issues too. The paper reports aggregate accuracy without confidence intervals, and the ablation never isolates the multi-agent contribution from the tools: the agent loses to GPT-4o on Q5 and Q13, and to Gemini on Q8 and Q12, so the "leading results across TextQA, ImageQA, and VideoQA" is only true in aggregate. Since several tools (UniSoccer, MatchTime, SoccerReplay-1988, SoccerWiki) come from the same group, the evaluation could favor the in-house pipeline even absent any intent. That is not disqualifying, but it deserves an external re-run. And the authors say the benchmark and code are not released, which limits verification.

Who is this for: anyone working on sports QA, tool-augmented VLMs, or domain-specific benchmarks. It deserves a serious referee, not a desk reject. My recommendation: engage with it, but require a measured QA audit — a random sample of N questions independently re-annotated, per-task agreement, and a release of labels plus code. If the audit comes back clean, this becomes a standard resource. If not, the rankings need to be recomputed.

Referee Report

5 major / 7 minor

Summary. The paper proposes a three-part framework for soccer understanding: SoccerWiki, a multimodal knowledge base of players, teams, referees, and venues; SoccerBench, a 10K-sample multiple-choice QA benchmark spanning 13 tasks over text, image, and video; and SoccerAgent, a multi-agent system with 18 tools that plans and executes tool chains to answer soccer questions. The authors evaluate SoccerAgent against commercial and open-source MLLMs, reporting the highest aggregate accuracy on TextQA, ImageQA, and VideoQA, and they provide ablations over task descriptions and execution examples, cost analysis, and a failure analysis.

Significance. If validated, SoccerBench would be a useful unifying evaluation resource for soccer understanding, and SoccerAgent would be a strong tool-based baseline. The paper has several concrete strengths: it integrates multiple external soccer datasets into a standardized QA format, releases 17 of 18 tools as open-source, provides a transparent failure analysis in Appendix B.3, and reports cost and deployment details. The aggregate results in Table 2 are broadly consistent with the claim that an agent with retrieval and specialized tools outperforms generic MLLMs on this benchmark. The main risks are that the benchmark labels themselves are not independently validated, the ablation does not isolate the multi-agent contribution, and the reported comparisons lack uncertainty quantification; these issues are fixable but are load-bearing for the paper's headline claims.

major comments (5)

[Section 3.3.2 / Appendix C.3] The benchmark labels are the reference standard for every accuracy number in Table 2, yet their correctness and ambiguity are never measured. The pipeline synthesizes 100K QA pairs with DeepSeek-v3 and manually selects 10K, but the paper reports no inter-annotator agreement, no per-task label error rate, no distractor-plausibility measurement, and no external audit. Appendix C.3 states that ImageQA and VideoQA items answerable without visual context were filtered, but it gives no count of removed items and no reliability estimate for that filter. For the LLM-generated tasks (Q1, Q2, Q4, Q6, Q11, Q12), a hallucinated answer or an easily eliminated distractor directly changes the rankings. I request a measured audit: independent re-annotation of a stratified sample (e.g., 200 items per task), per-task agreement and label-accuracy numbers, a distractor-plausibility check (e.g., how often human annotators select the correct option at above chance), and a report of how many items were removed by each manual filter. Without this, the 'challenging benchmark' and 'superiority' claims are not yet established.
[Section 5.3, Table 3] The ablation does not isolate the multi-agent contribution. Table 3 varies only whether the planning agent receives task descriptions and whether the execution agent receives 20 examples; every variant still uses the full tool-based multi-agent architecture. There is no single-agent control that calls the same tools without decomposition, no 'no-tools' control that asks the agent core to answer directly, and no control that removes the retrieval tools. Consequently, the large gains on Q1 and Q4 could be driven by the SoccerWiki retrieval tools themselves rather than by the multi-agent decomposition and collaboration. I request ablations that separate (a) the agent core alone, (b) the agent with tools but no decomposition, and (c) the full system, so that the specific contribution of the multi-agent design is identifiable.
[Section 5.2, Table 2] No confidence intervals or significance tests are reported, so the headline comparisons are not yet statistically grounded. Several aggregate differences are small relative to the per-task sample sizes: for example, SoccerAgent (MCQ) achieves 60.9% VideoQA versus GPT-4o's 57.5% over approximately 4,600 video questions, while individual tasks have only 200-1,000 items. I request bootstrap confidence intervals for all cells and per-task paired tests (e.g., McNemar) for SoccerAgent versus the strongest baseline. This is particularly important because the paper makes comparative claims such as 'leading results across TextQA, ImageQA, and VideoQA' that require more than point estimates.
[Section 5.2, Table 2 (per-task rows)] The paper's superiority claim is stronger than the per-task data support. In Table 2, SoccerAgent (MCQ) is below GPT-4o on Q5 (85.7 vs. 89.6), Q8 (35.7 vs. 40.0), and Q13 (55.5 vs. 59.7), and below Gemini 2.0 Flash on Q12 (46.0 vs. 52.4). The claim 'leading results across TextQA, ImageQA, and VideoQA' is accurate only at the aggregate-category level, and the abstract's 'superiority' should be qualified accordingly. The authors should either report the specific tasks where the agent wins and loses or soften the claim to 'highest aggregate accuracy.'
[Section 4.2 / Appendix C.1 / Table 1] The evaluation may favor the in-house pipeline because the agent's tools and the benchmark draw on the same research-group datasets: UniSoccer tools are used for Q9/Q10, and SoccerReplay-1988 and MatchTime are both benchmark sources and agent tool sources. Appendix C.1 states that only test sets of the respective datasets are used to avoid leakage, but the paper does not demonstrate that the tool models have no access to those test splits or that the benchmark templates do not directly echo the tool outputs. I request a concrete contamination check: report the train/test split provenance for every source dataset used by the tools, and measure how SoccerAgent performs when the same-group datasets are excluded from the benchmark or when the involved tools are replaced by external ones. This is a correctness-risk concern, not an allegation of misconduct.

minor comments (7)

[Appendix Table 4] In the task-coverage table, the SoccerNet-v2 row appears to mark 'Jersey Number' as covered, but jersey-number annotations are introduced by SoccerNet-JN; please verify this entry against the cited dataset.
[References and Appendix C.1] The MatchTime and SoccerReplay-1988 references are used inconsistently: Appendix C.1.2 cites MatchTime as [43] and SoccerReplay-1988 as [44], while the main text and Table 1 use [43] for SoccerReplay-1988 and [44] for MatchTime. Please align all citations.
[Appendix D.1] The task-description prompt lists 14 tasks, including 'Task3: Match Events and Statistical QA,' but Table 1 defines only 13 benchmark tasks. Please reconcile the numbering or explain the extra task.
[Table 2 and Section 5.1.3] Q10 is described as commentary generation, yet it is scored as multiple-choice accuracy in Table 2. Please clarify what the four options are for this task and how template-generated commentary options are selected, since the underlying annotations are free-form captions.
[Appendix B.2] The cost statement 'each inference costs within $0.1' should specify whether this is per question, per tool call, or per full SoccerAgent trajectory, and whether it includes the local GPU cost or only the DeepSeek API fee.
[Figure 4] There is a small typo in the figure: the final step result is written as '</Anwer>' instead of '</Answer>'.
[General] The paper does not state where the dataset, code, and SoccerWiki will be released, despite the 'open-source' emphasis for the tools. Please add a public-release plan or an explicit statement of availability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: benchmark and agent share a data family, but held-out test splits and frozen pre-trained tools keep the evaluation independent.

full rationale

I walked the claimed derivation chain (SoccerWiki -> SoccerBench -> SoccerAgent -> robust performance) and found no step where a result is equivalent to its inputs by construction. The benchmark labels are generated from SoccerWiki and from existing datasets (SoccerReplay-1988, MatchTime, SoccerNet-v2, SoccerNet-XFoul, SoccerNet-Caption, SoccerNet-JN) via templates and DeepSeek-v3 prompting, then manually filtered (Sec. 3.3 and App. C.1-C.3). SoccerAgent's retrieval tools access SoccerWiki and its specialist tools include Action Classifier and Commentary Generation adopted from UniSoccer, the same authors' prior work (Sec. 4.2.1). This is a real overlap in data family, but it is not circular: Appendix C.1 states that 'for both textual commentary and event labels, we utilize only the test sets from the respective datasets to avoid potential data leakage to the trained models used later,' so the benchmark labels are not the training values of the tools. The agent's tool weights are pre-trained and frozen; no parameter is fitted to SoccerBench labels. The default SoccerAgent configuration uses only task descriptions and no execution examples (Table 3, Sec. 5.3), so the reported accuracy is not a renamed fit. The central comparisons are external and falsifiable: representative MLLMs are scored on the same held-out multiple-choice QA pairs (Table 2), and the benchmark is anchored to externally published SoccerNet datasets. The skeptic's concern about LLM-generated label quality is a validity/audit risk, not a circularity step, because even noisy external labels would not make the derivation reduce to itself. The self-citations to prior datasets/models are normal citation of published, reproducible work and are not load-bearing as an argumentative premise. No equation or metric in the paper is defined in terms of the outcome it is used to predict.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

SoccerWiki, SoccerBench, and SoccerAgent are constructed resources, not explanatory postulates in the physics sense, so they are not listed as invented entities. Their independent validity depends on public release and external evaluation, which are currently missing.

assumptions (5)

domain assumption SoccerWiki content scraped from Wikipedia and Flashscore is accurate, current, and sufficiently complete for the 13 benchmark tasks.
SoccerWiki is the ground truth for knowledge-based QA tasks Q1, Q2, Q4, Q11, Q12 and a primary retrieval source for SoccerAgent; if it contains stale or wrong facts, benchmark labels and agent answers are both wrong. See Sections 3.2 and 4.2.
domain assumption The annotations reused from SoccerNet-v2, SoccerNet-XFoul, SoccerNet-Caption, SoccerNet-JN, MatchTime, and SoccerReplay-1988 are correct and complete enough to serve as ground truth for the perception tasks.
Tasks Q3, Q5-Q10, Q13 are built from these external labels without re-adjudication, so label noise transfers directly to SoccerBench. See Table 1 and Section 3.3.
ad hoc to paper DeepSeek-v3 generated open-ended QA pairs and distractors are plausible and correct after the authors' manual filtering.
Around 100K QA pairs were synthesized by LLM and manually reduced to 10K; no inter-annotator agreement or error-rate statistics are reported, making QA quality an unmeasured assumption. See Section 3.3 and Appendix C.3.
domain assumption The DeepSeek-v3 agent core, given task descriptions and the recommended tool chains, plans and executes the correct tool sequence for the benchmark questions.
System performance depends on the planner choosing appropriate tools; the authors report planning failures below 5% with task descriptions (Appendix B.3) but do not provide the full planner success logs. See Appendix D.1 and D.3.
domain assumption The pretrained tools (UniSoccer action and commentary models, Qwen2.5-VL-7B based tools, face recognition, CLIP, GroundingDINO) generalize to the benchmark videos and images.
The appendix shows tool errors dominate failures, e.g., Entity Recognition's 63.4% player ID accuracy caps Q4 at 65.3% success; SoccerAgent inherits all tool limitations. See Appendix B.3.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multi-Agent System for Comprehensive Soccer Understanding." pith.science (2026). https://pith.science/paper/IWLW4T6T

@misc{pith2026250503735,
  author       = {Pith},
  title        = {Pith review of: Multi-Agent System for Comprehensive Soccer Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IWLW4T6T}},
  note         = {Machine review of arXiv:2505.03735}
}

read the original abstract

Recent advances in soccer understanding have demonstrated rapid progress, yet existing research predominantly focuses on isolated or narrow tasks. To bridge this gap, we propose a comprehensive framework for holistic soccer understanding. Concretely, we make the following contributions in this paper: (i) we construct SoccerWiki, the first large-scale multimodal soccer knowledge base, integrating rich domain knowledge about players, teams, referees, and venues to enable knowledge-driven reasoning; (ii) we present SoccerBench, the largest and most comprehensive soccer-specific benchmark, featuring around 10K multimodal (text, image, video) multi-choice QA pairs across 13 distinct tasks; (iii) we introduce SoccerAgent, a novel multi-agent system that decomposes complex soccer questions via collaborative reasoning, leveraging domain expertise from SoccerWiki and achieving robust performance; (iv) extensive evaluations and comparisons with representative MLLMs on SoccerBench highlight the superiority of our agentic system.

Figures

Figures reproduced from arXiv: 2505.03735 by the authors.

**Figure 2.** SoccerBench QA Generation Pipeline. We construct multi-choice QA samples based on SoccerWiki and other existing [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** SoccerAgent Architecture Overview. We design a multi-agent system to decompose and solve the given multi-modal [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Qualitative Results. Here, we demonstrate several representative examples showing the entire process of tool planning [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

93 extracted references · 58 canonical work pages

[1]

Anthropic. 2025. Claude 3.7 Sonnet. https://www.anthropic.com

2025
[2]

Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the International Conference on Computer Vision . 2425–2433

2015
[3]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. 2025. Qwen2.5-VL Technical Rep...

arXiv 2025
[4]

Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Pi- otr Dollár, and C Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325 (2015)

arXiv 2015
[5]

Anthony Cioppa, Adrien Deliege, Silvio Giancola, Bernard Ghanem, and Marc Van Droogenbroeck. 2022. Scaling up SoccerNet with multi-view spatial localiza- tion and re-identi/f_ication.Scienti/f_ic Data9, 1 (2022), 355

2022
[6]

Anthony Cioppa, Silvio Giancola, Vladimir Somers, Victor Joos, Floriane Magera, Jan Held, Seyed Abolfazl Ghasemzadeh, Xin Zhou, Karolina Seweryn, Mateusz Kowalczyk, et al . 2024. SoccerNet 2024 Challenges Results. arXiv preprint arXiv:2409.10587 (2024)

arXiv 2024
[7]

Adrien Deliege, Anthony Cioppa, Silvio Giancola, Meisam J Seikavandi, Jacob V Dueholm, Kamal Nasrollahi, Bernard Ghanem, Thomas B Moeslund, and Marc Van Droogenbroeck. 2021. Soccernet-v2: A dataset and benchmarks for holistic understanding of broadcast soccer videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop...

2021
[8]

Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Wang Siyuan, Zhongyu Wei, and Fei Huang. 2025. AI Hospital: Benchmarking Large Language Models in a Multi- agent Medical Interaction Simulator. In Proceedings of the International Conference on Computational Linguistics . 10183–10213

2025

Show all 93 references

[9]

Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al . 2023. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. arXiv preprint arXiv:2306.13394 (2023)

2023 arXiv
[10]

Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al . 2025. Video-mme: The /f_irst-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In Proceedings of the IEEE Conferen...

2025
[11]

Adam Geitgey. [n. d.]. Face Recognition. https://github.com/ageitgey/face_ recognition?tab=readme-ov-/f_ile
[12]

Alireza Ghafarollahi and Markus J Buehler. 2024. SciAgents: Automating Scien- ti/f_ic Discovery Through Bioinspired Multi-Agent Intelligent Graph Reasoning. Advanced Materials (2024), 2413523

2024
[13]

Silvio Giancola, Mohieddine Amine, Tarek Dghaily, and Bernard Ghanem. 2018. Soccernet: A scalable dataset for action spotting in soccer videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . 1711–1721

2018
[14]

Google. 2025. Gemini 2.0 Flash. https://developers.googleblog.com/en/ experiment-with-gemini-20-/f_lash-native-image-generation/

2025
[15]

Google. 2025. Gemini 2.5 Pro Experimental. https://ai.googleblog.com/2025/03/ gemini-25-pro-experimental.html

2025
[16]

Xiaofan Gu, Xinwei Xue, and Feng Wang. 2020. Fine-grained action recognition on a novel basketball dataset. In International Conference on Acoustics, Speech, and Signal Processing. 2563–2567

2020
[17]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

2025 arXiv
[18]

Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: a survey of progress and challenges. In Proceedings of the Interna- tional Joint Conference on Arti/f_icial Intell...

2024
[19]

Xudong Guo, Kaixuan Huang, Jiale Liu, Wenhui Fan, Natalia Vélez, Qingyun Wu, Huazheng Wang, Thomas L Griﬃths, and Mengdi Wang. 2024. Embodied llm agents learn to cooperate in organized teams. arXiv preprint arXiv:2403.12482 (2024)

2024 arXiv
[20]

Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, et al . 2024. Mmworld: Towards multi-discipline multi-faceted world model evaluation in videos. arXiv preprint arXiv:2406.08407 (2024)

2024 arXiv
[21]

Jan Held, Anthony Cioppa, Silvio Giancola, Abdullah Hamdi, Bernard Ghanem, and Marc Van Droogenbroeck. 2023. Vars: Video assistant referee system for automated soccer decision making from multiple views. In Proceedings of the IEEE Conference on Computer Vision and Pattern Reco...

2023
[22]

Jan Held, Hani Itani, Anthony Cioppa, Silvio Giancola, Bernard Ghanem, and Marc Van Droogenbroeck. 2024. X-vars: Introducing explainability in football refereeing with multi-modal large language models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog...

2024
[23]

Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al
[24]

Pedro Calciolari Jardim, Leonardo Mauro Pereira Moraes, and Cristina Dutra Aguiar. 2023. Qasports: A question answering dataset about sports. In Dataset Showcase Workshop. 1–12

2023
[25]

Maria Koshkina and James H. Elder. 2024. A General Framework for Jersey Number Recognition in Sports Video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . 3235–3244

2024
[26]

Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. 2024. Seed-bench: Benchmarking multimodal large language models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 13299–13308

2024
[27]

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2025. LLaV A- OneVision: Easy Visual Task Transfer. Transactions on Machine Learning Research (2025)

2025
[28]

Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society. In Advances in Neural Information Processing Systems

2023
[29]

Haopeng Li, Andong Deng, Qiuhong Ke, Jun Liu, Hossein Rahmani, Yulan Guo, Bernt Schiele, and Chen Chen. 2024. Sports-qa: A large-scale video question answering benchmark for complex and professional sports. arXiv preprint arXiv:2401.01505 (2024)

2024
[30]

Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: work/f_low, infrastructure, and challenges.Vicinagearth 1, 1 (2024), 9

2024
[31]

Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al . 2024. Videochat- /f_lash: Hierarchical compression for long-context video modeling.arXiv preprint arXiv:2501.00574 (2024)

2024 arXiv
[32]

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al . 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024)

2024 arXiv
[33]

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning. In Advances in Neural Information Processing Systems

2023
[34]

Qianying Liu, Sicong Jiang, Yizhong Wang, and Sujian Li. 2020. LiveQA: A question answering dataset over sports live. In Chinese Computational Linguistics: 19th China National Conference, CCL 2020, Hainan, China, October 30–November 1, 2020, Proceedings 19 . 316–328

2020
[35]

Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al . 2024. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In Proceedings of the European Conference on Computer Vision

2024
[36]

Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al . 2024. Mmbench: Is your multi-modal model an all-around player?. In Proceedings of the European Conference on Computer Vision . 216–233

2024
[37]

Fanqing Meng, Chuanhao Li, Jin Wang, Quanfeng Lu, Hao Tian, Tianshuo Yang, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, et al . 2025. MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models. In Proceedings of the International Conference on Learn...

2025
[38]

Hassan Mkhallati, Anthony Cioppa, Silvio Giancola, Bernard Ghanem, and Marc Van Droogenbroeck. 2023. SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . 5074–5085

2023
[39]

OpenAI. 2024. GPT-4o. https://openai.com

2024
[40]

Ji Qi, Jifan Yu, Teng Tu, Kunyu Gao, Yifan Xu, Xinyu Guan, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li, et al . 2023. GOAL: A challenging knowledge-grounded video captioning benchmark for real-time soccer commentary generation. In Proceedings of the ACM International Conference o...

2023
[41]

Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al . 2024. ChatDev: Communicative Agents for Software Development. In Association for Computational Linguistics . 15174–15186

2024
[42]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, 9 MM ’25, October 27–31, 2025, Dublin, Ireland Jiayuan Rao, Zifeng Li, Haoning Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie et al. 2...

2025
[43]

Jiayuan Rao, Haoning Wu, Hao Jiang, Ya Zhang, Yanfeng Wang, and Weidi Xie
[44]

Jiayuan Rao, Haoning Wu, Chang Liu, Yanfeng Wang, and Weidi Xie. 2024. MatchTime: Towards Automatic Soccer Game Commentary Generation. In Pro- ceedings of the Conference on Empirical Methods in Natural Language Processing

2024
[45]

Dian Shao, Yue Zhao, Bo Dai, and Dahua Lin. 2020. Finegym: A hierarchical video dataset for /f_ine-grained action understanding. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2616–2625

2020
[46]

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Re/f_lexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems , Vol. 36. 8634–8652

2023
[47]

Vladimir Somers, Victor Joos, Anthony Cioppa, Silvio Giancola, Seyed Abolfazl Ghasemzadeh, Floriane Magera, Baptiste Standaert, Amir M Mansourian, Xin Zhou, Shohreh Kasaei, et al . 2024. SoccerNet game state reconstruction: End-to- end athlete tracking and identi/f_ication on ...

2024
[48]

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adri Garriga- Alonso, et al. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions ...

2023
[49]

Sinan Tan, Weilai Xiang, Huaping Liu, Di Guo, and Fuchun Sun. 2020. Multi-agent embodied question answering in interactive environments. In Proceedings of the European Conference on Computer Vision . 663–678

2020
[50]

Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023)

2023 arXiv
[51]

Graham Thomas, Rikke Gade, Thomas B Moeslund, Peter Carr, and Adrian Hilton. 2017. Computer vision for sports: Current applications and research topics. Computer Vision and Image Understanding 159 (2017), 3–18

2017
[52]

Jize Wang, Ma Zerun, Yining Li, Songyang Zhang, Cailian Chen, Kai Chen, and Xinyi Le. 2024. GTA: a benchmark for general tool agents. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track

2024
[53]

Zhe Wang, Petar Veličković, Daniel Hennes, Nenad Tomašev, Laurel Prince, Michael Kaisers, Yoram Bachrach, Romuald Elie, Li Kevin Wenliang, Federico Piccinini, et al. 2024. TacticAI: an AI assistant for football tactics. Nature Com- munications 15, 1 (2024), 1–13

2024
[54]

Di Wu, Xian Wei, Guang Chen, Hao Shen, Xiangfeng Wang, Wenhao Li, and Bo Jin. 2025. Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review. arXiv preprint arXiv:2502.11518 (2025)

2025 arXiv
[55]

Dekun Wu, He Zhao, Xingce Bao, and Richard P Wildes. 2022. Sports video anal- ysis on large-scale data. In Proceedings of the European Conference on Computer Vision

2022
[56]

Haoning Wu, Xiao Huang, Yaohui Chen, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2025. SpatialScore: Towards Uni/f_ied Evaluation for Multimodal Spatial Understanding. arXiv preprint arXiv:2505.17012 (2025)

2025 arXiv
[57]

Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al . 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation. In First Conference on Language Modeling

2023
[58]

Haotian Xia, Zhengbang Yang, Yuqing Wang, Rhys Tracy, Yun Zhao, Dongdong Huang, Zezhi Chen, Yan Zhu, Yuan-fang Wang, and Weining Shen. 2024. Sportqa: A benchmark for sports understanding in large language models. In Proceedings of the Conference of the North American Chapter o...

2024
[59]

Haotian Xia, Zhengbang Yang, Junbo Zou, Rhys Tracy, Yuqing Wang, Chi Lu, Christopher Lai, Yanjun He, Xun Shao, Zhuoqing Xie, et al. 2025. SPORTU: A Com- prehensive Sports Understanding Benchmark for Multimodal Large Language Models. In Proceedings of the International Conferen...

2025
[60]

Jinglin Xu, Yongming Rao, Xumin Yu, Guangyi Chen, Jie Zhou, and Jiwen Lu
[61]

Yuchen Yang, Wei Wang, Yifei Liu, Linfeng Dong, Hao Wu, Mingxin Zhang, Zhihang Zhong, and Xiao Sun. 2025. SGA-INTERACT: A 3D Skeleton-based Benchmark for Group Activity Understanding in Modern Basketball Tactic. arXiv preprint arXiv:2503.06522 (2025)

2025 arXiv
[62]

Shunyu Yao, Jeﬀrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In Proceedings of the International Conference on Learning Representations

2023
[63]

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al . 2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE Conference o...

2024
[64]

Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al . 2024. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813 (2024)

2024 arXiv
[65]

Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al . 2025. VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding. arXiv preprint arXiv:2501.13106 (2025)

2025 arXiv
[66]

Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chun- yuan Li. 2024. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713 (2024)

2024 arXiv
[67]

What is the camera position in this picture?

Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. 2024. Mlvu: A comprehensive bench- mark for multi-task long video understanding. arXiv preprint arXiv:2406.04264 (2024). 10 Multi-Agent System for Comprehensive...

2024 arXiv
[71]

Maintain the question’s core meaning while adapting it for multiple-choice format
[72]

Generate four options where: - The correct option MUST exactly match the reference answer - Distractors should: * Be thematically relevant and plausible * Avoid obvious errors (e.g., incorrect club names) * Include common misconceptions or easily confused elements * Maintain c...
[73]

Randomize option order (correct answer not /f_ixed to any position, you must randomize the order of the options)
[74]

Use this exact format: Q: [Question text] A: [Correct option letter] O1: [Option 1] O2: [Option 2] O3: [Option 3] O4: [Option 4] Example 1: Q: Which player holds the record for most goals in a single Champions League season? A: O3 O1: Cristiano Ronaldo O2: Lionel Messi O3: Rob...

1930
[75]

Problem Decomposition (Part 1) - Identify available information - Break down the question into sequential steps
[76]

Sequential Tool Application (Part 2) - Execute one tool at a time - Record each tool’s output - Continue until suﬃcient information is gathered
[77]

How does the viewpoint of the camera shift in the video?

Solution Synthesis (Part 3) - Integrate all results - Generate /f_inal answer ## Available Tools For all the QA, you need to decompose them and Here are the tools that you can use to answer the questions: 18 Multi-Agent System for Comprehensive Soccer Understanding MM ’25, Oct...

2025
[78]

You should only use the tools provided in the toolbox to answer the questions and provide the exact tool names
[79]

Use exact item category names with $$ to represent the information categories
[80]

Use exact tool category names with ** as shown above to represent the tools
[81]

Only respond with Part 1 analysis - Parts 2 & 3 will be addressed in subsequent interactions
[82]

Connect tools using -> symbol
[83]

If the template /f_its the question, you can directly use the recommended tool chain

Try your best to decompose the question and identify the required tools, you can /f_irst reference the common QA tasks to get some ideas. If the template /f_its the question, you can directly use the recommended tool chain. If not, you can try to decompose the question and ide...

2025
[84]

This part is important for me to understand the context of the execution

You should be clear about the tool name (must be chosen from toolbox), /f_ile path and query/question in the instruction. This part is important for me to understand the context of the execution. You cannot change any of the information in the instruction
[85]

You should not repeat the same instruction again

If I have given you the feedback of the execution, you should analyze what you should write in the next call based on the feedback considering the tool chain I gave you and the task descriptions and tool descriptions. You should not repeat the same instruction again
[86]

You should not add any other information in the instruction

If my prompt leaves you to generate the /f_irst call, you should directly return me with the call in the form from <> to </>. You should not add any other information in the instruction
[87]

{question}

Otherwise, if in the prompt I have given you some <StepResult>, you should consider the total process of the execution and continue to return me exactly with the form from <> to </>. You should not add any other information in the instruction. Once again, I repreat that the qu...

2025
[88]

If **any of the team’s name in original query information** is sure not to be in team names from candidates, such candidate cannot be returned anymore, you cannot let such candidate take place in your return answer
[89]

Chelsea" and

For example, if the original query information contains "Chelsea" and "West Ham", but candidates contains "chelsea FC" and "Liverpool", since such candidate cannot be returned anymore since West Ham is not in candidate information
[90]

Chelsea" and

For example, if the original query information contains "Chelsea" and "West Ham", but candidates contains "Chelsea FC" and "West Ham United", since such candidate is still possible to be returned since both team names are in candidate information
[91]

Chelsea", but candidates contains

For example, if the original query information contains only "Chelsea", but candidates contains "Bayern Munich" and "Real Madrid", since such candidate cannot be returned since Chelsea is not in candidate information. After considering the above situation and exclude those can...
[92]

[Here give some recommendation to complete the information if possible, for example, provide the date or the score of the match, or which team is the home/away team etc

If there are still **obviously** probable answer with all known information correct, please return the /f_ile path of that match EXACTLY in the following format: "The given information seems incomplete, but we found the most probable match in the database with this /f_ile path...
[93]

How many goals did Lionel Messi score last season?

If no match is signi/f_icantly more likely among all the candidates, please return all candidate matches with information of league, season, date, time, score, home team, away team, venue and referee (without /f_ile path), and explain that the information provided is too vague...

2025
[2022]

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

Finediving: A /f_ine-grained dataset for procedure-aware action quality assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2949–2958
[2024]

In Proceedings of the International Conference on Learning Representations

MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In Proceedings of the International Conference on Learning Representations
[2025]

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

Towards Universal Soccer Video Understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Anthropic. 2025. Claude 3.7 Sonnet. https://www.anthropic.com

2025

[2] [2]

Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the International Conference on Computer Vision . 2425–2433

2015

[3] [3]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. 2025. Qwen2.5-VL Technical Rep...

arXiv 2025

[4] [4]

Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Pi- otr Dollár, and C Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325 (2015)

arXiv 2015

[5] [5]

Anthony Cioppa, Adrien Deliege, Silvio Giancola, Bernard Ghanem, and Marc Van Droogenbroeck. 2022. Scaling up SoccerNet with multi-view spatial localiza- tion and re-identi/f_ication.Scienti/f_ic Data9, 1 (2022), 355

2022

[6] [6]

Anthony Cioppa, Silvio Giancola, Vladimir Somers, Victor Joos, Floriane Magera, Jan Held, Seyed Abolfazl Ghasemzadeh, Xin Zhou, Karolina Seweryn, Mateusz Kowalczyk, et al . 2024. SoccerNet 2024 Challenges Results. arXiv preprint arXiv:2409.10587 (2024)

arXiv 2024

[7] [7]

Adrien Deliege, Anthony Cioppa, Silvio Giancola, Meisam J Seikavandi, Jacob V Dueholm, Kamal Nasrollahi, Bernard Ghanem, Thomas B Moeslund, and Marc Van Droogenbroeck. 2021. Soccernet-v2: A dataset and benchmarks for holistic understanding of broadcast soccer videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop...

2021

[8] [8]

Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Wang Siyuan, Zhongyu Wei, and Fei Huang. 2025. AI Hospital: Benchmarking Large Language Models in a Multi- agent Medical Interaction Simulator. In Proceedings of the International Conference on Computational Linguistics . 10183–10213

2025