Pith. sign in

REVIEW 3 major objections 3 minor 30 references

Four Eyes Are Better Than Two: Harnessing the Collaborative Potential of Large Models via Differentiated Thinking and Complementary Ensembles

T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free ensemble of seven large-model “thinking modes” answers long egocentric video questions at 79% accuracy, beating the prior 75% state of the art.

desk verdict The headline 79% is in-sample fitted because the ensemble weights are computed from ground-truth labels on the very test set used for scoring; the genuinely useful material is the validation-side comparison of prompts, CoT, and 1-stage vs 2-stage pipelines. read the letter →

arxiv 2505.16784 v2 pith:AAYEEE6V submitted 2025-05-22 cs.CV

Jun Xie , Xiongjun Guan , Yingjian Zhu , Zhaoran Zhao , Xinming Wang , Hongzhu Yi , Feng Chen , Zhepeng Wang This is my paper
classification cs.CV
keywords EgoSchemavideoquestionansweringvision-languagemodelspromptengineeringchain-of-thoughtmodelensembleegocentrictraining-freeadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that, for long-form video question answering, a single forward pass through a modern vision-language model with carefully chosen prompts can outperform multi-stage pipelines that first caption a video and then reason over the caption. The authors show that a one-stage design, in which the model outputs captions, summary, reasoning, answer, and confidence in one JSON response, reaches 75% or better on its own. Combining seven such 'modes'—different prompt styles, chain-of-thought structures, and models—through similarity-weighted voting raises accuracy to 79%, which placed second on the EgoSchema challenge leaderboard. The practical claim is that collaboration between large models can be harvested without any additional training or expensive multi-stage orchestration.

What carries the argument

The load-bearing mechanism is the complementary-ensemble voting rule. Each mode $k$ is assigned a weight $w_k$ equal to the fraction of samples on which it agrees with ground truth ($w = \sum_i \mathbb{I}(y_i = \hat{y}_i)/n$), and each pair of modes has a similarity $\mathrm{sim}(A,B)$ equal to their answer-overlap fraction. The ensemble score for option $c$ is $\sum_k w_k \sum_b \mathrm{sim}(a,b) \, \mathbb{I}(\hat{y}_k = c)$, and the answer with the highest score wins. The similarity term is meant to stop near-duplicate modes from dominating the vote. Equally important is the one-stage prompt/CoT construction: the model is asked to emit caption, summary, reason, answer, and confidence in a single JSON response, which the paper finds both simpler and more accurate than the earlier two-stage caption-then-reason pipeline.

What would settle it

Compute the seven mode outputs on the 5,000 test questions, hold out one half of the test set, estimate the ensemble weights and mode selection from that half only, and score the ensemble on the held-out half. If the held-out accuracy falls to or below the single-mode baseline (around 75%), the 79% figure is largely an artifact of test-set weighting. A weaker check: perturb the weight vector by small noise and see whether the final answer changes; if it does, the ensemble result is fragile.

Watch

Extended reading notes

Core claim

On the EgoSchema benchmark, the paper's central discovery is that the bottleneck is not video understanding capability but how the model's attention is directed. A 1-stage prompt that asks a vision-language model to produce a clip-by-clip caption, a global summary, a reason, an answer, and a confidence score in one JSON block outperforms the previous state-of-the-art 75% method, which used a separate captioning stage plus a reasoning stage. The paper then treats each prompt style and model as a distinct 'mode' of thinking and ensembles the most accurate, least redundant modes. With seven modes, ensemble weights are set to each mode's accuracy and modulated by pairwise answer similarity, and the final answer is the option receiving the highest similarity-weighted vote; this reaches 79% on the 5,000-question test set. In the paper's implementation, these weights and similarities are computed on the test set itself.

Load-bearing premise

The reported 79% rests on the assumption that choosing ensemble weights and modes by looking at the test set's correct answers, and then reporting accuracy on that same test set, gives an honest measure of performance; if that assumption is false, the number is not a generalizable result.

Editorial extensions

If this is right

  • A single strong vision-language model with a well-structured one-stage prompt can replace a multi-stage caption-then-reason system for long-form video QA, cutting cost and complexity.
  • Differentiated prompt styles and chain-of-thought output structures produce answers that are complementary enough that voting over them beats any single mode.
  • Because all model calls are parallel and the ensemble is a lightweight vote, the final solution remains effectively one stage and near-zero-cost at inference.
  • If the 79% transfers to other benchmarks, training-free prompt-and-ensemble recipes become a practical baseline that future video-QA methods must beat.
  • Careful ablation shows that long, example-heavy prompts can hurt performance, so prompt design should be treated as a first-class experimental variable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's design uses the test set's ground-truth labels to choose ensemble weights and to select the seven modes, then reports accuracy on that same test set; unless those choices generalize to a held-out split, 79% is an upper bound on what a deployment would see.
  • A direct testable extension is to estimate weights on a randomly chosen half of the test set and score the ensemble on the other half; the gap, if any, would quantify how much of the gain is genuine complementarity versus fitting.
  • The paper's 'confidence' output component does not appear to enter the ensemble rule (weights are answer-accuracy, not self-reported confidence); a natural variant is to blend self-confidence with empirical accuracy to reduce reliance on labels.
  • The same prompt-differentiation recipe could be applied to other multiple-choice video QA benchmarks where a few hundred labeled examples are available, but the reported advantage may shrink when the label budget is too small to estimate weights reliably.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper describes the runner-up solution for the Ego4D EgoSchema Challenge at CVPR 2025. It proposes a 1-stage prompting pipeline for video-language models, systematically compares prompt styles and chain-of-thought output structures, and combines seven complementary modes with a weighted ensemble. The headline claim is that this ensemble reaches 79% accuracy on the 5000-sample EgoSchema test set, improving on the previous SOTA method iLearn at 75%, and that even a single mode (75.9%) already surpasses that SOTA. The supplementary material provides ablation studies on a 500-sample validation set and detailed equations for the ensemble weights and mode similarity.

Significance. If the reported result were a valid out-of-sample estimate, this would be a useful practical demonstration that training-free, 1-stage prompting and ensembling of accessible VLMs can compete at the top of a long-form video QA leaderboard. The paper has concrete strengths: it makes code available, reports a systematic ablation of CoT components, and is transparent about the configuration choices and their evolution. However, the central 79% figure is computed through a circular evaluation procedure: the ensemble weights and mode selection are derived from ground-truth labels on the same test set that is used to report the final accuracy. This invalidates the main quantitative claim as an unbiased performance estimate, and it also undermines the secondary claim that a single mode beats the prior SOTA. The validation-set ablations are less affected by this issue, but they are not the paper's headline result.

major comments (3)
  1. [Section 6.5, Eq. (1), Table 3, Figure 6] The reported 79% accuracy is an in-sample fitted statistic. Eq. (1) defines each mode's weight w as the fraction of samples for which that mode matches the ground-truth label y_i, and Table 3 explicitly reports "Accuracy (%) of different modes in 5000 test dataset." Figure 6 computes the similarity matrix on the same 5000 test samples, and Section 6.5 says the seven modes are selected because they have "high accuracy and low similarity." The final weighted vote (Eq. (3)) then uses these test-derived weights and test-derived mode selection to report accuracy on that same test set. This is circular: the same y_i appear in the weight computation and in the final evaluation. The claim in Section 3 that the method "outperforms almost all previously proposed methods (75%→79%)" is therefore not supported as an unbiased, generalizable result. The authors should recompute weights and select modes using only the validation split, or clearly re-frame the reported numbers as post-hoc fits to the test set rather than predictive performance.
  2. [Abstract; Section 3; Table 1] The abstract's claim that "directly utilizing an individual multimodal model already outperforms the previous SOTA" rests on a 0.9-point gap (Mode 1 at 75.9% versus iLearn at 75.0% in Table 1/Table 3). No confidence intervals, significance tests, or repeated-run variance are reported, and Mode 1 was itself selected after inspecting its accuracy on the test set during the ensemble construction (Section 6.5). A 0.9-point difference on 5000 samples, with no uncertainty quantification, is not sufficient to support a claim of superiority, especially when the mode was chosen post hoc on the same test set.
  3. [Section 6.2; Section 6.5] The validation-set ablations are more credible because they are out-of-sample with respect to the 500-sample validation split, but the paper itself notes "a stark difference between the trends observed in the validation set and the test set, indicating a significant data bias." This means the validation set may not reliably select the best CoT configuration, and the later test-set-based selection of the seven modes (Section 6.5) is exactly the kind of multiple-comparison procedure that inflates apparent accuracy. The authors should provide separate validation-based and test-based estimates, and should avoid drawing conclusions about the best configuration from test-set numbers alone.
minor comments (3)
  1. [Section 2.2] There is a typo in "we develope a 1-stage process" which should read "we develop a 1-stage process."
  2. [Figure 2] Figure 2 is visually dense and the legend for the diagonal line and green dashed line is easy to miss; the caption should define all symbols directly in the figure caption.
  3. [Section 2.5] The text says "The similarity between any two modes A and A" but should say "A and B"; the equation is correct but the surrounding sentence is sloppy.

Circularity Check

1 steps flagged · score 6.0 of 10

The 79% headline is an in-sample fitted statistic: ensemble weights and mode selection are computed from ground-truth labels on the same test set used for scoring, so the reported accuracy is not a valid out-of-sample estimate.

  1. fitted input called prediction [Section 2.5, Eqs. (1)-(3); Section 6.5, Table 3 and Figure 6]
    "Let y_i and ŷ_i represent the ground truth and predicted result, respectively, the confidence level of a certain mode is calculated as w = Σ_i I(y_i = ŷ_i)/n (1) ... A total of 7 sets of results are adopted, which have high accuracy and low similarity. Table 3 shows the accuracy (weights) of different modes calculated from Equation 1."

    The weight of each mode is defined as its accuracy on the same 5000 test samples whose ground-truth labels y_i are then used to compute the reported final accuracy. Equation 3 votes with these test-derived weights, and the selection of the seven modes (high accuracy, low similarity) is also based on Table 3 and Figure 6, both computed on the 5000 test set. The reported 79% is therefore an in-sample fitted statistic: the voting weights and voter set were chosen using the exact labels that define the headline number. The ensemble 'prediction' is not an unbiased out-of-sample estimate; it is constructed from the target labels by Eq. 1.

full rationale

The central numerical claim (79%, rank 2) is not a clean out-of-sample evaluation. By the paper's own Eq. 1, each ensemble weight is the per-mode accuracy on the 5000-sample test set; Table 3 is literally titled 'Accuracy (%) of different modes in 5000 test dataset,' and the same test-set predictions are used in the similarity matrix of Figure 6 to choose which modes to ensemble. Eq. 3 then applies these test-fitted weights to produce the final answer, and the paper reports the accuracy of that same test set as the headline result. This is a textbook in-sample fit: the free parameters of the ensemble (weights and mode membership) are selected on the target labels, so the reported accuracy is biased upward and cannot be read as a prediction of performance on unseen data. The validation-set ablations (Table 2) are out-of-sample and more credible, but they do not support the 79% headline. A secondary concern is that the claim that a single VLM already beats prior SOTA (75.9% vs 75.0%) relies on picking the best of several test-evaluated modes, a milder post-hoc selection effect rather than an equation-level circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several free parameters and domain assumptions. Most critically, the ensemble weights are fitted to the test-set ground truth, and the reported accuracy is evaluated on the same test set. The fixed 4-second caption granularity and the validation-based prompt choices are additional unverified hyperparameters. No new entities are introduced.

free parameters (4)
  • Ensemble weights w_k = 75.9, 73.7, 75.2, 74.0, 73.0, 74.4, 73.7 (percent, per mode)
    Computed from test-set ground-truth labels via Eq. (1) and used to weight votes in Eq. (3).
  • Seven ensembled modes = modes 1-7 in Table 3
    Selected based on high test-set accuracy and low pairwise similarity (Figure 6), a post-hoc selection on the evaluation set.
  • Prompt style and CoT component choices = P1/P2/P3; CAPTION/SUMMARY/REASON/ANSWER/CONFIDENCE inclusions
    Hand-chosen and validated on the 500-sample validation set; not fitted to test but still manual hyperparameters.
  • Caption segment duration (4s) = 4 seconds
    Fixed choice for generating video captions in Section 2.4; no ablation on segment length reported.
assumptions (5)
  • domain assumption Commercial VLM outputs are stable and reproducible across API calls.
    The method relies on Gemini, DeepSeek, and GPT-4o outputs without version pinning or seeds; prompt engineering results may shift as APIs update.
  • domain assumption EgoSchema test-set ground-truth labels used to compute weights are correct.
    Eq. (1) uses y_i (ground truth) to compute mode weights; if labels contain errors the weights and final accuracy are affected.
  • domain assumption Four-second caption intervals capture the information needed to answer the questions.
    The CoT construction uses 4s captions as fixed granularity (Section 2.4); no sensitivity analysis for segment length is provided.
  • domain assumption The 500-sample validation set is representative for selecting prompt styles.
    Prompt choices are made on 500 validation samples while the paper reports a 'significant data bias' between validation and test (Section 6.2), so the selection may not generalize.
  • domain assumption The leaderboard accuracy is the official, comparable metric across teams.
    Rank and accuracy comparisons assume identical evaluation protocol across competition entries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Four Eyes Are Better Than Two: Harnessing the Collaborative Potential of Large Models via Differentiated Thinking and Complementary Ensembles." pith.science (2026). https://pith.science/paper/AAYEEE6V

@misc{pith2026250516784,
  author       = {Pith},
  title        = {Pith review of: Four Eyes Are Better Than Two: Harnessing the Collaborative Potential of Large Models via Differentiated Thinking and Complementary Ensembles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AAYEEE6V}},
  note         = {Machine review of arXiv:2505.16784}
}
read the original abstract

In this paper, we present the runner-up solution for the Ego4D EgoSchema Challenge at CVPR 2025 (Confirmed on May 20, 2025). Inspired by the success of large models, we evaluate and leverage leading accessible multimodal large models and adapt them to video understanding tasks via few-shot learning and model ensemble strategies. Specifically, diversified prompt styles and process paradigms are systematically explored and evaluated to effectively guide the attention of large models, fully unleashing their powerful generalization and adaptability abilities. Experimental results demonstrate that, with our carefully designed approach, directly utilizing an individual multimodal model already outperforms the previous state-of-the-art (SOTA) method which includes several additional processes. Besides, an additional stage is further introduced that facilitates the cooperation and ensemble of periodic results, which achieves impressive performance improvements. We hope this work serves as a valuable reference for the practical application of large models and inspires future research in the field. Our Code is available at https://github.com/XiongjunGuan/EgoSchema-CVPR25.

Figures

Figures reproduced from arXiv: 2505.16784 by the authors.

Figure 1
Figure 1. The framework of our proposed algorithm. The specific text content is omitted in the flowchart and reported in the main text. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The comprehensive roadmap of our technological evo [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Example of prompt words in Style P1. 1 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Example of prompt words in Style P2. You are a visual question answering expert. You can choose the correct answer from five options based on the video content. Watch the video carefully and answer the following question by selecting the most appropriate option. [QUEST…
Figure 5
Figure 5. Figure 5: Example of prompt words in Style P3. gests that the structure and content of the output terms can heavily influence the inference performance of VLMs. This finding underscores the importance of carefully designing the reasoning process and output format in tasks involv…
Figure 6
Figure 6. Figure 6: Similarity matrix (%) of results from different modes in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 19 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 3 70 7874 2-stage [g, r1, r1] 2-stage [g, r1, v3] 2-stage [L, r1, v3] 1-stage [g2.0] prompt [-> p1] prompt [-> p2] COT[cap,rs,ans,conf] COT[sum,ans] COT[rs, ans,conf] 2-stage [+ qa focus] 2-...

    arXiv

  2. [2]

    Mem- ory consolidation enables long-context video understanding

    Ivana Bala ˇzevi´c, Yuge Shi, Pinelopi Papalampidi, Rahma Chaabouni, Skanda Koppula, and Olivier J H ´enaff. Mem- ory consolidation enables long-context video understanding. arXiv preprint arXiv:2402.05861, 2024. 1

    arXiv 2024

  3. [3]

    Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning.Advances in Neural Infor- mation Processing Systems, 37:110805–110853, 2024

    Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Kai Wang, Yuxi- ang Lin, Zheng Lian, Xiaojiang Peng, and Alexander Haupt- mann. Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning.Advances in Neural Infor- mation Processing Systems, 37:110805–110853, 2024. 1

    work page 2024

  4. [4]

    Zero-shot video question answering with pro- cedural programs.arXiv preprint arXiv:2312.00937, 2023

    Rohan Choudhury, Koichiro Niinuma, Kris M Kitani, and L´aszl´o A Jeni. Zero-shot video question answering with pro- cedural programs.arXiv preprint arXiv:2312.00937, 2023. 3

    arXiv 2023

  5. [5]

    Second joint egocentric vision (EgoVis) workshop,

    egovis. Second joint egocentric vision (EgoVis) workshop,

    work page

  6. [6]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 18995–19012, 2022. 1

    2022

  7. [7]

    Parameter-efficient transfer learning for nlp

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. InInternational conference on machine learning, pages 2790–2799. PMLR, 2019. 1

    work page 2019

  8. [8]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022. 1

    2022

Show all 30 references
  1. [9]

    Egoschema: A diagnostic benchmark for very long- form video language understanding.Advances in Neural In- formation Processing Systems, 36:46212–46244, 2023

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic benchmark for very long- form video language understanding.Advances in Neural In- formation Processing Systems, 36:46212–46244, 2023. 1

    2023

  2. [10]

    A simple recipe for contrastively pre-training video-first en- coders beyond 16 frames

    Pinelopi Papalampidi, Skanda Koppula, Shreya Pathak, Justin Chiu, Joe Heyward, Viorica Patraucean, Jiajun Shen, Antoine Miech, Andrew Zisserman, and Aida Nematzdeh. A simple recipe for contrastively pre-training video-first en- coders beyond 16 frames. InProceedings of the IEE...

    2024

  3. [11]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text.arXiv preprint arXiv:2403.05530, 2024

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text.arXiv preprint arXiv:2403.05530, 2024. 1, 3

    2024 arXiv

  4. [12]

    Videoagent: Long-form video understand- ing with large language model as agent.arXiv preprint arXiv:2403.10517, 2024

    Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understand- ing with large language model as agent.arXiv preprint arXiv:2403.10517, 2024. 3

    2024 arXiv

  5. [13]

    Lifelongmem- ory: Leveraging llms for answering queries in long-form egocentric videos.arXiv preprint arXiv:2312.05269, 2023

    Ying Wang, Yanlai Yang, and Mengye Ren. Lifelongmem- ory: Leveraging llms for answering queries in long-form egocentric videos.arXiv preprint arXiv:2312.05269, 2023. 1, 2, 3

    2023 arXiv

  6. [14]

    Internvideo2: Scaling foundation models for mul- timodal video understanding

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al. Internvideo2: Scaling foundation models for mul- timodal video understanding. InEuropean Conference on Computer Vision, pages 396–416. Springer, 2024. 1, 3

    2024

  7. [15]

    mplug-owl: Modularization empowers large language models with multimodality.arXiv preprint arXiv:2304.14178, 2023

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality.arXiv preprint arXiv:2304.14178, 2023. 3

    2023 arXiv

  8. [16]

    A sim- ple llm framework for long-range video question-answering

    Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. A sim- ple llm framework for long-range video question-answering. arXiv preprint arXiv:2312.17235, 2023. 1, 3

    2023 arXiv

  9. [17]

    HCQA@Ego4D egoschema challenge 2024.arXiv preprint arXiv:2406.15771, 2024

    Haoyu Zhang, Yuquan Xie, Yisen Feng, Zaijing Li, Meng Liu, and Liqiang Nie. HCQA@Ego4D egoschema challenge 2024.arXiv preprint arXiv:2406.15771, 2024. 1, 2, 3, 4

    2024 arXiv

  10. [18]

    Learning video representations from large lan- guage models

    Yue Zhao, Ishan Misra, Philipp Kr ¨ahenb¨uhl, and Rohit Girdhar. Learning video representations from large lan- guage models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6586– 6597, 2023. 2, 1 4 Four Eyes Are Better Than Two: Harn...

    2023

  11. [20]

    Among them, the underline indicates the content that needs to be filled based on the sample

    Prompt Word The example of prompt words in three style (introduced in section 2.3) are show in Figure 3, 4 and 5. Among them, the underline indicates the content that needs to be filled based on the sample. The italics indicate the corresponding guiding content, which can prom...

  12. [21]

    CAPTION":

    Ablation Study 6.1. Macro Design Our technological evolution starts with the SOTA solution, which is the 2-stage framework of iLearn [17]. The selec- tion of VLMs and the data flow paradigm have been thor- oughly evaluated. In the top three rows, Gemini 2.0 flash (g) and LaViL...

  13. [22]

    Select the option that best suits the question and video content

    Based on the provided video content and the question below, evaluate the plausibility of each option step-by-step. Select the option that best suits the question and video content. Please detail your reasoning process and results

  14. [23]

    THINK": [your chain of thoughts]

    You must output in the JSON format. Output the reasoning process, results, and confidence in sequence. [FORMAT] ```json {{ "THINK": [your chain of thoughts] "ANSWER": [option number], "CONFIDEN CE": [confidence level from 1-5] }}``` [EXAMPLE] [QUESTION] (...) [OPTION] (...) [O...

  15. [24]

    Please first analyze the video and generate 4-second text descriptions in "CAPTION" to help answer the question

  16. [25]

    Please provide a further summary of the entire video content in the "SUMMARY" section

  17. [26]

    Descriptions must explicitly state the subject

  18. [27]

    The #C indicates the image seen from your point of view, and the #O indicates the other people in the image you seen

    Subject Pronouns: Use C to represent the primary agent. The #C indicates the image seen from your point of view, and the #O indicates the other people in the image you seen

  19. [28]

    Focus descriptions on elements relevant to answering potential questions

  20. [29]

    REASON". Please provide the results and confidence level in

    Based on the description, summary, questions, and options of the video and clips, provide the analysis process and results. Please think step by step and provide a brief reason in "REASON". Please provide the results and confidence level in "ANSWER" and "CONFIDENCE"

  21. [30]

    CAPTION": {{

    YOU MUST output in the JSON format. Output the caption, summary, reasoning process, results, and confidence in sequence. [FORMAT] ```json {{ "CAPTION": {{ "0:00-0:04": "#C C drops a card on the table with her left", ..., "2:08-2:12": "#O A man picks a card from the stack of ca...

  22. [2025]

    Accessed: May 21, 2025. 1

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.