Pith. sign in

REVIEW 3 major objections 5 minor 30 references

MANTA: Cross-Modal Semantic Alignment and Information-Theoretic Optimization for Long-form Multimodal Understanding

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MANTA claims that long-form video reasoning reduces to selecting query-relevant text under a token budget with a provable near-optimality ratio.

desk verdict A plausible RAG-for-video pipeline wrapped in an invalid optimality theorem, with unverifiable benchmark claims. read the letter →

arxiv 2507.00068 v1 pith:X6TGSR66 submitted 2025-06-28 cs.CV cs.AIcs.CL

Ziqi Zhong , Daniel Tang This is my paper
classification cs.CVcs.AIcs.CL
keywords long-formvideoquestionansweringcross-modalalignmentinformation-theoreticcontentselectiontoken-budgetoptimizationmulti-scaletemporalmodelingretrieval-augmentedgenerationmultimodallargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MANTA tries to show that long-form video question answering can be handled by translating both video frames and audio into text segments, scoring each segment by how much information it carries about the query, and selecting a budgeted set of segments to feed an LLM. The paper's central theoretical claim is that this selection is provably near-optimal: under stated assumptions, the density-score greedy choice achieves at least $1 - (\epsilon + \delta + \gamma)$ of the information that the best possible budgeted selection would achieve. Empirically, the paper reports that attaching MANTA to strong video-QA models raises accuracy by up to 22.6 percentage points on Video-MME, with the largest gains on videos over 30 minutes and on temporal and cross-modal reasoning. A careful reader should care because the method offers a transferable design principle: choose context by information density under a token cap, a recipe that would apply to any long multimodal document.

What carries the argument

The load-bearing object is the per-segment information density score $$D($s_i^{{(l)}}$) = -\log p($s_i^{{(l)}}$|s_{<i}^{(l)}) + \$\alpha$ H($s_i^{{(l)}}$) + \$\beta$ I($c_i^{{(l)}}$; $t_i^{{(l)}}$) - \gamma R($s_i^{{(l)}}$),$$ whose terms respectively penalize redundancy, reward internal richness, reward visual-audio coherence, and penalize overlap with already-selected content. The proof carries the argument by casting budgeted segment selection as a knapsack over these scores: with near-independence of segment information, value density $D/|s|$ becomes the right greedy key, and the approximation ratio follows from the standard greedy knapsack bound plus the assumed closeness of $D$ to $I(s;Q)$. A second mechanism is the hierarchical three-scale segmentation (micro 1-3 seconds, meso 10-30 seconds, macro 1-5 minutes) that feeds the scorer, and a contrastive InfoNCE-style alignment loss that makes visual and auditory representations comparable before they are fused.

What would settle it

Take a labeled set of video-QA pairs, compute $D(s_i)$ for every segment, estimate $I(s_i;Q)$ with a calibrated mutual-information estimator or a strong proxy such as an LLM's pointwise relevance judgments, and compare the two; if the mean absolute error exceeds $\gamma$ on a meaningful fraction of segments, the claimed $1-(\epsilon+\delta+\gamma)$ guarantee cannot hold for that setting.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is a reduction: the problem of building an LLM prompt from a long video, under a token budget, can be treated as a knapsack problem in which each candidate text segment has value equal to its mutual information with the question and weight equal to its token length. MANTA's Theorem 2 asserts that a greedy selection by the density score $D(s_i)$ from Eq. (5) is within a factor $1 - (\epsilon + \delta + \gamma)$ of the optimal query-relevant segment set, provided segment contributions are nearly independent, length and information are nearly uncorrelated, and $D$ approximates the query mutual information within $\gamma$. The paper pairs this with a contrastive alignment objective that pulls visual captions and audio transcripts into one semantic space, and reports that the full pipeline lifts Video-MME accuracy from 78.9% to 98.2% on its strongest baseline and improves temporal-ordering, causal, and cross-modal reasoning by roughly 23 to 27 percentage points.

Load-bearing premise

The guarantee depends on assumption (iii) of Theorem 2: the hand-designed density score $D(s_i)$ must be within $\gamma$ of the true query mutual information $I(s_i; Q)$, and the paper does not measure that gap directly, so if $D$ is a poor proxy the near-optimality claim is not established.

Editorial extensions

If this is right

  • If Theorem 2 holds, any question-answering system with a token budget can use MANTA's density score as a drop-in selection rule and inherit a concrete optimality guarantee rather than a heuristic ranking.
  • The reported Video-MME numbers imply that the method is most valuable exactly where long-form systems fail: videos longer than 30 minutes gain 27.3% in accuracy, and rare-event detection improves 26.2%.
  • The ablations imply that multi-scale temporal modeling and hierarchical fusion carry most of the benefit, so coarse single-scale captioning would not achieve the same gains.
  • Because the framework is modality-agnostic once content is textualized, the same selection and alignment recipe applies to other paired streams, such as images plus OCR, or sensor logs plus natural language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Left implicit is a testable stronger claim: because the proof only needs a token budget and a query-score proxy, MANTA should transfer to long documents with embedded figures, podcasts, or surveillance feeds by simply swapping the visual and audio captioners; a reader could probe this by running the same scorer on an audio-only or document-only dataset.
  • The optimality guarantee is stated for a fixed query, so in settings where many queries share one prompt or no query is available at indexing time, the approximation argument does not directly apply and the paper's success there would be entirely empirical.
  • An editor would want a direct check of assumption (iii): estimating $I(s_i;Q)$ on a labeled subset and comparing it to $D(s_i)$ would either confirm the proxy or bound how far the reported approximation ratio can be trusted.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MANTA, a pipeline for long-form video question answering that transcribes video and audio into textual segments, scores segments with a query-independent information-density heuristic, aligns visual and textual representations with contrastive learning, and retrieves query-relevant segments under a token budget for a downstream LLM. The authors claim a provable near-optimality guarantee for this selection procedure (Theorem 2) and report large accuracy gains on Video-MME plus two newly introduced benchmarks, LVU-QA and MultiModal-TempRel. The manuscript also includes a convergence theorem for contrastive alignment, ablation studies, and qualitative case studies.

Significance. If the theoretical guarantee and the empirical gains were both valid, the paper would be significant: it would give a principled, provably near-optimal mechanism for constructing LLM context from long-form multimodal input, and the reported improvements over strong baselines would be practically important. The contrastive alignment result, however, is a standard InfoNCE-style statement, and the central optimality theorem is not established: its main assumption effectively assumes the query-independent heuristic equals the query-dependent mutual information, and the knapsack approximation step is asserted rather than derived. The empirical evaluation is also difficult to assess because the two new benchmarks are not released, no error bars or significance tests are given, and several baselines are not cited. The paper is therefore currently a plausible engineering recipe with an unsupported optimality claim and incomplete validation.

major comments (3)
  1. [Section 3.5, Theorem 2, assumption (iii)] The optimality theorem is not applicable to the system as described because the density score D(s_i) defined in Eq. (5) is query-independent, while the quantity it is assumed to approximate, I(s_i; Q), is query-dependent. Each term in Eq. (5) -- -log p(s_i|s_<i), H(s_i), I(c_i; t_i), and R(s_i) -- is a property of the video segment alone. For a fixed video, D(s_i) is the same for every query, whereas I(s_i; Q) varies with Q. Applying assumption (iii) to two queries Q1 and Q2 gives |I(s_i; Q1) - I(s_i; Q2)| < 2γ for every segment, which would mean all queries have nearly identical relevance scores across all segments. This contradicts any nontrivial retrieval setting and is also contradicted by the paper's own query-dependent two-stage retrieval in Eq. (12), which projects the query and reranks candidates by query similarity. Theorem 2 therefore cannot justify the optimization of Eq. (1).
  2. [Section 3.5, proof of Theorem 2] The approximation-ratio derivation has unsupported steps that are load-bearing. Eq. (9) bounds the gap between total mutual information and the sum of per-segment mutual informations by C(|S|,2)·ε, yet the theorem's conclusion uses only ε in the ratio 1-(ε+δ+γ); no argument is given for how the quadratic term is absorbed. More seriously, the proof states that under assumption (ii) 'a greedy algorithm selecting items based on value density achieves an approximation ratio of 1-δ,' but this is asserted without proof. The standard greedy guarantee for the knapsack problem is 1/2, and the uncorrelatedness bound δ does not by itself yield a 1-δ approximation. Consequently, the claimed ratio is unsupported independently of the proxy issue in assumption (iii).
  3. [Section 4.1 and Tables 1-3] The empirical validation is insufficient for the strength of the claims. LVU-QA and MultiModal-TempRel are introduced in Section 4.1 as 'newly collected' benchmarks, but no release information, dataset construction details, annotation protocols, or basic statistics beyond video/question counts are provided, and no evaluation code or data is made available. No table in the paper reports error bars, multiple seeds, or statistical significance tests, despite the main results showing differences of more than 20 percentage points. Given that the central claims of the paper rest on these numbers, the evaluation as presented cannot be independently verified or reproduced.
minor comments (5)
  1. [Eq. (5) and Theorem 2] The symbol γ is used both as the redundancy-penalty weight in Eq. (5) and as the error tolerance in Theorem 2, which is confusing because the two quantities play entirely different roles; please use distinct notations.
  2. [Eq. (9)] The displayed equation in the proof of Theorem 2 appears garbled, with absolute-value bars rendered inconsistently; if the bound is retained, it should be written cleanly and the combinatorial factor should be addressed.
  3. [Table 1 and References] Several baselines in Table 1 -- MLLM-Projection, MCA-VILLA, Vision-Flan, VideoGPT-4, and MultiVision-7B -- are not identified by citations, so readers cannot determine what systems were used or whether the comparisons are apples-to-apples.
  4. [Section 5 and Abstract] The abstract states a '27.3%' gain on videos exceeding 30 minutes, but the closest evidence in the paper is the 27.3% improvement on Long-Range Dependencies in Table 2, which is not the same as a duration-stratified result; please clarify which claim is supported.
  5. [Section 3.5 and Appendix B] The phrase 'information density estimation' suggests estimation of a probability density, but Eq. (5) is a heuristic weighted sum of novelty, entropy, cross-modal coherence, and redundancy terms; the terminology should be adjusted to avoid overclaiming.

Circularity Check

1 steps flagged · score 7.0 of 10

Theorem 2's near-optimality guarantee is built into assumption (iii): the query-independent density score D(s_i) is assumed to approximate I(s_i;Q), so the central 'provable optimality' claim reduces to a hand-designed proxy assumption.

  1. self definitional [Section 3.5, Theorem 2, assumption (iii), with Eq. (5) in Section 3.3]
    "Theorem 2 (Optimality of Information-Density Selection). ... (iii) The density scores D(si) approximate mutual information: |D(si) − I(si; Q)| < γ. Then selecting segments based on information density scores D(si) achieves an approximation ratio of 1−(ϵ+δ +γ) compared to the optimal solution for maximizing mutual information with the query subject to context length constraints."

    D(s_i) in Eq. (5) contains no query term: it is −log p(s_i|s_<i)+αH(s_i)+βI(c_i;t_i)−γR(s_i), all independent of Q. Assumption (iii) therefore asserts that this fixed, query-free score is within γ of I(s_i;Q), the very query-relevance quantity the theorem claims to optimize. The conclusion that D-based selection is 1−(ε+δ+γ)-optimal is the proxy assumption restated as a guarantee; no independent derivation of the score's fidelity is given. Since D is identical for every question on a given video while I(s_i;Q) varies by question, assumption (iii) cannot hold across the queries used in inference (Eq. 12 uses a query embedding q for retrieval). The central optimality claim thus reduces by construction to the unverified equation D≈I.

full rationale

The paper's empirical claims are benchmarked externally and are not circular. Theorem 1 is the standard InfoNCE mutual-information bound, and the related-work self-citation (Zhong, 2025) is not load-bearing for any result. The circularity is localized to the paper's headline theoretical contribution: Section 3.5's Theorem 2. There, the hand-designed density score D(s_i) from Eq. (5) is not query-dependent, yet assumption (iii) stipulates |D(s_i)−I(s_i;Q)|<γ, and the theorem then 'proves' D-based selection is near-optimal. That guarantee is the assumption wearing a theorem's clothing. The proof also asserts, without derivation, that a density-greedy selection achieves ratio 1−δ from assumption (ii); standard knapsack greedy only guarantees 1/2 unless additional small-item conditions are quantified, so this is an unsupported step (correctness risk) rather than circularity per se. Because the central optimality claim reduces to the proxy assumption, the circularity score is 7 rather than lower.

Assumptions & free parameters 10 free parameters · 5 assumptions · 2 invented entities

The central content-selection theorem leans on an ad hoc assumption that the paper's own density score approximates the target mutual information, plus a hand-set set of hyperparameters and the sufficiency of textual projection. The benchmarks and code are not released, so the empirical claims rest entirely on the authors' unreported evaluation setup.

free parameters (10)
  • alpha (α) = 0.35
    Information density score weight for entropy term in Eq. 5; hand-set in Section 4.1, no sensitivity analysis.
  • beta (β) = 0.25
    Weight for cross-modal mutual information term in Eq. 5; hand-set.
  • gamma (γ) = 0.15
    Weight for redundancy penalty in Eq. 5; hand-set; also the allowed approximation error in Theorem 2 assumption (iii).
  • tau_dedup = 0.85
    Coverage overlap threshold in Algorithm 1; hand-set.
  • tau_length = 10 tokens
    Minimum unique content length in Algorithm 1; hand-set.
  • tau_conf (confidence threshold) = not specified
    ASR filtering threshold in Algorithm 2; value not reported.
  • lambda (λ) = not specified
    Coverage decay rate in Algorithm 1; value not reported.
  • temperature tau (contrastive) = not specified
    Temperature in Eq. 6; value not reported.
  • context window w = not specified
    Window size in Eq. 11; value not reported.
  • coherence threshold τ = not specified
    Constraint in Eq. 1; value not reported.
assumptions (5)
  • standard math Contrastive loss (InfoNCE) is a lower bound on mutual information and minimizing it maximizes MI between corresponding segments.
    Invoked in Theorem 1 proof, Section 3.3; standard InfoNCE result, but the paper does not account for the finite-sample bias ϵ.
  • ad hoc to paper Segment information contributions are ϵ-approximately independent (Theorem 2 assumption i).
    This assumption is needed to turn the selection problem into a knapsack but is not validated for real video segments.
  • ad hoc to paper Segment length and information content are uncorrelated (Theorem 2 assumption ii).
    Used to claim greedy density selection has 1-delta approximation; no empirical support and the implication is not derived.
  • ad hoc to paper Density score D(s_i) approximates query mutual information within gamma (Theorem 2 assumption iii).
    This is the circular core: the theorem assumes the heuristic score is already a proxy for the target quantity.
  • domain assumption Textual caption/transcript is a sufficient representation of visual/audio content for QA.
    The whole pipeline projects modalities into text, losing fine visual detail; no analysis of information loss.
invented entities (2)
  • LVU-QA benchmark
    purpose: To evaluate long-range temporal reasoning in long videos.
    Introduced in Section 4.1 as a new benchmark with 500 videos and 3,000 questions, but no download link or external validation is provided.
  • MultiModal-TempRel benchmark
    purpose: To evaluate temporal relationships across modalities.
    Introduced in Section 4.1 with 300 videos and 1,800 questions, but not released and no external validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MANTA: Cross-Modal Semantic Alignment and Information-Theoretic Optimization for Long-form Multimodal Understanding." pith.science (2026). https://pith.science/paper/X6TGSR66

@misc{pith2026250700068,
  author       = {Pith},
  title        = {Pith review of: MANTA: Cross-Modal Semantic Alignment and Information-Theoretic Optimization for Long-form Multimodal Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X6TGSR66}},
  note         = {Machine review of arXiv:2507.00068}
}
read the original abstract

While multi-modal learning has advanced significantly, current approaches often treat modalities separately, creating inconsistencies in representation and reasoning. We introduce MANTA (Multi-modal Abstraction and Normalization via Textual Alignment), a theoretically-grounded framework that unifies visual and auditory inputs into a structured textual space for seamless processing with large language models. MANTA addresses four key challenges: (1) semantic alignment across modalities with information-theoretic optimization, (2) adaptive temporal synchronization for varying information densities, (3) hierarchical content representation for multi-scale understanding, and (4) context-aware retrieval of sparse information from long sequences. We formalize our approach within a rigorous mathematical framework, proving its optimality for context selection under token constraints. Extensive experiments on the challenging task of Long Video Question Answering show that MANTA improves state-of-the-art models by up to 22.6% in overall accuracy, with particularly significant gains (27.3%) on videos exceeding 30 minutes. Additionally, we demonstrate MANTA's superiority on temporal reasoning tasks (23.8% improvement) and cross-modal understanding (25.1% improvement). Our framework introduces novel density estimation techniques for redundancy minimization while preserving rare signals, establishing new foundations for unifying multimodal representations through structured text.

Figures

Figures reproduced from arXiv: 2507.00068 by the authors.

Figure 1
Figure 1. Flowchart showing the MANTA framework. Raw videos are first processed through [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 11 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Feng Cheng and Gedas Bertasius. 2022. https://doi.org/10.1007/978-3-031-19830-4_29 Tallformer: Temporal action localization with&nbsp;a&nbsp;long-memory transformer . In Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXIV, page 503–521, Berlin, Heidelberg. Springer-Verlag

    doi:10.1007/978-3-031-19830-4_29 2022

  4. [4]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. https://arxiv.org/abs/2401.08281 The faiss library . ArXiv

    arXiv 2024

  5. [5]

    Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. 2024. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075

    arXiv 2024

  6. [6]

    Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, and Zicheng Liu. 2021. https://api.semanticscholar.org/CorpusID:244527662 Violet : End-to-end video-language transformers with masked visual-token modeling . ArXiv, abs/2111.12681

    arXiv 2021

  7. [7]

    Daya Guo, Jiangshui Hong, Binli Luo, Qirui Yan, and Zhangming Niu. 2019. https://api.semanticscholar.org/CorpusID:201065921 Multi-modal representation learning for short video understanding and recommendation . 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pages 687--690

    work page 2019

  8. [8]

    Hai Huang, Yan Xia, Shengpeng Ji, Shulei Wang, Hanting Wang, Jieming Zhu, Zhenhua Dong, and Zhou Zhao. 2024. https://api.semanticscholar.org/CorpusID:268297134 Unlocking the potential of multimodal unified discrete representation through training-free codebook optimization and hierarchical alignment . ArXiv, abs/2403.05168

    arXiv 2024

Show all 30 references
  1. [9]

    Weizhe Lin and Bill Byrne. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.772 Retrieval augmented visual question answering with outside knowledge . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11238--11254, Abu Dhabi, Uni...

    2022 doi

  2. [10]

    Weizhe Lin, Jingbiao Mei, Jinghong Chen, and Bill Byrne. 2024. https://aclanthology.org/2024.acl-long.289 P re FLMR : Scaling up fine-grained late-interaction multi-modal retrievers . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V...

    2024

  3. [11]

    Weizhe Lin, Zhilin Wang, and Bill Byrne. 2023. https://aclanthology.org/2023.findings-eacl.11 FVQA 2.0: Introducing adversarial samples into fact-based visual question answering . In Findings of the Association for Computational Linguistics: EACL 2023, pages 149--157, Dubrovni...

    2023

  4. [12]

    Ruyang Liu, Chen Li, Haoran Tang, Yixiao Ge, Ying Shan, and Ge Li. 2023. St-llm: Large language models are effective temporal learners. https://arxiv.org/abs/2404.00308

    2023 arXiv

  5. [13]

    Long Qian, Juncheng Li, Yu Wu, Yaobo Ye, Hao Fei, Tat-Seng Chua, Yueting Zhuang, and Siliang Tang. 2024. Momentor: advancing video large language model with fine-grained temporal reasoning. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org

    2024

  6. [14]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org

    2023

  7. [15]

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. 2023. https://arxiv.org/abs/2312.02051 Timechat: A time-sensitive multimodal large language model for long video understanding . Preprint, arXiv:2312.02051

    2023 arXiv

  8. [16]

    Yan Shu, Peitian Zhang, Zheng Liu, Minghao Qin, Junjie Zhou, Tiejun Huang, and Bo Zhao. 2024. https://api.semanticscholar.org/CorpusID:272827076 Video-xl: Extra-long vision language model for hour-scale video understanding . ArXiv, abs/2409.14485

    2024 arXiv

  9. [17]

    Enxin Song, Wenhao Chai, Tian Ye, Jenq-Neng Hwang, Xi Li, and Gaoang Wang. 2024. https://arxiv.org/abs/2404.17176 Moviechat+: Question-aware sparse memory for long video question answering . Preprint, arXiv:2404.17176

    2024 arXiv

  10. [18]

    Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari S. Morcos. 2023. D4: improving llm pretraining via document de-duplication and diversification. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA. C...

    2023

  11. [19]

    Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, Sen Xing, Guo Chen, Junting Pan, Jiashuo Yu, Yali Wang, Limin Wang, and Yu Qiao. 2022. https://api.semanticscholar.org/CorpusID:254275041 Internvideo: General video...

    2022 arXiv

  12. [20]

    Ziyue Wang, Chi Chen, Peng Li, and Yang Liu. 2023. https://arxiv.org/abs/2311.11598 Filling the image information gap for vqa: Prompting large language models to proactively ask questions . Preprint, arXiv:2311.11598

    2023 arXiv

  13. [21]

    Yuetian Weng, Mingfei Han, Haoyu He, Xiaojun Chang, and Bohan Zhuang. 2024. https://doi.org/10.1007/978-3-031-73414-4_26 Longvlm: Efficient long video understanding via large language models . In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29...

    2024 doi

  14. [22]

    Chao-Yuan Wu, Christoph Feichtenhofer, Haoqi Fan, Kaiming He, Philipp Krähenbühl, and Ross Girshick. 2019. https://doi.org/10.1109/CVPR.2019.00037 Long-term feature banks for detailed video understanding . In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition ...

    2019

  15. [23]

    Yan Xia, Hai Huang, Jieming Zhu, and Zhou Zhao. 2024. Achieving cross modal generalization with multimodal unified representation. Advances in Neural Information Processing Systems, 36

    2024

  16. [24]

    Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, and Afshin Dehghan. 2024. https://api.semanticscholar.org/CorpusID:271329151 Slowfast-llava: A strong training-free baseline for video large language models . ArXiv, abs/2407.15841

    2024 arXiv

  17. [25]

    Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. 2022. Zero-shot video question answering via frozen bidirectional language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, ...

    2022

  18. [26]

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yi Zhou, Junyan Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qiang Qi, Ji Zhang, and Feiyan Huang. 2023. https://api.semanticscholar.org/CorpusID:258352455 mplug-owl: Modul...

    2023 arXiv

  19. [27]

    Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1209 A simple LLM framework for long-range video question-answering . In Proceedings of the 2024 Conference on Empirical Metho...

    2024 doi

  20. [28]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. https://doi.org/10.48550/arXiv.2210.03493 Automatic chain of thought prompting in large language models

    2022 doi arXiv

  21. [29]

    Ziqi Zhong. 2025. https://doi.org/10.2139/ssrn.5255370 Ai-driven privacy policy optimisation for sustainable data strategy

    2025 doi

  22. [30]

    Yi Zhu and Xiu Li. 2023. https://api.semanticscholar.org/CorpusID:259121925 Iterative uni-modal and cross-modal clustered contrastive learning for image-text retrieval . 2023 International Conference on Pattern Recognition, Machine Vision and Intelligent Algorithms (PRMVIA), p...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.