Pith. sign in

REVIEW 3 major objections 6 minor 251 references

VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training-free decoder intervention that diagnoses each video–question pair and reweights attention toward video evidence can reduce event-level hallucination in video language models, reaching 72.60% on EventHallusion with LLaVA-Video-7B.

desk verdict Training-free hallucination mitigation with real gains, but the adaptive mechanism is under-validated and the headline result is tuned on the same benchmark it leads with. read the letter →

arxiv 2608.08622 v1 pith:LSHD77SK submitted 2026-08-09 cs.CV

Dong Xing , Jiaxin Chen , Hang Yang , Peixun Liu , Qiushi Yang , Yuqing Wang This is my paper · ORCID
classification cs.CV
keywords videohallucinationcontrastivedecodingattentionreweightingtraining-freedebiasingevent-levelgroundingtemporalconsistencygradientsaliencylargevision-languagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VADER is a training-free decoding method that tries to stop video language models from producing fluent answers unsupported by the video. Its central claim is that hallucination can be reduced by making the intervention per-input rather than fixed: for each video–question pair, the model's own attention and gradients identify which decoder layers carry visual-to-text evidence, and those layers get a calibrated reallocation of pre-softmax attention from system tokens toward video tokens. A second branch erases the most-attended visual tokens within each frame, so predictions that stay confident without that evidence are exposed as prior-driven and suppressed by contrastive decoding. The paper argues this works because video frames are redundant enough that input-level perturbations can be recovered, while decoder-internal, per-frame erasure cannot be. If correct, this gives a plug-and-play way to improve grounding and temporal consistency across existing video LLMs without retraining.

What carries the argument

The load-bearing object is the gradient-attention saliency tensor $I^{(l)}_{ij}=\frac{1}{H}\sum_h |A^{(l,h)}_{ij} \odot \partial \mathcal{L}_{\mathrm{diag}}/\partial A^{(l,h)}_{ij}|$, computed from the model's own first prediction $\hat y_1$; it claims to measure which token interactions are both attended to and influential for the current output. VFR reads the cross-modal part of this tensor (text positions attending to video positions) to choose the intervention layers and the logarithmic amplitude $a=\log(1+\beta x)$. SEE's mechanism is an additive $-\infty$ mask on the attention-score columns of high-importance video tokens at a fixed insertion layer $l^\star$, with a per-frame retention budget that keeps the lowest-attended tokens in every frame. The contrastive fusion $\bar z=(1+\lambda)z_{\mathrm{gnd}}-\lambda z_{\mathrm{prior}}$ is the final mechanism that converts the two branches into a suppression rule.

What would settle it

Replace VADER's adaptive layer selection with attention-mass-only selection and with a random selection of five layers on EventHallusion while keeping SEE fixed; if either replacement matches the adaptive result within noise, the attribution signal is not doing the work the paper assigns it.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a dual-branch contrastive decoding scheme, with both branches constructed inside the frozen decoder, meaningfully reduces event-level hallucination in video LLMs. The grounded branch (VFR) uses gradient–attention saliency to select the top-K layers with the strongest cross-modal flow and reweights the system-token and video-token blocks of the pre-softmax score matrix by $(1-a)$ and $(1+a)$. The prior branch (SEE) masks, at a fixed insertion layer, the highest-attended visual tokens per frame and keeps low-attended context, producing logits that reflect language priors; the contrast $\bar z=(1+\lambda)z_{\mathrm{gnd}}-\lambda z_{\mathrm{prior}}$ suppresses candidates that remain confident after erasure. On LLaVA-Video-7B, the method reaches 72.60% accuracy on EventHallusion, above the training-based TPO and RRPO, and it improves average VideoHallucer scores by +7.29 points while leaving MVBench, VideoMME, and TVBench averages positive or neutral.

Load-bearing premise

The method assumes that the product of attention mass and gradient magnitude, read from the model's own first prediction, reliably shows which decoder layers and which video tokens actually drive the current answer.

Editorial extensions

If this is right

  • On LLaVA-Video-7B, VADER improves EventHallusion accuracy from 63.57% to 72.60%, surpassing the training-based TPO (63.33%) and RRPO (67.97%) without any parameter updates.
  • VADER improves average VideoHallucer accuracy by +3.90, +10.30, and +7.29 points on Qwen3-VL-8B, LLaVA-OneVision-7B, and LLaVA-Video-7B, with the largest gains on event-ordering and temporal-consistency subsets.
  • Ablations show that the adaptive policy matters: fixed-layer VFR yields smaller gains than per-video layer selection, and VFR-only or SEE-only settings each underperform the combined method.
  • General video understanding is preserved: on MVBench, VideoMME, and TVBench, VADER raises the average by +0.6 and +1.5 points on LLaVA-OV-7B and LLaVA-Video-7B, while several competing interventions are neutral or slightly negative.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The gradient-attention saliency is never validated against a ground-truth attribution; a direct test would be to replace the saliency-based layer selection with attention-mass-only selection and with random layers on the same benchmarks, which would show how much of the gain depends on the attribution being correct.
  • Because the method is training-free and architecture-agnostic, the same two-branch recipe may transfer to other multimodal decoding settings, such as image-based LVLMs with long contexts or audio-visual models, where per-frame redundancy has a distinct analogue.
  • The contrastive formulation suggests a testable prediction: VADER should help most on questions where the wrong answer is supported by coarse scene context and where the correct answer depends on localized, sparse evidence — a comparison the paper only illustrates qualitatively in Figure 5.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes VADER, a training-free framework for reducing hallucinations in video large language models (VideoLLMs). VADER combines two modules: Visual Focus Reallocation (VFR), which diagnoses layer-wise visual-to-text attention flow via a gradient-attention saliency measure and adaptively selects decoder layers and a reweighting amplitude to shift pre-softmax attention from system tokens to video tokens; and Selective Evidence Erasure (SEE), which masks per-frame high-importance visual tokens inside the decoder to construct a prior-biased contrastive branch. The two branches are fused through contrastive decoding. Experiments on VidHalluc, VideoHallucer, EventHallusion, and general video understanding benchmarks across three VideoLLMs report substantial gains; the headline result is 72.60% accuracy on EventHallusion with LLaVA-Video-7B, surpassing the training-based methods TPO and RRPO.

Significance. If the results hold, VADER shows that training-free, input-adaptive decoding interventions can improve video grounding without retraining, and the module ablations support the complementarity of VFR and SEE. Strengths include a clearly specified algorithm, a consistent training-free protocol across backbones and benchmarks, and evaluations on both hallucination-specific and general video-understanding tasks. However, the evidence for the adaptive mechanism itself is weaker than the evidence for the overall gains, and the hyperparameter selection procedure calls the headline number into question; both issues need to be addressed before the central claims are fully convincing.

major comments (3)
  1. [§4.1, §4.4, Table 1] The global configuration (β=0.05, λ=1.5, K_l=5, l*=8, ρ=0.3) and the SEE insertion depth are explicitly selected using EventHallusion with LLaVA-Video-7B, and the same setting is then reported as the headline result (72.60% on EventHallusion in Table 1). This constitutes test-set tuning: the hyperparameters are fit to the benchmark on which the main improvement over TPO and RRPO is claimed. Please either report results with hyperparameters chosen on a disjoint validation split of EventHallusion, or provide a held-out benchmark that was not used for any configuration choice; without this, the comparison on EventHallusion is not conclusive.
  2. [§3.2, Eq. (3), Table 3] The gradient-attention saliency I^{(l)}_{ij} is computed with respect to the model's own initial prediction ŷ1, which may itself be a hallucination. There is no validation that this saliency identifies the visual-to-text evidence flow that supports the correct answer; the only supporting evidence is the qualitative visualization in Figure 3 and a comparison to a single fixed-layer set in Table 3. If the attribution is noisy or systematically miscalibrated for hallucinated inputs, the adaptive policy could select the wrong layers and the mechanism claim would not hold. Please add a quantitative sanity check (e.g., correlation of saliency with some ground-truth measure of evidence importance, or a control comparing adaptive selection with randomly chosen layer sets) and justify using the first-token diagnostic for all subsequent decoding steps.
  3. [§3.3, Figure 5] The claim that SEE yields a prior-biased branch that is 'difficult to compensate' through cross-frame redundancy is not directly measured. Masking at a single insertion layer l* may be partially undone in higher layers, and the paper does not show that the branch is systematically biased toward language priors beyond the one case study in Figure 5. Please provide an analysis of the negative branch (e.g., agreement between z_prior and a text-only model, or a measure of how much evidence survives the mask across layers), or at least explicitly qualify this claim.
minor comments (6)
  1. [§3.2, Eq. (10)] The set T_dec ('decoder-side positions actually intervened by VFR') is never precisely defined; please clarify whether it includes all textual positions (system plus context) and whether it affects rows only for the current decoding token or all rows.
  2. [§4.2, Tables 3–4] No error bars or multiple runs are reported, so the gains of 0.5–2.1 points over the fixed-layer baseline could be within run-to-run variance; deterministic decoding is mentioned for the prior branch, but the main branch sampling strategy is not specified.
  3. [§4.4] The term 'source-setting' is used without definition; please clarify what 'source' refers to in this context.
  4. [Figure 3] The heatmaps in Figure 3 are difficult to read in print; consider adding color bars and axis labels to make the saliency visualizations interpretable.
  5. [Abstract] The acronym VADER is used immediately after the expansion 'Video-Adaptive Debiasing via Evidence Reweighting,' but the acronym definition appears in the full text; consider defining it at first use in the abstract for clarity.
  6. [Table 1 caption] The comparison with TPO and RRPO is presented without noting that those methods involve additional training or reward-model supervision; the caption should state this to avoid misleading readers about the training-free claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the adaptive policy is computed from each input's own attention gradients and all benchmark results are external measurements with a fixed configuration.

full rationale

VADER's derivation chain is self-contained and does not reduce to its inputs by construction. The VFR policy (Eqs. 3-7) is computed from the frozen model's own forward/backward pass on each video-question input, using the detached self-prediction y_hat and the diagnostic loss L_diag = -log p(y_hat|V,q); no ground-truth labels or benchmark scores enter this computation. Equation (10) is a deterministic reweighting of pre-softmax attention scores, Eq. (15) is a fixed masking operation, and Eq. (16) is standard contrastive decoding. The reported improvements are external benchmark accuracies measured after applying these operations, with a single global configuration stated transparently in Section 4.1. The only mild caveat is that the global hyperparameters are selected on the EventHallusion/LLaVA-Video-7B setting before being applied elsewhere; this is a data-selection or test-set-tuning concern, not a circular derivation, since the benchmark numbers are not used to construct the policy or to define the diagnostic quantities. There is no load-bearing self-citation chain: comparisons and related work cite external methods and benchmarks, and no central premise is justified solely by the present authors' prior work. The skeptical concern about whether gradient-attention saliency is causally informative is a robustness/validity issue, not a circularity issue. The paper therefore receives a circularity score of 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces five hand-set hyperparameters selected on the EventHallusion benchmark, and relies on two interpretability assumptions about attention-gradient saliency and per-frame token retention. No new entities or forces are postulated.

free parameters (5)
  • beta = 0.05
    Mapping from saliency flow x to reweighting amplitude a in Eq. 7; hand-chosen as part of the global configuration tuned on EventHallusion.
  • lambda = 1.5
    Contrastive strength in Eq. 16; tuned on EventHallusion.
  • K_l = 5
    Number of selected layers for VFR in Eq. 6; tuned on EventHallusion.
  • l_star = 8
    SEE insertion depth; calibrated on EventHallusion in Section 4.4.
  • rho = 0.3
    Retention ratio for SEE in Eq. 13; tuned on EventHallusion.
assumptions (4)
  • standard math Standard scaled dot-product attention and softmax are computed as in Eq. 2 and Eq. 11.
    This is the standard attention mechanism in transformer decoders, used as the substrate for all interventions.
  • domain assumption The gradient-attention saliency product |A * dL/dA| in Eq. 3 identifies token interactions that influence the model's output distribution.
    VFR's layer selection and strength estimation depend on this attribution being meaningful; the paper provides qualitative visualizations but no formal validation.
  • domain assumption Cross-modal saliency flow averaged over text-video token pairs (Eq. 4) indicates where visual evidence competes with system tokens in a given layer.
    VFR chooses top-K layers by this quantity; if the flow is not a reliable proxy for evidence competition, the adaptive policy may pick unhelpful layers.
  • domain assumption Retaining the bottom-k visual tokens per frame (Eq. 13) preserves low-response temporal context while removing discriminative evidence.
    SEE's prior-biased branch relies on this to avoid cross-frame compensation; this is an empirical design hypothesis tested only via ablations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models." pith.science (2026). https://pith.science/paper/LSHD77SK

@misc{pith2026260808622,
  author       = {Pith},
  title        = {Pith review of: VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LSHD77SK}},
  note         = {Machine review of arXiv:2608.08622}
}
read the original abstract

Large vision-language models (LVLMs) have demonstrated strong performance in open-ended video understanding, yet they remain prone to fluent responses unsupported by video evidence. Existing training-free methods typically apply a globally fixed visual intervention or construct a contrastive branch through input perturbation. The former cannot accommodate video-dependent fusion paths, while the latter can be compensated by cross-frame redundancy. We therefore propose Video-Adaptive Debiasing via Evidence Reweighting (VADER), a training-free framework with two complementary modules. Visual Focus Reallocation (VFR) automatically instantiates an intervention policy for each video-question input: it diagnoses layer-wise visual-to-text evidence flow, determines where to intervene, and derives how strongly to reallocate pre-softmax attention from system-token to video-token blocks. Selective Evidence Erasure (SEE) independently masks high-importance visual tokens in every frame, constructing a prior-biased branch that is difficult to compensate through neighboring frames. Contrastive decoding then down-weights predictions that remain confident after selective evidence erasure. Across multiple VideoLLMs, VADER yields substantial improvements on event-level grounding and temporal consistency; on LLaVA-Video-7B, it reaches 72.60% accuracy on EventHallusion.

Figures

Figures reproduced from arXiv: 2608.08622 by the authors.

Figure 1
Figure 1. Overview of the VADER framework. VFR automatically derives a video-conditioned intervention policy—the layer locations and response strength—and reallocates attention toward video evidence. SEE suppresses high-response visual tokens independently in every frame to expose prior-biased predictions. Joint temporal contrastive suppression contrasts the two branches. tokens. The same decoding state is then fed into two d… view at source ↗
Figure 2
Figure 2. Attention rebalancing by fixed-layer VFR. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. One-time calibration of the SEE insertion depth on the source setting. Left: query-conditioned saliency visualization [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Sensitivity to the SEE keep ratio. Frame 0 Frame 1 Frame 2 Frame 3 Frame 4 (a) (b) A B C D 0.0 0.2 0.4 0.6 0.8 1.0 Probability 0.44 0.56 0.00 0.00 Original logits A B C D 0.30 0.70 0.00 0.00 SEE logits A B C D 0.72 0.28 0.00 0.00 Original - SEE [PITH_FULL_IMAGE:figure…
Figure 5
Figure 5. Figure 5: SEE exposes prior-driven bias through erasure and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

251 extracted references · 66 linked inside Pith

  1. [2]

    Bae, K.; Kim, J.; Lee, S.; Lee, S.; Lee, G.; and Choi, J. 2025. MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representations. In Proceedings of the Computer Vision and Pattern Recognition Conference, 13744--13753

    2025

  2. [5]

    Cai, J.; Hong, J.; Zhang, Z.; Zhou, W.; zhannianji; and Li, H. 2025. Mitigating Hallucination in Video LLM s via Temporal-Aware Activation Engineering. In The Thirty-ninth Annual Conference on Neural Information Processing Systems

    2025

  3. [7]

    Ding, X.; Han, J.; Xu, H.; Liang, X.; Zhang, W.; and Li, X. 2024. Holistic autonomous driving understanding by bird's-eye-view injected multi-modal large models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13668--13677

    2024

  4. [10]

    Fu, C.; Dai, Y.; Luo, Y.; Li, L.; Ren, S.; Zhang, R.; Wang, Z.; Zhou, C.; Shen, Y.; Zhang, M.; et al. 2025. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, 24108--24118

    2025

  5. [11]

    Huang, Q.; Dong, X.; Zhang, P.; Wang, B.; He, C.; Wang, J.; Lin, D.; Zhang, W.; and Yu, N. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13418--13427

    2024

  6. [12]

    H.; Jo, Y.; and Seo, M

    Lee, S.; Park, S. H.; Jo, Y.; and Seo, M. 2024. VOLCANO: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision. In 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2024, 391--404. Association for Computational Linguistics (ACL)

    2024

  7. [13]

    Lei, J.; Berg, T.; and Bansal, M. 2023. Revealing single frame bias for video-and-language learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 487--507

    2023

  8. [14]

    Leng, S.; Zhang, H.; Chen, G.; Li, X.; Lu, S.; Miao, C.; and Bing, L. 2024. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13872--13882

    2024

Show all 251 references
  1. [16]

    W.; and Fazli, P

    Li, C.; Im, E. W.; and Fazli, P. 2025. Vidhalluc: Evaluating temporal hallucinations in multimodal large language models for video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, 13723--13733

    2025

  2. [18]

    Li, K.; Wang, Y.; He, Y.; Li, Y.; Wang, Y.; Liu, Y.; Wang, Z.; Xu, J.; Chen, G.; Luo, P.; et al. 2024 b . Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 22195--22206

    2024

  3. [20]

    Lin, B.; Ye, Y.; Zhu, B.; Cui, J.; Ning, M.; Jin, P.; and Yuan, L. 2024. Video-LLaVA: Learning United Visual Representation by Alignment Before Projection. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 5971--5984. Miami, Florida, US...

    2024

  4. [21]

    Liu, F.; Lin, K.; Li, L.; Wang, J.; Yacoob, Y.; and Wang, L. 2023 a . Aligning large multi-modal model with robust instruction tuning. CoRR

    2023

  5. [22]

    Liu, F.; Lin, K.; Li, L.; Wang, J.; Yacoob, Y.; and Wang, L. 2024 a . Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning. In The Twelfth International Conference on Learning Representations

    2024

  6. [23]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023 b . Visual instruction tuning. Advances in neural information processing systems, 36: 34892--34916

    2023

  7. [25]

    Ma, F.; Jin, X.; Wang, H.; Xian, Y.; Feng, J.; and Yang, Y. 2024. Vista-llama: Reducing hallucination in video language models via equal distance to visual tokens. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13151--13160

    2024

  8. [26]

    Maaz, M.; Rasheed, H.; Khan, S.; and Khan, F. 2024. Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 12585--12602. Bangk...

    2024

  9. [28]

    Song, E.; Chai, W.; Wang, G.; Zhang, Y.; Zhou, H.; Wu, F.; Chi, H.; Guo, X.; Ye, T.; Zhang, Y.; et al. 2024. Moviechat: From dense token to sparse memory for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18221--18232

    2024

  10. [29]

    Tang, X.; Qiu, J.; Xie, L.; Tian, Y.; Jiao, J.; and Ye, Q. 2025. Adaptive keyframe sampling for long video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, 29118--29128

    2025

  11. [32]

    Wang, X.; Pan, J.; Ding, L.; and Biemann, C. 2024 b . Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding. In Findings of the Association for Computational Linguistics ACL 2024, 15840--15853

    2024

  12. [38]

    Yin, S.; Fu, C.; Zhao, S.; Xu, T.; Wang, H.; Sui, D.; Shen, Y.; Li, K.; Sun, X.; and Chen, E. 2024. Woodpecker: Hallucination correction for multimodal large language models. Science China Information Sciences, 67(12): 220105

    2024

  13. [39]

    Yu, Q.; Li, J.; Wei, L.; Pang, L.; Ye, W.; Qin, B.; Tang, S.; Tian, Q.; and Zhuang, Y. 2024 a . Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12944--12953

    2024

  14. [40]

    Yu, T.; Yao, Y.; Zhang, H.; He, T.; Han, Y.; Cui, G.; Hu, J.; Liu, Z.; Zheng, H.-T.; Sun, M.; et al. 2024 b . Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision an...

    2024

  15. [41]

    Q.; Stepputtis, S.; Ramanan, D.; Salakhutdinov, R.; Morency, L.-P.; Sycara, K

    Zhang, C.; Wan, Z.; Kan, Z.; Ma, M. Q.; Stepputtis, S.; Ramanan, D.; Salakhutdinov, R.; Morency, L.-P.; Sycara, K. P.; and Xie, Y. 2025. Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models. In The Thirteenth Internati...

    2025

  16. [43]

    j.; Gui, L.; Fu, D.; Feng, J.; Liu, Z.; and Li, C

    Zhang, Y.; Li, B.; Liu, h.; Lee, Y. j.; Gui, L.; Fu, D.; Feng, J.; Liu, Z.; and Li, C. 2024 b . LLaVA-NeXT: A Strong Zero-shot Video Understanding Model

    2024

  17. [45]

    Zhao, L.; Deng, Y.; Zhang, W.; and Gu, Q. 2025. Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, 77461--...

    2025

  18. [46]

    Zhou, Y.; Cui, C.; Yoon, J.; Zhang, L.; Deng, Z.; Finn, C.; Bansal, M.; and Yao, H. 2023. Analyzing and Mitigating Object Hallucination in Large Vision-Language Models. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following

    2023

  19. [47]

    Zhuang, X.; Zhu, Z.; Xie, Y.; Liang, L.; and Zou, Y. 2025. Vasparse: Towards efficient visual hallucination mitigation via visual-aware token sparsification. In Proceedings of the Computer Vision and Pattern Recognition Conference, 4189--4199

    2025

  20. [48]

    FirstName LastName , title =

  21. [49]

    FirstName Alpher , title =

  22. [50]

    Journal of Foo , volume = 13, number = 1, pages =

    FirstName Alpher and FirstName Fotheringham-Smythe , title =. Journal of Foo , volume = 13, number = 1, pages =

  23. [51]

    Journal of Foo , volume = 14, number = 1, pages =

    FirstName Alpher and FirstName Fotheringham-Smythe and FirstName Gamow , title =. Journal of Foo , volume = 14, number = 1, pages =

  24. [52]

    FirstName Alpher and FirstName Gamow , title =

  25. [53]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

  26. [54]

    arXiv preprint arXiv:2409.12191 , year=

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution , author=. arXiv preprint arXiv:2409.12191 , year=

    arXiv

  27. [55]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    arXiv

  28. [56]

    arXiv preprint arXiv:2312.11805 , year=

    Gemini: a family of highly capable multimodal models , author=. arXiv preprint arXiv:2312.11805 , year=

    arXiv

  29. [57]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Image captioning with semantic attention , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  30. [58]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Msr-vtt: A large video description dataset for bridging video and language , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  31. [59]

    Proceedings of the IEEE international conference on computer vision , pages=

    Vqa: Visual question answering , author=. Proceedings of the IEEE international conference on computer vision , pages=

  32. [60]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Activitynet-qa: A dataset for understanding complex web videos via question answering , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  33. [61]

    Icml , volume=

    Is space-time attention all you need for video understanding? , author=. Icml , volume=

  34. [62]

    arXiv preprint arXiv:2305.06355 , year=

    Videochat: Chat-centric video understanding , author=. arXiv preprint arXiv:2305.06355 , year=

    arXiv

  35. [63]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Detecting and preventing hallucinations in large vision language models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  36. [64]

    arXiv preprint arXiv:2306.14565 , year=

    Mitigating hallucination in large multi-modal models via robust instruction tuning , author=. arXiv preprint arXiv:2306.14565 , year=

    arXiv

  37. [65]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Vidhalluc: Evaluating temporal hallucinations in multimodal large language models for video understanding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  38. [66]

    arXiv preprint arXiv:2406.16338 , year=

    Videohallucer: Evaluating intrinsic and extrinsic hallucinations in large video-language models , author=. arXiv preprint arXiv:2406.16338 , year=

    arXiv

  39. [67]

    arXiv preprint arXiv:2409.16597 , year=

    EventHallusion: Diagnosing Event Hallucinations in Video LLMs , author=. arXiv preprint arXiv:2409.16597 , year=

  40. [68]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Mitigating object hallucinations in large vision-language models through visual contrastive decoding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  41. [69]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Multi-modal hallucination control by visual information grounding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  42. [70]

    arXiv preprint arXiv:2405.17820 , year=

    Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models , author=. arXiv preprint arXiv:2405.17820 , year=

    arXiv

  43. [71]

    Proceedings of the 42nd International Conference on Machine Learning , pages=

    Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance , author=. Proceedings of the 42nd International Conference on Machine Learning , pages=. 2025 , volume=

    2025

  44. [72]

    arXiv preprint arXiv:2408.03326 , year=

    Llava-onevision: Easy visual task transfer , author=. arXiv preprint arXiv:2408.03326 , year=

    arXiv

  45. [73]

    arXiv preprint arXiv:2410.02713 , year=

    Video instruction tuning with synthetic data , author=. arXiv preprint arXiv:2410.02713 , year=

    arXiv

  46. [74]

    5-vl technical report , author=

    Qwen2. 5-vl technical report , author=. arXiv preprint arXiv:2502.13923 , year=

    arXiv

  47. [75]

    Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10) , pages=

    Causalbench: A comprehensive benchmark for evaluating causal reasoning capabilities of large language models , author=. Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10) , pages=

  48. [76]

    Advances in Neural Information Processing Systems , volume=

    Rextime: A benchmark suite for reasoning-across-time in videos , author=. Advances in Neural Information Processing Systems , volume=

  49. [77]

    arXiv preprint arXiv:2402.11435 , year=

    Momentor: Advancing video large language model with fine-grained temporal reasoning , author=. arXiv preprint arXiv:2402.11435 , year=

    arXiv

  50. [78]

    arXiv preprint arXiv:2406.11303 , year=

    Videovista: A versatile benchmark for video understanding and reasoning , author=. arXiv preprint arXiv:2406.11303 , year=

    arXiv

  51. [79]

    arXiv preprint arXiv:2501.13919 , year=

    Temporal preference optimization for long-form video understanding , author=. arXiv preprint arXiv:2501.13919 , year=

    arXiv

  52. [80]

    arXiv preprint arXiv:2504.12083 , year=

    Self-alignment of large video language models with refined regularized preference optimization , author=. arXiv preprint arXiv:2504.12083 , year=

  53. [81]

    arXiv preprint arXiv:2506.03340 , year=

    Seeing the Arrow of Time in Large Multimodal Models , author=. arXiv preprint arXiv:2506.03340 , year=

  54. [82]

    arXiv preprint arXiv:2403.00476 , year=

    Tempcompass: Do video llms really understand videos? , author=. arXiv preprint arXiv:2403.00476 , year=

    arXiv

  55. [83]

    arXiv preprint arXiv:2410.07752 , year=

    Lost in Time: A New Temporal Benchmark for VideoLLMs , author=. arXiv preprint arXiv:2410.07752 , year=

    arXiv

  56. [84]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  57. [85]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Mvbench: A comprehensive multi-modal video understanding benchmark , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  58. [86]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  59. [87]

    arXiv preprint arXiv:2302.13971 , year=

    Llama: Open and efficient foundation language models , author=. arXiv preprint arXiv:2302.13971 , year=

    arXiv

  60. [88]

    Journal of Machine Learning Research , volume=

    Scaling instruction-finetuned language models , author=. Journal of Machine Learning Research , volume=

  61. [89]

    CoRR , year=

    Aligning large multi-modal model with robust instruction tuning , author=. CoRR , year=

  62. [90]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  63. [91]

    Transactions on Machine Learning Research , issn=

    Maxime Oquab and Timoth. Transactions on Machine Learning Research , issn=. 2024 , url=

    2024

  64. [92]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  65. [93]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representations , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  66. [94]

    arXiv preprint arXiv:2504.05810 , year=

    PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning , author=. arXiv preprint arXiv:2504.05810 , year=

    arXiv

  67. [95]

    Mitigating Hallucination in Video

    Jianfeng Cai and Jiale Hong and Zongmeng Zhang and Wengang Zhou and zhannianji and Houqiang Li , booktitle=. Mitigating Hallucination in Video. 2025 , url=

    2025

  68. [96]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Test of time: Instilling video-language models with a sense of time , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  69. [97]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Vista-llama: Reducing hallucination in video language models via equal distance to visual tokens , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  70. [98]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    MHBench: Demystifying Motion Hallucination in VideoLLMs , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  71. [99]

    arXiv preprint arXiv:2411.12591 , year=

    Thinking before looking: Improving multimodal llm reasoning via mitigating visual hallucination , author=. arXiv preprint arXiv:2411.12591 , year=

    arXiv

  72. [100]

    Findings of the Association for Computational Linguistics ACL 2024 , pages=

    TempCompass: Do Video LLMs Really Understand Videos? , author=. Findings of the Association for Computational Linguistics ACL 2024 , pages=

    2024

  73. [101]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Video-LLaVA: Learning United Visual Representation by Alignment Before Projection , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=. 2024 , address=. doi:10.18653/v1/2024.emnlp-main.342 , url=

    2024 doi

  74. [102]

    arXiv preprint arXiv:2404.16994 , year=

    Pllava: Parameter-free llava extension from images to videos for video dense captioning , author=. arXiv preprint arXiv:2404.16994 , year=

    arXiv

  75. [103]

    The Thirteenth International Conference on Learning Representations , year=

    LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token , author=. The Thirteenth International Conference on Learning Representations , year=

  76. [104]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=. 2024 , address=. doi:10.18653/v1/2024.acl-long.679 , url=

    2024 doi

  77. [105]

    arXiv preprint arXiv:2505.19678 , year=

    Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs , author=. arXiv preprint arXiv:2505.19678 , year=

  78. [106]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  79. [107]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Devils in middle layers of large vision-language models: Interpreting, detecting and mitigating object hallucinations via attention lens , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  80. [108]

    The Thirteenth International Conference on Learning Representations , year=

    CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs , author=. The Thirteenth International Conference on Learning Representations , year=

  81. [109]

    The Twelfth International Conference on Learning Representations , year=

    Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning , author=. The Twelfth International Conference on Learning Representations , year=

  82. [110]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  83. [111]

    arXiv preprint arXiv:2504.13122 , year=

    VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models , author=. arXiv preprint arXiv:2504.13122 , year=

    arXiv

  84. [112]

    Advances in Neural Information Processing Systems , volume=

    Flashattention-3: Fast and accurate attention with asynchrony and low-precision , author=. Advances in Neural Information Processing Systems , volume=

  85. [113]

    Advances in neural information processing systems , volume=

    Space-time correspondence as a contrastive random walk , author=. Advances in neural information processing systems , volume=

  86. [114]

    International conference on machine learning , pages=

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation , author=. International conference on machine learning , pages=. 2022 , organization=

    2022

  87. [115]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  88. [116]

    arXiv preprint arXiv:2304.15010 , year=

    Llama-adapter v2: Parameter-efficient visual instruction model , author=. arXiv preprint arXiv:2304.15010 , year=

    arXiv

  89. [117]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages=

    Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages=

    2023

  90. [118]

    arXiv preprint arXiv:2501.05901 , year=

    Valley2: Exploring multimodal models with scalable vision-language design , author=. arXiv preprint arXiv:2501.05901 , year=

    arXiv

  91. [119]

    The Twelfth International Conference on Learning Representations , year=

    LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment , author=. The Twelfth International Conference on Learning Representations , year=

  92. [120]

    Forty-second International Conference on Machine Learning , year=

    The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models Via Visual Information Steering , author=. Forty-second International Conference on Machine Learning , year=

  93. [121]

    2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2024 , pages=

    VOLCANO: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision , author=. 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2024 , pages=. 2024 , organization=

    2024

  94. [122]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  95. [123]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  96. [124]

    Proceedings of the 33rd ACM International Conference on Multimedia , pages=

    Draw with Thought: Unleashing Multimodal Reasoning for Scientific Diagram Generation , author=. Proceedings of the 33rd ACM International Conference on Multimedia , pages=

  97. [125]

    arXiv preprint arXiv:2508.06206 , year=

    Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model , author=. arXiv preprint arXiv:2508.06206 , year=

    arXiv

  98. [126]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  99. [127]

    Nature , volume=

    The problem of the random walk , author=. Nature , volume=

  100. [128]

    Advances in Neural Information Processing Systems , volume=

    Region mutual information loss for semantic segmentation , author=. Advances in Neural Information Processing Systems , volume=

  101. [129]

    European Conference on Computer Vision , pages=

    Llama-vid: An image is worth 2 tokens in large language models , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  102. [130]

    arXiv preprint arXiv:2505.12826 , year=

    Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineering , author=. arXiv preprint arXiv:2505.12826 , year=

    arXiv

  103. [131]

    Advances in Neural Information Processing Systems , volume=

    Sharegpt4video: Improving video understanding and generation with better captions , author=. Advances in Neural Information Processing Systems , volume=

  104. [132]

    arXiv preprint arXiv:2504.08809 , year=

    Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models , author=. arXiv preprint arXiv:2504.08809 , year=

  105. [133]

    arXiv preprint arXiv:2512.01282 , year=

    Kardia-R1: Unleashing LLMs to Reason toward Understanding and Empathy for Emotional Support via Rubric-as-Judge Reinforcement Learning , author=. arXiv preprint arXiv:2512.01282 , year=

  106. [134]

    arXiv preprint arXiv:2505.15255 , year=

    MentalMAC: Enhancing Large Language Models for Detecting Mental Manipulation via Multi-Task Anti-Curriculum Distillation , author=. arXiv preprint arXiv:2505.15255 , year=

  107. [135]

    LLaVA-NeXT: A Strong Zero-shot Video Understanding Model , url=

    Zhang, Yuanhan and Li, Bo and Liu, haotian and Lee, Yong jae and Gui, Liangke and Fu, Di and Feng, Jiashi and Liu, Ziwei and Li, Chunyuan , month=. LLaVA-NeXT: A Strong Zero-shot Video Understanding Model , url=

  108. [136]

    2025 , eprint=

    VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding , author=. 2025 , eprint=

    2025

  109. [137]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    Zhao, Yilun and Zhang, Haowei and Xie, Lujing and Hu, Tongyan and Gan, Guo and Long, Yitao and Hu, Zhiyuan and Chen, Weiyuan and Li, Chuhan and Xu, Zhijian and Wang, Chengye and Shangguan, Ziyao and Liang, Zhenwen and Liu, Yixin and Zhao, Chen and Cohan, Arman , title =. Proce...

    2025

  110. [138]

    Attention is All you Need , volume =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle =. Attention is All you Need , volume =

  111. [139]

    arXiv preprint arXiv:2512.18671 , year=

    SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention Collapse , author=. arXiv preprint arXiv:2512.18671 , year=

  112. [140]

    arXiv preprint arXiv:2601.04778 , year=

    CounterVid: Counterfactual Video Generation for Mitigating Action and Temporal Hallucinations in Video-Language Models , author=. arXiv preprint arXiv:2601.04778 , year=

  113. [141]

    arXiv preprint arXiv:2503.19622 , year=

    Exploring hallucination of large multimodal models in video understanding: Benchmark, analysis and mitigation , author=. arXiv preprint arXiv:2503.19622 , year=

    arXiv

  114. [142]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  115. [143]

    arXiv preprint arXiv:1711.05101 , year=

    Decoupled weight decay regularization , author=. arXiv preprint arXiv:1711.05101 , year=

    arXiv

  116. [144]

    arXiv preprint arXiv:2505.14582 , year=

    Can pruning improve reasoning? revisiting long-cot compression with capability in mind for better reasoning , author=. arXiv preprint arXiv:2505.14582 , year=

  117. [145]

    Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding

    Wang, Xintong and Pan, Jingheng and Ding, Liang and Biemann, Chris. Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding. Findings of the Association for Computational Linguistics ACL 2024. 2024

    2024

  118. [146]

    The Thirteenth International Conference on Learning Representations , year=

    Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models , author=. The Thirteenth International Conference on Learning Representations , year=

  119. [147]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Vasparse: Towards efficient visual hallucination mitigation via visual-aware token sparsification , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  120. [148]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Moviechat: From dense token to sparse memory for long video understanding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  121. [149]

    arXiv preprint arXiv:2405.21075 , year=

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis , author=. arXiv preprint arXiv:2405.21075 , year=

    arXiv

  122. [150]

    arXiv preprint arXiv:2306.02858 , year=

    Video-llama: An instruction-tuned audio-visual language model for video understanding , author=. arXiv preprint arXiv:2306.02858 , year=

    arXiv

  123. [151]

    arXiv preprint arXiv:2304.10592 , year=

    Minigpt-4: Enhancing vision-language understanding with advanced large language models , author=. arXiv preprint arXiv:2304.10592 , year=

    arXiv

  124. [152]

    InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning , author=

  125. [153]

    arXiv preprint arXiv:2304.08485 , year=

    Visual instruction tuning , author=. arXiv preprint arXiv:2304.08485 , year=

    arXiv

  126. [154]

    2024 , eprint=

    Qwen2 Technical Report , author=. 2024 , eprint=

    2024

  127. [155]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Frozen in time: A joint video and image encoder for end-to-end retrieval , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  128. [156]

    Proceedings of the ieee conference on computer vision and pattern recognition , pages=

    Activitynet: A large-scale video benchmark for human activity understanding , author=. Proceedings of the ieee conference on computer vision and pattern recognition , pages=

  129. [157]

    arXiv preprint arXiv:2310.01852 , year=

    Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment , author=. arXiv preprint arXiv:2310.01852 , year=

    arXiv

  130. [158]

    Preprint at https://arxiv

    Parameter-efficient image-to-video transfer learning for action recognition , author=. Preprint at https://arxiv. org/abs/2206.13559 , year=

    arXiv

  131. [159]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Scalability in perception for autonomous driving: Waymo open dataset , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  132. [160]

    X3D: Expanding Architectures for Efficient Video Recognition , url=

    Feichtenhofer, Christoph , year=. X3D: Expanding Architectures for Efficient Video Recognition , url=. doi:10.1109/cvpr42600.2020.00028 , booktitle=

    2020

  133. [161]

    2022 , month=

    Constitutional AI: Harmlessness from AI Feedback , author=. 2022 , month=

    2022

  134. [162]

    2023 , month=

    GPT-4 Technical Report , author=. 2023 , month=

    2023

  135. [163]

    2022 , month=

    Vision Transformer Adapter for Dense Predictions , author=. 2022 , month=

    2022

  136. [164]

    arXiv preprint arXiv:2301.12597 , year=

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models , author=. arXiv preprint arXiv:2301.12597 , year=

    arXiv

  137. [165]

    Deep Residual Learning for Image Recognition , url=

    He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian , year=. Deep Residual Learning for Image Recognition , url=. doi:10.1109/cvpr.2016.90 , booktitle=

    2016 doi

  138. [166]

    International Conference on Machine Learning , pages=

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation , author=. International Conference on Machine Learning , pages=. 2022 , organization=

    2022

  139. [167]

    2023 , eprint=

    PerceptionGPT: Effectively Fusing Visual Perception into LLM , author=. 2023 , eprint=

    2023

  140. [168]

    2023 , eprint=

    G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model , author=. 2023 , eprint=

    2023

  141. [169]

    and Cogswell, Michael and Das, Abhishek and Vedantam, Ramakrishna and Parikh, Devi and Batra, Dhruv , year=

    Selvaraju, Ramprasaath R. and Cogswell, Michael and Das, Abhishek and Vedantam, Ramakrishna and Parikh, Devi and Batra, Dhruv , year=. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization , url=. doi:10.1109/iccv.2017.74 , booktitle=

    doi

  142. [170]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , month =

    Huang, Xinyu and Cheng, Xinjing and Geng, Qichuan and Cao, Binbin and Zhou, Dingfu and Wang, Peng and Lin, Yuanqing and Yang, Ruigang , title =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , month =

  143. [171]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    nuscenes: A multimodal dataset for autonomous driving , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  144. [172]

    arXiv preprint arXiv:2302.00673 , year=

    Adapt: Action-aware driving caption transformer , author=. arXiv preprint arXiv:2302.00673 , year=

    arXiv

  145. [173]

    Proceedings of the European conference on computer vision (ECCV) , pages=

    Textual explanations for self-driving vehicles , author=. Proceedings of the European conference on computer vision (ECCV) , pages=

  146. [174]

    Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

    DRAMA: Joint Risk Localization and Captioning in Driving , author=. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

  147. [175]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Grounding human-to-vehicle advice for self-driving vehicles , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  148. [176]

    Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

    Bleu: a method for automatic evaluation of machine translation , author=. Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

  149. [177]

    2006 , publisher=

    Mastering regular expressions , author=. 2006 , publisher=

    2006

  150. [178]

    2023 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Bevfusion: Multi-task multi-sensor fusion with unified bird's-eye view representation , author=. 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2023 , organization=

    2023

  151. [179]

    arXiv preprint arXiv:2306.00890 , year=

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day , author=. arXiv preprint arXiv:2306.00890 , year=

    arXiv

  152. [180]

    See https://vicuna

    Vicuna: An open-source chatbot impressing gpt-4 with 90\ author=. See https://vicuna. lmsys. org (accessed 14 April 2023) , year=

    2023

  153. [181]

    arXiv preprint arXiv:2303.03378 , year=

    Palm-e: An embodied multimodal language model , author=. arXiv preprint arXiv:2303.03378 , year=

    arXiv

  154. [182]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Imagebind: One embedding space to bind them all , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  155. [183]

    arXiv preprint arXiv:2306.15195 , year=

    Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic , author=. arXiv preprint arXiv:2306.15195 , year=

    arXiv

  156. [184]

    arXiv preprint arXiv:2306.14824 , year=

    Kosmos-2: Grounding Multimodal Large Language Models to the World , author=. arXiv preprint arXiv:2306.14824 , year=

    arXiv

  157. [185]

    arXiv preprint arXiv:2010.11929 , year=

    An image is worth 16x16 words: Transformers for image recognition at scale , author=. arXiv preprint arXiv:2010.11929 , year=

    2010 arXiv

  158. [186]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Swin transformer: Hierarchical vision transformer using shifted windows , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  159. [187]

    Computational Visual Media , volume=

    Pvt v2: Improved baselines with pyramid vision transformer , author=. Computational Visual Media , volume=. 2022 , publisher=

    2022

  160. [188]

    Proceedings of the IEEE , year=

    Object detection in 20 years: A survey , author=. Proceedings of the IEEE , year=

  161. [189]

    OpenAI blog , volume=

    Language models are unsupervised multitask learners , author=. OpenAI blog , volume=

  162. [190]

    The Journal of Machine Learning Research , volume=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. The Journal of Machine Learning Research , volume=. 2020 , publisher=

    2020

  163. [191]

    arXiv preprint arXiv:2305.18279 , year=

    Contextual Object Detection with Multimodal Large Language Models , author=. arXiv preprint arXiv:2305.18279 , year=

    arXiv

  164. [192]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

    Agent-centric risk assessment: Accident anticipation and risky region localization , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

  165. [193]

    2019 International Conference on Robotics and Automation (ICRA) , pages=

    Goal-oriented object importance estimation in on-road driving videos , author=. 2019 International Conference on Robotics and Automation (ICRA) , pages=. 2019 , organization=

    2019

  166. [194]

    Proceedings of the ieee conference on computer vision and pattern recognition workshops , pages=

    Dr (eye) ve: a dataset for attention-based tasks with applications to autonomous and assisted driving , author=. Proceedings of the ieee conference on computer vision and pattern recognition workshops , pages=

  167. [195]

    2018 21st International Conference on Intelligent Transportation Systems (ITSC) , pages=

    Learning to attend to salient targets in driving videos using fully convolutional rnn , author=. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) , pages=. 2018 , organization=

    2018

  168. [196]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Titan: Future forecast using action priors , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  169. [197]

    Proceedings of the IEEE international conference on computer vision , pages=

    Interpretable learning for self-driving cars by visualizing causal attention , author=. Proceedings of the IEEE international conference on computer vision , pages=

  170. [198]

    2019 International Conference on Robotics and Automation (ICRA) , pages=

    Deep object-centric policies for autonomous driving , author=. 2019 International Conference on Robotics and Automation (ICRA) , pages=. 2019 , organization=

    2019

  171. [199]

    2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=

    Who make drivers stop? towards driver-centric risk assessment: Risk object identification via causal inference , author=. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=. 2020 , organization=

    2020

  172. [200]

    Advances in neural information processing systems , volume=

    Pytorch: An imperative style, high-performance deep learning library , author=. Advances in neural information processing systems , volume=

  173. [201]

    Talk2Car: Taking Control of Your Self-Driving Car , author=. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages=

    2019

  174. [202]

    , TITLE =

    Simons-Morton, Bruce and Ehsani, Johnathon P. , TITLE =. Safety , VOLUME =. 2016 , NUMBER =

    2016

  175. [203]

    1989 , publisher=

    A Guide to the SQL Standard , author=. 1989 , publisher=

    1989

  176. [204]

    Object Referring in Videos with Language and Human Gaze , journal=

    Vasudevan, ArunBalajee and Dai, Dengxin and Gool, LucVan , year=. Object Referring in Videos with Language and Human Gaze , journal=

  177. [205]

    arXiv preprint arXiv:2309.04379 , year=

    Language prompt for autonomous driving , author=. arXiv preprint arXiv:2309.04379 , year=

    arXiv

  178. [206]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Referring Multi-Object Tracking , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  179. [207]

    arXiv preprint arXiv:2303.11403 , year=

    eP-ALM: Efficient Perceptual Augmentation of Language Models , author=. arXiv preprint arXiv:2303.11403 , year=

    arXiv

  180. [208]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Planning-oriented autonomous driving , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  181. [209]

    arXiv preprint arXiv:2106.08417 , volume=

    Scene transformer: A unified multi-task model for behavior prediction and planning , author=. arXiv preprint arXiv:2106.08417 , volume=

    arXiv

  182. [210]

    arXiv preprint arXiv:2305.11175 , year=

    Visionllm: Large language model is also an open-ended decoder for vision-centric tasks , author=. arXiv preprint arXiv:2305.11175 , year=

    arXiv

  183. [211]

    European conference on computer vision , pages=

    End-to-end object detection with transformers , author=. European conference on computer vision , pages=. 2020 , organization=

    2020

  184. [212]

    arXiv preprint arXiv:1608.03983 , year=

    Sgdr: Stochastic gradient descent with warm restarts , author=. arXiv preprint arXiv:1608.03983 , year=

    arXiv

  185. [213]

    Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages=

    Multi-view 3d object detection network for autonomous driving , author=. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages=

  186. [214]

    Proceedings of robotics: science and systems IV, Zurich, Switzerland , volume=

    Model based vehicle tracking for autonomous driving in urban environments , author=. Proceedings of robotics: science and systems IV, Zurich, Switzerland , volume=

  187. [215]

    2019 International Conference on Robotics and Automation (ICRA) , pages=

    DeepSignals: Predicting intent of drivers through visual signals , author=. 2019 International Conference on Robotics and Automation (ICRA) , pages=. 2019 , organization=

    2019

  188. [216]

    Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages=

    Fast and furious: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net , author=. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages=

  189. [217]

    Conference on Robot Learning , pages=

    Intentnet: Learning to predict intention from raw sensor data , author=. Conference on Robot Learning , pages=. 2018 , organization=

    2018

  190. [218]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    X3d: Expanding architectures for efficient video recognition , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  191. [219]

    arXiv preprint arXiv:2106.09685 , year=

    Lora: Low-rank adaptation of large language models , author=. arXiv preprint arXiv:2106.09685 , year=

    arXiv

  192. [220]

    arXiv preprint arXiv:2112.11790 , year=

    Bevdet: High-performance multi-camera 3d object detection in bird-eye-view , author=. arXiv preprint arXiv:2112.11790 , year=

    arXiv

  193. [221]

    2024 , publisher=

    Lmms-eval: Accelerating the development of large multimoal models , author=. 2024 , publisher=

    2024

  194. [222]

    Science China Information Sciences , volume=

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites , author=. Science China Information Sciences , volume=. 2024 , publisher=

    2024

  195. [223]

    arXiv preprint arXiv:2408.15542 , year=

    Kangaroo: A powerful video-language model supporting long-context video input , author=. arXiv preprint arXiv:2408.15542 , year=

    arXiv

  196. [224]

    arXiv preprint arXiv:2406.16852 , year=

    Long context transfer from language to vision , author=. arXiv preprint arXiv:2406.16852 , year=

    arXiv

  197. [225]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Vila: On pre-training for visual language models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  198. [226]

    arXiv preprint arXiv:2403.05530 , year=

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context , author=. arXiv preprint arXiv:2403.05530 , year=

    arXiv

  199. [227]

    2024 , eprint=

    PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning , author=. 2024 , eprint=

    2024

  200. [228]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    Xiao, Junbin and Shang, Xindi and Yao, Angela and Chua, Tat-Seng , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2021 , pages =

    2021

  201. [229]

    arXiv preprint arXiv:2208.02797 , year=

    Vision-centric bev perception: A survey , author=. arXiv preprint arXiv:2208.02797 , year=

    arXiv

  202. [230]

    Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XIV 16 , pages=

    Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d , author=. Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XIV 16 , pages=. 2020 , organization=

    2020

  203. [231]

    European conference on computer vision , pages=

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers , author=. European conference on computer vision , pages=. 2022 , organization=

    2022

  204. [232]

    arXiv preprint arXiv:2305.14836 , year=

    NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario , author=. arXiv preprint arXiv:2305.14836 , year=

    arXiv

  205. [233]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Holistic autonomous driving understanding by bird's-eye-view injected multi-modal large models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  206. [234]

    Department of Computer Science & Technology, Nanjing University, Tech

    Multi-instance learning: A survey , author=. Department of Computer Science & Technology, Nanjing University, Tech. Rep , volume=

  207. [235]

    arXiv preprint arXiv:2309.05186 , year=

    HiLM-D: Towards High-Resolution Understanding in Multimodal Large Language Models for Autonomous Driving , author=. arXiv preprint arXiv:2309.05186 , year=

    arXiv

  208. [236]

    arXiv preprint arXiv:2305.14167 , year=

    DetGPT: Detect What You Need via Reasoning , author=. arXiv preprint arXiv:2305.14167 , year=

    arXiv

  209. [237]

    Advances in Neural Information Processing Systems , volume=

    Flamingo: a visual language model for few-shot learning , author=. Advances in Neural Information Processing Systems , volume=

  210. [238]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    arXiv

  211. [239]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  212. [240]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Eyes wide shut? exploring the visual shortcomings of multimodal llms , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  213. [241]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Vcoder: Versatile vision encoders for multimodal large language models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  214. [242]

    The Twelfth International Conference on Learning Representations , year=

    Mitigating hallucination in large multi-modal models via robust instruction tuning , author=. The Twelfth International Conference on Learning Representations , year=

  215. [243]

    Advances in Neural Information Processing Systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in Neural Information Processing Systems , volume=

  216. [244]

    Advances in Neural Information Processing Systems , volume=

    Flashattention: Fast and memory-efficient exact attention with io-awareness , author=. Advances in Neural Information Processing Systems , volume=

  217. [245]

    NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following , year=

    Analyzing and Mitigating Object Hallucination in Large Vision-Language Models , author=. NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following , year=

    2023

  218. [246]

    Science China Information Sciences , volume=

    Woodpecker: Hallucination correction for multimodal large language models , author=. Science China Information Sciences , volume=. 2024 , publisher=

    2024

  219. [247]

    arXiv preprint arXiv:2410.10818 , year=

    TemporalBench: Towards Fine-grained Temporal Understanding for Multimodal Video Models , author=. arXiv preprint arXiv:2410.10818 , year=

    arXiv

  220. [248]

    2024 , eprint=

    Video Instruction Tuning With Synthetic Data , author=. 2024 , eprint=

    2024

  221. [249]

    Proceedings of EMNLP 2024 , year=

    mDPO: Conditional Preference Optimization for Multimodal Large Language Models , author=. Proceedings of EMNLP 2024 , year=

    2024

  222. [250]

    arXiv preprint arXiv:2501.16629 , year=

    CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs , author=. arXiv preprint arXiv:2501.16629 , year=

    arXiv

  223. [251]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Improved baselines with visual instruction tuning , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  224. [252]

    European Conference on Computer Vision , pages=

    Clip-dpo: Vision-language models as a source of preference for fixing hallucinations in lvlms , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  225. [253]

    arXiv preprint arXiv:2404.01258 , year=

    Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward , author=. arXiv preprint arXiv:2404.01258 , year=

    arXiv

  226. [254]

    arXiv preprint arXiv:2402.00253 , year=

    A survey on hallucination in large vision-language models , author=. arXiv preprint arXiv:2402.00253 , year=

    arXiv

  227. [255]

    LLaVA-NeXT: Improved reasoning, OCR, and world knowledge , url=

    Liu, Haotian and Li, Chunyuan and Li, Yuheng and Li, Bo and Zhang, Yuanhan and Shen, Sheng and Lee, Yong Jae , month=. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge , url=

  228. [256]

    Proceedings of the 32nd ACM International Conference on Multimedia , pages=

    Self-supervised visual preference alignment , author=. Proceedings of the 32nd ACM International Conference on Multimedia , pages=

  229. [257]

    arXiv preprint arXiv:2405.14734 , year=

    Simpo: Simple preference optimization with a reference-free reward , author=. arXiv preprint arXiv:2405.14734 , year=

    arXiv

  230. [258]

    European Conference on Computer Vision , pages=

    Strengthening multimodal large language model with bootstrapped preference optimization , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  231. [259]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  232. [260]

    arXiv preprint arXiv:2307.12981 , year=

    3d-llm: Injecting the 3d world into large language models , author=. arXiv preprint arXiv:2307.12981 , year=

    arXiv

  233. [261]

    Microsoft Auton

    Chatgpt for robotics: Design principles and model abilities , author=. Microsoft Auton. Syst. Robot. Res , volume=

  234. [262]

    arXiv preprint arXiv:2312.10665 , year=

    Silkie: Preference distillation for large visual language models , author=. arXiv preprint arXiv:2312.10665 , year=

    arXiv

  235. [263]

    arXiv preprint arXiv:2405.17220 , year=

    RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness , author=. arXiv preprint arXiv:2405.17220 , year=

  236. [264]

    https://openai.com/index/hello-gpt-4o/ , author=

    Hello gpt-4o. https://openai.com/index/hello-gpt-4o/ , author=

  237. [265]

    arXiv preprint arXiv:2310.01412 , year=

    DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model , author=. arXiv preprint arXiv:2310.01412 , year=

    arXiv

  238. [266]

    arXiv preprint arXiv:2310.01415 , year=

    Gpt-driver: Learning to drive with gpt , author=. arXiv preprint arXiv:2310.01415 , year=

    arXiv

  239. [267]

    arXiv preprint arXiv:2310.02251 , year=

    Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving , author=. arXiv preprint arXiv:2310.02251 , year=

    arXiv

  240. [268]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Adaptive keyframe sampling for long video understanding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  241. [269]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Revealing single frame bias for video-and-language learning , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  242. [270]

    arXiv preprint arXiv:2511.21631 , year=

    Qwen3-vl technical report , author=. arXiv preprint arXiv:2511.21631 , year=

    arXiv

  243. [271]

    arXiv preprint arXiv:2512.04643 , year=

    SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding , author=. arXiv preprint arXiv:2512.04643 , year=

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.