Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behavior

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that fine-tuning large multimodal models on fine-grained, dual-view driver-behavior descriptions and explanations transfers general visual models to driving understanding, improving PDB-QA zero-shot question answering by…

desk verdict A genuinely useful dual-view driving benchmark with a plausible transfer story; the load-bearing claims need a driver-disjoint split and a less circular evaluation before they hold. read the letter →

arxiv 2507.18447 v1 pith:BU2LD5FU submitted 2025-07-24 cs.CV

Junda Wu , Jessica Echterhoff , Kyungtae Han , Amr Abdelraouf , Rohit Gupta , Julian McAuley This is my paper
classification cs.CV
keywords PDB-Evallargemultimodalmodelspersonalizeddriverbehaviordescriptionandexplanationvisualquestionansweringdualdash-camvideoinstructionfine-tuningdrivingintentionprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that large multimodal models can be brought into the driving domain by training them on fine-grained, personalized descriptions of what a driver does and why. To do this it builds PDB-Eval, a benchmark with two parts: PDB-X, which pairs in-cabin driver-behavior descriptions with external-dashcam visual evidence explaining them, and PDB-QA, a question-answering task generated from those pairs. The paper reports that fine-tuning BLIP-2 and VTimeLLM on PDB-X improves zero-shot PDB-QA performance by up to 73.2%, and that the same models transfer to Brain4Cars turn-intention prediction and AIDE driving-recognition tasks without extra fine-tuning. If the claims hold, PDB-Eval would be a reusable intermediate-supervision resource that adds interpretable driver reasoning to general multimodal models.

What carries the argument

The load-bearing object is the PDB-Eval construction pipeline, whose key mechanism is comparative prompting: two drivers with the same intention are shown in concatenated internal-dashcam frames, and an MLLM must describe their behavioral differences, after which the outputs are filtered by identity and intention consistency, summarized into behavior-type guidelines, and reused as instructions to generate grounded descriptions and explanations from paired internal and external frames. Human filtering removes hallucinated and irrelevant aspects, and text-only LLMs turn the surviving description-explanation pairs into PDB-QA questions and answers. This pipeline is what turns raw dual-cam video into the training signal claimed to transfer MLLMs to driving.

What would settle it

Give the same dual-cam clips to independent human experts to write driver-behavior descriptions and explanations from scratch; if the experts agree poorly with PDB-X, or if models fine-tuned on the expert-written labels perform worse than models fine-tuned on PDB-X, the claim that PDB-X teaches genuine driving comprehension is not supported.

Watch

Extended reading notes

Core claim

The central discovery is that descriptions and explanations of personalized driver behavior, built by comparing two drivers performing the same maneuver and then grounding each description in external-camera visual evidence, act as effective intermediate supervision for aligning MLLMs with driving. The paper constructs PDB-X by having an MLLM contrast two drivers with the same intention, filtering the outputs for identity and intention consistency, summarizing the results into behavior-type guidelines, and prompting with those guidelines to generate fine-grained descriptions plus external-view explanations; human annotators then filter hallucinated or irrelevant samples. On PDB-QA, the fine-tuned models beat both their pre-trained zero-shot performance and GPT-4V's zero-shot n-gram overlap metric, and the same fine-tuned models improve turn-intention prediction on Brain4Cars and driver, emotion, traffic, and vehicle recognition on AIDE without additional fine-tuning. In the paper's own framing, this shows that a generic generative learning task can bridge the driving domain gap without sacrificing generalizability.

Load-bearing premise

The labels are assumed to be correct because a computer model generated them and people filtered them, with no independent check that the explanations match what really happened on the road.

Editorial extensions

If this is right

  • Fine-tuned MLLMs can produce textual descriptions of driver behavior grounded in external visual evidence, a capability that plain zero-shot prompting does not provide.
  • PDB-QA can serve as an instruction-following learning task that improves MLLMs on driving question answering while preserving their generality.
  • The gains on Brain4Cars turn intention suggest personalized behavior descriptions are useful textual evidence for predicting imminent maneuvers even without vehicle speed or GPS.
  • The cross-domain gains on AIDE indicate that PDB-X supervision transfers to driver emotion, traffic context, and vehicle condition recognition.
  • The remaining gap in BLEU-4 scores shows that fine-grained, temporally aware driving description is still unsolved, so PDB-X can continue serving as an evaluation target.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the benchmark labels were replaced by independent expert annotations and the same training recipe still improved downstream tasks, the central claim would be much stronger; the paper does not make that comparison.
  • Comparative prompting could be exported to other paired-view settings, such as operator monitoring in control rooms or human-robot collaboration, where an internal behavior stream must be explained by external scene evidence.
  • The reported gains might partly reflect learning guideline-specific language style rather than causal reasoning; a test that swaps or perturbs external frames during inference would separate language mimicry from genuine evidence grounding.
  • The pipeline suggests a data flywheel for safety systems: each new driver's behavior can be described and explained by an MLLM, filtered by humans, and folded back into fine-tuning for personalized driver models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces PDB-Eval, a benchmark built from the Brain4Cars dual-camera driving dataset, with two tasks: PDB-X, which asks MLLMs to produce fine-grained descriptions of driver behavior and explanations grounded in external-view visual evidence, and PDB-QA, a visual-explanation question-answering task used for instruction fine-tuning. The data are generated through a multi-step pipeline that includes comparative prompting with GPT-4V, identity/intention consistency filtering, guideline construction with a text-only LLM, and human annotator filtering. The authors fine-tune BLIP-2 and VTimeLLM on PDB-X and report BLEU-4 results on PDB-X and PDB-QA, as well as downstream results on Brain4Cars turn-intention prediction and AIDE recognition tasks, claiming up to a 73.2% improvement on PDB-QA and up to 12.5% and 11.0% improvements on the downstream tasks.

Significance. If the reported improvements genuinely arise from the PDB-Eval supervision, the benchmark would fill a real gap: it targets fine-grained, temporally aligned reasoning across internal and external driving views, and it provides a concrete recipe for using comparative prompting and human filtering to generate instruction data for MLLMs. The inclusion of external benchmarks (Brain4Cars and AIDE) is a strength, because it tests transfer beyond the benchmark's own reference distribution. However, the significance is currently conditional on resolving three load-bearing concerns: potential driver leakage between the training and test splits, the circularity of measuring gains against GPT-4V/LLM-generated references, and the absence of statistical reliability measures. The benchmark and pipeline are valuable as a proposal, but the central claim that PDB-Eval 'bridges the gap' is not yet established by the evidence as presented.

major comments (4)
  1. [Section IV, Table II; Section III-A] The paper does not establish that the PDB-Eval train/test split is driver-disjoint. Section III-A states that Brain4Cars was collected from only 10 drivers, and Table II reports 478 train and 116 test clips, but no driver IDs or split guarantees are given. If clips from the same drivers appear on both sides, the 73.2% PDB-QA gain in Table IV and the 12.5% Brain4Cars turn-intention gain in Table V could be inflated by memorizing driver-specific appearance or behavior rather than by learning generalizable visual evidence. The authors should report the driver distribution per split and provide a leave-drivers-out evaluation (e.g., training on 8-9 drivers and testing on the held-out drivers) before attributing the gains to driving comprehension.
  2. [Section III-E; Section IV] The reference ground truth is generated by GPT-4V for descriptions/explanations and by LLMs for QA pairs, with human filtering described only qualitatively. No inter-annotator agreement, number of annotators, or residual hallucination analysis is reported, so the filtering step is not empirically validated. Because PDB-QA answers are automatically generated from PDB-X texts (Section IV) and the fine-tuned models are trained to match this same generation style, the BLEU-4 improvements in Tables III and IV partly measure fit to the reference-generation distribution rather than semantic correctness. The authors should add a human evaluation of generated descriptions/explanations and/or validate a sample of explanations against independently annotated driver actions, and should make the annotation and verification protocol reproducible.
  3. [Section V-C, Table IV; Section V-D, Tables V-VI] All reported improvements are single point estimates with no error bars, confidence intervals, or significance tests. With only 116 test clips and 10 underlying drivers, differences such as the PDB-QA gain of 48.61 vs. 28.07 for VTimeLLM in Table IV could be within run-to-run or driver-sampling variability. Additionally, the downstream evaluation protocol in Section V-D is underspecified: it is not stated how the fine-tuned MLLMs' textual evidence is combined with the video features to produce the precision/recall numbers in Table V and the accuracy/F1 numbers in Table VI, nor whether any classifier was trained on the AIDE/Brain4Cars splits. The authors should report multiple seeds, bootstrap confidence intervals, and a precise protocol for the downstream tasks.
  4. [Abstract; Section V-C, Table IV] The abstract's claim that fine-tuning 'improves zero-shot performance on question-answering tasks by up to 73.2%' is ambiguous and potentially misleading. In Table IV, the 73.2% improvement corresponds to VTimeLLM's fine-tuned BLEU-4 (48.61) relative to its pre-trained baseline (28.07), not to GPT-4V's zero-shot score (16.08). The abstract should clarify the baseline (pre-trained model vs. GPT-4V zero-shot) and state whether improvements are relative or absolute, since different choices lead to very different conclusions about the method's benefit.
minor comments (5)
  1. [Throughout] The notation is inconsistent: 'MLLM' and 'LMM' are used interchangeably (e.g., Figure 3 caption and Section V-D), and the same model families are sometimes called 'LMMs' and 'MLLMs' in the same paragraph. Please unify the terminology.
  2. [Table II] The entry 'T rain' contains a typo and should read 'Train'.
  3. [Section IV] The description of pair sampling is unclear: 'we sampled 20 pairs of drivers from each annotated driving intention subset' is difficult to reconcile with the fact that Brain4Cars contains only 10 drivers. Please clarify whether pairs can reuse drivers and how the 20 pairs per intention were constructed.
  4. [Section V-A] The video preprocessing details are under-specified: for BLIP-2 and VTimeLLM, the exact frame extraction rates, the handling of variable-length clips, and the concatenation layout (vertical vs. horizontal) are not fully described, which limits reproducibility. Please provide exact preprocessing code or a precise algorithmic description.
  5. [General] The paper does not state whether the dataset, code, or fine-tuned model checkpoints will be released, nor does it give a download link or license information. For a benchmark paper, such artifacts are important for the community to verify and build on the results.

Circularity Check

2 steps flagged · score 6.0 of 10

PDB-QA is generated from PDB-X, so the headline 73.2% QA gain is partly a fit to a self-derived distribution; the Brain4Cars turn-intention gain uses PDB-X text built from the same intention labels, making that downstream result non-independent. AIDE provides independent cross-domain support.

  1. self definitional [Section IV (PDB-Eval construction, after Table II) and Section V-C (PDB-QA evaluation)]
    "Based on the descriptions and explanations from PDB-X, we can enable automatic question-answer pair generation using text-only LLMs [21], [22]. ... We further evaluate MLLMs on PDB-QA to showcase their capacities in answering complex questions related to personalized driving behavior."

    PDB-QA ground-truth answers are algorithmically generated from the PDB-X description/explanation texts. Fine-tuning on PDB-X therefore trains the model to produce text in the same style and content distribution that PDB-QA references are drawn from. The headline 73.2% relative improvement (VTimeLLM PT 28.07 to FT 48.61, Table IV) is thus substantially a fit to a self-derived reference distribution, not independent evidence that the model has acquired general driving-domain comprehension.

  2. fitted input called prediction [Section III-A, III-C and Section V-D1 (Table V)]
    "we augment dual-cam videos from the Brain4Cars [1] dataset, with personalized driver behavior descriptions and explanations. ... we validate if the generated intentions ITu, ITv are also the same. ... The BLIP-2 and VTimeLLM columns are obtained with textual evidence output from LMMs fine-tuned on PDB-X."

    PDB-X is constructed from the same Brain4Cars clips, and its creation pipeline uses Brain4Cars ground-truth intention labels in the consistency filter (Section III-C); PDB-X also includes INT (intention) as one of the nine behavior types (Table I). Fine-tuning on PDB-X therefore conditions the text generator on Brain4Cars intention labels. Feeding this text as 'textual evidence' into the Brain4Cars turn intention prediction task (Table V) evaluates the target label with an input derived from that same label, so the reported 12.5% improvement is not an independent test of generalizable driving comprehension.

full rationale

Two load-bearing evaluation steps reduce to construction. The 73.2% PDB-QA gain is measured on QA pairs generated from PDB-X, so fine-tuning on PDB-X inflates the score by construction. The Brain4Cars turn-intention improvement uses textual evidence from models fine-tuned on PDB-X, which was built from Brain4Cars clips with intention-consistency filtering and includes intention as a behavior type, making the target labels part of the training signal rather than an external check. The AIDE results (Table VI) use a genuinely separate dataset and provide independent evidence for cross-domain transfer; no self-citation chain is load-bearing. The paper's central claim has independent content, but the headline and in-domain gains are overstated by these non-independent evaluations.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the quality of the generated annotation dataset, the hand-chosen preprocessing (frame counts, pair counts), and the validity of BLEU-4 as the evaluation metric. There are no fitted physical parameters or newly postulated entities.

free parameters (2)
  • Temporal frame sampling count N = 10 frames per camera view
    Models receive 10 frames per camera per clip (Section V-A). The number is hand-chosen and affects the temporal information available to the models and the resulting scores.
  • Comparative prompting pair count per intention = 20 pairs per intention (100 pairs total)
    The seed for guideline construction samples 20 driver pairs per intention (Section IV). This determines the diversity of the resulting prompting guidelines.
assumptions (4)
  • domain assumption Human annotator filtering is sufficient to remove hallucinated driver behavior descriptions from the MLLM-generated references.
    The benchmark's ground truth is GPT-4V output filtered by humans (Section III-E). If residual hallucinations pass the filter, the reference text is not a valid target for behavior understanding.
  • domain assumption Brain4Cars intention labels are reliable ground truth for intention filtering and evaluation.
    The pipeline filters samples using the original Brain4Cars annotations (Section III-C), and Table V uses those labels as evaluation targets.
  • domain assumption Internal and external dashcam frames are temporally synchronized so the same frame index refers to the same moment in both views.
    The explanation task assumes external evidence at a given frame can explain the driver's behavior in the same frame (Section III-D). No synchronization analysis is provided.
  • domain assumption BLEU-4 is a meaningful measure of description and explanation quality.
    BLEU-4 measures n-gram overlap with a reference and is weakly correlated with semantic correctness, but it is used as the primary metric for PDB-X and PDB-QA without calibration against human judgment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behavior." pith.science (2026). https://pith.science/paper/BU2LD5FU

@misc{pith2026250718447,
  author       = {Pith},
  title        = {Pith review of: PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behavior},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BU2LD5FU}},
  note         = {Machine review of arXiv:2507.18447}
}
read the original abstract

Understanding a driver's behavior and intentions is important for potential risk assessment and early accident prevention. Safety and driver assistance systems can be tailored to individual drivers' behavior, significantly enhancing their effectiveness. However, existing datasets are limited in describing and explaining general vehicle movements based on external visual evidence. This paper introduces a benchmark, PDB-Eval, for a detailed understanding of Personalized Driver Behavior, and aligning Large Multimodal Models (MLLMs) with driving comprehension and reasoning. Our benchmark consists of two main components, PDB-X and PDB-QA. PDB-X can evaluate MLLMs' understanding of temporal driving scenes. Our dataset is designed to find valid visual evidence from the external view to explain the driver's behavior from the internal view. To align MLLMs' reasoning abilities with driving tasks, we propose PDB-QA as a visual explanation question-answering task for MLLM instruction fine-tuning. As a generic learning task for generative models like MLLMs, PDB-QA can bridge the domain gap without harming MLLMs' generalizability. Our evaluation indicates that fine-tuning MLLMs on fine-grained descriptions and explanations can effectively bridge the gap between MLLMs and the driving domain, which improves zero-shot performance on question-answering tasks by up to 73.2%. We further evaluate the MLLMs fine-tuned on PDB-X in Brain4Cars' intention prediction and AIDE's recognition tasks. We observe up to 12.5% performance improvements on the turn intention prediction task in Brain4Cars, and consistent performance improvements up to 11.0% on all tasks in AIDE.

Figures

Figures reproduced from arXiv: 2507.18447 by the authors.

Figure 1
Figure 1. Existing vehicle movement understanding ( [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the pipeline for creating a dataset that captures drivers’ personalized behavior characteristics, including five [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A comparative example of comparative prompted description and guideline instruction prompted description. The context [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Illustration of three tasks for drivers’ personalized behaviors understanding. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DriveDNA: A Large-Scale Multimodal Naturalistic Driving Dataset and Benchmark for Driving Style Identification

    cs.LG 2026-07 conditional novelty 7.0 of 10

    A multi-vehicle naturalistic benchmark finds learned driving embeddings retain driver identity under condition matching, while descriptors collapse and video re-ID is mostly route leakage.

Reference graph

Works this paper leans on

23 extracted references · 9 canonical work pages · cited by 1 Pith paper

  1. [1]

    Brain4cars: Car that knows before you do via sensory-fusion deep learning architecture,

    A. Jain, H. S. Koppula, S. Soh, B. Raghavan, A. Singh, and A. Saxena, “Brain4cars: Car that knows before you do via sensory-fusion deep learning architecture,” arXiv preprint arXiv:1601.00740, 2016

    arXiv 2016

  2. [2]

    Aide: A vision-driven multi- view, multi-modal, multi-tasking dataset for assistive driving perception,

    D. Yang, S. Huang, Z. Xu, Z. Li, S. Wang, M. Li, Y. Wang, Y. Liu, K. Yang, Z. Chenet al., “Aide: A vision-driven multi- view, multi-modal, multi-tasking dataset for assistive driving perception,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 20459–20470

    work page 2023

  3. [3]

    Textual explanations for self-driving vehicles,

    J. Kim, A. Rohrbach, T. Darrell, J. Canny, and Z. Akata, “Textual explanations for self-driving vehicles,”Proceedings of the European Conference on Computer Vision (ECCV) , 2018

    work page 2018

  4. [4]

    Vtimellm: Empower llm to grasp video moments,

    B. Huang, X. Wang, H. Chen, Z. Song, and W. Zhu, “Vtimellm: Empower llm to grasp video moments,”arXiv preprint arXiv:2311.18445, 2023

    arXiv 2023

  5. [5]

    Personalized multimodal large language models: A survey,

    J. Wu, H. Lyu, Y. Xia, Z. Zhang, J. Barrow, I. Kumar, M. Mirtaheri, H. Chen, R. A. Rossi, F. Dernoncourtet al., “Personalized multimodal large language models: A survey,” arXiv preprint arXiv:2412.02142, 2024

    arXiv 2024

  6. [6]

    Visual prompting in multimodallargelanguagemodels:Asurvey,

    J. Wu, Z. Zhang, Y. Xia, X. Li, Z. Xia, A. Chang, T. Yu, S. Kim, R. A. Rossi, R. Zhanget al., “Visual prompting in multimodallargelanguagemodels:Asurvey,” arXiv preprint arXiv:2409.15310, 2024

    arXiv 2024

  7. [7]

    Mvbench: A comprehen- sive multi-modal video understanding benchmark,

    K. Li, Y. Wang, Y. He, Y. Li, Y. Wang, Y. Liu, Z. Wang, J. Xu, G. Chen, P. Luo et al. , “Mvbench: A comprehen- sive multi-modal video understanding benchmark,” arXiv preprint arXiv:2311.17005, 2023

    arXiv 2023

  8. [8]

    Large language models are temporal and causal reasoners for video questionanswering,

    D. Ko, J. S. Lee, W. Kang, B. Roh, and H. J. Kim, “Large language models are temporal and causal reasoners for video questionanswering,” arXiv preprint arXiv:2310.15747,2023

    arXiv 2023

Show all 23 references
  1. [9]

    List items one by one: A new data source and learning paradigm for multimodal llms,

    A. Yan, Z. Yang, J. Wu, W. Zhu, J. Yang, L. Li, K. Lin, J. Wang, J. McAuley, J. Gaoet al., “List items one by one: A new data source and learning paradigm for multimodal llms,” arXiv preprint arXiv:2404.16375, 2024

    2024 arXiv

  2. [10]

    Uncertainty-basedtrafficacci- dent anticipation with spatio-temporal relational learning,

    W.Bao,Q.Yu,andY.Kong,“Uncertainty-basedtrafficacci- dent anticipation with spatio-temporal relational learning,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 2682–2690

    2020

  3. [11]

    Cocot: Contrastive chain-of-thought prompting for large multimodal models with multiple image inputs,

    D. Zhang, J. Yang, H. Lyu, Z. Jin, Y. Yao, M. Chen, and J. Luo, “Cocot: Contrastive chain-of-thought prompting for large multimodal models with multiple image inputs,”arXiv preprint arXiv:2401.02582, 2024

    2024 arXiv

  4. [12]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Has- son, K. Lenc, A. Mensch, K. Millican, M. Reynoldset al., “Flamingo: a visual language model for few-shot learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 23716–23736, 2022

    2022

  5. [13]

    Dipnet: Driver intention prediction for a safe takeover tran- sition in autonomous vehicles,

    M. Bonyani, M. Rahmanian, S. Jahangard, and M. Rezaei, “Dipnet: Driver intention prediction for a safe takeover tran- sition in autonomous vehicles,” IET Intelligent Transport Systems, 2023

    2023

  6. [14]

    A study of situational reasoning for traffic understanding,

    J. Zhang, F. Ilievski, K. Ma, A. Kollaa, J. Francis, and A. Oltramari, “A study of situational reasoning for traffic understanding,” arXiv preprint arXiv:2306.02520, 2023

    2023 arXiv

  7. [15]

    Driving through the concept gridlock: Unraveling explainability bottlenecks in automated driving,

    J. Echterhoff, A. Yan, K. Han, A. Abdelraouf, R. Gupta, and J. McAuley, “Driving through the concept gridlock: Unraveling explainability bottlenecks in automated driving,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 7346–7355

    2024

  8. [16]

    Drive like a human: Rethinking autonomous driving with large language models,

    D. Fu, X. Li, L. Wen, M. Dou, P. Cai, B. Shi, and Y. Qiao, “Drive like a human: Rethinking autonomous driving with large language models,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 910–919

    2024

  9. [17]

    Drivegpt4: Interpretable end-to-end au- tonomous driving via large language model,

    Z. Xu, Y. Zhang, E. Xie, Z. Zhao, Y. Guo, K. K. Wong, Z. Li, and H. Zhao, “Drivegpt4: Interpretable end-to-end au- tonomous driving via large language model,”arXiv preprint arXiv:2310.01412, 2023

    2023 arXiv

  10. [18]

    Gollie: Annotation guidelines improve zero-shot information-extraction,

    O. Sainz, I. García-Ferrero, R. Agerri, O. L. de Lacalle, G. Rigau, and E. Agirre, “Gollie: Annotation guidelines improve zero-shot information-extraction,” arXiv preprint arXiv:2310.03668, 2023

    2023 arXiv

  11. [19]

    A survey on halluci- nation in large vision-language models,

    H. Liu, W. Xue, Y. Chen, D. Chen, X. Zhao, K. Wang, L. Hou, R. Li, and W. Peng, “A survey on halluci- nation in large vision-language models,” arXiv preprint arXiv:2402.00253, 2024

    2024 arXiv

  12. [20]

    Analyzing and mitigating ob- ject hallucination in large vision-language models,

    Y. Zhou, C. Cui, J. Yoon, L. Zhang, Z. Deng, C. Finn, M. Bansal, and H. Yao, “Analyzing and mitigating ob- ject hallucination in large vision-language models,” arXiv preprint arXiv:2310.00754, 2023

    2023 arXiv

  13. [21]

    From images to textual prompts: Zero-shot visual question answering with frozen large language models,

    J. Guo, J. Li, D. Li, A. M. H. Tiong, B. Li, D. Tao, and S. Hoi, “From images to textual prompts: Zero-shot visual question answering with frozen large language models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10867–10877

    2023

  14. [22]

    Generalizing visual question answering from synthetic to human-written questions via a chain of qa with a large language model,

    T. Kim, Y. Cho, H. Shin, Y. Jo, and D. Shin, “Generalizing visual question answering from synthetic to human-written questions via a chain of qa with a large language model,” arXiv preprint arXiv:2401.06400, 2024

    2024 arXiv

  15. [23]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” arXiv preprint arXiv:2301.12597 , 2023

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.