Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Facial Dynamics in Video: Instruction Tuning for Improved Facial Expression Perception and Contextual Awareness

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces dynamic facial expression captioning with a 5,033-clip instruction dataset, a face-tracking multimodal model, a benchmark, and an event-order-aware metric, and claims the model beats every compared method on the…

desk verdict The DFEC task and FDA dataset are the real contribution; the Table 1 SOTA claim is confounded by fine-tuning and needs rework. read the letter →

arxiv 2501.07978 v1 pith:734JDURI submitted 2025-01-14 cs.CV cs.AI

Jiaxing Zhao , Boyuan Sun , Xiang Chen , Xihan Wei This is my paper · ORCID
classification cs.CVcs.AI
keywords dynamicfacialexpressioncaptioningvideomultimodallargelanguagemodelsinstructiontuningfacetrackingtemporaleventmatchingbenchmarkmulti-personunderstandingLoRAfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video multimodal large language models can describe what happens in a clip, but they under-describe faces because every frame is compressed into a small number of visual tokens. The paper's proposal is that this can be fixed by giving the model a dedicated, tracked view of the main character's face: it introduces a new task, dynamic facial expression captioning; a 5,033-clip manually annotated instruction dataset (FDA); and FaceTrack-MM, a model that adds a face-tracking module and a face-specific encoder on top of an existing video MLLM. It also contributes FEC-Bench, a 1,000-clip benchmark, and a metric, TEM, that scores both event content and event order. The paper's central claim is that FaceTrack-MM outperforms all 15 compared open-source and proprietary models on FEC-Bench, with the largest gains on lexical and event-based metrics. If true, this gives downstream applications a compact recipe for turning video MLLMs into reliable, temporally ordered describers of facial behavior.

What carries the argument

The central mechanism is the dynamic video face tracking module. It downsamples the video to 16 fps, runs face detection (DaMOFD) and face-feature extraction (TransFace), links detections into trajectories with StrongSORT, and uses K-means on trajectory area and internal cosine similarity to pick the main character's trajectory; the chosen face is then encoded by FaceXFormer and projected into the LLM as extra tokens. The paper's other mechanism is the Temporal Event Matching (TEM) metric, which extracts face-related events with ChatGPT, classifies each generated-versus-reference event pair as Same Meaning, Opposite Meaning, or No Relation, and averages an event F-measure with a longest-common-subsequence score on the ordered event lists.

What would settle it

Construct caption pairs that describe the same facial events in the same order but use synonymous wording, such as 'his eyes widen' versus 'his eyes become wide', run TEM on them, and check whether the order component stays unchanged; if the score drops, the metric is measuring surface wording rather than event order.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that the bottleneck in facial-expression captioning is visual-token allocation rather than language understanding, and that the bottleneck is removable. FaceTrack-MM takes the VideoLLaMA2 architecture and inserts a dynamic face pathway: frames are downsampled to 16 fps, faces are detected and tracked into trajectories, K-means clustering separates the main character's trajectory from background faces, and FaceXFormer encodes that trajectory into compact 'facial prior' tokens that enter the LLM alongside the usual CLIP/STC video tokens. After LoRA instruction tuning on FDA, the model records the best score in every column of FEC-Bench, including ChatGPT correctness, detail, context, and temporal scores, CIDEr, ROUGE-L, AutoDQ, and the proposed TEM. The ablation in the paper attributes the gains specifically to the trajectory-based face selection and to the specialized face encoder, with plain face-crop concatenation giving almost no improvement.

Load-bearing premise

The metric's event-order score assumes that ChatGPT extracts identical strings for the same facial event in the generated and reference captions; if the wording differs, the order match fails even when the events and their sequence are correct.

Editorial extensions

If this is right

  • A 7B open video MLLM fine-tuned only with LoRA can surpass GPT-4o and Claude-3.5-Sonnet on facial-expression captioning once the face pathway and instruction data are added.
  • The FDA dataset's separation of objective facial changes from marked subjective inferences offers a reusable annotation pattern for subtle video behavior.
  • TEM gives an automatic, LLM-based way to penalize caption errors that preserve content but scramble event order, which CIDEr, ROUGE-L, and AutoDQ do not do.
  • FaceTrack-MM's trajectory selection should let the model describe the intended person's face in multi-person video instead of blending or switching between faces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The face-pathway recipe likely transfers to other small-region video understanding tasks such as hand, gaze, or gesture description, but the paper does not test this.
  • TEM could be made more robust by feeding the relation-classification results into the longest-common-subsequence matching instead of requiring identical event strings; the paper does not do this.
  • Because the benchmark uses a random split of one annotated video distribution, the reported gains may partly reflect learning the annotation style, and cross-domain generalization remains untested.
  • Expanding the 5,033-clip manual annotation pipeline with model-assisted pre-labeling plus human correction is a natural next step toward scaling this task.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Dynamic Facial Expression Captioning (DFEC), a new task of generating natural-language descriptions of a main character's facial changes in video. It contributes the FDA dataset (5,033 manually annotated clips), the FaceTrack-MM model (VideoLLaMA2 augmented with a dynamic face-tracking module and a FaceXFormer facial encoder), the TEM evaluation metric (event extraction, relation classification, and LCS), and the FEC-Bench benchmark comparing 15 MLLMs. The central empirical claim is that FaceTrack-MM 'significantly outperforms all other methods' on FEC-Bench, and that its face-tracking design is responsible for the improvement.

Significance. If the claims were fully supported, the paper would make a useful contribution: a new dataset and benchmark for a task (DFEC) that existing MLLMs handle poorly, a modular architecture for injecting facial priors into video LLMs, and a metric that tries to capture both event content and temporal order. The dataset and benchmark are potentially reusable resources, and the ablation study (Table 2) is a good start toward understanding where gains come from. However, the headline comparison is confounded by in-distribution fine-tuning, and the TEM metric has an internal inconsistency between its stated relation-classification step and its actual LCS implementation. These issues do not invalidate the dataset or the architectural idea, but they do undermine the paper's main comparative and metric claims as currently presented.

major comments (4)
  1. [§5.2, Table 1 and Table 2] The headline comparison is confounded. FaceTrack-MM is fine-tuned on the FDA training split, while all 15 baselines in Table 1 are evaluated zero-shot. Table 2 shows that instruction tuning alone raises VideoLLaMA2's AutoDQ from 0.344 to 0.422 and TEM from 0.258 to 0.321, and the full face-tracking plus FaceXFormer pipeline adds only +0.061 AutoDQ and +0.043 TEM. Therefore, the large margins over zero-shot baselines in Table 1 mostly reflect in-distribution fine-tuning rather than the proposed face-tracking architecture. The claim in §5.2 that the method 'significantly outperforms all other methods on the FEC-Bench' is not supported without a comparably fine-tuned strong baseline (e.g., VideoLLaMA2 instruction-tuned on the same FDA training data) as the primary control.
  2. [§3.4, Algorithm 1] The TEM metric is internally inconsistent. The text describes relation classification into Same Meaning, Opposite Meaning, and No Relation, but Algorithm 1 line 14 computes LCS using exact string equality (EG[i-1] == ER[j-1]). ChatGPT-extracted event strings will rarely be identical when they carry the same meaning, so the LCS temporal-order component does not measure semantic order consistency as claimed. The F-measure computation in line 24 is also unspecified: the paper does not state how the relation classifications are converted into precision and recall. Without a precise matching rule, TEM's numerical values are not reproducible, and the temporal-order component should be either implemented via the relation classifications or clearly described as an exact-string proxy.
  3. [§5.2, Fig. 8] The ChatGPT-based reformatting of generated text before evaluation biases the benchmark in favor of the fine-tuned model. The paper states that all generated text is converted into a format similar to the reference 'without altering its content,' but no evidence is provided that content is preserved; for n-gram metrics such as CIDEr and ROUGE-L, even a paraphrase with the same meaning can change token overlap. Because FaceTrack-MM was fine-tuned on the reference style, it naturally matches the reformatted references better than zero-shot baselines. Please report scores both before and after reformatting, and show that the reformatting does not systematically change the relative ranking.
  4. [Abstract and §5.4] The claim of 'superior performance ... even in intricate multi-person scenarios' is not directly tested. No experiment isolates multi-person videos, and the qualitative example in Fig. 4 appears to be a single person. Please provide a breakdown of FEC-Bench results by number of detected faces per video, or a dedicated multi-person subset, to support this specific claim about multi-person robustness.
minor comments (5)
  1. [§3.4, Algorithm 1 line 21] The LCS score is normalized by m (the number of generated events), which is asymmetric and not justified; if a model generates many spurious events, the score is penalized even when all reference events are matched in order. Please state the normalization choice explicitly and consider reporting both order-aware recall and precision, or normalizing by max(m,n).
  2. [§4.2.2] The main-trajectory selection uses K-means with exactly two clusters, implicitly assuming that all videos contain only two roles (main and background). Many videos may have multiple salient characters; please justify this fixed choice or make the number of clusters adaptive.
  3. [Table 1] The table caption lists a 'Consistency' metric, but the table header shows only Correctness, Detail, Context, and Temporal. Please align the caption with the columns actually reported.
  4. [§3.2] The annotation pipeline involves three annotators and a final reviewer, but no inter-annotator agreement statistic is reported. A measure such as Cohen's kappa or a sample-level agreement rate would strengthen confidence in the dataset's quality.
  5. [Various] There are several typos and formatting inconsistencies, e.g., 'PLLaV A' in Table 1, 'VideoLLaMA2' vs 'VideoLLaMA2-7B' vs 'Mistral-7B-Instruct' in §4.2.1, and 'Fig. 1' used to refer to both the task-comparison figure and the architecture figure. Please copyedit throughout.

Circularity Check

0 steps flagged · score 2.0 of 10

No definitional circularity; the SOTA claim is confounded by a fine-tuned-versus-zero-shot comparison, but the model is not derived from its own metric.

full rationale

The paper's central derivation is not circular. FaceTrack-MM is trained with a standard language-modeling objective on the FDA training split, and the TEM metric is used only for evaluation; the model never optimizes TEM or FEC-Bench scores, so Algorithm 1 is not fed back into the model. Results are also reported on external n-gram metrics (CIDEr, ROUGE-L), which do not depend on the proposed pipeline. The self-citation [93] appears only in related work and is not load-bearing. The strongest caveat is evaluation fairness, not circularity: all 15 baselines are zero-shot while FaceTrack-MM is instruction-tuned on FDA, and Table 2 shows instruction tuning alone accounts for most of the gain; the paper's own Sec. 5.2 admits ChatGPT-based scores are biased by format mismatch and that outputs are reformatted before scoring. This is a validity concern about the 'significantly outperforms' headline, but it does not reduce the model or metric to the input by construction. The TEM LCS step uses exact string equality (Algorithm 1, line 14) despite relation classification, an internal inconsistency that weakens the metric's claim but is not circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central empirical claim rests on the validity of the ChatGPT-based evaluation, the adequacy of the 2-cluster face trajectory separation, and the assumption that a test set sampled from the same distribution as training gives a fair zero-shot comparison.

free parameters (1)
  • K-means cluster count for main/background trajectory separation = 2
    The main trajectory selection in Section 4.2.2 uses K-means with 2 cluster centers, representing one main character cluster and one background character cluster. This hand-chosen value assumes exactly two roles in every video.
assumptions (3)
  • domain assumption ChatGPT can reliably extract and classify facial expression events for evaluation.
    The TEM metric in Section 3.4 relies on proprietary ChatGPT API outputs for event extraction and relation classification, without human validation or correlation analysis.
  • domain assumption K-means with two clusters always separates the main character from background characters in multi-person scenes.
    Section 4.2.2 assumes that the trajectory with larger area and higher cosine similarity is the main character, which may fail when multiple people speak or when the main character changes.
  • standard math The standard longest common subsequence dynamic programming algorithm is valid.
    Algorithm 1 uses LCS as a standard algorithmic tool, which is correct as a substring ordering measure, though its application here uses exact equality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Facial Dynamics in Video: Instruction Tuning for Improved Facial Expression Perception and Contextual Awareness." pith.science (2026). https://pith.science/paper/734JDURI

@misc{pith2026250107978,
  author       = {Pith},
  title        = {Pith review of: Facial Dynamics in Video: Instruction Tuning for Improved Facial Expression Perception and Contextual Awareness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/734JDURI}},
  note         = {Machine review of arXiv:2501.07978}
}
read the original abstract

Facial expression captioning has found widespread application across various domains. Recently, the emergence of video Multimodal Large Language Models (MLLMs) has shown promise in general video understanding tasks. However, describing facial expressions within videos poses two major challenges for these models: (1) the lack of adequate datasets and benchmarks, and (2) the limited visual token capacity of video MLLMs. To address these issues, this paper introduces a new instruction-following dataset tailored for dynamic facial expression caption. The dataset comprises 5,033 high-quality video clips annotated manually, containing over 700,000 tokens. Its purpose is to improve the capability of video MLLMs to discern subtle facial nuances. Furthermore, we propose FaceTrack-MM, which leverages a limited number of tokens to encode the main character's face. This model demonstrates superior performance in tracking faces and focusing on the facial expressions of the main characters, even in intricate multi-person scenarios. Additionally, we introduce a novel evaluation metric combining event extraction, relation classification, and the longest common subsequence (LCS) algorithm to assess the content consistency and temporal sequence consistency of generated text. Moreover, we present FEC-Bench, a benchmark designed to assess the performance of existing video MLLMs in this specific task. All data and source code will be made publicly available.

Figures

Figures reproduced from arXiv: 2501.07978 by the authors.

Figure 1
Figure 1. Comparison of annotation styles between different facial [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Some statistics of proposed DFEC dataset. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Architecture of FaceTrack-MM. Our FaceTrack-MM leverages FaceXFormer [56] as the auxiliary facial visual encoder to extract facial features of the main characters and uses CLIP-ViT-Large [64] as the visual encoder. We utilize the STC module [11] as the visual projector to inject temporal information and use Mistral-7B-Instruct [23] for the pretrained large language model. fully crafted prompts with GPT-4 [61] to gen… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Qualitative result comparison. We highlight the content related to emotional and expressive changes in different methods as well as in the ground truth (GT). Our model demonstrates superior capability in capturing changes in facial expressions. Method VC Avg. AutoDQ TE…
Figure 5
Figure 5. Figure 5: Pipeline of the FDA annotation process. We use ChatGPT to preliminarily annotate the emotions and facial changes of the person in the video and introduce manual correcton and consolidation to get the refined final annotations. 3 [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Event extraction. We use ChatGPT with the specific prompt to extract event from video captions. Prompt: Given two lists of events. For each pair of events (one from each list), classify the relationship between the events into three classes: Same Meaning, Opposite Mean…
Figure 8
Figure 8. Figure 8: Format alignment before calculating the VideoChatGPT scores. - Please give a thorough account of the character's facial features and expression variations throughout the video, highlighting their expressions at both the start and end of the footage. - Could you provide…
Figure 9
Figure 9. Figure 9: Prompt used to generate instructions and some instances of FDA instructions. 5 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: More visualizations of dataset annotations. 6 [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: More visualizations of dataset annotations. 7 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: More visualizations of dataset annotations. 8 [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A training-free token compression method using semantic connected components in space and time keeps video understanding accuracy high even when retaining only 5-10% of visual tokens.

Reference graph

Works this paper leans on

95 extracted references · 44 canonical work pages · cited by 1 Pith paper

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems , 35:23716–23736,

  2. [2]

    Emotion recognition in speech using cross- modal transfer in the wild, 2018

    Samuel Albanie, Arsha Nagrani, Andrea Vedaldi, and An- drew Zisserman. Emotion recognition in speech using cross- modal transfer in the wild, 2018. 2

    2018

  3. [3]

    Claude-3.5, 2024

    Anthropic. Claude-3.5, 2024. 3, 7

    2024

  4. [5]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. 2

    arXiv 2023

  5. [6]

    Collecting highly paral- lel data for paraphrase evaluation

    David Chen and William B Dolan. Collecting highly paral- lel data for paraphrase evaluation. InProceedings of the 49th annual meeting of the association for computational linguis- tics: human language technologies , pages 190–200, 2011. 2

    2011

  6. [7]

    Finecliper: Multi-modal fine-grained clip for dynamic facial expression recognition with adapters

    Haodong Chen, Haojian Huang, Junhao Dong, Mingzhe Zheng, and Dian Shao. Finecliper: Multi-modal fine-grained clip for dynamic facial expression recognition with adapters. arXiv preprint arXiv:2407.02157, 2024. 1

    arXiv 2024

  7. [8]

    Sharegpt4video: Improving video understand- ing and generation with better captions

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, et al. Sharegpt4video: Improving video understand- ing and generation with better captions. arXiv preprint arXiv:2406.04325, 2024. 7

    arXiv 2024

  8. [9]

    Stcam: Spatial-temporal and channel attention module for dynamic facial expression recognition

    Weicong Chen, Dong Zhang, Ming Li, and Dah-Jye Lee. Stcam: Spatial-temporal and channel attention module for dynamic facial expression recognition. IEEE TAFFC, 14(1): 800–810, 2023. 1

    2023

Show all 95 references
  1. [10]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Zhong Muyan, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238, 2023. 2

    2023 arXiv

  2. [11]

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video- llms

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video- llms. arXiv preprint arXiv:2406.07476, 2024. 2, 3, 4, 6, 7, 8

    2024 arXiv

  3. [12]

    Transface: Calibrating trans- former training for face recognition from a data-centric per- spective, 2023

    Jun Dan, Yang Liu, Haoyu Xie, Jiankang Deng, Haoran Xie, Xuansong Xie, and Baigui Sun. Transface: Calibrating trans- former training for face recognition from a data-centric per- spective, 2023. 6

    2023

  4. [13]

    Diffusionrig: Learning personal- ized priors for facial appearance editing

    Zheng Ding, Cecilia Zhang, Zhihao Xia, Lars Jebe, Zhuowen Tu, and Xiuming Zhang. Diffusionrig: Learning personal- ized priors for facial appearance editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2

    2023

  5. [14]

    Internlm-xcomposer2: Mastering free-form text-image composition and compre- hension in vision-language large model

    Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. Internlm-xcomposer2: Mastering free-form text-image composition and compre- hension in vision-language large model. arXiv preprint arXiv:2401.1...

    2024 arXiv

  6. [15]

    Strongsort: Make deep- sort great again

    Yunhao Du, Zhicheng Zhao, Yang Song, Yanyun Zhao, Fei Su, Tao Gong, and Hongying Meng. Strongsort: Make deep- sort great again. IEEE Transactions on Multimedia, 2023. 6

    2023

  7. [16]

    EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Ex- pression Recognition

    Niki Maria Foteinopoulou and Ioannis Patras. EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Ex- pression Recognition. In IEEE FG, 2024. 1

    2024

  8. [17]

    Video-mme: The first-ever compre- hensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever compre- hensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075, 2024. 2

    2024 arXiv

  9. [18]

    Llama-adapter v2: Parameter-efficient vi- sual instruction model

    Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xi- angyu Yue, et al. Llama-adapter v2: Parameter-efficient vi- sual instruction model. arXiv preprint arXiv:2304.15010 ,

    arXiv

  10. [19]

    Music Emotion Recognition: Toward new, robust standards in personalized and context-sensitive ap- plications

    Juan Sebasti ´an G ´omez-Ca˜n´on, Estefan ´ıa Cano, Tuomas Eerola, Perfecto Herrera, Xiao Hu, Yi-Hsuan Yang, and G´omez Emilia. Music Emotion Recognition: Toward new, robust standards in personalized and context-sensitive ap- plications. IEEE Signal Processing Magazine, 38:106–114,

  11. [20]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InIn- ternational Conference on Learning Representations , 2022. 6

    2022

  12. [21]

    Multimodal pretraining for dense video cap- tioning

    Gabriel Huang, Bo Pang, Zhenhai Zhu, Clara Rivera, and Radu Soricut. Multimodal pretraining for dense video cap- tioning. arXiv preprint arXiv:2011.11760, 2020. 2

    2011 arXiv

  13. [22]

    Video recap: Recursive captioning of hour-long videos

    Md Mohaiminul Islam, Ngan Ho, Xitong Yang, Tushar Na- garajan, Lorenzo Torresani, and Gedas Bertasius. Video recap: Recursive captioning of hour-long videos. arXiv preprint arXiv:2402.13250, 2024. 2

    2024 arXiv

  14. [23]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Deven- dra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guil- laume Lample, L ´elio Renard Lavaud, Lucile Saulnier, Mari...

    2024

  15. [24]

    Dfew: A large-scale database for recognizing dynamic facial expres- sions in the wild, 2020

    Xingxun Jiang, Yuan Zong, Wenming Zheng, Chuangao Tang, Wanchuang Xia, Cheng Lu, and Jiateng Liu. Dfew: A large-scale database for recognizing dynamic facial expres- sions in the wild, 2020. 1, 3, 2

    2020

  16. [25]

    Chat-univi: Unified visual representation em- 9 powers large language models with image and video under- standing

    Peng Jin, Ryuichi Takanobu, Caiwan Zhang, Xiaochun Cao, and Li Yuan. Chat-univi: Unified visual representation em- 9 powers large language models with image and video under- standing. arXiv preprint arXiv:2311.08046, 2023. 7

    2023 arXiv

  17. [26]

    Expression, affect, action unit recognition: Aff-wild2, multi-task learning and arcface, 2019

    Dimitrios Kollias and Stefanos Zafeiriou. Expression, affect, action unit recognition: Aff-wild2, multi-task learning and arcface, 2019. 2

    2019

  18. [27]

    Afew-va database for valence and arousal estimation in-the-wild

    Jean Kossaifi, Georgios Tzimiropoulos, Sinisa Todorovic, and Maja Pantic. Afew-va database for valence and arousal estimation in-the-wild. Image and Vision Computing, 65:23– 36, 2017. Multimodal Sentiment Analysis and Mining in the Wild Image and Vision Computing. 3, 2

    2017

  19. [28]

    Dense-captioning events in videos

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense-captioning events in videos. In Proceedings of the IEEE international conference on com- puter vision, pages 706–715, 2017. 2

    2017

  20. [29]

    Context-aware emotion recognition net- works

    Jiyoung Lee, Seungryong Kim, Sunok Kim, Jungin Park, and Kwanghoonn Sohn. Context-aware emotion recognition net- works. In Proceedings of the IEEE/CVF international con- ference on computer vision, 2019. 3, 2

    2019

  21. [30]

    Llava-next: What else influences visual instruction tun- ing beyond data?, 2024

    Bo Li, Hao Zhang, Kaichen Zhang, Dong Guo, Yuanhan Zhang, Renrui Zhang, Feng Li, Ziwei Liu, and Chunyuan Li. Llava-next: What else influences visual instruction tun- ing beyond data?, 2024. 3

    2024

  22. [31]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. 3, 7, 8

    2024 arXiv

  23. [32]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024. 3

    2024 arXiv

  24. [33]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Inter- national Conference on Machine Learning, ICML 2023, 23- 29 July 2023, Honolulu, Hawaii, USA, pages 19730–197...

    2023

  25. [34]

    Videochat: Chat-centric video understanding

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355, 2023. 2, 7

    2023 arXiv

  26. [35]

    Dual-sti: Dual-path spatial-temporal interac- tion learning for dynamic facial expression recognition

    Min Li, Xiaoqin Zhang, Chenxiang Fan, Tangfei Liao, and Guobao Xiao. Dual-sti: Dual-path spatial-temporal interac- tion learning for dynamic facial expression recognition. In- formation Sciences, page 120953, 2024. 1

    2024

  27. [36]

    Facial affective behavior analysis with instruction tuning, 2024

    Yifan Li, Anh Dao, Wentao Bao, Zhen Tan, Tianlong Chen, Huan Liu, and Yu Kong. Facial affective behavior analysis with instruction tuning, 2024. 6

    2024

  28. [37]

    Llama-vid: An image is worth 2 tokens in large language models

    Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. 2024. 3, 7

    2024

  29. [38]

    Photomaker: Customizing realistic human photos via stacked id embedding

    Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming- Ming Cheng, and Ying Shan. Photomaker: Customizing realistic human photos via stacked id embedding. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  30. [39]

    Video-llava: Learning united visual represen- tation by alignment before projection

    Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual represen- tation by alignment before projection. arXiv preprint arXiv:2311.10122, 2023. 2

    2023 arXiv

  31. [40]

    Saanet: Siamese action-units attention network for improving dynamic facial expression recogni- tion

    Daizong Liu, Xi Ouyang, Shuangjie Xu, Pan Zhou, Kun He, and Shiping Wen. Saanet: Siamese action-units attention network for improving dynamic facial expression recogni- tion. Neurocomputing, 413:145–157, 2020. 1

    2020

  32. [41]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744, 2023. 3

    2023 arXiv

  33. [42]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. 2023. 3

    2023

  34. [43]

    World model on million-length video and language with ringattention

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with ringattention. arXiv preprint arXiv:2402.08268, 2024. 2

    2024 arXiv

  35. [44]

    One for all: Video conversation is fea- sible without video instruction tuning

    Ruyang Liu, Chen Li, Yixiao Ge, Ying Shan, Thomas H Li, and Ge Li. One for all: Video conversation is fea- sible without video instruction tuning. arXiv preprint arXiv:2309.15785, 2023. 7

    2023 arXiv

  36. [45]

    St-llm: Large language models are effective tem- poral learners

    Ruyang Liu, Chen Li, Haoran Tang, Yixiao Ge, Ying Shan, and Ge Li. St-llm: Large language models are effective tem- poral learners. https://arxiv.org/abs/2404.00308, 2023. 7

    2023 arXiv

  37. [46]

    Mafw: A large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild, 2023

    Yuanyuan Liu, Wei Dai, Chuanxu Feng, Wenbin Wang, Guanghao Yin, Jiabei Zeng, and Shiguang Shan. Mafw: A large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild, 2023. 3, 4, 2

    2023

  38. [47]

    DamoFD: Digging into backbone de- sign on face detection

    Yang Liu, Jiankang Deng, Fei Wang, Lei Shang, Xuansong Xie, and Baigui Sun. DamoFD: Digging into backbone de- sign on face detection. In The Eleventh International Con- ference on Learning Representations, 2023. 6

    2023

  39. [48]

    Livingstone and Frank A

    Steven R. Livingstone and Frank A. Russo. The ryer- son audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english. PLOS ONE, 2018. 3, 2

    2018

  40. [49]

    Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews

    Patrick Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Com- puter Society Conference on Computer Vision and Pattern ...

    2010

  41. [50]

    The extended cohn- kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression

    Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. The extended cohn- kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In CVPRW, pages 94–101,

  42. [51]

    Learning multi-dimensional edge feature- based au relation graph for facial action unit recognition

    Cheng Luo, Siyang Song, Weicheng Xie, Linlin Shen, and Hatice Gunes. Learning multi-dimensional edge feature- based au relation graph for facial action unit recognition. In Proceedings of the Thirty-First International Joint Confer- ence on Artificial Intelligence, IJCAI-22 , ...

  43. [52]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fa- had Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424, 2023. 2

    2023 arXiv

  44. [53]

    Video-chatgpt: Towards detailed video 10 understanding via large vision and language models, 2024

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fa- had Shahbaz Khan. Video-chatgpt: Towards detailed video 10 understanding via large vision and language models, 2024. 5, 7, 1

    2024

  45. [54]

    Egoschema: A diagnostic benchmark for very long- form video language understanding

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic benchmark for very long- form video language understanding. Advances in Neural In- formation Processing Systems, 36, 2024. 2

    2024

  46. [55]

    The importance of emotional regulation in mental health

    Johnston CA Menefee DS, Ledoux T. The importance of emotional regulation in mental health. American Journal of Lifestyle Medicine, 2022. 1

    2022

  47. [56]

    Facexformer: A unified transformer for fa- cial analysis

    Kartik Narayan, Vibashan VS, Rama Chellappa, and Vishal M Patel. Facexformer: A unified transformer for fa- cial analysis. arXiv preprint arXiv:2403.12960, 2024. 4, 6

    2024 arXiv

  48. [57]

    Repre- sentation learning and identity adversarial training for facial behavior understanding, 2024

    Mang Ning, Albert Ali Salah, and Itir Onal Ertugrul. Repre- sentation learning and identity adversarial training for facial behavior understanding, 2024. 2

    2024

  49. [58]

    OpenAI. ChatGPT. https://openai.com/blog/ chatgpt/, 2023. 2, 4, 5, 1

    2023

  50. [59]

    Gpt-4v(ision) system card

    OpenAI. Gpt-4v(ision) system card. 2023. 3

    2023

  51. [60]

    Gpt-4 technical report, 2023

    OpenAI. Gpt-4 technical report, 2023. 2, 3

    2023

  52. [61]

    Gpt-4o system card, 2024

    OpenAI. Gpt-4o system card, 2024. 3, 4, 7, 1

    2024

  53. [62]

    Digihuman: A con- versational digital human with facial expressions

    Kasım ¨Ozacar and Munya Alkhalıfa. Digihuman: A con- versational digital human with facial expressions. Turkish Journal of Science and Technology, 19(1):25–37, 2024. 2

    2024

  54. [63]

    Pantic, M

    M. Pantic, M. Valstar, R. Rademaker, and L. Maat. Web- based database for facial expression analysis. In 2005 IEEE International Conference on Multimedia and Expo , pages 5 pp.–, 2005. 2

    2005

  55. [64]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  56. [65]

    Movie description

    Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Pal, Hugo Larochelle, Aaron Courville, and Bernt Schiele. Movie description. International Journal of Computer Vision, 123:94–120, 2017. 2

    2017

  57. [66]

    Multi-view dynamic facial action unit detection

    Andr ´es Romero, Juan Le´on, and Pablo Arbel´aez. Multi-view dynamic facial action unit detection. Image and Vision Com- puting, 2018. 2

    2018

  58. [67]

    Deep adaptive attention for joint facial action unit detection and face alignment

    Zhiwen Shao, Zhilei Liu, Jianfei Cai, and Lizhuang Ma. Deep adaptive attention for joint facial action unit detection and face alignment. In European Conference on Computer Vision, pages 725–740. Springer, 2018. 2

    2018

  59. [68]

    Driver’s emotion and behavior classification system based on internet of things and deep learning for advanced driver assistance system (adas)

    Mariya Tauqeer, Saddaf Rubab, Muhammad Attique Khan, Rizwan Ali Naqvi, Kashif Javed, Abdullah Alqahtani, Sht- wai Alsubai, and Adel Binbusayyis. Driver’s emotion and behavior classification system based on internet of things and deep learning for advanced driver assistance sys...

    2022

  60. [69]

    Gemini: A family of highly capable multi- modal models, 2024

    Gemini Team. Gemini: A family of highly capable multi- modal models, 2024. 3

    2024

  61. [70]

    Qwen2.5: A party of foundation models, 2024

    Qwen Team. Qwen2.5: A party of foundation models, 2024. 3

    2024

  62. [71]

    Induced disgust, hap- piness and surprise: an addition to the mmi facial expres- sion database

    Michel Valstar, Maja Pantic, et al. Induced disgust, hap- piness and surprise: an addition to the mmi facial expres- sion database. In Proc. 3rd Intern. Workshop on EMOTION, page 65, 2010. 1

    2010

  63. [72]

    Cider: Consensus-based image description evalua- tion

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evalua- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015. 5, 7

    2015

  64. [73]

    A survey on the pipeline evolution of facial capture and tracking for digital humans

    Carlos Vilchis, Carmina Perez-Guerrero, Mauricio Mendez- Ruiz, and Miguel Gonzalez-Mendoza. A survey on the pipeline evolution of facial capture and tracking for digital humans. Multimedia Systems, 29(4):1917–1940, 2023. 2

    1917

  65. [74]

    Gross, Kristina H ¨o¨ok, Regan Mandryk, and Petr Slovak

    Greg Wadley, Vassilis Kostakos, Peter Koval, Wally Smith, Sarah Webber, Anna Cox, James J. Gross, Kristina H ¨o¨ok, Regan Mandryk, and Petr Slovak. The future of emotion in human-computer interaction. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computi...

    2022

  66. [75]

    Tarsier: Recipes for training and evaluating large video description models, 2024

    Jiawei Wang, Liping Yuan, Yuchen Zhang, and Haomiao Sun. Tarsier: Recipes for training and evaluating large video description models, 2024. 2, 5, 7, 1

    2024

  67. [76]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Jun- yang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024 arXiv

  68. [77]

    Vatex: A large-scale, high- quality multilingual dataset for video-and-language research

    Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, and William Yang Wang. Vatex: A large-scale, high- quality multilingual dataset for video-and-language research. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4581–4591, 2019. 2

    2019

  69. [78]

    Ferv39k: A large-scale multi-scene dataset for fa- cial expression recognition in videos, 2022

    Yan Wang, Yixuan Sun, Yiwen Huang, Zhongying Liu, Shuyong Gao, Wei Zhang, Weifeng Ge, and Wenqiang Zhang. Ferv39k: A large-scale multi-scene dataset for fa- cial expression recognition in videos, 2022. 3, 2

    2022

  70. [79]

    Internvideo2: Scaling video foundation mod- els for multimodal video understanding

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Jilan Xu, Zun Wang, et al. Internvideo2: Scaling video foundation mod- els for multimodal video understanding. arXiv preprint arXiv:2403.15377, 2024. 2

    2024 arXiv

  71. [80]

    Msr-vtt: A large video description dataset for bridging video and language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5288–5296, 2016. 2

    2016

  72. [81]

    Pllava : Parameter-free llava extension from images to videos for video dense captioning, 2024

    Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng. Pllava : Parameter-free llava extension from images to videos for video dense captioning, 2024. 2, 3, 7

    2024

  73. [82]

    xgen-mm (blip-3): A family of open large multimodal models, 2024

    Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael S Ryoo, Shrikant Kendre, Jieyu Zhang, Can Qin, Shu Zhang, Chia-Chih Chen, Ning Yu, Juntao Tan, Tulika Manoj Awalgaonkar, Shelby Heinecke, Huan Wang, Yejin C...

    2024

  74. [84]

    mplug-owl: Modularization empowers large language models with multimodality

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178, 2023. 3

    2023 arXiv

  75. [85]

    mplug- owl2: Revolutionizing multi-modal large language model with modality collaboration, 2023

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mplug- owl2: Revolutionizing multi-modal large language model with modality collaboration, 2023. 3

    2023

  76. [86]

    Spatio-temporal convolutional features with nested lstm for facial expression recognition.Neurocomputing, 317: 50–57, 2018

    Zhenbo Yu, Guangcan Liu, Qingshan Liu, and Jiankang Deng. Spatio-temporal convolutional features with nested lstm for facial expression recognition.Neurocomputing, 317: 50–57, 2018. 1

    2018

  77. [87]

    Auformer: Vision transformers are parameter-efficient facial action unit detectors

    Kaishen Yuan, Zitong Yu, Xin Liu, Weicheng Xie, Huanjing Yue, and Jingyu Yang. Auformer: Vision transformers are parameter-efficient facial action unit detectors. In European Conference on Computer Vision. Springer, 2024. 2

    2024

  78. [88]

    Video-llama: An instruction-tuned audio-visual language model for video un- derstanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video un- derstanding. arXiv preprint arXiv:2306.02858, 2023. 3, 7

    2023 arXiv

  79. [89]

    Vi- sion transformer with quadrangle attention

    Qiming Zhang, Jing Zhang, Yufei Xu, and Dacheng Tao. Vi- sion transformer with quadrangle attention. arXiv preprint arXiv:2303.15105, 2023. 3

    2023 arXiv

  80. [90]

    Cohn, Shaun Canavan, Michael Reale, Andy Horowitz, Peng Liu, and Jeffrey M

    Xing Zhang, Lijun Yin, Jeffrey F. Cohn, Shaun Canavan, Michael Reale, Andy Horowitz, Peng Liu, and Jeffrey M. Girard. Bp4d-spontaneous: a high-resolution spontaneous 3d dynamic facial expression database. Image and Vision Computing, 32(10):692–706, 2014. Best of Automatic Face...

    2014

  81. [91]

    Llava- next: A strong zero-shot video understanding model, 2024

    Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava- next: A strong zero-shot video understanding model, 2024. 2, 3, 7

    2024

  82. [92]

    Facial expression recognition from near- infrared videos

    Guoying Zhao, Xiaohua Huang, Matti Taini, Stan Z Li, and Matti Pietik¨aInen. Facial expression recognition from near- infrared videos. Image and vision computing , 29(9):607– 619, 2011. 1

    2011

  83. [93]

    Llava-octopus: Unlocking instruction-driven adaptive projector fusion for video understanding

    Jiaxing Zhao, Boyuan Sun, Xiang Chen, Xihan Wei, and Qibin Hou. Llava-octopus: Unlocking instruction-driven adaptive projector fusion for video understanding. arXiv preprint arXiv:2501.05067, 2025. 2

    2025 arXiv

  84. [94]

    Deep region and multi-label learning for facial action unit detec- tion

    Kaili Zhao, Wen-Sheng Chu, and Honggang Zhang. Deep region and multi-label learning for facial action unit detec- tion. In CVPR, 2016. 2

    2016

  85. [95]

    Mlvu: A comprehensive benchmark for multi-task long video understanding

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264,

    arXiv

  86. [96]

    Towards automatic learning of procedures from web instructional videos

    Luowei Zhou, Chenliang Xu, and Jason Corso. Towards automatic learning of procedures from web instructional videos. In Proceedings of the AAAI Conference on Artificial Intelligence, 2018. 2

    2018

  87. [97]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 2 12

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.