Pith. sign in

REVIEW 4 major objections 6 minor 58 references

FaceInsight: A Multimodal Large Language Model for Face Perception

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FaceInsight claims that adding segmentation maps, co-occurrence priors, and logic rules to an MLLM beats nine general models on face attribute, demographic, and expression tasks.

desk verdict Plausible architecture and a useful problem, but the evaluation as reported cannot support the headline performance claim. read the letter →

arxiv 2504.15624 v2 pith:IILFVPSS submitted 2025-04-22 cs.CV

Jingzhi Li , Changjiang Luo , Ruoyu Chen , Hua Zhang , Wenqi Ren , Jianhou Gan , Xiaochun Cao This is my paper · ORCID
classification cs.CV
keywords faceperceptionmultimodallargelanguagemodelvisual-textualalignmentattributerecognitionagegenderandraceestimationfacialexpressionsegmentationlogicalconstraints
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FaceInsight argues that general multimodal large language models fail at face perception because they lean too heavily on language-driven reasoning and do not capture the dense, overlapping structure of facial attributes. The paper's central claim is that feeding an MLLM three face-specific signals—segmentation maps, an attribute co-occurrence correlation map refined by a graph convolutional network, and first-order logic rules penalizing contradictory outputs—makes it outperform nine general MLLMs on face attribute recognition, age/gender/race estimation, and expression prediction. If true, this would give a practical recipe for making vision-language models reliable in face-critical applications such as authentication, human-computer interaction, and healthcare, where hallucinated or contradictory facial descriptions are costly. The paper reports the largest gains not in accuracy but in recall, precision, and F1, which is where the comparison models produce many incorrect or conflicting attributes.

What carries the argument

The load-bearing mechanism is the correlation constraint module, built on an asymmetric conditional-probability matrix $P=(p_{ij})$ with $p_{ij}=m_{ij}/n_i$, where $m_{ij}$ is the co-occurrence count of facial attributes $i$ and $j$ and $n_i$ is the frequency of $i$ in the training set. Thresholding $P$ at $\tau$ gives a sparse adjacency map $\hat{C}$, reweighted so each attribute keeps weight $1-\omega$ on itself and distributes $\omega$ over its co-occurring neighbors; $L$ graph-convolutional layers refine prompt features by $T_p^l = \rho(\hat{C} T_p^{l-1} W^{l-1})$ with a residual connection, producing the final text embedding $\hat{T}_p = T_p^0 + T_p^L$. A parallel logical constraint module turns 22 first-order rules into a violation probability and adds a logical loss $\mathcal{L}_c$ to the binary cross-entropy loss, so the full objective is $\mathcal{L} = \mathcal{L}_{bce} + \mathcal{L}_c$. Face segmentation maps generated by Pyfacer are encoded by a second pretrained ViT and injected as extra tokens, giving the LLM both the global image and localized structural cues.

What would settle it

Compute the correlation priors only from the training split, fine-tune the general MLLMs on the same instruction-following splits, and evaluate both on held-out identities; if FaceInsight's recall and F1 advantage over the fine-tuned general models collapses, the gains come from label leakage rather than the proposed modules.

Watch

Extended reading notes

Core claim

FaceInsight claims that a multimodal large language model can be made to perceive faces accurately by aligning facial knowledge through three targeted additions: feeding face segmentation maps as an auxiliary perceptual modality; refining text-prompt embeddings with a graph convolutional network driven by an asymmetric conditional-probability correlation map among facial attributes; and adding a logical-loss term that penalizes violations of 22 hand-coded rules such as 'bald implies not bangs and not wavy hair.' Trained with binary cross-entropy plus the logical loss, the model reports the best accuracy, recall, precision, and F1 across face attribute recognition on MAAD and CelebA, age/gender/race estimation on FairFace and UTKFace, and expression prediction on ExpW and RAF-DB, beating nine general MLLMs in training-free comparisons and three fine-tuned MLLMs in the fine-tuned setting. Ablations on CelebA attribute the average 2.3% mAP gain to the three components, with each step adding a positive increment.

Load-bearing premise

The reported margins assume that the labels used to train FaceInsight are strictly separated from the images used to test it, and that measuring it against general models that were not trained on face-task labels is a fair comparison.

Editorial extensions

If this is right

  • Adding the three modules to a LLaVA-style MLLM does not require retraining the LLM or the image encoder; only the adapter and graph convolutional network are updated.
  • The logical constraint loss can be reused in any vision-language task with mutually exclusive or co-occurring labels to suppress contradictory outputs.
  • The recipe is claimed to transfer to other fine-grained visual perception tasks, as the paper states in its conclusion.
  • Because the biggest gains are in recall, precision, and F1, the method most helps applications where false or contradictory facial descriptions matter more than raw accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims, the training-set-dependent correlation priors make the evaluation protocol the main open question: comparing a model trained on the six datasets against zero-shot general MLLMs conflates task-specific supervision with architectural benefit.
  • A cleaner test of the mechanism would compute the correlation matrix only from a training split, hold out entire identities, and fine-tune the general MLLMs on the same split; the paper does not describe such a protocol.
  • The hand-coded 22 rules suggest a natural scaling path: mine logical rules from a larger face-description corpus or learn them as soft constraints, which would let the logical module cover attribute vocabularies beyond the training sets.
  • The asymmetry of the correlation map is a general idea: directional attribute dependencies like 'bald implies no bangs' could improve consistency in other dense attribute-recognition domains, such as medical image description.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces FaceInsight, a multimodal large language model for face perception that combines three components: (1) face segmentation maps as an auxiliary perceptual modality, (2) a correlation constraint module (CCM) that injects pairwise attribute-dependency priors into the text embedding via a graph convolutional network, and (3) a logical constraint module that adds a loss derived from 22 hand-authored first-order logic rules enforcing mutual exclusions and group constraints among facial attributes. The model is built on the LLaVA-v1.5 architecture with a CLIP ViT image encoder pre-trained on FLIP. Experiments cover face attribute recognition (MAAD, CelebA), age/gender/race estimation (FairFace, UTKFace), and facial expression prediction (ExpW, RAF-DB). The authors compare FaceInsight against nine MLLMs, claim consistent superior performance under both 'training-free' and fine-tuned settings, and provide ablations on the contribution of each module.

Significance. If the central performance claim were established under a clean evaluation protocol, FaceInsight would be a meaningful contribution: it addresses a real deficiency of general MLLMs on fine-grained face perception, and the combination of segmentation maps, correlation priors, and logic constraints is a plausible and well-motivated design. The paper also includes ablation studies that isolate the effect of each component, which is a strength. However, the reported evidence does not currently support the headline claim of 'consistently outperforms nine compared MLLMs across all datasets' because the evaluation protocol has a potentially load-bearing train/test overlap, the comparison is not apples-to-apples (task-trained FaceInsight versus mostly zero-shot MLLMs), no error bars or statistical tests are provided, and no code or model weights are released. These issues are fixable in principle but require substantial additional experimentation and reporting.

major comments (4)
  1. [§4.1–4.2, Tables 1–5] The central evaluation claim is not supported because the training and test sets are the same six datasets. Section 4.2 states 'We trained our framework simultaneously on six training datasets,' and the six datasets listed in Section 4.1 are exactly MAAD, CelebA, FairFace, UTKFace, ExpW, and RAF-DB, which are also the test sets. The paper does not describe any train/test split protocol, does not identify held-out partitions, and provides no code or released checkpoints. Without disjoint train and test images, and without evidence that the annotation population used for training is independent of the test population, the reported margins in Tables 1–5 cannot be attributed to face perception ability rather than to memorization or label-distribution leakage. Please specify the exact split protocol (including identity-exclusion rules), release the splits or code, and additionally evaluate on at least one benchmark not used for training.
  2. [§3.2, Eqs. (5)–(6)] The correlation prior maps that drive the CCM are computed directly from co-occurrence frequencies in the training set, and this same training set is drawn from the benchmark pool used for evaluation. This means the CCM encodes the label-distribution statistics of the evaluation domains, including attribute base rates and pairwise associations, and can inflate performance even if the visual features are weak. The authors acknowledge overfitting concerns and apply a threshold, but they do not address the more direct circularity: the 'facial knowledge' injected is distribution-specific rather than an independent source of knowledge. Please either derive the priors from an external/unrelated corpus, or evaluate on held-out datasets with different label distributions, and ablate the CCM against a randomized or shuffled prior matrix to quantify how much of the gain comes from prior leakage.
  3. [§4.2, §4.3, Figs. 3–6] The comparison protocol is not balanced. FaceInsight is trained with a binary cross-entropy loss on the exact task labels of the six evaluation datasets, while the nine comparison MLLMs are evaluated zero-shot (only three are fine-tuned, on 'task-specific datasets' with unspecified splits). The paper still reports that FaceInsight outperforms the MLLMs in both settings, but this conflates task-specific training with general face perception ability. The label 'training-free' for FaceInsight is misleading because the model has already been trained on the six datasets. Please add strong specialist baselines (e.g., FaceXFormer, FairFace, MIVolo, or other dedicated face-perception models) trained and evaluated on the same splits, report the fine-tuning protocol for comparison MLLMs in detail, and include error bars or confidence intervals across multiple runs so that the 'consistently outperforms' claim is statistically grounded.
  4. [§4.5, Fig. 8] The ablation study reports only single-point mAP numbers on CelebA with no error bars, no number of runs, and no statistical test. The text is also internally unclear about how the component contributions add up: it states an average mAP improvement of 2.3%, then reports gains of 0.82%, 1.48%, and 0.95% for the sequential additions, which do not obviously sum to 2.3%. Please clarify whether the reported numbers are cumulative or incremental, and provide means and standard deviations over multiple seeds. In addition, Fig. 8b sweeps only ω on CelebA; the choice ω=0.8 should be justified on a validation split that is disjoint from the reported test set.
minor comments (6)
  1. [§4.1] The dataset name appears as 'TRAF-DB' in the text but as 'RAF-DB' everywhere else, including Table 5; please correct the typo.
  2. [Abstract and §4.2] The phrase 'training-free settings' is applied to FaceInsight, but the model is trained on the six datasets before evaluation. Please rephrase to distinguish zero-shot evaluation of comparison MLLMs from the fully trained FaceInsight, or introduce a genuinely zero-shot variant of FaceInsight.
  3. [Fig. 1] The figure contains a passage of garbled path-like tokens ('/uni00000018/uni00000018/...') that appears to be a text corruption artifact; this should be removed or replaced with the intended caption or annotation text.
  4. [§4.3.2, Table 4] The text mentions FaRL as a baseline, but Table 4 lists only FairFace, FaceXFormer, and FaceInsight. Either add FaRL to the table or remove the reference to it in this comparison.
  5. [§3.1] The text refers to 'Pyfacer [50]' and cites RetinaFace [50]; please clarify whether Pyfacer is a library wrapping RetinaFace and provide a proper citation or URL for the library used.
  6. [§3.2, Eq. (9)] The logical loss multiplies V_i(rule_j) by P(y_ij), but it is not clear how P(y_ij) is selected for rules with multiple antecedents or how V_i is computed for rules involving negation. A short derivation or pseudocode would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the correlation prior is computed from training-set co-occurrence, not from test labels, and the logic rules are externally authored constraints.

full rationale

The claimed derivation chain is not circular. In Section 3.2, Eq. (5) constructs the conditional probability matrix from co-occurrence frequencies 'in the training set'; this is a fitted prior derived from the training labels, not from the evaluation labels. Section 4.1 lists the six evaluation datasets (MAAD, CelebA, FairFace, UTKFace, ExpW, RAF-DB), and Section 4.2 states 'We trained our framework simultaneously on six training datasets.' If 'training datasets' refers to the standard official training splits of these benchmarks, then evaluation on the corresponding held-out splits is the normal supervised-learning loop, not a self-definitional reduction: the model outputs are not equal to the fitted CCM by construction, and the CCM does not encode test-set labels. The 22 first-order logic rules in Section 3.2 (e.g., Bald(x) -> ...) are hand-specified external constraints, not learned from data, so they do not reduce to fitted values. There are no load-bearing self-citations: references [1]-[58] are all external works. The 'training-free' wording refers to the comparison MLLMs being evaluated without fine-tuning while FaceInsight is trained, which is an asymmetric but not circular protocol. The paper's failure to describe exact train/test splits, seed control, code release, and error bars is a genuine reproducibility and validity risk, but it is a reporting limitation rather than a circular derivation. Under the standard interpretation that training and evaluation used disjoint splits, the benchmark comparison is an empirical claim, not a tautology.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or ontological entities. Its contributions are architectural: a segmentation modality, a GCN-based correlation module, and a logical constraint loss. The free parameters are tuning knobs of the proposed framework, and the axioms are the domain assumptions needed for the evaluation and the auxiliary inputs to be meaningful.

free parameters (4)
  • tau
    Threshold in Eq. (6) that sparsifies the co-occurrence-derived correlation map; its value is never reported in the paper.
  • omega = 0.8
    Reweighting hyperparameter in Eq. (7); selected by cross-validation on CelebA (Section 4.5, Fig. 8b).
  • L (number of GCN layers)
    Depth of the GCN in Eq. (8); not specified in the paper.
  • training hyperparameters
    Batch size, learning rate, and early stopping details are only described qualitatively in Section 4.2.
assumptions (3)
  • domain assumption Co-occurrence conditional probabilities estimated on the training set (Eq. 5) generalize to the test distribution
    The correlation prior map C is built from training-label statistics and used to guide text features at inference; validity rests on train/test distributional similarity.
  • ad hoc to paper The 22 hand-authored first-order logic rules capture the true mutual exclusion and membership constraints among facial attributes
    Section 3.2: rules such as Bald(x) -> not Bangs(x) and not WavyHair(x) are authored by the researchers; no evidence is provided that this rule set is complete or optimal.
  • domain assumption Face segmentation maps from Pyfacer (RetinaFace) provide accurate region-level cues
    Segmentation maps are treated as a reliable auxiliary modality (Section 3.1); errors in the segmenter would propagate to the model input.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FaceInsight: A Multimodal Large Language Model for Face Perception." pith.science (2026). https://pith.science/paper/IILFVPSS

@misc{pith2026250415624,
  author       = {Pith},
  title        = {Pith review of: FaceInsight: A Multimodal Large Language Model for Face Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IILFVPSS}},
  note         = {Machine review of arXiv:2504.15624}
}
read the original abstract

Recent advances in multimodal large language models (MLLMs) have demonstrated strong capabilities in understanding general visual content. However, these general-domain MLLMs perform poorly in face perception tasks, often producing inaccurate or misleading responses to face-specific queries. To address this gap, we propose FaceInsight, the versatile face perception MLLM that provides fine-grained facial information. Our approach introduces visual-textual alignment of facial knowledge to model both uncertain dependencies and deterministic relationships among facial information, mitigating the limitations of language-driven reasoning. Additionally, we incorporate face segmentation maps as an auxiliary perceptual modality, enriching the visual input with localized structural cues to enhance semantic understanding. Comprehensive experiments and analyses across three face perception tasks demonstrate that FaceInsight consistently outperforms nine compared MLLMs under both training-free and fine-tuned settings.

Figures

Figures reproduced from arXiv: 2504.15624 by the authors.

Figure 1
Figure 1. Performance of Multimodal Large Models (MLLMs) on Face Perception Tasks. The left panel shows qualitative results, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of FaceInsight. To fully integrate fine-grained knowledge of facial information, we design a correlation [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison of FaceInsight and nine [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Performance comparison of FaceInsight and nine [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Performance comparison of FaceInsight and nine [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Performance comparison of FaceInsight and nine [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results for FaceInsight and all MLLMs. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Ablation study results for FaceInsight. To investigate the impact of different values of 𝜔 in Eq.7 on attribute accuracy, we evaluated 𝜔 across the set 0,0.2,0.5,0.8,1, as shown in Fig. 8b. The figure illustrates the significance of balanc￾ing the weights between the a…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 28 canonical work pages

  1. [1]

    Facexbench: Evaluating multimodal llms on face understanding

    Kartik Narayan, Vibashan VS, and Vishal M Patel. Facexbench: Evaluating multimodal llms on face understanding. arXiv preprint arXiv:2501.10360, 2025

    arXiv 2025

  2. [2]

    Face- mllm: A large face perception model

    Haomiao Sun, Mingjie He, Tianheng Lian, Hu Han, and Shiguang Shan. Face- mllm: A large face perception model. arXiv preprint arXiv:2410.20717, 2024

    arXiv 2024

  3. [3]

    Favchat: Unlocking fine-grained facail video understanding with multimodal large language models

    Fufangchen Zhao, Ming Li, Linrui Xu, Wenhao Jiang, Jian Gao, and Danfeng Yan. Favchat: Unlocking fine-grained facail video understanding with multimodal large language models. arXiv preprint arXiv:2503.09158, 2025

    arXiv 2025

  4. [4]

    Emo-llama: Enhancing facial emotion understanding with instruction tuning

    Bohao Xing, Zitong Yu, Xin Liu, Kaishen Yuan, Qilang Ye, Weicheng Xie, Huan- jing Yue, Jingyu Yang, and Heikki Kälviäinen. Emo-llama: Enhancing facial emotion understanding with instruction tuning. arXiv preprint arXiv:2408.11424, 2024

    arXiv 2024

  5. [5]

    Open- flamingo: An open-source framework for training large autoregressive vision- language models

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Open- flamingo: An open-source framework for training large autoregressive vision- language models. arXiv preprint arXiv:2308.01390, 2023

    arXiv 2023

  6. [6]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems , 36:34892–34916, 2023

    work page 2023

  7. [7]

    Llava-next: Improved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024

    2024

  8. [8]

    Visual large language models for generalized and specialized applications

    Yifan Li, Zhixin Lai, Wentao Bao, Zhen Tan, Anh Dao, Kewei Sui, Jiayi Shen, Dong Liu, Huan Liu, and Yu Kong. Visual large language models for generalized and specialized applications. arXiv preprint arXiv:2501.02765, 2025

    arXiv 2025

Show all 58 references
  1. [9]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , p...

    2021

  2. [10]

    Vcoder: Versatile vision encoders for multimodal large language models

    Jitesh Jain, Jianwei Yang, and Humphrey Shi. Vcoder: Versatile vision encoders for multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 27992–28002, 2024

    2024

  3. [11]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023) , ...

    2023

  4. [12]

    Cogcom: Train large vision-language models diving into details through chain of manipulations

    Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, et al. Cogcom: Train large vision-language models diving into details through chain of manipulations. arXiv preprint arXiv:2402.04236, 2024

    2024 arXiv

  5. [13]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  6. [14]

    Minigpt-4: Enhancing vision-language understanding with advanced large lan- guage models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large lan- guage models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  7. [15]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730–19742. PMLR, 2023

    2023

  8. [16]

    Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023

    2023

  9. [17]

    mplug-owl: Modular- ization empowers large language models with multimodality

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Jun- yang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modular- ization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178, 2023

    2023 arXiv

  10. [18]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 26296–26306, 2024

    2024

  11. [19]

    Lion: Empowering multimodal large language model with dual-level visual knowl- edge

    Gongwei Chen, Leyang Shen, Rui Shao, Xiang Deng, and Liqiang Nie. Lion: Empowering multimodal large language model with dual-level visual knowl- edge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26540–26550, 2024

    2024

  12. [20]

    Minigpt-v2: large language model as a unified interface for vision- language multi-task learning

    Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model as a unified interface for vision- language multi-task learning. arXiv preprint arXiv:231...

    arXiv

  13. [21]

    Vila: On pre-training for visual language models

    Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 26689– 26699, 2024

    2024

  14. [22]

    Moe-llava: Mixture of experts for large vision-language models

    Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Jinfa Huang, Junwu Zhang, Yatian Pang, Munan Ning, et al. Moe-llava: Mixture of experts for large vision-language models. arXiv preprint arXiv:2401.15947, 2024

    2024 arXiv

  15. [23]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024

    2024 arXiv

  16. [24]

    Minicpm-v: A gpt-4v level mllm on your phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  17. [25]

    Mantis: Interleaved multi-image instruction tuning

    Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. arXiv preprint arXiv:2405.01483, 2024

    2024 arXiv

  18. [26]

    Monkey: Image resolution and text label are important things for large multi-modal models

    Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xiang Bai. Monkey: Image resolution and text label are important things for large multi-modal models. In proceedings of the IEEE/CVF conference on computer vision and pattern recogni...

    2024

  19. [27]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024

    2024 arXiv

  20. [28]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  21. [29]

    Deepveil: deep learning for identification of face, gender, expression recognition under veiled conditions

    Ahmad BA Hassanat, Abeer Ahmad Albustanji, Ahmad S Tarawneh, Malek Alrashidi, Hani Alharbi, Mohammed Alanazi, Mansoor Alghamdi, Ibrahim S Alkhazi, and VB Surya Prasath. Deepveil: deep learning for identification of face, gender, expression recognition under veiled conditions. ...

    2022

  22. [30]

    Spl- net: Spatial-semantic patch learning network for facial attribute recognition with limited labeled data

    Yan Yan, Ying Shu, Si Chen, Jing-Hao Xue, Chunhua Shen, and Hanzi Wang. Spl- net: Spatial-semantic patch learning network for facial attribute recognition with limited labeled data. International Journal of Computer Vision , 131(8):2097–2121, 2023

    2023

  23. [31]

    Logical consistency and greater descriptive power for facial hair attribute learning

    Haiyu Wu, Grace Bezold, Aman Bhatta, and Kevin W Bowyer. Logical consistency and greater descriptive power for facial hair attribute learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8588–8597, 2023

    2023

  24. [32]

    Logicnet: A log- ical consistency embedded face attribute learning network

    Haiyu Wu, Sicong Tian, Huayu Li, and Kevin W Bowyer. Logicnet: A log- ical consistency embedded face attribute learning network. arXiv preprint arXiv:2311.11208, 2023

    2023 arXiv

  25. [33]

    A survey on facial emotion recognition techniques: A state-of-the-art literature review

    Felipe Zago Canal, Tobias Rossi Müller, Jhennifer Cristine Matias, Gustavo Gino Scotton, Antonio Reis de Sa Junior, Eliane Pozzebon, and Antonio Carlos Sobier- anski. A survey on facial emotion recognition techniques: A state-of-the-art literature review. Information Sciences,...

    2022

  26. [34]

    Semi-supervised multimodal emotion recognition with expression mae

    Zebang Cheng, Yuxiang Lin, Zhaoru Chen, Xiang Li, Shuyi Mao, Fan Zhang, Daijun Ding, Bowen Zhang, and Xiaojiang Peng. Semi-supervised multimodal emotion recognition with expression mae. In Proceedings of the 31st ACM Inter- national Conference on Multimedia , pages 9436–9440, 2023

    2023

  27. [35]

    Facial affective behavior analysis with instruction tuning

    Yifan Li, Anh Dao, Wentao Bao, Zhen Tan, Tianlong Chen, Huan Liu, and Yu Kong. Facial affective behavior analysis with instruction tuning. InEuropean Conference on Computer Vision, pages 165–186, 2024. Jingzhi Li1,2, Changjiang Luo 1,2, Ruoyu Chen 1,2, Hua Zhang 1,2, Wenqi Ren...

    2024

  28. [36]

    Rank consistent ordinal regression for neural networks with application to age estimation

    Wenzhi Cao, Vahid Mirjalili, and Sebastian Raschka. Rank consistent ordinal regression for neural networks with application to age estimation. Pattern Recognition Letters, 140:325–331, 2020

    2020

  29. [37]

    Learning probabilistic ordinal embeddings for uncertainty-aware regression

    Wanhua Li, Xiaoke Huang, Jiwen Lu, Jianjiang Feng, and Jie Zhou. Learning probabilistic ordinal embeddings for uncertainty-aware regression. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 13896–13905, 2021

    2021

  30. [38]

    Mivolo: Multi-input transformer for age and gender estimation

    Maksim Kuprashevich and Irina Tolstykh. Mivolo: Multi-input transformer for age and gender estimation. In International Conference on Analysis of Images, Social Networks and Texts, pages 212–226. Springer, 2023

    2023

  31. [39]

    Hyperface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition

    Rajeev Ranjan, Vishal M Patel, and Rama Chellappa. Hyperface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE transactions on pattern analysis and machine intelligence, 41(1):121–135, 2017

    2017

  32. [40]

    An all-in-one convolutional neural network for face analysis

    Rajeev Ranjan, Swami Sankaranarayanan, Carlos D Castillo, and Rama Chellappa. An all-in-one convolutional neural network for face analysis. In 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017) , pages 17–24. IEEE, 2017

    2017

  33. [41]

    Swinface: a multi-task transformer for face recognition, expression recog- nition, age estimation and attribute estimation

    Lixiong Qin, Mei Wang, Chao Deng, Ke Wang, Xi Chen, Jiani Hu, and Weihong Deng. Swinface: a multi-task transformer for face recognition, expression recog- nition, age estimation and attribute estimation. IEEE Transactions on Circuits and Systems for Video Technology , 2023

    2023

  34. [42]

    Facexformer: A unified transformer for facial analysis

    Kartik Narayan, Vibashan VS, Rama Chellappa, and Vishal M Patel. Facexformer: A unified transformer for facial analysis. arXiv preprint arXiv:2403.12960, 2024

    2024 arXiv

  35. [43]

    Task- adaptive q-face

    Haomiao Sun, Mingjie He, Shiguang Shan, Hu Han, and Xilin Chen. Task- adaptive q-face. arXiv preprint arXiv:2405.09059, 2024

    2024 arXiv

  36. [44]

    Faceptor: A generalist model for face perception

    Lixiong Qin, Mei Wang, Xuannan Liu, Yuhang Zhang, Wei Deng, Xiaoshuai Song, Weiran Xu, and Weihong Deng. Faceptor: A generalist model for face perception. In European Conference on Computer Vision , pages 240–260. Springer, 2025

    2025

  37. [45]

    General facial repre- sentation learning in a visual-linguistic manner

    Yinglin Zheng, Hao Yang, Ting Zhang, Jianmin Bao, Dongdong Chen, Yangyu Huang, Lu Yuan, Dong Chen, Ming Zeng, and Fang Wen. General facial repre- sentation learning in a visual-linguistic manner. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognit...

    2022

  38. [46]

    Pre-training strategies and datasets for facial representation learning

    Adrian Bulat, Shiyang Cheng, Jing Yang, Andrew Garbett, Enrique Sanchez, and Georgios Tzimiropoulos. Pre-training strategies and datasets for facial representation learning. In European Conference on Computer Vision , pages 107–125. Springer, 2022

    2022

  39. [47]

    Label2label: A language modeling framework for multi-attribute learning

    Wanhua Li, Zhexuan Cao, Jianjiang Feng, Jie Zhou, and Jiwen Lu. Label2label: A language modeling framework for multi-attribute learning. In European Conference on Computer Vision , pages 562–579. Springer, 2022

    2022

  40. [48]

    Prompting visual-language models for dy- namic facial expression recognition

    Zengqun Zhao and Ioannis Patras. Prompting visual-language models for dy- namic facial expression recognition. arXiv preprint arXiv:2308.13382, 2023

    2023 arXiv

  41. [49]

    Emoclip: A vision-language method for zero-shot video facial expression recognition

    Niki Maria Foteinopoulou and Ioannis Patras. Emoclip: A vision-language method for zero-shot video facial expression recognition. In 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG) , pages 1–10. IEEE, 2024

    2024

  42. [50]

    Retinaface: Single-shot multi-level face localisation in the wild

    Jiankang Deng, Jia Guo, Evangelos Ververas, Irene Kotsia, and Stefanos Zafeiriou. Retinaface: Single-shot multi-level face localisation in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5203–5212, 2020

    2020

  43. [51]

    Maad-face: A massively annotated attribute dataset for face images

    Philipp Terhörst, Daniel Fährmann, Jan Niklas Kolf, Naser Damer, Florian Kirch- buchner, and Arjan Kuijper. Maad-face: A massively annotated attribute dataset for face images. IEEE Transactions on Information Forensics and Security , 16:3942– 3957, 2021

    2021

  44. [52]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pages 3730–3738, 2015

    2015

  45. [53]

    Fairface: Face attribute dataset for bal- anced race, gender, and age for bias measurement and mitigation

    Kimmo Karkkainen and Jungseock Joo. Fairface: Face attribute dataset for bal- anced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 1548– 1558, 2021

    2021

  46. [54]

    Age progression/regression by con- ditional adversarial autoencoder

    Song Yang Zhang, Zhifei and Hairong Qi. Age progression/regression by con- ditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017

    2017

  47. [55]

    From facial expression recognition to interpersonal relation prediction

    Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. From facial expression recognition to interpersonal relation prediction. International Journal of Computer Vision, 126:550–569, 2018

    2018

  48. [56]

    Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild

    Shan Li, Weihong Deng, and JunPing Du. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2852–2861, 2017

    2017

  49. [57]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  50. [58]

    Nvila: Efficient frontier visual language models

    Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. Nvila: Efficient frontier visual language models. arXiv preprint arXiv:2412.04468, 2024

    2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.