Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluator

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that an MLLM fine-tuned with chain-of-thought prompting can produce defect heatmaps, plausibility scores, and written explanations of AI-generated image flaws, and that these three outputs improve one another enough to…

desk verdict Solid heatmap/score system paper; explanation claim needs independent evaluation before it can be believed. read the letter →

arxiv 2411.17261 v2 pith:SOWULOXP submitted 2024-11-26 cs.CV cs.AI

Fan Yang , Ru Zhen , Jianing Wang , Yanhao Zhang , Haoxiang Chen , Haonan Lu , Sicheng Zhao , Guiguang Ding This is my paper · ORCID
classification cs.CVcs.AI
keywords AIGCimagequalityassessmentdefectheatmappredictionexplainableevaluationmultimodallargelanguagemodelschain-of-thoughtreasoninghierarchicalimplausibilitymappinguncertainty-basedfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HEIE is a single multimodal large language model that is fine-tuned to grade AI-generated images in three coordinated ways: a pixel-level heatmap of implausible regions, an overall plausibility score, and a written explanation that names the problem areas and why they fail. The paper argues that specialized defect-localization models cannot explain their findings or generalize well, while general-purpose MLLMs cannot localize tiny defects or emit pixel-level maps. Its central claim is that a chain-of-thought structure that makes the model describe the image, locate problematic regions, analyze their causes, and then score them lets all three outputs reinforce one another, producing state-of-the-art heatmap and score predictions while adding explanation that users can act on. If true, quality assessment of AI-generated images moves from a bare score or heatmap to a form that directly supports human understanding and targeted model repair.

What carries the argument

The load-bearing mechanism is a pair of special tokens inside the LLM sequence. A dynamic number of [MAP] tokens carries implausibility-region information: the hidden state of each token is extracted from the final LLM layer and fused with vision-encoder features through stacked two-way cross-attention to predict a heatmap, with more tokens assigned to higher-resolution images. A separate global [MAP] token yields a coarse whole-image heatmap, and an uncertainty-weighted fusion, using Laplace distribution standard deviations, combines the local and global maps into the final output. A [SCORE] token's hidden state is regressed through a small network and then calibrated against a score derived from the predicted heatmap, while the chain-of-thought prompt organizes the text generation so that description, localization, and analysis feed directly into the map and score tokens.

What would settle it

Construct a test set of AIGC images with defects that are independently verified by multiple humans and whose causes are known, then check whether HEIE's explanation names the verified cause for each defect while ignoring non-defective areas; if explanations match plausible-sounding generic text as often as verified causes, the explainability claim fails. A simpler version: if human raters cannot distinguish HEIE's explanations from fluent template text, the explanation evaluation is rewarding style, not content.

Watch

Extended reading notes

Core claim

The paper's central claim is that an MLLM can be adapted into a full explainable image implausibility evaluator rather than merely a textual critic. The CoT-Driven Explainable Trinity Evaluator decomposes the task into five ordered subtasks: image description, problematic-region identification, map-token injection, issue analysis, and score-token regression; the earlier textual steps are not decorative but feed the map and score tokens that produce the spatial and numeric outputs. The Adaptive Hierarchical Implausibility Mapper predicts a global heatmap and several local patch heatmaps, then fuses them with weights derived from Laplace-distribution uncertainty, so that small defects such as a distorted hand or eye do not get washed out by the global view. On the RichHF-18K benchmark the authors report the best heatmap metrics among the compared models, including an all-data MSE of 0.00825, and on their new Expl-AIGI-Eval dataset they report the best explanation scores by both GPT-4o-based comparison and human evaluation.

Load-bearing premise

The load-bearing premise is that the human-annotated defect masks are correct and that the LLM-written explanations, produced by prompting a commercial model on red-circled defect regions, are a trustworthy stand-in for true explanations of those defects; if either fails, the reported heatmap or explanation numbers measure something other than real implausibility understanding.

Editorial extensions

If this is right

  • If the central claim holds, one fine-tuned MLLM can replace the usual pipeline of separate heatmap and scoring models for AIGC quality control.
  • The chain-of-thought text becomes a working part of the architecture: forcing the model to describe and localize before scoring should make the spatial and scalar outputs more reliable, not just more readable.
  • Uncertainty-weighted local-global heatmap fusion should transfer to images of arbitrary resolution and aspect ratio without re-tuning the patch grid.
  • The explainable dataset makes it possible to train and benchmark other models on jointly predicting heatmaps, scores, and textual causes, so the contribution outlives this particular model.
  • Zero-shot transfer results between two defect datasets suggest that the world knowledge in the MLLM supplies generalization that small specialized models lack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the explanation metrics likely measure agreement with LLM-written reference text; they do not directly prove the explanations name the true physical cause of each defect.
  • Editorial inference: because the dataset's explanations are generated by a commercial LLM from red-circled regions, HEIE's explanation quality ceiling is that teacher LLM's ability to describe circled defects.
  • Editorial inference: the uncertainty-based fusion is a self-contained module that could be reused in non-MLLM heatmap predictors, for example segmenting localized artifacts in video frames or medical images.
  • Editorial inference: a sharper test would ask humans to localize the defect the explanation refers to or to judge whether the explanation's cause matches the heatmap region; the current human evaluation score does not separate fluency from accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes HEIE, an MLLM-based evaluator that jointly outputs implausibility heatmaps, a verisimilitude score, and free-form textual explanations for AIGC images. The method introduces an Adaptive Hierarchical Implausibility Mapper, which uses two-way cross-attention between [MAP] token features and image features and fuses local and global heatmaps via a Laplace-uncertainty weighted sum, plus a CoT-Driven Explainable Trinity Evaluator that sequences image description, problem-region identification, map-token injection, issue analysis, and score-token regression. The authors also construct Expl-AIGI-Eval by adding LLM-generated explanations to RichHF-18K and AbHuman. Experiments report state-of-the-art heatmap metrics on RichHF-18K (MSE 0.00825, KLD 1.634, CC 0.574, SIM 0.417, AUC-Judd 0.915) and AbHuman, improved score correlations, zero-shot cross-dataset generalization, and ablations for each proposed component.

Significance. If the results hold, HEIE is a meaningful advance: it is an MLLM-based system that produces pixel-level defect heatmaps together with textual rationales, and it outperforms specialized baselines on the heatmap and score benchmarks. The heatmap and score evaluations are grounded in external human-annotated masks from RichHF-18K and AbHuman, and the component ablations in Tables 5, 6, and 7 are internally consistent and support the main design choices. The principal caveat is that the explanation-quality evaluation is not independent of the label-generation process, so the headline 'explainable' claim is only partially secured by the current experiments.

major comments (4)
  1. [§3.4, Table 2, Fig. 3] The explanation branch is trained on and evaluated against texts produced by Claude-3.5-sonnet and reformatted by GPT-4o, and the two automated metrics in Table 2 are perplexity against those same texts and GPT-4o similarity to those same texts. Because HEIE is fine-tuned on exactly this annotation pipeline, the large margins in Table 2 likely measure how well HEIE imitates the LLM annotation style rather than how accurately it explains the actual image defects; the comparison against GPT-4o and Claude-3.5-Sonnet is particularly fragile because those are the models that generated the gold text. The human evaluation cited in Table 2 is reported without sample size, annotation protocol, rubric, or inter-annotator agreement, so it does not break the circularity. To support the central explainability claim, the authors should evaluate explanation factuality against the human-annotated masks (for example, whether each mentioned region overlaps a mask), use held-out LLM labelers that were not involved in training-data generation, or provide a complete human study with agreement statistics.
  2. [§3.2.1, Eq. (3)] The loss used to train the Implausibility Mapper is not precisely specified. In §3.2.1 the text says 'we simply adopt focal loss to overcome the data imbalance issue,' but Eq. (3) is a Laplace negative-log-likelihood with an additional log(σ) term, and no equation or text states how focal loss is combined with the uncertainty loss, nor how σ_l and σ_g are parameterized and optimized. The hyperparameters of the focal loss (α, γ) and the relative weighting between the heatmap loss and the uncertainty terms are also omitted. Because Table 5 attributes the final gain (Exp. e vs. d) to the uncertainty-based adaptive fusion, the reader cannot determine whether the reported improvements come from the fusion rule or from unstated training details; please provide the full combined loss and all hyperparameters.
  3. [Tables 1–7] All quantitative claims are reported as point estimates without error bars, multiple seeds, or significance tests. For example, the headline improvements over RAHF (augmented prompt) in Table 1 are small on several metrics (AUC-Judd 0.915 vs 0.913; PLCC 0.697 vs 0.693; SRCC 0.683 vs 0.681), and the zero-shot gains in Table 4 are smaller still; without variance estimates these differences cannot be distinguished from training noise. Please report mean±std over at least three runs or a paired significance test for the main comparisons.
  4. [§3.3.1, Table 6] The verisimilitude scorer is not fully specified: the calibration function S = Calib(S_token, S_map) is introduced without giving the concrete form used in the experiments, and the text states only that 'weighted summation and dynamic fusion' exhibited similar performance. Since Table 6 reports that combining token and heatmap features (Exp. d) is a key component of the score result, the exact calibration function and any learned parameters must be reported to make the method reproducible and to interpret the ablation.
minor comments (4)
  1. [§1, §3.1] There are typographical errors: 'possessing extensive knowledge and common' in §1 should end with 'sense,' and 'Co-driven' in §3.1 should be 'CoT-driven.'
  2. [Tables 3 and 4] Baseline names are inconsistent across tables (InternViT, InternViT-300MB, InternViT-300M), and the text does not state whether these refer to the same model variant; please unify the notation.
  3. [§4.2, Table 1] The RAHF numbers are taken from the original paper; the manuscript should state explicitly that the evaluation split, preprocessing, and metrics are identical, since RAHF is not available for re-evaluation.
  4. [§3.4] The Expl-AIGI-Eval dataset description lacks basic statistics, such as number of images, number of explanations, average explanation length, and number of regions per image, as well as quality-control details beyond the ethical filtering mentioned in the text; these should be reported for the dataset contribution to be assessable.

Circularity Check

1 steps flagged · score 4.0 of 10

Explanation-quality evaluation is partially circular: HEIE is trained on Claude-3.5-sonnet/GPT-4o-generated explanation labels and then judged by perplexity and GPT-4o similarity to those same labels; heatmap and score claims rest on independent human masks.

  1. fitted input called prediction [Sec. 3.4; Sec. 4.1.1; Sec. 4.2 (Tab. 2)]
    "(2) LLM Free-Form Outputting: Using Claude-3.5-sonnet [3], we generate detailed analyses of the defect regions. ... (3) In-Context Learning-Based Formatting: ... we employ GPT-4o and in-context learning to format the defect analysis texts obtained in Stage 2. ... We employ three evaluation methods to evaluate the models' text explanation capability: (1) Compare the perplexity (ppl) of model outputs to ground truth; (2) Employ GPT-4o to compare the content similarity between the model outputs and the ground truth."

    The Expl-AIGI-Eval 'ground-truth' explanations are themselves LLM outputs: Claude-3.5-sonnet wrote the free-form defect analyses and GPT-4o reformatted them. HEIE is fine-tuned on this exact corpus (Sec. 4.1.1). Table 2 then measures explanation quality as perplexity of outputs against those LLM-generated labels and as GPT-4o's content-similarity score between outputs and those same labels. Consequently, the automatic explanation metrics reward close imitation of the annotation pipeline's style rather than independent verification that the stated defects exist in the image; HEIE's large margins over GPT-4o and Claude-3.5-Sonnet in Table 2 are the expected result of supervised training on their output distribution.

full rationale

The central heatmap and score claims are self-contained: Tables 1, 3, and 4 compare against human-annotated defect masks in RichHF-18K and AbHuman, and the ablations (Tables 5-7) support the mapper and scorer design without relying on self-citation. The explanation-quality claim is the only partially circular component. The gold explanations in Expl-AIGI-Eval were generated by Claude-3.5-sonnet and reformatted by GPT-4o, and Table 2's automatic metrics (perplexity to ground truth and GPT-4o similarity to ground truth) evaluate against those same LLM-generated texts. Because HEIE is fine-tuned on those texts, the GPT-4o Eval and perplexity scores partly measure distribution matching to the annotation pipeline. The paper does report a human evaluation row, which would be an independent check, but no protocol details are provided. Numerous self-citations appear in the related-work and generalization sections, but none is load-bearing for the architecture or the main results, so they do not raise the score. Overall, the system's heatmap and score performance is independently supported; only the 'explainability' superiority claim is undercut by the training and evaluation loop, yielding a score of 4 rather than 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The model is a standard supervised deep learning system; the free parameters are the learned uncertainty and calibration weights plus unspecified hyperparameters. The main ad hoc-to-paper assumption is treating LLM-generated text as ground truth for explanations. No new physical entities are introduced; the [MAP] and [SCORE] tokens are internal architecture tokens.

free parameters (4)
  • Uncertainty standard deviations sigma_l and sigma_g for local-global heatmap fusion = not reported
    Learned via Eq. 3 and converted into confidence weights p = e^{-sigma} at inference; these fitted uncertainties directly determine the final heatmap and are central to the 'adaptive' fusion claim.
  • Score calibration parameters in Calib(S_token, S_map) = not reported
    Sec. 3.3.1 says several calibration functions (weighted sum, dynamic fusion) were tried and exhibited similar performance, but the chosen form and weights are unspecified; they are fitted to score labels.
  • Local patch number N and adaptive token count = not reported
    Sec. 3.2.3 adjusts the number of local map tokens based on image resolution and aspect ratio; the exact schedule is not given and affects local heatmap quality.
  • Focal loss hyperparameters and loss weighting = not reported
    The paper says focal loss is adopted to address data imbalance but does not specify how it is combined with the Laplace uncertainty loss in Eq. 3 or the values of focal loss hyperparameters.
assumptions (5)
  • domain assumption AIGC implausibility can be decomposed into local patch-level defects and one global whole-image defect map.
    Sec. 3.2.2 assumes this dichotomy; defects crossing patch boundaries or requiring mid-level context may not be captured.
  • domain assumption The hidden state of a special [MAP] token, after two-way cross-attention with ViT image features, can be decoded into a pixel-wise heatmap.
    Sec. 3.2.1 states this mechanism without proof; its validity is the core empirical bet of the mapper.
  • domain assumption Chain-of-thought text (image description, region identification, issue analysis) improves heatmap and score prediction.
    Supported only by the paper's own ablation in Tab. 7; treated as a general property of LLMs.
  • ad hoc to paper LLM-generated explanations (Claude-3.5-sonnet plus GPT-4o formatting) are valid ground truth for image implausibility explanations.
    Sec. 3.4 constructs the dataset without human verification; this assumption is load-bearing for all explanation results.
  • domain assumption RichHF-18K and AbHuman human mask annotations are accurate enough to supervise heatmap prediction.
    Taken as given from prior datasets; no relabeling or quality check is reported in this paper.
invented entities (2)
  • [MAP] special token
    purpose: Aggregates implausibility-region information from the LLM and, via cross-attention to image features, decodes into heatmaps.
    An internal learnable token with no external falsifiable handle; its value is only measured through downstream heatmap metrics.
  • [SCORE] special token
    purpose: Carries verisimilitude information regressed by a feed-forward network to a scalar score.
    Internal token; no independent evidence outside the paper's score prediction experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluator." pith.science (2026). https://pith.science/paper/SOWULOXP

@misc{pith2026241117261,
  author       = {Pith},
  title        = {Pith review of: HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluator},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SOWULOXP}},
  note         = {Machine review of arXiv:2411.17261}
}
read the original abstract

AIGC images are prevalent across various fields, yet they frequently suffer from quality issues like artifacts and unnatural textures. Specialized models aim to predict defect region heatmaps but face two primary challenges: (1) lack of explainability, failing to provide reasons and analyses for subtle defects, and (2) inability to leverage common sense and logical reasoning, leading to poor generalization. Multimodal large language models (MLLMs) promise better comprehension and reasoning but face their own challenges: (1) difficulty in fine-grained defect localization due to the limitations in capturing tiny details, and (2) constraints in providing pixel-wise outputs necessary for precise heatmap generation. To address these challenges, we propose HEIE: a novel MLLM-Based Hierarchical Explainable Image Implausibility Evaluator. We introduce the CoT-Driven Explainable Trinity Evaluator, which integrates heatmaps, scores, and explanation outputs, using CoT to decompose complex tasks into subtasks of increasing difficulty and enhance interpretability. Our Adaptive Hierarchical Implausibility Mapper synergizes low-level image features with high-level mapper tokens from LLMs, enabling precise local-to-global hierarchical heatmap predictions through an uncertainty-based adaptive token approach. Moreover, we propose a new dataset: Expl-AIGI-Eval, designed to facilitate interpretable implausibility evaluation of AIGC images. Our method demonstrates state-of-the-art performance through extensive experiments. Our project is at https://yfthu.github.io/HEIE/.

Figures

Figures reproduced from arXiv: 2411.17261 by the authors.

Figure 1
Figure 1. (a) Specialized models lack the ability to explain and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The Implausibility Mapper processes a dynamic number of special [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Three stages in Expl-AIGI-Eval dataset construction: Stage 1 - Visual Prompting: Defect regions are circled on images to aid Claude-3.5-sonnet in accurately locating problem areas. Stage 2 - LLM Free-Form Output: Claude-3.5-sonnet generates free-form defect location and analysis. Stage 3 - In-Context Learning-Based Formatting: GPT-4o is used for format standardization. gion information into the special map token for… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Model outputs of HEIE. HEIE not only predicts implausibility heatmaps but also provides image descriptions, problematic regions identification, analysis of issues, and score, achieving reliable and explainable implausibility evaluation. Note that for the last AIGC imag…
Figure 5
Figure 5. Figure 5: Comparison with Baselines. Each set of images, from left to right, includes: (a) Input Image, (b) Output of InternViT-300M￾448px, (c) Output of CLIP-ViT-Base-Patch16, (d) Output of ours HEIE, (e) Ground Truth. prediction tasks, we report the Pearson linear correlation …
Figure 6
Figure 6. Figure 6: Results of Hierarchical Implausibility Mapper. Columns (c) and (d) are the global and local heatmaps from our Hierarchical Implausibility Mapper. Column (e) illustrates the fi￾nal heatmap after adaptively fusing the global and local heatmaps. Perplexity ↓ GPT-4o Eval ↑…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models

    cs.CV 2025-07 conditional novelty 6.0 of 10

    AIGI-Holmes combines visual expert pretraining, SFT on explanation data, and direct preference optimization to deliver human-verifiable explanations and top detection accuracy on unseen AI generators.

  2. DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for Parameter-Efficient Video-Text Retrieval

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A parameter-efficient video-text retrieval method that trains only 0.56M parameters on top of frozen CLIP and achieves 50.5% R@1 on MSRVTT.

Reference graph

Works this paper leans on

58 extracted references · 32 canonical work pages · cited by 2 Pith papers

  1. [1]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv: 2010.11929, 2020

    Dosovitskiy Alexey. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv: 2010.11929, 2020. 2, 4

    arXiv 2010

  2. [2]

    Explainable ar- tificial intelligence: an analytical review.Wiley Interdisci- plinary Reviews: Data Mining and Knowledge Discovery, 11(5):e1424, 2021

    Plamen P Angelov, Eduardo A Soares, Richard Jiang, Nicholas I Arnold, and Peter M Atkinson. Explainable ar- tificial intelligence: an analytical review.Wiley Interdisci- plinary Reviews: Data Mining and Knowledge Discovery, 11(5):e1424, 2021. 3 8

    work page 2021

  3. [3]

    Claude 3.5 sonnet: A next-generation language model

    Anthropic. Claude 3.5 sonnet: A next-generation language model. Online, 2024. Version 3.5, Accessed: 2025-03-12. 5

    work page 2024

  4. [4]

    Qwen-vl: A frontier large vision-language model with versatile abilities.arXiv preprint arXiv:2308.12966, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities.arXiv preprint arXiv:2308.12966, 2023. 2, 7

    arXiv 2023

  5. [5]

    Improving image generation with better captions.Computer Science

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions.Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 2

    2023

  6. [6]

    What do different evaluation metrics tell us about saliency models?IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(3):740–757, 2018

    Zoya Bylinskii, Tilke Judd, Aude Oliva, Antonio Torralba, and Fr´edo Durand. What do different evaluation metrics tell us about saliency models?IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(3):740–757, 2018. 5, 6

    work page 2018

  7. [7]

    Visualgpt: Data-efficient adaptation of pretrained language models for image captioning

    Jun Chen, Han Guo, Kai Yi, Boyang Li, and Mohamed El- hoseiny. Visualgpt: Data-efficient adaptation of pretrained language models for image captioning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18030–18040, 2022. 5

    work page 2022

  8. [8]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhang- wei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024. 4

    arXiv 2024

Show all 58 references
  1. [9]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. InPro- ceedings of the IEEE/CVF Conference on Computer ...

    2024

  2. [10]

    Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks

    Xiaohan Ding, Yuchen Guo, Guiguang Ding, and Jungong Han. Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks. InProceedings of the IEEE/CVF international conference on computer vision, pages 1911–1920, 2019. 7

    1911

  3. [11]

    Repvgg: Making vgg-style convnets great again

    Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun. Repvgg: Making vgg-style convnets great again. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 13733–13742, 2021. 7

    2021

  4. [12]

    Ex- plainable artificial intelligence: A survey

    Filip Karlo Do ˇsilovi´c, Mario Br ˇci´c, and Nikica Hlupi ´c. Ex- plainable artificial intelligence: A survey. In2018 41st International Convention on Information and Communica- tion Technology, Electronics and Microelectronics (MIPRO), pages 0210–0215. IEEE, 2018. 3

    2018

  5. [13]

    Hu- manrefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance.arXiv preprint arXiv:2407.06937, 2024

    Guian Fang, Wenbiao Yan, Yuanfan Guo, Jianhua Han, Zu- tao Jiang, Hang Xu, Shengcai Liao, and Xiaodan Liang. Hu- manrefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance.arXiv preprint arXiv:2407.06937, 2024. 3, 5, 7, 8

    2024 arXiv

  6. [14]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools.arXiv preprint arXiv:2406.12793, 2024

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools.arXiv preprint arXiv:2406.12793, 2024. 2, 7

    2024 arXiv

  7. [15]

    Quantized prompt for efficient generalization of vision-language models.arXiv preprint arXiv:2407.10704, 2024

    Tianxiang Hao, Xiaohan Ding, Juexiao Feng, Yuhong Yang, Hui Chen, and Guiguang Ding. Quantized prompt for efficient generalization of vision-language models.arXiv preprint arXiv:2407.10704, 2024. 7

    2024 arXiv

  8. [16]

    Secret: Self-consistent pseudo label refinement for unsupervised domain adaptive person re-identification

    Tao He, Leqi Shen, Yuchen Guo, Guiguang Ding, and Zhen- hua Guo. Secret: Self-consistent pseudo label refinement for unsupervised domain adaptive person re-identification. InProceedings of the AAAI conference on artificial intelli- gence, pages 879–887, 2022. 7

    2022

  9. [17]

    Tifa: Accu- rate and interpretable text-to-image faithfulness evaluation with question answering

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accu- rate and interpretable text-to-image faithfulness evaluation with question answering. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages...

    2023

  10. [18]

    Aesexpert: Towards multi-modality foun- dation model for image aesthetics perception

    Yipo Huang, Xiangfei Sheng, Zhichao Yang, Quan Yuan, Zhichao Duan, Pengfei Chen, Leida Li, Weisi Lin, and Guangming Shi. Aesexpert: Towards multi-modality foun- dation model for image aesthetics perception. InProceedings of the 32nd ACM International Conference on Multimedia, ...

    2024

  11. [19]

    Aesbench: An expert benchmark for multimodal large language models on image aesthetics perception.arXiv preprint arXiv:2401.08276, 2024

    Yipo Huang, Quan Yuan, Xiangfei Sheng, Zhichao Yang, Haoning Wu, Pengfei Chen, Yuzhe Yang, Leida Li, and Weisi Lin. Aesbench: An expert benchmark for multimodal large language models on image aesthetics perception.arXiv preprint arXiv:2401.08276, 2024. 3

    2024 arXiv

  12. [20]

    Visualcritic: Making lmms perceive visual quality like humans.arXiv preprint arXiv:2403.12806, 2024

    Zhipeng Huang, Zhizheng Zhang, Yiting Lu, Zheng-Jun Zha, Zhibo Chen, and Baining Guo. Visualcritic: Making lmms perceive visual quality like humans.arXiv preprint arXiv:2403.12806, 2024. 3

    2024 arXiv

  13. [21]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 4

    2023

  14. [22]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36: 36652–36663, 2023. 1, 3

    2023

  15. [23]

    Large language models are zero-shot reasoners.Advances in Neural Information Pro- cessing Systems, 35:22199–22213, 2022

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners.Advances in Neural Information Pro- cessing Systems, 35:22199–22213, 2022. 2

    2022

  16. [24]

    Agiqa-3k: An open database for ai-generated image quality assessment.IEEE Transactions on Circuits and Sys- tems for Video Technology, 2023

    Chunyi Li, Zicheng Zhang, Haoning Wu, Wei Sun, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, and Weisi Lin. Agiqa-3k: An open database for ai-generated image quality assessment.IEEE Transactions on Circuits and Sys- tems for Video Technology, 2023. 3

    2023

  17. [25]

    Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds.Advances in Neural Information Pro- cessing Systems, 36, 2024

    Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds.Advances in Neural Information Pro- cessing Systems, 36, 2024. 2

    2024

  18. [26]

    Rich human feedback for 9 text-to-image generation

    Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, et al. Rich human feedback for 9 text-to-image generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2024

  19. [27]

    Deepseek-vl: towards real-world vision- language understanding.arXiv preprint arXiv:2403.05525,

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. Deepseek-vl: towards real-world vision- language understanding.arXiv preprint arXiv:2403.05525,

    arXiv

  20. [28]

    Geometry uncer- tainty projection network for monocular 3d object detection

    Yan Lu, Xinzhu Ma, Lei Yang, Tianzhu Zhang, Yating Liu, Qi Chu, Junjie Yan, and Wanli Ouyang. Geometry uncer- tainty projection network for monocular 3d object detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3111–3121, 2021. 4

    2021

  21. [29]

    Midjourney: Ai-based image synthesis tool

    MidJourney. Midjourney: Ai-based image synthesis tool. https://www.midjourney.com, 2023. Accessed: 2023-10-01. 2

    2023

  22. [30]

    Dreambench++: A human-aligned bench- mark for personalized image generation.arXiv preprint arXiv:2406.16855, 2024

    Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. Dreambench++: A human-aligned bench- mark for personalized image generation.arXiv preprint arXiv:2406.16855, 2024. 3

    2024 arXiv

  23. [31]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InInternational Conference on Machine Learning,...

    2021

  24. [32]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 2

    2022

  25. [33]

    Focal loss for dense object detection

    Tsung-Yi Lin Priya Goyal Ross and Girshick Kaiming He Pi- otr Doll ´ar. Focal loss for dense object detection. InPro- ceedings of the IEEE International Conference on Computer Vision, page 2980–2988, 2017. 4

    2017

  26. [34]

    Tempme: Video temporal token merging for efficient text- video retrieval.arXiv preprint arXiv:2409.01156, 2024

    Leqi Shen, Tianxiang Hao, Tao He, Sicheng Zhao, Yifeng Zhang, Pengzhang Liu, Yongjun Bao, and Guiguang Ding. Tempme: Video temporal token merging for efficient text- video retrieval.arXiv preprint arXiv:2409.01156, 2024. 3

    2024 arXiv

  27. [35]

    Fastvid: Dynamic density pruning for fast video large language mod- els.arXiv preprint arXiv:2503.11187, 2025

    Leqi Shen, Guoqiang Gong, Tao He, Yifeng Zhang, Pengzhang Liu, Sicheng Zhao, and Guiguang Ding. Fastvid: Dynamic density pruning for fast video large language mod- els.arXiv preprint arXiv:2503.11187, 2025

    2025

  28. [36]

    Llava-mlb: Mitigating and leveraging attention bias for training-free video llms.arXiv preprint arXiv:2503.11205,

    Leqi Shen, Tao He, Guoqiang Gong, Fan Yang, Yifeng Zhang, Pengzhang Liu, Sicheng Zhao, and Guiguang Ding. Llava-mlb: Mitigating and leveraging attention bias for training-free video llms.arXiv preprint arXiv:2503.11205,

    arXiv

  29. [37]

    Eva-clip- 18b: Scaling clip to 18 billion parameters.arXiv preprint arXiv:2402.04252, 2024

    Quan Sun, Jinsheng Wang, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, and Xinlong Wang. Eva-clip- 18b: Scaling clip to 18 billion parameters.arXiv preprint arXiv:2402.04252, 2024. 7, 8

    2024 arXiv

  30. [38]

    Let me speak freely? a study on the impact of format restrictions on performance of large language models.arXiv preprint arXiv:2408.02442,

    Zhi Rui Tam, Cheng-Kuang Wu, Yi-Lin Tsai, Chieh-Yen Lin, Hung-yi Lee, and Yun-Nung Chen. Let me speak freely? a study on the impact of format restrictions on performance of large language models.arXiv preprint arXiv:2408.02442,

    arXiv

  31. [39]

    Zhiyu Tan, Xiaomeng Yang, Luozheng Qin, Mengping Yang, Cheng Zhang, and Hao Li. Evalalign: Evaluating text- to-image models through precision alignment of multimodal large models with supervised fine-tuning to human annota- tions.arXiv preprint arXiv:2406.16562, 2024. 3

    2024 arXiv

  32. [40]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017. 4

    2017

  33. [41]

    Repvit: Revisiting mobile cnn from vit perspective

    Ao Wang, Hui Chen, Zijia Lin, Jungong Han, and Guiguang Ding. Repvit: Revisiting mobile cnn from vit perspective. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15909–15920, 2024. 7

    2024

  34. [42]

    Yolov10: Real-time end-to-end object de- tection.Advances in Neural Information Processing Systems, 37:107984–108011, 2024

    Ao Wang, Hui Chen, Lihao Liu, Kai Chen, Zijia Lin, Jun- gong Han, et al. Yolov10: Real-time end-to-end object de- tection.Advances in Neural Information Processing Systems, 37:107984–108011, 2024. 7

    2024

  35. [43]

    Lode: Deep local deblurring and a new benchmark

    Zerun Wang, Liuyu Xiang, Fan Yang, Jinzhao Qian, Jie Hu, Haidong Huang, Jungong Han, Yuchen Guo, and Guiguang Ding. Lode: Deep local deblurring and a new benchmark. arXiv preprint arXiv:2109.09149, 2021. 7

    2021 arXiv

  36. [44]

    Chain-of-thought prompting elicits reasoning in large lan- guage models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022. 2

    2022

  37. [45]

    Q-bench: A benchmark for general-purpose foundation models on low-level vision

    Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, et al. Q-bench: A benchmark for general-purpose foundation models on low-level vision. arXiv preprint arXiv:2309.14181, 2023. 3

    2023 arXiv

  38. [46]

    Q-align: Teaching lmms for visual scoring via discrete text-defined levels.arXiv preprint arXiv:2312.17090, 2023

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching lmms for visual scoring via discrete text-defined levels.arXiv preprint arXiv:2312.17090, 2023. 3

    2023 arXiv

  39. [47]

    Q-instruct: Improving low-level visual abilities for multi-modality foundation models

    Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, et al. Q-instruct: Improving low-level visual abilities for multi-modality foundation models. In Proceedings of the IEEE/CVF Conference on Computer V...

    2024

  40. [48]

    Towards open-ended visual quality comparison

    Haoning Wu, Hanwei Zhu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Annan Wang, Wenxiu Sun, Qiong Yan, et al. Towards open-ended visual quality comparison. InEuropean Conference on Computer Vision, pages 360–377. Springer, 2025. 3

    2025

  41. [49]

    A comprehensive study of multimodal large lan- guage models for image quality assessment.arXiv preprint arXiv:2403.10854, 2024

    Tianhe Wu, Kede Ma, Jie Liang, Yujiu Yang, and Lei Zhang. A comprehensive study of multimodal large lan- guage models for image quality assessment.arXiv preprint arXiv:2403.10854, 2024. 3

    2024 arXiv

  42. [50]

    Human preference score: Better aligning text- to-image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hong- sheng Li. Human preference score: Better aligning text- to-image models with human preference. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 2096–2105, 2023. 1, 3

    2023

  43. [51]

    Imagere- 10 ward: Learning and evaluating human preferences for text- to-image generation.Advances in Neural Information Pro- cessing Systems, 36, 2024

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- 10 ward: Learning and evaluating human preferences for text- to-image generation.Advances in Neural Information Pro- cessing Systems, 36, 2024. 1, 3

    2024

  44. [52]

    Ground plane matters: Pick- ing up ground plane prior in monocular 3d object detection

    Fan Yang, Xinhao Xu, Hui Chen, Yuchen Guo, Jungong Han, Kai Ni, and Guiguang Ding. Ground plane matters: Pick- ing up ground plane prior in monocular 3d object detection. arXiv preprint arXiv:2211.01556, 2022. 7

    2022 arXiv

  45. [53]

    Gpro3d: Deriving 3d bbox from ground plane in monocular 3d object detection.Neurocom- puting, 562:126894, 2023

    Fan Yang, Xinhao Xu, Hui Chen, Yuchen Guo, Yuwei He, Kai Ni, and Guiguang Ding. Gpro3d: Deriving 3d bbox from ground plane in monocular 3d object detection.Neurocom- puting, 562:126894, 2023. 7

    2023

  46. [54]

    Geometry-guided do- main generalization for monocular 3d object detection

    Fan Yang, Hui Chen, Yuwei He, Sicheng Zhao, Chenghao Zhang, Kai Ni, and Guiguang Ding. Geometry-guided do- main generalization for monocular 3d object detection. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 6467–6476, 2024. 7

    2024

  47. [55]

    Llmi3d: Empowering llm with 3d perception from a single 2d image.arXiv preprint arXiv:2408.07422, 2024

    Fan Yang, Sicheng Zhao, Yanhao Zhang, Haoxiang Chen, Hui Chen, Wenbo Tang, Haonan Lu, Pengfei Xu, Zhenyu Yang, Jungong Han, et al. Llmi3d: Empowering llm with 3d perception from a single 2d image.arXiv preprint arXiv:2408.07422, 2024. 3

    2024 arXiv

  48. [56]

    Depicting beyond scores: Advanc- ing image quality assessment through multi-modal language models.arXiv preprint arXiv:2312.08962, 2023

    Zhiyuan You, Zheyuan Li, Jinjin Gu, Zhenfei Yin, Tianfan Xue, and Chao Dong. Depicting beyond scores: Advanc- ing image quality assessment through multi-modal language models.arXiv preprint arXiv:2312.08962, 2023. 3

    2023 arXiv

  49. [57]

    Automatic chain of thought prompting in large language models.arXiv preprint arXiv:2210.03493, 2022

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models.arXiv preprint arXiv:2210.03493, 2022. 2

    2022 arXiv

  50. [58]

    A-bench: Are lmms masters at evaluat- ing ai-generated images?arXiv preprint arXiv:2406.03070,

    Zicheng Zhang, Haoning Wu, Chunyi Li, Yingjie Zhou, Wei Sun, Xiongkuo Min, Zijian Chen, Xiaohong Liu, Weisi Lin, and Guangtao Zhai. A-bench: Are lmms masters at evaluat- ing ai-generated images?arXiv preprint arXiv:2406.03070,

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.