Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-body

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HumanCalibrator detects and repairs impossible body parts in AI-generated people, reporting 80.69% average accuracy on absent abnormalities and 58.57% on redundant ones.

desk verdict Useful new task and datasets, but the headline localization accuracy is not demonstrated because the evaluation is self-referential. read the letter →

arxiv 2411.14205 v1 pith:DO5KH55S submitted 2024-11-21 cs.CV cs.AI

Zeqing Wang , Qingyang Ma , Wentao Wan , Haojie Li , Keze Wang , Yonghong Tian This is my paper · ORCID
classification cs.CVcs.AI
keywords FHADabnormalhumanbodydetectionAIGCvision-languagemodeldiffusioninpaintingbody-partcorrelationimagerepairtext-to-imageevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Fine-grained Human-body Abnormality Detection (FHAD): given an image or video frame of a person, a system must say which body part is abnormal, whether it is absent or redundant, and where it is located. The authors build two datasets, one synthesized from COCO by masking body parts and one hand-labeled from AI-generated videos, and show that powerful vision-language models perform near random guessing on this task. They then propose HumanCalibrator, which detects absent parts with a vision-language model trained on masked real photos to exploit body-part correlations, and detects redundant parts by using a diffusion inpainter to regenerate each grounded body part and comparing semantics before and after. On the AIGC Human-Aware 1K test set, the framework reaches 80.69% mean accuracy for absent parts and 58.57% for redundant parts, and it repairs the abnormal region while preserving the rest of the image. The central claim is that a post-processing step can turn anatomically impossible generated people into plausible ones without regenerating the whole picture.

What carries the argument

The load-bearing mechanism is the pairing of two complementary detectors. AHD is a LLaVA-1.5-7B vision-language model fine-tuned with an autoregressive objective on automatically masked COCO images, so it learns correlations among body parts and can name both the missing part and its bounding box. The redundant-part detector uses a grounding model to list body parts, a Stable Diffusion inpainting model to regenerate each part in place, and a semantic comparison to flag parts whose regenerated content differs strongly from the original; this makes it sensitive to parts appearing anywhere in the frame. The same inpainter then repairs flagged regions with a prompt template, and the loop terminates when AHD reports no further absences.

What would settle it

Hand-annotate the abnormal regions in AIGC Human-Aware 1K with bounding boxes and compute IoU between HumanCalibrator's predicted boxes and the human boxes; if the repair-quality-based accuracy stays high while IoU is near chance, the central detection claim is not supported, whereas high IoU would corroborate the paper's evaluation.

Watch

Extended reading notes

Core claim

HumanCalibrator is a two-part detection-and-repair framework for generated human images. For absent body parts, a fine-tuned vision-language model called the Absent Human-body Detector (AHD) learns from COCO images in which one body part has been replaced by background, so it can infer a missing hand, ear, arm, leg, foot, or head from the position and identity of the remaining parts. For redundant body parts, HumanCalibrator grounds all visible body parts, inpaints each one with a diffusion model conditioned on the part name, and compares semantic content before and after; a large difference marks that part as redundant. The same inpainting model then repairs the flagged regions, cycling until no new absences appear. The paper reports 80.69% average accuracy on absent abnormalities and 58.57% on redundant abnormalities on the hand-labeled AIGC Human-Aware 1K, with false discovery rates of 8.48% and 2.54% respectively, and better visual consistency than a pose-conditioned baseline (FID 16.55 versus 98.86).

Load-bearing premise

The paper measures how accurately HumanCalibrator locates an abnormality by how well the repaired image looks after inpainting that region, because the test set has no bounding-box annotations; if repair quality is not a faithful proxy for localization accuracy, the reported detection numbers do not establish that the box is the right box.

Editorial extensions

If this is right

  • Generated human images can be post-processed automatically: absent hands, ears, and limbs get regenerated, and extra body parts get removed, without regenerating the background or the rest of the person.
  • Video generation can be fixed at low cost by repairing only the first and last frames and letting a keyframe interpolation model rebuild the intermediate frames.
  • The FHAD datasets give the community a benchmark on which current vision-language models score near random, so progress on anatomical abnormality perception becomes measurable.
  • Because AHD was trained only on real COCO images and still transfers to AI-generated frames, the correlation-based training recipe may extend to other structured-object defects without expensive AIGC annotation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper's localization metric is repair-based, detection accuracy and inpainting strength are entangled; an independent bounding-box benchmark would be needed to tell whether HumanCalibrator finds the right region or merely repairs whatever region it is given.
  • The same masked-part correlation training could be applied to other objects with fixed part geometry, such as animals, faces, or vehicles, making the approach a general post-hoc sanity checker for generative models.
  • A concrete testable extension is to run HumanCalibrator on images with one obvious correctable abnormality and a second, subtler abnormality, to see whether the cyclic detection stops prematurely once the first repair succeeds.
  • Because the redundant-part detector relies on the inpainting model's prior over normal bodies, its accuracy on a new generator may degrade if that generator's artifacts violate the inpainter's learned normality; a per-generator calibration study would reveal this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper introduces the Fine-grained Human-body Abnormality Detection (FHAD) task and proposes HumanCalibrator, a framework that detects and repairs abnormal human body parts in AIGC images and videos. The authors construct two datasets: COCO Human-Aware Val, with synthetically masked body parts, and AIGC Human-Aware 1K, with manually labeled abnormality types but no bounding-box annotations. The method has two detection branches: an Absent Human-body Detector (AHD), a fine-tuned VLM trained on masked COCO images, and a redundant-part detector that compares original body-part regions with inpainted versions via a grounding model. Detected regions are repaired using a diffusion inpainting model. The main reported results are 80.69% average absent-detection accuracy and 58.57% redundant-detection accuracy on AIGC Human-Aware 1K, together with repair-quality and visual-consistency metrics.

Significance. The paper identifies a genuinely under-studied and practically important failure mode of AIGC human image generation, and its evidence that strong VLMs such as GPT-4o perform near random guessing on this task is a useful negative result. The proposed datasets, especially the filtered AIGC Human-Aware 1K, have potential value as benchmarks. The framework is architecturally sensible, and the image, video, and cross-model case studies are suggestive. However, the central detection-accuracy claim for the AIGC domain currently rests on a self-referential evaluation in which repair quality is used to validate detection localization. If that evaluation is corrected with human-annotated boxes or a human study, the contribution would be substantially strengthened; as presented, the accuracy numbers in Table 1 should be read as measuring detector-inpainter coupling rather than localization quality.

major comments (4)
  1. [Section 5.1 / Figure 3 caption / Table 1] The evaluation of detection localization on AIGC Human-Aware 1K is circular. The caption of Figure 3 and Section 5.1 state that, because abnormal-region locations are ambiguous, the authors do not annotate bounding boxes and instead evaluate bounding-box accuracy by repair quality. However, the repair is generated by the same inpainting model R used in the redundant detector (Eq. 8) and in the absent-detection loop (Eq. 9). A predicted box whose contents are easy to regenerate can therefore be counted as correct even if it does not cover the actual abnormality, while a correct box that the inpainter fills poorly would be counted as wrong. Table 1 thus does not demonstrate localization accuracy on AIGC Human-Aware 1K. I ask for human-annotated boxes (or a human study judging predicted boxes against manually marked regions) on this dataset, or a re-framing of the reported numbers as repair-quality scores rather than detection accuracy.
  2. [Table 1] The per-category sample sizes in Table 1 are small and unreported, and no confidence intervals or error bars are given. For example, the redundant rows contain values such as 33.33% for leg, 83.33% for ear, and 66.67% for foot; given the 158 redundant instances reported in Table S3, these percentages likely correspond to only a few examples per class. With denominators this small, the per-category ranking of methods is not statistically meaningful, and even the aggregate 58.57% redundant accuracy needs an uncertainty estimate. Please report exact per-class counts and, where possible, binomial confidence intervals.
  3. [Eq. (8) / Section 4.2] The redundant-part detector depends on a free threshold tau in Eq. (8), but the paper does not specify its value, how it was selected, or how sensitive the Table 1 results are to it. Moreover, the expression 'G(R(pg_i, bg_i), pg_i) < tau' is not formally defined as a semantic-difference quantity; the comparison needs a precise grounding-confidence or similarity formulation. Without this information the redundant-detection accuracy is not reproducible and could reflect threshold tuning rather than the method itself.
  4. [Table 2 / Section 5.1] The repair-quality evidence in Table 2 is weak: the reported improvements are 22.59 to 22.77 in Human Concept Score, 41.87 to 41.97 in CLIP Score, and 26.36 to 26.42 in Human CLIP Score, with no variance or significance testing. These differences are within the noise one would expect from CLIP embedding similarity, so the conclusion that HumanCalibrator improves repair quality is not established by these metrics. The FID comparison against a pose-conditioned full-image regeneration (98.86 vs. 16.55) is also not an apples-to-apples comparison with a fine-grained repair method, since full-image regeneration is expected to have higher FID by construction.
minor comments (6)
  1. [Title / Abstract] The title 'Is this Generated Person Existed in Real-world?' and the abstract phrase 'the realistic of generated human photos' contain grammatical errors that should be corrected.
  2. [Figure 6] Figure 6 contains a stray '图表标题' ('chart title' in Chinese) placeholder that appears to be a leftover editing artifact and should be removed.
  3. [Appendix A.2] Please clarify whether the HumanCalibrator results in Table 2 include the 2x super-resolution step; the text says no SR is applied in comparisons, but the pipeline description includes SR as a component, leaving readers unsure which configuration is evaluated.
  4. [Eq. (5)] The definition of L as 'the length of the concatenated instruction Ia and the perception and position of the current absent body part' is awkward and should be rephrased; the superscript notation in the autoregressive probability is also difficult to parse and should be simplified.
  5. [Section 3 / Appendix D] The paper says 'we highly recommend reading this section' about Appendix D; this informal phrasing, combined with the fact that the annotation protocol is only in the supplement, suggests the essential dataset-construction details should be summarized in the main text.
  6. [Conclusion / Figure S5(b)] The failure cases shown in Figure S5(b) are not discussed in the main text; a brief failure analysis would help readers calibrate when the method is likely to produce incorrect abnormality identification or inaccurate localization.

Circularity Check

2 steps flagged · score 6.0 of 10

Detection accuracy on AIGC Human-Aware 1K is not independently established: the paper scores its own bounding boxes by the repair quality of the same inpainting model used for detection.

  1. self definitional [Figure 3 caption; Section 5.1 'Body Part Abnormality Repair']
    "Since the location of the abnormalities is ambiguous, we do not annotate the bounding box. Instead, we evaluate the accuracy of the bounding box location by assessing the repair quality."

    The Table 1 accuracy numbers for AIGC Human-Aware 1K are based on a metric in which a detected bounding box is judged correct or incorrect by how well the HumanCalibrator's own inpainting model repairs that region. This makes the evaluation self-referential: the detector's localization is validated by the behavior of the same regeneration model that participates in the detection pipeline (especially for redundant parts, where the inpainter is the detector). A box that is easy to inpaint can score as correct even if it misses the true abnormality, and a box that correctly localizes an abnormality can score as incorrect if inpainting fails. Thus the reported 80.69%/58.57% accuracies do not demonstrate independent localization accuracy on AIGC data.

  2. other [Section 4.2, Eq. 8]
    "To determine if the original body part bg_i is indeed the redundant body br, we compare the corresponding semantics pg_i to pR. If a significant semantic difference is detected (with the assistance of G), it indicates that the body part ⟨pg_i, bg_i⟩ is redundant."

    The redundant-body detector is defined operationally as the set of grounded parts whose inpainted regeneration differs beyond threshold τ. The positive output of the detector is therefore true by stipulation relative to the inpainting model's behavior, not by independent evidence of redundancy. This is a legitimate heuristic design, but it becomes circular when the same inpainter's repair quality is then used as the ground-truth proxy for bounding-box localization on AIGC Human-Aware 1K, closing the evaluation loop.

full rationale

The central circular step is the evaluation protocol for AIGC Human-Aware 1K. The paper explicitly declines to annotate bounding boxes and instead uses repair quality — produced by the same inpainting model that is part of HumanCalibrator and that defines the redundant-part detector — to score detection localization. Consequently, Table 1's accuracy numbers measure the coupling between the detector and its own inpainting module rather than independent agreement with ground-truth locations. This is a genuine self-referential metric, not merely a weak proxy. The paper does provide some independent evidence: COCO Human-Aware Val contains synthetic boxes and is used to train/evaluate the absent detector, and qualitative case studies show plausible generalization. But COCO Human-Aware Val is generated by the same masking procedure used for AHD training data, and it contains only absent abnormalities, so it does not independently validate AIGC-domain localization for either absent or redundant cases. The score is 6 rather than 8 because the abnormality type and category labels on AIGC Human-Aware 1K are human-annotated and independent, and the repair-quality proxy still correlates with localization quality; the circularity is real but partial, confined to the localization-evaluation link in the main AIGC benchmark.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework depends on an inpainting model behaving a certain way, a threshold chosen by hand, and an evaluation protocol that uses the repair model itself to score detection. This makes the empirical claims sensitive to unstated implementation choices.

free parameters (3)
  • tau (grounding threshold in Eq. 8) = not reported
    Used to decide whether the semantic difference between the original body part and its inpainted version is large enough to mark it redundant. No ablation or sensitivity analysis is provided.
  • Bounding box expansion ratio for inpainting = not reported
    In Appendix A.2 the authors state they expand the abnormal region bounding box before inpainting to improve repair quality. The ratio is not specified and it affects the repair quality metrics used as detection proxy.
  • Background replacement method for COCO Human-Aware Val = not reported
    The automated construction replaces a body part with 'background' to simulate an absent part. The inpainting/replacement algorithm is not specified, which changes the difficulty and domain of the benchmark.
assumptions (4)
  • domain assumption Human body parts are correlated, so the presence and location of an absent part can be inferred from the remaining parts.
    Section 4.1: the AHD training strategy masks one body part and learns to predict it from the others. If body parts were independent, the training objective would have no signal.
  • domain assumption An inpainting model trained on normal human bodies regenerates a redundant body part into semantically different content, while a normal body part is regenerated similarly.
    Section 4.2 and Eq. 8: redundant detection is based on comparing original body part semantics with inpainted semantics. This assumes Stable Diffusion 2 Inpainting behaves this way for all predefined parts.
  • ad hoc to paper Repair quality is a valid proxy for detection localization accuracy on AIGC Human-Aware 1K, where bounding boxes are not annotated.
    Figure 3 caption and Section 5.1: because locations are 'ambiguous', the authors assess bounding box accuracy by whether the repair looks good. This ties the evaluation of detection to the same inpainting model used by the method.
  • domain assumption The manually filtered AIGC Human-Aware 1K labels are objective and complete.
    Appendix D: multi-round and multi-reviewer checks are claimed, but inter-annotator agreement is not reported, so label reliability is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-body." pith.science (2026). https://pith.science/paper/DO5KH55S

@misc{pith2026241114205,
  author       = {Pith},
  title        = {Pith review of: Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-body},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DO5KH55S}},
  note         = {Machine review of arXiv:2411.14205}
}
read the original abstract

Recent improvements in visual synthesis have significantly enhanced the depiction of generated human photos, which are pivotal due to their wide applicability and demand. Nonetheless, the existing text-to-image or text-to-video models often generate low-quality human photos that might differ considerably from real-world body structures, referred to as "abnormal human bodies". Such abnormalities, typically deemed unacceptable, pose considerable challenges in the detection and repair of them within human photos. These challenges require precise abnormality recognition capabilities, which entail pinpointing both the location and the abnormality type. Intuitively, Visual Language Models (VLMs) that have obtained remarkable performance on various visual tasks are quite suitable for this task. However, their performance on abnormality detection in human photos is quite poor. Hence, it is quite important to highlight this task for the research community. In this paper, we first introduce a simple yet challenging task, i.e., \textbf{F}ine-grained \textbf{H}uman-body \textbf{A}bnormality \textbf{D}etection \textbf{(FHAD)}, and construct two high-quality datasets for evaluation. Then, we propose a meticulous framework, named HumanCalibrator, which identifies and repairs abnormalities in human body structures while preserving the other content. Experiments indicate that our HumanCalibrator achieves high accuracy in abnormality detection and accomplishes an increase in visual comparisons while preserving the other visual content.

Figures

Figures reproduced from arXiv: 2411.14205 by the authors.

Figure 1
Figure 1. Fine-Grained Human-body Abnormality Detection [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Fine-Grained Human-body Abnormality Detection [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Absent Human-body Detector (AHD) training strategy. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: The illustration of our HumanCalibrator. The HumanCalibrator consists of two parts: perception and regeneration. In the [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Comparison of accuracy in COCO Human-aware Val. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Case study of the image repair quality of HumanCalibrator. In (a), the HumanCalibrator detects the absent hand of the woman and [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Case study of the video repair quality of HumanCalibrator. The first row shows frames from the original video, which contain [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Examples of HumanCalibrator applied to frames [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Detecting Human Artifacts from Text-to-Image Models

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A new dataset and detector suite localize human artifacts in text-to-image outputs and feed back into generation to reduce them.

  2. Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A new large benchmark for AI-generated human-centric video quality with pairwise preferences, plus a Mixture-of-Experts MLLM that outperforms prior methods on rating, comparison, and Q&A.

  3. Survey on AI-Generated Media Detection: From Non-MLLM to MLLM

    cs.CV 2025-02 unverdicted novelty 3.0 of 10

    A survey organizing AI-generated media detection into Non-MLLM and MLLM based methods, with task and benchmark taxonomies.

Reference graph

Works this paper leans on

67 extracted references · 17 canonical work pages · cited by 3 Pith papers

  1. [1]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. 2

    arXiv 2023

  2. [2]

    Videophy: Evaluating physical commonsense for video generation

    Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai- Wei Chang, and Aditya Grover. Videophy: Evaluating physical commonsense for video generation. arXiv preprint arXiv:2406.03520, 2024. 2, 4

    arXiv 2024

  3. [3]

    Improving image genera- tion with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, Wesam Manassra, Prafulla Dhariwal, Casey Chu, Yunxin Jiao, and Aditya Ramesh. Improving image genera- tion with better captions. OpenAI, 2023. 1

    work page 2023

  4. [4]

    Deepseek llm: Scaling open- source language models with longtermism

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. Deepseek llm: Scaling open- source language models with longtermism. arXiv preprint arXiv:2401.02954, 2024. 2

    arXiv 2024

  5. [5]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in neural in- formation processing systems, 33:1877–1901, 2020. 2

    1901

  6. [6]

    What matters in detecting ai-generated videos like sora? arXiv preprint arXiv:2406.19568, 2024

    Chirui Chang, Zhengzhe Liu, Xiaoyang Lyu, and Xiaojuan Qi. What matters in detecting ai-generated videos like sora? arXiv preprint arXiv:2406.19568, 2024. 2

    arXiv 2024

  7. [7]

    Motionllm: Understanding human behaviors from human motions and videos

    Ling-Hao Chen, Shunlin Lu, Ailing Zeng, Hao Zhang, Benyou Wang, Ruimao Zhang, and Lei Zhang. Motionllm: Understanding human behaviors from human motions and videos. arXiv preprint arXiv:2405.20340, 2024. 2

    arXiv 2024

  8. [8]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhang- wei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821, 2024. 2, 1

    arXiv 2024

Show all 67 references
  1. [9]

    Scaling instruction-finetuned language models

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. 2

    arXiv

  2. [10]

    Pa-llava: A large language-vision assistant for human pathology image under- standing

    Dawei Dai, Yuanhui Zhang, Long Xu, Qianlan Yang, Xiao- jing Shen, Shuyin Xia, and Guoyin Wang. Pa-llava: A large language-vision assistant for human pathology image under- standing. arXiv preprint arXiv:2408.09530, 2024. 1

    2024 arXiv

  3. [11]

    Instructblip: Towards general- purpose vision-language models with instruction tuning,

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general- purpose vision-language models with instruction tuning,

  4. [12]

    Cogview: Mastering text-to-image generation via transformers

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34:19822–19835, 2021. 3

    2021

  5. [13]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arxiv 2020. arXi...

    2020 arXiv

  6. [14]

    Hu- manrefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance

    Guian Fang, Wenbiao Yan, Yuanfan Guo, Jianhua Han, Zu- tao Jiang, Hang Xu, Shengcai Liao, and Xiaodan Liang. Hu- manrefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance. arXiv preprint arXiv:2407.06937, 2024. 1, 2, 4, 7, 8

    2024 arXiv

  7. [15]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 2

    2014

  8. [16]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning. Interna- tional Conference on Learning Representations, 2024. 8

    2024

  9. [17]

    Efficient multi- modal learning from data-centric perspective

    Muyang He, Yexin Liu, Boya Wu, Jianhao Yuan, Yueze Wang, Tiejun Huang, and Bo Zhao. Efficient multi- modal learning from data-centric perspective. arXiv preprint arXiv:2402.11530, 2024. 2

    2024 arXiv

  10. [18]

    Imagen video: High definition video generation with diffusion mod- els

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022. 3

    arXiv

  11. [19]

    Bliva: A simple multimodal llm for better handling of text-rich visual questions

    Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, and Zhuowen Tu. Bliva: A simple multimodal llm for better handling of text-rich visual questions. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2256– 2264, 2024. 2

    2024

  12. [20]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 6700–6709, 2019. 2

    2019

  13. [21]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lam- ple, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. 2

    arXiv

  14. [22]

    Humansd: A native skeleton-guided diffusion model for human image generation

    Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, and Qiang Xu. Humansd: A native skeleton-guided diffusion model for human image generation. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 15988–15998, 2023. 1

    2023

  15. [23]

    Reflective decoding network for image captioning

    Lei Ke, Wenjie Pei, Ruiyu Li, Xiaoyong Shen, and Yu-Wing Tai. Reflective decoding network for image captioning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8888–8897, 2019. 2

    2019

  16. [24]

    Text2video-zero: Text-to- image diffusion models are zero-shot video generators.arXiv preprint arXiv:2303.13439, 2023

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text-to- image diffusion models are zero-shot video generators.arXiv preprint arXiv:2303.13439, 2023. 8

    2023 arXiv

  17. [25]

    Open-sora-plan, 2024

    PKU-Yuan Lab and Tuzhan AI etc. Open-sora-plan, 2024. 1, 3

    2024

  18. [26]

    Pyramid- flow: High-resolution defect contrastive localization using pyramid normalizing flow

    Jiarui Lei, Xiaobo Hu, Yue Wang, and Dong Liu. Pyramid- flow: High-resolution defect contrastive localization using pyramid normalizing flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14143–14152, 2023. 8

    2023

  19. [27]

    Agiqa-3k: An open database for ai-generated image quality assessment

    Chunyi Li, Zicheng Zhang, Haoning Wu, Wei Sun, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, and Weisi Lin. Agiqa-3k: An open database for ai-generated image quality assessment. IEEE Transactions on Circuits and Sys- tems for Video Technology, 2023. 2

    2023

  20. [28]

    Llava-med: Training a large language- and-vision assistant for biomedicine in one day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language- and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36, 2024. 1

    2024

  21. [29]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023. 2

    2023 arXiv

  22. [30]

    Mini-gemini: Mining the potential of multi-modality vision language models

    Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia. Mini-gemini: Mining the potential of multi-modality vision language models. arXiv:2403.18814, 2023. 2

    2023 arXiv

  23. [31]

    K-sort arena: Efficient and reliable benchmarking for generative models via k-wise human preferences

    Zhikai Li, Xuewen Liu, Dongrong Fu, Jianquan Li, Qingyi Gu, Kurt Keutzer, and Zhen Dong. K-sort arena: Efficient and reliable benchmarking for generative models via k-wise human preferences. arXiv preprint arXiv:2408.14468, 2024. 2

    2024 arXiv

  24. [32]

    Evaluation of text-to-video gen- eration models: A dynamics perspective

    Mingxiang Liao, Hannan Lu, Xinyu Zhang, Fang Wan, Tianyu Wang, Yuzhong Zhao, Wangmeng Zuo, Qixiang Ye, and Jingdong Wang. Evaluation of text-to-video gen- eration models: A dynamics perspective. arXiv preprint arXiv:2407.01094, 2024. 2

    2024 arXiv

  25. [33]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 2, 6, 1

    2023

  26. [34]

    Global tex- ture enhancement for fake face detection in the wild

    Zhengzhe Liu, Xiaojuan Qi, and Philip HS Torr. Global tex- ture enhancement for fake face detection in the wild. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8060–8069, 2020. 2

    2020

  27. [35]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems , 35:2507–2521,

  28. [36]

    Lareˆ 2: Latent reconstruction error based method for diffusion-generated image detection

    Yunpeng Luo, Junlong Du, Ke Yan, and Shouhong Ding. Lareˆ 2: Latent reconstruction error based method for diffusion-generated image detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17006–17015, 2024. 2

    2024

  29. [37]

    Generating images from captions with attention

    Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, and Rus- lan Salakhutdinov. Generating images from captions with attention. arXiv preprint arXiv:1511.02793, 2015. 2

    2015 arXiv

  30. [38]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 2024. 1

    2024

  31. [39]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 3

    2021 arXiv

  32. [40]

    Towards uni- versal fake image detectors that generalize across genera- tive models

    Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards uni- versal fake image detectors that generalize across genera- tive models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24480– 24489, 2023. 2

    2023

  33. [41]

    Aigc image quality assessment via image-prompt correspondence

    Fei Peng, Huiyuan Fu, Anlong Ming, Chuanming Wang, Huadong Ma, Shuai He, Zifei Dou, and Shu Chen. Aigc image quality assessment via image-prompt correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , pages 6432–6...

    2024

  34. [42]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1, 3

    2023 arXiv

  35. [43]

    Evaluation and comparison of vi- sual language models for transportation engineering prob- lems

    Sanjita Prajapati, Tanu Singh, Chinmay Hegde, and Pranamesh Chakraborty. Evaluation and comparison of vi- sual language models for transportation engineering prob- lems. arXiv preprint arXiv:2409.02278, 2024. 1

    2024 arXiv

  36. [44]

    Thinking in frequency: Face forgery detection by min- ing frequency-aware clues

    Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by min- ing frequency-aware clues. In European conference on com- puter vision, pages 86–103. Springer, 2020. 2

    2020

  37. [45]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  38. [46]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 1, 3

    2021

  39. [47]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1, 3

    2022 arXiv

  40. [48]

    Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S

    Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdel- rahman Shaker, Salman Khan, Hisham Cholakkal, Rao M. Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S. Khan. Glamm: Pixel grounding large multimodal model. The IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2

    2024

  41. [49]

    Timechat: A time-sensitive multimodal large language model for long video understanding

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long video understanding. ArXiv, abs/2312.02051,

    arXiv

  42. [50]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3

    2022

  43. [51]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  44. [52]

    A-okvqa: A benchmark for visual question answering using world knowl- edge

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-okvqa: A benchmark for visual question answering using world knowl- edge. arXiv, 2022. 2

    2022

  45. [53]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  46. [54]

    Gemma: Open models based on gemini research and tech- nology

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi `ere, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and tech- nology. arXiv preprint arXiv:2403.08295, 2024. 2

    2024 arXiv

  47. [55]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms,

    Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal llms,

  48. [56]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 2

    2023 arXiv

  49. [57]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023. 2

    2023 arXiv

  50. [58]

    Vidprom: A million-scale real prompt-gallery dataset for text-to-video diffusion models

    Wenhao Wang and Yi Yang. Vidprom: A million-scale real prompt-gallery dataset for text-to-video diffusion models. arXiv preprint arXiv:2403.06098, 2024. 3

    2024 arXiv

  51. [59]

    Attngan: Fine- grained text to image generation with attentional generative adversarial networks

    Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He. Attngan: Fine- grained text to image generation with attentional generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , page...

    2018

  52. [60]

    xgen-mm (blip-3): A family of open large multimodal models

    Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yu- tong Dai, Michael S Ryoo, et al. xgen-mm (blip-3): A family of open large multimodal models. arXiv preprint arXiv:2408.08872, 2024. 2

    2024

  53. [61]

    Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024. 2

    2024 arXiv

  54. [62]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 1, 3, 8

    2024 arXiv

  55. [63]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 3

    2022 arXiv

  56. [64]

    Glm-130b: An open bilingual pre-trained model

    Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414, 2022. 2

    2022 arXiv

  57. [65]

    Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks

    Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE international conference on computer vision , p...

    2017

  58. [66]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 1

    2023

  59. [67]

    ab- sent

    Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, and Pan Zhou. Let’s think outside the box: Exploring leap-of-thought in large language models with creative humor generation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion an...

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.