Pith. sign in

REVIEW 5 major objections 7 minor 47 references

Towards Interactive Deepfake Analysis

T0 review · 5 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that deepfake analysis can be turned into an interactive dialogue: an instruction-tuned multimodal language model can detect forged faces, classify the manipulation technique, describe the artifacts, and answer follow-up…

desk verdict A useful first dataset/benchmark for interactive deepfake analysis, with believable detection/classification numbers, but the artifact-description evaluation is circular and the release is missing. read the letter →

arxiv 2501.01164 v1 pith:FOKQ4L4N submitted 2025-01-02 cs.CV

Lixiong Qin , Ning Jiang , Yang Zhang , Yuhan Qiu , Dingheng Zeng , Jiani Hu , Weihong Deng This is my paper · ORCID
classification cs.CV
keywords interactivedeepfakeanalysismulti-modallargelanguagemodelsinstructiontuninglow-rankadaptationartifactdescriptiondetectionclassificationDFA-Bench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Today's deepfake detectors mostly output a single fake/real label, which limits how much they help a human examiner. This paper argues that the next step is interactive analysis: a system that can say whether a face is forged, which technique produced it, where the artifacts are, and how to interpret them. To pursue that goal, the authors build a large instruction-following dataset, a benchmark that scores detection, classification, and artifact description, and a system called DFA-GPT obtained by instruction-tuning a multimodal language model with a low-rank adapter. Their experiments report that DFA-GPT beats vision-only detectors on detection and classification while also producing artifact descriptions those detectors cannot provide. If the approach holds, forensic work could become a conversation with the model rather than a bare verdict.

What carries the argument

The load-bearing piece is the training-data pipeline. It starts from face images spanning four forgery categories, uses an automated multimodal annotation model to write artifact descriptions for each forged image, optionally paired with its genuine original, and expands all labels into question-answer pairs, including free conversations. The system itself is a frozen vision encoder and language model connected by a trainable projector, with Low-Rank Adaptation (LoRA) modules learning low-rank residual updates so only a small parameter set is trained. DFA-Bench turns the held-out test images into standardized questions, manually rewrites artifact answers that do not match the images, and scores detection by accuracy and ACER, classification by accuracy, and descriptions by ROUGE-L.

What would settle it

Take a set of forged images whose artifacts were inserted by known pipelines, have independent human examiners mark the exact regions that differ from the genuine originals, and compare DFA-GPT's artifact descriptions against those marked regions; if the descriptions name the marked regions no better than descriptions trained on shuffled labels do, the claimed artifact-description capability is primarily imitation rather than localization.

Watch

Extended reading notes

Core claim

The paper claims that deepfake analysis can be recast as an interactive language task rather than a single verdict from a discriminative classifier. It defines four capabilities—detecting whether a face image is forged, classifying the forgery technique, describing the visible artifacts, and freely conversing about them—and claims that an instruction-tuned multimodal large language model can acquire all four at once. On its DFA-Bench test set, the resulting system DFA-GPT reaches 95.22% detection accuracy, 5.04 ACER, 92.74% classification accuracy, and a 42.54 ROUGE-L artifact-description score, outperforming vision-only baselines and general-purpose multimodal models. The authors also report that adding artifact-description supervision slightly improves both detection and classification, which they read as evidence that richer language supervision strengthens forensic robustness.

Load-bearing premise

The approach depends on the automated annotation model's artifact descriptions actually pointing at real manipulation traces in the images, because those descriptions are both the training signal and the benchmark's gold answers.

Editorial extensions

If this is right

  • A single instruction-tuned model can deliver detection, classification, artifact description, and free conversation, replacing pipelines that need separate discriminative models for the first two tasks.
  • Artifact description becomes a measurable task: DFA-Bench's standardized questions and ROUGE-L score let researchers quantify and compare a capability that previously had no evaluation.
  • Adding artifact-description and classification supervision improves detection (ACER drops from 5.72 to 4.46 in the ablation), so language-level annotations act as useful extra training signal, not just output formatting.
  • General-purpose multimodal models are far behind on deepfake-specific questions, so domain instruction tuning is currently necessary for interactive forensics.
  • Because only the projector and low-rank adapter are trained, the approach can be reproduced and updated under limited compute as new forgery methods appear.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the benchmark's gold artifact descriptions inherit whatever biases the annotation model has, so a high ROUGE-L score may partly measure stylistic imitation; an independent test with human-verified region masks would settle whether the model truly localizes artifacts.
  • Beyond the paper: the same three-step annotation recipe could be applied to video or audio forgeries, extending interactive analysis beyond single face images.
  • Beyond the paper: a straightforward stress test is to evaluate DFA-GPT on forgeries from techniques absent from the 40 used in training, since real-world deployment will meet new generators the dataset has not seen.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes an interactive deepfake analysis framework built on instruction-tuned multimodal LLMs. It introduces DFA-Instruct, a GPT-assisted dataset of 127.3K face images and 891.6K question-answer pairs covering deepfake detection (DF-D), classification (DF-C), artifact description (AD), and free conversation (FC); DFA-Bench, a benchmark with ACC/ACER for DF-D, ACC for DF-C, and ROUGE-L for AD; and DFA-GPT, a LLaVA-1.5-7B model tuned with LoRA. Experiments on DFA-Bench report 95.22% DF-D accuracy, 5.04 ACER, 92.74% DF-C accuracy, and 42.54 ROUGE-L, outperforming vision-only and general MLLM baselines. The claimed novelty is that DFA-GPT is the first interactive deepfake analysis system that can describe artifacts and engage in free conversation about forgeries.

Significance. If the DF-D and DF-C results survive a more controlled evaluation and the AD capability is independently validated, the paper would make a useful contribution by defining four capabilities for interactive deepfake analysis, constructing a large instruction dataset, and demonstrating parameter-efficient MLLM tuning for forensics. The data-construction pipeline and the LoRA design are sensible, and the comparison with general MLLMs in Table III is informative. However, the absence of independent AD ground truth, the lack of error bars or cross-dataset generalization tests, the un-evaluated FC capability, and the non-availability of code and data currently prevent verification of the core claims.

major comments (5)
  1. [III.A Step 2; III.B] The artifact-description (AD) evaluation is circular. In Section III.A Step 2, GPT-4o generates the AD annotations from the forgery images (using single-image and image-pair prompts), and Section III.B states that DFA-Bench AD gold answers are the same GPT-4o outputs after manual rewriting of descriptions that 'do not match the images.' DFA-GPT is instruction-tuned on DFA-Instruct, whose AD targets are exactly these GPT-4o texts, and then scored with ROUGE-L against the same type of GPT-4o-derived references. A high ROUGE-L (42.54) can therefore be achieved by learning GPT-4o's phrasing habits rather than by localizing real manipulation traces. The manual rewrite step is not quantified (no number of rewrites, protocol, or inter-annotator agreement), and no independent forensic ground truth (e.g., pixel-level manipulation masks or expert annotations) is provided. This makes the central 'interactive artifact description' capability unvalidated.
  2. [V.B; Table I] DF-D and DF-C results are single point estimates on one in-distribution test split and are reported without error bars or significance tests. The test set comes from the same DFA-Instruct distribution as the training set (with non-overlapping identities), and Section III.A explicitly excludes DF-40 subsets with too few samples. For a forensic application, cross-dataset generalization (e.g., to FaceForensics++ or Celeb-DF) is a load-bearing requirement; without it, the 95.22% ACC / 5.04 ACER numbers cannot support the claim of a robust interactive system.
  3. [V.B; Table I] The comparison with vision-only models in Table I is not controlled. DFA-GPT is a 7B-parameter MLLM with a LoRA adapter and MLP projector trained on 891.6K instruction-following pairs, whereas the vision baselines use frozen backbones and only a final projection layer trained on binary DF-D / multi-class DF-C labels. The statement in Section V.B that 'introducing LLM and natural language supervision enhances the robustness' is confounded by differences in architecture, parameter count, and training objective. Controlled baselines (e.g., the same vision encoder with a large-capacity head, or an ablation of the language component) are needed to support this attribution.
  4. [III.B; V.B] The free-conversation (FC) capability is presented in Section I and Fig. 1 as a fundamental capability of the interactive system, and Section V.B states that DFA-GPT 'offers the abilities of AD and FC,' but no metric or experiment for FC appears anywhere in the paper. The interactive claim is therefore not evaluated. Either FC should be removed from the claimed contributions, or an evaluation protocol (e.g., human or LLM-based response-quality scoring) should be added.
  5. [Abstract; III.A] The dataset and code are only promised ('will be made available') and no working link or release is provided, so the contributions are not reproducible. In addition, Section III.A does not specify the numerical threshold for excluding DF-40 'subsets with too few samples' or the video frame sampling interval; these details are needed to reconstruct DFA-Instruct and to interpret the benchmark.
minor comments (7)
  1. [V.B] The section header contains a typo: 'Comparsion' should be 'Comparison.'
  2. [References] Reference [16] is identical to reference [1] ('On the detection of digital face manipulation,' CVPR 2020); please deduplicate.
  3. [III.B; Table I] No metric is defined for free conversation (FC), and the tables also omit any FC column; consider adding a placeholder or explaining why FC is not quantitatively evaluated.
  4. [III.A] The prompts used to query GPT-4o and ChatGPT are only described in words; including the actual prompt templates in an appendix would substantially strengthen reproducibility.
  5. [V.D; Table III] The evaluation protocol for GPT-4V is not specified (API settings, temperature, prompt template, number of samples); this is important because GPT-4V is a closed model.
  6. [IV; Eq. (1)] The variables x and h in Eq. (1) are not defined; x should be the input activation to the linear layer (not the input image), and h the output activation.
  7. [Fig. 3] The panel labels in Fig. 3 are difficult to read (e.g., the pie-chart labels appear as 'Dec.', 'TC', 'CE' instead of the DF-D/DF-C/AD/FC abbreviations used in the text); please regenerate the figure with legible labels.

Circularity Check

1 steps flagged · score 6.0 of 10

The AD benchmark is circular: GPT-4o writes both the training targets and the gold answers, so the reported ROUGE-L may measure imitation of GPT-4o's phrasing rather than verified artifact localization; the DF-D and DF-C axes remain independently grounded.

  1. fitted input called prediction [Section III.A Step 2 (AD annotation generation); Section III.B (DFA-Bench AD metric)]
    "Step 2: Generate annotations for AD. We design two types of prompts as instructions for querying GPT-4o [42] to generate AD annotations. ... We use ACC, ERR, and ACER to assess the DF-D ability of MLLMs, ACC to evaluate their DF-C ability, and ROUGE-L to measure their AD ability. ... Notably, to ensure the validity of the evaluation, we manually review AD answers and rewrite artifact descriptions that do not match the images."

    The same GPT-4o outputs that define the AD training targets for DFA-GPT are reused as the gold references for AD evaluation in DFA-Bench. ROUGE-L therefore measures how closely the model reproduces GPT-4o's wording, not whether the descriptions localize real manipulation traces. Because DFA-GPT was instruction-tuned to imitate these exact GPT-4o-generated annotations, a high AD score is rewarded by construction. The manual rewriting step is unquantified and does not supply an independent forensic ground truth for artifact localization.

full rationale

The central circularity is confined to the artifact description (AD) axis, which is the paper's novel extended capability. GPT-4o generates the AD annotations in Section III.A Step 2; DFA-GPT is trained on those annotations via DFA-Instruct; and Section III.B scores AD by ROUGE-L against the same GPT-4o-derived answers, with only unquantified manual rewriting of mismatches. Consequently, the 42.54 ROUGE-L value in Table I is not evidence that DFA-GPT can independently identify forensic artifacts; it is evidence of stylistic and lexical convergence to the model that authored both the training targets and the evaluation references. The DF-D and DF-C labels, by contrast, come from known generation techniques in DF-40 with held-out identities, so those results have independent grounding and are not circular. There is no load-bearing self-citation or imported uniqueness theorem; the Faceptor citation is limited to face alignment. The overall score reflects partial circularity: the detection and classification claims are independently supported, but the benchmark's central novel capability, artifact description, reduces by construction to reproduction of GPT-4o text.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The detection and classification results rest on DF-40's technique labels and an identity-disjoint split, which are externally grounded. The artifact-description results rest on GPT-4o-written annotations, the least externally grounded component. Training hyperparameters are hand-chosen and not swept. No new physical entities are introduced; the invented component is the AD evaluation protocol.

free parameters (5)
  • LoRA rank = 128
    Rank of A and B in Section IV; chosen without an ablation study, it controls how much new deepfake knowledge the model can store.
  • Initial learning rate = 2e-4
    Set in Section V.A; no sweep or sensitivity analysis is reported.
  • Training epochs = 1
    Section V.A trains for one epoch; no convergence or overfitting analysis is provided.
  • DF-40 subset exclusion threshold = not reported
    Section III.A Step 1 excludes subsets with too few samples without stating the cutoff, which changes the class balance of DFA-Instruct.
  • Video frame sampling interval = not reported
    Section III.A Step 1 retains frames at specific intervals for video-based FS and FR subsets; the interval is not specified, affecting temporal redundancy and dataset size.
assumptions (4)
  • domain assumption DF-40 technique labels are correct ground truth for DF-D and DF-C.
    Section III.A Step 1 builds annotations directly from DF-40's 40 forgery subsets, and DFA-Bench inherits these labels as gold.
  • ad hoc to paper GPT-4o artifact descriptions are faithful enough to serve as training and evaluation targets after manual rewriting.
    Section III.A Step 2 uses GPT-4o outputs as AD annotations, and Section III.B rewrites only mismatched descriptions; no independent forensic verification exists.
  • domain assumption Face alignment and CLIP-L/14 feature extraction preserve manipulation artifacts.
    Section IV feeds aligned face images through frozen CLIP-L/14; if alignment or preprocessing removes small artifacts, detection performance would drop.
  • domain assumption LLaVA-1.5-7B retains sufficient visual instruction-following ability when only the projector and LoRA are tuned.
    Section V.A initializes from LLaVA-1.5-7B and tunes only the projector and LoRA, assuming the frozen backbone keeps enough competence.
invented entities (1)
  • Artifact description (AD) capability and its ROUGE-L evaluation protocol on DFA-Bench
    purpose: Defines a new open-ended generation task for MLLMs to describe where and how a face image appears manipulated.
    The AD gold answers come from GPT-4o with selective manual rewriting (Section III.A Step 2, Section III.B). No external dataset of verified artifact locations exists, so the protocol cannot independently confirm that higher ROUGE-L means more accurate forensics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Interactive Deepfake Analysis." pith.science (2026). https://pith.science/paper/FOKQ4L4N

@misc{pith2026250101164,
  author       = {Pith},
  title        = {Pith review of: Towards Interactive Deepfake Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FOKQ4L4N}},
  note         = {Machine review of arXiv:2501.01164}
}
read the original abstract

Existing deepfake analysis methods are primarily based on discriminative models, which significantly limit their application scenarios. This paper aims to explore interactive deepfake analysis by performing instruction tuning on multi-modal large language models (MLLMs). This will face challenges such as the lack of datasets and benchmarks, and low training efficiency. To address these issues, we introduce (1) a GPT-assisted data construction process resulting in an instruction-following dataset called DFA-Instruct, (2) a benchmark named DFA-Bench, designed to comprehensively evaluate the capabilities of MLLMs in deepfake detection, deepfake classification, and artifact description, and (3) construct an interactive deepfake analysis system called DFA-GPT, as a strong baseline for the community, with the Low-Rank Adaptation (LoRA) module. The dataset and code will be made available at https://github.com/lxq1000/DFA-Instruct to facilitate further research.

Figures

Figures reproduced from arXiv: 2501.01164 by the authors.

Figure 1
Figure 1. An interactive deepfake analysis system. Questions representing the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Step 1: Acquire bona fide and corresponding forgery face images, and add annotations for DF-D and DF￾C. Bona fide images can be sourced from the internet or existing datasets, while corresponding forgery images are generated from bona fide images using various deepfake FS FR FE EFS Bona Fide Image Step 1 Forgery Images Internet Other Datasets DF-C Annotation DF-D Annotation Collect Add Add Add Add Instruction: Compa… view at source ↗
Figure 4
Figure 4. The overall architecture of our DFA-GPT. [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Statistics of our proposed DFA-Instruct. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 37 canonical work pages

  1. [2]

    Multi- attentional deepfake detection,

    H. Zhao, W. Zhou, D. Chen, T. Wei, W. Zhang, and N. Yu, “Multi- attentional deepfake detection,” in CVPR, pp. 2185–2194, Computer Vision Foundation / IEEE, 2021

    work page 2021

  2. [3]

    Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection,

    L. Chen, Y . Zhang, Y . Song, L. Liu, and J. Wang, “Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection,” in CVPR, pp. 18689–18698, IEEE, 2022

    work page 2022

  3. [4]

    Implicit identity leakage: The stumbling block to improving deepfake detection generalization,

    S. Dong, J. Wang, R. Ji, J. Liang, H. Fan, and Z. Ge, “Implicit identity leakage: The stumbling block to improving deepfake detection generalization,” in CVPR, pp. 3994–4004, IEEE, 2023

    work page 2023

  4. [5]

    UCF: uncovering common features for generalizable deepfake detection,

    Z. Yan, Y . Zhang, Y . Fan, and B. Wu, “UCF: uncovering common features for generalizable deepfake detection,” in ICCV, pp. 22355– 22366, IEEE, 2023

    work page 2023

  5. [6]

    Gpt-4v (ision) system card

    OpenAI, “Gpt-4v (ision) system card.” https://cdn.openai.com/papers/ GPTV System Card.pdf, 2023

    work page 2023

  6. [7]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” in NeurIPS, 2023

    2023

  7. [8]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” in ICLR, OpenReview.net, 2024

    work page 2024

  8. [9]

    Pandagpt: One model to instruction-follow them all,

    Y . Su, T. Lan, H. Li, J. Xu, Y . Wang, and D. Cai, “Pandagpt: One model to instruction-follow them all,” CoRR, vol. abs/2305.16355, 2023

    arXiv 2023

Show all 47 references
  1. [10]

    Qwen-vl: A frontier large vision-language model with versatile abilities,

    J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou, “Qwen-vl: A frontier large vision-language model with versatile abilities,” CoRR, vol. abs/2308.12966, 2023

    2023 arXiv

  2. [11]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, B. Li, P. Luo, T. Lu, Y . Qiao, and J. Dai, “Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,” CoRR, vol. abs/2312.14238, 2023

    2023 arXiv

  3. [12]

    Deepfakes: a new threat to face recogni- tion? assessment and detection,

    P. Korshunov and S. Marcel, “Deepfakes: a new threat to face recogni- tion? assessment and detection,” CoRR, vol. abs/1812.08685, 2018

    2018 arXiv

  4. [13]

    Exposing deep fakes using inconsistent head poses,

    X. Yang, Y . Li, and S. Lyu, “Exposing deep fakes using inconsistent head poses,” in ICASSP, pp. 8261–8265, IEEE, 2019

    2019

  5. [14]

    Faceforensics++: Learning to detect manipulated facial images,

    A. R ¨ossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “Faceforensics++: Learning to detect manipulated facial images,” in ICCV, pp. 1–11, IEEE, 2019

    2019

  6. [15]

    Celeb-df: A new dataset for deepfake forensics,

    Y . Li, X. Yang, P. Sun, H. Qi, and S. Lyu, “Celeb-df: A new dataset for deepfake forensics,” CoRR, vol. abs/1909.12962, 2019

    1909 arXiv

  7. [16]

    On the detection of digital face manipulation,

    H. Dang, F. Liu, J. Stehouwer, X. Liu, and A. K. Jain, “On the detection of digital face manipulation,” in CVPR, pp. 5780–5789, Computer Vision Foundation / IEEE, 2020

    2020

  8. [17]

    DF40: toward next-generation deepfake detection,

    Z. Yan, T. Yao, S. Chen, Y . Zhao, X. Fu, J. Zhu, D. Luo, L. Yuan, C. Wang, S. Ding, and Y . Wu, “DF40: toward next-generation deepfake detection,” CoRR, vol. abs/2406.13495, 2024

    2024 arXiv

  9. [18]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in ICLR, OpenReview.net, 2022

    2022

  10. [19]

    Simswap: An efficient framework for high fidelity face swapping,

    R. Chen, X. Chen, B. Ni, and Y . Ge, “Simswap: An efficient framework for high fidelity face swapping,” CoRR, vol. abs/2106.06340, 2021

    2021 arXiv

  11. [20]

    Blendface: Re-designing identity encoders for face-swapping,

    K. Shiohara, X. Yang, and T. Taketomi, “Blendface: Re-designing identity encoders for face-swapping,” in ICCV, pp. 7600–7610, IEEE, 2023

    2023

  12. [21]

    Designing one unified framework for high-fidelity face reenactment and swapping,

    C. Xu, J. Zhang, Y . Han, G. Tian, X. Zeng, Y . Tai, Y . Wang, C. Wang, and Y . Liu, “Designing one unified framework for high-fidelity face reenactment and swapping,” in ECCV (15), vol. 13675 of Lecture Notes in Computer Science , pp. 54–71, Springer, 2022

    2022

  13. [22]

    Mobilefaceswap: A lightweight framework for video face swapping,

    Z. Xu, Z. Hong, C. Ding, Z. Zhu, J. Han, J. Liu, and E. Ding, “Mobilefaceswap: A lightweight framework for video face swapping,” in AAAI, pp. 2973–2981, AAAI Press, 2022

    2022

  14. [23]

    Fine-grained face swapping via regional GAN inversion,

    Z. Liu, M. Li, Y . Zhang, C. Wang, Q. Zhang, J. Wang, and Y . Nie, “Fine-grained face swapping via regional GAN inversion,” in CVPR, pp. 8578–8587, IEEE, 2023

    2023

  15. [24]

    Latent image animator: Learning to animate images via latent space navigation,

    Y . Wang, D. Yang, F. Br ´emond, and A. Dantcheva, “Latent image animator: Learning to animate images via latent space navigation,” in ICLR, OpenReview.net, 2022

    2022

  16. [25]

    Depth-aware generative adver- sarial network for talking head video generation,

    F. Hong, L. Zhang, L. Shen, and D. Xu, “Depth-aware generative adver- sarial network for talking head video generation,” in CVPR, pp. 3387– 3396, IEEE, 2022

    2022

  17. [26]

    Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,

    W. Zhang, X. Cun, X. Wang, Y . Zhang, X. Shen, Y . Guo, Y . Shan, and F. Wang, “Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,” in CVPR, pp. 8652–8661, IEEE, 2023

    2023

  18. [27]

    Implicit identity representation conditioned mem- ory compensation network for talking head video generation,

    F. Hong and D. Xu, “Implicit identity representation conditioned mem- ory compensation network for talking head video generation,” in ICCV, pp. 23005–23015, IEEE, 2023

    2023

  19. [28]

    Hyperreenact: One-shot reenactment via jointly learning to refine and retarget faces,

    S. Bounareli, C. Tzelepis, V . Argyriou, I. Patras, and G. Tzimiropoulos, “Hyperreenact: One-shot reenactment via jointly learning to refine and retarget faces,” in ICCV, pp. 7115–7125, IEEE, 2023

    2023

  20. [29]

    Designing an encoder for stylegan image manipulation,

    O. Tov, Y . Alaluf, Y . Nitzan, O. Patashnik, and D. Cohen-Or, “Designing an encoder for stylegan image manipulation,” ACM Trans. Graph. , vol. 40, no. 4, pp. 133:1–133:14, 2021

    2021

  21. [30]

    Stargan: Uni- fied generative adversarial networks for multi-domain image-to-image translation,

    Y . Choi, M. Choi, M. Kim, J. Ha, S. Kim, and J. Choo, “Stargan: Uni- fied generative adversarial networks for multi-domain image-to-image translation,” in CVPR, pp. 8789–8797, Computer Vision Foundation / IEEE Computer Society, 2018

    2018

  22. [31]

    Stargan v2: Diverse image synthesis for multiple domains,

    Y . Choi, Y . Uh, J. Yoo, and J. Ha, “Stargan v2: Diverse image synthesis for multiple domains,” in CVPR, pp. 8185–8194, Computer Vision Foundation / IEEE, 2020

    2020

  23. [32]

    Styleclip: Text-driven manipulation of stylegan imagery,

    O. Patashnik, Z. Wu, E. Shechtman, D. Cohen-Or, and D. Lischinski, “Styleclip: Text-driven manipulation of stylegan imagery,” in ICCV, pp. 2065–2074, IEEE, 2021

    2021

  24. [33]

    Taming transformers for high- resolution image synthesis,

    P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high- resolution image synthesis,” in CVPR, pp. 12873–12883, Computer Vision Foundation / IEEE, 2021

    2021

  25. [34]

    Alias-free generative adversarial networks,

    T. Karras, M. Aittala, S. Laine, E. H ¨ark¨onen, J. Hellsten, J. Lehtinen, and T. Aila, “Alias-free generative adversarial networks,” in NeurIPS, pp. 852–863, 2021

    2021

  26. [35]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in CVPR, pp. 10674–10685, IEEE, 2022

    2022

  27. [36]

    Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis,

    J. Chen, J. Yu, C. Ge, L. Yao, E. Xie, Z. Wang, J. T. Kwok, P. Luo, H. Lu, and Z. Li, “Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis,” in ICLR, OpenReview.net, 2024

    2024

  28. [37]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” in ICCV, pp. 4172–4182, IEEE, 2023

    2023

  29. [38]

    Language models are few-shot learners,

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B...

    2020

  30. [39]

    GPT understands, too,

    X. Liu, Y . Zheng, Z. Du, M. Ding, Y . Qian, Z. Yang, and J. Tang, “GPT understands, too,” CoRR, vol. abs/2103.10385, 2021

    2021 arXiv

  31. [40]

    Parameter-efficient transfer learning for NLP,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for NLP,” in ICML, vol. 97 of Proceedings of Machine Learning Research, pp. 2790–2799, PMLR, 2019

    2019

  32. [41]

    Faceptor: A generalist model for face perception,

    L. Qin, M. Wang, X. Liu, Y . Zhang, W. Deng, X. Song, W. Xu, and W. Deng, “Faceptor: A generalist model for face perception,” CoRR, vol. abs/2403.09500, 2024

    2024 arXiv

  33. [42]

    Hello gpt-4o

    OpenAI, “Hello gpt-4o.” https://openai.com/index/hello-gpt-4o/, 2024

    2024

  34. [43]

    Chatgpt

    OpenAI, “Chatgpt.” https://openai.com/blog/chatgpt/, 2023

    2023

  35. [44]

    Learning transferable visual models from natural language supervi- sion,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervi- sion,” in ICML, vol. 139 of Proceedings of Machine Learning Research ...

    2021

  36. [45]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

    W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalez, et al. , “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” See https://vicuna. lmsys. org (accessed 14 April 2023) , vol. 2, no. 3, p. 6, 2023

    2023

  37. [46]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines with visual instruction tuning,” CoRR, vol. abs/2310.03744, 2023

    2023 arXiv

  38. [47]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR, pp. 770–778, IEEE Computer Society, 2016

    2016

  39. [48]

    Training data-efficient image transformers & distillation through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J ´egou, “Training data-efficient image transformers & distillation through attention,” in ICML, vol. 139 of Proceedings of Machine Learning Research, pp. 10347–10357, PMLR, 2021

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.