pith. sign in

arxiv: 2604.23247 · v1 · submitted 2026-04-25 · 💻 cs.CV

Micro-Expression-Aware Avatar Fingerprinting via Inter-Frame Feature Differencing

Masoumeh Chapariniya , Jean-Marc Odobez , Volker Dellwo , Teodora Vukovi\'c This is my paper

Pith reviewed 2026-05-08 08:29 UTC · model grok-4.3

classification 💻 cs.CV
keywords avatar fingerprintingmicro-expressionfeature differencingsynthetic video verificationmotion dynamicsend-to-end learningtalking-head videoidentity separation
0
0 comments X

The pith

Inter-frame feature differencing on a micro-expression-aware backbone enables end-to-end avatar fingerprinting directly from raw video pixels.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that avatar fingerprinting, which verifies the human driver behind a synthetic talking-head video, can be achieved without any external preprocessing or fixed landmark extraction stages. It accomplishes this by subtracting deep feature maps from consecutive frames inside a backbone tuned to micro-expressions, so that stable appearance cancels out while driver-specific motion remains. A sympathetic reader would care because current methods are limited by non-differentiable landmark steps that block joint optimization from pixels. Ablations demonstrate that temporal motion supplies the large majority of the discriminative signal and that raw appearance features actively impair identity separation. The resulting model reaches an overall AUC of 0.877 on NVFAIR and matches or exceeds the landmark baseline on most cross-generator pairs.

Core claim

The central claim is that a preprocessing-free avatar fingerprinting system can be built by running raw video frames through a micro-expression-aware backbone and subtracting consecutive feature maps. This operation zeros out temporally stable appearance dimensions and retains driver-specific motion dynamics in the learned feature space. Both the backbone choice and the differencing step are required: generic encoders produce near-identical representations across frames that collapse under subtraction, whereas the chosen backbone preserves measurable motion variation that differencing can exploit. A controlled ablation on NVFAIR confirms that temporal motion accounts for most performance and

What carries the argument

Inter-frame feature differencing on a micro-expression-aware backbone, which subtracts learned feature maps from adjacent frames to isolate motion-based identity signals while nulling stable appearance.

If this is right

  • Temporal motion between frames supplies the dominant signal for distinguishing which human drives an avatar video.
  • Raw appearance features degrade identity separation when used without differencing.
  • End-to-end training from raw pixels becomes possible and competitive with methods that depend on external landmark detectors.
  • The approach generalizes across the majority of tested synthetic video generators without retraining.
  • Both the micro-expression-aware backbone and the differencing operation are jointly necessary for the observed performance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same subtraction principle could be tested on full-body or multi-person synthetic videos to see whether motion patterns remain identity-specific.
  • Generators might be hardened against this style of fingerprinting by deliberately reducing micro-expression variation in the output.
  • The method could be inserted into live video pipelines for on-the-fly driver verification without landmark computation delays.
  • Extending the temporal window beyond adjacent frames might capture longer motion signatures for finer identity discrimination.

Load-bearing premise

The micro-expression-aware backbone must retain enough distinct motion variation between adjacent frames for subtraction to isolate driver-specific signals rather than producing near-zero or noisy differences.

What would settle it

A direct test would be to measure feature-map differences on same-driver videos: if the backbone outputs near-identical maps across consecutive frames, or if performance falls to chance levels on held-out generator pairs, the claim that motion is preserved and useful would be falsified.

Figures

Figures reproduced from arXiv: 2604.23247 by Jean-Marc Odobez, Masoumeh Chapariniya, Teodora Vukovi\'c, Volker Dellwo.

Figure 1
Figure 1. Figure 1: End-to-end pipeline. Feature extraction: A video clip V composed of T grayscale frames from a generated video pass independently through the F5C backbone (ConvStack → FCC → CCC) to produce per-frame feature maps ft ∈ R128×16×16 . FCC provides global row/column receptive fields, allowing each spatial location to attend across the full face; CCC models inter-regional correlations via a k-NN graph (k=4), capt… view at source ↗
Figure 2
Figure 2. Figure 2: Cross-generator generalization for FEAT_DIFF (solid) versus the NVFAIR baseline (dashed). Each panel corresponds to a model trained on a single generator and evaluated on all three generators. TABLE VII CLIP-LENGTH ABLATION (F5C + FEAT_DIFF, SUPCON, 150 EPOCHS, ALL GENERATORS JOINTLY). T =64 IS THE DEFAULT CONFIGURATION. T Overall FV2V TPS LIA 16 0.813 0.808 0.817 0.816 32 0.862 0.853 0.868 0.866 64 0.877 … view at source ↗
read the original abstract

Avatar fingerprinting, i.e., verifying who drives a synthetic talking-head video rather than whether it is real, is a critical safeguard for authorized use of face-reenactment technology. Existing methods rely on a fixed, non-differentiable landmark extraction stage that prevents the fingerprinting model from being optimized end-to-end from raw pixels. We propose a preprocessing-free system built on a micro-expression-aware backbone operating on raw video frames, with inter-frame feature differencing as the core design principle: consecutive feature maps are subtracted in the learned deep feature space, so that temporally stable appearance dimensions contribute zero to the output while driver-specific motion dynamics are preserved. A controlled ablation on NVFAIR confirms that temporal motion accounts for the large majority of discriminative performance, and that raw appearance features actively degrade identity separation. Both the choice of backbone and the differencing principle are essential: differencing alone is insufficient when applied to a generic encoder, as appearance-dominated features collapse to near-identical representations across adjacent frames, while the micro-expression-aware F5C backbone retains measurable motion variation that the differencing operation can exploit. Without any external preprocessing, our model achieves an overall AUC of 0.877 on NVFAIR and matches or exceeds the landmark-based baseline on the majority of cross-generator pairs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes a preprocessing-free avatar fingerprinting system that employs a micro-expression-aware F5C backbone on raw video frames combined with inter-frame feature differencing to preserve driver-specific motion while nullifying stable appearance features. An ablation study on the NVFAIR dataset shows that temporal motion drives most of the discriminative power, with raw appearance degrading performance. The model achieves an AUC of 0.877 and matches or exceeds a landmark-based baseline on the majority of cross-generator pairs.

Significance. Should the empirical results and the attribution to the micro-expression-aware backbone hold upon closer inspection, this approach could represent a meaningful step toward fully differentiable, preprocessing-free methods for verifying identities in synthetic talking-head videos. It highlights the potential of exploiting subtle temporal dynamics in deep feature space rather than relying on explicit landmarks, which may have implications for robustness in avatar authentication tasks.

major comments (3)
  1. [Ablation study] The claim that only the micro-expression-aware F5C backbone retains usable motion under differencing, while generic encoders collapse, is central to attributing the performance to the proposed design. However, the ablation lacks sufficient characterization of the generic encoder (e.g., its architecture and training regime) and does not include controls to isolate whether the retained variation is due to micro-expression awareness specifically or incidental properties of the F5C training.
  2. [Methods] The F5C backbone is referred to as 'micro-expression-aware' but the manuscript provides no details on its architecture, pre-training objective, dataset used for fine-tuning, or how micro-expression awareness is achieved. This makes it impossible to verify the mechanism or reproduce the results independently.
  3. [Experiments] The NVFAIR dataset is used for evaluation, but details such as the number of videos, identities, generators involved, and the exact definition of cross-generator pairs are insufficient to assess the strength of the AUC 0.877 claim and the comparison to the landmark baseline.
minor comments (2)
  1. [Abstract] The acronym 'F5C' is used without expansion or prior definition.
  2. [Abstract] The phrase 'without any external preprocessing' may be misleading if the F5C backbone involves pre-training on external data.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their thorough review and constructive suggestions. We address each major comment below, indicating where revisions will be made to improve the manuscript's clarity, reproducibility, and rigor.

read point-by-point responses

  1. Referee: [Ablation study] The claim that only the micro-expression-aware F5C backbone retains usable motion under differencing, while generic encoders collapse, is central to attributing the performance to the proposed design. However, the ablation lacks sufficient characterization of the generic encoder (e.g., its architecture and training regime) and does not include controls to isolate whether the retained variation is due to micro-expression awareness specifically or incidental properties of the F5C training.

    Authors: We agree that the ablation would be strengthened by additional characterization of the generic encoder and further controls. In the revised manuscript, we will fully specify the architecture and training regime of the generic encoder employed in the ablation study. We will also incorporate additional control experiments to better isolate the contribution of micro-expression awareness from other properties of the F5C training. This addresses the concern about attributing the retained motion variation specifically to the proposed design. revision: yes

  2. Referee: [Methods] The F5C backbone is referred to as 'micro-expression-aware' but the manuscript provides no details on its architecture, pre-training objective, dataset used for fine-tuning, or how micro-expression awareness is achieved. This makes it impossible to verify the mechanism or reproduce the results independently.

    Authors: We acknowledge the lack of details on the F5C backbone in the current manuscript. The revised version will expand the Methods section with a full description of the F5C architecture, its pre-training objective for micro-expression recognition, the specific dataset used for fine-tuning, and the mechanisms through which micro-expression awareness is instilled. This will facilitate independent verification and reproduction of our results. revision: yes

  3. Referee: [Experiments] The NVFAIR dataset is used for evaluation, but details such as the number of videos, identities, generators involved, and the exact definition of cross-generator pairs are insufficient to assess the strength of the AUC 0.877 claim and the comparison to the landmark baseline.

    Authors: We agree that insufficient details on the NVFAIR dataset hinder assessment of the results. In the revision, we will provide complete specifications including the total number of videos and identities, the specific generators involved, and the precise definition of cross-generator pairs. We will also detail the evaluation protocol to better contextualize the overall AUC of 0.877 and the comparisons to the landmark-based baseline. revision: yes

Circularity Check

0 steps flagged

No circularity in empirical method and ablation results

full rationale

The paper advances an empirical pipeline for avatar fingerprinting that combines a micro-expression-aware backbone with inter-frame feature differencing, then validates performance via controlled ablations and AUC measurements on the NVFAIR dataset. No equations, parameter-fitting steps, or self-referential definitions appear in the derivation; the claim that temporal motion dominates and that generic encoders collapse under subtraction is presented as an experimental observation rather than a quantity forced by construction. The central result (AUC 0.877 without preprocessing) is therefore an independent measurement against external data and baselines, not a renaming or tautological restatement of the inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The approach relies on the domain assumption that the F5C backbone preserves usable motion information and that the NVFAIR dataset constitutes a valid benchmark; no free parameters or new invented entities are introduced in the abstract.

axioms (1)
  • domain assumption The micro-expression-aware F5C backbone retains measurable motion variation across frames while generic encoders do not
    Invoked to explain why differencing succeeds only with the chosen backbone.

pith-pipeline@v0.9.0 · 5545 in / 1299 out tokens · 30840 ms · 2026-05-08T08:29:16.355829+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages

  1. [1]

    One-shot free-view neural talking-head synthesis for video conferencing,

    T.-C. Wang, A. Mallya, and M.-Y . Liu, “One-shot free-view neural talking-head synthesis for video conferencing,” inIEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 10 034–10 044

    work page 2021

  2. [2]

    Thin-plate spline motion model for image animation,

    J. Zhao and H. Zhang, “Thin-plate spline motion model for image animation,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 3657–3666

    work page 2022

  3. [3]

    Latent image animator: Learning to animate images via latent space navigation,

    Y . Wang, D. Yang, F. Brémond, and A. Dantcheva, “Latent image animator: Learning to animate images via latent space navigation,” in International Conference on Learning Representations (ICLR), 2023

    work page 2023

  4. [4]

    LivePortrait: Efficient portrait animation with stitching and retargeting control,

    J. Guo, D. Zhang, X. Liu, Z. Zhong, Y . Zhang, P. Wan, and D. Zhang, “LivePortrait: Efficient portrait animation with stitching and retargeting control,” 2024

    work page 2024

  5. [5]

    EMO: Emote portrait alive – generating expressive portrait videos with audio2video diffusion model under weak conditions,

    L. Tian, Q. Wang, B. Zhang, and L. Bo, “EMO: Emote portrait alive – generating expressive portrait videos with audio2video diffusion model under weak conditions,” inEuropean Conference on Computer Vision (ECCV), 2024

    work page 2024

  6. [6]

    Identity deepfake threats to biometric authentication systems: Public and expert perspectives,

    L. Goncharov, I. Petrov, D. Malakhov, and V . Fedotov, “Identity deepfake threats to biometric authentication systems: Public and expert perspectives,” 2025

    work page 2025

  7. [7]

    Lips don’t lie: A generalisable and robust approach to face forgery detection,

    A. Haliassos, K. V ougioukas, S. Petridis, and M. Pantic, “Lips don’t lie: A generalisable and robust approach to face forgery detection,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 5039–5049

    work page 2021

  8. [8]

    Avatar fingerprinting for authorized use of synthetic talking-head videos,

    E. Prashnani, K. Nagano, S. De Mello, D. Luebke, and O. Gallo, “Avatar fingerprinting for authorized use of synthetic talking-head videos,” in European Conference on Computer Vision (ECCV), 2024, pp. 209–228

    work page 2024

  9. [9]

    Categorizing sex and identity from the biological motion of faces,

    H. Hill and A. Johnston, “Categorizing sex and identity from the biological motion of faces,”Current Biology, vol. 11, no. 11, pp. 880–885, 2001

    work page 2001

  10. [10]

    Facial motion can determine facial identity,

    B. Knappmeyer, I. Thornton, and H. Bülthoff, “Facial motion can determine facial identity,”Journal of Vision, vol. 1, no. 3, pp. 293–293, 2001

    work page 2001

  11. [11]

    Recognizing moving faces: A psychological and neural synthesis,

    A. J. O’Toole, D. A. Roark, and H. Abdi, “Recognizing moving faces: A psychological and neural synthesis,”Trends in Cognitive Sciences, vol. 6, no. 6, pp. 261–266, 2002

    work page 2002

  12. [12]

    Leveraging avatar fingerprinting: A multi-generator photorealistic talking-head public database and benchmark,

    L. Pedrouzo-Rodriguez, L. F. Gomez, R. Tolosana, R. Vera-Rodriguez, R. Daza, A. Morales, and J. Fierrez, “Leveraging avatar fingerprinting: A multi-generator photorealistic talking-head public database and benchmark,” 2026

    work page 2026

  13. [13]

    Bias-averse learning for mitigating source dataset bias in avatar fingerprinting,

    M. Sahadewa, M. Marchellus, and I. K. Park, “Bias-averse learning for mitigating source dataset bias in avatar fingerprinting,”IEEE Access, vol. 14, pp. 34 790–34 801, 2026

    work page 2026

  14. [14]

    Is it really you? exploring biometric verification scenarios in photorealistic talking-head avatar videos,

    L. Pedrouzo-Rodriguez, P. Delgado-DeRobles, L. F. Gomez, R. Tolosana, R. Vera-Rodriguez, A. Morales, and J. Fierrez, “Is it really you? exploring biometric verification scenarios in photorealistic talking-head avatar videos,” 2025

    work page 2025

  15. [15]

    Unmasking puppeteers: Leveraging bio- metric leakage to disarm impersonation in AI-based videoconferencing,

    D. S. Vahdati, T. D. Nguyen, E. Prashnani, K. Nagano, D. Luebke, O. Gallo, and M. C. Stamm, “Unmasking puppeteers: Leveraging bio- metric leakage to disarm impersonation in AI-based videoconferencing,” 2025

    work page 2025

  16. [16]

    ID-Reveal: Identity-aware deepfake video detection,

    D. Cozzolino, A. Rössler, J. Thies, M. Nießner, and L. Verdoliva, “ID-Reveal: Identity-aware deepfake video detection,” inIEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 15 108– 15 117

    work page 2021

  17. [17]

    Protecting world leaders against deep fakes,

    S. Agarwal, H. Farid, Y . Gu, M. He, K. Nagano, and H. Li, “Protecting world leaders against deep fakes,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019, pp. 38–45

    work page 2019

  18. [18]

    Audio-visual person-of- interest deepfake detection,

    D. Cozzolino, M. Nießner, and L. Verdoliva, “Audio-visual person-of- interest deepfake detection,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024

    work page 2024

  19. [19]

    Ti2Net: Temporal identity inconsistency network for deepfake detection,

    B. Liu, B. Liu, M. Ding, T. Zhu, and X. Yu, “Ti2Net: Temporal identity inconsistency network for deepfake detection,”IEEE Transactions on Information Forensics and Security, vol. 19, pp. 1–14, 2024

    work page 2024

  20. [20]

    Generalizable deepfake detection with phase-based motion analysis,

    Y . Sun, H. H. Nguyen, J. Yamagishi, and I. Echizen, “Generalizable deepfake detection with phase-based motion analysis,”IEEE Transac- tions on Image Processing, vol. 34, pp. 100–112, 2025

    work page 2025

  21. [21]

    Face forgery video detection via temporal forgery cue unraveling,

    L. Guo, F. Guo, T. Lv, J. Zhang, R. Feng, and C. Lu, “Face forgery video detection via temporal forgery cue unraveling,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    work page 2025

  22. [22]

    To- wards a universal synthetic video detector: From face or background manipulations to fully AI-generated content,

    R. Kundu, A. Saha, A. Bhattacharya, and A. Roy-Chowdhury, “To- wards a universal synthetic video detector: From face or background manipulations to fully AI-generated content,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    work page 2025

  23. [23]

    ExpressionAuth: Utilizing avatar expression blendshapes for behavioral biometrics in VR,

    T. Jitpanyoyos, Y . Sato, S. Maeda, M. Nishigaki, and T. Ohki, “ExpressionAuth: Utilizing avatar expression blendshapes for behavioral biometrics in VR,” in2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2024, pp. 679–680

    work page 2024

  24. [24]

    FacialMotionID: Identifying users of mixed reality headsets using abstract facial motion representations,

    A. Castro, S. Hanisch, M. Fallahi, and T. Strufe, “FacialMotionID: Identifying users of mixed reality headsets using abstract facial motion representations,” 2025

    work page 2025

  25. [25]

    Investigating identity signals in conversational facial dynamics via disentangled expression features,

    M. Chapariniya, P. Vuillecard, J.-M. Odobez, V . Dellwo, and T. Vukovi´c, “Investigating identity signals in conversational facial dynamics via disentangled expression features,” 2025

    work page 2025

  26. [26]

    Beyond ap- pearance: Transformer-based person identification from conversational dynamics,

    M. Chapariniya, T. Vukovi ´c, S. Ebling, and V . Dellwo, “Beyond ap- pearance: Transformer-based person identification from conversational dynamics,” in2025 15th International Conference on Computer and Knowledge Engineering (ICCKE). IEEE, 2025, pp. 1–6

    work page 2025

  27. [27]

    MOL: Joint estimation of micro-expression, optical flow, and landmark via transformer-graph-style convolution,

    Z. Shao, Y . Cheng, F. Li, Y . Zhou, X. Lu, Y . Xie, and L. Ma, “MOL: Joint estimation of micro-expression, optical flow, and landmark via transformer-graph-style convolution,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 10, pp. 8756–8768, 2025

    work page 2025

  28. [28]

    Supervised contrastive learning,

    P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y . Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 18 661–18 673

    work page 2020