Pith. sign in

REVIEW 3 major objections 7 minor 56 references

Geometry-Aware Video Inpainting for Joint Headset Occlusion Removal and Face Reconstruction in Social XR

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a monocular RGB video of a headset-wearing user, guided by facial landmarks and a single clean reference frame, can be inpainted to remove the headset and regressed into a complete 3D face model, preserving identity…

desk verdict Solid RGB inpainting benchmark, but the 3D geometry evaluation is circular—GT meshes come from the same SynergyNet backbone used for prediction. read the letter →

arxiv 2508.12336 v1 pith:324VNO2Y submitted 2025-08-17 cs.CV

Fatemeh Ghorbani Lohesara , Karen Eguiazarian , Sebastian Knorr This is my paper · ORCID
classification cs.CV
keywords extendedrealityhead-mounteddisplayremovalvideoinpainting3DfacereconstructionMorphableModelfaciallandmarksgenerativeadversarialnetworkssocialXR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Head-mounted displays hide the eyes and eyebrows, breaking the eye contact and expression cues that make video teleconferencing feel social. This paper claims that the occlusion can be erased from ordinary RGB video and the user's full 3D face geometry recovered at the same time, using only one front-facing camera, a single reference frame taken without the headset, and facial landmarks as geometric guidance. The mechanism is a loop: landmarks steer a GAN-based video inpainter, the inpainted frames are fed to a 3D Morphable Model regressor, and refined landmarks from that regressor supply a dense landmark loss that makes appearance restoration and mesh recovery pull in the same direction. On a large public benchmark of news presenter videos, the pipeline outperforms existing video inpainting baselines on image-quality metrics, and the reported geometry errors decrease as landmark density rises from 10 to 216. If the claim holds, social XR teleconferencing could restore gaze and expression without infrared cameras, depth sensors, or per-user calibration.

What carries the argument

The load-bearing object is a landmark-to-3DMM feedback loop. Dense 3D facial landmarks (up to 216 points in the occluded region) are fed into the inpainting network alongside a masked RGB sequence and one occlusion-free reference frame; a gated temporal shift module with self-attention fills the missing upper face; a convolutional network refines landmarks on the inpainted frames by injecting 3D Morphable Model semantics; a SynergyNet-style backbone regresses 3DMM identity, expression, and pose parameters from the frames and refined landmarks, yielding a mesh with per-vertex $(x,y,z)$ coordinates; and a Huber-based dense landmark loss compares predicted landmarks against ground-truth landmarks to optimize the whole chain. Depth is never estimated by a separate network; it comes out of the 3DMM parameters.

What would settle it

An independent 3D capture, such as a depth camera or multi-view scan, of the same people with and without the headset would settle the geometry claim: if the recovered meshes show large errors against that scan while round-trip scores look good, the claimed reconstruction of true 3D face shape is refuted. A second check is to drive the pipeline with landmarks from real headset eye trackers rather than landmarks detected on unobstructed frames; if inpainting quality collapses, the practical claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that headset removal and 3D face reconstruction belong in one geometry-aware pipeline rather than two separate stages. Dense facial landmarks extracted from unobstructed frames, or available from headset eye trackers, provide spatial structure that guides the video inpainter; the inpainted RGB frames are then passed to a 3D Morphable Model based regressor that produces a face mesh; and the regressor's refinement of landmarks supplies a dense Huber loss that supervises both stages. In the authors' evaluation, the full pipeline using 216 landmarks achieves the best scores on image quality metrics and beats the inpainting-only baselines, while the same framework with only 10 eyelid landmarks remains close, showing graceful degradation. The recovered meshes, textured for visualization, are claimed to be photorealistic and identity-preserving.

Load-bearing premise

The evaluation's ground-truth 3D meshes are produced by the same reconstruction network that later processes the inpainted frames, so the geometric error scores measure round-trip agreement with that network's own outputs rather than absolute face shape.

Editorial extensions

If this is right

  • Video calls in XR can restore eye contact and eyebrow and expression cues from a single webcam stream plus one clean reference photo, without infrared cameras, depth sensors, or per-user calibration.
  • Because the output includes a 3D mesh, remote participants could view the user from novel angles rather than only seeing the fixed frontal video.
  • The gradual degradation from 216 to 10 landmarks means consumer headsets with sparse eye-tracking landmark data may still produce usable avatars.
  • Geometry-aware supervision improves the inpainted video itself, so appearance-only inpainting baselines are not sufficient for high-quality headset removal.
  • The same landmark-to-3DMM loop should transfer to other fixed upper-face occlusions, such as surgical masks or safety goggles, without changing the pipeline's logic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 3D geometry metrics are self-referential, so the 'photorealistic 3D face geometry' claim should be read as round-trip fidelity to the chosen reconstruction backbone until independent 3D ground truth is provided.
  • Because the reference frame comes from the same video sequence, the framework may depend on similar pose, lighting, and expression between reference and target; the paper's acknowledged shadowing and large-head-movement limitations suggest this dependence is real.
  • A natural next experiment is to remove the landmark channel entirely and drive the inpainter with gaze and eyelid estimates from headset sensors; if quality is preserved, the geometry-aware loop could run with near-zero external inputs.
  • Synthetic headset masks do not reproduce shadows, reflections, or skin deformation under the device, so the reported gains may shrink on real headset footage; a real-world capture benchmark with varied lighting is the missing test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes a geometry-aware framework that jointly removes head-mounted display (HMD) occlusions from monocular RGB video and reconstructs 3D face geometry. The pipeline uses an EVI-HRnet-based GAN for video inpainting, a dense landmark regression module, and a SynergyNet backbone that regresses 3DMM parameters from the inpainted frames. Training combines adversarial, FER, style, VGG, reconstruction, and dense-landmark Huber losses, with a two-stage protocol that fine-tunes SynergyNet jointly with the generator. Experiments on FaceForensics with synthetic HMD masks compare RGB inpainting performance against EVI-HRnet, CombCN, and LGTSM, and an ablation study varies the number of facial landmarks from 10 to 216 to measure both RGB quality and 3D mesh distances. The paper also includes a qualitative real-world demonstration with a Meta Quest Pro and a webcam.

Significance. The RGB inpainting contribution is evaluated carefully against relevant baselines on a standard dataset, and the consistent improvements in FID, MSE, LPIPS, SSIM, and PSNR are a credible strength. The systematic ablation over landmark density is also useful, and the promise of public code and output videos would aid reproducibility. However, the central claim of accurate 3D face reconstruction is not yet supported: the 3D ground truth is generated by the same SynergyNet module used for prediction, and the geometry metrics therefore measure round-trip consistency rather than absolute accuracy. The paper would be valuable if the 3D component were validated against independent ground truth or if the claims were appropriately restricted.

major comments (3)
  1. [§3.4, §4.1, §4.2, Table 2] The 3D geometry evaluation is circular. The GT meshes are produced by applying SynergyNet to the original unoccluded frames (Section 3.4 and Section 4.1), while the predicted meshes are produced by the same SynergyNet backbone, fine-tuned jointly with the generator in Stage 2, from the inpainted frames. The Chamfer, RMS, and Hausdorff distances therefore measure how consistently the inpainted frames reproduce SynergyNet's own 3DMM output, not how accurately the recovered geometry matches true face shape. I ask for independent validation: for example, multi-view or depth-sensor ground truth, or at minimum a cross-check with a different single-image 3D face reconstruction method applied to both inpainted and original frames, together with a report of SynergyNet's own error against independent GT on the original frames. Without this, Table 2 cannot support the abstract's claim of accurate 3D face reconstruction.
  2. [§3.4, Eq. (2), §4.1] The geometry metrics are partially optimized at training time. DenseLMLoss (Eq. 2) trains the model so that SynergyNet landmarks extracted from inpainted frames match MediaPipe landmarks from the original unoccluded frames, and in Stage 2 SynergyNet itself is fine-tuned jointly with the generator. Consequently, the reported mesh distances in Table 2 are not an independent measure of geometric fidelity; they reflect agreement with the specific 3DMM/landmark signature used as supervision. Please either evaluate on a held-out source of 3D ground truth or clearly restrict the claim to consistency with SynergyNet reconstruction rather than absolute geometric accuracy.
  3. [§4.2, Table 2] The reported numbers for the 10-landmark configuration are internally inconsistent: Table 2 lists an Average Chamfer Distance of 2.754, while the text states 2.934. Since these values are load-bearing for the landmark-density conclusion, please correct the discrepancy and ensure that all quantitative claims in the text match the corresponding tables exactly.
minor comments (7)
  1. [§2] There is a typo in the paragraph on the EVI-HRnet baseline: 'EVI-HRnnet' should read 'EVI-HRnet'.
  2. [§4.1, Table 1] The second-best values in the Ours (68 LM) row are not visible in the rendered table. Please fix the table formatting so that every reported value is readable.
  3. [§3.4, Eq. (1)] The Huber loss definition appears garbled in the manuscript. Please typeset Eq. (1) properly and state the numerical value of the threshold delta used during training.
  4. [§3.4, Eq. (4)] The six loss weights in Eq. (4) are listed without any sensitivity analysis. If all weights are equal to unity, this should be stated explicitly; otherwise a brief justification of the chosen values would help reproducibility.
  5. [§3.2, §4.1] For the proposed method the reference frame is masked outside the HMD region, while for baseline methods the first frame is left fully unmasked. Please clarify whether this input difference affects the fairness of the comparison in Table 1.
  6. [§5, Figure 6] The real-world comparison in Figure 6 uses different capture setups for each method, and the caption notes that identical input images are not available. The text should clearly separate this illustrative demonstration from the benchmark evaluation so that readers do not interpret Figure 6 as a quantitative comparison.
  7. [§5] The qualitative demonstration says the reconstructed 3D meshes are textured using UV maps from FFHQ-UV, not by the proposed method. Please state explicitly in the main text which parts of the rendered results are generated by the framework and which are borrowed from external data.

Circularity Check

2 steps flagged · score 6.0 of 10

Table 2's 3D geometry metrics are circular: GT meshes come from the same SynergyNet pipeline that reconstructs from inpainted frames, so Chamfer/RMS/Hausdorff measure round-trip consistency, not absolute face geometry accuracy.

  1. self definitional [Section 3.4 (Geometry Reconstruction) and Section 4.1 (Implementation Details), used by Table 2]
    "It is noteworthy that, due to the unavailability of the person’s face mesh model in real-world scenarios, the GT face geometry models are also generated from the original RGB video frames without HMD occlusion, using the SynergyNet method. This helps the model to learn the desired features during optimization and training of the whole framework."

    The predicted 3D meshes are produced by the same SynergyNet backbone from inpainted frames, and SynergyNet is fine-tuned jointly with the generator in Stage 2. The Chamfer, RMS, and Hausdorff distances in Table 2 therefore compare SynergyNet(inpainted frame) against SynergyNet(original unoccluded frame), i.e., a round trip through one regressor, not a comparison against independent 3D ground truth. The claimed 'accurate 3D face reconstruction' is evaluated against meshes generated by the very same estimator family, so the ground truth is defined in terms of the predictor rather than measured externally.

  2. fitted input called prediction [Section 3.5 (Optimization), DenseLMLoss (Eq. 2), and Section 3.5 Training Protocol]
    "DenseLMLoss: predicted 3D landmarks extracted from SynergyNet output (inpainted frames) vs. GT 3D landmarks extracted by MediaPipe from the original non-occluded frames. ... In Stage 2, it is fine-tuned together with the generator to adapt its latent 3D representation to the evolving inpainted frames and refined dense landmark maps."

    The DenseLMLoss explicitly trains the generator and SynergyNet so that landmarks detected from inpainted frames match MediaPipe landmarks extracted from the original frames. Table 2 then reports Chamfer/RMS/Hausdorff distances between SynergyNet meshes obtained from those two frame sets. The geometry agreement being reported as evidence of reconstruction accuracy is itself a training objective of the pipeline, so the Table 2 numbers are partly optimized into the model rather than being an independent assessment of face geometry fidelity.

full rationale

The RGB inpainting contribution is benchmarked against external baselines (EVI-HRnet, LGTSM, CombCN) using standard image-quality metrics, so the 2D inpainting claim is not circular and retains independent support. The circularity is confined to the 3D geometry evaluation that underlies the central 'accurate 3D face reconstruction' claim. The paper states that ground-truth face meshes are generated with SynergyNet from unoccluded frames, while the predicted meshes are generated with the same SynergyNet backbone from inpainted frames, and this backbone is fine-tuned during Stage 2. Consequently, the Chamfer, RMS, and Hausdorff metrics in Table 2 measure the consistency of one 3DMM regressor across original and inpainted images rather than absolute geometric accuracy; no independent depth-scan or multi-view calibration is supplied. The DenseLMLoss further optimizes the landmark agreement between the inpainted and original frames, making part of the reported geometry metric a fitted training signal. The paper's own future-work statement that a multi-view calibrated setup is needed confirms the absence of absolute geometry validation. The RGB results, however, mean the paper is only partially circular, not wholly so; hence a score of 6 rather than higher.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The only fitted values in the paper are the manually chosen loss weights and the Huber delta, both unreported in usable form. The evaluation depends on the domain assumptions that SynergyNet and MediaPipe produce accurate geometry and landmarks, and that synthetic masks are adequate proxies. No new physical or network entities are introduced.

free parameters (2)
  • Loss weights (λ_Adv, λ_FER, λ_Style, λ_VGG, λ_Recon, λ_DenseLM) = Unspecified in text
    Equation (4) values are garbled; weights are hand-chosen and directly affect the balance between inpainting realism and geometric fidelity.
  • Huber loss threshold δ = Not reported
    Equation (1) defines δ, but its value is not given; it controls the L1/L2 transition for landmark regression.
assumptions (3)
  • domain assumption SynergyNet regresses accurate 3DMM parameters from small RGB face images
    The entire 3D reconstruction stage relies on SynergyNet's pretrained weights being an accurate estimator of facial geometry; no independent 3D ground truth is used to verify this in the HMD-removal setting.
  • domain assumption MediaPipe Face Landmarker provides reliable dense 3D landmarks on GT frames and inpainted frames
    Landmarks from MediaPipe are treated as ground truth for both the inpainting guidance and the DenseLMLoss; no manual verification or cross-check is reported.
  • ad hoc to paper Synthetic HMD masks approximate real HMD occlusion
    The authors state that using synthetic masks does not capture real-world HMD shadows; this simplification is assumed throughout training and evaluation and is acknowledged in the Discussion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Geometry-Aware Video Inpainting for Joint Headset Occlusion Removal and Face Reconstruction in Social XR." pith.science (2026). https://pith.science/paper/324VNO2Y

@misc{pith2026250812336,
  author       = {Pith},
  title        = {Pith review of: Geometry-Aware Video Inpainting for Joint Headset Occlusion Removal and Face Reconstruction in Social XR},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/324VNO2Y}},
  note         = {Machine review of arXiv:2508.12336}
}
read the original abstract

Head-mounted displays (HMDs) are essential for experiencing extended reality (XR) environments and observing virtual content. However, they obscure the upper part of the user's face, complicating external video recording and significantly impacting social XR applications such as teleconferencing, where facial expressions and eye gaze details are crucial for creating an immersive experience. This study introduces a geometry-aware learning-based framework to jointly remove HMD occlusions and reconstruct complete 3D facial geometry from RGB frames captured from a single viewpoint. The method integrates a GAN-based video inpainting network, guided by dense facial landmarks and a single occlusion-free reference frame, to restore missing facial regions while preserving identity. Subsequently, a SynergyNet-based module regresses 3D Morphable Model (3DMM) parameters from the inpainted frames, enabling accurate 3D face reconstruction. Dense landmark optimization is incorporated throughout the pipeline to improve both the inpainting quality and the fidelity of the recovered geometry. Experimental results demonstrate that the proposed framework can successfully remove HMDs from RGB facial videos while maintaining facial identity and realism, producing photorealistic 3D face geometry outputs. Ablation studies further show that the framework remains robust across different landmark densities, with only minor quality degradation under sparse landmark configurations.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 49 canonical work pages

  1. [1]

    S. F. Langa, M. Montagud, G. Cernigliaro, et al. , ``Multiparty holomeetings: Toward a new era of low-cost volumetric holographic meetings in virtual reality,'' IEEE Access 10 , 81856--81876 (2022)

    work page 2022

  2. [2]

    M. A. Rojas-S \'a nchez, P. R. Palos-S \'a nchez, and J. A. Folgado-Fern \'a ndez, ``Systematic literature review and bibliometric analysis on virtual reality and education,'' Education and Information Technologies 28 (1), 155--192 (2023)

    work page 2023

  3. [3]

    S. G. Ali, X. Wang, P. Li, et al. , ``A systematic review: Virtual-reality-based techniques for human exercises and health improvement,'' Frontiers in Public Health 11 , 1143947 (2023)

    work page 2023

  4. [4]

    Simons, I

    A. Simons, I. Wohlgenannt, S. Zelt, et al. , ``Intelligence at play: game-based assessment using a virtual-reality application,'' Virtual Reality , 1--17 (2023)

    work page 2023

  5. [5]

    Numan, F

    N. Numan, F. Ter Haar, and P. Cesar, ``Generative rgb-d face completion for head-mounted display removal,'' in 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) , 109--116, IEEE (2021)

    work page 2021

  6. [6]

    J. Lou, Y. Wang, C. Nduka, et al. , ``Realistic facial expression reconstruction for vr hmd users,'' IEEE Transactions on Multimedia 22 (3), 730--743 (2019)

    work page 2019

  7. [7]

    Chen, Y.-K

    S.-Y. Chen, Y.-K. Lai, S. Xia, et al. , ``3d face reconstruction and gaze tracking in the hmd for virtual interaction,'' IEEE Transactions on Multimedia (2022)

    work page 2022

  8. [8]

    Thies, M

    J. Thies, M. Zollh \"o fer, M. Stamminger, et al. , ``Facevr: Real-time facial reenactment and eye gaze control in virtual reality,'' arXiv preprint arXiv:1610.03151 (2016)

    arXiv 2016

Show all 56 references
  1. [9]

    M. Wang, X. Wen, and S.-M. Hu, ``Faithful face image completion for hmd occlusion removal,'' in 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) , 251--256, IEEE (2019)

    2019

  2. [10]

    Y. Zhao, W. Chen, J. Xing, et al. , ``Identity preserving face completion for large ocular region occlusion,'' arXiv preprint arXiv:1807.08772 (2018)

    2018 arXiv

  3. [11]

    Ghorbani Lohesara, K

    F. Ghorbani Lohesara, K. Egiazarian, and S. Knorr, ``Expression-aware video inpainting for hmd removal in xr applications,'' in Proceedings of the 20th ACM SIGGRAPH European Conference on Visual Media Production , 1--9 (2023)

    2023

  4. [12]

    Chang, Z

    Y.-L. Chang, Z. Y. Liu, K.-Y. Lee, et al. , ``Learnable gated temporal shift module for deep video inpainting,'' arXiv preprint arXiv:1907.01131 (2019)

    2019 arXiv

  5. [13]

    C.-Y. Wu, Q. Xu, and U. Neumann, ``Synergy between 3dmm and 3d landmarks for accurate 3d facial geometry,'' in 2021 International Conference on 3D Vision (3DV) , 453--463, IEEE (2021)

    2021

  6. [14]

    E. Wood, T. Baltru s aitis, C. Hewitt, et al. , ``3d face reconstruction with dense landmarks,'' in European Conference on Computer Vision , 160--177, Springer (2022)

    2022

  7. [15]

    Dijkstra-Soudarissanane, K

    S. Dijkstra-Soudarissanane, K. E. Assal, S. Gunkel, et al. , ``Multi-sensor capture and network processing for virtual reality conferencing,'' in Proceedings of the 10th ACM Multimedia Systems Conference , 316--319 (2019)

    2019

  8. [16]

    S. N. Gunkel, R. Hindriks, K. M. E. Assal, et al. , ``Vrcomm: an end-to-end web system for real-time photorealistic social vr communication,'' in Proceedings of the 12th ACM multimedia systems conference , 65--79 (2021)

    2021

  9. [17]

    G. A. Moutsinas, J. A. Esponda-P \'e rez, B. Senapati, et al. , ``Application of virtual reality in education,'' in Computer Science On-line Conference , 319--326, Springer (2023)

    2023

  10. [18]

    P. Wang, Y. Wang, M. Billinghurst, et al. , ``Behere: a vr/sar remote collaboration system based on virtual replicas sharing gesture and avatar in a procedural task,'' Virtual Reality , 1--22 (2023)

    2023

  11. [19]

    C. G. Fidalgo, M. Sousa, D. Mendes, et al. , ``Magic: Manipulating avatars and gestures to improve remote collaboration,'' in 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR) , 438--448, IEEE (2023)

    2023

  12. [20]

    Y. Zhao, Q. Xu, W. Chen, et al. , ``Mask-off: Synthesizing face images in the presence of head-mounted displays,'' in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) , 267--276 (2019)

    2019

  13. [21]

    S.-Y. Chen, L. Gao, Y.-K. Lai, et al. , ``Real-time 3d face reconstruction and gaze tracking for virtual reality,'' in 2018 IEEE Conference on virtual reality and 3d user interfaces (VR) , 525--526, IEEE (2018)

    2018

  14. [22]

    Frueh, A

    C. Frueh, A. Sud, and V. Kwatra, ``Headset removal for virtual and mixed reality,'' in ACM SIGGRAPH 2017 Talks , SIGGRAPH '17 , Association for Computing Machinery, (New York, NY, USA) (2017)

    2017

  15. [23]

    G. Chen, G. Zhang, Z. Yang, et al. , ``Multi-scale patch-gan with edge detection for image inpainting,'' Applied Intelligence 53 (4), 3917--3932 (2023)

    2023

  16. [24]

    Y. Chen, R. Xia, K. Yang, et al. , ``Dgca: high resolution image inpainting via dr-gan and contextual attention,'' Multimedia Tools and Applications , 1--21 (2023)

    2023

  17. [25]

    Lahiri, A

    A. Lahiri, A. K. Jain, S. Agrawal, et al. , ``Prior guided gan based semantic inpainting,'' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 13696--13705 (2020)

    2020

  18. [26]

    J. Yu, Z. Lin, J. Yang, et al. , ``Free-form image inpainting with gated convolution,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision , 4471--4480 (2019)

    2019

  19. [27]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, et al. , ``Generative adversarial nets,'' Advances in neural information processing systems 27 (2014)

    2014

  20. [28]

    Gupta, A

    S. Gupta, A. Shetty, and A. Sharma, ``Attention based occlusion removal for hybrid telepresence systems,'' in 2022 19th Conference on Robots and Vision (CRV) , 167--174, IEEE (2022)

    2022

  21. [29]

    Gupta, S

    S. Gupta, S. S. Jinka, A. Sharma, et al. , ``Supervision by landmarks: An enhanced facial de-occlusion network for vr-based applications,'' in European Conference on Computer Vision , 323--337, Springer (2022)

    2022

  22. [30]

    J. Yu, Z. Lin, J. Yang, et al. , ``Generative image inpainting with contextual attention,'' in Proceedings of the IEEE conference on computer vision and pattern recognition , 5505--5514 (2018)

    2018

  23. [31]

    Z. Wan, J. Zhang, D. Chen, et al. , ``High-fidelity pluralistic image completion with transformers,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision , 4692--4701 (2021)

    2021

  24. [32]

    Z. Liu, Y. Lin, Y. Cao, et al. , ``Swin transformer: Hierarchical vision transformer using shifted windows,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision , 10012--10022 (2021)

    2021

  25. [33]

    Lugmayr, M

    A. Lugmayr, M. Danelljan, A. Romero, et al. , ``Repaint: Inpainting using denoising diffusion probabilistic models,'' Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 11461--11471 (2022)

    2022

  26. [34]

    Saharia, W

    C. Saharia, W. Chan, S. Saxena, et al. , ``Palette: Image-to-image diffusion models,'' in ACM SIGGRAPH 2022 Conference Proceedings , 1--10 (2022)

    2022

  27. [35]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al. , ``An image is worth 16x16 words: Transformers for image recognition at scale,'' arXiv preprint arXiv:2010.11929 (2020)

    2020 arXiv

  28. [36]

    J. Ho, A. Jain, and P. Abbeel, ``Denoising diffusion probabilistic models,'' Advances in Neural Information Processing Systems 33 , 6840--6851 (2020)

    2020

  29. [37]

    Bau, J.-Y

    D. Bau, J.-Y. Zhu, J. Wulff, et al. , ``Seeing what a gan cannot generate,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision , 4502--4511 (2019)

    2019

  30. [38]

    X. Zou, L. Yang, D. Liu, et al. , ``Progressive temporal feature alignment network for video inpainting,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 16448--16457 (2021)

    2021

  31. [39]

    H. Bai, D. Kang, H. Zhang, et al. , ``Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 362--371 (2023)

    2023

  32. [40]

    J. Guo, X. Zhu, Y. Yang, et al. , ``Towards fast, accurate and stable 3d dense face alignment,'' in European Conference on Computer Vision , 152--168, Springer (2020)

    2020

  33. [41]

    Zhang, I

    H. Zhang, I. Goodfellow, D. Metaxas, et al. , ``Self-attention generative adversarial networks,'' in International conference on machine learning , 7354--7363, PMLR (2019)

    2019

  34. [42]

    Simonyan and A

    K. Simonyan and A. Zisserman, ``Very deep convolutional networks for large-scale image recognition,'' arXiv preprint arXiv:1409.1556 (2014)

    2014 arXiv

  35. [43]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, et al. , ``Imagenet large scale visual recognition challenge,'' International journal of computer vision 115 , 211--252 (2015)

    2015

  36. [44]

    L. A. Gatys, A. S. Ecker, and M. Bethge, ``A neural algorithm of artistic style,'' arXiv preprint arXiv:1508.06576 (2015)

    2015 arXiv

  37. [45]

    Arjovsky, S

    M. Arjovsky, S. Chintala, and L. Bottou, ``Wasserstein generative adversarial networks,'' in International conference on machine learning , 214--223, PMLR (2017)

    2017

  38. [46]

    A. V. Savchenko, ``Video-based frame-level facial analysis of affective behavior on mobile devices using efficientnets,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2359--2366 (2022)

    2022

  39. [47]

    Dhall, R

    A. Dhall, R. Goecke, S. Lucey, et al. , ``Collecting large, richly annotated facial-expression databases from movies,'' IEEE multimedia 19 (3), 34 (2012)

    2012

  40. [48]

    C. Wang, H. Huang, X. Han, et al. , ``Video inpainting by jointly learning temporal structure and spatial details,'' in Proceedings of the AAAI Conference on Artificial Intelligence , 33 (01), 5232--5239 (2019)

    2019

  41. [49]

    R \"o ssler, D

    A. R \"o ssler, D. Cozzolino, L. Verdoliva, et al. , ``Faceforensics: A large-scale video dataset for forgery detection in human faces,'' arXiv preprint arXiv:1803.09179 (2018)

    2018 arXiv

  42. [50]

    Z. Wang, A. C. Bovik, H. R. Sheikh, et al. , ``Image quality assessment: from error visibility to structural similarity,'' IEEE transactions on image processing 13 (4), 600--612 (2004)

    2004

  43. [51]

    Zhang, P

    R. Zhang, P. Isola, A. A. Efros, et al. , ``The unreasonable effectiveness of deep features as a perceptual metric,'' in Proceedings of the IEEE conference on computer vision and pattern recognition , 586--595 (2018)

    2018

  44. [52]

    Heusel, H

    M. Heusel, H. Ramsauer, T. Unterthiner, et al. , ``Gans trained by a two time-scale update rule converge to a local nash equilibrium,'' Advances in neural information processing systems 30 (2017)

    2017

  45. [53]

    Suzuki, F

    K. Suzuki, F. Nakamura, J. Otsuka, et al. , ``Recognition and mapping of facial expressions to avatar by embedded photo reflective sensors in head mounted display,'' in 2017 IEEE Virtual Reality (VR) , 177--185, IEEE (2017)

    2017

  46. [54]

    Olszewski, J

    K. Olszewski, J. J. Lim, S. Saito, et al. , ``High-fidelity facial and speech animation for vr hmds,'' ACM Transactions on Graphics (TOG) 35 (6), 1--14 (2016)

    2016

  47. [55]

    Frueh, A

    C. Frueh, A. Sud, and V. Kwatra, ``Headset removal for virtual and mixed reality,'' in ACM SIGGRAPH 2017 Talks , 1--2 (2017)

    2017

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION blank.sep after.quote 'output.state := FUNCTION fin.entry output.state after.quoted.block = 'skip 'add.period if write newline FUNCTION new.block output.state before.all = 'skip output.state after.quote = after.quoted.blo...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.