Pith. sign in

REVIEW 3 major objections 6 minor 6 cited by

Progressive Face Super-Resolution via Attention to Facial Landmark

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Landmark heatmaps, used as attention weights, let a progressive network super-resolve faces 8x with photo-realistic detail.

desk verdict A solid incremental face SR paper whose perceptual gains look real, but the abstract overclaims quantitative superiority and the missing error bars need fixing before publication. read the letter →

arxiv 1908.08239 v1 pith:NUZW3GHW submitted 2019-08-22 cs.CV

Deokyun Kim , Minseon Kim , Gihyun Kwon , Dae-Shik Kim This is my paper · ORCID
classification cs.CV
keywords facesuper-resolutionprogressivetrainingfaciallandmarkheatmapattentionlossnetworkdistillationgenerativeadversarialperceptualquality8xupscaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that face super-resolution in the extreme 8x regime can be made stable and perceptually realistic by training in progressive 2x steps and by steering the reconstruction loss with facial landmark heatmaps. The network is claimed to restore fine details around eyes, nose, and mouth without distortion, and to beat prior face SR methods on structural similarity and human perceptual ratings, even though its pixel PSNR is slightly below a very deep non-face baseline. The authors argue this works because the heatmaps come from a lightweight distilled face alignment network whose responses concentrate on non-occluded landmark areas.

What carries the argument

The load-bearing mechanism is the facial attention loss, defined as $L_{\text{attention}} = \frac{1}{r^2 WH}\sum_{x,y} M^*_{x,y}\, |I^{HR}_{x,y} - G(I^{LR})_{x,y}|$, where $M^*$ is the channel-wise maximum of the target image's landmark heatmap after min-max normalization to [0,1]. This loss is applied at the 4x and 8x steps of a progressively grown generator–discriminator pair that uses residual blocks, transpose convolutions, and a Wasserstein adversarial loss with gradient penalty. The heatmaps come from a distilled face alignment network, trained by hint-based regression to match the original FAN's heatmaps; because it lacks the encoder–decoder and skip-layer structure, its responses are broad over visible landmark regions and low over occluded ones, which the paper argues makes them suitable attention weights. The machinery also includes a heatmap loss that penalizes disagreement between heatmaps of the generated and target images to preserve structural consistency.

What would settle it

Retrain the same progressive network with the attention heatmaps deliberately corrupted—shifted by several pixels, replaced by random maps, or swapped between paired images—and compare SSIM and MOS on the test set; if the reported gains do not clearly disappear, the facial attention loss is not responsible for the claimed detail restoration.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an 8x face SR generator can be trained progressively—growing from 2x to 4x to 8x—and that at each later stage a facial attention loss should multiply the L1 pixel difference by a min-max-normalized landmark heatmap. The heatmap is the channel-wise max of the heatmaps produced by a distilled face alignment network, so the loss focuses the generator on regions around facial landmarks while suppressing occluded regions. A corresponding heatmap loss keeps the structure of the generated face aligned with the target. The result is a network that produces 128x128 images from 16x16 inputs with preserved facial attributes, and the authors report that it outperforms prior methods on SSIM, MS-SSIM, and a mean-opinion-score perceptual test.

Load-bearing premise

The load-bearing premise is that the distilled landmark heatmaps reliably mark which pixels carry facial detail worth preserving, so that weighting the reconstruction loss by those maps improves actual facial fidelity rather than merely sharpening landmark regions.

Editorial extensions

If this is right

  • 8x face super-resolution can be trained stably without attribute vectors or other priors that are hard to obtain in the wild.
  • Applying the landmark attention loss at intermediate progressive steps yields measurable gains in SSIM, MS-SSIM, and human MOS over applying the same losses without progressive training.
  • A distilled landmark network with roughly 1.35% of the original FAN's parameters produces heatmaps that work better for attention and cuts training time from about 3 days to about 1 day.
  • The method generalizes to both aligned and unaligned face crops, suggesting it does not depend on tight face pre-registration.
  • The reported MOS gap over prior face SR methods implies the main practical advantage is perceptual realism rather than raw pixel accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same heatmap-weighted loss could be applied to other landmark-structured objects, such as hands, bodies, or medical structures, whenever a cheap landmark estimator is available, making the attention mechanism task-agnostic.
  • Editorial inference: the distilled FAN's low response on occluded landmarks suggests the attention loss doubles as an occlusion-aware weighting; a direct test would compare aligned versus misaligned heatmap weights on occluded faces.
  • Editorial inference: because PSNR is slightly lower than VDSR while MOS is much higher, the metric story implies pixel-level fidelity and human-perceived quality diverge more sharply for faces than for general images; this could be tested by ablating the adversarial loss while keeping the attention loss.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a progressive face super-resolution method that upscales faces 8x using a generator trained in three steps, a facial attention loss that weights pixel differences by landmark heatmaps, and a distilled (compressed) Face Alignment Network used to produce those heatmaps. The method is evaluated on aligned and unaligned CelebA and AFLW test sets with PSNR, SSIM, MS-SSIM, and a MOS perceptual test, and the authors report state-of-the-art results especially in perceptual quality, with a compact landmark network and reduced training time.

Significance. If the reported results hold, the paper offers a useful combination of progressive training, landmark-guided attention, and model distillation for face super-resolution. The ablation study in Table 2 supports the contribution of the facial attention loss and progressive training, and the GitHub release makes the approach reproducible. The main value is the perceptual-quality improvement shown in the MOS test, though that test currently lacks the statistical detail needed to verify the strength of the claim.

major comments (3)
  1. [Abstract; Table 3] The abstract's claim that the method 'outperforms state-of-the-art methods in both qualitative and quantitative measurements' is not supported by the paper's own Table 3: VDSR reports higher PSNR on both the aligned (22.94 vs. 22.66) and unaligned (23.70 vs. 22.96) test sets, and PSNR is a quantitative measurement. The text's explanation that VDSR outputs are blurred is reasonable, but the quantitative claim must be narrowed to structural and perceptual metrics (SSIM, MS-SSIM, MOS) with an explicit acknowledgment that the method does not lead on PSNR.
  2. [Section 4.4 (MOS)] The MOS test is the key evidence for the perceptual-quality claim, but no confidence intervals, per-rater variance, or significance tests are reported, and the paper refers to 'Supplementary Materials' for details that are not included in the submission. Without these, the statement that the method shows 'overwhelming performance' in Figure 7 cannot be assessed; at minimum the authors should report the number of scenes rated, inter-rater agreement, and a paired significance test against the closest competitor.
  3. [Section 4.2; Table 1] The claim that the distilled FAN produces heatmaps 'suitable for facial attention weights' is validated only indirectly: Table 1 reports NME computed against the original FAN's predictions, which measures agreement with the teacher rather than landmark accuracy, and the PSNR/SSIM improvements attributed to the distilled FAN are small and given without variance estimates. The authors should provide an independent landmark-localization evaluation (e.g., against manual annotations) and error bars, or explicitly restrict the claim to 'agrees with the original FAN while improving SR metrics in our pipeline.'
minor comments (6)
  1. [Table 1] Table 1 contains malformed entries (e.g., '(64x64)- (128x128)-', '0.830% 2.2643%22.96') and the column labels are garbled; please reformat the table so each metric is a separate numeric column.
  2. [Section 3.2] Equation (1) is defined with G(ILR), but the loss is applied at Steps 2 and 3 with different upscale factors; please clarify which step's generator output is used in each application.
  3. [Section 3.2] The normalization of M is unclear: the text says the heatmap is 'min-max normalized into [0,1]' and that M* is the 'channel-wise max values'; please specify whether normalization is applied per landmark channel or globally across all channels.
  4. [Section 3.1] The notation N∈{2,3} is used for the step number, but N is also used for the number of landmarks in Eq. (5); please use distinct symbols for these two quantities.
  5. [Section 4.1; Eq. (6)] The overall loss in Eq. (6) depends on α, β, γ, λ, and η, but their values are never reported; please include them in the implementation details.
  6. [References] The gradient penalty is attributed to reference [23], but WGAN-GP is normally credited to Gulrajani et al. (NIPS 2017); please verify the citation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SR pipeline is trained and evaluated against external benchmarks; the distilled FAN is an empirical component whose contribution is ablated, not assumed.

full rationale

The claimed derivation chain is self-contained. The face SR network is trained with pixel, perceptual, adversarial, heatmap, and attention losses (Eqs. 1-6), and the contribution of each loss is tested by ablation against the external PSNR/SSIM/MS-SSIM metrics (Table 2) and against published baselines (Table 3). The distilled FAN is trained by hint-based regression to the published FAN's heatmaps and is evaluated with NME and parameter counts (Table 1), so its behavior is anchored to an external network rather than to the SR objective. The only close call is the Sec. 4.2 claim that the distilled FAN's heatmaps are 'suitable for facial attention weights'; this is validated by an internal comparison of SR metrics using the original versus distilled FAN, but that comparison is an ordinary ablation whose outcome was not forced by construction and whose metrics are external to the attention mechanism. No fitted parameter is relabeled as a prediction, and no self-citation chain supports the central claim. The abstract's unqualified claim of superiority in 'both qualitative and quantitative measurements' conflicts with Table 3, where VDSR achieves the highest PSNR on both test sets; this is a claim-accuracy / overreach issue, not a circularity issue.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical entities, forces, or conserved quantities are introduced. The distilled FAN is a trained neural network, not a postulated entity, so the invented-entities ledger is empty.

free parameters (2)
  • Loss weights alpha, beta, gamma, lambda, eta = not reported
    Eq. 6 defines the total loss with these weights, but their numerical values are never given; the exact objective cannot be reproduced from the text.
  • Training iterations per progressive step = 50K, 50K, 100K
    Chosen empirically in Section 4.1 rather than by a stopping criterion.
assumptions (3)
  • domain assumption Facial landmark heatmaps encode the regions where facial detail matters and are reliable guides for super-resolution.
    Underlies the facial attention loss (Eq. 1) and heatmap loss (Eq. 5). If the heatmaps mislocate features, the weighting would emphasize the wrong pixels.
  • domain assumption Adversarial training with WGAN-GP and perceptual loss improves perceived realism.
    Adopted from prior work [3,7,23] without re-derivation; the paper's perceptual quality claim depends on it.
  • domain assumption MOS ratings from 26 raters are a valid estimator of perceptual quality despite no reported variance.
    Figure 7 and Table 3 treat the MOS difference 3.73 vs. 2.55 as meaningful without confidence intervals or significance tests.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Progressive Face Super-Resolution via Attention to Facial Landmark." pith.science (2026). https://pith.science/paper/NUZW3GHW

@misc{pith2026190808239,
  author       = {Pith},
  title        = {Pith review of: Progressive Face Super-Resolution via Attention to Facial Landmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NUZW3GHW}},
  note         = {Machine review of arXiv:1908.08239}
}
read the original abstract

Face Super-Resolution (SR) is a subfield of the SR domain that specifically targets the reconstruction of face images. The main challenge of face SR is to restore essential facial features without distortion. We propose a novel face SR method that generates photo-realistic 8x super-resolved face images with fully retained facial details. To that end, we adopt a progressive training method, which allows stable training by splitting the network into successive steps, each producing output with a progressively higher resolution. We also propose a novel facial attention loss and apply it at each step to focus on restoring facial attributes in greater details by multiplying the pixel difference and heatmap values. Lastly, we propose a compressed version of the state-of-the-art face alignment network (FAN) for landmark heatmap extraction. With the proposed FAN, we can extract the heatmaps suitable for face SR and also reduce the overall training time. Experimental results verify that our method outperforms state-of-the-art methods in both qualitative and quantitative measurements, especially in perceptual quality.

Figures

Figures reproduced from arXiv: 1908.08239 by the authors.

Figure 1
Figure 1. Our network architecture overview. (k : kernel size, n : output channel, s : stride) tors to lower the ambiguity in facial attributes. Lee et al. [13] fuses the information of both image domain and attribute domain in order to reflect facial attributes in super-resolved im￾ages. These methods preserve facial attributes indicated by facial attribute vectors. However, attribute vectors are not only difficult to acquir… view at source ↗
Figure 2
Figure 2. Distilled face alignment network hint based training To generate heatmaps suitable for giving atten￾tion to accurate facial landmark area, we construct the network with neither encoder-decoder architec￾ture nor skip-layer so as to predict landmarks based on single-scale feature maps. Also, in order to reduce overall training time and achieve comparable perfor￾mance to state-of-the-art FAN, we compress the FAN into t… view at source ↗
Figure 3
Figure 3. Distilled FAN results(Ours) comparison with FAN results [ [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Our image results (a) with the original FAN, (b) with the distilled FAN. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Ablation study results on aligned and unaligned datasets. (a) [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison with aligned and unaligned datasets [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: MOS result with aligned and unaligned datasets [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PASDiff: Physics-Aware Semantic Guidance for Joint Real-World Low-Light Face Enhancement and Restoration

    cs.CV 2026-03 conditional novelty 6.0 of 10

    PASDiff steers an unconditional diffusion model with inverse-intensity exposure maps, Retinex reflectance anchors, and AdaIN-aligned structural priors to restore identity-consistent faces from compound low-light degra...

  2. Robust ID-Specific Face Restoration via Alignment Learning

    cs.CV 2025-07 conditional novelty 6.0 of 10

    RIDFR injects a reference person's identity into diffusion-based face restoration and uses Alignment Learning across multiple same-identity references to suppress pose, expression, and makeup interference.

  3. RefSTAR: Blind Facial Image Restoration with Reference Selection, Transfer, and Reconstruction

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A reference-based face restoration method that explicitly selects which reference regions to transfer, uses dual-stream attention to force feature transfer, and adds a mask-compatible cycle loss, achieving state-of-th...

  4. DiffusionReward: Enhancing Blind Face Restoration through Reward Feedback Learning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A reward-feedback fine-tuning framework trains a face reward model and uses its gradient plus structural and regularization losses to improve diffusion face restoration models.

  5. F-Bench: Rethinking Human Preference Evaluation Metrics for Benchmarking Face Generation, Customization, and Restoration

    cs.CV 2024-12 conditional novelty 6.0 of 10

    FaceQ, a new 12K-image benchmark with multi-dimensional human preference scores, reveals that existing quality metrics poorly match human judgment on AI-generated faces, and F-Eval, an instruction-tuned LMM, outperforms them.

  6. InstantRestore: Single-Step Personalized Face Restoration with Shared-Image Attention

    cs.CV 2024-12 conditional novelty 6.0 of 10

    A single-step diffusion-based face restoration model uses reference-image attention to preserve identity in about 0.5 seconds per image, with no per-identity tuning.

Reference graph

Works this paper leans on

27 extracted references · 21 canonical work pages · cited by 6 Pith papers

  1. [1]

    Super-fan: Integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with gans

    Georgios Tzimiropoulos Adrian Bulat. Super-fan: Integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with gans. In CVPR, 2018

    work page 2018

  2. [2]

    Image super-resolution via progressive cascading residual network

    Namhyuk Ahn, Byungkon Kang, and Kyung-Ah Sohn. Image super-resolution via progressive cascading residual network. In CVPR Workshops, 2018

    work page 2018

  3. [3]

    Wasserstein generative adversarial net- works

    Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial net- works. In ICML, 2017

    work page 2017

  4. [4]

    How far are we from solving the 2d & 3d face align- ment problem? (and a dataset of 230,000 3d facial landmarks)

    Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2d & 3d face align- ment problem? (and a dataset of 230,000 3d facial landmarks). In ICCV, 2017

    work page 2017

  5. [5]

    Fsrnet: End-to-end learning face super-resolution with facial priors

    Yu Chen, Ying Tai, Xiaoming Liu, Chunhua Shen, and Jian Yang. Fsrnet: End-to-end learning face super-resolution with facial priors. In CVPR, 2018

    work page 2018

  6. [6]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016

    2016

  7. [7]

    Perceptual losses for real-time style transfer and super-resolution

    Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV, 2016

    2016

  8. [8]

    Accurate image super-resolution using very deep convolutional networks

    Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. Accurate image super-resolution using very deep convolutional networks. In CVPR, 2016

    2016

Show all 27 references
  1. [9]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015

    2015

  2. [10]

    Annotated facial land- marks in the wild: A large-scale, real-world database for facial landmark localization

    Martin Koestinger, Paul Wohlhart, Peter M Roth, and Horst Bischof. Annotated facial land- marks in the wild: A large-scale, real-world database for facial landmark localization. In ICCV workshop, 2011

    2011

  3. [11]

    Deep laplacian pyramid networks for fast and accurate super-resolution

    Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR, 2017

    2017

  4. [12]

    Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi

    Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew P. Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo- realistic single image super-resolution using a generative adversarial network. In CVPR, 2017

    2017

  5. [13]

    Attribute augmented convolutional neural network for face hallucination

    Cheng-Han Lee, Kaipeng Zhang, Hu-Cheng Lee, Chia-Wen Cheng, and Winston Hsu. Attribute augmented convolutional neural network for face hallucination. In CVPR Workshops, 2018

    2018

  6. [14]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015

    2015

  7. [15]

    Stacked hourglass networks for human pose esti- mation

    Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hourglass networks for human pose esti- mation. In ECCV, 2016

    2016

  8. [16]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NeurIPS workshop, 2017

    2017

  9. [17]

    Fitnets: Hints for thin deep nets

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In ICLR, 2015

    2015

  10. [18]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015. 12 D. KIM, M. KIM, G. KWON ET AL.: PROGRESSIVE FACE SUPER-RESOLUTION

    2015

  11. [19]

    Progressive growing of gans for improved quality, stability, and variation

    Samuli Laine Jaakko Lehtinen Tero Karras, Timo Aila. Progressive growing of gans for improved quality, stability, and variation. In ICLR, 2018

    2018

  12. [20]

    A fully progressive approach to single-image super- resolution

    Yifan Wang, Federico Perazzi, Brian McWilliams, Alexander Sorkine-Hornung, Olga Sorkine- Hornung, and Christopher Schroers. A fully progressive approach to single-image super- resolution. In CVPR Workshops, 2018

    2018

  13. [21]

    Simoncelli, and Alan C

    Zhou Wang, Eero P. Simoncelli, and Alan C. Bovik. Multi-scale structural similarity for image quality assessment. In IEEE Asilomar Conference on Signals, Systems and Computers , 2003

    2003

  14. [22]

    Bovik, Hamid R

    Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions Image Processing, 2004

    2004

  15. [23]

    Improving the improved train- ing of wasserstein gans: A consistency term and its dual effect

    Xiang Wei, Boqing Gong, Zixia Liu, Wei Lu, and Liqiang Wang. Improving the improved train- ing of wasserstein gans: A consistency term and its dual effect. In ICLR, 2018

    2018

  16. [24]

    Ultra-resolving face images by discriminative generative net- works

    Xin Yu and Fatih Murat Porikli. Ultra-resolving face images by discriminative generative net- works. In ECCV, 2016

    2016

  17. [25]

    Face super- resolution guided by facial component heatmaps

    Xin Yu, Basura Fernando, Bernard Ghanem, Fatih Porikli, and Richard Hartley. Face super- resolution guided by facial component heatmaps. In ECCV, 2018

    2018

  18. [26]

    Super-resolving very low-resolution face images with supplementary attributes

    Xin Yu, Basura Fernando, Richard Hartley, and Fatih Porikli. Super-resolving very low-resolution face images with supplementary attributes. In CVPR, 2018

    2018

  19. [27]

    Face detection, pose estimation, and landmark localization in the wild

    Xiangxin Zhu and Deva Ramanan. Face detection, pose estimation, and landmark localization in the wild. In CVPR, 2012

    2012

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.