{"id":"c3de8b6e-d287-4997-8200-51bd6e7e20c7","arxiv_id":"2507.15709","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A self-trained teacher and knowledge distillation produce a 1.18M-parameter face image quality model that nearly matches the 86.88M-parameter teacher on ICCV 2025 VQualA FIQA challenge data.","lead":"This paper builds a lightweight face image quality assessment model by training a large teacher model and distilling it into a small student using pseudo-labels from unlabeled face images. The student nearly matches the teacher's accuracy on a 2025 ICCV challenge with about 1% of the parameters and 100x fewer computations.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's teacher-student comparison is not tied to a specified evaluation set; the official challenge score (0.9664) is far below Table 1's 0.9855, so the 'comparable' claim lacks auditable support.","rationale":"The reader's verdict is CONDITIONAL, and my concern reinforces that conditionality without moving it to a different verdict. I partially agree with the reader's weakest_assumption about pseudo-label reliability and distribution shift: that is a genuine threat to generalization, and the unlabeled data are unavailable. However, the more immediate load-bearing problem is that the paper's headline evidence—the teacher-student comparison—is not auditable because the evaluation split is ambiguous and the official challenge score (0.9664) is much lower than Table 1's numbers (average ~0.9884). This discrepancy means the central claim may be supported only on the validation split or on an unreported aggregation, not on the official Flickr-only test. The reader's rationale did mention 'evaluation-set reporting is ambiguous,' but their formal weakest_assumption focused on pseudo-labels; I would elevate the evaluation ambiguity to the primary concern because it affects even the internal consistency of the reported results. The paper's first-place challenge finish is real and provides independent support for the method's competitiveness, but it does not by itself establish that the student is comparable to the teacher, since the teacher's official test score is absent. A clean per-split, multi-seed evaluation would resolve this, and the current CONDITIONAL verdict appropriately requires it.","tokens_in":13154,"tokens_out":5717,"duration_ms":62458,"concrete_test":"Ask the authors to release per-split evaluation of both teacher and student on the official challenge validation and test splits, at both 352x352 and 288x288, across at least 3 training seeds, reporting SRCC/PLCC and standard deviations. If the student's Flickr-only test SRCC is within about 0.005 of the teacher's on the same split and resolution, the claim stands; if the gap is large or Table 1's numbers cannot be reproduced on any defined split, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on Table 1, where the student reaches SRCC 0.9855 / PLCC 0.9914 versus the teacher's 0.9863 / 0.9924. However, the paper never states which evaluation split Table 1 uses. Section 4.1 defines a challenge validation set (1,000 images, CelebA+Flickr) and a test set (889 images, Flickr-only), then Section 4.3 says methods were 'retrained on the three test datasets'—a set that is never defined. Meanwhile, Table 4 reports the official challenge score of the same student architecture as 0.9664 (average of SRCC and PLCC), far below Table 1's average of about 0.9884. The official submission uses a 288x288 input (0.3313 GFLOPs), while Table 1's student uses 352x352 (0.4949 GFLOPs), but Table 3 shows resolution changes of only ~0.001 in SRCC/PLCC, so resolution alone cannot explain a ~0.022 gap. This strongly suggests Table 1 reports results on the easier validation split or on an unreported aggregation, not on the Flickr-only official test. If so, the claimed teacher-student comparability may not hold on the official test, and the teacher's official test score is not reported, making the comparison unreconstructable. The single-run protocol and absent confidence intervals further prevent assessing whether a 0.0008 SRCC difference is meaningful.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage framework for efficient face image quality assessment (FIQA). A Swin-B teacher is first trained on labeled challenge data, then used to pseudo-label 200k unlabeled Internet face images. These pseudo-labels are used both to fine-tune an enhanced teacher (self-training) and to distill a lightweight EdgeNeXt student; a second pseudo-labeled set is generated by the enhanced teacher and also used for student training. The loss is a weighted combination of MSE and a PLCC-based term. The authors report that the student (1.18M parameters, 0.4949 GFLOPs) achieves SRCC 0.9855 / PLCC 0.9914 versus the teacher's 0.9863 / 0.9924, and that the method ranked first in the ICCV 2025 VQualA FIQA Challenge.","tokens_in":13477,"tokens_out":5007,"duration_ms":52711,"significance":"If the empirical claims hold, the paper offers a practically useful result: a very lightweight FIQA model with accuracy close to a much larger teacher, backed by a first-place challenge result and released code. The method is clearly described and the ablations in Table 2 give some support for the contribution of the pseudo-labeling stages. The significance is currently conditional, however, because the central teacher–student comparison in Table 1 is not tied to a specified evaluation split, and the reported official challenge score for the same student architecture is substantially lower. The single-run protocol also makes the small SRCC/PLCC differences difficult to interpret.","major_comments":[{"comment":"Table 1 is the only direct evidence for the core claim that the student is comparable to the teacher, but the paper never states which evaluation split this table reports. Section 4.1 defines a validation set (1,000 images, CelebA+Flickr) and a test set (889 images, Flickr-only), yet Table 1 gives only one SRCC/PLCC pair per method. The ambiguity is consequential: Table 4 reports an official challenge score of 0.9664 (the mean of SRCC and PLCC) for the same student architecture, whereas Table 1 implies a mean of about 0.9884. Section 4.7 says the challenge submission used 288×288 input while Table 1's student uses 352×352, but Table 3 shows that resolution changes of this magnitude affect SRCC by only about 0.001, so the 0.022 gap cannot be explained by resolution alone. The authors must specify the evaluation set for Table 1, report teacher and student results on both the validation and official test splits, and report the teacher's official test score; without this, the 'comparable performance' claim is not auditable.","section":"§4.1 and Table 1"},{"comment":"The sentence 'For fairness, all methods were retrained on the three test datasets, except NIQE' is not interpretable as written and, taken literally, describes training on test data, which would invalidate the comparisons in Table 1. Please clarify whether the intended meaning is that all methods were trained on the challenge training set and evaluated on both validation and test splits, and state exactly which data splits were used for training each baseline. This is essential for reproducibility of the baseline numbers.","section":"§4.3"},{"comment":"All reported results come from a single training run, with no error bars, confidence intervals, or significance tests. The central conclusion of teacher–student comparability rests on differences as small as 0.0008 in SRCC (teacher 0.9863 vs student 0.9855) and the claimed self-training gain is 0.0004 (teacher 0.9863 vs teacher+ 0.9867). These differences may be within run-to-run variation. Please report at least three independent training runs (mean ± std) for Tables 1 and 2, or otherwise justify the stability of these small differences.","section":"§4.5, Tables 1 and 2"},{"comment":"The self-training premise is that pseudo-labels produced by the teacher on roughly 400,000 unlabeled Internet face images are reliable and sufficiently aligned with the Flickr-only test distribution. The paper does not report any confidence filtering, domain alignment, or analysis of the pseudo-label distribution. Because both the enhanced teacher and the student are trained on these pseudo-labels, a systematic bias would be inherited by the student. Since the teacher's official test score is not reported, the paper currently cannot demonstrate that the self-training step improves generalization on the actual test distribution. I would like to see at least a distributional/agreement analysis of the pseudo-labels, or an evaluation of teacher vs teacher+ on the official test split, to support the claim that the self-training step helps.","section":"§3.1 and §4.1"}],"minor_comments":[{"comment":"The notation in Eq. (3) is malformed: the loss expression has unbalanced parentheses, and the symbol θt+ is not defined. It should probably be θ_t^+ (or similar), and the objective should be written with explicit parentheses around each loss term.","section":"Eq. (3)"},{"comment":"The third term in Eq. (4) uses ¯y_i without a definition. Please define ¯y_i as the pseudo-label generated by the enhanced teacher for the second unlabeled set, and ensure the notation matches Section 3.1.","section":"Eq. (4)"},{"comment":"There is a typo in the caption: 'The the enhanced teacher model' should be 'The enhanced teacher model'.","section":"Figure 1 caption"},{"comment":"The sentence 'The lightweight baseline model, MobileNetV2, achieving moderate performance' is grammatically incomplete; it should read 'achieves moderate performance'.","section":"§4.5"},{"comment":"Table 4 reports only the overall challenge score, which is the average of SRCC and PLCC. If the individual SRCC and PLCC values for the challenge submission are available, they should be reported so that readers can compare the official test behavior with Table 1.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The core concern is not the internal logic of the method but the unreported evaluation split for Table 1 and the single-run protocol. I believe this is fixable with additional experiments: report both validation and official-test results for teacher and student, report the teacher's official test score, and add multiple-seed statistics. The challenge ranking is a genuine external signal, but it should not replace a fully specified evaluation inside the paper. There are no concerns about novelty, citation behavior, or scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a practical distillation paper for face IQA with a real first-place challenge finish, but the headline comparability claim rests on a table whose evaluation split is never specified, and the official challenge score is far lower. The gap is too big for resolution to explain, so the central claim is not auditable as written.\n\nWhat is actually new: the two-stage pseudo-labeling recipe — using the initial teacher to pseudo-label the first 200k unlabeled images, then using those to refine the teacher, then pseudo-labeling another 200k with the enhanced teacher, and distilling the student from all three supervision sources. That is a modest but legitimate extension of E-VQA and Noisy Student, and it is applied to perceptual FIQA for the first time. The ablation in Table 2 is clean: each pseudo-label source adds a small but consistent gain. The efficiency numbers are real: 1.18M parameters and 0.4949 GFLOPs for the student versus 86.88M and 50.13 for the teacher. The challenge first place is external evidence that the method works in a competitive setting, and the code is public.\n\nThe soft spots are in the reporting, not the idea. Table 1 never says which evaluation set it uses. Section 4.1 defines a 1,000-image validation set (CelebA+Flickr) and an 889-image Flickr-only test set. Table 4 reports the official test score for the same student as 0.9664 at 288x288; Table 1 shows ~0.9884 average at 352x352. Table 3 shows changing resolution from 336 to 352 shifts SRCC by 0.0007, so the 0.022 gap is not resolution. It almost certainly means Table 1 is the validation split or some unreported aggregation. If so, the 'comparable to teacher' claim is not established on the official Flickr-only test, and the teacher's official test score is missing entirely. Also, Section 4.3's phrase 'retrained on the three test datasets' is incoherent and could suggest training on test data; that passage needs a rewrite. Minor issues: single run, no error bars, lambda fixed at 1 without sensitivity analysis, unlabeled data not released.\n\nWho this is for: practitioners working on efficient FIQA or mobile IQA deployments, and challenge participants. The conceptual novelty is moderate. With a revision that pins down the evaluation split, reports teacher and student on the official test, and adds basic error bars, this would be a solid systems paper. As is, the claim outruns the evidence, despite the genuine challenge win.\n\nRecommendation: send it to peer review; the challenge result and efficiency gains deserve referee time, but the referee should make the evaluation-set clarification a condition of acceptance.","headline":"Useful distillation recipe with a real challenge win, but the comparability claim rests on an unspecified eval split that doesn't square with the official test score.","tokens_in":13992,"tokens_out":3420,"would_cite":false,"duration_ms":34523,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A distilled student model with 1.18 million parameters matches a Swin-B teacher on face image quality assessment (SRCC 0.9855 vs 0.9863) while using about one hundredth of the computation.","keywords":["face image quality assessment","knowledge distillation","self-training","pseudo-labeling","model compression","Swin Transformer","EdgeNeXt","VQualA challenge"],"falsifier":"An experiment that would settle this: train the identical student on the labeled data alone, with no pseudo-labels, and compare its SRCC/PLCC on a fresh FIQA test set from a different source (e.g., synthetic distortions or a different demographic mix). If the gap between the distilled student and the teacher shrinks or reverses on that set, the pseudo-labeling gain is distribution-specific rather than general.","tokens_in":1252,"feed_emoji":"📷","tokens_out":1483,"duration_ms":52334,"temperature":0.7,"pith_summary":"The paper aims to show that a lightweight face image quality assessment (FIQA) model can match a much heavier one when the teacher is improved by self-training on its own pseudo-labels before distillation. The teacher, a Swin-B, is trained on 27,686 labeled face images, then used to pseudo-label 400,000 unlabeled Internet face images; those pseudo-labels both refine the teacher and supervise a compact student. The resulting EdgeNeXt-based student has 1.18M parameters and 0.4949 GFLOPs, and reaches SRCC 0.9855 / PLCC 0.9914 on the ICCV 2025 VQualA FIQA test set, within 0.001 of the teacher on both metrics. The method took first place in that challenge. This matters because accurate FIQA is currently too computationally heavy for many mobile and real-time applications.","feed_headline":"1.18M-param student matches Swin-B teacher on face quality scoring","feed_subtitle":"Distillation with self-trained teacher keeps SRCC 0.9855 vs 0.9863 at ~100x lower FLOPs.","key_machinery":"The engine is an iterative pseudo-labeling and distillation loop. A teacher trained on labeled data (Eq. 1) produces pseudo-labels for unlabeled images (Eq. 2); those pseudo-labels are added to the labeled set and used to fine-tune the teacher (Eq. 3), yielding an enhanced teacher that pseudo-labels a fresh set of unlabeled images; the student is then trained with a loss that sums labeled supervision, initial-teacher pseudo-labels, and enhanced-teacher pseudo-labels (Eq. 4). The regression loss combines MSE with a Pearson correlation (PLCC) term (Eqs. 5–7), which keeps predictions both close to the labels and rank-consistent with human ratings.","core_discovery":"The central claim is that a two-stage pseudo-labeling loop makes knowledge distillation work far better for FIQA: the teacher is first trained on labeled data, then used to pseudo-label a pool of unlabeled faces; those pseudo-labeled samples are fed back into the teacher to produce an enhanced teacher, which pseudo-labels a second pool; the student is trained on labeled data plus both pseudo-labeled pools. Ablations show each added source of pseudo-labels improves the student (SRCC 0.9807 with labeled only, 0.9845 with the first pseudo-label set, 0.9855 with both), and the enhanced teacher slightly outperforms the original (SRCC 0.9867 vs 0.9863). The student lands at 0.9855 SRCC / 0.9914 PLCC with 1.18M parameters and 0.4949 GFLOPs, a 99% parameter reduction and roughly 100x fewer FLOPs than the teacher, while staying within 0.001 of the teacher's accuracy.","pith_inferences":["Because the test set is exclusively Flickr images while unlabeled data is general Internet faces, the student's edge may shrink on other distributions (e.g., synthetic distortions or demographic subgroups); a cross-dataset test would reveal how much of the gain is distribution-specific.","The self-training loop assumes the teacher's pseudo-labels are unbiased; if they are systematically wrong on certain face types, the student inherits the bias. A simple probe would be to compare student vs teacher errors on low-quality or under-represented faces.","The same two-stage pseudo-labeling recipe could transfer to other perceptual regression tasks (e.g., video quality, aesthetic scoring) where unlabeled data is cheap and the teacher is large."],"forward_implications":["A student with 1.18M parameters and 0.4949 GFLOPs can replace a Swin-B teacher in deployment without a meaningful quality drop, enabling FIQA on mobile or real-time pipelines.","Adding pseudo-labels from the enhanced teacher (second stage) improves the student further than using only the initial teacher's pseudo-labels, so the extra pseudo-labeling round pays off.","The teacher's own accuracy rises slightly after self-training on its own pseudo-labels, so unlabeled Internet faces can supplement limited labeled FIQA data.","At 288x288 inference the model uses only 0.3313 GFLOPs, which is competitive with the other top challenge entries on both accuracy and cost."],"supporting_citations":[{"why":"Introduces knowledge distillation, the base mechanism the student training relies on.","marker":"[15]"},{"why":"Noisy Student self-training shows iterative pseudo-labeling with an equal-or-larger student improves generalization, the template for the teacher enhancement loop.","marker":"[53]"},{"why":"Demonstrates billion-scale self-training with pseudo-labels, justifying the use of large unlabeled pools for FIQA.","marker":"[54]"},{"why":"E-VQA supplies the distillation-with-pseudo-labels recipe for quality assessment that this paper adapts to faces.","marker":"[46]"},{"why":"GFIQA-20K provides the perceptual FIQA dataset and generative-prior model that frame the labeled-data baseline.","marker":"[37]"},{"why":"The ICCV 2025 VQualA FIQA challenge paper defines the benchmark, data split, and evaluation protocol the results are measured on.","marker":"[25]"}],"fun_headline_variants":["Self-training loop distills tiny student for fast face quality scoring","Pseudo-label feedback loop yields 100x lighter FIQA student","Tiny student matches teacher on face quality after self-training","Self-training boosts distilled FIQA student to near-teacher accuracy","1.18M-param student rivals teacher on face quality scoring"],"cache_read_input_tokens":16128,"weakest_assumption_plain":"The load-bearing premise is that the pseudo-labels generated by the teacher on 400,000 unlabeled Internet face images are reliable enough and sufficiently aligned with the test distribution (Flickr-only images) that training on them improves both the enhanced teacher and the student.","fun_headline_variants_meta":{"raw":{"variants":["Self-training loop distills tiny student for fast face quality scoring","Pseudo-label feedback loop yields 100x lighter FIQA student","Tiny student matches teacher on face quality after self-training","Self-training boosts distilled FIQA student to near-teacher accuracy","1.18M-param student rivals teacher on face quality scoring"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000486,"raw_usage":{"total_tokens":2446,"prompt_tokens":1042,"completion_tokens":1404,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":1317}},"tokens_in":658,"tokens_out":1404,"duration_ms":11259,"temperature":1.0,"reasoning_tokens":1317,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:24:55.913032+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An experiment that would settle this: train the identical student on the labeled data alone, with no pseudo-labels, and compare its SRCC/PLCC on a fresh FIQA test set from a different source (e.g., synthetic distortions or a different demographic mix). If the gap between the distilled student and the teacher shrinks or reverses on that set, the pseudo-labeling gain is distribution-specific rather than general.","supporting_citations":[{"cited_title":"Going the extra mile in face image quality assess- ment: A novel database and model","cited_arxiv_id":null,"evidence_quote":"GFIQA-20K provides the perceptual FIQA dataset and generative-prior model that frame the labeled-data baseline."}],"review_version":1}