{"id":"85a547b4-c9d6-47ef-a85a-c85877f6c7c9","arxiv_id":"2412.11779","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Aligning face images before quality assessment systematically raises FIQA scores, and the effect grows as image conditions worsen.","lead":"This paper investigates whether face image quality scores change when faces are aligned using landmarks instead of simply cropped. Across three datasets, four quality assessment methods, and two detectors, aligned faces received consistently higher quality scores, with larger gaps in low-quality surveillance images.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'alignment' comparison is confounded with box scaling, margins, and resampling, so the central claim is not yet isolated; a matched-pair perturbation test is needed.","rationale":"The reader's verdict is CONDITIONAL with high confidence, and my stress-test agrees: the empirical observation is internally consistent and is supported by multiple datasets (LFW, IJB-B, SCFace), two detectors, and four FIQA methods. The aligned distributions are consistently higher and tighter than the cropped distributions, and the SCFace distance trend is plausible. However, the central claim is about a causal factor—alignment—and the paper's protocol in Section 3.3 does not isolate that factor. Because 'Cropped' and 'Aligned' are generated by different pipelines, the comparison bundles alignment together with box scaling, margins, face resolution, and resampling. This is the weakest load-bearing point: if the score gap persists when those factors are held constant and only alignment is perturbed, the conclusion stands; if it disappears, the paper's headline overstates what is shown. The concern is not that the data are fabricated or that the paper is internally contradictory; it is that the inference from observed score differences to 'sensitivity to alignment' is underdetermined. An explicit limitation in the Discussion ('the FIQA methods were trained using aligned face images') strengthens the possibility that the gap reflects distribution shift rather than alignment per se. The proposed test is straightforward and decisive: a matched-pair perturbation study on aligned images, with output size and interpolation fixed. If that test is run and supports the paper, the conditional acceptance can be upgraded; if not, the claim should be weakened to 'cropped images receive lower scores than aligned images under these pipelines.' Statistical testing and code release would further strengthen the paper, but they are secondary. Verdict: UNCHANGED—the reader's CONDITIONAL remains appropriate.","tokens_in":9927,"tokens_out":4232,"duration_ms":38835,"concrete_test":"On a random subset of LFW (e.g., 1,000 detections), start from the paper's aligned 112×112 image and create controlled variants that apply known misalignment in the aligned coordinate frame: rotations of ±5° and ±10°, scale perturbations of ±10%, and translations of ±12 px, keeping the output canvas and interpolation method identical. Measure SER-FIQ and DifFIQA scores on the unperturbed aligned image, each perturbed variant, and the original cropped image. If the perturbed variants reproduce most of the cropped-vs-aligned gap and scores degrade monotonically with perturbation magnitude, the alignment-sensitivity claim is supported; if not, the original gap is driven by box scaling, margins, or resampling rather than by alignment itself.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the abstract and conclusion, is that FIQA methods are sensitive to alignment and that 'misaligned face images – only cropped versions – produce lower quality scores than properly aligned ones.' For that claim to be established, the Cropped versus Aligned comparison must isolate alignment as the cause. Section 3.3 does not do this. Cropped images are produced by taking detector bounding boxes (MTCNN boxes scaled by 1.2, RetinaFace boxes used directly), while Aligned images are produced by a similarity transformation (Eqs. 1–3) computed on the original image with the ArcFace template. Consequently the two conditions differ not only in geometric alignment but also in output size and margins (MTCNN's 1.2 scaling vs direct boxes vs ArcFace template), in the face resolution within the frame, and in interpolation/resampling introduced by the warp. Any of these confounds could plausibly lower FIQA scores for the cropped images, especially since the evaluated FIQA models were trained on aligned images. The paper's own Discussion acknowledges the training-distribution bias, but that does not address the crop/margin/resampling confound. Without a matched control that varies alignment while holding scale, margins, and resampling fixed, the reported effect cannot be attributed specifically to alignment; it could be an artifact of the different preprocessing pipelines.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper empirically compares FIQA scores computed on 'Cropped' faces versus 'Aligned' faces across three datasets (LFW, IJB-B, SCFace), two detectors (MTCNN, RetinaFace), and four FIQA methods (SER-FIQ, FaceQAN, DifFIQA, SDD-FIQA). The authors report that aligned images consistently receive higher quality scores than cropped ones, with the gap larger for SCFace at longer distances, and conclude that FIQA methods are sensitive to alignment and that misaligned (cropped-only) faces yield lower quality scores.","tokens_in":10178,"tokens_out":2063,"duration_ms":21694,"significance":"If the central claim is established, the paper would be a useful cautionary note for the FIQA community: quality scores are not comparable across preprocessing pipelines unless alignment is standardized, and alignment errors could be an additional quality-degradation factor in surveillance scenarios. The paper covers a relevant combination of detectors and state-of-the-art FIQA models and presents results on three standard benchmarks. However, the current experimental design does not isolate alignment as the causal factor, so the main conclusion remains unsubstantiated as stated.","major_comments":[{"comment":"The 'Cropped' and 'Aligned' conditions differ in several properties beyond geometric alignment. For MTCNN, the bounding box is scaled by 1.2 before cropping, while RetinaFace boxes are used directly; the aligned images are produced by a similarity transformation (Eqs. 1–3) applied to the original image with the ArcFace template. As a result, the two sets differ in output size, face scale within the frame, margins, interpolation/resampling, and potentially resolution. Any of these confounds could drive the observed score gap, especially because the four evaluated FIQA models were trained on aligned images. The authors should compare conditions that vary only the alignment transformation while keeping scale, margins, and resampling fixed—for example, by aligning to the template and then cropping with identical margins, or by applying a fixed similarity transform with perturbed landmark targets to the same crop. Without such a matched-pair control, the paper's central claim that 'FIQA methods are sensitive to alignment' is not established.","section":"Section 3.3"},{"comment":"The results are presented as aggregated distributions and mean scores without any measure of uncertainty or statistical testing. The paper reports no standard deviations, confidence intervals, or per-pair comparisons, and does not state the number of images used for each distribution. Since the claim is a systematic score difference, the authors should report paired differences (cropped vs. aligned for the same identity and image), effect sizes, and significance tests (e.g., Wilcoxon signed-rank or bootstrap confidence intervals). This is particularly important because the qualitative reading of Fig. 4 and Fig. 5 may depend on kernel density smoothing choices, and Table 1 reports only means. Without this, the reader cannot judge whether the observed gaps are reliable or within sampling variability.","section":"Section 5, Figs. 4–5, Table 1"},{"comment":"The Discussion acknowledges that 'the FIQA methods were trained using aligned face images,' but this fact is also a potential full explanation for the observed gap: the tested quality models may simply assign lower scores to any input that differs from their training distribution, regardless of whether the difference is due to alignment or to the crop/margin/resampling confounds identified above. To support the paper's interpretation, the authors should include a control experiment that decouples alignment from the training distribution—for example, using a FIQA model trained on unaligned crops (if available) or evaluating a generic image-quality metric (e.g., BRISQUE or NIQE) that does not depend on aligned face training. The current wording in Section 6, 'misaligned face images – only cropped versions – produce lower quality scores than properly aligned ones,' overstates what the experiments can distinguish.","section":"Section 5.1 and Section 6"}],"minor_comments":[{"comment":"The paper uses 'misaligned' to refer to the cropped-only images, but a tightly cropped face is not necessarily misaligned; the term should be defined more precisely, e.g., as 'not geometrically normalized by a landmark-based transform.'","section":"Section 3.3"},{"comment":"The sample images and the FIQA scores printed below them are too small to be readable in the provided figure; the authors should enlarge the images or present the scores in a separate table for the samples.","section":"Fig. 3"},{"comment":"The text says 'the quality score distributions of aligned images are very similar, whether MTCNN or RetinaFace is used,' but no quantitative similarity measure is given; a simple two-sample test or distribution-distance value would support this statement.","section":"Section 5"},{"comment":"There are minor citation inconsistencies, e.g., 'GraFIQs' is listed as [14] in the background but the style of the reference list is not uniform; also, the citation to the SCFace database [10] is a journal article while others are conference papers, which is fine, but the caption of Table 1 should restate the N per cell (the paper says SCFace contains 15 images per subject, but the table reports averages with no count).","section":"References"},{"comment":"The sentence 'This much quality drop when using the cropped images still remains unexpected' is informal; it would be clearer to state the expected drop under a specific hypothesis and compare the observed magnitude to that expectation.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant practical question, but the main experiment has a design confound that is acknowledged only partially in the Discussion. The authors should be encouraged to add a matched-pair control and significance testing, as these are feasible within the scope of the manuscript. I would not recommend rejection because the observed effect is plausible and the datasets/models are relevant; the issue is that the causal attribution to 'alignment' is not yet isolated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something genuinely useful: it runs four FIQA methods over three datasets with two detectors and shows that cropped faces get consistently lower quality scores than aligned faces. The effect is large, consistent across methods, and grows in the surveillance-distance SCFace data. As a descriptive finding — \"preprocessing pipelines change FIQA scores\" — this is solid and worth knowing. The figures and tables are clear, and the authors are honest that all four FIQA models were trained on aligned images, which likely biases them against crops.\n\nThe core problem is that the abstract and conclusion claim this is about alignment sensitivity, but Section 3.3 does not isolate alignment as the causal factor. Cropped images come from detector bounding boxes, with MTCNN boxes scaled by 1.2 and RetinaFace boxes used directly. Aligned images come from a similarity transform applied to the original image. So the two conditions differ in scale, margins, face resolution within the frame, and resampling, not just geometric alignment. Any of those could lower the scores for the cropped condition, especially given the models' training distribution. The paper's own Discussion acknowledges the training bias but never addresses the crop/margin/resampling confound.\n\nThere are also no error bars or significance tests. The distributions look separable, but with 13k LFW images that's not needed to believe the raw difference; it matters for the finer claims, like which FIQA method is most sensitive.\n\nThe fix is straightforward: a matched-pair perturbation test that warps already-cropped faces by controlled rotations or landmark jitter while holding scale and margins fixed, or a comparison where crops are resized to the aligned template's dimensions. That would separate alignment from the other preprocessing differences. The authors do cite relevant prior work, including ARoFace for alignment robustness, and the citation pattern looks fair.\n\nThis is a workshop- or conference-level empirical study, not a finished claim. The descriptive result deserves to be cited with caution; the attribution to alignment should be softened until the confounds are controlled. I would send it to peer review, but with a strong request for a controlled experiment, significance testing, and ideally code or exact preprocessing details. A serious referee could get this into publishable shape.","headline":"Useful empirical pattern, but the headline attribution to alignment is confounded by differences in scale, margins, and resampling; worth a careful revision.","tokens_in":10685,"tokens_out":1460,"would_cite":true,"duration_ms":16578,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that face image quality assessment methods are sensitive to face alignment: cropped-only images receive systematically lower quality scores than landmark-aligned images, with the gap widening under challenging real-world…","keywords":["face alignment","face image quality assessment","FIQA","MTCNN","RetinaFace","SER-FIQ","SCFace","surveillance face recognition"],"falsifier":"A controlled experiment that fixes the crop scale, margins, and interpolation for both versions and varies only the landmark-based rotation and translation would settle the claim: if quality scores do not decrease when the face is geometrically rotated or translated relative to the template, then alignment itself is not the cause of the observed score drop.","tokens_in":9746,"feed_emoji":"📷","tokens_out":3829,"duration_ms":34814,"temperature":0.7,"pith_summary":"This paper asks whether the preprocessing step of face alignment changes the quality score a face image receives from a face image quality assessment (FIQA) method. The authors compare two versions of each detected face, one simply cropped from the detector's bounding box and one geometrically aligned using facial landmarks, across three datasets and four FIQA methods. They report that aligned faces consistently score higher and with less variability than cropped faces, and that the difference grows in low-quality surveillance images. If correct, FIQA evaluations that skip or vary alignment are not measuring image quality alone; they are also measuring alignment-induced degradation, so alignment must be controlled for scores to be comparable.","feed_headline":"Misaligned face images score far lower on quality checks","feed_subtitle":"Four FIQA methods rank aligned faces higher across three datasets; the gap grows for distant surveillance shots.","key_machinery":"The central object is the similarity transformation $T$ that maps five detected facial landmarks (eye corners, nose tip, mouth corners) onto the fixed ArcFace template, combining scaling, rotation, and translation while minimizing the sum of squared landmark distances. This transformation is what separates the 'Aligned' images from the 'Cropped' images in every comparison. The argument's force comes from pairing this transformation with four FIQA scorers and comparing the resulting score distributions and means, treating alignment as the independent variable.","core_discovery":"The paper establishes that FIQA methods are alignment-sensitive: converting a detected face into a landmark-aligned image via a similarity transformation to the ArcFace template, instead of using the raw detector crop, yields higher and more consistent quality scores. This pattern holds across SER-FIQ, FaceQAN, DifFIQA, and SDD-FIQA on LFW and IJB-B, and it becomes more pronounced with distance on the surveillance dataset SCFace, where SER-FIQ mean scores at 4.2m roughly double or more after alignment (e.g., from 0.187 to 0.474 for MTCNN and from 0.070 to 0.426 for RetinaFace). The paper interprets these results as showing that misaligned face images produce lower quality scores, and it suggests the likely reason is that FIQA methods are trained on aligned images and therefore penalize inputs that deviate from that distribution.","pith_inferences":["Because the cropped and aligned versions in this study also differ in scale, margins, interpolation, and resolution (MTCNN boxes are scaled by 1.2, RetinaFace boxes are used directly, and alignment is applied to the original image), the measured gap is an upper bound on the pure alignment effect; a controlled study holding crop dimensions fixed while varying only landmark-based rotation and transl","The same sensitivity likely carries into FIQA-driven face recognition pipelines: if quality scores are used to discard low-quality faces before recognition, alignment errors could cause recognizable faces to be discarded as low quality.","A practical takeaway not stated by the authors is that standardizing the alignment template before scoring could reduce FIQA variance on low-quality surveillance data at near-zero computational cost relative to the recognition step.","Future FIQA benchmarking could treat alignment as a protocol parameter, reporting scores for both cropped and aligned versions so that robustness to preprocessing becomes a measurable property."],"forward_implications":["FIQA scores from different studies are not directly comparable unless the alignment step is specified and controlled.","Quality gaps attributed to capture distance in surveillance scenarios may be partly caused by alignment failure rather than by resolution alone.","Deployment pipelines that use FIQA to filter or rank faces should align faces before scoring, or they will systematically undervalue unaligned faces.","FIQA method design should consider robustness to alignment as a separate axis, since training on aligned images can create an unintended sensitivity to preprocessing.","Benchmark conclusions about which FIQA method performs best could change if the preprocessing protocol were held fixed across methods."],"supporting_citations":[{"why":"Supplies SER-FIQ, one of the four FIQA methods whose scores are compared between cropped and aligned faces.","marker":"[24]"},{"why":"Supplies FaceQAN, another FIQA method used to measure alignment sensitivity.","marker":"[2]"},{"why":"Supplies DifFIQA, another FIQA method used in the distribution and distance analyses.","marker":"[3]"},{"why":"Supplies SDD-FIQA, the FIQA method that shows the least sensitivity to alignment in the results.","marker":"[18]"},{"why":"Supplies MTCNN, the detector whose bounding boxes and landmarks generate one set of cropped and aligned images.","marker":"[29]"},{"why":"Supplies RetinaFace, the detector generating the second set of cropped and aligned images.","marker":"[5]"},{"why":"Supplies the ArcFace landmark template and the face recognition model underlying the alignment transformation and SER-FIQ.","marker":"[6]"},{"why":"Supplies the LFW dataset used for the quality-score distribution analysis under relatively normal real-world conditions.","marker":"[13]"},{"why":"Supplies the IJB-B dataset used to observe alignment sensitivity under more challenging unconstrained conditions.","marker":"[26]"},{"why":"Supplies the SCFace surveillance dataset whose distance-based protocol shows the widening alignment gap.","marker":"[10]"}],"fun_headline_variants":["Face alignment boosts image quality scores across FIQA methods","Misalignment drops face quality scores, especially at distance","Alignment raises quality scores; gap widens at surveillance distances","FIQA scores favor aligned faces, more so in real-life conditions","Aligning faces improves quality scores across all tested FIQA methods"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The two compared pipelines differ in more than alignment alone: MTCNN crops are scaled by 1.2, RetinaFace crops use raw bounding boxes, and alignment is applied to the original image, so scale, margins, interpolation, and resolution also differ between 'Cropped' and 'Aligned'; if those confounds rather than landmark geometry drive the score gap, the central claim is not established.","fun_headline_variants_meta":{"raw":{"variants":["Face alignment boosts image quality scores across FIQA methods","Misalignment drops face quality scores, especially at distance","Alignment raises quality scores; gap widens at surveillance distances","FIQA scores favor aligned faces, more so in real-life conditions","Aligning faces improves quality scores across all tested FIQA methods"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000678,"raw_usage":{"total_tokens":3096,"prompt_tokens":974,"completion_tokens":2122,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":2040}},"tokens_in":590,"tokens_out":2122,"duration_ms":15074,"temperature":1.0,"reasoning_tokens":2040,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:35:06.412246+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled experiment that fixes the crop scale, margins, and interpolation for both versions and varies only the landmark-based rotation and translation would settle the claim: if quality scores do not decrease when the face is geometrically rotated or translated relative to the template, then alignment itself is not the cause of the observed score drop.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies SER-FIQ, one of the four FIQA methods whose scores are compared between cropped and aligned faces."},{"cited_title":"In: 2022 26th International Conference on Pattern Recognition (ICPR)","cited_arxiv_id":null,"evidence_quote":"Supplies FaceQAN, another FIQA method used to measure alignment sensitivity."},{"cited_title":"In: 2023 IEEE International Joint Con- ference on Biometrics (IJCB)","cited_arxiv_id":null,"evidence_quote":"Supplies DifFIQA, another FIQA method used in the distribution and distance analyses."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies SDD-FIQA, the FIQA method that shows the least sensitivity to alignment in the results."},{"cited_title":"IEEE Signal Processing Letters 23(10), 1499–1503 (2016)","cited_arxiv_id":null,"evidence_quote":"Supplies MTCNN, the detector whose bounding boxes and landmarks generate one set of cropped and aligned images."},{"cited_title":"In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies RetinaFace, the detector generating the second set of cropped and aligned images."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the ArcFace landmark template and the face recognition model underlying the alignment transformation and SER-FIQ."},{"cited_title":"In: Workshop on faces in’Real-Life’Images: detection, alignment, and recognition (2008)","cited_arxiv_id":null,"evidence_quote":"Supplies the LFW dataset used for the quality-score distribution analysis under relatively normal real-world conditions."},{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops","cited_arxiv_id":null,"evidence_quote":"Supplies the IJB-B dataset used to observe alignment sensitivity under more challenging unconstrained conditions."},{"cited_title":"Mul- timedia Tools and Applications 51, 863–879 (2011)","cited_arxiv_id":null,"evidence_quote":"Supplies the SCFace surveillance dataset whose distance-based protocol shows the widening alignment gap."}],"review_version":1}