{"id":"5bba12bd-e2c3-42ac-afe9-59116eb85060","arxiv_id":"2501.12319","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper introduces BW(iqa), a metric that multiplies face-matcher similarity scores by image quality scores to evaluate demorphing outputs.","lead":"This paper proposes a new metric, biometrically cross-weighted IQA, for evaluating reference-free face demorphing methods, combining biometric similarity with SSIM and PSNR. It argues that existing metrics like TMR and RA can be trivially gamed by a solution that replicates the morph, and benchmarks three demorphing methods on six datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"BW lacks any term enforcing output distinctness, so the claimed penalty on trivial demorphing is absent and untested; the trivial morphe-copying solution may score highly.","rationale":"The reader's weakest_assumption correctly identifies the missing distinctness penalty, and I agree. The paper makes two valid points: TMR/RA are vulnerable to trivial morph-replication, and SSIM/PSNR ignore identity. However, the proposed BW metric is just a biometric-weighted IQA; it does not encode the demorphing-specific constraint that the two outputs be distinct and both match different constituents. The claim that BW 'penalizes trivial demorphing outputs' is therefore not entailed by the metric's definition. An empirical demonstration could have saved the paper, but Table 2 omits the trivial solution's BW score, and the results are said to be 'supplied by the original authors' despite the claimed common protocol, further weakening the benchmark. Since the paper's main contribution is the metric and its claimed property, and that property is untested and not guaranteed, the central claim fails. I recommend REJECT. Credit where due: the analysis of existing metrics' flaws is reasonable and the experiments are extensive in scale, but they do not test the specific claim.","tokens_in":9864,"tokens_out":6044,"duration_ms":57979,"concrete_test":"Compute BW(SSIM) and BW(PSNR) for the trivial demorpher o1=o2=x on all six test datasets using AdaFace and ArcFace with the same protocol as Table 2, and compare against the reported averages. If the trivial output's BW score is not lower than every method's average—or, worse, if it ranks first—the metric demonstrably fails to penalize trivial demorphing. As an analytic companion check, re-derive Eq. (9) to confirm that B(o1,o2) appears nowhere in the expression.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central claim is that BW(iqa) penalizes trivial demorphing outputs (Section 1, contribution 2). Equation (9) defines BW(iqa)=E_x max( Σ_i B(o_i,i_i)·iqa(o_i,i_i), Σ_i B(o_i,i_j)·iqa(o_i,i_j) ) with j=i%2+1. This expression contains no term involving B(o1,o2) and no enforcement of the distinctness condition Eq. (3) (B(o1,o2)<θ). Consequently, a demorpher that returns the input morph for both outputs (o1=o2=x) is not explicitly penalized. Since a morph by construction has high biometric similarity to both constituents (B(x,i_k)>τ), the products B(x,i_k)·iqa(x,i_k) can be substantial, potentially exceeding the BW scores of the three benchmarked methods (Table 2, averages 0.11-0.23 for BW(SSIM)). The paper never reports the BW score of this trivial solution, leaving the claim unverified. Moreover, because max(·) selects the better of the two output-to-GT pairings, an output set containing one very good match and one poor match can receive a score similar to a set with two mediocre matches, further decoupling BW from the requirement that both reconstructed images be useful and distinct. Thus the metric does not by design penalize the failure mode the paper itself identifies, and the central contribution is not supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses evaluation of reference-free face demorphing. It argues that existing metrics are flawed: TMR and RA can be gamed by trivial morph replication, while SSIM/PSNR ignore identity information. It proposes a new metric, biometrically cross-weighted IQA (BW), defined in Eq. (9), which multiplies biometric similarity scores with IQA values over possible pairings of outputs and ground truths. The authors benchmark three demorphing methods on six datasets with AdaFace and ArcFace, reporting TMR, RA, PSNR/SSIM, and BW scores, and conclude that BW better reflects visual quality.","tokens_in":10105,"tokens_out":4666,"duration_ms":48587,"significance":"The paper identifies a real problem: reference-free demorphing evaluation lacks a consensus metric, and Table 3 convincingly shows that a trivial solution achieves perfect TMR on all six datasets. If a metric truly penalized morph replication while combining biometric and image-quality information, it would be a useful contribution. The paper also usefully demonstrates that SSIM/PSNR can rank distorted outputs incorrectly. However, the proposed BW metric does not contain any explicit mechanism to penalize the trivial solution, and the experimental protocol is internally inconsistent as reported. As a result, the central claim is not supported by the presented evidence.","major_comments":[{"comment":"The paper claims that BW 'penalizes trivial demorphing outputs,' but Eq. (9) contains no term involving B(o1,o2) and does not enforce the distinctness condition in Eq. (3). If a method outputs o1=o2=x, the two sums inside max(·) become identical, and BW reduces to E_x[ B(x,i1)·iqa(x,i1)+B(x,i2)·iqa(x,i2) ]. Since a morph is constructed so that B(x,ik)>τ, these products can be substantial. The paper never reports the BW score of this trivial solution in Table 2 or elsewhere, so the central claim is neither guaranteed by construction nor verified empirically.","section":"§4.2, Eq. (9); §1 contribution 2"},{"comment":"Section 5 says the methods are trained under a common protocol, but Table 2 states that the scores are 'supplied by the original authors.' If the three methods were evaluated under their originally published training protocols (scenarios 1 and 3 as described in §2), the columns are not comparable; if they were re-trained under the common protocol, the scores cannot be supplied by the original authors. Either way, the cross-method comparison that underpins the efficacy claims for BW is not supported by the reported numbers.","section":"§5 vs. Table 2 caption"},{"comment":"The max over the two possible pairings in Eq. (7) and Eq. (9) means that one well-reconstructed output and one poor output can receive a score similar to two moderately reconstructed outputs. This is inconsistent with the problem statement in Eq. (4), which requires both outputs to align with their corresponding ground-truth images. Thus BW can be inflated by partially successful decompositions, and it does not measure whether the demorpher satisfies the two-output recovery condition.","section":"§4.1, Eq. (7)"}],"minor_comments":[{"comment":"The header column labeled 'Facial Demorphing [33]' refers to [1] in the text; [33] is SDeMorph, so the citation is inconsistent.","section":"Table 2 header"},{"comment":"The sentence 'Note that the test morphs include those generated using includes both conventional landmark-based techniques...' contains a typo: 'using includes' should be 'using both' or similar.","section":"§5, last sentence"},{"comment":"The caption says TMR is 'averaged over two subjects,' but the text in §7 indicates averaging over two face matchers; please clarify which quantity is averaged.","section":"Table 3 caption"},{"comment":"The impostor score computation described in the Evaluation Criterion paragraph is not used in Eq. (7), Eq. (8), or Eq. (9); either it should be removed or its role in the proposed metric should be explained.","section":"§4.1"}],"recommendation":"reject","confidential_remarks":"The manuscript has a central conceptual mismatch: the proposed metric does not penalize the trivial morph-replication solution that the paper itself identifies as the key failure of TMR/RA. In addition, the benchmark protocol is internally inconsistent because Table 2 reports scores supplied by the original authors while Section 5 claims a common training protocol. The demonstration that TMR is vacuous is useful, but the proposed metric would need a substantially revised formulation with an explicit distinctness constraint and a re-run benchmark to support the central claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is worth reading if you work on face morphing evaluation, but the central claim does not survive contact with its own equation. The authors correctly point out that TMR and RA are trivially satisfied by a demorpher that just reproduces the morph, since the morph matches both constituent identities. Table 3 makes that case cleanly. That is a real contribution. The proposed fix, biometrically cross-weighted IQA (BW), multiplies biometric similarity by SSIM/PSNR and takes the max over two output-to-ground-truth pairings. The idea of combining a biometric term with an image-quality term is natural and not crazy.\n\nThe problem is that the metric contains no term enforcing that the two outputs are distinct, despite Eq. (3) in the paper stating that a valid demorphing must satisfy B(o1,o2) < theta. A method that returns the morph for both outputs would, by construction, have high biometric similarity to both constituents, so the products B(·)·iqa(·) could be substantial, possibly higher than the benchmarked methods' scores. The paper never reports the BW score of this trivial solution, so the central claim that BW \"penalizes trivial demorphing outputs\" is unverified. This is a load-bearing gap, not a nuance.\n\nThe benchmark also has a protocol problem. Table 2 says the scores were \"supplied by the original authors,\" yet Section 5 claims the methods were retrained under a common protocol. Those two statements cannot both be true in a way that supports the metric comparison. Either the scores are independent, in which case the protocol claim is false, or they are self-reported, in which case the comparison is not a level playing field. The paper needs to say which.\n\nMinor issue: the max over pairings can reward an output set with one excellent reconstruction and one garbage one, since it only takes the better pairing. And there are no confidence intervals or significance tests across the three methods, so the ranking differences are suggestive at best.\n\nIf the authors add a distinctness penalty (or at least report the trivial baseline's BW score) and clarify the protocol, the metric could be useful. As is, the paper overstates what its metric does. I would send it to peer review for major revision; a serious referee should push on these points. The critique of TMR/RA alone is worth publishing somewhere.\n\nRecommendation: engage, but require the trivial-baseline experiment and a protocol clarification before acceptance.","headline":"Correctly identifies a real evaluation gap in face demorphing, but the proposed metric does not penalize the trivial solution it claims to fix, and the benchmark protocol is murky.","tokens_in":10612,"tokens_out":3124,"would_cite":false,"duration_ms":29334,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes a new metric, biometrically cross-weighted IQA, that scores reference-free face demorphing by multiplying face-matcher similarity with image quality and selecting the correct output pairing.","keywords":["face morphing","reference-free demorphing","evaluation metric","biometric similarity","image quality assessment","true match rate","morph replication","face recognition"],"falsifier":"Run a demorpher that outputs the same constituent image twice, say $o_1 = o_2 = i_1$, evaluate BW(SSIM) with any face matcher, and check whether its score is comparable to the genuine demorphers reported in Table 2; if it is, the metric does not penalize trivial outputs.","tokens_in":9647,"feed_emoji":"📏","tokens_out":6186,"duration_ms":56890,"temperature":0.7,"pith_summary":"Reference-free face demorphing tries to recover the two constituent face images from a single morphed image, but the field has no agreed way to score the recovered images. This paper argues that the two metric families currently used are both broken for this task: biometric rates such as True Match Rate and Restoration Accuracy give a perfect score to a trivial demorpher that simply returns the morph unchanged, while pixel-space SSIM and PSNR ignore identity altogether. To replace them, the paper introduces biometrically cross-weighted IQA, which multiplies each candidate output's image quality score by its face-matcher similarity to the ground-truth constituents and takes the best pairing. The paper benchmarks three demorphing methods on six datasets with two face matchers and reports that the new metric ranks Identity Preserving Decomposition first, a ranking that agrees with visual inspection of the outputs.","feed_headline":"Demorphing metric fuses biometric match with image quality","feed_subtitle":"Existing scores give a perfect result to a demorpher that just copies the morph; the new BW metric does not.","key_machinery":"The load-bearing object is the biometrically cross-weighted IQA score defined in Equation (9) of the paper. Given a morph $x$, outputs $o_1, o_2$, and ground truths $i_1, i_2$, the metric computes biometric similarity $B(o,i)$ with a face matcher and image quality $\\text{iqa}(o,i)$ with an IQA function such as SSIM or PSNR, multiplies the two per candidate pair, and takes the maximum over the two possible bijections between outputs and ground truths. The max operation is what handles the fact that demorpher outputs are unordered, and the multiplicative weighting is what forces a high score to require both identity preservation and image fidelity.","core_discovery":"The central claim is that a demorphing metric should be a product of two factors: the biometric match score between a demorphed output and a ground-truth constituent image, and the image quality of that same output measured by an IQA function. For unordered outputs, the metric computes both possible output-to-ground-truth pairings and keeps the larger weighted sum, then averages over all morphs. This single formula, $\\text{BW}(\\text{iqa}) = \\mathbb{E}_{x \\in X} \\max\\left(\\sum_{i\\in\\{1,2\\}} B(o_i,i_i)\\cdot\\text{iqa}(o_i,i_i), \\sum_{i\\in\\{1,2\\}, j=i\\%2+1} B(o_i,i_j)\\cdot\\text{iqa}(o_i,i_j)\\right)$, is claimed to overcome the morph-replication loophole that makes TMR and RA uninformative and to correct the identity blindness of SSIM and PSNR. The paper's experiments support this by showing that a trivial demorpher reaches 100% TMR while the proposed metric separates the three methods in a way consistent with visual quality.","pith_inferences":["Editorial inference: the metric inherits the blind spots of both components; if the face matcher cannot distinguish two identities, the biometric weight will not penalize identity confusion, and if the IQA is insensitive to artifacts, a distorted output can still score well.","Editorial inference: the definition of BW(iqa) contains no term enforcing the paper's own Equation (3), the requirement that the two outputs $o_1$ and $o_2$ be dissimilar, so a demorpher that returns the same image for both outputs could still score high if that image matches one ground truth.","Editorial inference: the Table 2 benchmark numbers are described as supplied by the original authors rather than recomputed under the paper's common protocol, so the reported ranking of methods under the new metric may partly reflect the original papers' own training conditions.","Editorial inference: a natural testable extension is to add the distinctness penalty $B(o_1,o_2)<\\theta$ directly into the metric, or to generalize the formula to morphs created from more than two identities by summing over all possible output-to-identity assignments."],"forward_implications":["If the metric is adopted, demorphing papers will need to report a single number that can be compared across methods, datasets, and face matchers.","TMR and RA should be dropped as primary evidence, because a trivial demorpher that regurgitates the morph achieves a perfect score by construction.","Methods trained under different assumptions can be ranked under a common protocol using the same score, as the paper does for three open-source demorphing methods.","The metric extends straightforwardly to any face matcher and any IQA measure, so future demorphing methods can be tuned directly against it.","The benchmark result, with Identity Preserving Decomposition scoring highest on BW, gives a concrete reference point for subsequent work."],"supporting_citations":[{"why":"Supplies the diffusion-based demorphing method whose reported Restoration Accuracy the paper re-examines under the proposed metric.","marker":"[33]"},{"why":"Identity Preserving Decomposition is the method the proposed metric ranks first, so it anchors the validation of BW.","marker":"[35]"},{"why":"The Facial Demorphing GAN method is the third benchmark and the original user of TMR@10%FMR, the metric the paper shows is vulnerable to trivial solutions.","marker":"[1]"},{"why":"AdaFace is one of two face matchers used to compute biometric similarity scores in BW.","marker":"[17]"},{"why":"ArcFace is the second face matcher used to compute biometric similarity scores in BW.","marker":"[9]"},{"why":"AMSL supplies one of the morphed-face test datasets used to evaluate the three demorphing methods.","marker":"[21]"},{"why":"FRLL supplies the bonafide face images from which several test morph datasets are built.","marker":"[8]"},{"why":"SMDD supplies the synthetic training faces used to train the demorphing models under the common protocol.","marker":"[4]"}],"fun_headline_variants":["New metric beats demorphing evaluation loopholes","Biometric + quality: better demorphing metric","Demorphing metric that ignores morph copies","BW metric: fusing biometrics and IQA","Reference-free demorphing: new evaluation metric"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that multiplying biometric match scores by image-quality scores makes the trivial demorpher score poorly, but the formula contains no term requiring the two outputs to be distinct, so a demorpher that returns the same image twice could still receive a high score.","fun_headline_variants_meta":{"raw":{"variants":["New metric beats demorphing evaluation loopholes","Biometric + quality: better demorphing metric","Demorphing metric that ignores morph copies","BW metric: fusing biometrics and IQA","Reference-free demorphing: new evaluation metric"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000453,"raw_usage":{"total_tokens":2260,"prompt_tokens":905,"completion_tokens":1355,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":1283}},"tokens_in":521,"tokens_out":1355,"duration_ms":10487,"temperature":1.0,"reasoning_tokens":1283,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:15:41.216049+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a demorpher that outputs the same constituent image twice, say $o_1 = o_2 = i_1$, evaluate BW(SSIM) with any face matcher, and check whether its score is comparable to the genuine demorphers reported in Table 2; if it is, the metric does not penalize trivial outputs.","supporting_citations":[{"cited_title":"SDeMorph: Towards Better Facial De- morphing from Single Morph","cited_arxiv_id":null,"evidence_quote":"Supplies the diffusion-based demorphing method whose reported Restoration Accuracy the paper re-examines under the proposed metric."},{"cited_title":"Facial Demorphing via Iden- tity Preserving Image Decomposition","cited_arxiv_id":null,"evidence_quote":"Identity Preserving Decomposition is the method the proposed metric ranks first, so it anchors the validation of BW."},{"cited_title":"Facial De-morphing: Extracting Component Faces from a Single Morph","cited_arxiv_id":null,"evidence_quote":"The Facial Demorphing GAN method is the third benchmark and the original user of TMR@10%FMR, the metric the paper shows is vulnerable to trivial solutions."},{"cited_title":"Privacy-Friendly Synthetic Data for the Development of Face Morphing Attack Detectors","cited_arxiv_id":null,"evidence_quote":"SMDD supplies the synthetic training faces used to train the demorphing models under the common protocol."}],"review_version":1}