{"id":"acd396aa-0c0f-430d-8c79-2d3e744b34da","arxiv_id":"2412.08976","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"FLT improves identity preservation in landmark-conditioned character animation by combining reference face shape with driving expressions in 3D and re-rendering landmarks.","lead":"A simple preprocessing trick, Facial Landmarks Transformation (FLT), re-fits driving facial landmarks to a reference identity using a 3D Morphable Model before feeding them to a pose-guided character animation model. The paper reports modest gains in face-similarity metrics on two datasets, but the evidence is thin without error bars or stronger baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FLT's identity gains may stem from suppressing driving expressions; the paper never measures expression fidelity, so the claim of improved pose-guided animation is unverified.","rationale":"The reader's verdict is CONDITIONAL, and my concern reinforces the need for additional evaluation rather than overturning the verdict. The reader identified the 3DMM shape/expression separation as the weakest assumption; my concern is a specific failure mode of that assumption: the paper measures identity similarity but never verifies that the transformed landmarks preserve driving expressions. This is load-bearing because the method's purpose is pose/expression-driven animation with preserved identity—losing expression would make the method ineffective even if identity scores improve. The reader's rationale focuses on missing error bars and modest gains; while those are valid, the most technically central gap is the absence of any expression-preservation metric. A concrete expression-fidelity test would settle whether FLT actually delivers both identity and expression, and could be required as a condition for acceptance. The verdict remains CONDITIONAL because the paper could be made acceptable with that additional experiment and statistical reporting; no rejection is warranted solely on the current evidence.","tokens_in":6722,"tokens_out":6667,"duration_ms":73186,"concrete_test":"Run the same shuffled-pair evaluation with an additional expression metric: for each generated frame and its corresponding driving frame, extract expression coefficients (e.g., via the same 3DMM fitting used in FLT) or facial Action Unit intensities, and compute the mean absolute error or cosine similarity between generated and driving expressions for FLT versus the raw-landmark baseline. If FLT's expression error is significantly larger than baseline (paired test over videos, with multiple generation seeds), the identity gains are obtained at the expense of expression fidelity and the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation (2) constructs Mesh_trans using only the reference shape coefficients S_ref and the driving expression coefficients E_drive, then re-renders with the driving pose and reference texture before re-detecting landmarks. The core claim that FLT improves identity preservation while retaining driving expressions depends entirely on the re-rendered face and landmark detector faithfully encoding E_drive. Yet the evaluation (Table 1) reports only identity metrics—ArcFace cosine similarity and FID—and never measures whether the generated expressions match the driving video. A method that averages away or discards expression content would trivially increase identity similarity to the reference (since expression variation reduces average similarity), while failing the actual task of pose/expression-driven animation. The observed gains are tiny (e.g., +0.004 on UBC Fashion with AnimateAnyone, +0.023 on TikTok with ControlNeXt), consistent with an expression-smoothing artifact rather than a genuine identity-preserving improvement. Without an expression-fidelity metric, the central claim—that FLT improves pose-guided character animation—is not established; the paper's own limitations section acknowledges that rapid motion and occlusion challenge landmark accuracy, which is exactly when expression loss would be most severe.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Facial Landmarks Transformation (FLT), a training-free, plug-and-play preprocessing step for pose-guided image-to-video character animation. FLT takes 2D facial landmarks from a reference image and a driving image, fits a 3D Morphable Model to both, and constructs a transformed mesh (Eq. 2) that uses the reference shape coefficients and the driving expression coefficients. This mesh is re-rendered with the reference texture and the driving pose, and landmarks re-detected from the re-rendered face are fed into any landmark-conditioned animation model. The authors evaluate FLT with AnimateAnyone and ControlNeXt on TikTok and UBC Fashion datasets, using a dataset shuffle protocol and reporting ArcFace cosine similarity, similarity variance, and face FID. They report consistent but small improvements in identity similarity over the raw-landmark baselines.","tokens_in":6917,"tokens_out":3609,"duration_ms":37551,"significance":"If the method works as claimed, it would be an attractive drop-in module for improving identity consistency in diffusion-based character animation without retraining. The idea of using a 3DMM to merge reference shape with driving expression is reasonable and the paper demonstrates modest gains in ArcFace similarity across two models and two datasets. The authors also state that the code is open-sourced, which would aid reproducibility. However, the evaluation is incomplete in a load-bearing way: the paper never measures whether driving expressions and poses are preserved, and the identity metric is biased by construction because Eq. (2) forcibly imposes the reference geometry. The reported improvements are small and are not accompanied by error bars or significance tests, so the central claim of improved pose-guided animation is not yet established.","major_comments":[{"comment":"The evaluation reports only identity-oriented metrics (ArcFace cosine similarity, similarity variance, and face FID) and never measures expression or pose fidelity of the generated videos. Since Eq. (2) explicitly discards the driving shape coefficients and the method is claimed to preserve both reference identity and driving expressions, the absence of an expression/pose metric is critical. A baseline that averages away or suppresses expression variation could trivially increase ArcFace similarity to the reference, because expression variation typically reduces the average similarity to a neutral reference. The paper should add a quantitative expression-fidelity metric (e.g., distance between the generated and driving landmark coordinates or 3DMM expression coefficients) and, ideally, a perceptual user study, to support the claim that FLT retains the driving dynamics.","section":"§4.2, Table 1"},{"comment":"All results are point estimates from a single fixed-seed shuffle. Given the small gains (e.g., +0.023 on TikTok ControlNeXt, +0.004 on UBC AnimateAnyone), the improvements could lie within run-to-run or shuffle-seed variability. The paper should report results over at least 3–5 random shuffles, including standard deviations, and use an appropriate paired significance test (e.g., paired t-test or Wilcoxon signed-rank test) for the ArcFace similarity and FID differences. Without this, the claim that FLT 'significantly improves' identity preservation is unsupported.","section":"§4.2, Table 1"},{"comment":"The identity-preservation improvement is partly by construction: the method sets the 3DMM shape coefficients to the reference's, and the evaluation measures cosine similarity to the reference image. A control experiment is needed to determine whether the gains come from genuine expression-aware alignment or simply from biasing all generated faces toward the reference geometry. For example, replacing the driving landmarks with the reference landmarks (which destroys all expression/pose information) would likely also raise ArcFace similarity. The paper should compare FLT against such a control and against a variant that uses the same 3DMM recombination but skips the re-rendering/re-detection step, to isolate the contribution of each component.","section":"§3.3, Eq. (2) and §4.2"},{"comment":"The 'Target' baseline is not defined precisely enough for the reader to interpret the results. The text says it represents the ideal matched condition where the driving landmarks and the reference image come from the same video, but it is unclear whether this means running the same model on same-video landmarks without shuffle, or whether the reference image and driving frame are identical. Additionally, the FID metric is computed between the distribution of cropped reference faces and the distribution of cropped generated faces, which is a dataset-level measure and does not directly assess per-video identity preservation. Clarify the construction of 'Target' and state explicitly whether FID is paired or unpaired.","section":"§4.2"},{"comment":"The re-rendering and landmark re-detection step is under-specified. The authors state that the reference image is used as a texture source 'via a simple shader,' but do not describe the UV mapping, the rendering resolution, the face cropping procedure, or the landmark detector used for the re-rendered face. These details determine whether the re-detected landmarks faithfully encode the driving expression, especially under large pose variations. Without a precise description, the method is not reproducible, and it is difficult for a reader to assess when the pipeline might fail.","section":"§3.4"}],"minor_comments":[{"comment":"The phrase 'To address this limitation' appears twice in the abstract, and the sentence 'FLT converts 2D landmarks into a 3D face model, adjusts the 3D face model to align with the reference identity' is slightly redundant; consider tightening the wording.","section":"Abstract and §1"},{"comment":"The manuscript says 'we extract a 1-second clip' for each video but does not state the frame rate or the resulting number of frames used for evaluation; please specify these details for both TikTok and UBC Fashion.","section":"§4.1"},{"comment":"In the shuffle definition, there is a typo: 'ensuring that each Vi is is paired with' should read 'ensuring that each Vi is paired with.'","section":"§4.2"},{"comment":"The variance formula uses the population variance (division by N); please state whether this is intentional or whether the sample variance (N−1) is more appropriate for the reported statistics.","section":"§4.2, Eq. (3)"},{"comment":"The reference to 'Tab. 4.2' should be 'Table 1'; also, the text states 'A nimateAnyone' in Section 4.3 heading with an extra space.","section":"§4.3"},{"comment":"The paper claims 'we have open-sourced our approach' in the introduction, but no code repository or link is provided anywhere in the manuscript; please add the URL or remove the claim.","section":"§1 and §6"}],"recommendation":"major_revision","confidential_remarks":"This manuscript reads like a short conference submission (e.g., ICIP/ICASSP) rather than a full journal paper. The proposed method is simple and plausible, but the experimental evidence is too thin for a journal: no expression-fidelity metric, no statistical significance, and a single-seed shuffle protocol. The identity gain is partly a consequence of the construction in Eq. (2), so the paper needs either a strong control experiment or a clear re-framing of the contribution as 'landmark re-targeting toward the reference identity' rather than 'identity-preserving pose-guided animation.' I would also note that the manuscript claims open-source code but provides no link; that is a reproducibility concern that should be fixed before review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a clean, training-free preprocessing trick that probably does help identity preservation, but the paper doesn't prove the important half of its claim — that expressions survive the transformation. The evaluation only measures identity similarity to the reference, so the observed gains could partly be expression smoothing.\n\nWhat's new: the specific composition — fit a 3DMM to driving and reference landmarks, swap shape and expression coefficients, re-render with the driving pose and reference texture, re-detect landmarks, and feed those into an existing landmark-conditioned I2V model — is a sensible drop-in that nobody seems to have published. Each step is known, but the packaging is neat and it's genuinely plug-and-play. The paper also shows consistent ArcFace improvements across two models and two datasets, which is more than many such papers do. The method section is clear and the math is trivial.\n\nWhere it's soft: the evaluation has no error bars, no multiple seeds (one shuffle), no statistical tests. The gains are small: e.g., +0.023 on TikTok with ControlNeXt, +0.004 on UBC Fashion with AnimateAnyone. The 'Target' rows show the method closes only about a quarter of the gap to the matched-identity upper bound on TikTok, so the effect is modest. More importantly, there is no metric for expression fidelity. The whole point of a pose-guided animation model is to transfer the driving expression, and FLT could in principle be increasing identity similarity by averaging out or discarding expression content. The paper never checks whether the generated faces match the driving expressions. The stress-test worry about expression smoothing isn't refuted by anything in the paper. That's a genuine gap, not a nitpick.\n\nAlso, the code is promised but no link appears in the manuscript, which matters for a plug-and-play claim.\n\nWho it's for: people working on landmark-conditioned image-to-video generation who want a cheap way to reduce identity drift. It's not a breakthrough, but it's a useful tool if the expression-preservation question gets answered.\n\nRecommendation: send it to peer review, but push for an expression-fidelity metric (e.g., compare generated expressions to driving expressions with a face-recognition expression model or landmark distance) and error bars. The method is plausible and the writing is honest; it just needs the missing evidence.","headline":"Useful training-free preprocessing for landmark-conditioned animation, but the paper never checks whether expressions survive, so the headline claim is only half-proven.","tokens_in":7458,"tokens_out":2777,"would_cite":false,"duration_ms":26665,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A landmark preprocessing step prevents facial identity drift in pose-driven character animation.","keywords":["pose-guided character animation","image-to-video generation","facial landmarks transformation","3D morphable model","identity preservation","expression transfer","diffusion video generation","training-free plug-in"],"falsifier":"A direct test is to compare FLT against the target condition on the paper's own shuffle benchmark: if transformed landmarks do not move average ArcFace similarity closer to the target value than raw landmarks do, the central claim fails. A sharper version uses clips with large out-of-plane head rotation or occlusion, where the authors concede landmark accuracy degrades.","tokens_in":6524,"feed_emoji":"🎭","tokens_out":7118,"duration_ms":68071,"temperature":0.7,"pith_summary":"Pose-guided character animation models condition on facial landmarks from a driving video, but when the driver's face geometry differs from the reference character's, the generated face drifts toward the driver's identity. This paper proposes a training-free preprocessing step, Facial Landmarks Transformation (FLT), that replaces raw driving landmarks with transformed ones before they enter the animation model. FLT fits a 3D Morphable Model to the reference and driving faces, keeps the reference's shape coefficients, takes the driving's expression coefficients, re-renders the combined mesh with the reference texture under the driving pose, and re-detects landmarks from that render. If it works as claimed, any landmark-conditioned image-to-video model can gain identity fidelity at inference time with no retraining.","feed_headline":"Landmark preprocessor stops facial identity drift in animation","feed_subtitle":"Fusing a reference face's 3D shape with a driver's expressions improves pose-guided video generation without retraining.","key_machinery":"The load-bearing object is the identity-preserving reconstruction step expressed by Eq. 2: a 3D Morphable Model represents a face as the mean shape plus PCA shape components plus expression blendshapes, and FLT swaps the coefficients so the transformed mesh keeps the reference's shape and adopts the driving's expressions. Around this sits a four-stage pipeline: landmark extraction with a standard detector, 3DMM fitting, the coefficient fusion of Eq. 2, and a re-rendering plus re-detection stage in which the mesh is projected with the driving camera matrix, textured by the reference image, and re-annotated to obtain guiding landmarks. The re-render is what makes the method usable under pose variation: instead of projecting mesh vertices directly, FLT synthesizes a face image and lets a landmark detector find the final points.","core_discovery":"On the paper's own terms, the central discovery is that identity drift in pose-guided character animation can be corrected by operating on the conditioning landmarks rather than on the generation model. Given a reference image and a driving image, FLT extracts 2D landmarks from both, fits a 3D Morphable Model to each, and constructs a transformed mesh $\\mathrm{Mesh}_{\\mathrm{trans}} = \\bar{V} + \\sum_i S^{\\mathrm{ref}}_i V^S_i + \\sum_j E^{\\mathrm{drive}}_j V^E_j$, meaning the reference's shape PCA coefficients with the driving's expression blendshape coefficients. This mesh is projected with the driving camera matrix, textured with the reference image, rendered to a 2D face, and re-annotated with a landmark detector; those re-detected landmarks become the generation condition. The authors report that when FLT is plugged into AnimateAnyone and ControlNeXt on TikTok and UBC Fashion, average ArcFace similarity to the reference rises and FID drops relative to raw landmarks, approaching the upper-bound target setting where driver and reference come from the same video.","pith_inferences":["Beyond the paper's claims, the same coefficient-swap idea could be tested on full-body skeletons: keeping a reference character's body-shape coefficients while taking the driver's joint angles may reduce body-shape leakage in whole-character animation.","A second untested consequence is training-time use: if transformed landmarks are generated as augmentation during model training, the generation model may learn to be more robust to raw landmark mismatches, possibly making the inference-time transform unnecessary.","A third extension is to disentangle why similarity improves: measuring temporal smoothness and landmark-flow consistency of transformed versus raw landmarks would show whether the gain comes from identity alignment alone or from steadier conditioning signals."],"forward_implications":["Any landmark-conditioned pose-guided image-to-video model can adopt FLT at inference time without retraining, since it only replaces the conditioning landmarks.","Characters whose facial geometry differs strongly from the driver's will keep their own face shape, jawline, and eye structure instead of inheriting the driver's features.","The gap between the ideal same-video target condition and the mismatched-driver condition shrinks, because transformed landmarks approximate perfect alignment between driver and reference.","A single unified pipeline can serve both dancing-style dynamic videos and static-camera fashion videos, with larger reported gains on the dynamic TikTok set."],"supporting_citations":[{"why":"Pose-guided animation model into which FLT is plugged and against which identity preservation is measured.","marker":"[4]"},{"why":"Second integration target and baseline for verifying FLT's generality.","marker":"[5]"},{"why":"Supplies the 3DMM fitting framework, PCA shape model, and expression blendshapes used in Eq. 2.","marker":"[6]"},{"why":"TikTok dataset used for dynamic-motion evaluation with dancing and quick head movements.","marker":"[7]"},{"why":"UBC Fashion dataset used for static-camera evaluation with less dynamic facial variation.","marker":"[8]"},{"why":"Landmark detector used both to extract input landmarks and to re-detect landmarks from the rendered face.","marker":"[15]"},{"why":"DWPose used to obtain pose skeletons and facial landmarks for driving frames.","marker":"[16]"},{"why":"FID metric used to compare the generated-face distribution with the reference-face distribution.","marker":"[17]"},{"why":"ArcFace used to compute cosine-similarity and variance metrics for identity fidelity.","marker":"[18]"}],"fun_headline_variants":["Landmark transform fuses reference face with driver's expressions","No retraining: landmark fix keeps character identity stable","3D face alignment stops identity drift in pose-guided animation","Transform landmarks, not models, to preserve facial identity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach depends on the assumption that a 3D face model fitted from a small set of 2D landmark points can cleanly separate a person's facial shape from their expression, and that re-rendering the merged face and re-detecting landmarks preserves both sides; under fast motion or occlusion this separation and re-render can degrade.","fun_headline_variants_meta":{"raw":{"variants":["Landmark transform fuses reference face with driver's expressions","No retraining: landmark fix keeps character identity stable","3D face alignment stops identity drift in pose-guided animation","Transform landmarks, not models, to preserve facial identity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000823,"raw_usage":{"total_tokens":3598,"prompt_tokens":938,"completion_tokens":2660,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":2595}},"tokens_in":554,"tokens_out":2660,"duration_ms":18807,"temperature":1.0,"reasoning_tokens":2595,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:20:50.796798+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to compare FLT against the target condition on the paper's own shuffle benchmark: if transformed landmarks do not move average ArcFace similarity closer to the target value than raw landmarks do, the central claim fails. A sharper version uses clips with large out-of-plane head rotation or occlusion, where the authors concede landmark accuracy degrades.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Pose-guided animation model into which FLT is plugged and against which identity preservation is measured."},{"cited_title":"This enables the generation of transformed land- marks that preserve the reference image’s facial identity while adopting the driving face’s expressive dynamics","cited_arxiv_id":null,"evidence_quote":"Second integration target and baseline for verifying FLT's generality."},{"cited_title":"Pose-guided human animation from a sin- gle image in the wild,","cited_arxiv_id":null,"evidence_quote":"Supplies the 3DMM fitting framework, PCA shape model, and expression blendshapes used in Eq. 2."},{"cited_title":"Align your latents: High-resolution video syn- thesis with latent diffusion models,","cited_arxiv_id":null,"evidence_quote":"TikTok dataset used for dynamic-motion evaluation with dancing and quick head movements."},{"cited_title":"Magicpose: Re- alistic human poses and facial expressions retargeting with identity-aware diffusion,","cited_arxiv_id":null,"evidence_quote":"UBC Fashion dataset used for static-camera evaluation with less dynamic facial variation."},{"cited_title":"Accurate 3d face recon- struction with weakly-supervised learning: From single image to image set,","cited_arxiv_id":null,"evidence_quote":"Landmark detector used both to extract input landmarks and to re-detect landmarks from the rendered face."},{"cited_title":"3d face morphable models","cited_arxiv_id":null,"evidence_quote":"DWPose used to obtain pose skeletons and facial landmarks for driving frames."},{"cited_title":"Deep video portraits,","cited_arxiv_id":null,"evidence_quote":"FID metric used to compare the generated-face distribution with the reference-face distribution."},{"cited_title":"Dreampose: Fashion video synthesis with stable diffusion,","cited_arxiv_id":null,"evidence_quote":"ArcFace used to compute cosine-similarity and variance metrics for identity fidelity."}],"review_version":1}