{"id":"da9f518f-6d42-4255-8d9b-84de778371cc","arxiv_id":"2505.22416","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A neural face-skinning model with segmentation-supervised per-vertex weights improves expression retargeting accuracy and editability across arbitrary face meshes.","lead":"This paper introduces a neural method that transfers facial expressions between 3D face meshes with different shapes and topologies, using predicted per-vertex skinning weights to localize a global expression code. The approach improves retargeting fidelity and provides editable FACS-based control, which could speed up facial animation workflows.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim assumes the ICT-supervised per-vertex skinning weights (Sec. 3.3) transfer reliably to unseen scales/topologies, yet the only cross-mesh evidence is qualitative and the acknowledged scale-mismatch failure (Fig.","rationale":"The reader's weakest assumption and my own analysis converge on the same point: the skinning encoder's per-vertex weights are supervised only on the ICT model and must transfer to arbitrary topologies and scales. The paper's own Limitations section and Figure 11 show that this transfer can fail under scale mismatch, which is not a minor edge case because the abstract and introduction emphasize 'regardless of underlying structures' and 'proportions that deviate from typical human faces.' The proposed scaled-ICT test is feasible because ground truth exists, and it would convert the qualitative observations of Figures 10 and 11 into a quantitative robustness curve. It would also isolate whether the skinning encoder, rather than the decoder's per-vertex features, is responsible for any cross-scale robustness. Given the good-faith reading of the paper, this concern does not require rejection; it supports keeping the CONDITIONAL verdict pending that verification, especially since no code or data are provided for an independent check.","tokens_in":14985,"tokens_out":7095,"duration_ms":92165,"concrete_test":"Generate scaled versions (e.g., 0.4x, 0.6x, 0.8x, 1.2x, 1.6x, 2.0x bounding-box scale) of held-out ICT test identities, keeping the same source expression meshes and using the correspondingly scaled expression mesh as ground truth. Test the full model and the w/o-SE ablation from Table 4, reporting per-segment MSE and the semantic agreement of predicted zSkin with the ground-truth segment labels. If the full model's advantage over w/o-SE shrinks or reverses as scale departs from the training distribution, and zSkin accuracy degrades, the skinning-transfer assumption is the bottleneck and the mesh-agnostic claim must be qualified to a scale band.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The mesh-agnostic claim — 'even for facial meshes with proportions that deviate from typical human faces' — depends on the skinning encoder producing semantically correct per-vertex localization weights ωSkin for arbitrary target meshes. The supervision for this is one-sided: the NLL loss (Eq. 2) is applied only to zSkin on ICT data, and zSkin is passed through an MLP to form ωSkin, so nothing directly couples ωSkin to the FACS displacement fields. On Multiface and stylized meshes, no segmentation labels or blendshape-coefficient ground truth are available (Eq. 9), so ωSkin is shaped only by reconstruction losses. The quantitative evaluations are self-retargeting on training-like identities (Table 2) and inverse rigging on ICT (Table 3), the latter being partly circular because zGE is trained to match ICT coefficients (Eq. 4) and then evaluated by multiplication with the same ICT basis (Eq. 7). Cross-mesh evidence on stylized meshes is qualitative (Fig. 10). The paper itself documents that scale-mismatched targets receive inaccurate skinning weights (Fig. 11), which is precisely the failure mode that would break the central claim. Therefore, transfer of the skinning weights is the load-bearing assumption, and it is not quantitatively verified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a facial expression retargeting method that combines a global, interpretable expression code (supervised to match FACS-based ICT blendshape coefficients) with per-vertex skinning weights predicted by a skinning encoder from segmentation labels. The skinning weights localize the global code, and a per-vertex decoder outputs displacements. The model is trained on synthetic ICT data and Multiface scans using losses for vertex/normal/Jacobian reconstruction, encoder regularization, blendshape projection and reconstruction, and a segmentation NLL loss. Experiments compare against NFR and ZPT on self-retargeting and inverse rigging, with ablations for the proposed components.","tokens_in":15261,"tokens_out":4239,"duration_ms":45404,"significance":"The core idea—localizing a global, user-interpretable expression code through learned per-vertex skinning weights—is interesting and potentially useful, as it could offer a better trade-off between intuitive control and local detail than existing global or local methods. The architecture and losses are clearly specified, and the ablation study in Tables 4–5 provides evidence that the skinning encoder and the blendshape-based losses contribute to the reported performance. If the mesh-agnostic generalization claim were quantitatively validated on truly unseen topologies and scales, this would be a valuable contribution to facial animation retargeting. However, as the paper stands, the central generalization claim is supported mainly by self-retargeting experiments and qualitative stylized-mesh results, with a partly circular inverse-rigging evaluation, so the evidence is not yet fully convincing.","major_comments":[{"comment":"","section":"Section 4.2, Table 3, Eqs. (4) and (7)"},{"comment":"","section":"Section 4, paragraph on ZPT training"},{"comment":"","section":"Sections 3.3 and 5, Figure 11"}],"minor_comments":[{"comment":"","section":"Section 4.4 heading"},{"comment":"","section":"Figure 5 caption"},{"comment":"","section":"Equation (2)"},{"comment":"","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a sensible architecture and the ablations are informative, but the evaluation of the central mesh-agnostic claim is not yet convincing due to the partly circular inverse-rigging metric on ICT and the biased ZPT baseline on Multiface. The authors should be asked to provide an independent cross-mesh quantitative evaluation and to re-examine the fairness of the baseline comparison. If these issues are addressed, the paper could be suitable for publication at a good venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take on arXiv:2505.22416. The core idea is genuinely new: instead of using a single global expression code as NFR and ZPT do, they predict per-vertex skinning weights on the target mesh (supervised by face-region segmentation labels), and use those weights to localize the global code before the decoder. That simple mechanism buys them measurable improvements in self-retargeting on ICT and Multiface, and the ablation makes a convincing case that the skinning encoder—not just the extra losses—is what helps.\n\nWhat the paper does well: the architecture and training losses are clearly specified, the ablation isolates the components (skinning encoder, L_BP, L_BR), and the qualitative results on unseen stylized meshes (Mery, Malcolm, Bonnie, Morphy) show the method handling very different topologies and proportions without the visible artifacts of NFR (shifted eyebrows, rescaled meshes) or ZPT's failures. The paper is also honest about limitations: it acknowledges scale-mismatched meshes can get inaccurate skinning weights (Fig. 11), and suggests FLAME for head pose/neck movement.\n\nThe soft spots are in the evaluation rather than the method. The inverse-rigging numbers in Table 3 are largely self-consistency: z_GE is trained to match ICT expression coefficients (Eq. 4) and L_BP directly penalizes what z_GE·B reconstructs, so measuring how well z_GE·B reconstructs the source is in part measuring the encoder's own training objective. It still shows the code is more interpretable than NFR's, but it shouldn't be read as an external benchmark. Second, the ZPT baseline on Multiface is trained using the authors' own expression encoder's predictions as pseudo ground truth for pose codes. That makes the comparison to ZPT less clean, because ZPT is effectively wrapping the proposed method's encoder. Third, there's no code or data release, so independent reproduction is limited. The segmentation NLL loss (Eq. 2) is also written in a way that looks like binary cross-entropy per vertex, but z_Skin is a per-vertex L-vector; the notation needs cleanup.\n\nThe stress-test's worry that the skinning-weight transfer is unverified is only partly right. The quantitative Multiface self-retargeting (different mesh topology from ICT) does exercise the transfer across topologies, and the stylized-mesh results, though qualitative, are consistent with the claim. The failure mode is scale mismatch, and the paper says so. So the central claim holds, with a narrower range than the abstract implies.\n\nBottom line: this is a solid, incremental contribution to facial animation retargeting, worth a serious referee. The evaluator should push for code release and a fairer ZPT baseline, and should descope the inverse-rigging claim from 'outperforms' to 'more interpretable and more aligned with the FACS grammar.' But the idea is real and the paper is competently built.\n\nRecommendation: send it to peer review.","headline":"Skinning-weighted localization of a global FACS code is a real improvement over NFR/ZPT, but the paper's quantitative claims are softer than they look due to a circular inverse-rigging metric and a pseudo-GT ZPT baseline.","tokens_in":15798,"tokens_out":6078,"would_cite":true,"duration_ms":67094,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Skinning weights localize facial expression cloning to any target mesh.","keywords":["facial expression retargeting","mesh-agnostic","skinning weights","FACS blendshapes","global-local deformation","DiffusionNet","inverse rigging","expression cloning"],"falsifier":"Run a trained model on a target mesh that is a uniform two-fold upscale of a training identity. If the predicted skinning weights no longer follow semantic regions and the deformations move the wrong parts of the face, for instance the mouth action deforming the forehead, the transfer assumption fails. A second check is to compute predicted skinning weights on the same face represented with two different triangulations; if the weights diverge strongly for corresponding regions, they depend on mesh discretization rather than on facial semantics.","tokens_in":14781,"feed_emoji":"🎭","tokens_out":6635,"duration_ms":61870,"temperature":0.7,"pith_summary":"This paper claims that deep facial expression retargeting fails where a single global latent code flattens local detail, while local deformation methods sacrifice overall control. The proposed fix is to localize a global expression code at every vertex of the target mesh using predicted skinning weights, trained with indirect supervision from facial segmentation labels. The code is anchored to FACS-based blendshapes, so the result remains interpretable and editable. If the claim holds, arbitrary face meshes, including stylized characters with proportions far from human faces, can be animated with both regional fidelity and global controllability.","feed_headline":"Skinning weights make facial expression cloning fit any mesh","feed_subtitle":"Localizing a global expression code preserves local detail while keeping results editable via FACS blendshapes.","key_machinery":"The skinning encoder and skinning block carry the argument. A shared CNN + DiffusionNet frontend produces per-vertex features; the skinning encoder emits a per-vertex skinning feature, and a small MLP maps it through softmax to skinning weights $\\omega_{\\text{Skin}} \\in \\mathbb{R}^{128}$. The Hadamard product of those weights with the global expression code $z_{GE}$ yields a localized expression code $z_{LE}$ for every vertex, which conditions an eight-layer MLP decoder that outputs per-vertex displacements. The FACS-blendshape losses $L_{BP}$ and $L_{BR}$ are what tie this mechanism back to an interpretable, editable expression code.","core_discovery":"The central discovery is that multiplying a global expression embedding by a per-vertex skinning vector converts a single global code into a spatially localized conditioning signal, and that the needed skinning vectors can be learned without manual rigs by supervising with segmentation labels. The model predicts per-vertex skinning weights from the target mesh geometry, localizes the global latent, and decodes per-vertex displacements, so retargeting requires no correspondences and no fixed mesh structure. Two blendshape-based losses tie the latent to the FACS grammar of the ICT model, giving the user an interpretable, editable expression code. The paper reports lower error than its baselines on expression fidelity and inverse rigging, and successful cloning on stylized meshes outside the training distribution.","pith_inferences":["The same per-vertex latent-localization trick should generalize to full-body or hand retargeting, where joint skinning weights already provide a natural prior.","The reliance on hand-labeled segments could eventually be removed by clustering blendshape influence regions in an unsupervised way, making the whole training pipeline automatic.","A stress test the paper does not run is feeding meshes with non-uniform scale changes or disconnected components to see whether predicted skinning weights stay semantically consistent.","The architecture suggests a generic recipe, global code multiplied by a per-element localization mask, for any deformation-transfer task that needs local fidelity without losing global control."],"forward_implications":["Expression cloning transfers to meshes with unseen topology and proportions, including stylized characters, because per-vertex skinning weights are learned from geometry rather than from a fixed template.","The FACS-aligned latent code makes inverse rigging and interactive editing possible: users can modify blendshape coefficients instead of raw vertex positions.","Because the decoder predicts per-vertex displacements directly, the method sidesteps Poisson-integration artifacts that appear in per-triangle Jacobian approaches.","Substituting a more expressive blendshape basis, such as one that includes head pose and neck motion, would extend the same pipeline to full-head animation."],"supporting_citations":[{"why":"Supplies the ICT-Facekit parametric model with 100 identity and 53 FACS-based expression blendshapes used for training and supervision.","marker":"[LBZ+]"},{"why":"Provides the Multiface real-scanned range-of-motion data used alongside synthetic ICT meshes to train the network.","marker":"[WZA+]"},{"why":"Defines the NFR baseline and the overall encoder/decoder design with CNN and DiffusionNet that this method extends with the skinning encoder.","marker":"[QSA+]"},{"why":"Provides the DiffusionNet backbone that gives the encoders mesh-agnostic surface features.","marker":"[SACO]"},{"why":"Supplies the facial muscle-group reference used to create the 20-segment segmentation maps that supervise the skinning encoder.","marker":"[Win]"},{"why":"Establishes the Facial Action Coding System that the expression latent is aligned to for interpretability and editing.","marker":"[EF]"},{"why":"Provides the ZPT baseline, a zero-shot pose-transfer method with per-vertex displacement prediction used as a comparison.","marker":"[WLL+]"},{"why":"Introduces Linear Blend Skinning, the joint-based deformation idea the paper adapts to localize expression codes.","marker":"[MLT]"}],"fun_headline_variants":["Skinning weights localize expression codes for any face mesh","Learn skinning from geometry to clone faces on unseen meshes","Mesh-agnostic expression cloning via learned per-vertex skinning","No rigs needed: predict skinning weights for facial retargeting","Localized latent + FACS editing for any mesh expression cloning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that soft skinning weights learned from segmentation labels on the ICT model transfer reliably to arbitrary target meshes with different topologies and proportions; if the weights are inaccurate, the localized expression code produces poor deformations on unseen shapes.","fun_headline_variants_meta":{"raw":{"variants":["Skinning weights localize expression codes for any face mesh","Learn skinning from geometry to clone faces on unseen meshes","Mesh-agnostic expression cloning via learned per-vertex skinning","No rigs needed: predict skinning weights for facial retargeting","Localized latent + FACS editing for any mesh expression cloning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000695,"raw_usage":{"total_tokens":3124,"prompt_tokens":910,"completion_tokens":2214,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":2126}},"tokens_in":526,"tokens_out":2214,"duration_ms":17076,"temperature":1.0,"reasoning_tokens":2126,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:07:24.133104+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a trained model on a target mesh that is a uniform two-fold upscale of a training identity. If the predicted skinning weights no longer follow semantic regions and the deformations move the wrong parts of the face, for instance the mouth action deforming the forehead, the transfer assumption fails. A second check is to compute predicted skinning weights on the same face represented with two different triangulations; if the weights diverge strongly for corresponding regions, they depend on mesh discretization rather than on facial semantics.","supporting_citations":[],"review_version":1}