{"id":"1d1189ae-8ef4-433f-b5b1-89ec7eebc52e","arxiv_id":"2412.06248","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"RefSD combines 3D pose rendering with Stable Diffusion to pseudonymize people in images while preserving posture, and reports that models trained on its synthetic data can match or beat real-data training.","lead":"This paper presents RefSD, a pipeline that replaces people in photos with synthetic humans generated by Stable Diffusion, guided by 3D-rendered poses to preserve the original posture. It also introduces HumanGenAI, a framework for evaluating how well these pseudonymized images keep attributes and stay useful for training AI models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pose preservation, the core differentiator of RefSD, is never quantitatively validated; the SMPL→render→Canny→ControlNet chain may fail to preserve posture.","rationale":"The reader's weakest assumption identifies exactly this: the pipeline presumes the Canny edge map of the rendered SMPL avatar encodes the original pose accurately enough for ControlNet to preserve it. I agree that this is the single most load-bearing assumption because the entire novelty of RefSD over standard Stable Diffusion or GAN-based anonymization is posture preservation. Without quantitative evidence, the headline claim cannot be accepted as established. The paper's human perception studies (ϕA–ϕD) evaluate attribute fidelity, not pose fidelity; the utility experiments (ψA, ψB) evaluate downstream task performance, which can be high even with pose drift. Thus no existing experiment tests the core mechanism. The concern is not that the authors are dishonest; it is that a plausible failure mode—SMPL estimation error or ControlNet ignoring fine-grained edge details—would collapse the central differentiator. The proposed concrete test directly measures pose alignment and includes a control to isolate the contribution of the rendered edge map. This is a standard, inexpensive evaluation that the authors can perform. The reader's CONDITIONAL verdict is appropriate: the paper should be accepted only if such a quantitative pose-preservation evaluation is added, along with the other stated conditions (privacy metrics, statistical rigor). My review does not change that verdict; it reinforces it with a precise technical locus.","tokens_in":122,"tokens_out":3124,"duration_ms":44300,"concrete_test":"Run an off-the-shelf pose estimator (e.g., ViTPose or OpenPose) on the original and RefSD-generated images for a random sample of at least 500 instances from the OpenImages subset used in ψB. Compute PCK@0.1 and OKS between corresponding persons. If mean PCK@0.1 is below 0.9 or the average keypoint error is comparable to a baseline that simply inpaints without pose conditioning, the pose-preservation claim is refuted. Additionally, create a control condition where the Canny edge map is computed from the original image rather than the rendered avatar (or use dense pose as in DP2); if pose metrics do not improve with the rendered mesh, the 3D rendering step provides no measurable benefit, undermining the core contribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that RefSD preserves the original human pose rests on an unverified chain: 4DHuman's SMPL parameters {θi, βi} accurately capture the pose; the rendered avatar mi = R(θi, βi) matches that pose; Canny edges of mi provide sufficient structural guidance; and SDXL+ControlNet, conditioned on those edges (Eq. 3), maintains the pose in the final output. The paper provides only qualitative comparisons (Fig. 3, Fig. 9) and no quantitative pose metric (e.g., PCK, OKS, or keypoint distance). This is especially concerning because 4DHuman is known to struggle with occlusions, loose clothing, and unusual viewpoints, and because the rendered SMPL mesh lacks clothing, hair, and facial detail, so its Canny edges differ systematically from the original person's visual structure. Even if the edge map is correct, ControlNet is a soft guidance, not a hard constraint, so the generated human can drift in joint locations while still roughly following the edges. The utility results in Table 3 do not rescue the pose claim: a detector trained on synthetic data could achieve competitive mAP even if poses are altered, since the task only requires detecting persons, not matching original keypoints. Therefore, the load-bearing assumption of posture preservation is not just under-reported; it is the defining advantage over standard diffusion-based anonymization, and it remains unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RefSD, a pseudonymization pipeline that combines SMPL-based 3D rendering with Canny-edge-conditioned SDXL to replace human subjects in images while preserving posture and scene context, and introduces HumanGenAI, a two-part evaluation framework covering human perception of attribute fidelity and downstream utility in classification and detection. The experiments generate 73,130 synthetic images and collect 147,942 annotator ratings; the utility section reports that detectors trained on RefSD data match or exceed those trained on real data (Table 3: mAP@[.5:.95] 26.4 vs 25.3, and 30.8 with synthetic pretraining), and that combining synthetic with real data improves classification accuracy for several attributes (Table 2). The central technical claim is that the rendered SMPL avatar provides sufficient structural guidance for SDXL to preserve the original pose, but this is supported only by qualitative comparisons in Figures 3, 9, and 10, and no quantitative pose metric is reported anywhere in the paper.","tokens_in":20409,"tokens_out":5047,"duration_ms":49344,"significance":"If the posture-preservation claim were quantitatively validated, RefSD would be a useful contribution to privacy-oriented dataset release, and the HumanGenAI framework is a valuable attempt to standardize attribute-level evaluation of synthetic human generation. The large-scale human perception study, with 73,130 generated images and 147,942 graded annotations across prompt complexity, fine-grained translation, and full-body attributes, is a concrete empirical resource. However, the paper's defining advantage over standard diffusion-based anonymization is exactly the pose-preservation claim, and that claim currently lacks direct evidence; the utility benchmarks, while competently designed, do not by themselves establish pose fidelity because detection and classification do not require per-instance keypoint alignment. The significance is therefore conditional: with added quantitative pose metrics and statistical rigor on the utility tables, the contribution would be solid, but as written the central claim remains unverified.","major_comments":[{"comment":"The paper's core claim of posture-preserving pseudonymization rests on the pipeline in Section 3.3: 4DHuman SMPL parameter extraction, rendering, Canny edge extraction, and ControlNet-conditioned SDXL generation (Eq. 3). However, no quantitative pose metric is reported anywhere in the paper or supplementary material. Figures 3, 9, and 10 provide only qualitative side-by-side comparisons with SD and DeepPrivacy2; the reader cannot determine whether joint locations, limb orientations, or body proportions are actually preserved, or how often the chain fails when SMPL estimation is inaccurate. The detection results in Table 3 do not rescue this claim, because person detection does not require matching original keypoints. Please add quantitative pose-preservation metrics on a held-out set (e.g., PCK, OKS, or mean keypoint distance between original and pseudonymized instances), ideally with per-stage errors along the chain (SMPL, render, final output), and report failure modes such as occlusion, loose clothing, and unusual viewpoints.","section":"Section 3.3, Eq. (3), Fig. 3"},{"comment":"The text states that 'incorporating synthetic data consistently enhances performance across all attributes' and that pre-training on synthetic then fine-tuning on real 'offers the most substantial gains.' This is contradicted by the table itself: for Age, ViT-Tiny S→R (55.7) is below R (57.0) and ViT-Base S→R (58.1) is below R (58.4); for Emotion, ViT-Tiny S→R (42.2) is only 0.7 points above R while S alone (39.6) is far below R; for ViT-Base Age, S+R (59.9) is only 1.5 points above R. All numbers are from single runs with no error bars or significance tests. Please either soften the 'consistent' claim, or provide multiple seeds and a paired significance test, so that the reported 0.5% to 11.1% differences can be evaluated against run-to-run variance.","section":"Table 2, Section 5.2 (ψA)"},{"comment":"The claimed 1.1-point mAP advantage of synthetic data over real data is based on a single training run with no error bars, no multiple seeds, and no statistical test. Detector training on 75,000 images is stochastic, and a 1.1 mAP difference can easily fall within run-to-run noise, especially on a validation set with only 722 Person and 227 Human Face instances out of a 600-class evaluation. The S→R gain of 5.5 mAP points is more substantial, but it still needs variance estimates before the conclusion that RefSD data 'consistently improves results without any negative impact.' Please report at least 2-3 seeds with mean and standard deviation, or a significance test, for both mAP@[.5:.95] and mAP@0.5.","section":"Table 3, Section 5.2 (ψB)"},{"comment":"The title and abstract claim 'Privacy Compliant' synthetic data, and the introduction invokes GDPR and CCPA requirements, but the paper contains no privacy evaluation: there is no re-identification attack, no membership-inference test, no face-identification metric, and no measure of residual personally identifiable information beyond the qualitative statement that PII regions are inpainted. Section 5.3 does acknowledge that pose and location may carry re-identification risk, which is a useful caveat, but it does not substantiate the compliance claim. Please add a privacy-leakage evaluation (e.g., face re-identification or attribute inference on the pseudonymized outputs) or explicitly scope the claim to 'privacy-preserving intent' rather than legal compliance.","section":"Title, Section 5.3"}],"minor_comments":[{"comment":"The notation [·]bi is introduced as 'cropping with bounding box bi' but the exact tensor indexing and resizing operation are not specified; please define the crop operation unambiguously and state how the mask ai and rendered avatar mi are aligned with the cropped region in Eq. (2).","section":"Section 3.3, Eq. (2)"},{"comment":"The text says the S→R setting 'further improved by 5.5%' and '6.6%' compared to real data alone; since the baseline mAP values are 25.3 and 32.2, the differences are 5.5 and 6.6 mAP points, not percentages. Please use 'mAP points' to avoid conflating absolute and relative improvements.","section":"Table 3, Section 5.2 (ψB)"},{"comment":"The reported Cronbach's alpha of 0.340 for the fine-grained attribute translation evaluation indicates low inter-annotator reliability, yet the insights drawn from these scores are presented as model limitations. Please discuss how this reliability level affects the conclusions, or report additional agreement statistics.","section":"Section 5.1, Fig. 7 (ϕC)"},{"comment":"The rendering block says synthetic avatars are rendered using 'various appearance textures,' but no details are given on how many textures are used, how they are sampled, or how they affect downstream conditioning; this is material for reproducibility.","section":"Section 3.2, Eq. (1)"},{"comment":"In the pseudocode, x'[cropi] is set using the full image x before cropping, while the rendered avatar mi and mask ai are also involved; please clarify that all three terms are cropped with the same bounding box and resized consistently before being passed to the generative model.","section":"Algorithm 1, line 10"},{"comment":"The 'footballer' example in Figure 1(b) is not described in the main text; either reference it explicitly in Section 3.3 or remove it to avoid confusion.","section":"Fig. 1(b)"}],"recommendation":"major_revision","confidential_remarks":"The paper would benefit from releasing code, exact prompts, and seeds for the 73,130 generated images, as the current supplementary provides templates but not full reproducibility. The central pose-preservation claim is the main risk; if the authors add quantitative pose metrics and tighten the statistical claims in Tables 2 and 3, the contribution could be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"RefSD is a legitimate new pipeline worth a serious look: it replaces humans in place using SMPL-rendered avatars as Canny-edge conditioning for SDXL+ControlNet, and it ships a genuinely large perception evaluation. The catch is that its defining claim—posture preservation—is shown only qualitatively, and the GDPR-compliance language outruns the evidence.\n\nWhat is genuinely new: the assembly of 4DHuman SMPL estimation, pyrender, Canny edges, and ControlNet for in-place pseudonymization. It is the diffusion-era successor to DeepPrivacy2, and the modular design is sensible. The real asset is the HumanGenAI evaluation: 147,942 human annotations across four perception studies, honestly analyzed (they report Cronbach's alpha 0.340 on fine-grained pairs, and failures like 'no beard' producing beards), plus utility experiments with the right baseline configurations (synthetic, real, S→R, S+R). The qualitative comparisons against plain SD and DP2 show a visible difference, and the S→R/S+R classification numbers mostly beat real-only training.\n\nSoft spots, in proportion. Largest: posture preservation is never measured. No PCK, OKS, or keypoint distances—only Figs. 3, 9, 10. The 4DHuman→render→Canny→ControlNet chain can drift joints while looking plausible, and a detection mAP gain of 1.1 (26.4 vs 25.3) does not test pose fidelity. The stress-test note is right that this is load-bearing and under-supported; I'd add that the mechanism is plausible and the qualitative evidence is fairly clear, so the fix is a metric, not a redesign. Second: privacy is asserted, not measured. Calling the outputs 'GDPR-compliant images' (Sec. 3.4) is an overclaim without a re-identification or face-matching evaluation, and Sec. 5.3's own admission that pose and location carry re-identification risk tensions with that language. Third, minor-to-moderate: utility tables have no error bars or significance tests, and one number (ViT-Tiny Age S→R 55.7 vs real 57.0) contradicts the 'consistently improves' sentence. No code is released, though all components are off-the-shelf.\n\nWho this is for: people working on privacy-preserving computer vision or synthetic human data. It deserves a serious referee: the design is new, the evaluation is substantial, and the gaps are quantifiable follow-ups rather than category errors. I'd send it out, asking for pose metrics, error bars, a bounded privacy/GDPR claim, and code.","headline":"RefSD is a sensible new assembly of SMPL rendering plus ControlNet for in-place pseudonymization, with a serious evaluation suite, but the posture-preservation and GDPR claims outrun the measurements.","tokens_in":20931,"tokens_out":6283,"would_cite":true,"duration_ms":59426,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RefSD claims that replacing people in photos with synthetic humans—guided by a rendered 3D pose—preserves posture and scene context while keeping the images as useful for training detectors and classifiers as the originals.","keywords":["image pseudonymization","privacy-preserving synthetic data","Stable Diffusion","posture preservation","SMPL body model","ControlNet","downstream utility evaluation","GDPR compliance"],"falsifier":"Run a keypoint estimator (or 4DHuman itself) on both the original and the pseudonymized versions of a large image set and measure the mean per-joint distance or percentage of correct keypoints between the two. If pseudonymized images are no closer to the original pose than images produced by SDXL without the rendered edge guidance, the central posture-preservation claim collapses; an ablation that switches off the ControlNet edge condition would isolate exactly how much of the pose fidelity comes from the rendering block.","tokens_in":19990,"feed_emoji":"🕵️","tokens_out":13668,"duration_ms":112426,"temperature":0.7,"pith_summary":"RefSD is a proposed answer to a regulatory squeeze: privacy laws such as GDPR and CCPA push dataset owners to pseudonymize the people in their images, while the usual fixes—blurring, masking, cropping—destroy the very information the images exist to provide. The paper argues that a person can be replaced in place by a synthetic human with a different face, age, ethnicity, or clothing while the original posture and scene context survive, so the edited image stays useful for training vision models. The evidence is two-sided: annotator studies of attribute fidelity, and utility experiments in which a detector trained on RefSD-pseudonymized OpenImages reaches mAP@[.5:.95] of 26.4, versus 25.3 for the same detector trained on real images, with classification accuracy on RAF-DB also rising when synthetic and real data are combined. If the claims hold, photographic datasets could be legally shared or commercially used under privacy regulations without paying the usual utility tax.","feed_headline":"Pseudonymized photos beat real data for training detectors","feed_subtitle":"Swaps identities while preserving pose; synthetic-plus-real training beats real-only by 5.5 mAP points.","key_machinery":"The load-bearing object is the rendered SMPL avatar used as a structural bridge between the two blocks. 4DHuman extracts per-subject pose and shape parameters $\\{\\theta_i, \\beta_i\\}$ from the original image, and a synthetic human is rendered from them; the Canny edge map of that render is then fed, via ControlNet (a conditioning mechanism that lets an edge map steer the diffusion output), into SDXL alongside a text prompt that specifies the new person's attributes. This way the diffusion process is free to change identity, age, clothing, and other appearance, but is geometrically pinned to the original posture. The reintegration step composites each generated crop back into the untouched background using feathered person masks and $\\alpha$ blending, and a Grounding DINO pass detects and inpaints any remaining personal information such as faces or license plates.","core_discovery":"The paper's central claim is that rendering a 3D body model of a person and feeding its edge map into a text-guided diffusion model produces pseudonymized images that are both identity-replacing and posture-preserving, and that these images retain enough information to train downstream models. In the RefSD pipeline, 4DHuman extracts SMPL pose and shape parameters from each subject, a featureless avatar is rendered and blended over the original person, and the avatar's Canny edge map is passed through ControlNet to condition SDXL, so the generated person is geometrically locked to the original pose while a text prompt freely changes appearance. The quantitative anchor of the utility claim is Table 3: a DINOv2-Adapter with Faster R-CNN detector trained on 75,000 pseudonymized images reaches mAP@[.5:.95] of 26.4 against 25.3 for the same detector trained on real images, and 30.8 when synthetic pretraining is followed by real-data fine-tuning. Table 2 reports consistent classification gains on RAF-DB, up to 11.1 points for gender with ViT-Base, when synthetic data is combined with or used to pretrain on real data.","pith_inferences":["The annotator data imply a ceiling on fine-grained prompt control: closely related ethnicities and subtle emotions were nearly indistinguishable (annotator consistency 0.340 for the translation test), so users should expect attribute control to follow SDXL's underlying biases rather than a neutral attribute axis.","A quantitative pose-fidelity metric, such as keypoint distance between original and pseudonymized images, is the missing piece that would turn the qualitative posture-preservation claim into a testable specification other pipelines could adopt.","The paper's own discussion concedes that pose and location survive pseudonymization and remain re-identification cues; going further, a stricter privacy regime would have to perturb them, which would directly trade away part of the measured detection utility."],"forward_implications":["Photographic datasets containing people could be pseudonymized for sharing or commercial use under GDPR- and CCPA-style rules without the utility loss that blurring or masking imposes.","A detector trained only on pseudonymized images can match or beat one trained on the originals: 26.4 versus 25.3 mAP@[.5:.95] on OpenImages.","Using pseudonymized data for pretraining and real data for fine-tuning lifts detection mAP to 30.8, a 5.5-point gain over real-only training.","Combining synthetic and real data consistently improves attribute classification on RAF-DB across emotion, age, gender, and ethnicity.","Because the pipeline is modular, future rendering and diffusion models could be swapped in while carrying the same posture-preserving, attribute-controllable pseudonymization forward."],"supporting_citations":[{"why":"Supplies the SMPL pose and shape parameters extracted from each person, grounding the rendered avatar in the original posture.","marker":"[7]"},{"why":"Provides the edge-conditioning mechanism that lets the rendered avatar's Canny map steer SDXL, the load-bearing step for posture preservation.","marker":"[56]"},{"why":"The SDXL diffusion model that generates the pseudonymized humans in the generative block.","marker":"[39]"},{"why":"The latent diffusion formulation that RefSD's generative block builds on for realistic image synthesis.","marker":"[45]"},{"why":"The GAN-based full-body anonymization baseline whose posture preservation and realism RefSD must outperform.","marker":"[13]"},{"why":"The OpenImages dataset whose 75,000 images ground the detection-utility result in Table 3.","marker":"[20]"},{"why":"The RAF-DB dataset whose labeled attributes ground the classification-utility result in Table 2.","marker":"[23]"},{"why":"Zero-shot segmentation used to detect and inpaint residual personal information beyond the human subjects.","marker":"[25]"}],"fun_headline_variants":["Pseudonymized photos beat real data for training detectors","Synthetic data with swapped identities outperforms real data in detection","RefSD: identity-swapped synthetic data boosts detector training","Pose-preserving pseudonymization beats real data for detectors","Combining synthetic and real data beats real-only for detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline assumes that the Canny edge map of a rendered SMPL avatar gives SDXL enough structural guidance to reproduce the original posture exactly, and that 4DHuman's SMPL extraction captured that posture correctly in the first place; the paper verifies this only with qualitative side-by-side images, never with a numeric pose-fidelity measurement.","fun_headline_variants_meta":{"raw":{"variants":["Pseudonymized photos beat real data for training detectors","Synthetic data with swapped identities outperforms real data in detection","RefSD: identity-swapped synthetic data boosts detector training","Pose-preserving pseudonymization beats real data for detectors","Combining synthetic and real data beats real-only for detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0012,"raw_usage":{"total_tokens":4961,"prompt_tokens":974,"completion_tokens":3987,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":3905}},"tokens_in":590,"tokens_out":3987,"duration_ms":29875,"temperature":1.0,"reasoning_tokens":3905,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:52:10.749142+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a keypoint estimator (or 4DHuman itself) on both the original and the pseudonymized versions of a large image set and measure the mean per-joint distance or percentage of correct keypoints between the two. If pseudonymized images are no closer to the original pose than images produced by SDXL without the rendered edge guidance, the central posture-preservation claim collapses; an ablation that switches off the ControlNet edge condition would isolate exactly how much of the pose fidelity comes from the rendering block.","supporting_citations":[{"cited_title":"Humans in 4D: Reconstructing and tracking humans with transformers","cited_arxiv_id":null,"evidence_quote":"Supplies the SMPL pose and shape parameters extracted from each person, grounding the rendered avatar in the original posture."},{"cited_title":"Adding conditional control to text-to-image diffusion models, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the edge-conditioning mechanism that lets the rendered avatar's Canny map steer SDXL, the load-bearing step for posture preservation."},{"cited_title":"Blattmann, Dominik Lorenz, Patrick Esser, and B","cited_arxiv_id":null,"evidence_quote":"The latent diffusion formulation that RefSD's generative block builds on for realistic image synthesis."},{"cited_title":"Deepprivacy2: To- wards realistic full-body anonymization","cited_arxiv_id":null,"evidence_quote":"The GAN-based full-body anonymization baseline whose posture preservation and realism RefSD must outperform."},{"cited_title":"The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale","cited_arxiv_id":null,"evidence_quote":"The OpenImages dataset whose 75,000 images ground the detection-utility result in Table 3."},{"cited_title":"Reliable crowd- sourcing and deep locality-preserving learning for expres- sion recognition in the wild","cited_arxiv_id":null,"evidence_quote":"The RAF-DB dataset whose labeled attributes ground the classification-utility result in Table 2."}],"review_version":1}