{"id":"a65fdee7-2f9f-4f98-9507-6659e715c613","arxiv_id":"2411.14205","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"HumanCalibrator detects absent and redundant body parts in AI-generated human photos and repairs them while preserving the rest of the image.","lead":"This paper introduces a new task, Fine-grained Human-body Abnormality Detection, for finding and repairing body parts in AI-generated images that do not match real human anatomy. It also presents a framework called HumanCalibrator and two datasets, claiming much higher detection accuracy than powerful vision-language models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bounding-box accuracy on AIGC Human-Aware 1K is not demonstrated because the evaluation metric is self-referential.","rationale":"The reader's weakest assumption is exactly the repair-quality-as-proxy for localization on AIGC Human-Aware 1K, and the strongest claim is the high accuracy in Table 1 that this proxy underwrites. I agree that this is the load-bearing concern: the dataset statistics in Table S3 confirm only 649 absent and 158 redundant samples, and the framework's components (GroundingDINO, StableDiffusion-inpainting, AHD) are not released, so the headline numbers cannot be independently verified without annotations. I am not raising the lack of ablations or the modest repair metrics as decisive because those do not threaten the central claim as directly. My concrete check is deliberately smaller than full re-annotation: 150 samples targeted at the reported accuracy would suffice to calibrate the self-referential metric. I recommend the verdict remain CONDITIONAL: accept the FHAD benchmark and framework idea as a contribution, but require the conditional on either releasing the AIGC Human-Aware 1K bounding-box annotations, providing box-IoU evaluation, or clearly reframing the paper's headline as 'repair-guided proxy accuracy' rather than localization accuracy.","tokens_in":20070,"tokens_out":2121,"duration_ms":16760,"concrete_test":"Re-annotate 100-200 samples from AIGC Human-Aware 1K with tight, carefully-defined ground-truth bounding boxes for each abnormal part (with double-review for ambiguity, as already used in dataset construction). Re-run HumanCalibrator and measure box-level intersection-over-union at IoU>=0.5 and IoU>=0.7 per category, plus report the accuracy/FDR of the repaired-image proxy on that subset. If the proxy and the box-IoU agreement is high (e.g., >10 percentage points of each other), the concern is resolved. If IoU@0.5 accuracy is substantially below the reported 80.69%/58.57%, Table 1 overstates localization accuracy.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central accuracy claim (80.69% absent / 58.57% redundant, Table 1) rests on evaluating detection localization on AIGC Human-Aware 1K, where bounding boxes are never annotated. Section 5.1 states that the accuracy of the bounding box location is assessed by repair quality, and Figure 3 explicitly says the location is ambiguous. The procedure appears to be: run HumanCalibrator; run the same or associated inpainting model on the detected region; if the repair is visually/quantitatively acceptable, the detected box is counted as correct. This is circular in the sense that a detector that proposes boxes whose contents are easy to re-generate (e.g., a tiny, generic, or background-like region) can achieve high repair quality and be counted as accurate even if the proposed box misses the actual abnormality. Conversely, a box that correctly locates the abnormality but where inpainting produces poor results would be counted as inaccurate. Thus the Table 1 accuracy numbers may measure the coupling between the detector and the inpainting model rather than the correctness of the localization itself. Independent evidence that HumanCalibrator locates abnormalities correctly is limited: on COCO Human-Aware Val, boxes are synthetic and a 76.91% accuracy is reported, but that dataset contains only absent abnormalities, so it is not a direct validation of AIGC-domain localization for either absent or redundant cases. Without per-box annotations on AIGC Human-Aware 1K, the strongest claim that the framework can 'pinpoint' abnormalities is not supported by Table 1.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces the Fine-grained Human-body Abnormality Detection (FHAD) task and proposes HumanCalibrator, a framework that detects and repairs abnormal human body parts in AIGC images and videos. The authors construct two datasets: COCO Human-Aware Val, with synthetically masked body parts, and AIGC Human-Aware 1K, with manually labeled abnormality types but no bounding-box annotations. The method has two detection branches: an Absent Human-body Detector (AHD), a fine-tuned VLM trained on masked COCO images, and a redundant-part detector that compares original body-part regions with inpainted versions via a grounding model. Detected regions are repaired using a diffusion inpainting model. The main reported results are 80.69% average absent-detection accuracy and 58.57% redundant-detection accuracy on AIGC Human-Aware 1K, together with repair-quality and visual-consistency metrics.","tokens_in":20382,"tokens_out":5899,"duration_ms":56235,"significance":"The paper identifies a genuinely under-studied and practically important failure mode of AIGC human image generation, and its evidence that strong VLMs such as GPT-4o perform near random guessing on this task is a useful negative result. The proposed datasets, especially the filtered AIGC Human-Aware 1K, have potential value as benchmarks. The framework is architecturally sensible, and the image, video, and cross-model case studies are suggestive. However, the central detection-accuracy claim for the AIGC domain currently rests on a self-referential evaluation in which repair quality is used to validate detection localization. If that evaluation is corrected with human-annotated boxes or a human study, the contribution would be substantially strengthened; as presented, the accuracy numbers in Table 1 should be read as measuring detector-inpainter coupling rather than localization quality.","major_comments":[{"comment":"The evaluation of detection localization on AIGC Human-Aware 1K is circular. The caption of Figure 3 and Section 5.1 state that, because abnormal-region locations are ambiguous, the authors do not annotate bounding boxes and instead evaluate bounding-box accuracy by repair quality. However, the repair is generated by the same inpainting model R used in the redundant detector (Eq. 8) and in the absent-detection loop (Eq. 9). A predicted box whose contents are easy to regenerate can therefore be counted as correct even if it does not cover the actual abnormality, while a correct box that the inpainter fills poorly would be counted as wrong. Table 1 thus does not demonstrate localization accuracy on AIGC Human-Aware 1K. I ask for human-annotated boxes (or a human study judging predicted boxes against manually marked regions) on this dataset, or a re-framing of the reported numbers as repair-quality scores rather than detection accuracy.","section":"Section 5.1 / Figure 3 caption / Table 1"},{"comment":"The per-category sample sizes in Table 1 are small and unreported, and no confidence intervals or error bars are given. For example, the redundant rows contain values such as 33.33% for leg, 83.33% for ear, and 66.67% for foot; given the 158 redundant instances reported in Table S3, these percentages likely correspond to only a few examples per class. With denominators this small, the per-category ranking of methods is not statistically meaningful, and even the aggregate 58.57% redundant accuracy needs an uncertainty estimate. Please report exact per-class counts and, where possible, binomial confidence intervals.","section":"Table 1"},{"comment":"The redundant-part detector depends on a free threshold tau in Eq. (8), but the paper does not specify its value, how it was selected, or how sensitive the Table 1 results are to it. Moreover, the expression 'G(R(pg_i, bg_i), pg_i) < tau' is not formally defined as a semantic-difference quantity; the comparison needs a precise grounding-confidence or similarity formulation. Without this information the redundant-detection accuracy is not reproducible and could reflect threshold tuning rather than the method itself.","section":"Eq. (8) / Section 4.2"},{"comment":"The repair-quality evidence in Table 2 is weak: the reported improvements are 22.59 to 22.77 in Human Concept Score, 41.87 to 41.97 in CLIP Score, and 26.36 to 26.42 in Human CLIP Score, with no variance or significance testing. These differences are within the noise one would expect from CLIP embedding similarity, so the conclusion that HumanCalibrator improves repair quality is not established by these metrics. The FID comparison against a pose-conditioned full-image regeneration (98.86 vs. 16.55) is also not an apples-to-apples comparison with a fine-grained repair method, since full-image regeneration is expected to have higher FID by construction.","section":"Table 2 / Section 5.1"}],"minor_comments":[{"comment":"The title 'Is this Generated Person Existed in Real-world?' and the abstract phrase 'the realistic of generated human photos' contain grammatical errors that should be corrected.","section":"Title / Abstract"},{"comment":"Figure 6 contains a stray '图表标题' ('chart title' in Chinese) placeholder that appears to be a leftover editing artifact and should be removed.","section":"Figure 6"},{"comment":"Please clarify whether the HumanCalibrator results in Table 2 include the 2x super-resolution step; the text says no SR is applied in comparisons, but the pipeline description includes SR as a component, leaving readers unsure which configuration is evaluated.","section":"Appendix A.2"},{"comment":"The definition of L as 'the length of the concatenated instruction Ia and the perception and position of the current absent body part' is awkward and should be rephrased; the superscript notation in the autoregressive probability is also difficult to parse and should be simplified.","section":"Eq. (5)"},{"comment":"The paper says 'we highly recommend reading this section' about Appendix D; this informal phrasing, combined with the fact that the annotation protocol is only in the supplement, suggests the essential dataset-construction details should be summarized in the main text.","section":"Section 3 / Appendix D"},{"comment":"The failure cases shown in Figure S5(b) are not discussed in the main text; a brief failure analysis would help readers calibrate when the method is likely to produce incorrect abnormality identification or inaccurate localization.","section":"Conclusion / Figure S5(b)"}],"recommendation":"major_revision","confidential_remarks":"The dataset and task formulation are useful contributions, and the negative result for strong VLMs is likely to interest the community. The main obstacle is the circular localization evaluation on AIGC Human-Aware 1K; if the authors add human-annotated boxes or a human localization study, the paper could become publishable. I recommend major revision rather than reject because the framework and datasets retain value and the evaluation gap is addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper introduces a useful new task—fine-grained human-body abnormality detection (FHAD)—and two datasets: a synthetic COCO-based one and a hand-labeled AIGC set drawn from VidProM. The task is real, and the paper shows convincingly that strong VLMs (GPT-4o, InternVL2) are near random at it. The absent-part detector trained with a body-part correlation mask strategy is a clever, low-cost idea, and the case studies show plausible repair outputs. That's the good stuff.\n\nThe soft spot is the evaluation of localization on the main AIGC dataset. The paper deliberately doesn't annotate bounding boxes because the locations are ambiguous (Figure 3 caption), and instead judges box accuracy by repair quality. But the same inpainting model that proposes redundant regions is then used to repair them and 'verify' correctness. That's circular: a detector that proposes an easy-to-inpaint region will score well even if it missed the actual abnormality, and a correct box that the inpainter mishandles will score poorly. So the 80.69% / 58.57% numbers in Table 1 don't actually demonstrate accurate localization. The stress-test note is right about this. The paper is transparent about the choice, which is good, but the headline claim is overreaching.\n\nThere are other softer issues: no error bars, small per-category sample sizes (redundant leg is 33.33%—likely 3 examples), free parameters like tau and the box expansion ratio, and the COCO val only covers absent abnormalities so redundant detection isn't independently validated. The HumanRefiner comparison isn't a faithful reproduction—they use a pose-conditioned regeneration, not the actual method. No code or data is released, which makes reproducibility hard. The authors note the predefined-class limitation in the conclusion.\n\nI'd still send this to a serious referee. The task and datasets are new and likely to be used by others; the method is a reasonable first solver; and the negative result on VLMs is a useful data point. But I'd expect heavy revision: add box annotations on at least a subset or a human study for localization, report error bars, and release code/data. Without that, the accuracy claims are just not substantiated.","headline":"Useful new task and datasets, but the headline localization accuracy is not demonstrated because the evaluation is self-referential.","tokens_in":20907,"tokens_out":3050,"would_cite":false,"duration_ms":28514,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HumanCalibrator detects and repairs impossible body parts in AI-generated people, reporting 80.69% average accuracy on absent abnormalities and 58.57% on redundant ones.","keywords":["FHAD","abnormal human body detection","AIGC","vision-language model","diffusion inpainting","body-part correlation","human image repair","text-to-image evaluation"],"falsifier":"Hand-annotate the abnormal regions in AIGC Human-Aware 1K with bounding boxes and compute IoU between HumanCalibrator's predicted boxes and the human boxes; if the repair-quality-based accuracy stays high while IoU is near chance, the central detection claim is not supported, whereas high IoU would corroborate the paper's evaluation.","tokens_in":19874,"feed_emoji":"🧍","tokens_out":6151,"duration_ms":54229,"temperature":0.7,"pith_summary":"The paper introduces Fine-grained Human-body Abnormality Detection (FHAD): given an image or video frame of a person, a system must say which body part is abnormal, whether it is absent or redundant, and where it is located. The authors build two datasets, one synthesized from COCO by masking body parts and one hand-labeled from AI-generated videos, and show that powerful vision-language models perform near random guessing on this task. They then propose HumanCalibrator, which detects absent parts with a vision-language model trained on masked real photos to exploit body-part correlations, and detects redundant parts by using a diffusion inpainter to regenerate each grounded body part and comparing semantics before and after. On the AIGC Human-Aware 1K test set, the framework reaches 80.69% mean accuracy for absent parts and 58.57% for redundant parts, and it repairs the abnormal region while preserving the rest of the image. The central claim is that a post-processing step can turn anatomically impossible generated people into plausible ones without regenerating the whole picture.","feed_headline":"Finds and fixes impossible body parts in AI-generated people","feed_subtitle":"New benchmark and method catch the hands, ears, and arms that image models get wrong — and correct them without touching the rest of the…","key_machinery":"The load-bearing mechanism is the pairing of two complementary detectors. AHD is a LLaVA-1.5-7B vision-language model fine-tuned with an autoregressive objective on automatically masked COCO images, so it learns correlations among body parts and can name both the missing part and its bounding box. The redundant-part detector uses a grounding model to list body parts, a Stable Diffusion inpainting model to regenerate each part in place, and a semantic comparison to flag parts whose regenerated content differs strongly from the original; this makes it sensitive to parts appearing anywhere in the frame. The same inpainter then repairs flagged regions with a prompt template, and the loop terminates when AHD reports no further absences.","core_discovery":"HumanCalibrator is a two-part detection-and-repair framework for generated human images. For absent body parts, a fine-tuned vision-language model called the Absent Human-body Detector (AHD) learns from COCO images in which one body part has been replaced by background, so it can infer a missing hand, ear, arm, leg, foot, or head from the position and identity of the remaining parts. For redundant body parts, HumanCalibrator grounds all visible body parts, inpaints each one with a diffusion model conditioned on the part name, and compares semantic content before and after; a large difference marks that part as redundant. The same inpainting model then repairs the flagged regions, cycling until no new absences appear. The paper reports 80.69% average accuracy on absent abnormalities and 58.57% on redundant abnormalities on the hand-labeled AIGC Human-Aware 1K, with false discovery rates of 8.48% and 2.54% respectively, and better visual consistency than a pose-conditioned baseline (FID 16.55 versus 98.86).","pith_inferences":["Because the paper's localization metric is repair-based, detection accuracy and inpainting strength are entangled; an independent bounding-box benchmark would be needed to tell whether HumanCalibrator finds the right region or merely repairs whatever region it is given.","The same masked-part correlation training could be applied to other objects with fixed part geometry, such as animals, faces, or vehicles, making the approach a general post-hoc sanity checker for generative models.","A concrete testable extension is to run HumanCalibrator on images with one obvious correctable abnormality and a second, subtler abnormality, to see whether the cyclic detection stops prematurely once the first repair succeeds.","Because the redundant-part detector relies on the inpainting model's prior over normal bodies, its accuracy on a new generator may degrade if that generator's artifacts violate the inpainter's learned normality; a per-generator calibration study would reveal this."],"forward_implications":["Generated human images can be post-processed automatically: absent hands, ears, and limbs get regenerated, and extra body parts get removed, without regenerating the background or the rest of the person.","Video generation can be fixed at low cost by repairing only the first and last frames and letting a keyframe interpolation model rebuild the intermediate frames.","The FHAD datasets give the community a benchmark on which current vision-language models score near random, so progress on anatomical abnormality perception becomes measurable.","Because AHD was trained only on real COCO images and still transfers to AI-generated frames, the correlation-based training recipe may extend to other structured-object defects without expensive AIGC annotation."],"supporting_citations":[{"why":"Supplies the prior post-processing baseline for abnormal human generation that HumanCalibrator is compared against on visual consistency.","marker":"[14]"},{"why":"Provides the LLaVA v1.5 base vision-language model that is fine-tuned to become the Absent Human-body Detector.","marker":"[33]"},{"why":"Is the source of generated videos from which the AIGC Human-Aware 1K test frames are sampled and annotated.","marker":"[58]"},{"why":"Underpins the latent diffusion inpainting model used for redundant-part regeneration and final repair.","marker":"[50]"},{"why":"Supplies the open-source VLM baseline family whose near-random accuracy motivates the claim that existing VLMs lack abnormality perception.","marker":"[8]"}],"fun_headline_variants":["AI image models now get a fix for extra hands and missing ears","New tool spots and repairs abnormal bodies in AI photos","HumanCalibrator: a scalpel for AI-generated anatomy","Detects and repairs the hands AI draws wrong","A self-correcting model for impossible body parts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper measures how accurately HumanCalibrator locates an abnormality by how well the repaired image looks after inpainting that region, because the test set has no bounding-box annotations; if repair quality is not a faithful proxy for localization accuracy, the reported detection numbers do not establish that the box is the right box.","fun_headline_variants_meta":{"raw":{"variants":["AI image models now get a fix for extra hands and missing ears","New tool spots and repairs abnormal bodies in AI photos","HumanCalibrator: a scalpel for AI-generated anatomy","Detects and repairs the hands AI draws wrong","A self-correcting model for impossible body parts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000655,"raw_usage":{"total_tokens":3040,"prompt_tokens":1028,"completion_tokens":2012,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":644,"completion_tokens_details":{"reasoning_tokens":1932}},"tokens_in":644,"tokens_out":2012,"duration_ms":12864,"temperature":1.0,"reasoning_tokens":1932,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:24:53.557684+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hand-annotate the abnormal regions in AIGC Human-Aware 1K with bounding boxes and compute IoU between HumanCalibrator's predicted boxes and the human boxes; if the repair-quality-based accuracy stays high while IoU is near chance, the central detection claim is not supported, whereas high IoU would corroborate the paper's evaluation.","supporting_citations":[{"cited_title":"Visual instruction tuning, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the LLaVA v1.5 base vision-language model that is fine-tuned to become the Absent Human-body Detector."}],"review_version":1}