{"id":"05748215-4dc4-4e62-9388-47bedea8b1b7","arxiv_id":"2412.09920","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"PIHOT adds depth estimation and object inpainting to human-object contact detection, reporting top scores on HOT benchmarks, but its model-selection comparison contains a contradiction.","lead":"PIHOT is a new computer vision method that combines depth maps and image inpainting to better detect where people touch objects in photos, even when objects are partly hidden. It reports state-of-the-art results on three human-object contact benchmarks, though some reported comparisons are internally inconsistent.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Depth-difference signal ds (Eqs. 5-6) may encode the human silhouette rather than contact-specific spatial relations, because ZoeDepth's depth on inpainted regions is unvalidated; the paper's own appendix shows the body dominates ds.","rationale":"I agree with the reader that the depth-difference logic is the least secure assumption. The reader's verdict is CONDITIONAL; my analysis supports that: the mechanism needs empirical validation before the SOTA claim can be accepted. I also note the Table 4 inconsistency: MAE+MDENet achieves mIoU 0.279 vs Lama+ZoeDepth 0.236, while the text claims the latter is superior by 6% on mIoU. This is a concrete reporting error that further weakens confidence in the experimental narrative, but it is secondary to the mechanistic concern. No code is available, so the proposed test (ablation replacing ds) is a feasible way to adjudicate the core claim without the full release. I therefore leave the verdict unchanged at CONDITIONAL until the depth signal's causal role is tested.","tokens_in":12457,"tokens_out":11469,"duration_ms":121633,"concrete_test":"Replace ds in Eq. 8 with (a) the normalized dilated human mask, and (b) a constant-zero map, while keeping all other modules fixed, and retrain on the three datasets. If SC-Acc/mIoU drop by less than ~1 point relative to the reported PIHOT for both replacements, then the depth difference is not the active ingredient and the claimed depth-awareness is not supported. Additionally, on a small synthetic subset with known ground-truth depth, compare the pipeline using ZoeDepth versus ground-truth depth for the inpainted regions; if the gains vanish or reverse, the depth model's behavior on inpainted content is the decisive factor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central novelty is that subtracting the depth map of the human-removed, inpainted image from the depth map of the original image isolates the human and the occluded object, providing a physical depth cue for contact. This requires the monocular depth model to produce geometrically consistent depth for hallucinated content that was not present during training. No evidence is provided that ZoeDepth satisfies this; in fact, the appendix (Section 'The role of depth map ds') states that in the difference map 'the body is brighter, the background and object parts are darker, and occlusions have intermediate brightness.' That pattern is exactly what one would expect from the human mask alone: the body region differs, the rest is mostly unchanged. If ds carries no contact-specific information beyond a human silhouette, then the SPO/IDSI gains in Tables 2-3 and the reported SOTA improvements may be caused by the additional attention parameters or by leaky use of the mask, not by the claimed depth-aware disambiguation. The paper's own visualizations are qualitative and do not separate these explanations. This is load-bearing because the title and abstract promise that depth information 'prevents false interaction detection'; if the depth signal is arbitrary or mask-like, the mechanism is unsupported, even if the empirical numbers are later reproduced.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PIHOT, a human-object contact (HOT) detection method that aims to handle occlusion by combining object inpainting with depth-map cues. Specifically, it removes the human from the input via a dilated mask and an inpainting model, computes depth maps of the original and inpainted images, and uses their normalized absolute difference as a spatial-relation feature. Two attention mechanisms (IPI and IDSI) are introduced to fuse object, contact, and depth features, and a contact perception operation combines them for final segmentation. Experiments on HOT-Annotated and HOT-Generated (and the combined Full Set) report consistent improvements over the DHOT baseline, e.g., Table 1 shows +11%/+14%/+10%/+10% on SC-Acc., C-Acc., mIoU, and wIoU for HOT-Annotated and larger gains on HOT-Generated. The abstract claims state-of-the-art performance on three benchmarks with average improvements of 13%, 27.5%, 16%, and 18.5% over DHOT.","tokens_in":12706,"tokens_out":6440,"duration_ms":62872,"significance":"If the reported results hold, the paper makes a useful engineering contribution to a relatively new task (HOT detection) under occlusion, and the idea of using depth differences between original and inpainted images as a spatial prior is interesting. The paper also provides ablation studies and a comparison of inpainting/depth model choices, which is helpful for reproducibility. However, the central mechanistic claim that the depth-difference signal encodes contact-specific spatial relations is not validated, and the model-selection table (Table 4) contains an internal inconsistency. The empirical gains, while consistent, are based on single runs without error bars. Thus the current evidence supports the method's practical usefulness but not the depth-aware interpretation that motivates it.","major_comments":[{"comment":"Table 4 lists MAE+MDENet with mIoU 0.279, which is higher than the selected LaMa+ZoeDepth mIoU of 0.236, yet the text states that MAE+MDENet 'fall[s] short of the selected models by 4%, 6%, 6%, and 2% across the four metrics.' For mIoU the alternative is better, not worse. Since the selection of LaMa+ZoeDepth is justified by this comparison, the discrepancy is load-bearing and must be corrected or explained.","section":"Table 4"},{"comment":"The appendix states that in the difference map ds 'the body is brighter, the background and object parts are darker, and occlusions have intermediate brightness.' This pattern is exactly what one would expect from the human mask alone (the body region changes after inpainting, the rest does not), so it is not evidence that ds encodes contact-specific spatial relations. Equations (5)-(6) are the core of the proposed depth-aware disambiguation, but no quantitative analysis (e.g., correlation of ds with the human mask, or an ablation replacing ds with a dilated human mask) is provided to show that ds carries information beyond the human silhouette. Without such evidence, the claimed mechanism for 'preventing false interaction detection' is unsupported, even if the endpoint metrics are positive.","section":"Appendix, 'The role of depth map ds'"},{"comment":"Equation (3) uses the symbol xb in the definition of Q, but xb is never defined in the paper; the surrounding text says Q is derived from the object feature xo. Also, Eq. (7) defines S as a concatenation of ds with two identical terms E(conv2d(oa)), which appears to be a typographical error. These undefined/inconsistent quantities make the IPI and IDSI modules formally incomplete and should be corrected.","section":"Eq. (3) and Eq. (7)"},{"comment":"No error bars or multiple-seed results are reported in any of the experimental tables. The headline claim of state-of-the-art performance rests on single runs; the improvements over DHOT vary widely across metrics (e.g., 10% vs 41% on C-Acc. for the two datasets in Table 1), so without variance estimates the statistical significance of the gains cannot be assessed. Reporting mean and standard deviation over at least three seeds is needed to support the SOTA claim.","section":"Tables 1-5"}],"minor_comments":[{"comment":"The abstract reports average improvements of 13%, 27.5%, 16%, and 18.5% without stating that these are averages over the HOT-Annotated and HOT-Generated datasets; the corresponding numbers are not directly visible in a single table, so the averaging should be defined.","section":"Abstract"},{"comment":"In Table 2, row 2, the SC-Acc. improvement from 40.5 to 42.4 is labeled +7%, but the relative increase is 4.7%; this percentage should be recalculated or the convention (percentage points vs relative) clarified.","section":"Table 2"},{"comment":"Equation (9) applies a per-pixel binary cross-entropy loss independently to each of the Cy channels, but the task is described as segmentation with 17 foreground classes plus background. If a softmax over classes is intended, the loss is mis-specified; if multi-label binary CE is intended, this should be stated explicitly.","section":"Eq. (9)"},{"comment":"Figure 5 reports Full Set results visually, but the corresponding numeric values are never tabulated; including a table would make the SOTA claim on the Full Set easier to verify.","section":"Figure 5"},{"comment":"There are minor typographical and stylistic issues (e.g., 'twofolds' in Related Work, repeated sentence fragments such as 'Occlusion' as a standalone label), and the paper would benefit from a careful proofread.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The Table 4 inconsistency and the undefined xb in Eq. (3) suggest that the manuscript was not carefully proofread. More substantively, the depth-cue validation concern is the main scientific issue: the authors need to demonstrate that ds carries contact-specific information beyond the human mask, or temper their mechanistic claims. If they can provide such an analysis, the paper may be salvageable as an empirical contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, it is a genuine extension of DHOT: it is the first to bring object inpainting and depth-map differencing to human-object contact detection, and the main results in Table 1 are internally consistent. Second, the paper has a real flaw in its model-selection comparison and a mechanism that may not do what the authors claim.\n\nWhat is new: applying known techniques (LaMa inpainting, ZoeDepth) plus two attention modules to the HOT task. That is incremental but not trivial—occlusion is a real problem for contact segmentation, and the ablation suggests each module adds something. The reported gains over DHOT on HOT-Annotated and HOT-Generated look plausible, and the visualizations show cleaner boundaries in some cases.\n\nNow the soft spots. Table 4 is simply wrong as written: MAE+MDENet gets mIoU 0.279 while the selected LaMa+ZoeDepth gets 0.236, and the text says the selected model is better on mIoU by 6%. That is a direct contradiction a referee will catch immediately. No error bars or multiple seeds are reported, so we cannot tell whether the gains over DHOT are noise. The depth-differencing signal d_s is the bigger conceptual concern. The appendix admits that in the difference map the body is brighter and the object/background darker, which is exactly what you would expect from subtracting the inpainted image's depth from the original: the human region dominates. That makes d_s closer to a human-position prior than a contact-specific spatial cue. The authors are honest about this in the appendix, but then they still lean on it as the core of the title's claim that depth \"prevents false interaction detection.\" The attention modules or the mask itself could be doing the work.\n\nNone of this kills the empirical result—the method may still help in practice—but it shifts the paper from a clean demonstration of depth-aware disambiguation to a more modest claim: an inpainting-and-attention augmentation that works better than the baseline, with an unexplained depth component.\n\nWho is this for? Researchers working on HOT or fine-grained HOI will want to see it, but they should treat the numbers as preliminary. It deserves a serious referee because the task is real and the baseline comparisons, once corrected, could be useful. I would recommend engaging with it, but only if the authors resolve the Table 4 inconsistency, release code or detailed implementation notes, and provide uncertainty estimates.","headline":"PIHOT is a plausible incremental extension of DHOT with consistent-looking gains, but the depth-differencing mechanism is less well supported than the title claims, and the paper contains a clear numerical inconsistency in its model-selection table.","tokens_in":13288,"tokens_out":1833,"would_cite":false,"duration_ms":22726,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The PIHOT detector restores occluded object textures and compares depth maps to separate true human-object contact from mere 2D overlap, improving accuracy on all three HOT benchmarks.","keywords":["human-object contact detection","depth-difference signal","occlusion handling","image inpainting","contact segmentation","cross-attention mechanism","zero-shot depth estimation","HOT benchmarks"],"falsifier":"Replace ZoeDepth with a depth model that is known to output constant or random depth for inpainted regions, then measure PIHOT's gains over its baseline; if SC-Acc. and C-Acc. still improve, the depth channel is not the active mechanism. Alternatively, build a synthetic dataset with ground-truth geometry and check whether $d_s$ is zero at true contact areas and positive at mere 2D overlaps.","tokens_in":12252,"feed_emoji":"🤝","tokens_out":8552,"duration_ms":78146,"temperature":0.7,"pith_summary":"This paper sets out to fix a blind spot in human-object contact detection: when a person's body hides the object they touch, current methods misplace or blur the contact boundary. Its proposed detector, PIHOT, restores the hidden object texture by inpainting the person out of the photo, then compares depth maps of the original and restored images so that only objects actually touching the person remain highlighted. On three established HOT benchmarks the method reports consistent gains over the previous DHOT approach, with Contact Accuracy improving by 14% on annotated images and 41% on generated images. If these results hold, occlusion no longer needs to be a hard barrier for contact-aware perception in robots, AR, and video analysis.","feed_headline":"Depth-map difference uncovers hidden human-object contact zones","feed_subtitle":"Restoring occluded object textures and comparing depths lifts segmentation accuracy by up to 41% over the prior method.","key_machinery":"The load-bearing mechanism is the depth-difference map $d_s$: the pipeline runs the original image and the human-masked, inpainted image through the same zero-shot depth estimator (ZoeDepth), subtracts the two depth maps, and min–max normalizes the absolute difference. In the ideal case this leaves only the person and the contacted object, since both are present in the original but only the object remains in the inpainted image. Two attention modules then carry the signal: the Instances Perspective Interaction (IPI) module applies cross-attention between the restored-object features and the original contact features, and the Instances Depth Space Interaction (IDSI) module feeds $d_s$ through a second attention stage. The Contact Perception Operation fuses these features, with $d_s$ acting as a spatial gate that highlights exactly the pixels whose depth changes when the person is removed.","core_discovery":"On the paper's own terms, the central discovery is that the depth map of a photograph with the person inpainted away, subtracted from the depth map of the original photograph, yields a signal $d_s = |d_i - d_o|$ that isolates the occluding person and the once-hidden object. The paper argues that this depth-divergence channel, fed through cross-attention with restored object features, gives a segmentation network what it needs to draw contact boundaries that were previously invisible. The reported numbers—up to 45.3 SC-Acc. and 80.7 C-Acc. on the annotated set, and up to 34.9 SC-Acc. and 76.3 C-Acc. on the generated set—exceed every baseline in the paper, including the full DHOT model.","pith_inferences":["This suggests the depth-divergence trick could apply to amodal segmentation, where the inpainted object shape supplies a plausible estimate of the hidden region.","Because the depth estimator is zero-shot, the pipeline may transfer to new scene types without retraining the depth component, bound mainly by the restorer's quality.","A testable extension would replace the provided human masks with automatic human segmentation at inference to measure how sensitive PIHOT is to mask errors.","The ablations accumulate modules in a fixed order, so the individual contribution of the depth channel without inpainting is not isolated; a factorial ablation would pin down the cause of each gain."],"forward_implications":["Contact regions hidden by occluding body parts can be segmented without explicit 3D scene reconstruction.","The depth-difference channel transfers to all three HOT benchmarks, where adding the SPO and IDSI modules improves mIoU and wIoU in the ablations.","The object-inpainting model is a plug-in component: swapping LaMa for MAE changes results but the pipeline still functions.","The gap over DHOT reported in Table 1 suggests the earlier approach's occlusion failure mode is addressable by depth and texture cues."],"supporting_citations":[{"why":"Defines the HOT task, supplies the three benchmark datasets and four metrics, and provides the DHOT baselines PIHOT is compared against.","marker":"(Chen et al. 2023)"},{"why":"Provides the LaMa inpainting model used to remove the person and restore occluded object texture.","marker":"(Suvorov et al. 2022)"},{"why":"Provides the ZoeDepth zero-shot depth estimator that produces the depth maps whose difference forms the spatial signal.","marker":"(Bhat et al. 2023)"},{"why":"Supplies the MAE restoration alternative used in Table 4 to show the chosen restoration model's effect.","marker":"(He et al. 2022)"},{"why":"Supplies the UperNet segmentation baseline compared in the HOT benchmarks.","marker":"(Xiao et al. 2018)"},{"why":"Supplies the PPM segmentation baseline compared in the HOT benchmarks.","marker":"(Zhao et al. 2017)"}],"fun_headline_variants":["Depth-subtraction trick reveals occluded human-object contacts","PIHOT uses depth difference to see through occlusions","Depth-aware interaction detector beats prior by up to 41%","Texture restoration plus depth cues boost contact segmentation","New method lifts human-object contact accuracy across three benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The depth-difference logic assumes that subtracting the depth map of the restored image from the depth map of the original cleanly isolates the human and the contacted object; if the depth model assigns arbitrary depth to hallucinated inpainted regions, the signal carries no physical contact information.","fun_headline_variants_meta":{"raw":{"variants":["Depth-subtraction trick reveals occluded human-object contacts","PIHOT uses depth difference to see through occlusions","Depth-aware interaction detector beats prior by up to 41%","Texture restoration plus depth cues boost contact segmentation","New method lifts human-object contact accuracy across three benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1322,"prompt_tokens":908,"completion_tokens":414,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":336}},"tokens_in":524,"tokens_out":414,"duration_ms":4757,"temperature":1.0,"reasoning_tokens":336,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:34:16.013135+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace ZoeDepth with a depth model that is known to output constant or random depth for inpainted regions, then measure PIHOT's gains over its baseline; if SC-Acc. and C-Acc. still improve, the depth channel is not the active mechanism. Alternatively, build a synthetic dataset with ground-truth geometry and check whether $d_s$ is zero at true contact areas and positive at mere 2D overlaps.","supporting_citations":[{"cited_title":"K.; Black, M","cited_arxiv_id":null,"evidence_quote":"Defines the HOT task, supplies the three benchmark datasets and four metrics, and provides the DHOT baselines PIHOT is compared against."}],"review_version":1}