{"id":"10e12251-e31c-42cd-bc0f-ff813c09c475","arxiv_id":"2412.08506","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"InterProDa models each HOI category as a Gaussian distribution over soft prompt embeddings, sampled with a learnable noise factor, and reports SOTA results on HICO-DET and V-COCO.","lead":"This paper proposes a method for detecting human-object interactions in images by representing each interaction category as a learned distribution of soft prompts, rather than a single fixed query. The approach reports state-of-the-art results on two standard benchmarks and shows it can be bolted onto existing detectors with small parameter overhead.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (3) in §3.3 is sign-reversed: the dynamic orthogonal loss penalizes low cosine similarity, pulling categories together rather than separating them; the central constraint cannot be reproduced as written.","rationale":"The reviewer's Gaussian-distribution concern is mitigated by the paper's own ablation: Table 5(c) shows that a Fourier basis achieves comparable or better performance than the Gaussian basis (Full 42.40 vs 42.01; Rare 44.42 vs 44.44), so the SOTA claim is not contingent on the Gaussian form. The more load-bearing issue is the sign reversal in Eq. (3). The dynamic orthogonal loss is described as enforcing separation between categories, but as written it penalizes low cosine similarity and therefore pulls category distributions together toward a fixed positive cosine value. Since the +Ldo ablation is the largest single component gain in Table 3, the mismatch between the stated mechanism and the equation is not a minor typo: it affects reproducibility and the interpretation of nearly two mAP points. The existing CONDITIONAL verdict should stand, but the acceptance conditions must include either correcting Eq. (3) to match the implemented loss or providing the implementation so the actual constraint can be verified. This is a concrete internal inconsistency rather than a disagreement with community consensus, and it can be settled by a focused reimplementation experiment.","tokens_in":12410,"tokens_out":11344,"duration_ms":123961,"concrete_test":"Train InterProDa on HICO-DET default split with identical hyperparameters (seed 42, K=8, Ns=2, α=0.5, λ=5e-2) under two variants: (A) Ldo exactly as Eq. (3) is written, and (B) the sign-corrected hinge max(ε, cos−Δ)^2. If variant (B) reproduces the reported +Ldo row (Full 41.85, Rare 43.80) and variant (A) does not, Eq. (3) is a typo and the stated constraint characterization is wrong; if variant (A) reproduces it, the 'orthogonal' semantics in §3.3 are false and the mechanism must be redescribed. Either way, the manuscript's central constraint specification is not currently trustworthy.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"In Section 3.3, Eq. (2)-(3) define the dynamic orthogonal loss. With Δ(i,j)=α(1−cos(i,j)), the loss term is max(ε, Δ−cos)^2 = max(ε, α−(1+α)cos)^2. For α=0.5, a pair with cos=1 yields max(ε,−1)=ε, so highly similar categories are essentially unpenalized, while a pair with cos=0 yields 0.5, a large penalty. Minimization therefore drives cos upward toward the fixed point α/(1+α)=0.333 for all pairs, i.e., it is an attractive constraint, not an orthogonal one. This directly contradicts the text's stated goal of increasing the distance between ambiguous categories and expanding separation among semantically distinct categories. Table 3 attributes the single largest ablation gain to +Ldo (39.56→41.85 Full, 43.80 Rare), so the sign error is not cosmetic: either the implementation differs from the published equation, making the paper non-reproducible, or the implemented loss has the opposite effect from the claimed one. A reader cannot tell which from the manuscript.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes InterProDa, a prompt-distribution-learning approach for HOI detection. For each HOI category, the method learns a collection of soft prompts fed through a CLIP text encoder, estimates a Gaussian distribution over the resulting embeddings, and samples from these distributions with a learnable noise factor to form category-distribution queries that are added to transformer decoder queries. Three structurally different prompt groups are used for subjects, objects, and interactions, and a dynamic orthogonal loss is applied to the object and interaction distribution spaces. Experiments report state-of-the-art results on HICO-DET and V-COCO, zero-shot results, ablations, and portability improvements when the module is added to HOICLIP and ADA-CM.","tokens_in":12673,"tokens_out":7147,"duration_ms":79048,"significance":"If the method is correctly implemented, the idea of distribution-valued queries is a plausible and useful step beyond fixed or learned point prompts, and the reported gains on rare HOI categories are substantial. The portability study in Table 4 is a practical strength, and the ablation set in Tables 3 and 5 is fairly comprehensive. However, the manuscript currently contains a sign-reversed loss equation for the component that Table 3 credits with the largest single improvement, and the reported benchmark numbers come from a single stochastic inference pass. Both issues must be resolved before the empirical claims can be relied upon.","major_comments":[{"comment":"The dynamic orthogonal loss as written is sign-reversed with respect to the stated goal. Since Δ(i,j)=α(1−cos(i,j)), the penalty term becomes max(ε, α−(1+α)cos(i,j))². For α=0.5, a pair with cos=1 gives loss ε, while a pair with cos=0 gives 0.25, so well-separated pairs are penalized more than confused pairs; minimization drives cos toward approximately α/(1+α)=1/3. This is an attractive constraint, not an orthogonal one, and it contradicts the text's claim that Ldo 'increases the distance between ambiguous interaction categories'. This matters because Table 3 attributes the largest single ablation gain to +Ldo (39.56→41.85 Full). The authors must either correct Eq. (3), e.g. to max(ε, cos(i,j)−Δ(i,j))², or explicitly state how the implemented loss differs from the published equation; as written, the central constraint is not reproducible.","section":"§3.3, Eq. (3)"},{"comment":"The reported results have no uncertainty quantification despite the inference procedure being stochastic. The sampling step S(P_c)=μ_c+γnσ_c draws fresh Gaussian noise at inference, and the implementation section specifies only a single random seed (42). The headline numbers in Table 1 (42.67 Full, 45.21 Rare) and Table 4 (41.71/42.92) are therefore single-rollout values. Since the claimed improvements over the closest baselines are on the order of 2–3 mAP, the paper should report mean and standard deviation over at least three seeds, or state that a fixed noise draw is used at test time; otherwise the SOTA claim is not statistically grounded.","section":"§4.2–4.3"}],"minor_comments":[{"comment":"The cosine similarity notation is ambiguous: the numerator is written as if it were a norm of the pair, and the denominator uses ℓ1 norms. Please write it as ⟨P̄_ci, P̄_cj⟩ / (‖P̄_ci‖_1‖P̄_cj‖_1+ε) and clarify whether P̄ is the mean-pooled token vector or a matrix.","section":"§3.3, Eq. (2)"},{"comment":"The dynamic orthogonal loss is applied only to the object and interaction spaces, but no ablation shows the effect of excluding the subject space. A short experiment or a more detailed motivation would help.","section":"§3.3"},{"comment":"The Gaussian assumption on prompt embeddings is asserted with citations to domain-generalization work, but no empirical check is provided for CLIP embeddings of HOI prompts. Given that Table 5(c) shows a Fourier basis achieves similar performance, the paper should either add a normality diagnostic or explicitly state that the method is robust to the distributional choice.","section":"§3.2"},{"comment":"Several table entries run numbers together, e.g. '42.6745.21' and '40.8839.58'; these need spacing or a column separator for readability.","section":"Table 1, Table 2"},{"comment":"The reference list has formatting problems: 'Part et al. (2023)' in Table 1 should be 'Park et al. (2023)', 'Tamura et al.(2021)' is mis-formatted, and two different works are cited as 'Zhang et al. (2022)', making the in-text citations ambiguous.","section":"References"},{"comment":"The section heading contains a typo: 'Conlusion' should be 'Conclusion'.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"The paper's novelty relative to ProDA and DreamDistribution is adequately acknowledged, and the HOI-specific prompt structures plus the portability experiments are genuine contributions. The sign error in Eq. (3) is the main technical blocker; if the authors confirm the implementation matches the intended orthogonal objective and add multi-seed evaluation, the paper would be a reasonable candidate for acceptance. I would also ask the editor to enforce journal reference style, as the current bibliography is unusually sloppy."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things. The method genuinely moves the needle on HICO-DET, especially rare classes, and the ablation story is coherent. But Eq. (3) is sign-reversed: the 'orthogonal' loss actually drives cosine similarity up toward a fixed point, which is the opposite of the stated goal. That makes the central constraint non-reproducible as written.\n\nThe application of prompt distribution learning to HOI detection with subject/object/interaction prompt structures is new relative to ProDA and DreamDistribution. The gains on rare categories are large (45.21 rare mAP), and the portability results in Table 4 show real value: +2M params on HOICLIP and <1M on ADA-CM give meaningful boosts. The ablations are thorough and each component contributes. The learnable noise factor in reparameterization is a sensible fix for VAE-style instability.\n\nThe sign issue is not cosmetic: Table 3 credits Ldo with the largest single gain (39.56 to 41.85 Full). If the implementation matches the equation, the loss is attractive, not orthogonal. If it doesn't, the paper doesn't tell you. Either way, a reader cannot reproduce the result from the text. Also missing: standard deviations across runs, no code or weights, and the main Table 1 has formatting issues (numbers run together like '42.6745.21'). The Gaussian assumption for prompt embeddings is asserted with citations but not tested; that is a minor concern given the gains, but it is an assumption.\n\nThis paper deserves a serious referee despite my skepticism about the reproducibility. The core idea is worth evaluating and the empirical gains, if they hold, are valuable to the subfield. An accept should be conditioned on correcting Eq. (3) or explaining the actual loss, releasing code/weights, and adding variance estimates. I wouldn't cite it until the sign error is resolved.","headline":"Solid HOI gains from prompt distribution learning, but the dynamic orthogonal loss as written pulls categories together rather than apart, and the paper ships no code or error bars.","tokens_in":13177,"tokens_out":2709,"would_cite":false,"duration_ms":27030,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Modeling each interaction as a distribution beats fixed queries","keywords":["human-object interaction detection","prompt distribution learning","soft prompts","category distribution queries","Gaussian distribution","reparameterization sampling","zero-shot HOI detection","transformer-based detection"],"falsifier":"Sample many random prompts per HOI category, embed them with the CLIP text encoder, and apply a multivariate normality test (e.g., Henze-Zirkler) to each per-category cloud; if the clouds are clearly multi-modal or heavily skewed, the Gaussian assumption underlying InterProDa's mean/variance estimates fails, undermining the claimed fidelity of the learned distributions.","tokens_in":12215,"feed_emoji":"🎯","tokens_out":14116,"duration_ms":118890,"temperature":0.7,"pith_summary":"The paper argues that a major reason transformer-based HOI detectors miss rare or ambiguous human-object interactions is that each interaction category is represented by a single query or prompt, which cannot capture the many visual forms one action can take. To fix this, the authors introduce InterProDa, which learns several soft prompts per category, estimates a Gaussian distribution over them, and samples from that distribution to form category-aware detector queries. With this change, a single category can express a near-infinite range of visual patterns, and distinct categories can be pushed apart in a shared distribution space. On the standard benchmarks HICO-DET and V-COCO, the approach reaches 42.67 full mAP, 45.21 rare mAP, and 67.6 scenario-1 AP, outperforming prior methods and boosting existing detectors when plugged in with only a few extra parameters.","feed_headline":"Modeling each interaction as a distribution beats fixed queries","feed_subtitle":"New HICO-DET and V-COCO results show rare interactions gain the most; the approach also boosts other detectors.","key_machinery":"The central object is the category distribution query: each HOI category is represented not by a fixed prompt embedding but by a Gaussian distribution estimated from $K$ learnable soft prompts, with mean $\\mu$ and variance $\\sigma$ in the CLIP text-embedding space. Before feeding queries to the detector, the distribution is sampled via a VAE-style reparameterization with a learnable noise scale $\\gamma$, producing a finite set of samples $S(\\hat{P}_c)=\\hat{\\mu}_c+\\gamma n \\hat{\\sigma}_c$ that are linearly projected and added to the decoder queries. This gives each query a continuous range of visual patterns, while a dynamic orthogonal loss on the mean embeddings keeps ambiguous categories separated in the shared distribution space.","core_discovery":"InterProDa claims that HOI category knowledge is better represented as a distribution than as a point. For each HOI category, the method learns a collection of K soft prompt embeddings, assumes these embeddings follow a Gaussian, and estimates their mean and variance. These per-category Gaussians are stacked into distribution spaces for subjects, objects, and interactions, trained jointly with a dynamic orthogonal loss that keeps semantically distinct categories apart. During inference, the distribution space is sampled using a reparameterization trick with a learnable noise scale, and the samples are fused into the decoder queries of a transformer-based HOI detector. The result, the authors report, is a detector that captures diverse intra-category visual patterns and inter-category dependencies, achieving state-of-the-art performance on HICO-DET and V-COCO and improving existing one-stage and two-stage detectors with small parameter overhead.","pith_inferences":["The paper does not test whether the Gaussian assumption is essential; if replaced with a mixture model or a non-parametric estimator, rare-category recall could improve further, since high-variance categories like 'hit a ball' appear visually multi-modal in the paper's own variance plots.","The paper's own ablation shows that replacing sampled queries with the distribution mean loses only about 0.28 mAP (42.39 vs 42.67), so the gains may come from variance-aware training rather than from stochastic sampling; a direct comparison of $\\gamma=0$ versus larger $\\gamma$ would settle that.","The three-way prompt structure (subject, object, interaction) is not confined to still images; the same distributional queries could be applied to video-based interaction detection or fine-grained action recognition, where intra-category variation is even larger.","The dynamic orthogonal loss only uses pooled category means; using full covariance structure or a learned distance between distributions could capture richer inter-category relationships."],"forward_implications":["HOI detectors that adopt InterProDa should improve disproportionately on rare and visually diverse categories, since the rare-split mAP gain (45.21 vs 42.67 full) is larger than the non-rare gain.","The approach transfers to other transformer-based HOI detectors: HOICLIP gains over 3 full mAP and ADA-CM gains over 3 full mAP with under 1-2 million extra parameters.","Zero-shot HOI detection also improves, with InterProDa outperforming prior prompt-based methods on both rare-first and non-rare-first unseen splits, even compared with multi-modal prompt methods.","Because the distribution representation is compatible with different space bases, the Gaussian choice is not critical: a Fourier basis yields similar performance, suggesting the benefit comes from the distributional query structure itself."],"supporting_citations":[{"why":"ProDA originated prompt distribution learning, from which InterProDa borrows the idea of estimating a category distribution from multiple prompts.","marker":"(Lu et al. 2022)"},{"why":"Supplies the CLIP text encoder whose embedding space hosts the learned prompt distributions.","marker":"(Radford et al. 2021)"},{"why":"QPIC provides the query-transformer detector and set-prediction loss that serve as the base architecture for InterProDa.","marker":"(Tamura et al. 2021)"},{"why":"HOICLIP's decoder with CLIP knowledge transfer is adopted as the detector backbone and used as a portability testbed.","marker":"(Ning et al. 2023)"},{"why":"ADA-CM is the closest prompt-learning-based HOI detector and the second portability testbed for the distribution learning pipeline.","marker":"(Lei et al. 2023)"},{"why":"The VAE reparameterization trick is the sampling mechanism that keeps distribution learning differentiable.","marker":"(Kingma and Welling 2022)"},{"why":"CQL introduced category queries as learnable priors for HOI classification, the representation that InterProDa extends from a point to a distribution.","marker":"(Xie et al. 2023)"},{"why":"GEN-VLKT establishes the zero-shot HOI evaluation protocol and is a primary baseline in the zero-shot comparison.","marker":"(Liao et al. 2022)"}],"fun_headline_variants":["Interactions as distributions beat fixed queries","HOI detection: distribution over queries beats fixed points","Soft prompt distributions capture rare interaction patterns","InterProDa: learn a Gaussian per interaction, boost HOI","Model each interaction as a distribution, not a point"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that all prompt embeddings describing one HOI category form a single Gaussian cloud in CLIP's embedding space, so the estimated mean and variance faithfully summarize that category's visual patterns.","fun_headline_variants_meta":{"raw":{"variants":["Interactions as distributions beat fixed queries","HOI detection: distribution over queries beats fixed points","Soft prompt distributions capture rare interaction patterns","InterProDa: learn a Gaussian per interaction, boost HOI","Model each interaction as a distribution, not a point"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000185,"raw_usage":{"total_tokens":1281,"prompt_tokens":863,"completion_tokens":418,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":479,"completion_tokens_details":{"reasoning_tokens":345}},"tokens_in":479,"tokens_out":418,"duration_ms":5498,"temperature":1.0,"reasoning_tokens":345,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:42:53.485877+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Sample many random prompts per HOI category, embed them with the CLIP text encoder, and apply a multivariate normality test (e.g., Henze-Zirkler) to each per-category cloud; if the clouds are clearly multi-modal or heavily skewed, the Gaussian assumption underlying InterProDa's mean/variance estimates fails, undermining the claimed fidelity of the learned distributions.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CQL introduced category queries as learnable priors for HOI classification, the representation that InterProDa extends from a point to a distribution."}],"review_version":1}