{"id":"fe702d39-1dcf-4f43-b71f-ba6b263653ca","arxiv_id":"2501.10869","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Diffusion behavior cloning can generate facilitator poses, and conditioning on plotted pose keypoints lowers MPJPE but increases processing time versus raw images.","lead":"The paper applies an existing diffusion behavior cloning model to generate a facilitator's body poses in group discussions, comparing two ways to feed camera images to the model. It finds that pre-processed pose plots improve prediction error but slow processing, while neither setup runs in real time.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported MPJPE advantage of plotted images may reflect within-session memorization rather than generalization, because the 80-20 split is not specified as session-disjoint and actions are consecutive-frame deltas.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the train-eval split is not described as session-disjoint, so the reported MPJPE may partly measure memorization. I agree, and the risk is actually stronger than the reader stated because the action representation is a consecutive-frame delta. A random frame-level split means an evaluation observation can be a near-duplicate of a training observation one frame away, and the evaluation label is derived from poses visible in training. This directly threatens the central comparison in Table I, because plotted images remove background and isolate exactly the keypoints used to define actions, making memorization easier for that condition. The paper does provide some independent support: the comparison is concrete, all three facilitators show the same direction of improvement, and the processing-time trade-off is documented in Table III. However, with one run and no session-disjoint evaluation, the core claim remains conditional. The reader's conditional verdict is appropriate, and no verdict change is needed; the revision should require a session-disjoint evaluation and ideally multiple seeds or confidence intervals. If the proposed test shows the advantage persists on held-out sessions, the central claim would be substantially strengthened.","tokens_in":6841,"tokens_out":5085,"duration_ms":63952,"concrete_test":"Re-run the full pipeline with leave-one-session-out cross-validation: for each facilitator type, train on two of the three sessions and evaluate on the held-out session, repeating for all three folds. Report MPJPE per fold and the mean and standard deviation across folds for both raw and plotted conditioning, and also report the same metrics on a frame-level random split to quantify the leakage effect. If the plotted-image MPJPE advantage does not persist on held-out sessions, or flips sign in any fold, the headline result is attributable to within-session leakage rather than to the conditioning representation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that conditioning on plotted keypoint images yields lower MPJPE than raw images for facilitator pose generation (Table I). For this comparison to support the stated conclusion, the MPJPE difference must reflect generalization to new social interactions, not memorization of the training sessions. Section II-B says only that the authors used an 80-20 train-eval split on roughly 90,000 frames from nine sessions (three per facilitator), without stating that the split is session-disjoint. If frames were split randomly, the same facilitator session appears in both training and evaluation, and because actions are defined as per-joint coordinate differences between consecutive frames (Section II-B), adjacent frames are near-duplicates. An evaluation observation at time t can be almost identical to a training observation at time t+1, and the evaluation target p_{t+1} - p_t is computed from poses that also appear in training observations. This is a form of temporal leakage that can inflate apparent accuracy. The risk is not symmetric: plotted images remove background and isolate keypoints, which may make frame-level memorization easier than with raw images, so the raw-vs-plotted comparison could be biased in favor of plotted images. The absence of error bars or repeated-seed evaluation compounds the problem, since a single run cannot distinguish a genuine representation benefit from split-specific noise. Thus the main empirical claim is not securely established without a session-disjoint evaluation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper adapts the diffusion behavior cloning framework of Pearce et al. (2023) to generate facilitator joint displacements in a multiparty group discussion. The authors train a transformer-based denoising network on two conditioning representations: raw 360-degree camera images and plotted pose keypoints on a white background, using the FUMI-MPF dataset with nine sessions across three facilitator types. They report mean per-joint position error (MPJPE) and per-frame processing times, claiming that plotted keypoint images reduce MPJPE at a modest processing-time cost, and they use wrist-displacement statistics to interpret the performance differences. The paper frames its main contribution as a new application of diffusion behavior cloning to social pose generation and an evaluation of two conditioning modalities.","tokens_in":7162,"tokens_out":5015,"duration_ms":51994,"significance":"The contribution is incremental but potentially useful: it demonstrates an applied use of diffusion behavior cloning for social pose generation and tests a pose-only visual conditioning modality. If the results were securely established, the finding that plotted-keypoint conditioning yields lower MPJPE than raw images would be a practical data-representation insight for social robotics. The authors are transparent about limitations, including background GPU processes, the lack of human evaluation, and the need for larger datasets. However, the evaluation is currently too thin to support the headline claim: there are no error bars, no repeated-seed runs, no session-disjoint split, and no code or data release. The paper's value would be improved substantially by a more rigorous evaluation protocol.","major_comments":[{"comment":"The paper does not state whether the 80-20 train-eval split is session-disjoint. Because the data comprise nine sessions and actions are consecutive-frame joint displacements, a frame-level random split can place near-duplicate observations in both training and evaluation, making the reported MPJPE partly a measure of temporal memorization rather than generalization. This concern is especially relevant to the raw-vs-plotted comparison, since plotted images remove background and may make frame-level memorization easier. The authors should specify the split construction and report results on a session-disjoint split, for example by training on two sessions per facilitator and evaluating on the held-out session.","section":"Section II-B, Tables I and III"},{"comment":"The quantitative comparison consists of a single MPJPE value per facilitator-condition combination, with no error bars, no repeated-seed runs, and no significance tests. The differences in Table I range from 9.82% to 54.19%; without an estimate of variance, the small Teacher-session improvement may be within split-specific noise. The authors should report means and standard deviations over multiple seeds or sessions and, where appropriate, paired significance tests.","section":"Tables I and III"},{"comment":"The processing-time comparison is confounded by uncontrolled GPU background processes, as the authors themselves note for the Teacher raw-image condition. Since the central trade-off claim of 8.1-18.9% higher processing time for plotted images relies on these per-run timings, the processing-time comparison should be repeated under controlled conditions or the results should be reported with caveats that appear in the abstract and results summary, not only in the discussion.","section":"Section IV and Table III"},{"comment":"The abstract and conclusion state that the model generates \"realistic social behavior,\" but the only supporting evidence is MPJPE, a geometric error metric. The authors explicitly defer human evaluation to future work. The claim should be softened to \"quantitatively closer to ground-truth joints\" or complemented by a perceptual study; otherwise the conclusion overreaches the evidence.","section":"Abstract, Section II-B, and Section V"}],"minor_comments":[{"comment":"The caption contains a typo: \"RWRIST\" should be \"WRIST.\"","section":"Table II caption"},{"comment":"There is a capitalization error: \"To prepare the training set, We calculated\" should be \"we calculated.\"","section":"Section II-B"},{"comment":"There is a sentence fragment: \"Since behavior cloning has always been trained on expert behavior. [13], we decided...\" The period after \"behavior\" should be a comma.","section":"Section I"},{"comment":"The method section does not state how MPJPE is computed from the delta-action outputs (for example, whether predicted displacements are accumulated from a reference frame) or what coordinate units are used; please specify this for reproducibility.","section":"Section II-B"},{"comment":"The paper does not report key training hyperparameters other than T and M, such as learning rate, batch size, number of epochs, and transformer dimensions, which limits reproducibility.","section":"Section II-B"}],"recommendation":"major_revision","confidential_remarks":"This manuscript reads like a workshop or short conference paper. In its current form, the empirical rigor is below what I would expect for a full archival publication. I would encourage the authors to rerun the experiments with a session-disjoint split and multiple seeds; if those results support the current conclusion, the paper could be suitable after revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read on arXiv:2501.10869. It adapts Pearce et al.'s diffusion behavior cloning model to generate a conversation facilitator's pose from 360-degree images, and compares two conditionings: raw images vs plotted keypoints. That's a real but modest new application. The paper is honest about its limits; it explicitly says neither implementation is real-time and defers human evaluation to future work. Credit where due: the wrist-displacement distribution analysis is a sensible attempt to explain per-facilitator differences, and the training/inference trade-off plot is useful.\n\nNow the soft spots. The main empirical claim—plotted keypoints lower MPJPE—is not securely established. The 80-20 split is not described as session-disjoint, and because actions are consecutive-frame coordinate deltas, random frame splits create near-duplicate observations across train and eval. That's exactly the kind of temporal leakage that inflates accuracy and could bias the raw-vs-plotted comparison, since plotted images strip background and may make frame-level memorization easier. No error bars or repeated-seed runs either, so one run can't separate signal from split noise. The processing-time comparison is also confounded by background GPU processes, as the authors themselves admit. These aren't gotchas; the authors flag some of them. But they mean the central quantitative conclusion should be treated as tentative, not settled.\n\nWhat this paper is for: researchers working on social robotics or imitation learning for HRI who want a quick baseline result and a caveat about input representations. It's not a methods paper. With a session-disjoint split, error bars, and a clearer statement about generalization, it could be a solid short workshop or conference paper.\n\nI'd send it to peer review if I were an editor—the question is legitimate and the work is reproducible enough to evaluate. But I'd want heavy revision, mainly the split and statistical rigor. Bring to reading group? Maybe, as a useful example of temporal leakage in BC evaluation.","headline":"A small, honest application of diffusion BC to facilitator pose generation, but the headline MPJPE advantage rests on a train-eval split that may leak temporally.","tokens_in":7634,"tokens_out":1603,"would_cite":false,"duration_ms":17156,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A diffusion model can learn a facilitator's social gestures, and pose-keypoint images condition it more accurately than raw 360-degree frames.","keywords":["diffusion models","behavior cloning","pose generation","social interaction","facilitator","MPJPE","imitation learning","image conditioning"],"falsifier":"Re-train the same two models with an explicitly session-disjoint split, putting all frames from held-out facilitator sessions into evaluation only, and compare MPJPE; if the plotted-image advantage shrinks below the reported 9.8–54% gaps or reverses, the paper's central claim is not supported.","tokens_in":6685,"feed_emoji":"🤖","tokens_out":8343,"duration_ms":76059,"temperature":0.7,"pith_summary":"This paper tries to establish that diffusion-based behavior cloning, previously applied to game-playing and robotic control, can also replicate the nonverbal pose behavior of a human facilitator in a multi-party social interaction. The paper compares two conditioning inputs for the same model: raw 360-degree camera frames and images made by plotting extracted pose keypoints on a white background. Across three facilitator sessions, the pose-only plotted images yield lower mean per-joint position error (MPJPE), with improvements of about 10%, 50%, and 54% relative to raw images. The cost is 8.1% to 18.9% higher processing time per frame, and neither variant runs in real time. A sympathetic reader would care because a pose-only conditioning signal is simpler, more portable across camera setups, and slightly more accurate for generating social cues.","feed_headline":"Plotted pose images cut social-pose error by up to 54%","feed_subtitle":"Pose-only conditioning lowers joint-position error in every facilitator session, with the largest gain on subtle movements.","key_machinery":"The central object is the conditioning observation fed to the denoising network: either a raw 360-degree frame downsampled to 128×128 pixels, or an image of the same size made by plotting the extracted pose keypoints of all six participants on a white background. The actions the model must predict are the facilitator's per-frame joint displacements ($\\Delta x$, $\\Delta y$). The machinery carrying the claim is the diffusion behavior-cloning pipeline from reference [6]—a transformer-based denoising network sampled with the Diffusion-X algorithm using 50 denoising timesteps and 8 refinement steps—because it is the fixed component across both conditioning conditions. The plotted image is what strips away scene noise and lets the model attend to movement cues, which is why the paper treats it as the source of the accuracy gain.","core_discovery":"On the paper's own terms, the central discovery is that conditioning a transformer-based diffusion behavior-cloning model on plotted pose-keypoint images—instead of on the raw equirectangular 360-degree frames—reduces mean per-joint position error (MPJPE) for the facilitator's predicted joint displacements in all three tested sessions. The improvement is 9.82% for the teacher, 50% for the music teacher, and 54.19% for the musician. This accuracy gain comes with an 8.1% to 18.9% increase in per-frame processing time, which the paper attributes mainly to pose-keypoint extraction for every participant. The paper interprets the result as evidence that the feature-extracted pose representation removes background and visual noise, allowing the diffusion model to capture subtle nonverbal cues like small wrist movements.","pith_inferences":["A natural stress test, not run in the paper, would be a session-disjoint train/eval split; if the plotted-image advantage shrinks or disappears when training and evaluation frames come from different facilitator sessions, the reported MPJPE partly measures memorization.","The pose-only representation could make the approach usable in privacy-sensitive settings where raw RGB video is not available, since the model never sees the original scene; the paper does not make this claim.","Because the plotted images differ from raw frames in both content (skeleton only) and apparent cleanliness, the comparison does not isolate the pose-information channel; a control with keypoints plotted onto cluttered backgrounds would separate those factors.","The wrist-distribution analysis suggests a tuning principle: facilitators whose cues are subtle and low-range may gain the most from pose-only conditioning, which could inform which social roles are best suited to this method."],"forward_implications":["For a fixed diffusion behavior-cloning architecture, switching from raw RGB frames to plotted pose keypoints improves MPJPE in every session tested, with the largest gains on the facilitator whose wrist movements are most subtle.","The preprocessing stage, not the diffusion sampling, is the main source of the added latency, so optimizing or hardware-accelerating pose-keypoint extraction is a direct path toward real-time social gesture generation.","Because neither conditioning variant nears real-time performance (about 530–630 ms per frame), real-world robot facilitation with this model would require either a faster denoiser, fewer timesteps, or a smaller observation size.","The size of the plotted-image advantage depends on the movement distribution of the facilitator: the teacher's broad-range motions benefit least (9.82%) and the musician's subtle motions benefit most (54.19%)."],"supporting_citations":[{"why":"Supplies the diffusion behavior-cloning model, the transformer denoising network, and the Diffusion-X sampling algorithm that the paper adapts to social pose generation.","marker":"[6]"},{"why":"Provides the FUMI-MPF multiparty facilitation dataset from which the three facilitator sessions are drawn.","marker":"[20]"},{"why":"Motivates the facilitator role in social interactions and gives the example of a robot supporting group discussions.","marker":"[12]"},{"why":"Establishes the denoising diffusion probabilistic model formalism underlying the generative process.","marker":"[5]"},{"why":"Supports the choice of mean per-joint position error (MPJPE) as the standard metric for evaluating predicted poses.","marker":"[21]"},{"why":"Supports the combination of diffusion models with behavior cloning to improve robustness over standard supervised behavior cloning.","marker":"[17]"}],"fun_headline_variants":["Pose-image conditioning cuts social-pose error by 54%","Diffusion behavior cloning yields 54% lower social pose error","Pose keyframes cut error 54%, but cost up to 19% more time","Pose-image diffusion model cuts social error up to 54%","Trade speed for social pose accuracy: 54% error drop"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim depends on the 80-20 train/eval split being session-disjoint, because if frames from the same facilitator session appear in both training and evaluation, the reported MPJPE partly measures memorization rather than generalization to new interactions.","fun_headline_variants_meta":{"raw":{"variants":["Pose-image conditioning cuts social-pose error by 54%","Diffusion behavior cloning yields 54% lower social pose error","Pose keyframes cut error 54%, but cost up to 19% more time","Pose-image diffusion model cuts social error up to 54%","Trade speed for social pose accuracy: 54% error drop"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001217,"raw_usage":{"total_tokens":5024,"prompt_tokens":980,"completion_tokens":4044,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":3950}},"tokens_in":596,"tokens_out":4044,"duration_ms":27653,"temperature":1.0,"reasoning_tokens":3950,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:52:37.877368+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-train the same two models with an explicitly session-disjoint split, putting all frames from held-out facilitator sessions into evaluation only, and compare MPJPE; if the plotted-image advantage shrinks below the reported 9.8–54% gaps or reverses, the paper's central claim is not supported.","supporting_citations":[{"cited_title":"Who to teach a robot to facilitate multi-party social interactions?","cited_arxiv_id":null,"evidence_quote":"Provides the FUMI-MPF multiparty facilitation dataset from which the three facilitator sessions are drawn."},{"cited_title":"Robot-supported collaborative learning (rscl): Social robots as teaching assistants for higher education small group facilitation,","cited_arxiv_id":null,"evidence_quote":"Motivates the facilitator role in social interactions and gives the example of a robot supporting group discussions."},{"cited_title":"Deep 3d human pose estimation: A review,","cited_arxiv_id":null,"evidence_quote":"Supports the choice of mean per-joint position error (MPJPE) as the standard metric for evaluating predicted poses."}],"review_version":1}