{"id":"6f06e5a3-cf33-4327-882d-ea6753f0f7a5","arxiv_id":"2507.20445","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Robots with different bodies can learn human interaction skills by imitating a compact graph representation of the demonstration, rather than matching exact joint angles.","lead":"This paper introduces BuddyImitation, a framework that lets robots with very different body shapes learn human interaction skills like handshaking and sparring from video demonstrations. It works by compressing the interaction into a small 'interaction graph' and then training the robot to match that graph in a physics simulation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Vertex correspondence in §3.3.1 may not preserve semantic graph roles for embodiments with fewer or differently arranged end-effectors; without reported mappings, the transfer reward could drive the wrong body parts.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the automatic vertex correspondence in Section 3.3.1 may fail to preserve the semantic roles of interaction edges. This is indeed the most critical link in the argument. The entire transfer module depends on the correspondence to define the interaction consistency reward; if the mapping is wrong, the policy is trained toward incorrect graph features and the central claim of cross-embodiment semantic transfer collapses. The paper provides no evidence—quantitative or qualitative—that the neutral-pose inner-product heuristic yields semantically correct mappings for the reported embodiments, and the heuristic is especially suspect for Stretch and Go2Ar, which have fewer or differently arranged end-effectors than a human. This is a concrete, testable gap rather than a vague concern about reproducibility. The lack of code/data and the weak user-study statistics are secondary; they affect verification but not the internal soundness of the method. A conditional acceptance is appropriate because the concern can be resolved by reporting the actual vertex mappings and/or running a comparison against manually specified semantic correspondences. If the mappings turn out to be semantically incorrect, the verdict should be downgraded; if they are correct, the framework may be sound. Thus the reader's conditional verdict remains unchanged.","tokens_in":15440,"tokens_out":8789,"duration_ms":95524,"concrete_test":"Ask the authors to report, for each reported embodiment (Go2Ar, Stretch, child) and interaction type (handshaking, sparring, dancing), the actual vertex correspondence produced by Section 3.3.1 for every vertex present in the embedded graphs. Verify explicitly that each human arm/hand vertex in the embedded graph maps to the robot's manipulator/gripper (or an explicitly justified equivalent), not to a paw, base, or unmapped vertex. Additionally, run the transfer with a manually specified semantic correspondence (human arms to the robot manipulator, legs to appropriate legs) and compare the learned behavior, interaction consistency reward, and user-study ratings against the automatic mapping. If the manual mapping changes the learned behavior or improves semantic alignment, the automatic correspondence is not preserving semantics.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The transfer pipeline's only task-specific training signal is the interaction consistency reward (Eqs. 8–12), which requires mapping the human embedded interaction graph onto each new embodiment. Section 3.3.1 proposes an automatic vertex assignment based on the inner product of neutral-pose root-to-end-effector vectors, with remaining vertices assigned by 'relative sequence to the end-effectors.' This heuristic is not guaranteed to preserve semantic roles when the embodiment has a different number of end-effectors or a different kinematic topology. For the Stretch robot (one gripper, two wheels) and the Go2Ar (four paws plus one gripper), the human graph can include both left- and right-arm vertices. Because each new-character end-effector can be assigned to only one human end-effector, at least one arm vertex must map to a paw/base or remain unmapped, yet the paper never specifies how such vertices are handled or reports the actual correspondences used. If a human arm vertex maps to a paw or base, then whenever the embedded graph contains an edge incident to that vertex—e.g., during the left-arm phase of sparring—the reward will drive the robot to move the wrong body part, and the policy will be optimized toward incorrect interaction semantics. A failure here directly invalidates the central claim of semantic cross-embodiment transfer.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents BuddyImitation, a two-stage framework for learning physical interaction skills (dancing, handshaking, sparring, rock-paper-scissors, circling) from human-human demonstration data and transferring them to agents with substantially different morphologies (humanoid, child, Go2Ar legged manipulator, Stretch mobile manipulator). In the Interaction Embedding stage, a sparse Embedded Interaction Graph (EIG) is learned via multi-head cross-attention edge selection and future-pose prediction, and is evaluated quantitatively against random-graph and full-graph baselines. In the Interaction Transfer stage, the EIG is used to define an interaction-consistency reward that guides reinforcement learning of a centralized hierarchical control policy for new embodiments. The transfer results are presented primarily through qualitative video stills, descriptive analysis, and a user study that compares the method to an IK-based retargeting baseline.","tokens_in":15664,"tokens_out":4634,"duration_ms":48774,"significance":"If the claims are substantiated, the paper makes a valuable contribution to cross-embodiment imitation learning: it proposes a compact, interpretable, and sparse graph representation of interaction dynamics that is explicitly designed to be transferable across morphologically distinct agents. The learned EIG is shown to be more predictive than random or fully connected graphs in the embedding module, which is a concrete, reproducible quantitative result. The idea of using the learned graph as an imitation objective rather than raw joint positions is novel and potentially impactful for robotics and character animation. However, the central claim of the paper—semantic cross-embodiment transfer—currently rests on qualitative demonstrations and a user study lacking statistical details, and on a vertex-correspondence heuristic that is not adequately validated. The framework is well motivated and the two-stage architecture is sensible, but the evidence for the main claim needs strengthening.","major_comments":[{"comment":"The automatic vertex correspondence procedure is underspecified and may not preserve the semantic roles of interaction graph vertices when the new embodiment has a different number or arrangement of end-effectors. For example, human demonstrations have two arm end-effectors and two leg end-effectors, while Go2Ar has four paws and one gripper and Stretch has one gripper and two wheels. The text states that 'each end-effector in the new character is assigned to a unique end-effector in the embedded graph' but does not specify what happens when the embedded graph contains end-effectors with no counterpart in the new character, nor how the 'relative sequence to the end-effectors' rule assigns non-end-effector vertices such as the head or spine. If a human arm vertex is mapped to a paw or base, then any edge incident to that vertex in the embedded graph will drive the reward (Eqs. 8–12) toward moving the wrong body part, directly undermining the claimed semantic transfer. The paper never reports the actual vertex correspondences used in the experiments. Please provide the full mappings for each embodiment, describe how unmatched graph vertices are handled, and ideally add an analysis or ablation that verifies the assigned correspondences lead to the intended interaction semantics.","section":"§3.3.1"},{"comment":"The user study is the only quantitative evaluation of the Interaction Transfer module, yet the paper reports no error bars, no sample sizes per condition, and no statistical tests. The claim that 'our method significantly outperforms the baseline in both activity recognition and semantic consistency' is not backed by any p-values, confidence intervals, or effect sizes. This is a load-bearing gap because the central contribution of the paper is cross-embodiment interaction transfer, not just the pose-prediction accuracy of the embedding module. Please provide a full statistical analysis of the user study results, and consider supplementing it with an objective measure of interaction consistency—for example, the time-varying distance between the reference and generated embedded graph features (dl_t, ded_t, dcp_t)—so that the transfer quality can be assessed without relying solely on subjective ratings.","section":"§1.2, Figure 4D"},{"comment":"The normalization lengths L_hat and L in the length and center-point metrics are described only as 'morphology-dependent length values ... predefined parameters defined according to the morphology of the agent.' No values or derivation are given for any of the four embodiments, even though these quantities directly scale two of the three terms in the interaction consistency reward and therefore affect the learned behavior. If these are hand-tuned, please report the values and justify them; if they are computed from the morphology (e.g., a characteristic body dimension), state the formula. Without this information, the reward design—and hence the transfer results—cannot be reproduced or properly assessed.","section":"§3.3.2, Eqs. (9) and (11)"}],"minor_comments":[{"comment":"Typo: 'wholebbody' should be 'whole-body'.","section":"Abstract"},{"comment":"Typo: 'correspondingtod' should be 'corresponding to'.","section":"§1.1"},{"comment":"Typos: 'neural poses' should be 'neutral poses'; 'crossponding' should be 'corresponding'; 'elaboration' should be 'elaborate'; 'consistancy' should be 'consistency'.","section":"§3.3.1"},{"comment":"The quantitative comparison of prediction error across graph configurations is described only in text; adding exact error values and error bars to the figure or a table would improve clarity and reproducibility.","section":"Figure 3d"},{"comment":"The KL divergence weight β is said to be tuned, and the chosen value β = 0.3 is reported, but no sensitivity results are shown; a brief statement of the range explored would be helpful.","section":"§3.2.3"},{"comment":"Reference [68] is attributed to 'Authors, G.'; the citation should be updated to the actual authors of the Genesis environment.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting and timely problem, and the EIG representation is a plausible step toward interpretable cross-embodiment interaction learning. The main risk is that the central transfer claim is not yet convincingly validated: the vertex-correspondence heuristic in §3.3.1 could be silently assigning wrong semantic roles, and the only quantitative transfer evaluation is a user study with no statistics. Both issues are testable and fixable, but they need to be resolved before the paper can be accepted. I recommend major revision rather than rejection because the core architecture and the embedding-module results are sound and the missing evidence is within the manuscript's scope to provide."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing you should know: BuddyImitation is a real step forward for cross-embodiment interaction learning. It learns a sparse, interpretable interaction graph from human demonstrations and uses it as an RL reward to train policies for non-humanoid robots. The individual pieces (Interaction Graph, MVAE, CrossLoco-style RL) are known, but the combination—learning which edges matter and transferring that semantics across morphologies—is new and clearly useful.\n\nThe embedding stage is the strongest part. The paper quantifies pose prediction against a random graph of the same size and a fully connected graph, and shows the learned 4-edge graph beats both. The ablations (E0, E1, E4, E8, EF) are well designed and show the sparsity tradeoff. That part earns its keep.\n\nThe soft spot is exactly where the reader put it: the transfer stage is evaluated by videos and a user study without error bars or tests. The user study (50 participants) reports \"significantly outperforms\" but gives no standard deviations or p-values. That is fixable in revision. The automated vertex correspondence in §3.3.1 is the load-bearing heuristic for the whole transfer reward, and the paper never reports the actual mappings for Stretch or Go2Ar. If an arm vertex maps to a paw or base, the reward silently drives the wrong body part. The qualitative videos suggest it usually works—Go2Ar switching from left to right arm based on the dominant sparring hand is a nice piece of evidence—but \"usually\" is not a metric. The authors should provide quantitative interaction metrics (e.g., edge-feature errors over rollouts) and at least a table of correspondences.\n\nThe circularity concern is minor. The graph is learned on the demonstrations and then used as an objective; that is standard reward design, not a hidden derivation of the result.\n\nOverall: this is a promising robotics/animation paper with a sound core idea and a clear gap in evaluation. It deserves a serious referee—the novelty is there and the problems are addressable. I'd like to see it in a reading group, and I'd cite it if I work on cross-embodiment imitation.\n\nRecommendation: send to peer review, with a request for transfer-stage metrics and code/data release.","headline":"A genuinely useful framework for cross-embodiment interaction learning, with the main transfer claim under-evidenced by qualitative-only evaluation.","tokens_in":16222,"tokens_out":2174,"would_cite":true,"duration_ms":23277,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a sparse, learned Embedded Interaction Graph can carry the semantics of two-person physical interactions from human demonstrations to robots with very different bodies, and that training policies to reproduce this…","keywords":["cross-embodiment imitation","learning from demonstration","interaction graph","motion retargeting","physics-based reinforcement learning","multi-agent coordination","interaction semantics","sparse attention"],"falsifier":"Retrain one transferred interaction, such as Go2Ar handshaking, with the automatic vertex correspondence replaced by a deliberately permuted mapping that assigns the demonstrator's right-hand graph edge to the robot's left manipulator or rear leg. If the resulting policy still produces a recognizable handshake, the correspondence assumption is not load-bearing; if the behavior loses its interaction semantics or collapses, the assumption is the critical link.","tokens_in":15195,"feed_emoji":"🤝","tokens_out":7431,"duration_ms":71528,"temperature":0.7,"pith_summary":"Physical interaction skills such as handshaking, sparring, and dancing are difficult to transfer from human demonstrations to robots because a quadruped or mobile manipulator does not share the demonstrator's skeleton. This paper proposes BuddyImitation, a two-stage framework that compresses a two-person interaction into a sparse, time-varying Embedded Interaction Graph and then trains a reinforcement-learning policy to reproduce that graph in a physics-based simulator. The central claim is that this graph, rather than raw joint trajectories, is the right thing to imitate: it selects the few inter-agent edges that predict future motion, so a one-armed robot can switch between imitating the demonstrator's left and right arms and adapt handshake height to its partner. If the claim is correct, cross-embodiment imitation can proceed without handcrafted interaction objectives or matching skeletons.","feed_headline":"Robots copy human handshakes via a sparse interaction graph","feed_subtitle":"The graph keeps the meaning of a handshake or sparring match while each robot adapts it to its own body.","key_machinery":"The load-bearing object is the Embedded Interaction Graph (EIG), a sparse, time-indexed subgraph of the Interaction Graph in which nodes are joints and edges carry a 6D feature describing the relative position and midpoint of paired joints between the two characters. The embedding stage selects one edge per attention head via hard attention between the character's current pose and all edge embeddings, so the chosen edges are those most predictive of the next pose when passed through a pretrained motion decoder. The transfer stage converts this graph into an imitation objective through an interaction consistency reward on normalized edge length, root-edge XY direction, and edge center-point height, which rewards a new character for reproducing the relational geometry of the reference interaction rather than its joint angles.","core_discovery":"On the paper's own terms, the discovery is that interaction semantics can be distilled into a time-varying sparse graph $G_{\\mathrm{emb}}^t$, a subset of the fully connected Interaction Graph, whose edges are chosen by multi-head cross-attention to minimize future-pose prediction error. The paper shows that this embedded graph predicts the next 120 frames of a demonstration more accurately than random sparse graphs or the full 484-edge graph, and that the selected edges shift with context: right-arm and root edges dominate handshaking, mirrored hand edges appear in circling, and alternating dominant-arm edges appear in sparring. It then uses the graph as the reward signal for a centralized hierarchical policy, with an interaction consistency reward that compares normalized edge lengths, root-edge direction, and edge center-point heights. Reported results show Go2Ar, Stretch, and humanoid agents producing recognizable interaction patterns while adapting limb use and posture to their own bodies.","pith_inferences":["Not claimed by the paper: because the selected edges are semantically interpretable, the same graph could be used as an explanation of what an interaction is about, not only as a control objective for a robot.","The paper trains separate policies per interaction and character pair; a natural extension it leaves implicit is conditioning one policy on the interaction identity, which would require a shared observation space across embodiments.","One testable extension is to feed the time-varying graph attention into a downstream task planner, so that the interaction semantics can be re-targeted online when a partner changes body mid-interaction; the paper does not address online switching.","The vertex-correspondence method is based on neutral-pose geometry, so a stronger test would be to evaluate it on embodiments with asymmetric or redundant limbs, where the inner-product mapping has multiple plausible answers."],"forward_implications":["A robot with a single manipulator can learn bimanual human interactions by switching its reference limb to whichever demonstrator arm dominates the current interaction state.","Agents adapt the spatial details of an interaction to their partner: handshake height rises or falls with the relative sizes of the two characters.","Training in physics-based simulation makes the policy robust to physically infeasible or noisy demonstration frames, because such frames lead to falls and low rewards and are avoided.","A four-edge embedded graph predicts future interaction poses better than a fully connected graph, supporting the use of sparse representations as imitation objectives."],"supporting_citations":[{"why":"defines the Interaction Graph representation, the 6D edge features, and the fully connected graph that the EIG sparsifies","marker":"[1]"},{"why":"supplies the root-to-end-effector positional alignment used for automatic vertex correspondence between demonstrator and new characters","marker":"[16]"},{"why":"provides the guided unsupervised reinforcement learning formulation and the root-height, orientation, and torque regularization terms","marker":"[20]"},{"why":"provides the two-person human interaction motion dataset used for demonstrations and quantitative evaluation","marker":"[30]"},{"why":"provides the Motion VAE architecture and scheduled-sampling scheme used to pretrain the human motion decoder","marker":"[69]"},{"why":"provides the imitate-and-repurpose pretraining procedure for character-specific low-level policies","marker":"[70]"}],"fun_headline_variants":["Sparse graph lets robots learn handshakes from video","Robots learn to spar and dance via interaction graph","Interaction graph teaches diverse robots human moves","Embedded graph transfers human interaction skills to robots","Robots pick up handshaking from humans via attention graph"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes that the automatic vertex correspondence, computed by inner products of root-to-end-effector vectors in neutral pose, maps each semantic role of the demonstration graph onto the correct body part of the new character; if that mapping is wrong, the interaction consistency reward measures the wrong graph features and the policy is trained toward incorrect semantics.","fun_headline_variants_meta":{"raw":{"variants":["Sparse graph lets robots learn handshakes from video","Robots learn to spar and dance via interaction graph","Interaction graph teaches diverse robots human moves","Embedded graph transfers human interaction skills to robots","Robots pick up handshaking from humans via attention graph"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000401,"raw_usage":{"total_tokens":2078,"prompt_tokens":917,"completion_tokens":1161,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":1086}},"tokens_in":533,"tokens_out":1161,"duration_ms":9179,"temperature":1.0,"reasoning_tokens":1086,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:42:30.600050+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain one transferred interaction, such as Go2Ar handshaking, with the automatic vertex correspondence replaced by a deliberately permuted mapping that assigns the demonstrator's right-hand graph edge to the robot's left manipulator or rear leg. If the resulting policy still produces a recognizable handshake, the correspondence assumption is not load-bearing; if the behavior loses its interaction semantics or collapses, the assumption is the critical link.","supporting_citations":[{"cited_title":"1–11 (2023)","cited_arxiv_id":null,"evidence_quote":"defines the Interaction Graph representation, the 6D edge features, and the fully connected graph that the EIG sparsifies"},{"cited_title":"In: SIGGRAPH Asia 2023 Conference Papers, pp","cited_arxiv_id":null,"evidence_quote":"supplies the root-to-end-effector positional alignment used for automatic vertex correspondence between demonstrator and new characters"},{"cited_title":"International Conference on Learning Representations (2024)","cited_arxiv_id":null,"evidence_quote":"provides the guided unsupervised reinforcement learning formulation and the root-height, orientation, and torque regularization terms"},{"cited_title":"International Journal of Computer Vision (IJCV) (2024)","cited_arxiv_id":null,"evidence_quote":"provides the two-person human interaction motion dataset used for demonstrations and quantitative evaluation"},{"cited_title":"ACM Transactions on Graphics (TOG) 39(4), 40–1 (2020)","cited_arxiv_id":null,"evidence_quote":"provides the Motion VAE architecture and scheduled-sampling scheme used to pretrain the human motion decoder"}],"review_version":2}