{"id":"d29afe0b-3472-4a31-a520-17d5b4a48f5e","arxiv_id":"2501.10857","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Using Implicit Behavior Cloning, the authors report a gaze-generation model for robotic facilitators that achieves higher target-reaching success (96% vs 93%) and smoother trajectories than an MSE behavior cloning baseline.","lead":"This paper applies an existing imitation learning method, Implicit Behavioral Cloning, to generate a human facilitator's gaze movements in group conversations. It reports that the implicit method reaches the correct gaze endpoints more often and produces smoother trajectories than a standard behavior cloning baseline, though the matched trajectory fit is worse.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Action/setpoint mismatch: policy outputs gaze deltas but environment expects setpoints, an undocumented conversion that ASM results depend on.","rationale":"The reader's weakest_assumption concerned the suitability of the PD-controlled particle environment as a simulator of human gaze dynamics. I agree that this is a concern, but I identify a more immediate and concrete internal issue: the paper's description contains a contradiction between the action representation (delta) and the environment's stated setpoint semantics. This is not merely about external validity; it is about whether the reported metrics are computed correctly at all. The reader's conditional verdict already flags the need for more detail, and my concern reinforces that condition. The correct test is to inspect the implementation and re-run the evaluation with a clearly documented conversion. If the conversion is as I suspect (delta added to current position), the paper likely works, but this must be stated; if not, the ASM results are invalid. Since the paper neither provides code nor describes this conversion, the claim is not fully verifiable. I therefore keep the reader's CONDITIONAL verdict unchanged, while sharpening the specific technical condition that must be met.","tokens_in":7488,"tokens_out":7705,"duration_ms":80197,"concrete_test":"Inspect or request the environment code from the authors. Verify how the policy output is converted into a PD controller setpoint: check whether the code computes setpoint = current_position + action (delta semantics) or setpoint = action directly (absolute semantics). Then re-run the ASM evaluation with the correct, documented conversion. If the reported values (IBC 0.96, MSE 0.93) change materially, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section II-B0b states that expert actions are 'the differences in the facilitator’s pitch and yaw between consecutive frames,' i.e., a delta. Section II-C3 states the environment 'receives actions from the policy as setpoints for the particle to achieve.' A delta is not a setpoint; either the environment adds the delta to the current gaze position to form a setpoint (s_t = g_t + δ), or the policy's output is actually an absolute target. The paper never specifies this conversion. This is load-bearing because the average success metric (ASM) — the central evidence for IBC outperforming MSE (0.96 vs 0.93) — requires the environment to move the particle toward the goal. If the delta were directly used as an absolute setpoint, the particle would be commanded to a tiny offset near zero and the success rate would be near 0, not 0.96. The reported numbers therefore presuppose an undocumented interpretation. Without this detail, the evaluation pipeline is not reproducible and the central claim is not verifiable from the text.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript reports a replication of Implicit Behavior Cloning (IBC), originally proposed by Florence et al. (CoRL 2022), applied to the generation of nonverbal gaze cues for a robotic facilitator in multiparty social interactions. Using a subset of the FUMI-MPF dataset with teacher, musician, and music-teacher facilitators, the authors train an energy-based IBC policy and an explicit MSE behavior-cloning policy with the same MLP backbone and a PD-controlled particle environment. They evaluate both policies with the average success metric (ASM), R2, and SPARC, and report that IBC outperforms MSE on ASM (0.96 vs. 0.93) and produces smoother trajectories according to SPARC, while MSE achieves higher R2 values for trajectory fit. The paper concludes that IBC was successfully replicated and extended to gaze generation for social interaction tasks.","tokens_in":7719,"tokens_out":6563,"duration_ms":68629,"significance":"If the central comparison is reliable, the paper provides a useful data point: IBC can be transferred from the original particle-control tasks to a real social-signal domain, and the authors honestly report a metric (R2) on which the MSE baseline wins. The manuscript is also transparent about the absence of human evaluators and about the challenges of extending the method to pose generation. However, the current evidentiary value is weakened by an unspecified action-to-setpoint conversion and by the complete absence of variance or significance information, so the central claim that IBC outperforms MSE on the success metric is not yet fully supported.","major_comments":[{"comment":"Section II-B0b defines expert actions as 'the differences in the facilitator’s pitch and yaw between consecutive frames' and later as gaze velocity, while Section II-C3 says the environment 'receives actions from the policy as setpoints for the particle to achieve.' A delta (or velocity) is not a setpoint, and the paper never states whether the policy output is added to the current gaze position to form the target setpoint or is itself treated as an absolute target. This ambiguity is load-bearing: the ASM values in Table I (0.96 vs. 0.93) presuppose that the particle is commanded toward the correct target, and the opposite interpretation would make the reported success rates impossible. Please specify the exact conversion and verify that it was applied identically in training and inference.","section":"II-B0b, II-C3"},{"comment":"Section III-1 and Table I report ASM values as point estimates, but the paper gives no standard deviation, confidence interval, per-fold breakdown, or statistical test. Given the small dataset described in Section II-B0a (seven sessions, two-fold validation), the 0.03 gap between IBC and MSE could easily arise from sampling noise. Please provide per-session or per-fold numbers and, if appropriate, a paired significance test; alternatively, state explicitly that the difference is not statistically evaluated.","section":"III-1, II-E2"},{"comment":"Section II-D1 and II-D2 state that hyperparameters are taken from the public repository of [11], but no values are listed (learning rate, MCMC step count, noise schedule, batch size, layer sizes, dropout rate). Because this is a replication study, one of the central contributions is verifying that the method works under the original configuration; without these details, the reader cannot reproduce the training or assess whether the 'replication' used settings comparable to Florence et al. Please include a hyperparameter table and the exact MLP architecture.","section":"II-D1, II-D2"},{"comment":"Section II-E2 introduces R2 as a measure of how well predicted trajectories match ground truth, but the text does not define what is being compared: policy outputs, integrated gaze trajectories, or rolled-out environment states. The very low yaw R2 values in Table II (e.g., 0.13 for Teacher IBC) are hard to interpret without this definition, and Section III-2 uses them to qualify the ASM results. Please state the precise R2 computation and the units of the underlying signals.","section":"II-E2, III-2"}],"minor_comments":[{"comment":"The 'Reported' column in Table I is not defined in the text; state that it refers to the ASM reported by Florence et al. and explain why that comparison is meaningful for a different task.","section":"Table I"},{"comment":"The expression '{amax, amin}' should be written as an interval [amin, amax]; the sentence about clipping is also ambiguous and should specify whether actions are clipped before or after the Langevin updates.","section":"II-D1"},{"comment":"Since the SPARC values in Table III are negative, the phrase 'values closer to zero are smoother' should clarify whether the displayed values are raw SPARC scores or a scaled variant.","section":"III-3"},{"comment":"References [16] and [18] are duplicate entries for the same L2CS-Net paper; keep one and cite consistently in the text.","section":"References"},{"comment":"Reference [6] is incomplete: 'F. Agent, An agent that facilitates crowd discussion' has no author list, no year, and no publication venue.","section":"References"},{"comment":"The Langevin update equations use both a¯i and a−i for negative samples; unify the notation to avoid confusion.","section":"II-D1, II-E1"}],"recommendation":"major_revision","confidential_remarks":"I found no circularity in the evaluation: the models are trained on expert demonstrations and evaluated against held-out setpoints, so the stress-test concern about action/setpoint conversion is a genuine reproducibility gap rather than evidence of bad faith. The manuscript fits the scope of a robotics venue, and the replication framing is valuable if the requested details are added. I would not require new data collection; the paper can be made acceptable by clarifying the action-to-setpoint conversion, adding variance or significance information, and reporting the exact hyperparameters and R2 definition."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a small, honest replication study. It applies Florence et al.'s implicit behavior cloning to facilitator gaze in a multiparty dataset and reports that IBC beats an MSE baseline on average success (0.96 vs 0.93) and produces smoother trajectories per SPARC. The contribution is mainly empirical—a new domain for IBC, not new algorithms. It will be useful to HRI practitioners who want to try IBC, but there are a few soft spots that need attention before the claims are solid.\n\nWhat's good: (1) Real human demonstration data from the FUMI-MPF dataset, not synthetic. (2) Three evaluation metrics, with honest reporting that MSE matches ground truth better on R2 while IBC wins on success and smoothness. (3) Transparent about the failed pose extension. (4) Use of the public IBC codebase helps replication.\n\nThe main issue: the action representation. Section II-B0b says expert actions are the differences in facilitator pitch/yaw between consecutive frames, i.e., deltas. Section II-C3 says the environment receives actions from the policy as setpoints for the particle to achieve. A delta is not a setpoint. The paper never states whether the delta is added to the current gaze position to form a setpoint or whether the policy actually outputs absolute targets. This matters because the average success metric depends on the particle reaching a target; if a delta were fed directly as an absolute setpoint, success would be near zero. So the reported 0.96 ASM presupposes an undocumented conversion. The authors likely have a working pipeline, but the experiment is not reproducible from the text as written, and the central claim cannot be verified.\n\nOther soft spots: no error bars or statistical tests, so the 0.96 vs 0.93 gap could be noise. The dataset description is confusing—seven sessions for gaze and nine additional for pose, but it's unclear how they split for the main results. Hyperparameters come from the repo rather than the paper, which is fine, but should be stated explicitly. No comparison to prior gaze generation models, so we don't know if IBC is better than simpler baselines for this specific facilitation task.\n\nOverall, this is a decent workshop-level paper with a real empirical result, but the missing conversion detail is a blocker for full verification. A serious referee could pin this down, and then it would be a solid contribution to HRI practice. I'd send it to review with the expectation of major revisions.","headline":"A modest replication of IBC for gaze generation in social robotics, with a real reproducibility gap in the action-to-setpoint conversion that undercuts the headline success numbers.","tokens_in":8211,"tokens_out":3338,"would_cite":false,"duration_ms":32653,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that Implicit Behavior Cloning can be replicated to generate a facilitator's gaze cues in multiparty social interactions, outperforming a standard behavior-cloning baseline on success and smoothness while fitting ground…","keywords":["Implicit Behavior Cloning","nonverbal cue generation","gaze behavior","robotic facilitator","multiparty interaction","behavior cloning","energy-based model","human-robot interaction"],"falsifier":"Re-run the same training and evaluation with the PD controller replaced by a head-dynamics model calibrated to human neck movement, or with a physical robot head, and compare which policy wins the success metric. If the implicit policy no longer beats the MSE baseline, the reported replication rests on the simulator rather than on gaze behavior.","tokens_in":7345,"feed_emoji":"🤖","tokens_out":5334,"duration_ms":53148,"temperature":0.7,"pith_summary":"The paper tries to show that Implicit Behavior Cloning (IBC), an energy-based behavior-cloning technique, transfers from simulated particle control to a real social setting: generating the gaze movements of a human facilitator leading a multiparty conversation. The authors train both IBC and a standard mean-squared-error behavior-cloning baseline on recorded sessions with teacher, musician, and music-teacher facilitators, then compare how often each policy reaches the observed gaze target and how smooth the resulting gaze trajectories are. They report that IBC reaches the target more often on average (0.96 vs 0.93) and produces smoother pitch and yaw trajectories by the SPARC metric, while the baseline fits the ground-truth trajectory better on $R^2$. The point of the exercise is that a robot could take over the facilitator's nonverbal role when no human facilitator is present.","feed_headline":"Implicit cloning beats baseline at generating facilitator gaze","feed_subtitle":"A replication study finds the energy-based policy hits more gaze targets and produces smoother trajectories in multiparty sessions.","key_machinery":"The load-bearing machinery is Implicit Behavior Cloning (IBC): an energy-based model $E_\\theta(o,a)$ scores observation-action pairs, and at inference an initial action sampled from a uniform distribution is refined by Langevin MCMC gradient updates to find a low-energy action. Training uses negative action samples and an InfoNCE loss. The policy outputs setpoints that a PD controller tracks in a particle environment adapted from the original IBC paper, with episodes of 50 frames and the facilitator's gaze target at frame 50. The same MLP, environment, and evaluation metrics are used for the MSE baseline so that the comparison isolates the implicit versus explicit policy update.","core_discovery":"The paper's central claim is that Implicit Behavior Cloning, an energy-based method that refines candidate actions by gradient-based sampling, can be successfully replicated for generating nonverbal gaze cues from real multiparty facilitation data. Using gaze pitch and yaw extracted from recorded sessions with three facilitator types, the authors report an average success metric of 0.96 for IBC versus 0.93 for an MSE behavior-cloning baseline, aligning with the 0.95 success level reported in the original IBC study. On SPARC smoothness, IBC values are closer to zero for both pitch and yaw, which the paper interprets as more natural motion. The paper acknowledges that on $R^2$ the MSE baseline better matches the ground-truth trajectories (0.76 vs 0.73 average pitch; 0.56 vs 0.41 average yaw), and explains this as IBC reaching the correct final gaze position by a different, smoother route.","pith_inferences":["An implication the paper leaves implicit is that the same energy-based refinement could be applied per joint to overcome the pose-generation failure, since the paper attributes that failure to PD overshoot and action-space complexity rather than to the implicit objective itself.","A testable extension would separate controller effects from policy effects by re-running the success metric with different setpoint-tracking gains; if the IBC advantage disappears under a more realistic head-dynamics model, the reported gain is partly an artifact of the simulator.","The unexplained musician-facilitator advantage could be probed by inspecting per-session action ranges and gaze variability; the paper notes it has not investigated why that session type learns more easily."],"forward_implications":["A robot facilitator could use IBC to choose where to look, succeeding at hitting observed gaze target directions at about the same rate as the original simulated IBC results.","Because IBC-produced gaze trajectories are smoother under SPARC, generated nonverbal behavior may appear more natural in human-robot interaction, although the paper does not yet test human perception.","The comparison shows that an explicit MSE baseline can fit ground-truth trajectories more tightly on $R^2$ while still missing goals more often, so the choice of evaluation metric changes which method looks better.","Extending the same method to full body pose is not yet viable: the paper reports that pose generation required an unreasonably high success threshold and lost humanoid quality due to controller overshoot and action-space complexity."],"supporting_citations":[{"why":"Supplies the IBC and MSE policy architectures, the Langevin training and inference procedure, the particle environment, and the average success metric this paper replicates.","marker":"[11]"},{"why":"Provides the recordings of teacher, musician, and music-teacher facilitators that form the custom training and evaluation dataset.","marker":"[17]"},{"why":"Estimates pitch and yaw gaze for every participant in each frame, producing the observations and actions used for training.","marker":"[18]"},{"why":"Underlies the energy-based model and the Langevin MCMC sampling used to refine candidate actions.","marker":"[20]"},{"why":"Defines the InfoNCE loss used to train the energy model against negative action samples.","marker":"[21]"},{"why":"Provide the SPARC metric used to measure trajectory smoothness and support the interpretation of smoother motion as more natural.","marker":"[13]–[15]"}],"fun_headline_variants":["IBC beats MSE at robot gaze in multiparty sessions","Replicating implicit cloning for facilitator gaze generation","Energy-based policy outshines baseline for social robot gaze","Implicit cloning hits gaze targets, smoother than MSE baseline","Study replicates IBC for nonverbal cues, beats MSE on success"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results hinge on the assumption that the particle-environment controller, which steers a simulated point toward setpoint goals, is a faithful enough model of real human head and gaze motion for success in that environment to count as good gaze behavior.","fun_headline_variants_meta":{"raw":{"variants":["IBC beats MSE at robot gaze in multiparty sessions","Replicating implicit cloning for facilitator gaze generation","Energy-based policy outshines baseline for social robot gaze","Implicit cloning hits gaze targets, smoother than MSE baseline","Study replicates IBC for nonverbal cues, beats MSE on success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1419,"prompt_tokens":1008,"completion_tokens":411,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":331}},"tokens_in":624,"tokens_out":411,"duration_ms":4703,"temperature":1.0,"reasoning_tokens":331,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:54:28.728776+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same training and evaluation with the PD controller replaced by a head-dynamics model calibrated to human neck movement, or with a physical robot head, and compare which policy wins the success metric. If the implicit policy no longer beats the MSE baseline, the reported replication rests on the simulator rather than on gaze behavior.","supporting_citations":[{"cited_title":"Implicit behavioral cloning,","cited_arxiv_id":null,"evidence_quote":"Supplies the IBC and MSE policy architectures, the Langevin training and inference procedure, the particle environment, and the average success metric this paper replicates."},{"cited_title":"L2cs-net: Fine-grained gaze estimation in unconstrained environments,","cited_arxiv_id":null,"evidence_quote":"Estimates pitch and yaw gaze for every participant in each frame, producing the observations and actions used for training."},{"cited_title":"Implicit generation and modeling with energy based models,","cited_arxiv_id":null,"evidence_quote":"Underlies the energy-based model and the Langevin MCMC sampling used to refine candidate actions."}],"review_version":1}