{"id":"acd5eaf6-d4e5-43b9-8ed1-3c189989d7f2","arxiv_id":"2501.08946","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A robot using two zero-shot general turn-taking models (TurnGPT and VAP) responds faster and interrupts less than a traditional silence-threshold system in a 39-person user study.","lead":"Researchers wired two AI turn-taking models, trained on human conversations, into a Furhat robot and asked 39 people to chat with it. Compared with a standard silence-detection system, the robot responded faster, interrupted less, and was preferred by participants.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is underdetermined because the proposed system bundles general turn-taking models with response pre-generation, self-monitoring, and a different interruption-handling protocol; the reported gains cannot be uniquely attributed to TurnGPT/VAP.","rationale":"The reader's weakest assumption focused on out-of-distribution reliability of the pretrained models, which is a legitimate generalization concern. My stress-test identifies a more fundamental attribution problem: even if VAP and TurnGPT work perfectly, the current experiment does not demonstrate their contribution because the proposed system differs from the baseline in multiple concurrent ways. The paper itself notes the pre-generation could have been added to the baseline (Section V-B), which shows the authors were aware of one confound but did not control for it. I do not see fraud or fabrication; the engineering result is plausible and well-powered. However, the causal claim that general turn-taking models are responsible is not supported. This warrants keeping the conditional verdict, but the condition should be extended to include a component ablation or explicit weakening of the attribution claim. The concern is concrete and testable, so I recommend CONDITIONAL rather than UNCHANGED with a different rationale.","tokens_in":18550,"tokens_out":1748,"duration_ms":18982,"concrete_test":"Run a component-wise ablation in the same Furhat/LLM setup: (A) baseline plus incremental-ASR response pre-generation but without TurnGPT/VAP (pre-generate on every ASR update using the same semantic-similarity gate), keeping the same silence-based turn-taking and no barge-in; (B) proposed system without pre-generation (TurnGPT/VAP only decide when to start preparing, not mid-utterance). If (A) matches the proposed system's response-time and interruption-rate improvements, the general models contribute little beyond pre-generation. If (B) is no better than baseline, the central attribution fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that general turn-taking models (TurnGPT, VAP) benefit HRI turn-taking. But the proposed system differs from the baseline in several bundled ways: (1) it pre-generates tentative responses from incremental ASR (Section V-B), a mechanism the paper itself acknowledges could have been added to the baseline; (2) it uses self-monitoring by feeding the robot's own TTS back into VAP and the dialogue history into TurnGPT (Section V-D); (3) it uses a different interruption-handling rule (Section V-C); and (4) participants experienced different interaction protocols (explicitly told they could interrupt vs. only speak when the red light is off, Section VI-C). The dramatic response-time reduction (median 1.5 s vs. 2.7 s) is largely explainable by pre-generation: the baseline waits for end-of-speech before starting LLM/TTS, while the proposed system starts preparing responses before the user finishes. The interruption-rate reduction (6.9% vs. 16.6%) is confounded because participants knew which condition they were in and annotated interruptions themselves, so expectations could bias both behavior and annotation. Thus the observed improvements do not isolate the contribution of the general models. This is a real soft spot in the central argument, not just an external-scope caveat.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an HRI turn-taking system in which two pre-trained general turn-taking models, TurnGPT and Voice Activity Projection (VAP), are used zero-shot to decide when the robot should prepare a response, start speaking, and yield to user interruptions. The proposed pipeline also includes incremental ASR-based response pre-generation, self-monitoring of the robot's own TTS, and VAP-based barge-in handling. The authors compare this system with a traditional silence-threshold baseline (with red LED and gaze-aversion cues) in a within-subject study with 39 participants. They report a significantly shorter median response time (1.5 s vs 2.7 s), a significantly lower interruption rate (6.9% vs 16.6%), and a significant preference for the proposed system.","tokens_in":18922,"tokens_out":6832,"duration_ms":69078,"significance":"If the improvements were caused by the general turn-taking models themselves, this would be a valuable first demonstration of zero-shot transfer from human-human telephone dialogue to face-to-face HRI. The study is properly powered, counterbalanced, and uses both objective timing and participant ratings; the effect sizes are large and statistically significant. The authors are also transparent about the arbitrary threshold values and the external validity risks. The main weakness is attribution: the proposed system differs from the baseline in several coupled design choices, so the reported gains do not isolate the contribution of TurnGPT and VAP. Because the central claim is about the general models, this needs to be resolved or the claim reframed.","major_comments":[{"comment":"The reduction in response time is confounded with response pre-generation. The proposed system begins LLM/TTS preparation on incremental ASR results whenever TurnGPT or the ASR timeout allows (pseudo-code lines 45–54), while the baseline waits for end-of-speech before generating anything. The paper itself acknowledges in §V-B that incremental preparation 'could potentially have been added to the baseline system as well.' Consequently, the median gap of 1.5 s vs 2.7 s and the 0.6 s histogram peak in Figure 6 may reflect the pre-generation mechanism rather than the turn-yield predictions of TurnGPT/VAP. To support the central claim, the authors should add a control condition or analysis that separates the contribution of the turn-taking models from that of pre-generation, for example a baseline with identical incremental response preparation but silence-threshold turn-taking, or an analysis in the proposed condition that distinguishes pre-generated from non-pre-generated turns.","section":"§V-B and §VII-A"},{"comment":"The interruption-rate comparison does not isolate the models because the two conditions use different interaction protocols. The baseline is simplex (red LED signals that the robot is not listening; participants are instructed to 'only speak when there is no light'), whereas the proposed system is duplex and participants are told they 'can interrupt the robot anytime.' The VAP-based barge-in handler in §V-C is a separate component from turn-yield prediction, and the instruction difference changes participant behavior and expectations. In addition, interruptions were self-annotated by participants who knew which condition they were in, so the 16.6% vs 6.9% difference may reflect protocol differences and annotation bias as well as model quality. The authors should either hold the interruption instructions and affordances constant across conditions, use blind annotators, or report a sensitivity analysis that addresses this confound.","section":"§V-C and §VI-C"},{"comment":"The system-level hyperparameters are tuned on the target application: the text before the pseudo-code states that their current values 'were tuned through four pilot experiments.' This means the comparison is not a pure zero-shot evaluation of the general models; thresholds such as VAP_PNOW_YIELD_THRESHOLD, TURNGPT_PREPARE_THRESHOLD, and MIN_GAP_TIME were adapted to the robot and task. The manuscript should specify how pilots were used, whether pilot participants were excluded from the main study, and ideally report the sensitivity of the main results to threshold variation. Otherwise the 'general models without domain-specific tuning' claim is only partially supported.","section":"Appendix A and §V"},{"comment":"The statistical summary is incomplete in ways that affect interpretation. Table I reports only means, medians, modes, and W statistics; it does not give the number of turns per participant, per-participant distributions, effect sizes, or confidence intervals. Because response times within a participant are not independent, the Wilcoxon test should be on per-participant summary values, but the paper does not explicitly say this. Please report the exact test setup, the per-participant differences, and effect sizes for response time and interruption rate.","section":"§VII-A"}],"minor_comments":[{"comment":"The claim that TurnGPT 'responds in about 20 ms' would benefit from clarification about whether this is per token, per utterance, or for a single turn-completion evaluation; as written it is ambiguous.","section":"§III-A"},{"comment":"The baseline's end-of-speech threshold is described as likely based on a silence threshold, but no estimate of the actual threshold value is given despite Figure 2 showing a roughly 2.5 s total response time; adding this would help readers compare the baseline with the 0.5–3 s timeouts in the proposed system.","section":"§IV-A"},{"comment":"The instruction 'only speak when there is no light' seems to conflict with the fact that participants annotated robot interruptions; please clarify whether the red light was expected to suppress all user speech and how the baseline's 16.6% interruption rate relates to this protocol.","section":"§VI-C"},{"comment":"The reporting of Q8 is unclear: the median of 2 and W=98 would be easier to interpret if the scale endpoints and the direction of preference were stated explicitly in the table.","section":"§VII-B and Appendix D"},{"comment":"Figure 4 is discussed as showing both a user interruption and a backchannel; labeling the two panels would improve readability and prevent confusion about which part of the figure corresponds to which event.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a solid systems paper, but the title and abstract claim that general turn-taking models improve HRI. If the authors add an ablation or re-frame the contribution as an integrated system, the paper would be strong. I do not see grounds for rejection, but the attribution issue is real and needs to be fixed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the integration: first HRI system that runs TurnGPT and VAP together, with self-monitoring of the robot's own speech, and a full within-subject study behind it. The algorithm in Appendix A is concrete and reproducible enough for someone to reimplement, and the evaluation is properly powered, counterbalanced, and shows large, significant effects on response time, interruption rate, and user preference. I believe the effects are real in this particular setup.\n\nBut the central claim is underdetermined. The proposed system differs from the baseline in at least four bundled ways: it pre-generates tentative responses from incremental ASR, it feeds the robot's TTS back into VAP and dialogue history into TurnGPT, it uses a different interruption-handling rule, and participants were told they could interrupt in one condition but not the other. The paper even admits the pre-generation could have been added to the baseline. That matters because the median response time drop from 2.7s to 1.5s is mostly explained by pre-generation, not by better turn-yield prediction. The interruption rate reduction is also confounded: participants knew which condition they were in and annotated interruptions themselves, so expectations could bias both behavior and annotation. The pilot-tuned thresholds add a mild self-tuning concern, though not a fatal one.\n\nNone of this makes the paper worthless. The integration is a useful template, the zero-shot transfer of telephone-trained models to a Furhat robot is an interesting empirical datapoint, and the authors are appropriately cautious in the discussion. The stress-test note says the central argument has a load-bearing soft spot, and I agree. But it's a soft spot in attribution, not in the existence of the effect. The paper deserves peer review: a serious referee could ask for an ablation or a baseline-with-pre-generation control, and with that the contributions would be much cleaner.\n\nWho is this for? HRI and spoken dialogue researchers, especially those thinking about incremental response generation and turn-taking prediction. I'd recommend engaging with it, but with eyes open about what is actually shown. The answer to \"did general turn-taking models help\" is \"we think so, but we didn't test that directly.\"","headline":"A solid first HRI deployment of TurnGPT and VAP in tandem, but the user study bundles several changes at once, so the headline claim that the general models are responsible for the gains is not actually isolated.","tokens_in":19378,"tokens_out":1346,"would_cite":true,"duration_ms":14997,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two zero-shot turn-taking models, used in tandem, make a robot's conversation faster and less interruptive, and people prefer it.","keywords":["turn-taking","human-robot interaction","Voice Activity Projection","TurnGPT","self-supervised learning","response delay","interruptions","conversational AI"],"falsifier":"Take the published algorithm, replace the stereo VAP input with the user-only channel while keeping every other hyperparameter, and compare interruption rates: if the 6.9% rate does not degrade toward the 16.6% baseline, the claimed role of self-monitoring is not supported.","tokens_in":18370,"feed_emoji":"🤖","tokens_out":5647,"duration_ms":57988,"temperature":0.7,"pith_summary":"Conversational robots usually decide when to speak by waiting for silence, which forces a trade-off between slow replies and interruptions. This paper argues that general self-supervised models of human turn-taking can replace that heuristic without any domain-specific retraining. The authors combine TurnGPT, which predicts syntactic and pragmatic turn completion from text, with Voice Activity Projection (VAP), which continuously predicts upcoming speech activity from audio, and feed the robot's own synthesized speech back into both models. In a within-subject study with 39 adults talking to a Furhat robot, the proposed system reduced median response time from 2.7 s to 1.5 s and interruption rate from 16.6% to 6.9%, and participants significantly preferred it. If the result holds, turn-taking in human-robot interaction no longer needs hand-tuned silence thresholds or per-domain training data.","feed_headline":"Robot chat gets faster with zero-shot turn-taking models","feed_subtitle":"General models trained on phone calls halved median response time and cut interruptions in a 39-person robot study.","key_machinery":"The machinery is the tandem of TurnGPT and VAP plus self-monitoring. TurnGPT is a GPT-2-style language model trained on 385K text conversations with a special turn-completion token, so the probability it assigns to that token measures whether the current utterance is pragmatically complete. VAP is a transformer trained on roughly 1,000 hours of telephone dialogue (Fisher and Switchboard) to project, ten times per second, who will speak in the next two seconds, summarized as $p_{\\mathrm{now}}$ and $p_{\\mathrm{future}}$; a version taking stereo input lets one channel be the robot's own synthesized voice. Feeding the robot's speech back is what lets the system distinguish a genuine interruption from a backchannel and decide when gaze aversion is needed. A small set of thresholds on these two signals, listed in the appendix, converts the predictions into the robot's turn-taking behavior.","core_discovery":"The central discovery is that turn-taking models trained as third-person predictors of human-human dialogue can serve as first-person controllers of a robot's turn-taking behavior when used in tandem and fed the robot's own speech. TurnGPT supplies a turn-completion probability from the incremental ASR transcript, capturing syntactic and longer-range pragmatic completion; VAP supplies two continuous acoustic projections, $p_{\\mathrm{now}}$ (0-600 ms) and $p_{\\mathrm{future}}$ (600-2000 ms), from stereo audio of user and robot. The control algorithm lets the robot start preparing LLM+TTS responses before the user finishes, allows a turn shift only after both VAP projections favor the robot or a TurnGPT-graded timeout expires, and stops the robot mid-utterance only when both projections signal a genuine interruption. The evaluation against a silence-threshold baseline with gaze aversion and an LED listening signal found the proposed system significantly faster and less interruptive, with users rating it more fluent, more human-like, and easier to interrupt.","pith_inferences":["The authors do not test this, but the same VAP signal could also drive synthesized fillers or prosodic turn-holding cues, since they note VAP is sensitive to fillers.","The paper does not test non-English or multi-party interaction, so a direct stress test of zero-shot generality would use other languages, accents, or group settings.","Their proposed semantic-similarity gating on incremental ASR could be combined with TurnGPT rollouts to prepare responses even earlier, an idea they mention as future work."],"forward_implications":["Spoken dialogue systems can drop fixed silence thresholds and instead gate turns on continuous projections of when the other speaker will finish.","Response generation can begin on incremental ASR before the user stops speaking, so the floor can be taken close to the 0.5 s minimum allowed by the algorithm whenever the LLM and TTS are fast enough.","The same VAP signal can separate genuine barge-in interruptions from brief backchannels, enabling duplex listening without the false-trigger problems of naive barge-in.","Because the models are used zero-shot, new HRI applications need not collect and annotate domain-specific turn-taking data to get natural timing.","The preference result suggests that more human-like timing is not automatically worse for users, though a minority preferred the explicit LED pacing of the baseline."],"supporting_citations":[{"why":"Defines the standard silence-threshold turn-taking approach and the response-time/interruption trade-off that the proposed system is compared against.","marker":"[3]"},{"why":"Supplies TurnGPT, the text-based model that predicts syntactic and pragmatic turn completion.","marker":"[19]"},{"why":"Supplies Voice Activity Projection, the self-supervised acoustic model that predicts upcoming speech activity.","marker":"[20]"},{"why":"Provides the Furhat robot platform used in both the baseline and proposed systems.","marker":"[21]"},{"why":"Provides the SODA dataset of 385K text conversations on which TurnGPT was trained.","marker":"[43]"},{"why":"Provides the stereo-input VAP version that lets the robot's own synthesized speech be fed back as one channel.","marker":"[45]"},{"why":"Fisher corpus is part of the telephone dialogue training data for VAP.","marker":"[46]"},{"why":"Switchboard corpus is part of the telephone dialogue training data for VAP.","marker":"[47]"}],"fun_headline_variants":["Zero-shot turn-taking models speed up robot chat","Robot conversations get quicker with zero-shot models","TurnGPT and VAP cut robot response delays","General turn-taking models improve robot dialogue","Zero-shot models make robot chat more natural"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach depends on predictions from models trained on telephone conversations between US English speakers staying reliable when applied, without retraining, to a face-to-face robot whose synthetic voice is fed back into the models.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot turn-taking models speed up robot chat","Robot conversations get quicker with zero-shot models","TurnGPT and VAP cut robot response delays","General turn-taking models improve robot dialogue","Zero-shot models make robot chat more natural"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000252,"raw_usage":{"total_tokens":1538,"prompt_tokens":903,"completion_tokens":635,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":568}},"tokens_in":519,"tokens_out":635,"duration_ms":6610,"temperature":1.0,"reasoning_tokens":568,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:13:27.030123+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the published algorithm, replace the stereo VAP input with the user-only channel while keeping every other hyperparameter, and compare interruption rates: if the 6.9% rate does not degrade toward the 16.6% baseline, the claimed role of self-monitoring is not supported.","supporting_citations":[{"cited_title":"Turn-taking in conversational systems and human-robot interaction: A review,","cited_arxiv_id":null,"evidence_quote":"Defines the standard silence-threshold turn-taking approach and the response-time/interruption trade-off that the proposed system is compared against."},{"cited_title":"TurnGPT: A Transformer-based language model for predicting turn-taking in spoken dialog,","cited_arxiv_id":null,"evidence_quote":"Supplies TurnGPT, the text-based model that predicts syntactic and pragmatic turn completion."},{"cited_title":"V oice Activity Projection: Self-supervised learning of turn-taking events,","cited_arxiv_id":null,"evidence_quote":"Supplies Voice Activity Projection, the self-supervised acoustic model that predicts upcoming speech activity."},{"cited_title":"Furhat: A back-projected human-like robot head for multiparty human-machine interaction,","cited_arxiv_id":null,"evidence_quote":"Provides the Furhat robot platform used in both the baseline and proposed systems."},{"cited_title":"Predictive models of turn-taking in spoken dialogue,","cited_arxiv_id":null,"evidence_quote":"Provides the stereo-input VAP version that lets the robot's own synthesized speech be fed back as one channel."},{"cited_title":"The fisher corpus: a resource for the next generations of speech-to-text,","cited_arxiv_id":null,"evidence_quote":"Fisher corpus is part of the telephone dialogue training data for VAP."},{"cited_title":"Switchboard: Telephone speech corpus for research and development,","cited_arxiv_id":null,"evidence_quote":"Switchboard corpus is part of the telephone dialogue training data for VAP."}],"review_version":1}