{"id":"66c99648-0b2d-4da3-9b70-3d83100dfff4","arxiv_id":"2501.01568","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A real-time system that classifies user interruptions into four intents and adapts robot responses, achieving 93.69% successful handling in a 21-participant study.","lead":"This paper builds a system that lets a social robot detect when a person interrupts it, classify the interruption as cooperative or disruptive, and respond with the right strategy. The authors integrated it into an LLM-powered robot and report that it handled 93.69% of user interruptions in two lab tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 93.69% success rate does not isolate the intended contribution: 56 of 111 handled interruptions bypass the intent classifier, and success was coder-judged with no baseline condition.","rationale":"The reader's weakest-assumption concern is that the intent classifier uses only text transcripts and elapsed time, without prosody or visual cues, so the foundation of strategy selection may be faulty. That is a plausible concern, but the paper's own data already show 80% accuracy on the 55 classifier-passed cases, and several classification errors still led to successful handling. The more load-bearing problem is that the headline 93.69% success rate is not attributable to the classifier: a large fraction of handled interruptions bypass the module, and the success metric is a subjective coder judgment with no baseline condition. This directly threatens the central claim that intention-aware handling is what makes the system effective. The paper is still a useful systems demonstration with transparent reporting and released code, but its causal claim about the benefit of intention classification needs stronger support. Therefore I partially agree with the reader's weakest assumption but would keep the CONDITIONAL verdict, with the concrete requirement being an explicit baseline comparison or a reanalysis isolating the classifier's contribution.","tokens_in":11998,"tokens_out":7406,"duration_ms":72724,"concrete_test":"Reanalyze the 111 interruption episodes with the 43 wakeword and 13 finish-up cases excluded, and compare the classifier-only 55-case subset against a scripted always-yield baseline scored with the same coder rubric on the same transcripts. Compute the success rate under both policies; if the classifier-only success rate is not significantly above the always-yield baseline (e.g., by McNemar's test on the matched episodes), the paper should qualify the causal claim that intention classification drives the 93.69% success. Report whether coders were blind to the policy being evaluated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim is that intention-based handling achieves 93.69% success, but the reported number cannot be attributed to the novel intent-classification module. Of the 111 handled interruptions, 43 were resolved by wakeword-triggered immediate yield and 13 were 'finish-up' cases with less than two seconds of planned speech remaining; both paths bypass the intent classifier entirely (Section VI-B, Fig. 3). Only 55 interruptions passed through the classifier, and its accuracy there was 80.00% (44/55). Success itself was coded by the authors as 'effectively responded to the interrupter without causing a conversation breakdown' (Section V-E), with no baseline condition; the Discussion's comparison against 'always yield' (84.68%) and 'ignore' (27.03%) is a post hoc relabeling of the same data, not a measured control. Because 43 wakeword yields are successful by construction and several misclassifications were counted as successful (e.g., disruptive interruptions misclassified as clarification were handled successfully, Section VI-D), the headline result is consistent with a system that mostly yields politely. It does not demonstrate that real-time intention classification improves interruption handling over a simpler always-yield policy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a real-time interruption-handling system for conversational robots. The system detects user-initiated interruptions via overlapping speech, classifies the interrupter's intent into four categories (cooperative agreement, cooperative assistance, cooperative clarification, disruptive interruption) using an LLM (GPT-4o-mini) with text transcripts and elapsed time, and selects handling strategies derived from interaction patterns observed in human-human YouTube conversations. The authors integrate the system into an LLM-powered social robot and evaluate it in a timed decision-making task and a contentious discussion task with 21 participants. They report that 104 of 111 handled interruptions (93.69%) were successfully handled, that intent classification was correct in 88.78% of cases, and that the system is, to the best of their knowledge, the first robotic system to integrate intention classification into its interruption-handling framework.","tokens_in":12236,"tokens_out":6529,"duration_ms":61254,"significance":"The work addresses an important gap in human-robot interaction: moving beyond always-yield or ignore policies toward context- and intent-aware interruption handling. The paper provides a complete system implementation with open-source code and supplementary materials, a user study with 206 minutes of interaction data, and a two-coder annotation process with high inter-coder reliability (Cohen's Kappa = 0.92). The interaction-pattern analysis from human-human data is a useful design resource for the community. However, the current evaluation does not isolate the contribution of the novel intent-classification module, and the abstract's headline success rate omits important exclusions and lacks a measured baseline. If these evaluation issues are addressed, the work could be a solid empirical contribution to real-time conversational robot design.","major_comments":[{"comment":"The headline claim 'successfully handled 93.69% (n=104/111) of user-initiated interruptions' does not isolate the paper's intended contribution. Of the 111 handled interruptions, 43 were resolved by wakeword-triggered immediate yield and 13 were 'finish-up' cases with less than two seconds of planned robot speech remaining; both paths bypass the intent classifier entirely (Section VI-B, Fig. 3). Only 55 interruptions passed through the classifier, with 80.00% accuracy (44/55). The Discussion's comparison against 'always yield' (84.68%) and 'ignore' (27.03%) is a post hoc relabeling of the same data, not a measured control condition. Consequently, the aggregate success rate is consistent with a system that mostly yields politely, and it does not by itself demonstrate that real-time intention classification improves interruption handling. The authors should report success separately for classifier-based vs. bypass paths and add a measured baseline condition (e.g., always-yield or wakeword-only) to support the causal claim.","section":"Abstract; §VI-B–VI-C"},{"comment":"The success criterion is subjective and lacks an experimental baseline. Coders assessed whether the robot 'effectively responded to the interrupter without causing a conversation breakdown,' but no operational definition of 'conversation breakdown' is given, and the coding was performed by the authors on their own system. In addition, several misclassifications were counted as successful handling (e.g., disruptive interruptions misclassified as clarification, Section VI-D), which makes the metric lenient. Without a baseline condition under the same tasks and participant pool, the 93.69% figure cannot be interpreted as evidence that intention-based handling outperforms simpler policies. I recommend adding a between-subjects baseline (always-yield, ignore, or wakeword-only) or, at minimum, reporting per-path success rates and a more objective outcome measure such as task completion or interrupter goal attainment.","section":"§V-E; §VI-C"},{"comment":"The intent classifier relies solely on text transcripts and elapsed time, without prosodic or visual cues, and its accuracy is only 80.00% (44/55) on the classifier-passing subset (95% confidence interval for 44/55 is roughly 67–89%). Because the system's strategy selection is predicated on this classification, the paper should provide a confusion matrix, per-type accuracy (especially for cooperative vs. disruptive), and a discussion of how text-only input biases the classification (e.g., short utterances like 'no' are ambiguous without prosody). The current reporting of aggregate 'intent correctly classified' (88.78%) mixes bypass paths with classifier output and is not sufficient to establish the reliability of the novel component.","section":"§IV-B; §VI-B"},{"comment":"The abstract reports 93.69% (n=104/111) without disclosing that 10 of the 121 observed interruptions were excluded from analysis: five triggered pre-programmed events that bypass the system, and five occurred during a network-related speech-to-text delay (Section VI-B). These exclusions are disclosed in the body, but the abstract's 'user-initiated interruptions' phrase is inaccurate because it omits these cases. The abstract should either use a qualified phrasing (e.g., 'of the 111 interruptions handled by the system') or report the overall rate including all 121 interruptions.","section":"Abstract; §VI-B"}],"minor_comments":[{"comment":"The metro-map diagram is visually dense, and the text references 'Line <color>' without ensuring that colors are distinguishable in grayscale or accessible to color-blind readers; please add line labels or numbers.","section":"Fig. 3"},{"comment":"The table formatting is difficult to parse; the columns and counts for the 'LLM error handling' row and the footnote markers are unclear. Please reformat with explicit column headers and align counts with rows.","section":"Table II"},{"comment":"The sentence 'The timed decision-making task (n=86) had more than twice as many user-initiated interruptions as the contentious discussion task (n=35)' is ambiguous about whether these are per-participant or total counts; please clarify.","section":"§VI-A"},{"comment":"The inter-coder reliability (Cohen's Kappa = 0.92) is reported for the full coding including interruption type and success; please also report reliability separately for the 'successfully handled' judgment, since this is the key outcome variable.","section":"§V-E"},{"comment":"The novelty claim 'no existing robotic system has integrated intention classification into its interruption-handling framework' is strong; a systematic survey of prior work in spoken dialogue systems and voice assistants would strengthen it, or the claim should be softened.","section":"§I"},{"comment":"In the example, the robot says 'I suggested a parachute' after previously discussing a flashlight; the inconsistency between the example text and the item under discussion is confusing and should be corrected.","section":"§VI-D"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable fit for a systems-oriented HRI venue, and the open-source implementation is a positive element. The main concern is that the central empirical claim is currently overstated relative to the evidence. A revision that adds a baseline condition or at least a clear per-path breakdown, together with a more careful abstract, could make this a solid contribution. I would encourage the editor to consider the revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful thing here is the system, not the headline number. They built a real robot that detects overlapping speech, classifies the user's intent into four types with an LLM, and picks a handling strategy from patterns they coded in human-human YouTube data. That integration is new, as far as the cited literature goes, and the implementation is released. The reporting is also unusually transparent: they give the full breakdown of how the 111 handled interruptions were routed, they disclose the ten excluded cases, and they analyze every failure rather than hiding it. Credit where it's due: this is a solid systems contribution and a good template for future HRI interruption work.\n\nThe soft spot is the central empirical claim. The abstract says 93.69% of interruptions were successfully handled, but 56 of those 111 cases bypassed the intent classifier entirely: 43 were wakeword-triggered yields and 13 were finish-up because less than two seconds of planned speech remained. Only 55 interruptions actually went through the classifier, and its accuracy there was 80%. The success criterion is also coder-judged, and some misclassifications were counted as successful (e.g., disruptive interruptions misclassified as clarification still got answered). The Discussion's comparison against always-yield (84.68%) and ignore (27.03%) is a post hoc relabeling of the same data, not a control condition, so it doesn't demonstrate that intent-based handling improves outcomes. The study is small (21 participants, two lab tasks) and the classifier uses only text and elapsed time, which is a real limitation, though a fair one to leave to future work.\n\nNone of this makes the paper a write-off. The claimed contribution is not fraudulent; it's just under-supported. What the paper actually shows is that a system combining wakeword yield with a decent LLM classifier can handle most interruptions acceptably in a lab setting. Whether the intent classifier adds value over simply always yielding is not proven, and the trend in their own failure data (misclassified disruptive interruptions sometimes still handled fine) makes me think the added value may be modest. That is an empirical question, and an important one for the field. The right fix is a proper within-subjects or between-subjects baseline comparing their full system against an always-yield policy and maybe a no-system control.\n\nFor whom: HRI researchers and conversational-agent designers will get value from the system architecture, the taxonomy, and the honest failure analysis. It deserves serious peer review, and I would cite it as a systems reference even while being skeptical of the headline effect. My recommendation: engage with it, send it to review, and require the baseline comparison before publication.","headline":"A genuinely novel integrated system for real-time interruption handling, but the headline 93.69% success number does not isolate the intent-classifier contribution and the always-yield comparison is not a measured baseline.","tokens_in":12749,"tokens_out":1826,"would_cite":true,"duration_ms":19326,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A robot that classifies the intent behind user interruptions handles 93.69% of them in live tasks.","keywords":["human-robot interaction","interruption handling","intent classification","large language models","conversational robots","turn-taking","social robots","real-time dialogue"],"falsifier":"Record the same 21-participant interactions and re-classify each of the 111 interruptions from transcript plus elapsed time, then compare against prosodic and visual features; if adding pitch, volume, and gaze information changes the classification of more than a few of the 11 misclassified cases, the text-only premise is testably incomplete. More directly, run the system with participants instructed to interrupt using only non-lexical sounds such as sighs or laughter, and measure whether the pipeline can handle those cases at all.","tokens_in":11820,"feed_emoji":"🤖","tokens_out":8981,"duration_ms":78221,"temperature":0.7,"pith_summary":"Interruptions are a normal and often useful part of conversation, but most conversational robots either ignore overlapping speech or yield to every interruption, breaking the flow. This paper argues that a robot should first infer why the user is interrupting—agreeing, helping, asking for clarification, or disrupting—and then choose a handling strategy matched to that intent. To support this, the authors built a real-time system that detects overlapping speech, uses a large language model to classify the interrupter's intent from the transcript and elapsed turn time, and applies strategies derived from human-human interaction patterns. Integrated into an LLM-powered social robot and tested with 21 participants in a timed survival task and a contentious debate task, the system successfully handled 104 of 111 user-initiated interruptions (93.69%). The result matters because it moves robots from rigid turn-taking rules toward fluid, intent-sensitive conversation.","feed_headline":"Robot reads interruption intent, handles 93.69% live","feed_subtitle":"An LLM-powered social robot sorts interruptions into agreement, help, clarification, or disruption and adapts its response.","key_machinery":"The load-bearing mechanism is a three-module pipeline. Interruption detection monitors for overlapping speech and treats speech as a non-interruption if less than two seconds of planned robot speech remains. Intent classification uses a prompt-engineered large language model that receives the conversational history and the elapsed time since the robot's turn began and returns one of four intents. Strategy selection then follows interaction patterns coded from human-human videos: acknowledge-and-continue for agreement and assistance, answer-and-continue for clarification, and either hold the floor or yield for disruptive interruptions depending on how early in the turn they occur. A wakeword or the word \"stop\" bypasses the classifier and triggers immediate yielding. The design also separates one-to-two-word backchannels from longer cooperative agreements and treats disruptive interruptions within five seconds of the robot's turn start as aggressive, triggering a floor-holding response.","core_discovery":"The paper claims a first for robotic systems: interruption handling driven by intention classification rather than a fixed reaction to overlapping speech. Its central empirical claim is that a system which classifies interruptions into four categories—cooperative agreement, cooperative assistance, cooperative clarification, and disruptive interruption—and selects among strategies such as continue, acknowledge-and-continue, clarify-and-continue, yield immediately, and acknowledge-and-wrap-up achieves 93.69% successful handling in live interaction. The paper also reports that the intent classifier was correct on 88.78% of handled interruptions, that 76.03% of interruption attempts were disruptive, and that both ignoring interruptions and always yielding perform worse on the same data (27.03% and 84.68% successful, respectively). In the paper's own words, no existing robotic system has integrated intention classification into its interruption-handling framework.","pith_inferences":["Beyond the paper, the text-only classifier's success may not transfer to languages or dialects where agreement and disagreement are carried by prosody rather than words; adding pitch, energy, and speaking-rate features and comparing accuracy on the same recorded interruptions is a direct test.","Beyond the paper, the five-second aggression rule and the one-to-two-word backchannel rule are thresholds fitted to this study's data, and the optimal cutoffs likely shift with task context and culture.","Beyond the paper, the finding that participants viewed the robot as an assistive tool and expected it to yield immediately suggests interruption handling can be used to signal robot role: holding the floor may cast the robot as more authoritative, which could help or harm depending on the application.","Beyond the paper, the dyadic evaluation leaves multi-party interruptions unaddressed; extending the system would require deciding whose interruption to honor when several users speak at once."],"forward_implications":["A robot that ignores interruptions would have handled only 27.03% of the interruptions observed in this study, because it would miss 73 disruptive and 8 clarification attempts.","A robot that always yields would handle 84.68%, failing on cooperative agreement and cooperative assistance, so intent-based handling adds roughly nine percentage points of successful handling.","Unsuccessful interruption handling correlates with lower perceived inclusion ($\\rho = -0.43$, $p = .005$) and lower discussion satisfaction ($\\rho = -0.35$, $p = .021$), so handling errors carry social cost.","Speech recognition errors, not the language model itself, caused most intent misclassifications, so improving automatic speech recognition robustness is the next lever for the approach.","Users' habits from commercial voice assistants, such as saying the wakeword before every query, can make interactions rigid even when the system supports natural interruption."],"supporting_citations":[{"why":"It supplies the four-part taxonomy of interruption intent (agreement, assistance, clarification, disruptive) that the system classifies.","marker":"[23]"},{"why":"It provides prior multimodal classification of interruptions as cooperative or competitive, which the paper extends from post hoc analysis to real-time robot handling.","marker":"[24]"},{"why":"It defines interruptions in terms of overlap and documents prior agent architectures that ignore or mishandle overlapping speech.","marker":"[8]"},{"why":"It represents the existing design of always yielding the floor after an interruption, the baseline the paper argues against.","marker":"[7]"},{"why":"It recommends distinguishing cooperative from disruptive interruptions to decide how an agent should respond.","marker":"[19]"},{"why":"It establishes cooperative and disruptive interruption categories and links interruption patterns to gender and nonverbal behavior, informing the robot's persona.","marker":"[13]"},{"why":"It shows interruption type and frequency vary with conversational power dynamics, motivating the choice of human-human videos with different power structures.","marker":"[3]"}],"fun_headline_variants":["Robot reads intent, handles 93.69% of live interruptions","Intent-aware robot handles interruptions 93.69% of the time","Robot classifies interruption intent, hits 93.69% success","93.69% of interruptions handled by intent-aware robot"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole handling strategy rests on the language model being able to infer the user's true intent from a text transcript plus elapsed turn time, without hearing tone of voice or seeing the user's face; if text alone cannot reliably separate cooperation from disruption, the strategy selection has no solid foundation.","fun_headline_variants_meta":{"raw":{"variants":["Robot reads intent, handles 93.69% of live interruptions","Intent-aware robot handles interruptions 93.69% of the time","Robot classifies interruption intent, hits 93.69% success","93.69% of interruptions handled by intent-aware robot"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000774,"raw_usage":{"total_tokens":3394,"prompt_tokens":883,"completion_tokens":2511,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":2438}},"tokens_in":499,"tokens_out":2511,"duration_ms":17535,"temperature":1.0,"reasoning_tokens":2438,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:25:02.925994+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record the same 21-participant interactions and re-classify each of the 111 interruptions from transcript plus elapsed time, then compare against prosodic and visual features; if adding pitch, volume, and gaze information changes the classification of more than a few of the 11 misclassified cases, the text-only premise is testably incomplete. More directly, run the system with participants instructed to interrupt using only non-lexical sounds such as sighs or laughter, and measure whether the pipeline can handle those cases at all.","supporting_citations":[{"cited_title":"What If I Interrupt You","cited_arxiv_id":null,"evidence_quote":"It supplies the four-part taxonomy of interruption intent (agreement, assistance, clarification, disruptive) that the system classifies."},{"cited_title":"Multimodal classification of interruptions in humans’ interaction","cited_arxiv_id":null,"evidence_quote":"It provides prior multimodal classification of interruptions as cooperative or competitive, which the paper extends from post hoc analysis to real-time robot handling."},{"cited_title":"Pardon the interrup- tion: Managing turn-taking through overlap resolution in embodied artificial agents","cited_arxiv_id":null,"evidence_quote":"It defines interruptions in terms of overlap and documents prior agent architectures that ignore or mishandle overlapping speech."},{"cited_title":"Designing the Impres- sion of Social Agents’ Real-time Interruption Handling","cited_arxiv_id":null,"evidence_quote":"It represents the existing design of always yielding the floor after an interruption, the baseline the paper argues against."},{"cited_title":"Continuous interac- tion with a virtual human","cited_arxiv_id":null,"evidence_quote":"It recommends distinguishing cooperative from disruptive interruptions to decide how an agent should respond."},{"cited_title":"Interruptions and nonverbal gender differences","cited_arxiv_id":null,"evidence_quote":"It establishes cooperative and disruptive interruption categories and links interruption patterns to gender and nonverbal behavior, informing the robot's persona."},{"cited_title":"Interruption in conversational in- teraction, and its relation to the sex and status of the interactants","cited_arxiv_id":null,"evidence_quote":"It shows interruption type and frequency vary with conversational power dynamics, motivating the choice of human-human videos with different power structures."}],"review_version":1}