{"id":"6f3862d7-d068-4757-8194-52f18841a1ed","arxiv_id":"2608.02254","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Homebot's architecture separates channel-scoped chat sessions from wake-word-bounded voice sessions and adds a three-state dialogue protocol for hands-free turn control.","lead":"Homebot is a home assistant AI that handles voice, Telegram, and Feishu requests through one shared engine while keeping each conversation separate. Its design centers on an explicit voice turn-taking protocol, so hands-free interactions know when to listen, reply, or end.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The voice turn-control protocol in §2.4 depends on the LLM emitting a valid, semantically correct dialogue_state every turn, but no validation, fallback, or reliability data is described; without it the state machine can stall, so the 'practical foundation' claim is unprotected at its core.","rationale":"The reader identifies the broad lack of empirical validation of the voice pipeline and tool-calling reliability. My concern is narrower and more specific: even with a perfect speech front end, the voice state machine depends on a single unvalidated LLM output field (dialogue_state) whose failure mode is not specified in §2.4.1. This is internally load-bearing because Figure 3's transitions are driven entirely by that field, and the paper's own text emphasizes that 'the dialogue state determines the transition after playback.' The absence of validation or a fallback path is not merely an unmeasured quantity; it is a design gap that can produce a stuck interaction. The public code and the paper's honest scoping of planned extensions are real strengths, but they do not close this gap. The proposed harness would settle the concern by exercising the actual channel logic; if the implementation already contains fallback handling not described in the paper, the concern resolves. Since the reader's verdict is already CONDITIONAL and this concern reinforces rather than overturns that verdict, I recommend no change to the verdict; the condition should be expanded to require either a demonstrated fallback or a reliability measurement for the dialogue_state contract.","tokens_in":4631,"tokens_out":3278,"duration_ms":34275,"concrete_test":"Extract the voice channel and run a scripted harness with a mock model: for 200 turns, randomly inject (a) invalid JSON, (b) a missing dialogue_state, (c) contradictory states such as follow_up after a skill-completing answer, and (d) valid but semantically wrong states; record whether the channel reaches STOPPED/LISTENING or remains stuck, and whether any fallback path exists. Also audit the code for schema validation, retry logic, or constrained decoding around dialogue_state. If realistic failure rates exceed 1% and no fallback exists, the 'practical foundation' claim needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central 'practical foundation' claim rests on the voice turn-control protocol in §2.4 being reliable in real use. The protocol assigns the semantic next-state decision to the LLM via a structured JSON dialogue_state (end/follow_up/continuous), and the channel's state machine transitions only after parsing this field. The paper specifies no validation of this field, no retry or fallback for malformed or missing JSON, no constrained decoding, and no measurement of how often the model emits a semantically correct state in household dialogues. If the model ever returns invalid JSON or a contradictory state (e.g., end while a skill still expects a follow-up answer), the channel has no defined recovery path; the interaction can hang or silently drop the session. Because every hands-free turn depends on this single contract, the architecture's 'practical' claim is load-bearing on an unverified and unprotected component. This is distinct from the broader 'no field data' concern: even with perfect wake-word detection and speech recognition, the state machine can fail at its final transition. The contrast with tool calls is telling: §2.2 explicitly verifies tool names and validates parameters against schemas, but §2.4.1 describes no equivalent protection for the dialogue_state field, which is the linchpin of voice turn control.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"Homebot is described as a locally deployable AI agent for conversational household assistance, supporting voice, Telegram, and Feishu through a shared runtime. The paper's contribution is architectural: channels normalize platform events into a common message representation; a Message Bus and Agent Runtime interleave LLM reasoning with validated tool calls; session management scopes chat history to channel:chat_id while wake-word activations create bounded voice sessions; and the voice pipeline combines local wake-word detection, streaming ASR/TTS, and a structured dialogue_state (end, follow_up, continuous) that gates post-playback transitions. The paper concludes that these channel, tool, and skill contracts provide an extensible, practical foundation for home assistance. The source code is publicly available, but the manuscript reports no experiments or field measurements.","tokens_in":4972,"tokens_out":4160,"duration_ms":42931,"significance":"Taken as a design proposal, the architecture is clear and internally consistent, and the paper usefully separates a shared request path from session-ownership boundaries, which is a genuine design tension for household agents. The explicit contracts for channels, tools, and skills, plus the public code, are concrete assets. However, the central claim of practical suitability is not supported by empirical evidence; the value currently rests on the soundness of the design and on trusting untested speech and dialogue components. If the robustness and evaluation gaps are addressed, this could be a useful reference architecture for local home assistants.","major_comments":[{"comment":"The post-playback transition of every voice interaction depends on the language model emitting a valid and semantically correct dialogue_state in structured JSON, but the manuscript specifies no validation, constrained decoding, retry, or fallback for malformed or contradictory values. If the model returns invalid JSON or, for example, 'end' while a skill still expects a follow-up answer, the state machine has no defined recovery path and the interaction can stall. The contrast with tool calls is sharp: Section 2.2 verifies tool names and validates parameters against JSON Schema, but Section 2.4.1 gives the dialogue_state field no equivalent protection. Please specify a recovery contract and, ideally, measure the field's reliability in representative household dialogues.","section":"Section 2.4.1"},{"comment":"The conclusion's 'practical foundation' claim and Section 2.4's 'reliable hands-free conversation' claim require evidence that the components compose correctly in real use. The paper reports no user study, no wake-word false-accept/false-reject rates, no ASR/TTS error or latency measurements, no dialogue-state correctness statistics, and no tool-call success data. Without at least a small deployment study or a reproducible benchmark, the central claim remains an untested design assertion. Adding an evaluation section, or explicitly narrowing the claim to an architecture description, is necessary.","section":"General evaluation"}],"minor_comments":[{"comment":"The phrase 'The exact /new command' appears to be a typo; it should likely be 'The explicit /new command' or simply 'The /new command'.","section":"Section 2.3"},{"comment":"The question-mark transition labels in Figure 3 are not explained in the prose, and the text does not define default timeout durations or the behavior after ASR or TTS failures; please spell out each transition and the timeout policy.","section":"Figure 3"},{"comment":"The statement that 'dependency requirements determine whether a skill is available' is vague; specify how skill dependencies are declared and what happens when they are unmet.","section":"Section 2.5"},{"comment":"The example exit phrase is given as 'goodbye, ' with a trailing comma and space; use a complete example phrase or a placeholder such as 'goodbye, Homebot'.","section":"Section 2.4.1"}],"recommendation":"major_revision","confidential_remarks":"This manuscript is more a systems-design report than a completed study. If the venue regularly accepts design papers without evaluation, the missing evaluation might be less critical, but the dialogue-state robustness gap is still load-bearing for the voice interaction claim. I recommend major revision so that the authors either add a recovery/validation mechanism and a small evaluation, or explicitly soften the 'practical foundation' claim to an architectural proposal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things: this is a systems design paper, not an empirical one, and it is upfront about that; and the most load-bearing component, the dialogue_state protocol in §2.4, is the least protected part of the architecture.\n\nWhat is actually new is modest but real. Homebot separates channel-scoped chat sessions from wake-word-bounded voice sessions behind a shared agent runtime, and it makes the voice turn-control decision an explicit three-valued field (end / follow_up / continuous). That combination is not in the cited prior work, so the design artifact is novel enough. What the paper does well is describe the architecture clearly, ship public code, and define clean extension contracts for channels, tools, and skills. The progressive-disclosure skill mechanism is a nice practical touch, and the tool-call validation against JSON Schema in §2.2 is concrete. Credit where due: the authors are honest that they introduce no new models, and the citation pattern looks fine—they cite their own Venus work only as related work, not to justify design choices.\n\nThe soft spots are proportionate. The biggest one is exactly what the stress-test note flags. The entire voice turn-control state machine in §2.4.1 hinges on the LLM emitting a valid, semantically correct dialogue_state on every turn. The paper specifies no validation, no retry, no constrained decoding, and no fallback for malformed or contradictory JSON. If the model says \"end\" while a skill is mid-interaction, the channel has no recovery path; the session can hang or silently drop. That is not a minor omission because the paper's conclusion calls the design a \"practical foundation\"—the claim is load-bearing and untested. It is telling that tool calls get schema validation but dialogue_state gets nothing.\n\nA second, related weakness is the absence of any evaluation: no wake-word reliability data, no ASR accuracy numbers, no user study, no measurement of dialogue_state correctness in household dialogues. That is fine for a design report, but it means the \"practical\" claim is an assertion, not a demonstrated property.\n\nOverall the paper is coherent and internally consistent. The stress-test concern holds up on reading. This is worth a serious referee, especially for a systems or HCI venue, but the authors should either soften the \"practical foundation\" language to \"design proposal\" or add a robustness analysis and some field data. I would not cite it in my own work, but I would bring it to a reading group interested in local AI agent architectures.\n\nRecommendation: send it to peer review, clearly labeled as a systems design paper, and ask the referees to focus on the dialogue_state failure modes and what it would take to make the protocol robust.","headline":"A clear, honest systems design report with public code; the only real problem is that the central 'practical foundation' claim leans on an unvalidated and unprotected LLM-emitted dialogue_state field.","tokens_in":5388,"tokens_out":1045,"would_cite":false,"duration_ms":12967,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Homebot's central claim is that a single shared agent runtime can serve voice and messaging channels without merging their conversation histories, and that this design is an extensible foundation for local home assistance.","keywords":["home assistance","voice interaction","agent runtime","tool use","session management","wake-word detection","multi-channel messaging","smart home automation"],"falsifier":"Run Homebot in a typical household for 100 wake-word activations at measured background noise levels; record the fraction of activations that reach RECOGNIZING and THINKING and the fraction in which the model's dialogue_state matches the user's intent, for example whether follow_up actually restarts recognition. A high wake-word false-reject rate or a high dialogue-state mismatch rate would show that the voice turn-control design does not hold up under the conditions it targets.","tokens_in":4417,"feed_emoji":"🏠","tokens_out":7744,"duration_ms":71667,"temperature":0.7,"pith_summary":"Homebot is a locally deployable home assistant that handles voice and text chat through one shared processing core. The paper's central claim is that a single Agent Runtime can serve both modalities while keeping every conversation's history separate: messaging state is scoped to a channel and chat identifier, while each wake-word activation opens a fresh voice session. Voice is treated as a full pipeline, combining local wake-word detection, streaming speech recognition, and incremental speech synthesis with an explicit dialogue-state field that tells the channel whether to end, follow up, or continue after an answer. The system separates common request processing from extension points, defining contracts for channels, tools, and skills so new capabilities can be added without rewriting the core path. If the architecture works as described, a household can extend its assistant through one uniform mechanism rather than maintaining modality-specific code.","feed_headline":"Homebot routes voice and messaging through one agent runtime","feed_subtitle":"A shared runtime keeps tool use and skills common while each mode keeps its own conversation history.","key_machinery":"The load-bearing mechanism is the separation of Session Management from the Agent Runtime: every request, voice or text, is normalized into a common message and processed through the same tool-use loop, but whether a request may reuse history is decided outside that loop by session keys. Messaging uses channel:chat_id, voice uses voice:<UUID>, so modality-specific boundaries survive the shared path. On the voice side, the controlling object is the dialogue_state protocol, a three-valued field (end, follow_up, continuous) that tells the channel what to do after playback, separating the model's semantic judgment from the channel's state transitions. The voice channel also runs an explicit five-state machine—STOPPED, LISTENING, RECOGNIZING, THINKING, PLAYING—and skills use progressive disclosure, with always-on skills contributing full instructions while other skills are loaded from SKILL.md only when relevant.","core_discovery":"Homebot's central claim is that conversational home assistance can be built as a layered system in which all channels normalize their events into a common message representation and share one Agent Runtime, without flattening session boundaries. Messaging history is keyed by channel:chat_id and isolated per chat; voice history is keyed by voice:<UUID> and lasts only as long as one wake-word activation. The voice channel is a complete interaction pipeline: continuous local keyword spotting starts a bounded session, streaming recognition turns speech into a normal inbound message, streaming synthesis plays replies sentence by sentence, and a structured dialogue_state with three values—end, follow_up, continuous—decides whether the interaction stops, waits for an answer, or stays in a persistent multi-turn mode. The paper positions this architecture as a practical, extensible foundation for local deployment and states that it does not train wake-word, speech-recognition, speaker-recognition, or language models.","pith_inferences":["A decisive test not reported in the paper is whether the dialogue_state field stays accurate in a noisy home, since a wrong end-versus-follow_up choice breaks the interaction; collecting that field's error rate would quantify the design's main risk.","The same separation of response text from turn-control state could transfer to other voice agents beyond the home, because it removes the need to commit to a next-state decision before playback finishes.","The three-valued dialogue_state is a minimal declarative contract; extending it to states such as display-only or defer-to-later would let a channel handle more situations without changing the runtime.","Measuring per-turn token consumption with and without progressive disclosure would turn the paper's qualitative efficiency argument into a quantitative one, and could be done with the public implementation."],"forward_implications":["Adding a new channel requires only implementing the common lifecycle and sending contract; it automatically reuses the runtime's tools, skills, and session rules.","Voice and messaging histories remain separate by default, so a spoken exchange does not leak into a text chat and vice versa.","Because voice requests become ordinary inbound messages after recognition, any tool or skill written for text also works for hands-free voice.","The explicit dialogue_state lets the model decide whether the interaction ends, follows up, or continues, while the channel handles the actual state transitions.","With optional speaker verification attached to the request context, the runtime can distinguish a member from a guest, laying the groundwork for profile-conditioned memory and member-specific skills."],"supporting_citations":[{"why":"Motivates the household context by showing an LLM-driven smart-home system with goal-oriented reasoning.","marker":"[2]"},{"why":"Grounds the voice design in the finding that everyday voice interaction is embedded in shared physical and social settings.","marker":"[5]"},{"why":"Supplies the underlying idea that language models can learn to use external tools, which the tool loop builds on.","marker":"[6]"},{"why":"Provides the interleaved reasoning-and-acting loop that the Agent Runner's request–action cycle implements.","marker":"[9]"},{"why":"Represents the existing personal-agent systems whose design Homebot extends toward the household setting.","marker":"[1]"}],"fun_headline_variants":["Homebot: one agent runtime for voice and chat","Two session scopes, one Homebot brain","Homebot keeps chat and voice histories apart","From wake word to multi-turn with Homebot","Homebot's shared runtime, separate memories"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The design's practical value rests on untested reliability assumptions: local wake-word detection and streaming speech recognition must work in a noisy, shared household, and the language model must reliably emit valid tool calls and the correct one of the three state values that decide whether to end, ask a follow-up, or continue a conversation; Section 2.4 specifies the protocol but reports no field data or reliability measurements.","fun_headline_variants_meta":{"raw":{"variants":["Homebot: one agent runtime for voice and chat","Two session scopes, one Homebot brain","Homebot keeps chat and voice histories apart","From wake word to multi-turn with Homebot","Homebot's shared runtime, separate memories"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000142,"raw_usage":{"total_tokens":1107,"prompt_tokens":826,"completion_tokens":281,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":211}},"tokens_in":442,"tokens_out":281,"duration_ms":3257,"temperature":1.0,"reasoning_tokens":211,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:24:58.291855+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Homebot in a typical household for 100 wake-word activations at measured background noise levels; record the fraction of activations that reach RECOGNIZING and THINKING and the fraction in which the model's dialogue_state matches the user's intent, for example whether follow_up actually restarts recognition. A high wake-word false-reject rate or a high dialogue-state mismatch rate would show that the voice turn-control design does not hold up under the conditions it targets.","supporting_citations":[],"review_version":2}