{"id":"d992a718-1b3d-43ec-af5f-670dcbfef1fb","arxiv_id":"2608.07631","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Playback-aligned context repair lifts referent anchoring after user interruptions from 25.0% to 96.3% on a new 108-case full-duplex voice benchmark.","lead":"This paper describes PACE, middleware that tracks how much of an AI assistant's audio actually played on the user's device, and cancels or rewrites conversational context that the user never heard. It shows that this playback anchoring fixes a failure mode called Generative Context Mis-anchoring, lifting referent accuracy from 25% to 96.3% on a new benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 96.3% RAA rests on ChatGPT-5.5 judgments for the 'next' operation, where the judge cannot observe the generated-but-unplayed continuation; the paper itself concedes this in §5.3 and §8, so the pooled accuracy may be inflated.","rationale":"The reader's weakest assumption is the same concern I would flag: the automated judge's inability to verify 'next' responses. The paper is unusually transparent about this limitation, explicitly stating in Section 5.3 that posterior judging is weaker for 'next' and in Section 8 that human-validated evaluation of this operation is left to future work. Since 'next' constitutes 36 of 108 GCM-Bench cases, the pooled RAA and the paired McNemar counts are directly affected. This is a load-bearing concern for the headline number, not a peripheral caveat.\n\nAt the same time, the concern does not justify rejecting the paper or even moving the verdict away from CONDITIONAL. The 'elaborate' operation improves from 2.78% to 94.44% and the 'repeat' operation from 11.11% to 97.22%; both are far less affected by the judge limitation because the played item alone determines correctness. The FDB compatibility result is reported descriptively and the quality-score change is small, so it is not the main risk. The remaining weaknesses (single model, hyperparameters tuned on the same set, lookback batches collected separately) are real but secondary; they also argue for a conditional rather than full acceptance.\n\nThe concrete test I propose would settle the main uncertainty directly: human annotation of the 36 'next' cases, with access to the unplayed continuation, would reveal whether the 96.3% pooled figure is inflated and by how much. Until that is done, the paper's own caveat should be taken at face value and the headline should be read as provisional.","tokens_in":18352,"tokens_out":4134,"duration_ms":42342,"concrete_test":"Take all 36 'next'-operation GCM-Bench trials (both baseline and PACE outputs) and have two independent human annotators label whether the response advances from the item at the playback boundary, giving annotators the playback boundary, the full generated-but-unplayed assistant transcript, the before/after text, and the interruption utterance. Compute RAA for 'next' from human labels while retaining judge labels for 'elaborate' and 'repeat'. If human-validated 'next' accuracy for PACE falls materially below 97.22% (or baseline below 61.11%), the pooled 96.3% headline is overstated and should be reported with 'next' marked as unvalidated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that PACE raises Referent Anchoring Accuracy from 25.0% to 96.3% on GCM-Bench. That number is only as sound as the automated judge used to label each response. For the 'next' operation, which is exactly one third of the 108 cases, the judge is given the played prefix, the post-interruption response, and the operation type, but not the generated-but-unplayed continuation. The paper concedes in Section 5.3 that for 'next' the judge 'cannot always verify the identity and order of the true next item' and 'may accept a new, non-repeated item even when the model has advanced from an internal item beyond the playback boundary.' Section 8 repeats this limitation and says reliable resolution may require case-level human inspection against both the playback boundary and the generated-but-unplayed sequence, which the paper does not perform. Because 'next' is one third of the benchmark, the pooled 96.3% and the 78:1 repair-to-regression count depend on unvalidated judgments on exactly the cases where the referent is least observable. The 'elaborate' and 'repeat' operations provide cleaner evidence, and the direction of the effect is unlikely to reverse, but the headline number as stated is not yet settled.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies Generative Context Mis-anchoring (GCM), a failure mode in full-duplex LLM-based voice dialogue where user speech is interpreted against assistant content that was generated but never played. It proposes PACE, a middleware layer that tracks the client playback boundary via an output-turn ledger and playback acknowledgments, and after an interruption projects only the played prefix into the model context, using an audio re-injection adapter for black-box speech models. The authors implement this path in a browser-based assistant with a black-box model, release a new 108-case benchmark (GCM-Bench), and report a Referent Anchoring Accuracy improvement from 25.0% to 96.3% over a cancellation-only baseline, along with a compatibility study on 200 Full-Duplex-Bench v1 samples showing no loss in response quality and modest added latency. The paper also includes ablations of delimiter wording and lookback window and discusses the limitations of the automated judge for the 'next' operation.","tokens_in":147,"tokens_out":6563,"duration_ms":78830,"significance":"If the results hold, the paper makes a valuable contribution: it identifies and names a real distributed-consistency failure (GCM) that existing full-duplex benchmarks do not measure, and it demonstrates a provider-independent, deployment-relevant remedy. The work is notable for pairing a controlled, reproducible benchmark with a live end-to-end implementation in which playback boundaries come from actual browser PlaybackAck events rather than simulation. The paired 108-case design with McNemar testing, the 78:1 improvement-to-regression count, the delimiter and lookback ablations, and the external Full-Duplex-Bench compatibility check are concrete strengths. The released dataset and the explicit separation of physical vs. semantic commit boundaries will be useful to the community even if the headline accuracy changes after human validation.","major_comments":[{"comment":"The pooled RAA of 96.3% is not fully supported for the 'next' operation. As the paper acknowledges in §5.3 and §8, the ChatGPT-5.5 judge observes only the played prefix and the post-interruption response, not the generated-but-unplayed continuation, so it cannot verify whether a response advances from the item at the playback boundary or from an unheard internal item. Since 'next' constitutes 36 of the 108 cases, the 35/36 next successes and the resulting 78:1 improvement-to-regression ratio may be inflated by judge acceptance of plausible novel items. The central claim in the abstract and conclusion should be conditioned on human validation of these cases (or on a re-analysis that excludes or re-labels them), or the headline should be explicitly restricted to elaborate and repeat operations.","section":"§5.3, Table 2"},{"comment":"Several design choices that affect the reported RAA were selected using the same 108 evaluation cases: the interruption delays (12/16/20 s) were chosen after pilot runs on the same task, and the delimiter wording (P1) and lookback window (5 s) were selected by ablating on the same 108 cases. This does not invalidate the paired comparison, but it means the reported 96.3% is a tuned estimate on the evaluation set rather than an out-of-sample performance. Please provide a validation split or otherwise clarify what, if anything, was held out; without this, the absolute accuracy is difficult to interpret, even though the direction of the effect is likely robust.","section":"§5.2, §6.3"},{"comment":"The lookback ablation is not fully paired: the 2.5-s and 10-s conditions were collected in a later realtime batch than the 5-s condition. The paper states this and interprets the near-tie cautiously, but the conclusion that '5 s is a conservative default' and the interpretation of the 10-s degradation depend on cross-batch comparisons that cannot separate batch effects from window length. At minimum, the claim should be softened to acknowledge that the 5-s versus 2.5-s comparison is not a same-batch paired test.","section":"§6.3, Table 4"}],"minor_comments":[{"comment":"There is a formatting error in the abstract: 'failureGenerative Context Mis-anchoring' should be 'failure Generative Context Mis-anchoring' with a space.","section":"Abstract"},{"comment":"The notation P_delim is used in the audio re-injection adapter description before it is formally defined; please introduce it earlier or add a brief parenthetical definition at first use.","section":"§3.3"},{"comment":"The definition of TOR is only given implicitly as 'turn-obedience rate' from Full-Duplex-Bench; the manuscript should state explicitly how a response is judged as turn-obedient in GCM-Bench, especially since TOR is reported as 100% for all conditions.","section":"§5.3"},{"comment":"The FDB compatibility result is reported with a 0.020-point quality difference and a 58.7 ms latency increase; it would be helpful to report the distribution of the quality scores (e.g., fraction of 5s) to show the effect is not driven by a ceiling artifact.","section":"§6.4"},{"comment":"The discussion of future work mentions 'case-level human inspection against both the playback boundary and the generated-but-unplayed sequence'; this is precisely the validation needed for the current headline result, and the paper would be stronger if a small-scale version of this protocol were included in this revision.","section":"§8"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a speech/dialogue systems venue and the core idea is timely. The main concern is whether the headline RAA can be trusted given the acknowledged judge limitation for one-third of the dataset; the authors' own discussion in §5.3 and §8 suggests they are aware of the fix. A major revision that adds human validation or re-scopes the claim would make the contribution publishable. I would also encourage the editor to ask for a clearer statement of what was selected on the test set versus held out, as this affects the interpretability of the absolute accuracy."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the thing to know: PACE is not another turn-taking or overlap model. It targets a distinct failure mode—the model's dialogue state advancing past what the user has actually heard—and it makes that precise with a playback boundary abstraction. That's a real problem in commercial full-duplex voice agents, and this paper gives it a name, a framework, and a working prototype. That alone makes it worth reading.\n\nWhat's well done: The design separates generation from grounding, uses turn-local coordinates for acknowledgments, and defines projection adapters for different runtimes. They implement the audio-only path end-to-end in a browser against a black-box speech API without modifying the model. The evaluation uses 108 paired live trials with actual playback tracking, not simulation. They release GCM-Bench, and they run ablations on delimiter wording and lookback length. The paper is also honest: it flags the next-operation judging limitation twice and lists applicable scope explicitly.\n\nThe soft spots are real but manageable. The headline 96.3% RAA depends on a ChatGPT-5.5 judge that, for the 'next' operation (one third of GCM-Bench), cannot observe the generated-but-unplayed continuation. The paper says this itself in Section 5.3, confirming the stress-test concern. So the pooled accuracy is likely an overestimate. The good news is that the direction is unlikely to reverse—the elaborate and repeat operations improve from near floor to mid-90s, and those don't depend on the unplayed continuation. Still, the next-operation numbers should be reported separately with a human-validated protocol before the headline is trusted.\n\nTwo smaller concerns: hyperparameters (lookback length, delimiter wording, interruption delays) were tuned on the same 108 cases that produce the headline, so some overfitting is baked in. And the study uses a single model and TTS-generated user trajectories, so it demonstrates the mechanism rather than estimating real-world prevalence. The FDB compatibility check is fine as a sanity check but is descriptive.\n\nOverall, the core idea is sound, the implementation is real, and the limitations are stated rather than hidden. This deserves a serious peer review conversation. I'd want the next-operation scoring fixed or at least human-annotated before accepting the paper's central number, but the contribution stands on its own.","headline":"A real failure mode, a clean abstraction, and a solid end-to-end implementation—but the headline number rests on a judge that can't see the unplayed continuation for a third of the cases.","tokens_in":19193,"tokens_out":2676,"would_cite":true,"duration_ms":24848,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a middleware layer which anchors model-facing dialogue context to the client's playback boundary can repair full-duplex voice assistants' tendency to answer questions about assistant content the user never heard.","keywords":["full-duplex voice dialogue","generative context mis-anchoring","playback grounding","interruption handling","referent anchoring","context repair","GCM-Bench"],"falsifier":"Have human annotators judge all 'next' cases with access to the full generated-but-unplayed continuation and compare their verdicts to the automated judge's; if agreement is clearly below the reported 97.2% 'next' accuracy, the headline 96.3% Referent Anchoring Accuracy is inflated. A second check is to re-run GCM-Bench with two additional LLM judges and measure inter-judge agreement.","tokens_in":18133,"feed_emoji":"🗣️","tokens_out":9159,"duration_ms":78375,"temperature":0.7,"pith_summary":"This paper tries to establish that the failure mode it calls Generative Context Mis-anchoring (GCM) is real, measurable, and repairable without touching the underlying speech model. GCM happens because a server generates assistant speech faster than the client can play it, so the model's conversation state runs ahead of what the user could have heard; a user interruption can then be answered against unheard content. The proposed fix, PACE, is a middleware layer that tracks how far playback has actually progressed and, after an interruption, re-presents only the played audio prefix to the model before releasing the user's request. On a new 108-case benchmark, this raises Referent Anchoring Accuracy from 25.0% to 96.3% relative to a cancellation-only baseline, and on 200 official interruption samples it preserves response quality while adding roughly 59 ms only on the interruption path. A sympathetic reader should care because this is a correctness problem that conventional interruption metrics miss: both repaired and unrepaired systems answer fluently, but only the playback-aligned system answers about what the user actually heard.","feed_headline":"Playback-aligned context repair lifts anchoring 25% to 96%","feed_subtitle":"A middleware layer anchors what the model hears to what the user heard, fixing mis-anchored interruptions.","key_machinery":"The load-bearing object is the playback boundary, represented as a turn-local pair (audio_turn_id, played_samples) that the client reports on every uplink tick and the server snapshots at speech onset. Around it PACE builds an OutputTurnLedger that records generated audio, optional text, and optional semantic spans for each assistant turn, plus a forward gate that forces the ordering cancel, injected context, then held user audio. The implemented adhesive is the audio re-injection adapter: it slices the most recent five seconds of played audio from the turn-indexed buffer, resamples it to the model's input rate, appends a pre-synthesized spoken delimiter such as \"That was what I heard before interrupting...\", and sends the concatenation to the model's audio input before releasing post-interruption speech. That adapter carries the argument because it grounds the next response at the playback boundary using only audio, which every target runtime accepts, and it never assumes text transcripts, conversation-item truncation, or KV-cache access.","core_discovery":"The paper's central claim is that the divergence among generated output, client playback, and model-facing dialogue state is a distributed consistency failure, and that a turn-local playback boundary is the right system-observable anchor for repairing it. PACE assigns each assistant response a turn id, keeps a server-side ledger with a copy of the delivered audio, and has the client continuously report how many samples of that turn have been rendered. On a confirmed interruption it snapshots this boundary, cancels and revokes everything beyond it, then projects the played prefix into the model before the user's post-interruption speech, so the model's next response is grounded at the point the user could have heard. The implemented audio-only path works against a black-box speech model with no transcript or cache access: it slices the last five seconds of played audio, appends a spoken delimiter, and injects that audio into the model's input buffer. The paper reports 96.3% Referent Anchoring Accuracy on its 108-case GCM-Bench (versus 25.0% for cancellation alone, McNemar p<0.001), repairing 78 baseline failures while introducing one regression, and unchanged interruption quality on the 200-sample compatibility study.","pith_inferences":["Editorial inference: the same boundary could gate irreversible actions — PACE already separates generation from action commitment, so a natural extension is to block any tool call whose triggering proposal was never played, which the paper motivates but does not implement.","Editorial inference: because the paper concedes that the automated judge cannot verify 'next' responses without the unplayed continuation, the true gain on that third of the benchmark is the number to scrutinize; a human-annotated version of GCM-Bench would either confirm or lower the headline 96.3%.","Editorial inference: the audio re-injection adapter should transfer to cascaded ASR-LLM-TTS pipelines by substituting transcript re-injection for audio re-injection, giving a testable cross-runtime extension the paper leaves implicit."],"forward_implications":["Any full-duplex voice assistant whose context advances with generation can silently answer about unheard content; the paper turns this into a named, measurable failure (GCM) and a benchmark (GCM-Bench) that interruption metrics do not detect.","Playback-aligned context repair works through a black-box speech model: the implemented path needs only an audio input buffer and a way to cancel output, not provider-specific truncation or model internals.","A short spoken delimiter and a five-second lookback are enough to anchor referents in the tested setting; widening the lookback to ten seconds degrades anchoring because it introduces competing candidates.","Ordinary topic-switching interruptions are not harmed by the repair: on the 200 official samples every interruption receives a response and judged quality is essentially unchanged, with the added latency confined to the interruption path.","The mechanism generalizes to cascaded systems in principle: when a runtime provides reliable audio-to-text alignment, the same playback boundary can rewrite prompt history instead of re-injecting audio."],"supporting_citations":[{"why":"Supplies the 200 official interruption samples used for the compatibility and latency study.","marker":"[13]"},{"why":"The closest existing repair mechanism (provider item truncation); PACE generalizes beyond its conversation-item abstraction.","marker":"[17]"},{"why":"Grounds the premise that rendered content is a conservative proxy for what the user could have heard.","marker":"[4]"},{"why":"Represents learned full-duplex models that handle overlapping speech but do not address the generation-playback gap.","marker":"[6]"},{"why":"Production live API that cancels generation on interruption while leaving playback-to-dialogue mapping to the app.","marker":"[8]"},{"why":"Shows why transport-level delivery feedback alone cannot supply semantic dialogue position.","marker":"[21]"},{"why":"Provides the client audio clock and scheduling used to compute played_samples.","marker":"[26]"}],"fun_headline_variants":["Playback anchoring lifts voice AI references from 25% to 96%","Middleware anchors AI context to what user heard, hitting 96% accuracy","Full-duplex voice fixed: PACE uses playback boundary, 96% reference accuracy","Playback-aligned context repair boosts referent anchoring to 96%","Anchor model context to playback: PACE cuts mis-anchoring from 75% to 3.7%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline accuracy figure assumes the automated judge can tell which list item a response anchors to, and the paper concedes that for one third of the benchmark ('next' requests) the judge cannot always verify the true next item because the generated-but-unplayed continuation is invisible to it.","fun_headline_variants_meta":{"raw":{"variants":["Playback anchoring lifts voice AI references from 25% to 96%","Middleware anchors AI context to what user heard, hitting 96% accuracy","Full-duplex voice fixed: PACE uses playback boundary, 96% reference accuracy","Playback-aligned context repair boosts referent anchoring to 96%","Anchor model context to playback: PACE cuts mis-anchoring from 75% to 3.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000477,"raw_usage":{"total_tokens":2411,"prompt_tokens":1039,"completion_tokens":1372,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":655,"completion_tokens_details":{"reasoning_tokens":1263}},"tokens_in":655,"tokens_out":1372,"duration_ms":10551,"temperature":1.0,"reasoning_tokens":1263,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:28:03.549090+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have human annotators judge all 'next' cases with access to the full generated-but-unplayed continuation and compare their verdicts to the automated judge's; if agreement is clearly below the reported 97.2% 'next' accuracy, the headline 96.3% Referent Anchoring Accuracy is inflated. A second check is to re-run GCM-Bench with two additional LLM judges and measure inter-judge agreement.","supporting_citations":[{"cited_title":"2003.RTP: A Transport Protocol for Real-Time Applications","cited_arxiv_id":null,"evidence_quote":"Shows why transport-level delivery feedback alone cannot supply semantic dialogue position."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the client audio clock and scheduling used to compute played_samples."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The closest existing repair mechanism (provider item truncation); PACE generalizes beyond its conversation-item abstraction."},{"cited_title":"Clark and Susan E","cited_arxiv_id":null,"evidence_quote":"Grounds the premise that rendered content is a conservative proxy for what the user could have heard."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Production live API that cancels generation on interruption while leaving playback-to-dialogue mapping to the app."}],"review_version":1}