{"id":"2e494636-1309-4825-a0e3-645c5ed30776","arxiv_id":"2607.17900","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Wrapping a TTS engine with an LLM-driven prompt-tool router improves instruction-following and naturalness over direct instruction control in synthetic evaluations.","lead":"This paper introduces Harness TTS, a control layer that chooses among a fixed set of style “tools”—each linked to a prompt audio—so a TTS engine can match the right tone to context. A smart generalist might read it because it shows how LLM-based tool routing is being used to make voice assistants respond with context-appropriate expression rather than just following literal instructions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Synthesis comparison conflates planner routing with style-matched prompt audio; no random/retrieval-prompt ablation isolates the Harness contribution.","rationale":"The reader identified LLM-generated test cases, teacher labels, and preference judges as the weakest assumption. That is a real threat to external validity, and the authors themselves acknowledge it in the Limitations section. However, the more immediate threat to the central causal claim is internal to the experimental design. The synthesis experiment varies two things at once: whether the planner selected the prompt, and whether the executor received a style-matched prompt audio instead of a default one. Since the TTS backends are reference-conditioned zero-shot models, the latter difference alone could produce large win-rate margins. Without a condition that uses the same registry audio but with a non-routing selection mechanism (random, retrieval, or oracle), the paper does not demonstrate that the LLM planner is responsible for the synthesis gains. This does not invalidate the system as a whole — a curated registry plus a planner may still be practical — but it weakens the specific contribution claim. The verdict remains CONDITIONAL because the missing ablation is addressable and the existing routing results provide partial support.","tokens_in":10995,"tokens_out":5307,"duration_ms":53147,"concrete_test":"Run the §4.2 synthesis protocol with a Random-Tool condition: replace Qwen3-4B with uniform random selection from the same 25-clip registry, keep CosyVoice3/VoxCPM2, instructions, and Gemini-3.1-Pro judge identical; report Harness, Random-Tool, and Instruct win rates with bootstrap CIs over the 135 cases. If Random-Tool's margin over Instruct approaches Harness's, the planner's routing is not the driver; if Random-Tool is near chance or Instruct, the routing decision matters.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing issue is that the §4.2 synthesis comparison cannot separate the routing layer's decision from the acoustic benefit of a style-matched prompt audio. Harness (planner + selected prompt audio) is compared only against Instruct (instruction text + default reference audio). Both CosyVoice3 and VoxCPM2 are zero-shot TTS models that condition strongly on reference audio; the Harness condition always supplies a prompt audio explicitly selected to match the requested style, while Instruct uses a single default clip. The reported 13.8–35.6 point win margins may therefore reflect the reference-audio advantage rather than the planner's routing quality. The routing task (§4.1) does show Qwen3-4B beats retrieval on teacher-aligned metrics, but no synthesis ablation links routing accuracy to speech quality: there is no random-prompt, retrieval-prompt, or oracle-prompt synthesis condition. The paper's central contribution is the externalized decision layer, but the end-to-end experiment only tests 'system with style-matched audio' vs 'system without it,' leaving the causal role of the planner unestablished.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Harness TTS, an external control layer for prompt-based TTS. It constructs a tool registry of style prompt audios with structured metadata, and uses a lightweight LLM planner to route multi-source observations (system defaults, user profile, scenario, implicit cues, explicit instructions) to a tool according to a priority policy. The selected prompt audio then conditions a zero-shot TTS engine. The paper evaluates routing accuracy against Gemini-generated teacher labels, planner latency, and synthesis quality measured by a Gemini preference judge plus UTMOSv2 and speaker similarity, comparing Harness against instruction-only control on CosyVoice3 and VoxCPM2. The reported results favor Harness on routing accuracy, latency, instruction-following win rates, and naturalness.","tokens_in":11220,"tokens_out":4427,"duration_ms":41446,"significance":"If the findings hold, the paper offers a practical, auditable way to add context-aware style control to existing TTS systems without retraining. Strengths include the explicit priority hierarchy, interpretable tool IDs, a realistic latency measurement setup with vLLM, and an honest limitations section that acknowledges the lack of human validation. However, because the evaluation is entirely LLM-generated and the synthesis comparison confounds planner routing with reference-audio choice, human validation and targeted ablations are needed before the central synthesis claim can be accepted.","major_comments":[{"comment":"The central claim that Harness improves instruction following over Instruct is not established because the comparison changes two variables simultaneously: the planner's routing decision and the prompt audio. Both CosyVoice3 and VoxCPM2 are zero-shot TTS models that condition on reference audio. In the Harness condition the executor always receives a style-matched prompt audio selected by the planner; in the Instruct condition it receives the instruction text plus a single default reference clip. The 13.8–35.6 point win margins in Fig. 3 could therefore be driven entirely by the acoustic benefits of a matched reference, not by the planner's routing. I request ablations that isolate the planner: e.g., Harness vs. random-prompt selection, retrieval-prompt selection, and oracle-prompt selection (the best possible prompt for the instruction), all with the same executor and the same registry.","section":"§4.2, Fig. 3"},{"comment":"The evaluation loop is entirely LLM-based: Gemini-2.5-Pro generates test cases and teacher labels (§4.1), Gemini-3.1-Pro judges synthesis preferences (§4.2), and §6 acknowledges that metrics 'await subjective validation' and that all test data are generated by LLMs. The T-* routing metrics measure agreement with a Gemini teacher, not ground-truth correctness; implicit and conflict subsets are especially vulnerable to teacher bias. Likewise, the Inst. win% is a pairwise preference by an LLM judge that may favor longer, structured, or otherwise distinctive outputs. For a paper whose contribution is practical voice-assistant behavior, at least one human listening test (e.g., MOS for style match and naturalness on a sample of the 135 cases) and a report of inter-annotator agreement are necessary. Without them, the headline margins are not evidence about human perception.","section":"§4.1, §4.2, §6"},{"comment":"The synthesis results are reported as point estimates without confidence intervals or significance tests. Figure 3 reports win rates over 135 test cases per backend (27 tags × 5 utterances), and Fig. 4 reports UTMOSv2 means with no variability. Many margins are sizable, but 'consistently achieves' requires statistical support; please report bootstrap CIs or pairwise significance tests (e.g., Wilcoxon signed-rank on per-tag scores). This is especially important because the win-rate differences are the paper's central quantitative claim.","section":"§5.3"}],"minor_comments":[{"comment":"Margin arithmetic: the abstract states CosyVoice3 margins of 23.1–35.6, but the figure shows 61.5 vs. 38.5 for the parameter subset (margin 23.0); the VoxCPM2 parameter margin reads 13.9 (55.4 vs. 41.5) rather than 13.8. Please align numbers or clarify rounding.","section":"Fig. 3 / Abstract"},{"comment":"The routing task uses a simulated 42-tool registry while the synthesis task uses a 25-clip voice library. The relationship between the two registries and how tools map to audio clips should be stated explicitly.","section":"§3.2, §4.1"},{"comment":"No code, data, or audio samples are released; release would substantially help reproducibility, especially for the LLM-judge evaluation.","section":"General"},{"comment":"The abstract's 'standard mode' is not defined in the text; clarify that it corresponds to the w/o CoT setting and cite the P95 First-ID latency of 41.2 ms.","section":"Table 5 / Abstract"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the routing/latency results are solid as automated benchmarks, but the synthesis claim needs the ablations and human validation described above. I do not see grounds for reject; the issues are addressable in a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the system is a genuine engineering contribution: a closed-set registry of style prompt audios with dual annotations (tags plus captions), a priority-aware observation schema, and a lightweight LLM router that picks the audio. Second, the headline synthesis result is confounded and does not yet isolate what the router does.\n\nThe routing experiments are the strongest part. Qwen3-4B with CoT beats keyword and embedding-retrieval baselines by large margins on explicit, implicit, and conflict subsets, and the largest gap appears exactly where retrieval should struggle—conflict resolution. The first-ID latency of roughly 41 ms at P95 for the 4B planner is a useful practical number. That part of the paper is coherent and worth taking seriously.\n\nThe soft spots are two, and one is load-bearing. The synthesis comparison in Section 4.2 changes two variables at once: Harness gets a style-matched prompt audio, Instruct gets a single default reference audio. Both CosyVoice3 and VoxCPM2 are zero-shot TTS models that condition heavily on reference audio. So the 13.8–35.6 point win margins could reflect the acoustic benefit of a matched prompt rather than the router's decision quality. There is no ablation with a random prompt, a retrieval prompt, or an oracle prompt. The paper's central claim—that the externalized decision layer improves instruction following—is not yet established by the synthesis data.\n\nThe second soft spot is the fully synthetic evaluation loop: Gemini generates the test cases, provides the teacher labels, and judges the synthesis preferences. The limitations section admits this. No human listening, no confidence intervals, no released audio or code. For routing, teacher-alignment is a defensible proxy; for synthesis, it is thin. The authors are honest about it, but the honesty does not turn an LLM preference score into a perceptual result.\n\nWho should read this: practitioners building voice assistants and researchers working on controllable TTS. It is an engineering contribution, not a new training paradigm. The architecture is plausible and the writing is straightforward.\n\nRecommendation: send it to peer review, but expect the authors to add the missing ablations and either collect human judgments or substantially temper the synthesis claims. The routing results can stand; the win-rate headline needs more work.","headline":"A plausible TTS control layer with a confounded synthesis evaluation: the win margins don't isolate the router, and the authors need ablations plus human or at least varied-prompt comparisons.","tokens_in":11719,"tokens_out":2715,"would_cite":false,"duration_ms":28116,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Expressive voice-assistant speech does not need a stronger TTS engine; it needs a Harness layer that reduces style control to closed-set selection among recorded prompt tools, and the paper's experiments show this routing approach beats dir","keywords":["expressive speech synthesis","harness layer","prompt-based TTS","LLM planner","tool routing","context-aware TTS","voice assistants","style control"],"falsifier":"A human A/B listening study using the same 135 instruction utterances, pitting Harness-generated speech against direct-instruction speech: if untrained listeners do not show preference margins near the reported 13.8–35.6-point gaps, the synthesis claim does not transfer to real users.","tokens_in":10897,"feed_emoji":"🎙️","tokens_out":5957,"duration_ms":57682,"temperature":0.7,"pith_summary":"The paper argues that the bottleneck for expressive voice assistants is not the generative capability of TTS engines but the absence of a structured interface between contextual signals and the synthesizer. It proposes a lightweight Harness layer that externalizes style decisions by turning them into closed-set prompt-tool selection: a curated registry of recorded style prompts, chosen by a compact LLM planner guided by a priority-aware observation schema. Experiments indicate that speech generated from the selected prompt audio follows instructions more reliably than speech generated by direct instruction conditioning, with win-rate margins of roughly 14 to 36 percentage points across two TTS backends, naturalness gains, and under 50 ms latency for the first routing decision. A sympathetic reader would take this as evidence that expressive control can be decoupled from the generator itself.","feed_headline":"A routing layer lifts TTS style-following by up to 36 points","feed_subtitle":"A Harness layer routes style to recorded prompt tools, improving expressiveness while keeping latency under 50 ms.","key_machinery":"The central mechanism is the Harness layer itself, instantiated as a lightweight LLM-based router. Three load-bearing pieces carry the argument: the tool registry, a small, speaker-specific set of style prompt audios with dual-annotation metadata; the observation schema, which organizes contextual signals into five decision fields with a fixed priority order so that conflicting requirements resolve predictably; and the closed-set routing interface, which converts unbounded style generation into the selection of an audio ID. The work performed by this mechanism is the separation of contextual decision-making from acoustic generation, making stylistic choices auditable, bounded, and stable wit","core_discovery":"The paper tries to establish that style control in text-to-speech can be externalized and made reliable by reformulating it as closed-set prompt-tool routing. A registry stores style prompt audios, each described by discrete tags (speed, volume, emotion) and a free-text caption. A text-only LLM planner receives a structured observation built from five prioritized fields — explicit instruction, implicit intent, scenario, user profile, and system default — and ranks candidate tools, with explicit user requirements overriding lower-priority context. The selected prompt audio then conditions an unmodified prompt-based TTS executor. The claimed result is that this arrangement outperforms direct i","pith_inferences":["The closed-set registry caps expressive range: any style not represented by a recorded prompt tool is unavailable, so the practical ceiling moves from the generator to the curation of the prompt library; a natural extension is per-user or dynamically growing registries.","Because the Harness condition supplies the planner with a longer, structured observation while the Instruct condition receives only a bare instruction, part of the win-rate gap may come from the LLM judge rewarding richer textual context rather than the audio itself — a human listening test is the direct check.","The harness pattern is not TTS-specific: any generative model whose output style is currently controlled by free-form prompting could be wrapped in the same priority-aware tool-selection layer, inheriting the same auditability benefits."],"forward_implications":["Existing prompt-based TTS engines gain context-aware style control without any retraining of the generator.","Every stylistic choice becomes auditable: the selected tool ID and prompt audio explain why a given utterance sounded the way it did.","Real-time interaction remains feasible, because synthesis only needs the first tool ID and the planner emits it in under 50 ms.","High-level scenario instructions (such as 'bedtime story' or 'news broadcast') can be realized by matching a preset tool or composing acoustic axes, which direct instruction conditioning handled poorly in the experiments.","The fixed priority policy gives system designers a predictable conflict-resolution rule: explicit instructions always override contextual signals."],"fun_headline_variants":["TTS style routing beats instruction-only by up to 36 points","Harness TTS: LLM planner routes prompt tools for expressive speech","Prompt-tool routing improves TTS style control, latency under 50 ms","Context-aware TTS: Externalized style control via harness layer","Style control as routing: LLM selects prompt audio for TTS"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The central claim stands or falls on whether LLM-generated test cases, teacher labels, and preference judgments are a faithful proxy for how humans perceive style and naturalness, which the paper's own limitations section says remains unvalidated.","fun_headline_variants_meta":{"raw":{"variants":["TTS style routing beats instruction-only by up to 36 points","Harness TTS: LLM planner routes prompt tools for expressive speech","Prompt-tool routing improves TTS style control, latency under 50 ms","Context-aware TTS: Externalized style control via harness layer","Style control as routing: LLM selects prompt audio for TTS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000322,"raw_usage":{"total_tokens":1681,"prompt_tokens":809,"completion_tokens":872,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":779}},"tokens_in":553,"tokens_out":872,"duration_ms":8390,"temperature":1.0,"reasoning_tokens":779,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T16:38:49.744119+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A human A/B listening study using the same 135 instruction utterances, pitting Harness-generated speech against direct-instruction speech: if untrained listeners do not show preference margins near the reported 13.8–35.6-point gaps, the synthesis claim does not transfer to real users.","supporting_citations":[],"review_version":1}