{"id":"f081b832-00e2-4c2a-a6ce-a331e65024a3","arxiv_id":"2606.30294","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Rhetor automates rehearsed live web-app demos with segment-synchronized narration and real-time voice QA using cross-modal UI-plus-code features, a grounded scripter, rehearsal loops, and timing invariants, with case-study metrics on four applications.","lead":"Rhetor is a proposed multi-agent system that automates live product demonstrations for web applications by generating rehearsed narrations synchronized to actions and handling real-time voice questions. A smart generalist might read it to understand potential AI-driven reductions in the cost and effort of software product demos in organizations.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Evaluation limited to case studies; proposed benchmark not executed","rationale":"The reader's weakest_assumption directly identifies the cross-modal representation and rehearsal convergence as the key unverified link; the absence of the proposed benchmark results is the concrete place where that assumption remains least secure. Full text would be required to check whether additional evidence appears beyond the abstract, but the abstract itself flags the gap.","tokens_in":1816,"tokens_out":323,"duration_ms":31243,"concrete_test":"Execute the ten-metric benchmark protocol on the six application categories (with ablations removing the cross-modal tiers or rehearsal loop); if the metrics show no statistically significant positive contribution from those components relative to baselines, the convergence claim does not generalize beyond the reported sessions.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim depends on the cross-modal feature representation (merging UI exploration with source-code analysis via discrete focus tiers) enabling a grounded scripter whose scripts converge reliably in the rehearsal loop. The abstract reports sigma-bar values (0.31-1.00 across 147 actions; ~0.92 on 53-action workload; convergence to 1.00 at iteration 2 on Excalidraw) from six sessions on four applications. However, it explicitly defines a benchmark protocol of ten metrics across six application categories that \"would establish, beyond the case study, whether each design choice contributes positively,\" indicating the protocol was not executed. This leaves the reliability claim supported only by limited case studies without systematic ablations or comparisons.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes Rhetor, a multi-agent system that takes a running web application and its source-code repository as input and outputs a rehearsed live demonstration featuring segment-synchronized narration and real-time voice question answering. Architectural elements include a cross-modal feature representation merging UI exploration with source-code analysis via discrete focus tiers, a grounded scripter constrained to observed UI elements and using multi-strategy semantic locators, a pre-presentation rehearsal loop with explicit convergence criteria and graceful degradation, and a runtime synchronization invariant linking browser actions to narration audio-end events. Empirical results are reported from six pipeline sessions on four deployed applications (including Excalidraw), with internal locator-firing rates (sigma-bar) spanning 0.31-1.00 over 147 actions; on a 53-action workload sigma-bar is ~0.92 and on Excalidraw the repair step yields convergence to 1.00 at iteration 2. The paper also defines (but does not execute) a benchmark protocol of ten metrics across six application categories.","tokens_in":1968,"tokens_out":558,"duration_ms":26166,"significance":"If the cross-modal representation and rehearsal loop reliably produce convergent scripts, the work could meaningfully reduce the recurring human cost of live product demonstrations in software organizations. The case-study metrics provide concrete evidence of feasibility on real deployed applications, including a public-domain reference, and the explicit definition of a benchmark protocol is a constructive contribution toward falsifiable evaluation. However, because the reported results are limited to six sessions without ablations, comparisons, or execution of the proposed benchmark, the demonstrated impact remains preliminary rather than conclusive.","major_comments":[{"comment":"Abstract: The central claim that the cross-modal feature representation (with discrete focus tiers) enables a grounded scripter whose scripts converge reliably via the rehearsal loop rests on sigma-bar values from six sessions on four applications. The manuscript itself states that the benchmark protocol of ten metrics across six categories 'would establish, beyond the case study, whether each design choice contributes positively,' indicating this protocol was not executed. This is load-bearing because the reliability claim cannot be assessed without systematic ablations or the defined benchmark.","section":"Abstract"},{"comment":"Abstract: The reported performance figures (sigma-bar of 0.92 on the 53-action workload; convergence to 1.00 at iteration 2 on Excalidraw) are presented without error bars, statistical tests, or a full experimental protocol. This weakens the empirical grounding for the claim that the rehearsal loop drives reliable convergence across the 147 scripted actions.","section":"Abstract"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed and constructive review. The manuscript presents the reported results explicitly as a case study on four deployed applications while defining (but not executing) a benchmark protocol for stronger claims. We address each major comment below and propose targeted revisions to improve clarity.","responses":[{"response":"We agree that the benchmark protocol was defined but not executed, as the manuscript already states. The central claim is scoped to the observed behavior in the six pipeline sessions: the cross-modal features, grounded scripter, and rehearsal loop produced sigma-bar values of 0.31-1.00 (with 0.92 on the 53-action workload and convergence to 1.00 at iteration 2 on Excalidraw). These are internal, per-session metrics of locator success during rehearsal, not a general reliability guarantee. We will revise the abstract to more explicitly frame the contribution as a feasibility case study on real applications (including a public-domain reference) and to note that the defined benchmark remains future work for systematic ablations.","revision_made":"partial","referee_comment":"[Abstract] Abstract: The central claim that the cross-modal feature representation (with discrete focus tiers) enables a grounded scripter whose scripts converge reliably via the rehearsal loop rests on sigma-bar values from six sessions on four applications. The manuscript itself states that the benchmark protocol of ten metrics across six categories 'would establish, beyond the case study, whether each design choice contributes positively,' indicating this protocol was not executed. This is load-bearing because the reliability claim cannot be assessed without systematic ablations or the defined benchmark."},{"response":"The figures are descriptive metrics computed directly from the six individual pipeline sessions on specific applications and workloads (147 actions total). Because each session constitutes a complete run rather than a sample from a larger population, standard error bars or inferential statistical tests are not applicable. We will add a clarifying sentence in the abstract and results section stating that these values are case-study observations of the rehearsal process rather than aggregated experimental statistics.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The reported performance figures (sigma-bar of 0.92 on the 53-action workload; convergence to 1.00 at iteration 2 on Excalidraw) are presented without error bars, statistical tests, or a full experimental protocol. This weakens the empirical grounding for the claim that the rehearsal loop drives reliable convergence across the 147 scripted actions."}],"tokens_in":1652,"tokens_out":497,"duration_ms":23905,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core of this paper is a system called Rhetor that chains UI exploration, source-code analysis, a rehearsal loop with explicit convergence, and a runtime sync rule to produce live demos that can handle real-time voice questions. The combination of those pieces into one pipeline for this specific task is what stands out as new compared to the general browser agents or fixed video tools mentioned.\n\nIt does a few things cleanly. The authors report locator-firing rates (sigma-bar) from actual runs on four applications, reaching about 0.92 on a 53-action workload and full convergence to 1.00 after two rehearsal iterations on Excalidraw. They also spell out a ten-metric benchmark across six application categories that would let future work test each component. That protocol is a concrete step even if it was not run here.\n\nThe main limitation is that all numbers come from six sessions on four apps with no error bars, no statistical tests, and no ablations. The paper itself notes that the benchmark would be needed to show whether the cross-modal features or the rehearsal loop actually drive the results, but that step was skipped. Scope is also narrow to web apps that accept the kinds of inputs the system can locate.\n\nThis is worth a look for anyone building multi-agent tools that need to stay grounded in both UI state and code. A reader who wants ideas for rehearsal loops or cross-modal tagging could pull useful details. It is not yet strong enough to change how demos are done in practice.\n\nI would send it to peer review. The architecture is described clearly enough and the case results give a starting point, but referees would need to press on the missing benchmark and the lack of controlled comparisons.","headline":"Rhetor assembles a multi-agent pipeline for rehearsed live demos with voice QA but rests on case-study metrics without executing its own proposed benchmark.","tokens_in":2472,"tokens_out":422,"would_cite":false,"duration_ms":19230,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Rhetor is a multi-agent system that converts a running web application and its source code into a rehearsed live demonstration with synchronized narration and real-time voice question answering.","keywords":["multi-agent systems","live demonstrations","web applications","source code analysis","narration synchronization","voice question answering","rehearsal loop","UI exploration"],"falsifier":"If a new application yields a locator-firing rate below 0.5 after multiple rehearsal iterations, the claim of reliable convergence would be falsified.","tokens_in":2720,"feed_emoji":"🤖","tokens_out":629,"duration_ms":24115,"temperature":0.7,"pith_summary":"Live product demonstrations require selecting features, performing interactions, narrating them, and answering questions in real time, which is expensive for software teams. The paper presents Rhetor as a system that accepts a running web app and its source-code repository to automate this process. It combines UI exploration with code analysis through a cross-modal representation, uses a constrained scripter with semantic locators, runs a rehearsal loop for convergence, and enforces synchronization between actions and narration audio. Experiments on four applications show high locator success rates after rehearsal, with a proposed benchmark to measure each component.","feed_headline":"Multi-agent system turns web apps into rehearsed live demos","feed_subtitle":"Rhetor merges UI and code analysis, rehearses scripts, and syncs narration for real-time voice answers.","key_machinery":"The cross-modal feature representation that merges UI exploration with source-code analysis into features tagged with discrete focus tiers, which supports a grounded scripter and rehearsal loop for reliable script convergence.","core_discovery":"Rhetor takes a running web application and its source-code repository as input and produces a rehearsed live demonstration with segment-synchronized narration and real-time voice question answering, using a cross-modal feature representation, grounded scripter, pre-presentation rehearsal loop, and runtime synchronization invariant.","pith_inferences":["Organizations could reduce presenter preparation time by running the system on updated codebases before each demo event.","The same tiered feature approach might support automated testing scripts that remain stable across UI changes.","Real-time voice Q&A could be extended by linking answers directly to the same cross-modal features used for narration."],"forward_implications":["The rehearsal loop with graceful degradation permits demos to continue even when some actions cannot be executed.","The runtime synchronization invariant ensures each browser action aligns exactly with the end of its narration segment.","Locator repair during rehearsal drives convergence to full success on applications like Excalidraw.","A ten-metric benchmark protocol across six application categories can test whether each architectural choice improves outcomes."],"fun_headline_variants":["Rhetor rehearses live demos from web apps with voice QA","Multi-agent system syncs narration to browser actions","Rehearsed demos use cross-modal UI and code features","Rhetor runs pre-presentation rehearsal for voice QA"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Merging UI exploration with source-code analysis into tiered features allows the scripter to produce scripts that converge reliably via the rehearsal loop.","fun_headline_variants_meta":{"raw":{"variants":["Rhetor rehearses live demos from web apps with voice QA","Multi-agent system syncs narration to browser actions","Rehearsed demos use cross-modal UI and code features","Rhetor runs pre-presentation rehearsal for voice QA"]},"model":"grok-4.3","cost_usd":0.006739,"raw_usage":{"total_tokens":3172,"prompt_tokens":738,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":67387000,"prompt_tokens_details":{"text_tokens":738,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2369,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":738,"tokens_out":65,"duration_ms":20885,"temperature":1.0,"reasoning_tokens":2369,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T06:00:38.521004+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"If a new application yields a locator-firing rate below 0.5 after multiple rehearsal iterations, the claim of reliable convergence would be falsified.","supporting_citations":[],"review_version":1}