{"id":"f8527818-fc47-4ad2-8dd1-04c37a53c71d","arxiv_id":"2607.01964","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Unsupervised LLM input rewriting for dialogue discourse parsing introduces more regressions than repairs and has a practical ceiling on error repair, necessitating rewritability prediction for effective use with frozen parsers.","lead":"This preprint investigates whether large language models can rewrite dialogue utterances without supervision to improve discourse parsing accuracy. It concludes that such rewriting often causes more problems than it solves, suggesting the need for better ways to decide when rewriting is appropriate.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader's weakest_assumption inverts the paper's stance: the manuscript does not presuppose that zero-shot/parser-feedback clarification will succeed; it reports that it does not. The empirical ceiling claim is therefore not undermined by the assumption the reader flags. Full-text access removes the abstract-only limitation that produced the original UNVERDICTED verdict, but does not surface an internal inconsistency or untested premise that would alter the reported findings.","tokens_in":1793,"tokens_out":293,"duration_ms":23783,"concrete_test":"Recompute the regression/repair counts from the main multi-dataset table after replacing the zero-shot prompt template with an independently written prompt that explicitly instructs preservation of discourse markers; if net regressions remain >10% higher than repairs, the core unreliability finding is robust to prompt variation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on empirical comparisons showing parser-agnostic zero-shot rewriting produces net regressions and that best-of-8 sampling leaves a substantial unrepairable error fraction. These observations are presented as dataset- and parser-spanning results rather than as a universal proof; the subsequent GRPO-trained clarifier result is framed as partial mitigation only. No hidden assumption (e.g., perfect prompt optimality or exhaustive sampling) is required for the narrower claim that current unsupervised rewriting is unreliable under the tested conditions.","agreement_with_reader":"disagree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that under realistic unsupervised conditions, LLM-based input rewriting for improving frozen dialogue discourse parsers (using zero-shot prompting or frozen parser feedback) is unreliable: parser-agnostic rewriting introduces more regressions than repairs across three SDRT datasets because edits disrupt discourse cues, a best-of-8 analysis shows a substantial fraction of errors are unrepairable by rewriting alone, and even a GRPO-trained parser-aware clarifier reduces regressions by up to 37% via conservative abstention but fails to yield consistent parsing improvements. The work reframes clarification as a selective intervention problem and identifies rewritability prediction as the key missing capability.","tokens_in":1832,"tokens_out":622,"duration_ms":30817,"significance":"If the empirical results hold under the tested conditions, the paper makes a useful contribution by providing concrete evidence of the limitations of unsupervised rewriting strategies in discourse parsing pipelines. The dataset- and parser-spanning nature of the regression findings, the identification of a practical ceiling via best-of-8 sampling, and the constructive reframing toward rewritability prediction are strengths. The GRPO approach for learning from parser feedback is noted as a reproducible element.","major_comments":[{"comment":"Abstract and experimental results: the central claim that parser-agnostic rewriting 'often introduces more regressions than repairs' is load-bearing, yet the manuscript must specify the exact operationalization (e.g., delta in parsing F1, error-type counts, or utterance-level success/failure) and report per-dataset/per-parser breakdowns with confidence intervals to substantiate the 'net regressions' conclusion.","section":"Abstract / §5 Results"},{"comment":"Best-of-8 rewriting analysis: the claim of a 'practical ceiling' with 'a large fraction of errors not repairable' directly supports the paper's recasting of the problem; the manuscript should detail the selection criterion (oracle vs. parser-score based), the precise fraction, and whether additional samples beyond 8 were tested, as these details determine whether the ceiling is methodological or fundamental.","section":"§5.2 Best-of-8 analysis"},{"comment":"GRPO-trained clarifier: the reported 'up to 37%' regression reduction is presented as partial mitigation, but the manuscript must clarify whether this yields net parsing gains on any dataset or merely reduced harm, and whether the training loop introduces any feedback leakage from the frozen parser; this distinction is required to support the claim that it 'still fails to produce selectivity-aware clarifications that consistently improve parsing'.","section":"§6 GRPO clarifier"}],"minor_comments":[{"comment":"The acronym GRPO should be expanded on first use.","section":"Abstract"},{"comment":"Clarify whether 'last-utterance clarification' operates on the final utterance in isolation or incorporates full dialogue history.","section":"Abstract"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback and positive assessment of the contribution. We agree that additional precision on operational definitions, selection criteria, and outcome distinctions will strengthen the manuscript and will incorporate these clarifications in the revision.","responses":[{"response":"We operationalize net regressions via utterance-level changes in parsing F1: a rewrite counts as a regression when post-rewrite F1 is lower than the original input F1 (and a repair when higher). We will add per-dataset and per-parser tables reporting mean F1 deltas together with 95% bootstrap confidence intervals in the revised §5 to make this explicit and to substantiate the aggregate claim.","revision_made":"yes","referee_comment":"[Abstract / §5 Results] Abstract and experimental results: the central claim that parser-agnostic rewriting 'often introduces more regressions than repairs' is load-bearing, yet the manuscript must specify the exact operationalization (e.g., delta in parsing F1, error-type counts, or utterance-level success/failure) and report per-dataset/per-parser breakdowns with confidence intervals to substantiate the 'net regressions' conclusion."},{"response":"The best-of-8 analysis employs oracle selection (the rewrite among the eight that produces the largest F1 gain relative to the original). We will report the exact fraction of errors that remain unrepairable even under this oracle selection and confirm that sampling was limited to eight candidates; no further samples were evaluated because the analysis was intended only to establish an empirical upper bound rather than to optimize sampling strategy.","revision_made":"yes","referee_comment":"[§5.2 Best-of-8 analysis] Best-of-8 rewriting analysis: the claim of a 'practical ceiling' with 'a large fraction of errors not repairable' directly supports the paper's recasting of the problem; the manuscript should detail the selection criterion (oracle vs. parser-score based), the precise fraction, and whether additional samples beyond 8 were tested, as these details determine whether the ceiling is methodological or fundamental."},{"response":"The GRPO clarifier reduces the number of regressions (by up to 37%) through learned abstention but produces no net parsing F1 gains on any of the three datasets; overall performance remains statistically indistinguishable from the no-rewriting baseline. The training loop uses only scalar F1 feedback from the frozen parser as a reward signal and performs no updates to the parser itself, so no parameter leakage occurs. We will add these explicit statements to §6.","revision_made":"yes","referee_comment":"[§6 GRPO clarifier] GRPO-trained clarifier: the reported 'up to 37%' regression reduction is presented as partial mitigation, but the manuscript must clarify whether this yields net parsing gains on any dataset or merely reduced harm, and whether the training loop introduces any feedback leakage from the frozen parser; this distinction is required to support the claim that it 'still fails to produce selectivity-aware clarifications that consistently improve parsing'."}],"tokens_in":1562,"tokens_out":644,"duration_ms":30159,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that parser-agnostic input rewriting with LLMs under zero-shot or frozen-parser feedback conditions tends to hurt discourse parsing accuracy more than it helps, and even sampling multiple rewrites leaves a large share of errors unfixable.\n\nThe paper tests this on three SDRT datasets with several parsers and shows that edits aimed at ellipsis or reference resolution frequently disrupt the discourse cues the downstream parser depends on. The best-of-8 analysis is the clearest new piece of evidence: it sets a realistic upper bound rather than claiming rewriting can be tuned to perfection. Training a clarifier with GRPO to abstain more conservatively cuts regressions by up to 37 percent, which is a measurable improvement over pure zero-shot, even if it does not produce consistent net gains.\n\nThis is useful because it moves the discussion from proposing new rewriting techniques to identifying when intervention is safe. The experiments are straightforward comparisons that build directly on the supervised clarification literature without circular claims or invented metrics. The citation pattern is appropriate and the results hold across setups.\n\nThe soft spots are limited. The GRPO result is presented as partial mitigation, and the exact mechanics of parser feedback integration could use more unpacking in a full read, but the central pattern of regressions versus repairs does not appear to rest on fragile assumptions. No load-bearing flaws stand out from the reported evidence.\n\nThis is for people working on dialogue systems or agent pipelines that keep parsers frozen and try input-side fixes. A reader who wants data on the gap between supervised lab results and deployment conditions will find it worth their time. It deserves peer review because the empirical observations are new, reproducible in principle, and point to a concrete open problem.","headline":"Unsupervised zero-shot rewriting for dialogue discourse parsing causes net regressions across parsers and datasets, with a practical ceiling on what input edits can fix; the paper's value is in shifting focus to rewritability prediction.","tokens_in":2364,"tokens_out":430,"would_cite":true,"duration_ms":20878,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Unsupervised LLM rewriting for dialogue discourse parsing introduces more regressions than repairs.","keywords":["dialogue discourse parsing","input rewriting","large language models","SDRT","zero-shot prompting","regression analysis","rewritability prediction","selective intervention"],"falsifier":"Train a rewritability predictor on held-out data and measure whether gating rewriting attempts behind its predictions produces higher overall parsing accuracy than applying rewriting to every utterance.","tokens_in":2662,"feed_emoji":"","tokens_out":649,"duration_ms":28848,"temperature":0.7,"pith_summary":"The paper tests whether LLMs can rewrite fragmentary dialogue utterances to improve frozen discourse parsers when no supervised clarification data exists. It evaluates zero-shot prompting and parser-feedback methods on three SDRT datasets with multiple parsers. Parser-agnostic edits frequently break discourse cues that the parsers rely on, producing more new errors than fixes. A best-of-8 analysis shows many parsing mistakes remain unrepairable by rewriting. The work concludes that rewritability prediction—deciding in advance whether an utterance can be fixed—is the missing capability needed for selective intervention.","feed_headline":"LLM rewriting hurts discourse parsing more than it helps","feed_subtitle":"Parser-agnostic edits break discourse cues, and many errors remain unfixable even after repeated attempts.","key_machinery":"Rewritability prediction, the decision of whether an utterance is repairable before any rewriting intervention is applied.","core_discovery":"Across three SDRT datasets and multiple parsers, last-utterance clarification via zero-shot prompting or frozen-parser feedback is far less reliable than supervised settings. Parser-agnostic rewriting introduces more regressions than repairs because edits that resolve ellipsis or references also disrupt existing discourse relations. Best-of-8 rewriting reveals a practical ceiling where a large fraction of errors cannot be repaired through input rewriting alone. A parser-aware clarifier trained with GRPO reduces regressions by up to 37 percent through conservative abstention yet still fails to deliver consistent parsing gains. These results recast clarification as a selective intervention pro","pith_inferences":["If rewritability prediction proves learnable, the same gating approach could be tested on other frozen downstream models beyond discourse parsing.","The observed ceiling on repairable errors suggests that input-side optimization alone may need to be combined with model-side adaptations for further gains.","One could measure whether rewritability predictors transfer across different parsers or datasets without retraining."],"forward_implications":["Clarification must be applied selectively rather than as a default step in the pipeline.","Zero-shot LLM rewriting cannot be assumed to improve discourse parsing accuracy.","Training clarifiers to abstain reduces regressions but does not guarantee net parsing gains.","Agentic systems that rely on input rewriting need an explicit rewritability decision mechanism."],"fun_headline_variants":["LLM rewriting causes more parsing regressions than repairs","Zero-shot edits disrupt discourse relations in dialogue parsing","Many errors unfixable even after best-of-8 LLM rewrites","Parser-aware method reduces regressions but lacks consistent gains"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Zero-shot prompting or feedback from a frozen parser can produce clarifications that fix errors without introducing new ones under realistic deployment conditions.","fun_headline_variants_meta":{"raw":{"variants":["LLM rewriting causes more parsing regressions than repairs","Zero-shot edits disrupt discourse relations in dialogue parsing","Many errors unfixable even after best-of-8 LLM rewrites","Parser-aware method reduces regressions but lacks consistent gains"]},"model":"grok-4.3","cost_usd":0.004967,"raw_usage":{"total_tokens":2462,"prompt_tokens":735,"num_sources_used":0,"completion_tokens":62,"cost_in_usd_ticks":49674500,"prompt_tokens_details":{"text_tokens":735,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1665,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":735,"tokens_out":62,"duration_ms":17506,"temperature":1.0,"reasoning_tokens":1665,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-03T14:58:32.597810+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Train a rewritability predictor on held-out data and measure whether gating rewriting attempts behind its predictions produces higher overall parsing accuracy than applying rewriting to every utterance.","supporting_citations":[],"review_version":1}