{"id":"e6b026b6-844f-4cae-99d9-fcd8435d6b29","arxiv_id":"1908.11790","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"The paper finds that adding discourse relation labels or delta vectors between sentences helps a neural language model generate missing middle sentences of a paragraph, with discourse labels helping most.","lead":"The paper tests two ways of capturing how sentences connect in a paragraph and uses them to generate missing middle sentences. The discourse-relation version generally wins, but gains over baselines are small and measured only by automatic scores.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's 'both FlowNet variants outperform baselines' is contradicted by Table 2: FlowNet(delta) scores 3.1 METEOR on Papers vs 3.7 for both baselines and 69.7 VE on SciFi vs 71.0/73.0.","rationale":"The reader's CONDITIONAL verdict is appropriate. However, the most load-bearing concern is not only that METEOR and VectorExtrema may fail to measure coherence; it is that the paper's own Table 2 contradicts the categorical claim in the abstract and §4.4. FlowNet(delta) is worse than both baselines on Papers METEOR and on SciFi VE, and FlowNet(discourse) is worse than FlowNet(delta) on VE in two of three datasets. These are internal inconsistencies, not just questions of metric validity. A revision must either qualify the claim to 'on most metrics/datasets' and add significance testing, or demonstrate that the observed deficits are within noise; otherwise the central claim is unsupported. I did not base the verdict on the architecture omission I also noticed (FlowNet's conditioning in Eq. 5 uses only the previous sentence and delta, while the task description says both first and last sentences are given), because an undisclosed decoding procedure might use the last sentence; the Table 2 contradiction is direct and sufficient. The provided code and detailed hyperparameters are genuine supports, but they do not repair the mismatch between the reported numbers and the stated conclusion.","tokens_in":8829,"tokens_out":8142,"duration_ms":73249,"concrete_test":"Run a paired bootstrap or per-paragraph significance test (e.g., 10,000 resamples) over the test paragraphs for each model pair, metric, and dataset in Table 2. Report 95% confidence intervals for the deltas FlowNet(delta) − S2S and FlowNet(delta) − HS2S on Papers METEOR and SciFi VE, and for FlowNet(discourse) − FlowNet(delta) on VE across all datasets. If any observed deficit is significant, the abstract's 'outperform' must be withdrawn or qualified. If all observed deficits are non-significant, the paper should say 'statistically indistinguishable' and re-derive its conclusions accordingly.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim, stated in the abstract and echoed in §4.4, is that both FlowNet variants 'outperform the baselines' on the bridging task, with the discourse-driven variant best by the largest margin. Table 2 does not support this categorical claim. On Papers, FlowNet(delta) METEOR is 3.1, lower than S2S (3.7) and HS2S (3.7). On SciFi, FlowNet(delta) VectorExtrema is 69.7, lower than S2S (71.0) and HS2S (73.0), while FlowNet(discourse) VE is 70.3, lower than HS2S (73.0). On Fantasy, FlowNet(discourse) VE (71.8) is lower than FlowNet(delta) (73.9). The paper's own sentence, 'except for VecterExtrema on SciFi,' omits the Papers METEOR deficit and the Fantasy VE reversal between the two proposed models. Thus the point estimates fail the universal comparison stated in the abstract. This is independent of whether METEOR and VectorExtrema are valid coherence measures; even accepting those metrics, the reported numbers contradict the strongest claim. Without significance testing or a qualified, per-metric/per-dataset claim, the central conclusion is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two paragraph-level language models for partially conditioned generation, which the authors call the bridging task: given the first and last sentences of a paragraph, the model generates the middle sentences. FlowNet(discourse) augments an LSTM sequence-to-sequence model with an auxiliary CRF that predicts flattened RST discourse relations, while FlowNet(delta) conditions decoding on an RNN over differences of adjacent sentence representations. The models are evaluated against S2S and HS2S baselines on Papers, SciFi, and Fantasy datasets with METEOR and VectorExtrema metrics. The paper claims that both FlowNet variants outperform the baselines and that the discourse-driven variant is best, with larger gains on longer paragraphs and larger training sets.","tokens_in":9144,"tokens_out":5400,"duration_ms":48361,"significance":"The comparison of linguistically supervised discourse relations and unsupervised latent relations for paragraph generation is a potentially useful contribution, and the bridging task is a sensible way to probe paragraph-level coherence. The manuscript also ships public code and data, uses an external parser only at training time, and tunes the CRF scaling parameter on validation, which are clear strengths. However, the headline empirical claim is not actually established by the numbers in Table 2, the automatic metrics do not directly measure coherence, and no variance or significance information is reported. In the current form, the conclusions are considerably weaker than the abstract states, although the underlying ideas are sound and the problems are fixable.","major_comments":[{"comment":"The categorical claim in the abstract and in Section 4.4 that both proposed models outperform the baselines is contradicted by the point estimates in Table 2. On Papers, FlowNet(delta) METEOR is 3.1, lower than both S2S (3.7) and HS2S (3.7); on SciFi, FlowNet(delta) VectorExtrema is 69.7 versus 71.0 for S2S and 73.0 for HS2S, and FlowNet(discourse) VectorExtrema is 70.3, below HS2S; on Fantasy, FlowNet(discourse) VectorExtrema (71.8) is below FlowNet(delta) (73.9). The sentence 'except for VecterExtrema on SciFi' accounts for only one of these failures. Without significance tests, the abstract's universal comparison is not supported.","section":"Section 4.4, Table 2; Abstract"},{"comment":"The claim that discourse relations 'help produce more coherent text' rests on METEOR and VectorExtrema, which compare generated tokens to the reference, not on any judgment of the generated middle sentences. The human study reports only that human writers beat all models by large margins; it does not rate model outputs for coherence. Since there is no established automatic coherence metric for this task, the paper needs either human ratings of model outputs or a demonstration that METEOR and VectorExtrema track human coherence judgments on this task. As it stands, coherence itself is not measured.","section":"Section 4.2, Section 4.4, Figure 4"},{"comment":"There is a direct inconsistency about whether discourse labels are used at test time. Section 3.1 states that RST relations are 'only used during training ... but not at test time,' yet Section 4.4 explains FlowNet(discourse)'s diverse output by saying 'each sentence is generated based on the representation conditioned on the predicted RST discourse relation.' The decoding rule described in Section 3.1 also greedily predicts the label sequence, which appears to be a test-time use of the discourse labels. This ambiguity affects what the model actually conditions on and must be resolved.","section":"Section 3.1 versus Section 4.4"},{"comment":"Only single point estimates are reported for every comparison, with no standard deviations, confidence intervals, or significance tests. Several margins are very small (for example, SciFi METEOR 3.5 versus 3.6, and SciFi VectorExtrema 70.3 versus 71.0), so the results cannot be separated from training noise. Re-running each condition with multiple seeds and reporting variance, or applying paired significance tests, is necessary before statements such as 'achieve significant improvements' in Section 5 can be made.","section":"Section 4.4, Tables 2-3, Figure 4"}],"minor_comments":[{"comment":"The text contains the typo 'VecterExtrema' in Table 2 and the results paragraph, and the metric definition 'by calculating cosine similarity of averaged word embeddings' is incomplete: VectorExtrema normally uses elementwise extrema of word vectors, not a simple average.","section":"Section 4.2"},{"comment":"Table 5 reports models named FT2SEQ and H-FT2SEQ, but the main text defines only S2S and HS2S; please use consistent names.","section":"Appendix, Table 5"},{"comment":"The claim that improvements grow with training size conflates dataset size with domain differences, because Papers, SciFi, and Fantasy come from different genres; the observed trend is not interpretable as a pure size effect.","section":"Section 4.4"},{"comment":"The rule 'If two consecutive RST relations are given, the deeper level of relation is chosen' is not fully specified; please define 'deeper' with respect to the RST tree and provide a concrete example.","section":"Section 3.1"},{"comment":"The hyperparameter section does not report how alpha in Eq. (3) was chosen or which value was used; please report the selected alpha per domain.","section":"Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is well founded: Table 2 contradicts the abstract as written, and the point estimates do not support the universal comparison. The paper is within scope and the core idea is reasonable, so I see this as a major-revision case rather than a rejection. The main fixes are to correct the overclaims, add variance or significance information, clarify the test-time role of discourse labels, and either add human coherence judgments or justify the automatic metrics."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is a genuinely useful empirical comparison of two ways of modeling inter-sentence relations for paragraph generation, but the central claim as stated is too strong. The abstract says both FlowNet variants outperform baselines; Table 2 contradicts it on Papers METEOR and SciFi VectorExtrema, and the discourse model loses to delta on Fantasy VE.\n\nWhat the paper does well: it defines a clean bridging task (generate middle sentences given first and last), compares a discourse-CRF joint model and a delta-conditioned hierarchical RNN against two seq2seq baselines, and releases code and data. Discourse FlowNet does win on METEOR everywhere and beats both baselines on most metrics, so the overall direction is believable. The delta ablation (subtract vs add vs MLP) is a nice touch, and the Appendix B discussion of theoretical differences between linguistic and latent relations is thoughtful.\n\nThe main problem is the mismatch between the blanket claim and the reported numbers. Point estimates show the delta model below baselines on Papers METEOR and SciFi VE, and the discourse model below delta on Fantasy VE. The paper's own hedge ('except for VectorExtrema on SciFi') misses the Papers METEOR deficit and the Fantasy reversal. Even if those differences would flip with error bars, a claim of universal superiority needs either significance tests or a qualified statement. Second, the evaluation relies on METEOR and VectorExtrema; there is no human judgment of the model-generated middle sentences, so 'more coherent' is only as good as those metrics. Third, improvements are small in absolute terms, and the human comparison in Figure 4 is used only to show humans are much better, which doesn't tell us much about model ordering. These are real but not fatal flaws; the comparison is still informative as a first pass.\n\nWho is this for? Anyone working on paragraph generation or discourse-aware generation will want to know these results, even if they should treat the superiority claim with caution. If this came across my desk, I'd send it to review with a request for significance testing, per-metric and per-dataset qualification, and ideally some human pairwise ratings on a sample. Worth engaging.","headline":"A useful empirical comparison of discourse and latent sentence relations for paragraph generation, but the abstract's universal superiority claim is contradicted by the paper's own Table 2 and there are no error bars.","tokens_in":9623,"tokens_out":1535,"would_cite":true,"duration_ms":14169,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that injecting explicit discourse relations into a paragraph language model produces more coherent text than latent relation signals, and that both beat standard sequence-to-sequence baselines on a bridging task.","keywords":["paragraph generation","discourse relations","Rhetorical Structure Theory","latent relations","hierarchical recurrent neural network","language modeling","text coherence","bridging task"],"falsifier":"Ask human annotators to rate the coherence of the middle sentences generated by FlowNet (discourse), FlowNet (delta), S2S, and HS2S on the same test set; if the discourse model is not rated more coherent than the baselines, the paper's central claim fails.","tokens_in":8648,"feed_emoji":"📝","tokens_out":11130,"duration_ms":86037,"temperature":0.7,"pith_summary":"The paper tries to establish that paragraph-level coherence in generation is driven by relations between sentences, and that these relations can be supplied in two very different forms: explicit linguistic labels or latent vector differences. It proposes two FlowNet language models, one trained jointly to predict next words and Rhetorical Structure Theory (RST) discourse relations, and one conditioned on the vector difference between previous sentence representations. On a bridging task--given a paragraph's first and last sentences, generate the middle--both variants beat standard sequence-to-sequence baselines on the automatic metrics METEOR and VectorExtrema, with the discourse-driven variant ahead by the largest margin. The result matters because it suggests that coarse human-defined discourse structure is a usable training signal for making long-form generation more coherent, even though that signal is only needed at training time.","feed_headline":"Discourse relations beat latent signals for coherent paragraphs","feed_subtitle":"Training a language model to predict discourse labels along with words yields the best bridged middle sentences.","key_machinery":"The load-bearing object is the paragraph flow: the logical connection between sentences, covering tense, coreference, and discourse plans. For the discourse variant, flow is encoded as a flattened sequence of Rhetorical Structure Theory (RST) relations between adjacent sentences, and the model predicts these labels alongside the next word through a conditional random field (CRF), with objective $L_{\\text{disc}}=L_{\\text{s2s}}+\\alpha L_{\\text{CRF}}$. For the latent variant, flow is the delta vector $d_{i-1}=s_i-s_{i-1}$ obtained by subtracting adjacent sentence hidden states, encoded by another recurrent network, and fed into decoding together with the previous word and previous sentence. Both mechanisms convert a coherence notion into an extra learning signal for the language model, and the discourse signal is the one that generalizes best on the bridging task.","core_discovery":"The central discovery is that explicit discourse relations and latent delta relations are both effective inductive biases for paragraph language models, but the explicit linguistic signal is the stronger one. FlowNet (discourse) adds a sequence-labeling objective over flattened Rhetorical Structure Theory (RST) relations while learning the standard next-word objective; FlowNet (delta) encodes words, sentences, and the subtraction $d_{i-1}=s_i-s_{i-1}$ between adjacent sentence representations. Across three domains, the discourse variant produces the strongest METEOR and VectorExtrema scores on the bridging task, and the advantage grows with paragraph length and training-set size. The paper interprets this as evidence that a human-annotated relation structure captures paragraph flow better than a learned latent relation does.","pith_inferences":["A natural control would replace the RST labels with random or permuted labels; if the gain disappears, the advantage comes from auxiliary supervision rather than from discourse meaning itself. That control is not in the paper.","Because delta relations are always adjacent, the latent variant may be structurally unable to capture long-distance discourse relations such as Solutionhood, which the paper's own appendix notes; combining both signals in one model might beat either alone.","The reported advantage of discourse relations grows with data size, which suggests the explicit signal is most valuable in data-rich regimes; in small-data settings the annotation-free delta model may be the safer choice."],"forward_implications":["A paragraph generator trained with discourse-relation prediction as an auxiliary objective should produce more coherent middle sentences than a plain or hierarchical sequence-to-sequence generator when only the first and last sentences are given.","The improvement from discourse conditioning should grow as paragraphs get longer and as training data increases, since the discourse signal has more structure to exploit.","Latent delta conditioning should remain a useful fallback where reliable discourse parses are unavailable, since it needs no external annotation.","If the comparison holds, future paragraph generators can treat discourse labels as a training-time-only supervision, leaving the test-time model free of parsing requirements."],"supporting_citations":[{"why":"Defines Rhetorical Structure Theory, the discourse relation scheme the paper flattens into training labels.","marker":"Mann and Thompson, 1988"},{"why":"Provides the discourse parser used to turn paragraphs into RST trees for the supervised objective.","marker":"Ji and Eisenstein, 2014"},{"why":"Supplies the hierarchical recurrent architecture whose sentence-level recurrence the delta model extends with delta relations.","marker":"Lin et al., 2015"},{"why":"Supplies the conditional random field objective used to predict discourse labels jointly with words.","marker":"Lafferty et al., 2001"},{"why":"Defines METEOR, one of the two automatic metrics used to compare generated middle sentences.","marker":"Banerjee and Lavie, 2005"},{"why":"Defines VectorExtrema, the embedding-based similarity metric used as the second evaluation signal.","marker":"Liu et al., 2016"},{"why":"Provides the attentional sequence-to-sequence baseline that both FlowNet variants must beat.","marker":"Bahdanau et al., 2014"},{"why":"Provides the hierarchical encoder-decoder baseline (HRED) that encodes the given context and decodes the middle words.","marker":"Serban et al., 2017; Sordoni et al., 2015"}],"fun_headline_variants":["Explicit discourse beats latent for paragraph flow","Linguistic relations trump latent for coherent paragraphs","Discourse labels beat latent vectors in paragraph generation","Supervised discourse edges out unsupervised latent","RST relations outperform latent for paragraph coherence"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that METEOR and VectorExtrema actually measure paragraph coherence; the paper never collects human judgments of the generated middle sentences, so if those automatic scores do not track coherence the central comparison is not established.","fun_headline_variants_meta":{"raw":{"variants":["Explicit discourse beats latent for paragraph flow","Linguistic relations trump latent for coherent paragraphs","Discourse labels beat latent vectors in paragraph generation","Supervised discourse edges out unsupervised latent","RST relations outperform latent for paragraph coherence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1228,"prompt_tokens":855,"completion_tokens":373,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":471,"completion_tokens_details":{"reasoning_tokens":306}},"tokens_in":471,"tokens_out":373,"duration_ms":3796,"temperature":1.0,"reasoning_tokens":306,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:06:05.941214+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Ask human annotators to rate the coherence of the middle sentences generated by FlowNet (discourse), FlowNet (delta), S2S, and HS2S on the same test set; if the discourse model is not rated more coherent than the baselines, the paper's central claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Rhetorical Structure Theory, the discourse relation scheme the paper flattens into training labels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the discourse parser used to turn paragraphs into RST trees for the supervised objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the hierarchical recurrent architecture whose sentence-level recurrence the delta model extends with delta relations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the hierarchical encoder-decoder baseline (HRED) that encodes the given context and decodes the middle words."}],"review_version":1}