Pith. sign in

REVIEW 4 major objections 5 minor 40 references

Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that injecting explicit discourse relations into a paragraph language model produces more coherent text than latent relation signals, and that both beat standard sequence-to-sequence baselines on a bridging task.

desk verdict A useful empirical comparison of discourse and latent sentence relations for paragraph generation, but the abstract's universal superiority claim is contradicted by the paper's own Table 2 and there are no error bars. read the letter →

arxiv 1908.11790 v1 pith:JZ3REQ3K submitted 2019-08-30 cs.CL cs.AI

Dongyeop Kang , Hiroaki Hayashi , Alan W Black , Eduard Hovy This is my paper · ORCID
classification cs.CLcs.AI
keywords paragraphgenerationdiscourserelationsRhetoricalStructureTheorylatenthierarchicalrecurrentneuralnetworklanguagemodelingtextcoherencebridgingtask
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that paragraph-level coherence in generation is driven by relations between sentences, and that these relations can be supplied in two very different forms: explicit linguistic labels or latent vector differences. It proposes two FlowNet language models, one trained jointly to predict next words and Rhetorical Structure Theory (RST) discourse relations, and one conditioned on the vector difference between previous sentence representations. On a bridging task--given a paragraph's first and last sentences, generate the middle--both variants beat standard sequence-to-sequence baselines on the automatic metrics METEOR and VectorExtrema, with the discourse-driven variant ahead by the largest margin. The result matters because it suggests that coarse human-defined discourse structure is a usable training signal for making long-form generation more coherent, even though that signal is only needed at training time.

What carries the argument

The load-bearing object is the paragraph flow: the logical connection between sentences, covering tense, coreference, and discourse plans. For the discourse variant, flow is encoded as a flattened sequence of Rhetorical Structure Theory (RST) relations between adjacent sentences, and the model predicts these labels alongside the next word through a conditional random field (CRF), with objective $L_{\text{disc}}=L_{\text{s2s}}+\alpha L_{\text{CRF}}$. For the latent variant, flow is the delta vector $d_{i-1}=s_i-s_{i-1}$ obtained by subtracting adjacent sentence hidden states, encoded by another recurrent network, and fed into decoding together with the previous word and previous sentence. Both mechanisms convert a coherence notion into an extra learning signal for the language model, and the discourse signal is the one that generalizes best on the bridging task.

What would settle it

Ask human annotators to rate the coherence of the middle sentences generated by FlowNet (discourse), FlowNet (delta), S2S, and HS2S on the same test set; if the discourse model is not rated more coherent than the baselines, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that explicit discourse relations and latent delta relations are both effective inductive biases for paragraph language models, but the explicit linguistic signal is the stronger one. FlowNet (discourse) adds a sequence-labeling objective over flattened Rhetorical Structure Theory (RST) relations while learning the standard next-word objective; FlowNet (delta) encodes words, sentences, and the subtraction $d_{i-1}=s_i-s_{i-1}$ between adjacent sentence representations. Across three domains, the discourse variant produces the strongest METEOR and VectorExtrema scores on the bridging task, and the advantage grows with paragraph length and training-set size. The paper interprets this as evidence that a human-annotated relation structure captures paragraph flow better than a learned latent relation does.

Load-bearing premise

The load-bearing premise is that METEOR and VectorExtrema actually measure paragraph coherence; the paper never collects human judgments of the generated middle sentences, so if those automatic scores do not track coherence the central comparison is not established.

Editorial extensions

If this is right

  • A paragraph generator trained with discourse-relation prediction as an auxiliary objective should produce more coherent middle sentences than a plain or hierarchical sequence-to-sequence generator when only the first and last sentences are given.
  • The improvement from discourse conditioning should grow as paragraphs get longer and as training data increases, since the discourse signal has more structure to exploit.
  • Latent delta conditioning should remain a useful fallback where reliable discourse parses are unavailable, since it needs no external annotation.
  • If the comparison holds, future paragraph generators can treat discourse labels as a training-time-only supervision, leaving the test-time model free of parsing requirements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural control would replace the RST labels with random or permuted labels; if the gain disappears, the advantage comes from auxiliary supervision rather than from discourse meaning itself. That control is not in the paper.
  • Because delta relations are always adjacent, the latent variant may be structurally unable to capture long-distance discourse relations such as Solutionhood, which the paper's own appendix notes; combining both signals in one model might beat either alone.
  • The reported advantage of discourse relations grows with data size, which suggests the explicit signal is most valuable in data-rich regimes; in small-data settings the annotation-free delta model may be the safer choice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two paragraph-level language models for partially conditioned generation, which the authors call the bridging task: given the first and last sentences of a paragraph, the model generates the middle sentences. FlowNet(discourse) augments an LSTM sequence-to-sequence model with an auxiliary CRF that predicts flattened RST discourse relations, while FlowNet(delta) conditions decoding on an RNN over differences of adjacent sentence representations. The models are evaluated against S2S and HS2S baselines on Papers, SciFi, and Fantasy datasets with METEOR and VectorExtrema metrics. The paper claims that both FlowNet variants outperform the baselines and that the discourse-driven variant is best, with larger gains on longer paragraphs and larger training sets.

Significance. The comparison of linguistically supervised discourse relations and unsupervised latent relations for paragraph generation is a potentially useful contribution, and the bridging task is a sensible way to probe paragraph-level coherence. The manuscript also ships public code and data, uses an external parser only at training time, and tunes the CRF scaling parameter on validation, which are clear strengths. However, the headline empirical claim is not actually established by the numbers in Table 2, the automatic metrics do not directly measure coherence, and no variance or significance information is reported. In the current form, the conclusions are considerably weaker than the abstract states, although the underlying ideas are sound and the problems are fixable.

major comments (4)
  1. [Section 4.4, Table 2; Abstract] The categorical claim in the abstract and in Section 4.4 that both proposed models outperform the baselines is contradicted by the point estimates in Table 2. On Papers, FlowNet(delta) METEOR is 3.1, lower than both S2S (3.7) and HS2S (3.7); on SciFi, FlowNet(delta) VectorExtrema is 69.7 versus 71.0 for S2S and 73.0 for HS2S, and FlowNet(discourse) VectorExtrema is 70.3, below HS2S; on Fantasy, FlowNet(discourse) VectorExtrema (71.8) is below FlowNet(delta) (73.9). The sentence 'except for VecterExtrema on SciFi' accounts for only one of these failures. Without significance tests, the abstract's universal comparison is not supported.
  2. [Section 4.2, Section 4.4, Figure 4] The claim that discourse relations 'help produce more coherent text' rests on METEOR and VectorExtrema, which compare generated tokens to the reference, not on any judgment of the generated middle sentences. The human study reports only that human writers beat all models by large margins; it does not rate model outputs for coherence. Since there is no established automatic coherence metric for this task, the paper needs either human ratings of model outputs or a demonstration that METEOR and VectorExtrema track human coherence judgments on this task. As it stands, coherence itself is not measured.
  3. [Section 3.1 versus Section 4.4] There is a direct inconsistency about whether discourse labels are used at test time. Section 3.1 states that RST relations are 'only used during training ... but not at test time,' yet Section 4.4 explains FlowNet(discourse)'s diverse output by saying 'each sentence is generated based on the representation conditioned on the predicted RST discourse relation.' The decoding rule described in Section 3.1 also greedily predicts the label sequence, which appears to be a test-time use of the discourse labels. This ambiguity affects what the model actually conditions on and must be resolved.
  4. [Section 4.4, Tables 2-3, Figure 4] Only single point estimates are reported for every comparison, with no standard deviations, confidence intervals, or significance tests. Several margins are very small (for example, SciFi METEOR 3.5 versus 3.6, and SciFi VectorExtrema 70.3 versus 71.0), so the results cannot be separated from training noise. Re-running each condition with multiple seeds and reporting variance, or applying paired significance tests, is necessary before statements such as 'achieve significant improvements' in Section 5 can be made.
minor comments (5)
  1. [Section 4.2] The text contains the typo 'VecterExtrema' in Table 2 and the results paragraph, and the metric definition 'by calculating cosine similarity of averaged word embeddings' is incomplete: VectorExtrema normally uses elementwise extrema of word vectors, not a simple average.
  2. [Appendix, Table 5] Table 5 reports models named FT2SEQ and H-FT2SEQ, but the main text defines only S2S and HS2S; please use consistent names.
  3. [Section 4.4] The claim that improvements grow with training size conflates dataset size with domain differences, because Papers, SciFi, and Fantasy come from different genres; the observed trend is not interpretable as a pure size effect.
  4. [Section 3.1] The rule 'If two consecutive RST relations are given, the deeper level of relation is chosen' is not fully specified; please define 'deeper' with respect to the RST tree and provide a concrete example.
  5. [Section 4.3] The hyperparameter section does not report how alpha in Eq. (3) was chosen or which value was used; please report the selected alpha per domain.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the predictions are empirical outputs of a generation model with external inputs, not re-statements of fitted parameters or self-citations.

full rationale

The paper's derivation chain is self-contained and does not reduce to its inputs. The discourse-driven FlowNet uses RST relations obtained from an external pre-trained parser (Ji and Eisenstein, 2014) as auxiliary training labels, and the paper explicitly states these labels are not used at test time, so the generated text is not constructed from the discourse labels themselves. The delta-driven FlowNet uses an unsupervised subtraction of hidden representations, which is a modeling choice rather than a fitted quantity, and its comparison with additive and MLP alternatives in Table 3 is an empirical ablation. The CRF scaling parameter alpha is selected on a validation set, which is an ordinary hyperparameter search, not a parameter fitted to the test outputs being predicted. The central claims are evaluated against external baselines (S2S, HS2S) using standard automatic metrics METEOR and VectorExtrema, and the comparisons are not forced by any equation that defines the result in terms of a fitted input. Although the abstract's categorical claim that both variants outperform baselines is not fully supported by every point estimate in Table 2 (e.g., FlowNet(delta) METEOR on Papers is 3.1 versus 3.7 for both baselines), that concern is a matter of correctness or evidence, not circularity. Self-citations in the paper are limited to data collection (Kang et al., 2018a) and earlier related work, and none of these citations is load-bearing for the central generation result. Therefore, the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central comparisons are empirical rather than derivational. The model choices that carry weight are the CRF scaling, the delta definition, and the RST parser and flattening heuristic. No new physical or categorical entities are introduced.

free parameters (2)
  • CRF objective scaling alpha = not reported; selected by grid search on validation set
    Equation (3) combines the language modeling loss and the discourse CRF loss; the balance is tuned empirically and can change the contribution of discourse labels.
  • delta operation type = SUBTRACT (Eq. 4), though ADD scores higher on METEOR in Table 3
    Defining delta as subtraction is a hand-chosen linear operation; alternatives (ADD, MLP) are compared in Table 3, so the reported FlowNet(delta) results depend on this choice.
assumptions (3)
  • domain assumption The pre-trained RST parser of Ji and Eisenstein (2014) produces reliable-enough discourse trees for Papers, SciFi, and Fantasy.
    Section 3.1 and Appendix A parse every paragraph and drop parser failures; poor parses would make discourse labels noisy.
  • domain assumption METEOR and VectorExtrema are valid proxies for paragraph coherence.
    Section 4.2 defines evaluation with these metrics only; no human ratings of generated outputs are collected, so the coherence claim depends on this proxy.
  • ad hoc to paper Flattening RST trees to relations between adjacent phrases preserves the flow information needed for generation.
    Section 3.1 reduces full discourse trees by keeping only adjacent phrase relations, preferring deeper relations, and excluding some relations; this heuristic is not validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs." pith.science (2026). https://pith.science/paper/JZ3REQ3K

@misc{pith2026190811790,
  author       = {Pith},
  title        = {Pith review of: Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JZ3REQ3K}},
  note         = {Machine review of arXiv:1908.11790}
}
read the original abstract

Generating a long, coherent text such as a paragraph requires a high-level control of different levels of relations between sentences (e.g., tense, coreference). We call such a logical connection between sentences as a (paragraph) flow. In order to produce a coherent flow of text, we explore two forms of intersentential relations in a paragraph: one is a human-created linguistical relation that forms a structure (e.g., discourse tree) and the other is a relation from latent representation learned from the sentences themselves. Our two proposed models incorporate each form of relations into document-level language models: the former is a supervised model that jointly learns a language model as well as discourse relation prediction, and the latter is an unsupervised model that is hierarchically conditioned by a recurrent neural network (RNN) over the latent information. Our proposed models with both forms of relations outperform the baselines in partially conditioned paragraph generation task. Our codes and data are publicly available.

Figures

Figures reproduced from arXiv: 1908.11790 by the authors.

Figure 1
Figure 1. FlowNet with linguistic (i.e., discourse) versus latent (i.e., delta) relation. (a) For each word, a form [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Bridging task: given [1] and [4] sentences, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of paragraph lengths. Best viewed in color [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison (METEOR) with human perfor￾mance (black bars): S2S (blue), HS2S (red), Flow:delta (yellow), and Flow:disc. (green). Best viewed in color. We conduct a comparison with human perfor￾mance (See [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Douglas E Appelt. 1982. Planning natural-language utterances to satisfy multiple goals. Technical report, SRI INTERNATIONAL MENLO PARK CA ARTIFICIAL INTELLIGENCE CENTER

    work page 1982

  4. [4]

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

    arXiv 2014

  5. [5]

    Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65--72

    2005

  6. [6]

    Donn Byrne. 1979. Teaching writing skills. Longman

    work page 1979

  7. [7]

    Junyoung Chung, Sungjin Ahn, and Yoshua Bengio. 2016. Hierarchical multiscale recurrent neural networks. CoRR, abs/1609.01704

    arXiv 2016

  8. [8]

    Ronan Collobert, Jason Weston, L \'e on Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug):2493--2537

    2011

Show all 40 references
  1. [9]

    Hercules Dalianis and Eduard Hovy. 1996. Aggregation in natural language generation. In Trends in Natural Language Generation An Artificial Intelligence Perspective, pages 88--105. Springer

    1996

  2. [10]

    John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(Jul):2121--2159

    2011

  3. [11]

    Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. 2017. Creating training corpora for nlg micro-planning. In 55th annual meeting of the Association for Computational Linguistics (ACL)

    2017

  4. [12]

    Albert Gatt and Emiel Krahmer. 2018. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. Journal of Artificial Intelligence Research, 61:65--170

    2018

  5. [13]

    Sepp Hochreiter and J \"u rgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9:1735--1780

    1997

  6. [14]

    Eduard H. Hovy. 1985. Integrating text planning and production in generation. In IJCAI

    1985

  7. [15]

    Eduard H Hovy. 1990. Pragmatics and natural language generation. Artificial Intelligence, 43(2):153--197

    1990

  8. [16]

    Eduard H Hovy. 1991. Approaches to the planning of coherent text. In Natural language generation in artificial intelligence and computational linguistics, pages 83--102. Springer

    1991

  9. [17]

    Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. CoRR, abs/1508.01991

    2015 arXiv

  10. [18]

    Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 13--24

    2014

  11. [19]

    Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine van Zuylen, Sebastian Kohlmeier, Eduard Hovy, and Roy Schwartz. 2018 a . A dataset of peer reviews (peerread): Collection, insights and nlp applications. In Proceedings of NAACL-HLT

    2018

  12. [20]

    Dongyeop Kang, Varun Gangal, Ang Lu, Zheng Chen, and Eduard Hovy. 2017. Detecting and explaining causes from text for a time series event. In Conference on Empirical Methods on Natural Language Processing

    2017

  13. [21]

    Dongyeop Kang, Tushar Khot, Ashish Sabharwal, and Eduard Hovy. 2018 b . Adventure: Adversarial training for textual entailment with knowledge-guided examples. In The 56th Annual Meeting of the Association for Computational Linguistics (ACL), Melbourne, Australia

    2018

  14. [22]

    Lafferty, Andrew McCallum, and Fernando Pereira

    John D. Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML

    2001

  15. [23]

    Jiwei Li, Minh-Thang Luong, and Daniel Jurafsky. 2015. A hierarchical neural autoencoder for paragraphs and documents. In ACL

    2015

  16. [24]

    Rui Lin, Shujie Liu, Muyun Yang, Mu Li, Ming Zhou, and Sheng Li. 2015. Hierarchical recurrent neural network for document modeling. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 899--907

    2015

  17. [25]

    Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: A n empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023

    2016 arXiv

  18. [26]

    William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243--281

    1988

  19. [27]

    Kathleen R McKeown. 1985. Discourse strategies for generating natural-language text. Artificial Intelligence, 27(1):1--41

    1985

  20. [28]

    Johanna D Moore and C \'e cile L Paris. 1993. Planning text for advisory dialogues: Capturing intentional and rhetorical information. Computational linguistics, 19(4):651--694

    1993

  21. [29]

    Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543

    2014

  22. [30]

    Ehud Reiter and Robert Dale. 2000. Building natural language generation systems. Cambridge university press

    2000

  23. [31]

    Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In AAAI, pages 3295--3301

    2017

  24. [32]

    Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. 2015. A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In Proceedings of the 24th ACM International on Conference on Information an...

    2015

  25. [33]

    Judith A. Swan. 2002. The science of scientific writing. In Book

    2002

  26. [34]

    Kai Sheng Tai, Richard Socher, and Christopher D Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075

    2015 arXiv

  27. [35]

    Silvan S Tomkins. 1978. Script theory: Differential magnification of affects. In Nebraska symposium on motivation. University of Nebraska Press

    1978

  28. [36]

    Qingyun Wang, Xiaoman Pan, Lifu Huang, Boliang Zhang, Zhiying Jiang, Heng Ji, and Kevin Knight. 2018. Describing a knowledge base. In CoRR, volume abs/1809.01797

    2018 arXiv

  29. [37]

    Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language...

    2016

  30. [38]

    R Michael Young and Johanna D Moore. 1994. Dpocl: A principled approach to discourse planning. In Proceedings of the Seventh International Workshop on Natural Language Generation, pages 13--20. Association for Computational Linguistics

    1994

  31. [39]

    Biao Zhang, Deyi Xiong, Jinsong Su, Qun Liu, Rongrong Ji, Hong Duan, and Min Zhang. 2016. Variational neural discourse relation recognizer. In EMNLP

    2016

  32. [40]

    Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler

    Yukun Zhu, Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. 2015 IEEE International Conference on Computer Visio...

    2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.