Pith. sign in

REVIEW 4 major objections 6 minor 35 references

ESURF: Simple and Effective EDU Segmentation

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A random forest trained only on words and character fragments around each candidate boundary segments text into discourse units more accurately than large neural models, and swapping it into a top RST parser improves parse scores.

desk verdict A plausible simple EDU segmenter whose SOTA claim rests on uncontrolled literature baselines and an undisclosed CNN/Daily Mail label source. read the letter →

arxiv 2501.07723 v1 pith:7MI7NGIM submitted 2025-01-13 cs.CL cs.LG

Mohammadreza Sediqin , Shlomo Engelson Argamon This is my paper · ORCID
classification cs.CLcs.LG
keywords EDUsegmentationrhetoricalstructuretheoryrandomforestdiscourseparsinglexicalfeaturescharactern-gramsRST-DTCNN/DailyMail
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a deliberately simple classifier—a random forest trained on the words and character fragments around each candidate boundary—segments text into Elementary Discourse Units (EDUs), the leaf nodes of Rhetorical Structure Theory, more accurately than far more complex neural models. The authors report that ESURF reaches 95.8% F1 on the RST-DT test set, slightly beating the previous best of 95.5%, and that swapping ESURF into the Yu et al. (2022) RST parser improves its span, relation, and full-structure scores. If true, the result matters because it suggests that local lexical and morphological cues carry most of the information needed for EDU identification, pointing toward cheaper and more training-efficient discourse analysis.

What carries the argument

The central object is the nine-token context window around a candidate EDU boundary, with tokens tagged as Before, Leading, or Continuing, combined with filtered character n-grams from those tokens. A random forest classifier assigns a boundary/non-boundary label to each window, and consecutive tokens between positive boundaries become EDUs. This design isolates local surface cues and deliberately avoids parse trees, hand-crafted syntax features, and pretrained language models, which is what makes the comparison against neural baselines informative.

What would settle it

Re-run all baselines on the same RST-DT split and the same CNN/Daily Mail subset under identical preprocessing and label definitions; if the CNN/Daily Mail EDU labels come from an automatic segmenter, check whether ESURF's advantage persists when those labels are replaced with human-annotated boundaries.

Watch

Extended reading notes

Core claim

ESURF treats EDU segmentation as binary classification of each token gap: for every nine-token window with three tokens before the candidate boundary and six after, it extracts the tokens themselves and informative character substrings, marks each by position (before, leading, or continuing), and trains a random forest. The paper's central discovery is that this low-tech feature set matches or beats state-of-the-art EDU segmenters: on RST-DT it achieves 0.958 F1 against 0.955 for the Joint Model, and when used with the shift-reduce parser of Yu et al. (2022), parse span F1 rises from 0.764 to 0.768 and relation F1 from 0.545 to 0.558. The authors interpret this as evidence that lexical and morphological context, rather than deep syntax or pretrained representations, are the dominant cues for basic discourse unit boundaries.

Load-bearing premise

The central claim depends on all methods being compared under identical data, preprocessing, and EDU label definitions; if the baselines were not run in the same setup, or if the CNN/Daily Mail EDU labels come from an automatic system, the reported advantage may not be real.

Editorial extensions

If this is right

  • If ESURF's advantage holds, simpler feature-based EDU segmentation can replace heavy neural segmenters without losing accuracy, saving compute and training data.
  • Plugging ESURF into existing RST parsers yields immediate parse-quality gains (span 0.768, relation 0.558, full 0.538 on the reported metrics), showing segmentation quality directly limits parser performance.
  • The success of position-tagged word and character features supports semi-supervised use on unlabeled corpora, as the model needs only boundary labels and no syntactic preprocessing.
  • The approach promises easier adaptation to lower-resource languages, since random forests with n-gram features need far less data than fine-tuned transformers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit: applying the same position-tagged word and character features inside a linear classifier would reveal whether the random forest's nonlinearities are necessary or whether the features themselves carry the signal.
  • Because ESURF needs only boundary-annotated text and no parse trees, the authors' future plan implies a semi-supervised loop—train on a small gold corpus, segment a large unlabeled corpus, retrain on machine-labeled output—though the paper does not demonstrate this.
  • The paper does not ablate its feature groups, but its framing implies a diagnostic: removing character n-grams while keeping word tokens (and vice versa) would quantify how much morphology contributes, and if character cues dominate, the method should transfer more readily to morphologically rich languages.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes ESURF, a random-forest EDU boundary classifier that uses nine-token windows with lexical and character n-gram features. The system is evaluated in three ways: against generic classifiers on RST-DT and CNN/Daily Mail (Table 1), against established EDU segmenters on RST-DT (Table 2), and as the segmentation component inside the Yu et al. (2022) RST parser (Table 3). The authors report a state-of-the-art F1 of 95.8 on RST-DT segmentation, 0.3 points above the Joint Model (Lin, 2023), and small improvements in parser Span, Relation, and Full scores when ESURF replaces the original segmenter.

Significance. If substantiated, the result is valuable: it would show that simple local lexical and morphological features can rival or exceed large neural models on EDU segmentation, with potential training-efficiency and interpretability benefits. The simplicity of the model and the use of an external parser for downstream evaluation are positive features. However, the empirical support as reported is incomplete: the CNN/Daily Mail EDU label provenance is undisclosed, the literature baselines in Tables 2 and 3 are not shown to be evaluated under the same protocol, and no significance or stability testing accompanies the small reported margins. The paper also does not include a code or data availability statement, which limits reproducibility of the comparisons.

major comments (4)
  1. [Section 4.1, Table 1] The paper never states where the EDU labels for CNN/Daily Mail come from; this corpus has no gold RST annotations normally associated with it. Without disclosing whether the labels were produced by an automatic segmenter, projected from another corpus, or manually annotated, the CNN/Daily Mail columns in Table 1 are uninterpretable and the claim of a two-dataset evaluation is not established.
  2. [Section 4.2, Table 2] The comparison with prior segmenters (HILDA, SPADE, CRF, JCN, WYL, Fisher and Roark, and the Joint Model) relies on scores taken from earlier publications rather than re-running those systems under the same evaluation protocol. Section 4.2 specifies a balanced 50/50 subset and 'preprocessing consistent with our previous approach' only for the Table 1 classifiers, so it is unclear whether the literature baselines in Table 2 were evaluated under the same boundary-matching, sentence-splitting, and class-balancing setup. In addition, a 0.3-point F1 gap over the Joint Model is within typical test-set noise, and no confidence intervals, bootstrap resampling, or significance tests are reported.
  3. [Section 4.3, Table 3] The parser experiment does not state whether the Yu et al. (2022) baseline numbers were re-run under identical conditions or taken from the original paper, and it does not report variance across runs or paired tests. The reported improvements (+0.004 Span, +0.013 Relation, +0.003 Full, with Full tying Zhang et al. 2021b at 0.538) are small enough that they may not be distinguishable from noise. The paper should clarify the setup and provide significance or stability evidence before claiming that ESURF improves a state-of-the-art parser.
  4. [Table 1, XLNet row] The reported XLNet F1 of 0.330 with precision 0.253 is near chance and inconsistent with published results for XLNet on text classification. This suggests a degenerate training setup, an implementation issue, or a label misalignment. The authors should either report the exact training configuration, diagnose the failure, or remove the XLNet comparison, since as presented the row does not constitute a meaningful baseline.
minor comments (6)
  1. [Throughout] The manuscript contains several typos and stylistic errors, including 'well-acccepted', 'assymetrical', 'the the', and 'highly effectiveness'; a careful proofread is needed.
  2. [Section 3 and Figure 1] Figure 1 is referenced but no diagram appears in the manuscript text; either include the figure or remove the reference.
  3. [References] There are duplicate references: Yu et al. 2018a and 2018b are identical, and Zhang et al. 2021a and 2021b are identical; these should be consolidated.
  4. [References, Joint Model] The 'Joint Model (Lin, 2023)' citation points to a book titled 'Natural language processing as autoregressive generation', which does not appear to describe an EDU segmentation system; please verify the citation and provide the correct source.
  5. [Section 3] The feature description lacks details needed for reproducibility: the character n-gram lengths, the frequency filter thresholds, and the random forest hyperparameters are not specified.
  6. [Section 6, Limitation] The limitation section states that the method assumes accurate sentence segmentation, but Section 3 says that windows do not cross sentence boundaries; the paper should clarify whether the evaluation used gold sentence boundaries or an automatic sentence segmenter.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ESURF is a supervised classifier trained on gold EDU labels and evaluated on held-out data, with parser evaluation using an external parser.

full rationale

I walked the paper's derivation chain. ESURF formulates EDU segmentation as a classification problem over nine-token windows with lexical and character n-gram features, trained on gold EDU boundaries and evaluated on held-out RST-DT test data. There is no equation or fitted parameter that later reappears as a 'prediction'; the random-forest classifier is an independent model, not a re-statement of the training labels. The parser experiment uses an external transition-based RST parser from Yu et al. (2022), and the reported improvements are comparisons of parser outputs under different segmenters; no parser metric is defined in terms of ESURF's own features. The phrase 'preprocessing consistent with our previous approach' (Section 4.2) references unspecified prior work, but it does not import the target result or define ESURF in terms of its evaluation. The undisclosed CNN/Daily Mail label source and the reliance on literature-reported baselines are legitimate evidence-quality concerns, but they are not examples of circularity: the paper does not state that CNN labels were produced by ESURF or by a model whose outputs ESURF is designed to reproduce, and I cannot exhibit a specific reduction without such a statement. No self-citation is load-bearing. Therefore the derivation is self-contained with respect to the tested claims, and the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claim rests on standard supervised classification assumptions plus several unstated evaluation choices. The main ledger items are the hand-set token window, the unstated character n-gram frequency filter, unstated random forest hyperparameters, and the assumption that EDU labels exist for CNN/Daily Mail. No new theoretical entities are introduced.

free parameters (4)
  • 9-token context window = 3 before, 6 after
    The window size is a hand-set modeling choice made in Section 3; no ablation or sensitivity analysis is reported.
  • character n-gram frequency filter = unspecified
    Section 3 states character subsequences are kept if they appear in multiple corpus texts but not most of them, but the exact frequency bounds are not given.
  • random forest hyperparameters = not stated
    Number of trees, maximum depth, and other random forest settings are never reported, so the exact model is underspecified.
  • balanced evaluation subset = 50% positive / 50% negative
    Section 4.2 says evaluation uses a similarly sized balanced subset, but the sampling procedure is not described, which can affect classifier comparisons.
assumptions (4)
  • domain assumption EDU boundaries are predictable from local lexical and morphological context in a nine-token window.
    The entire method is built on this premise; Section 3 defines features only from the local window and provides no evidence that longer-range context is unnecessary.
  • domain assumption Sentence boundaries are gold and windows never cross sentence boundaries.
    Footnote 1 states sequences do not cross sentence boundaries, and the limitation section acknowledges that accurate sentence segmentation is assumed.
  • ad hoc to paper CNN/Daily Mail articles have EDU labels obtainable by the same procedure as RST-DT.
    Section 4.1 lists CNN/Daily Mail as an evaluation dataset but never describes how EDU labels are obtained for a corpus without gold RST annotations.
  • domain assumption Reported scores of prior systems are directly comparable to ESURF under the same conditions.
    Tables 2 and 3 mix literature-reported numbers with new runs without controlling for preprocessing, train/test splits, or hyperparameter tuning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ESURF: Simple and Effective EDU Segmentation." pith.science (2026). https://pith.science/paper/7MI7NGIM

@misc{pith2026250107723,
  author       = {Pith},
  title        = {Pith review of: ESURF: Simple and Effective EDU Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7MI7NGIM}},
  note         = {Machine review of arXiv:2501.07723}
}
read the original abstract

Segmenting text into Elemental Discourse Units (EDUs) is a fundamental task in discourse parsing. We present a new simple method for identifying EDU boundaries, and hence segmenting them, based on lexical and character n-gram features, using random forest classification. We show that the method, despite its simplicity, outperforms other methods both for segmentation and within a state of the art discourse parser. This indicates the importance of such features for identifying basic discourse elements, pointing towards potentially more training-efficient methods for discourse analysis.

Figures

Figures reproduced from arXiv: 2501.07723 by the authors.

Figure 1
Figure 1. Architecture diagram for ESURF Model Acc. (CNN) Prec. (CNN) Rec. (CNN) F1 (CNN) Acc. (RST-DT) Prec. (RST-DT) Rec. (RST-DT) F1 (RST-DT) SVM 0.851 0.843 0.858 0.850 0.889 0.885 0.881 0.886 CRF 0.893 0.878 0.852 0.865 0.927 0.903 0.875 0.885 Gradient Boosted 0.871 0.875 0.866 0.873 0.890 0.904 0.891 0.885 ESURF 0.912 0.894 0.921 0.907 0.950 0.935 0.979 0.958 BERT 0.903 0.888 0.910 0.899 0.943 0.909 0.953 0.947 XLNet 0.… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 28 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Omar Ali. 2023. Fuzzy text segmentation using syntactic features for rhetorical structure theory. Ph.D. thesis, University of Portsmouth

    work page 2023

  4. [4]

    Lynn Carlson, Mary Ellen Okurowski, and Daniel Marcu. 2002. RST discourse treebank. Linguistic Data Consortium, University of Pennsylvania

    work page 2002

  5. [5]

    Zeyu Dai and Ruihong Huang. 2019. A regularization approach for incorporating event knowledge and coreference relations into neural discourse parsing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2976--2987

    work page 2019

  6. [6]

    Vanessa Wei Feng and Graeme Hirst. 2014 a . A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 511--521

    work page 2014

  7. [7]

    Vanessa Wei Feng and Graeme Hirst. 2014 b . Two-pass discourse segmentation with pairing and global features. arXiv preprint arXiv:1407.8215

    work page Pith review arXiv 2014

  8. [8]

    Seeger Fisher and Brian Roark. 2007. The utility of parse-derived features for automatic discourse segmentation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 488--495

    work page 2007

Show all 35 references
  1. [9]

    Grigorii Guz and Giuseppe Carenini. 2020. Coreference for discourse parsing: A neural approach. In Proceedings of the First Workshop on Computational Approaches to Discourse, pages 160--167

    2020

  2. [10]

    Hugo Hernault, Helmut Prendinger, David A du Verle, and Mitsuru Ishizuka. 2010. Hilda: A discourse parser using support vector machine classification. Dialogue & Discourse, 1(3):1--33

    2010

  3. [11]

    Shafiq Joty, Giuseppe Carenini, and Raymond Ng. 2012. A novel discriminative framework for sentence-level discourse analysis. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 904--915

    2012

  4. [12]

    Naoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Top-down rst parsing utilizing granularity levels in documents. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8099--8106

    2020

  5. [13]

    Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2021 a . https://doi.org/10.18653/v1/2021.eacl-main.60 Top-down discourse parsing via sequence labelling . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,...

    2021 doi

  6. [14]

    Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2021 b . Top-down discourse parsing via sequence labelling. arXiv preprint arXiv:2102.02080

    2021 arXiv

  7. [15]

    Man Lan, Yu Xu, and Zheng-Yu Niu. 2013. Leveraging synthetic discourse data via multi-task learning for implicit discourse relation recognition. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 476--485

    2013

  8. [16]

    Jiaqi Li, Ming Liu, Bing Qin, and Ting Liu. 2022. A survey of discourse parsing. Frontiers of Computer Science, 16(5):165329

    2022

  9. [17]

    Xiang Lin. 2023. Natural language processing as autoregressive generation

    2023

  10. [18]

    Jinxian Liu, Bingbing Ni, Caiyuan Li, Jiancheng Yang, and Qi Tian. 2019. Dynamic points agglomeration for hierarchical point sets learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7546--7555

    2019

  11. [19]

    Zhengyuan Liu, Ke Shi, and Nancy F Chen. 2021. Dmrst: A joint framework for document-level multilingual rst discourse segmentation and parsing. arXiv preprint arXiv:2110.04518

    2021 arXiv

  12. [20]

    Minh-Thang Luong, Quoc V Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2015. Multi-task sequence to sequence learning. arXiv preprint arXiv:1511.06114

    2015 arXiv

  13. [21]

    Amandla Mabona, Laura Rimell, Stephen Clark, and Andreas Vlachos. 2019. Neural generative rhetorical structure parsing. arXiv preprint arXiv:1909.11049

    2019 arXiv

  14. [22]

    William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-interdisciplinary Journal for the Study of Discourse, 8(3):243--281

    1988

  15. [23]

    Mathieu Morey, Philippe Muller, and Nicholas Asher. 2017. How much progress have we made on rst discourse parsing? a replication study of recent results on the rst-dt. In Conference on Empirical Methods on Natural Language Processing (EMNLP 2017), pages pp--1330

    2017

  16. [24]

    Philippe Muller, Chlo \'e Braud, and Mathieu Morey. 2019. Tony: Contextual embeddings for accurate multilingual discourse segmentation of full documents. In Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019, pages 115--124

    2019

  17. [25]

    Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023

    2016 arXiv

  18. [26]

    Thanh-Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty, and Xiaoli Li. 2021. Rst parsing from scratch. arXiv preprint arXiv:2105.10861

    2021 arXiv

  19. [27]

    Radu Soricut and Daniel Marcu. 2003. Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 228--235

    2003

  20. [28]

    Yizhong Wang, Sujian Li, and Jingfeng Yang. 2018. Toward fast and accurate neural discourse segmentation. arXiv preprint arXiv:1808.09147

    2018 arXiv

  21. [29]

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2020. https://arxiv.org/abs/1906.08237 Xlnet: Generalized autoregressive pretraining for language understanding . Preprint, arXiv:1906.08237

    2020 arXiv

  22. [30]

    Nan Yu, Meishan Zhang, and Guohong Fu. 2018 a . Transition-based neural rst parsing with implicit syntax features. In Proceedings of the 27th International Conference on Computational Linguistics, pages 559--570

    2018

  23. [31]

    Nan Yu, Meishan Zhang, and Guohong Fu. 2018 b . Transition-based neural rst parsing with implicit syntax features. In Proceedings of the 27th International Conference on Computational Linguistics, pages 559--570

    2018

  24. [32]

    Nan Yu, Meishan Zhang, Guohong Fu, and Min Zhang. 2022. Rst discourse parsing with second-stage edu-level pre-training. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4269--4280

    2022

  25. [33]

    Longyin Zhang, Fang Kong, and Guodong Zhou. 2021 a . https://doi.org/10.18653/v1/2021.acl-long.305 Adversarial learning for discourse rhetorical structure parsing . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Internat...

    2021 doi

  26. [34]

    Longyin Zhang, Fang Kong, and Guodong Zhou. 2021 b . Adversarial learning for discourse rhetorical structure parsing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Proc...

    2021

  27. [35]

    Longyin Zhang, Yuqing Xing, Fang Kong, Peifeng Li, and Guodong Zhou. 2020. A top-down neural architecture towards text-level parsing of discourse rhetorical structure. arXiv preprint arXiv:2005.02680

    2020 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.