Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization

T0 review · 5 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Summarization systems and datasets each carry a measurable bias toward position, diversity, or importance—and news is the only domain where position dominates.

desk verdict Broad and practical empirical survey of summarization biases; the position findings are solid, but the diversity and importance measurements need validation before the cross-domain claims can be trusted. read the letter →

arxiv 1908.11723 v1 pith:ZUNDOBCN submitted 2019-08-30 cs.CL

Taehee Jung , Dongyeop Kang , Lucas Mentch , Eduard Hovy This is my paper · ORCID
classification cs.CL
keywords textsummarizationpositionbiasextractivecorpussystemsemanticdiversitysentenceimportanceROUGE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that "earlier isn't always better": the well-known tendency of summarizers to pick opening sentences holds strongly for news, but not for academic papers, meeting minutes, or several other domains. It decomposes summarization into three sub-aspects—position, importance, and diversity—and measures each with a purpose-built extractive algorithm across nine corpora. The measurements show that every corpus and every summarization system has a distinct bias profile; neural extractive systems are comparatively balanced, and simple ensembles of aspect-specific selectors match or beat individual systems on most domains. If true, this means benchmark results on news overstate the role of sentence position and understate the need for importance and diversity in other genres.

What carries the argument

The measurement instrument is a set of three extractive probe algorithms operating on BERT sentence embeddings. Position is probed by selecting the first, last, or middle k sentences; diversity by ConvexFall, which picks sentences whose convex hull in embedding space has maximum volume and then prunes least-volume-reducing sentences, plus a greedy Heuristic baseline; importance by N-Nearest and K-Nearest, which select sentences most similar to their neighbors. Each probe's output is compared with reference summaries or a system's output using averaged ROUGE, sentence overlap with the extractive oracle, and volume-overlap ratio, yielding a position/diversity/importance profile for every corpus and system.

What would settle it

Take a new domain absent from the nine (for instance, legal opinions) and run the same three probe algorithms against human summaries. If First-k, ConvexFall, and N-Nearest all score no better than random selection, the claimed aspect structure does not exist there; if their relative ordering matches the news pattern in a non-news domain, the claim that position bias is domain-specific would be contradicted.

Watch

Extended reading notes

Core claim

The central claim is that both corpora and systems can be profiled by how much they rely on sentence position, content importance, and semantic diversity, and that these profiles differ systematically. Concretely, First-k position selection achieves high ROUGE and sentence overlap on CNNDM, Newsroom, Reddit, and BookSum, but roughly random-level performance on PubMed, AMI, and XSum; academic papers (PeerRead, PubMed) need all three aspects at once. On the system side, LexRank is heavily position-biased, MMR leans importance, and neural extractive systems (CL, SumRun, S2SExt) are more balanced, while an ensemble built from the three aspect algorithms outperforms single-aspect systems on five of eight datasets. The paper also finds that reference summaries in most corpora contain fewer than 15% novel words, except academic abstracts, and that semantic volume overlap between model and reference summaries is not correlated with ROUGE.

Load-bearing premise

The whole comparison rests on the assumption that the three probe algorithms—First for position, ConvexFall for diversity, N-Nearest for importance—each cleanly isolate one sub-aspect, so their ROUGE scores against reference summaries reveal how much that aspect contributes to a corpus or system.

Editorial extensions

If this is right

  • Benchmarking on news alone will keep rewarding lead-biased models; new datasets should report an aspect profile so position-biased gains are not mistaken for general progress.
  • For academic-paper summarization, a system must balance all three aspects; position-only baselines near random on PubMed show why abstract-based evaluation needs importance-aware selection.
  • Long documents (books, movie scripts) and conversations (meetings) remain far from solved; current aspect probes score near random, indicating the task needs additional mechanisms beyond these three aspects or noise filtering.
  • Simple ensembles of aspect-specific selectors are a cheap, effective alternative: on five of eight datasets they beat single systems in ROUGE or sentence overlap while keeping the three aspects in balance.
  • The weak correlation between volume overlap and ROUGE implies semantic diversity and lexical overlap are measuring different qualities; systems should be evaluated on both.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same probe procedure could be run on evaluation metrics themselves, scoring each metric's own position/diversity/importance bias, which would turn the paper's closing suggestion into a concrete benchmark.
  • The near-random performance on meetings, books, and scripts suggests these domains may need sub-aspects beyond the three studied (for example, discourse structure or speaker role); testing additional probes would show whether the three-aspect space is closed.
  • A direct test of the decomposition: construct synthetic source documents with known aspect composition and check whether First, ConvexFall, and N-Nearest recover the planted profile; that would validate or invalidate the measurement itself.
  • Dataset designers could publish an aspect profile alongside each corpus, letting users match a summarizer's bias to the target domain before training.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper characterizes summarization behavior through three sub-aspects—position, importance, and diversity—and measures each by the ROUGE score of a heuristic extractive algorithm (First, ConvexFall, N-Nearest) against target summaries on nine corpora and against system outputs on a further set of systems. It reports that news and personal posts are strongly position-biased, academic papers are more balanced across aspects, and meeting, book, and movie-script summarization is very difficult. It also reports system-level aspect biases and finds that simple ensembles of aspect systems perform comparably to single-aspect baselines. The central empirical claim is that sub-aspect biases vary systematically across domains and across system families.

Significance. The study's breadth is its main strength: it compares position, importance, and diversity biases across nine domains and multiple system families in a single framework, and the position aspect is transparently defined via First, Last, and Middle baselines. The oracle-extraction and sentence-overlap controls are sensible for extractive comparisons. If the diversity and importance operationalizations are valid, the results would give useful guidance for dataset collection and system design. However, the diversity and importance algorithms are geometric proxies whose construct validity is not established, and at least one headline conclusion ('neural systems rarely have bias') is contradicted by the paper's own table. The paper would be substantially strengthened by validation studies for the aspect algorithms and by uncertainty estimates for the cross-corpus comparisons.

major comments (5)
  1. [§3.2, Table 2, §6.1] The diversity sub-aspect is operationalized solely as convex-hull volume in BERT embedding space, with no validation that hull vertices or the pruned subset correspond to diverse summary content. The paper itself states in §6.1 that ConvexFall 'chooses the outlier sentences over the convex hull,' yet the corpus-level diversity bias in Figure 1(a) is derived from the ROUGE score of this same algorithm. Without a validation against human annotations or against a controlled diversity baseline such as MMR with varying redundancy penalties, a low diversity score for a corpus can simply mean that hull-vertex extraction is a poor extractive strategy for that corpus, not that the corpus lacks diversity bias.
  2. [§3.3, Table 4] The importance aspect is measured by N-Nearest and K-Nearest average Pearson correlations, a centrality heuristic that may select generic or repetitive sentences rather than summary-relevant ones. No comparison is made with established importance baselines such as LexRank, TextRank, or the MMR relevance term, and no ablation on the sentence encoder is provided. Because the system-bias columns R(P/D/I) in Table 4 are ROUGE scores between system outputs and these aspect-algorithm summaries, the system-bias analysis inherits the validity of the heuristics; an unvalidated importance proxy makes the reported system-level importance biases (e.g., MMR's high importance on XSum and Reddit) difficult to interpret.
  3. [§4, Eq. (3), footnote 6] The volume-overlap metric is computed by reducing sentence embeddings to 2D PCA before computing convex-hull overlap, but the paper gives no evidence that 2D hull overlap preserves the intended semantic-volume relation. The very high VO values for ConvexFall (e.g., 98.6 on AMI, 99.7 on BookSum, 93.5 on PubMed) likely reflect hull vertices spanning the 2D projection rather than semantic overlap, and VO is undefined for single-sentence target summaries such as XSum and Reddit. These values should be accompanied by a sensitivity analysis over PCA dimensions or replaced by a high-dimensional overlap measure.
  4. [§7, Table 4, Conclusion] The conclusion that 'neural systems rarely do [have bias]' is contradicted by Table 4: on CNNDM, CL, SumRun, and S2SExt have position components of 86, 68, and 74, respectively, with much smaller diversity and importance components. This is a load-bearing claim in the system-bias analysis and should be reworded or reconciled with the reported numbers.
  5. [§6, Table 1] All corpus-bias comparisons are reported as point estimates without confidence intervals or significance tests. Several corpora are small (Reddit has 404 training documents and AMI has 98), and some are preprocessed by truncation (BookSum uses the first 1,000 source sentences and first 50 target sentences), so the cross-corpus differences highlighted in Figure 1(a) may not be stable. The main claims, especially the news-versus-academic and news-versus-meeting position findings, should include bootstrap intervals or per-document variance.
minor comments (5)
  1. [Table 5] The Reddit Last-k ROUGE-1 score is listed as 116.4, which is impossible for an F-measure and appears to be a typo for 16.4.
  2. [Figures 7 and 8] The captions use 'KN' where the text and Figure 4 use 'NN' for the N-Nearest algorithm; the notation should be made consistent.
  3. [§3.2] The text says ConvexFall 'guarantees the maximum volume size,' but the subsequent pruning step removes this guarantee; the claim should be stated as applying only before pruning.
  4. [General] The paper does not state whether code and data splits will be released; a reproducibility statement would strengthen confidence in the empirical claims.
  5. [General] Minor typos include 'anarphorcity' in the Related Work section, 'V enn' in the Appendix B heading, and 'the the sub-aspects' in the Figure 6 caption.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the sub-aspect measurements use fixed, pre-defined algorithms and external benchmarks; the only self-citations are dataset/related-work references and are not load-bearing.

full rationale

The paper does not derive its conclusions from fitted parameters or from results that are equivalent to its own inputs by construction. The three sub-aspects are operationalized with explicit algorithms (Section 3): position by First/Last/Middle selection, diversity by volume maximization (ConvexFall/Heuristic), and importance by N-Nearest/K-Nearest correlation. These algorithms are fixed before evaluation; no target-summary-derived constants are fit. Corpus bias is then computed as ROUGE/SO between the pre-defined aspect summaries and target summaries, and system bias as ROUGE between systems and the same aspect summaries. This is an operationalization of the aspects, not a circular prediction: 'position bias' is not a hidden construct that is then used to define the position algorithm, but rather the algorithm's measured performance is presented as evidence of the aspect's prominence. The shared volume assumption of Yogatama et al. (2015) appears both in the diversity algorithm and in the VO metric, which is a modeling assumption rather than a circular reduction; moreover, the headline corpus-bias claims rely on ROUGE, not on VO. The only self-citations are Kang et al. (2018) for the PeerRead dataset and Lin and Hovy (1997) for the well-known position hypothesis; neither is load-bearing for the paper's central analysis, since the paper independently re-measures position with the First-k baseline and uses the dataset only as an external corpus. Thus there is no exhibited reduction of any claimed result to its input, and the analysis is self-contained given its stated operational assumptions.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper's analysis rests mainly on domain assumptions about what the embeddings and geometric operations measure; no physical or mathematical entities are invented. The sub-aspect taxonomy is inherited from prior theory but applied as if exhaustive.

free parameters (3)
  • Number of sentences k selected by aspect algorithms = equal to number of target summary sentences
    Each aspect algorithm selects the same number of sentences as the reference summary; this is a design choice, not fitted to optimize claims.
  • K in K-Nearest importance algorithm
    The neighborhood size K is not specified in the paper; the algorithm is described as averaging distances to the K nearest sentences, but K is never given.
  • PCA dimension for volume overlap = 2
    Volume overlap is computed after projecting BERT embeddings to 2D PCA space; the choice of 2 is due to the Shapely library limitation and is not validated.
assumptions (5)
  • domain assumption Sentence embeddings from the last layer of BERT, averaged over tokens, capture sufficient semantics for aspect analysis.
    Used in Section 3 for all aspect algorithms and VO; if BERT embeddings do not reflect the intended notions of importance/diversity, the measurements inherit the error.
  • domain assumption Maximizing convex hull volume of sentence vectors corresponds to semantic diversity.
    Adopted from Yogatama et al. (2015) in Section 3.2; the paper does not validate that hull vertices are semantically diverse rather than outliers.
  • domain assumption ROUGE F-measure is an adequate evaluation for comparing aspect algorithms.
    Used throughout Sections 6 and 7 to measure the strength of each aspect and system; the authors acknowledge ROUGE limitations but do not correct for them.
  • domain assumption The oracle extractive summary, maximizing ROUGE-L F against the reference, is a valid upper bound for extractive summarization.
    Defined in Section 4 and used for the SO metric; for abstractive targets the extractive oracle may be a loose proxy.
  • ad hoc to paper Three sub-aspects (position, importance, diversity) are sufficient to characterize summarization bias.
    Taken from Lin and Bilmes' sub-aspect theory but the paper does not check whether additional aspects are needed; the Venn oracle-recall numbers show large fractions of oracle sentences not covered by any aspect (up to 99%).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization." pith.science (2026). https://pith.science/paper/ZUNDOBCN

@misc{pith2026190811723,
  author       = {Pith},
  title        = {Pith review of: Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZUNDOBCN}},
  note         = {Machine review of arXiv:1908.11723}
}
read the original abstract

Despite the recent developments on neural summarization systems, the underlying logic behind the improvements from the systems and its corpus-dependency remains largely unexplored. Position of sentences in the original text, for example, is a well known bias for news summarization. Following in the spirit of the claim that summarization is a combination of sub-functions, we define three sub-aspects of summarization: position, importance, and diversity and conduct an extensive analysis of the biases of each sub-aspect with respect to the domain of nine different summarization corpora (e.g., news, academic papers, meeting minutes, movie script, books, posts). We find that while position exhibits substantial bias in news articles, this is not the case, for example, with academic papers and meeting minutes. Furthermore, our empirical study shows that different types of summarization systems (e.g., neural-based) are composed of different degrees of the sub-aspects. Our study provides useful lessons regarding consideration of underlying sub-aspects when collecting a new summarization dataset or developing a new system.

Figures

Figures reproduced from arXiv: 1908.11723 by the authors.

Figure 1
Figure 1. Corpus and system biases with the three sub [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Volume maximization functions. Black dots [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. PCA projection of extractive summaries cho [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: Intersection of averaged summary sentence [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 5
Figure 5. Figure 5: Sentence overlap proportion of each sub-aspect (row) with the oracle summary across corpora (column). [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Venndiagram of averaged summary sentence overlaps across the the sub-aspects for all datasets. We [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: PCA projection of extractive summaries chosen by multiple aspects of algorithms ( [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: PCA projection of extractive summaries chosen by multiple aspects of algorithms ( [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: System biases with the three sub-aspects per each corpus, showing what portion of aspect is used for [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reranking-based Generation for Unbiased Perspective Summarization

    cs.CL 2025-06 conditional novelty 6.0 of 10

    LLM-based metrics (LLM-Coverage and ALIGN SCORE) are more reliable than ROUGE and BERTScore for political perspective summaries, and reranking plus DPO training improves both coverage and faithfulness.

Reference graph

Works this paper leans on

55 extracted references · 32 canonical work pages · cited by 1 Pith paper

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A convolutional attention network for extreme summarization of source code. arXiv preprint arXiv:1602.03001

    arXiv 2016

  4. [4]

    Siddhartha Banerjee, Prasenjit Mitra, and Kazunari Sugiyama. 2015. Multi-document abstractive summarization using ilp based multi-sentence compression. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015)

    work page 2015

  5. [5]

    C Bradford Barber, David P Dobkin, David P Dobkin, and Hannu Huhdanpaa. 1996. The quickhull algorithm for convex hulls. ACM Transactions on Mathematical Software (TOMS), 22(4):469--483

    work page 1996

  6. [6]

    Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems, pages 1171--1179

    2015

  7. [7]

    Taylor Berg-Kirkpatrick, Dan Gillick, and Dan Klein. 2011. Jointly learning to extract and compress. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 481--490. Association for Computational Linguistics

    work page 2011

  8. [8]

    Florian Boudin and Emmanuel Morin. 2013. Keyphrase extraction for n-best reranking in multi-sentence compression. In North American Chapter of the Association for Computational Linguistics (NAACL)

    work page 2013

Show all 55 references
  1. [9]

    Florian Boudin, Hugo Mougard, and Benoit Favre. 2015. Concept-based summarization using integer linear programming: From concept pruning to multiple optimal solutions. In Conference on Empirical Methods in Natural Language Processing (EMNLP) 2015

    2015

  2. [10]

    Jaime Carbonell and Jade Goldstein. 1998. The use of mmr, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pages 335--336. ACM

    1998

  3. [11]

    Jean Carletta, Simone Ashby, Sebastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, et al. 2005. The ami meeting corpus: A pre-announcement. In International Workshop on Machine Learning for Multimodal...

    2005

  4. [12]

    Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. arXiv preprint arXiv:1603.07252

    2016 arXiv

  5. [13]

    Trevor Cohn and Mirella Lapata. 2008. Sentence compression beyond word deletion. In Proceedings of the 22nd International Conference on Computational Linguistics-Volume 1, pages 137--144. Association for Computational Linguistics

    2008

  6. [14]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  7. [15]

    Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein. 2016. Learning-based single-document summarization with compression and anaphoricity constraints. arXiv preprint arXiv:1603.08887

    2016 arXiv

  8. [16]

    Harold P Edmundson. 1969. New methods in automatic extracting. Journal of the ACM (JACM), 16(2):264--285

    1969

  9. [17]

    G \"u nes Erkan and Dragomir R Radev. 2004. Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research, pages 457--479

    2004

  10. [18]

    Katja Filippova. 2010. Multi-sentence compression: finding shortest paths in word graphs. In Proceedings of the 23rd International Conference on Computational Linguistics, pages 322--330. Association for Computational Linguistics

    2010

  11. [19]

    Kavita Ganesan, ChengXiang Zhai, and Jiawei Han. 2010. Opinosis: a graph-based approach to abstractive summarization of highly redundant opinions. In Proceedings of the 23rd international conference on computational linguistics, pages 340--348. Association for Computational Li...

    2010

  12. [20]

    Sebastian Gehrmann, Yuntian Deng, and Alexander M Rush. 2018. Bottom-up abstractive summarization. arXiv preprint arXiv:1808.10792

    2018 arXiv

  13. [21]

    Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of EMNLP

    2014

  14. [22]

    Dan Gillick and Benoit Favre. 2009. A scalable global model for summarization. In Proceedings of the Workshop on Integer Linear Programming for Natural Langauge Processing, pages 10--18. Association for Computational Linguistics

    2009

  15. [23]

    Philip John Gorinski and Mirella Lapata. 2015. Movie script summarization as graph-based scene extraction. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1066--1076

    2015

  16. [24]

    David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2003. English gigaword. Linguistic Data Consortium, Philadelphia, 4(1):34

    2003

  17. [25]

    Max Grusky, Mor Naaman, and Yoav Artzi. 2018. http://aclweb.org/anthology/N18-1065 Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguis...

    2018

  18. [26]

    Kai Hong and Ani Nenkova. 2014. Improving the estimation of word importance for news multi-document summarization. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 712--721

    2014

  19. [27]

    Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine van Zuylen, Sebastian Kohlmeier, Eduard Hovy, and Roy Schwartz. 2018. https://arxiv.org/abs/1804.09635 A dataset of peer reviews (peerread): Collection, insights and nlp applications . In Meeting of the North American Chapt...

    2018 arXiv

  20. [28]

    Chris Kedzie, Kathleen McKeown, and Hal Daume III. 2018. Content selection in deep learning models of summarization. arXiv preprint arXiv:1810.12343

    2018 arXiv

  21. [29]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out: Proceedings of the ACL-04 workshop, volume 8

    2004

  22. [30]

    Chin-Yew Lin and Eduard Hovy. 1997. Identifying topics by position. In Fifth Conference on Applied Natural Language Processing

    1997

  23. [31]

    Chin-Yew Lin and Eduard Hovy. 2000. The automated acquisition of topic signatures for text summarization. In Proceedings of the 18th conference on Computational linguistics-Volume 1, pages 495--501. Association for Computational Linguistics

    2000

  24. [32]

    Hui Lin and Jeff Bilmes. 2010. Multi-document summarization via budgeted maximization of submodular functions. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 912--920. Association...

    2010

  25. [33]

    Hui Lin and Jeff Bilmes. 2011. A class of submodular functions for document summarization. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 510--520. Association for Computational Linguistics

    2011

  26. [34]

    Hui Lin and Jeff A Bilmes. 2012. Learning mixtures of submodular shells with application to document summarization. arXiv preprint arXiv:1210.4871

    2012 arXiv

  27. [35]

    Fei Liu, Jeffrey Flanigan, Sam Thomson, Norman Sadeh, and Noah A Smith. 2015. Toward abstractive summarization using semantic representations. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T...

    2015

  28. [36]

    Daniel Marcu. 1999. Discourse trees are good indicators of importance in text. Advances in automatic text summarization, 293:123--136

    1999

  29. [37]

    Ryan McDonald. 2007. A study of global inference algorithms in multi-document summarization. Springer

    2007

  30. [38]

    Yashar Mehdad, Giuseppe Carenini, and Raymond Ng. 2014. Abstractive summarization of spoken and written conversations based on phrasal queries. In Proc. of ACL, pages 1220--1230

    2014

  31. [39]

    Rada Mihalcea and Hakan Ceylan. 2007. Explorations in automatic book summarization. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL)

    2007

  32. [40]

    Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In Proceedings of the 2004 conference on empirical methods in natural language processing, pages 404--411

    2004

  33. [41]

    Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Thirty-First AAAI Conference on Artificial Intelligence

    2017

  34. [42]

    Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 280--290

    2016

  35. [43]

    Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018 a . Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745

    2018 arXiv

  36. [44]

    Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018 b . Ranking sentences for extractive summarization with reinforcement learning. arXiv preprint arXiv:1802.08636

    2018 arXiv

  37. [45]

    Jessica Ouyang, Serina Chang, and Kathy McKeown. 2017. Crowd-sourced iterative annotation for narrative summarization corpora. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 46--51

    2017

  38. [46]

    Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304

    2017 arXiv

  39. [47]

    Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543

    2014

  40. [48]

    Maxime Peyrard. 2019 a . https://www.aclweb.org/anthology/P19-1101 A simple theoretical model of importance for summarization . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1059--1073, Florence, Italy. Association for Comput...

    2019

  41. [49]

    Maxime Peyrard. 2019 b . https://www.aclweb.org/anthology/P19-1502 Studying summarization evaluation metrics in the appropriate scoring range . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5093--5100, Florence, Italy. Associ...

    2019

  42. [50]

    Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685

    2015 arXiv

  43. [51]

    Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368

    2017 arXiv

  44. [52]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998--6008

    2017

  45. [53]

    Kristian Woodsend and Mirella Lapata. 2011. Learning to simplify sentences with quasi-synchronous grammar and integer programming. In Proceedings of the conference on empirical methods in natural language processing, pages 409--420. Association for Computational Linguistics

    2011

  46. [54]

    Dani Yogatama, Fei Liu, and Noah A Smith. 2015. Extractive summarization by maximizing semantic volume. In EMNLP, pages 1961--1966

    2015

  47. [55]

    David Zajic, Bonnie Dorr, and Richard Schwartz. 2004. Bbn/umd at duc-2004: Topiary. In Proceedings of the HLT-NAACL 2004 Document Understanding Workshop, Boston, pages 112--119

    2004

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.