Pith. sign in

REVIEW 3 major objections 5 minor 59 references

Decoupling Generation and Selection for Budget-Constrained Faithful Summarization

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Global sentence selection after generation consistently improves summary factuality while lowering reference-overlap scores, across six generators and four benchmarks.

desk verdict Solid, honest system paper; the main result is probably real but the paper never isolates the selector from the candidate pool, and that gap should be fixed before publication. read the letter →

arxiv 2608.03655 v1 pith:B7IP7QSX submitted 2026-08-04 cs.CL cs.AI

Zeyu Wang , Guanghua Wang , Meng Xu This is my paper · ORCID
classification cs.CLcs.AI
keywords abstractivesummarizationfactualconsistencysentenceselectionbudgetconstraintdeterminantalpointprocessesgeneration-and-selectionhallucinationreductionmulti-document
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the way to make abstractive summarizers faithful is to stop letting a single decoding run decide the final text. Its framework generates several candidate summaries, splits them into sentences, and then treats the final summary as a combinatorial optimization problem: pick the subset of candidate sentences that best balances source relevance, factual support, and redundancy, under a hard sentence budget. Across CNN/DailyMail, Multi-News, FaithBench, and TofuEval, this selection stage improves every factuality-oriented metric relative to direct generation, by more than twenty points on MiniCheck and AlignScore in multi-document settings, while ROUGE and BERTScore drop. A blind human evaluation prefers the selected summaries 61 to 26 and rates them higher on consistency, relevance, clarity, and conciseness. The claim matters because it offers a retraining-free, model-agnostic lever for factual grounding: any generator can be wrapped with the selector, and the trade-off with reference overlap is explicit and controllable.

What carries the argument

The load-bearing object is the candidate-utility function combined with the log-determinant subset criterion. Each sentence c_i in the deduplicated pool receives utility u_i = λ_cov·g_i + λ_fact·m_i, where g_i is a ROUGE-based coverage score and m_i is a MiniCheck factuality score, and pairs receive a redundancy score R_ij. The DPP-inspired selector forms L = QKQ + εI from these, Q being a diagonal quality matrix built from the utilities and K the scaled redundancy similarity matrix, then greedily maximizes log det(L_A) over subsets of size B with a positivity check and fallback. The same utilities and similarities feed MMR and ILP, so the determinant objective is the proposed mechanism that

What would settle it

Have expert annotators label unsupported spans, as FaithBench's gold labels do, on direct versus DPP-selected outputs, and check whether the large automatic gains—for example, BART+DPP on FaithBench raising MiniCheck from 86.62 to 92.89—appear as an actual reduction in human-flagged errors. If the automatic scores improve while expert-flagged errors stay flat or rise, the claim that selection improves faithfulness is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that factual grounding in abstractive summarization is largely a selection problem rather than a generation problem. Because supported information recurs across multiple decoding trajectories while hallucinated content is less stable, a combinatorial selector operating on the pooled sentences of several candidate summaries can filter out weak claims and reassemble complementary facts. The paper instantiates this with a DPP-inspired log-determinant objective built from per-sentence coverage–factuality utilities and pairwise redundancy scores, and shows it improves source-grounding metrics across task-specific and instruction-tuned generators, with th

Load-bearing premise

The sentence-level factuality score used during selection (MiniCheck) genuinely tracks whether a claim is supported by the source, so that choosing sentences with higher scores really does produce more faithful summaries rather than just higher scores.

Editorial extensions

If this is right

  • Any pretrained summarizer or instruction-tuned LLM can improve source grounding without retraining, by adding the selection stage over its own multiple decoded outputs.
  • Multi-document summarization is the biggest winner: PRIMERA+DPP lifts MiniCheck from 63.06 to 85.38 and AlignScore from 53.30 to 71.27 on Multi-News.
  • The sentence budget becomes a hard, reliably enforced constraint: the DPP selector meets the target exactly 100% of the time across B = 2–5, where direct generation's hit rate falls to 13.7–66.4%.
  • Factuality and reference-overlap are trading dimensions, not interchangeable measures: gains in grounding metrics systematically come with lower ROUGE and BERTScore, and the balance can be shifted with the weight configuration.
  • Selector choice matters but does not change the qualitative conclusion: MMR, ILP, and DPP all improve factuality under matched pools; DPP and ILP favor grounding while MMR preserves more reference overlap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the selection signal is an explicit factuality score, the framework's ceiling is tied to the quality of that score: replacing MiniCheck with a stronger or domain-adapted fact-checker should directly translate into better selected summaries, making the approach a testbed for factuality estimators.
  • The 'stable content repeats, hallucinations do not' premise predicts that the method will help least when hallucinations are systematic rather than stochastic—when a generator confidently repeats the same unsupported claim across all trajectories; a targeted test on such cases would delimit the framework's reach.
  • The source-aligned ordering step implies the method can be extended to produce extractive-style ordering with abstractive content; discourse coherence might improve if a learned planner orders the selected sentences instead of their source positions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a generation-and-selection framework for sentence-budget-constrained abstractive summarization. Instead of using a single autoregressive output, the system generates multiple candidate summaries, decomposes them into sentence-level candidates, and selects a budget-constrained subset by balancing coverage, factuality, and redundancy (Eqs. 1, 6, 9). The selector is instantiated as MMR, ILP, or a DPP-inspired log-determinant objective, with the selected sentences ordered by source position. Experiments on CNN/DailyMail, Multi-News, FaithBench, and TofuEval compare the +DPP pipeline against direct generation and several baselines, reporting consistent improvements on factuality-oriented metrics (FactCC, MiniCheck, AlignScore, FactKB, FaithLens) at the cost of lower ROUGE/BERTScore, plus a blind human preference study. The manuscript includes ablations over weights, budgets, and candidate-pool size, and paired statistical tests with Holm correction.

Significance. If the central claim is validated, the framework offers a model-agnostic, retraining-free mechanism for improving factual grounding in abstractive summarization, which would be a useful contribution to the field. The paper's strengths include a modular design, code release, careful paired bootstrap/permutation statistics, and evaluation across multiple factuality metrics and backbones (BART, PRIMERA, Llama-3-8B). The explicit reporting of the reference-overlap trade-off is also commendable. However, the main empirical claim is currently weakened by a confound between the selection objective and the size of the candidate pool, and by the use of MiniCheck both as the selection signal and as a headline evaluation metric. The human evaluation is a positive addition but is limited in scale and annotator reliability. These issues are addressable and do not necessarily invalidate the approach, but they must be resolved before the paper can substantiate its central claim.

major comments (3)
  1. [§4.1, §4.3, Appendix B.3, Appendix D.3 (Table 8)] The +DPP comparisons in Tables 1–2 confound the selection objective with candidate-pool size. Direct baselines use a single generated summary, while +DPP generates 12 hypotheses (Appendix B.3) before sentence-level selection. Appendix D.3 (Table 8) shows that factuality metrics improve monotonically as the beam grows from 4 to 12 (e.g., PRIMERA MiniCheck 62.26→85.38, AlignScore 52.24→71.27). Therefore the large gains in Table 1 (e.g., PRIMERA+DPP MiniCheck 63.06→85.38) may be largely due to having 12 candidates rather than to the balanced coverage–factuality–redundancy objective in Eq. 1/Eq. 9. Table 3 holds the pool fixed, but it compares selectors to each other, not against direct generation on the same cached pool, and it does not include a trivial selector (random B sentences, first B sentences, or best full candidate by MiniCheck). Without such controls, the paper's claim that the c
  2. [§3.3 (Eq. 3), Appendix B.3, Tables 1–3, 7, 8, 10, §6] MiniCheck is used both as the factuality score in the selection objective (Eq. 3; the balanced weighting in Appendix B.3 sets wfact=0.33 on MiniCheck) and as the first factuality metric in the main tables and statistical appendix. Consequently, improvements on MiniCheck are partly a selection artifact: the selector is explicitly optimizing that score. The manuscript acknowledges this in §6 ('improvements on related automatic evaluation metrics may therefore partly reflect metric alignment'), but the abstract and conclusion state unqualified 'improvements in factuality.' This is a load-bearing issue because the reader must be able to verify that the gains are not merely metric optimization. The paper does report gains on four other factuality metrics not used in the objective (FactCC, AlignScore, FactKB, FaithLens), which mitigates the concern. Still, I recommend either (a) augmenting the
  3. [§4.4, Appendix E, Table 4, §6] The human evaluation uses only 100 CNN/DailyMail examples, one annotator per comparison, and no inter-annotator reliability measure. This is explicitly acknowledged in §6, and the results are described as a preference study. However, the report in §4.4 states that DPP-selected summaries receive higher mean ratings for consistency, relevance, clarity, and conciseness without any inferential statistics (e.g., paired tests on ratings or at least a permutation test on preference counts). Given that each rating is a single annotator's judgment, the numerical differences in Table 4(b) (e.g., clarity +0.765, conciseness +0.892) could be driven by annotator idiosyncrasy. Furthermore, the higher preference for DPP may reflect the 'conciseness' dimension (which the selector explicitly enforces via budget and redundancy) rather than improved factual fidelity. I recommend adding statistical tests, a
minor comments (5)
  1. [§3.4, Eq. 9] Eq. 9 uses the constraint |A| = min(B, |C|), while the general formulation in Eq. 1 uses |A| ≤ B. The paper should clarify whether the implemented selector always fills the budget and why the equality constraint does not affect the reported budget-adherence results in Table 7.
  2. [§4.3] The sentence 'This result supports the modularity of the framework and shows that the observed gains do not arise solely from generating a larger candidate pool' is not justified by Table 3, since Table 3 does not include a direct-generation baseline using the same candidate pool. The conclusion that the gains are not solely due to pool size requires the trivial-selector controls described in the first major comment.
  3. [Abstract and §1] Minor typographical issues: 'generation-and-selectionproblem' lacks a space (Abstract and Introduction); 'LLMs) have substantially improved the fluency and semantic quality of abstractive summarization' is fine, but some references are inconsistently formatted. The paper should also define 'm' in Eq. 3 (the factuality model) explicitly; it is only implicit from the text.
  4. [Table 8] The column header 'Beam' with values 4/8/12 is clear, but the table caption and text refer to 'beam size' as candidate-generation width. Since the same width may produce different pool sizes after deduplication, it would be informative to report the average number of unique sentence candidates per pool for each beam setting, to better connect pool size to the observed gains.
  5. [Appendix E, Table 9] The annotation protocol is clear, but the 'Not sure' option is ambiguous: it could mean the annotator cannot decide or the outputs are equally good in different ways. Consider clarifying this in the instructions or merging with 'Same' in the analysis.

Circularity Check

1 steps flagged · score 4.0 of 10

MiniCheck gains are partly by construction, but independent metrics and human evaluation keep the central claim substantive.

  1. fitted input called prediction [Section 3.3, Eq. 3; Appendix B.3; Section 6; Tables 1-3 and Appendix F]
    "we compute a sentence-level factuality score: mi = Pfact(consistent | D, ci) ... sentence-level factuality is estimated using MiniCheck (Tang et al., 2024a) ... Improvements on related automatic evaluation metrics may therefore partly reflect metric alignment and should not be interpreted as guarantees of error-free factuality."

    The factuality term in the selection objective (Eq. 3) is implemented as MiniCheck, and MiniCheck is also the primary faithfulness metric reported in Tables 1-3 and the paired statistics in Appendix F. Selecting sentences to maximize a MiniCheck-based utility and then reporting MiniCheck improvements as evidence of better factuality is a selection artifact: the measured metric is the same function that was optimized. This is partial circularity, not total, because FactCC, AlignScore, FactKB, FaithLens, and the human evaluation are not used in the objective and still improve; the paper's own Limitations section explicitly concedes metric alignment.

full rationale

The paper's derivation chain is otherwise self-contained: Eq. 1-9 define a generation-and-selection objective over candidate sentences with coverage, factuality, redundancy, and a greedy log-determinant selector, and the comparisons among MMR/ILP/DPP hold the candidate pools fixed (Table 3). The central claim that decoupling generation from selection improves source grounding has independent support from factuality metrics that are not in the objective (FactCC, AlignScore, FactKB, FaithLens) and from a blind human preference study. However, the MiniCheck column is not independent evidence: Eq. 3's factuality score is MiniCheck (Appendix B.3), so the reported MiniCheck gains are partly forced by the objective. The paper acknowledges this in Section 6. The main +DPP-versus-direct comparisons also confound sentence selection with a larger candidate pool (12 summaries vs 1; Table 8 shows pool size alone improves factuality), but that is an attribution/control issue rather than an equivalence-by-construction circularity. No load-bearing self-citations or imported uniqueness theorems were found. Score 4 reflects one genuine partial circularity while the central claim retains substantial independent content.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The framework depends on the validity of automatic scoring functions, the recurrence assumption about generated content, and several hand-chosen hyperparameters. It introduces no new theoretical entities.

free parameters (8)
  • coverage weight wcov = 0.33 (balanced setting)
    Chosen by hand, controls the contribution of ROUGE-based coverage in Eq. 4; varied in sensitivity analysis (Appendix D.1).
  • factuality weight wfact = 0.33 (balanced setting)
    Chosen by hand for MiniCheck-based factuality; sensitivity analysis shows faithfulness-heavy weighting improves factuality further.
  • redundancy weight wred = 0.34 (balanced setting)
    Chosen by hand; used to scale pairwise redundancy and in alpha = wred / max(1, B-1).
  • sentence budget B = 3 (main experiments), varied {2,4,5}
    Explicit budget constraint set by the user, not learned; appendix shows the selector satisfies it 100% of the time.
  • candidate generation width (beam size) = 12 (main), varied {4,8}
    Number of generated hypotheses; larger pools improve factuality but increase cost.
  • epsilon_q quality floor = 0.01
    Clipping floor for quality scores in Eq. 7 to avoid zero entries in the Q matrix.
  • epsilon regularization for L = 1e-6
    Numerical regularization added to the L kernel in Eq. 8.
  • ILP redundancy threshold = not stated
    The hard ILP formulation uses a configured threshold for mutual exclusion; the threshold value is not reported in the paper.
assumptions (5)
  • domain assumption MiniCheck provides a reliable estimate of the factual consistency of a candidate sentence
    Used as the factuality score in Eq. 3 and as a headline evaluation metric; if biased, selection optimizes the wrong objective. Acknowledged in Section 6.
  • domain assumption Salient, source-supported information recurs across multiple decoding trajectories, while hallucinated content is less stable
    Motivates generating multiple candidates and expecting the pool to contain the correct facts (Sections 1 and 3.2). Not directly tested.
  • domain assumption Sentence-level decomposition and source-aligned concatenation preserve summary quality sufficient for the downstream metrics
    The framework selects individual sentences and orders them by source position; human eval shows a small coherence drop, but the final text is treated as a valid summary.
  • domain assumption The factuality evaluation metrics (FactCC, AlignScore, FactKB, FaithLens) measure true factual consistency
    Used as ground truth for evaluation; these are automatic classifiers known to be imperfect, and the paper itself notes metric alignment concerns.
  • standard math Greedy log-determinant maximization is an adequate approximation to the combinatorial objective in Eq. 9
    Exact optimization is combinatorial; the authors use greedy selection with a non-PSD kernel and explicitly call the method DPP-inspired rather than a true DPP.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decoupling Generation and Selection for Budget-Constrained Faithful Summarization." pith.science (2026). https://pith.science/paper/B7IP7QSX

@misc{pith2026260803655,
  author       = {Pith},
  title        = {Pith review of: Decoupling Generation and Selection for Budget-Constrained Faithful Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B7IP7QSX}},
  note         = {Machine review of arXiv:2608.03655}
}
read the original abstract

Abstractive summarization models remain vulnerable to factual inconsistency, redundancy, and weak length control. We propose a modular generation-and-selection framework for sentence-budget-constrained summarization. A pretrained generator produces multiple candidate summaries, which are decomposed into sentence-level candidates. A combinatorial selector then constructs the final summary by balancing relevance, factuality, and redundancy under an explicit budget. The framework supports MMR, ILP, and a DPP-inspired log-determinant objective without retraining the generator. Experiments on CNN/DailyMail, Multi-News, FaithBench, and TofuEval show consistent improvements in factuality and source-grounding metrics, especially for multi-document summarization, at the cost of lower reference-overlap scores. Human evaluation further indicates higher perceived consistency, relevance, clarity, and conciseness, with a small reduction in coherence. These results show that decoupling generation from selection provides a model-agnostic mechanism for improving factual grounding. Code is available at https://anonymous.4open.science/r/bcfs-D05E/.

Figures

Figures reproduced from arXiv: 2608.03655 by the authors.

Figure 1
Figure 1. From generation to planning for abstractive [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed framework. A pretrained generator first produces multiple candidate summaries, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 19 canonical work pages

  1. [1]

    Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling

    Balachandran, Vidhisha and Hajishirzi, Hannaneh and Cohen, William and Tsvetkov, Yulia. Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.667

    work page doi:10.18653/v1/2022.emnlp-main.667 2022

  2. [2]

    BART : Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

    Lewis, Mike and Liu, Yinhan and Goyal, Naman and Ghazvininejad, Marjan and Mohamed, Abdelrahman and Levy, Omer and Stoyanov, Veselin and Zettlemoyer, Luke. BART : Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguisti...

    doi:10.18653/v1/2020.acl-main.703 2020

  3. [3]

    2026 , eprint=

    Qwen3.5-Omni Technical Report , author=. 2026 , eprint=

    2026

  4. [4]

    S im CLS : A Simple Framework for Contrastive Learning of Abstractive Summarization

    Liu, Yixin and Liu, Pengfei. S im CLS : A Simple Framework for Contrastive Learning of Abstractive Summarization. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 2021. doi:10.18653/v1/2021.acl-short.135

    doi:10.18653/v1/2021.acl-short.135 2021

  5. [5]

    BRIO : Bringing Order to Abstractive Summarization

    Liu, Yixin and Liu, Pengfei and Radev, Dragomir and Neubig, Graham. BRIO : Bringing Order to Abstractive Summarization. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.207

    doi:10.18653/v1/2022.acl-long.207 2022

  6. [6]

    Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality

    Dixit, Tanay and Wang, Fei and Chen, Muhao. Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2023. doi:10.18653/v1/2023.acl-short.78

    doi:10.18653/v1/2023.acl-short.78 2023

  7. [7]

    S umma R eranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization

    Ravaut, Mathieu and Joty, Shafiq and Chen, Nancy. S umma R eranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.309

    doi:10.18653/v1/2022.acl-long.309 2022

  8. [8]

    2025 , eprint=

    gpt-oss-120b and gpt-oss-20b Model Card , author=. 2025 , eprint=

    2025

Show all 59 references
  1. [9]

    2026 , howpublished =

    Gemma 4 E4B IT , author =. 2026 , howpublished =

    2026

  2. [10]

    2024 , url =

    Llama 3 Model Card , author=. 2024 , url =

    2024

  3. [11]

    Abstractive Text Summarization using Sequence-to-sequence RNN s and Beyond

    Nallapati, Ramesh and Zhou, Bowen and dos Santos, Cicero and Gu l c ehre, C a g lar and Xiang, Bing. Abstractive Text Summarization using Sequence-to-sequence RNN s and Beyond. Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. 2016. doi:10.1...

    2016 doi

  4. [12]

    Multi-News: A Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model

    Fabbri, Alexander and Li, Irene and She, Tianwei and Li, Suyi and Radev, Dragomir. Multi-News: A Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. ...

    2019 doi

  5. [13]

    ROUGE : A Package for Automatic Evaluation of Summaries

    Lin, Chin-Yew. ROUGE : A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out. 2004

    2004

  6. [14]

    2020 , eprint=

    BERTScore: Evaluating Text Generation with BERT , author=. 2020 , eprint=

    2020

  7. [15]

    Evaluating the Factual Consistency of Abstractive Text Summarization

    Kryscinski, Wojciech and McCann, Bryan and Xiong, Caiming and Socher, Richard. Evaluating the Factual Consistency of Abstractive Text Summarization. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18653/v1/2020.emnlp...

    2020 doi

  8. [16]

    M ini C heck: Efficient Fact-Checking of LLM s on Grounding Documents

    Tang, Liyan and Laban, Philippe and Durrett, Greg. M ini C heck: Efficient Fact-Checking of LLM s on Grounding Documents. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.499

    2024 doi

  9. [17]

    A lign S core: Evaluating Factual Consistency with A Unified Alignment Function

    Zha, Yuheng and Yang, Yichi and Li, Ruichen and Hu, Zhiting. A lign S core: Evaluating Factual Consistency with A Unified Alignment Function. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023. doi:10.18653/v1/...

    2023 doi

  10. [18]

    F act KB : Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge

    Feng, Shangbin and Balachandran, Vidhisha and Bai, Yuyang and Tsvetkov, Yulia. F act KB : Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. do...

    2023 doi

  11. [19]

    PRIMERA : Pyramid-based Masked Sentence Pre-training for Multi-document Summarization

    Xiao, Wen and Beltagy, Iz and Carenini, Giuseppe and Cohan, Arman. PRIMERA : Pyramid-based Masked Sentence Pre-training for Multi-document Summarization. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:...

    2022 doi

  12. [20]

    Benchmarking Large Language Models for News Summarization

    Zhang, Tianyi and Ladhak, Faisal and Durmus, Esin and Liang, Percy and McKeown, Kathleen and Hashimoto, Tatsunori B. Benchmarking Large Language Models for News Summarization. Transactions of the Association for Computational Linguistics. 2024. doi:10.1162/tacl_a_00632

    2024 doi

  13. [21]

    On Context Utilization in Summarization with Large Language Models

    Ravaut, Mathieu and Sun, Aixin and Chen, Nancy and Joty, Shafiq. On Context Utilization in Summarization with Large Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.153

    2024 doi

  14. [22]

    Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias

    Chhabra, Anshuman and Askari, Hadi and Mohapatra, Prasant. Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computat...

    2024 doi

  15. [23]

    Discrete Mathematics, Algorithms and Applications , volume=

    Surveying the landscape of text summarization with deep learning: A comprehensive review , author=. Discrete Mathematics, Algorithms and Applications , volume=. 2024 , publisher=

    2024

  16. [24]

    On Learning to Summarize with Large Language Models as References

    Liu, Yixin and Shi, Kejian and He, Katherine and Ye, Longtian and Fabbri, Alexander and Liu, Pengfei and Radev, Dragomir and Cohan, Arman. On Learning to Summarize with Large Language Models as References. Proceedings of the 2024 Conference of the North American Chapter of the...

    2024 doi

  17. [25]

    Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization

    Liu, Yixin and Fabbri, Alexander and Chen, Jiawen and Zhao, Yilun and Han, Simeng and Joty, Shafiq and Liu, Pengfei and Radev, Dragomir and Wu, Chien-Sheng and Cohan, Arman. Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllab...

    2024 doi

  18. [26]

    Exploring Iterative Controllable Summarization with Large Language Models

    Ryu, Sangwon and Do, Heejin and Kim, Daehui and Yu, Hwanjo and Kim, Dongwoo and Kim, Yunsu and Lee, Gary and Ok, Jungseul. Exploring Iterative Controllable Summarization with Large Language Models. Findings of the A ssociation for C omputational L inguistics: EACL 2026. 2026. ...

    2026 doi

  19. [27]

    and Marshall, Iain J

    DeYoung, Jay and Martinez, Stephanie C. and Marshall, Iain J. and Wallace, Byron C. , title =. Transactions of the Association for Computational Linguistics , volume =. 2024 , month =. doi:10.1162/tacl_a_00687 , url =

    2024 doi

  20. [28]

    Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles

    Huang, Kung-Hsiang and Laban, Philippe and Fabbri, Alexander and Choubey, Prafulla Kumar and Joty, Shafiq and Xiong, Caiming and Wu, Chien-Sheng. Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Informatio...

    2024 doi

  21. [29]

    Tell me what I need to know: Exploring LLM -based (Personalized) Abstractive Multi-Source Meeting Summarization

    Kirstein, Frederic and Ruas, Terry and Kratel, Robert and Gipp, Bela. Tell me what I need to know: Exploring LLM -based (Personalized) Abstractive Multi-Source Meeting Summarization. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Indust...

    2024 doi

  22. [30]

    Structure-Aware Chunking for Abstractive Summarization of Long Legal Documents

    Sonowal, Himadri and Sadhu, Saisab. Structure-Aware Chunking for Abstractive Summarization of Long Legal Documents. Proceedings of the 1st Workshop on NLP for Empowering Justice (JUST-NLP 2025). 2025. doi:10.18653/v1/2025.justnlp-main.19

    2025 doi

  23. [31]

    CCS um: A Large-Scale and High-Quality Dataset for Abstractive News Summarization

    Jiang, Xiang and Dreyer, Markus. CCS um: A Large-Scale and High-Quality Dataset for Abstractive News Summarization. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa...

    2024 doi

  24. [32]

    Extractive Summarization with Text Generator

    Le, Thang and Luu, Anh Tuan. Extractive Summarization with Text Generator. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.naacl-long.9

    2024 doi

  25. [33]

    Extract, Select and Rewrite: A Modular Sentence Summarization Method

    Guan, Shuo and Padmakumar, Vishakh. Extract, Select and Rewrite: A Modular Sentence Summarization Method. Proceedings of the 4th New Frontiers in Summarization Workshop. 2023. doi:10.18653/v1/2023.newsum-1.4

    2023 doi

  26. [34]

    On Faithfulness and Factuality in Abstractive Summarization

    Maynez, Joshua and Narayan, Shashi and Bohnet, Bernd and McDonald, Ryan. On Faithfulness and Factuality in Abstractive Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.173

    2020 doi

  27. [35]

    Asking and Answering Questions to Evaluate the Factual Consistency of Summaries

    Wang, Alex and Cho, Kyunghyun and Lewis, Mike. Asking and Answering Questions to Evaluate the Factual Consistency of Summaries. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.450

    2020 doi

  28. [36]

    QAF act E val: Improved QA -Based Factual Consistency Evaluation for Summarization

    Fabbri, Alexander and Wu, Chien-Sheng and Liu, Wenhao and Xiong, Caiming. QAF act E val: Improved QA -Based Factual Consistency Evaluation for Summarization. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Huma...

    2022 doi

  29. [37]

    Evaluating the Factual Consistency of Large Language Models Through News Summarization

    Tam, Derek and Mascarenhas, Anisha and Zhang, Shiyue and Kwan, Sarah and Bansal, Mohit and Raffel, Colin. Evaluating the Factual Consistency of Large Language Models Through News Summarization. Findings of the Association for Computational Linguistics: ACL 2023. 2023. doi:10.1...

    2023 doi

  30. [38]

    Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistency

    Li, Taiji and Li, Zhi and Zhang, Yin. Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistency. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLI...

    2024

  31. [39]

    F aith B ench: A Diverse Hallucination Benchmark for Summarization by M odern LLM s

    Bao, Forrest Sheng and Li, Miaoran and Qu, Renyi and Luo, Ge and Wan, Erana and Tang, Yujia and Fan, Weisi and Tamber, Manveer Singh and Kazi, Suleman and Sourabh, Vivek and Qi, Mike and Tu, Ruixuan and Xu, Chenyu and Gonzales, Matthew and Mendelevitch, Ofer and Ahmad, Amin. F...

    2025

  32. [40]

    On Positional Bias of Faithfulness for Long-form Summarization

    Wan, David and Vig, Jesse and Bansal, Mohit and Joty, Shafiq. On Positional Bias of Faithfulness for Long-form Summarization. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies...

    2025 doi

  33. [41]

    Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models

    Huang, Yanwen and Zhang, Yong and Cheng, Ning and Li, Zhitao and Wang, Shaojun and Xiao, Jing. Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models. Findings of the Association for Computational Linguistics: ACL ...

    2025 doi

  34. [42]

    Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information

    Chae, Kyubyung and Choi, Jaepill and Jo, Yohan and Kim, Taesup. Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information. Findings of the Association for Computational Linguistics: NAACL 2024. 2024. doi:10.18653/v1/2024.findings-naacl.117

    2024 doi

  35. [43]

    Hallucination Diversity-Aware Active Learning for Text Summarization

    Xia, Yu and Liu, Xu and Yu, Tong and Kim, Sungchul and Rossi, Ryan and Rao, Anup and Mai, Tung and Li, Shuai. Hallucination Diversity-Aware Active Learning for Text Summarization. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computati...

    2024 doi

  36. [44]

    MAMM -Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration

    Wan, David and Chen, Justin and Stengel-Eskin, Elias and Bansal, Mohit. MAMM -Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computationa...

    2025 doi

  37. [45]

    1998 , isbn =

    Carbonell, Jaime and Goldstein, Jade , title =. 1998 , isbn =. doi:10.1145/290941.291025 , booktitle =

    1998

  38. [46]

    A Scalable Global Model for Summarization

    Gillick, Dan and Favre, Benoit. A Scalable Global Model for Summarization. Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing. 2009

    2009

  39. [47]

    A Class of Submodular Functions for Document Summarization

    Lin, Hui and Bilmes, Jeff. A Class of Submodular Functions for Document Summarization. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011

    2011

  40. [48]

    Foundations and Trends in Machine Learning , volume =

    Kulesza, Alex and Taskar, Ben , title =. Foundations and Trends in Machine Learning , volume =. 2012 , month =. doi:10.1561/2200000044 , url =

    2012 doi

  41. [49]

    Lightweight reranking for language model generations

    Jain, Siddhartha and Ma, Xiaofei and Deoras, Anoop and Xiang, Bing. Lightweight reranking for language model generations. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.376

    2024 doi

  42. [50]

    Reranking-based Generation for Unbiased Perspective Summarization

    Ri, Narutatsu and Deas, Nicholas and McKeown, Kathleen. Reranking-based Generation for Unbiased Perspective Summarization. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18653/v1/2025.findings-acl.1268

    2025 doi

  43. [51]

    Submodular-based In-context Example Selection for LLM s-based Machine Translation

    Ji, Baijun and Duan, Xiangyu and Qiu, Zhenyu and Zhang, Tong and Li, Junhui and Yang, Hao and Zhang, Min. Submodular-based In-context Example Selection for LLM s-based Machine Translation. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Lan...

    2024

  44. [52]

    An End-to-End Submodular Framework for Data-Efficient In-Context Learning

    Kumari, Lilly and Wang, Shengjie and Das, Arnav and Zhou, Tianyi and Bilmes, Jeff. An End-to-End Submodular Framework for Data-Efficient In-Context Learning. Findings of the Association for Computational Linguistics: NAACL 2024. 2024. doi:10.18653/v1/2024.findings-naacl.209

    2024 doi

  45. [53]

    SMART : Submodular Data Mixture Strategy for Instruction Tuning

    Renduchintala, H S V N S Kowndinya and Bhatia, Sumit and Ramakrishnan, Ganesh. SMART : Submodular Data Mixture Strategy for Instruction Tuning. Findings of the Association for Computational Linguistics: ACL 2024. 2024. doi:10.18653/v1/2024.findings-acl.766

    2024 doi

  46. [54]

    arXiv preprint arXiv:1904.09675 , year=

    Bertscore: Evaluating text generation with bert , author=. arXiv preprint arXiv:1904.09675 , year=

    1904 arXiv

  47. [55]

    Journal of social computing , volume=

    Segmented summarization and refinement: A pipeline for long-document analysis on social media , author=. Journal of social computing , volume=. 2024 , publisher=

    2024

  48. [56]

    2020 , editor =

    Zhang, Jingqing and Zhao, Yao and Saleh, Mohammad and Liu, Peter , booktitle =. 2020 , editor =

    2020

  49. [57]

    arXiv preprint arXiv:2404.01261 , year=

    Fables: Evaluating faithfulness and content selection in book-length summarization , author=. arXiv preprint arXiv:2404.01261 , year=

    arXiv

  50. [58]

    T ofu E val: Evaluating Hallucinations of LLM s on Topic-Focused Dialogue Summarization

    Tang, Liyan and Shalyminov, Igor and Wong, Amy and Burnsky, Jon and Vincent, Jake and Yang, Yu ' an and Singh, Siffi and Feng, Song and Song, Hwanjun and Su, Hang and Sun, Lijia and Zhang, Yi and Mansour, Saab and McKeown, Kathleen. T ofu E val: Evaluating Hallucinations of LL...

    2024

  51. [59]

    F aith L ens: Detecting and Explaining Faithfulness Hallucination

    Si, Shuzheng and Wang, Qingyi and Zhao, Haozhe and Bai, Yuzhuo and Chen, Guanqiao and Luo, Kangyang and Chen, Gang and Qi, Fanchao and Zhang, Minjia and Chang, Baobao and Sun, Maosong. F aith L ens: Detecting and Explaining Faithfulness Hallucination. Findings of the A ssociat...

    2026 doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.