A Rate-Distortion Framework for Summarization

Aylin Yener; Enes Arda

REVIEW 3 major objections 5 minor 26 references

A Rate-Distortion Framework for Summarization

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes that any summarizer's rate is bounded below by a source-dependent rate-distortion function, giving a fundamental limit on how short summaries can be at a fixed fidelity.

desk verdict A genuinely new rate-distortion formulation for summarization with a likely-true converse, but the proof's independence WLOG is a real, repairable gap and the Gaussian approximation is only heuristic. read the letter →

arxiv 2501.13100 v2 pith:ZEJA4MZR submitted 2025-01-22 cs.IT cs.CLcs.LGmath.IT

Enes Arda , Aylin Yener This is my paper · ORCID

classification cs.ITcs.CLcs.LGmath.IT MSC 94A1594A1794A29

keywords summarizerrate-distortionfunctiontextsummarizationinformationtheoryfundamentallimitsBlahut-Arimotoalgorithmreversewater-fillingGaussianembeddingsCNN/DailyMail

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes a fundamental lower bound on text summarization by importing the logic of rate-distortion theory. It defines a summarizer rate-distortion function $R_S(D)$ as the minimum, over single-document summarizers, of the conditional mutual information between text and summary given the text's length, normalized by the average text length, under an expected-distortion constraint. The main theorem shows that any summarizer processing $n$ documents whose expected summary length is at most $R$ times its input length and whose expected distortion is at most $D$ must have $R \geq R_S(D)$. This makes $R_S(D)$ a source-dependent floor on how short summaries can be at a given fidelity, and therefore a benchmark against which practical summarizers can be measured. The paper also gives a Blahut-Arimoto-style algorithm for computing $R_S(D)$ and a practical approximation based on embedding texts in a Gaussian model.

What carries the argument

The load-bearing object is the summarizer rate-distortion function $R_S(D) = \min_{p_{S|T}} \frac{1}{\bar L} I(T; S | \ell(T))$ subject to $E[d(T,S)] \leq D$. It is a conditional mutual information per average input length: conditioning on $\ell(T)$ removes the length component from the text distribution, so the quantity measures only how much information about the text, beyond its length, the summary must carry. The proof of the lower bound works by a Chebyshev-based typical-set argument: for each length sequence it bounds the probability that the summary length exceeds its budget, bounds the entropy of the summary sequence in the typical and atypical cases, and then chains the resulting per-symbol conditional mutual informations to $R_S(D)$. For computation, a Blahut-Arimoto iteration over each length class solves the minimization in Definition 3, and under a Gaussian embedding model the solution reduces to reverse water-filling over the eigenvalues of the length-conditioned covariance matrices.

What would settle it

Run Algorithm 1 on the four-string source of Example 1 to compute $R_S(D)$ exactly, then brute-force search over all two-document summarizers $p_{S^2|T^2}$ satisfying $E[\ell(S^2)|\ell(T^2)] \leq R\,\ell(T^2)$ with probability 1 and $E[d(T^2,S^2)] \leq D$. If any such summarizer achieves $R < R_S(D)$, Theorem 1 is false; if none does, the bound is verified on the smallest nontrivial block length.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1: for every summarizer $p_{S^n|T^n}$ satisfying $E[\ell(S^n)|\ell(T^n)] \leq R\ell(T^n)$ with probability 1 and $E[d(T^n,S^n)] \leq D$, the rate $R$ satisfies $R \geq R_S(D)$, where $R_S(D) = \min_{p_{S|T}} \frac{1}{\bar L} I(T; S | \ell(T))$ subject to $E[d(T,S)] \leq D$. The conditioning on $\ell(T)$ is deliberate: the rate is the ratio of summary length to input length, and the length of the input is known when the rate is measured, so only the residual information in $T$ beyond its length is charged to the rate. The paper proves the bound by a classical converse that upper-bounds the entropy of the summary sequence given the length sequence, splits the problem across length classes, and then uses convexity and Jensen's inequality to pass from single-document summaries to block summarizers. It also shows $R_S(D)$ is convex and non-increasing, equals zero at and above $D_{\max}$, and is strictly decreasing below $D_{\max}$ when $R_S(0)>0$, so the curve has the qualitative shape of an ordinary rate-distortion function.

Load-bearing premise

The proof of the lower bound assumes that, given the input lengths, the summary lengths of the $n$ documents are conditionally independent, so their fluctuations add only linearly; if a summarizer coordinates its output lengths across documents, the fluctuations can add in a correlated way and the entropy bound no longer follows.

Editorial extensions

If this is right

Any summarizer that maintains a fixed expected distortion $D$ must spend at least $R_S(D)$ bits per unit of input length, so arbitrarily aggressive compression at fixed fidelity is impossible.
Because $R_S(D)$ is non-increasing and convex, there is a continuum of efficient rate-distortion pairs, and the curve reaches zero exactly at the distortion achievable by mapping every text to its best summary per length class.
Algorithm 1 computes $R_S(D)$ exactly for finite sources, giving a way to compare a practical summarizer against the information-theoretic optimum on a toy source.
Algorithm 2 turns the lower bound into a data-driven benchmark: on CNN/DailyMail with BGE-M3 embeddings, the approximated curve lies below the distortion-rate points of BART and PEGASUS summarizers, consistent with the theorem.
The Gaussian approximation diverges as $D \to 0$, while the identity summarizer achieves zero distortion at rate $R=1$, so the practical curve should not be read as a lower bound in the very-low-distortion regime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The framework gives every distortion metric its own fundamental curve; choosing ROUGE, BERTScore, or an LLM-based score as $d$ changes $R_S(D)$, so metric choice becomes a choice about what counts as fidelity.
The theorem's proof assumes conditionally independent output lengths; if a summarizer correlates its output lengths across documents, testing whether such a summarizer can fall below $R_S(D)$ would determine whether the bound is genuinely universal or only valid for memoryless summarizers.
A natural extension is to impose a perception constraint on the summary distribution, turning $R_S(D)$ into a three-way trade-off between rate, distortion, and naturalness.
The same conditional-mutual-information construction should apply to prompt compression when pruning is generalized to arbitrary rewrites, giving a rate-distortion curve for LLM prompting that interpolates between the pruning-only result and full summarization.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take on arXiv:2501.13100. The paper gives a new information-theoretic formulation of summarization: a rate-distortion function where the rate is a length ratio and the mutual information is conditioned on text length. That is a sensible and genuinely new way to think about summarizer limits, and the converse statement (Theorem 1) is likely true. The Blahut-Arimoto style computation and the empirical comparison with three LLM summarizers are nice extras.

The soft spot is in the proof of Theorem 1. Appendix A says 'without loss of generality, we consider conditionally independent ℓ(S_i)' to get the variance bound in Eq. (41). That WLOG is false: a summarizer can use a common random bit to either copy all inputs or empty all inputs, giving perfectly correlated lengths, variance Θ((Σ l_i)^2), and breaking the Chebyshev step. The summarizer still satisfies the rate constraint, so it's inside the theorem's hypotheses. The good news is the gap is repairable: instead of Chebyshev, bound H(S^n|L^n) directly via H(ℓ(S^n)|L^n) + E[ℓ(S^n)|L^n] ≤ log(e(nR\bar{L}+1)) + nR\bar{L} + εn, and the O(log n) slack is absorbed by the εn term. So I think the theorem is true, but the submitted proof has a genuine hole.

The Gaussian section is fine as a heuristic. Theorem 2 is just reverse water-filling for Gaussian sources, and the paper doesn't oversell it. The empirical part is explicitly limited: the Gaussian approximation diverges at low distortion, which the authors note, and they call the curve 'plausible' rather than proven. That is honest.

Overall: the central idea is valuable and the flaws are manageable. The proof needs a real revision before Theorem 1 is accepted as proven. I'd send this to peer review; a good referee can help the authors fix the WLOG step and clarify the status of the Gaussian approximation.

Referee Report

3 major / 5 minor

Summary. The paper introduces a rate-distortion framework for text summarization. It defines the summarizer rate-distortion function R_S(D) as the minimum, over one-shot summarizers p_{S|T}, of (1/E[ℓ(T)]) I(T; S | ℓ(T)) subject to E[d(T,S)] ≤ D. Theorem 1 claims that any block summarizer p_{S^n|T^n} whose expected output length satisfies E[ℓ(S^n)|ℓ(T^n)] ≤ R ℓ(T^n) must have R ≥ R_S(D) whenever its expected distortion is at most D. The paper also proposes a Blahut-Arimoto-type algorithm for computing R_S(D) and, for real text data, a practical approximation based on embeddings and a Gaussian model, which is compared against several summarizers on CNN/DailyMail.

Significance. If Theorem 1 is proven, the paper establishes a clean information-theoretic lower bound for summarization: every summarizer's compression ratio is bounded below by a source-dependent rate-distortion curve, analogous to classical rate-distortion theory. The convexity lemmas, the Blahut-Arimoto algorithm, and the reverse water-filling result are natural and potentially useful tools. The paper is also transparent about the practical limitations of the empirical method, listing the Gaussian assumption, the embedding model, and the sample size as open concerns in Appendix C.G. The empirical curve is computed from source statistics rather than fitted to the summarizer points, which mitigates circularity concerns. However, both the main converse proof and the Gaussian equality claim currently contain gaps, so the significance is conditional on repair.

major comments (3)

[Appendix A, Eq. (41)] The step 'without loss of generality, we consider conditionally independent ℓ(S_i)' is false for arbitrary summarizers. A valid summarizer can draw one common random bit U: if U=0 it copies every input text and if U=1 it outputs the empty string for every input. This summarizer satisfies E[ℓ(S^n)|ℓ(T^n)] = ℓ(T^n)/2, so it is inside the hypotheses of Theorem 1 with R=1/2, but the output lengths are perfectly correlated given ℓ(T^n), making Var[ℓ(S^n)|ℓ(T^n)] = Θ((Σ_i ℓ(T_i))²) and invalidating the Chebyshev bound in Eq. (42). This is a genuine hole in the proof as written. The theorem statement is likely correct: using only the length constraint, one can bound H(ℓ(S^n)|L^n) = O(log L^n) and absorb this slack into the εn term in the converse, so the proof is repairable. The submitted Appendix A nevertheless needs to be revised before Theorem 1 can be accepted as proven.
[Theorem 2 and Appendix B] The proof establishes at most a lower bound, not the claimed equality. For any summarizer, the data processing inequality gives I(T;S|L) ≥ I(E_T;E_S|L), where E_T and E_S are the embeddings of T and S, and the reverse water-filling calculation lower-bounds I(E_T;E_S|L). However, achievability is shown only for the embedding test channel p_{E_S|E_T,L}; the paper does not construct a conditional distribution p_{S|T} over text summaries whose embeddings realize the optimal Gaussian channel, nor does it bound the gap I(T;S|L) − I(E_T;E_S|L). Therefore Eq. (68) is not established as the summarizer rate-distortion function. The theorem should be restated as a lower bound under the Gaussian assumption, or an achievability argument over the text alphabet S must be supplied.
[Section V and Figure 2] The empirical claim that the approximated curve is a 'plausible lower bound' is not fully supported. First, the lower-bound property depends on the Gaussian conditional model, which the authors themselves list as an open question in Appendix C.G. Second, the computed curve diverges to infinity at zero distortion, whereas the identity summarizer achieves D=0 at R=1, so the approximation is not a valid lower bound in the low-distortion regime. The text acknowledges this ('becomes less accurate at low distortion'), but the paper should state the range of D over which the lower-bound claim is intended and preferably quantify the effect of the Gaussian assumption, for example by comparing against a non-Gaussian baseline or by reporting the fitted Gaussian's goodness of fit.

minor comments (5)

[Definition 1] The rate R is defined as the 'smallest real number' satisfying Eq. (1); since the feasible set may be open, the definition should use an infimum rather than a minimum.
[Definition 4] The argmin in Eq. (18) may not exist when S is infinite; the paper should state finiteness or compactness assumptions, or otherwise handle the supremum/infimum explicitly.
[Appendix C.C] There is a typo in the phrase 'summarizer rate-distrtion curve'; it should read 'rate-distortion curve'.
[Appendix C.F and Figure 2] The intended rates deviate from the actual rates, but Figure 2 plots only the points; reporting the actual rate values in a table or in the figure would improve reproducibility.
[Appendix C.E] The claim that principal components are in the same direction for every length interval is not verified directly; Figure 5 compares the resulting curves, but a direct comparison of the eigenvectors would be more convincing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 1 is a genuine information-theoretic converse and the empirical curve is not fitted to the summarizer points.

full rationale

The paper's central lower bound, Theorem 1, is derived as a standard information-theoretic converse. The summarizer rate R is defined as an almost-sure conditional length ratio (Definition 1), while R_S(D) is defined as a length-normalized conditional mutual information minimization (Definition 3). These are distinct quantities, and the proof of Theorem 1 uses entropy and mutual-information inequalities plus the definition of R_S(D) as a constrained minimum; it does not assume the conclusion. No fitted parameter is renamed as a prediction: the practical curve in Algorithm 2 is computed from source embedding covariances, not from the summarizer points, and the comparison with summarizers is a sanity check under an explicitly stated Gaussian and squared-Euclidean modeling assumption. The cited works are external (e.g., [5] for Gaussian embedding modeling, [13] for rate-distortion water-filling), and no load-bearing argument reduces to a self-citation. The empirical validation has a self-referential flavor because the same embedding model and distortion metric define both the curve and the summarizer operating points, but this is a modeling/validation weakness, not a circular derivation: the lower-bound inequality is not forced by fitting to the summarizer data. Appendix C explicitly lists the Gaussianity and covariance-estimation reliability as open questions, and Appendix A contains a genuine proof gap in the 'without loss of generality, we consider conditionally independent ℓ(S_i)' assertion; however, a proof gap is a correctness issue, not a circular reduction, and the theorem's statement is not equivalent to its inputs by construction. Therefore no significant circularity is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The theoretical lower bound requires few assumptions beyond i.i.d. source and finite third moment. The practical approximation adds strong unverified assumptions: Gaussianity of embeddings, validity of squared-distance distortion, and a transfer of the bound from embedding space to string space. The main data-driven parameters are the sample covariance eigenvalues and the length grid.

free parameters (4)

Length grid endpoints = {71, 588, 866, 1233, 3419}
Chosen so each interval has at least 2,000 texts, affecting the covariance estimates and the computed curve.
Minimum samples per length interval = 2000
Selected to obtain stable sample covariance estimates for 1024-dimensional embeddings.
Sample covariance eigenvalues = Estimated from CNN/DailyMail test split
The reverse water-filling curve is computed from these eigenvalues, so the curve is data-dependent.
Embedding model and pooling = BGE-M3 with CLS pooling, no normalization
Model choice determines the embedding space; mean pooling was rejected because it underestimated variance.

assumptions (4)

domain assumption Text embeddings follow a multivariate Gaussian distribution conditional on text length.
Invoked in Section V and Theorem 2 to derive the closed-form reverse water-filling rate-distortion function.
domain assumption Squared Euclidean distance between text and summary embeddings is a valid distortion function for summarization.
The entire rate-distortion bound and all empirical comparisons use this distortion measure.
domain assumption The rate-distortion function computed in the embedding space lower-bounds the length-ratio rate of discrete-text summarizers.
No theorem connects the Gaussian embedding result to Definition 1's length-ratio rate; this is assumed in the empirical section.
standard math Text strings are generated i.i.d. from a source with finite third moment of length.
Used to prove the converse in Theorem 1 and to justify the entropy bounds.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Rate-Distortion Framework for Summarization." pith.science (2026). https://pith.science/paper/ZEJA4MZR

@misc{pith2026250113100,
  author       = {Pith},
  title        = {Pith review of: A Rate-Distortion Framework for Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZEJA4MZR}},
  note         = {Machine review of arXiv:2501.13100}
}

read the original abstract

This paper introduces an information-theoretic framework for text summarization. We define the summarizer rate-distortion function and show that it provides a fundamental lower bound on summarizer performance. We describe an iterative procedure, similar to Blahut-Arimoto algorithm, for computing this function. To handle real-world text datasets, we also propose a practical method that can calculate the summarizer rate-distortion function with limited data. Finally, we empirically confirm our theoretical results by comparing the summarizer rate-distortion function with the performances of different summarizers used in practice.

Figures

Figures reproduced from arXiv: 2501.13100 by the authors.

**Figure 2.** The summarizer rate-distortion curve, computed via Algorithm 2 for [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** The distribution of text lengths in the CNN/DailyMail test split, [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Comparison of the sample covariance eigenvalues for each interval. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 5.** Figure 5: Comparison of summarizer rate-distortion curves computed with [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 20 canonical work pages

[1]

Abstractive summarization: An overview of the state of the art,

S. Gupta and S. K. Gupta, “Abstractive summarization: An overview of the state of the art,” Expert Systems with Applications , vol. 121, pp. 49–65, 2019. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0957417418307735

work page 2019
[2]

Summeval: Re-evaluating summarization evaluation,

A. R. Fabbri, W. Kry ´sci´nski, B. McCann, C. Xiong, R. Socher, and D. Radev, “Summeval: Re-evaluating summarization evaluation,” Transactions of the Association for Computational Linguistics , vol. 9, pp. 391–409, 04 2021. [Online]. Available: https://doi.org/10.1162/tacl_ a_00373

doi:10.1162/tacl_ 2021
[3]

Re- evaluating evaluation in text summarization,

M. Bhandari, P. N. Gour, A. Ashfaq, P. Liu, and G. Neubig, “Re- evaluating evaluation in text summarization,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), B. Webber, T. Cohn, Y . He, and Y . Liu, Eds. Online: Association for Computational Linguistics, Nov. 2020, pp. 9347–9359. [Online]. Available: http...

work page 2020
[4]

COSMIC: Mutual information for task-agnostic summarization evaluation,

M. Darrin, P. Formont, J. Cheung, and P. Piantanida, “COSMIC: Mutual information for task-agnostic summarization evaluation,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , L.-W. Ku, A. Martins, and V . Srikumar, Eds. Bangkok, Thailand: Association for Computational Linguistics, Aug. 20...

work page 2024
[5]

Mutual information divergence: A unified metric for multimodal generative models,

J.-H. Kim, Y . Kim, J. Lee, K. M. Yoo, and S.-W. Lee, “Mutual information divergence: A unified metric for multimodal generative models,” in Advances in Neural Information Processing Systems , A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022. [Online]. Available: https://openreview.net/forum?id=wKd2XtSRsjl

work page 2022
[6]

Unsupervised extractive summarization using pointwise mutual information,

V . Padmakumar and H. He, “Unsupervised extractive summarization using pointwise mutual information,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , P. Merlo, J. Tiedemann, and R. Tsarfaty, Eds. Online: Association for Computational Linguistics, Apr. 2021, pp. 2505–2512. [Onlin...

work page 2021
[7]

Mutual information alleviates hallucinations in abstractive summarization,

L. van der Poel, R. Cotterell, and C. Meister, “Mutual information alleviates hallucinations in abstractive summarization,” ArXiv, vol. abs/2210.13210, 2022. [Online]. Available: https://api.semanticscholar. org/CorpusID:253097869

arXiv 2022
[8]

Coding theorems for a discrete source with a fidelity criterion,

C. E. Shannon, “Coding theorems for a discrete source with a fidelity criterion,” IRE Nat. Conv. Rec , vol. 4, no. 142-163, p. 1, 1959

work page 1959

Show all 26 references

[9]

Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression, ser

T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression, ser. Prentice-Hall electrical engineering series. Prentice- Hall, 1971. [Online]. Available: https://books.google.com/books?id= -HV1QgAACAAJ

1971
[10]

LLMLingua: Compressing prompts for accelerated inference of large language models,

H. Jiang, Q. Wu, C.-Y . Lin, Y . Yang, and L. Qiu, “LLMLingua: Compressing prompts for accelerated inference of large language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore:...

2023
[11]

Fundamental limits of prompt compression: A rate-distortion framework for black-box language models,

A. Nagle, A. Girish, M. Bondaschi, M. Gastpar, A. V . Makkuva, and H. Kim, “Fundamental limits of prompt compression: A rate-distortion framework for black-box language models,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024. [Online]. A...

2024
[12]

R. W. Yeung, Information Theory and Network Coding, 1st ed. Springer Publishing Company, Incorporated, 2008

2008
[13]

T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) . USA: Wiley- Interscience, 2006

2006
[14]

Computation of channel capacity and rate-distortion func- tions,

R. Blahut, “Computation of channel capacity and rate-distortion func- tions,” IEEE Transactions on Information Theory , vol. 18, no. 4, pp. 460–473, 1972

1972
[15]

An algorithm for computing the capacity of arbitrary dis- crete memoryless channels,

S. Arimoto, “An algorithm for computing the capacity of arbitrary dis- crete memoryless channels,” IEEE Transactions on Information Theory , vol. 18, no. 1, pp. 14–20, 1972

1972
[16]

Teaching machines to read and comprehend,

K. M. Hermann, T. Kocisky, E. Grefenstette, L. Espeholt, W. Kay, M. Suleyman, and P. Blunsom, “Teaching machines to read and comprehend,” in Advances in Neural Information Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, Eds., vol. 28. Curran As...

2015
[17]

Get to the point: Summarization with pointer-generator networks,

A. See, P. J. Liu, and C. D. Manning, “Get to the point: Summarization with pointer-generator networks,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), R. Barzilay and M.-Y . Kan, Eds. Vancouver, Canada: Asso...

2017
[18]

ROUGE: A package for automatic evaluation of summaries,

C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013/

2004
[19]

Bertscore: Evaluating text generation with bert,

T. Zhang*, V . Kishore*, F. Wu*, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=SkeHuCVFDr

2020
[20]

Bartscore: Evaluating generated text as text generation,

W. Yuan, G. Neubig, and P. Liu, “Bartscore: Evaluating generated text as text generation,” in Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 27 263– 27 2...

2021
[21]

FineSurE: Fine-grained summarization evaluation using LLMs,

H. Song, H. Su, I. Shalyminov, J. Cai, and S. Mansour, “FineSurE: Fine-grained summarization evaluation using LLMs,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , L.-W. Ku, A. Martins, and V . Srikumar, Eds...

2024
[22]

BERT: Pre- training of deep bidirectional transformers for language understanding,

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota: ...

2019
[23]

Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation,

J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu, “Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation,” 2024. [Online]. Available: https://arxiv.org/abs/2402.03216 APPENDIX A APPENDIX : PROOF OF TH...

2024 arXiv
[24]

How accurately can the chosen language model capture the dataset’s semantics in the embedding space?
[25]

To what extent do the embeddings follow a Gaussian distribution?
[26]

How reliably can the sample covariance matrices be estimated from the dataset? Fig. 5. Comparison of summarizer rate-distortion curves computed with Algorithm 2 using different number of length intervals

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Abstractive summarization: An overview of the state of the art,

S. Gupta and S. K. Gupta, “Abstractive summarization: An overview of the state of the art,” Expert Systems with Applications , vol. 121, pp. 49–65, 2019. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0957417418307735

work page 2019

[2] [2]

Summeval: Re-evaluating summarization evaluation,

A. R. Fabbri, W. Kry ´sci´nski, B. McCann, C. Xiong, R. Socher, and D. Radev, “Summeval: Re-evaluating summarization evaluation,” Transactions of the Association for Computational Linguistics , vol. 9, pp. 391–409, 04 2021. [Online]. Available: https://doi.org/10.1162/tacl_ a_00373

doi:10.1162/tacl_ 2021

[3] [3]

Re- evaluating evaluation in text summarization,

M. Bhandari, P. N. Gour, A. Ashfaq, P. Liu, and G. Neubig, “Re- evaluating evaluation in text summarization,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), B. Webber, T. Cohn, Y . He, and Y . Liu, Eds. Online: Association for Computational Linguistics, Nov. 2020, pp. 9347–9359. [Online]. Available: http...

work page 2020

[4] [4]

COSMIC: Mutual information for task-agnostic summarization evaluation,

M. Darrin, P. Formont, J. Cheung, and P. Piantanida, “COSMIC: Mutual information for task-agnostic summarization evaluation,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , L.-W. Ku, A. Martins, and V . Srikumar, Eds. Bangkok, Thailand: Association for Computational Linguistics, Aug. 20...

work page 2024

[5] [5]

Mutual information divergence: A unified metric for multimodal generative models,

J.-H. Kim, Y . Kim, J. Lee, K. M. Yoo, and S.-W. Lee, “Mutual information divergence: A unified metric for multimodal generative models,” in Advances in Neural Information Processing Systems , A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022. [Online]. Available: https://openreview.net/forum?id=wKd2XtSRsjl

work page 2022

[6] [6]

Unsupervised extractive summarization using pointwise mutual information,

V . Padmakumar and H. He, “Unsupervised extractive summarization using pointwise mutual information,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , P. Merlo, J. Tiedemann, and R. Tsarfaty, Eds. Online: Association for Computational Linguistics, Apr. 2021, pp. 2505–2512. [Onlin...

work page 2021

[7] [7]

Mutual information alleviates hallucinations in abstractive summarization,

L. van der Poel, R. Cotterell, and C. Meister, “Mutual information alleviates hallucinations in abstractive summarization,” ArXiv, vol. abs/2210.13210, 2022. [Online]. Available: https://api.semanticscholar. org/CorpusID:253097869

arXiv 2022

[8] [8]

Coding theorems for a discrete source with a fidelity criterion,

C. E. Shannon, “Coding theorems for a discrete source with a fidelity criterion,” IRE Nat. Conv. Rec , vol. 4, no. 142-163, p. 1, 1959

work page 1959