Pith. sign in

REVIEW 4 major objections 5 minor 40 references

RELexED: Retrieval-Enhanced Legal Summarization with Exemplar Diversity

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A retrieval-enhanced framework for legal summarization that selects exemplar summaries by balancing relevance and diversity with a determinantal point process, scored by influence functions, reports consistent gains on two legal datasets.

desk verdict A plausible two-stage exemplar-selection recipe for legal summarization, with a headline significance claim that the reported statistics do not actually back. read the letter →

arxiv 2501.14113 v1 pith:VYOWERKQ submitted 2025-01-23 cs.CL

T.Y.S.S. Santosh , Chen Jia , Patrick Goroncy , Matthias Grabmair This is my paper · ORCID
classification cs.CL
keywords legalsummarizationretrieval-augmentedgenerationdeterminantalpointprocessinfluencefunctionsexemplardiversityabstractiveNLP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that legal summarization improves when the summarizer is given not just the source opinion but also a small set of reference summaries from the training corpus, and that the way those reference summaries are chosen matters. Its proposed framework, RELexED, first uses BM25 lexical retrieval to cut candidate examples, then applies a determinantal point process to pick a final set that balances each example's relevance to the query against the set's overall diversity. The relevance and similarity scores that feed the selection are computed from gradient-based influence functions on an auxiliary model rather than from lexical overlap. On SuperSCOTUS and CivilSum, the authors report that this two-stage selection outperforms the no-exemplar baseline and a similarity-only BM25 selector, with the largest gains in coherence, fluency, and factual-consistency metrics.

What carries the argument

The central object is the DPP kernel $L_{ij}=q_i s_{ij} q_j$ with quality score $q_i$ and similarity score $s_{ij}$. RELexED estimates both from influence functions: $q_i$ is the TracIn gradient dot product of example $i$ on the query, and $s_{ij}$ is the gradient dot product between examples $i$ and $j$, taken at the first encoder layer of an auxiliary summarization model trained without exemplars. The determinant of the selected subset's kernel submatrix measures squared volume, so greedy MAP inference selects examples that are individually relevant and mutually dissimilar.

What would settle it

An ablation that substitutes randomly drawn scores for the influence scores inside the same DPP selection would determine whether the influence-function component is load-bearing: if the random-scores variant still beats BM25-based DPP, the claimed mechanism is falsified. A second check is to compute influence scores from the full encoder rather than the first layer; if results do not change, the first-layer proxy is not responsible for the gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that exemplar diversity, not just exemplar similarity to the query, is what drives retrieval-enhanced gains in legal summarization, and that gradient-derived influence scores are a better input to a diversity-aware selector than lexical BM25 scores. Concretely, the paper reports that the RELexED configuration (BM25 candidates plus DPP with influence-function quality and similarity) outperforms both a model trained without exemplars and the same DPP pipeline using BM25 scores on both SuperSCOTUS and CivilSum across ROUGE-1/2/L, BERTScore, AlignScore, coherence, and fluency. The paper interprets these results as evidence that highly similar exemplars are redundant, while a diverse set selected with a quality-diversity trade-off supplies broader information and better writing style guidance.

Load-bearing premise

The method depends on the assumption that the overlap between two examples' internal learning signals is a reliable measure of both how relevant an example is to the query and how similar two examples are; the paper does not test this assumption.

Editorial extensions

If this is right

  • Retrieval-enhanced legal summarizers can get large style and faithfulness gains from four to eight exemplar summaries, without increasing model size.
  • A diversity-aware selector beats a similarity-only selector, so future systems should treat redundancy among retrieved examples as a cost.
  • Gradient-based similarity estimates can be used inside standard DPP selection for supervised summarization.
  • Lexical BM25 retrieval is still useful as a first-stage filter to keep the influence-score computation tractable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same two-stage selection could be tested in other highly templated summarization domains, such as patents, medical notes, or financial filings, where writing style carries as much information as content.
  • A cleaner test of the paper's mechanism would be to compare influence-function scores against a cheap embedding-diversity baseline inside the same DPP; if the embedding baseline matches RELexED, the contribution would reduce to DPP diversity selection.
  • Because the paper's limitation section acknowledges no expert legal validation, an evaluation where lawyers judge whether the diverse exemplars actually produce more useful summaries would be the natural next step, and it is not yet in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces RELexED, a retrieval-augmented framework for legal summarization. Given a query document, the method first retrieves candidate exemplar summaries using BM25 and then applies a determinantal point process (DPP) to choose a final set of k exemplars, where the quality and similarity scores in the DPP kernel are computed from TracIn influence functions. The selected exemplars are concatenated with the source document and fed into a Longformer encoder-decoder. Experiments on SuperSCOTUS and CivilSum compare four conditions: no exemplars, BM25-only retrieval, BM25 plus DPP with BM25 scores, and BM25 plus DPP with influence-function scores. The paper reports ROUGE, BERTScore, AlignScore, coherence, and fluency, along with a descriptive exemplar diversity analysis and a case study. The central claim is that RELexED significantly outperforms both no-exemplar models and models relying solely on similarity-based exemplar selection.

Significance. If fully supported, the contribution would be practically valuable: it shows that diversity-aware exemplar selection with influence-based relevance scores can improve legal summarization across lexical, semantic, factual-consistency, and style metrics on two public datasets. The method itself is clearly described, the evaluation uses standard metrics and held-out test splits, and the limitations section is candid about cross-jurisdiction generality, lack of expert evaluation, and metric limitations. However, the paper's headline claim is not currently backed by the statistical evidence it reports: significance testing is performed only against the no-exemplar baseline, while the central comparison to similarity-based selection is left untested. Because the margin over similarity-based selection is small or negative on several CivilSum metrics, the main empirical claim needs additional evidence before the result can be considered established.

major comments (4)
  1. [Abstract and Table 1] The abstract states that RELexED 'significantly outperforms models that do not utilize exemplars and those that rely solely on similarity-based exemplar selection,' but Table 1's caption and Section 3.2 report a Wilcoxon signed-rank test only against the w/o-exemplars baseline. No significance test is reported between BM25+DPP(IF) and either BM25 or BM25+DPP(BM25). This matters because the novel component is the influence-function-based DPP selection, and the raw differences are small on several metrics: on CivilSum, BM25+DPP(IF) has lower BERTScore (59.68 vs 59.82) and lower fluency (77.75 vs 78.12) than BM25+DPP(BM25). With no error bars, standard deviations, or repeated-seed comparisons, the claim of significant gains over similarity-based selection is not supported by the evidence as reported. Please add paired comparisons with an appropriate test, report variance across seeds, and adjust the abstract and conclusions if those differences are not significant.
  2. [Section 2.2] The method's core assumption is that TracIn gradient dot products computed on the first encoder layer of an auxiliary model fine-tuned without exemplars measure both query relevance (qi) and redundancy (sij) among legal exemplars. The paper follows Thakkar et al. (2023) without any experiment validating this proxy in the legal summarization setting. No ablation varies the encoder layer, compares TracIn-based scores against BM25 or embedding-based scores within the same DPP pipeline, or checks whether the selected exemplars correlate with downstream summary quality. If this assumption fails, the DPP kernel becomes a noise source and the apparent gains in Table 1 are not attributable to the proposed mechanism. Please add an ablation or at least a diagnostic correlation between influence scores and relevance/diversity judgments.
  3. [Appendix B and Section 3.1] Several selection hyperparameters are fixed without sensitivity analysis: the first-stage pool size k1=40, the number of exemplars k=4 for SuperSCOTUS and k=8 for CivilSum, and the choice to fill the encoder budget with 4 or 8 exemplars. The paper reports only one configuration per dataset. Because the proposed method has more moving parts than the BM25 baseline, a sensitivity study (e.g., varying k1, varying k, and possibly varying the TracIn layer) is needed to show that the gains are robust rather than tied to a particular tuning. Please also state how k and k1 were chosen.
  4. [Table 2 and Section 3.2] The diversity analysis in Table 2 reports average cosine similarities (EQ and IE) but provides no statistical test and no direct quantitative link to downstream summary quality. Both EQ and IE decrease when moving from BM25 to the DPP-based methods, so the table alone does not establish that reduced inter-exemplar similarity is the cause of improved summaries; it could simply reflect lower relevance. The paper argues that diverse exemplars improve performance, but the connection between these descriptive statistics and the ROUGE/BERTScore/AlignScore improvements is made only qualitatively. Please report paired tests or correlations between IE/EQ and summary metrics, and discuss the relevance-diversity trade-off quantitatively.
minor comments (5)
  1. [Section 3.1] There are typos in the metrics paragraph: 'beetween' should be 'between' and 'referecne' should be 'reference'; please proofread the manuscript.
  2. [Equation (1)] Equation (1) is typeset confusingly: the denominator should make clear that det(L+I) is the normalization constant, and the displayed expression 'P k det(Lk) =det(L + I)' is not a standard equality. Please rewrite the equation so that the DPP probability is stated correctly.
  3. [Throughout] The method name is spelled inconsistently as RELexED, ReLexED, and RElexED; please standardize to a single spelling.
  4. [Appendix B] Appendix B reports 10 training epochs with early stopping and mixed precision, but does not report the number of runs, initialization seeds, or standard deviations. Reporting seeds and multiple runs would support the variance estimates requested above and improve reproducibility.
  5. [Table 1 and Appendix B] The Wilcoxon signed-rank test is described only in the Table 1 caption. Please state in Section 3.2 or Appendix B exactly what is paired (e.g., individual test documents) and whether any correction for multiple metrics or multiple comparisons was applied.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RELexED's selection and evaluation are self-contained on external benchmarks; the unsupported significance claim is a statistical-support gap, not a circular chain.

full rationale

The derivation chain is self-contained. Section 2.2 defines the DPP quality and similarity scores as TracIn gradient dot products (qi = influence of item i on query x; sij = influence of item i on item j) from an auxiliary model trained without exemplars, and the final summarization model is trained and evaluated on held-out test splits of SuperSCOTUS and CivilSum. No equation is defined in terms of a fitted parameter that is later reported as a prediction, and no result is forced by construction. The only self-citations (Santosh et al. 2024a-d; Tyss et al. 2024) appear in motivation and future-work framing and are not load-bearing. The abstract's claim of significant gains over similarity-only selection is not backed by the reported Wilcoxon test, since Table 1's caption restricts significance to the w/o-exemplars baseline; this is a statistical-support weakness, not a circularity. The acknowledged limitations (no legal-expert evaluation, no temporal/multi-aspect features) are external-validity caveats. No circular step is present.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method introduces no new theoretical entities. It rests on established DPP and TracIn machinery, two hand-chosen hyperparameters (k1=40, k=4/8), and an unvalidated proxy assumption that first-layer gradient inner products encode legal-summary relevance and redundancy. No sensitivity analysis, error bars, or code are provided, so the ledger cannot be further verified.

free parameters (3)
  • first-stage candidate pool size k1 = 40
    Appendix B: 'We set to filter out top 40 exemplars from first stage selection.' Hand-chosen; no sensitivity analysis reported.
  • final exemplar count k = 4 (SuperSCOTUS), 8 (CivilSum)
    Section 3.1: 'We use 4 and 8 exemplars for SuperSCOTUS and CivilSum respectively.' Dictated by the 16384-token encoder budget rather than tuned.
  • TracIn encoder layer = first layer only
    Section 2.2: 'use only first layer of the encoder to compute the influence score.' Adopted from Thakkar et al. (2023) without validation for this task.
assumptions (5)
  • standard math The determinant of the DPP kernel submatrix measures joint quality and diversity of a selected set (standard DPP decomposition L_ij = q_i * s_ij * q_j).
    Invoked in Appendix A to justify the two-stage selection; based on Kulesza et al. (2012).
  • domain assumption Reference summaries in the training corpus are useful exemplars for guiding style and content in legal summarization.
    Introduction: 'one scalable way is to use reference summaries in the training corpus as exemplars' (citing An et al. 2021; Wang et al. 2022).
  • ad hoc to paper Gradient dot products (TracIn) between samples, computed on the first encoder layer, capture semantic relevance and redundancy of legal examples.
    Section 2.2: 'Gradients offer a more nuanced notion of similarity, as similar examples often result in nearly identical gradients.' No empirical validation is provided for legal text.
  • standard math Greedy DPP MAP inference (Chen et al. 2018) is a sufficient approximation to the NP-hard optimal diverse selection.
    Section 2.1: 'MAP inference for DPP involves sub-modular maximization, which is NP-hard. Therefore we use greedy algorithm for faster inference.'
  • ad hoc to paper Auxiliary model gradients remain informative when computed on a model fine-tuned without exemplars, even though the summarizer is later fine-tuned with exemplars.
    Section 2.2: the auxiliary model 'fine-tuned to generate a summary directly from the input alone without exemplars' provides all score computations; the train/test setup for this auxiliary model is matched to the exemplar-free baseline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RELexED: Retrieval-Enhanced Legal Summarization with Exemplar Diversity." pith.science (2026). https://pith.science/paper/VYOWERKQ

@misc{pith2026250114113,
  author       = {Pith},
  title        = {Pith review of: RELexED: Retrieval-Enhanced Legal Summarization with Exemplar Diversity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VYOWERKQ}},
  note         = {Machine review of arXiv:2501.14113}
}
read the original abstract

This paper addresses the task of legal summarization, which involves distilling complex legal documents into concise, coherent summaries. Current approaches often struggle with content theme deviation and inconsistent writing styles due to their reliance solely on source documents. We propose RELexED, a retrieval-augmented framework that utilizes exemplar summaries along with the source document to guide the model. RELexED employs a two-stage exemplar selection strategy, leveraging a determinantal point process to balance the trade-off between similarity of exemplars to the query and diversity among exemplars, with scores computed via influence functions. Experimental results on two legal summarization datasets demonstrate that RELexED significantly outperforms models that do not utilize exemplars and those that rely solely on similarity-based exemplar selection.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 20 canonical work pages

  1. [1]

    Chenxin An, Ming Zhong, Zhichao Geng, Jianqiang Yang, and Xipeng Qiu. 2021. Retrievalsum: A retrieval enhanced framework for abstractive summarization. arXiv preprint arXiv:2109.07943

    arXiv 2021

  2. [2]

    Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150

    arXiv 2020

  3. [3]

    Paheli Bhattacharya, Kaustubh Hiware, Subham Rajgaria, Nilay Pochhi, Kripabandhu Ghosh, and Saptarshi Ghosh. 2019. A comparative study of summarization algorithms applied to legal case judgments. In Advances in Information Retrieval: 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14--18, 2019, Proceedings, Part I 41, pages 413...

    work page 2019

  4. [4]

    Paheli Bhattacharya, Soham Poddar, Koustav Rudra, Kripabandhu Ghosh, and Saptarshi Ghosh. 2021. Incorporating domain knowledge for extractive summarization of legal case documents. In Proceedings of the eighteenth international conference on artificial intelligence and law, pages 22--31

    work page 2021

  5. [5]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165

    arXiv 2020

  6. [6]

    Ziqiang Cao, Wenjie Li, Sujian Li, and Furu Wei. 2018. Retrieve, rerank and rewrite: Soft template based neural summarization. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 152--161

    work page 2018

  7. [7]

    Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. Legal-bert: The muppets straight out of law school. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2898--2904

    work page 2020

  8. [8]

    Laming Chen, Guoxin Zhang, and Eric Zhou. 2018. Fast greedy map inference for determinantal point process to improve recommendation diversity. Advances in Neural Information Processing Systems, 31

    2018

Show all 40 references
  1. [9]

    Aniket Deroy, Kripabandhu Ghosh, and Saptarshi Ghosh. 2023. How ready are pre-trained abstractive models and llms for legal case judgement summarization? arXiv preprint arXiv:2306.01248

    2023 arXiv

  2. [10]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234

    2022 arXiv

  3. [11]

    Mohamed Elaraby and Diane Litman. 2022. Arglegalsumm: Improving abstractive summarization of legal documents with argument mining. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6187--6194

    2022

  4. [12]

    Biaoyan Fang, Trevor Cohn, Timothy Baldwin, and Lea Frermann. 2023. Super-scotus: A multi-sourced dataset for the supreme court of the us. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 202--214

    2023

  5. [13]

    Atefeh Farzindar. 2004. Atefeh farzindar and guy lapalme,'letsum, an automatic legal text summarizing system in t. gordon (ed.), legal knowledge and information systems. jurix 2004: The seventeenth annual conference. amsterdam: Ios press, 2004, pp. 11-18. In Legal knowledge an...

    2004

  6. [14]

    Shen Gao, Xiuying Chen, Piji Li, Zhangming Chan, Dongyan Zhao, and Rui Yan. 2019. How to write summaries with patterns? learning towards abstractive summarization through prototype editing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processi...

    2019

  7. [15]

    Claire Grover, Ben Hachey, Ian Hughson, and Chris Korycinski. 2003 a . Automatic summarisation of legal documents. In Proceedings of the 9th international conference on Artificial intelligence and law, pages 243--251

    2003

  8. [16]

    Claire Grover, Ben Hachey, and Chris Korycinski. 2003 b . Summarising legal texts: Sentential tense and argumentative roles. In Proceedings of the HLT-NAACL 03 Text Summarization Workshop, pages 33--40

    2003

  9. [17]

    Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  10. [18]

    Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885--1894. PMLR

    2017

  11. [19]

    Alex Kulesza, Ben Taskar, et al. 2012. Determinantal point processes for machine learning. Foundations and Trends in Machine Learning , 5(2--3):123--286

    2012

  12. [20]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  13. [21]

    Manuj Malik, Zheng Zhao, Marcio Fonseca, Shrisha Rao, and Shay B Cohen. 2024. Civilsum: A dataset for abstractive summarization of indian court decisions. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages ...

    2024

  14. [22]

    Gianluca Moro and Luca Ragazzi. 2022. Semantic self-segmentation for abstractive summarization of long documents in low-resource regimes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11085--11093

    2022

  15. [23]

    Tatsuro Oya, Yashar Mehdad, Giuseppe Carenini, and Raymond Ng. 2014. A template-based abstractive meeting summarization: Leveraging summary and source text relationships. In Proceedings of the 8th International Natural Language Generation Conference (INLG), pages 45--53

    2014

  16. [24]

    Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. 2020. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33:19920--19930

    2020

  17. [25]

    TYS Santosh, Mahmoud Aly, and Matthias Grabmair. 2024 a . Lexabsumm: Aspect-based summarization of legal decisions. arXiv preprint arXiv:2404.00594

    2024 arXiv

  18. [26]

    TYS Santosh, Vatsal Venkatkrishna, Saptarshi Ghosh, and Matthias Grabmair. 2024 b . Beyond borders: Investigating cross-jurisdiction transfer in legal case summarization. arXiv preprint arXiv:2403.19317

    2024 arXiv

  19. [27]

    TYS Santosh, Tuan-Quang Vuong, and Matthias Grabmair. 2024 c . Chronoslex: Time-aware incremental training for temporal generalization of legal classification tasks. arXiv preprint arXiv:2405.14211

    2024 arXiv

  20. [28]

    TYSS Santosh, Cornelius Weiss, and Matthias Grabmair. 2024 d . Lexsumm and lext5: Benchmarking and modeling legal summarization tasks in english. arXiv preprint arXiv:2410.09527

    2024 arXiv

  21. [29]

    Murali Saravanan, Balaraman Ravindran, and Shivani Raman. 2006. Improving legal document summarization using graphical models. Frontiers in Artificial Intelligence and Applications, 152:51

    2006

  22. [30]

    Zejiang Shen, Kyle Lo, Lauren Yu, Nathan Dahlberg, Margo Schlanger, and Doug Downey. 2022. Multi-lexsum: Real-world summaries of civil rights lawsuits at multiple granularities. Advances in Neural Information Processing Systems, 35:13158--13173

    2022

  23. [31]

    Abhay Shukla, Paheli Bhattacharya, Soham Poddar, Rajdeep Mukherjee, Kripabandhu Ghosh, Pawan Goyal, and Saptarshi Ghosh. 2022. Legal case document summarization: Extractive and abstractive methods and their evaluation. In Proceedings of the 2nd Conference of the Asia-Pacific C...

    2022

  24. [32]

    Megh Thakkar, Tolga Bolukbasi, Sriram Ganapathy, Shikhar Vashishth, Sarath Chandar, and Partha Talukdar. 2023. Self-influence guided data reweighting for language model pre-training. arXiv preprint arXiv:2311.00913

    2023 arXiv

  25. [33]

    Santosh Tyss, Hassan Sarwat, Ahmed Mohamed Abdelaal Abdou, and Matthias Grabmair. 2024. Mind your neighbours: Leveraging analogous instances for rhetorical role labeling for legal documents. In Proceedings of the 2024 Joint International Conference on Computational Linguistics...

    2024

  26. [34]

    Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, and Michael Zeng. 2022. Training data is more valuable than you think: A simple and effective method by retrieving from training data. arXiv preprint arXiv:2203.08773

    2022 arXiv

  27. [35]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R \'e mi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical met...

    2020

  28. [36]

    Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. 2023. Alignscore: Evaluating factual consistency with a unified alignment function. arXiv preprint arXiv:2305.16739

    2023 arXiv

  29. [37]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations

    2019

  30. [38]

    Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022. Towards a unified multi-dimensional evaluator for text generation. arXiv preprint arXiv:2210.07197

    2022 arXiv

  31. [39]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  32. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.