Classic readability formulas correlate weakly with human readability judgments for plain-language summaries (FKGL r=0.16), the best LLM judge reaches r=0.56, and the two evaluator families rank datasets nearly opposite.
The Use of Readability Metrics in Legal Text: A Systematic Literature Review
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
Understanding the text in legal documents can be challenging due to their complex structure and the inclusion of domain-specific jargon. Laws and regulations are often crafted in such a manner that engagement with them requires formal training, potentially leading to vastly different interpretations of the same texts. Linguistic complexity is an important contributor to the difficulties experienced by readers. Simplifying texts could enhance comprehension across a broader audience, not just among trained professionals. Various metrics have been developed to measure document readability. Therefore, we adopted a systematic review approach to examine the linguistic and readability metrics currently employed for legal and regulatory texts. A total of 3566 initial papers were screened, with 34 relevant studies found and further assessed. Our primary objective was to identify which current metrics were applied for evaluating readability within the legal field. Sixteen different metrics were identified, with the Flesch-Kincaid Grade Level being the most frequently used method. The majority of studies (73.5%) were found in the domain of "informed consent forms". From the analysis, it is clear that not all legal domains are well represented in terms of readability metrics and that there is a further need to develop more consensus on which metrics should be applied for legal documents.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Evaluating the Evaluators: Are readability metrics good measures of readability?
Classic readability formulas correlate weakly with human readability judgments for plain-language summaries (FKGL r=0.16), the best LLM judge reaches r=0.56, and the two evaluator families rank datasets nearly opposite.