REVIEW 3 major objections 7 minor 40 references
LengClaro2023: A Dataset of Administrative Texts in Spanish with Plain Language adaptations
T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read LengClaro2023 pairs each of seven Spanish Social Security pages with two simplified versions, one based on arText claro's recommendations and one on additional plain-language guidelines, forming a dataset for evaluating automatic text…
desk verdict A small, honestly-built Spanish administrative simplification dataset whose benchmark claim is undercut by simplified targets that sometimes add or correct information not present in the source. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The organizing object is the HTML trio: each of the seven documents is stored in three parallel files following the naming pattern document number_original.html, document number_artext.html, and document number_lengclaro.html. The two simplification protocols are the second mechanism: arText claro's automatic recommendations, classified into discourse, morphosyntactic, and lexical levels with numeric thresholds such as 25 words per sentence and 135 words per paragraph, and the LengClaro manual guidelines, built from a review of plain-language manuals and web-writing guides and including recommendations such as getting to the point, addressing the reader directly, avoiding parenthetical remarks, using active constructions, and organizing ideas under headings. The trio structure is what makes the dataset usable for evaluating automatic simplification systems.
What would settle it
Inspect the repository's HTML trios and check whether any lengclaro version changes a legal condition, for example whether the child-benefit supplement is still described as grantable independently of the Minimum Vital Income; or run a comprehension test with pension-age readers comparing original and simplified versions. If any simplified version alters a legal condition or fails to improve comprehension, the dataset's central promise of faithful, clearer equivalents does not hold for that text.
Extended reading notes
Core claim
The central claim is that a usable linguistic resource exists: a collection of seven legal-administrative Spanish texts, each in three versions — original, artext (rewritten following arText claro), and lengclaro (rewritten following an expanded set of plain-language recommendations for legal-administrative and web writing). The paper argues this structure supports pairing the original with each simplification as a complex-simple binomial for automatic text simplification. It further claims that the qualitative analysis exposes systematic weaknesses in arText claro — paragraph and sentence boundary detection, passive-voice recognition, and over-flagging of participles and nominalizations — and that the manual process illustrates linguistic challenges of plain-language adaptation, such as resolving ambiguity by consulting external legal sources.
Load-bearing premise
The dataset's usefulness as a benchmark depends on the two simplified versions being both faithful to the legal content and genuinely easier for citizens to understand, but they were produced by a single professional translator and not checked by a second annotator or by comprehension testing, as stated in the paper's Limitations section.
Editorial extensions
If this is right
- Researchers can pair original-artext and original-lengclaro as gold simplified outputs to score automatic text simplification systems on legal-administrative Spanish.
- arText claro's rule set can be refined: average sentence-length criteria, broader passive detection, and more targeted flagging of participles and nominalizations would make its recommendations easier to apply.
- The manual simplification cost of 30 to 90 minutes per page-length text quantifies the gap between automatic assistance and fully manual plain-language adaptation.
- Because readability and comprehensibility are distinct, an automatic text simplification benchmark built on this dataset should separate surface readability metrics from meaning-level comprehension.
- Because the original web pages may have been updated since extraction, the dataset versions should be checked against the live site before being used in evaluation.
Reading between the lines
- Used naively as a benchmark, the dataset could reward systems that imitate one translator's phrasing rather than general plain-language competence; a multi-annotator validation or reader-comprehension test would make it a more stable gold standard.
- The two-condition design could be exploited to isolate what automatic detection contributes: comparing the artext and lengclaro versions could show whether the additional manual guidelines add measurable clarity or mostly change style.
- The paper's ambiguity-resolution examples suggest a boundary for current text simplification systems: simplifying legal-administrative content often requires external legal knowledge, so evaluation should separate sentence-level simplification from content-preserving legal inference.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces LengClaro2023, a small Spanish dataset built from seven pages of the Spanish Social Security website. For each original HTML page, the authors provide two simplified versions: one produced by following the recommendations of the arText claro writing assistant ('artext') and one produced by manually applying a broader set of plain-language guidelines ('lengclaro'). The paper documents the simplification process in detail, organizes the applied recommendations into discourse, morphosyntactic, lexical, planning, revision, and orthographic categories, and proposes improvements to arText claro based on the authors' experience as users. The stated purpose of the dataset is to serve as a resource for evaluating automatic text simplification (ATS) systems in Spanish, with each original-simplified pair forming a 'complex-simple binomial.' The limitations section acknowledges that the simplifications were produced by a single professional translator and were not checked by a second annotator or by comprehension testing.
Significance. If the dataset is accepted as a valid simplification resource, it would fill a gap for Spanish administrative text simplification: it is openly licensed, preserves the HTML structure of the originals, and provides two distinct simplification strategies, which is useful for comparative evaluation. The qualitative analysis of arText claro from a user perspective is also valuable, particularly the detailed examples of false positives and missed detections across the tool's recommendations. The paper is honest about many limitations and does not overclaim quantitative gains. However, the central evaluation use of the dataset rests on an assumption that the simplified versions are meaning-preserving rewrites of the paired originals, and this assumption is not validated in the manuscript. Because that premise is load-bearing for the stated ATS benchmark purpose, the resource as currently documented cannot be considered a reliable evaluation benchmark without additional validation or a reframing of its intended use.
major comments (3)
- [§5.2, §4(d3), and §6] The reference simplifications are not consistently meaning-preserving rewrites of the source texts. Section 5.2 describes consulting legal experts and external sources such as the BOE and Revista de la Seguridad Social to resolve ambiguities, and Section 4, recommendation d3, states that an omission in original Document 6 was 'corrected' in the LengClaro version, with the simplified text explicitly adding 'A cualquier persona que necesite certificar que percibe o no percibe una prestación.' Similarly, the child-benefit supplement example in Section 5.2 introduces the proposition that the supplement 'se puede conceder de forma independiente,' which is confirmed only by an external source. These targets contain information that is not present in, or not derivable from, the paired original. Under the paper's own definition of simplification in Section 1 ('retaining the original information content and meaning'), such edits are editorial reconstructions rather than simplifications. This directly undermines the claim in Section 6 that the dataset enables 'complex-simple binomials' for ATS evaluation, because an ATS system would be penalized for not producing facts that never appeared in its input. I recommend that the authors either document the provenance and type of each non-derivable edit (e.g., error correction, legal clarification, omission repair) or split the dataset into a strict-simplification subset and an editorial-reconstruction subset, with the ATS benchmark claim restricted to the former.
- [Limitations] The lack of any reliability or validity check is a load-bearing gap for a resource whose stated purpose is evaluation. The limitations section states that the texts were simplified by a single professional translator and that the simplifications were not checked by a second annotator or by comprehension testing. Without inter-annotator agreement on a sample, or at least an independent verification of meaning preservation, there is no evidence that the 'lengclaro' and 'artext' versions reflect stable plain-language norms rather than one person's idiosyncratic rewriting choices. This is particularly important because many of the simplified sentences in Sections 3 and 4 involve substantial restructuring, deletion, and reordering. I would ask the authors to add a small reliability study, for example two independent annotators rating whether each target preserves the meaning of its source, or to add a clear statement that the dataset is a single-author resource whose evaluation use is illustrative rather than normative. The current phrasing frames the dataset as suitable for benchmarking, which the unvalidated single-author status does not support.
- [§5.2 and §3] The paper does not quantify the degree of simplification it claims to achieve, which would be straightforward to provide and would strengthen the descriptive claims. For example, the discussion repeatedly states that sentences were shortened, lists introduced, and difficult words replaced, but no aggregate statistics are reported for the seven documents across the three versions (e.g., mean sentence length, word frequency profiles, number of subordinate clauses, or readability indices). Such statistics are standard in text-simplification resource papers and would also help readers understand the difference between the 'artext' and 'lengclaro' versions. I am not asking for a comprehension study as a condition for publication, since the authors explicitly exclude that claim, but reporting descriptive complexity measures for the final versions would substantiate that the 'simplified' labels correspond to measurable reductions in linguistic complexity.
minor comments (7)
- [§1] The footnote markers for URLs interrupt the prose; consider moving the repository and website URLs to a dedicated availability section with standard citation formatting.
- [§6] The text contains the typo 'LengClaro3023' where 'LengClaro2023' is intended.
- [§4(a9)] The phrase 'the the fact' contains a duplicated article and should read 'the fact.'
- [§4(c11)] The phrase 'in with the generic masculine forms are in bold' should be 'in which the generic masculine forms are in bold.'
- [§3(b7)] The quoted original sentence 'Sino son coincidentes' should be 'Si no son coincidentes'; if this is a typo in the source text, it would be helpful to mark it as such.
- [Throughout] The capitalization of 'arText claro' is inconsistent: the paper uses 'arText claro', 'artext', and 'Artext Claro' in different places. The same applies to 'LengClaro' vs. 'lengClaro'. Please standardize the dataset and tool names.
- [§2.2] The dataset mixes content under different licenses (original web pages vs. CC BY-NC-SA adaptations); a machine-readable per-file license field or a table in the repository would clarify downstream reuse conditions.
Circularity Check
No circularity: the dataset's simplified versions are produced by applying external plain-language rules to selected source texts; the ATS-evaluation claim is a downstream use, not an input to the construction.
full rationale
The paper's derivation chain is resource construction: select seven Spanish Social Security web pages, extract them, then produce two simplified versions, one by applying arText claro's published recommendations and one by manually applying an independently compiled set of plain-language guidelines. Nothing in this chain fits a parameter to a target and then predicts that target, and no equation or definition makes the simplified outputs equivalent to an evaluation score. The central claim is existence of the HTML trios, not a derived empirical result. The only self-citation (Gonzalez-Dios et al., 2022) supports a side discussion about hierarchical processing and sentence-length limits; it is not load-bearing for the dataset's validity. The Limitations section does state that simplifications were made by one professional translator and were not checked by a second annotator or by comprehension testing, and Sections 4 (d3) and 5.2 show that some simplifications involved legal expertise, external sources, and correction of source omissions. These are legitimate validity and benchmark-quality concerns, because such editorial additions could make an ATS target contain propositions not inferable from the source. However, that is an evidence gap or correctness risk, not circularity: the reference texts were not produced by, fitted to, or defined in terms of the ATS systems they might later evaluate, and no claim reduces by construction to its own input. Likewise, the proposed arText claro improvements are untested, but untested proposals are not circular. Accordingly, no specific circular step can be quoted, and the appropriate score is 0.
Assumptions & free parameters
assumptions (5)
- domain assumption Plain-language simplification improves citizen comprehension of administrative texts.
- domain assumption The pages in the 'A un click' and 'Lo más visitado' homepage sections are the most relevant communications for citizens.
- domain assumption arText claro's thresholds for long sentences (25 words) and long paragraphs (135 words) are valid indicators of clarity.
- domain assumption The rewrites by one professional translator preserve the legal content and meaning of the originals.
- domain assumption Hypertext web reading differs enough from linear reading to justify excluding some plain-language recommendations.
Cite this review
Pith. "Pith review of LengClaro2023: A Dataset of Administrative Texts in Spanish with Plain Language adaptations." pith.science (2026). https://pith.science/paper/26D5VOGQ
@misc{pith2026250605927,
author = {Pith},
title = {Pith review of: LengClaro2023: A Dataset of Administrative Texts in Spanish with Plain Language adaptations},
year = {2026},
howpublished = {\url{https://pith.science/paper/26D5VOGQ}},
note = {Machine review of arXiv:2506.05927}
}
read the original abstract
In this work, we present LengClaro2023, a dataset of legal-administrative texts in Spanish. Based on the most frequently used procedures from the Spanish Social Security website, we have created for each text two simplified equivalents. The first version follows the recommendations provided by arText claro. The second version incorporates additional recommendations from plain language guidelines to explore further potential improvements in the system. The linguistic resource created in this work can be used for evaluating automatic text simplification (ATS) systems in Spanish.
Reference graph
Works this paper leans on
-
[1]
F. Alva Manchego, C. Scarton, and L. Specia. 2020. https://doi.org/10.1162/coli_a_00370 Data-driven sentence simplification: Survey and benchmark . Computational Linguistics, 46(1):135--187
-
[2]
N. Beks van Raaij, D. Kolkman, and K. Podoynitsyna. 2024. https://aclanthology.org/2024.determit-1.15 Clearer governmental communication: Text simplification with ChatGPT evaluated by quantitative and qualitative research . In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context @ LREC-COLING 2024, pages 152--178, ...
work page 2024
-
[3]
A. Camps. 1990. Modelos del proceso de redacci \'o n: algunas implicaciones para la ense \ n anza. Infancia y aprendizaje, 13(49):3--19
work page 1990
-
[4]
C. Carretero González. 2019. https://biblioteca-tirant-com.eu1.proxy.openathens.net/cloudLibrary/ebook/info/9788413137582 Comunicación para juristas . Tirant lo Blanch
-
[5]
C. Carretero González and J. C. Fuentes Gómez. 2019. La claridad del lenguaje jurídico. Revista del Ministerio Fiscal, (8):7--40
work page 2019
-
[6]
C. Carretero González, J. Pérez, L. Lanne-Lenne, and G. de los Reyes. 2017. http://hdl.handle.net/11531/17353 Lenguaje claro. comprender y hacernos entender . Technical report, Instituto de Lectura Fácil y Clarity
work page 2017
- [7]
-
[8]
Comisión Europea . 2015. https://publications.europa.eu/resource/cellar/725b7eb0-d92e-11e5-8fea-01aa75ed71a1.0007.03/DOC_1 Cómo escribir con claridad . Oficina de Publicaciones de la Unión Europea
work page 2015
Show all 40 references
-
[9]
Comunidad de Madrid . 2020. https://www.comunidad.madrid/servicios/sede-electronica/comunicacion-clara Gu \' a de comunicaci \'o n clara: Pautas para aplicar a los tr \'a mites de la comunidad de madrid
2020
-
[10]
da Cunha
I. da Cunha. 2022 a . El sistema ar T ext claro: el primer redactor asistido para el espa \ n ol que ayuda a escribir textos en lenguaje claro. In Lenguaje claro y tecnolog \' a en la Administraci \'o n , pages 151--187
2022
-
[11]
da Cunha
I. da Cunha. 2022 b . Un redactor asistido para adaptar textos administrativos a lenguaje claro. Procesamiento del lenguaje natural, 69:39--49
2022
-
[12]
da Cunha and M
I. da Cunha and M. A. Escobar. 2021. https://doi.org/10.25267/Pragmalinguistica.2021.i29.07 Recomendaciones sobre lenguaje claro en espa \ n ol en el ámbito jurídico-administrativo: análisis y clasificación . Pragmalingüística, (29):129–148
2021 doi
-
[13]
de Miguel
E. de Miguel. 2000. El texto jur \' dico-administrativo: an \'a lisis de una orden ministerial. C \' rculo de ling \"u \' stica aplicada a la comunicaci \'o n , (4):3
2000
-
[14]
M. Dols. 2018. Brieven aan burgers: Analyse en evaluatie. Master's thesis, Faculteit Geesteswetenschappen Universiteit van Tilburg, may
2018
-
[15]
Garc \' a-Cabrera
L. Garc \' a-Cabrera. 2018. M etodolog \' a remediavagos para escribir textos eficientes para la web: 20 buenas pr \'a cticas. In Innovaci \'o n y esfuerzo investigador en la educaci \'o n medi \'a tica contempor \'a nea , pages 25--38. Ediciones Egregius
2018
-
[16]
P. Gavora. 2012. Text comprehension and text readability: Findings on lower secondary school pupils in slovakia. Forum Dydaktyczne (9-10), pages 9--21
2012
-
[17]
J. J. Giraldo Ortiz. 2008. An \'a lisis y descripci \'o n de las siglas en el discurso especializado de genoma humano y medio ambiente . Universitat Pompeu Fabra, Barcelona
2008
-
[18]
Gobierno de Aragón . 2020. Manual de estilo de lenguaje claro. C riterios para la redacci \'o n de textos claros en internet. https://www.aragon.es/documents/20127/6649155/Manual [Accessed: 24th October 2023]
2020
-
[19]
Gonzalez-Dios, I
I. Gonzalez-Dios, I. Guti \'e rrez-Fandi \ n o, O. M. Cumbicus-Pineda, and A. Soroa. 2022. https://doi.org/10.18653/v1/2022.tsar-1.8 I rekia LF es: a new open benchmark and baseline systems for S panish automatic text simplification . In Proceedings of the Workshop on Text Sim...
2022 doi
-
[20]
González Delgado and B
G. González Delgado and B. Navarro Colorado. 2024. https://aclanthology.org/2024.determit-1.12 The simplification of the language of public administration: The case of ombudsman institutions . In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilin...
2024
-
[21]
González Soria
B. González Soria. 2021. https://elibro.net/es/ereader/ual/269324 Simplificación del lenguaje contractual . In El jurista y el reto de un derecho comprensible para todos, pages 79--94. Reus
2021
-
[22]
R. M. Jiménez Yá \ n ez. 2016. Escribir bien es de justicia. Aranzadi
2016
-
[23]
Lentz, L
L.R. Lentz, L. Nell, and H. Pander Maat. 2017. https://doi.org/10.5117/TVT2017.2.LENT Begrijpelijkheid van pensioencommunicatie: effecten van wetgeving, geletterdheid en revisies . T ijdschrift voor T aalbeheersing , 39(2):191--208
2017 doi
-
[24]
Ministerio de Justicia . 2011. Informe de la C omisi \'o n de M odernizaci \'o n del Lenguaje J ur \' dico . Ministerio de Justicia
2011
-
[25]
M. Moliner. D.L.1998. Diccionario de uso del espa \ n ol , 2nd edition. Gredos, Madrid
1998
-
[26]
Montolío Durán
E. Montolío Durán. 2019. Hacer m \'a s claro el discurso judicial: propuestas ling \"u \' sticas de optimizaci \'o n. Revista del Ministerio Fiscal, (8):72--95
2019
-
[27]
Montolío Durán and M
E. Montolío Durán and M. Tascón. 2017. https://www.madrid.es/UnidadesDescentralizadas/Calidad/LenguajeClaro/ComunicacionClara/Documentos/GuiaPracticaCClara.pdf Comunicación Clara. Guía Práctica . Ayuntamiento de Madrid
2017
-
[28]
T. H. Nelson. 1981. https://api.semanticscholar.org/CorpusID:61176382 Literary machines: the report on, and of, project Xanadu concerning word processing, electronic publishing, hypertext, thinkertoys, tomorrow's intellectual revolution, and certain other topics including know...
1981
-
[29]
C. Otaola. 1988. La modalidad (con referencia especial a la lengua espa \ n ola). Revista de Filolog \' a espa \ n ola , 68:1--2
1988
-
[30]
Pander Maat and N
H. Pander Maat and N. Dekker. 2016. Tekstgenres analyseren op lexicale complexiteit met t-scan. Tijdschrift voor Taalbeheersing 38(3), pages 263--304
2016
-
[31]
Pistola Grille and I
S. Pistola Grille and I. da Cunha. 2022. https://doi.org/10.14198/ELUA.22442 La resolución administrativa publicada en portales web institucionales: un análisis basado en corpus . ELUA: Estudios de Lingüística. Universidad de Alicante, (38):17–45
2022 doi
-
[32]
Poblete Olmedo and P
C. Poblete Olmedo and P. Yepes Villegas. 2023. Propuesta de un instrumento para medir la claridad de los textos jurídico-administrativos. In Crónica de la lengua espa \ n ola 2022-2023 , pages 629--639. Espasa
2023
-
[33]
Real Academia Espa\ n ola y Asociaci\'on de Academias de la Lengua Espa\ n ola . 2024. Guía panhisp\'anica de lenguaje claro y accesible. Espasa, Madrid
2024
-
[34]
Región de Murcia . 2022. Guía de lenguaje claro para la administración pública de la Comunidad Autónoma de la Región de Murcia : recomendaciones para la descripción de procedimientos y documentos administrativos. Inspección General de Servicios. Dirección General de Modernizac...
2022
-
[35]
J. Renkema. 2011. Een kwalitatief onderzoek naar de begrijpelijkheid van digitale informatie van de belastingdienst
2011
-
[36]
Siddharthan
A. Siddharthan. 2014. https://doi.org/10.1075/itl.165.2.06sid A survey of research on text simplification . ITL - International Journal of Applied Linguistics, 165(2):259--298
2014 doi
-
[37]
G. Sierra. 2022. Palabras largas vs. palabras cortas: una estrategia para el lenguaje claro basada en la variaci \'o n sinon \' mica. In Lenguaje claro y tecnolog \' a en la Administraci \'o n , pages 137--150. Comares
2022
-
[38]
Vilches and R
F. Vilches and R. Sarmiento. 2010. Manual de lenguaje jur \' dico-administrativo . Dykinson
2010
-
[39]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[40]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.