Pith. sign in

REVIEW 3 major objections 7 minor 23 references

SimplifyMyText: An LLM-Based System for Inclusive Plain Language Text Simplification

T0 review · 3 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SimplifyMyText is an online system that uses large language models to rewrite complex text into plain language, tailored to a reader group chosen by the user, and the authors report evaluation scores showing readable, well-aligned…

desk verdict A plausible demo system whose audience-specific claim is not supported by the reported evaluation; worth a demo-track referee but not strong enough to cite as evidence. read the letter →

arxiv 2504.14223 v1 pith:Q2GHERWT submitted 2025-04-19 cs.CL cs.AIcs.HC

Michael Färber , Parisa Aghdam , Kyuri Im , Mario Tawfelis , Hardik Ghoshal This is my paper · ORCID
classification cs.CLcs.AIcs.HC
keywords textsimplificationplainlanguagelargemodelsaccessibilityaudience-specificcustomizationzero-shotlearningdemosystemwebplatform
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces SimplifyMyText, an online system that uses large language models to rewrite complex input into plain language, with the target reader selected by the user. The authors describe it as the first such system to accept multiple input formats—typed text, PDFs, and Word documents—and to offer audience-specific customization rather than a one-size-fits-all simplification. They evaluate two models, GPT-4o and Llama 3.1, on a Wikipedia-based simplification dataset and report that both produce outputs with high BLEU and SARI scores and readable Flesch-Kincaid levels across the five offered audience groups. The stated goal is to make tailored plain-language communication easier to produce, thereby supporting literacy and social inclusion.

What carries the argument

The core mechanism is zero-shot prompting: the user selects one of five audience options (Scientists and Researchers, Students and Academics, Industry Professionals, Journalists and Media, General Public) and the system sends the text to a large language model with instructions to simplify for that audience, defaulting to the General Public setting when no audience is chosen. An expert editing mode lets users click any word for synonyms or definitions and any sentence for rephrasing at different complexity levels, and a rating feature collects feedback. On the evaluation side, the load-bearing tools are the PKWP Wikipedia sentence-pair dataset and the BLEU, SARI, and Flesch-Kincaid metrics, which together quantify alignment with human references and readability.

What would settle it

A comprehension experiment with human participants from each of the five target audiences: if simplified outputs, despite high BLEU and SARI scores and favorable Flesch-Kincaid levels, do not improve factual comprehension over the original texts, the paper's accessibility claim would be unsupported.

Watch

Extended reading notes

Core claim

The central claim is that an LLM-powered demo system can generate plain-language versions of a text that stay close to human-written references and are readable for a chosen audience, and that the two tested models, GPT-4o and Llama 3.1, both do this effectively. On the PKWP dataset, Llama 3.1 earns the higher BLEU and SARI scores in most user groups, while GPT-4o yields slightly better Flesch-Kincaid Ease and Grade levels; the paper takes these results as evidence that audience-specific plain-language simplification is feasible with zero-shot prompting. It further claims that providing audience selection, word-level synonyms and definitions, sentence-level rephrasing, and a feedback rating feature makes the platform a purpose-built alternative to generic AI text tools.

Load-bearing premise

The evaluation assumes that a Wikipedia simplification dataset plus the BLEU, SARI, and Flesch-Kincaid scores are valid proxies for whether the five target audiences actually understand the simplified text.

Editorial extensions

If this is right

  • If the system works as described, government agencies could automate plain-language compliance for public documents, reducing the manual labor that currently makes simplification inconsistent.
  • Educators and content producers could generate audience-appropriate versions of the same material on demand instead of writing separate versions by hand.
  • Because the architecture accepts any large language model, organizations could plug in open-source or privacy-preserving models rather than relying on a single commercial API.
  • The built-in rating and feedback loop could steadily improve simplification quality without retraining the underlying model.
  • The audience-specific options could make the same text usable by diverse readers, extending plain-language practice beyond a single simplified style for everyone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • In our reading, the evaluation establishes feasibility, not superiority: the paper does not compare against existing simplification systems or fine-tuned models, so a head-to-head benchmark would be a natural next test.
  • A direct way to extend the work would be human comprehension testing: ask people from each target audience to read original and simplified versions and answer questions, since BLEU, SARI, and Flesch-Kincaid do not measure understanding directly.
  • The same platform design could be generalized to non-English plain language, for example German or Spanish, where the paper's own motivation notes large populations with reading challenges; the multi-format upload feature would carry over unchanged.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces SimplifyMyText, a web-based demo system that uses LLMs (GPT-4o and Llama 3.1) to generate plain-language simplifications of user-provided text, with five audience-specific prompt options and an expert mode. The authors evaluate the two models on the PWKP dataset (called PKWP in the paper) using BLEU, SARI, and Flesch-Kincaid scores, and report that both models produce well-aligned, readable simplifications. The paper claims to be the first system to offer tailored plain-language simplification for multiple audiences from multiple input formats.

Significance. If the evaluation were sound, the paper would provide a practical, open-source accessible-simplification platform that could benefit researchers and practitioners in accessibility and NLP. The public demo, GitHub repository, and comparison of a proprietary and an open-weights LLM are concrete contributions. However, the current evidence does not establish the central claim of audience-specific accessibility: the metrics and dataset are not matched to the five target audiences, no baselines or human studies are included, and the prompt-optimization procedure is not separated from the evaluation. The system is plausible, but the paper's main quantitative claim needs substantial revision.

major comments (3)
  1. [3 Evaluation] Section 3 states that BLEU, Flesch-Kincaid, and SARI are used 'to assess performance and optimize prompts' but provides no information about a train/dev/test split. If the same sentences are used both for prompt engineering and for final scoring, the scores in Table 2 may be optimistically biased and are not independent. This directly affects the conclusion that both models 'generate well-aligned, readable simplifications' and must be addressed by documenting the split or demonstrating that prompt choices were made on a separate development set.
  2. [3 Evaluation, Table 2] The PKWP dataset (the paper repeats the name PKWP, but the cited source [23] is the PWKP corpus) consists of Wikipedia/Simple Wikipedia sentence pairs with generic simple-English references. It contains no audience-specific references for the five categories in Table 2 (Scientists and Researchers, Students and Academics, Industry Professionals, Journalists and Media, General Public). Therefore BLEU and SARI scores computed against the same generic references cannot show that the audience-specific prompts produce different or appropriate output for each target group; the audience-tailoring claim is not supported by the reported numbers.
  3. [3 Evaluation, Table 2] The Flesch-Kincaid Ease and Grade scores in Table 2 are computed only on the generated text, not on the original inputs, so the table does not demonstrate that simplification improves readability relative to the source. Additionally, there is no baseline (e.g., a generic 'simplify this text' prompt) and no human comprehension test with participants drawn from the target audiences. For a system whose purpose is inclusive plain-language communication, the absence of any human evaluation is a serious gap; the conclusion that accessibility is 'effectively enhanc[ed]' rests on automatic metrics alone, which is insufficient.
minor comments (7)
  1. [Abstract] The sentence 'we https://simplifymytext.org' is missing the verb 'introduce' and is not a complete sentence; please add the missing word.
  2. [3 Evaluation, Table 2] The metric names should be defined precisely: 'Flesch Reading Ease (FRE)' and 'Flesch-Kincaid Grade Level'; the current 'FK Ease' and 'FK Grade' columns are not introduced in the text.
  3. [3 Evaluation, Table 2] BLEU scores are reported as values between 0.4 and 0.5 while the text says scores range from 0 to 1; BLEU is conventionally reported on a 0-100 scale. Please state the scaling convention used.
  4. [3 Evaluation] The parameter count 'approximately 1.8 trillion' for GPT-4o is not officially documented; it should be removed or cited to a reliable source.
  5. [3 Evaluation, Table 2] The dataset name 'PKWP' in Section 3 and Table 2 should be 'PWKP' to match reference [23] (Zhu et al., 2010); the current name appears to be a typo.
  6. [1 Introduction, Table 1] The simplified example in Table 1 contains the phrase 'a mistake only 37.5% of the top time and 17.0% of the top five times', which misrepresents 'top-1 and top-5 error rates' and should be corrected or the example replaced with a clearer one.
  7. [Throughout] The model name appears both as 'Llama 3.1' and 'LLaMA 3.1'; please use one consistent spelling.

Circularity Check

1 steps flagged · score 6.0 of 10

Evaluation scores are partly fitted: the same PKWP set and metrics used to optimize prompts are then reported as evidence of quality, with no held-out split.

  1. fitted input called prediction [Section 3, first paragraph]
    "We evaluate our system using the PKWP dataset [23], which contains 108,016 paired sentences from 65,133 Wikipedia articles. To assess performance and optimize prompts, we use three metrics: BLEU, Flesch-Kincaid, and SARI."

    The paper states that the three metrics are used both 'to assess performance and optimize prompts' on the same PKWP dataset, and no train/test split is described. If prompts were tuned to maximize BLEU/SARI/FK on these sentences, then the Table 2 scores are the values of the objective used for prompt selection, not independent measurements of generalization. The conclusion that 'both models generate well-aligned, readable simplifications' is therefore partly self-referential: the reported scores are partly a fitted result rather than a prediction on unseen data.

full rationale

The central contribution is the SimplifyMyText web system, which is not circular: it is a demo platform with audience-specific prompts, and no derived quantity is defined in terms of its own output. There are no load-bearing self-citations; no referenced uniqueness theorem is invoked. The main circularity concern is in the evaluation. The paper explicitly says the same three metrics are used for both performance assessment and prompt optimization, presumably on the same PKWP set, without documenting any held-out split. This makes the reported BLEU, SARI, and Flesch-Kincaid scores partly the result of fitting prompts to the evaluation data, so Table 2 cannot serve as an independent confirmation of the accessibility claim. Additionally, PKWP only contains generic Simple English references, so the audience-specific rows are not scored against audience-matched references; however, that is an external-validity limitation rather than definitional circularity. The system itself remains a functional contribution independent of these scores, which prevents the circularity from consuming the entire paper, but the evaluation evidence is substantially compromised.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The core contribution is a demo system, so the ledger is dominated by empirical assumptions rather than mathematical axioms. The main load-bearing choices are the hand-tuned prompt templates, the use of PKWP as a proxy for plain-language quality, and the assumption that automatic readability metrics reflect comprehension for the target audiences.

free parameters (1)
  • Audience-specific prompt templates = Unspecified, five hand-tuned variants
    The authors state in Section 3 that they use the metrics to optimize prompts. The exact wording of the five audience prompts is not published, so the customization results depend on undisclosed, hand-tuned strings that may have been adjusted on the evaluation data.
assumptions (3)
  • domain assumption BLEU, SARI, and Flesch-Kincaid scores are meaningful proxies for plain-language quality and audience accessibility.
    Section 3 uses only these automatic metrics to conclude that outputs are readable and accessible, with no human comprehension test or comparison to target readers.
  • domain assumption The PKWP Wikipedia simplification pairs approximate plain-language simplification for the five advertised audiences.
    PKWP contains simple Wikipedia rewrites, not audience-specific plain-language texts, and the paper does not justify using it to validate versions for Scientists, Journalists, or the General Public.
  • domain assumption LLM zero-shot simplification preserves the meaning of the source text.
    The paper does not evaluate factuality or semantic equivalence, yet it claims simplification while preserving meaning and presents the tool as enhancing accessibility.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SimplifyMyText: An LLM-Based System for Inclusive Plain Language Text Simplification." pith.science (2026). https://pith.science/paper/Q2GHERWT

@misc{pith2026250414223,
  author       = {Pith},
  title        = {Pith review of: SimplifyMyText: An LLM-Based System for Inclusive Plain Language Text Simplification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q2GHERWT}},
  note         = {Machine review of arXiv:2504.14223}
}
read the original abstract

Text simplification is essential for making complex content accessible to diverse audiences who face comprehension challenges. Yet, the limited availability of simplified materials creates significant barriers to personal and professional growth and hinders social inclusion. Although researchers have explored various methods for automatic text simplification, none fully leverage large language models (LLMs) to offer tailored customization for different target groups and varying levels of simplicity. Moreover, despite its proven benefits for both consumers and organizations, the well-established practice of plain language remains underutilized. In this paper, we https://simplifymytext.org, the first system designed to produce plain language content from multiple input formats, including typed text and file uploads, with flexible customization options for diverse audiences. We employ GPT-4 and Llama-3 and evaluate outputs across multiple metrics. Overall, our work contributes to research on automatic text simplification and highlights the importance of tailored communication in promoting inclusivity.

Figures

Figures reproduced from arXiv: 2504.14223 by the authors.

Figure 1
Figure 1. Screenshot of our demo system at https://simplifymytext.org ences. As the first system of its kind, it enables professionals to refine outputs for specific audiences. With a user-friendly design, the platform lets users gener￾ate customized plain language texts by pasting content or uploading documents, producing versions aligned with their intended audience. 2 Demonstration System Our code and demonstration video a… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 16 canonical work pages

  1. [23]

    In: Proceedings of the 23rd International Conference on Computational Linguistics

    Zhu, Z., Bernhard, D., Gurevych, I.: A Monolingual Tree-based Translation Model for Sentence Simplification. In: Proceedings of the 23rd International Conference on Computational Linguistics. p. 1353–1361. COLING ’10, Association for Com- putational Linguistics, USA (2010)

    work page 2010

  2. [1]

    https://www.zeit.de/news/2020-08/29/literatur-muss-nicht-kompliziert- sein (2020), [Accessed 03-11-2024]

    Trend zu einfacher Sprache: Literatur muss nicht kompliziert sein — zeit.de. https://www.zeit.de/news/2020-08/29/literatur-muss-nicht-kompliziert- sein (2020), [Accessed 03-11-2024]

    work page 2020

  3. [2]

    Anschütz, M., Oehms, J., Wimmer, T., Jezierski, B., Groh, G.: Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style- specificPre-training.In:FindingsoftheAssociationforComputationalLinguistics. pp. 1147–1158. ACL’23 (2023). https://doi.org/10.18653/v1/2023.findings- acl.74

    doi:10.18653/v1/2023.findings- 2023

  4. [3]

    In: Perspectives and Trends in Education and Tech- nology, Smart Innovation, Systems and Technologies, vol

    Araújo, S., Aguiar, M.: Simplifying Specialized Texts with AI: A ChatGPT- Based Learning Scenario. In: Perspectives and Trends in Education and Tech- nology, Smart Innovation, Systems and Technologies, vol. 366. Singapore (2023). https://doi.org/10.1007/978-981-99-5414-8_55

    work page doi:10.1007/978-981-99-5414-8_55 2023

  5. [4]

    In: Proceedings of the 33th Annual Conference on Neural Informa- tion Processing Systems

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language Models are Few- Shot Learners. In: Proceedings of the 33th Annual Conference on Neural Informa- tion Processing Systems. NeurIPS’20, vol. 33, pp. 1877–1901 (2020)

    work page 2020

  6. [5]

    js case study

    De Sousa, M., Gonçalves, A.: humanportal–a react. js case study. In: Proceedings of the 15th Iberian Conference on Information Systems and Technologies. pp. 1–6. CISTI’20, IEEE (2020) 6 M. Färber et al

    work page 2020

  7. [6]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics

    Devaraj, A., Sheffield, W., Wallace, B., Li, J.J.: Evaluating Factuality in Text Simplification. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. pp. 7331–7345. ACL’22, Dublin, Ireland (2022). https://doi.org/10.18653/v1/2022.acl-long.506

    doi:10.18653/v1/2022.acl-long.506 2022

  8. [7]

    OECD Education Working Papers (131) (2016).https://doi.org/10.1787/ 5jm0v44bnmnx-en, https://doi.org/10.1787/5jm0v44bnmnx-en

    Grotlüschen, A., et al.: Adults with Low Proficiency in Literacy or Numer- acy. OECD Education Working Papers (131) (2016).https://doi.org/10.1787/ 5jm0v44bnmnx-en, https://doi.org/10.1787/5jm0v44bnmnx-en

    work page doi:10.1787/5jm0v44bnmnx-en 2016

Show all 23 references
  1. [8]

    Standard, Interna- tionalOrganizationforStandardization,Geneva,CH(Jun2023),status:Published, Stage: International Standard [60.60]

    Plain language — Part 1: Governing principles and guidelines. Standard, Interna- tionalOrganizationforStandardization,Geneva,CH(Jun2023),status:Published, Stage: International Standard [60.60]

  2. [9]

    In: Proceedings ofthe 26th Annual Conference on Neural Information Processing Systems

    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet Classification with Deep Convolutional Neural Networks. In: Proceedings ofthe 26th Annual Conference on Neural Information Processing Systems. pp. 1106–1114. NIPS’12 (2012)

    2012

  3. [10]

    Maaß, C.: Easy Language – Plain Language – Easy Language Plus, vol. 3. Frank & Timme, 1 edn. (2020)

    2020

  4. [11]

    In: Sprache bar- rierefrei gestalten

    Maaß,C.,Rink,I.,Zehrer,C.:LeichteSpracheinderSprach-undÜbersetzungswis- senschaft [Plain Language in Linguistics and Translation Studies]. In: Sprache bar- rierefrei gestalten. Perspektiven aus der Angewandten Linguistik [Designing Lan- guage for Accessibility: Perspectives fr...

    2014

  5. [12]

    In: Proceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies

    Maddela, M., Alva-Manchego, F., Xu, W.: Controllable Text Simplification with Explicit Paraphrasing. In: Proceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies. pp. 3536–3553. NAACL-HLT...

    2021 doi

  6. [13]

    In: Proceedings of the Twelfth Language Resources and Eval- uation Conference

    Martin, L., de la Clergerie, É., Sagot, B., Bordes, A.: Controllable Sentence Simplification. In: Proceedings of the Twelfth Language Resources and Eval- uation Conference. pp. 4689–4698. LREC’20, Marseille, France (2020), https: //aclanthology.org/2020.lrec-1.577

    2020

  7. [14]

    https://www.un.org/en/chronicle/article/education-all- rising-challenge (2010), [Accessed 03-11-2024]

    Nations, U.: Education for All: Rising to the Challenge | United Na- tions — un.org. https://www.un.org/en/chronicle/article/education-all- rising-challenge (2010), [Accessed 03-11-2024]

    2010

  8. [15]

    In: Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications

    Omelianchuk, K., Raheja, V., Skurzhanskyi, O.: Text Simplification by Tagging. In: Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications. pp. 11–25. Association for Computational Linguistics, Online (2021), https://aclanthology.org/202...

    2021

  9. [16]

    In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a Method for Automatic Evaluation of Machine Translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. pp. 311–318. ACL’02, Yorktown Heights, NY, USA (2002)

    2002

  10. [17]

    https://plainlanguagenetwork.org/plain-language/plain-language- around-the-world/ (2024), [Accessed 07-01-2025]

    Plain Language Association International: Plain language around the world. https://plainlanguagenetwork.org/plain-language/plain-language- around-the-world/ (2024), [Accessed 07-01-2025]

    2024

  11. [18]

    In: Proceedings of the 14th International Con- ference on Natural Language Generation

    Sheang, K.C., Saggion, H.: Controllable Sentence Simplification with a Unified Text-to-Text Transfer Transformer. In: Proceedings of the 14th International Con- ference on Natural Language Generation. pp. 341–352. INLG’21, Aberdeen, Scot- land, UK (2021).https://doi.org/10.186...

    2021 doi

  12. [19]

    International Journal of Applied Linguistics165(2), 259–298 (2014) An LLM-Based System for Inclusive Plain Language Text Simplification 7

    Siddharthan, A.: A survey of research on text simplification. International Journal of Applied Linguistics165(2), 259–298 (2014) An LLM-Based System for Inclusive Plain Language Text Simplification 7

    2014

  13. [20]

    PLOS Digit Health3(4), e0000489 (Apr 2024)

    Swanson, K., He, S., Calvano, J., Chen, D., Telvizian, T., Jiang, L., Chong, P., Schwell, J., Mak, G., Lee, J.: Biomedical text readability after hypernym substi- tution with fine-tuned large language models. PLOS Digit Health3(4), e0000489 (Apr 2024). https://doi.org/10.1371/...

    2024 doi

  14. [21]

    arXiv preprint arXiv:2201.08239 (2022)

    Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.T., Jin, A., Bos, T., Baker, L., Du, Y., et al.: Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022)

    2022 arXiv

  15. [22]

    Transactions of the Association for Computational Linguistics 4, 401–415 (2016)

    Xu, W., Napoles, C., Pavlick, E., Chen, Q., Callison-Burch, C.: Optimizing statisti- cal machine translation for text simplification. Transactions of the Association for Computational Linguistics 4, 401–415 (2016). https://doi.org/10.1162/tacl_ a_00107

    2016 doi

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.