Pith. sign in

REVIEW 2 major objections 4 minor 84 references

Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilities

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DPO post-training improves readability of LLM-generated German text simplifications but lowers meaning preservation, and target-group preference data is too inconsistent for any DPO checkpoint to beat its SFT baseline at the group level.

desk verdict Solid, transparent empirical study with a valuable new dataset, but the headline negative result about DPO and target-group preferences is confounded by training on noisy annotators and evaluating on consistent ones. read the letter →

arxiv 2507.01479 v1 pith:6JU33DZN submitted 2025-07-02 cs.CL cs.AI

Yingqiang Gao , Kaede Johnson , David Froehlich , Luisa Carrer , Sarah Ebling This is my paper
classification cs.CLcs.AI
keywords automatictextsimplificationdirectpreferenceoptimizationGermanintellectualdisabilitieshumandataHF4ATSreadabilityLLMalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests whether direct preference optimization (DPO) can personalize large language models for simplifying German texts for persons with intellectual disabilities, using pairwise preferences collected from the target group and from text simplification experts. It finds that DPO post-training consistently improves readability as measured by the Wiener Sachtextformel 4 (WSTF4) across all tested backbones and both preference sources, while also consistently lowering meaning preservation as measured by BERTScore. On held-out preference pairs, expert-supervised DPO checkpoints achieve win rates above 0.50 and, for the two German-tuned backbones, statistically significant group-level preference over their SFT precursors; no target-group DPO checkpoint reaches significance. The paper concludes that DPO-style alignment relies on consistency in supervision signals that is difficult to achieve when preference data is collected from persons with intellectual disabilities, and it releases HF4ATS, a German ATS preference dataset, to support further research.

What carries the argument

The central mechanism is direct preference optimization (DPO), a contrastive alignment objective that replaces explicit reward modeling with an implicit reward margin. For a complex text $x$, a preferred simplification $y_w$, and a dispreferred simplification $y_l$, DPO maximizes the log-sigmoid of $\beta$ times the difference between the log-odds of the preferred and dispreferred completions under the policy model relative to a frozen reference model (initialized as the SFT checkpoint). The paper operationalizes alignment through the win rate $W_{y_w \succ y_l}$, the fraction of held-out pairs for which the implicit reward margin is positive, and through the DPO supremacy score $S_{\text{DPO} \succ \text{SFT}}$, the proportion of pairwise human choices favoring DPO inferences over SFT inferences.

What would settle it

A within-subjects reading-comprehension experiment with persons with intellectual disabilities, where each participant reads SFT and DPO simplifications of the same complex sentences and answers simple multiple-choice questions, would settle whether the WSTF4 gain corresponds to genuine comprehension improvement; if comprehension is equal or better for SFT outputs, the claimed readability gain is an artifact of the formula.

Watch

Extended reading notes

Core claim

The paper's central discovery is that DPO post-training of LLM-based German ATS models produces a reliable readability gain at the cost of semantic fidelity, and that the effectiveness of DPO is governed by the consistency of the preference signal. Across three 7-8 billion parameter backbones, DPO checkpoints improve WSTF4 readability relative to their SFT precursors while consistently reducing BERTScore. On held-out pairwise preferences, all expert-supervised DPO models exceed a 0.50 win rate and the German-tuned backbones show statistically significant group-level DPO supremacy over SFT, whereas no target-group-supervised checkpoint reaches significance. Target-group annotator agreement is near zero (Cohen's kappa ranging from about -0.04 to 0.30, Krippendorff's alpha near 0), while expert agreement is moderate. The authors frame this as a fundamental challenge: methods like DPO rely on consistency in supervision signals that is often difficult to achieve when preference data comes from persons with intellectual disabilities.

Load-bearing premise

The load-bearing premise is that the Wiener Sachtextformel 4 (WSTF4) readability score measures the accessibility that matters to persons with intellectual disabilities, since the headline 'readability improves' result is measured with that formula rather than with target-group comprehension data.

Editorial extensions

If this is right

  • If DPO post-training is applied to ATS for readability, practitioners should expect a trade-off: WSTF4 readability improves while BERTScore meaning preservation drops, regardless of backbone or preference source.
  • Expert-supervised DPO can produce group-level preference alignment for German-tuned backbones, with DiscoLeo-Llama-DPO-1080 and LeoLM-Mistral-DPO-2280 showing statistically significant DPO supremacy over their SFT precursors.
  • Target-group DPO checkpoints should not be assumed to capture group preferences: none of the three reaches significant group-level supremacy, and win rates hover near the 0.50 threshold.
  • Filtering training pairs to those annotated by high-intra-consistency annotators can boost win rates, e.g., a 31.93% improvement for the target-group LeoLM-Mistral-DPO model, indicating that preference consistency matters more than information-equality labels or model-pair source.
  • HF4ATS provides a reusable German dataset of complex sentences with paired simplifications and both target-group and expert preferences for future alignment research.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the near-zero inter-annotator agreement among target-group annotators suggests that pairwise preference annotation itself may be the wrong elicitation format for this population; a single-item acceptability judgment (as in KTO) may yield more consistent supervision, a direction the paper itself floats for future work.
  • Editorial inference: because WSTF4 is a formula based on multisyllabic words and sentence length, the readability gain could partly reflect DPO's tendency to shorten and simplify surface form rather than improve comprehension; a reading-comprehension study with target-group readers would test whether the WSTF4 improvement translates to actual understanding.
  • Editorial inference: the failure of expert-trained models to generalize to target-group preferences (win rates oscillating around 0.50 when evaluated on target annotations) suggests that group-level personalization may require target-group data at every stage, not just evaluation.
  • Editorial inference: the same pipeline could be applied to other low-resource accessibility domains, such as German easy-to-understand health information, where preference consistency might be higher and DPO's trade-off more favorable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies whether direct preference optimization (DPO) can personalize German automatic text simplification (ATS) models for persons with intellectual disabilities. The authors collect HF4ATS, a new preference dataset containing paired simplifications generated by SFT models and annotated by both target-group participants and text-simplification experts. They fine-tune three German-capable LLMs (DiscoLeo-Llama-3-8B, Llama-3.1-8B, LeoLM-Mistral-7B) as SFT checkpoints and then apply DPO using target-group or expert preferences, evaluating through automatic metrics (SARI, BERTScore, WSTF4, win rates) and a human evaluation in which four target-group evaluators and four expert evaluators compare DPO outputs against their SFT precursors. The main findings are that DPO post-training improves readability as measured by WSTF4 but causes meaning loss as measured by BERTScore, that expert-supervised DPO achieves statistically significant group-level preference over SFT for German-tuned backbones, and that no target-group DPO checkpoint achieves statistically significant group-level preference. The paper concludes that preference inconsistency among target-group annotators poses a fundamental challenge for DPO-style alignment.

Significance. If the central negative result holds, the paper makes a useful contribution to accessible NLP by providing a concrete, well-documented case study showing that noisy preference signals from a user group can undermine offline preference optimization, and by releasing the first German ATS preference dataset collected directly from the target group, along with open-sourced models and code. The strengths are the held-out preference splits, the inclusion of both intra- and inter-annotator agreement measures, the human evaluation with real target-group participants, and the explicit, honest discussion of limitations in Section 5.3. The main weakness is that the target-group negative result is confounded: the evaluated DPO checkpoints were trained on all target annotations, including annotators with near-zero intra-annotator agreement, while human evaluation used only the four most consistent target annotators. The paper's own Table 5 shows that consistency-filtered target training can substantially improve win rates, so the central claim needs additional evidence before it can be considered established.

major comments (2)
  1. [Section 5.3, Table 6] The central claim that DPO cannot capture collective target-group preferences is confounded by training-data noise. The target DPO checkpoints in Table 6 were trained on the 'all' subset of HF4ATS-DPO, which includes annotations from participants with very low or negative intra-annotator agreement (Table 1: ta01 = -0.037, ta03 = -0.026, ta07 = -0.045, ta14 = 0.008), while the human evaluation used only evaluators ta04, ta05, ta10, and ta12, who have the highest Intra-AA scores. The authors explicitly acknowledge this possibility in Section 5.3: 'It is therefore possible that human evaluation of a DPO checkpoint trained on a different subset of our target group preference pairs would better capture group-level preferences.' This is not merely hypothetical: Table 5 shows that training on the max Intra-AA subset raises target-group win rates from 0.5145 to 0.6094 for Llama and from 0.4382 to 0.5781 for LeoLM-Mistral. Since no human DPO-supremacy evaluation was run for consistency-filtered target checkpoints, the failure of the all-target DPO models cannot be attributed to an incompatibility between DPO and target-group preferences rather than to noisy training labels. The conclusion in Section 6 that 'methods like DPO rely on consistency in supervision signals that is often difficult to achieve' therefore lacks support as stated; the paper should either add a human evaluation of the max Intra-AA target checkpoints or substantially weaken the general claim.
  2. [Section 5.1, Table 4] The positive RQ1 finding that 'DPO post-training benefits the ATS readability in general' rests entirely on WSTF4 as a readability proxy, but no validation is provided linking WSTF4 to actual comprehension or perceived accessibility for persons with intellectual disabilities. WSTF4 is a general German readability formula based on multisyllabic word percentage and sentence length, and the paper itself notes in Section 4.1 that WSTF4 scores of 4 and 15 correspond to 'very simple' and 'very complex' texts, yet there is no evidence that this formula tracks what the target group finds readable. In addition, the differences reported in Table 4 are not accompanied by significance tests or confidence intervals; several expert-DPO WSTF4 improvements are large, but some target-DPO differences, such as Llama-DPO-1440 at 5.796 versus Llama-SFT-2400 at 5.852, are small relative to the reported standard deviations. The claim that DPO improves readability 'in general' would be more defensible if the authors either validated WSTF4 against target-group judgments or reported uncertainty intervals and a statistical comparison.
minor comments (4)
  1. [Section 4.2, Table 6] The DPO supremacy scores are computed from only 50 pairs per model, and the group-level binomial test uses majority voting with random assignment of ties; with four evaluators and a small number of pairs, the test may have low power to detect a true group-level preference. This does not change the direction of the negative result, but a statement about statistical power would help readers interpret the absence of significant target-group effects.
  2. [Table 2] The target group's 'Pref. % 1st ATS' of 36.65% is far from the expected 50%, which suggests a potential left/right side bias or task non-adherence among some target annotators. Appendix C and Figure 8 discuss one such annotator (ta06), but the implications of this bias for the DPO training data and for the win-rate interpretation deserve explicit discussion in the main text.
  3. [Section 3.2.1] The Gaussian sampling weight for APA-LHA introduces a mean shift of η · (μ_DLHA − μ_DDEplain\DSFT) with η = 4800/8000, but no justification is given for why the mean should be shifted in this particular way. A brief explanation of the intended length-matching behavior would improve reproducibility.
  4. [Section 5.2, Table 5] The paper states that '30 winning DPO checkpoints' were evaluated, but the reader must infer that this corresponds to 3 backbones × 2 annotation groups × 5 training subsets. Spelling this out explicitly would avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the DPO comparisons are held-out evaluations and the central claim is not forced by construction.

full rationale

The paper's derivation chain is self-contained and externally grounded. The DPO objective is taken from the external literature [52] and applied as a standard training loss; the SFT reference checkpoints are selected by SARI/WSTF4 on a development set, and the final win rates are computed on the withheld DPO test set Dtest_DPO, not on training pairs. The DPO supremacy scores in Table 6 are based on newly collected human pairwise judgments between DPO and SFT inferences, assembled from test-set complex sentences, so they are not a re-statement of the training signal. WSTF4 is a fixed, published readability formula [7], not a quantity fitted from the model outputs, so the 'readability improves' conclusion is an external metric comparison rather than a self-defined outcome. The only mildly self-referential element is that each model is evaluated on preferences from the same annotator group that supplied its supervision, which is a standard setup for measuring group-level alignment and does not make the measured preference identical to the training objective. The paper's own caution in Section 5.3 that target DPO checkpoints were trained on all annotators but evaluated by the four most consistent ones is a genuine confound and a validity threat, but it is not a circular step: the result is not equivalent to its input by construction. No fitted parameter is renamed as a prediction, and the cited prior work by the same authors is used for datasets and background, not as the sole justification for the central 'fundamental challenge' conclusion.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its load-bearing assumptions are the validity of DPO as an alignment method, the validity of the readability and meaning metrics for the target group, and the representativeness of the small participant pool. The free parameters are heuristic thresholds and standard hyperparameters in the data filtering and training pipeline.

free parameters (5)
  • SFT entailment cosine threshold = 0.5
    Used to filter DEplain-APA pairs in Section 3.1.2, chosen based on empirical analysis without a reported sensitivity study.
  • SFT ROUGE overlap threshold = F1 > 0.8 removed
    Removes pairs whose simplifications are too similar to complex inputs, Section 3.1.2; threshold is heuristic.
  • SFT max simplification words = 30
    Removes 116 overlong simplifications in Section 3.1.2; chosen by hand.
  • Gaussian length sampling mean and sigma = mu=15, sigma=3
    Used to balance sentence lengths when sampling 5,200 pairs for the SFT split, Section 3.1.2; parameters chosen by hand.
  • DPO beta = 0.1
    DPO regularization coefficient, Section 3.2.3; a standard hyperparameter not fitted to target data.
assumptions (6)
  • standard math Pairwise human preferences follow the Bradley-Terry model, P(y_w > y_l | x) = sigma(R(x,y_w)-R(x,y_l)).
    Invoked in Section 2.2 to derive the DPO objective, as in Rafailov et al. (2023).
  • domain assumption The DPO loss with a frozen reference model is a valid alignment objective that approximates RLHF without a separate reward model.
    The paper relies on DPO as its personalization method in Section 2.2, citing [52]; it does not re-derive or independently validate the objective.
  • domain assumption WSTF4 is a valid reference-free measure of accessibility for persons with intellectual disabilities.
    Used as the primary readability metric in Section 4.1 and the basis for the RQ1 conclusion that DPO improves readability; no validation against target-group comprehension is provided.
  • domain assumption SARI and BERTScore are valid reference-based measures of simplification quality and meaning preservation for German ATS.
    Adopted in Section 4.1 from the ATS literature; their suitability for German Leichte Sprache outputs is assumed.
  • domain assumption The 15 target-group participants with mild to moderate intellectual disabilities suffice to represent group-level preferences for personalization.
    Section 3.2.2 describes the participant pool; the paper treats group-level personalization as achievable through aggregated pairwise choices despite low annotator agreement.
  • domain assumption Manually constructed ATS preference pairs by 13 expert pair creators are of sufficient quality without reported inter-creator agreement.
    Section 3.2.1 describes the rubric, but no agreement statistics among pair creators are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilities." pith.science (2026). https://pith.science/paper/6JU33DZN

@misc{pith2026250701479,
  author       = {Pith},
  title        = {Pith review of: Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6JU33DZN}},
  note         = {Machine review of arXiv:2507.01479}
}
read the original abstract

Automatic text simplification (ATS) aims to enhance language accessibility for various target groups, particularly persons with intellectual disabilities. Recent advancements in generative AI, especially large language models (LLMs), have substantially improved the quality of machine-generated text simplifications, thereby mitigating information barriers for the target group. However, existing LLM-based ATS systems do not incorporate preference feedback on text simplifications during training, resulting in a lack of personalization tailored to the specific needs of target group representatives. In this work, we extend the standard supervised fine-tuning (SFT) approach for adapting LLM-based ATS models by leveraging a computationally efficient LLM alignment technique -- direct preference optimization (DPO). Specifically, we post-train LLM-based ATS models using human feedback collected from persons with intellectual disabilities, reflecting their preferences on paired text simplifications generated by mainstream LLMs. Furthermore, we propose a pipeline for developing personalized LLM-based ATS systems, encompassing data collection, model selection, SFT and DPO post-training, and evaluation. Our findings underscore the necessity of active participation of target group persons in designing personalized AI accessibility solutions aligned with human expectations. This work represents a step towards personalizing inclusive AI systems at the target-group level, incorporating insights not only from text simplification experts but also from target group persons themselves.

Figures

Figures reproduced from arXiv: 2507.01479 by the authors.

Figure 1
Figure 1. Our personalization pipeline for LLM-based ATS models. (a) Data Filtering: We selected high-quality sentence-level [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A web application was developed to collect prefer [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Win rates on development sets during training across different subsets of the HF4ATS-DPO training data. Development [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Average reward margins with respect to the number of training instances from different subsets of the HF4ATS-DPO [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Cross-model comparison for SFT checkpoint evaluation. This figure compares different SFT configurations across [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: shows that pair creators exhibited individual preferences for specific SFT checkpoints despite the model-blind pair creation procedure [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: presents the percentage of pairs labeled as having equal or differing information, grouped by pair creator. The shaded bottom row (Avg.) shows the overall average, with nearly 80% of pairs labeled as having equal information [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: suggests that some target group annotators consistently favored one side, indicating possible non-adherence to task instructions [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 67 canonical work pages

  1. [1]

    Sweta Agrawal and Marine Carpuat. 2024. Do Text Simplification Systems Pre- serve Meaning? A Human Evaluation via Reading Comprehension. Transactions of the Association for Computational Linguistics 12 (2024), 432–448

    2024

  2. [2]

    Suha S Al-Thanyyan and Aqil M Azmi. 2021. Automated Text Simplification: A Survey. ACM Computing Surveys (CSUR) 54, 2 (2021), 1–36

    2021

  3. [3]

    Fernando Alva-Manchego, Louis Martin, Antoine Bordes, Carolina Scarton, Benoît Sagot, and Lucia Specia. 2020. ASSET: A Dataset for Tuning and Evalua- tion of Sentence Simplification Models with Multiple Rewriting Transformations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 4668–4679

    2020

  4. [4]

    Miriam Anschütz, Joshua Oehms, Thomas Wimmer, Bartłomiej Jezierski, and Georg Groh. 2023. Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-training. In Findings of the Association for Computational Linguistics: ACL 2023 . 1147–1158

    2023

  5. [5]

    Dennis Aumiller and Michael Gertz. 2022. Klexikon: A German Dataset for Joint Summarization and Simplification. In Proceedings of the Thirteenth Language Resources and Evaluation Conference . 2693–2701

    work page 2022

  6. [6]

    Nguyen Bach, Qin Gao, Stephan Vogel, and Alex Waibel. 2011. TriS: A Statistical Sentence Simplifier with Log-linear Models and Margin-based Discriminative Training. In Proceedings of 5th International Joint Conference on Natural Language Processing. 474–482

    work page 2011

  7. [7]

    Richard Bamberger and Erich Vanacek. 1984. Lesen-Verstehen-Lernen-Schreiben. Diesterweg

    work page 1984

  8. [8]

    Alessia Battisti, Dominik Pfütze, Andreas Säuberli, Marek Kostrzewa, and Sarah Ebling. 2020. A Corpus for Automatic Readability Assessment and Text Sim- plification of German. In Proceedings of the Twelfth Language Resources and Evaluation Conference. 3302–3311

    work page 2020

Show all 84 references
  1. [9]

    Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022. Power to the people? Opportunities and challenges for participatory AI. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms,...

    2022

  2. [10]

    Ralph Allan Bradley and Milton E Terry. 1952. Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrika 39, 3/4 (1952), 324–345

    1952

  3. [11]

    Andrew Cashin, Julia Morphet, Nathan J Wilson, and Amy Pracilio. 2024. Barriers to Communication with People with Developmental Disabilities: A Reflexive Thematic Analysis. Nursing & health sciences 26, 1 (2024), e13103

    2024

  4. [12]

    Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, and He He. 2022. Meta- learning via Language Model In-context Tuning. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 719–730

    2022

  5. [13]

    Paul F Christiano, Jan Leike, Tom B Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep Reinforcement Learning from Human Preferences. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 4302–4310

    2017

  6. [14]

    Jacob Cohen. 1960. A Coefficient of Agreement for Nominal Scales. Educational and psychological measurement 20, 1 (1960), 37–46

    1960

  7. [15]

    Liam Cripwell, Joël Legrand, and Claire Gardent. 2023. Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplification. In Proceed- ings of the 2023 Conference on Empirical Methods in Natural Language Processing . 12053–12059

    2023

  8. [16]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The LLaMA 3 Herd of Models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  9. [17]

    Sarah Ebling, Alessia Battisti, Marek Kostrzewa, Dominik Pfütze, Annette Rios, Andreas Säuberli, and Nicolas Spring. 2022. Automatic Text Simplification for German. Frontiers in Communication 7 (2022), 706718

    2022

  10. [18]

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Model alignment as prospect theoretic optimization. In Forty-first International Conference on Machine Learning

    2024

  11. [19]

    Federal Ministry of Social Affairs, Health, Care and Consumer Protection (BMS- GPK). 2022. National Action Plan on Disability 2022–2030. Vienna: BMSGPK (2022)

    2022

  12. [20]

    Adam Fisch, Jacob Eisenstein, Vicky Zayats, Alekh Agarwal, Ahmad Beirami, Chirag Nagpal, Pete Shaw, and Jonathan Berant. 2024. Robust Preference Opti- mization Through Reward Model Distillation. arXiv preprint arXiv:2405.19316 (2024)

    2024 arXiv

  13. [21]

    Rudolph Flesch. 1948. A New Readability Yardstick.Journal of applied psychology 32, 3 (1948), 221

    1948

  14. [22]

    Annette Rios Gonzales, Nicolas Spring, Tannon Kew, Marek Kostrzewa, Andreas Säuberli, Mathias Müller, and Sarah Ebling. 2021. A New Dataset and Efficient Baselines for Document-level Text Simplification in German. In Proceedings of the Third Workshop on New Frontiers in Summar...

    2021

  15. [23]

    Silvia Hansen-Schirra, Walter Bisang, Arne Nagels, Silke Gutermuth, Julia Fuchs, Liv Borghardt, Silvana Deilen, Anne-Kathrin Gros, Laura Schiffl, and Johanna Sommer. 2020. Intralingual Translation into Easy Language–or How to Reduce Cognitive Processing Costs. Easy Language Re...

    2020

  16. [24]

    Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2022. Towards a Unified View of Parameter-Efficient Transfer Learning. In Proceedings of the Tenth International Conference on Learning Representations

    2022

  17. [25]

    David Heineman, Yao Dou, Mounica Maddela, and Wei Xu. 2023. Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 3466–3495

    2023

  18. [26]

    Freya Hewett, Hadi Asghari, and Manfred Stede. 2024. Elaborative Simplification for German-language Texts. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue . 29–39

    2024

  19. [27]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. LoRA: Low-Rank Adaptation of Large Lan- guage Models. In Proceedings of the Tenth International Conference on Learning Representations

    2022

  20. [28]

    Xiangkun Hu, Tong He, and David Wipf. [n. d.]. New Desiderata for Direct Preference Optimization. In ICML 2024 Workshop on Models of Human Feedback for AI Alignment

    2024

  21. [29]

    Hamish Ivison, Yizhong Wang, Jiacheng Liu, Zeqiu Wu, Valentina Pyatkin, Nathan Lambert, Noah A Smith, Yejin Choi, and Hannaneh Hajishirzi. 2024. Unpacking DPO and PPO: Disentangling Best Practices for Learning from Pref- erence Feedback. In Proceedings of the 38st Internationa...

    2024

  22. [30]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al . 2023. Mistral 7B. arXiv preprint arXiv:2310.06825 (2023)

    2023 arXiv

  23. [31]

    J Peter Kincaid, Robert P Fishburne Jr, Richard L Rogers, and Brad S Chissom

  24. [32]

    David Klaper, Sarah Ebling, and Martin Volk. 2013. Building a German/Simple German Parallel Corpus for Automatic Text Simplification. In Proceedings of the Second Workshop on Predicting and Improving Text Readability for Target Reader Populations. 11–19

    2013

  25. [33]

    Lars Klöser, Mika Beele, Jan-Niklas Schagen, and Bodo Kraft. 2024. German Text Simplification: Finetuning Large Language Models with Semi-Synthetic Data. In Proceedings of the Fourth Workshop on Language Technology for Equality, Diversity, Inclusion. 63–72

    2024

  26. [34]

    Klaus Krippendorff. 2004. Reliability in Content Analysis: Some Common Mis- conceptions and Recommendations. Human communication research 30, 3 (2004), 411–433

    2004

  27. [35]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58t...

    2020

  28. [36]

    Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, 74–81

    2004

  29. [37]

    Jiahong Liu, Zexuan Qiu, Zhongyang Li, Quanyu Dai, Jieming Zhu, Minda Hu, Menglin Yang, and Irwin King. 2025. A Survey of Personalized Large Language Models: Progress and Future Directions. arXiv preprint arXiv:2502.11528 (2025)

    2025

  30. [38]

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvinine- jad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual Denoising Pre-training for Neural Machine Translation. (2020), 726–742

    2020

  31. [39]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In Proceedings of the Seventh International Conference on Learning Representations (ICLR 2019)

    2019

  32. [40]

    Ilya Loshchilov and Frank Hutter. 2022. SGDR: Stochastic Gradient Descent with Warm Restarts. In Proceedings of the Tenth International Conference on Learning Representations (ICLR 2022)

    2022

  33. [41]

    Christiane Maaß. 2015. Leichte Sprache. Das Regelbuch . Deutsche Nationalbiblio- thek

    2015

  34. [42]

    Mounica Maddela, Fernando Alva-Manchego, and Wei Xu. 2021. Controllable Text Simplification with Explicit Paraphrasing. In Proceedings of the 2021 Confer- ence of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies. 3536–3553

    2021

  35. [43]

    Mounica Maddela, Yao Dou, David Heineman, and Wei Xu. 2023. LENS: A Learnable Evaluation Metric for Text Simplification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 16383–16408

    2023

  36. [44]

    Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, Dietrich Klakow, and Yanai Elazar. 2023. Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation. In Findings of the Association for Computational Linguistics: ACL

    2023

  37. [45]

    Eduardo Mosqueira-Rey, Elena Hernández-Pereira, David Alonso-Ríos, José Bobes-Bascarán, and Ángel Fernández-Leal. 2023. Human-in-the-loop Machine Learning: A State of the Art.Artificial Intelligence Review 56, 4 (2023), 3005–3054

    2023

  38. [46]

    Akifumi Nakamachi, Tomoyuki Kajiwara, and Yuki Arase. 2020. Text Simplifica- tion with Reinforcement Learning using Supervised Rewards on Grammaticality, Meaning Preservation, and Simplicity. In Proceedings of the 1st conference of the Asia-Pacific chapter of the association f...

    2020

  39. [47]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training Language Models to Follow Instructions with Human Feedback. In Proceedings of the 36st International Conference ...

    2022

  40. [48]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics . 311–318

    2002

  41. [49]

    Venkatesh Balavadhani Parthasarathy, Ahtsham Zafar, Aafaq Khan, and Ar- salan Shahid. 2024. The Ultimate Guide to Fine-tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities. arXiv...

    2024 arXiv

  42. [50]

    Biqing Qi, Pengfei Li, Fangyuan Li, Junqi Gao, Kaiyan Zhang, and Bowen Zhou

  43. [51]

    Jipeng Qiang and Xindong Wu. 2019. Unsupervised Statistical Text Simplification. IEEE Transactions on Knowledge and Data Engineering 33, 4 (2019), 1802–1806

    2019

  44. [52]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. 2023. Direct Preference Optimization: Your Lan- guage Model is Secretly A Reward Model. In Proceedings of the 37th International Conference on Neural Information Processing S...

    2023

  45. [53]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the Limits of Transfer Learning with A Unified Text-to-text Transformer. Journal of machine learning research 21, 140 (2020), 1–67

    2020

  46. [54]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCN...

    2019

  47. [55]

    Horacio Saggion, Sanja Štajner, Stefan Bott, Simon Mille, Luz Rello, and Biljana Drndarevic. 2015. Making it Simplext: Implementation and Evaluation of A Text Simplification System for Spanish. ACM Transactions on Accessible Computing (TACCESS) 6, 4 (2015), 1–36

    2015

  48. [56]

    Andreas Säuberli, Sarah Ebling, and Martin Volk. 2020. Benchmarking Data- driven Automatic Text Simplification for German. In Proceedings of the 1st work- shop on tools and resources to empower people with reading dIfficulties (READI) . 41–48

    2020

  49. [57]

    Andreas Säuberli, Franz Holzknecht, Patrick Haller, Silvana Deilen, Laura Schiffl, Silvia Hansen-Schirra, and Sarah Ebling. 2024. Digital Comprehensibility As- sessment of Simplified Texts among Persons with Intellectual Disabilities. In Proceedings of the CHI Conference on Hu...

    2024

  50. [58]

    Carolina Scarton and Lucia Specia. 2018. Learning Simplifications for Specific Target Audiences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . 712–718

    2018

  51. [59]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  52. [60]

    Laura Seiffe, Fares Kallel, Sebastian Möller, Babak Naderi, and Roland Roller

  53. [61]

    Kim Cheng Sheang and Horacio Saggion. 2021. Controllable Sentence Simplifi- cation with a Unified Text-to-Text Transfer Transformer. InProceedings of the 14th International Conference on Natural Language Generation . 341–352

    2021

  54. [62]

    Zhengyan Shi, Adam X Yang, Bin Wu, Laurence Aitchison, Emine Yilmaz, and Aldo Lipani. 2024. Instruction Tuning with Loss Over Instructions. InProceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

    2024

  55. [63]

    Advaith Siddharthan and Angrosh Mandya. 2014. Hybrid Text Simplification Us- ing Synchronous Dependency Grammars with Hand-written and Automatically Harvested Rules. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics ...

    2014

  56. [64]

    Nicolas Spring, Annette Rios Gonzales, and Sarah Ebling. 2021. Exploring German Multi-Level Text Simplification. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021) . 1339–1349

    2021

  57. [65]

    Nicolas Spring, Marek Kostrzewa, David Fröhlich, Annette Rios, Dominik Pfütze, Alessia Battisti, and Sarah Ebling. 2023. Analyzing Sentence Alignment for Automatic Simplification of German Texts. In Emerging Fields in Easy Language and Accessible Communication Research . Sprin...

    2023

  58. [66]

    Nicolas Spring, Marek Kostrzewa, Annette Rios, and Sarah Ebling. 2022. En- sembling and Score-Based Filtering in Sentence Alignment for Automatic Sim- plification of German Texts. In International Conference on Human-Computer Interaction. 137–149

    2022

  59. [67]

    Regina Stodden, Omar Momen, and Laura Kallmeyer. 2023. DEplain: A German Parallel Corpus with Intralingual Translations into Plain Language for Sentence and Document Simplification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol...

    2023

  60. [68]

    Julia Suter, Sarah Ebling, and Martin Volk. 2016. Rule-based Automatic Text Simplification for German. In Proceedings of the 13th Conference on Natural Language Processing (KONVENS 2016) . 279–287

    2016

  61. [69]

    Vanessa Toborek, Moritz Busch, Malte Boßert, Christian Bauckhage, and Pascal Welke. 2023. A New Aligned Simple German Corpus. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 11393–11412

    2023

  62. [70]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  63. [71]

    Xingjiao Wu, Luwei Xiao, Yixuan Sun, Junhang Zhang, Tianlong Ma, and Liang He. 2022. A Survey of Human-in-the-loop for Machine Learning. Future Genera- tion Computer Systems 135 (2022), 364–381

    2022

  64. [72]

    Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison- Burch. 2016. Optimizing Statistical Machine Translation for Text Simplification. Transactions of the Association for Computational Linguistics 4 (2016), 401–415. Gao and Johnson et al

    2016

  65. [73]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. InProceedings of the 2021 Conference of the North American Chapter of the Association ...

    2021

  66. [74]

    Daichi Yamaguchi, Rei Miyata, Sayuka Shimada, and Satoshi Sato. 2023. Gauging the Gap Between Human and Machine Text Simplification Through Analytical Evaluation of Simplification Strategies and Errors. In Findings of the Association for Computational Linguistics: EACL 2023 . 359–375

    2023

  67. [75]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi

  68. [76]

    Xingxing Zhang and Mirella Lapata. 2017. Sentence Simplification with Deep Reinforcement Learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing . 584–594

    2017

  69. [77]

    Siyan Zhao, Mingyi Hong, Yang Liu, Devamanyu Hazarika, and Kaixiang Lin

  70. [78]

    Le- icht:

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. LIMA: Less is More for Alignment. In Proceedings of the 37th International Conference on Neural Information Processing Systems. 55006–55021. Evaluating...

    2023

  71. [1975]

    Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel. (1975)

    1975

  72. [2017]

    arXiv preprint arXiv:1707.06347 (2017)

    Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  73. [2019]

    In Proceedings of the Seventh International Conference on Learning Representations (ICLR 2019)

    BERTScore: Evaluating Text Generation with Bert. In Proceedings of the Seventh International Conference on Learning Representations (ICLR 2019)

    2019

  74. [2022]

    In Proceedings of the Thirteenth Language Resources and Evaluation Conference

    Subjective text complexity assessment for German. In Proceedings of the Thirteenth Language Resources and Evaluation Conference . 707–714

  75. [2024]

    arXiv preprint arXiv:2406.05534 (2024)

    Online DPO: Online Direct Preference Optimization with Fast-slow Chas- ing. arXiv preprint arXiv:2406.05534 (2024)

    2024 arXiv

  76. [2025]

    InProceedings of the 15th International Conference on Learning Representations (ICLR 2025)

    Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs. InProceedings of the 15th International Conference on Learning Representations (ICLR 2025)

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.