Pith. sign in

REVIEW 3 major objections 5 minor 42 references

Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training a language model on a small human-ranked paraphrase dataset with Direct Preference Optimization raises generation accuracy to 57 percent and top-ranked human preference to 40 percent.

desk verdict Useful but confounded: without an SFT-on-APTY control, the claimed DPO gains in paraphrase type generation can't be attributed to DPO, even though the paper ships code, models, and a real human evaluation. read the letter →

arxiv 2506.02018 v2 pith:WJA5IKG2 submitted 2025-05-28 cs.CL

Christopher Lee Lübbers This is my paper
classification cs.CL
keywords paraphrasetypegenerationdetectionDirectPreferenceOptimizationhumanalignmentRLHFatomictypesreference-freeevaluationhuman-annotateddataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a small set of human-ranked paraphrases—each labeled with a fine-grained atomic paraphrase type—can be used with Direct Preference Optimization (DPO) to align a language model's paraphrase generation with what people actually prefer. On 260 test sentences spanning ten paraphrase types, the DPO-trained Llama-3.1-8B model generated the correct transformation with 57% mean accuracy, versus 54% for the supervised fine-tuned baseline and 8% for the base model, and human annotators ranked its output first 40% of the time versus 33% for the baseline. The paper also reports that standard lexical-overlap metrics (ROUGE, BLEU) correlate with human rankings below 0.3, and that a newly built paraphrase-type detector reaches F1 scores of 0.91 for addition/deletion, 0.78 for same-polarity substitution, and 0.70 for punctuation changes. If these results hold, they show that human preference data, not larger reference sets, is the missing ingredient for semantically faithful paraphrase generation, and that reference-free detection can evaluate transformations more granularly than BLEU/ROUGE.

What carries the argument

The load-bearing mechanism is Direct Preference Optimization (DPO), a training objective that updates the language model directly from pairs of chosen and rejected responses, eliminating the separate reward model used by RLHF. The object it acts on is the APTY-ranked dataset, whose 333 examples each contain an original sentence, a target atomic paraphrase type (a fine-grained transformation such as addition/deletion or same-polarity substitution), and human-ranked chosen/rejected paraphrases. The evaluation machinery is a paraphrase-type detection (PTD) model, a DeBERTa-based classifier first trained on Quora Question Pairs for binary paraphrase recognition and then fine-tuned on the ten most frequent ETPC paraphrase types, which the paper uses to verify whether generated paraphrases actually exhibit the requested transformation without a reference sentence.

What would settle it

A sufficient check is to re-annotate the same 1,040 model outputs with a larger, independent annotator pool and recompute per-model accuracy and top-rank preference; if DPO/APTY no longer beats SFT/ETPC by roughly 3 accuracy points and 7 preference points, the central claim fails.

Watch

Extended reading notes

Core claim

The core discovery is that Direct Preference Optimization on the APTY-ranked dataset—333 human-ranked examples in which each item pairs a chosen and a rejected paraphrase for a given atomic paraphrase type—moves a supervised fine-tuned Llama-3.1-8B toward the transformations humans judge correct. The paper reports 57% mean accuracy (SD=9) for DPO/APTY versus 54% (SD=8) for SFT/ETPC and 8% (SD=14) for the base model, with a one-way ANOVA over all 1,040 human-annotated outputs significant at $p < 10^{-12}$. In the preference ranking task, DPO/APTY received the top rank in 40% of cases versus 33% for SFT/ETPC and 6% for the base model. An identity-preference-optimization (IPO) variant reached 52% accuracy, and a reward-model RLHF pipeline was discontinued because its reward model only reached 49% accuracy. The paper presents these numbers as evidence that human-ranked preference data plus DPO yields paraphrases that are both more technically accurate and more user-aligned, and that automated lexical metrics systematically miss this improvement.

Load-bearing premise

The load-bearing premise is that the two annotators' rankings of the 1,040 generated paraphrases are accurate enough to serve as ground truth; if those rankings are noisy or biased, the reported 3-point accuracy gain and 7-point preference gain could be artifacts of the annotation, not of DPO.

Editorial extensions

If this is right

  • Paraphrase-type generation can be improved without a reward model: a few hundred human-ranked examples fed through DPO beat supervised fine-tuning on both accuracy and top-rank preference.
  • Because ROUGE and BLEU correlate with human rankings below 0.3, evaluations that rely on lexical overlap are likely understating DPO's advantage; future paraphrase-type-generation work should include human or detection-based evaluation.
  • A reference-free paraphrase-type detector can give per-type feedback (F1 0.91 for addition/deletion), enabling model-specific error analysis, though complex semantic types remain hard for it.
  • DPO/APTY also improved the team-allocation subset of the MuSR reasoning benchmark by up to 38 percent, suggesting human-aligned paraphrase training can transfer to some multistep reasoning tasks.
  • The newly created human-annotated dataset provides a reusable testbed for comparing generation and detection methods on paraphrase types.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit: the same DPO-on-ranked-data recipe should be applied to other fine-grained generation tasks (e.g., text simplification with type labels) to see whether the 3-point gain generalizes beyond APTY/ETPC.
  • The annotated evaluation used only two annotators, so a useful next check is measuring variance across annotator pools; if agreement is higher with more annotators, the moderate kappa was a measurement artifact, and if lower, the accuracy gap may shrink.
  • Because the PTD detector scores far lower on semantic-based (0.16) and derivational changes (0.12), framing it as a general reference-free evaluator is premature; the evidence supports reliable detection only for high-frequency concrete transformations.
  • The paper's weak metric correlations (<0.3) imply that conventional leaderboards could be re-ranked by swapping BLEU/ROUGE for a PTD-style detector; this is a direct consequence of the reported numbers but is not a claim the paper itself makes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript studies paraphrase-type generation (PTG) where a Llama-3.1-8B model is first supervised-fine-tuned on the Extended Typology Paraphrase Corpus (ETPC) and then further trained on the human-ranked APTY dataset using Direct Preference Optimization (DPO), Identity Preference Optimization (IPO), or reward-model-based RLHF. The authors report that DPO/APTY reaches 57% human-annotated PTG accuracy versus 54% for SFT/ETPC, that human evaluators rank DPO/APTY first in 40% of cases versus 33% for SFT/ETPC, and that automated metrics such as ROUGE and BLEU correlate weakly with human judgments. They also train a DeBERTa-based paraphrase type detection (PTD) model with reported F1 scores up to 0.91, and they evaluate the PTG models on Open LLM Leaderboard v2 tasks, noting a MuSR improvement. The paper concludes that DPO with human-ranked data improves paraphrase-type generation and motivates reference-free evaluation.

Significance. If the attribution of the observed gains to DPO were rigorously supported, this would be a useful demonstration that preference optimization can improve fine-grained linguistic transformation tasks and that human-ranked data can complement lexical metrics. The paper's strengths include public release of code and models, a human-annotated evaluation effort, confidence intervals for the PTD F1 scores, and evaluation on a standard benchmark suite. However, the central empirical claim is currently not well identified: the DPO condition differs from the SFT condition in both training data and training objective, and several reported numbers are internally inconsistent. The contribution is therefore conditional on additional control experiments and corrected statistical reporting.

major comments (3)
  1. [§3.2.2, §4.1, §4.2] The central comparison is confounded. DPO/APTY is initialized from SFT/ETPC and then trained on APTY-ranked, whereas SFT/ETPC is trained on ETPC only; no model is trained by continued supervised learning on the APTY chosen responses or by an SFT loss on APTY. The reported 3-point accuracy gain and 7-point preference gain therefore bundle the effect of the DPO/IPO objective with the effect of exposure to the APTY dataset and additional training steps. The abstract and conclusion attribute the gain to DPO ('DPO-based training increases...'), which is not supported without an SFT-on-APTY control. This is load-bearing: even with perfect annotations, the current design cannot separate data-addition effects from preference-optimization effects.
  2. [§4.1, §4.2, §5] Pairwise statistical support for the headline differences is missing, and several reported numbers are internally inconsistent. Section 4.1 gives only an omnibus ANOVA (F(3,1036)=49.4) for the four-model accuracy comparison; it does not report a pairwise test or confidence interval for the DPO/APTY versus SFT/ETPC 57% vs 54% difference. Section 4.2 reports only an omnibus chi-square, and the text (40% vs 33%) conflicts with the Figure 6 caption (49% vs 33%), the table 5 chi-square value (92.34) conflicts with the text value (231.9), and the conclusion's '16%' user-preference gain conflicts with the abstract's '7 percentage points'. The authors should provide pairwise tests with multiplicity correction, confidence intervals for all headline differences, and a single consistent set of top-rank values.
  3. [§3.2.3, §4.1, §4.2] The human-annotation premise is thin for the claimed effect sizes. All 1,040 paraphrases were judged by two annotators with only moderate agreement (Cohen's kappa 0.69; Krippendorff's alpha 0.63), and the ranking protocol assigns all invalid paraphrases a common rank of 4. With top-rank shares of 40% versus 33% based on 260 sets, the 7-point difference is within the range that annotator noise or adjudication choices could materially affect. The authors should report per-annotator accuracies and rankings, adjudication or disagreement handling, and confidence intervals around the top-rank proportions, or reduce the strength of the causal claim.
minor comments (5)
  1. [§4.4] The text contains a typo ('We evalauted') and should clarify whether the reported F1 values are macro-averaged or weighted; the abstract cites per-type F1 scores while §4.4 reports a weighted F1 of 0.71.
  2. [§1, Figure 1] The title and Figure 1 prominently mention RLHF, but RLHF was discontinued because the reward model accuracy was only 49%; the presentation should avoid implying that RLHF was part of the successful pipeline.
  3. [§4.3] The correlation analysis uses only 30 paraphrase sets (120 paraphrases); the text should state explicitly that this is a supplementary subset and explain how it was chosen.
  4. [§3.2.3] The logistic transformation uses 2.5 as a fixed midpoint, but no justification or sensitivity analysis is provided for this choice; a sentence explaining the rationale would help.
  5. [Table 5, §4.1] The table caption reports a chi-square value of 92.34 with p=5.5e-16 while the main text reports chi-square(9)=231.9 with p<10^-44; these values should be reconciled.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the DPO-versus-SFT comparison is a genuine experiment with fresh human evaluation; same-group citations are used as building blocks, not as a forced premise.

full rationale

The central claim (DPO/APTY improves paraphrase-type generation accuracy by 3 points and human preference by 7 points over SFT/ETPC) rests on a real experiment: DPO/APTY is initialized from SFT/ETPC and further trained on APTY-ranked preferences, and the evaluation uses fresh human annotations of 1,040 model-generated paraphrases on ETPC base sentences. The evaluation accuracy is human judgment of type correctness, not the DPO training objective, and the preference rankings are new annotations, not the APTY training labels. Thus the result is not equivalent to its inputs by construction. The paper does lean on same-group resources (APTY from Meier et al. [22], the SFT/ETPC model from Wahle et al. [37], and the author's own GitHub/HuggingFace [20,21]), but these are used as experimental building blocks and benchmarks, not as an unverified premise that forces the conclusion. The lack of an SFT-on-APTY control is a real confound: the 3-point gain may reflect extra data and training steps rather than DPO specifically. However, a missing control is an experimental-validity concern, not a circular reduction. Manual hyperparameter tuning to maximize reward margins and accuracy is mild selection, not a fitted parameter renamed as a prediction. No equation or definition in the paper reduces the reported gains to the training data or to a self-citation chain; the reported inconsistencies in top-rank percentages and the 16% versus 7% preference increase are correctness issues, not circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the reliability of human annotations and on prior same-group datasets (APTY, ETPC). No new theoretical entities are introduced; the free parameters are training hyperparameters and a correlation-smoothing constant.

free parameters (5)
  • DPO beta = 0.2
    Controls the KL penalty in the DPO loss; set by hand, tuned to maximize reward margins and stability (section 3.2.2).
  • DPO learning rate = 1e-6
    Learning rate for DPO training, chosen to maximize reward margins (section 3.2.2).
  • DPO weight decay = 4e-1
    Weight decay for DPO, unusually high; chosen during tuning (section 3.2.2).
  • Logistic transformation midpoint = 2.5
    Ad hoc midpoint used to smooth human ranks before computing correlations with automatic metrics; affects the reported correlation coefficients (section 3.2.3).
  • Top-10 paraphrase type filter = 10 types
    Limits PTD training and evaluation to the most frequent types, which improves reported F1 and aligns with APTY; a selection choice (section 3.3.1).
assumptions (5)
  • domain assumption The APT taxonomy is a valid and complete typology of paraphrase transformations.
    Adopted from Vila et al. [36] and used as the label space in sections 3.1 and 3.2.
  • domain assumption The human rankings in the APTY-ranked dataset reflect genuine user preferences.
    The training signal for DPO; if the rankings are noisy or biased, the preference alignment claim weakens (section 3.2.1).
  • domain assumption The ETPC annotations for the top-10 types are correct and complete examples of those types.
    Used to train and evaluate the PTD model (section 3.3.1); errors here propagate to F1 scores.
  • standard math DPO and IPO theoretical guarantees apply to this fine-tuning setting.
    The paper relies on Rafailov et al. [28] and Azar et al. [1] for the optimization objectives (sections 2 and 3.2.2).
  • domain assumption The chosen base models (LLaMA-3.1-8B, LLaMA-2-7B, DeBERTa, BART) are appropriate for the tasks.
    The study does not justify model choice beyond open-source availability (section 3.2.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data." pith.science (2026). https://pith.science/paper/WJA5IKG2

@misc{pith2026250602018,
  author       = {Pith},
  title        = {Pith review of: Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WJA5IKG2}},
  note         = {Machine review of arXiv:2506.02018}
}
read the original abstract

Paraphrasing re-expresses meaning to enhance applications like text simplification, machine translation, and question-answering. Specific paraphrase types facilitate accurate semantic analysis and robust language models. However, existing paraphrase-type generation methods often misalign with human preferences due to reliance on automated metrics and limited human-annotated training data, obscuring crucial aspects of semantic fidelity and linguistic transformations. This study addresses this gap by leveraging a human-ranked paraphrase-type dataset and integrating Direct Preference Optimization (DPO) to align model outputs directly with human judgments. DPO-based training increases paraphrase-type generation accuracy by 3 percentage points over a supervised baseline and raises human preference ratings by 7 percentage points. A newly created human-annotated dataset supports more rigorous future evaluations. Additionally, a paraphrase-type detection model achieves F1 scores of 0.91 for addition/deletion, 0.78 for same polarity substitution, and 0.70 for punctuation changes. These findings demonstrate that preference data and DPO training produce more reliable, semantically accurate paraphrases, enabling downstream applications such as improved summarization and more robust question-answering. The PTD model surpasses automated metrics and provides a more reliable framework for evaluating paraphrase quality, advancing paraphrase-type research toward richer, user-aligned language generation and establishing a stronger foundation for future evaluations grounded in human-centric criteria.

Figures

Figures reproduced from arXiv: 2506.02018 by the authors.

Figure 1
Figure 1. Comparison of direct preference optimization (DPO), identity preference optimization (IPO), and reinforcement [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Paraphrase Type Generation (PTG) workflow. The model is first fine-tuned on the Extended Typology Paraphrase [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Direct Preference Optimization (DPO) framework, adapted from Rafailov et al [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Paraphrase Type Detection pipeline. Initial training on the Quora Question Pairs (QQP) dataset teaches paraphrase [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Accuracy of paraphrase-type generation across methods. The DPO/APTY model achieves 57 % accuracy, surpassing [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Human preference rankings of model-generated paraphrases. The DPO/APTY model produces 49 % top-ranked [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Correlation between automated metrics and human quality judgments. For 120 paraphrases from four models, Pearson [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: F1 scores for paraphrase type detection (PTD) across ten atomic transformations. The PTD model excels at simpler [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Model performance on various NLP benchmarks, including Multistep Soft Reasoning (MuSR) and Big Bench Hard [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Accuracy of paraphrase-type generation by Llama-2-7B and a DPO/APTY model, based on human annotations. While [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: F1 scores for agreement of paraphrase type detection (PTD) with human annotators. The PTD model scores better, [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 25 canonical work pages

  1. [1]

    Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Rémi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello

    work page

  2. [2]

    Rahul Bhagat and Eduard Hovy. 2013. What Is a Paraphrase?Computational Linguistics39, 3 (2013), 463–472. https://doi.org/10.1162/ COLI_a_00166 arXiv:https://direct.mit.edu/coli/article-pdf/39/3/463/1801912/coli_a_00166.pdf

    work page 2013

  3. [3]

    David Chen and William Dolan. 2011. Collecting Highly Parallel Data for Paraphrase Evaluation. InProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Dekang Lin, Yuji Matsumoto, and Rada Mihalcea (Eds.). Association for Computational Linguistics, Portland, Oregon, USA, 190–200. https://aclan...

    work page 2011

  4. [4]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Jill Burstein, Christy ...

    doi:10.18653/v1/n19 2019

  5. [5]

    Dolan and Chris Brockett

    William B. Dolan and Chris Brockett. 2005. Automatically Constructing a Corpus of Sentential Paraphrases. InProceedings of the Third International Workshop on Paraphrasing (IWP2005). https://aclanthology.org/I05-5002

    work page 2005

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and others. 2024. The llama 3 herd of models.ArXiv preprintabs/2407.21783 (2024). https://arxiv.org/abs/2407. 21783

    arXiv 2024

  7. [7]

    Clémentine Fourrier, Nathan Habib, Alina Lozovskaya, Konrad Szafer, and Thomas Wolf. 2024. Open LLM Leaderboard v2. https: //huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard

    work page 2024

  8. [8]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: decoding-Enhanced Bert with Disentangled Attention. In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https: //openreview.net/forum?id=XPZIaotutsD

    work page 2021

Show all 42 references
  1. [9]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring Mathematical Problem Solving With the MATH Dataset. https://arxiv.org/abs/2103.03874

    2021 arXiv

  2. [10]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. InThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 202...

    2022

  3. [11]

    Kuan-Hao Huang, Varun Iyer, I-Hung Hsu, Anoop Kumar, Kai-Wei Chang, and Aram Galstyan. 2023. ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volu...

    2023 doi

  4. [12]

    Shankar Iyer, Nikhil Dandekar, , and Kornél Csernai. 2017. First Quora Dataset Release: Question Pairs. Accessed: 2024-12-24. https://quoradata.quora.com/First-Quora-Dataset-Release-Question-Pairs

    2017

  5. [13]

    Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial Example Generation with Syntactically Controlled Paraphrase Networks. InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan...

    2018 doi

  6. [14]

    Antònia Martí, and Maria Salamó

    Venelin Kovatchev, M. Antònia Martí, and Maria Salamó. 2018. ETPC - A Paraphrase Identification Corpus Annotated with Extended Paraphrase Typology and Negation. InProceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Nicoletta C...

    2018

  7. [15]

    2024.Content Analysis: An Introduction to Its Methodology(fourth edition ed.)

    Klaus Krippendorff. 2024.Content Analysis: An Introduction to Its Methodology(fourth edition ed.). SAGE Publications, Inc., Thousand Oaks, California. https://doi.org/10.4135/9781071878781

    2024 doi

  8. [16]

    J Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data.Biometrics33 1 (1977), 159–74. https://api.semanticscholar.org/CorpusID:11077516

    1977

  9. [17]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettle- moyer. 2020. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. InProceedings of the 58...

    2020

  10. [18]

    Zichao Li, Xin Jiang, Lifeng Shang, and Hang Li. 2018. Paraphrase Generation with Deep Reinforcement Learning. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). A...

    2018 doi

  11. [19]

    Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. InText Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74–81. https://aclanthology.org/W04-1013

    2004

  12. [20]

    Luebbers

    Christopher L. Luebbers. 2024. dpo-rhlf-paraphrase-types. Accessed: 2024-12-24. https://github.com/cluebbers/dpo-rlhf-paraphrase- types

    2024

  13. [21]

    Luebbers

    Christopher L. Luebbers. 2024. Enhancing Paraphrase Type Generation Huggingface Collection. Accessed: 2024-12-24. https: //huggingface.co/collections/cluebbers/enhancing-paraphrase-type-generation-673ca8d75dfe2ce962a48ac0

    2024

  14. [22]

    Dominik Meier, Jan Philip Wahle, Terry Lima Ruas, and Bela Gipp. 2025. Towards Human Understanding of Paraphrase Types in Large Language Models. InProceedings of the 31st International Conference on Computational Linguistics, Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend ...

    2025

  15. [23]

    Affan Hilmy Natsir, Indriana Hidayah, and Teguh Bharata Adji. 2023. Deep Learning in Paraphrase Generation: A Systematic Literature Review. In2023 IEEE 7th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE). Institute ...

    2023

  16. [24]

    Shinhyeok Oh, Hyojun Go, Hyeongdon Moon, Yunsung Lee, Myeongho Jeong, Hyun Seung Lee, and Seungtaek Choi. 2023. Evaluation of Question Generation Needs More References. InFindings of the Association for Computational Linguistics: ACL 2023, Anna Rogers, Jordan Boyd-Graber, and ...

    2023 doi

  17. [25]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Pierre Isabelle, Eugene Charniak, and Dekang Lin (Eds....

    2002

  18. [26]

    Karl Pearson. 1895. Note on Regression and Inheritance in the Case of Two Parents.Proceedings of the Royal Society of London Series I 58 (1895), 240–242

  19. [27]

    Alec Radford and Karthik Narasimhan. 2018. Improving Language Understanding by Generative Pre-Training. https://api. semanticscholar.org/CorpusID:49313245

    2018

  20. [28]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Inf...

    2023

  21. [29]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. GPQA: A Graduate-Level Google-Proof Q&A Benchmark. https://arxiv.org/abs/2311.12022

    2023 arXiv

  22. [30]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. https://arxiv.org/abs/1707.06347 Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data•15

    2017 arXiv

  23. [31]

    Lingfeng Shen, Lemao Liu, Haiyun Jiang, and Shuming Shi. 2022. On the Evaluation Metrics for Paraphrase Generation. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for...

    2022 doi

  24. [32]

    Reece Shuttleworth, Jacob Andreas, Antonio Torralba, and Pratyusha Sharma. 2024. LoRA vs Full Fine-tuning: An Illusion of Equivalence. https://arxiv.org/abs/2410.21228

    2024

  25. [33]

    Zayne Sprague, Xi Ye, Kaj Bostrom, Swarat Chaudhuri, and Greg Durrett. 2024. MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,

    2024

  26. [34]

    Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them. InFindings of the Association for Comput...

    2023 doi

  27. [35]

    https://openreview.net/forum?id=jenyYQzue1

    OpenReview.net. https://openreview.net/forum?id=jenyYQzue1

  28. [36]

    Antònia Martí, and Horacio Rodríguez

    Marta Vila, M. Antònia Martí, and Horacio Rodríguez. 2014. Is This a Paraphrase? What Kind? Paraphrase Boundaries and Typology. Open Journal of Modern Linguistics04, 01 (2014), 205–218. https://doi.org/10.4236/ojml.2014.41016

    2014

  29. [37]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. https://arxiv.org/abs/2307.09288

    2023 arXiv

  30. [38]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. 2024. MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark. InAdvances in Neural Information Processing Syste...

    2024

  31. [39]

    Jan Philip Wahle, Bela Gipp, and Terry Ruas. 2023. Paraphrase Types for Generation and Detection. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguisti...

    2023 doi

  32. [40]

    wrong". All correct paraphrases are ranked from

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction- Following Evaluation for Large Language Models. https://arxiv.org/abs/2311.07911 A Technical Information A requirements file is available on Github [20...

    2023 arXiv

  33. [41]

    Chao Zhou, Cheng Qiu, Lizhen Liang, and Daniel E. Acuna. 2025. Paraphrase Identification With Deep Learning: A Review of Datasets and Methods.IEEE Access13 (2025), 65797–65822. https://doi.org/10.1109/ACCESS.2025.3556899

    2025

  34. [2024]

    InInternational Conference on Artificial Intelligence and Statistics, 2-4 May 2024, Palau de Congressos, Valencia, Spain (Proceedings of Machine Learning Research, Vol

    A General Theoretical Paradigm to Understand Learning from Human Preferences. InInternational Conference on Artificial Intelligence and Statistics, 2-4 May 2024, Palau de Congressos, Valencia, Spain (Proceedings of Machine Learning Research, Vol. 238), Sanjoy Dasgupta, Stephan...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.