Pith. sign in

REVIEW 4 major objections 6 minor 75 references

ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ReSURE claims that adaptively down-weighting high-loss dialogues by turn depth keeps multi-turn fine-tuning robust to noisy and mixed-quality data, with positive scaling where standard fine-tuning degrades.

desk verdict A plausible, well-scoped loss-reweighting trick for multi-turn dialogue fine-tuning, but the headline scaling claim rests on Spearman correlations that cannot be computed from the three points shown. read the letter →

arxiv 2508.19996 v1 pith:PS54MJH5 submitted 2025-08-27 cs.CL

Yiming Du , Yifan Xiang , Bin Liang , Dahua Lin , Kam-Fai Wong , Fei Tan This is my paper · ORCID
classification cs.CL
keywords multi-turndialoguesupervisedfine-tuninglossreweightingnoisysupervisionWelfordonlinestatisticsinstructiontuningdataqualityLLMrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that multi-turn dialogue fine-tuning can absorb low-quality and off-task training data without any pre-filtering step, by continuously estimating which dialogues are unreliable and softly damping their loss. The claim is that supervision mistakes in early turns propagate into later turns, so a turn-depth-aware reweighting scheme should reduce that cascade and keep optimization positive as data scales. The authors test this on high-, normal-, and low-quality dialogue sets and on mixtures that add math problems, reporting that their method maintains or improves response quality while standard tuning degrades. If the claim holds, large mixed-quality corpora become usable for dialogue instruction tuning without manual curation.

What carries the argument

The central object is a per-dialogue-depth online estimate of the mean and standard deviation of whole-dialogue loss, maintained with Welford's algorithm. For each group of dialogues with the same maximum supervised turn index, the running statistics define a threshold $\tau^{(b)}_s = \mu^{(b)}_s + \alpha \sigma^{(b)}_s$; losses above it are softly scaled down by $\exp(-(\ell_s-\tau^{(b)}_s)/\tau^{(b)}_s)$, with a percentile floor. This mechanism is what lets the training loop detect unreliable supervision dynamically and regulate trust by dialogue depth without discarding samples.

What would settle it

Train on a synthetic multi-turn corpus with noise injected only into specified later turns, and compare per-turn response accuracy of ReSURE versus ordinary fine-tuning; if final-turn accuracy is not higher, the claimed early-turn error propagation mechanism is not what drives the gains.

Watch

Extended reading notes

Core claim

ReSURE reweights every training dialogue by comparing its whole-dialogue supervised cross-entropy loss against running statistics maintained separately for each dialogue-depth group. After a warm-up phase, a dialogue whose loss exceeds the group mean by more than one standard deviation is labeled unreliable and its gradient contribution is decayed exponentially, with a floor so that hard examples are not zeroed out; such dialogues are also excluded from the statistics update, so outliers do not contaminate the baseline. The paper reports that this procedure improves or preserves scores on in-domain held-out data, MT-Bench, and MT-Bench-Ext across six datasets, and yields positive Spearman correlations between dataset complexity and performance (0.21, 1.00, 0.80) while Vicuna-Tuning degrades. It also reports that mixing in GSM8K math data does not collapse dialogue quality, and that combining ReSURE with static pre-filtering improves over either alone.

Load-bearing premise

The load-bearing premise is that a whole-dialogue loss more than one standard deviation above its turn-depth group mean marks unreliable supervision rather than a hard but useful example, and that down-weighting the whole dialogue can repair errors originating in early turns.

Editorial extensions

If this is right

  • Adding larger or noisier multi-turn datasets to fine-tuning should no longer systematically hurt response quality; ReSURE's positive Spearman correlations (0.21, 1.00, 0.80) are the paper's evidence for this.
  • Off-task data mixed into dialogue training, such as GSM8K, can be absorbed without collapsing conversational performance, and the paper reports a small improvement in math accuracy as well.
  • Static pre-filtering and adaptive reweighting address different weaknesses, since ReSURE plus pre-filtering outperforms either approach alone.
  • The mechanism transfers across the four instruction-tuned models tested in the paper, so it is not tied to one architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the method reweights a whole dialogue on one loss value, it cannot by itself tell which turn inside that dialogue is unreliable; a natural extension is to run the same statistics per turn and test whether the gains grow.
  • The Spearman correlations are computed over a small number of mixture points, so the claim of positive scaling regardless of data quality would be stronger with many more dataset sizes and noise levels.
  • Down-weighting high-loss samples may also down-weight genuinely hard but valuable examples, so combining ReSURE with a difficulty-aware sampling policy is a plausible next test.
  • The same loss-calibration idea could apply to other supervised fine-tuning settings where noise arrives in structured groups, such as by task or domain, though the paper only tests multi-turn dialogue.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes ReSURE, an online loss-reweighting method for fine-tuning multi-turn dialogue models. The method groups training dialogues by their maximum turn depth, maintains online per-group loss statistics with Welford's algorithm, flags samples whose whole-dialogue loss exceeds a turn-group-specific threshold, and softly down-weights those samples. Experiments compare ReSURE against Vicuna-Tuning, Baize, and ChatGLM3 on in-domain, MT-Bench, and MT-Bench-Ext, using mixtures of datasets labeled high, normal, and low quality. The paper reports that ReSURE consistently improves or stabilizes performance as more mixed-quality data is added, and claims positive Spearman correlations (0.21, 1.00, 0.80) between dataset complexity and performance across the three benchmarks, interpreting this as evidence that ReSURE can leverage increasing amounts of mixed-quality data without degradation.

Significance. If the empirical claims hold, ReSURE is a simple, computationally light alternative to static data filtering that could easily be dropped into existing instruction-tuning pipelines, and the authors do provide public code. The turn-depth-grouped loss statistics are a reasonable heuristic, and the soft-reweighting design is more flexible than hard filtering. However, the headline scaling claim is central to the paper's contribution and is currently not supported by the reported statistics: the Spearman values in Table 4 are inconsistent with the three mixture levels described in the experiment. The method's motivation and implementation also diverge (whole-dialogue reweighting versus per-turn loss reweighting), and the empirical evaluation is thin (single seed, three-point scaling curves, a 20-dialogue human evaluation). These issues materially affect the paper's validity and require substantial revision rather than minor copy edits.

major comments (4)
  1. [Section 4.4.2 and Table 4] The reported Spearman correlations (ReSURE: 0.211, 1.000, 0.800) cannot be obtained from the three mixture levels (H, H+N, H+N+L) described in Section 4.4.2 and Figure 2. For three untied points, Spearman's rho can only take values in {-1, -0.5, 0.5, 1}; the values 0.211 and 0.800 are impossible without additional data points or a different definition of dataset complexity. Even allowing ties among the three performance values, no tie pattern reproduces these coefficients. This makes the abstract's scaling claim unsupported. Please specify exactly how many points were used, provide the underlying data, and recompute the correlations; if the correlations are over individual datasets rather than mixture levels, clarify that. The unresolved "Figure X" placeholder in this section also prevents verification of the scaling curves.
  2. [Section 3.2 and 3.3, Equations (5)-(7)] The method computes turn-group statistics but applies a single weight to the whole-dialogue loss ℓ_s. The threshold in Equation (5) uses the maximum turn-group index b of the sample, and Equation (6) down-weights the entire sample loss. This is whole-dialogue reweighting, not per-turn loss reweighting. The paper's motivation emphasizes early-turn error propagation, but the implementation cannot isolate which turn caused the high loss, so the claimed mechanism is not directly realized. Either change the method to compute and reweight per-turn losses, or revise the framing to describe ReSURE as down-weighting entire dialogues whose aggregate loss is anomalous within a turn-depth group.
  3. [Section 4.4.1 and Table 1] The sentence "ReSURE consistently outperforms the base model by 6.11%, 9.82%, and 2.86% on the in-domain benchmark, MT-Bench, and MT-Bench-Ext, respectively" is misleading. These percentages are the maximum gains, obtained on ShareGPT, not representative of all datasets. Table 1 shows negative relative changes for M2Lingual (-0.56% on In-Domain-Test), WildChat (+0.88% only; -0.56% on In-Domain? actually +0.88% on In-Domain, +1.12% on MT-Bench, +1.20% on Ext), and other cases where ReSURE is below the base model. Please report the mean and per-dataset relative changes, and avoid claiming uniform superiority without statistical support.
  4. [Section 4.4.2 and Figure 2] The scaling conclusion rests on three points per curve, a single random seed (seed=42), and no error bars or confidence intervals. Small evaluation noise at any level could change the sign of the trend, and the 0.211 in-domain correlation is close to zero. The paper should include multiple seeds or bootstrap intervals, and the discussion should acknowledge the fragility of three-point trend claims. This is particularly important because the positive-scaling claim is the paper's main advertised contribution.
minor comments (6)
  1. [Section 3.1, Equation (4)] Equation (4) uses "M2(b)_s" which is not defined; the text defines SSD(b)_s and calls it the Sum of Squared Deviations. Please unify the notation with Welford's standard M2 term, or consistently use SSD throughout.
  2. [Section 3.1, Equation (2)] The subscripts in Equation (2) are inconsistent: the left-hand side and update use µ(b)_s and µ(b)_s-1, while the right-hand side reference µ(b)_t-1 and t(b). Clarify the indexing so the recursion is unambiguous.
  3. [Section 4.4.2] The reference "Figure X" is an unresolved placeholder; the associated claims about Vicuna-Tuning dropping by 0.75 on MT-Bench and 0.90 on MT-Bench-Ext cannot be checked. Include the figure or cite the correct existing figure.
  4. [Appendix C] The human evaluation uses only 20 dialogues (10 each from MT-Bench and MT-Bench-Ext). With three annotators, the claim that ReSURE outperforms baselines "across all dimensions" needs inter-annotator agreement statistics and confidence intervals, or the claim should be softened.
  5. [Figure 3 caption] The caption contains a typo: "Vicuna-T uning" should be "Vicuna-Tuning".
  6. [References] The reference list contains duplicates: Wu et al. 2023a and 2023b are the same paper, and the same for other entries. Please deduplicate and use consistent citation labels.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: ReSURE is an empirical reweighting heuristic whose losses and weights depend only on training statistics, not on the benchmark outcomes being claimed.

full rationale

ReSURE is not a derived first-principles result; it is an adaptive reweighting heuristic. The objective in Eq. (1) uses a sample weight w_s computed from Eqs. (2)-(6), where the threshold tau = mu + alpha*sigma and the soft weight exp(-(l_s - tau)/tau) depend only on the running mean and standard deviation of the training loss within a turn group, plus the hyperparameters alpha=1.0 and a percentile floor. None of these quantities is defined in terms of the downstream evaluation scores (In-Domain-Test, MT-Bench, MT-Bench-Ext). The reported Spearman correlations in Table 4 are descriptive summaries of the observed experimental results, not predictions emitted by the method's equations; thus they cannot be circular. The method's design choices are justified by external robust-optimization literature (Ren et al., 2018; Zhang and Sabuncu, 2020), and the authors' own prior works appear only in background or related work, never as a load-bearing justification for the core mechanism. Welford's algorithm is an external, well-known numerical routine cited to Welford (1962). No equation in the paper reduces to its input by construction, and no fitted parameter is renamed as a prediction. (Two reporting defects lie outside the circularity scope: the Spearman values 0.211 and 0.800 are mathematically impossible for the three mixture levels described in Section 4.4.2, and Section 4.4.2 references an unresolved 'Figure X'; these are internal-consistency or documentation issues, not circularity.)

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the assumption that loss outliers indicate unreliable supervision and on the dataset quality ordering used to create mixed training sets. The hyperparameters alpha, the percentile floor, and the warm-up size were tuned on validation data and are free parameters.

free parameters (3)
  • anomaly factor alpha = 1.0
    Tuned on validation sets; the paper states that 1.0 balances sensitivity to moderate deviations with stability, while more aggressive or moderate cutoffs underperform.
  • weight floor percentile epsilon = 5th percentile of batch weight distribution
    Tuned on validation sets; 1st percentile over-penalizes valid hard examples, while 10th or 25th percentiles reduce noise suppression efficiency.
  • warm-up sample count = 640
    Used to initialize baseline mean and variance statistics; no sensitivity analysis is reported for this choice.
assumptions (3)
  • domain assumption High-loss dialogues are predominantly unreliable supervision, not hard-but-useful examples.
    The threshold in Equation (5) assumes that loss outliers within a turn-depth group are noise; this conflates noise with difficulty and is not validated.
  • domain assumption Per-turn supervision error propagation can be mitigated by reweighting whole-dialogue losses.
    The paper motivates turn-level error propagation, but the implementation uses a single per-dialogue loss, so this causal link is assumed rather than modeled.
  • domain assumption Dataset quality labels derived from GPT-4o and reward-model scores are correct.
    Section 4.1 uses these labels to construct mixed-quality datasets; if the quality ordering is wrong, the mixed-quality experiments lose meaning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning." pith.science (2026). https://pith.science/paper/PS54MJH5

@misc{pith2026250819996,
  author       = {Pith},
  title        = {Pith review of: ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PS54MJH5}},
  note         = {Machine review of arXiv:2508.19996}
}
read the original abstract

Fine-tuning multi-turn dialogue systems requires high-quality supervision but often suffers from degraded performance when exposed to low-quality data. Supervision errors in early turns can propagate across subsequent turns, undermining coherence and response quality. Existing methods typically address data quality via static prefiltering, which decouples quality control from training and fails to mitigate turn-level error propagation. In this context, we propose ReSURE (Regularizing Supervision UnREliability), an adaptive learning method that dynamically down-weights unreliable supervision without explicit filtering. ReSURE estimates per-turn loss distributions using Welford's online statistics and reweights sample losses on the fly accordingly. Experiments on both single-source and mixed-quality datasets show improved stability and response quality. Notably, ReSURE enjoys positive Spearman correlations (0.21 ~ 1.0 across multiple benchmarks) between response scores and number of samples regardless of data quality, which potentially paves the way for utilizing large-scale data effectively. Code is publicly available at https://github.com/Elvin-Yiming-Du/ReSURE_Multi_Turn_Training.

Figures

Figures reproduced from arXiv: 2508.19996 by the authors.

Figure 1
Figure 1. Overview of Training Paradigms: Traditional Fine-tuning, Pre-filtering, and ReSURE. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Performance scaling with Hierarchical Data Integration (H, H+N, H+N+L): (a) In-Domain-Test Perfor [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison between Vicuna￾Tuning and ReSURE on ChatAlpaca and ChatAl￾paca+GSM8K across three evaluation benchmarks: In￾Domain-Test, MT-Bench, and MT-Bench-Ext. Model In-Domain￾Test MT-Bench MT-Bench￾Ext VT -1.000 -1.000 -1.000 Baize -1.000 0.000 -0.400 ChatGLM3 -1.000 0.211 -0.800 ReSURE 0.211 1.000 0.800 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Win Rates of ReSURE vs. VT on MT-Bench and MT-Bench-Ext. Metric ReSURE w/o Welford ∆ (%) In-Domain-Test 8.26 8.20 -0.73% MT-Bench 7.29 7.19 -1.37% MT-Bench-Ext 6.76 6.70 -0.89% [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Case study. spite this variability, ReSURE demonstrates robust improvements across both model families, high￾lighting its effectiveness and broad applicability. ReSURE enhances response performance by effectively skipping low-quality data. To bet￾ter understand the imp…
Figure 6
Figure 6. Figure 6: Case study. Dimension Score Description Faithfulness 1 Completely irrelevant or ignores prior context, leading to a fundamentally incorrect answer. 2 Contains substantial irrelevant or contradictory content, but barely addresses the request. 3 Accurately addresses the …
Figure 7
Figure 7. Figure 7: Prompts of the evaluation of connection. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Prompts of the evaluation of quality [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Prompts of the evaluation of information density. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Prompts of the evaluation of friendliness. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

75 extracted references · 18 canonical work pages

  1. [1]

    Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, and Wanli Ouyang. 2024. https://doi.org/10.18653/v1/2024.acl-long.401 MT -bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues . In Proceedings of the 62nd Annual Meeting of the Association for Com...

    doi:10.18653/v1/2024.acl-long.401 2024

  2. [2]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862

    arXiv 2022

  3. [3]

    Ning Bian, Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun, and Ben He. 2023. Chatalpaca: A multi-turn dialogue corpus based on alpaca instructions. https://github.com/cascip/ChatAlpaca

    work page 2023

  4. [4]

    Yihan Cao, Yanbin Kang, Chi Wang, and Lichao Sun. 2023. Instruction mining: Instruction data selection for tuning large language models. arXiv preprint arXiv:2307.06290

    arXiv 2023

  5. [5]

    Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. 2023. Alpagasus: Training a better alpaca with fewer data. arXiv preprint arXiv:2307.08701

    arXiv 2023

  6. [6]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ https://lmsys.org/blog/2023-03-30-vicuna/. Accessed: 2025-02-10

    work page 2023

  7. [7]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems, 2021. URL https://arxiv. org/abs/2110.14168, 9

    arXiv 2021

  8. [8]

    OpenAssistant Contributors. 2023. Openassistant conversations - democratizing large language model alignment. https://arxiv.org/abs/2304.07327. Accessed: 2023-04-17

    arXiv 2023

Show all 75 references
  1. [9]

    Nina Dethlefs, Helen Hastie, Heriberto Cuay\' a huitl, Yanchao Yu, Verena Rieser, and Oliver Lemon. 2016. https://doi.org/10.1016/j.csl.2015.11.001 Information density and overlap in spoken dialogue . Comput. Speech Lang., 37(C):82–97

    2016 doi

  2. [10]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36

    2024

  3. [11]

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233

    2023 arXiv

  4. [12]

    Qianlong Du, Chengqing Zong, and Jiajun Zhang. 2023. Mods: Model-oriented data selection for instruction tuning. arXiv preprint arXiv:2311.15653

    2023 arXiv

  5. [13]

    Yiming Du, Bingbing Wang, Yang He, Bin Liang, Baojun Wang, Zhongyang Li, Lin Gui, Jeff Z Pan, Ruifeng Xu, and Kam-Fai Wong. 2025. Bridging the long-term gap: A memory-active policy for multi-session task-oriented dialogue. arXiv preprint arXiv:2505.20231

    2025 arXiv

  6. [14]

    Yann Dubois, Bal \'a zs Galambosi, Percy Liang, and Tatsunori B Hashimoto. 2024. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475

    2024 arXiv

  7. [15]

    Hashimoto

    Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. http://arxiv.org/abs/2305.14387 Alpacafarm: A simulation framework for methods that learn from human feedback

    2023 arXiv

  8. [16]

    Andrey A Efanov, Sergey A Ivliev, and Alexey G Shagraev. 2021. Welford’s algorithm for weighted statistics. In 2021 3rd International Youth Conference on Radio Electronics, Electrical and Power Engineering (REEPE), pages 1--5. IEEE

    2021

  9. [17]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie H...

    2024 arXiv

  10. [18]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  11. [19]

    Jiawei Gu, Zacc Yang, Chuanghao Ding, Rui Zhao, and Fei Tan. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.903 CMR scaling law: Predicting critical mixture ratios for continual pre-training of language models . In Proceedings of the 2024 Conference on Empirical Methods in ...

    2024 doi

  12. [20]

    Peter Hase, Mohit Bansal, Peter Clark, and Sarah Wiegreffe. 2024. https://doi.org/10.18653/v1/2024.acl-long.378 The unreasonable effectiveness of easy training data for hard tasks . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol...

    2024 doi

  13. [21]

    Alex Havrilla. 2023. https://doi.org/10.57967/hf/1428 synthetic-instruct-gptj-pairwise (revision cc92d8d)

    2023 doi

  14. [22]

    Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543

    2021 arXiv

  15. [23]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654

    2020 arXiv

  16. [24]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  17. [25]

    Hanjiang Hu, Alexander Robey, and Changliu Liu. 2025. Steering dialogue dynamics for robustness against multi-turn jailbreaking attacks. arXiv preprint arXiv:2503.00187

    2025

  18. [26]

    Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Lee. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.319 LLM -adapters: An adapter family for parameter-efficient fine-tuning of large language models . In Proceedings of ...

    2023 doi

  19. [27]

    o pf, Yannic Kilcher, Dimitri von R \

    Andreas K \"o pf, Yannic Kilcher, Dimitri von R \"u tte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Rich \'a rd Nagyfi, et al. 2024. Openassistant conversations-democratizing large language model alignment. Advances in Neura...

    2024

  20. [28]

    Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, and Kam-Fai Wong. 2024 a . Mt-eval: A multi-turn capabilities evaluation benchmark for large language models. arXiv preprint arXiv:2401.16745

    2024 arXiv

  21. [29]

    Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, and Kam-Fai Wong. 2024 b . https://doi.org/10.18653/v1/2024.emnlp-main.1124 MT -eval: A multi-turn capabilities evaluation benchmark for large language models . In Proceeding...

    2024 doi

  22. [30]

    Yutian Lei, Luping Ji, and Pei Liu. 2025. Mining in-distribution attributes in outliers for out-of-distribution detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18181--18188

    2025

  23. [31]

    Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. 2024 a . https://doi.org/10.18653/v1/2024.naacl-long.421 From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning ...

    2024 doi

  24. [32]

    Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis. 2023 a . Self-alignment with instruction backtranslation. arXiv preprint arXiv:2308.06259

    2023 arXiv

  25. [33]

    Hashimoto

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023 b . Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval

    2023

  26. [34]

    Yunshui Li, Binyuan Hui, Xiaobo Xia, Jiaxi Yang, Min Yang, Lei Zhang, Shuzheng Si, Ling-Hao Chen, Junhao Liu, Tongliang Liu, Fei Huang, and Yongbin Li. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.252 One-shot learning as instruction data prospector for large language mo...

    2024 doi

  27. [35]

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024 a . Dora: Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09353

    2024 arXiv

  28. [36]

    Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2024 b . https://openreview.net/forum?id=BTKAeLqLMw What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning . In The Twelfth International Conference on Learning Re...

    2024

  29. [37]

    Jinghui Lu, Dongsheng Zhu, Weidong Han, Rui Zhao, Brian Mac Namee, and Fei Tan. 2023 a . https://doi.org/10.18653/v1/2023.acl-long.128 What makes pre-trained language models better zero-shot learners? In Proceedings of the 61st Annual Meeting of the Association for Computation...

    2023 doi

  30. [38]

    Junru Lu, Siyu An, Mingbao Lin, Gabriele Pergola, Yulan He, Di Yin, Xing Sun, and Yunsheng Wu. 2023 b . Memochat: Tuning llms to use memos for consistent long-range open-domain conversation. arXiv preprint arXiv:2308.08239

    2023 arXiv

  31. [39]

    Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, and Chang Zhou. 2023 c . \# instag: Instruction tagging for diversity and complexity analysis. arXiv preprint arXiv:2308.07074

    2023 arXiv

  32. [40]

    Rishabh Maheshwary, Vikas Yadav, Hoang Nguyen, Khyati Mahajan, and Sathwik Tejaswi Madhusudhan. 2024. M2lingual: Enhancing multilingual, multi-turn instruction alignment in large language models. arXiv preprint arXiv:2406.16783

    2024 arXiv

  33. [41]

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2024. Pissa: Principal singular values and singular vectors adaptation of large language models. arXiv preprint arXiv:2404.02948

    2024 arXiv

  34. [42]

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2021...

    2021

  35. [43]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . arXiv preprint arXiv:2303.08774

    2023 arXiv

  36. [44]

    OpenAssistant . 2023. Openassistant/reward-model-deberta-v3-large-v2. https://huggingface.co/OpenAssistant/reward-model-deberta-v3-large-v2. Reward model trained from human feedback to predict which generated answer is better judged by a human, given a question

    2023

  37. [45]

    Jiao Ou, Jiayu Wu, Che Liu, Fuzheng Zhang, Di Zhang, and Kun Gai. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.964 Inductive-deductive strategy reuse for multi-turn instructional dialogues . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Pr...

    2024 doi

  38. [46]

    Nicole M Radziwill and Morgan C Benton. 2017. Evaluating quality of chatbots and intelligent conversational agents. arXiv preprint arXiv:1704.04579

    2017 arXiv

  39. [47]

    Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In International conference on machine learning, pages 4334--4343. PMLR

    2018

  40. [48]

    RyokoAI. 2023. Sharegpt. https://huggingface.co/datasets/RyokoAI/ShareGPT52K

    2023

  41. [49]

    Lior Shani, Aviv Rosenberg, Asaf Cassel, Oran Lang, Daniele Calandriello, Avital Zipori, Hila Noga, Orgad Keller, Bilal Piot, Idan Szpektor, et al. 2024. Multi-turn reinforcement learning from preference human feedback. arXiv preprint arXiv:2405.14655

    2024 arXiv

  42. [50]

    Shivalika Singh, Freddie Vargus, Daniel Dsouza, B \"o rje F Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, et al. 2024. Aya dataset: An open-access collection for multilingual instruction tuning. arXiv preprint arXiv:...

    2024 arXiv

  43. [51]

    Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano

    Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2020. Learning to summarize from human feedback. In NeurIPS

    2020

  44. [52]

    Yuchong Sun, Che Liu, Kun Zhou, Jinwen Huang, Ruihua Song, Xin Zhao, Fuzheng Zhang, Di Zhang, and Kun Gai. 2024. https://doi.org/10.18653/v1/2024.acl-long.525 Parrot: Enhancing multi-turn instruction following for large language models . In Proceedings of the 62nd Annual Meeti...

    2024 doi

  45. [53]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  46. [54]

    Qwen Team. 2024. https://arxiv.org/abs/2412.15115 Qwen2.5 technical report . arXiv preprint arXiv:2412.15115

    2024 arXiv

  47. [55]

    Jiahao Wang, Bolin Zhang, Qianlong Du, Jiajun Zhang, and Dianhui Chu. 2024 a . A survey on data selection for llm instruction tuning. arXiv preprint arXiv:2402.05123

    2024 arXiv

  48. [56]

    Shiqi Wang, Zhengze Zhang, Rui Zhao, Fei Tan, and Nguyen Cam-Tu. 2024 b . https://doi.org/10.18653/v1/2024.findings-emnlp.115 Reward difference optimization for sample reweighting in offline RLHF . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages...

    2024 doi

  49. [57]

    Zhaowei Wang, Wei Fan, Qing Zong, Hongming Zhang, Sehyun Choi, Tianqing Fang, Xin Liu, Yangqiu Song, Ginny Y Wong, and Simon See. 2024 c . Absinstruct: Eliciting abstraction ability from llms through explanation tuning with plausibility estimation. arXiv preprint arXiv:2402.10646

    2024 arXiv

  50. [58]

    Lai Wei, Zihao Jiang, Weiran Huang, and Lichao Sun. 2023. Instructiongpt-4: A 200-instruction paradigm for fine-tuning minigpt-4. arXiv preprint arXiv:2308.12067

    2023 arXiv

  51. [59]

    B. P. Welford. 1962. Note on a method for calculating corrected sums of squares and products. Technometrics, 4(3):419--420

    1962

  52. [60]

    Chuhan Wu, Fangzhao Wu, Tao Qi, Yongfeng Huang, and Xing Xie. 2022. Noisytune: A little noise can help you finetune pretrained language models better. arXiv preprint arXiv:2202.12024

    2022 arXiv

  53. [61]

    Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Yanwei Fu, Qin Liu, et al. 2025. Reasoning or memorization? unreliable results of reinforcement learning due to data contamination. arXiv preprint arXiv:2507.10532

    2025

  54. [63]

    Shengguang Wu, Keming Lu, Benfeng Xu, Junyang Lin, Qi Su, and Chang Zhou. 2023 b . Self-evolved diverse data sampling for efficient instruction tuning. arXiv preprint arXiv:2311.08182

    2023 arXiv

  55. [64]

    Heng-Da Xu, Xian-Ling Mao, Puhai Yang, Fanshu Sun, and Heyan Huang. 2024. https://doi.org/10.18653/v1/2024.acl-long.152 Rethinking task-oriented dialogue systems: From complex modularity to zero-shot autonomous agent . In Proceedings of the 62nd Annual Meeting of the Associati...

    2024 doi

  56. [65]

    Yang Xu, Yongqiang Yao, Yufan Huang, Mengnan Qi, Maoquan Wang, Bin Gu, and Neel Sundaresan. 2023. Variety and quality over quantity: Towards versatile instruction curation. arXiv preprint arXiv:2312.11508

    2023 arXiv

  57. [66]

    Zihao Yi, Jiarui Ouyang, Yuwen Liu, Tianhao Liao, Zhe Xu, and Ying Shen. 2024. A survey on recent advances in llm-based multi-turn dialogue systems. arXiv preprint arXiv:2402.18013

    2024 arXiv

  58. [67]

    Shaoxiong Zhan, Yanlin Lai, Ziyu Lu, Dahua Lin, Ziqing Yang, and Fei Tan. 2025. Mathsmith: Towards extremely hard mathematical reasoning by forging synthetic problems with a reinforced policy. arXiv preprint arXiv:2508.05592

    2025

  59. [68]

    Hengyuan Zhang, Yanru Wu, Dawei Li, Sak Yang, Rui Zhao, Yong Jiang, and Fei Tan. 2024. https://doi.org/10.18653/v1/2024.findings-acl.445 Balancing speciality and versatility: a coarse to fine framework for supervised fine-tuning large language model . In Findings of the Associ...

    2024 doi

  60. [69]

    Zhengze Zhang, Shiqi Wang, Yiqun Shen, Simin Guo, Dahua Lin, Xiaoliang Wang, Nguyen Cam-Tu, and Fei Tan. 2025. https://doi.org/10.18653/v1/2025.findings-acl.796 da DPO : Distribution-aware DPO for distilling conversational abilities . In Findings of the Association for Computa...

    2025 doi

  61. [70]

    Zhilu Zhang and Mert R Sabuncu. 2020. Generalized cross entropy loss for training deep neural networks with noisy labels. In NeurIPS

    2020

  62. [71]

    Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, and Yuandong Tian. 2024 a . Galore: Memory-efficient llm training by gradient low-rank projection. arXiv preprint arXiv:2403.03507

    2024 arXiv

  63. [72]

    Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024 b . https://openreview.net/forum?id=Bl8u7ZRlbM Wildchat: 1m chatgpt interaction logs in the wild . In The Twelfth International Conference on Learning Representations

    2024

  64. [73]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595--46623

    2023

  65. [74]

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2024. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36

    2024

  66. [75]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  67. [76]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.