Pith. sign in

REVIEW 4 major objections 7 minor 47 references

MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Simulated teacher-student dialogues, converted into fine-tuning data, lift LLM scores on reasoning and coding benchmarks.

desk verdict A plausible but unverified data-augmentation recipe; the post-hoc MCQ claim undermines the strongest headline, but the core idea deserves a serious look if artifacts are released. read the letter →

arxiv 2506.02689 v2 pith:PXXQFF4H submitted 2025-06-03 cs.CL

Liang Yue , Yihong Tang , Kehai Chen , Jie Liu , Min Zhang This is my paper
classification cs.CL
keywords multi-agentLLMsimulationinstructionfine-tuningdataaugmentationsimulatedclassroomteachingBOOST-QAdatasetreasoningbenchmarksmultiple-choiceaccuracyerrorcorrectionmoduleanalogicalretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that instruction fine-tuning data can be made more learnable by wrapping original question-answer pairs in simulated teacher-student dialogues. It introduces MASTER, a multi-agent framework that stages three pedagogical scenarios—error correction, debate, and analogical problem solving—and concatenates the resulting agent utterances into ShareGPT-format samples. The resulting dataset, BOOST-QA, is used to fine-tune LLaMA3-8B, Qwen2.5-7B, and Mistral-7B with only 19,000 samples. Across benchmarks the augmented data generally beats the original data and several baselines, with the largest reported gain on multiple-choice tasks: 31.46% on MATHQA and an average of 15.50% across eight such tests. The paper concludes that multi-scenario simulated teaching, not any single scenario, is what drives the improvement.

What carries the argument

The load-bearing mechanism is MACLASS, a multi-agent classroom simulator. Three modules generate the augmented dialogues: the Error Correction Module has a Qwen2.5-0.5B student produce imperfect answers and a Qwen2.5-14B teacher correct them before the student revises; the Debate Module has two 7B student agents argue and a 14B summarizer produce a final answer; and the Analogical Retrieval Module uses cosine similarity on MiniLM embeddings to pair each question with a similar one for a second-round solve. The agent utterances are concatenated into ShareGPT format (a multi-turn conversation transcript), so each training sample is a dialogue rather than a single question-answer pair.

What would settle it

Inspect a random sample of BOOST-QA error-correction dialogues and annotate how many contain an uncorrected or incorrectly corrected student error; then fine-tune on the filtered subset with those dialogues removed. If the benchmark gains disappear or shrink substantially, the claim that the error-correction dialogues drive the improvement would be falsified; if they persist, the gain would instead be attributable to the other scenarios or to format alone.

Watch

Extended reading notes

Core claim

Stated in the paper's own terms, the central discovery is that simulating classroom interactions among LLM agents with different cognitive levels—a deliberately fallible student, a correcting teacher, debating students, and analogous-question retrieval—transforms ordinary QA data into instruction data that base models learn more effectively from. Fine-tuning on BOOST-QA improves reasoning, coding, and multiple-choice performance over fine-tuning on the original data, and the improvement is especially pronounced on complex multiple-choice benchmarks. The paper attributes this to the dialogues making the data more learnable, to gradient smoothing from structured noise, and to the model internalizing longer reasoning chains.

Load-bearing premise

The pipeline assumes the 14B teacher model reliably catches and corrects every error made by the deliberately error-prone 0.5B student, with no filtering, human validation, or agreement check to remove residuals.

Editorial extensions

If this is right

  • If the central claim holds, any existing QA or instruction dataset can be upgraded to a multi-turn teaching dialogue without collecting new human data, lowering the cost of high-quality fine-tuning data.
  • Models fine-tuned on BOOST-QA generate longer reasoning chains, which the paper ties to better generalization on complex multiple-choice problems.
  • No single teaching scenario suffices; the three scenarios are complementary and must be combined for the full gain.
  • The 19,000-sample BOOST-QA improves three different base models (LLaMA-3-8B, Qwen2.5-7B, Mistral-7B), suggesting the effect is not tied to one architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit: because the three source datasets are math, code, and general chat, the same pipeline may transfer to specialized domains such as medical or legal QA, where human annotation is expensive.
  • The reported gains on multiple-choice tasks could partly reflect longer output tokens and chain-of-thought style responses rather than the pedagogical content per se; a control that matches response length would isolate the teaching-dialogue effect.
  • A testable extension would be to replace the unfiltered 14B-teacher correction with a verifier or ensemble, since the current pipeline has no guard against uncorrected student errors.
  • Because the method produces conversations rather than single steps, it may interact with context-length limits and inference cost; that trade-off is not quantified in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces MASTER, a multi-agent simulated teaching framework that converts existing QA instruction datasets into multi-turn teacher-student dialogues across three pedagogical scenarios: error correction, debate, and analogical reasoning. The resulting dataset, BOOST-QA, is constructed from 19K samples drawn from Orca-Math-200K, ProcQA, and OpenHermes2.5, and is used to fine-tune three base models (LLaMA-3-8B, Qwen2.5-7B, Mistral-7B) with LoRA. The authors report that BOOST-QA improves performance over the original data on several benchmarks and outperforms four baselines (RandomAug, SpellingAug, TAGCOS, CoT Collection), with particularly large gains on a set of eight multiple-choice benchmarks, peaking at a 31.46% improvement. The central claim is that multi-agent simulated teaching dialogues make instruction-tuning data more learnable and improve reasoning, coding, and multiple-choice accuracy.

Significance. If the reported results are robust, the MASTER framework offers a simple and general recipe for data augmentation in instruction tuning, combining pedagogical principles with multi-agent LLM interactions. The paper includes useful ablations showing that all three modules contribute to the final performance and compares against several established baselines. However, the significance is currently limited by the lack of uncertainty quantification, the post-hoc selection of evaluation benchmarks for the headline multiple-choice claim, and the absence of quality analysis for the synthetically generated dialogues. These issues must be addressed before the empirical claims can be considered reliable.

major comments (4)
  1. [Section 4.3, Figure 2] The paper's headline claim of 'consistent improvements exceeding 5%' on multiple-choice tasks is based on benchmarks that were added after observing the phenomenon, as stated in the text: 'Upon observing this phenomenon, we incorporated additional multiple-choice benchmark tests.' This is post-hoc selection on the dependent variable, so the reported average improvement of 15.50% and the maximum of 31.46% are not unbiased estimates of the method's typical effect on MCQ tasks. Furthermore, Table 2 provides a direct counterexample under the same evaluation protocol: Mistral-7B fine-tuned on BOOST-QA scores 49.10 on SCI-Q, a 21.40-point drop from the 70.50 achieved with Ori-Data. The authors should report the full set of multiple-choice benchmarks attempted, including any with negative results, and clarify whether the reported eight benchmarks are the complete set or a selected subset.
  2. [Tables 1 and 2] All results are reported as single numbers with no error bars, no repeated seeds, and no significance tests. This is problematic because several differences are extreme: Qwen2.5-7B ARC rises from 20.90 to 68.52 and MMLU from 24.05 to 50.12, while Mistral-7B SCI-Q drops from 70.50 to 49.10. Without variance estimates, the reader cannot determine whether these differences reflect the method's effect or training noise. Please report mean and standard deviation over at least three independent fine-tuning runs, or provide code, data, and random seeds to allow exact reproduction.
  3. [Section 3.3, Error Correction Module] The error correction module deliberately prompts a 0.5B student model to produce erroneous answers and relies exclusively on a 14B teacher model to correct them, with no filtering, agreement measure, or human validation. The paper provides no analysis of the correction success rate or of the factual and pedagogical quality of the generated dialogues. Since the method's benefit depends on the correctness of the teacher's corrections, the authors should provide evidence (e.g., a sample of generated dialogues, automated answer-consistency checks, or a small human evaluation) that erroneous corrections do not propagate into the training data.
  4. [Section 4.2, Baselines] The baseline construction is difficult to interpret. The text states that TAGCOS was used to select 5K/5K/9K samples from the original datasets, and that 'RandomAug, SpellingAug, CoT Collection, and our MASTER were then applied to randomly augment an equal amount of original data.' This suggests that the TAGCOS baseline uses a selected subset, while the other methods are applied to randomly sampled data of equal size, but the details are unclear. In particular, CoT Collection is a pre-existing external dataset rather than an augmentation of the original data; its sample size, selection procedure, and prompt format need to be specified to make the comparison fair. Please clarify the exact construction of each baseline and ensure that the training data amounts are matched.
minor comments (7)
  1. [Introduction] The Introduction contains a duplicated sentence: 'For example, Yoo et al. combined subsets of training examples and embedded them into model prompts...' appears twice.
  2. [Section 2 heading] 'Relate Work' should be 'Related Work'.
  3. [Figure 2] Figure 2 does not specify which base model(s) were used, the number of runs, or whether the reported accuracies are single-run results; please add this information.
  4. [Table 4] The average reported for 'Ori' in Table 4 (37.52) differs from the LLaMA3 Ori average in Tables 1-2 (45.98) because different benchmark subsets are included; please state explicitly which benchmarks contribute to each average.
  5. [Section 3.3] Equations (1), (2), and (4) are introduced as 'the gradient expression' but the actual training loss is standard cross-entropy on the concatenated ShareGPT dialogues; please either remove these equations or clearly label them as intuitive illustrations that are not used in the training objective.
  6. [Section 4.1] The evaluator for subjective questions is Qwen2.5-14B-Instruct; please provide the exact evaluation prompt and report a validation of the judge's agreement with human annotations.
  7. [General] The paper lacks a discussion of limitations, including the computational cost of generating multi-agent dialogues and the risk of error propagation in synthetic teaching interactions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MASTER's central claims rest on external benchmark comparisons with no fitted parameters or self-citation chain; post-hoc MCQ benchmark selection is a validity concern, not circularity.

full rationale

The paper's load-bearing claim is that models fine-tuned on BOOST-QA outperform models fine-tuned on Ori-Data and on baseline-augmented data across external benchmarks. No parameter of the MASTER pipeline is fit to any evaluation benchmark, and no benchmark result is used to construct BOOST-QA. The informal gradient expressions (Eqs. 1, 2, 4) are explanatory sketches of the augmentation objective, not derivations of benchmark accuracy, so they cannot reduce the empirical claim to an assumption. The paper contains no self-citation chain: pedagogical citations [29-31] motivate scenario design, and dataset citations [33-47] provide inputs and baselines, but none of these are invoked as a uniqueness theorem or as a substitute for the reported experiments. The closest concern to circularity is the statement in Sec. 4.3 that 'upon observing this phenomenon, we incorporated additional multiple-choice benchmark tests,' which is post-hoc selection on the dependent variable and may inflate the reported 31.46% maximum and 15.50% average improvement. That is a statistical-validity and selection-bias concern, but it is not circularity under the definition used here: no benchmark value is an input to the method, and the comparison to Ori-Data on pre-registered benchmarks in Tables 1 and 2 remains an external, falsifiable test. Therefore the core derivation is self-contained against external benchmarks and the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The main method parameters are hand-set temperatures and an unspecified retrieval top-k; none are fitted to the evaluation benchmarks. The approach assumes unverified dialogue quality, correctness preservation after teacher correction, and validity of the Qwen2.5-14B judge for subjective answers. No new physical or theoretical entities are introduced.

free parameters (5)
  • student error-generation temperature = 0.8
    Chosen to make the 0.5B student model produce diverse, imperfect answers; not fit to benchmarks.
  • teacher and student correction temperature = 0.2
    Chosen for the 14B model in Error Correction and Analogical modules for accurate, low-entropy outputs.
  • debate participant temperature = 0.6
    Chosen for the 7B S1 and S2 debate agents to encourage diverse arguments.
  • debate summarizer temperature = 0.2
    Chosen for the 14B S3 summarizer.
  • similar-question retrieval top-k = not specified
    Equation (3) samples from top-k similar questions, but k is never reported.
assumptions (4)
  • domain assumption LLMs can reliably simulate teacher and student roles with the specified prompts.
    Section 3.2; no validation of role fidelity or dialogue quality is provided.
  • domain assumption The generated multi-turn dialogues preserve the correctness of the standard answer after teacher correction.
    Section 3.3; no filtering or human check is described.
  • domain assumption The Qwen2.5-14B-Instruct model is a valid grader for subjective answers on the evaluation benchmarks.
    Section 4.3; no human agreement or calibration is reported.
  • domain assumption Training on error-containing multi-turn data improves downstream performance rather than teaching the model to reproduce errors.
    Section 3.3; the claim that this injects structured noise and smooths the loss landscape is asserted, not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching." pith.science (2026). https://pith.science/paper/PXXQFF4H

@misc{pith2026250602689,
  author       = {Pith},
  title        = {Pith review of: MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXXQFF4H}},
  note         = {Machine review of arXiv:2506.02689}
}
read the original abstract

Instruction fine-tuning is crucial in NLP tasks, enhancing pretrained models' instruction-following capabilities and task-specific performance. However, obtaining high-quality fine-tuning data for large models is challenging due to data collection difficulties and high production costs. To address this, we propose MASTER, a novel data augmentation method that enriches original data through interactions among multiple agents with varying cognitive levels. We simulate three pedagogically grounded teaching scenarios, leveraging multi-agent conversations to generate high-quality teacher-student interaction data. Utilizing MASTER, we construct BOOST-QA, a fine-tuning dataset augmented from existing datasets like Orca-Math-200k, ProcQA, and OpenHermes2.5. Experiments show that models fine-tuned with BOOST-QA perform excellently across multiple benchmarks, demonstrating strong multitask generalization. Notably, MASTER significantly improves models' reasoning abilities in complex tasks, providing valuable insights for future research.

Figures

Figures reproduced from arXiv: 2506.02689 by the authors.

Figure 1
Figure 1. A multi-agent system-based data augmentation pipeline that simulates three different [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Results from the complex multiple-choice question test show a maximum improvement [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 15 canonical work pages

  1. [1]

    Parameter-efficient fine-tuning for large models: A comprehensive survey

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608, 2024

    arXiv 2024

  2. [2]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems , 35:27730–27744, 2022

    2022

  3. [3]

    Gpt3mix: Leveraging large-scale language models for text augmentation

    Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, and Woomyeong Park. Gpt3mix: Leveraging large-scale language models for text augmentation. arXiv preprint arXiv:2104.08826, 2021

    arXiv 2021

  4. [4]

    Enhancing chat language models by scaling high-quality instructional conversations

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233, 2023

    arXiv 2023

  5. [5]

    Baize: An open-source chat model with parameter-efficient tuning on self-chat data

    Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. arXiv preprint arXiv:2304.01196, 2023

    arXiv 2023

  6. [6]

    Data augmentation approaches in natural language processing: A survey

    Bohan Li, Yutai Hou, and Wanxiang Che. Data augmentation approaches in natural language processing: A survey. Ai Open, 3:71–90, 2022

    work page 2022

  7. [7]

    CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding

    Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Jiawei Han, and Weizhu Chen. Coda: Contrast- enhanced and diversity-promoting data augmentation for natural language understanding. arXiv preprint arXiv:2010.08670, 2020

    work page Pith review arXiv 2010

  8. [8]

    Eda: Easy data augmentation techniques for boosting performance on text classification tasks

    Jason Wei and Kai Zou. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv:1901.11196, 2019

    arXiv 1901

Show all 47 references
  1. [9]

    Text data augmentation for deep learning

    Connor Shorten, Taghi M Khoshgoftaar, and Borko Furht. Text data augmentation for deep learning. Journal of big Data, 8(1):101, 2021

    2021

  2. [10]

    Pga-scire: Harnessing llm on data augmentation for enhancing scientific relation extraction

    Yang Zhou, Shimin Shan, Hongkui Wei, Zhehuan Zhao, and Wenshuo Feng. Pga-scire: Harnessing llm on data augmentation for enhancing scientific relation extraction. arXiv preprint arXiv:2405.20787, 2024

    2024 arXiv

  3. [11]

    Flipda: Effective and robust data augmentation for few-shot learning

    Jing Zhou, Yanan Zheng, Jie Tang, Jian Li, and Zhilin Yang. Flipda: Effective and robust data augmentation for few-shot learning. arXiv preprint arXiv:2108.06332, 2021

    2021 arXiv

  4. [12]

    Wizardlm: Empowering large language models to follow complex instructions

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244, 2023

    2023 arXiv

  5. [13]

    Self-instruct: Aligning language models with self-generated instructions

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560, 2022

    2022 arXiv

  6. [14]

    Condor: Enhance llm alignment with knowledge-driven data synthesis and refinement

    Maosong Cao, Taolin Zhang, Mo Li, Chuyu Zhang, Yunxin Liu, Haodong Duan, Songyang Zhang, and Kai Chen. Condor: Enhance llm alignment with knowledge-driven data synthesis and refinement. arXiv preprint arXiv:2501.12273, 2025

    2025 arXiv

  7. [15]

    Exploring collaboration mechanisms for llm agents: A social psychology view

    Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. Exploring collaboration mechanisms for llm agents: A social psychology view. arXiv preprint arXiv:2310.02124, 2023

    2023 arXiv

  8. [16]

    Investigating the personality consistency in quantized role-playing dialogue agents

    Yixiao Wang, Homa Fashandi, and Kevin Ferreira. Investigating the personality consistency in quantized role-playing dialogue agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 239–255, 2024

    2024

  9. [17]

    Prospects for multi-agent collaboration and gaming: challenge, technology, and application

    Yu Liu, Zhi Li, Zhizhuo Jiang, and You He. Prospects for multi-agent collaboration and gaming: challenge, technology, and application. Frontiers of Information Technology & Electronic Engineering , 23(7): 1002–1009, 2022

    2022

  10. [18]

    Reflective multi-agent collaboration based on large language models

    Xiaohe Bo, Zeyu Zhang, Quanyu Dai, Xueyang Feng, Lei Wang, Rui Li, Xu Chen, and Ji-Rong Wen. Reflective multi-agent collaboration based on large language models. Advances in Neural Information Processing Systems, 37:138595–138631, 2024

    2024

  11. [19]

    Com- peteai: Understanding the competition dynamics in large language model-based agents

    Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie. Com- peteai: Understanding the competition dynamics in large language model-based agents. arXiv preprint arXiv:2310.17512, 2023. 10

    2023 arXiv

  12. [20]

    Unveiling the truth and facilitating change: Towards agent-based large-scale social movement simulation

    Xinyi Mou, Zhongyu Wei, and Xuanjing Huang. Unveiling the truth and facilitating change: Towards agent-based large-scale social movement simulation. arXiv preprint arXiv:2402.16333, 2024

    2024 arXiv

  13. [21]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology , pages 1–22, 2023

    2023

  14. [22]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  15. [23]

    Training shallow and thin networks for acceleration via knowledge distillation with conditional adversarial networks

    Zheng Xu, Yen-Chang Hsu, and Jiawei Huang. Training shallow and thin networks for acceleration via knowledge distillation with conditional adversarial networks. arXiv preprint arXiv:1709.00513, 2017

    2017 arXiv

  16. [24]

    Deep mutual learning

    Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. Deep mutual learning. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4320–4328, 2018

    2018

  17. [25]

    Similarity-preserving knowledge distillation

    Frederick Tung and Greg Mori. Similarity-preserving knowledge distillation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 1365–1374, 2019

    2019

  18. [26]

    Relational knowledge distillation

    Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3967–3976, 2019

    2019

  19. [27]

    Speculative knowledge distillation: Bridging the teacher-student gap through interleaved sampling

    Wenda Xu, Rujun Han, Zifeng Wang, Long T Le, Dhruv Madeka, Lei Li, William Yang Wang, Rishabh Agarwal, Chen-Yu Lee, and Tomas Pfister. Speculative knowledge distillation: Bridging the teacher-student gap through interleaved sampling. arXiv preprint arXiv:2410.11325, 2024

    arXiv

  20. [28]

    Tinyr1-32b-preview: Boosting accuracy with branch-merge distillation

    Lin Sun, Guangxiang Zhao, Xiaoqi Jian, Yuhan Wu, Weihong Lin, Yongfu Zhu, Linglin Zhang, Jinzhu Wu, Junfeng Ran, Sai-er Hu, et al. Tinyr1-32b-preview: Boosting accuracy with branch-merge distillation. arXiv preprint arXiv:2503.04872, 2025

    2025 arXiv

  21. [29]

    Secondary school students learning from reflections on the rationale behind self-made errors: A field experiment

    Tim Heemsoth and Aiso Heinze. Secondary school students learning from reflections on the rationale behind self-made errors: A field experiment. The Journal of Experimental Education , 84(1):98–118, 2016

    2016

  22. [30]

    Debate: a teaching-learning strategy for developing competence in communication and critical thinking

    Michele Darby. Debate: a teaching-learning strategy for developing competence in communication and critical thinking. Journal of dental hygiene, 81(4), 2007

    2007

  23. [31]

    Schema induction in children’s analogical problem solving

    Zhe Chen. Schema induction in children’s analogical problem solving. Journal of Educational Psychology, 91(4):703, 1999

    1999

  24. [32]

    Teacher-student interactions for enhanced learning in upper secondary mathematics classroom

    Aline Dorimana, Alphonse Uworwabayeho, and Gabriel Nizeyimana. Teacher-student interactions for enhanced learning in upper secondary mathematics classroom. International Journal of Evaluation and Research in Education, 11(2):507–515, 2022

    2022

  25. [33]

    Orca-math: Unlocking the potential of slms in grade school math

    Arindam Mitra, Hamed Khanpour, Corby Rosset, and Ahmed Awadallah. Orca-math: Unlocking the potential of slms in grade school math. arXiv preprint arXiv:2402.14830, 2024

    2024 arXiv

  26. [34]

    Procqa: A large- scale community-based programming question answering dataset for code search

    Zehan Li, Jianfei Zhang, Chuantao Yin, Yuanxin Ouyang, and Wenge Rong. Procqa: A large- scale community-based programming question answering dataset for code search. arXiv preprint arXiv:2403.16702, 2024

    2024 arXiv

  27. [35]

    Evaluating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021

    2021 arXiv

  28. [36]

    Program synthesis with large language models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021

    2021 arXiv

  29. [37]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  30. [38]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural Information Pr...

    2024

  31. [39]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. 11

    2009 arXiv

  32. [40]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  33. [41]

    Crowdsourcing multiple choice science questions

    Johannes Welbl, Nelson F Liu, and Matt Gardner. Crowdsourcing multiple choice science questions. arXiv preprint arXiv:1707.06209, 2017

    2017 arXiv

  34. [42]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  35. [43]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  36. [44]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    doi arXiv

  37. [45]

    Nlp augmentation

    Edward Ma. Nlp augmentation. https://github.com/makcedward/nlpaug, 2019

    2019

  38. [46]

    Tagcos: Task-agnostic gradient clustered coreset selection for instruction tuning data

    Jipeng Zhang, Yaxuan Qin, Renjie Pi, Weizhong Zhang, Rui Pan, and Tong Zhang. Tagcos: Task-agnostic gradient clustered coreset selection for instruction tuning data. arXiv preprint arXiv:2407.15235, 2024

    2024 arXiv

  39. [47]

    The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning

    Seungone Kim, Se June Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo. The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning. arXiv preprint arXiv:2305.14045, 2023. 12

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.