Pith. sign in

REVIEW 3 major objections 5 minor 34 references

The paper claims that a ~$21 synthetic dataset of 21,250 reworded GSM8K problems, combined with full-layer LoRA fine-tuning on a 16 GB laptop, lifts two small Qwen3 models' exact-match accuracy on GSM8K by 12.6 and 13.0 percentage points.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 13:21 UTC pith:YA5GHZSH

load-bearing objection Honest, cheap synthetic-data recipe for small-model arithmetic, but the reported gains may be inflated by a parser that rewards the fine-tuned models' learned '####' format. the 3 major comments →

arxiv 2607.18266 v1 pith:YA5GHZSH submitted 2026-05-21 cs.AI

Structured Synthetic Reasoning Data for Arithmetic Fine-Tuning of Small Language Models

Jake O'Grady , Effirul Ramlan This is my paper
classification cs.AI
keywords synthetic dataarithmetic reasoningsmall language modelsLoRA fine-tuningGSM8KSocratic-style guidanceself-consistencyconsumer hardware
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that a relatively cheap, fully synthetic dataset can make small language models noticeably better at multi-step arithmetic. Starting from the GSM8K training set, the authors used a teacher model to generate 21,250 reworded and re-structured word problems with natural-language solution traces, brief Socratic guiding questions, and irrelevant distractors, then fine-tuned two Qwen3 models with LoRA on a single 16 GB laptop. Exact-match accuracy on GSM8K rose from 36.5% to 49.1% for the 0.6B model and from 53.5% to 66.5% for the 1.7B model, and transfer to MultiArith reached 92.2% and 98.9% with four self-consistency samples. The authors read the results as evidence for structured synthetic reasoning data as a whole, not as a causal proof that Socratic cues alone cause the gains.

Core claim

The central claim is that a filtered corpus of teacher-generated GSM8K variants—free-form reasoning traces with light Socratic prompts, structural variation, and mandatory distractor sentences—plus full-layer LoRA adapters is sufficient to shift small-model arithmetic behaviour. On the GSM8K test set, accuracy rises from 36.5% to 49.1% for Qwen3-0.6B (481 to 647 correct of 1,319) and from 53.5% to 66.5% for Qwen3-1.7B (706 to 877). Transfer reaches 98.9% on MultiArith and 73.0% on SVAMP for the 1.7B model with four self-consistency samples. The paper also reports design findings: adapting all 28 layers matters more than increasing rank; natural-language traces beat a rigid checklist format;

What carries the argument

The central object is the filtered synthetic corpus and the LoRA adapter configuration. Each example pairs a rewritten grade-school word problem with a natural-language solution that embeds one or two Socratic-style guiding questions, at least one sentence of irrelevant context, and a numeric answer after a '####' delimiter. A four-stage filter removes malformed entries, very short or long solutions, near-duplicates (5-gram Jaccard above 0.85), and rhetorical questions before the answer. The LoRA setup adapts query, key, value, output, gate, up, and down projections in all 28 transformer layers; the paper reports that layer coverage, not rank, carries the gain in the 0.6B runs.

Load-bearing premise

The claims assume the teacher-generated training variants are disjoint from the GSM8K test set; the paper reports no decontamination filter against test questions, so if the teacher memorized those questions the gains could reflect recall rather than learned arithmetic.

What would settle it

Generate the same 21,250-example corpus with a teacher model that provably has no exposure to GSM8K test questions (or filter out any generated example with high n-gram overlap with the 1,319 test problems), repeat the LoRA runs, and compare GSM8K exact-match accuracy; a drop toward the base rates would indicate contamination drove part of the gain. A matched non-Socratic corpus would additionally isolate the Socratic component.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the recipe is right, a 0.6B model fine-tuned on roughly $21 of teacher-generated data outperforms a 7B 8-bit baseline on GSM8K under identical evaluation (49.1% vs 47.2%), and a 1.7B model beats a 3B instruct model (66.5% vs 56.4%).
  • Fine-tuned models produce more concise reasoning traces and use irrelevant distractors less often, making 4-sample self-consistency voting more reliably helpful than for base models.
  • The design lesson that full-layer attention-plus-feed-forward LoRA coverage is necessary, while rank 16 to 32 adds little, gives a concrete starting point for other small-model arithmetic fine-tuning.
  • Transfer being much stronger on MultiArith than on SVAMP implies the corpus teaches calculation and operation selection more than robustness to phrasing variation; future data design should add linguistic diversity for the latter.
  • Because the paper combines several design choices, a matched non-Socratic dataset is the next experiment required to attribute gains to Socratic cues.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The pipeline as reported does not decontaminate teacher outputs against the GSM8K test set. If the teacher memorized those test questions, the headline gains could be partly recall rather than newly learned arithmetic; a decontamination filter or a teacher with verified no exposure is the natural check.
  • The LoRA-layer result suggests the effective capacity ceiling for this dataset is below rank 32; an interpretation not explored in the paper is that larger synthetic corpora, not larger adapters, are the next scaling lever.
  • Because MultiArith transfer is near-saturated at 98.9%, a sharper test of the corpus's robustness would be a harder arithmetic benchmark with more variable language, or adding linguistic-distractor diversity to the generation prompt.
  • The self-consistency improvements suggest the fine-tuned model's output distribution is more concentrated on correct answers; measuring predictive entropy or answer-pair agreement on the test set would quantify this directly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a low-cost pipeline for constructing structured synthetic arithmetic reasoning data. Starting from GSM8K training problems, the authors use GPT-5-mini to generate 21,250 variants that combine natural-language solution traces, light Socratic-style cues, structural variation, and irrelevant distractor context, followed by multi-stage filtering. They LoRA fine-tune Qwen3-0.6B and Qwen3-1.7B on this corpus under a consumer-hardware budget (Apple M4, 16 GB RAM) and report exact-match gains on GSM8K (36.5% to 49.1% for 0.6B; 53.5% to 66.5% for 1.7B), plus transfer gains on MultiArith and SVAMP with self-consistency. The paper includes qualitative error analysis, design lessons on format sensitivity and layer coverage, and explicit caveats that the intervention is not a causal test of Socratic guidance alone.

Significance. If the results hold, the paper provides a practical, low-cost recipe for improving arithmetic adaptation in small models, with a credible transfer signal beyond GSM8K. The authors ship code, prompts, and evaluation logs, report API costs transparently, and are appropriately cautious about causal attribution. The MultiArith and SVAMP gains, especially the 98.9% MultiArith result for Qwen3-1.7B, are notable and go beyond a simple GSM8K-only improvement. However, the central quantitative claim depends on two testable assumptions—parser fairness and train/test disjointness—that are not yet established.

major comments (3)
  1. [§3.3, §3.1, Appendix A] The answer extractor searches for the last occurrence of '#### <number>'; outputs without this pattern are recorded as empty and scored incorrect. The training data, by construction, always uses '#### <answer>', so fine-tuned models are directly reinforced to emit this delimiter, while base models under 4-shot prompting may produce correct answers without it. The paper does not report parse rates or a lenient-parser comparison on any benchmark. Given the paper's own Section 6 finding that small models are highly sensitive to surface formatting, this asymmetry is a plausible confound for the headline GSM8K gains and for the MultiArith/SVAMP gains. Please report parse rates for base vs. fine-tuned models and re-evaluate with a lenient parser that accepts a final numeric answer regardless of delimiter.
  2. [§3.1] The synthetic corpus is generated by GPT-5-mini from GSM8K training problems, but the paper reports no decontamination step against the GSM8K test set. If the teacher model's pretraining included the test set, generated variants or close paraphrases could leak into the training data and inflate the GSM8K results. Because the corpus itself is not released, this cannot be independently checked. Please report n-gram overlap between the generated variants and the GSM8K test questions and make the corpus available; if overlap is non-trivial, retrain on filtered data. The MultiArith/SVAMP transfer results mitigate this concern, but the primary GSM8K claim requires the training and evaluation distributions to be disjoint.
  3. [§4.2, Tables 2–3] The reported improvements are based on a single run per configuration, and for Qwen3-1.7B the headline 66.5% is the best of three hyperparameter settings selected after seeing test results. There are no confidence intervals, seed variance estimates, or a held-out hyperparameter selection procedure. Since the central claim is quantitative ('materially improve'), please add repeated runs (at least three seeds) for the best configuration and report mean ± std, or bootstrap confidence intervals over the test set. Also clarify whether validation loss was used to select hyperparameters or only to select checkpoints.
minor comments (5)
  1. [Abstract and §4.3] The benchmark name is 'SVAMP', but the text inconsistently uses 'SV AMP' with a space. Please make this consistent.
  2. [§3.4, Eq. (2)] There is a typo in Eq. (2): 'N as the number of test cases' should read 'N is the number of test cases'.
  3. [Appendix B, Table 6] The learning-rate entries in Table 6 (e.g., '24.01×10^-4' and '8.01×10^-4') are inconsistent with the values reported in Tables 2–3 (e.g., '1×10^-4'). Please standardize the notation and verify the actual values used.
  4. [§3.4] The self-consistency sampling parameters (temperature 0.7, top-p 0.95, top-k 20) are reported only for the transfer evaluations. Please specify the sampling parameters for the primary GSM8K single-sample evaluation as well.
  5. [Data and Code Availability] The availability statement mentions code, prompts, and processed evaluation logs, but not the 21,250-example synthetic corpus. Please state explicitly whether the corpus will be released; without it, the central recipe is not fully reproducible and decontamination cannot be verified independently.

Circularity Check

0 steps flagged

No significant circularity: the paper's central claim is an empirical fine-tuning measurement with disjoint train/test splits and no derivation chain that reduces to its inputs.

full rationale

The paper's central claim is that LoRA fine-tuning on a synthetic GSM8K-derived corpus improves exact-match arithmetic accuracy on GSM8K, MultiArith, and SVAMP. This is an empirical measurement, not a derivation. The training corpus is generated from the GSM8K training split (Section 3.1), while primary evaluation uses the disjoint GSM8K test set (Section 3.4), and transfer is measured on MultiArith and SVAMP, which are not used for training. No equation in the paper equates a predicted quantity with a fitted parameter or with the data-generation prompt by construction. Hyperparameters and checkpoint selection are tuned on a held-out validation split, which is standard practice and does not make the test-set result definitionally equal to an input. The paper explicitly disclaims a causal test of Socratic guidance and lists the missing matched non-Socratic baseline as a limitation, so it does not rename an input as a discovered cause. There are no self-citations of the authors' prior work, no imported uniqueness theorem, and no ansatz smuggled in via citation. The evaluation-parser asymmetry and possible GSM8K contamination of the teacher model are external validity or measurement concerns, not circularity: they do not show that the reported accuracy gain is equivalent to the training objective or prompt by construction. Therefore the appropriate finding is no significant circularity, score 0.

Axiom & Free-Parameter Ledger

6 free parameters · 3 axioms · 0 invented entities

The central claim rests on an empirical pipeline rather than a derivation; the ledger captures hand-selected hyperparameters and two unverified assumptions about data generation and evaluation.

free parameters (6)
  • Jaccard deduplication threshold = 0.85
    Hand-chosen in filtering stage 3 (Sec 3.1); controls near-duplicate removal and affects effective training diversity.
  • Best LoRA rank (Qwen3-1.7B) = 16
    Selected via validation loss among rank 16/32/48; the 66.5% GSM8K result uses this rank.
  • Best LoRA learning rate (Qwen3-1.7B) = 1e-4
    Selected via validation loss; Table 3 run 3 achieves 66.5% with rank 16 and this LR.
  • Self-consistency sampling temperature = 0.7
    Used for all MultiArith/SVAMP transfer evaluations (Sec 3.4); hand-chosen, not swept.
  • Self-consistency top-p / top-k = 0.95 / 20
    Fixed inference parameters for transfer evaluations; hand-chosen.
  • Variants generated per GSM8K problem = 3
    Data-generation prompt requests 3 variants per source problem, giving the 21,250 corpus size; fixed choice with no ablation.
axioms (3)
  • domain assumption GPT-5-mini generates variants only from GSM8K training problems and does not reproduce GSM8K test content from its pretraining memory.
    The corpus is built by a closed teacher model; the paper describes no test-set decontamination. If false, GSM8K test gains may be inflated by memorized test problems.
  • domain assumption The 90/10 validation split and checkpoint selection by validation loss are uncontaminated and representative.
    Checkpoints are chosen by lowest validation loss (Sec 3.3); if validation examples duplicate training variants from the same synthetic process, selection could be biased.
  • domain assumption Exact-match parsing of the last '#### <number>' pattern is a faithful proxy for arithmetic correctness.
    All reported accuracies depend on this regex; malformed but conceptually correct outputs are scored as wrong (Sec 3.3).

pith-pipeline@v1.3.0-alltime-deepseek · 10993 in / 13245 out tokens · 135192 ms · 2026-08-02T13:21:47.114965+00:00 · methodology

0 comments
read the original abstract

Small language models are attractive for local deployment, but they often struggle with multi-step arithmetic reasoning. We study whether structured synthetic reasoning data can improve this behaviour under consumer-hardware constraints. Starting from GSM8K, we generated a 21,250-example corpus of grade-school arithmetic word-problem variants using GPT-5-mini, combining natural-language solution traces, light Socratic-style cues, structural variation, and irrelevant distractor context. We then fine-tuned Qwen3-0.6B and Qwen3-1.7B with LoRA on consumer hardware (Apple M4, 16 GB RAM). Exact-match accuracy on GSM8K improved from 36.5% to 49.1% for Qwen3-0.6B and from 53.5% to 66.5% for Qwen3-1.7B. For Qwen3-1.7B, transfer to related arithmetic benchmarks was stronger, reaching 98.9% on MultiArith and 73.0% on SVAMP, compared with 54.4% and 45.3% for the base model. Qualitative analysis suggests that fine-tuned models produce shorter reasoning traces, make fewer arithmetic and distractor-use errors, and benefit more consistently from self-consistency sampling. These results show that low-cost synthetic data design can materially improve arithmetic adaptation in small language models. Because the intervention combines Socratic-style cues with other data-design choices, we interpret the gains as evidence for structured synthetic reasoning data rather than as a causal test of Socratic guidance alone.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 2 canonical work pages

  1. [1]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott G...

    1901

  2. [2]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwa- sawa. Large language models are zero-shot reasoners. InAdvances in Neural In- formation Processing Systems, volume 35, pages 22199–22213. Curran Associates, Inc.,

  3. [3]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InAdvances in Neural Information Processing Systems, vol- ume 35, pages 24824–24837. Curran Associates, Inc., 2022. doi: 10.48550/arXiv.2201. 11903. URL https://proceedings.n...

    doi:10.48550/arxiv.2201 2022

  4. [4]

    Parameter-efficient fine-tuning for large models: A comprehensive survey.arXiv preprint arXiv:2403.14608, 2024

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey.arXiv preprint arXiv:2403.14608, 2024. doi: 10.48550/arXiv. 2403.14608. URLhttps://arxiv.org/abs/2403.14608

    Pith/arXiv arXiv doi:10.48550/arxiv.2403.14608 2024

  5. [5]

    Retrieval-augmented generation for knowledge-intensive NLP tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. InAd- vances in Neural Information Processing Systems, volume 33, pages 9459–9474. Curran As- sociate...

    2020

  6. [6]

    Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. InAdvances in Neural Information Processing Sys- tems, volume 35, pages 27730–27744. Curran Associates, Inc., 2022. doi: 10.48550/arXiv...

    doi:10.48550/arxiv.2203 2022

  7. [7]

    Teaching large language models to reason with reinforcement learning.arXiv preprint arXiv:2403.04642, 2024

    Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, Sainbayar Sukhbaatar, and Roberta Raileanu. Teaching large language models to reason with reinforcement learning.arXiv preprint arXiv:2403.04642, 2024. doi: 10.48550/arXiv.2403.04642. URLhttps://arxiv.org/abs/2403.04642

    Pith/arXiv arXiv doi:10.48550/arxiv.2403.04642 2024

  8. [8]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on 10 Learning Representations. OpenReview.net, 2022. doi: 10.48550/arXiv.2106.09685. URL https: //openreview.net/forum?id=nZeVKeeFYf9

    Pith/arXiv arXiv doi:10.48550/arxiv.2106.09685 2022

  9. [10]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13484–13508, Toronto, Canada, July 2023. Associa...

    doi:10.18653/v1/2023.acl-long.754 2023

  10. [11]

    Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022. doi: 10.48550/arXiv.2203.11171. URL https://arxiv.org/ abs/2203.11171

    Pith/arXiv arXiv doi:10.48550/arxiv.2203.11171 2022

  11. [12]

    LIMA: Less is more for alignment

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: Less is more for alignment. InAdvances in Neural Information Processing Systems, volume 36. Curran Associates, Inc., 2023. doi: 10.48550/arXiv.2305. 11206. URL https:/...

    doi:10.48550/arxiv.2305 2023

  12. [13]

    Socratic method as an approach to teaching.European Researcher, 111(10):511–517, 2016

    Hasan Delic and Senad Becirovic. Socratic method as an approach to teaching.European Researcher, 111(10):511–517, 2016. doi: 10.13187/er.2016.111.511. URL https://doi.org/10.13187/ er.2016.111.511

    work page doi:10.13187/er.2016.111.511 2016

  13. [14]

    Reyes, Ron Mhel Francis L

    Englevert P. Reyes, Ron Mhel Francis L. Blanco, Defanee Rose L. Doroon, Jay Lord B. Limana, and Ana Marie A. Torcende. Feynman technique as a heutagogical learning strategy for independent and remote learning.Recoletos Multidisciplinary Research Journal, 9(2):1–13, 2021. URL https: //ejournals.ph/article.php?id=16804

    2021

  14. [15]

    Metacognition in schools: What does the literature suggest about the effectiveness of teaching metacognition in schools?Educational Review, 71(4): 483–500, 2019

    Jonathan Perry, David Lundie, and Gill Golder. Metacognition in schools: What does the literature suggest about the effectiveness of teaching metacognition in schools?Educational Review, 71(4): 483–500, 2019. doi: 10.1080/00131911.2018.1441127. URL https://doi.org/10.1080/ 00131911.2018.1441127

    arXiv 2019

  15. [16]

    The art of SOCRATIC QUESTIONING: Recursive thinking with large language models

    Jingyuan Qi, Zhiyang Xu, Ying Shen, Minqian Liu, Di Jin, Qifan Wang, and Lifu Huang. The art of SOCRATIC QUESTIONING: Recursive thinking with large language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4177–4199, Singapore, 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emn...

    work page doi:10.18653/v1/2023.emnlp-main.255 2023

  16. [17]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021. doi: 10.48550/ arXiv.2110.14168. URLhttps://arxiv.org/abs/2110.14168. 11

    Pith/arXiv arXiv doi:10.48550/arxiv.2110.14168 2021

  17. [18]

    AdaLoRA: Adaptive budget allocation for parameter-efficient fine-tuning

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. AdaLoRA: Adaptive budget allocation for parameter-efficient fine-tuning. InInternational Conference on Learning Representations. OpenReview.net, 2023. doi: 10.48550/arXiv.2303.10512. URLhttps://openreview.net/forum?id=lq62uWRJjiY

    Pith/arXiv arXiv doi:10.48550/arxiv.2303.10512 2023

  18. [19]

    DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019. doi: 10.48550/ arXiv.1910.01108. URLhttps://arxiv.org/abs/1910.01108

    Pith/arXiv arXiv doi:10.48550/arxiv.1910.01108 1910

  19. [20]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford Alpaca: An instruction-following LLaMA model. Stanford CRFM, 2023. URLhttps://crfm.stanford.edu/2023/03/13/alpaca.html

    2023

  20. [21]

    Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. MetaMath: Bootstrap your own mathematical questions for large language models.arXiv preprint arXiv:2309.12284, 2023. doi: 10.48550/arXiv.2309.12284. URL https://arxiv.org/abs/2309.12284

    Pith/arXiv arXiv doi:10.48550/arxiv.2309.12284 2023

  21. [22]

    WizardMath: Empowering mathematical reasoning for large language models via reinforced evol-instruct.arXiv preprint arXiv:2308.09583,

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, Yansong Tang, and Dongmei Zhang. WizardMath: Empowering mathematical reasoning for large language models via reinforced evol-instruct.arXiv preprint arXiv:2308.09583,

    Pith/arXiv arXiv

  22. [23]

    Boundless Socratic learning with language games.arXiv preprint arXiv:2411.16905, 2024

    Tom Schaul. Boundless Socratic learning with language games.arXiv preprint arXiv:2411.16905, 2024. doi: 10.48550/arXiv.2411.16905. URLhttps://arxiv.org/abs/2411.16905

    Pith/arXiv arXiv doi:10.48550/arxiv.2411.16905 2024

  23. [24]

    MLX LM: Language model training and inference for Apple Silicon, 2024

    Apple Inc. MLX LM: Language model training and inference for Apple Silicon, 2024. URL https: //github.com/ml-explore/mlx-lm. GitHub repository

    2024

  24. [25]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. doi: 10.48550/arXiv.2505.09388. URLhttps://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.09388 2025

  25. [26]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations. OpenReview.net, 2019. doi: 10.48550/arXiv.1711.05101. URL https: //openreview.net/forum?id=Bkg6RiCqY7

    Pith/arXiv arXiv doi:10.48550/arxiv.1711.05101 2019

  26. [27]

    SGDR: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. InInterna- tional Conference on Learning Representations. OpenReview.net, 2017. doi: 10.48550/arXiv.1608. 03983. URLhttps://openreview.net/forum?id=Skq89Scxx

    doi:10.48550/arxiv.1608 2017

  27. [28]

    The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024. doi: 10.48550/arXiv.2407.21783. URL https: //arxiv.org/abs/2407.21783

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.21783 2024

  28. [29]

    Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7B.arXiv preprint arXiv:2310.06825, 2023. doi: 10.48550/arXiv.2310.06825. URL https: //arxiv.org/abs/2310.06825

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.06825 2023

  29. [30]

    Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    Gemma Team. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025. doi: 10.48550/ arXiv.2503.19786. URLhttps://arxiv.org/abs/2503.19786. 12

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.19786 2025

  30. [31]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080–2094, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2...

    doi:10.18653/v1/2021.naacl-main.168 2021

  31. [32]

    Solving general arithmetic word problems

    Subhro Roy and Dan Roth. Solving general arithmetic word problems. InProceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1743–1752, Lisbon, Portugal,

    2015

  32. [2015]

    Question:

    Association for Computational Linguistics. doi: 10.18653/v1/D15-1202. URL https:// aclanthology.org/D15-1202/. 13 A Data Generation Prompt You are a math tutor tasked with generating training data by rewriting math solutions into multiple concise reasoning variants, using gentle Socratic-style guidance. Rules: - Generate 3 altered questions similar in str...

    doi:10.18653/v1/d15-1202

  33. [2022]

    URL https://proceedings.neurips.cc/paper_files/paper/2022/hash/ 8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html

    2022

  34. [2023]

    URLhttps://arxiv.org/abs/2308.09583

    doi: 10.48550/arXiv.2308.09583. URLhttps://arxiv.org/abs/2308.09583

    Pith/arXiv arXiv doi:10.48550/arxiv.2308.09583