Pith. sign in

REVIEW 4 major objections 7 minor 65 references

Typhoon T1: An Open Thai Reasoning Model

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Supervised fine-tuning alone, on 42k structured long-thinking records, converts a 3B Thai instruct model into a reasoning model that improves on math, code, and science benchmarks and can think in Thai.

desk verdict Open Thai reasoning recipe with real artifacts and a clean thinking-format comparison, but the unvalidated synthetic traces are the main thing to fix. read the letter →

arxiv 2502.09042 v2 pith:AXO6EMVS submitted 2025-02-13 cs.CL cs.AI

Pittawat Taveekitworachai , Potsawee Manakul , Kasima Tharnpipitchai , Kunat Pipatanakul This is my paper · ORCID
classification cs.CLcs.AI
keywords reasoningmodelssupervisedfine-tuningThailanguagestructuredthinkinglongchain-of-thoughtlow-resourceopen-sourceLLMsyntheticdata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that supervised fine-tuning on roughly 42,000 machine-generated long-thinking records, formatted with XML structural tags, converts a 3B Thai instruct model into a reasoning model that beats its base on GSM8K, HumanEval+, and GPQA, and can produce Thai thinking traces after adding 1,565 translated records. It argues that a structured 'plan-then-step' thinking format outperforms semi-structured and unstructured formats, and that safety data is the most influential domain in the mixture. If correct, it establishes a reproducible, low-cost SFT-only recipe for building small reasoning models in low-resource languages without distillation from an existing reasoning model.

What carries the argument

The load-bearing mechanism is the transformation-and-refinement data pipeline: starting from open datasets with ground-truth answers, a frontier language model (GPT-4o mini) converts each answer into a long structured-thinking trace using three hand-curated few-shot exemplars, then a larger open instructor model (Qwen2.5-32B-Instruct) refines the trace for structural and factual correctness. The structured thinking format itself—with explicit planning, per-step scratchpads, summaries, and next-step decisions—is the syntactic object that carries the reasoning behavior, and it can be reduced to semi-structured or unstructured formats by tag removal so that the formats are compared with equivalent information.

What would settle it

Run the same SFT recipe on traces whose intermediate steps have been independently verified correct, and compare against traces generated without verification; if benchmark gains vanish, the effect came from format or dataset overlap rather than reasoning quality.

Watch

Extended reading notes

Core claim

The central discovery is that long-chain reasoning behavior in a 3B model can be elicited by supervised fine-tuning alone on synthetically transformed open datasets, without reinforcement learning or knowledge distillation. The paper introduces structured thinking—a format with XML tags such as <plan>, <step>, <scratch_pad>, <summary>, and <next_step> inside <thoughts>—and shows it improves performance over semi-structured and unstructured formats, especially on math and code. It further finds that 75% of the full dataset is better than 100%, that safety data is the most impactful domain, and that training on a small number of Thai-translated records lets the model reason in Thai without a catastrophic drop on English tasks.

Load-bearing premise

The pipeline assumes that the LLM-generated thinking traces correctly reflect the ground truth and teach genuine multi-step reasoning; if they merely mimic the format or contain incorrect steps, the benchmark improvements would not demonstrate real reasoning.

Editorial extensions

If this is right

  • An open, fully documented SFT recipe can produce a reasoning model with about 42k records and no RL, making reasoning-model training accessible outside large labs.
  • Structured thinking with XML scratchpads yields better math/code scores and more token-efficient reasoning than free-form or simply delimited thought.
  • Training with too much long-thinking data can hurt performance; the sweet spot in this setup was 75% of the mixture.
  • Adding a small set of Thai-translated traces (around 1.5k records) is enough to switch the model's thinking language to Thai while preserving most English benchmark performance.
  • Forcing the model to reason in one language degrades accuracy, so multilingual reasoning models should be allowed to pick their own thinking language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's result suggests that the value of RL for reasoning models may be largely in generating high-quality long thoughts, which can instead be curated once and distilled via SFT; a testable extension is to compare a model trained on these traces against an RL-trained baseline on the same base model.
  • The structured thinking format could be viewed as a lightweight inductive bias that encourages planning and self-checking; one could test this by ablating each tag (plan, scratch_pad, summary, next_step) to see which contributes the gains.
  • The leave-one-out results hint that the 'safety' domain in this mixture may proxy for general helpfulness and response formatting diversity rather than safety per se; a dataset-diversity ablation with matched sizes would separate those effects.
  • The Thai-translation experiment opens a low-cost path for other low-resource languages: translate a small subset of a high-quality thinking dataset and SFT, rather than building reasoning data from scratch.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper presents Typhoon T1, an open Thai reasoning model built by supervised fine-tuning Typhoon 2 3B Instruct on about 55,677 synthetic long-thinking records. The authors introduce a 'structured thinking' format using XML tags (<plan>, <step>, <title>, <scratch_pad>, <summary>, <next_step>) and a two-stage transformation-and-refinement pipeline (GPT-4o-mini few-shot transformation followed by Qwen2.5-32B-Instruct refinement) that converts existing ground-truth answers into long reasoning traces. The paper reports ablations on thinking format, dataset size, domain mixture, and Thai-translated data, claiming that structured thinking is best overall, that 75% of the full dataset is optimal, that safety data matters most, and that adding a small amount of Thai-translated data enables Thai reasoning traces at some cost to English benchmarks. All datasets, pipeline details, training configurations, and model weights are released.

Significance. If the central claim holds, the paper provides a valuable, reproducible, and cost-effective SFT-only recipe for building reasoning models in low-resource languages, an area where open documentation is scarce. The explicit release of the data mixture, training configuration, model weights, and benchmark evaluation setup is a genuine strength, as is the systematic ablation over thinking formats, data quantity, and domain composition. The structured-thinking idea is clearly described and could be useful beyond the Thai setting. However, the significance is currently limited by two load-bearing gaps: the correctness of the generated thinking traces is not validated, and the choice of the 75% dataset size is made on the same benchmarks used to report the final model's performance. These issues need to be addressed before the recipe can be taken as a reliable guide for future work.

major comments (4)
  1. [Section 2.2.1 and Section 2.2.2] The load-bearing assumption of the entire paper is that the transformation-and-refinement pipeline turns correct ground truths into correct long thinking traces that teach genuine reasoning. This is not validated. The paper reports no automatic check that the refined <response> still matches the original ground truth, no human evaluation of the logical validity of the generated steps, and no analysis of the PRM800K conversions where incorrect steps are deliberately included and then 'corrected.' If a substantial fraction of traces are incorrect or merely format-mimicking, the benchmark gains in Section 3.1 could reflect output length or dataset overlap rather than improved reasoning. Please add a validation study: for example, exact-match or LLM-judge consistency between the final answer and the original ground truth on a random sample, human ratings of step-level correctness, and a quantitative analysis of how often the PRM800K self-correction step actually corrects the deliberately wrong step.
  2. [Section 3.2, Figure 3, and Table 7] The 75% dataset size is selected by comparing model performance on the same six benchmarks that are later used to report the final Typhoon T1-EN results in Tables 3 and 8. Since the selection is made on the evaluation benchmarks themselves, the reported scores are optimistically biased. This is a load-bearing issue for the paper's claim about the optimal data quantity and for the final model's benchmark numbers. Please either evaluate the selected configuration on a held-out set that was not used for model selection, or explicitly frame the procedure as model selection and report the selected model's performance on a separate test set.
  3. [Section 3.1 and Table 2] The section title and text overclaim the benefits of structured thinking. The title says structured thinking improves mathematics and coding, but Table 2 shows semi-structured thinking outperforming structured thinking on HumanEval+ (72.87 vs. 69.76) and IFEval (55.27 vs. 53.60), while structured thinking is best only on GSM8K, GPQA, and MMLU Pro. The coding claim is therefore not supported by the table. In addition, all numbers come from single runs without error bars or multiple seeds; differences of 2-3 points on some benchmarks (e.g., GSM8K 62.02 vs. 57.24, GPQA 27.23 vs. 27.68) may be within run-to-run noise. Please either revise the title and text to a more accurate summary, or provide additional evidence, such as confidence intervals or repeated runs, before concluding that structured thinking is the best format.
  4. [Section 3.3 and Table 8] The conclusion that 'the safety domain has the most impact' is only partially supported by Table 8. Removing safety yields the largest drop in GSM8K, HumanEval+, IFEval, and ThaiExam, but removing coding also produces large drops (e.g., IFEval 41.96 vs. 41.59; ThaiExam 19.57 vs. 17.19), and on GPQA and MMLU Pro removing safety actually improves or matches the baseline. The differences are small in several columns (e.g., GSM8K 56.71 vs. 56.94), and without variance estimates these rankings may be unstable. Please report confidence intervals, multiple seeds, or a more nuanced interpretation that acknowledges the role of coding data and the mixed effect on multiple-choice benchmarks.
minor comments (7)
  1. [Title] The title contains a typo: 'A N OPEN' should be 'AN OPEN'.
  2. [Figure 5 caption] The caption reads 'This figures show domain distribution'; it should be 'This figure shows' or 'These figures show.'
  3. [Appendix A.10] The text says 'The exact prompt used for this purpose is provided bekiw'; 'bekiw' should be 'below.'
  4. [Figure 4] The figure caption refers to 'Typhoon T1 3B Instruct' as the baseline, but the rest of the paper consistently calls the baseline 'Typhoon 2 3B Instruct.' Please correct this inconsistency.
  5. [Appendix A.5] The three few-shot exemplars are hosted on pastes.io, which is ephemeral and not a stable archival location. They should be included in the paper or in the project's Hugging Face repository so that the data-generation pipeline is actually reproducible.
  6. [Table 3] The '+ 1.5k, CSFT' row shows a dramatic drop in MMLU Pro from 27.39 to 4.30, which is more than a 'significant reduction' and deserves a dedicated explanation; this pattern is not discussed in the text.
  7. [Section 3.1] The sentence about average response length says Typhoon T models generate longer responses 'except when prompting Typhoon 2 with zero-shot chain-of-thought reasoning'; as written this is confusing because the exception refers to the baseline model, not the Typhoon T models. Please rephrase.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper's claims are measured against external benchmarks and no result reduces to its inputs by construction.

full rationale

Typhoon T1 is an empirical SFT recipe evaluated on external benchmarks (GSM8K, HumanEval+, IFEval, GPQA, MMLU Pro) and a fixed Thai exam set. The central claims—that structured long-thinking SFT improves math and coding, that 75% of the dataset is the best size, and that Thai-translated SFT enables Thai reasoning traces—are established by measured benchmark deltas, not derived from definitions or from a fitted parameter renamed as a prediction. The authors' own Typhoon 2 model is used only as the starting checkpoint, and ThaiExam is a pre-existing fixed evaluation set; neither is defined in terms of the model's outputs, and no benchmark gain reduces to a quantity fitted on those same benchmarks. The mild concern that the 75% proportion was chosen using the same benchmarks later reported as final performance is selection on the test set rather than a circular derivation, and the core improvement claim holds across all data sizes. The absence of validation of the correctness of the LLM-generated thinking traces is a substantive correctness risk, but it is not a circularity because the evaluated gains are measured externally and are not defined by the traces. No load-bearing self-citation chain or equation-level reduction is present.

Assumptions & free parameters 7 free parameters · 5 assumptions · 1 invented entities

The central recipe depends on several hand-chosen hyperparameters (dataset size fraction, epochs, learning rate, filtering thresholds, data mixture) and on unverified assumptions about the fidelity of the synthetic long-thinking data and the validity of the benchmarks. These are the real costs of the paper's contribution: the reader must accept them to trust the reported gains.

free parameters (7)
  • Dataset size fraction = 75% (41,755 records)
    Selected via ablation over {100,75,50,25,10,5}% using the same six benchmarks used for final reporting; this is model selection on the eval set.
  • Training epochs = 2
    Fixed across all experiments by hand, with best checkpoint chosen on a 5% validation split.
  • Learning rate = 2e-5
    Chosen by hand; standard for full SFT with cosine scheduler and no warmup.
  • UltraFeedback rating threshold = 4.25
    Post-processing filter to select high-quality responses; threshold chosen by hand, not optimized.
  • Number of few-shot exemplars = 3
    Manually curated exemplars for transformation and refinement; not systematically varied.
  • Thai translated dataset size = 1,565 records
    Result of translating 25,000 records and filtering by heuristics; the retained count is treated as the training amount for Section 3.4.
  • Data mixture proportions = 55,677 total: 21,941 math, 13,188 instruction following, 10,814 code, 5,300 safety, 4,434 finance
    Designed by hand based on domain rationale; not optimized quantitatively except via leave-one-out after the fact.
assumptions (5)
  • domain assumption SFT on long-thinking traces can elicit reasoning behavior in a 3B model without RL
    Underlies the entire method; supported by prior SFT reasoning work but not re-established here.
  • domain assumption The six chosen benchmarks are valid proxies for reasoning ability
    Evaluation relies on GSM8K, HumanEval+, IFEval, GPQA, MMLU Pro, and ThaiExam; no uncertainty quantification or significance testing across seeds.
  • ad hoc to paper The transformation-and-refinement pipeline preserves answer correctness and teaches genuine reasoning
    Data quality is not measured; the pipeline uses GPT-4o mini and Qwen2.5-32B-Instruct with only three exemplars.
  • ad hoc to paper Randomly including incorrect PRM800K steps teaches self-correction without harmful noise
    Section 2.2.1: the conversion script randomly selects correct or incorrect steps and adds a correction step; correctness of these resamples is not verified.
  • ad hoc to paper Thai translation via fine-tuned Llama 3.1 8B and strict filtering yields usable training data
    Section 3.4 and A.10: only 1,565 of 25,000 translated records survive; translation quality and diversity are not evaluated.
invented entities (1)
  • Structured thinking XML tag schema (<plan>, <step>, <title>, <scratch_pad>, <summary>, <next_step>) independent evidence
    purpose: Guide the model to generate a plan, per-step scratchpads, summaries, and next-step decisions within <thoughts>
    A novel format introduced in Section 2.1; its benefit is tested in the Section 3.1 ablation where structured thinking improves several benchmarks, though on coding semi-structured scores higher.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Typhoon T1: An Open Thai Reasoning Model." pith.science (2026). https://pith.science/paper/AXO6EMVS

@misc{pith2026250209042,
  author       = {Pith},
  title        = {Pith review of: Typhoon T1: An Open Thai Reasoning Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AXO6EMVS}},
  note         = {Machine review of arXiv:2502.09042}
}
read the original abstract

This paper introduces Typhoon T1, an open effort to develop an open Thai reasoning model. A reasoning model is a relatively new type of generative model built on top of large language models (LLMs). A reasoning model generates a long chain of thought before arriving at a final answer, an approach found to improve performance on complex tasks. However, details on developing such a model are limited, especially for reasoning models that can generate traces in a low-resource language. Typhoon T1 presents an open effort that dives into the details of developing a reasoning model in a more cost-effective way by leveraging supervised fine-tuning using open datasets, instead of reinforcement learning. This paper shares the details about synthetic data generation and training, as well as our dataset and model weights. Additionally, we provide insights gained from developing a reasoning model that generalizes across domains and is capable of generating reasoning traces in a low-resource language, using Thai as an example. We hope this open effort provides a foundation for further research in this field.

Figures

Figures reproduced from arXiv: 2502.09042 by the authors.

Figure 1
Figure 1. Top: The transformation-and-refinement pipeline used for long-thinking data generation described in Sections 2.2.1 and 2.2.2. Bottom-Left: The structured long-thinking (the best thinking format) training pipeline for Typhoon T, as described in Section 3.1. Bottom-Right: The bilingual English-Thai Typhoon T1 model training pipeline detailed in Section 3.4. form response. This allows better control over thought qualit… view at source ↗
Figure 2
Figure 2. Differences between three thinking formats: (a) Unstructured thinking, where no XML [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Increasing the proportion of the training [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Final performance comparison of Ty￾phoon T1-EN and Typhoon T1 against the base￾line Typhoon T1 3B Instruct model across six evaluation benchmarks. We compare the final performance of each model against Typhoon T1 3B Instruct in Fig￾ure 4. Based on all experiments in th…
Figure 5
Figure 5. Figure 5: This figures show domain distribution of the training set for the experiments. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: This figure shows Typhoon T1’s Thai thinking trace and Typhoon T1-EN’s English think [PITH_FULL_IMAGE:figures/full_fig_p025_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 25 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [5]

    Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws . In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=FxNNiUgtfa

    work page 2025

  5. [6]

    Scaling test-time compute with open models , 2024

    Edward Beeching, Lewis Tunstall, and Sasha Rush. Scaling test-time compute with open models , 2024. URL https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling-test-time-compute

    work page 2024

  6. [7]

    Graph of Thoughts: Solving Elaborate Problems with Large Language Models

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of Thoughts: Solving Elaborate Problems with Large Language Models . Proceedings of the AAAI Conference on Artificial Intelligence, 38 0 (16): 0 17682--17690, Mar. 20...

    doi:10.1609/aaai.v38i16.29720 2024

  7. [8]

    Wealth Alpaca

    Gaurang Bharti. Wealth Alpaca . https://huggingface.co/datasets/gbharti/wealth-alpaca_lora, 2023

    work page 2023

  8. [9]

    Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning , 2025

    Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang. Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning , 2025. URL https://arxiv.org/abs/2412.09078

    arXiv 2025

Show all 65 references
  1. [10]

    Language Models are Few-Shot Learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey W...

    1901

  2. [11]

    Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations

    Nuo Chen, Zinan Zheng, Ning Wu, Ming Gong, Dongmei Zhang, and Jia Li. Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations . In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Lingu...

    2024 doi

  3. [12]

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks , 2023. URL https://arxiv.org/abs/2211.12588

    2023 arXiv

  4. [13]

    Training Verifiers to Solve Math Word Problems , 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training Verifiers to Solve Math Word Problems , 2021. URL https://arxiv.org/abs/2110.14168

    2021 arXiv

  5. [14]

    UltraFeedback: Boosting Language Models with Scaled AI Feedback , 2024

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. UltraFeedback: Boosting Language Models with Scaled AI Feedback , 2024. URL https://arxiv.org/abs/2310.01377

    2024 arXiv

  6. [15]

    FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning , 2023

    Tri Dao. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning , 2023. URL https://arxiv.org/abs/2307.08691

    2023 arXiv

  7. [16]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , 2025

    DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , 2025. URL https://arxiv.org/abs/2501.12948

    2025 arXiv

  8. [17]

    Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation

    Ruomeng Ding, Chaoyun Zhang, Lu Wang, Yong Xu, Minghua Ma, Wei Zhang, Si Qin, Saravan Rajmohan, Qingwei Lin, and Dongmei Zhang. Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation . In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Finding...

    2024 doi

  9. [18]

    Prompt-Based Monte Carlo Tree Search for Mitigating Hallucinations in Large Models , 2025

    Zhihua Duan and Jialin Wang. Prompt-Based Monte Carlo Tree Search for Mitigating Hallucinations in Large Models , 2025. URL https://arxiv.org/abs/2501.13942

    2025 arXiv

  10. [19]

    PAL: Program-aided Language Models , 2023

    Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL: Program-aided Language Models , 2023. URL https://arxiv.org/abs/2211.10435

    2023 arXiv

  11. [20]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  12. [21]

    OLMES: A Standard for Language Model Evaluations , 2024

    Yuling Gu, Oyvind Tafjord, Bailey Kuehl, Dany Haddad, Jesse Dodge, and Hannaneh Hajishirzi. OLMES: A Standard for Language Model Evaluations , 2024. URL https://arxiv.org/abs/2406.08446

    2024 arXiv

  13. [22]

    rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking , 2025

    Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking , 2025. URL https://arxiv.org/abs/2501.04519

    2025 arXiv

  14. [23]

    Measuring Mathematical Problem Solving With the MATH Dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring Mathematical Problem Solving With the MATH Dataset . In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track...

    2021

  15. [24]

    Liger Kernel: Efficient Triton Kernels for LLM Training , 2025

    Pin-Lun Hsu, Yun Dai, Vignesh Kothapalli, Qingquan Song, Shao Tang, Siyu Zhu, Steven Shimizu, Shivam Sahni, Haowen Ning, and Yanning Chen. Liger Kernel: Efficient Triton Kernels for LLM Training , 2025. URL https://arxiv.org/abs/2410.10989

    2025 arXiv

  16. [25]

    Tianze Hua, Tian Yun, and Ellie Pavlick. m O thello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models? In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL...

    2024 doi

  17. [26]

    Not All Languages Are Created Equal in LLM s: Improving Multilingual Capability by Cross-Lingual-Thought Prompting

    Haoyang Huang, Tianyi Tang, Dongdong Zhang, Xin Zhao, Ting Song, Yan Xia, and Furu Wei. Not All Languages Are Created Equal in LLM s: Improving Multilingual Capability by Cross-Lingual-Thought Prompting . In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Ass...

    2023 doi

  18. [27]

    AdaCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Chain-of-Thought , 2025

    Xin Huang, Tarun Kumar Vangani, Zhengyuan Liu, Bowei Zou, and Ai Ti Aw. AdaCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Chain-of-Thought , 2025. URL https://arxiv.org/abs/2501.16154

    2025 arXiv

  19. [28]

    O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson? , 2024

    Zhen Huang, Haoyang Zou, Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern, Shijie Xia, Yiwei Qin, Weizhe Yuan, and Pengfei Liu. O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson? , 2024. URL https://arxiv.org/ab...

    2024 arXiv

  20. [29]

    Challenges in Adapting Multilingual LLM s to Low-Resource Languages using L o RA PEFT Tuning

    Omkar Khade, Shruti Jagdale, Abhishek Phaltankar, Gauri Takalikar, and Raviraj Joshi. Challenges in Adapting Multilingual LLM s to Low-Resource Languages using L o RA PEFT Tuning . In Kengatharaiyer Sarveswaran, Ashwini Vaidya, Bal Krishna Bal, Sana Shams, and Surendrabikram T...

    2025

  21. [30]

    Large Language Models are Zero-Shot Reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large Language Models are Zero-Shot Reasoners . In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://o...

    2022

  22. [31]

    Understanding Catastrophic Forgetting in Language Models via Implicit Inference

    Suhas Kotha, Jacob Mitchell Springer, and Aditi Raghunathan. Understanding Catastrophic Forgetting in Language Models via Implicit Inference . In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=VrHiF2hsrm

    2024

  23. [32]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention . In Proceedings of the ACM SIGOPS 29th Symposium on Operating System...

    2023

  24. [33]

    Bespoke-Stratos: The unreasonable effectiveness of reasoning distillation

    Bespoke Labs. Bespoke-Stratos: The unreasonable effectiveness of reasoning distillation . www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation, 2025. Accessed: 2025-01-22

    2025

  25. [34]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luc...

    2025 arXiv

  26. [35]

    Teaching Small Language Models to Reason for Knowledge-Intensive Multi-Hop Question Answering

    Xiang Li, Shizhu He, Fangyu Lei, JunYang JunYang, Tianhuang Su, Kang Liu, and Jun Zhao. Teaching Small Language Models to Reason for Knowledge-Intensive Multi-Hop Question Answering . In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Comp...

    2024 doi

  27. [36]

    Let's Verify Step by Step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's Verify Step by Step . In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/f...

    2024

  28. [37]

    Is Your Code Generated by Chat GPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is Your Code Generated by Chat GPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation . In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openre...

    2023

  29. [38]

    Evaluating Language Models for Efficient Code Generation

    Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, and Lingming Zhang. Evaluating Language Models for Efficient Code Generation . In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=IBCBMeAhmC

    2024

  30. [39]

    An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning , 2025

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning , 2025. URL https://arxiv.org/abs/2308.08747

    2025 arXiv

  31. [40]

    WizardCoder: Empowering Code Large Language Models with Evol-Instruct , 2023

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. WizardCoder: Empowering Code Large Language Models with Evol-Instruct , 2023

    2023

  32. [41]

    s1: Simple test-time scaling , 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling , 2025. URL https://arxiv.org/abs/2501.19393

    2025 arXiv

  33. [42]

    Show Your Work: Scratchpads for Intermediate Computation with Language Models , 2021

    Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. Show Your Work: Scratchpads for Intermediate Computation with Language Models , 2021....

    2021 arXiv

  34. [43]

    Skywork-o1 Open Series

    Skywork o1 Team. Skywork-o1 Open Series . https://huggingface.co/Skywork, November 2024. URL https://huggingface.co/Skywork

    2024

  35. [44]

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Mi...

    2025 arXiv

  36. [45]

    Typhoon: Thai Large Language Models

    Kunat Pipatanakul, Phatrasek Jirabovonvisut, Potsawee Manakul, Sittipong Sripaisarnmongkol, Ruangsak Patomwong, Pathomporn Chokchainant, and Kasima Tharnpipitchai. Typhoon: Thai Large Language Models . arXiv preprint arXiv:2312.13951, 2023

    2023 arXiv

  37. [46]

    Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models , 2024

    Kunat Pipatanakul, Potsawee Manakul, Natapong Nitarach, Warit Sirichotedumrong, Surapon Nonesung, Teetouch Jaknamon, Parinthapat Pengpun, Pittawat Taveekitworachai, Adisai Na-Thalang, Sittipong Sripaisarnmongkol, Krisanapong Jirayoot, and Kasima Tharnpipitchai. Typhoon 2: A Fa...

    2024 arXiv

  38. [47]

    O1 Replication Journey: A Strategic Progress Report -- Part 1 , 2024

    Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, and Pengfei Liu. O1 Replication Journey: A Strategic Progress Report -- Part 1 , 2024. URL https://arxiv.org/abs/2410.18982

    2024 arXiv

  39. [48]

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct Preference Optimization: Your Language Model is Secretly a Reward Model . In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Ne...

    2023

  40. [49]

    Rush, and Thomas Wolf

    Nazneen Rajani, Lewis Tunstall, Edward Beeching, Nathan Lambert, Alexander M. Rush, and Thomas Wolf. No Robots . https://huggingface.co/datasets/HuggingFaceH4/no_robots, 2023

    2023

  41. [50]

    ZeRO: Memory optimizations Toward Training Trillion Parameter Models

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. ZeRO: Memory optimizations Toward Training Trillion Parameter Models . In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp.\ 1--16, 2020. doi:10.1109/SC41405.2020.00024

    2020 arXiv

  42. [51]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA : A Graduate-Level Google-Proof Q&A Benchmark . In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=Ti67584b98

    2024

  43. [52]

    Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters , 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters , 2024. URL https://arxiv.org/abs/2408.03314

    2024 arXiv

  44. [53]

    Sky-T1: Train your own O1 preview model within \ 450

    NovaSky Team. Sky-T1: Train your own O1 preview model within \ 450 . https://novasky-ai.github.io/posts/sky-t1, 2025. Accessed: 2025-01-09

    2025

  45. [54]

    Ni, Linyi Yang, Ying Wen, and Weinan Zhang

    Jun Wang, Meng Fang, Ziyu Wan, Muning Wen, Jiachen Zhu, Anjie Liu, Ziqin Gong, Yan Song, Lei Chen, Lionel M. Ni, Linyi Yang, Ying Wen, and Weinan Zhang. OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models , 2024 a . URL https://arxiv.org/abs/2410.09671

    2024 arXiv

  46. [55]

    Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models

    Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models . In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st ...

    2023 doi

  47. [56]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-Consistency Improves Chain of Thought Reasoning in Language Models . In The Eleventh International Conference on Learning Representations, 2023 b . URL https...

    2023

  48. [57]

    MMLU -Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU -Pro: A More Robust and Challenging Multi-Task Language Under...

    2024

  49. [58]

    HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM , 2023 c

    Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Polak Scowcroft, Neel Kant, Aidan Swope, and Oleksii Kuchaiev. HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM , 2023 c . URL https://arxiv.org/abs/2311.09528

    2023 arXiv

  50. [59]

    Chi, Quoc V Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of Thought Prompting Elicits Reasoning in Large Language Models . In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in ...

    2022

  51. [60]

    Thinking LLMs: General Instruction Following with Thought Generation , 2024

    Tianhao Wu, Janice Lan, Weizhe Yuan, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. Thinking LLMs: General Instruction Following with Thought Generation , 2024. URL https://arxiv.org/abs/2410.10630

    2024 arXiv

  52. [61]

    Griffiths, Yuan Cao, and Karthik R Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. Tree of Thoughts: Deliberate Problem Solving with Large Language Models . In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openrev...

    2023

  53. [62]

    OpenThaiGPT 1.5: A Thai-Centric Open Source Large Language Model , 2024

    Sumeth Yuenyong, Kobkrit Viriyayudhakorn, Apivadee Piyatumrong, and Jillaphat Jaroenkantasima. OpenThaiGPT 1.5: A Thai-Centric Open Source Large Language Model , 2024. URL https://arxiv.org/abs/2411.07238

    2024 arXiv

  54. [63]

    LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System...

    2024 arXiv

  55. [64]

    Instruction-Following Evaluation for Large Language Models , 2023 a

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-Following Evaluation for Large Language Models , 2023 a . URL https://arxiv.org/abs/2311.07911

    2023 arXiv

  56. [65]

    Thread of Thought Unraveling Chaotic Contexts , 2023 b

    Yucheng Zhou, Xiubo Geng, Tao Shen, Chongyang Tao, Guodong Long, Jian-Guang Lou, and Jianbing Shen. Thread of Thought Unraveling Chaotic Contexts , 2023 b . URL https://arxiv.org/abs/2311.08734

    2023 arXiv

  57. [66]

    Tab- C o T : Zero-shot Tabular Chain of Thought

    Jin Ziqi and Wei Lu. Tab- C o T : Zero-shot Tabular Chain of Thought . In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp.\ 10259--10277, Toronto, Canada, July 2023. Association for Computation...

    2023 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.