REVIEW 4 cited by
Typhoon: Thai Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Typhoon is a series of Thai large language models (LLMs) developed specifically for the Thai language. This technical report presents challenges and insights in developing Thai LLMs, including data preparation, pretraining, instruction-tuning, and evaluation. As one of the challenges of low-resource languages is the amount of pretraining data, we apply continual training to transfer existing world knowledge from a strong LLM. To evaluate the Thai knowledge encapsulated in each model from the pretraining stage, we develop ThaiExam, a benchmark based on examinations for high-school students and investment professionals in Thailand. In addition, we fine-tune Typhoon to follow Thai instructions, and we evaluate instruction-tuned models on Thai instruction datasets as well as translation, summarization, and question-answering tasks. Experimental results on a suite of Thai benchmarks show that Typhoon outperforms all open-source Thai language models, and its performance is on par with GPT-3.5 in Thai while having only 7 billion parameters and being 2.62 times more efficient in tokenizing Thai text.
Forward citations
Cited by 4 Pith papers
-
Mangosteen: An Open Thai Corpus for Language Model Pretraining
An open 47B-token Thai pre-training corpus and a Thai-adapted data cleaning pipeline, with ablations showing quality gains and an 8B model that improves on Thai benchmarks.
-
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
An adaptive, per-language data filtering and deduplication pipeline produces multilingual LLM pre-training corpora that beat prior public datasets on 11 of 14 evaluated languages, and a 20TB, 1,868 language-script dat...
-
Weakly Supervised Data Refinement and Flexible Sequence Compression for Efficient Thai LLM-based ASR
EThai-ASR combines a self-refined Zipformer encoder with a Thai LLM and reports SOTA CER on Thai test sets plus a cosine-similarity frame pruning that gives 1.5-2.1x speedups in some modes.
-
CortexDebate: Debating Sparsely and Equally for Multi-Agent Debate
CortexDebate prunes the multi-agent debate graph every round using a McKinsey-style trust score per directed link, reporting accuracy gains over full-debate baselines on eight datasets with shorter per-agent contexts.
Discussion (0). Sign in to comment.