Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

TTS-1 Technical Report

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that an 8.8-billion-parameter speech-language model, trained with pre-training, supervised fine-tuning, and GRPO alignment, produces 48 kHz multilingual speech and is preferred over several commercial TTS systems in a…

desk verdict A genuinely useful open TTS training recipe that overclaims 'state-of-the-art' on an arena too small and too internally controlled to support it. read the letter →

arxiv 2507.21138 v1 pith:BAKCGOM6 submitted 2025-07-22 cs.CL cs.AIcs.LGcs.SDeess.AS

Oleg Atamanenko , Anna Chalova , Joseph Coombes , Nikki Cope , Phillip Dang , Zhifeng Deng , Jimmy Du , Michael Ermolenko
show 24 more authors
Feifan Fan Yufei Feng Cheryl Fichter Pavel Filimonov Louis Fischer Kylan Gibbs Valeria Gusarova Pavel Karpik Andreas Assad Kottner Ian Lee Oliver Louie Jasmine Mai Mikhail Mamontov Suri Mao Nurullah Morshed Igor Poletaev Florin Radu Dmytro Semernia Evgenii Shingarev Vikram Sivaraja Peter Skirko Rinat Takhautdinov Robert Villahermosa Jean Wang
This is my paper
classification cs.CLcs.AIcs.LGcs.SDeess.AS
keywords text-to-speechspeech-languagemodelvoicecloningGroupRelativePolicyOptimizationreinforcementlearningalignmentaudiocodecstreamingspeechsynthesismarkupcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the standard large-language-model recipe—large-scale pre-training on raw audio, supervised fine-tuning on filtered high-quality pairs, and reinforcement-learning alignment with Group Relative Policy Optimization—transfers directly to speech synthesis. The authors introduce two autoregressive speech-language models that clone an unseen speaker from a short reference clip and produce 48 kHz audio in 11 languages, with eight speaking styles and seven non-verbal vocalizations controlled by text markups. If the reported evaluations hold, the larger 8.8B model wins 55 to 61 percent of blind head-to-head votes against several commercial TTS systems, and the 1.6B model is fast enough for real-time streaming and on-device use. Training, modeling, and benchmarking code is released under an MIT license, while the trained weights are withheld.

What carries the argument

The argument rides on three coupled mechanisms. First, the SpeechLM training pipeline: pre-training on 1M hours of raw audio mixed with roughly 20B text tokens, SFT on 200k hours of filtered pairs, then GRPO with reward $R(p,c)=\alpha R_{wer}(c)+\beta R_{similarity}(p,c)+\gamma R_{dnsmos}(c)$. Second, a 48 kHz audio codec built on the X-codec2 architecture with a super-resolution decoder and an added RMS loudness loss, which turns audio and semantic information into a single 65,536-token codebook. Third, the streaming and control layer: concatenation only at non-voicing regions, decoder context extension to stabilize volume and speaker identity, and LoRA fine-tuning on pairs of neutral and stylized same-speaker utterances so text markups like [whispering] or [laugh] work without an explicit speaker embedding.

What would settle it

Re-run the paper's speaker-similarity evaluation with the same reference clips but target sentences taken from a different recording, different room, or different microphone; if SIM scores drop substantially below the reported continuation-based values, the purely in-context cloning claim is overstated.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a sequential three-stage training pipeline—pre-training a SpeechLM on roughly one million hours of raw audio mixed with text, fine-tuning it on 200,000 hours of DNSMOS-filtered, pace-filtered audio–text pairs, then aligning it with GRPO against a composite reward of word error rate, speaker similarity, and DNSMOS—is what separates the models from earlier TTS systems. The audio decoder is upgraded to 48 kHz with a super-resolution module and an RMS loudness loss, and the LLaMA-based backbone's vocabulary is expanded with 65,536 audio tokens. Because the model learns voices purely in context, no speaker embedding model is needed for cloning; emotional control is taught separately by pairing neutral and stylized utterances from the same speaker during a LoRA fine-tune. The headline evidence is an internal blind arena in which TTS-1-Max is preferred in 59.1% of comparisons against 11LABS Multilingual V2, 60.9% against Cartesia Sonic 2, and 60.7% against OpenAI TTS-1-HD.

Load-bearing premise

The main load-bearing assumption is that the voice-cloning evaluation is realistic: each utterance is split in half, the model hears the first half as the reference and is scored on how well it continues the second half of the same recording, with the same speaker, microphone, and acoustic conditions.

Editorial extensions

If this is right

  • TTS-1-Max is preferred in 59.1% of blind comparisons against 11LABS Multilingual V2, 60.9% against Cartesia Sonic 2, 60.7% against OpenAI TTS-1-HD, and 55.3% against TTS-1 in the internal arena.
  • RL alignment improves content accuracy over SFT alone: average WER drops from 7.9% to 6.3% for TTS-1 and from 6.9% to 5.1% for TTS-1-Max on the English length-varying sets.
  • The streaming pipeline cuts time-to-first-two-seconds by about 70% relative to a vanilla vLLM-based serving stack while keeping 48 kHz audio.
  • A single model can clone voices, stream in real time, and switch emotional delivery via markup tags, with markups generalizing to non-English languages at reduced fidelity per the paper's human preference checks.
  • Because the reward framework is modular, the same GRPO loop can later add style, emotion, or non-verbal rewards without retraining from scratch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The bisection protocol likely makes reported speaker-similarity numbers an optimistic upper bound for real deployments; a disjoint reference utterance with different target text would give a stricter test.
  • The same three-stage recipe may transfer to other backbone sizes and codecs; trying GRPO on a smaller pre-trained SpeechLM would test whether the alignment stage is the main driver.
  • Conditional reward activation—applying style or emotion rewards only when markup tags are present—could make emotional control a trained behavior rather than a side effect of LoRA pairing, which the paper leaves untested.
  • An independent blind arena with fixed prompt sets and varied speakers would show whether the reported win rates persist outside the internal setup.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Inworld TTS-1 and TTS-1-Max, two autoregressive Transformer-based TTS models built on LLaMA backbones (1B and 8B SpeechLM components, 1.6B and 8.8B total parameters). It describes an X-codec2-based audio codec extended to 48 kHz with a super-resolution module, a three-stage training pipeline (pre-training on ~1M hours of audio, SFT on ~200k hours of filtered audio-text pairs, and GRPO-based RL alignment with a composite WER/SIM/DNSMOS reward), audio markup tags for speaking styles and non-verbal vocalizations, and a streaming inference pipeline. The evaluation section reports internal multilingual WER/SIM results on a Gemini-generated 1100-sentence benchmark, English WER results across input lengths, and an internal blind arena with about 20 annotators and over 400 votes comparing the two models against three commercial TTS systems. The central claim is that both models achieve state-of-the-art quality and exceptional zero-shot voice cloning purely through in-context learning.

Significance. If the central claims were fully supported, this would be a significant systems contribution: the paper gives an unusually transparent account of the full training pipeline, releases training and modeling code under an MIT license, reports detailed infrastructure and hyperparameters, and describes practical techniques for 48 kHz streaming synthesis and style control. The three-stage recipe and the modular GRPO reward framework are potentially reusable. However, the evidence for the headline 'state-of-the-art' and 'exceptional quality' claims is currently limited: the main external comparison is an underpowered internal arena, the primary objective metrics overlap with the RL reward, and the voice-cloning protocol uses same-recording continuation rather than a true zero-shot setup. The paper is more convincing as an engineering report than as a validated empirical demonstration of SOTA status.

major comments (4)
  1. [Abstract; §4.2, Table 9] The headline claim that 'both models achieve state-of-the-art performance' is not supported by the reported arena data. The decisive cells for TTS-1-Max are 59.1% (26/44) vs 11Labs, 60.9% (28/46) vs Cartesia, 60.7% (17/28) vs OpenAI, and 55.3% (26/47) vs Inworld TTS-1; TTS-1's best cells are similar in size. With n between 28 and 52, a 60% observed win rate has a 95% confidence interval of roughly ±14-18 percentage points, so a true 50% win rate (parity) is not excluded for any pairing. No p-values, confidence intervals, or inter-annotator agreement are reported. Please add a proper statistical treatment, report the number of unique texts and how speakers/texts were matched, and either provide adequately powered evidence or temper the abstract's SOTA claim to 'competitive'.
  2. [§3.5, Eqs. (8)-(11); §4, Fig. 8, Table 8] There is a circularity between the RL reward and the main objective evaluation. The reward function is a weighted sum of WER, speaker similarity, and DNSMOS, and the quality evaluation in Section 4 reports WER and SIM as primary evidence of improvement. Improvements on these metrics after RL are expected by construction because the model is optimized against them; they do not independently confirm perceptual superiority. The only human evaluation is the small internal arena, and DNSMOS is not re-evaluated in the final quality section. Please add an external human preference study or an independent benchmark that was not used in the reward, or explicitly frame the Section 4 results as reward-alignment outcomes rather than independent quality measurements.
  3. [§3.3, Fig. 4; §4, Fig. 8] The voice-cloning evaluation bisects each utterance: the first half is used as the reference prompt and the model generates the second half, which is compared with the ground truth from the same recording. This protocol fixes the speaker, microphone, and acoustic environment and uses the ground-truth continuation as the target text, so it is not representative of zero-shot cloning with a different reference utterance and different target text. The reported SIM values are therefore likely to overstate practical cloning fidelity. Please evaluate on a disjoint reference/target benchmark and connect the result to the Section 7 caveat that speaker similarity can fluctuate with emotionally expressive speech delivery.
  4. [§4, first paragraph] The 1100-sentence multilingual benchmark is generated with Gemini 2.5 Pro and contains no external TTS systems; it only compares TTS-1 against TTS-1-Max. The claim of 'state-of-the-art performance on a variety of benchmarks' therefore rests on this internal comparison plus the underpowered arena. Please add standard public benchmarks with published baselines (e.g., common English and multilingual TTS test sets) or revise the claim to describe the results as internal comparisons only.
minor comments (6)
  1. [§5.1] The text says 'Given the fact (in Table. 9) that audio decoding is much faster than audio token generation'; this should reference Figure 9, since Table 9 is the head-to-head arena table.
  2. [§3.5] The claim that disabling reward scaling in GRPO makes training 'more robust' is stated without an ablation or a comparison to the scaled-reward baseline; please provide the supporting experiment or cite a specific result from reference [43] that covers this setting.
  3. [§3.6] The non-English generalization of audio markup tags is described only as an observed qualitative phenomenon ('emergent capability ... although with reduced fidelity'); if this is part of the contribution, please quantify it or mark it explicitly as anecdotal.
  4. [§5.3] The limitation that caching reference prompt audio tokens can cause emotional and stylistic bleed from the reference into the generated speech is important, but it is not tied to any measurement; please give an example or an internal evaluation illustrating the effect.
  5. [§6] The paper says it open-sources training, modeling, and benchmarking code but does not release model weights; please clarify in the text what 'open-source' means for reproducibility, since the safety-motivated weight withholding limits direct replication.
  6. [§3.4, Table 4] The statement that initializing the SFT learning rate to the final pre-training learning rate was 'crucial' is made without an ablation; please add the comparison or soften the claim.

Circularity Check

1 steps flagged · score 6.0 of 10

RL reward and final quality evaluation use the same WER/SIM metrics, making part of the quality claim a re-statement of the training objective.

  1. fitted input called prediction [Section 3.5 (Reward function design, Eqs. 8–11) and Section 4 (Quality Evaluation, Figure 8 / Table 8)]
    "R(p, c) = αRwer(c) + βRsimilarity (p, c) + γRdnsmos(c) (8) ... The quality was assessed using the same Word Error Rate (WER) and Speaker Similarity (SIM) metrics described previously."

    Equations 9–11 define the RL rewards as monotone transformations of WER, WavLM speaker-embedding cosine similarity, and DNSMOS. Section 4 then reports “the same WER and SIM metrics described previously” as the main quality evidence, and Table 8 explicitly attributes WER improvements to RL alignment. Because GRPO is run to maximize a composite of these reward terms, post-RL gains on WER and SIM are expected consequences of the training objective, not independent confirmation of quality. The internal human arena is the only external signal, but the objective-metric portion of the “exceptional quality” and “state-of-the-art” claim reduces by construction to the reward function used to train the model.

full rationale

The one clear circular step is that the model is RL-optimized against exactly the metrics later presented as proof of quality: WER, speaker similarity, and DNSMOS enter the reward function in Eqs. 8–11, and Section 4 evaluates quality using “the same WER and SIM metrics described previously.” Table 8 even labels the comparison as SFT-only versus the complete pipeline with RL alignment, so those WER improvements are a check that the optimization worked, not an independent measurement. The internal human arena (Table 9) is independent of the reward metrics, which keeps the paper from being fully circular, but it is small (28–52 judgments per pairing) and internally run; that is a statistical limitation rather than circularity. No load-bearing self-citation chain is present: the architecture, codec, and training-method citations are external prior work. The codec DNSMOS evaluation and the decoder-prompt SIM comparison are not circular because those components were not optimized on those specific metrics. Overall, the central quality claim is partially circular because a substantial portion of its evidence reduces by construction to the reward objective, while the independent arena evidence is too under-powered to carry the headline claim by itself.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claims rest on a small number of hand-chosen hyperparameters (λRMS, k, reward weights, codec strides, evaluation decoding settings) and on several domain assumptions about the validity of automatic metrics (WavLM SIM, DNSMOS, Whisper WER) that are used both as rewards and as evaluation criteria. The audio markup training strategy is an ad hoc data construction that the paper itself introduced to overcome a failed initial approach. No new physical or conceptual entities are postulated.

free parameters (6)
  • λRMS loudness loss weight = 1.0
    Chosen empirically in Section 3.2 for both 24 kHz and 48 kHz audio; affects decoder volume consistency and thus perceived quality.
  • k in WER reward = 2.5
    Controls sensitivity of Rwer(c) = exp(-k·WER) in Section 3.5; no derivation given, set by hand.
  • Reward weights α, β, γ = α=β=γ=1.0
    In Section 3.5 the combined RL model uses equal weights for WER, similarity, and DNSMOS; the paper notes these can be tuned, so the reported result depends on this choice.
  • 48 kHz decoder stride/hop configuration = strides 3,2; hop 160
    In Section 2.1 the super-resolution strides and iSTFT hop length are empirically chosen (the authors note they 'empirically found' better quality with a larger hop length than prior work).
  • Evaluation decoding parameters = topk=50, topp=1.0, temperature=1.0
    Used for the pre-training evaluation in Section 3.3; decoding settings affect WER/SIM results, and the paper states inference parameters must be tuned per use case.
  • SFT data filtering thresholds = 20% lowest DNSMOS removed; 5% fastest/slowest CPS removed
    Section 3.4 describes these thresholds as part of the multi-stage filtering that produced the 200k-hour SFT set; they are heuristic choices that affect which data the model sees.
assumptions (6)
  • domain assumption WavLM-large speaker embeddings (cosine similarity) reliably measure speaker similarity.
    Used as the SIM metric in Sections 3.3 and 4; if WavLM similarity does not track human perception of voice identity, the reported SIM gains are not meaningful.
  • domain assumption DNSMOS scores align with human perceptual quality.
    Used both as an RL reward and as the quality metric for codec sample-rate comparison in Section 3.2; the paper relies on DNSMOS as a proxy for listening quality.
  • domain assumption Whisper-large-v3 word/character error rate accurately measures content accuracy of synthesized speech.
    Whisper is used as the ASR in the WER reward and evaluation; ASR errors could be mistaken for synthesis errors or vice versa.
  • domain assumption The X-codec2 50-token-per-second single-codebook representation retains enough information for high-fidelity 48 kHz speech.
    Section 2.1 assumes the codec, originally 16 kHz, can be extended to 48 kHz with a super-resolution decoder while preserving quality.
  • ad hoc to paper Pairing neutral and stylized utterances from the same speaker teaches style control without harming speaker identity.
    Section 3.6 introduces this data construction to solve the observed failure of simple style-tag prepending; no independent evidence is given that this strategy generalizes across speakers and languages.
  • ad hoc to paper The GRPO advantage without reward scaling is more robust than with scaling.
    Section 3.5 disables reward scaling following [43]; the paper adopts this as a default for TTS without a dedicated ablation on speech data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TTS-1 Technical Report." pith.science (2026). https://pith.science/paper/BAKCGOM6

@misc{pith2026250721138,
  author       = {Pith},
  title        = {Pith review of: TTS-1 Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BAKCGOM6}},
  note         = {Machine review of arXiv:2507.21138}
}
read the original abstract

We introduce Inworld TTS-1, a set of two Transformer-based autoregressive text-to-speech (TTS) models. Our largest model, TTS-1-Max, has 8.8B parameters and is designed for utmost quality and expressiveness in demanding applications. TTS-1 is our most efficient model, with 1.6B parameters, built for real-time speech synthesis and on-device use cases. By scaling train-time compute and applying a sequential process of pre-training, fine-tuning, and RL-alignment of the speech-language model (SpeechLM) component, both models achieve state-of-the-art performance on a variety of benchmarks, demonstrating exceptional quality relying purely on in-context learning of the speaker's voice. Inworld TTS-1 and TTS-1-Max can generate high-resolution 48 kHz speech with low latency, and support 11 languages with fine-grained emotional control and non-verbal vocalizations through audio markups. We additionally open-source our training and modeling code under an MIT license.

Figures

Figures reproduced from arXiv: 2507.21138 by the authors.

Figure 1
Figure 1. The architecture of Inworld TTS-1. The audio encoder tokenizes a refer￾ence audio into a sequence of discrete audio tokens. These tokens are concatenated with the tokenized reference text and the text to be synthesized to form a prompt for the SpeechLM. The SpeechLM autoregressively generates audio tokens, which are then converted back into a 48 kHz waveform by the audio decoder. Inworld TTS-1 and TTS-1-Max are buil… view at source ↗
Figure 2
Figure 2. Speech data quantities per language and dataset. 3 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Pre-training loss for the TTS-1 and TTS-1-Max SpeechLM models. For training we use a standard technique to perform unsupervised language modeling via next token prediction [31] using the hyperparameters shown in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Performance of TTS-1 and TTS-1-Max pre-trained SpeechLM check￾points on the test set. Cross-language results are not directly comparable as each language subset contains distinct utterances. 3.4 SpeechLM Supervised Fine-Tuning For supervised fine-tuning (SFT), we adopt…
Figure 5
Figure 5. Figure 5: Ablation study on the impact of audio pre-training for SFT of TTS-1. The plot compares SFT loss curves for models initialized from the base LLaMA-3.2-1B checkpoint versus an audio pre-trained version. Both were trained on ∼100k audio hours under identical settings. 7 …
Figure 6
Figure 6. Figure 6: Training curves for GRPO experiments. Left: separate reward signal training, where each model was trained with a single reward (WER, Similarity, or DNSMOS) and evaluated on the corresponding metric. Right: unified model with combined reward function. 3.6 Audio Markups …
Figure 7
Figure 7. Figure 7: Histogram of occurrences of each audio markup tag. Experiment setup. A series of ablation studies revealed that both TTS-1 and TTS-1-Max SpeechLMs achieve better generalization, as indicated by lower evaluation loss, when fine-tuned using Low-Rank Adaptation (LoRA) [22…
Figure 8
Figure 8. Figure 8: Multilingual evaluation results comparing WER and SIM scores by language. Left: WER (lower is better). Right: SIM (higher is better). 4.1 Evaluation on Varying Input Lengths To further assess the models’ performance on English, we conducted an evaluation across three d…
Figure 9
Figure 9. Figure 9: Audio decoder latency comparison across different sample rates and audio durations. 5.1 Decoder While streaming support helped to reduce the time to get the first audio chunk, it also introduced new challenges for the decoder. Specifically, the decoder must generate au…
Figure 10
Figure 10. Figure 10: P90 latency comparison for the first 2-second audio chunk between the vanilla vLLM solution and the Modular-optimized implementation. As a result of these combined optimizations, the streaming API delivers the first two seconds of synthesized audio on average 70% fast…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating and Rewarding LALMs for Expressive Role-Play TTS via Mean Continuation Log-Probability

    cs.SD 2026-01 conditional novelty 6.0 of 10

    A new metric, MCLP, uses a frozen audio-LLM's continuation likelihood to grade speaking-style consistency and doubles as a reward that improves role-play TTS on a new 1,435-hour drama dataset.

Reference graph

Works this paper leans on

48 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    The people’s speech: A large-scale diverse english speech recognition dataset for commercial usage

    Daniel Galvez, Greg Diamos, Juan Ciro, Juan Felipe Cerón, Keith Achorn, Anjali Gopi, David Kanter, Maximilian Lam, Mark Mazumder, and Vijay Janapa Reddi. The people’s speech: A large-scale diverse english speech recognition dataset for commercial usage. arXiv preprint arXiv:2111.09344, 2021

    arXiv 2021

  2. [2]

    Yodas: Youtube-oriented dataset for audio and speech

    Xinjian Li, Shinnosuke Takamichi, Takaaki Saeki, William Chen, Sayaka Shiota, and Shinji Watanabe. Yodas: Youtube-oriented dataset for audio and speech. pages 1–8, 2023

    work page 2023

  3. [3]

    Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation

    Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, et al. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation. pages 885–890, 2024

    work page 2024

  4. [4]

    Styletts 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models

    Yinghao Aaron Li, Cong Han, Vinay Raghavan, Gavin Mischler, and Nima Mesgarani. Styletts 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models. Advances in Neural Information Processing Systems , 36:19594–19621, 2023

    work page 2023

  5. [5]

    Fastspeech: Fast, robust and controllable text to speech

    Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. Fastspeech: Fast, robust and controllable text to speech. Advances in neural information processing systems , 32, 2019

    work page 2019

  6. [6]

    Better speech synthesis through scaling

    James Betker. Better speech synthesis through scaling. arXiv preprint arXiv:2305.07243, 2023

    arXiv 2023

  7. [7]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech

    Jaehyeon Kim, Jungil Kong, and Juhee Son. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. pages 5530–5540, 2021

    work page 2021

  8. [8]

    V oicebox: Text-guided multilingual universal speech generation at scale

    Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, et al. V oicebox: Text-guided multilingual universal speech generation at scale. Advances in neural information processing systems , 36:14005–14034, 2023

    work page 2023

Show all 48 references
  1. [9]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  2. [10]

    Training compute-optimal large language models

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022

    2022 arXiv

  3. [11]

    Neural codec language models are zero-shot text to speech synthesizers

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111, 2023

    2023 arXiv

  4. [12]

    Minimax-speech: Intrinsic zero-shot text-to-speech with a learnable speaker encoder

    Bowen Zhang, Congchao Guo, Geng Yang, Hang Yu, Haozhe Zhang, Heidi Lei, Jialong Mai, Junjie Yan, Kaiyue Yang, Mingqi Yang, et al. Minimax-speech: Intrinsic zero-shot text-to-speech with a learnable speaker encoder. arXiv preprint arXiv:2505.07916, 2025

    2025 arXiv

  5. [13]

    Seed-tts: A family of high-quality versatile speech generation models

    Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, et al. Seed-tts: A family of high-quality versatile speech generation models. arXiv preprint arXiv:2406.02430, 2024

    2024 arXiv

  6. [14]

    Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens

    Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407, 2024

    2024 arXiv

  7. [15]

    Cosyvoice 2: Scalable streaming speech synthesis with large language models

    Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, et al. Cosyvoice 2: Scalable streaming speech synthesis with large language models. arXiv preprint arXiv:2412.10117, 2024

    2024 arXiv

  8. [16]

    Redpajama: an open dataset for training large language models

    Maurice Weber, Dan Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, et al. Redpajama: an open dataset for training large language models. Advances in neural information processing systems , 37:116462–116492, 2024

    2024

  9. [17]

    Open instruction generalist (oig) dataset

    LAION. Open instruction generalist (oig) dataset. https://laion.ai/blog/oig-dataset/, 2023

    2023

  10. [18]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv. org/abs/2402.03300. 18

    2024 arXiv

  11. [19]

    Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis

    Zhen Ye, Xinfa Zhu, Chi-Min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi DAI, et al. Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis. arXiv preprint arXiv:2502.04128, 2025

    2025 arXiv

  12. [20]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing , 16(6)...

    2022

  13. [21]

    Dnsmos p

    Chandan KA Reddy, Vishak Gopal, and Ross Cutler. Dnsmos p. 835: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. In ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages 886–890. IEEE, 2022

    2022

  14. [22]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  15. [23]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770–778, 2016

    2016

  16. [24]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  17. [25]

    Matrix multiplication background user’s guide

    NVIDIA Corporation. Matrix multiplication background user’s guide. https://docs.nvidia. com/deeplearning/performance/dl-performance-matrix-multiplication/index.html# requirements-tc, 2023

    2023

  18. [26]

    Initializing new word embeddings for pretrained language models

    John Hewitt. Initializing new word embeddings for pretrained language models. https://nlp. stanford.edu/~johnhew//vocab-expansion.html, 2021

    2021

  19. [27]

    Bigcodec: Pushing the limits of low-bitrate neural speech codec

    Detai Xin, Xu Tan, Shinnosuke Takamichi, and Hiroshi Saruwatari. Bigcodec: Pushing the limits of low-bitrate neural speech codec. arXiv preprint arXiv:2409.05377, 2024

    2024 arXiv

  20. [28]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis

    Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems , 33:17022–17033, 2020

    2020

  21. [29]

    High fidelity neural audio compres- sion

    Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compres- sion. arXiv preprint arXiv:2210.13438, 2022

    2022 arXiv

  22. [30]

    Adding instruc- tions during pretraining: Effective way of controlling toxicity in language models

    Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. Adding instruc- tions during pretraining: Effective way of controlling toxicity in language models. arXiv preprint arXiv:2302.07388, 2023

    2023 arXiv

  23. [31]

    A neural probabilistic language model

    Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137–1155, 2003

    2003

  24. [32]

    Flashattention-2: Faster attention with better parallelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023

    2023 arXiv

  25. [33]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  26. [34]

    Pytorch distributed: Experiences on accelerating data parallel training

    Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, et al. Pytorch distributed: Experiences on accelerating data parallel training. arXiv preprint arXiv:2006.15704, 2020

    2006 arXiv

  27. [35]

    Pytorch fsdp: experiences on scaling fully sharded data parallel

    Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, et al. Pytorch fsdp: experiences on scaling fully sharded data parallel. arXiv preprint arXiv:2304.11277, 2023

    2023 arXiv

  28. [36]

    Parler-tts

    Lacombe Yoach, Vaibhav Srivastav, and Sanchit Gandhi. Parler-tts. https://github.com/ huggingface/parler-tts, 2024

    2024

  29. [37]

    Zero: Memory optimizations toward training trillion parameter models

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis , pages 1–16. IEEE, 2020. 19

    2020

  30. [38]

    Cosyvoice 3: Towards in-the-wild speech generation via scaling-up and post-training

    Zhihao Du, Changfeng Gao, Yuxuan Wang, Fan Yu, Tianyu Zhao, Hao Wang, Xiang Lv, Hui Wang, Chongjia Ni, Xian Shi, et al. Cosyvoice 3: Towards in-the-wild speech generation via scaling-up and post-training. arXiv preprint arXiv:2505.17589, 2025

    2025 arXiv

  31. [39]

    Enhancing zero-shot text-to-speech synthesis with human feedback

    Chen Chen, Yuchen Hu, Wen Wu, Helin Wang, Eng Siong Chng, and Chao Zhang. Enhancing zero-shot text-to-speech synthesis with human feedback. arXiv preprint arXiv:2406.00654, 2024

    2024 arXiv

  32. [40]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  33. [41]

    Deepseek llm: Scaling open-source language models with longtermism

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954, 2024

    2024 arXiv

  34. [42]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741, 2023

    2023

  35. [43]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  36. [44]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492–28518. PMLR, 2023

    2023

  37. [45]

    Huggingface’s transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019

    1910 arXiv

  38. [46]

    Pytorch: An imperative style, high-performance deep learning library

    A Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703, 2019

    1912 arXiv

  39. [47]

    Pytorch lightning

    William A Falcon. Pytorch lightning. https://github.com/PyTorchLightning/ pytorch-lightning, 2019

    2019

  40. [48]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. pages 611–626, 2023. 20

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.