{"id":"91b6da5d-1dc9-48bb-a326-c21a03a4ab72","arxiv_id":"2505.18609","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new 13,000-hour dataset with 24 million LLM-generated text descriptions enables the first open-source text-driven TTS for 24 Indian languages, with reported high speaker, emotion, and cross-lingual control.","lead":"Researchers assembled RASMALAI, a 13,000-hour multilingual speech dataset with 24 million text descriptions spanning 24 languages, and trained IndicParlerTTS, a text-prompted voice synthesizer, on it. If the evaluations hold, this could make expressive, controllable text-to-speech broadly accessible for Indian language speakers and enable new voice applications at scale.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Instruction-following scores rest on the same acoustic-attribute pipeline that generated the training descriptions, so IF-BLEU 93.18 may reflect label consistency rather than human-perceived prompt adherence; a human attribute-perception test is needed.","rationale":"The reader's weakest assumption identifies the same load-bearing point: the instruction-adherence metrics are generated by the same pipeline that created the descriptions, so they are not grounded in human perception. My read does not change the conditional verdict. The paper has real independent support: human MUSHRA naturalness on seen speakers (81.7 vs 89.7), human emotion classification (most Ekman emotions above 70%), and intelligibility metrics (CER 12%, WER 24%). Those results justify conditional acceptance. However, the paper's headline claim of 'reliably follow text descriptions' depends on an evaluation that is at least partly circular and, as written, uses an ill-defined IF-BLEU procedure. This is an addressable issue: release the code and data, and add a human perceptual forced-choice test of attribute following. If that test fails, the central controllable-TTS claim would need to be weakened; if it passes, the concern is resolved. Therefore the appropriate verdict remains CONDITIONAL, i.e., unchanged from the reader's judgment.","tokens_in":8807,"tokens_out":7956,"duration_ms":60780,"concrete_test":"For a stratified sample of utterances across at least 4 languages, synthesize two versions of the same text from paired descriptions that differ in exactly one attribute (e.g., 'fast' vs 'slow', 'high pitch' vs 'low pitch', 'reverberant' vs 'dry'). Have native listeners, blind to condition, choose which audio matches each description. If human selection accuracy is not significantly above chance for attributes with automatic accuracy above 95% in Table 6, the instruction-adherence claim is overstated. As a secondary check, recompute IF-BLEU using an independent natural-language style captioner on the generated audio and compare with 93.18.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that IndicParlerTTS 'reliably follows text descriptions' is supported mainly by Table 5 (IF-BLEU 93.18) and Table 6 (attribute accuracy 80-99%). The load-bearing assumption is that the evaluation pipeline is a valid proxy for human perception of the described attributes. Section 2.2 constructs training descriptions by extracting acoustic attributes (pitch, C50, SNR, speaking rate, PESQ), binning them, and asking an LLM to verbalize them. Section 3 then evaluates instruction adherence by 're-annotating acoustic features of the synthesised samples using our pipeline up to the binning stage.' Thus the same deterministic extractor/binner defines both the training labels and the scoring function. A high IF-BLEU/accuracy primarily shows that the model's output audio re-maps to the same bins used to create the prompt; it does not show that a listener would perceive 'slightly fast pace,' 'slightly roomy environment,' or 'great speech quality.' This is an internal-consistency result, not an external validity result. There is also an interpretability problem with the IF-BLEU computation itself: it is described as comparing a comma-separated sequence of binned values to 'the original instruction.' If that original instruction is the LLM-generated natural-language sentence, BLEU against comma-separated attribute tokens would be near zero; if the comparison is instead to the original binned attributes, the number is an attribute-reconstruction score, not IF-BLEU as defined in LibriTTS-P. Either way, the reported 93.18 cannot be interpreted without the released code. The human-rated MUSHRA naturalness and emotion-classification results are independent and partially support expressiveness, but they cover only coarse dimensions and do not validate the fine-grained attribute-control claims that distinguish this system.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RASMALAI, a large-scale speech dataset of roughly 13,000 hours across 24 languages, with 24 million automatically generated text descriptions built from acoustic attributes (pitch, C50, SNR, speaking rate, PESQ) and metadata (age, gender, emotion, style, accent). Using this dataset, the authors train IndicParlerTTS, a multilingual text-prompted TTS system. They evaluate it with MUSHRA naturalness tests on seen speakers in the Rasa-13 benchmark, automatic metrics (CER, WER, MOS, S-SIM, IF-BLEU), attribute-level accuracy, a human emotion classification test, and zero-shot expressive synthesis across native, proximal, and distal speaker groups. The paper's central claims are that IndicParlerTTS achieves near-human naturalness on seen speakers, reliably follows text descriptions, accurately synthesizes specified attributes, and transfers expressive characteristics within and across languages.","tokens_in":9172,"tokens_out":6399,"duration_ms":53559,"significance":"If validated, this would be a substantial contribution: RASMALAI is much larger and more language-diverse than existing text-prompted TTS datasets for Indic languages, and the public release of the dataset and model would fill a clear gap. The human MUSHRA results (Table 3) and the human emotion classification experiment (Figure 2) are genuine strengths and provide credible evidence for naturalness and emotional expressiveness, independently of the pipeline-based evaluations. The attribute coverage comparison in Table 2 is useful. However, the instruction-following claim, which is central to the paper's 'controllable TTS' contribution, currently rests on an evaluation that reuses the same acoustic-attribute extraction and binning pipeline that generated the training descriptions. This makes the headline IF-BLEU of 93.18 an internal-consistency result rather than evidence of human-perceived prompt adherence. The contribution would be strong once this is addressed with a human attribute-perception test or a careful reframing of the claim.","major_comments":[{"comment":"The instruction-adherence evaluation is circular. Section 2.2 constructs training descriptions by extracting acoustic attributes, binning them, and asking an LLM to verbalize them, while Section 3 evaluates adherence by 're-annotating acoustic features of the synthesised samples using our pipeline up to the binning stage' and comparing the binned values against the original attributes. Consequently, IF-BLEU (93.18) and attribute accuracy (80-99%) primarily measure the model's ability to invert the same deterministic feature extractor and binner used to create the training labels, not whether a listener perceives 'slightly fast pace' or 'great speech quality' as described. To support the Section 4.2 claim that the model 'reliably follows text descriptions,' a human attribute-perception test is required, in which listeners match or rate synthesized samples against the natural-language prompts; without this, the headline instruction-following claim should be reframed as acoustic-attribute reconstruction accuracy.","section":"Section 3, Tables 5 and 6"},{"comment":"The definition of IF-BLEU is ambiguous as written. The text says the metric is computed by 'constructing prompts from binned values as comma-separated sequences and compar[ing] them to the original instruction from which this sample was synthesised.' If 'the original instruction' is the LLM-generated natural-language sentence from Section 2.2, BLEU between comma-separated attribute tokens and natural-language tokens would be near zero under standard tokenization; if the comparison is instead to the original binned attributes, the metric is an attribute-reconstruction score, not the instruction-following BLEU introduced in LibriTTS-P. Please specify exactly what the reference string is, how tokenization and punctuation are handled, and whether the score is averaged over the three prompt types or only over one of them. This is necessary for interpreting the reported value of 93.18.","section":"Section 3, IF-BLEU definition"},{"comment":"The evaluation scope is narrower than the claims. The main naturalness and instruction-following results (Tables 3-6) are reported only for the Rasa-13 test set, despite the system covering 24 languages. Table 7 reports zero-shot expressive MUSHRA scores for Native, Proximal, and Distal groups, but does not state the number of speakers, languages, or utterances per group, nor does it include a baseline comparator. In addition, the only comparative baseline, IndicVC, is a zero-shot speech-prompted TTS, not a text-description-guided system, so the comparison in Tables 4 cannot benchmark instruction-following capability. Please either extend the evaluation to additional languages, add an appropriate text-prompted baseline, or temper the 'multilingual' and 'new standard' claims to match the evidence.","section":"Section 4.1 and Section 4.4"},{"comment":"The automatic metrics in Table 5 (CER, WER, MOS, S-SIM, IF-BLEU) are reported as point estimates without confidence intervals or significance tests, and the size of the evaluation subset is not specified. Given that Section 4.1 uses language such as 'significantly surpasses' for the MUSHRA comparisons, the same standard should be applied to the automatic metrics. Please provide confidence intervals or significance tests for these values, and state how many utterances were used in the TTS subset of the test set.","section":"Section 3, Table 5"}],"minor_comments":[{"comment":"There is a typo in the evaluation paragraph: 'by by re-annotating' should read 'by re-annotating.'","section":"Section 3"},{"comment":"The text contains the erroneous token 'LERTTS' where 'IndicParlerTTS' is evidently intended.","section":"Section 4.1"},{"comment":"The example in Table 1 is internally inconsistent: the attribute list says 'Slightly close sounding,' but the descriptive caption says 'slightly roomy environment.' These are opposite descriptions; please correct the example so the caption matches the attribute list.","section":"Table 1"},{"comment":"PESQ is a full-reference metric, but the paper does not state how it is computed for utterances that lack a reference signal. Please clarify the estimation procedure used in the attribute extraction pipeline.","section":"Section 2.2"},{"comment":"The abstract says the authors 'will release' the dataset and code, while the conclusion says 'We release RASMALAI, along with our models and code.' Please make the availability status consistent and state the intended release license and platform.","section":"Section 6"},{"comment":"The caption for Table 6 refers to 'the TTS subset of the test set,' but this subset is not defined anywhere. Please define it and report the number of utterances used for the attribute-accuracy evaluation.","section":"Table 6"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of a speech/audio journal and the dataset resource is potentially valuable. The central risk is the circularity of the instruction-following evaluation, which is fixable with a human attribute-perception experiment or by reframing the claims. I did not see citation or novelty concerns beyond what is stated in the report: the proposed evaluation extensions (more languages, a text-prompted baseline, error bars) are feasible within a major revision and do not require new data collection on the scale of the original dataset."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline: this paper's real contribution is the dataset, not the model. RASMALAI bundles 13,000 hours of Indic speech with LLM-generated text descriptions in 24 languages, which is a genuinely useful resource. The model, IndicParlerTTS, is an adapted Parler-TTS with an expanded tokenizer; the novelty is in the data, not the architecture.\n\nWhat the paper does well: the human evaluations are credible. MUSHRA on Rasa-13 shows 81.7 versus 89.7 for human speech, with a clear win over IndicVoiceCraft. The emotion confusion matrix is honest—sadness is confused with fear, neutral is ambiguous—and the authors say so. The zero-shot expressive transfer results, while based on limited subjective tests, are plausibly the first evidence that a text-prompted TTS can do cross-lingual style transfer for Indic languages. The dataset also covers attributes (accent, environment, native-language descriptions) that earlier text-prompted datasets lack, and the comparison table with LibriTTS-P etc. is useful.\n\nThe soft spots are mostly around the instruction-following evaluation. The paper reports IF-BLEU of 93.18 and attribute accuracies of 80–99% by re-running the same acoustic feature extraction and binning pipeline that produced the training descriptions. That is a circular measurement: it shows the model reproduces the annotation process, not that a listener would perceive 'slightly fast pace' or 'great speech quality.' The stress-test note is right, and the ambiguity in the IF-BLEU definition makes it worse. Comparing comma-separated binned values against the LLM-generated natural-language instruction should give near-zero BLEU; comparing against the original binned attribute sequence is attribute reconstruction, not instruction following. The paper doesn't disambiguate, so the 93.18 is uninterpretable without code. Also, automatic metrics have no error bars, and the baseline comparison is only IndicVC, which is a reasonable choice but a narrow one.\n\nThese are fixable. A human attribute-perception test, or at least a clear description of the IF-BLEU computation plus a non-circular baseline, would address the main concern. The naturalness and emotion claims are independent of this and stand on human ratings.\n\nWho should read this: anyone building TTS datasets or systems for Indian languages, and anyone working on evaluating text-prompted TTS. The dataset alone justifies the paper if the release actually happens.\n\nRecommendation: send it to peer review. It deserves referee time; the circularity should be raised as a major comment, and artifact release should be required. But the resource is real and the human eval supports the central qualitative claims.","headline":"Useful multilingual TTS dataset with honest human eval, but the headline instruction-following number is circular and uninterpretable as reported.","tokens_in":9750,"tokens_out":3041,"would_cite":true,"duration_ms":24119,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RASMALAI pairs 13,000 hours of Indian-language speech with 24 million text descriptions, and the resulting TTS approaches human naturalness on seen voices.","keywords":["text-prompted text-to-speech","expressive speech synthesis","Indian languages","multilingual TTS","speech description dataset","instruction following","emotion synthesis","zero-shot style transfer"],"falsifier":"Run a blind listening test in which human raters match synthesized clips to text descriptions, and compare those judgments with IF-BLEU and the attribute-accuracy scores; if the automatic scores do not track listener judgments for attributes like 'slightly fast pace' or 'great speech quality,' the instruction-following claim is not supported.","tokens_in":1818,"feed_emoji":"🗣️","tokens_out":1834,"duration_ms":54560,"temperature":0.7,"pith_summary":"This paper claims that controllable, expressive text-to-speech for Indian languages is now possible at scale, and it backs the claim with two artifacts: RASMALAI, a 13,000-hour, 24-million-description corpus across 23 Indian languages plus English, and IndicParlerTTS, the first open-source text-description-guided TTS for these languages. The model is reported to produce near-human naturalness on seen speakers, with a MUSHRA score of 81.7 versus 89.7 for human speech, and to follow text instructions with high fidelity, reaching an IF-BLEU score of 93.18. It also transfers expressive style to speakers and languages it never saw with expressive training data. If true, this lets a user control voice, emotion, accent, and environment by typing a description instead of supplying a reference audio clip.","feed_headline":"First text-prompted TTS for 24 Indian languages","feed_subtitle":"A 13,000-hour corpus with 24 million descriptions lets users type a voice, emotion, and accent.","key_machinery":"The central mechanism is an attribute-extraction and LLM-captioning pipeline: acoustic features such as mean and standard deviation of pitch, C50, SNR, speaking rate, and PESQ are combined with available metadata like age, gender, and speaker identity, plus Ekman emotion labels, then binned into structured tags. A LLaMA-3.1-8B-Instruct model turns each tag set into three caption variants, Descriptive, Concise, and Attribute-Robust, and IndicTrans2 translates the Descriptive captions into the target native languages. These captions convert existing {audio, transcript} pairs into {audio, transcript, description} training triplets, which is what lets a TTS model condition generation on text rather than on a reference audio sample. The model itself is Parler-TTS mini v1 adapted with an expanded Llama2 tokenizer for better subword segmentation in Indian languages.","core_discovery":"The central discovery this paper argues for is that an automated pipeline can convert existing speech corpora into a richly described TTS training set large enough to make text-prompted synthesis work for Indian languages. RASMALAI aggregates read, extempore, conversational, studio-recorded, and accented English speech; extracts metadata, emotion labels, and acoustic attributes; discretizes them into bins; and asks an LLM to write three kinds of captions per utterance, with native-language translations for the descriptive captions. Trained on this corpus, IndicParlerTTS is reported to be the first open-source text-description-guided TTS covering 24 languages, with near-human naturalness on seen speakers and strong instruction adherence. The paper also reports robust zero-shot expressive transfer, both to unseen speakers within a language and across related and distant languages, which is what makes the contribution a step toward controllable multilingual synthesis rather than a single-voice demo.","pith_inferences":["Because the captions are generated from automatically extracted acoustic tags rather than human listening, the dataset encodes machine-measurable attributes; a human annotation study could reveal which descriptions listeners actually perceive and would likely strengthen the instruction-following evaluation.","The cross-lingual expressive transfer results suggest the model builds language-independent representations of emotion and style, which could be exploited for tasks beyond TTS, such as style-consistent dubbing, voice editing, or expressive data augmentation for speech recognition.","The relatively lower PESQ accuracy hints that studio-only finetuning limits the model's ability to reproduce deliberately degraded or noisy recordings described in the dataset; adding matched noisy finetuning data could test and close that gap."],"forward_implications":["A user can generate speech in any of 24 languages by typing a description of the speaker's voice, accent, emotion, speaking rate, and recording environment, with no reference audio clip required.","On the Rasa-13 subjective test, the model scores 81.7 versus 89.7 for human speech, and in Bodo, Maithili, Marathi, and Telugu the gap is within a few points of human naturalness.","Automatic evaluation reports an IF-BLEU score of 93.18 for instruction adherence, with attribute-level accuracy between 80.44% and 99.26% across C50, F0 mean, F0 standard deviation, SNR, speaking rate, and PESQ.","The model renders emotions for speakers with no expressive training data, including cross-lingually, with MUSHRA scores of 86.73 for native speakers, 80.86 for speakers of related languages, and 76.01 for speakers of distant languages.","The released corpus of 13,000 hours and 24 million annotations provides a public training resource for controllable and expressive Indian-language TTS that previously did not exist."],"supporting_citations":[{"why":"Supplies the attribute-extraction and synthetic-annotation pipeline that RASMALAI extends to more languages and attributes.","marker":"[8]"},{"why":"Provides the Rasa expressive studio-quality dataset and the 13-language test set used in subjective naturalness evaluations.","marker":"[11]"},{"why":"Provides the IndicVoices-R restored speech corpus and the IndicVoiceCraft baseline that IndicParlerTTS is compared against.","marker":"[13]"},{"why":"Supplies the bulk of raw {audio, transcript} pairs across 22 Indian languages in the pretraining set.","marker":"[18]"},{"why":"Defines the IF-BLEU instruction-adherence metric and provides the LibriTTS-P dataset used for scale comparison.","marker":"[7]"},{"why":"Supplies the GLOBE accented English corpus that contributes diverse accent coverage to RASMALAI.","marker":"[19]"},{"why":"Supplies LIMMITS studio-recorded multilingual read-speech data used as a high-quality source.","marker":"[12]"},{"why":"Supplies IndicTTS studio-recorded read-speech data used as a high-quality source.","marker":"[10]"},{"why":"Provides IndicTrans2, used to translate descriptive captions into native Indian languages.","marker":"[17]"}],"fun_headline_variants":["Text-prompted TTS expands to 24 Indian languages","13K hours of Indian speech with text descriptions for TTS","IndicParlerTTS: open-source text-guided synthesis for India","Expressive Indian TTS: 13,000 hours, 24M descriptions","First open TTS for Indian languages driven by text prompts"],"cache_read_input_tokens":11776,"weakest_assumption_plain":"The claim that the model faithfully follows instructions rests on the assumption that re-annotating synthesized audio with the same acoustic feature pipeline used to build the captions is a valid measure of whether the descriptions were obeyed; if that pipeline does not track what listeners actually perceive, the high instruction-following scores could overstate true adherence.","fun_headline_variants_meta":{"raw":{"variants":["Text-prompted TTS expands to 24 Indian languages","13K hours of Indian speech with text descriptions for TTS","IndicParlerTTS: open-source text-guided synthesis for India","Expressive Indian TTS: 13,000 hours, 24M descriptions","First open TTS for Indian languages driven by text prompts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000391,"raw_usage":{"total_tokens":2025,"prompt_tokens":879,"completion_tokens":1146,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":1069}},"tokens_in":495,"tokens_out":1146,"duration_ms":9727,"temperature":1.0,"reasoning_tokens":1069,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:28:07.751821+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a blind listening test in which human raters match synthesized clips to text descriptions, and compare those judgments with IF-BLEU and the attribute-accuracy scores; if the automatic scores do not track listener judgments for attributes like 'slightly fast pace' or 'great speech quality,' the instruction-following claim is not supported.","supporting_citations":[{"cited_title":"StyleTTS 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models,","cited_arxiv_id":null,"evidence_quote":"Supplies the attribute-extraction and synthetic-annotation pipeline that RASMALAI extends to more languages and attributes."},{"cited_title":"PromptTTS 2: Describing and generating voices with text prompt,","cited_arxiv_id":null,"evidence_quote":"Provides the Rasa expressive studio-quality dataset and the 13-language test set used in subjective naturalness evaluations."},{"cited_title":"Libritts-p: A corpus with speaking style and speaker identity prompts for text-to-speech and style captioning,","cited_arxiv_id":null,"evidence_quote":"Provides the IndicVoices-R restored speech corpus and the IndicVoiceCraft baseline that IndicParlerTTS is compared against."},{"cited_title":"Indicvoices-r: Unlocking a massive multilingual multi-speaker speech corpus for scaling indian TTS,","cited_arxiv_id":null,"evidence_quote":"Supplies the GLOBE accented English corpus that contributes diverse accent coverage to RASMALAI."},{"cited_title":"Rasa: Building expressive speech synthesis systems for indian languages in low-resource settings,","cited_arxiv_id":null,"evidence_quote":"Provides IndicTrans2, used to translate descriptive captions into native Indian languages."}],"review_version":1}