{"id":"09d88152-d544-4e04-900a-acf85d23e3bc","arxiv_id":"2412.08112","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Training a CTC-based aligner on Mel-spectrograms and using its durations to train StyleSpeech reduces word error rate from 0.361 to 0.199 compared with using external Montreal Forced Aligner durations.","lead":"This paper trains a speech-recognition-style aligner to produce phoneme durations, then uses those durations to train a StyleSpeech text-to-speech model instead of relying on an external forced aligner. On a Chinese single-speaker dataset, the approach lowers word error rate by about 16 percentage points and the authors identify Mel-spectrograms as the best acoustic feature for alignment.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PDA's unconstrained CTC argmax can yield durations inconsistent with the target phoneme string, undermining the central WER comparison.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the PDA algorithm in Section II-B derives durations from a frame-wise argmax without enforcing that the resulting phoneme sequence matches the target X. I agree that this is the most fragile step in the central argument. The paper's headline improvement is specifically attributed to more accurate duration labelling, so if the duration labels can correspond to phonemes not in X, the comparison in Table I does not support the claimed mechanism. My proposed test directly checks whether this failure occurs and whether it matters by comparing unconstrained argmax durations with constrained CTC forced-alignment durations. Other issues, such as the '16% improvement' being an absolute percentage-point change rather than a relative improvement and the lack of significance testing, are real but secondary; they do not by themselves invalidate the central claim, and they are addressable. The absence of code and the small-scale single-dataset evaluation also support a CONDITIONAL stance but do not make the central argument impossible. Since my analysis converges with the reader's weakest assumption and does not identify a different, more severe flaw, the appropriate verdict remains CONDITIONAL as originally issued, and I mark the verdict as UNCHANGED.","tokens_in":6263,"tokens_out":4390,"duration_ms":49594,"concrete_test":"Compute the collapsed argmax phoneme sequence for all 1000 test utterances and compare it with the target sequence X using standard CTC collapse (remove blanks, merge repeats). Report the phoneme error rate and the fraction of utterances where the collapsed sequence differs from X. If that fraction is non-negligible (e.g., >2–5%), rerun the full Table I comparison using durations obtained from constrained CTC forced alignment (Viterbi decoding restricted to valid alignments of X, e.g., torchaudio.functional.forced_align). If the WER improvement persists with constrained alignment, the unconstrained argmax is not the cause; if it disappears, the central claim depends on invalid duration labels. This single check would settle whether the PDA algorithm's missing X-consistency constraint is load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is in Section II-B (Phoneme Duration Alignment). The PDA takes a frame-wise argmax over the CTC likelihood matrix and collapses consecutive identical or blank frames into durations, but it never verifies that the collapsed phoneme sequence equals the target phoneme sequence X. CTC training only guarantees that some alignment in the full lattice has high probability; the single most probable path can contain insertions, deletions, or substitutions relative to X. For example, an argmax path 'h e l k o' for target 'h e l o' would either yield a duration for a nonexistent phoneme or misassign durations, yet these labels are used as ground truth for the TTS duration adapter in Eq. (2). The reported WER gain in Table I (0.361→0.199) is attributed to 'accurate duration labelling'; if the labels are not constrained to X, the comparison is not actually testing aligner-guided duration quality. The paper asserts the duration sequence must satisfy T = ΣL_i, but not that it must be consistent with X. The description of blank handling is also ambiguous: it says silence frames increase 'its' duration without specifying whether the blank is assigned to the preceding phoneme, the following phoneme, or a separate silence token, which changes the resulting labels.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an Aligner-Guided Training Paradigm for text-to-speech: an ASR model with CTC loss is trained on acoustic features (Mel-spectrograms, MFCCs, or RAVE latent features), and a phoneme duration alignment (PDA) algorithm converts the frame-wise argmax of the CTC likelihood matrix into per-phoneme durations. These durations are then used as ground-truth labels to train a StyleSpeech TTS model. On the Baker Chinese dataset, the authors report that this approach improves WER from 0.361 (MFA-based durations) to 0.199 with Mel-spectrogram features, which they describe as a 16% improvement, and they conclude that Mel-spectrograms give the best alignment quality among the three feature types.","tokens_in":6470,"tokens_out":4087,"duration_ms":41521,"significance":"If the central claim holds, the work offers a practical alternative to external forced aligners such as MFA for obtaining phoneme durations in TTS training, and it highlights the role of acoustic feature choice in alignment quality. The idea is simple and potentially useful, and the experimental comparison across three feature types is a reasonable first step. However, the paper's load-bearing methodological components, especially the PDA algorithm, are underspecified, and the reported improvement percentage is statistically mislabeled. These issues must be resolved before the results can be taken as evidence for the claimed advantage.","major_comments":[{"comment":"The PDA algorithm does not guarantee that the collapsed argmax path equals the target phoneme sequence X. CTC training only ensures that some alignment in the entire lattice has high probability; the single most probable frame-wise path can contain insertions, deletions, or substitutions relative to X. As described, the algorithm would assign durations to phonemes not present in X, or omit phonemes from X, without any verification or correction. This is load-bearing because the duration labels L are used as ground truth in Eq. (2); if they are not actually consistent with X, the WER comparison in Table I does not measure the benefit of aligner-guided duration. The authors should specify how the argmax path is mapped to X (e.g., by using CTC prefix decoding with the target constraint, or by post-hoc alignment that forces the sequence to match X), and they should report the proportion of utterances where the argmax path actually matches X.","section":"II-B, Phoneme Duration Alignment"},{"comment":"The description of blank (silence) handling is ambiguous. The text states that if a frame's argmax is the same as the previous phoneme or is the blank symbol epsilon, 'we increase its duration by one,' but it does not define whether the blank frame is assigned to the preceding phoneme, the following phoneme, or a separate silence token. This ambiguity affects the consistency condition T = Σ L_i: if blanks are counted separately, the sum of the N phoneme durations cannot equal the total frame count T. The authors must clarify the assignment rule for blanks and explain how the sum constraint is satisfied under that rule.","section":"II-B, blank handling"},{"comment":"The reported '16% improvement' (and similarly the 9% and 14% figures) are absolute percentage-point differences in WER, not relative improvements. For Mel-spectrograms, the relative improvement is (0.361 − 0.199)/0.361 ≈ 45%. The abstract and the contributions list state 'up to a 16% improvement,' which is misleading unless explicitly labeled as an absolute decrease in WER. This is a load-bearing statistical reporting issue that affects the paper's headline claim and should be corrected.","section":"Abstract and Section III (Results)"}],"minor_comments":[{"comment":"The metrics WER-P and WER-S are used in Table I, but the paper never defines them. The conclusion claims improvements in 'tone alignment' without any defined tone-level metric; please provide precise definitions of these metrics or revise the claims accordingly.","section":"III, Metrics"},{"comment":"The 'Origin' row in Table II is not discussed in the analysis paragraph. The authors should clarify that 'Origin' refers to the StyleSpeech baseline trained with MFA-supplied durations, and they should comment on how the aligner-derived durations differ from the MFA durations for this example.","section":"III-A, Table II"},{"comment":"The paper says 'we follow the approach of RVAE' and 'We train an autoencoder to encode speech into a latent feature,' but gives no details about the autoencoder architecture, its training data, or whether the latent representation is fixed or fine-tuned during TTS training. Adding these details would improve reproducibility.","section":"II-C, latent features"},{"comment":"The claim 'more than a 15% improvement in speech accuracy' in contribution 1 is not defined; 'speech accuracy' is never measured. It should be rephrased to refer to the WER metric actually used in the experiments.","section":"I, Introduction"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is short and has several underspecified components, but the central idea is defensible. The main risk is the PDA algorithm's correctness; if the authors can constrain the alignment to the target sequence and clarify blank handling, the empirical comparison would be meaningful. The percentage reporting should also be fixed. I do not see grounds for rejection, but the authors need to address the PDA specification and statistical reporting before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper shows that replacing Montreal Forced Aligner durations with durations extracted from a CTC-trained aligner improves WER on a single Mandarin TTS dataset (0.361 to 0.199). That's a large relative gain, and it's the kind of practical result that could influence how people build TTS pipelines. What's genuinely new is the specific combination: a CTC-based aligner, run-length collapse of the argmax path, and a three-way comparison of acoustic features (MelSpec, MFCC, latent) feeding StyleSpeech. The feature comparison is a well-designed ablation, and the paper avoids circularity by checking durations against an external MFA baseline and evaluating with Whisper. Credit where due: the pipeline is simple, plausibly useful, and presented with enough detail to reproduce once the code is out.\n\nThe soft spots are real but fixable. The Phoneme Duration Alignment algorithm (Section II-B) is the load-bearing part and it's underspecified. The column-wise argmax over the CTC likelihood matrix can produce a phoneme sequence that does not match the target sequence X—insertions, deletions, and substitutions are all possible—but the paper never verifies consistency. If the argmax path diverges from X, the \"durations\" aren't aligned to the text, and the central WER comparison is weakened. The handling of CTC blanks is also ambiguous: does a blank get added to the preceding phoneme, the following phoneme, or become a separate silence token? The paper says \"silence frames increase its duration\" without saying whose duration. That's a genuine gap, not a nitpick. Second, the alleged \"16% improvement\" is an absolute percentage-point drop (0.361 to 0.199), which is actually a ~45% relative reduction. The reporting should be corrected. Third, there's no significance testing or error analysis beyond the WER standard deviations, and everything runs on one 12-hour single-speaker dataset. None of these are fatal, but they currently push the headline claim beyond what the evidence supports.\n\nWho gets value? TTS practitioners who want a cheap, internal alternative to MFA, and researchers studying duration modeling. I'd send this to a serious referee: the idea is plausible and the preliminary result is striking, but the paper needs a clearer PDA definition and a more honest reporting of the improvement. If the authors tighten the alignment description and add a sanity check that collapsed argmax equals the target phoneme string, it could be a solid workshop-level or short-paper contribution. The stress-test concern is on point and should be raised in review.","headline":"A useful empirical trick—CTC-derived durations beat MFA on one TTS benchmark—but the alignment algorithm is under-specified and the headline improvement is misreported.","tokens_in":6985,"tokens_out":1666,"would_cite":false,"duration_ms":19209,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing external forced-aligner durations with a CTC-trained aligner's durations improves TTS intelligibility, cutting word error rate from 0.361 to 0.199 on a Mandarin benchmark.","keywords":["text-to-speech","duration alignment","connectionist temporal classification","forced alignment","StyleSpeech","Mel-spectrogram","word error rate","Mandarin TTS"],"falsifier":"On the Baker test set, compute the fraction of utterances for which the collapsed argmax phoneme sequence from the aligner equals the target phoneme string. If that fraction is far below 100 percent, the duration labels are not actually aligned to the text, and Table I's comparison is not isolating aligner-guided duration quality.","tokens_in":6066,"feed_emoji":"🗣️","tokens_out":9411,"duration_ms":82715,"temperature":0.7,"pith_summary":"The paper argues that duration labels used to train text-to-speech models should come from an aligner trained on the same data, rather than from an external forced aligner, and that this substitution improves intelligibility. It proposes a two-stage paradigm: first train an ASR model with CTC loss on acoustic features, collapse its per-frame predictions into phoneme durations, then train StyleSpeech with those durations. On the Baker Mandarin dataset, the aligner-guided model with Mel-spectrogram features reduces overall word error rate from 0.361 to 0.199 compared with the same model trained on externally aligned durations. The paper also reports that Mel-spectrograms are the best acoustic feature for this purpose, ahead of MFCCs and latent features.","feed_headline":"Aligner-guided durations cut TTS word error to 0.199","feed_subtitle":"An internal aligner trained on Mel spectrograms replaces external forced-alignment labels, cutting WER from 0.361.","key_machinery":"The mechanism is the combination of Connectionist Temporal Classification (CTC) loss, which trains a frame-level speech recognizer to align a long acoustic feature sequence to a shorter phoneme sequence without frame-level labels, and the Phoneme Duration Alignment algorithm. The aligner outputs a per-frame likelihood matrix over phonemes plus a blank symbol; the algorithm takes the most-likely phoneme per frame and collapses consecutive identical labels and blanks into a duration count for each phoneme. These counts become the ground-truth duration labels that the StyleSpeech duration adapter is trained to predict.","core_discovery":"The central claim is that a CTC-trained aligner can produce duration labels for TTS training that are better than labels from an external forced aligner, and that the choice of acoustic feature fed to the aligner matters. With Mel-spectrogram features, StyleSpeech's overall WER drops from 0.361 with external aligner durations to 0.199 with aligner-guided durations; phoneme-level WER drops from 0.271 to 0.110 and style-level WER from 0.226 to 0.131. The paper attributes the gain to more accurate duration labels, and to Mel-spectrograms preserving clear phoneme boundaries that MFCCs and latent features blur.","pith_inferences":["A direct test of the PDA algorithm would be to compare its collapsed argmax sequence with the target text per utterance; until that comparison is reported, the WER gain should be read as evidence for the whole pipeline rather than for the duration labels specifically.","Because the PDA algorithm folds blank frames into the neighboring phoneme, it may conflate silence with phoneme duration; a variant that reserves blanks as explicit pauses could change prosody and intelligibility.","The same two-stage paradigm should transfer to other duration-based TTS backbones and to non-tonal languages, since nothing in the method is Mandarin- or StyleSpeech-specific.","A forced-alignment variant that respects the target phoneme sequence would settle whether the argmax collapse is better than simply obeying the text during alignment."],"forward_implications":["TTS pipelines can replace an external forced aligner with an aligner trained on the same data, removing a time-consuming preprocessing dependency.","Using aligner-guided durations from Mel-spectrogram features, StyleSpeech reaches a test WER of 0.199 versus 0.361 with external aligner durations, an absolute drop of about 16 points.","Phoneme-level and style-level WER also improve, reaching 0.110 and 0.131 respectively, suggesting better phoneme and tone alignment.","Mel-spectrogram features are the best aligner input among those tested, followed by MFCCs and latent features."],"supporting_citations":[{"why":"Supplies CTC loss, the training objective that learns frame-to-phoneme alignments without frame-level labels.","marker":"[7]"},{"why":"StyleSpeech is the TTS backbone whose duration adapter is retrained on aligner-guided durations.","marker":"[3]"},{"why":"Provides the external forced-aligner duration labels that form the Origin baseline in Table I.","marker":"[4]"},{"why":"Baker Mandarin corpus is the dataset used for training and evaluating the TTS and aligner models.","marker":"[9]"},{"why":"Whisper ASR transcribes the generated speech so word error rate can be measured.","marker":"[13]"},{"why":"RAVE autoencoder supplies the latent speech feature used as one of the aligner input variants.","marker":"[8]"}],"fun_headline_variants":["Aligner-guided durations cut TTS word error from 0.361 to 0.199","Internal aligner beats external forced aligner for TTS word error","Mel-spectrogram aligner delivers 45% lower TTS word error","Aligner-first training cuts TTS word error by 45%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the per-frame most-likely phoneme sequence, once collapsed, gives correct duration labels for the target text, but the paper does not check that the collapsed sequence actually matches the target phoneme string.","fun_headline_variants_meta":{"raw":{"variants":["Aligner-guided durations cut TTS word error from 0.361 to 0.199","Internal aligner beats external forced aligner for TTS word error","Mel-spectrogram aligner delivers 45% lower TTS word error","Aligner-first training cuts TTS word error by 45%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000682,"raw_usage":{"total_tokens":3060,"prompt_tokens":874,"completion_tokens":2186,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":2104}},"tokens_in":490,"tokens_out":2186,"duration_ms":15576,"temperature":1.0,"reasoning_tokens":2104,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:11:54.143710+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the Baker test set, compute the fraction of utterances for which the collapsed argmax phoneme sequence from the aligner equals the target phoneme string. If that fraction is far below 100 percent, the duration labels are not actually aligned to the text, and Table I's comparison is not isolating aligner-guided duration quality.","supporting_citations":[{"cited_title":"Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,","cited_arxiv_id":null,"evidence_quote":"Supplies CTC loss, the training objective that learns frame-to-phoneme alignments without frame-level labels."},{"cited_title":"Montreal forced aligner: Trainable text- speech alignment using kaldi.,","cited_arxiv_id":null,"evidence_quote":"Provides the external forced-aligner duration labels that form the Origin baseline in Table I."}],"review_version":1}