Intelligibility of Text-to-Speech Systems for Mathematical Expressions

H. G. Ranjani; Nishtha Paul; Siddhanth Iyengar; Subhadip Bandyopadhyay; Sujoy Roychowdhury; Sumit Soman

REVIEW 3 major objections 6 minor 40 references

Intelligibility of Text-to-Speech Systems for Mathematical Expressions

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Listening tests of five state-of-the-art text-to-speech models show that spoken mathematical expressions are often transcribed incorrectly, with accuracy varying by model and expression category and trailing human expert recordings in…

desk verdict First real listening-test data on TTS reading math, with a solid cross-model comparison in L1; the expert-rendition gap claim is weaker than the abstract suggests because it compares different text, not just different speech. read the letter →

arxiv 2506.11086 v1 pith:HX5YGELM submitted 2025-06-05 eess.AS cs.AI

Sujoy Roychowdhury , H. G. Ranjani , Sumit Soman , Nishtha Paul , Subhadip Bandyopadhyay , Siddhanth Iyengar This is my paper

classification eess.AScs.AI

keywords text-to-speechmathematicalexpressionsintelligibilitylisteningtestLaTeXlargelanguagemodelsmeanopinionscoreprosody

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether state-of-the-art text-to-speech (TTS) systems can read mathematical expressions aloud so that listeners can write them down correctly. The authors take LaTeX expressions, use two large language models to turn them into English pronunciations, feed those into five TTS models, and have listeners transcribe the audio. Across 120 expressions in eight categories, transcription is often far from perfect, and for most categories it is significantly worse than audio recorded by human experts. The results imply that current TTS models lack the prosody and phrasing that mathematics needs, and that evaluation of TTS for math must use human listeners, not automatic speech recognition.

What carries the argument

The evaluation cascade: LaTeX math is converted to a spoken-English string (MXText) by an LLM, then synthesized by one of five TTS models into audio (AudioMX), which listeners transcribe back to LaTeX. The intelligibility measurement rests on three complementary metrics: expert-judged count-of-correct, a normalized Levenshtein-based LaTeX character error rate, and TeXBLEU, a structural-grammar similarity score. A MUSHRA-style hidden-reference test quantifies the gap against expert recordings.

What would settle it

Give the same five TTS models hand-crafted expert English pronunciations (the kind the human experts used) for the same 120 expressions, rerun the transcription test, and compare counts-of-correct. If transcription accuracy jumps to near the expert-audio level, the intelligibility gap is caused by the LLM-generated text, not by TTS synthesis; if it stays low, the synthesis itself is the bottleneck.

Watch

Extended reading notes

Core claim

The paper's central claim is that TTS output for mathematical expressions is not necessarily intelligible, and that the intelligibility gap varies both with the TTS model and with the type of expression. Using three transcription metrics—count-of-correct, LaTeX character error rate, and TeXBLEU—the authors find that no model is consistently best; matrices and summations are hardest, while numerics are nearly perfect. Listeners' opinion scores overstate their actual comprehension, and the choice of LLM that produces the pronunciation has little effect once the pronunciation is correct. In a hidden-reference comparison, expert human recordings score higher than every TTS model for every expression, with large gaps for calculus, roots, and summation.

Load-bearing premise

The paper assumes that the LLM-generated English pronunciation fairly represents the mathematical content, so that transcription failures can be blamed on the TTS model rather than on the wording fed to it.

Editorial extensions

If this is right

No single public TTS model can be trusted for math audio; systems must be chosen or tuned by expression category.
User perception of understanding is not a reliable substitute for transcription: MOS scores ran ahead of actual correctness.
Automatic speech recognition metrics on a TTS-ASR pipeline do not predict human intelligibility, so ASR-based evaluation is insufficient.
Accessible math content, such as audio textbooks for vision-impaired readers, needs purpose-built prosody rather than stock voices.
Future work should fine-tune or train TTS models with math-specific prosody, with the category-level baselines here as a benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because expert reference audio was recorded without the LLM in the loop, the reported expert-versus-TTS gap is a property of the LLM+TTS cascade; a TTS model fed an expert's spoken text might narrow the gap substantially.
The roughly 12.5% judged-incorrect LLM pronunciations mean some intelligibility failures are upstream of the TTS; isolating TTS prosody from text errors would require controlling the text input.
The category-level difficulty ranking (numerics easiest, matrices and summations hardest) could serve as a stress-test battery for future math-aware TTS models.
The low correlation between ASR metrics and listener transcription suggests a blind spot: systems optimized for ASR word error rates may not be optimizing for human comprehension of structured notation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is the first paper I know of that actually puts TTS audio of math expressions in front of listeners and asks them to transcribe, and that alone makes it worth knowing. The L1 experiment—five TTS models, same LLM-generated verbalization for each expression, 49 listeners, three metrics—is a genuinely useful category-by-category map. The result that intelligibility varies by category and model, with numerics near ceiling and matrices/summation near floor, is credible and probably reproducible. The qualitative error analysis (pause placement changing 'a2-b2 over a' into 'a2 - b2/a', 'sin' mispronounced as [sIn]) is the kind of detail that will help people building math-aware TTS. The Whisper ASR control showing high TTS-to-ASR fidelity but low correlation with human transcription is a nice sanity check; it shows the problem is in the human-perception/prosody domain, not just garbled audio.

The soft spot is exactly where the stress-test note lands, and I think it lands. The L2 'gap with expert rendition' compares TTS audio synthesized from LLM-generated MXText with expert recordings made from the expert's own wording. Section 5.2 admits experts used 'whole divided by' and deliberate pauses. So Table 3 quantifies (LLM verbalizer + TTS) versus (expert verbalizer + human speech). The conclusion that 'general purpose TTS models do not include the complexities of prosody particular to MX' overshoots. You would need expert audio spoken from the same MXText, or TTS audio from expert verbalizations, to isolate prosody. The L1 cross-model comparisons are not affected, because all five TTS models read identical text, so Table 2 differences are attributable to TTS. Good.

Smaller issues: no confidence intervals or inter-annotator agreement on CoC; the 'significantly worse' in the abstract has no significance test attached in the paper; no data/code release; and the pronunciation-error rate (12.5%) is acknowledged and post-hoc filtered, but the absolute intelligibility numbers still inherit whatever LLM quirks remain.

Bottom line: the core observation holds—current TTS, even with a good LLM front-end, is often not intelligible for math, and category matters. The paper is not circular and does not overclaim its novelty. If I were refereeing it, I'd ask for a matched-text expert condition (or a clear qualification of what the L2 gap means), statistical detail, and release of audio/text transcriptions; with those, it would be a solid contribution. Worth a serious referee, and I'd cite it for the L1 results.

Referee Report

3 major / 6 minor

Summary. The paper evaluates the intelligibility and perceived quality of five text-to-speech (TTS) systems for mathematical expressions (MX). Because TTS models cannot directly process LaTeX, the authors use LLMs (QWEN2.5-7b and GPT4) to generate English verbalizations (MXText). Two listening tests are conducted: L1 measures transcription correctness (CoC, LCER, TeXBLEU) and MOS for 120 MX across 8 categories; L2 compares TTS audio against human expert reference recordings using a MUSHRA-style rating. The paper reports that intelligibility varies significantly by TTS model and MX category, and that for most categories there is a large gap between TTS and expert renditions, concluding that current TTS models lack MX-appropriate prosody.

Significance. The study addresses a real gap: there is little prior work on perceptual evaluation of TTS for mathematical content. The L1 results, based on 1800 listener ratings and three transcription metrics, provide useful evidence that intelligibility is model- and category-dependent. The paper also introduces LCER/TeXBLEU for MX transcription evaluation and shows that ASR-based cascade metrics are poorly correlated with human transcription. However, the RQ2 conclusion about the expert-rendition gap is not yet supported because the TTS and reference audio are not produced from the same text; a same-text control is needed. The post-hoc pronunciation correction check in Section 4 addresses the LLM front-end only for L1, not for L2.

major comments (3)

[Section 3.3 / Table 3] The L2 comparison conflates verbalization text with TTS performance. The TTS AudioMX is synthesized from LLM-generated MXText (Section 2.2.2), while the hidden reference RAudioMX is recorded by experts using their own wording (explicitly stated in Section 5.2). Consequently, the mean differences in Table 3 reflect differences in both the spoken text and the speech rendering, and the conclusion in Section 4 that 'public API or open source general purpose TTS models do not include the complexities of prosody particular to MX' is not warranted by the current design. A same-text control—either experts reading the MXText or TTS models synthesizing the expert verbalizations—is required to attribute the gap to TTS. Without it, the RQ2 gap should be described as a property of the LLM+TTS cascade.
[Abstract and Section 4] The abstract states that 'for most categories, performance of TTS models is significantly worse than that of expert rendition,' but no statistical test is reported for the L2 differences in Table 3. The ANOVA mentioned in Section 4 is on L1 LCER scores and does not support the L2 claim. Please add appropriate significance tests (e.g., paired tests across listeners or MX) on the per-MX reference-to-TTS differences.
[Section 4 (post-hoc check)] The manual filtering to correctly pronounced MXText does not address the L2 confound: it only shows that, in L1, the LLM factor becomes insignificant when incorrect pronunciations are removed. The conclusion that 'when the pronunciation is correct, LCER and MOS are statistically equivalent across both LLMs' is not a substitute for a same-text comparison with expert audio, and it does not support the abstract's claim about the gap with expert rendition.

minor comments (6)

[Section 3.3] The text says 'We sample 35 MX from 5 categories (excluding matrices)' but Table 3 reports 7 categories; please clarify the number of categories actually used in L2.
[Section 3.2] Please report inter-annotator agreement for the manual CoC evaluation; the current description says four evaluators mark each transcript but does not state how disagreements were resolved.
[Section 2.2.2] Please specify how the 120 MX were assigned to QWEN vs. GPT4 (e.g., random, category-balanced); the ANOVA on the 'LLM' factor assumes the factor is crossed with category.
[Section 4] Typo 'ANoV A' should be 'ANOVA'; please also report degrees of freedom and effect sizes for the significant factors.
[Section 2.1] The sentence 'we consolidate MX from different folders of HME to include categories such as matrices, de-duplicate and obtain a dataset of 3141 MX' is awkward; consider splitting into two sentences.
[Section 3.1] Minor language issues: 'a few of who did at most 3 batches' should be 'a few of whom'; 'have age range' should be 'are in the age range'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study's metrics and reference conditions are defined externally to the experimental pipeline, and no load-bearing result reduces to its inputs by construction.

full rationale

The paper is an empirical listening-study evaluation of five TTS systems for mathematical expressions. Its central quantities are listener transcriptions, MOS ratings, and comparative preference scores. These are all measured against external reference points: the original LaTeX ground truth from the HME dataset and human-expert reference audio. The three intelligibility metrics (CoC, LCER, TeXBLEU) are defined in Section 3.2 independently of the results and are not fitted to any subset of the data. The LLM-generated MXText is an input to the TTS systems, but the paper does not claim to predict intelligibility from MXText; rather, it tests whether the end-to-end output is intelligible. The ANOVA analyses compare factors (LLM, TTS model, category) against observed listener scores, and the post-hoc check on correctly pronounced MX additionally separates the LLM effect from the TTS effect. The comparison with expert rendition in L2 uses a hidden reference, and the finding that TTS scores fall short is a direct measurement, not a consequence of how the reference was constructed. The concern that the TTS and reference audio differ in verbalization text is a validity limitation, not circularity: it does not make the reported gap analytically equal to an input. There are no load-bearing self-citations; all cited prior work (MathBridge, TeXBLEU, MathSpeech, etc.) is by other authors and is used for dataset or metric context. The 'pronunciation errors' analysis in Section 4 is a quality check, not a fitted component of the main result. No step in the derivation chain equates a predicted quantity with a fitted parameter or renames a known result. The paper's conclusions may be debated on experimental-design grounds, but they are not circular.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The study is empirical and does not fit model parameters. It relies on statistical assumptions for ANOVA, on the sufficiency of three listeners per item, and on LaTeX as ground truth. No new entities are introduced.

assumptions (3)

standard math ANOVA assumptions (independence, normality, homoscedasticity) for LCER and MOS responses.
Used in Section 4 to infer significance of TTS Model and Category factors.
domain assumption Three listener ratings per AudioMX are sufficient to estimate intelligibility and MOS.
Section 3.1 assigns three listeners per item; no reliability analysis is reported.
domain assumption LaTeX rendering is a faithful ground-truth representation of the math expression.
Section 3.2 uses ground-truth LaTeX from HME to compute correctness metrics.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Intelligibility of Text-to-Speech Systems for Mathematical Expressions." pith.science (2026). https://pith.science/paper/HX5YGELM

@misc{pith2026250611086,
  author       = {Pith},
  title        = {Pith review of: Intelligibility of Text-to-Speech Systems for Mathematical Expressions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HX5YGELM}},
  note         = {Machine review of arXiv:2506.11086}
}

read the original abstract

There has been limited evaluation of advanced Text-to-Speech (TTS) models with Mathematical eXpressions (MX) as inputs. In this work, we design experiments to evaluate quality and intelligibility of five TTS models through listening and transcribing tests for various categories of MX. We use two Large Language Models (LLMs) to generate English pronunciation from LaTeX MX as TTS models cannot process LaTeX directly. We use Mean Opinion Score from user ratings and quantify intelligibility through transcription correctness using three metrics. We also compare listener preference of TTS outputs with respect to human expert rendition of same MX. Results establish that output of TTS models for MX is not necessarily intelligible, the gap in intelligibility varies across TTS models and MX category. For most categories, performance of TTS models is significantly worse than that of expert rendition. The effect of choice of LLM is limited. This establishes the need to improve TTS models for MX.

Figures

Figures reproduced from arXiv: 2506.11086 by the authors.

**Figure 1.** Dataset formats, experiment design and metrics. • RQ2:- Is there any perceived difference between an human expert rendered audio recording of a MX (RAudioMX) and TTS output MX (AudioMX) in terms of intelligibility, quality, clarity and/or speed? To the best of our knowledge, this is the first study of the TTS models for MX with a focus on user perception and intelligibility. Our main contributions are: • Establish… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 37 canonical work pages

[1]

Intelligibility of Text-to-Speech Systems for Mathematical Expressions

Introduction There have been significant advances in Text-to-Speech (TTS) models alongside those of Large language models (LLMs) in the recent years [1, 2, 3]. These have resulted in high qual- ity TTS outputs that are nearly indistinguishable from human speech [2, 3]. Typical applications for expressive speech con- sider and evaluate audio versions of eB...

work page Pith review arXiv 2025
[2]

Dataset There are limited datasets focusing on TTS using MX

Dataset and Methodology 2.1. Dataset There are limited datasets focusing on TTS using MX. HAMEX

work page
[3]

Did not understand

Experimental design Figure 1 shows our experimental design of two listening tests. 3.1. L1 - Intelligibility Listening Tests To address RQ1, the L1 listening test measures intelligibility of MX using only AudioMX. The listeners are not provided with the ground truth MX, the LLM or the TTS model corresponding to the AudioMX. The listeners are asked to tran...

work page
[4]

These met- rics address RQ1

Results and Analysis Table 2 tabulates the average MOS, LCER, CoC and TexBLEU metrics (detailed in Section 3.1 and 3.2) across categories and the considered TTS models for L1 listening test. These met- rics address RQ1. From Table 2(a)-(d), we observe that there is no TTS model which scores consistently high across categories and across metrics. It is als...

work page
[5]

Discussion 5.1. Qualitative Analysis of MX Transcription Error Some of the patterns and errors observed during process of manual verification of transcribed MX (discussed in Section 3.2), common feedbacks resulting from a focus-group discus- sion with listeners are detailed below. • Pronunciation related – Confusion among variables such as (‘b’, ‘p’), sim...

work page
[6]

We observe that LLMs, such as GPT4, can pro- vide the pronunciations from MX

Conclusions & Future Work Our research establishes the variability of TTS performance (for RQ1) based on category and the gap with human rendered au- dio (for RQ2). We observe that LLMs, such as GPT4, can pro- vide the pronunciations from MX. Any real-world system will need an analysis of the categories of expressions relevant in that application; the cho...

work page
[7]

These increase LCER and reduce CoC and TeXBLEU metrics

to be comprehended and transcribed as (−a − b2√ 2). These increase LCER and reduce CoC and TeXBLEU metrics. – The TTS models do not consider appropriate pauses nor adjust the speed of audio for more complex MX (such as calculus, summation, or those having equality signs). This results in multiple listenings per MX, causing listening fa- tigue. The incorre...

work page
[8]

Adaptive math-to-speech interface,

A. Bier and Z. Sroczy ´nski, “Adaptive math-to-speech interface,” in Proceedings of the Mulitimedia, Interaction, Design and Innno- vation, 2015, pp. 1–9

work page 2015

Show all 40 references

[9]

Recent advances in speech language models: A survey,

W. Cui, D. Yu, X. Jiao, Z. Meng, G. Zhang, Q. Wang, Y . Guo, and I. King, “Recent advances in speech language models: A survey,” arXiv preprint arXiv:2410.03751, 2024

2024 arXiv
[10]

Deep learning-based expressive speech synthesis: a systematic review of approaches, challenges, and resources,

H. Barakat, O. Turk, and C. Demiroglu, “Deep learning-based expressive speech synthesis: a systematic review of approaches, challenges, and resources,” EURASIP Journal on Audio, Speech, and Music Processing, vol. 2024, no. 1, p. 11, 2024

2024
[11]

E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts,

S. E. Eskimez, X. Wang, M. Thakker, C. Li, C.-H. Tsai, Z. Xiao, H. Yang, Z. Zhu, M. Tang, X. Tan, Y . Liu, S. Zhao, and N. Kanda, “E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts,” pp. 682–689, 2024

2024
[12]

Mathbridge: A large corpus dataset for translating spoken mathematical expressions into latex formulas for improved readability,

K. Jung, S. Hyeon, J. Y . Kwon, N.-J. Kim, H. G. Ryu, H.-J. Lee, and J. Do, “Mathbridge: A large corpus dataset for translating spoken mathematical expressions into latex formulas for improved readability,” CoRR, vol. abs/2408.07081, 2024. [Online]. Available: https://doi.org/...

2024 doi arXiv
[13]

Handwritten mathematical expressions dataset,

R. Tatman, “Handwritten mathematical expressions dataset,” https://www.kaggle.com/datasets/rtatman/ handwritten-mathematical-expressions/data, 2017

2017
[14]

Combining speech and handwriting modalities for mathematical expression recognition,

S. Medjkoune, H. Mouchere, S. Petitrenaud, and C. Viard-Gaudin, “Combining speech and handwriting modalities for mathematical expression recognition,” IEEE Transactions on Human-Machine Systems, vol. 47, no. 2, pp. 259–272, 2017

2017
[15]

Mathematical for- mulas in text to speech system,

P. Caky, J. Boron, M. Klimo, and K. Bachrata, “Mathematical for- mulas in text to speech system,” Scientifi Letters of the University of Zilina, no. 3, 2009

2009
[16]

Mathvision: An accessible intelligent agent for visually impaired people to understand mathematical equa- tions,

M. Awais, T. Ahmed, M. Aslam, A. Rehman, F. S. Alamri, S. A. Bahaj, and T. Saba, “Mathvision: An accessible intelligent agent for visually impaired people to understand mathematical equa- tions,” IEEE Access, 2024

2024
[17]

Rule-based intelligent system verbal- izing mathematical notation,

A. Bier and Z. Sroczynski, “Rule-based intelligent system verbal- izing mathematical notation,”Multimedia Tools and Applications, vol. 78, pp. 28 089–28 110, 2019

2019
[18]

Grammar- driven text-to-speech application for articulation of mathematical expressions

M. M. Mogale, T. J. Sefara, and T. B. Mokgonyane, “Grammar- driven text-to-speech application for articulation of mathematical expressions.” Southern Africa Telecommunication Networks and Applications Conference (SATNAC), 2020

2020
[19]

Provide a pronunciation for the mathematical expression given in Latex below

had handwritten MX and their audio recordings, was one of the earliest, but to the best of our knowledge, is no longer pub- licly available. The Kaggle dataset Handwritten Mathematical Equations (HME) [5], a superset of HAMEX, has the handwrit- ten MX and LATEX ground truth, b...
[20]

Mathspeech: Leveraging small lms for accurate conversion in mathematical speech-to-formula,

S. Hyeon, K. Jung, J. Won, N.-J. Kim, H. G. Ryu, H.-J. Lee, and J. Do, “Mathspeech: Leveraging small lms for accurate conversion in mathematical speech-to-formula,” arXiv preprint arXiv:2412.15655, 2024

2024 arXiv
[21]

Exploring the limits of transfer learning with a unified text-to-text transformer,

C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research , vol. 21, no. 140, pp. 1–67, 2020. [Online]. Available: h...

2020
[22]

A call for clarity in reporting BLEU scores,

M. Post, “A call for clarity in reporting BLEU scores,” in Proceedings of the Third Conference on Machine Translation: Research Papers , O. Bojar, R. Chatterjee, C. Federmann, M. Fishel, Y . Graham, B. Haddow, M. Huck, A. J. Yepes, P. Koehn, C. Monz, M. Negri, A. N ´ev´eol, M....

2018
[23]

Listening to formulas: Pioneering models and datasets for converting speech to latex equations,

D. Korzh, A. Iudin, N. Kuzmin, E. Karimov, M. Skripkin, O. Rogov, and I. Oseledets, “Listening to formulas: Pioneering models and datasets for converting speech to latex equations,”
[24]

Amazon polly,

A. W. Services, “Amazon polly,” https://aws.amazon.com/polly/, 2023, accessed: 2025-02-07

2023
[25]

Mathreader : Text-to-speech for mathematical documents,

S. Hyeon, K. Jung, N.-J. Kim, H. G. Ryu, and J. Do, “Mathreader : Text-to-speech for mathematical documents,” 2025. [Online]. Available: https://arxiv.org/abs/2501.07088

2025 arXiv
[26]

Techniques for automatically correcting words in text,

K. Kukich, “Techniques for automatically correcting words in text,” ACM computing surveys (CSUR) , vol. 24, no. 4, pp. 377– 439, 1992

1992
[27]

Towards a prosodic model for synthe- sized speech of mathematical expressions in mathml,

A. Souza and D. Freitas, “Towards a prosodic model for synthe- sized speech of mathematical expressions in mathml,” inProceed- ings of the 9th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info- Exclusion, 2020, pp. 105–110

2020
[28]

Evaluating leading text-to-speech models,

Labelbox, “Evaluating leading text-to-speech models,” 2024, accessed: 2025-02-11. [Online]. Available: https://labelbox.com/ guides/evaluating-leading-text-to-speech-models/

2024
[29]

Hamex-a handwritten and audio dataset of mathematical expressions,

S. Quiniou, H. Mouch `ere, C. Viard-Gaudin, E. Morin, S. Pe- titrenaud, S. Medjkoune et al., “Hamex-a handwritten and audio dataset of mathematical expressions,” in 2011 International Con- ference on Document Analysis and Recognition. IEEE, 2011, pp. 452–456

2011
[30]

Seamless: Multilingual expressive and streaming speech translation,

S. et al, “Seamless: Multilingual expressive and streaming speech translation,” 2023

2023
[31]

Google text-to-speech,

G. Cloud, “Google text-to-speech,” https://cloud.google.com/ text-to-speech, 2023, accessed: 2025-02-07

2023
[32]

Azure ai speech,

M. Azure, “Azure ai speech,” https://azure.microsoft.com/en-us/ products/ai-services/ai-speech, 2023, accessed: 2025-02-07

2023
[33]

Openai text-to-speech guide,

OpenAI, “Openai text-to-speech guide,” https://platform.openai. com/docs/guides/text-to-speech, 2023, accessed: 2025-02-07

2023
[35]

Texbleu: Automatic metric for evaluate latex format,

K. Jung, N.-J. Kim, H. Ryu, S. Hyeon, S.-j. Lee, and H.-j. Lee, “Texbleu: Automatic metric for evaluate latex format,” arXiv preprint arXiv:2409.06639, 2024

2024 arXiv
[37]

ITU-R BS.1534: Method for the Subjective Assessment of Intermediate Quality Levels of Coding Systems,

International Telecommunications Union, “ITU-R BS.1534: Method for the Subjective Assessment of Intermediate Quality Levels of Coding Systems,” International Telecommunications Union, Tech. Rep. BS.1534, July 2014

2014
[38]

Analysis of variance (anova),

L. St, S. Wold et al., “Analysis of variance (anova),” Chemomet- rics and intelligent laboratory systems, vol. 6, no. 4, pp. 259–272, 1989

1989
[39]

Robust speech recognition via large-scale weak supervision,

A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International conference on machine learning . PMLR, 2023, pp. 28 492–28 518

2023
[40]

Assessing speech-to- text translation quality: An overview of key metrics,

M. Labied, A. Belangour, and M. Banane, “Assessing speech-to- text translation quality: An overview of key metrics,” in 2024 International Conference on Decision Aid Sciences and Applica- tions (DASA). IEEE, 2024, pp. 1–6

2024
[191]

Available: https://aclanthology.org/W18-6319/

[Online]. Available: https://aclanthology.org/W18-6319/
[2024]

Available: https://openreview.net/forum?id= pflsJ6V6CL

[Online]. Available: https://openreview.net/forum?id= pflsJ6V6CL

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Intelligibility of Text-to-Speech Systems for Mathematical Expressions

Introduction There have been significant advances in Text-to-Speech (TTS) models alongside those of Large language models (LLMs) in the recent years [1, 2, 3]. These have resulted in high qual- ity TTS outputs that are nearly indistinguishable from human speech [2, 3]. Typical applications for expressive speech con- sider and evaluate audio versions of eB...

work page Pith review arXiv 2025

[2] [2]

Dataset There are limited datasets focusing on TTS using MX

Dataset and Methodology 2.1. Dataset There are limited datasets focusing on TTS using MX. HAMEX

work page

[3] [3]

Did not understand

Experimental design Figure 1 shows our experimental design of two listening tests. 3.1. L1 - Intelligibility Listening Tests To address RQ1, the L1 listening test measures intelligibility of MX using only AudioMX. The listeners are not provided with the ground truth MX, the LLM or the TTS model corresponding to the AudioMX. The listeners are asked to tran...

work page

[4] [4]

These met- rics address RQ1

Results and Analysis Table 2 tabulates the average MOS, LCER, CoC and TexBLEU metrics (detailed in Section 3.1 and 3.2) across categories and the considered TTS models for L1 listening test. These met- rics address RQ1. From Table 2(a)-(d), we observe that there is no TTS model which scores consistently high across categories and across metrics. It is als...

work page

[5] [5]

Discussion 5.1. Qualitative Analysis of MX Transcription Error Some of the patterns and errors observed during process of manual verification of transcribed MX (discussed in Section 3.2), common feedbacks resulting from a focus-group discus- sion with listeners are detailed below. • Pronunciation related – Confusion among variables such as (‘b’, ‘p’), sim...

work page

[6] [6]

We observe that LLMs, such as GPT4, can pro- vide the pronunciations from MX

Conclusions & Future Work Our research establishes the variability of TTS performance (for RQ1) based on category and the gap with human rendered au- dio (for RQ2). We observe that LLMs, such as GPT4, can pro- vide the pronunciations from MX. Any real-world system will need an analysis of the categories of expressions relevant in that application; the cho...

work page

[7] [7]

These increase LCER and reduce CoC and TeXBLEU metrics

to be comprehended and transcribed as (−a − b2√ 2). These increase LCER and reduce CoC and TeXBLEU metrics. – The TTS models do not consider appropriate pauses nor adjust the speed of audio for more complex MX (such as calculus, summation, or those having equality signs). This results in multiple listenings per MX, causing listening fa- tigue. The incorre...

work page

[8] [8]

Adaptive math-to-speech interface,

A. Bier and Z. Sroczy ´nski, “Adaptive math-to-speech interface,” in Proceedings of the Mulitimedia, Interaction, Design and Innno- vation, 2015, pp. 1–9

work page 2015