REVIEW 4 major objections 5 minor 1 cited by
MathReader : Text-to-Speech for Mathematical Documents
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A reading pipeline that turns LaTeX formulas into spoken English before synthesizing speech lowers word error rates on math documents far below what Microsoft Edge and Adobe Acrobat achieve.
desk verdict A reasonable, clearly described math-TTS pipeline with released code, but the headline WER gains are not yet established because the evaluation relies on a single ASR engine without human listening or error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a translation step placed between OCR and TTS. Nougat-small converts the PDF to a markup file; a delimiter-based splitter isolates formula segments using the math-delimiter patterns that the OCR step inserts; a fine-tuned T5-small model converts those segments into spoken English; and VITS synthesizes speech from the resulting plain text. The T5 translator is what carries the argument, because the ablation shows that removing it raises WER from 0.281 to 0.663.
What would settle it
Have independent listeners transcribe the audio produced by MathReader, Microsoft Edge, and Adobe Acrobat on the same set of PDFs, using a pre-registered spoken-English ground truth; if MathReader's word error rate is not lower than the commercial readers' under that human-listener comparison, the paper's central claim is refuted.
Extended reading notes
Core claim
The central discovery is that for mathematical PDFs, accurate spoken output depends less on the speech synthesizer than on what text is sent to it. If LaTeX code is read directly, special characters are dropped or mispronounced; if formulas are first verbalized as natural English sentences, a standard TTS engine produces understandable speech. The paper demonstrates this by building the full pipeline and comparing it to two widely used readers.
Load-bearing premise
The evaluation assumes that the word error rate computed by an automated speech recognizer on each system's synthesized voice is a valid measure of reading accuracy, even though the three systems use different voices and the ground-truth spoken-English transcriptions were typed by the same team that built the translator.
Editorial extensions
If this is right
- If the reported error rates hold, visually impaired readers could listen to math-heavy PDFs, including older scanned ones, and hear formulas rather than symbol salad.
- The measured time of 23.62 seconds per page on GPU hardware suggests the pipeline is practical for on-demand reading rather than batch-only processing.
- The ablation result implies that the formula-translation step, not the speech synthesizer, is what separates accurate from garbled math reading.
- Because the implementation is released with the paper, other document-reading tools could adopt the same OCR-translate-TTS chain.
Reading between the lines
- The paper's WER numbers come from an automated speech recognizer, so an independent listening test with human transcribers could change the ranking; this is my inference, not the paper's claim.
- The translation step should transfer to formulas rendered as web MathML or to equation images in lecture slides, since the pipeline only needs the formula's source text; the paper does not test these inputs.
- A broader test corpus spanning physics and engineering notation would test whether T5-small generalizes beyond the authors' manual transcription conventions; this is an extension I am proposing.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes MathReader, a pipeline that converts PDF documents containing mathematical formulas into speech. It uses Nougat-small to OCR the PDF into a markdown/math markup file, extracts LaTeX formula segments using delimiter patterns, translates those formulas into spoken English with a fine-tuned T5-small model trained on the MathBridge corpus, replaces the LaTeX with the spoken English, and synthesizes speech with VITS. The authors evaluate MathReader against Microsoft Edge and Adobe Acrobat by measuring WER and CER of Naver Clova Note ASR transcriptions of the synthesized audio, reporting substantial improvements (WER 0.281 vs 0.510 and 0.617), plus an ablation showing worse performance without T5. The paper also reports a per-page runtime of about 23.6 seconds on an H100.
Significance. If the reported results are reliable, MathReader would be a practical, lightweight assistive tool for reading math-heavy documents, with modest compute requirements (T5-small, VITS). The paper addresses a real accessibility problem and makes code available. However, the current evaluation is not sufficient to establish the central claim: the WER measure is confounded with ASR voice biases, the test set is not described quantitatively, and the ground truth is author-generated with conventions aligned to the training corpus. The contribution is therefore promising but not yet validated.
major comments (4)
- [IV-C] The evaluation in Section IV-C computes WER from transcripts produced by a single ASR system (Naver Clova Note) applied to audio synthesized with different TTS voices. Because MathReader uses VITS while Edge and Acrobat use their own voices, ASR error differences can masquerade as reading-accuracy differences; homophone examples mentioned in the paper (e.g., y vs why, T vs Tee) are exactly ASR-level confounds. The authors should provide human-listener transcriptions (or at least a second ASR and a plain-text control condition) to separate TTS/ASR artifacts from formula-reading quality.
- [IV-A] The test dataset description in Section IV-A reports no number of documents, pages, formulas, or source documents, and the ground-truth spoken English was manually typed by the authors. Since the T5 translator was fine-tuned on MathBridge [27], created by the same research group, and the test transcriptions are likely to follow the same conventions, the measured translation quality may reflect the label convention rather than general reading accuracy. Please report the test set size, make the full test data and annotation guidelines available, and include an independent or at least second-annotator ground truth.
- [IV-C / Table II] Table II reports only aggregate WER and CER for each reader, with no test-set size, per-document results, error bars, or significance tests. The observed differences could be within noise if the test corpus is small or heterogeneous. Please provide the number of documents/formulas and a paired statistical test (e.g., Wilcoxon signed-rank) on per-document WER/CER.
- [I / III-E] The statements in Section I ('Our pipeline can read all the formulas in documents without omissions') and Section III-E ('the TTS model outputs a voice without any errors') are stronger than what the experiments support: OCR (Nougat-small) can misrecognize formulas, delimiter-based extraction can miss undelimited or malformed math, and T5-small can mistranslate. The paper should quantify formula detection and translation coverage, not just overall WER.
minor comments (5)
- [Algorithm 1] The line 'Let pd fbe the input document' contains a typo; it should read 'Let pdf be the input document.'
- [III-B] The delimiter patterns are written as '\[ ] \ or \( ) \' with garbled spacing; the intended patterns are \[ ... \] and \( ... \). Also, 'Seperate' is misspelled as 'Separate' in Algorithm 1, Section III-B, and Table IV.
- [Table III] The formula column is rendered ambiguously (e.g., '5P n=1' does not convey the summation notation); a LaTeX rendering or clearer image would improve reproducibility.
- [IV-E] The runtime is reported for one page on an H100 with unspecified batching or model-loading details; please clarify whether the timing includes model initialization and whether it generalizes to multi-page documents.
- [References] References [22] (MathSpeech) and [27] (MathBridge) are from the same research group; the relationship between MathReader and these prior works should be stated explicitly in the text to avoid confusion.
Circularity Check
The central WER claim is evaluated against the authors' own spoken-English ground truth, which was generated by the same group that created the T5 training corpus; the comparison is therefore a self-consistency check rather than an independent test.
-
fitted input called prediction
[Section III-C (Fine-tuned T5) and Section IV-A (Test Dataset)]
"According to previous research [27], converting LaTeX into spoken English text can be considered a type of ”translation” task. Therefore, we used a fine-tuned T5 model to convert all LaTeX formulas in the mmd file into spoken English. To fine-tune T5 for the LaTeX translation task, we needed a dataset composed of (LaTeX - Spoken English) pairs, and because such a dataset already exists publicly [27], we used it. ... Therefore, we manually typed the LaTeX formulas in the documents in spoken English."
The training corpus MathBridge [27] was created by the same author group (Jung, Hyeon, Kim, Ryu, Lee, and Do), and the test ground truth is manually typed by the authors with no independent transcription standard. Thus the 'correct' spoken English for a formula is the authors' own convention, which is exactly the convention the T5 model was fine-tuned to reproduce. The WER comparison therefore measures how closely MathReader matches the authors' labeling convention rather than an external, independently verified reading. Edge and Acrobat are not privy to that convention, so the reported WER gap is partly a self-consistency score. This is a fitted-input-called-prediction pattern: the model is fit to a target convention and then 'predicts' that same convention on the test set.
full rationale
The paper's pipeline (OCR -> formula extraction -> T5 translation -> VITS TTS) is a straightforward engineering integration and does not, on its own, contain a derivation that reduces to its inputs. The circularity is confined to the evaluation of the T5 translation component. Section IV-A states that no spoken-English transcriptions existed for the test documents, so the authors manually typed the ground truth. Section III-C states that the T5 model was fine-tuned on MathBridge [27], a corpus created by the same author group. Because the test labels and the training target share the same author-defined convention for reading formulas aloud, the WER measured against those labels is a measure of the model's fit to that convention, not to an independent standard. The head-to-head comparison with Edge and Acrobat inherits this bias: those systems were not built to follow the authors' convention. A separate methodological concern, the use of a single ASR engine (Naver Clova Note) to score audio from different TTS voices, could confound WER with ASR voice familiarity; however, that is an evaluation-validity issue, not a circularity of the derivation. No other load-bearing self-citations or imported uniqueness claims appear. Overall score 5 reflects that the central quantitative claim is substantially supported only by self-referential evaluation.
Assumptions & free parameters
free parameters (4)
- T5-small fine-tuned weights =
unknown
- Nougat-small OCR weights =
unknown
- VITS TTS weights =
unknown
- T5 training hyperparameters =
learning rate 1e-4, batch size 48, 20 epochs, seq len 325
assumptions (5)
- domain assumption Nougat's mmd output reliably delimits LaTeX formulas with \[ ... \] or \( ... \) markers.
- domain assumption TTS models cannot read LaTeX special characters, so translation into spoken English is required.
- domain assumption LaTeX-to-spoken-English can be treated as a text-to-text translation task that T5-small can learn.
- domain assumption Naver Clova Note ASR produces WER/CER measurements that fairly compare different TTS outputs.
- domain assumption The manually typed spoken-English ground truth in Section IV-A represents correct reading of the formulas.
Cite this review
Pith. "Pith review of MathReader : Text-to-Speech for Mathematical Documents." pith.science (2026). https://pith.science/paper/JSR4VZHU
@misc{pith2026250107088,
author = {Pith},
title = {Pith review of: MathReader : Text-to-Speech for Mathematical Documents},
year = {2026},
howpublished = {\url{https://pith.science/paper/JSR4VZHU}},
note = {Machine review of arXiv:2501.07088}
}
read the original abstract
TTS (Text-to-Speech) document reader from Microsoft, Adobe, Apple, and OpenAI have been serviced worldwide. They provide relatively good TTS results for general plain text, but sometimes skip contents or provide unsatisfactory results for mathematical expressions. This is because most modern academic papers are written in LaTeX, and when LaTeX formulas are compiled, they are rendered as distinctive text forms within the document. However, traditional TTS document readers output only the text as it is recognized, without considering the mathematical meaning of the formulas. To address this issue, we propose MathReader, which effectively integrates OCR, a fine-tuned T5 model, and TTS. MathReader demonstrated a lower Word Error Rate (WER) than existing TTS document readers, such as Microsoft Edge and Adobe Acrobat, when processing documents containing mathematical formulas. MathReader reduced the WER from 0.510 to 0.281 compared to Microsoft Edge, and from 0.617 to 0.281 compared to Adobe Acrobat. This will significantly contribute to alleviating the inconvenience faced by users who want to listen to documents, especially those who are visually impaired. The code is available at https://github.com/hyeonsieun/MathReader.
Figures
Forward citations
Cited by 1 Pith paper
-
Intelligibility of Text-to-Speech Systems for Mathematical Expressions
State-of-the-art text-to-speech models are often unintelligible when reading mathematical expressions aloud, with accuracy varying sharply by expression category and model.
Reference graph
Works this paper leans on
-
[27]
Kyudan Jung, Sieun Hyeon, Jeong Youn Kwon, Nam-Joon Kim, Hyun Gon Ryu, Hyuk-Jae Lee, and Jaeyoung Do. Mathbridge: A large corpus dataset for translating spoken mathematical expressions into latex formulas for improved readability, 2024
work page 2024
-
[1]
Microsoft. Pdf reader in microsoft edge. https://learn.microsoft. com/en-us/deployedge/microsoft-edge-pdf#read-aloud, 2024. Accessed: 2024-09-13
work page 2024
-
[2]
Conveniently hear pdf files read aloud
Adobe. Conveniently hear pdf files read aloud. https://www.adobe.com/ acrobat/hub/how-to-read-pdf-aloud.html, 2024. Accessed: 2024-09-13
work page 2024
-
[3]
Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech
Jaehyeon Kim, Jungil Kong, and Juhee Son. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th Interna- tional Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research, pages 5530–5540. PMLR, 18–24 Jul 2021
work page 2021
-
[4]
V oicebox: Text-guided multilingual universal speech generation at scale
Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. V oicebox: Text-guided multilingual universal speech generation at scale. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems...
work page 2023
-
[5]
High-fidelity audio compression with improved rvqgan
Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar. High-fidelity audio compression with improved rvqgan. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 27980–27993. Curran Associates, Inc., 2023
work page 2023
-
[6]
Jungil Kong, Jihoon Park, Beomjeong Kim, Jeongmin Kim, Dohee Kong, and Sangjin Kim. Vits2: Improving quality and efficiency of single-stage text-to-speech with adversarial learning and architecture design. In INTERSPEECH 2023, pages 4374–4378, 2023
work page 2023
-
[7]
Naturalspeech: End-to-end text- to-speech synthesis with human-level quality
Xu Tan, Jiawei Chen, Haohe Liu, Jian Cong, Chen Zhang, Yanqing Liu, Xi Wang, Yichong Leng, Yuanhao Yi, Lei He, Sheng Zhao, Tao Qin, Frank Soong, and Tie-Yan Liu. Naturalspeech: End-to-end text- to-speech synthesis with human-level quality. IEEE Transactions on Pattern Analysis and Machine Intelligence , 46(6):4234–4245, 2024
work page 2024
Show all 28 references
-
[8]
Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers
Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. arXiv preprint arXiv:2304.09116, 2023
2023 arXiv
-
[9]
Pdf2latex: A deep learning system to convert mathematical documents from pdf to latex
Zelun Wang and Jyh-Charn Liu. Pdf2latex: A deep learning system to convert mathematical documents from pdf to latex. In Proceedings of the ACM Symposium on Document Engineering 2020, pages 1–10, 2020
2020
-
[10]
Nougat: Neural optical understanding for academic documents, 2023
Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents, 2023
2023
-
[11]
Upocr: Towards unified pixel- level ocr interface
Dezhi Peng, Zhenhua Yang, Jiaxin Zhang, Chongyu Liu, Yongxin Shi, Kai Ding, Fengjun Guo, and Lianwen Jin. Upocr: Towards unified pixel- level ocr interface. In Forty-first International Conference on Machine Learning, 2023
2023
-
[12]
Advancing ocr accuracy in image-to-latex conversion—a critical and creative exploration
Everistus Zeluwa Orji, Ali Haydar, ˙Ibrahim Ers ¸an, and Othmar Othmar Mwambe. Advancing ocr accuracy in image-to-latex conversion—a critical and creative exploration. Applied Sciences, 13(22):12503, 2023
2023
-
[13]
Sequence-to-sequence learning using deep learning for optical character recognition (ocr)
Vishal Mishra and Devinder Kaur. Sequence-to-sequence learning using deep learning for optical character recognition (ocr). In 2018 International Conference on Computational Science and Computational Intelligence (CSCI), pages 324–329. IEEE, 2018
2018
-
[14]
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Explor- ing the limits of transfer learning with a unified text-to-text transformer, 2023
2023
-
[15]
Latexvoice: Application based on latex to generate documents designed for visually impaired users
JAS Sanmiguel and Luiz Cesar Martini. Latexvoice: Application based on latex to generate documents designed for visually impaired users. International Journal of Artificial Intelligence-IJAI , 2:1–4, 2015
2015
-
[16]
Alap: Accessible latex based mathematical document authoring and presentation
Ahtsham Manzoor, Safa Arooj, Shaban Zulfiqar, Murayyiam Parvez, Suleman Shahid, and Asim Karim. Alap: Accessible latex based mathematical document authoring and presentation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, page 1–12, N...
2019
-
[17]
Web-alap: A web-based latex editor for blind individuals
Safa Arooj, Shaban Zulfiqar, Muhammad Qasim Hunain, Suleman Shahid, and Asim Karim. Web-alap: A web-based latex editor for blind individuals. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility , ASSETS ’20, New York, NY , USA, 2020...
2020
-
[18]
Texbleu: Automatic metric for evaluate latex format, 2024
Kyudan Jung, Nam-Joon Kim, Hyongon Ryu, Sieun Hyeon, Seung jun Lee, and Hyeok jae Lee. Texbleu: Automatic metric for evaluate latex format, 2024
2024
-
[19]
Using artificial-intelligence tools to make latex content accessible to blind readers
Gerd Kortemeyer. Using artificial-intelligence tools to make latex content accessible to blind readers. arXiv preprint arXiv:2306.02480 , 2023
2023 arXiv
-
[20]
Latex-gcl: Large language models (llms)-based data augmen- tation for text-attributed graph contrastive learning
Haoran Yang, Xiangyu Zhao, Sirui Huang, Qing Li, and Guandong Xu. Latex-gcl: Large language models (llms)-based data augmen- tation for text-attributed graph contrastive learning. arXiv preprint arXiv:2409.01145, 2024
2024 arXiv
-
[21]
An editing process for blind or visually impaired editors
Matthew J Baker, EM Nightingale, and Suzy Bills. An editing process for blind or visually impaired editors. IEEE Transactions on Professional Communication, 64(3):275–287, 2021
2021
-
[22]
Mathspeech: Leveraging small lms for accurate conversion in mathematical speech-to-formula, 2024
Sieun Hyeon, Kyudan Jung, Jaehee Won, Nam-Joon Kim, Hyun Gon Ryu, Hyuk-Jae Lee, and Jaeyoung Do. Mathspeech: Leveraging small lms for accurate conversion in mathematical speech-to-formula, 2024
2024
-
[23]
Microsoft speech platform
Microsoft, L. Microsoft speech platform. http://msdn.microsoft.com/ en-us/library/jj127449.aspx, 2024. Accessed: 2024-08-15
2024
-
[24]
Gpt-4 technical report
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023
2023 arXiv
-
[25]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...
2021
-
[26]
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 10012– 10022, October 2021
2021
-
[28]
Naver clovanote
NA VER Corp. Naver clovanote. https://clovanote.naver.com/, 2024. Accessed: 2024-08-15
2024
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.