Greek2MathTex: A Greek Speech-to-Text Framework for LaTeX Equations Generation

Elisavet Palogiannidi; Evangelia Gkritzali; Panagiotis Kaliosis; Sofia Galanaki; Theodoros Giannakopoulos

REVIEW 4 major objections 5 minor 12 references

Greek2MathTex: A Greek Speech-to-Text Framework for LaTeX Equations Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that dictating Greek mathematics aloud can produce LaTeX code through a pipeline of a fine-tuned speech recognizer and a large language model prompted with retrieved example equations.

desk verdict Useful dataset and a sane text-to-LaTeX pipeline, but the headline speech-to-LaTeX claim is untested: no audio goes through the system in the reported experiments. read the letter →

arxiv 2412.12167 v1 pith:5UCICYXB submitted 2024-12-11 cs.CL cs.AIeess.AS

Evangelia Gkritzali , Panagiotis Kaliosis , Sofia Galanaki , Elisavet Palogiannidi , Theodoros Giannakopoulos This is my paper · ORCID

classification cs.CLcs.AIeess.AS

keywords GreekspeechrecognitionLaTeXequationgenerationin-contextlearningretrieval-augmentedaccessibilitytechnologyautomaticLevenshteineditdistancemathdictation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a Greek-specific speech-to-LaTeX system is feasible by combining a fine-tuned speech recognizer with a large language model prompted with retrieved example equations. The authors built a dataset of 500 pairs of Greek spoken equations and their LaTeX forms, and they report that retrieval-based in-context learning markedly improves the text-to-LaTeX stage, lifting the share of outputs with edit distance below 0.1 from 27.45% to 37.67% at $k=5$ with cosine similarity and the English instruction prompt. They frame the contribution as an accessibility tool: users who struggle to type or read LaTeX can dictate mathematics and receive compilable code.

What carries the argument

The load-bearing mechanism is retrieval-augmented in-context learning: for each dictated equation, a $k$-nearest-neighbour search over the held-out Gr2Tex examples selects the $k$ most similar natural-language/LaTeX pairs, which are prepended to the prompt alongside an instruction, and the large language model then generates LaTeX for the query without updating its parameters. The paper's custom evaluation metric is the Levenshtein distance between normalized generated and ground-truth strings, used with thresholds 0.1 and 0.4. The retrieval mechanism and prompt design, not the speech recognizer, carry the reported accuracy gains.

What would settle it

Run the audio recordings of the test split through the fine-tuned recognizer and then through the best retrieval-augmented prompt, and compute the same edit-distance metrics on the final LaTeX output. If the end-to-end rate of edit distance below 0.1 is far below 37.67%, or the recognizer's error rate on dictated math is high, the central speech-to-LaTeX claim is unsupported.

Watch

Extended reading notes

Core claim

The central discovery the paper tries to establish is that an end-to-end Greek speech-to-LaTeX pipeline can be assembled from a fine-tuned XLS-R speech model, a retrieval mechanism over a small paired dataset (Gr2Tex), and GPT-3.5 with in-context learning. Evaluated on the test split of Gr2Tex, retrieval-augmented prompting outperforms no-example prompting on all reported metrics, with cosine similarity at $k=5$ and the English instruction prompt $p_2$ giving the best rate of outputs within edit distance 0.1 of the ground truth (37.67% versus a 27.45% baseline). English instructions slightly beat their Greek translations, and increasing $k$ to six does not improve over $k=5$. The authors claim this demonstrates a usable advancement in spoken-math accessibility for Greek speakers.

Load-bearing premise

The load-bearing premise is that evaluating on hand-transcribed text with a small test set reflects real speech performance, because the paper never measures recognition errors on dictated math audio.

Editorial extensions

If this is right

Adding retrieval examples to the prompt raises the share of outputs with edit distance below 0.1 from 27.45% to 37.67% at $k=5$ with cosine similarity and the English instruction prompt $p_2$.
Increasing the number of retrieved examples from five to six gives no further improvement, so the benefit of retrieval saturates.
Cosine similarity performs better than Manhattan or Euclidean distance in the tested configurations, and English prompts outperform a Greek translation of the same instruction.
Because in-context learning requires no retraining, the LaTeX generation stage can be updated by changing the prompt or the retrieved examples without modifying the speech recognizer.
The system's web interface lets a Greek speaker record an equation, play it back, and receive the generated LaTeX code, which is the accessibility scenario the paper targets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported accuracy is measured on hand-transcribed text rather than on the recognizer's output, so the true end-to-end speech-to-LaTeX accuracy remains untested and would likely be lower once recognition errors enter.
The same retrieval-plus-in-context-learning recipe should transfer to other languages that lack math-speech data, since it needs only a few hundred paired examples and a passable recognizer.
The observed advantage of English prompts over Greek suggests the underlying language model generates LaTeX more reliably from English framing; a Greek-specialized model or bilingual prompt tuning could close that gap.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this is a short, workshop-quality paper with one genuinely useful asset—the Gr2Tex dataset and open-source code—and one load-bearing empirical gap: the system advertised as speech-to-LaTeX is never evaluated on speech. Table 2 only feeds manually transcribed text into GPT-3.5 with retrieval ICL. No word error rate, no audio-to-LaTeX result, nothing. The authors describe fine-tuning XLS-R on Greek audio plus the Gr2Tex dataset, but they don't report how well it transcribes math dictation, which is the whole point of the title.

The new thing here is real: the first Greek-specific dataset for natural-language math expressions paired with LaTeX (500 pairs), released openly, and a sensible retrieval-augmented prompt recipe for GPT-3.5. The custom EL metric is a reasonable practical tool, and the web app lowers the barrier for Greek-speaking users. The related work is appropriate and the architecture choices are defensible.

The soft spots are proportionate to the missing evidence. First, the end-to-end claim collapses if the ASR stage is poor, and we have no data on that. Second, the evaluation is small: roughly 75 test equations, with k, similarity/distance function, and prompt all selected on that same test set. No error bars, no significance tests. Third, Section 4.1 says XLS-R was fine-tuned on the same Gr2Tex dataset that provides the test split, without explicitly stating that test recordings were excluded—so the reader cannot rule out leakage. The custom EL metric's validation against human annotation is also described in one sentence with no numbers.

The reader's REJECT is fair if the bar is "does the paper support its advertised claim." It does not. But the paper is not worthless: the dataset and the text-to-LaTeX component are contributions in their own right, and the gaps are fixable with additional experiments.

Who this is for: people working on speech-to-formula systems, accessibility, or Greek-language NLP. I would not cite the paper for its central claim, but I would cite the Gr2Tex dataset. A serious editor should send this to peer review, because the resource is useful and the authors can reasonably add the missing ASR evaluation or reframe the contribution as text-to-LaTeX generation plus a dataset. My recommendation: engage with it, ask for the end-to-end numbers, and let the authors decide whether to narrow the claims.

Referee Report

4 major / 5 minor

Summary. The paper presents Greek2MathTex, described as an end-to-end speech-to-LaTeX system for Greek mathematical dictation. The architecture combines a fine-tuned XLS-R ASR model, a k-NN retrieval mechanism over a self-built Greek dataset (Gr2Tex, 500 text/LaTeX pairs with audio), and GPT-3.5-turbo with in-context learning. The paper reports experiments in Table 2 showing that retrieval-based ICL improves text-to-LaTeX generation under a custom Levenshtein-based metric EL, and it releases code and a web application. No experiment feeds audio through the ASR component and measures the resulting LaTeX.

Significance. The intended contribution—an accessible Greek speech-to-LaTeX tool—is worthwhile, and the release of the Gr2Tex dataset and open-source code is a concrete asset. The paper also provides a useful comparison of prompts and retrieval configurations for LLM-based equation generation. However, the current evidence does not establish the headline speech-to-LaTeX claim because the quantitative evaluation is text-to-LaTeX only; the ASR component's accuracy and its interaction with retrieval are unmeasured. With an end-to-end evaluation or a narrowed claim, the contribution could be solid.

major comments (4)

[Section 5.2, Table 2] The only quantitative results are for text-to-LaTeX generation: Section 5.2 states the model was prompted to produce LaTeX for the 'provided transcribed text sequence,' and no word error rate, audio-to-text accuracy, or audio-to-LaTeX metric appears anywhere in the paper. The abstract and Section 1 claim the system works 'based solely on audio input,' so the central claim is currently unsupported. Please add an evaluation that feeds audio through the fine-tuned XLS-R model (e.g., WER on Gr2Tex test audio, followed by end-to-end EL/BLEU on the transcribed output), or explicitly rescope the paper's claim to text-to-LaTeX generation.
[Sections 4.1 and 3] Section 4.1 states that XLS-R was fine-tuned using 'our custom domain- and language-specific dataset' after Section 3 describes a 70/15/15 train/validation/test split of Gr2Tex. The text does not state that the test split's recordings were excluded from ASR fine-tuning. If they were not, the test set is contaminated and all downstream numbers are optimistic. Please state explicitly that the test split was held out from both retrieval database and ASR fine-tuning.
[Section 5.2, Table 2] The test set is only 15% of 500 pairs (about 75 equations), and the hyperparameters k, similarity/distance function, and prompt p1/p2/p3 are selected on that same test set. The reported improvements (EL<0.1 from 27.45% to 37.67%) are therefore not accompanied by any error bars, significance tests, or held-out validation. Please provide a separate validation split for hyperparameter selection or report confidence intervals; otherwise the ICL gains may reflect overfitting to the test queries.
[Section 5.1] The custom EL metric is said to have been assessed by comparing it to human annotations (match/almost match/no match), but no result of that comparison is reported. Without this validation, the EL<0.1 and EL>0.4 thresholds lack an independent interpretation. Please report the agreement/correlation between EL and the human labels, and if the comparison was only informal, state so.

minor comments (5)

[Table 2 and Section 5.2] The text says the fourth and fifth columns report percentages 'lower than 0.1 or 0.4,' but the table header reads 'EL > 0.4'; clarify whether the fifth column is the fraction above 0.4 (as the header and the discussion of reducing it suggest) and align the wording.
[Throughout] The dataset name is inconsistently written as Gr2Tex and Gr2TeX; choose one spelling.
[Figure 2] The caption contains the broken 'L ATEX' due to formatting; correct to LaTeX.
[Section 7] The first sentence calls the system 'speech-to-text,' although the contribution is speech-to-LaTeX; correct the terminology.
[Section 5.2] The text says the three instruction prompts are 'outlined in Table 2,' but they actually appear in Table 1; fix the cross-reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: results are measurements of a pipeline, not derivations; the main weakness is an untested ASR stage, which is a validity gap rather than a circular step.

full rationale

The paper contains no mathematical derivation chain whose conclusion could be equivalent to an input by construction. The central contribution is an engineered pipeline (fine-tuned XLS-R ASR, k-NN retrieval, GPT-3.5 prompting) evaluated with a Levenshtein-based EL metric, BLEU, and chrF. All references are external; there is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation. The evaluation in Section 5.2 measures only LaTeX generation from the "provided transcribed text sequence" and does not report an end-to-end audio-to-LaTeX result, and hyperparameters are selected on the same small test set; these are completeness and validity concerns rather than circular reductions. Similarly, the Gr2Tex dataset is self-created and the EL metric is validated only by an unpublished human comparison, which weakens external independence but does not make any result definitionally equal to its input. Therefore no circular step is identified and the score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its central empirical contribution depends on a small self-built dataset, several hyperparameters selected on the same test set, and implicit assumptions about data splits and metric reliability. These assumptions are the real 'free parameters' of the evaluation.

free parameters (5)

k (number of retrieved examples) = 5 (best EL<0.1); 6 for chrF
Selected by maximizing evaluation metrics on the test set; not tuned on a separate validation set.
Similarity/distance function = Cosine (slightly better in Table 2)
Cosine, Euclidean, and Manhattan were compared on the test set and cosine chosen.
Instruction prompt = p2
English prompt with examples outperformed p1 and p3 on test metrics.
EL thresholds = 0.1 and 0.4
Thresholds for declaring near-match in Levenshtein distance; chosen without stated justification.
Evaluation normalization rules = Replace Greek letters with Latin, remove delimiters/punctuation
These preprocessing choices change the distance values and are not validated against an external benchmark.

assumptions (5)

domain assumption The 500-pair Gr2Tex dataset is representative of Greek mathematical speech in real use.
The dataset is self-created and small; no external validation shows it covers the diversity of real user dictation.
domain assumption Human annotations used to validate the EL metric are reliable.
Section 5.1 mentions five annotators but reports no inter-annotator agreement or correlation values.
ad hoc to paper The retrieval database is disjoint from the test set.
The paper says a 'held-out set' is used for retrieval but does not specify the split or verify that test queries are absent.
ad hoc to paper Fine-tuning XLS-R on Gr2Tex does not include test recordings.
Section 4.1 does not state which split of Gr2Tex was used for ASR fine-tuning, so test leakage is possible.
domain assumption GPT-3.5-turbo output is treated as deterministic for evaluation.
Closed-source LLM responses can vary across calls, but the paper reports single-run scores without repeated sampling.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Greek2MathTex: A Greek Speech-to-Text Framework for LaTeX Equations Generation." pith.science (2026). https://pith.science/paper/5UCICYXB

@misc{pith2026241212167,
  author       = {Pith},
  title        = {Pith review of: Greek2MathTex: A Greek Speech-to-Text Framework for LaTeX Equations Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5UCICYXB}},
  note         = {Machine review of arXiv:2412.12167}
}

read the original abstract

In the vast majority of the academic and scientific domains, LaTeX has established itself as the de facto standard for typesetting complex mathematical equations and formulae. However, LaTeX's complex syntax and code-like appearance present accessibility barriers for individuals with disabilities, as well as those unfamiliar with coding conventions. In this paper, we present a novel solution to this challenge through the development of a novel speech-to-LaTeX equations system specifically designed for the Greek language. We propose an end-to-end system that harnesses the power of Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) techniques to enable users to verbally dictate mathematical expressions and equations in natural language, which are subsequently converted into LaTeX format. We present the architecture and design principles of our system, highlighting key components such as the ASR engine, the LLM-based prompt-driven equations generation mechanism, as well as the application of a custom evaluation metric employed throughout the development process. We have made our system open source and available at https://github.com/magcil/greek-speech-to-math.

Figures

Figures reproduced from arXiv: 2412.12167 by the authors.

**Figure 1.** An overview of the proposed system’s architecture. 4.1 Speech Recognition Component Our speech recognition component consisted of an instance of the XLS-R model developed by Meta AI [2]. While it is trained on thousands of hours of audio in multiple languages (including Greek), its performance on the Greek language was insufficient for integration into our proposed end-to-end pipeline. Thus, we underwent a fine-tu… view at source ↗

**Figure 2.** The UI when entering the application and during recording. The modal window appears when clicking on the LATEX button. [3] S. Batlouni, H. Karaki, F. Zaraket, and F. Karameh. 2011. Mathifier — Speech recognition of math equations. In 2011 18th IEEE International Conference on Electronics, Circuits, and Systems. 301–304. [4] Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, L. Li, and Z. Sui. 2023. A… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 11 canonical work pages

[1]

Ardila et al

R. Ardila et al. 2020. Common Voice: A Massively-Multilingual Speech Corpus. In Proceedings of the Twelfth Language Resources and Evaluation Conference . Marseille, France

work page 2020
[2]

A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y. Saraf, J. M. Pino, A. Baevski, A. Conneau, and M. Auli. 2021. XLS- R: Self-supervised Cross-lingual Speech Representation Learning at Scale. In Interspeech. Figure 2: The UI when entering the application and during record- ing. The modal window appears when clicking on...

work page 2021
[3]

Batlouni, H

S. Batlouni, H. Karaki, F. Zaraket, and F. Karameh. 2011. Mathifier — Speech recognition of math equations. In 2011 18th IEEE International Conference on Electronics, Circuits, and Systems . 301–304

work page 2011
[4]

Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, L. Li, and Z. Sui

work page
[5]

Li et al

R. Li et al. 2023. StarCoder: may the source be with you! Transactions on Machine Learning Research (2023)

work page 2023
[6]

Hanakovič and M

T. Hanakovič and M. Nagy. 2006. Speech Recognition Helps Visually Impaired People Writing Mathematical Formulas. In Computers Helping People with Special Needs. Springer Berlin Heidelberg, Berlin, Heidelberg, 1231–1234

work page 2006
[7]

Recent Approaches & Future Chal- lenges: Programs and Projects regarding the VI & MDVI

P. Katsoulis. 2008. The current educational situation for students with visual impairment in Greece: Trends and prospects. “Recent Approaches & Future Chal- lenges: Programs and Projects regarding the VI & MDVI” (2008)

work page 2008
[8]

Papineni, S

K. Papineni, S. Roukos, T. Ward, and W. Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, Philadelphia, Pennsylvania, USA

work page 2002

Show all 12 references

[9]

M. Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation . Association for Computational Linguistics, Lisbon, Portugal

2015
[10]

Wang and J

Z. Wang and J. Liu. 2021. Translating math formula images to LaTeX sequences using deep neural networks with sequence-level training. International Journal on Document Analysis and Recognition (IJDAR) 24 (06 2021), 1–13

2021
[11]

Wigmore, G

A. Wigmore, G. Hunter, E. Pfluegel, J. Denholm-Price, and V. Binelli. 2009. Us- ing Automatic Speech Recognition to Dictate Mathematical Expressions: The Development of the ’TalkMaths’ Application at Kingston University. (01 2009)

2009
[2023]

arXiv:2301.00234

A Survey on In-context Learning. arXiv:2301.00234

arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Ardila et al

R. Ardila et al. 2020. Common Voice: A Massively-Multilingual Speech Corpus. In Proceedings of the Twelfth Language Resources and Evaluation Conference . Marseille, France

work page 2020

[2] [2]

A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y. Saraf, J. M. Pino, A. Baevski, A. Conneau, and M. Auli. 2021. XLS- R: Self-supervised Cross-lingual Speech Representation Learning at Scale. In Interspeech. Figure 2: The UI when entering the application and during record- ing. The modal window appears when clicking on...

work page 2021

[3] [3]

Batlouni, H

S. Batlouni, H. Karaki, F. Zaraket, and F. Karameh. 2011. Mathifier — Speech recognition of math equations. In 2011 18th IEEE International Conference on Electronics, Circuits, and Systems . 301–304

work page 2011

[4] [4]

Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, L. Li, and Z. Sui

work page

[5] [5]

Li et al

R. Li et al. 2023. StarCoder: may the source be with you! Transactions on Machine Learning Research (2023)

work page 2023

[6] [6]

Hanakovič and M

T. Hanakovič and M. Nagy. 2006. Speech Recognition Helps Visually Impaired People Writing Mathematical Formulas. In Computers Helping People with Special Needs. Springer Berlin Heidelberg, Berlin, Heidelberg, 1231–1234

work page 2006

[7] [7]

Recent Approaches & Future Chal- lenges: Programs and Projects regarding the VI & MDVI

P. Katsoulis. 2008. The current educational situation for students with visual impairment in Greece: Trends and prospects. “Recent Approaches & Future Chal- lenges: Programs and Projects regarding the VI & MDVI” (2008)

work page 2008

[8] [8]

Papineni, S

K. Papineni, S. Roukos, T. Ward, and W. Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, Philadelphia, Pennsylvania, USA

work page 2002