Beyond Specialization: Benchmarking LLMs for Transliteration of Indian Languages

Erik Cambria; Gulfarogh Azam; Mohammad Nadeem; Mohammad Sultan Alam; Mohd Sadique; Saif Ali; Shahab Saquib Sohail

REVIEW 3 major objections 6 minor 34 references

Beyond Specialization: Benchmarking LLMs for Transliteration of Indian Languages

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that general-purpose LLMs, especially GPT-4.5 and fine-tuned GPT-4o, can match or beat the specialized IndicXlit model on Romanized-to-native-script transliteration for most of ten Indian languages.

desk verdict A useful benchmark comparison of LLMs vs IndicXlit, but the fine-tuning split is never confirmed to exclude the test sets, so the headline fine-tuned result could be memorization. read the letter →

arxiv 2505.19851 v1 pith:YDDY37II submitted 2025-05-26 cs.CL cs.AI

Gulfarogh Azam , Mohd Sadique , Saif Ali , Mohammad Nadeem , Erik Cambria , Shahab Saquib Sohail , Mohammad Sultan Alam This is my paper · ORCID

classification cs.CLcs.AI

keywords transliterationIndianlanguageslargelanguagemodelsIndicXlitDakshinadatasetAksharantarfine-tuningcharactererrorrate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether general-purpose large language models can transliterate Romanized Indian language text into native scripts as well as or better than IndicXlit, a specialized state-of-the-art model trained on a 26-million-pair corpus. Benchmarking GPT-4o, GPT-4.5, GPT-4.1, Gemma-3-27B-it, and Mistral-Large against IndicXlit across ten languages on the Dakshina and Aksharantar test sets, it reports that the GPT models beat the specialized system for most dataset-language pairs, with GPT-4.5 the strongest generalist. It further shows that fine-tuning GPT-4o on roughly one million transliteration pairs lifts accuracy above all other models for most scenarios, especially on named entities, and keeps the lowest character error rate under noisy inputs. The upshot is that a general foundation model, with minimal task-specific training, can replace a dedicated transliteration engine in practice.

What carries the argument

The evaluation machinery is a fixed comparison protocol: a shared test set of roughly 123,856 romanized word pairs across ten languages drawn from Dakshina and the AK-Freq, AK-NEF, and AK-NEI subsets of Aksharantar, scored by Top-1 Accuracy and Character Error Rate, the normalized Levenshtein distance between predicted and reference strings. Against this, each LLM is prompted in ChatML or a simplified instruction format to return a JSON array of native-script words, while IndicXlit, a character-level transformer encoder-decoder trained on Aksharantar, serves as the specialized baseline. The fine-tuning arm retrains GPT-4o through the OpenAI API on 982,381 training pairs in the same ChatML format and compares it on the identical test sets.

What would settle it

A contamination audit that checks overlap between the Dakshina and Aksharantar test words and the pretraining corpora of the GPT models; if most test words appear in training, the reported advantage could disappear on a fresh test set of unseen romanized words compiled after the models' training cutoffs. A simpler check would be to transliterate invented or low-frequency names not present in any public dataset.

Watch

Extended reading notes

Core claim

The central discovery is that transliteration skill is not locked inside task-specialized systems: GPT-4.5 achieves the highest overall Top-1 accuracy among off-the-shelf models, and GPT-4o fine-tuned on transliteration data surpasses IndicXlit in 9 of 10 languages when results are averaged across dataset categories, with particular gains on foreign and Indian named entities. The paper also finds that smaller open-weight models such as Gemma-3-27B-it and Mistral-Large trail both the GPT family and IndicXlit, and that IndicXlit retains an edge on high-frequency native words in some languages such as Telugu. Error analysis attributes IndicXlit's relative weakness to script-level character errors, while GPT-4.5 tends toward graphemic spelling errors that preserve pronunciation.

Load-bearing premise

The comparison assumes the benchmark test words were not already memorized by the closed GPT models during pretraining, since no contamination audit is provided and GPT-4.5's training data is unpublished.

Editorial extensions

If this is right

Off-the-shelf GPT-4.5 can serve as a drop-in transliteration engine for ten major Indian languages without task-specific training.
Fine-tuning GPT-4o on roughly a million pairs yields accuracy above all baselines for most languages and dataset splits, with the largest gains on named entities.
Character-level script errors, not graphemic spelling variants, separate a strong from a weak transliterator; IndicXlit's higher script-error rate explains its defeats.
Under simulated typos and case noise, the fine-tuned GPT-4o remains the most robust model, suggesting practical tolerance to noisy user input.
Open-weight models currently available lag both the closed GPT models and the specialized baseline on this task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the contamination concern is set aside, this result implies that foundation-model vendors can absorb a whole class of specialized NLP tools, and that buyers of transliteration services should weigh API cost and data privacy against the modest fine-tuning overhead.
A natural extension the paper does not run is the reverse direction, native-script to Romanized text, where a specialized model's character-level priors may matter differently; the paper itself flags this as future work.
Because IndicXlit still wins on high-frequency native words in several languages, a hybrid system that routes frequent words to the specialized model and rare or foreign words to an LLM could beat either alone.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a straightforward benchmark paper that mostly delivers what it promises: a systematic comparison of five LLMs plus a fine-tuned GPT-4o against IndicXlit on roughly 123k word pairs across ten Indian languages. The central zero-shot claim, that GPT-4.5 and the GPT family generally beat the specialist on most cells, is supported by Table 3. That part is a useful, if incremental, measurement.

What is new is the coverage: GPT-4.5, GPT-4.1, Gemma-3-27B-it, and Mistral-Large have not been benchmarked this way against IndicXlit before. The paper also does several solid things: it uses public datasets, standard metrics, internally consistent tables, and an error analysis that gives practical insight into graphemic versus script errors.

The soft spots, in order of severity. First, the fine-tuning section (2.5) says the 982,381 training pairs were "drawn from the Dakshina and Aksharantar datasets" but never explicitly states that the evaluation test splits were excluded. The test sets are part of those same released datasets, and the fine-tuning draw is under the authors' control. If test pairs leaked into the fine-tuning set, the reported 9/10 advantage over IndicXlit is memorization, not skill. The stress-test note is on target here. The authors need to state the exact split provenance, ideally release the train/validation/test IDs. Second, there is no contamination audit for LLM pretraining, so the zero-shot differences could also be inflated by memorization of public benchmarks. That is harder to fix but worth acknowledging. Third, no confidence intervals or repeated decoding; LLM outputs are stochastic, and a single run makes the rankings fragile, especially where margins are a few points. Fourth, the "Average" rows in Table 3 are unweighted means over categories of very different sizes (Dakshina ~5-10k, AK-NEF ~800), so they over-represent small categories. Fifth, no code or data are released, and fine-tuning hyperparameters are only partly described.

The paper is a plausible benchmark measurement, not a slam dunk. A serious referee should engage with it, but the fine-tuning split issue has to be resolved before the fine-tuned claim can be believed. Practitioners in Indic NLP will want this comparison; they just shouldn't act on the fine-tuned numbers until the split question is answered. I'd send it to review with a request for clarification and, ideally, released artifacts.

Referee Report

3 major / 6 minor

Summary. This paper benchmarks five general-purpose LLMs (GPT-4o, GPT-4.5, GPT-4.1, Gemma-3-27B-it, Mistral-Large) and a fine-tuned GPT-4o against the specialized IndicXlit model on Romanized-to-native-script transliteration for ten Indian languages. The evaluation uses the Dakshina test set and three Aksharantar subsets (AK-Freq, AK-NEF, AK-NEI), reporting Top-1 accuracy and character error rate (CER). The authors report that GPT-4.5 generally outperforms IndicXlit, that fine-tuned GPT-4o achieves higher average accuracy than IndicXlit in 9 of 10 languages, and they include error analysis and a small noisy-input robustness test. The paper concludes that general-purpose LLMs, especially with fine-tuning, can rival or surpass a dedicated transliteration system.

Significance. If the claimed results are taken at face value, the study offers a useful data point on when general-purpose LLMs can replace a specialized NLP component: it uses public benchmarks, standard metrics, a large aggregate test set (about 124k pairs), and documents its evaluation workflow (Algorithm 1), which are strengths for reproducibility. The main scientific claim—that GPT-4.5 and fine-tuned GPT-4o surpass IndicXlit on most language/category combinations—is, however, threatened by the absence of any contamination audit for LLM pretraining and by an apparently uncontrolled overlap between the fine-tuning data and the test sets. The paper also lacks confidence intervals or repeated decoding, so many of the reported cross-model differences are within plausible noise. The contribution is incremental, but if the leakage concerns are resolved, the benchmark is a solid empirical reference for practitioners.

major comments (3)

[Section 2.5, Table 2, Table 3] The fine-tuning section never states that the evaluation examples were excluded from the training data. The 982,381 training pairs are described as 'drawn from the Dakshina and Aksharantar datasets,' and the test sets in Table 1 (Dakshina test, AK-Freq, AK-NEF, AK-NEI) are subsets of those same released datasets. Because the training draw is roughly 100k pairs per language and the test subsets are publicly bundled with the training partitions, a random or full-dataset draw would very likely place gold-standard test pairs in the fine-tuning set. Under that scenario, the GPT-4o-Fine-Tuned accuracy in Table 3 partly reflects memorization, which would invalidate the headline claim in Section 3.2 that fine-tuned GPT-4o surpasses IndicXlit in 9/10 languages. The authors must provide an explicit hold-out split or verify non-overlap of the fine-tuning data with every test example.
[Section 3.1, Table 3] The zero-shot comparison of GPT-family models against IndicXlit assumes that the public benchmark words were not seen during LLM pretraining. No contamination check is reported for GPT-4.5, GPT-4o, or GPT-4.1, all of which were trained on massive web corpora that very plausibly include Aksharantar or Dakshina. Since the paper's second main claim is that general-purpose LLMs 'surpass IndicXlit for most instances' (Section 3.1), the absence of any contamination audit leaves the central result open to a memorization alternative. The authors should either report a contamination analysis (e.g., exact-match probing on training-subset queries, or temporal cutoffs and data-removal statements) or temper the zero-shot superiority claim accordingly.
[Section 2.6, Table 3, Section 3.4] All accuracy and CER numbers are single-run point estimates with no confidence intervals, significance tests, or repeated decoding. Several comparisons in Table 3 are very close—for example, Kannada average accuracy is 62.84 for GPT-4o-Fine-Tuned versus 62.92 for IndicXlit, and Bengali average CER is identical for GPT-4o and GPT-4.5 at 0.156—so the qualitative ranking 'achieves higher accuracy in 9 out of 10 languages' is not established as a real effect. Similarly, the noisy-input results in Table 4 are based on 100 manually altered words per language without multiple trials or statistical uncertainty, yet Section 3.4 concludes that the fine-tuned model 'consistently achieved the highest transliteration accuracy.' Reporting bootstrap confidence intervals at the cell level or a per-language significance test would make the comparative claims load-bearing and checkable.

minor comments (6)

[Section 2.5] The fine-tuning hyperparameter learning rate multiplier is set to 1.5, but the paper does not state the base learning rate or any other non-default hyperparameters, making the fine-tuning setup hard to reproduce.
[Section 3.2, Table 3] The text says the fine-tuned model 'achieved higher accuracy in 9 out of 10 languages,' but it does not identify Kannada as the single exception; naming the exception would remove ambiguity for the reader.
[Section 4, Conclusion] There are several typos and grammatical errors, including 'rominaized' in the Conclusion, 'langauge' in Section 3.1, 'Gujrati' and 'Telgu' in Figure 5, and the incomplete sentence in the Abstract beginning 'Our findings reveal that while...'.
[Section 2.1.2, Table 1] The text says approximately 123K word pairs total, and Table 1 sums to 123,856, but the paper does not explain that the Urdu row lacks AK-Freq, so the phrase 'across all four datasets' in Figure 5's caption is incorrect for Urdu, which has only three subsets.
[Section 2.4] The prompt illustration hardcodes Hindi in the system message and example, but the same prompt is used for nine other languages; the authors should state explicitly how the language name and example were adapted per language, since prompt wording can affect LLM output.
[Section 2.4] The paper does not state the decoding parameters (temperature, top-p, max tokens) used for the LLM API calls; because transliteration is nearly deterministic at low temperature but variable otherwise, reporting these is necessary for replication.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmark with standard held-out test sets, and the fine-tuning contamination risk is a data-hygiene concern, not a circular reasoning step.

full rationale

This is an empirical benchmark study rather than a derivation chain. The central claims—that GPT-4.5 and fine-tuned GPT-4o outperform IndicXlit on most transliteration benchmarks—are evaluated with Top-1 Accuracy and CER on fixed external test sets (Dakshina test set and Aksharantar AK-Freq, AK-NEF, and AK-NEI subsets) reported in Section 2.1 and Table 1, with results in Table 3. IndicXlit is an external system trained by AI4Bharat, and the LLM comparisons do not fit parameters to the test labels. The only apparent self-citation, Reference [12] (Researchbench, with author E. Cambria), is a background citation and is not load-bearing. The fine-tuning step in Section 2.5 uses 982,381 training and 90,140 validation pairs "drawn from the Dakshina and Aksharantar datasets"; the paper does not explicitly state that the official test subsets were excluded from that draw, so there is a potential data-leakage risk that could affect the fine-tuned GPT-4o results in Table 3. That is a correctness or contamination concern, not a circularity step, because nothing in the paper's equations or citations defines the reported advantage in terms of the training set. The test labels are external, the evaluation protocol is standard, and no self-definitional reduction, fitted parameter renamed as a prediction, or uniqueness argument imported from the authors' prior work is present. Therefore the paper shows no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities or fitted scientific constants. The central claim rests on data-separation assumptions, a hand-chosen fine-tuning learning rate multiplier, an ad hoc noise set, and an unweighted averaging choice. These are the main items a reader should audit before accepting the benchmark conclusion.

free parameters (1)

learning_rate_multiplier_finetune = 1.5
The only non-default hyperparameter in GPT-4o fine-tuning, chosen by hand and not justified by an ablation. It affects the reported fine-tuned performance in Section 2.5.

assumptions (4)

domain assumption Dakshina and Aksharantar test items were not seen during LLM pretraining or fine-tuning.
The benchmark comparison measures generalization only if test items are unseen. No contamination audit is provided, and this assumption is load-bearing for the claim that LLMs outperform IndicXlit.
domain assumption Fine-tuning used only the training and validation splits of Dakshina and Aksharantar, not the test splits.
Section 2.5 states the fine-tuning data was drawn from the datasets but does not explicitly confirm exclusion of test splits. If test items leaked into fine-tuning, the improvement would be inflated.
ad hoc to paper The manually constructed noise set represents realistic noisy Romanized input.
Robustness claims in Section 3.4 rest on 100 hand-picked words per language with two ad hoc noise types, not on a standard or released noise benchmark.
ad hoc to paper The unweighted average of per-category accuracies is a valid summary statistic.
Table 3 ranks models using simple means over Dakshina, AK-Freq, AK-NEF, and AK-NEI, despite category sizes differing by up to a factor of about nine. This can change comparative conclusions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Beyond Specialization: Benchmarking LLMs for Transliteration of Indian Languages." pith.science (2026). https://pith.science/paper/YDDY37II

@misc{pith2026250519851,
  author       = {Pith},
  title        = {Pith review of: Beyond Specialization: Benchmarking LLMs for Transliteration of Indian Languages},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YDDY37II}},
  note         = {Machine review of arXiv:2505.19851}
}

read the original abstract

Transliteration, the process of mapping text from one script to another, plays a crucial role in multilingual natural language processing, especially within linguistically diverse contexts such as India. Despite significant advancements through specialized models like IndicXlit, recent developments in large language models suggest a potential for general-purpose models to excel at this task without explicit task-specific training. The current work systematically evaluates the performance of prominent LLMs, including GPT-4o, GPT-4.5, GPT-4.1, Gemma-3-27B-it, and Mistral-Large against IndicXlit, a state-of-the-art transliteration model, across ten major Indian languages. Experiments utilized standard benchmarks, including Dakshina and Aksharantar datasets, with performance assessed via Top-1 Accuracy and Character Error Rate. Our findings reveal that while GPT family models generally outperform other LLMs and IndicXlit for most instances. Additionally, fine-tuning GPT-4o improves performance on specific languages notably. An extensive error analysis and robustness testing under noisy conditions further elucidate strengths of LLMs compared to specialized models, highlighting the efficacy of foundational models for a wide spectrum of specialized applications with minimal overhead.

Figures

Figures reproduced from arXiv: 2505.19851 by the authors.

**Figure 2.** Fine-tuning workflow using OpenAI’s API for GPT [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Training loss curve (with moving average smooth [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Mean token-level accuracy (with moving average [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: The average Top-1 accuracy of all models across all four datasets [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Bar graph illustrating mean CER values across models for each language. Lower bars indicate higher transliteration [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Distribution of error across different models and languages. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 26 canonical work pages

[1]

Decoding the diversity: A review of the indic ai research landscape,

S. KJ, V. Jain, S. Bhaduri, T. Roy, and A. Chadha, “Decoding the diversity: A review of the indic ai research landscape,” arXiv preprint arXiv:2406.09559, 2024

arXiv 2024
[2]

IndicVoices: Towards building an inclusive mul- tilingual speech dataset for Indian languages,

T. Javed, J. Nawale, E. George, S. Joshi, K. Bhogale, D. Mehen- dale, I. Sethi, A. Ananthanarayanan, H. Faquih, P. Palit, S. Rav- ishankar, S. Sukumaran, T. Panchagnula, S. Murali, K. Gandhi, A. R, M. M, C. Vaijayanthi, K. Karunganni, P. Kumar, and M. Khapra, “IndicVoices: Towards building an inclusive mul- tilingual speech dataset for Indian languages,” ...

work page 2024
[3]

Aksharantar: Open Indic-language transliteration datasets and models for the next billion users,

Y. Madhani, S. Parthan, P. Bedekar, G. Nc, R. Khapra, A. Kunchukuttan, P. Kumar, and M. Khapra, “Aksharantar: Open Indic-language transliteration datasets and models for the next billion users,” in Findings of the Association for Com- putational Linguistics: EMNLP 2023 , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Lingu...

work page 2023
[4]

Processing South Asian languages written in the Latin script: the Dakshina dataset,

B. Roark, L. Wolf-Sonkin, C. Kirov, S. J. Mielke, C. Johny, I. Demirsahin, and K. Hall, “Processing South Asian languages written in the Latin script: the Dakshina dataset,” in Proceed- ings of the Twelfth Language Resources and Evaluation Confer- ence, N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaar...

work page 2020
[5]

Machine transliteration for indian languages: Survey,

P. Kaur and K. D. Garg, “Machine transliteration for indian languages: Survey,” in 2022 Seventh International Conference on Parallel, Distributed and Grid Computing (PDGC) . IEEE, 2022, pp. 462–467

work page 2022
[6]

News 2018 whitepaper,

N. Chen, X. Duan, M. Zhang, R. E. Banchs, and H. Li, “News 2018 whitepaper,” in Proceedings of the Seventh Named Entities Workshop, 2018, pp. 47–54

work page 2018
[7]

Context-aware transliteration of romanized south asian lan- guages,

C. Kirov, C. Johny, A. Katanova, A. Gutkin, and B. Roark, “Context-aware transliteration of romanized south asian lan- guages,” Computational Linguistics, vol. 50, no. 2, pp. 475–534, 2024

work page 2024
[8]

Neural machine transliteration of in- dian languages,

A. Singh and J. Bansal, “Neural machine transliteration of in- dian languages,” in 2021 4th International Conference on Com- puting and Communications Technologies (ICCCT) . IEEE, 2021, pp. 91–96

work page 2021

Show all 34 references

[9]

Indian language transliteration using deep learning,

A. Joshi, K. Mehta, N. Gupta, and V. K. Valloli, “Indian language transliteration using deep learning,” in 2018 IEEE Re- cent Advances in Intelligent Computational Systems (RAICS) . IEEE, 2018, pp. 103–107

2018
[10]

Design challenges in named entity transliteration,

Y. Merhav and S. Ash, “Design challenges in named entity transliteration,” in Proceedings of the 27th International Confer- ence on Computational Linguistics , E. M. Bender, L. Derczyn- ski, and P. Isabelle, Eds. Santa Fe, New Mexico, USA: Associ- ation for Computational Lingu...

2018
[11]

Using large language models to transliterate endangered uralic languages,

N. Partanen, “Using large language models to transliterate endangered uralic languages,” in Proceedings of the 9th In- ternational Workshop on Computational Linguistics for Uralic Languages, 2024, pp. 81–88

2024
[12]

Researchbench: Bench- marking llms in scientific discovery via inspiration-based task decomposition,

Y. Liu, Z. Yang, T. Xie, J. Ni, B. Gao, Y. Li, S. Tang, W. Ouyang, E. Cambria, and D. Zhou, “Researchbench: Bench- marking llms in scientific discovery via inspiration-based task decomposition,” arXiv preprint arXiv:2503.21248 , 2025

2025 arXiv
[13]

mT5: A massively multilingual pre-trained text-to-text transformer,

L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Sid- dhant, A. Barua, and C. Raffel, “mT5: A massively multilingual pre-trained text-to-text transformer,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguist...

2021
[14]

Wikidata: a free collaborative knowledgebase,

D. Vrandečić and M. Krötzsch, “Wikidata: a free collaborative knowledgebase,” Commun. ACM, vol. 57, no. 10, p. 78–85, Sep
[15]

Samanantar: The largest publicly available parallel corpora collection for 11 indic languages,

G. Ramesh, S. Doddapaneni, A. Bheemaraj, M. Jobanputra, R. AK, A. Sharma, S. Sahoo, H. Diddee, M. J, D. Kakwani, N. Kumar, A. Pradeep, S. Nagaraj, K. Deepak, V. Raghavan, A. Kunchukuttan, P. Kumar, and M. S. Khapra, “Samanantar: The largest publicly available parallel corpora ...

2023 arXiv
[16]

IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages,

D. Kakwani, A. Kunchukuttan, S. Golla, G. N.C., A. Bhattacharyya, M. M. Khapra, and P. Kumar, “IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages,” in Findings of the Association for Computational Linguis...

2020
[17]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,”
[18]

Indonlp 2025: Shared task on real-time re- verse transliteration for romanized indo-aryan languages,

D. Sumanathilaka, I. Anuradha, R. Weerasinghe, N. Micallef, and J. Hough, “Indonlp 2025: Shared task on real-time re- verse transliteration for romanized indo-aryan languages,” arXiv preprint arXiv:2501.05816, 2025

2025 arXiv
[19]

Gpt-4o system card,

OpenAI, “Gpt-4o system card,” https://cdn.openai.com/ gpt-4o-system-card.pdf , 2024, accessed: 2025-04-10

2024
[20]

Gpt-4.5 system card,

——, “Gpt-4.5 system card,” https://cdn.openai.com/ gpt-4-5-system-card-2272025.pdf , 2025, accessed: 2025-04- 10

2025
[21]

Gpt-4.5 system card,

——, “Gpt-4.5 system card,” https://openai.com/index/ gpt-4-5-system-card/ , 2025, accessed: 2025-04-10

2025
[22]

Scale | seal leaderboard: Multichallenge,

Scale, “Scale | seal leaderboard: Multichallenge,” accessed: 2025-05-12. [Online]. A vailable: https://scale.com/leaderboard/ multichallenge

2025
[23]

Introducing gpt-4.1 - openai,

OpenAI, “Introducing gpt-4.1 - openai,” accessed: 2025-05-12. [Online]. A vailable: https://openai.com/index/gpt-4-1/

2025
[24]

Gemma 3,

G. Team, “Gemma 3,” 2025, accessed: 2025-04-10. [Online]. A vailable:https://goo.gle/Gemma3Report

2025
[25]

Gemma 3 technical report,

G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière, L. Rouillard, T. Mesnard, G. Cideron, J. bastien Grill, S. Ramos, E. Yvinec, M. Casbon, E. Pot, I. Penchev, G. Liu, F. Visin, K. Kenealy, L. Beyer, X. Zhai, A. T...

2025 arXiv
[26]

Mistral large model overview,

Mistral AI, “Mistral large model overview,” Accessed: April 10, 2025, 2024, https://mistral.ai/news/mistral-large/

2025
[27]

Mistral models on bedrock,

A. W. Services, “Mistral models on bedrock,” Accessed: April 10, 2025, 2024, https://aws.amazon.com/bedrock/mistral/

2025
[28]

Openai chatml,

OpenAI, “Openai chatml,” Accessed: April 12, 2025, 2023, https://github.com/openai/openai-python/blob/release-v0.28. 0/chatml.md

2025
[29]

Lora: Low-rank adaptation of large language models,

E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” 2021. [Online]. A vailable: https://arxiv.org/abs/2106.09685

2021 arXiv
[30]

Preparing your dataset - fine-tuning — openai api documentation,

OpenAI, “Preparing your dataset - fine-tuning — openai api documentation,” accessed: 2025-04-13. [Online]. A vail- able: https://platform.openai.com/docs/guides/fine-tuning/ preparing-your-dataset#preparing-your-dataset

2025
[31]

Fine-tuning — openai api documentation,

——, “Fine-tuning — openai api documentation,” accessed: 2025-04-13. [Online]. A vailable: https://platform.openai.com/ docs/guides/fine-tuning/

2025
[32]

Model - openai api,

——, “Model - openai api,” accessed: 2025-05-13. [Online]. A vailable: https://platform.openai.com/docs/models/gpt-4. 5-preview

2025
[2014]

A vailable: https://doi.org/10.1145/2629489

[Online]. A vailable: https://doi.org/10.1145/2629489

doi
[2023]

A vailable: https://arxiv.org/abs/1706.03762

[Online]. A vailable: https://arxiv.org/abs/1706.03762

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Decoding the diversity: A review of the indic ai research landscape,

S. KJ, V. Jain, S. Bhaduri, T. Roy, and A. Chadha, “Decoding the diversity: A review of the indic ai research landscape,” arXiv preprint arXiv:2406.09559, 2024

arXiv 2024

[2] [2]

IndicVoices: Towards building an inclusive mul- tilingual speech dataset for Indian languages,

T. Javed, J. Nawale, E. George, S. Joshi, K. Bhogale, D. Mehen- dale, I. Sethi, A. Ananthanarayanan, H. Faquih, P. Palit, S. Rav- ishankar, S. Sukumaran, T. Panchagnula, S. Murali, K. Gandhi, A. R, M. M, C. Vaijayanthi, K. Karunganni, P. Kumar, and M. Khapra, “IndicVoices: Towards building an inclusive mul- tilingual speech dataset for Indian languages,” ...

work page 2024

[3] [3]

Aksharantar: Open Indic-language transliteration datasets and models for the next billion users,

Y. Madhani, S. Parthan, P. Bedekar, G. Nc, R. Khapra, A. Kunchukuttan, P. Kumar, and M. Khapra, “Aksharantar: Open Indic-language transliteration datasets and models for the next billion users,” in Findings of the Association for Com- putational Linguistics: EMNLP 2023 , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Lingu...

work page 2023

[4] [4]

Processing South Asian languages written in the Latin script: the Dakshina dataset,

B. Roark, L. Wolf-Sonkin, C. Kirov, S. J. Mielke, C. Johny, I. Demirsahin, and K. Hall, “Processing South Asian languages written in the Latin script: the Dakshina dataset,” in Proceed- ings of the Twelfth Language Resources and Evaluation Confer- ence, N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaar...

work page 2020

[5] [5]

Machine transliteration for indian languages: Survey,

P. Kaur and K. D. Garg, “Machine transliteration for indian languages: Survey,” in 2022 Seventh International Conference on Parallel, Distributed and Grid Computing (PDGC) . IEEE, 2022, pp. 462–467

work page 2022

[6] [6]

News 2018 whitepaper,

N. Chen, X. Duan, M. Zhang, R. E. Banchs, and H. Li, “News 2018 whitepaper,” in Proceedings of the Seventh Named Entities Workshop, 2018, pp. 47–54

work page 2018

[7] [7]

Context-aware transliteration of romanized south asian lan- guages,

C. Kirov, C. Johny, A. Katanova, A. Gutkin, and B. Roark, “Context-aware transliteration of romanized south asian lan- guages,” Computational Linguistics, vol. 50, no. 2, pp. 475–534, 2024

work page 2024

[8] [8]

Neural machine transliteration of in- dian languages,

A. Singh and J. Bansal, “Neural machine transliteration of in- dian languages,” in 2021 4th International Conference on Com- puting and Communications Technologies (ICCCT) . IEEE, 2021, pp. 91–96

work page 2021