REVIEW 4 major objections 5 minor 44 references
Text2Cypher Across Languages: Evaluating and Finetuning LLMs
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Foundational LLMs on Text2Cypher are strongest in English, then Spanish, then Turkish; multilingual finetuning narrows the gap.
desk verdict Useful new multilingual Text2Cypher benchmark and plausible results, but the EN>ES>TR gap and finetuning effects may be partly translation artifacts; the finetuning comparison also has a data-size confound. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a multilingual dataset built by translating only the user questions from English into Spanish and Turkish while keeping the database schema, ground-truth Cypher queries, and metadata fixed, with named entities and quoted strings masked before translation and restored afterward. This design isolates the language of the question as the only variable, because the target query is identical across languages. The second load-bearing component is the finetuning contrast: one model trained on English-only data and one on matched multilingual data, evaluated with both ROUGE-L and execution-based exact match.
What would settle it
Take a random sample of Turkish and Spanish test questions, have native speakers translate them independently, and re-run the same foundational-model evaluation on the human-translated versions; if the English-first ordering or the finetuning-gap pattern disappears, the original results were artifacts of translation quality rather than genuine cross-lingual model capability.
Extended reading notes
Core claim
The paper reports a consistent cross-lingual ordering for Text2Cypher: on the same English-created benchmark, all evaluated instruction-tuned models score highest when the question is English, lower for Spanish, and lowest for Turkish, on both ROUGE-L and execution-based exact match. Translating the task prompt into Spanish or Turkish changes scores only slightly, about 1-1.5 percent on ROUGE-L with no change in exact match. Finetuning an 8-billion-parameter instruction-tuned model on English-only data improves overall accuracy across all three languages but enlarges the gap between English, Spanish, and Turkish. Finetuning on a balanced English-Spanish-Turkish dataset brings scores close together, with ROUGE-L around 0.86 for English, 0.85 for Spanish, and 0.83 for Turkish. The authors interpret this as evidence that language resourcedness and linguistic similarity shape query-generation ability, and that multilingual finetuning is a practical way to rebalance it.
Load-bearing premise
The load-bearing premise is that the automatic translations into Spanish and Turkish, scored around 0.79 to 0.81 by COMET-KIWI-22, preserve each question's exact database intent, so the measured performance differences reflect model capability rather than translation distortion.
Editorial extensions
If this is right
- Non-English speakers querying graph databases through LLMs will, without intervention, receive worse Cypher generations than English speakers, with the deficit growing as the language diverges from English.
- Merely translating the UI prompt into the user's language will not close this gap; the model must be trained on data in the target language.
- Multilingual finetuning can bring the performance of Spanish and Turkish close to English, so balanced training data is a concrete mitigation, not just a desirable property.
- English-only finetuning, while raising headline accuracy, is an equity risk: it can widen the cross-lingual gap even as average scores rise.
- The released English-Spanish-Turkish dataset gives the community a standard benchmark for measuring future models' cross-lingual Text2Cypher performance.
Reading between the lines
- Since the mechanism invoked is training coverage and linguistic similarity rather than Cypher-specific syntax, the same EN-over-ES-over-TR ordering may extend to other schema-driven query generation tasks such as Text2SQL and Text2SPARQL; the paper does not test this extension.
- Because the paper masked named entities and quoted strings before translation, the dataset avoids some of the hardest real-world translation failures, such as inflected or reordered entity mentions; a human-translated sample might expose a larger or different gap.
- The near-zero effect of prompt translation suggests the next bottleneck is the schema itself, which remains in English throughout the experiments; localising schema element names to match the question language is a direct, testable next step.
- A strict test of the finetuning result would hold total finetuning tokens equal between the English-only and multilingual runs and measure execution-based exact match per language, confirming that the ROUGE-based balance is not an artifact of dataset size.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a multilingual Text2Cypher dataset by machine-translating English questions from the existing Text2Cypher dataset into Spanish and Turkish, keeping the same Cypher ground truth. It evaluates three foundational LLMs (Gemma-2-9b-it, Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct) with standardized prompts and reports a consistent performance ordering by input language: English highest, Spanish intermediate, Turkish lowest. It also studies the effect of translating the task prompts into Spanish and Turkish, finding only minor impact, and compares finetuning on an English-only dataset versus a multilingual dataset, concluding that English-only finetuning widens the cross-lingual gap while multilingual finetuning narrows it. The test and training sets are publicly released, and results are reported with both ROUGE-L and execution-based exact-match metrics.
Significance. If the empirical patterns are robust, this paper makes a useful contribution to multilingual Text2Cypher and cross-lingual semantic parsing more broadly: it provides a new publicly available benchmark, identifies a consistent EN>ES>TR gap across multiple models, and offers evidence that multilingual finetuning can rebalance performance across languages. The strengths include the public dataset release, the use of standardized prompts, and the inclusion of both translation-based and execution-based evaluation, which goes beyond many prior studies that use only string-matching metrics. The main concerns are the lack of a task-level validation of the machine-translated test set and the absence of controlled comparisons in the finetuning experiments; these issues affect the central claims and need to be addressed before the results can be fully trusted.
major comments (4)
- [§3.2, Table 1] The test-set construction does not establish that the translated questions preserve the original query semantics, which is load-bearing for every cross-lingual comparison in Figures 2–4. The only reported validation, COMET-KIWI-22, is a general reference-free MT quality estimator (test: 0.8062 ES, 0.7934 TR; train: 0.7999 ES, 0.7849 TR), and the translation prompt instructs the model to preserve placeholders but does not guarantee that numbers, dates, negations, comparators, or relationship-relevant terms survive; no post-hoc check verifies these elements. Because the ground-truth Cypher is the English-derived query, any distortion that changes quantity, polarity, or schema-relevant predicates makes the item unsolvable and systematically penalizes ES and TR. The paper itself excludes 50 samples with masking-related issues, demonstrating that the pipeline is error-prone, yet no human validation or task-level fidelity check (e.g., back-translation, entity/value preservation tests, or human annotation of a sample) is reported. The EN>ES>TR gap and the finetuning effects are therefore confounded with translation error.
- [§3.3, §4.4, Figure 4] The comparison between English-only and multilingual finetuning is not controlled for the amount of training data. The English-only set has 20,512 unique English questions, while the multilingual set has roughly 36,000 instances (about 12,000 per language, with overlaps). Since the multilingual set includes the same English data plus translations, the improved balanced performance in Figure 4 could reflect greater total data volume or more training steps rather than the benefit of multilingual exposure per se. To support the claim that multilingual finetuning narrows the gap, the authors should match the number of training instances or total tokens across conditions, or run an ablation that holds data size constant.
- [§4.1, Figures 2(b), 3(b), 4(b)] Execution-based evaluation is only performed on approximately 50% of the test set (the subset with active database access), but the paper does not report how this subset was selected or show that it is representative of the full test set in terms of query types, difficulty, or language distribution. The Exact-Match scores in the execution-based figures are therefore not necessarily comparable across languages, and the cross-lingual conclusions drawn from execution-based metrics rest on an unverified subset.
- [§4.2, §4.4, Figures 2–4] All results are from single evaluation runs without error bars, confidence intervals, or significance tests. In particular, the claimed improvements of about 0.20 ROUGE-L for English and Spanish versus 0.15 for Turkish, and the characterization of the multilingual model's performance as 'more balanced', are differences of a few points that may be within run-to-run variance. The consistency across three foundational models supports the EN>ES>TR pattern, but the finetuning claims are based on a single model and need variance estimates or repeated runs.
minor comments (5)
- [Footnote 2] The URL for the training set contains a space ('translated text2cypher24 trainset sampled'), which likely breaks the link; please verify and correct it.
- [§3.3] The sentence 'This is a reference-free metric scores in the range [0,1], where higher values reflecting better quality' is ungrammatical; suggest 'This reference-free metric yields scores in [0,1], with higher values indicating better quality.'
- [§3.2] The masking example uses 'QUOTE 0' for the quoted string 'Neo4j', but the surrounding text says named entities are masked with placeholders; clarify that named entities inside quotes are treated as quoted text and that both masking types are used consistently.
- [§4.4] The paper does not specify finetuning hyperparameters (learning rate, number of epochs, LoRA rank, batch size, etc.) or the number of training steps; these details are needed for reproducibility and should be added to the experimental setup.
- [Figure 4] The text reports numeric ROUGE-L values for the multilingual finetuned model (0.86 EN, 0.85 ES, 0.83 TR) but not for the other conditions; consider adding a table with exact scores and standard deviations for all conditions and metrics.
Circularity Check
No significant circularity: the paper reports direct empirical measurements on a multilingual benchmark, with no fitted parameter or self-citation chain doing load-bearing work.
full rationale
The paper's central claims—EN-better-than-ES-better-than-TR performance, minor prompt-translation effects, and finetuning gap dynamics—are direct empirical observations of model outputs on a constructed test set. There is no derivation from first principles that needs to be checked for equivalence to its inputs, and no fitted parameter is repackaged as a prediction. The multilingual test set was created by translating English questions into Spanish and Turkish while preserving the original Cypher queries; the ground-truth queries are external to the behavior being measured, and the evaluation metrics (ROUGE-L, execution-based exact match) are applied to generated versus reference queries. The authors' prior Text2Cypher dataset [4] is used as the base corpus, but this is a data source rather than a load-bearing cited theorem or fitted result, and citing one's own dataset is not circular when the current study measures model outputs on that dataset. The acknowledged limitation that machine translation quality (COMET-KIWI ~0.79–0.81) might introduce task-level fidelity problems is a validity concern, not a circularity concern: even if translation artifacts confound the language ordering, the reported numbers still are not defined in terms of their own conclusions. No step in the paper reduces, by construction or by self-citation, to its own inputs.
Assumptions & free parameters
assumptions (4)
- domain assumption GPT-4o-mini preserves the meaning and database intent of the original English questions when translating to Spanish and Turkish.
- domain assumption COMET-KIWI-22 reference-free scores around 0.79 to 0.81 are an adequate proxy for translation quality.
- domain assumption 4-bit quantized Unsloth checkpoints behave like the original models for cross-lingual query generation.
- domain assumption The subset with active database access is representative of the full test set.
Cite this review
Pith. "Pith review of Text2Cypher Across Languages: Evaluating and Finetuning LLMs." pith.science (2026). https://pith.science/paper/LNISDA6E
@misc{pith2026250621445,
author = {Pith},
title = {Pith review of: Text2Cypher Across Languages: Evaluating and Finetuning LLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/LNISDA6E}},
note = {Machine review of arXiv:2506.21445}
}
read the original abstract
Recent advances in large language models (LLMs) have enabled natural language interfaces that translate user questions into database queries, such as Text2SQL, Text2SPARQL, and Text2Cypher. While these interfaces enhance database accessibility, most research today focuses on English, with limited evaluation in other languages. This paper investigates the performance of both foundational and finetuned LLMs on the Text2Cypher task across multiple languages. We create and release a multilingual dataset by translating English questions into Spanish and Turkish while preserving the original Cypher queries, enabling fair cross-lingual comparison. Using standardized prompts and metrics, we evaluate several foundational models and observe a consistent performance pattern: highest on English, followed by Spanish, and lowest on Turkish. We attribute this to differences in training data availability and linguistic features. We also examine the impact of translating task prompts into Spanish and Turkish. Results show little to no change in evaluation metrics, suggesting prompt translation has minor impact. Furthermore, we finetune a foundational model on two datasets: one in English only, and one multilingual. Finetuning on English improves overall accuracy but widens the performance gap between languages. In contrast, multilingual finetuning narrows the gap, resulting in more balanced performance. Our findings highlight the importance for multilingual evaluation and training to build more inclusive and robust query generation systems.
Reference graph
Works this paper leans on
-
[1]
ACM Computing Surveys (Csur) 54(4), 1–37 (2021)
Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G.D., Gutier- rez, C., Kirrane, S., Gayo, J.E.L., Navigli, R., Neumaier, S., et al.: Knowledge graphs. ACM Computing Surveys (Csur) 54(4), 1–37 (2021)
work page 2021
-
[2]
Emerging Science Journal 8(5), 2003–2022 (2024)
Jannuzzi, M., Perezhohin, Y., Peres, F., Castelli, M., Popoviˇ c, A.: Zero- shot prompting strategies for table question answering with a low-resource language. Emerging Science Journal 8(5), 2003–2022 (2024)
work page 2024
-
[3]
Geng, X., Zhu, M., Li, J., Lai, Z., Zou, W., She, S., Guo, J., Zhao, X., Li, Y., Li, Y., et al.: Why not transform chat large language models to non-english? arXiv preprint arXiv:2405.13923 (2024)
work page Pith review arXiv 2024
-
[4]
In: Proceedings of the Workshop on Generative AI and Knowledge Graphs (GenAIK), pp
Ozsoy, M.G., Messallem, L., Besga, J., Minneci, G.: Text2cypher: Bridging natural language and graph databases. In: Proceedings of the Workshop on Generative AI and Knowledge Graphs (GenAIK), pp. 100–108 (2025)
work page 2025
-
[5]
arXiv preprint arXiv:2306.07377 (2023)
Nicholas, G., Bhatia, A.: Lost in translation: large language models in non-english content analysis. arXiv preprint arXiv:2306.07377 (2023)
arXiv 2023
-
[6]
arXiv preprint arXiv:2406.01771 (2024)
Lai, W., Mesgar, M., Fraser, A.: Llms beyond english: Scaling the mul- tilingual capability of llms with cross-lingual feedback. arXiv preprint arXiv:2406.01771 (2024)
arXiv 2024
-
[7]
arXiv preprint arXiv:2305.18098 (2023)
Yang, W., Li, C., Zhang, J., Zong, C.: Bigtranslate: Augmenting large lan- guage models with multilingual translation capability over 100 languages. arXiv preprint arXiv:2305.18098 (2023)
arXiv 2023
-
[8]
arXiv preprint arXiv:2308.04948 (2023)
Zhu, W., Lv, Y., Dong, Q., Yuan, F., Xu, J., Huang, S., Kong, L., Chen, J., Li, L.: Extrapolating large language models to non-english by aligning languages. arXiv preprint arXiv:2308.04948 (2023)
arXiv 2023
Show all 44 references
-
[9]
arXiv preprint arXiv:2402.07827 (2024)
¨Ust¨ un, A., Aryabumi, V., Yong, Z.-X., Ko, W.-Y., D’souza, D., Onilude, G., Bhandari, N., Singh, S., Ooi, H.-L., Kayid, A., et al.: Aya model: An instruction finetuned open-access multilingual language model. arXiv preprint arXiv:2402.07827 (2024)
2024 arXiv
-
[10]
arXiv preprint arXiv:2312.14862 (2023)
Luo, Y., Kong, Q., Xu, N., Cao, J., Hao, B., Qu, B., Chen, B., Zhu, C., Zhao, C., Zhang, D., et al.: Yayi 2: Multilingual open-source large Text2Cypher Across Languages: Evaluating and Finetuning LLMs 13 language models. arXiv preprint arXiv:2312.14862 (2023)
2023 arXiv
-
[11]
arXiv preprint arXiv:2305.15011 (2023)
Li, H., Koto, F., Wu, M., Aji, A.F., Baldwin, T.: Bactrian-x: Multilingual replicable instruction-following models with low-rank adaptation. arXiv preprint arXiv:2305.15011 (2023)
2023 arXiv
-
[12]
arXiv preprint arXiv:2307.16039 (2023)
Lai, V.D., Van Nguyen, C., Ngo, N.T., Nguyen, T., Dernoncourt, F., Rossi, R.A., Nguyen, T.H.: Okapi: Instruction-tuned large language models in multiple languages with reinforcement learning from human feedback. arXiv preprint arXiv:2307.16039 (2023)
2023 arXiv
-
[13]
arXiv preprint arXiv:2305.07004 (2023)
Huang, H., Tang, T., Zhang, D., Zhao, W.X., Song, T., Xia, Y., Wei, F.: Not all languages are created equal in llms: Improving mul- tilingual capability by cross-lingual-thought prompting. arXiv preprint arXiv:2305.07004 (2023)
2023 arXiv
-
[14]
Etxaniz, J., Azkune, G., Soroa, A., de Lacalle, O.L., Artetxe, M.: Do multilingual language models think better in english? arXiv preprint arXiv:2308.01223 (2023)
2023 arXiv
-
[15]
arXiv preprint arXiv:2401.01055 (2024)
Zhao, J., Zhang, Z., Gao, L., Zhang, Q., Gui, T., Huang, X.: Llama beyond english: An empirical study on language capability transfer. arXiv preprint arXiv:2401.01055 (2024)
2024 arXiv
-
[16]
arXiv preprint arXiv:2410.05873 (2024)
Kargaran, A.H., Modarressi, A., Nikeghbal, N., Diesner, J., Yvon, F., Sch¨ utze, H.: Mexa: Multilingual evaluation of english-centric llms via cross-lingual alignment. arXiv preprint arXiv:2410.05873 (2024)
2024 arXiv
-
[17]
Zhong, C., Cheng, F., Liu, Q., Jiang, J., Wan, Z., Chu, C., Murawaki, Y., Kurohashi, S.: Beyond english-centric llms: What language do mul- tilingual language models think in? arXiv preprint arXiv:2408.10811 (2024)
2024 arXiv
-
[18]
Schut, L., Gal, Y., Farquhar, S.: Do multilingual llms think in english? arXiv preprint arXiv:2502.15603 (2025)
2025 arXiv
-
[19]
arXiv preprint arXiv:2410.01335 (2024)
Bandarkar, L., Muller, B., Yuvraj, P., Hou, R., Singhal, N., Lv, H., Liu, B.: Layer swapping for zero-shot cross-lingual transfer in large language models. arXiv preprint arXiv:2410.01335 (2024)
2024 arXiv
-
[20]
arXiv preprint arXiv:2409.07604 (2024)
Ozsoy, M.G.: Multilingual prompts in llm-based recommenders: Perfor- mance across languages. arXiv preprint arXiv:2409.07604 (2024)
2024 arXiv
-
[21]
In: Proceedings of the AAAI Conference on Artificial Intelligence, vol
Dou, L., Gao, Y., Pan, M., Wang, D., Che, W., Zhan, D., Lou, J.-G.: Mul- tispider: towards benchmarking multilingual text-to-sql semantic parsing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 12745–12753 (2023) 14 Text2Cypher Across Languages...
2023
-
[22]
In: Intelligent Systems: 10th Brazilian Conference, BRACIS 2021, Virtual Event, November 29–December 3, 2021, Proceedings, Part II 10, pp
Jos´ e, M.A., Cozman, F.G.: mrat-sql+ gap: a portuguese text-to-sql trans- former. In: Intelligent Systems: 10th Brazilian Conference, BRACIS 2021, Virtual Event, November 29–December 3, 2021, Proceedings, Part II 10, pp. 511–525 (2021). Springer
2021
-
[23]
arXiv preprint arXiv:2505.23838 (2025)
Huang, Y., Guo, J., Mao, W., Gao, C., Han, P., Liu, C., Ling, Q.: Explor- ing the landscape of text-to-sql with large language models: Progresses, challenges and opportunities. arXiv preprint arXiv:2505.23838 (2025)
2025 arXiv
-
[24]
arXiv preprint arXiv:1809.08887 (2018)
Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al.: Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887 (2018)
2018 arXiv
-
[25]
arXiv preprint arXiv:1909.13293 (2019)
Min, Q., Shi, Y., Zhang, Y.: A pilot study for chinese sql semantic parsing. arXiv preprint arXiv:1909.13293 (2019)
2019 arXiv
-
[26]
IEEE Access (2024)
Kanburoglu, A.B., Tek, F.B.: Turspider: A turkish text-to-sql dataset and llm-based study. IEEE Access (2024)
2024
-
[27]
In: Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing, pp
Almohaimeed, S., Almohaimeed, S., Al Ghanim, M., Wang, L.: Ar- spider: Text-to-sql in arabic. In: Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing, pp. 1024–1030 (2024)
2024
-
[28]
swiss: bilingual open data exploration in natural language
Nooralahzadeh, F., Zhang, Y., Smith, E., Maennel, S., Matthey-Doret, C., de Fondville, R., Stockinger, K.: Statbot. swiss: bilingual open data exploration in natural language. arXiv preprint arXiv:2406.03170 (2024)
2024 arXiv
-
[29]
In: Simp´ osio Brasileiro de Sistemas de Informa¸ c˜ ao (SBSI), pp
Pedroso, B.C., Pereira, M.R., Pereira, D.A.: Performance evaluation of llms in the text-to-sql task in portuguese. In: Simp´ osio Brasileiro de Sistemas de Informa¸ c˜ ao (SBSI), pp. 260–269 (2025). SBC
2025
-
[30]
In: Findings of the Association for Computational Linguistics: EMNLP 2022, pp
Bakshandaeva, D., Somov, O., Dmitrieva, E., Davydova, V., Tutubalina, E.: Pauq: Text-to-sql in russian. In: Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 2355–2376 (2022)
2022
-
[31]
Transactions of the Association for Computational Linguistics (2022)
Cui, R., Aralikatte, R., Lent, H., Hershcovich, D.: Compositional gener- alization in multilingual semantic parsing over wikidata. Transactions of the Association for Computational Linguistics (2022)
2022
-
[32]
arXiv preprint arXiv:2407.06041 (2024)
Srivastava, N., Ma, M., Vollmers, D., Zahera, H., Moussallem, D., Ngomo, A.-C.N.: Mst5–multilingual question answering over knowledge graphs. arXiv preprint arXiv:2407.06041 (2024)
2024 arXiv
-
[33]
Semantic Web 15(5), 2089–2124 (2024) Text2Cypher Across Languages: Evaluating and Finetuning LLMs 15
Perevalov, A., Both, A., Ngonga Ngomo, A.-C.: Multilingual question answering systems for knowledge graphs–a survey. Semantic Web 15(5), 2089–2124 (2024) Text2Cypher Across Languages: Evaluating and Finetuning LLMs 15
2024
-
[34]
https://text2sparql.aksw.org/
TEXT2SPARQL Challenge 2025. https://text2sparql.aksw.org/. Accessed: 2025-07-05 (2025)
2025
-
[35]
Text2KG (2024)
Perevalov, A., Both, A.: Towards LLM-driven Natural Language Gener- ation based on SPARQL Queries and RDF Knowledge Graphs. Text2KG (2024)
2024
-
[36]
JAIT 15(10), 1157–1162 (2024)
Lan, M., Xia, Y., Zhou, G., Huang, N., Li, Z., Wu, H.: Llm4qa: Leveraging large language model for efficient knowledge graph reasoning with sparql query. JAIT 15(10), 1157–1162 (2024)
2024
-
[37]
arXiv preprint arXiv:2004.09095 (2020)
Joshi, P., Santy, S., Budhiraja, A., Bali, K., Choudhury, M.: The state and fate of linguistic diversity and inclusion in the nlp world. arXiv preprint arXiv:2004.09095 (2020)
2020 arXiv
-
[38]
arXiv preprint arXiv:2109.10534 (2021)
Dhamecha, T.I., Murthy V, R., Bharadwaj, S., Sankaranarayanan, K., Bhattacharyya, P.: Role of language relatedness in multilingual fine- tuning of language models: A case study in indo-aryan languages. arXiv preprint arXiv:2109.10534 (2021)
2021 arXiv
-
[39]
arXiv preprint arXiv:2501.00593 (2024)
Bayram, M.A., Fincan, A.A., G¨ um¨ u¸ s, A.S., Diri, B., Yıldırım, S., Ayta¸ s, ¨O.: Setting standards in turkish nlp: Tr-mmlu for large language model evaluation. arXiv preprint arXiv:2501.00593 (2024)
2024 arXiv
-
[40]
In: Proceedings of the 2022 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies: System Demonstrations, pp
Alecakir, H., B¨ ol¨ uc¨ u, N., Can, B.: Turkishdelightnlp: A neural turkish nlp toolkit. In: Proceedings of the 2022 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies: System Demonstrations, pp. 17–26 (2022)
2022
-
[41]
Natural and Engineering Sciences 5(3), 204–211 (2020)
Tohma, K., Kutlu, Y.: Challenges encountered in turkish natural lan- guage processing studies. Natural and Engineering Sciences 5(3), 204–211 (2020)
2020
-
[42]
In: Proceedings of the 31st International Conference on Computational Linguistics, pp
Baucells, I., Aula-Blasco, J., de-Dios-Flores, I., Su´ arez, S.P., Perez, N., Salles, A., Docio, S.S., Falc˜ ao, J., Saiz, J.J., Sep´ ulveda-Torres, R., et al.: Iberobench: A benchmark for llm evaluation in iberian languages. In: Proceedings of the 31st International Conferenc...
2025
-
[43]
arXiv preprint arXiv:2209.06243 (2022)
Rei, R., Treviso, M., Guerreiro, N.M., Zerva, C., Farinha, A.C., Maroti, C., De Souza, J.G., Glushkova, T., Alves, D.M., Lavie, A., et al.: Cometkiwi: Ist-unbabel 2022 submission for the quality estimation shared task. arXiv preprint arXiv:2209.06243 (2022)
2022 arXiv
-
[44]
https://huggingface.co/ evaluate-metric (2024)
HuggingFace: HuggingFace Evaluate. https://huggingface.co/ evaluate-metric (2024)
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.