Pith. sign in

REVIEW 4 major objections 5 minor 46 references

This paper claims that a compact, Bangla-only code model family—TigerCoder—achieves the best reported Pass@1 scores on Bangla code benchmarks across five programming languages, beating models up to 27 times its size.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 19:40 UTC pith:HIML4IUW

load-bearing objection Real resource contribution with a plausible core result, but the headline numbers are softer than they look once you notice model selection on the test set and the under-specified multi-language test porting. the 4 major comments →

arxiv 2509.09101 v1 pith:HIML4IUW submitted 2025-09-11 cs.CL

TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla

Nishat Raihan , Antonios Anastasopoulos , Marcos Zampieri This is my paper
classification cs.CL
keywords Bangla code generationlow-resource language modelsinstruction tuningPass@1MBPP-Banglamultilingual code evaluationmachine translation for codeTigerCoder
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

TigerCoder is the first family of code-generation LLMs built specifically for Bangla, in two sizes (1B and 9B). The paper's central claim is that fine-tuning a strong Bangla base model on 300K curated Bangla instruction–code pairs yields large Pass@1 gains—about 11–18 percentage points over prior multilingual and Bangla-specific systems—and that the 1B model already beats models 27 times larger. To support this, the authors construct MBPP-Bangla, a 974-problem benchmark with human-translated prompts and reference solutions in Python, Java, C++, JavaScript, and Ruby. They also show that machine-translating Bangla prompts to English does not recover the performance lost from Bangla prompting. If correct, this is evidence that high-quality targeted data can outweigh model scale for low-resource code generation.

Core claim

On the paper's own terms, the discovery is that a small open model, fine-tuned on a purpose-built Bangla instruction corpus, can outperform both proprietary multilingual systems and all existing Bangla LLMs on Bangla code generation. The strongest result is TigerCoder-9B reaching 0.82 Pass@1 on MBPP-Bangla and 0.75 on mHumanEval-Bangla, with similar leads in C++, Java, JavaScript, and Ruby. The authors interpret this as demonstrating that curated, executable, language-specific instruction data is the decisive factor, challenging the assumption that scale alone drives code-generation quality.

What carries the argument

The operative mechanism is three 100K-pair instruction datasets—self-instructed, synthetic, and machine-translated—each filtered for syntax and execution validity, used to fine-tune the TigerLLM base. The evaluation anchor is MBPP-Bangla, a 974-task benchmark with independently human-translated Bangla prompts and expert-verified reference solutions in five languages, scored with Pass@K.

Load-bearing premise

The whole evaluation stands on the assumption that MBPP-Bangla's translated prompts and its ported Java, C++, JavaScript, and Ruby solutions faithfully preserve the original Python tasks and tests; if those ported tests do not truly match the Python semantics, every Pass@K number loses its meaning.

What would settle it

Inspect the released MBPP-Bangla records: if the non-Python reference solutions are paired with Python-specific test functions that were never semantically ported to each target language, or if re-running the ported solutions against the original MBPP Python test assertions fails for reasons other than trivial syntax, the multi-language Pass@K results are not measuring what they claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A 1B model that surpasses systems 27 times its size suggests practical, low-cost paths to code assistants for other low-resource languages.
  • Machine translation of prompts is not a substitute for native-language instruction data in code generation; future MT pipelines must preserve code-specific keywords.
  • The combined SI+Syn+TE dataset recipe—diverse instruction sources with execution validation—appears to be the configuration yielding the largest gains.
  • MBPP-Bangla extends code-generation evaluation beyond Python, allowing per-language analysis in five PLs.
  • The reported 9B leadership makes smaller open checkpoints a viable alternative to proprietary APIs for Bangla coding tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the data-quality thesis would be to train the same TigerLLM base on the same 300K pairs but with the Bangla instructions replaced by English translations; if Pass@1 drops, the benefit is linguistic, not merely instructional.
  • If the multi-language reference solutions are truly equivalent, the same benchmark could be used to study cross-lingual transfer and to expose whether models generate idiomatic code or merely token-mimicry.
  • The benchmark's portability hinges on the unstated test-conversion protocol; an explicit release of language-specific test harnesses would make the 0.82 number independently checkable.
  • The same curation template—native-speaker translation, multilingual expert verification, execution-validated instruction data—likely transfers to other under-resourced languages, with human verification costs as the main bottleneck.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper addresses Bangla code generation, a low-resource setting, and makes three main contributions: (1) MBPP-Bangla, a 974-problem benchmark with Bangla prompts and reference solutions in Python, Java, JavaScript, Ruby, and C++; (2) three 100K-pair Bangla code-instruction datasets (self-instruct, synthetic, and machine-translated); and (3) TigerCoder 1B and 9B models obtained by fine-tuning TigerLLM on combinations of those datasets. The authors also investigate two research questions: whether multilingual LLMs lose code-generation quality when prompted in Bangla instead of English (RQ1), and whether a simple Bangla-to-English machine-translation step helps (RQ2). Evaluation on mHumanEval-Bangla and MBPP-Bangla reports TigerCoder-9B reaching 0.82 Pass@1 on MBPP-Bangla and claims 11-18% gains over the strongest prior baseline, with similar cross-language gains in Appendix E. All models, datasets, and benchmarks are promised to be open-sourced.

Significance. If the results hold, this is a useful contribution to low-resource code generation: it provides a dedicated Bangla code benchmark, instruction-tuning datasets, and openly described models, and the comparison against external baselines (Gemma, GPT, Gemini, TigerLLM) is a strength. The paper also includes a sensible two-RQ design separating language comprehension from code-generation ability. However, the headline numerical claims currently rest on two procedures that are not yet adequately established: the conversion of Python test cases to four other programming languages, and the use of the evaluation benchmarks to select the final dataset combination and hyperparameters. Because both procedures are load-bearing for the abstract claim of 11-18% gains across all five programming languages, the reported numbers should be treated as unverified until the authors supply a proper test-conversion protocol and held-out evaluation.

major comments (4)
  1. [Appendix D, Step 4; Section 3] The five-language Pass@K results in Appendix E, and the abstract's 'across all 5 PLs' claim, rest entirely on the undocumented conversion of MBPP's Python test cases. Step 4 says the verifier patches ports 'until all variants satisfy the original tests,' but the paper never specifies how Python-specific tests/assertions are converted to Java, C++, JavaScript, and Ruby, whether the same edge cases are covered, or how equivalence is established. Without a concrete conversion protocol, examples, and a language-agnostic test harness, the non-Python numbers (Tables 11-14) may be measuring weaker or I/O-only behavior, making the reported 0.82 Pass@1 and the cross-PL deltas non-comparable. Please provide the full protocol, release the harness, and ideally include an independent re-run of a sample of ported tests.
  2. [Section 7, Figure 2, Table 6] The model-selection procedure appears to use the evaluation benchmarks as the validation set. Figure 2 chooses the dataset combination based on mHumanEval and MBPP performance, and Table 6 reports 'empirically selected hyperparameters'; Table 7 then reports final scores on the same two benchmarks. This is a form of test-set fitting, so the reported 11-18% gains are optimistic and do not estimate generalization to unseen Bangla coding problems. Please hold out a subset of MBPP-Bangla and mHumanEval-Bangla for final evaluation, or at minimum report all configurations explored and clearly label Figure 2 as validation-only with independent test results.
  3. [Section 4, Pass@K metric; Tables 3 and 7] No sampling details accompany any Pass@K estimate. The paper defines the formula but does not report n (number of sampled programs), temperature, maximum tokens, decoding scheme, random seeds, or repetitions. The metric is a point estimate with nontrivial variance at moderate n, and several headline deltas are only 4-5 points (e.g., Table 7: TigerCoder-1B 0.69 vs TigerLLM-1B 0.63/0.65 on mHumanEval; Table 3: TigerLLM-9B 0.61 vs Gemma-3 0.69 on MBPP-Bangla). Without variance estimates or confidence intervals, the word 'significant' in the abstract is unsupported. Please report n, temperature, standard errors or bootstrap confidence intervals, and reproducibility runs.
  4. [Section 6; Section 3] No decontamination step is described between the 300K instruction pairs and the two evaluation benchmarks. MBPP-Bangla is derived from the public MBPP, and the instruction datasets include GPT-4o/Claude self-instruct generations and translated Evol-Instruct data; such corpora often contain HumanEval/MBPP-style problems. If any training prompt overlaps with an evaluation task, the corresponding Pass@1 is inflated. Please conduct and report n-gram overlap and embedding-similarity analyses between Bangla-Code-Instruct and mHumanEval-Bangla/MBPP-Bangla, and describe any filtering applied.
minor comments (5)
  1. [Throughout] Please fix typos and inconsistent notation: 'comapred' (Section 8), inconsistent use of 'JAVA' vs 'Java', 'Bangla-Instruct-Syn' vs 'Bangla-Code-Instruct-Syn', and 'mHumanEval-ben' vs 'mHumanEval-Bangla'.
  2. [Appendix C, Appendix D] The NLLB entry is cited as 'NLLB ()' with no reference. TransCoder-ST (Appendix D, Step 4) is used without a citation. CometKiwi-22 QE is referenced to Rei et al. (2020), but that paper describes COMET, not the Kiwi-22 checkpoint; please update the references.
  3. [Table 1] The columns 'Paper?' and 'Reprod.' use 'Y/X' without a legend. Clarify what each symbol means, and consider adding a column for code-generation task support.
  4. [Table 6] Hyperparameter selection reports 'Save Steps Varies' and provides no search range or early-stopping details. Include the ranges searched and the criterion used for selecting final hyperparameters so the tuning process is reproducible.
  5. [Limitations] The Limitations paragraph calls MBPP-Bangla 'Python-focused' although the benchmark includes five programming languages and the paper claims cross-language evaluation. Rephrase to avoid contradicting the benchmark description in Section 3.

Circularity Check

0 steps flagged

No significant circularity: the central Pass@1 improvements are empirical measurements against external and newly constructed benchmarks, not derivations from fitted inputs.

full rationale

The paper's main claims (RQ1/RQ2 findings and TigerCoder Pass@1 gains) are direct experimental measurements. The model is fine-tuned from the authors' own TigerLLM base and evaluated on mHumanEval-Bangla and newly constructed MBPP-Bangla, but the reported improvements are relative to external baselines (GPT, Gemma, Phi, etc.) on the same benchmarks, so the result is not equivalent to a fitted parameter by construction. Selection of TigerLLM as base is justified by in-paper Table 3/4 results, not solely by citation. No uniqueness theorem or ansatz is imported from prior self-citations to force the outcome. Minor self-referential elements exist (TigerLLM lineage, mHumanEval-Bangla by the same first author), but they are not load-bearing: the evaluation would be meaningful even if those were replaced by external benchmarks. The Limitations section explicitly calls MBPP-Bangla 'Python-focused,' which raises a correctness/validity concern for the 5-PL claim, but not a circularity issue. The only notable methodological caveat is that the best dataset combination was selected using the same evaluation benchmarks (Figure 2/Section 7), which is selection bias rather than circular-by-construction; the reported 0.82 is the maximum over configurations, but this does not make the derivation equivalent to its inputs.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The central performance claim depends on a chain of curated data and hand-selected thresholds: base model choice, filtering cutoffs, training hyperparameters, benchmark translation quality, and cross-language test porting. None of these are derived from theory; they are empirical choices whose values materially affect the reported Pass@1 numbers.

free parameters (6)
  • learning_rate (1B/9B) = 1e-5 / 1e-6
    Reported in Table 6 as 'empirically selected'; affects fine-tuning and final performance.
  • number_of_epochs = 3
    Table 6; hand-chosen training duration.
  • SI instruction similarity threshold = cosine >= 0.95 discarded
    Appendix A.4; determines which self-instructed prompts enter the 100K set.
  • Syn BERTScore threshold = discard if BERTScore-F1 >= 0.7
    Appendix B.2; controls redundancy in synthetic set.
  • TE selection thresholds = Comet QE > 0.85 and BERTScore F1 > 0.95
    Appendix C.2; filters machine-translated instructions.
  • TE word-count filter = > 10 words
    Appendix C.1; pre-filter on English source instructions.
axioms (6)
  • standard math Pass@K formula is the correct evaluation metric for code generation
    Used in Section 4; standard metric from prior work.
  • domain assumption TigerLLM is a suitable base model for Bangla code generation
    Section 7 selects TigerLLM based on the authors' own Table 3 results; this is a premise, not a derived fact.
  • domain assumption Human translations in MBPP-Bangla preserve task semantics
    Section 3 Steps 2-4 describe two translators and one verifier, but no inter-annotator agreement or back-translation check is reported.
  • domain assumption Ported reference solutions for Java, C++, JS, Ruby pass the original MBPP tests
    Appendix D Step 4 asserts this, but the test porting procedure is not described, so the cross-language correctness claim is unverifiable from the text.
  • domain assumption Executable generated code is a valid training signal
    Sections 6.1-6.2 validate code by syntax and execution, not by hidden test correctness; semantically wrong code can still be executable.
  • domain assumption NLLB translations plus QE thresholds yield acceptable Bangla instructions in TE
    Section 6.3 relies on CometKiwi-22 and BERTScore cutoffs; no human evaluation of the translated instructions is reported.

pith-pipeline@v1.3.0-alltime-deepseek · 16739 in / 14152 out tokens · 133493 ms · 2026-08-04T19:40:18.544602+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla." pith.science (2026). https://pith.science/paper/HIML4IUW

@misc{pith2026250909101,
  author       = {Pith},
  title        = {Pith review of: TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HIML4IUW}},
  note         = {Machine review of arXiv:2509.09101}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Despite being the 5th most spoken language, Bangla remains underrepresented in Large Language Models (LLMs), particularly for code generation. This primarily stems from the scarcity of high-quality data to pre-train and/or finetune such models. Hence, we introduce the first dedicated family of Code LLMs for Bangla (1B & 9B). We offer three major contributions: (1) a comprehensive Bangla code instruction datasets for programming domain adaptation; (2) MBPP-Bangla, an evaluation benchmark for Bangla code generation; and (3) the TigerCoder-family of Code LLMs, achieving significant ~11-18% performance gains at Pass@1 over existing multilingual and general-purpose Bangla LLMs. Our findings show that curated, high-quality datasets can overcome limitations of smaller models for low-resource languages. We open-source all resources to advance further Bangla LLM research.

Figures

Figures reproduced from arXiv: 2509.09101 by Antonios Anastasopoulos, Marcos Zampieri, Nishat Raihan.

Figure 1
Figure 1. Figure 1: Incorrect keywords generated by machine [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance (Pass@1) comparison for different combinations of the [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 19 linked inside Pith

  1. [1]

    Kabir Ahuja, Anirudh Das, Sandipan Das, Ashwini Deshpande, Sebastian Gehrmann, Anup Gopinath, Arya Guha, Pooja Kumar-Jois, Prem Mani, Ashwin Paranjape, et al. 2023. Mega: Multilingual evaluation of generative ai. arXiv preprint arXiv:2310.10567

    Pith/arXiv arXiv 2023

  2. [2]

    Kabir Ahuja, Karan Sikka, Madhusudana Nallasamy, and Chandrika Singh. 2024. Few-shot learning for low-resource languages with large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics

    2024

  3. [3]

    Anthropic. 2023. Claude: The anthropic ai language model. Online documentation. Available at: https://www.anthropic.com

    2023

  4. [4]

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. In arXiv preprint arXiv:2108.07732

    Pith/arXiv arXiv 2021

  5. [5]

    Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M Sohel Rahman, and Rifat Shahriyar. 2022. Banglabert: Language model pretraining and benchmarks for low-resource language understanding evaluation in bangla. In Findings of the Association for Computational Linguistics: NAACL 2022

    2022

  6. [6]

    Pramit Bhattacharyya, Joydeep Mondal, Subhadip Maji, and Arnab Bhattacharya. 2023. Vacaspati : A diverse corpus of bangla literature. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics

    2023

  7. [7]

    Dami \'a n Blasi, Antonios Anastasopoulos, and Graham Neubig. 2022. Systematic inequalities in language technology performance across the world’s languages. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    2022

  8. [8]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. In arXiv preprint arXiv:2107.03374

    Pith/arXiv arXiv 2021

  9. [9]

    2008--2025

    Common Crawl Foundation . 2008--2025. Common crawl. https://commoncrawl.org

    2008

  10. [10]

    Marta R Costa-Juss \`a , James Cross, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    Pith/arXiv arXiv 2022

  11. [11]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Adam Afflerbach, Adriane Abramowitz, Aditya Singhal, Aditya Balaji, Akshat Agrawal, Akshat Ravinuthala, Akshay Ramakrishnan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    Pith/arXiv arXiv 2024

  12. [12]

    Syed Mohammed Sartaj Ekram, Adham Arik Rahman, Md Sajid Altaf, Mohammed Saidul Islam, Tareq Mahmood Jamil, Shadman Sakib Alam, Irfan Kabir, Mohammad Nasim, Enamul Hossain, and Nawshad Akhter. 2022. Banglarqa: A benchmark dataset for under-resourced bangla language reading comprehension-based question answering with diverse question-answer types. In Findin...

    2022

  13. [14]

    Google. 2024. Gemini 2.5: The next generation of multimodal ai models

    2024

  14. [15]

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, et al. 2022. Training compute-optimal large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems

    2022

  15. [16]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186

    Pith/arXiv arXiv 2024

  16. [17]

    Sai Iyer, Jiawei Baek, Daisy Li, Zoekook, and Spencer Weiss. 2022. Optimizing tokenization for low-resource languages in machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics

    2022

  17. [18]

    Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the nlp world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

    2020

  18. [19]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

    Pith/arXiv arXiv 2020

  19. [20]

    Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ali Payani, Ninghao Liu, and Mengnan Du. 2024. Quantifying multilingual performance of large language models across languages. arXiv preprint arXiv:2404.11553

    Pith/arXiv arXiv 2024

  20. [21]

    Microsoft. 2024. https://arxiv.org/abs/2406.18243 Phi-4: A 14b-parameter model with instruct-following and multi-modal capabilities . Preprint, arXiv:2406.18243

    Pith/arXiv arXiv 2024

  21. [22]

    Niklas Muennighoff, Alex Wang, Alena Fenogenova, Fangyu Huang, Francesca Toni, Adina Williams, and Colin Wang. 2023. Cross-lingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics

    2023

  22. [23]

    OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    Pith/arXiv arXiv 2023

  23. [24]

    OpenAI. 2024. https://arxiv.org/abs/2405.07133 Gpt-4o system card . Preprint, arXiv:2405.07133

    Pith/arXiv arXiv 2024

  24. [25]

    Liliana Pasquale, Antonino Sabetta, Marcelo d’Amorim, P \'e ter Heged u s, Mehdi Tarrit Mirakhorli, Hamed Okhravi, Mathias Payer, Awais Rashid, Joanna CS Santos, Jonathan M Spring, et al. 2025. Challenges to using large language models in code generation and repair. IEEE Security & Privacy, 23(2):81--88

    2025

  25. [26]

    Nishat Raihan, Antonios Anastasopoulos, and Marcos Zampieri. 2025 a . m H uman E val - a multilingual benchmark to evaluate large language models for code generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)

    2025

  26. [27]

    Nishat Raihan, Mohammed Latif Siddiq, Joanna CS Santos, and Marcos Zampieri. 2025 b . Large language models in computer science education: A systematic literature review. In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1, pages 938--944

    2025

  27. [28]

    Nishat Raihan and Marcos Zampieri. 2025. TigerLLM - a family of bangla large language models. In Proceedings of ACL

    2025

  28. [29]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685--2702

    2020

  29. [30]

    Pritika Rohera, Chaitrali Ginimav, Gayatri Sawant, and Raviraj Joshi. 2025. Better to ask in english? evaluating factual accuracy of multilingual llms in english and low-resource languages. arXiv preprint arXiv:2504.20022

    arXiv 2025

  30. [31]

    Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Iliç, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100

    Pith/arXiv arXiv 2022

  31. [32]

    Lisa Schut, Yarin Gal, and Sebastian Farquhar. 2025. Do multilingual llms think in english? arXiv preprint arXiv:2502.15603

    Pith/arXiv arXiv 2025

  32. [33]

    Sheikh Shafayat, H Hasan, Minhajur Mahim, Rifki Putri, James Thorne, and Alice Oh. 2024. BE n QA : A question answering benchmark for B engali and E nglish. In Findings of the Association for Computational Linguistics: ACL 2024

    2024

  33. [34]

    Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet : Masked and permuted pre-training for language understanding. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020)

    2020

  34. [35]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram \'e , Morgane Rivi \`e re, et al. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786

    Pith/arXiv arXiv 2025

  35. [36]

    Christopher Toukmaji and Jeffrey Flanigan. 2025. Prompt, translate, fine-tune, re-initialize, or instruction-tune? adapting llms for in-context learning in low-resource languages. In Proceedings of the ACL GEM Workshop. ArXiv:2506.19187

    Pith/arXiv arXiv 2025

  36. [37]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    Pith/arXiv arXiv 2023

  37. [38]

    Md Nafis Uddin, Masum Khan, Nabila Hasan, and Mahmudul Hossain. 2023. Exploring code-mixed bangla text in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  38. [39]

    Tianyi Wang, Yang Ye, Panupong Pasupat, Aohan Wan, Grant Friedman, Jiacheng Tu, Maya Schaar, Jason Wei, Suriya Gunasekar, Matthew Richardson, et al. 2023 a . Babelcode: Llm as a polyglot programmer. arXiv preprint arXiv:2303.03845

    Pith/arXiv arXiv 2023

  39. [40]

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, et al. 2023 b . Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics

    2023

  40. [41]

    Yuhan Wang, Xuanhe Zhou, Ruoxi Chen, et al. 2023 c . https://arxiv.org/abs/2303.10420 A comprehensive capability analysis of gpt-3 and gpt-3.5 series models . Preprint, arXiv:2303.10420

    Pith/arXiv arXiv 2023

  41. [42]

    Canwen Xu, Ruqing Wang, Yeyun Gong, et al. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244

    Pith/arXiv arXiv 2023

  42. [43]

    Xiang Yue, Yueqi Song, Akari Asai, et al. 2024. Pangea: A fully open multilingual multimodal llm for 39 languages. In The Thirteenth International Conference on Learning Representations

    2024

  43. [44]

    Abdullah Khan Zehady, Safi Al Mamun, Naymul Islam, and Santu Karmaker. 2024. Bongllama: Llama for bangla language. arXiv preprint arXiv:2410.21200

    arXiv 2024

  44. [45]

    Bertscore: Evaluating text generation with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations

  45. [46]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  46. [47]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...