Pith. sign in

REVIEW 3 major objections 5 minor 27 references

PL-Guard: Benchmarking Language Model Safety for Polish

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces a Polish-language safety benchmark and claims that a small HerBERT classifier outperforms much larger guard models, especially on adversarial noise.

desk verdict PL-Guard is a useful new dataset, but the HerBERT-superiority result is not yet credible because the test set may be contaminated by training-split leakage and test-set-tuned hyperparameters. read the letter →

arxiv 2506.16322 v1 pith:NE364UEM submitted 2025-06-19 cs.CL

Aleksandra Krasnodębska , Karolina Seweryn , Szymon {L}ukasik , Wojciech Kusa This is my paper
classification cs.CL
keywords Polish-languagesafetyclassificationadversarialperturbationsHerBERTLlamaGuardtaxonomycontentmoderationmultilingualLLMevaluationbenchmarkdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces PL-Guard, a manually annotated Polish-language safety benchmark for classifying prompts and model responses as safe or as one of fourteen hazard categories, together with PL-Guard-adv, a version of the test set corrupted by realistic character-level noise such as typos, OCR errors, and diacritic changes. The authors fine-tune three safety models on Polish data, including an 8-billion-parameter Llama-Guard model and a Polish-adapted Llama variant, and compare them with public guard models such as PolyGuard and GPT-4o-mini. Their central claim is that the smallest model, a HerBERT-based classifier, achieves the highest overall performance and is the most robust under adversarial perturbations. This matters because safety evaluation has been heavily English-centric, and if the claim holds, effective Polish safety moderation can be done with a compact, fast, language-specific model rather than a very large general-purpose one.

What carries the argument

The argument rests on two coupled resources. PL-Guard is a manually annotated Polish test set of 900 balanced examples (50 per hazard category plus 200 safe), using the Llama Guard taxonomy of 14 risk categories; PL-Guard-adv is the same set passed through a perturbation pipeline that applies 1 to 20 random character-level edits per sentence (typos, OCR errors, diacritic removal, swaps). The model comparison is carried by HerBERT, a base-sized Polish BERT derivative, fine-tuned as a single-label safety classifier and evaluated against Llama-Guard-3-8B, PLLuM-8B, PolyGuard, WildGuard, and GPT-4 variants. The mechanism that explains the headline result is native-language specialization: the winning classifier sees original Polish text, while the larger models are trained with machine-translated data from other taxonomies. The perturbation pipeline is what exposes the robustness gap.

What would settle it

Compare all models on a test set built from chatbot conversations with real Polish users rather than LLM-generated prompts, after removing any near-duplicates with training data; if HerBERT's adversarial margin disappears, the benchmark construction rather than model architecture caused the result.

Watch

Extended reading notes

Core claim

The paper's central claim is that a fine-tuned HerBERT-base-cased classifier is the best overall safety moderator for Polish among all tested models. On the clean PL-Guard test set its multiclass category F1 reaches 0.663 and its binary safety F1 reaches 0.935; on the adversarially perturbed PL-Guard-adv set it reaches 0.913 binary F1, the highest of any model, while the best fine-tuned Llama-Guard-3-8B drops to 0.814. The authors explain this by native-language specialization: HerBERT was trained on original Polish text, while the larger models had to rely partly on machine-translated examples, and the perturbed set exposes the larger models' sensitivity to surface noise. They conclude that compact, language-specific classifiers can be both more accurate and more robust than large general-purpose guard models in medium-resource languages.

Load-bearing premise

The ranking of models assumes PL-Guard's test set is a clean held-out measure; the appendix reports that the best Llama training configurations were chosen using results on this test set, and the train and test examples come from the same annotated pool with no described deduplication step.

Editorial extensions

If this is right

  • If the result holds, Polish-language content moderation can be run on a compact transformer rather than an 8B-parameter model, with lower serving cost and similar or better safety accuracy.
  • Benchmark designers for non-English languages should include adversarially perturbed splits; clean-set results alone would have ranked Llama-Guard ahead of HerBERT on binary safety.
  • Adding machine-translated PolyGuard and WildGuard data improves large Llama-based models, but for HerBERT the best binary adversarial safety score comes from the native Polish training set alone.
  • Polish-trained guard models do not generalize to English, so cross-lingual transfer cannot be assumed and language-specific safety classifiers remain necessary.
  • For fine-grained hazard classification, the specialized HerBERT model outperforms all tested baselines on Polish, suggesting that taxonomy coverage matters more than raw model scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The robustness gap may partly reflect how subword tokenizers handle typos; testing diacritic-only perturbations separately would show whether HerBERT's advantage is a tokenizer property rather than a training-data property.
  • Because PL-Guard train and test come from a single annotated pool and the appendix reports using the test set to choose Llama hyperparameters, a near-duplicate analysis and a truly held-out re-run would show how much of the reported ranking is benchmark artifact.
  • A direct inference-cost or latency measurement would make the practical claim about lightweight deployment explicit; the paper argues efficiency but does not report runtime.
  • The same annotation protocol could be repeated for other medium-resource languages; if the pattern recurs, it would form a design rule: build a small native classifier before renting a large multilingual guard API.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces PL-Guard, a Polish-language safety classification benchmark with manually verified labels and an adversarial perturbation variant (PL-Guard-adv). The authors fine-tune three models (HerBERT-based classifier, Llama-Guard-3-8B, and PLLuM, a Polish-adapted Llama) using combinations of PL-Guard, translated WildGuard, and PolyGuard data, and compare them against several baselines including GPT-4 variants and PolyGuard. The central claim is that the HerBERT-based classifier achieves the highest overall performance, particularly under adversarial conditions, despite being much smaller than the Llama-based models.

Significance. If the evaluation were clean, the paper would make a valuable contribution: a manually annotated Polish safety benchmark, an adversarial robustness test set, and evidence that a compact, language-specialized classifier can outperform much larger general-purpose guard models in a medium-resource language. The released resources (test sets and the HerBERT model) would be useful for future research on non-English safety moderation. The high inter-annotator agreement (Krippendorff's alpha = 0.92) and the explicit release of the dataset and model are strengths. However, the validity of the central claim currently hinges on whether the test set is a truly held-out set, which is questionable for the reasons detailed below.

major comments (3)
  1. [Appendix C.2 and Section 4.4] The statement in Appendix C.2 that 'the best results on the PL-Guard test set were obtained using the following configurations' indicates that the PL-Guard test set was used for hyperparameter selection for the Llama-based models. This violates the requirement that a test set remain untouched until final evaluation. While the bias introduced here runs against the paper's favored model (HerBERT was trained with fixed hyperparameters described in Appendix C.1), it nevertheless invalidates the reported Llama-guard and PLLuM results as honest held-out estimates, and it prevents a fair comparison. The authors must either use a separate validation split or clearly report that the test set was used for model selection and adjust the conclusions accordingly.
  2. [Section 3.1] The test set is carved from the same pool of over 7,000 manually annotated instances, with 900 items selected for testing and the remaining 6,487 used for training, and no deduplication or independence check is reported. Because the harmful prompts were generated with a template-based RainbowTeaming framework, the risk of near-duplicate or templatically overlapping content between train and test is substantial. A fine-tuned BERT classifier can exploit such surface-level overlap, and since PL-Guard-adv is a perturbed copy of exactly those test items, the reported adversarial robustness of HerBERT could reflect memorization of the clean test instances rather than genuine robustness. The paper should quantify the duplicate/near-duplicate rate between train and test, and if any significant overlap exists, re-split the data and re-run all experiments.
  3. [Abstract / Section 5.1] The abstract claims that HerBERT 'achieves the highest overall performance,' but in Table 3, for the binary safety F1 score on the original PL-Guard test set, Llama-Guard-3-8B trained on PL-Guard+WG+PG reaches 0.938 while HerBERT under the same condition reaches 0.935. HerBERT's advantage is clear on the adversarial set and on category-level F1, but 'highest overall performance' is an overstatement unless 'overall' is defined to exclude the binary clean-test comparison. The claim should be narrowed to, for example, the best performance on adversarial inputs and on fine-grained category classification, or the results should be reported with appropriate caveats.
minor comments (5)
  1. [Appendix C.2, Table 8] The learning-rate values '1e7' and '1e5' are almost certainly typos for 1e-7 and 1e-5; this should be corrected to avoid confusion.
  2. [Table 1 and Section 4.3.2] The WildGuard test subset size is given as 1,709 in Table 1 but as 1,308 in the text of Section 4.3.2; these numbers should be reconciled.
  3. [Section 4.3.3 (not present; see Section 3)] The paper says in Section 3 that the authors 'plan to release the test portion' of the dataset, but the abstract and conclusion state that the dataset is released. The release status should be consistent.
  4. [Table 2] The examples in Table 2 contain apparent rendering artifacts, such as '♂shield-altsafe' and '/exclamati⌢n-triangle', which make the adversarial examples difficult to read; these should be cleaned up.
  5. [Abstract and Section 1] There are minor grammatical issues, e.g., 'leaving majority of global languages underexamined' should be 'leaving the majority of global languages underexamined'; a careful copyedit is recommended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are empirical benchmark comparisons against human-annotated labels, not derivations from their own inputs.

full rationale

PL-Guard is an empirical benchmark paper. Its main claim—that the HerBERT-based classifier achieves the highest overall performance on PL-Guard and PL-Guard-adv—is supported by measured macro-F1 scores on a test set whose labels were manually reviewed, not by an equation or by a parameter fit that mathematically forces the result. The dataset construction does depend on the Llama Guard taxonomy and on Llama Guard pre-annotations, but those pre-annotations were re-annotated by humans and the paper reports agreement of 0.92, so the labels are not reducible to the very models being evaluated. The self-citations (Krasnodębska et al. 2025 for question generation, Chrabaszcz et al. 2025 for perturbation motivation) are methodological precursors rather than load-bearing evidence; nothing in the paper invokes a uniqueness theorem or an unverified prior claim to exclude alternative models. The Limitations section's admissions—no manual check of translated datasets, automatically generated prompts, multiclass instead of multilabel—are quality limitations, not circular steps. The more substantive concerns, namely that PL-Guard-test was carved from the same 7,000-instance annotation pool as the training set without a described deduplication step, that PL-Guard-adv is a perturbed copy of that same test set, and that Appendix C.2 states hyperparameters for the Llama models were selected using PL-Guard test results, are threats to benchmark validity and to the strength of the generalization claim, but they are not circularity as defined in this review: they do not make the reported HerBERT advantage equal by construction to a fitted parameter, a self-cited result, or a renamed known pattern. No step of the paper's claimed derivation chain reduces to its own inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the validity of the evaluation protocol, which in turn rests on the hyperparameter choices (one set tuned on the test set), the assumption that the Llama Guard taxonomy is suitable for Polish, and the reliability of machine translation for test data. These are the main non-data-driven inputs the paper relies on.

free parameters (3)
  • Llama-based model hyperparameters (epochs, lr, batch size) = Varies per model and data setting (e.g., lr 1e-7 to 1e-5, epochs 1 to 5, batch size 4)
    Appendix C.2 states that the best configurations for Llama-Guard-3-8B and Llama-PLLuM-8B-base were selected based on performance on the the PL-Guard test set, meaning these hyperparameters were fitted to the evaluation data rather than chosen via a separate validation split.
  • HerBERT training hyperparameters = 5 epochs, lr 1e-5, batch size 32, weight decay 0.01, max grad norm 5.0, 100 warmup steps
    These values are given in Appendix C.1 as fixed choices; they are hand-chosen and directly affect the reported performance of the HerBERT model, which is central to the paper's main claim.
  • Number of adversarial perturbations per example = Between 1 and 20, sampled uniformly
    The perturbation count is randomly sampled for each test sentence (Section 3.2), and this design choice controls the difficulty of PL-Guard-adv, directly influencing the measured robustness gap between models.
assumptions (3)
  • domain assumption The Llama Guard hazard taxonomy (S1-S14) is appropriate and sufficient for Polish-language safety classification.
    The entire dataset and all evaluations use this English-derived taxonomy without validating its applicability to Polish cultural, legal, or linguistic contexts (Section 3 and Appendix A).
  • domain assumption The human annotations provide reliable ground truth for safety labels.
    Only the first 100 instances were triple-annotated (Krippendorff alpha 0.92); the remaining over 7,000 instances were annotated by a single reviewer (Section 3), so annotator disagreement on the bulk of the data is unknown.
  • domain assumption The machine translation model (BiDi-eng-pol) preserves safety-relevant semantics.
    Used to create PL-Guard-en and the Polish WildGuard test set; translation quality was not manually checked for these test sets, as acknowledged in the Limitations section.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PL-Guard: Benchmarking Language Model Safety for Polish." pith.science (2026). https://pith.science/paper/NE364UEM

@misc{pith2026250616322,
  author       = {Pith},
  title        = {Pith review of: PL-Guard: Benchmarking Language Model Safety for Polish},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NE364UEM}},
  note         = {Machine review of arXiv:2506.16322}
}
read the original abstract

Despite increasing efforts to ensure the safety of large language models (LLMs), most existing safety assessments and moderation tools remain heavily biased toward English and other high-resource languages, leaving majority of global languages underexamined. To address this gap, we introduce a manually annotated benchmark dataset for language model safety classification in Polish. We also create adversarially perturbed variants of these samples designed to challenge model robustness. We conduct a series of experiments to evaluate LLM-based and classifier-based models of varying sizes and architectures. Specifically, we fine-tune three models: Llama-Guard-3-8B, a HerBERT-based classifier (a Polish BERT derivative), and PLLuM, a Polish-adapted Llama-8B model. We train these models using different combinations of annotated data and evaluate their performance, comparing it against publicly available guard models. Results demonstrate that the HerBERT-based classifier achieves the highest overall performance, particularly under adversarial conditions.

Figures

Figures reproduced from arXiv: 2506.16322 by the authors.

Figure 1
Figure 1. F1 score difference between the HerBERT and [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Performance drop between PL-Guard and PL [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Performance drop between PL-Guard and PL-Guard-Adv divided by safety categories across trained [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

27 extracted references · 13 canonical work pages

  1. [5]

    arXiv preprint arXiv:2404.05993

    Aegis: Online adaptive ai content safety moderation with ensemble of llm experts. arXiv preprint arXiv:2404.05993. Tsimur Hadeliya and Dariusz Kajtoch

    arXiv

  2. [6]

    Evaluation of Few-Shot Learning for Classification Tasks in the Polish Language

    Evalua- tion of few-shot learning for classification tasks in the polish language. Preprint, arXiv:2404.17832. Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri

    work page Pith review arXiv

  3. [7]

    Preprint, arXiv:2406.18495

    Wildguard: Open one-stop mod- eration tools for safety risks, jailbreaks, and refusals of llms. Preprint, arXiv:2406.18495. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar

    arXiv

  4. [9]

    arXiv preprint arXiv:2312.06674

    Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674. Krzysztof Jassem, Michał Ciesiółka, Filip Grali ´nski, Piotr Jabło ´nski, Jakub Pokrywka, Marek Kubis, Monika Jabło ´nska, and Ryszard Staruch

    arXiv

  5. [10]

    LLMzSz{\L}: a comprehensive LLM benchmark for Polish

    LLMzSzŁ: a comprehensive LLM benchmark for Polish. arXiv preprint arXiv:2501.02266. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bam- ford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, and 1 oth- ers

    work page Pith review arXiv

  6. [11]

    arXiv preprint arXiv:2401.04088

    Mixtral of experts. arXiv preprint arXiv:2401.04088. Art¯urs Kanepajs, Vladimir Ivanov, and Richard Moulange

    arXiv

  7. [12]

    Towards Safe Multilingual Frontier AI

    Towards safe multilingual frontier ai. Preprint, arXiv:2409.13708. Łukasz Kobylinski, Maciej Ogrodniczuk, Jan Kocon, Michał Marcinczuk, Aleksander Smywinski-Pohl, Krzysztof Wołk, Danijel Koržinek, Michal Ptaszyn- ski, Agata Pieciukiewicz, and Paweł Dybała

    work page Pith review arXiv

  8. [14]

    pl web service

    BAN-PL: A Polish dataset of banned harmful and offensive content from wykop. pl web service. In Proceedings of the 2024 Joint International Conference on Computational Linguis- tics, Language Resources and Evaluation (LREC- COLING 2024), pages 2107–2118. Artur Kot, Mikołaj Koszowski, Wojciech Chojnowski, Mieszko Rutkowski, Artur Nowakowski, Kamil Guttmann...

    work page 2024

Show all 27 references
  1. [15]

    Preprint, arXiv:2502.14509

    Mul- tislav: Using cross-lingual knowledge transfer to combat the curse of multilinguality. Preprint, arXiv:2502.14509. Aleksandra Krasnod˛ ebska, Maciej Chrabaszcz, and Wo- jciech Kusa

    arXiv

  2. [16]

    In Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025), pages 155–165, Albuquerque, New Mexico

    Rainbow-teaming for the Polish language: A reproducibility study. In Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025), pages 155–165, Albuquerque, New Mexico. Association for Computational Linguistics. Priyanshu Kumar, Devansh Jain, Akhila Yerukola, Li- wei J...

    2025

  3. [17]

    Preprint, arXiv:2504.04377

    Polyguard: A multilingual safety moderation tool for 17 languages. Preprint, arXiv:2504.04377. Teven Le Scao, Angela Fan, Christopher Akiki, El- lie Pavlick, Suzana Ili ´c, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, and 1 others

    arXiv

  4. [18]

    Preprint, arXiv:2407.21783

    The llama 3 herd of models. Preprint, arXiv:2407.21783. Robert Mroczkowski, Piotr Rybak, Alina Wróblewska, and Ireneusz Gawlik

    arXiv

  5. [19]

    arXiv preprint arXiv:2410.18565

    Bielik 7b v0.1: A polish language model – devel- opment, insights, and evaluation. arXiv preprint arXiv:2410.18565. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving

  6. [20]

    In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3419–3448, Abu Dhabi, United Arab Emirates

    Red teaming language models with language models. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3419–3448, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. PLLuM Consortium PLLuM Consortium

    2022

  7. [21]

    arXiv preprint arXiv:2405.10138

    PL-MTEB: Polish Massive Text Embedding Benchmark. arXiv preprint arXiv:2405.10138. Piotr Rybak, Robert Mroczkowski, Janusz Tracz, and Ireneusz Gawlik

    arXiv

  8. [22]

    arXiv preprint arXiv:2005.00630

    KLEJ: Comprehensive benchmark for Polish language understanding. arXiv preprint arXiv:2005.00630. Mikayel Samvelyan, Sharath Chandra Raparthy, An- drei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäsc...

    2005 arXiv

  9. [23]

    Preprint, arXiv:2402.16822

    Rainbow teaming: Open- ended generation of diverse adversarial prompts. Preprint, arXiv:2402.16822. The Mistral AI Team

    arXiv

  10. [24]

    Accessed: 2024-12-15

    Mistral nemo instruct 2407 model card. Accessed: 2024-12-15. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, and 1 others

    2024

  11. [25]

    In Pro- ceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 12574– 12584

    Pad: A robustness enhancement ensem- ble method via promoting attention diversity. In Pro- ceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 12574– 12584. Wenjun Zeng, Yuchi Liu, Ryan M...

    2024

  12. [26]

    Preprint, arXiv:2407.21772

    Shield- gemma: Generative ai content moderation based on gemma. Preprint, arXiv:2407.21772. Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang

    arXiv

  13. [27]

    arXiv preprint arXiv:2309.07045

    Safety- bench: Evaluating the safety of large language mod- els. arXiv preprint arXiv:2309.07045. A Detailed Annotation Process In Table 6 we present the used safety taxonomy. We follow original Llama Guard definition of hazard categories. Initially, we employed the Llama Guar...

    arXiv

  14. [2019]

    Anna Kolos, Inez Okulska, Kinga Gł ˛ abi´nska, Agnieszka Karli´nska, Emilia Wi´snios, Paweł Ellerik, and An- drzej Prałat

    Poleval 2019—the next chapter in evaluating natural language processing tools for polish. Anna Kolos, Inez Okulska, Kinga Gł ˛ abi´nska, Agnieszka Karli´nska, Emilia Wi´snios, Paweł Ellerik, and An- drzej Prałat

    2019

  15. [2020]

    In Findings of the Association for Computational Linguistics: EMNLP 2020 , pages 3356–3369, Online

    RealToxi- cityPrompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Computational Linguistics: EMNLP 2020 , pages 3356–3369, Online. Association for Computational Linguistics. Shaona Ghosh, Prasoon Varshney, Erick Galinkin, and Ch...

    2020

  16. [2022]

    arXiv preprint arXiv:2203.09509

    Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. arXiv preprint arXiv:2203.09509. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggin...

    arXiv

  17. [2023]

    arXiv preprint arXiv:2308.09662

    Red- teaming large language models using chain of utterances for safety-alignment. arXiv preprint arXiv:2308.09662. Maciej Chrab ˛ aszcz, Katarzyna Lorenc, and Karolina Seweryn

    arXiv

  18. [2024]

    Accessed: 2024-12-15

    Llama 3.1 70b instruct model card. Accessed: 2024-12-15. Lukasz Augustyniak, Kamil Tagowski, Albert Sawczyn, Denis Janiak, Roman Bartusiak, Adrian Szymczak, Arkadiusz Janz, Piotr Szyma´nski, Marcin W ˛ atroba, Mikołaj Morzy, and 1 others

    2024

  19. [2025]

    Preprint, arXiv:2506.07645

    Evaluating llms robustness in less resourced languages with proxy models. Preprint, arXiv:2506.07645. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.