Pith. sign in

REVIEW 4 major objections 5 minor 28 references

Controlling Language Confusion in Multilingual LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding an explicit penalty against undesired languages during fine-tuning suppresses language confusion in multilingual LLMs without degrading general QA performance.

desk verdict Useful, honest empirical note on ORPO for Korean language consistency, but the paper's own SFT baseline undercuts the claim that the penalty is the active ingredient. read the letter →

arxiv 2505.19116 v2 pith:7C5LXJ5F submitted 2025-05-25 cs.CL

Nahyun Lee , Yeongseo Woo , Hyunwoo Ko , Guijin Son This is my paper
classification cs.CL
keywords languageconfusioncode-mixingmultilingualLLMspreferenceoptimizationORPOlow-resourcelanguagesKoreaninstructiontuningconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models sometimes answer in the wrong language, mixing Korean, English, or Chinese in ways that make responses unusable. This paper argues that the root cause is the training objective: standard next-token prediction and supervised fine-tuning reward putting probability on the correct token but never penalise producing an entire response in the wrong language. The authors show that during pretraining, both SmolLM2 and OLMo2 models assign steadily falling loss to code-mixed text, so the model never learns to disfavor it. They then apply ORPO, a preference-optimization method that adds an explicit penalty against disfavored outputs, using Korean prompts with fully Korean chosen responses and either fully English or synthetically code-mixed rejected responses. After three epochs of ORPO, both models produce near-perfect word- and language-precision rates even at high decoding temperatures, while Korean multiple-choice QA accuracy stays about the same.

What carries the argument

The central mechanism is the ORPO objective (odds-ratio preference optimization), which augments standard supervised fine-tuning with a penalty term that widens the odds ratio between chosen and rejected responses without needing a reference model. The paper operationalizes this as triplet data: a Korean prompt, a fully Korean chosen response, and a rejected response that is fully English or Korean with randomly injected machine-translated English or Chinese tokens. The penalty inside ORPO is what creates the measurable loss gap between coherent and code-mixed outputs, and that gap is the paper's evidence that the model has internalized a preference for monolingual generation.

What would settle it

Test ORPO-trained models on naturally occurring code-mixed input or on code-mixed prompts drawn from real user logs, and score the outputs both with WPR/LPR and with human raters; if consistency drops or raters still flag confusion, the synthetic-rejection training did not generalize to genuine confusion patterns.

Watch

Extended reading notes

Core claim

The paper's central claim is that language confusion is not intrinsic to multilingual models but a consequence of an objective that never penalizes undesired languages, and that adding such a penalty through preference optimization fixes it. Evidence comes from two directions: loss trajectories during pretraining show no gap between monolingual and code-mixed text, and generation-level metrics (WPR and LPR) show ORPO-tuned models reach near-perfect language consistency at temperatures 0.7, 1.0, and 1.2, where the base models' consistency collapses at high temperature. ORPO's delta loss on code-mixed responses is the largest among SFT, DPO, and ORPO, indicating the penalty is internalized, and HAE-RAE accuracy does not degrade, so the gain is not paid for in general QA ability.

Load-bearing premise

The load-bearing premise is that the constructed rejected responses—fully English outputs and synthetically code-mixed outputs with randomly inserted translated tokens—capture the language confusion users actually encounter, so defeating them means defeating genuine confusion rather than memorizing an artificial training pattern.

Editorial extensions

If this is right

  • ORPO fine-tuning on language-consistency triplets raises WPR and LPR to near-perfect levels for both SmolLM2-1.7B and OLMo2-7B at temperatures 0.7, 1.0, and 1.2, while the base models degrade markedly as temperature rises.
  • The pretraining loss analysis indicates that models do not learn to disfavor code-mixed text at any scale up to 7B, so language confusion is better addressed through explicit preference signals than through model size alone.
  • Because HAE-RAE accuracy is preserved, the language-consistency gain from ORPO comes without a measurable trade-off in general Korean QA performance.
  • ORPO matches or exceeds SFT and clearly outperforms DPO on language fidelity across temperatures, making it a practical fine-tuning recipe for low-resource language consistency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the penalty mechanism is language-agnostic, the same triplet recipe could be applied to other language pairs, and even to non-linguistic style attributes such as verbosity or off-topic generation, by choosing the right rejected outputs.
  • The synthetic rejection data (random token injection) may be easier to learn than real, naturally occurring code-mixing; constructing triplets from observed confusion patterns would test whether the method transfers beyond its training signal.
  • If the loss-gap diagnostic is right, adding an ORPO-style penalty during pretraining rather than only at fine-tuning could prevent confusion from forming in the first place, an extension the paper only gestures at.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper addresses language confusion in multilingual LLMs, proposing that standard supervised fine-tuning lacks an explicit penalty against undesired languages and that ORPO-style preference optimization can supply this penalty. The authors construct Korean triplet datasets (OIG, HC3, KoAlpaca) in which chosen responses are pure Korean and rejected responses are either fully English or synthetically code-mixed, fine-tune SmolLM2-1.7B and OLMo2-7B with ORPO, and evaluate language consistency with WPR/LPR at temperatures 0.7, 1.0, and 1.2. They report near-perfect WPR/LPR scores for ORPO, loss trajectories suggesting that pretraining does not separate monolingual from code-mixed text, and HAE-RAE QA results that they describe as showing no significant degradation.

Significance. The problem is practically important, particularly for low-resource languages, and the paper provides a concrete, reproducible recipe for constructing preference data and fine-tuning models for language consistency. The loss-based diagnostics are a useful descriptive addition, and the paper is honest about several limitations. However, the central causal claim—that the explicit penalty in ORPO, rather than exposure to Korean-only text, is what suppresses language confusion—is not supported by the paper's own baselines: the SFT baseline in Appendix E matches or exceeds ORPO on the generation metrics. The evaluation also appears to be in-distribution and the metric is not independent of the training objective. The contribution is therefore currently a promising practical recipe rather than an established demonstration that preference penalties are necessary or uniquely effective.

major comments (4)
  1. [§5.1, Appendix E, Tables 4–5] The central claim that ORPO's explicit penalty, rather than mere exposure to Korean-only text, suppresses language confusion is not supported by the paper's own baselines. In Table 4, SmolLM2 SFT at T=1.2 achieves WPR>0.9 ratio of 100.0% and LPR>0.9 ratio of 99.7%, essentially identical to ORPO's 100.0% and 99.9%. In Table 5, OLMo2 SFT at T=1.2 actually reports slightly higher average WPR (0.9970 vs 0.9962) and average LPR (0.9915 vs 0.9881) than ORPO. Thus the generation-level evaluation cannot distinguish the proposed penalty mechanism from standard SFT on Korean responses. The authors should either provide an evaluation setting in which SFT fails and ORPO succeeds (e.g., naturally code-mixed prompts or held-out domains), report effect sizes with confidence intervals for the differences, or explicitly soften the causal conclusion to 'exposure to monolingual Korean data is sufficient under these conditions.'
  2. [§4.2, §3.1, Appendix A/D] The evaluation appears in-distribution and the metric is not independent of the training objective. The paper does not specify the source of the 1,000 evaluation prompts or whether they are disjoint from the roughly 10k triplet training sets derived from OIG, HC3, and KoAlpaca. If the prompts come from the same corpora, the near-perfect WPR/LPR scores may reflect learning a surface rule—emit Korean tokens and avoid English/Chinese tokens—that exactly matches the constructed training labels (full-English or randomly injected rejected responses). This concern is compounded by the fact that chosen responses were filtered to WPR/LPR = 1.0 during data construction and the WPR/LPR > 0.9 threshold was chosen by manual inspection (Appendix D). The authors should report prompt provenance and disjointness, add a held-out evaluation on naturally occurring Korean-English code-mixed inputs, and report sensitivity of the conclusions to the 0.9 threshold.
  3. [§5.2, Appendix A] The loss-based evidence for 'internalized penalties' is not independent of training. Section 3.1 lists HC3 as one of the three training datasets, but Section 5.2 reports delta-loss on 'the HC3 evaluation set' without stating that this subset is disjoint from the HC3-derived training triplets. If the evaluation set is drawn from the same HC3 corpus used in training, the larger ORPO delta-loss could reflect memorization of the training signal rather than a general internal preference. Additionally, Appendix A's HC3 description ('synthetically generated code-mixed rejected response') contradicts Section 3.1, which says HC3 rejected responses are written entirely in English; this discrepancy must be resolved because it changes what the HC3 diagnostic measures. Please specify the evaluation subset size, its overlap with training data, and define 'delta loss' precisely with per-sample variance.
  4. [§5.3, Figure 5] The claim that ORPO 'does not introduce measurable harm' to QA is not supported by the reported analysis. Figure 5 shows average accuracies without error bars, sample sizes, or significance tests, and the text uses 'no significant performance degradation' without a statistical test. Given the modest model sizes and the small differences likely involved, the authors should report per-subcategory scores with confidence intervals and, ideally, a paired comparison (e.g., bootstrap or McNemar) between Base, SFT, and ORPO on HAE-RAE. The same missing-variance issue affects Table 1 and Tables 4–5, where all WPR/LPR scores are reported to four decimal places without any measure of dispersion across the three repeated generations.
minor comments (5)
  1. [Eq. (1)] The notation 'T = SN i=1 Ti' should use a union symbol; as printed, 'S' appears to be a variable rather than the intended set union.
  2. [Table 1 caption] The phrase 'All metrics are higher is better' should read 'higher is better' or 'for all metrics, higher is better.'
  3. [§5.2] The term 'delta loss' is used without a definition; please state explicitly that it is mean loss on rejected responses minus mean loss on chosen responses, and report its standard deviation or confidence interval.
  4. [Appendix E, Tables 3–5] The ORPO-vs-DPO comparison is confounded by differences in optimizer (AdamW vs RMSprop), global batch size, and number of training steps; the authors should either match these settings or report a hyperparameter sensitivity analysis before claiming that ORPO 'substantially outperforms' DPO.
  5. [§4.1, Figure 2] The pretraining-loss diagnostic would be stronger with a control condition: a monotonic decrease in loss on any fixed held-out string is expected as pretraining improves, so parallel trajectories do not by themselves demonstrate the absence of an implicit penalty; reporting the loss gap with confidence intervals and comparing against matched non-Korean text would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: reported WPR/LPR gains are empirical outcomes with external controls; the metric/filter overlap is a design concern, not a by-construction equivalence.

full rationale

The central derivation—ORPO-tuned models achieve near-perfect WPR/LPR—is an empirical outcome, not an identity. Training triplets are filtered so chosen responses have WPR=LPR=1.0 (Appendix A), but nothing forces the fine-tuned model to generate Korean-only text; the DPO baseline on identical triplets degrades (Appendix E), showing that the filter alone does not produce the result. The 0.9 threshold (Appendix D) is a disclosed measurement choice, not a fitted parameter used to generate the reported numbers. The loss diagnostic in Section 5.2 is run on 'the evaluation subset HC3' and Appendix A describes HC3 as 'designed to evaluate ... unseen data during training,' so it is a held-out check rather than a re-measurement of the training objective. The HAE-RAE QA benchmark and SFT/DPO comparisons provide external control. The largest validity concern—SFT nearly matches ORPO (Appendix E), so the unique contribution of the penalty term is not isolated—is an experimental-design issue, not circularity, and the paper reports the SFT numbers transparently. No load-bearing self-citation or uniqueness argument appears.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the operationalization of language confusion via constructed triplet data and the WPR/LPR metric. No parameters are fitted to data in the derivation sense, but several hand-chosen thresholds and the fixed ORPO beta shape the results. The key assumptions are that the synthetic rejection distribution approximates real confusion and that the loss trajectory diagnostic measures internal preference. No invented entities are introduced.

free parameters (4)
  • WPR/LPR threshold = 0.9
    Hand-chosen by a native Korean speaker after manual inspection (Appendix D); used to define all pass-rate ratios in Tables 1, 4, and 5.
  • Chosen/rejected length ratio filter = 0.4 to 2.0
    Ad hoc filtering range applied during dataset construction (Appendix A) to discard unbalanced pairs.
  • ORPO weighting coefficient beta = 0.1
    Fixed to the value from the original ORPO paper (Hong et al., 2024), not tuned; affects the strength of the penalty and the loss-separation results.
  • Synthetic code-mixing injection rate = unspecified
    KoAlpaca rejected responses are created by injecting randomly selected English or Chinese tokens at random positions, but the fraction of tokens injected is not reported, making the difficulty of rejected examples uncontrolled (Appendix A).
assumptions (5)
  • domain assumption Language confusion is undesirable and should be suppressed in multilingual LLM outputs.
    Motivates the objective and evaluation; stated in Introduction and Section 2.1.
  • domain assumption WPR and LPR with the is_Korean and is_valid predicates correctly quantify target-language consistency.
    Metric definitions in Section 2.2 rely on these predicates, whose implementations are not given.
  • domain assumption Parallel loss decrease during pretraining on monolingual and code-mixed text indicates absence of internal penalization.
    Interpretation of Figure 2 and Appendix C; alternative explanations such as sequence difficulty confounds are not controlled.
  • domain assumption Synthetic rejected responses (fully English or randomly code-mixed) are representative of real language-confusion failures.
    Central to the triplet dataset construction in Section 3.1 and Appendix A.
  • standard math ORPO's odds-ratio objective can be applied to style control without reference model and without degrading generation quality.
    Inherited from Hong et al. (2024); the paper uses it as the training objective.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Controlling Language Confusion in Multilingual LLMs." pith.science (2026). https://pith.science/paper/7C5LXJ5F

@misc{pith2026250519116,
  author       = {Pith},
  title        = {Pith review of: Controlling Language Confusion in Multilingual LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7C5LXJ5F}},
  note         = {Machine review of arXiv:2505.19116}
}
read the original abstract

Large language models often suffer from language confusion, a phenomenon in which responses are partially or entirely generated in unintended languages. This critically degrades the user experience, especially in low-resource settings. We hypothesize that this issue stems from limitations in conventional fine-tuning objectives, such as supervised learning, which optimize the likelihood of correct tokens without explicitly penalizing undesired outputs such as cross-lingual mixing. Analysis of loss trajectories during pretraining further reveals that models fail to distinguish between monolingual and language-mixed texts, highlighting the absence of inherent pressure to avoid such confusion. In this work, we apply ORPO, which adds penalties for unwanted output styles to standard SFT, effectively suppressing language-confused generations. ORPO maintains strong language consistency, even under high decoding temperatures, while preserving general QA performance. Our findings suggest that incorporating appropriate penalty terms can effectively mitigate language confusion in multilingual models, particularly in low-resource scenarios.

Figures

Figures reproduced from arXiv: 2505.19116 by the authors.

Figure 1
Figure 1. Dataset structure (OIG, Chosen-Rejected pair) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Average loss for monolingual and code-mixed [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Loss of SmolLM2 models across tuning meth [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Average accuracy across training methods for [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 4
Figure 4. Figure 4: Loss of OLMo2 models across tuning methods [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 6
Figure 6. Figure 6: The average loss of original (monolingual) [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Samples of generated responses at varying WPR and LPR levels [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 7 canonical work pages

  1. [1]

    Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Mart \' n Bl \'a zquez, Guilherme Penedo, Lewis Tunstall, Andr \'e s Marafioti, Hynek Kydl \' c ek, Agust \' n Piqueres Lajar \' n, Vaibhav Srivastav, and 1 others. 2025. Smollm2: When smol goes big--data-centric training of a small language model. arXiv preprint arXiv:2502.02737

    arXiv 2025

  2. [2]

    Duarte M Alves, Jos \'e Pombal, Nuno M Guerreiro, Pedro H Martins, Jo \ a o Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, and 1 others. 2024. Tower: An open multilingual large language model for translation-related tasks. arXiv preprint arXiv:2402.17733

    arXiv 2024

  3. [3]

    Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Roee Aharoni, Melvin Johnson, and Wolfgang Macherey. 2019. Massively multilingual neural machine translation in the wild: Findings and challenges. arXiv preprint arXiv:1907.05019

    arXiv 2019

  4. [4]

    Beomi. 2023. https://github.com/beomi/KoAlpaca Koalpaca: Korean instruction-tuning dataset

    work page 2023

  5. [5]

    Yiyi Chen, Qiongxiu Li, Russa Biswas, and Johannes Bjerva. 2024. Large language models are easily confused: A quantitative metric, security implications and typological analysis. arXiv preprint arXiv:2410.13237

    work page Pith review arXiv 2024

  6. [6]

    Team Cohere, Arash Ahmadian, Marwan Ahmed, Jay Alammar, Yazeed Alnumay, Sophia Althammer, Arkady Arkhangorodsky, Viraat Aryabumi, Dennis Aumiller, Rapha \"e l Avalos, and 1 others. 2025. Command a: An enterprise-ready large language model. arXiv preprint arXiv:2504.00698

    arXiv 2025

  7. [7]

    Jessica Zosa Forde, Ruochen Zhang, Lintang Sutawika, Alham Fikri Aji, Samuel Cahyawijaya, Genta Indra Winata, Minghao Wu, Carsten Eickhoff, Stella Biderman, and Ellie Pavlick. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1085 Re-evaluating evaluation for multilingual summarization . In Proceedings of the 2024 Conference on Empirical Methods in Natura...

    doi:10.18653/v1/2024.emnlp-main.1085 2024

  8. [8]

    Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597

    arXiv 2023

Show all 28 references
  1. [9]

    Heegyu. 2023. Oig-small-chip2-ko. https://huggingface.co/datasets/heegyu/OIG-small-chip2-ko

    2023

  2. [10]

    Jiwoo Hong, Noah Lee, and James Thorne. 2024. Orpo: Monolithic preference optimization without reference model. arXiv preprint arXiv:2403.07691

    2024 arXiv

  3. [11]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  4. [12]

    LAION. 2022. Open instruction generalist (oig) dataset. https://laion.ai/blog/oig-dataset/

    2022

  5. [13]

    Kelly Marchisio, Wei-Yin Ko, Alexandre B \'e rard, Th \'e o Dehaze, and Sebastian Ruder. 2024. Understanding and mitigating language confusion in llms. arXiv preprint arXiv:2406.20052

    2024 arXiv

  6. [14]

    Yohan Na. 2023. Hc3-ko: Korean human chatgpt comparison corpus. https://huggingface.co/datasets/nayohan/HC3-ko. Accessed: 2025-05-17

    2023

  7. [15]

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, and 1 others. 2024. 2 olmo 2 furious. arXiv preprint arXiv:2501.00656

    2024 arXiv

  8. [16]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728--53741

    2023

  9. [17]

    Guijin Son, Jiwoo Hong, Hyunwoo Ko, and James Thorne. 2025. Linguistic generalizability of test-time scaling in mathematical reasoning. arXiv preprint arXiv:2502.17407

    2025 arXiv

  10. [18]

    Guijin Son, Hyunwoo Ko, Hoyoung Lee, Yewon Kim, and Seunghyeok Hong. 2024 a . Llm-as-a-judge & reward model: What they can and cannot do. arXiv preprint arXiv:2409.11239

    2024 arXiv

  11. [19]

    Guijin Son, Hanwool Lee, Suwan Kim, Huiseo Kim, Jaecheol Lee, Je Won Yeom, Jihyu Jung, Jung Woo Kim, and Songseong Kim. 2023. Hae-rae bench: Evaluation of korean knowledge in language models. arXiv preprint arXiv:2309.02706

    2023 arXiv

  12. [20]

    Guijin Son, Dongkeun Yoon, Juyoung Suk, Javier Aula-Blasco, Mano Aslan, Vu Trong Kim, Shayekh Bin Islam, Jaume Prats-Cristi \`a , Luc \' a Tormo-Ba \ n uelos, and Seungone Kim. 2024 b . Mm-eval: A multilingual meta-evaluation benchmark for llm-as-a-judge and reward models. arX...

    2024 arXiv

  13. [21]

    u/VictorRM . 2025. https://www.reddit.com/r/OpenAI/comments/1iflvc8/o3_thinks_in_chinese_for_no_reason_randomly/ O3 thinks in chinese for no reason randomly . Reddit, r/OpenAI. Accessed 2025-05-19

    2025

  14. [22]

    Bin Wang, Zhengyuan Liu, Xin Huang, Fangkai Jiao, Yang Ding, AiTi Aw, and Nancy F Chen. 2023. Seaeval for multilingual foundation models: From cross-lingual alignment to cultural reasoning. arXiv preprint arXiv:2309.04766

    2023 arXiv

  15. [23]

    Zhijun Wang, Jiahuan Li, Hao Zhou, Rongxiang Weng, Jingang Wang, Xin Huang, Xue Han, Junlan Feng, Chao Deng, and Shujian Huang. 2025. Investigating and scaling up code-switching for multilingual language model pre-training. arXiv preprint arXiv:2504.01801

    2025 arXiv

  16. [24]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

    2025 arXiv

  17. [25]

    Haneul Yoo, Cheonbok Park, Sangdoo Yun, Alice Oh, and Hwaran Lee. 2024. Code-switching curriculum learning for multilingual transfer in llms. arXiv preprint arXiv:2411.02460

    2024 arXiv

  18. [26]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595--46623

    2023

  19. [27]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  20. [28]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.