Pith. sign in

REVIEW 4 major objections 5 minor 18 references

Quality Estimation based Feedback Training for Improving Pronoun Translation

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that combining a reference-free quality-estimation reward with a pronoun-generation-likelihood reward in iterative fine-tuning improves both pronoun translation and overall English-to-German translation, reporting a…

desk verdict Plausible QE-plus-pronoun-likelihood training idea, but the evaluation rests on a circular metric and test-set tuning. read the letter →

arxiv 2501.03008 v1 pith:PXEUZM5C submitted 2025-01-06 cs.CL cs.AI

Harshit Dhankhar , Baban Gain , Asif Ekbal , Yogesh Mani Tripathi This is my paper · ORCID
classification cs.CLcs.AI
keywords pronountranslationneuralmachinequalityestimationreward-basedfine-tuningdocument-levelcontextiterativesupervisedgenerationlikelihoodEnglish-German
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ProNMT is a training framework that iteratively fine-tunes a pre-trained machine translation model using a reward signal built from a reference-free quality-estimation score plus a pronoun-specific bonus. The bonus, called pronoun generation likelihood (PGL), is the model's own probability of emitting the reference pronoun, signed negatively when the candidate pronoun does not match the reference. The authors claim that this feedback, applied to a distilled 600M-parameter NLLB model on a filtered English-to-German Europarl subset, improves both pronoun handling and overall translation quality—best context-aware scores are COMET 81.92 and BLEU 26.95, versus 73.95 and 15.2630 for the context-free baseline. The motivation is to capture part of the benefit of human-feedback training without human annotations.

What carries the argument

The key mechanism is the reward function $r(x,y) = \alpha R_{\text{translation}} + \beta R_{\text{PGL}}$. $R_{\text{translation}}$ is the score from a reference-free COMET quality-estimation model, and $R_{\text{PGL}}$ is the 'pronoun generation likelihood': the conditional probability the model assigns to the reference pronoun token at the pronoun position, taken with a positive sign if the candidate pronoun equals the reference, a negative sign if it does not, and zero if no pronoun appears. This reward is used to choose among $k=10$ sampled candidate translations for each source sentence, and the chosen translation becomes the supervised fine-tuning target for the next iteration. Context is provided by formatting the source as '<context> $x_{i-1}$ </context> $x_i$', and the training data is filtered to English 'it' aligned to German 'er', 'sie', or 'es' to reduce pronoun ambiguity.

What would settle it

Run the trained context-aware ProNMT checkpoint on the ContraPro contrastive pronoun test: if it does not select the correct pronoun significantly more often than the context-agnostic baseline despite reporting higher PGL, then PGL is not a faithful proxy for pronoun correctness and the central claim is falsified.

Watch

Extended reading notes

Core claim

The central discovery the paper argues for is that a linear reward $r(x,y) = \alpha R_{\text{translation}} + \beta R_{\text{PGL}}$, where $R_{\text{translation}}$ comes from a reference-free COMET quality-estimation model and $R_{\text{PGL}}$ is the pronoun generation likelihood (positive if the generated pronoun matches the reference, negative otherwise, zero if no pronoun is generated), can steer iterative supervised fine-tuning to improve pronoun translation and overall translation quality simultaneously. On their filtered Europarl EN→DE test set, the context-aware configuration with $\alpha=1.2$ and $\beta=1/\#\text{tokens}$ reaches COMET 81.92 and BLEU 26.9574, and the authors conclude that the combination matters because PGL-only training inflates PGL but collapses translation quality, while QE-only training improves translation without the pronoun-specific gains.

Load-bearing premise

The load-bearing premise is that the pronoun generation likelihood—the model's own probability of producing the reference pronoun token—faithfully measures whether the pronoun is translated correctly; the paper never validates PGL against human judgments or the standard ContraPro contrastive test, so if PGL can be inflated without real pronoun improvement, the reported pronoun gains support the central claim only circularly.

Editorial extensions

If this is right

  • Context-aware ProNMT exceeds the context-agnostic version on COMET, BLEU, QE, and PGL, so adding the previous sentence as context is part of the reported gains.
  • The joint QE-plus-PGL reward beats either reward alone; PGL-only training reaches PGL of 0.8998–0.9675 but collapses BLEU, so the two signals are complementary rather than redundant.
  • Reference-free quality estimation can substitute for human reward models in this pronoun-focused fine-tuning loop, removing the need for human annotations in the training pipeline.
  • Improvements are not confined to pronoun metrics: the same training run lifts overall COMET and BLEU, indicating the pronoun-focused signal does not trade away general quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because $R_{\text{PGL}}$ is the model's own probability of generating the reference token, it is a self-referential signal: optimizing it may inflate confidence without improving genuine pronoun disambiguation, and the observed collapse of PGL-only training is consistent with the model learning to output high-probability pronouns rather than correct ones.
  • A decisive check the paper does not run is the ContraPro contrastive test; if ProNMT's higher PGL does not translate into higher contrastive accuracy on pronouns, the claimed pronoun gains are an artifact of the reward metric.
  • The framework's design suggests it could extend beyond 'it' to other ambiguous pronouns or morphological agreement phenomena, but the reward definition assumes a unique reference pronoun, which limits it to cases where the target pronoun is unambiguous.
  • The paper's dismissal of ContraPro as too easy for fine-tuning is itself a claim worth testing: a head-to-head comparison with a model fine-tuned on ContraPro would show whether the filtered Europarl data is the reason for the gains or whether the gains come from the reward mechanism.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes ProNMT, an iterative fine-tuning framework for neural machine translation that uses a linear combination of a quality-estimation (QE) reward and a pronoun-specific reward to select candidate translations and fine-tune a pre-trained NLLB-200 model on a filtered English-to-German Europarl corpus. The pronoun reward is the Pronoun Generation Likelihood (PGL), defined as the model's probability of generating the reference pronoun token, with a positive sign if the generated pronoun matches the reference and a negative sign otherwise. The authors report that the context-aware ProNMT configuration achieves higher COMET, BLEU, QE, and PGL scores than a context-agnostic baseline, and they conclude that the framework consistently improves both pronoun translation and overall translation quality.

Significance. The core idea is timely and useful: using a reference-free QE model plus a pronoun-specific reward within an iterative SFT loop is a plausible recipe for improving pronoun translation without additional human annotation. The paper also deserves credit for including ablations that isolate PGL-only and QE-only rewards, which help illustrate the trade-off between pronoun likelihood and overall translation quality. However, as presented, the empirical claims are not established. The headline pronoun metric is the same quantity used as the training reward, the QE metric is the same model used as a reward component, and hyperparameters are selected on the test set. These issues are load-bearing for the paper's central conclusion.

major comments (4)
  1. [Section 3.2, Eq. (2); Section 4.3.2; Table 3] The reported pronoun metric PGL is exactly the RPGL component of the training reward in Eq. (2), and the test-set PGL values in Table 3 are averages of that same reward on the test set. The model is therefore trained to maximize the metric used to claim pronoun gains. The paper's own ablation makes the problem concrete: the 'ONLY PGL REWARD' row in Table 3 reaches PGL 0.9675 while COMET collapses to 16.79 and BLEU to 0.07, showing that PGL can be maximized while producing near-incoherent translations. Consequently, the reported improvement from 0.3672 to 0.4183 (Table 3) does not demonstrate better pronoun translation. The authors should validate on an external benchmark such as ContraPro or with human judgments; Section 4.1 dismisses ContraPro, but the replacement metric is not independently validated.
  2. [Table 2 and Table 3] Table 2 reports COMET, BLEU, QE, and PGL for four alpha/beta combinations evaluated directly on the test set, and Table 3 then presents the configuration with 'highest COMET and BLEU scores' as the headline ProNMT result. This is test-set model selection: the hyperparameters are chosen by looking at test performance, which invalidates the claim that ProNMT 'consistently outperforms' baselines on unseen data. The authors should select alpha and beta on a validation set, or report all configurations with confidence intervals and make clear how the reported configuration was chosen.
  3. [Section 4.3.2; Section 5] The evaluation metric labeled 'QE' is computed with 'wmt21-comet-qe-da', which is the same model used as Rtranslation in Eq. (2). Thus the QE column is not an independent measure of translation quality; it is another view of the training reward. The abstract and conclusion state that ProNMT improves performance 'across multiple metrics, including COMET, BLEU and QE models,' but the QE improvement is partly circular. COMET and BLEU are external, and they are the strongest evidence in the paper, but the QE-based claim should be removed or re-framed.
  4. [Sections 4.1 and 3.2] The evaluation is performed on a self-constructed filtered Europarl subset restricted to English 'it' aligned to German 'er', 'sie', or 'es', and the reward requires exact match to a single reference pronoun. The authors acknowledge in Appendix A.1 that pronoun translation is inherently ambiguous, and exact-match rewards can penalize valid alternatives. Without an external benchmark or human evaluation, it is unclear whether the reported PGL gains reflect genuine pronoun translation improvement or overfitting to the filtering and reference-matching scheme. The authors should supplement their test set with ContraPro, or at least provide a human evaluation of pronoun correctness on a sample.
minor comments (5)
  1. [Section 4.2 vs. Table 1] The text says the context fine-tuning runs for 3 epochs, while Table 1 lists 'Number of Train Epochs' as 10; please reconcile this discrepancy.
  2. [Tables 2 and 3] The 'BASELINE' row in Table 3 has exactly the same numeric values as the alpha=1.2, beta=1/#avg-len row in Table 2; the relationship between the baseline and the hyperparameter sweep should be clarified.
  3. [Table 3] The 'Loss' column is not described in the text; please explain how the loss is computed and why it is reported alongside the translation metrics.
  4. [Section 5] The paper uses the word 'significant' without any statistical significance testing; given a single seed and a limited test set, please add confidence intervals or at least soften the claim.
  5. [General] There are formatting issues such as 'ONLYPGLREWARD' and 'ONLYQE REWARD' in Table 3, and inconsistent figure references in the appendix (e.g., 'Fig. 5' appears before Figure 5 is introduced).

Circularity Check

3 steps flagged · score 7.0 of 10

Pronoun-quality claim rests on PGL, which is the training objective itself; QE test metric is the same reward model, while COMET/BLEU gains are independent.

  1. fitted input called prediction [Section 4.3.2, Table 3 caption]
    "QE (Rtranslation) and PGL (RP GL) refer to the respective average reward calculated on the test set."

    The test columns QE and PGL are not independent benchmarks; they are the components of the training reward r(x,y_j) = beta * R_PGL + alpha * R_translation in Eq. (2). Since ProNMT is iteratively fine-tuned to maximize this r, a higher test-set PGL/QE merely confirms that the optimization moved the reward, not that an external measure of pronoun quality improved. The paper's own ONLY-PGL ablation shows the metric is gameable: PGL reaches 0.9675 while COMET collapses to 16.79 and BLEU to 0.0703, meaning a model can maximize PGL while producing near-incoherent output.

  2. self definitional [Section 3.2, Eq. (2); Section 5]
    "Next, to assess the pronoun translation reward RP GL, we identify the pronoun token yp kj in candidate translation yp k and its "Pronoun Generation Likelihood (PGL)" defined as: P (yp kj | xp, yk p 1:j−1; θ). If the pronoun token matches with that in reference translation, then we set the pronoun reward to PGL itself. If it does not, then it is set to -PGL."

    PGL is defined as the model's own probability of generating the reference pronoun token, and the reward sets R_PGL = +PGL when the candidate pronoun matches the reference and -PGL otherwise. Thus 'pronoun translation accuracy' is measured by the very quantity the training loop maximizes. The conclusion that ProNMT yields 'significant gains in pronoun translation accuracy' is supported by PGL values such as 0.4183 vs 0.3672, which are averages of this training reward on the test set. No external validation against ContraPro or human judgments is provided; ContraPro is dismissed as containing 'shorter and easy-to-translate source sentences' and replaced by a filtered Europarl set, so the pronoun claim is self-referential.

1 more flagged steps
  1. fitted input called prediction [Section 4.3.2; Table 2 caption]
    "For quality estimation without reference translations, we use the ‘wmt21-comet-qe-da‘ model and present it under the QE column in the results. ... using WMT21-C OMET-QE-DA as reward model."

    The QE column in the results is computed with the same model used to compute R_translation in Eq. (2), namely wmt21-comet-qe-da. Therefore the reported 'QE' improvement is an increase in the training-reward component itself, not an independent quality estimate. The only truly independent metrics in Tables 2-3 are COMET (wmt22-comet-da, reference-based) and BLEU, which do improve in the best configuration; the paper's claim of improvement 'across multiple metrics, including COMET, BLEU and QE models' overstates the independence of the QE evidence.

full rationale

The central derivation chain is not circular for the main external metrics: the best configuration is selected by COMET/BLEU (Table 2 footnote), and the reported COMET and BLEU gains (81.92/26.95 vs 73.95/15.26) come from wmt22-comet-da with reference and BLEU, neither of which appears in the training reward. So there is genuine independent content in the paper's overall-translation claim. However, the paper's pronoun-specific claim, which is the titular contribution, rests entirely on PGL and on the QE reward. PGL is exactly R_PGL in Eq. (2), and the test PGL is defined as the average reward on the test set (Table 3 caption); the QE test column uses the same wmt21-comet-qe-da model that provides R_translation. Reporting these as evidence of pronoun improvement is reporting the optimized objective. The ONLY-PGL ablation makes the failure concrete: PGL reaches 0.9675 while COMET falls to 16.79 and BLEU to 0.07, so high PGL does not imply correct pronoun translation. The paper also declines to use the standard ContraPro benchmark, replacing it with a filtered Europarl set, so there is no external anchor for the pronoun metric. The self-citations to Appicharla et al. (2023, 2024) are related-work context and not load-bearing for the derivation, so they do not contribute to the score. Overall, the pronoun-quality result is circular by construction, while the COMET/BLEU result is independent; score 7 reflects this partial but central circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central empirical claim depends on the reward weighting alpha and beta, which were chosen using the test set, and on the PGL metric, which is both the training reward and the main pronoun evaluation. The loop is self-referential: the model is judged on the metric it was trained to maximize, with no external pronoun benchmark.

free parameters (3)
  • alpha (translation reward weight) = 1 or 1.2, best 1.2
    Weight of the QE reward in Equation 2; grid-searched and the best value was picked using test set scores (Table 2).
  • beta (pronoun reward weight) = 1/#tokens or 1/#avg_len, best 1/#tokens
    Weight of the PGL reward in Equation 2; grid-searched and the best value was picked using test set scores (Table 2).
  • candidate count k = 10
    Number of sampled candidate translations in Algorithm 1; hand-chosen, no ablation reported.
assumptions (4)
  • domain assumption wmt21-comet-qe-da provides a valid reference-free reward signal for translation quality
    The reward is defined as the QE score from this model (Section 3.2), and the same model is used as a test metric, so the framework assumes this proxy is reliable.
  • ad hoc to paper PGL is a valid proxy for pronoun translation correctness
    PGL is defined in Section 3.2 as the model's probability of generating a pronoun matching the reference; no external validation against human judgments or ContraPro is provided.
  • domain assumption Pronoun alignment in filtered Europarl is correct and unambiguous
    Dataset construction (Section 4.1) relies on part-of-speech tags and alignment to pair 'it' with er/sie/es; errors here would inject noise into the reward.
  • domain assumption Multinomial sampling with k=10 gives enough candidate diversity for reward selection
    Algorithm 1 samples k candidates; the paper does not ablate k or report sampling temperature.
invented entities (1)
  • Pronoun Generation Likelihood (PGL)
    purpose: Reward component and evaluation metric for pronoun translation quality
    Introduced in Section 3.2; it is the model's own output probability, and the paper provides no external benchmark or human correlation to show it measures real pronoun accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quality Estimation based Feedback Training for Improving Pronoun Translation." pith.science (2026). https://pith.science/paper/PXEUZM5C

@misc{pith2026250103008,
  author       = {Pith},
  title        = {Pith review of: Quality Estimation based Feedback Training for Improving Pronoun Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXEUZM5C}},
  note         = {Machine review of arXiv:2501.03008}
}
read the original abstract

Pronoun translation is a longstanding challenge in neural machine translation (NMT), often requiring inter-sentential context to ensure linguistic accuracy. To address this, we introduce ProNMT, a novel framework designed to enhance pronoun and overall translation quality in context-aware machine translation systems. ProNMT leverages Quality Estimation (QE) models and a unique Pronoun Generation Likelihood-Based Feedback mechanism to iteratively fine-tune pre-trained NMT models without relying on extensive human annotations. The framework combines QE scores with pronoun-specific rewards to guide training, ensuring improved handling of linguistic nuances. Extensive experiments demonstrate significant gains in pronoun translation accuracy and general translation quality across multiple metrics. ProNMT offers an efficient, scalable, and context-aware approach to improving NMT systems, particularly in translating context-dependent elements like pronouns.

Figures

Figures reproduced from arXiv: 2501.03008 by the authors.

Figure 1
Figure 1. Example illustrating the inter-sentential de [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Batchwise training translation reward (a) and [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Batchwise training translation reward (a) and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Batchwise training translation reward (a) and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Token counts distribution for (a) Contrapro [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 6 canonical work pages

  1. [1]

    Ramakrishna Appicharla, Baban Gain, Santanu Pal, and Asif Ekbal. 2023. https://aclanthology.org/2023.mtsummit-research.14 A case study on context encoding in multi-encoder based document-level neural machine translation . In Proceedings of Machine Translation Summit XIX, Vol. 1: Research Track, pages 160--172, Macau SAR, China. Asia-Pacific Association fo...

    work page 2023

  2. [2]

    Ramakrishna Appicharla, Baban Gain, Santanu Pal, Asif Ekbal, and Pushpak Bhattacharyya. 2024. https://aclanthology.org/2024.eamt-1.21 A case study on context-aware neural machine translation with multi-task learning . In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 246--257, Sheffield, UK....

    work page 2024

  3. [3]

    Zhiwei He, Xing Wang, Wenxiang Jiao, Zhuosheng Zhang, Rui Wang, Shuming Shi, and Zhaopeng Tu. 2024. https://doi.org/10.18653/v1/2024.naacl-long.451 Improving machine translation with human feedback: An exploration of quality estimation as a reward model . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computatio...

    doi:10.18653/v1/2024.naacl-long.451 2024

  4. [4]

    Marcin Junczys-Dowmunt. 2019. https://doi.org/10.18653/v1/W19-5321 M icrosoft translator at WMT 2019: Towards large-scale document-level neural machine translation . In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 225--233, Florence, Italy. Association for Computational Linguistics

    doi:10.18653/v1/w19-5321 2019

  5. [5]

    Julia Kreutzer, Shahram Khadivi, Evgeny Matusov, and Stefan Riezler. 2018 a . https://doi.org/10.18653/v1/N18-3012 Can neural machine translation be improved with user feedback? In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers) , pa...

    doi:10.18653/v1/n18-3012 2018

  6. [6]

    Julia Kreutzer, Joshua Uyheng, and Stefan Riezler. 2018 b . https://doi.org/10.18653/v1/P18-1165 Reliability and learnability of human bandit feedback for sequence-to-sequence reinforcement learning . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1777--1788, Melbourne, Australia. ...

    doi:10.18653/v1/p18-1165 2018

  7. [7]

    Bei Li, Hui Liu, Ziyang Wang, Yufan Jiang, Tong Xiao, Jingbo Zhu, Tongran Liu, and Changliang Li. 2020. https://doi.org/10.18653/v1/2020.acl-main.322 Does multi-encoder help? a case study on context-aware neural machine translation . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3512--3518, Online. Assoc...

    doi:10.18653/v1/2020.acl-main.322 2020

  8. [8]

    Mathias M \"u ller, Annette Rios, Elena Voita, and Rico Sennrich. 2018. A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation . In WMT 2018 , Brussels, Belgium. Association for Computational Linguistics

    work page 2018

Show all 18 references
  1. [9]

    Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kianté Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi, and Yejin Choi. 2023. https://arxiv.org/abs/2210.01241 Is reinforcement learning (not) for natural language processing: Benchmarks, baselines, and b...

    2023 arXiv

  2. [10]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. https://aclanthology.org/2020.wmt-1.101 Unbabel ' s participation in the WMT 20 metrics shared task . In Proceedings of the Fifth Conference on Machine Translation, pages 911--920, Online. Association for Computa...

    2020

  3. [11]

    Karin Sim Smith. 2017. https://doi.org/10.18653/v1/W17-4814 On integrating discourse in machine translation . In Proceedings of the Third Workshop on Discourse in Machine Translation, pages 110--121, Copenhagen, Denmark. Association for Computational Linguistics

    2017 doi

  4. [12]

    Zewei Sun, Mingxuan Wang, Hao Zhou, Chengqi Zhao, Shujian Huang, Jiajun Chen, and Lei Li. 2022. https://doi.org/10.18653/v1/2022.findings-acl.279 Rethinking document-level neural machine translation . In Findings of the Association for Computational Linguistics: ACL 2022, page...

    2022 doi

  5. [13]

    J \"o rg Tiedemann and Yves Scherrer. 2017. https://doi.org/10.18653/v1/W17-4811 Neural machine translation with extended context . In Proceedings of the Third Workshop on Discourse in Machine Translation, pages 82--92, Copenhagen, Denmark. Association for Computational Linguistics

    2017 doi

  6. [14]

    Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. 2018. https://doi.org/10.18653/v1/P18-1117 Context-aware neural machine translation learns anaphora resolution . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: L...

    2018 doi

  7. [15]

    Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, and Zhaopeng Tu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.1036 Document-level machine translation with large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natu...

    2023 doi

  8. [16]

    Longyue Wang, Zhaopeng Tu, Xing Wang, Li Ding, Liang Ding, and Shuming Shi. 2020. https://aclanthology.org/2020.wmt-1.60 Tencent AI lab machine translation systems for WMT 20 chat translation task . In Proceedings of the Fifth Conference on Machine Translation, pages 483--491,...

    2020

  9. [17]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  10. [18]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.