The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation

Aixiu An; Michael Jungo

REVIEW 3 major objections 5 minor 16 references

The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Enabling reasoning at inference improves translation quality, but matched training and inference modes dominate the cost-quality tradeoff for legal machine translation.

desk verdict Solid, clean ablation showing inference-time reasoning helps and training-with-reasoning compacts traces; the cost-quality headline is conditional on deployment volume and the single hourly rate, and the paper should say so. read the letter →

arxiv 2607.19226 v1 pith:LBHME7CP submitted 2026-07-21 cs.CL cs.AIcs.LG

Michael Jungo , Aixiu An This is my paper

classification cs.CLcs.AIcs.LG

keywords neuralmachinetranslationlegalreinforcementlearningwithverifiablerewardsreasoningtracescost-qualitytradeoffinferencecostGRPOSwiLTra-Bench

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the reasoning traces that improve legal machine translation are worth their cost. By switching reasoning on and off independently during training and inference, the authors show that enabling reasoning at inference consistently raises translation quality, even for models never trained to reason. But the reasoning mode must be consistent: models trained without thinking produce much longer, less effective reasoning at inference, tripling cost for slightly worse quality. The economic conclusion is that the cost-quality tradeoff favors matched configurations—both thinking or neither—and the main benefit of training with reasoning is keeping inference traces compact (up to 70% fewer tokens).

What carries the argument

The experiment uses GRPO training with a structural reward that requires the model to put reasoning inside <think> tags, and an empty pre-filled <think> tag to disable reasoning during training and inference. Quality is measured with chrF (the training reward), COMET, METEOR, and MetricX, while cost is estimated from a standardized hourly GPU cost ($0.79/hour) multiplied by execution time, following a depreciation-plus-power-plus-maintenance formula. The empty-tag switch cleanly separates the reasoning trace from the training paradigm, and checkpoints every 100 training samples reveal how quality and cost scale with data.

What would settle it

Recompute the cost-quality comparison with a different hourly GPU price (e.g., $0.30 or $2.00) and check whether the matched [T:✓, I:✓] and [T:✗, I:✗] configurations still Pareto-dominate; or probe the models' internal activations while generating with empty <think> tags to test whether reasoning is truly disabled.

Watch

Extended reading notes

Core claim

The central claim is that the value of reasoning in RLVR-trained translation models lies at inference, not in training. Enabling thinking during inference improves translation quality across all configurations, but training without thinking produces verbose reasoning traces at inference (up to 20.67M output tokens vs 6.27M for the 9B model) that cost far more and translate slightly worse. Therefore, the cost-quality tradeoff favors models that preserve the thinking mode between training and inference: either always think or never think. The best result is the 9B model trained and inferred with thinking, reaching a COMET score of 82.50, while its matched no-thinking variant sits on the same P

Load-bearing premise

The cost-quality conclusion rests on a fixed $0.79-per-GPU-hour price, and since the absolute cost differences between configurations are under a dollar, a different hardware or electricity price could change which configuration wins.

Editorial extensions

If this is right

Deployments should match the reasoning mode used during training; mismatched configurations are Pareto-dominated in cost and quality.
Training with reasoning is best justified as a cost-containment measure rather than a pure quality booster, reducing inference tokens by up to 70%.
RLVR's gains for translation do not require new reasoning capabilities; the base model already produces useful reasoning when asked at inference.
For the legal translation benchmark used, around 1,000–1,500 training pairs reach the quality plateau, so curating high-quality samples beats adding more data.
Matched local configurations can sit on the same cost-quality frontier as commercial API models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The $0.79/hour GPU price is the pivot: the absolute cost gaps between configurations are cents, so a sensitivity analysis across realistic hardware and electricity prices could reorder the Pareto frontier.
The empty-tag manipulation assumes reasoning is fully disabled when the tag is empty; if models reason implicitly, the 'no thinking' condition still contains hidden reasoning, making the attribution less clean than reported.
The trace-compaction effect suggests a testable extension: check whether the same 70% token reduction holds for other language pairs and domains, or whether it is specific to legal texts and the GRPO reward.
The Pareto frontier is a snapshot under one pricing model; changes in API prices or local hardware costs could shift which configurations are competitive.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This paper is worth a serious look. The core empirical finding holds up: enabling reasoning at inference improves translation quality regardless of whether the model was trained with reasoning, and training with reasoning keeps inference-time reasoning traces compact. The 2x2 factorial ablation is a genuine extension of the authors' prior work, and the token-compaction effect (e.g., Qwen3.5 9B output tokens dropping from 20.67M to 6.27M) is a non-obvious, useful observation. Quality results are consistent across four metrics with small standard errors, which is solid evidence. Code and data are available, and the experiments are reproducible in principle.

The main soft spot is the cost-quality framing. The Pareto analysis in Figure 2 plots inference cost only, justified by training being a one-time fixed cost. That justification is fine at large scale, but the paper does not quantify the scale. From Table 1, the always-thinking 9B model costs about $7 more to train than the inference-only-thinking model, but saves only about $0.75 per test-set inference. So you need roughly 170k translated sentence pairs before that training premium pays off. Below that volume, inference-only thinking is cheaper, and the COMET loss is only a hair (82.32 vs 82.50). The paper should report break-even volumes or explicitly frame the conclusion as "for high-volume deployment." The standardized $0.79/hour GPU cost is also a single point estimate with no sensitivity analysis; the Pareto ordering could shift under different hardware or electricity assumptions.

Two smaller issues: the best-checkpoint selection criterion is not stated, and the "no thinking" operationalization via empty tags is a reasonable proxy but does not rule out implicit reasoning. These are all addressable in revision.

For a short DocEng paper, this is a clean, honest empirical contribution. It is not ground-breaking, but it gives practitioners a concrete deployment rule for legal MT and similar RLVR tasks. I would send it to peer review, and I would expect acceptable after the authors add a break-even analysis, sensitivity on the hourly rate, and state the checkpoint selection protocol.

Referee Report

3 major / 5 minor

Summary. The paper studies whether explicit reasoning traces ('thinking') in RLVR-trained LLMs for legal NMT should be enabled during training and/or inference. Using Qwen3.5 4B/9B with GRPO/QLoRA on SwiLTra-Bench, the authors permute thinking on/off in the two phases, evaluate with chrF, COMET, METEOR, and MetricX, and measure training/inference cost and output tokens. They find that inference-time thinking improves translation quality regardless of the training condition, that training without thinking produces much longer thinking traces when thinking is enabled at inference, and that the cost-quality Pareto frontier favors matched modes (never thinking or always thinking). The central quantitative claim is that training with thinking is economically preferable mainly because it keeps inference-time reasoning traces compact, despite higher training cost.

Significance. The factorial design cleanly separates the role of the reasoning trace at training vs. inference, which is valuable for deployment decisions in legal MT. Strengths include open code, four complementary metrics with small standard errors, an explicit cost model borrowed from Zhuang et al., and training-sample curves (Fig. 1) supporting the plateau claim. If the conclusions hold, the result that RLVR's benefit is largely realized at inference time and that RL training compacts reasoning traces is practically important. The main weakness is that the cost-quality tradeoff is presented against inference cost only, without amortizing the substantial training-cost differences.

major comments (3)

[§4.2, Fig. 2, Table 1, Eq. (1)] The central cost-quality conclusion is based on inference cost only. The text argues that training is a one-time fixed cost and inference cost becomes the deciding factor, but no amortization or break-even analysis is provided. From Table 1, for Qwen3.5 9B the [T:✓, I:✓] configuration costs $11.61 to train + $0.28 to infer, while [T:✗, I:✓] costs $4.54 + $1.03. On a single SwiLTra-Bench test set the former is $6.32 more expensive; the claimed advantage appears only after ≈9.4 test-set equivalents (~170k sentence pairs), and ≈11.5 test sets (~207k pairs) for the 4B model. The Pareto frontier in Fig. 2 should include amortized training cost or a break-even volume, and should be accompanied by sensitivity analysis for the standardized $0.79/hour rate, since absolute cost differences are cents.
[§4.2, Table 1] The paper states 'we select the best checkpoints' but does not specify the selection criterion. Which metric and which validation/test set were used? If any test-set information influenced checkpoint selection, Table 1 and Fig. 2 overstate the achievable quality; if a validation COMET curve was used, this should be stated and the number of training samples for each selected checkpoint should be reported. This is load-bearing because the quality-cost ordering depends on the chosen checkpoint for each of the four training/inference configurations.
[§3, §4.1] The 'without thinking' condition is implemented by prefilling an empty <think></think> tag, and the paper later states that this 'does not destroy the thinking ability that the base model exhibits.' The manipulation therefore isolates the visible reasoning trace, not the presence or absence of reasoning as a cognitive process. The abstract and conclusion should phrase the finding as 'including a reasoning trace in the output' rather than 'reasoning has a positive effect,' unless a control for implicit reasoning is provided. This does not invalidate the quality comparisons, but it narrows the theoretical claim.

minor comments (5)

[§4.1] The sentence 'training without thinking ... does not destroy the thinking ability' is an interpretation; the evidence only shows that enabling thinking at inference still improves quality after such training. Please rephrase as an empirical observation about transfer, not an internal-ability claim.
[Table 1] Report for each row the number of training samples used for the selected checkpoint. Currently the training costs vary (e.g., 4B [T:✗] costs $5.46 while 9B [T:✗] costs $4.54), which is difficult to interpret without knowing the training data volume and the number of samples per checkpoint.
[§2.3, §4.2] State the hardware used for timing measurements (GPU type, vLLM configuration, batch settings) and the commercial API pricing version used in Fig. 2, so that the cost comparison is reproducible.
[§3] Provide GRPO/QLoRA hyperparameters (learning rate, LoRA rank/alpha, group size is already given as 4, number of steps, etc.) and the exact number of total training samples considered; Figure 1 shows up to 2,500 but the stopping criterion is not stated.
[Abstract] There is a formatting issue in the abstract ('bothtrainingandinference'); correct to 'both training and inference.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical ablations against an external benchmark; self-citations are not load-bearing.

full rationale

The paper's central claims are empirical comparisons, not derivations that reduce to their inputs. The main result—‘Enabling thinking during inference consistently improves the translation quality, regardless of whether the model has been trained to produce thinking traces’—is supported by direct experimental measurements in Table 1 and Figure 1, using external evaluation metrics (chrF, COMET, METEOR, MetricX). No fitted parameter is renamed as a prediction, and no equation is shown to be equivalent to another by construction. The cost-quality tradeoff uses an externally provided standardized hourly GPU cost ($0.79/hour from Zhuang et al. [16], Eq. 1), and the training/inference costs are measured from execution time; the relative ordering is an empirical observation, not a definitional consequence. The authors’ own prior work [1] is cited for the choice to weight COMET most heavily and for prior RLVR results, but this is a benchmark-selection dependency rather than a load-bearing circular argument: all four metrics agree on the main quality trends, so the central conclusion does not reduce to the self-cited COMET preference. There is no imported uniqueness theorem, no ansatz smuggled in via self-citation, and no renaming of a known result. The manuscript’s limitations—e.g., no sensitivity analysis on the $0.79/hour rate and no break-even training-cost amortization—are robustness concerns, not circularity. The derivation chain is therefore self-contained with respect to circularity.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The paper contributes empirical comparisons, not a derivation. No free parameter is fitted to make the central quality claim work; the cost model's $0.79/hour constant is an input that all cost conclusions inherit. The axioms are domain assumptions and one operationalization choice.

free parameters (1)

standardized GPU hourly cost = $0.79/hour
Chosen from Zhuang et al. [16] as a standardised cost for local inference; all training/inference cost estimates and the Pareto frontier depend on it. No sensitivity analysis is given.

assumptions (6)

domain assumption chrF is an adequate reward signal for translation quality during GRPO training.
Used as the translation-quality reward (§3); if chrF rewards are poorly aligned with human quality, the trained behavior may not reflect true translation quality.
domain assumption COMET is the most human-correlated metric on SwiLTra-Bench, as established in An et al. [1].
The paper weights COMET most heavily (§4); this relies on the prior authors' correlation analysis.
domain assumption A standardized GPU cost of $0.79/hour approximates real local inference cost.
Adopted from Zhuang et al. [16] (§4.2); all cost and Pareto-frontier statements inherit this assumption.
ad hoc to paper Prefilling empty <think> tags disables explicit reasoning during training and inference.
Operational definition of 'no thinking' (§3); implicit reasoning is not measured, so the paper's causal attribution to reasoning traces depends on this.
domain assumption SwiLTra-Bench is representative of legal translation demand.
Generalizations to legal MT and beyond (§5) assume the benchmark captures the relevant distribution.
domain assumption GRPO with group size G=4 under QLoRA is a valid RLVR setup.
The chosen training setup is inherited from prior RLVR practice and is not independently validated here.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation." pith.science (2026). https://pith.science/paper/LBHME7CP

@misc{pith2026260719226,
  author       = {Pith},
  title        = {Pith review of: The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LBHME7CP}},
  note         = {Machine review of arXiv:2607.19226}
}

read the original abstract

Reinforcement learning with verifiable rewards (RLVR) has been established as a viable paradigm for the post-training of Large Language Models (LLMs), including downstream tasks, such as Neural Machine Translation (NMT). With the latest research indicating that RLVR could be the preferred training method for translating legal documents due to the induced reasoning capabilities, it raises the question whether it is really attributed to the reasoning or more generally to the training paradigm. We investigate the importance of including the model's reasoning trace in the generated responses during both training and inference by systematically omitting it from one of the phases. Our experiments show that including the reasoning, specifically during inference, has a positive effect on the overall translation quality. Furthermore, we recognise that the reasoning leads to an increase in output tokens, hence we study the cost-quality tradeoff between the increased computational demands and the improved translation quality.

Figures

Figures reproduced from arXiv: 2607.19226 by the authors.

**Figure 2.** Reasoning Pareto frontier. Each point depicts the model’s translation quality in relation to its inference cost. The reasoning configuration is specified as [T:✓, I:✓], for whether thinking was employed during training and inference, respectively. The marker size is proportional to the parameter count of the model. 5 Conclusion The experiments conducted in this paper showed that including the model’s reasoning in … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 4 linked inside Pith

[1]

Aixiu An, Michael Jungo, Eloi Eynard, Mark Drenhaus, Andreas Fischer, Jean Hennebert, and Sébastien Rumley. 2026. Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning. InAI for Law Workshop @ ICML 2026. https://openreview.net/forum?id=822yKEGxe2

2026
[2]

Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. InProceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72

2005
[3]

Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2026. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?Advances in Neural Information Processing Systems 38 (2026), 57654–57689

2026
[4]

Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Sergey Levine, and Yi Ma. 2025. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training. InThe Second Conference on Parsi- mony and Learning (Recent Spotlight Track). https://openreview.net/forum?id= d3E3LWmTar

2025
[5]

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems36 (2023), 10088–10115

2023
[6]

Minggui He, Yilun Liu, Shimin Tao, Yuanchang Luo, Hongyong Zeng, Chang Su, Li Zhang, Hongxia Ma, Daimeng Wei, Weibin Meng, et al. 2025. R1-t1: Fully incentivizing translation capability in llms via reasoning learning.arXiv preprint arXiv:2502.19735(2025)

arXiv 2025
[7]

Michael Jungo and Andreas Fischer. 2025. Rule-Based Reinforcement Learning for Document Image Classification with Vision Language Models. InInternational Conference on Document Analysis and Recognition. Springer, 292–309

2025
[8]

Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. 2024. MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task. In Proceedings of the Ninth Conference on Machine Translation, Barry Haddow, Tom Kocmi, Philipp Koehn, and Christof Monz (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 492–504. https://a...

2024

Show all 16 references

[9]

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the 29th Symposium on Operating Systems Principl...

2023
[10]

Joel Niklaus, Jakob Merane, Luka Nenadic, Sina Ahmadi, Yingqiang Gao, Cyrill AH Chevalley, Claude Humbel, Christophe Gösken, Lorenzo Tanzi, Thomas Lüthi, et al. 2025. Swiltra-bench: The swiss legal translation benchmark.arXiv preprint arXiv:2503.01372(2025)

2025 arXiv
[11]

Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. InProceedings of the tenth workshop on statistical machine translation. 392–395

2015
[12]

Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A Neural Framework for MT Evaluation. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Associatio...

2020 doi
[13]

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024)

2024 arXiv
[14]

Jiaan Wang, Fandong Meng, and Jie Zhou. 2026. Deeptrans: Deep reasoning translation via reinforcement learning.Transactions of the Association for Com- putational Linguistics14 (2026), 47–63

2026
[15]

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

2025 arXiv
[16]

Boqin Zhuang, Jiacheng Qiao, Mingqian Liu, Mingxing Yu, Ping Hong, Rui Li, Xiaoxia Song, Xiangjun Xu, Xu Chen, Yaoyao Ma, et al. 2025. Beyond Benchmarks: The Economics of AI Inference.arXiv preprint arXiv:2510.26136(2025). Received 8 July 2026; accepted 20 July 2026

2025

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

Aixiu An, Michael Jungo, Eloi Eynard, Mark Drenhaus, Andreas Fischer, Jean Hennebert, and Sébastien Rumley. 2026. Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning. InAI for Law Workshop @ ICML 2026. https://openreview.net/forum?id=822yKEGxe2

2026

[2] [2]

Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. InProceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72

2005

[3] [3]

Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2026. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?Advances in Neural Information Processing Systems 38 (2026), 57654–57689

2026

[4] [4]

Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Sergey Levine, and Yi Ma. 2025. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training. InThe Second Conference on Parsi- mony and Learning (Recent Spotlight Track). https://openreview.net/forum?id= d3E3LWmTar

2025

[5] [5]

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems36 (2023), 10088–10115

2023

[6] [6]

Minggui He, Yilun Liu, Shimin Tao, Yuanchang Luo, Hongyong Zeng, Chang Su, Li Zhang, Hongxia Ma, Daimeng Wei, Weibin Meng, et al. 2025. R1-t1: Fully incentivizing translation capability in llms via reasoning learning.arXiv preprint arXiv:2502.19735(2025)

arXiv 2025

[7] [7]

Michael Jungo and Andreas Fischer. 2025. Rule-Based Reinforcement Learning for Document Image Classification with Vision Language Models. InInternational Conference on Document Analysis and Recognition. Springer, 292–309

2025

[8] [8]

Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. 2024. MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task. In Proceedings of the Ninth Conference on Machine Translation, Barry Haddow, Tom Kocmi, Philipp Koehn, and Christof Monz (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 492–504. https://a...

2024