{"id":"a08eaba3-bda2-40dc-a24c-6791c19bfe00","arxiv_id":"2412.14323","paper_version":2,"verdict":"REJECT","confidence":"LOW","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Manually inserting the omitted Chinese particle 的 into news titles and fine-tuning NMT models yields small BLEU and CHRF improvements, but the experiment has serious baseline and reporting gaps.","lead":"Translating Chinese to English is hard when the particle 的 (DE) is left out, making phrases ambiguous. This paper manually inserts those missing particles into news titles and then fine-tunes translation models, reporting small BLEU and CHRF gains.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains are not tied to the DE-insertion dataset: the fine-tuning data in §4 is the UM Corpus, not the manually modified Penn CTB titles, so the central claim is unsupported.","rationale":"The reader's weakest_assumption identifies the same fatal gap: the experiments fine-tune on UM Corpus News sentences, not on the DE-inserted title dataset. This is not a mere wording issue; it is the only mechanism by which the reported gains could support the paper's central claim. The 60-sentence condition's lack of effect and the 1k condition's substantial effect are exactly what one would expect from generic in-domain fine-tuning, making the dataset mismatch even more salient. I also note the absence of a control arm, error bars, or significance testing, but these would be secondary if the experiment actually used the described dataset. My proposed check would settle the attribution question by comparing the reported fine-tuning recipe against the actual DE-inserted corpus and a generic in-domain control. Since the concern fully aligns with the reader's rejection, no verdict change is needed.","tokens_in":5595,"tokens_out":3596,"duration_ms":29566,"concrete_test":"Re-run the fine-tuning comparison with three arms, fixing model, optimizer, batch size, epochs, and random seed: (A) fine-tune NLLB-200 on the 143 Penn CTB title pairs with DE manually inserted (source Chinese with 的, target Argos English); (B) fine-tune NLLB-200 on 1,000 random UM Corpus News sentences without DE reinsertion, disjoint from the evaluation sample; (C) use the pretrained checkpoint without fine-tuning. Evaluate all arms on the same held-out 1,000-sentence UM Corpus sample, excluding any sentence that appears in the fine-tuning sets. If arm B matches or exceeds arm A's BLEU/CHRF gains over C, the reported improvements are generic in-domain effects, not evidence for the DE-insertion dataset. If arm A beats B only on attributive-noun test items, the claim is partially supported; otherwise it fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest load-bearing step is the attribution of the BLEU/CHRF gains to the DE-inserted title corpus. Section 3.2 describes a parallel dataset built by taking 165 Penn CTB news titles, running Argos Translate before and after manually inserting the omitted particle 的, and keeping the modified English translations as targets. If the paper's hypothesis is right, fine-tuning should be performed on those altered source-target pairs. Instead, Section 4 states: \"The NMT models were fine-tuned with 2 subsets: 60 and 1k sentences from the approximately 67.5k sentences categorized under the News section of UM Corpus.\" The evaluation is also a 1,000-sentence sample from the UM Chinese-English parallel corpus. No train/eval split or deduplication is described, so the 1k fine-tuning sentences may overlap with the test sample. Crucially, Table 1 shows the 60-sentence condition produces essentially no change (Marian BLEU 37.8→37.8, mBART 36.7→36.7; only NLLB CHRF +0.1), while the 1k UM News condition produces the reported gains (e.g., NLLB BLEU 38.3→39.5, CHRF 72.1→75.1). Those gains can be explained entirely by generic in-domain fine-tuning on 1,000 news sentences, not by DE insertion. There is no control condition fine-tuning on UM News sentences without DE insertion and no condition fine-tuning on the actual DE-inserted Penn CTB titles. The paper's own limitations section admits the automated DE-insertion script failed and manual insertion was time-consuming, but it never says the fine-tuning data in §4 came from that manual process. As written, the central claim is internally inconsistent: the experiment does not use the dataset the hypothesis depends on.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper addresses the translation of Chinese attributive nouns into English, focusing on the omitted attributive particle 的 (DE). The authors describe a feasibility test of closed-class word omission, then construct a parallel dataset from 165 Penn Chinese Discourse Treebank news titles by manually inserting the omitted particle and using Argos Translate to generate modified English translations. They report that fine-tuning three NMT models (Marian, NLLB-200, mBART) on a subset of this data improves BLEU and CHRF scores on a 1,000-sentence sample from the UM corpus, and they conclude that targeted interventions on attributive-noun handling can enhance MT performance. The paper also reports a manual check in which 45 sentences improved and 42 regressed.","tokens_in":5912,"tokens_out":4372,"duration_ms":33442,"significance":"The underlying idea is practical and worth pursuing: a small, carefully curated resource targeting a specific linguistic ambiguity could complement generic domain adaptation. The authors are transparent about the manual effort required and about the failure of their automatic DE-insertion script, which is a useful negative result. However, the experimental evidence as reported does not test the stated hypothesis: the fine-tuning corpus in Section 4 is the UM Corpus News section, not the DE-inserted Penn CTB titles, and the evaluation design lacks a control condition and a documented train/test separation. As a result, the paper's central claim is unsupported by its own experiments, and the contribution reduces to a dataset description whose effectiveness is not demonstrated.","major_comments":[{"comment":"The paper's central claim is that fine-tuning on the DE-inserted title dataset improves translation, but Section 4 states that the models were fine-tuned with 60 and 1,000 sentences from the News section of the UM Corpus, not with the 135-sentence DE-inserted Penn CTB parallel dataset built in Section 3.2. Because the fine-tuning data and the DE-insertion intervention are never connected, the BLEU and CHRF gains in Table 1 cannot be attributed to handling attributive nouns, and the central claim is unsupported as reported. The Limitations section (6.2) candidly describes the manual effort and small corpus scope, but it does not acknowledge that the fine-tuning experiments were not run on that corpus at all.","section":"§4, Experiment Results"},{"comment":"The evaluation uses a 1,000-sentence sample from the UM Chinese-English parallel corpus, while the fine-tuning data is drawn from the UM Corpus News section; no train/test split or deduplication is described. The reported gains (e.g., NLLB-200 BLEU 38.3→39.5, CHRF 72.1→75.1) could therefore be inflated by overlap between fine-tuning and evaluation sentences, and in any case there is no control condition fine-tuned on the same 1,000 UM News sentences without DE insertion to establish that the intervention, rather than in-domain adaptation, caused the improvement.","section":"§4, Experiment Results"},{"comment":"The manual evaluation reports 45 improved and 42 regressed sentences, a nearly balanced outcome that does not support the claim that the intervention improves translation quality; the assertion that 'the magnitude and quality of improvements were substantially more significant than the regressions' is presented without examples, metrics, or inter-annotator agreement, and no error bars or significance tests are reported for the automatic scores in Table 1.","section":"§5, Results"},{"comment":"The interpretation states that fine-tuning 'the Helsinki NLP model' with the DE-enhanced dataset showed improvements, but Table 1 reports results for three models and the fine-tuning data was, per Section 4, the UM Corpus rather than the DE-enhanced dataset; this discrepancy between the narrative and the experimental record compounds the attribution problem identified above.","section":"§6.1, Interpretation of Results"}],"minor_comments":[{"comment":"The abstract contains a sentence fragment: 'By manually inserting the omitted particle X ('DE'). In news article titles from the Penn Chinese Discourse Treebank, we developed a targeted dataset...' This should be rewritten as a single grammatical sentence.","section":"Abstract"},{"comment":"The description of NLLB-200 as standing for 'Neural Language Lattice Based model' is incorrect; the model name expands to 'No Language Left Behind,' and the description should be corrected to reflect the actual architecture.","section":"§4.1, Models"},{"comment":"The phrase 'To test our test-dataset' is confusing because the experiments fine-tune on the UM Corpus rather than the DE-inserted dataset; the wording should clarify that the test is of the hypothesis, and should explicitly state which dataset is used for fine-tuning.","section":"§4, Experiment Results"},{"comment":"The phrase 'precision metrics' appears where the results table reports CHRF; since CHRF is an F-score over character n-grams, the wording should refer to it specifically rather than to precision.","section":"§5, Results"}],"recommendation":"reject","confidential_remarks":"The manuscript's experimental section does not implement the proposed method: the fine-tuning data is the UM Corpus rather than the DE-inserted Penn CTB titles, so the central claim is untested. This is a load-bearing error that cannot be fixed by clarification; the experiments would need to be redesigned and rerun with the correct dataset and appropriate control conditions. In addition, the manuscript contains numerous writing errors and an inaccurate model description, suggesting it is not yet at the standard for this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core hypothesis is plausible: Chinese attributive nouns often drop 的, and restoring it might help MT produce clearer English. The authors built a 143-title parallel dataset by manually inserting 的 into Penn CTB titles and re-translating them. That is a legitimate small artifact, and the feasibility test that singled out DE among 82 function words is a reasonable way to motivate the work. The related work on function words is adequate, and the limitations section is honest about the script failing and the manual process being slow.\n\nThe problem is that the experiment never actually uses that dataset. Section 4 says the models were fine-tuned on 60 and 1,000 sentences from the News section of the UM Corpus, not on the DE-inserted titles described in Section 3.2. The evaluation is also a 1,000-sentence UM sample. So the BLEU/CHRF gains in Table 1 can be fully explained by generic in-domain fine-tuning. The 60-sentence condition shows almost no change, and the 1k condition shows gains; that pattern is what you would expect from simply adding news-domain training data. There is no control condition fine-tuning on UM sentences without DE insertion, and no condition fine-tuning on the actual DE-inserted titles. As written, the paper's own experiment contradicts its stated method. The 45-improved versus 42-regressed manual check does not rescue this, and there are no error bars or released artifacts to check overlap.\n\nThis is not a minor flaw. The central claim collapses unless the authors clarify that the fine-tuning data was actually the DE-inserted dataset or provide a proper ablations. The paper reads like a promising workshop idea that got an experimental section written in haste and mismatched with the rest of the text.\n\nA serious referee should see this, because the hypothesis is worth testing and the dataset, if used correctly, could provide a small but real signal. But as it stands, the evidence does not support the conclusions. I would ask the authors to correct the experimental description, fine-tune on the DE-inserted titles themselves, and add a control condition. If they do that, the paper might be salvageable.","headline":"The claimed fine-tuning dataset and the actual fine-tuning data are different things: the reported BLEU gains come from generic UM Corpus news sentences, not from the manually DE-inserted titles, so the central claim is unsupported.","tokens_in":6465,"tokens_out":1618,"would_cite":false,"duration_ms":15675,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Restoring the omitted particle 的 in Chinese news titles and fine-tuning on the result improves Chinese-to-English translation scores.","keywords":["Chinese-English machine translation","attributive nouns","particle 的","function words","fine-tuning","news titles","BLEU","CHRF"],"falsifier":"Fine-tune the same three models on the same 60- and 1,000-sentence news-domain subsets without any DE restoration, evaluate on the same test sentences, and compare BLEU and CHRF; also check whether any evaluation sentence appears in the fine-tuning subset. If the gains match the DE-restored condition or disappear when overlap is removed, the claim that DE restoration drives the improvement is falsified.","tokens_in":5386,"feed_emoji":"🌐","tokens_out":14033,"duration_ms":109577,"temperature":0.7,"pith_summary":"Chinese frequently drops the attributive particle 的 (DE), so a phrase like 学生问题 can be read either as a noun phrase ('student question') or as a clause ('a student asks a question'). The paper argues that this ambiguity is a systematic, under-addressed error source in Chinese-to-English machine translation, and that a small targeted fine-tuning dataset can mitigate it. The authors built that dataset from 165 Chinese news titles, reinserting DE in 143 of them and using the corresponding English translations to train the model to infer the missing particle. They report small but measurable BLEU and CHRF gains after fine-tuning three pretrained translation models, with the largest gains at 1,000 training sentences. A sympathetic reading is that even minimal hand-curated linguistic annotation can improve a common translation failure mode without large-scale retraining.","feed_headline":"Restoring a missing 的 lifts Chinese-English translation scores","feed_subtitle":"A 1,000-sentence fine-tuning set with the omitted particle restored improves BLEU and CHRF across three translation models","key_machinery":"The mechanism is the parallel training pair built from the original Chinese title (particle omitted) and the English translation produced after the particle 的 was manually reinserted. In Chinese, attributive nouns usually omit 的, so the same string can be parsed either as a noun phrase or as a subject-verb clause; the fine-tuning data forces the decoder to map the ambiguous source to the intended English noun-phrase reading. The paper's controlled feasibility test narrowed 82 candidate function words to DE as the one omission that satisfies all three criteria (optional in Chinese, grammatical but ambiguous when deleted, and shifting the translation meaning), and then used the news-title pairs to expose the model to that particular ambiguity across many examples.","core_discovery":"On the paper's own terms, the discovery is that a closed-class word omitted from the source language can be treated as a trainable signal: by pairing Chinese titles as they actually appear (without 的) with English translations generated from the DE-restored versions, the model learns to recover the intended attributive reading. The authors report that this fine-tuning yields higher BLEU and CHRF scores across three pretrained NMT systems, that the improvement grows when the fine-tuning subset is enlarged from 60 to 1,000 sentences, and that a manual inspection of individual outputs found 45 improved translations against 42 regressions, with the improvements judged more substantial. This is offered as evidence that function-word restoration in the source text is a practical complement to large-scale domain adaptation.","pith_inferences":["A testable extension is to generate synthetic DE-restored training pairs by running a Chinese dependency parser over raw titles and inserting 的 before attributive nouns, which would remove the manual bottleneck and let the method scale.","The same omitted-particle ambiguity should appear in other clipped Chinese registers (headlines, captions, chat), so fine-tuning on DE-restored examples from those registers may transfer.","Because the gains are measured in BLEU and CHRF, a human evaluation of whether the corrected translations are actually the intended noun-phrase readings would clarify the practical value.","Repeating the feasibility test on other closed-class words omitted in Chinese could reveal whether DE is unique or one of several high-value targets."],"forward_implications":["A few hundred manually corrected examples can shift model behavior on a systematic error type, without retraining from scratch.","Increasing the fine-tuning subset from 60 to 1,000 sentences produced larger BLEU and CHRF gains, so the benefit scales with the amount of DE-restored data.","The method points to other omitted function words: the feasibility test identified DE as the only one meeting all three criteria among 82 candidates, so the same screening can be reused.","Translation engines that already perform well on general text can still be improved by targeted linguistic annotation, addressing error types that broad training data underrepresents."],"supporting_citations":[{"why":"Source of the 165 Chinese news titles used to build the DE-restored parallel dataset.","marker":"(Xue, 2005)"},{"why":"Supplies the BLEU metric used to measure fine-tuning gains.","marker":"(Papineni et al., 2002)"},{"why":"Supplies the CHRF metric used alongside BLEU.","marker":"(Popovi´c, 2015)"},{"why":"MarianNMT, one of the three pretrained models fine-tuned in the experiments.","marker":"Junczys-Dowmunt et al., 2018"},{"why":"NLLB-200, the model that showed the largest reported gains at the 1,000-sentence setting.","marker":"Costa-jussà et al., 2022"},{"why":"mBART, the third pretrained model fine-tuned and evaluated.","marker":"Liu et al., 2020"},{"why":"Prior evidence that function words affect NMT output, motivating the targeted intervention.","marker":"He et al., 2019"}],"fun_headline_variants":["Missing 的 in Chinese titles becomes trainable signal","1,000 restored particles boost Chinese-to-English scores","Fine-tune on restored 的 to lift NMT quality","How restoring an omitted word improves machine translation","A tiny dataset fixes a big translation gap"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported BLEU and CHRF gains come from the DE-inserted title dataset, because Section 4, as written, describes fine-tuning on 60- and 1,000-sentence subsets from the News section of a parallel corpus rather than on the DE-inserted dataset described in Sections 1 and 3.2; if generic in-domain fine-tuning or overlap between fine-tuning and evaluation sentences explains the gains, the central claim collapses.","fun_headline_variants_meta":{"raw":{"variants":["Missing 的 in Chinese titles becomes trainable signal","1,000 restored particles boost Chinese-to-English scores","Fine-tune on restored 的 to lift NMT quality","How restoring an omitted word improves machine translation","A tiny dataset fixes a big translation gap"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000822,"raw_usage":{"total_tokens":3526,"prompt_tokens":806,"completion_tokens":2720,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":422,"completion_tokens_details":{"reasoning_tokens":2646}},"tokens_in":422,"tokens_out":2720,"duration_ms":20421,"temperature":1.0,"reasoning_tokens":2646,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:20:01.980554+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fine-tune the same three models on the same 60- and 1,000-sentence news-domain subsets without any DE restoration, evaluate on the same test sentences, and compare BLEU and CHRF; also check whether any evaluation sentence appears in the fine-tuning subset. If the gains match the DE-restored condition or disappear when overlap is removed, the claim that DE restoration drives the improvement is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the 165 Chinese news titles used to build the DE-restored parallel dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"mBART, the third pretrained model fine-tuned and evaluated."}],"review_version":1}