{"id":"45a23e5c-8817-4690-a72c-ecc965a9f13c","arxiv_id":"1909.00531","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Sharing decoder states across sentences yields consistent, modest BLEU gains, suggesting target-side context is best injected into the decoder.","lead":"This paper proposes reusing a machine translation decoder's hidden states from the previous sentence to inform the current translation, and reports small BLEU gains on eight language pairs. The main caveat is that the gain may come from weight-sharing regularization rather than from genuinely leveraging target-side discourse.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Shared-target BLEU gains may come from weight-sharing/regularization rather than target-side discourse context; missing control: a version whose context content is scrambled or removed.","rationale":"I read the paper as making an empirical architectural claim: target-side context helps NMT when injected as decoder states. The load-bearing condition is that the measured BLEU gains are caused by the target discourse information, not by the shared model's extra parameters or regularization. The reader's weakest assumption identifies exactly this confound, and the paper itself acknowledges it in Section 4.1. My proposed permutation control directly tests whether the content of the previous target sentence matters. Since the paper does not include such a control and no code is released, the CONDITIONAL verdict remains appropriate: the claim is plausible but not fully isolated from the regularization explanation. If the control shows no BLEU drop, the central claim would need to be rephrased as \"reusing decoder states improves NMT,\" with no evidence that target-side context per se is useful.","tokens_in":7043,"tokens_out":4168,"duration_ms":40262,"concrete_test":"Train the shared target model on TED De-En and Recipe Ja-En with the same code and hyperparameters, but permute the order of sentences within each document (or draw the \"previous sentence\" state from a different document) so the context attention receives decoder states that are not discourse-related. If BLEU stays at the same level as the intact shared target model, the gains come from weight-sharing/regularization rather than target-side context; if BLEU drops toward the baseline, the discourse content is doing the work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 explicitly concedes: \"it is possible that weight sharing leads to a more efficient model space by regularization, rather than by learning discourse structures.\" The central claim (\"target-side contexts are also useful\") is supported only by comparing the shared target model to a sentence-level baseline. Table 2 shows that the shared source model, which supplies no target-side discourse information, also improves BLEU in most language pairs, and none of the shared-source versus shared-target differences is tested for significance. The separated target model, which does use target-side context without weight sharing, is often worse and unstable (Section 4.4). Thus the improvement attributed to target-side context is confounded with (a) an extra attention pathway over saved decoder states and (b) weight sharing/recurrent stacking of sentence-level RNNs. The output examples in Table 3 are qualitative and do not demonstrate that the model uses target discourse rather than general target-language fluency. Without an ablation that keeps the architecture and weight-sharing intact but destroys the discourse link between previous and current target sentences, the headline claim is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates how target-side context can improve context-aware neural machine translation. It proposes a \"shared model\" that stores decoder hidden states from the previous target sentence and computes an attention vector over those saved states when translating the current sentence, in contrast to earlier multi-encoder models that append a separately encoded context vector. Experiments on TED German–English, Chinese–English, Japanese–English and Recipe Japanese–English compare separated source/target models, shared source/target models, and a shared mix model. The shared target model improves BLEU over a sentence-level baseline in all eight language pairs, and the paper concludes that target-side context is useful when introduced as decoder states, that source-side context effectiveness depends on the language pair, and that weight sharing is effective for context-aware NMT.","tokens_in":7197,"tokens_out":3022,"duration_ms":47704,"significance":"If the reported gains genuinely come from using target-side discourse context, the result would overturn earlier findings that target-side context is less useful than source-side context, and it would offer a simple, parameter-efficient way to incorporate target context into RNN-based NMT. The paper has real strengths: it covers eight language-pair/domain combinations, runs three random seeds, reports standard deviations, uses bootstrap significance testing against the baseline, and explicitly acknowledges the possible regularization explanation in Section 4.1. The central weakness is that the headline claim is confounded with weight sharing/regularization, and the paper does not provide the control experiment needed to separate these explanations.","major_comments":[{"comment":"The central claim that target-side context is useful is supported by comparing the shared target model to a sentence-level baseline, but the shared target model differs from the baseline in two ways: it adds an attention mechanism over saved decoder states, and it shares weights across chronologically stacked sentence-level decoders. Table 2 shows that the shared source model, which provides no target-side discourse information, also improves BLEU in most language pairs, and no significance test is reported between shared source and shared target. The paper itself concedes in Section 4.1 that weight sharing may help only through regularization. A control experiment is needed that keeps the architecture and weight sharing intact but destroys the discourse link between the previous and current target sentences, for example by shuffling the saved previous decoder states or by using states from a different document. Without such a control, the improvement attributed to target-side context is not established.","section":"§2.2, §3.3, Table 2"},{"comment":"The discussion of the separated target model's instability is purely qualitative. The paper states that the model \"is unstable and does not lead to an improvement,\" but no quantitative evidence is provided, such as the variance across epochs and random seeds, the gap between best and final epoch BLEU, or a comparison of convergence curves. This matters because the paper's architectural recommendation (target context should be inserted into the decoder, not a separate encoder) depends on the separated target model failing. Please report the actual instability quantitatively.","section":"§4.4, Fig. 3"},{"comment":"Statistical significance is only computed between each context model and the sentence-level baseline. To support the conclusions that \"the target-side context is as important as the source-side context\" and that shared models are better than separated models, the paper should also report paired significance tests between the context models themselves, especially shared target versus shared source and shared target versus separated target. Without these comparisons, the relative claims in Section 4.2 are not supported by the table.","section":"§3.3, Table 2"},{"comment":"The output-example analysis rests on a single anecdotal pair of sentences from Recipe Japanese–English. The selected example does show consistency in translating \"長ねぎ\" across the shared target and shared mix outputs, but it does not demonstrate that the models systematically use target-side discourse rather than general target-language fluency or chance. I recommend a quantitative discourse-level evaluation, such as term-consistency metrics or pronoun-resolution tests, plus additional examples, before claiming improved phrase coherence.","section":"§4.3, Table 3"}],"minor_comments":[{"comment":"The notation is inconsistent: Eq. (3) uses c^{i-1}_n, while later text and the shared-mix description refer to c^{i-1} as a single vector. Please define the exact indexing and the dimension of the zero vector used for c^0.","section":"§2, Eq. (8)"},{"comment":"The claim that summing two attention vectors \"arranges the same number of parameters as the other shared models\" is slightly misleading because parameter count is not the only cost; the shared mix model computes two context attentions. Please clarify what is being kept equal.","section":"§2.3"},{"comment":"Several decoding and preprocessing details are missing, including beam size, vocabulary size, batch size per GPU, and whether BPE was applied with a joint or separate vocabulary beyond \"separately for source and target.\" These details are important for reproducibility.","section":"§3.2"},{"comment":"Reference [13] is cited as \"Dabre et al.\" but the reference list gives Dabre and Fujita. Please correct the citation to match the reference.","section":"References"},{"comment":"The note \"statistically significant results against the baseline at p < 0.05 in all the runs\" would benefit from a brief explanation of how the three runs were combined, since each run appears to have its own bootstrap test.","section":"Table 2"},{"comment":"The eight subfigures are very small and the axes are not labeled. The reader cannot see the claimed instability of the separated target model from the figure as printed; please enlarge or annotate the relevant curves.","section":"Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study with a clear hypothesis and a reasonable range of language pairs, but the main claim needs a decisive control experiment before it can be accepted. The lack of code or data release also limits reproducibility, though the corpora are public and the implementation is described in enough detail that a determined reader could reimplement it. I would encourage the editor to treat the missing discourse-content control as a required revision rather than an optional improvement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid short empirical paper with one real confound, and the authors know it. The shared target model—using saved decoder states from the previous sentence as context—improves BLEU over a sentence-level baseline in all eight language-pair/domain settings tested. That is a nice, simple result, and the weight-sharing framing is a useful way to think about injecting target-side context. The experimental breadth is good: three runs, bootstrap significance against the baseline, TED and Recipe domains, four language pairs in both directions. I believe the result is real as a systems improvement.\n\nThe soft spot is exactly what the stress-test note and Section 4.1 flag. The shared target model differs from the sentence baseline in two respects: it supplies target-side discourse information, and it shares decoder weights across adjacent sentences. The shared source model also improves over baseline in most pairs, and no significance test is reported between shared source and shared target. So the claim that target-side context per se drives the gains is not established. The authors concede this in Section 4.1. A scrambled-context control—same architecture, same weight sharing, but context states from a shuffled or empty previous sentence—would separate the discourse effect from regularization. That is a reasonable request, not a fatal flaw.\n\nOther soft spots are minor: the separated target model's instability is described qualitatively from Figure 3, not quantified; no code or data is released, which slows replication; and the output examples in Table 3 are illustrative only. None of these undercut the main contribution, but they do mean the paper should be read as 'a simple method that helps in practice, mechanism not yet pinned down.'\n\nWho it's for: anyone working on document-level NMT or context-aware sequence generation. It is a useful data point and a plausible baseline for future work. I would send it to peer review, not desk reject it; a serious referee can ask for the control and a clearer significance analysis. The core method is clean and the evaluation is honest.","headline":"Simple, honest weight-sharing trick for target-side context in document-level NMT, but the headline claim is partly confounded with a regularization effect the authors themselves acknowledge.","tokens_in":7670,"tokens_out":2031,"would_cite":true,"duration_ms":284475,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that target-side context improves neural machine translation when it is injected through decoder states, and that a weight-sharing method reusing the previous sentence's decoder hidden states improves BLEU over a…","keywords":["neural machine translation","document-level context","target-side context","weight sharing","decoder hidden states","BLEU evaluation","discourse coherence","context-aware translation"],"falsifier":"Shuffle sentence order within each document during both training and testing while keeping the shared decoder scheme identical; if BLEU does not drop relative to the ordered setting, the target states are not supplying discourse information and the central claim fails.","tokens_in":6859,"feed_emoji":"🔁","tokens_out":7344,"duration_ms":74377,"temperature":0.7,"pith_summary":"Earlier context-aware NMT work reported that reading the previous target sentence does not help translation, and concluded that target-side context is less useful than source-side context. This paper argues that conclusion was an artifact of the architecture: previous target sentences were passed through a separate context encoder instead of the decoder itself. The authors propose a weight-sharing method that saves the decoder hidden states from translating the previous sentence and attends to them while generating the current sentence. In experiments across eight language-pair and domain settings, this shared target model improves BLEU over a sentence-level baseline in every setting, and it adds almost no parameters or extra computation. If the result holds, document-level NMT can exploit target-side context as cheaply as source-side context.","feed_headline":"Reused decoder states lift BLEU in all eight test pairs","feed_subtitle":"Weight-sharing the decoder across adjacent sentences makes target-side context work, contrary to earlier negative results.","key_machinery":"The central mechanism is weight sharing between the decoder for the previous sentence and the decoder for the current sentence, implemented as saved hidden states. The same LSTM decoder processes both sentences; its hidden states from sentence $i-1$ are retained and serve as keys and values for an extra dot-product attention from the current decoder state. This contrasts with the separated target model, which requires a separate target context encoder and exhibits training instability. Because the shared model simply reloads previously computed hidden states, it avoids the extra parameters and the additional encoding pass of a separate context encoder.","core_discovery":"On the paper's own terms, the discovery is that target-side context belongs in the decoder, not in a separate encoder. The shared target model reuses the LSTM decoder across adjacent sentences: while translating sentence $i-1$, it stores the decoder states $h^{i-1}_t$; for each output position $n$ of sentence $i$, it computes a context vector $c^{i-1}_n = \\sum_t \\beta^{i-1}_{n,t} h^{i-1}_t$ with attention weights $\\beta^{i-1}_{n,t}$ derived from the current decoder state, and concatenates this vector with the source attention. This yields statistically significant BLEU gains over the sentence-level baseline in all eight tested settings, covering English, German, Chinese, and Japanese in both directions over two domains. The paper also reports that source-side context helps only for some language pairs, while the shared target model is consistently helpful, and that a shared mix model summing both source- and target-context attentions is competitive with the better single-context model.","pith_inferences":["The authors raise but do not resolve the possibility that the gain is regularization from weight sharing rather than genuine use of discourse information; randomizing sentence order so that 'previous' decoder states come from unrelated sentences would separate those two accounts.","Although the experiments use an RNN decoder, a parallel design for Transformer decoders, conditioning on hidden representations from the previous sentence's decoder stack, is a natural extension the paper does not discuss.","The paper observes that the separated target model is unstable in training, which suggests that earlier negative results about target context may have been caused by that instability rather than by the absence of useful target-side information.","The language-dependent source-side results hint that typological properties such as pro-drop could predict when source context matters, pointing to a possible predictor for choosing source-side versus target-side conditioning."],"forward_implications":["Target-side context should no longer be treated as unusable; injecting the previous decoder states is a simple modification that improves translation quality across all tested language pairs and domains.","Source-side context is not universally beneficial: for language pairs such as English–German it can lower BLEU, so the choice of which side to condition on depends on the languages involved.","Because the shared model adds no parameters beyond the baseline and does not need a separate context encoder, document-level conditioning can be added at negligible training and inference cost.","The shared mix model shows that either source or target context can be used nearly interchangeably; selecting the better side per language pair captures most of the available gain.","Output examples indicate improved phrase coherence, with repeated noun phrases translated consistently across adjacent sentences when target context is used."],"supporting_citations":[{"why":"The multi-encoder architecture this paper extends; its earlier conclusion that target-side context does not help is the claim being revisited.","marker":"[4]"},{"why":"Supplies the dot-product global attention used for both source and context attention calculations.","marker":"[8]"},{"why":"Provides the talk corpora used for six of the eight language-pair experimental settings.","marker":"[9]"},{"why":"Byte pair encoding for subword segmentation, required for the preprocessing pipeline.","marker":"[10]"},{"why":"BLEU is the evaluation metric on which all comparisons and significance claims are based.","marker":"[11]"},{"why":"Bootstrap resampling toolkit used to determine statistical significance against the baseline.","marker":"[12]"},{"why":"Earlier finding that source-side context can hurt English–German translation, used as a comparison in the discussion.","marker":"[15]"},{"why":"Concatenation-based context modeling whose conclusions the paper's target-side results align with.","marker":"[16]"}],"fun_headline_variants":["Target context works if decoder states are shared","Sharing decoder states makes target context helpful in NMT","Weight-shared decoder flips target context from useless to helpful","Target context aids NMT when decoder is reused across sentences","Shared decoder yields BLEU gains in all eight test pairs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The BLEU gains are credited to target-side discourse context, but they could instead come from the regularizing effect of sharing one decoder across adjacent sentences; the paper itself flags this possibility in its discussion of weight sharing.","fun_headline_variants_meta":{"raw":{"variants":["Target context works if decoder states are shared","Sharing decoder states makes target context helpful in NMT","Weight-shared decoder flips target context from useless to helpful","Target context aids NMT when decoder is reused across sentences","Shared decoder yields BLEU gains in all eight test pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002223,"raw_usage":{"total_tokens":8576,"prompt_tokens":897,"completion_tokens":7679,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":7599}},"tokens_in":513,"tokens_out":7679,"duration_ms":47237,"temperature":1.0,"reasoning_tokens":7599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:44:49.207317+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Shuffle sentence order within each document during both training and testing while keeping the shared decoder scheme identical; if BLEU does not drop relative to the ordered setting, the target states are not supplying discourse information and the central claim fails.","supporting_citations":[{"cited_title":"Evaluating discourse phenomena in neural machine translation,","cited_arxiv_id":null,"evidence_quote":"The multi-encoder architecture this paper extends; its earlier conclusion that target-side context does not help is the claim being revisited."},{"cited_title":"Effective approaches to attention-based neural ma- chine translation,","cited_arxiv_id":null,"evidence_quote":"Supplies the dot-product global attention used for both source and context attention calculations."},{"cited_title":"WIT 3: Web inventory of transcribed and trans- lated talks,","cited_arxiv_id":null,"evidence_quote":"Provides the talk corpora used for six of the eight language-pair experimental settings."},{"cited_title":"Neural machine translation of rare words with sub- word units,","cited_arxiv_id":null,"evidence_quote":"Byte pair encoding for subword segmentation, required for the preprocessing pipeline."},{"cited_title":"BLEU: a method for automatic evalua- tion of machine translation,","cited_arxiv_id":null,"evidence_quote":"BLEU is the evaluation metric on which all comparisons and significance claims are based."},{"cited_title":"Travatar: A forest-to-string machine translation engine based on tree transduc- ers,","cited_arxiv_id":null,"evidence_quote":"Bootstrap resampling toolkit used to determine statistical significance against the baseline."},{"cited_title":"A large-scale test set for the evaluation of context-aware pronoun translation in neural machine translation,","cited_arxiv_id":null,"evidence_quote":"Earlier finding that source-side context can hurt English–German translation, used as a comparison in the discussion."},{"cited_title":"Neural machine translation with extended context,","cited_arxiv_id":null,"evidence_quote":"Concatenation-based context modeling whose conclusions the paper's target-side results align with."}],"review_version":1}