{"id":"ce570fbb-e814-4af4-916d-c629e7b71b82","arxiv_id":"1906.11024","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Sharing attention weights in adjacent Transformer layers yields 1.3X inference speedup with negligible BLEU loss on ten WMT and NIST tasks.","lead":"This paper proposes sharing attention weights across adjacent layers in the Transformer to speed up auto-regressive decoding in machine translation. A smart generalist might read it to see a practical way to reduce latency in deployed NLP systems without retraining from scratch.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's weakest assumption correctly flags the empirical nature of the capacity claim, but once the full manuscript is consulted the experiments directly test that assumption on the exact tasks and models referenced in the strongest claim. No additional load-bearing gap is visible.","tokens_in":1671,"tokens_out":250,"duration_ms":20761,"concrete_test":"Reproduce the ten-task evaluation using the exact sharing policy and cache configuration described in the full paper; if average speedup falls below 1.1X or average BLEU drop exceeds 0.5 points, the headline result does not hold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an empirical report of 1.3X inference speedup (with negligible BLEU drop) on ten specific MT tasks when attention weights are shared between adjacent layers on top of an already-cached baseline. The paper directly measures this outcome across the stated tasks and also reports a larger gain when combined with the AAN model. Because the claim is scoped to the tested setting and the capacity-preservation assumption is evaluated by the reported BLEU numbers, no internal inconsistency or untested precondition is apparent.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes sharing attention weights between adjacent layers in the Transformer to enable vertical reuse of hidden states during auto-regressive inference for machine translation. The sharing policy is learned jointly with the model. On ten WMT and NIST OpenMT tasks, it reports an average 1.3X inference speedup (with almost no BLEU drop) on top of an already-cached state-of-the-art baseline, plus a 1.8X gain when combined with the AAN model (16X vs. uncached baseline).","tokens_in":1757,"tokens_out":356,"duration_ms":20962,"significance":"If the empirical results hold, the work supplies a lightweight, learnable inference optimization for Transformers that preserves task performance on the tested MT benchmarks. This is a practical contribution given the centrality of Transformer inference speed in deployed MT systems.","major_comments":[{"comment":"Results section: the central claims of 'consistent speed-ups' and 'almost no decrease in BLEU' across ten tasks are reported without error bars, variance estimates, or statistical significance tests, making it impossible to assess whether the 1.3X figure is robust or within noise of the cached baseline.","section":"Results"}],"minor_comments":[{"comment":"The description of how the sharing policy is parameterized and jointly optimized should be expanded with explicit equations or pseudocode to allow reproduction.","section":"Method"},{"comment":"Table or figure captions for the ten-task results should list per-task BLEU deltas and speed-up ratios rather than only averages.","section":"Experiments"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the positive evaluation and the recommendation for minor revision. We address the single major comment below regarding the presentation of empirical results.","responses":[{"response":"We agree that error bars, variance estimates, and statistical tests would strengthen the presentation. Each model was trained with a single run owing to the substantial computational cost of training large Transformers on the WMT and NIST corpora; multiple independent runs were not performed. The reported 1.3X average speedup (and near-zero BLEU change) is nevertheless observed uniformly across all ten tasks that differ in language pair, data size, and domain. In the revised manuscript we will add an explicit paragraph in the results section acknowledging the lack of variance estimates, justifying the single-run protocol, and emphasizing the cross-task consistency as supporting evidence for robustness.","revision_made":"partial","referee_comment":"[Results] Results section: the central claims of 'consistent speed-ups' and 'almost no decrease in BLEU' across ten tasks are reported without error bars, variance estimates, or statistical significance tests, making it impossible to assess whether the 1.3X figure is robust or within noise of the cached baseline."}],"tokens_in":1217,"tokens_out":264,"duration_ms":12672,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core result is straightforward: sharing attention weights vertically between adjacent layers, with hidden-state reuse and a jointly learned policy, delivers about 1.3X faster decoding than a strong cached baseline on WMT and NIST tasks, and 1.8X when combined with AAN. The work is mostly an engineering refinement rather than a conceptual shift, but it is scoped clearly to inference speed in autoregressive MT and measures the outcome directly against external baselines. What stands out is the consistent reporting across ten tasks and the additional gain when stacked on AAN, which shows the method is not limited to one setup. The joint learning of the sharing policy is a reasonable addition that avoids manual tuning. On the downside, the abstract gives no error bars, no ablation breakdowns, and no statistical tests, so the reliability of the 1.3X figure is hard to judge from the summary alone. The capacity-preservation assumption is tested only by the BLEU numbers on these particular tasks; nothing in the provided material suggests it would generalize without further checks. The citation pattern looks standard for the area and does not appear circular. This is the kind of paper that matters to groups shipping production MT systems who already use caching and want another lever for latency. It is not essential reading for core model research, but the empirical claim is narrow enough and the baseline strong enough that it should go to peer review rather than desk rejection. A referee could usefully press for ablations and variance numbers, but the central measurement is reproducible in principle from the described setup.","headline":"This paper reports a practical 1.3X inference speedup from sharing attention weights between adjacent layers on top of an existing cache, with little BLEU loss across ten MT tasks.","tokens_in":2220,"tokens_out":390,"would_cite":false,"duration_ms":13438,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"Empirical Transformer weight-sharing optimization unrelated to RS forcing chain","alignment":"orthogonal","rationale":"Paper's core mechanism (sharing attention matrices S across adjacent decoder layers, learned via JS-divergence policy) is a practical ML inference trick with no reference to J-cost, reciprocal symmetry, φ-ladder, 8-tick periodicity, or parameter-free constant derivation. Domain (cs.CL empirical speedup) lies outside RS theorems.","tokens_in":48753,"confidence":"high","tokens_out":110,"duration_ms":6674,"cache_read_input_tokens":38528,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Sharing attention weights between adjacent Transformer layers yields 1.3 times faster inference with almost no loss in BLEU score.","keywords":["attention sharing","transformer","machine translation","inference acceleration","weight sharing","auto-regressive decoding","WMT","NIST OpenMT"],"falsifier":"Running the shared-weight model on one of the ten tasks and measuring a BLEU drop larger than 0.5 points relative to the unshared version.","tokens_in":2582,"feed_emoji":"⚡","tokens_out":584,"duration_ms":17010,"temperature":0.7,"pith_summary":"The paper shows that attention weights can be shared across adjacent layers in the Transformer to enable vertical reuse of hidden states during auto-regressive decoding. This reduces the cost of repeated dot-product attention computations while the sharing policy itself is learned jointly with the translation model. Experiments across ten WMT and NIST OpenMT tasks demonstrate the resulting 1.3X speedup on top of an already-cached baseline implementation, and a larger gain when combined with the AAN model. A reader would care because the change requires no new hardware or data and keeps output quality nearly identical, making large attention models more usable at inference time.","feed_headline":"Sharing attention weights speeds Transformer 1.3 times","feed_subtitle":"Reusing weights across adjacent layers keeps BLEU nearly identical on ten translation tasks","key_machinery":"Attention weight sharing across adjacent layers, which permits vertical reuse of hidden states.","core_discovery":"By sharing attention weights in adjacent layers the model reuses hidden states vertically, producing an average 1.3X speed-up with almost no decrease in BLEU on ten WMT and NIST OpenMT tasks; the same approach gives 1.8X speed-up with the AAN model and reaches 16 times the speed of an uncached baseline.","pith_inferences":["The same sharing pattern could be tested on encoder-only or decoder-only Transformers outside machine translation.","Allowing different sharing patterns per head might recover any small accuracy gap observed in the experiments.","The learned sharing decisions may indicate which layer pairs perform redundant computations."],"forward_implications":["The shared model maintains translation quality within a negligible margin on standard benchmarks.","The technique stacks on top of existing attention caching for further gains.","The sharing decision can be optimized end-to-end with the translation loss.","The approach reaches 1.8X speed-up when combined with the AAN model."],"fun_headline_variants":["Shared weights speed Transformer 1.3X","Attention sharing speeds MT 1.3X","Reused weights cut decoding time 1.3X","Layer sharing speeds inference 1.3X","Vertical reuse speeds Transformer 1.3X"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Sharing attention weights between adjacent layers preserves enough model capacity to match the performance of the unshared model on the tested translation tasks.","fun_headline_variants_meta":{"raw":{"variants":["Shared weights speed Transformer 1.3X","Attention sharing speeds MT 1.3X","Reused weights cut decoding time 1.3X","Layer sharing speeds inference 1.3X","Vertical reuse speeds Transformer 1.3X"]},"model":"grok-4.3","cost_usd":0.007136,"raw_usage":{"total_tokens":3265,"prompt_tokens":606,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":71362000,"prompt_tokens_details":{"text_tokens":606,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2596,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":606,"tokens_out":63,"duration_ms":20779,"temperature":1.0,"reasoning_tokens":2596,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-25T15:47:14.051780+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the shared-weight model on one of the ten tasks and measuring a BLEU drop larger than 0.5 points relative to the unshared version.","supporting_citations":[],"review_version":1}