{"id":"f9fdc7c2-c564-4ed4-a532-760570340ad5","arxiv_id":"2502.07832","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Sharing one MLP layer's weights across several layers plus low-rank adapters recovers most of a pretrained LLM's quality with a fraction of the storage and faster phone inference.","lead":"SHARP is a way to shrink a large language model for phones by letting neighboring layers share one set of weights, patching the damage with small extra adapter weights. It reports cutting stored model size and inference time by roughly 40 percent while keeping in-distribution text quality close to the original.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The mobile speedup/storage headline (42.2% time, 42.8% storage) is measured on a simplified SHARP variant with the LoRA recovery parameters removed, so the numbers do not yet establish the claim for actual SHARP.","rationale":"I read the paper's central claim as two-part: (1) SHARP recovers perplexity with modest fine-tuning data while cutting MLP parameters, and (2) SHARP delivers large on-device storage and latency savings. The reader's weakest_assumption focuses on the low-rank-plus-warmup recovery assumption, which is plausible and empirically supported by Tables 2-4 and Appendix D.1. My own most load-bearing concern is different: the on-device savings in Table 7 are measured on a variant without the LoRA recovery parameters, and possibly on full-layer sharing rather than MLP-only sharing. This is not an attack on the recovery experiments; those remain the strongest part of the paper. But the abstract and the conclusion present the mobile numbers as properties of SHARP itself, and the reported experiment does not measure that object. This mismatch is concrete, internally verifiable from the text, and directly affects the headline claim. It is also fixable by a re-measurement, so it does not justify rejecting the paper; rather, it makes the acceptance conditional on the mobile claim being re-established for the actual method. The reader already reached CONDITIONAL, and this concern reinforces that verdict without changing it. I therefore recommend UNCHANGED, with the condition made explicit: the mobile efficiency numbers must be reproduced for the full SHARP model including recovery parameters and with an architecture matching the perplexity experiments.","tokens_in":27108,"tokens_out":4792,"duration_ms":46414,"concrete_test":"Re-run the ExecuTorch benchmark from Appendix C.3 with the fully realized SHARP Tnext model: quantize and include all rank-400 LoRA adapters for the replaced MLP layers, keep the architecture identical to the perplexity experiments (same reference/target layer sets, MLP-only sharing), and report load+init time, forward time, and on-device model size. If savings remain near 42% and the storage ratio matches Eq. (1), the headline stands; if the adapters erode the savings or the architecture must be changed to fit on device, the claims need to be revised to the adapter-free/direct-sharing variant.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central practical claim that SHARP 'saves 42.8% in model storage and reduces the total inference time by 42.2%' rests entirely on Table 7 in Section 3.5. However, Appendix C.3 states that the deployed model is 'a simplified version of SHARP (Tnext) where we removed the LoRA parameters' and that only the reference layers are stored, with target layers produced by calling the same layer multiple times. That is direct weight sharing without the recovery parameters that define SHARP. The actual method must store and load the rank-400 LoRA adapters for every replaced layer and must execute the adapter computation during each forward pass; these costs are nonzero and are omitted from the reported measurement. Moreover, the on-device model size of 2.31GB is approximately 56.25% of the 4.04GB original, which matches sharing 14 of 32 entire transformer layers, whereas the perplexity experiments in Tables 2 and 4 share only MLP weights while keeping attention weights intact. If the mobile benchmark shares whole layers, it measures a different architecture from the one whose quality is established. Either way, the headline savings are reported for a model that is not the SHARP model described and evaluated elsewhere in the paper. The claim should be re-measured with the actual SHARP architecture and with LoRA adapters included, or explicitly relabeled as a direct-sharing upper bound.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SHARP, a post-training method for compressing pretrained LLMs by sharing weights of adjacent layers and adding low-rank recovery parameters. A two-stage procedure is used: Single Layer Warmup (SLW) aligns the outputs of shared layers with L2 loss, followed by Supervised Fine-Tuning (SFT) of the recovery parameters. Experiments on Llama2-7b report that perplexity can be largely recovered on several in-distribution datasets with roughly 38%–65% reduction in stored MLP parameters, along with ablations on replacement patterns, candidate transformations, rank, and dataset size. On downstream tasks the method performs better on knowledge-memorization tasks than on reasoning tasks. The paper also reports mobile-device savings of 42.8% storage and 42.2% inference time.","tokens_in":27469,"tokens_out":5682,"duration_ms":52784,"significance":"If the perplexity-recovery claim holds, SHARP is a practically relevant compression method that requires only fine-tuning-scale data, not pretraining-scale resources. The observation that different low-rank parameterizations perform alike when parameter budgets are matched is a useful and somewhat surprising empirical finding. The cross-dataset results in Appendix D.1 and the LLaMA3.2-3B result in Appendix D.2 strengthen the generality of the core recovery claim beyond the in-distribution setting. The main caveat is that the headline mobile savings are measured on a simplified variant without the LoRA recovery parameters, so the paper's most visible practical claim is not yet supported for the method as defined.","major_comments":[{"comment":"The headline savings in Table 7 are measured on a model that is not the SHARP model evaluated elsewhere. Appendix C.3 states that the deployed model is 'a simplified version of SHARP (Tnext) where we removed the LoRA parameters' and that only reference layers are stored, with target layers produced by calling the same layer multiple times. This omits the storage and computation of the rank-400 recovery parameters that define SHARP. In addition, the on-device model size (2.31GB vs 4.04GB, about 56% of the original) matches sharing 14 of 32 entire transformer layers, whereas the quality experiments in Tables 2 and 4 focus on sharing MLP layers while keeping attention weights intact. The reported 42.8%/42.2% savings are therefore an upper bound for a direct-sharing architecture, not for SHARP with recovery parameters. Please re-measure with the actual SHARP architecture, including LoRA adapters and consistent layer-scope, or explicitly relabel the numbers as a direct-sharing upper bound.","section":"Section 3.5 and Appendix C.3"},{"comment":"The downstream evaluation shows substantial degradation on reasoning tasks even for the mild Tnext configuration: GSM8k drops from 14.2 to 3.6, ARC-Challenge from 43.3 to 34.7, and PIQA from 78.1 to 72.6. The text acknowledges this gap, but the abstract and conclusion present SHARP as generally 'recovering model performance' and accelerating inference without this scope qualifier. The central claim should be stated more precisely: recovery is demonstrated for perplexity and for several knowledge-memorization/commonsense tasks, while complex reasoning is not well recovered at the tested data budget. The paper should either add a prominent limitation statement or temper the general claim.","section":"Section 3.4 and Table 6"},{"comment":"All quantitative results appear to come from single runs, with no standard deviations, confidence intervals, or seeds reported. This is particularly important in Table 2, where the claimed recovery gaps are small (e.g., 3.2 vs 3.0 for Tnext on Arxiv-math), and in Table 6, where several differences are within a few points. Without repeated runs, it is difficult to know whether the ranking of replacement types in Table 4 or the comparison with baselines in Table 2 is statistically meaningful. Please report mean and variance across at least three seeds for the main tables, or provide a justification for why run-to-run variance is negligible.","section":"Section 3.1 and Tables 2, 4, 6"},{"comment":"The in-distribution perplexity evaluation is performed on a held-out 1% slice of the same dataset used for fine-tuning, so the primary evidence is partly self-referential. The independent evidence in Appendix D.1 (SHARP trained on GPT4-Alpaca and evaluated on four other tasks) and in Table 10 (LLaMA3.2-3B) is important and should be promoted into the main text, since it addresses the concern that the recovery could be specific to the fine-tuning distribution. At minimum, the main text should state clearly that the headline 'in-distribution' numbers are same-distribution held-out perplexity, while cross-distribution results appear only in the appendix.","section":"Section 3.1 and Appendix D.1"}],"minor_comments":[{"comment":"The title as rendered contains spacing artifacts ('A CCELERATING', 'SH ARING'); these should be corrected in the camera-ready version.","section":"Abstract and title"},{"comment":"The caption refers to a 'dotted line' for the original-model perplexity, but the text elsewhere refers to a 'dash line'; please make the line-style references consistent.","section":"Figure 2"},{"comment":"Appendix C.3 says the models use both 8-bit dynamic quantization and 4-bit weight quantization, while Table 7 labels them as 4-bit; please specify the final deployed precision unambiguously.","section":"Section 3.5 and Appendix C.3"},{"comment":"The rank values for g1, g2, and g3 are chosen to match total parameter counts, but the parameter-count formulas for these transformations are not given; please add them so the reader can verify the matching.","section":"Table 5"},{"comment":"The y-axis is defined as 'Baseline - Modified Model', so a larger value indicates a larger performance drop; the caption statement that 'the lower the value the better' appears to reverse this and should be corrected.","section":"Figure 4"},{"comment":"The term 'warmup' is used both for the SLW stage and for the learning-rate warmup within SFT; consider using a different name for one of them to avoid confusion.","section":"Section 2.2.2"}],"recommendation":"major_revision","confidential_remarks":"The central concern is the mismatch between the headline mobile benchmark and the SHARP method as defined; this is a load-bearing issue for the paper's practical claims. If the benchmark cannot be redone, the abstract and conclusion should be revised to present the numbers as a direct-sharing upper bound. The perplexity-recovery evidence is otherwise reasonable and potentially publishable after the above issues are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the quick take: SHARP is a real post-training compression recipe—share adjacent MLP layers with LoRA adapters, warm up with an L2 output-matching step, then fine-tune—and the perplexity recovery results are good enough to take seriously. But the headline mobile number, 42.2% time and 42.8% storage savings, is measured on a stripped-down variant with the LoRA parameters removed (Appendix C.3). That's not the method evaluated in the rest of the paper, so the practical claim as written is unsupported.\n\nWhat's new: applying MobileLLM-style adjacent-layer sharing post-hoc to a pretrained 7B model, with low-rank recovery parameters and a two-stage warmup+finetune pipeline. That combination isn't in the prior work I know. The ablation story is genuinely useful: back-of-model replacement beats front, and different low-rank parameterizations tie when parameter-matched. They also check cross-dataset recovery and include LLaMA3.2-3B, which is more than many compression papers do.\n\nSoft spots, in proportion: the mobile benchmark issue above. Credit to them for disclosing it in the appendix, but the abstract and Section 3.5 present the savings without the caveat. The in-distribution perplexity eval holds out only 1% of the same data used for training, so it's partly self-referential; the cross-dataset numbers help, but I'd like a truer held-out distribution. Single runs, no error bars, no code. That's typical for this area, but it caps how much I'd trust the exact figures. And on downstream tasks, reasoning capability drops a lot (GSM8k 14.2 to 3.6). The paper is honest about this, but 'recovering the model' overstates it; they recover perplexity, not full performance.\n\nWho's it for: anyone working on on-device LLM inference or structural compression. I'd send it to a serious referee; it deserves careful review, and the obvious request would be to benchmark the actual deployed SHARP with adapters, run multiple seeds, and release code. I wouldn't desk-reject.","headline":"Useful layer-sharing recovery recipe with credible perplexity results, but the mobile savings headline is measured on a variant without the recovery parameters and needs re-measurement.","tokens_in":27970,"tokens_out":3220,"would_cite":false,"duration_ms":28865,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SHARP keeps Llama2-7b near its original perplexity while sharing MLP layers and cutting mobile inference time by 42%.","keywords":["layer sharing","low-rank adaptation","model compression","LLM inference acceleration","mobile deployment","perplexity recovery","structural pruning","LoRA"],"falsifier":"Run direct adjacent-layer replacement on Llama2-7b and measure Arxiv-math perplexity: if replacing a middle MLP layer (say layer 15 with layer 14) raises perplexity from the 3.0 baseline by more than about 10%, the redundancy premise SHARP builds on is absent. A second decisive check is to run SHARP's Tnext schedule at rank 400 on a larger model with the 50k-example cap: the method's practical claim collapses if the required rank or data budget grows substantially with model scale.","tokens_in":26956,"feed_emoji":"📱","tokens_out":8589,"duration_ms":66326,"temperature":0.7,"pith_summary":"SHARP claims that a pretrained large language model can be compressed for inference by replacing many MLP layers with one reference layer plus small low-rank recovery parameters, without retraining from scratch. The authors show that adjacent MLP layers in Llama2-7b produce similar outputs even though their weights differ, and that a two-stage recovery -- a single-layer output-matching warmup followed by supervised fine-tuning -- brings perplexity back near the original using at most 50k examples. The payoff is concrete: 38--65% fewer stored MLP parameters and, on a mobile phone, 42.8% less storage and 42.2% less total inference time. The paper argues this makes layer-sharing practical for deploying existing LLMs in resource-constrained devices with only fine-tuning-scale data.","feed_headline":"Sharing adjacent layers slashes LLM inference time by 42%","feed_subtitle":"One MLP layer plus small low-rank adapters restores perplexity with at most 50k fine-tuning examples.","key_machinery":"The central object is the layer-sharing-with-recovery-parameters scheme: a stored reference layer $\\Theta_j$ is loaded once, and each target layer $l$ is recreated on the fly as $g(\\Theta_j, \\Delta\\Theta_l)$, where $g$ is a low-rank transformation such as LoRA addition $g_0(\\Theta_j,(\\alpha,A_l,B_l)) = \\alpha\\Theta_j + A_lB_l$; only reference layers plus small adapters are stored. The argument is carried by the compression identity $s = (32-X)/32 + (X/32)(4096r+11008r)/(4096\\times 11008) \\approx 1 - X/32 + X\\cdot r\\cdot 10^{-5}$ for Llama2-7b, and by the two-stage recovery procedure: SLW fits each adapter independently by minimizing $\\|f(X;g(\\Theta_j,\\Delta\\Theta_l))-f(X;\\Theta_l)\\|_2^2$ on sampled activations, giving a good initialization for the joint SFT stage that restores final model behavior.","core_discovery":"On its own terms, the paper's central discovery is that an LLM's MLP stack has a built-in redundancy that can be exploited after training: the current MLP layer can approximate a later MLP layer surprisingly well simply by being reused, and adding low-rank adapters (e.g., a LoRA-style update $g_0(\\Theta_j, (\\alpha, A, B)) = \\alpha\\Theta_j + AB$) nearly closes the residual gap. The paper establishes this by direct replacement experiments -- perplexity barely moves when a neighbor layer is substituted in most positions -- then shows that a two-stage recovery, SLW and SFT, recovers in-distribution perplexity with no more than 50k examples while storing only a fraction of the MLP weights. SLW fits each adapter to reproduce the original layer's output under $L_2$ loss on a 10% activation sample, and SFT jointly fine-tunes all adapters. Ablations indicate that later layers can be replaced more aggressively than early layers, and that different low-rank parameterizations behave similarly at matched parameter counts. The paper claims that Llama2-7b under the one-for-one adjacent replacement schedule called Tnext saves 42.8% model storage and 42.2% wall-clock inference time on a mobile benchmark.","pith_inferences":["Not claimed by the paper but directly testable: if the redundancy is a general property of Transformer MLPs, the same SLW-plus-SFT recipe should transfer to larger pretrained models with rank-400 adapters and a 50k-example budget; a negative result on a much larger model would bound the method's scope to the 3B-7B class studied here.","The layer-sensitivity ablation suggests a task-aware sharing policy: replace later layers that the target benchmark barely uses, and keep the layers that knowledge-memorization tasks depend on, which could push compression beyond the uniform Tnext schedule.","A natural extension left implicit is input-dependent sharing, where the number of reused layers is decided per token or per prompt, potentially recovering even more latency on easy inputs without sacrificing rare reasoning cases."],"forward_implications":["With the Tnext schedule (stored ratio 56%), SHARP closes most of the perplexity gap across Arxiv-math, DialogSum, GPT4-Alpaca, Dolly, and OpenOrca using at most 50k fine-tuning examples, so a single post-training pass can cut stored MLP parameters by 38%.","More aggressive schedules such as Tmore (storing only 8 of 32 layers) still recover perplexity to a small gap, suggesting the sharing limit is well beyond one-for-one layer replacement.","Because the original weights stay frozen and only low-rank adapters are trained, SHARP composes with 4-bit quantization at roughly a 1% performance drop, so further compression can be stacked on top.","The ablation result that Tback beats Tfront at matched storage means replacement should be concentrated in later layers, giving deployers a positional rule for where to share."],"supporting_citations":[{"why":"Supplies the adjacent-layer weight-sharing idea for small models and the motivation that memory loading dominates on-device latency.","marker":"Liu et al., 2024"},{"why":"Supplies the observation that consecutive layers have high representation similarity, which is the redundancy premise SHARP exploits.","marker":"Liu et al., 2023b"},{"why":"Supplies the LoRA low-rank parameterization used as the recovery transformation g0.","marker":"Hu et al., 2021"},{"why":"Provides the Llama2-7b model on which the main experiments are run.","marker":"Touvron et al., 2023"},{"why":"Provides the LayerPruning baseline whose schedules are adapted for comparison in in-distribution recovery.","marker":"Gromov et al., 2024"},{"why":"Provides the LLM-Pruner structural pruning baseline used for comparison.","marker":"Ma et al., 2023"},{"why":"Provides the Arxiv-math instruction dataset used for SLW activation sampling, SFT, and perplexity evaluation.","marker":"Kenny, 2023"},{"why":"Provides the Open-Instruct pipeline followed for the SFT stage.","marker":"Wang et al., 2023"}],"fun_headline_variants":["Sharing adjacent layers cuts LLM inference time by 42%","SHARP: share MLP layers, add adapters, slash 42% inference time","42% faster LLM inference by sharing adjacent layers","Retire 38-65% of MLP weights, recover perplexity with 50k examples","LLM layer sharing plus low-rank adapters: 42% faster on mobile"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a later MLP layer's function can be captured by an earlier layer's frozen weights plus low-rank adapters of rank 400, and that a per-layer output-matching warmup on only 10% of the activation data is a sufficient starting point for the joint fine-tune to restore the model.","fun_headline_variants_meta":{"raw":{"variants":["Sharing adjacent layers cuts LLM inference time by 42%","SHARP: share MLP layers, add adapters, slash 42% inference time","42% faster LLM inference by sharing adjacent layers","Retire 38-65% of MLP weights, recover perplexity with 50k examples","LLM layer sharing plus low-rank adapters: 42% faster on mobile"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000514,"raw_usage":{"total_tokens":2570,"prompt_tokens":1094,"completion_tokens":1476,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":710,"completion_tokens_details":{"reasoning_tokens":1373}},"tokens_in":710,"tokens_out":1476,"duration_ms":10135,"temperature":1.0,"reasoning_tokens":1373,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T13:39:30.946007+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run direct adjacent-layer replacement on Llama2-7b and measure Arxiv-math perplexity: if replacing a middle MLP layer (say layer 15 with layer 14) raises perplexity from the 3.0 baseline by more than about 10%, the redundancy premise SHARP builds on is absent. A second decisive check is to run SHARP's Tnext schedule at rank 400 on a larger model with the 50k-example cap: the method's practical claim collapses if the required rank or data budget grows substantially with model scale.","supporting_citations":[{"cited_title":"arxiv-math-instruct-50","cited_arxiv_id":null,"evidence_quote":"Provides the Arxiv-math instruction dataset used for SLW activation sampling, SFT, and perplexity evaluation."}],"review_version":1}