{"id":"7fdf192f-183e-4534-9c72-828243c059b6","arxiv_id":"2607.21291","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"AdaDSF uses per-layer cosine similarity to decide which tokens skip which layers, then distills the sparse model back toward the dense one, cutting FLOPs while holding accuracy close to dense.","lead":"Large language models are expensive to run because every token passes through every layer. This paper reports a method, AdaDSF, that lets some tokens skip some layers based on how much each layer transforms the input, and matches the sparse model to the dense one with an alignment loss. The authors report near-dense accuracy on GPT-NeoX and Qwen2.5 at up to ~30% lower FLOPs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1b/2 contradict the abstract's 'consistently yields smaller accuracy degradation': at 90% retention DLO has lower PPL (18.3 vs 18.5) and smaller accuracy drop (-2.2 vs -2.3), so the central comparative claim is overbroad.","rationale":"The reader's verdict is CONDITIONAL, and I agree. My stress-test focuses on the abstract's universal comparative claim. The strongest claim includes 'consistently yields smaller accuracy degradation than ... DLO'. The paper's own Table 2 (90% retention) shows DLO PPL 18.3 vs AdaDSF 18.5, and Table 1b shows DLO Diff -2.2 vs AdaDSF -2.3. These are direct counterexamples. The reader noted this as an overclaim but selected the unvalidated cosine-similarity proxy as the weakest assumption. I think the internal inconsistency is more load-bearing because it is a factual contradiction of the central claim, not a missing justification; it can be settled by simply reading the tables. The cosine-similarity premise is important, but even if it were validated, the abstract would still need revision; the empirical contradiction is dispositive for the claim as stated. A multi-seed paired comparison at 90% retention would determine if the contradiction is real or noise. I recommend keeping CONDITIONAL (UNCHANGED), because the core method at 70–80% retention appears to work and the overclaim can be fixed by qualification and variance reporting; rejection is too harsh.","tokens_in":8301,"tokens_out":6846,"duration_ms":67904,"concrete_test":"Run a paired, multi-seed evaluation (≥5 seeds) of AdaDSF and DLO at 90% retention on GPT-NeoX-130M/Wikitext103 and on Qwen2.5-0.5B commonsense reasoning, reporting means and 95% confidence intervals for PPL/accuracy and FLOPs. If the 90% deficit persists (DLO equal or better on both metrics), the abstract's 'consistently' must be revised. If the deficit vanishes under variance, the claim must be re-reported with error bars. Either outcome settles whether the overclaim is real.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that AdaDSF 'consistently yields smaller accuracy degradation than ... DLO' under comparable sparsity is directly contradicted by the paper's own tables at the 90% retention setting. Table 2 (GPT-NeoX, Wikitext103): at 90% retention, DLO achieves PPL 18.3 vs AdaDSF 18.5 (dense 17.9), so DLO's degradation is smaller; AdaDSF is better only at 80% and 70%. Table 1b (Qwen2.5-0.5B, reasoning): at 90%, DLO has average Diff -2.2 vs AdaDSF -2.3, so AdaDSF's accuracy degradation is slightly larger. The abstract and Section 4.2 assert 'consistent' superiority, which these numbers do not support at the highest retention ratio.\n\nThis is not merely a wording issue: the 90% case is the regime closest to the dense model and the most likely deployment setting. The contradiction also signals a reporting gap: no error bars or multiple seeds are provided, so one cannot determine whether the 0.1–0.2 point differences are noise. If 'comparable sparsity' is instead intended as matched FLOPs, the 90% DLO row (FLOPs 1.110) is not a valid sparse comparison, and the tables would need to be recomputed at matched FLOPs. Either way, the stated central claim is not established by the reported evidence.\n\nThe 80% and 70% results are plausible, and the alignment ablation (Table 4) helps, but the headline comparison to DLO must be qualified to lower retention ratios or matched-FLOP settings, and supported with variance estimates.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AdaDSF, a post-hoc depth-sparsification framework for pre-trained LLMs. It uses the cosine similarity between a layer's input and output hidden states (Eq. 4) to derive per-layer token retention ratios through temperature-normalized weighting, deviation scaling, sigmoid mapping, and global budget correction (Eqs. 5–8). A lightweight MLP router selects the top-K tokens for each layer; unselected tokens skip the Transformer block via the residual path. A feature-preserving alignment objective (Eq. 11), combining output-distribution KL and intermediate hidden-state alignment, is used to train the router and layers. Experiments on GPT-NeoX-130M and Qwen2.5-0.5B/1.5B report language-modeling PPL and six commonsense-reasoning accuracies against MoD, D-LLM, and DLO, together with normalized FLOPs. The central claim is that AdaDSF substantially reduces inference FLOPs while preserving dense-level performance and consistently outperforms strong depth-sparse baselines under comparable sparsity. The manuscript does not include code, error bars, or a validation protocol for hyperparameter selection.","tokens_in":8700,"tokens_out":8818,"duration_ms":89409,"significance":"If the claims are reproducible, the contribution is practically attractive: AdaDSF requires no retraining from scratch and no architectural change to attention, and it converts off-the-shelf checkpoints into depth-sparse models with a simple similarity-derived allocation. The method is easy to state, and the alignment objective is a sensible means of reducing representation shift. The empirical scope is reasonable for a short paper: two model families, three model sizes, and seven tasks. However, several load-bearing pieces of evidence are currently missing or under-specified: the headline comparison to DLO is contradicted at the 90% retention setting, the temperature is selected on the test set with no seeds, and the core 'similarity implies importance' premise is not directly validated. These issues prevent the paper from establishing its central claims as written, but they are addressable with additional analysis and reporting.","major_comments":[{"comment":"The abstract and §4.2 claim that AdaDSF 'consistently yields smaller accuracy degradation' than DLO under comparable sparsity. This is contradicted at the 90% retention setting. In Table 1b (Qwen2.5-0.5B), DLO has Diff −2.2 vs. AdaDSF −2.3; in Table 2 (GPT-NeoX, Wikitext-103), DLO has PPL 18.3 vs. AdaDSF 18.5. I note that DLO's normalized FLOPs at 90% is 1.110, i.e., it is not a sparse configuration, so the 90% row cannot serve as a 'comparable sparsity' comparison as it stands. The paper should either recompute matched-FLOP comparisons or explicitly restrict the consistency claim to retention ratios 70–80% / FLOPs<1. The current wording is overbroad, and the 0.1–0.2 point differences may be noise given the absence of error bars.","section":"§4.2, Tables 1b and 2"},{"comment":"The reported optimal PPL of 18.91 is obtained by sweeping τ ∈ {0.25, 0.1, 0.05} and selecting the best value. No validation split is described; if the selection is made on the Wikitext-103 test set, this is test-set leakage. Moreover, all numbers appear to come from a single run with no seeds or error bars. Given that several headline differences are 0.1–0.2 points (e.g., DLO vs. AdaDSF at 90%), variance estimates are essential. Please add multi-seed results and a clear train/validation/test protocol, including how τ, β, and the target retention t are chosen.","section":"§4.3, Table 3"},{"comment":"The efficiency metric 'normalized FLOPs' is defined only as 'sparse over dense' in §4.1, with no counting methodology: which operations are counted, how router overhead is handled, and how values such as DLO's 1.110 at 90% retention are computed. Without this, the claim of 'substantially reduces inference FLOPs' is not independently verifiable. Training details (optimizer, learning rate, epochs, batch size, router architecture, calibration-set size) are also absent, preventing reproduction.","section":"§4.1 and all experiments"},{"comment":"The core methodological premise is that input–output cosine similarity (Eq. 4) is a valid importance proxy, so that low-similarity layers should retain more tokens. Table 3 only varies τ, and Table 4 varies the loss; neither isolates the allocation mechanism. No comparison with uniform retention, inverse allocation, or random allocation at matched training budget is reported. As a result, the observed gains could be driven chiefly by the alignment objective in Eq. (11), not by similarity-driven allocation. Please add a direct ablation (e.g., uniform vs. similarity-derived ratios with identical alignment and budget) and, ideally, a correlation analysis between layer similarity and measured contribution to task performance.","section":"§3.3–3.5, Tables 3–4"},{"comment":"The D-LLM baseline has PPL ≈1955–2078 on Wikitext-103, three orders of magnitude worse than the dense model, and the text itself says it is 'not suitable' while still using it as a strong baseline. This suggests an implementation or protocol mismatch. If D-LLM cannot be configured successfully for this setting, it should be removed from the comparison; otherwise its inclusion inflates AdaDSF's relative improvements and weakens the 'strong baselines' claim.","section":"§4.2, Table 2"}],"minor_comments":[{"comment":"The symbol L is used both for the total loss and for the number of layers. Rename the total loss to L_total to avoid ambiguity.","section":"Eq. (11)"},{"comment":"The row 'GPT-Neox-AdaDSF (w/o τ)' is ambiguous: does it mean no temperature scaling (i.e., uniform weights), no calibration set, or something else? Please define what is removed.","section":"Table 3"},{"comment":"The overview figure is dense and the flow from similarity statistics to retention ratios to the router is hard to follow. Consider separating the three components or enlarging the diagram.","section":"Fig. 1"},{"comment":"Some related-work references (e.g., [9], [15], [21], [27], [28]) are not clearly connected to depth sparsity or LLM efficiency. Please state their specific relevance or remove them.","section":"§2, References"},{"comment":"The statement that the method is 'inherently scalable' is unsupported by experiments limited to models ≤1.5B. Either add a larger-scale experiment or temper the claim. Also consider adding an availability statement for code.","section":"§4.5"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a simple and potentially useful method, but the empirical validation is not yet at the level required for the central claims. The most serious issues are the DLO 90% contradiction, the test-set-based selection of τ, and the lack of a direct ablation of the similarity-driven allocation. These are fixable with additional experiments and careful re-reporting. I would also ask the editors to verify the relevance of references [9,15,21,27,28], some of which appear unrelated to LLM efficiency; this did not affect my verdict beyond the minor comment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper proposes AdaDSF: a way to make off-the-shelf LLMs depth-sparse by computing per-layer token retention ratios from cosine similarity between layer input and output, routing tokens with an MLP, and training with a feature-alignment loss. That combination is new, and the mechanism is sensible. The 80% and 70% retention results on GPT-NeoX and Qwen2.5 show real gains over MoD and DLO at roughly matched FLOPs, and the ablation shows the alignment loss helps. The method is simple and requires no architectural changes.\n\nThe central problem is the word \"consistently.\" Their own tables contradict it. At 90% retention on Wikitext103, DLO gets PPL 18.3 vs AdaDSF's 18.5 (dense 17.9), so DLO's degradation is smaller. On Qwen2.5-0.5B reasoning, DLO's average drop is -2.2 vs AdaDSF's -2.3 at 90%. That is not \"smaller accuracy degradation\" at the highest retention ratio — the setting closest to deployment. The stress-test note is right. The claim needs to be qualified to lower retention ratios or matched-FLOP comparisons.\n\nThe experimental rigor is also weaker than the claims. No error bars or seeds are reported, so the 0.1–0.2 PPL differences could be noise. The temperature τ is swept and the best value is reported without a described validation split. FLOPs accounting is not detailed. The D-LLM baseline is pathologically bad (PPL ~2000), which undermines the comparison rather than strengthening it. No code or training details are released. The scale (≤1.5B) is fine for a first pass but limits the generality claim.\n\nOn the substance: the cosine-similarity proxy for layer importance is plausible but unvalidated. The paper never shows that low-similarity layers really benefit from more retained tokens; the ablation only varies τ on one test set. So the mechanism behind the allocation is not independently supported, though the distillation objective likely explains much of the gain.\n\nThe paper deserves a serious referee because the idea is relevant and the 80%/70% results are promising. But it needs heavy revision: fix the overclaim, add variance estimates, disclose the selection procedure, and ideally release code. I would not cite it in its current form.","headline":"Plausible depth-sparse method with a real overclaim — the 90% retention results contradict the abstract's 'consistently' lower degradation.","tokens_in":9237,"tokens_out":2396,"would_cite":false,"duration_ms":23122,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AdaDSF converts off-the-shelf pre-trained LLMs into depth-sparse models by allocating compute according to cosine similarity between layer inputs and outputs, cutting inference FLOPs while keeping accuracy close to dense models.","keywords":["adaptive depth sparsity","cosine similarity","token routing","layer-wise retention","efficient LLM inference","knowledge distillation","Mixture-of-Depths","pre-trained LLMs"],"falsifier":"Compute per-layer cosine similarities on the same calibration set, then compare AdaDSF's allocation against an oracle that allocates tokens to maximize held-out accuracy, for example by grid-search or leave-one-layer-out importance. If the similarity-derived ratios do not correlate with oracle-optimal ratios across layers and models, the central allocation claim is unsupported. A simpler check: at fixed FLOPs, run AdaDSF with reversed ratios (more tokens to high-similarity layers) and see whether accuracy changes.","tokens_in":8162,"feed_emoji":"⚡","tokens_out":3503,"duration_ms":34617,"temperature":0.7,"pith_summary":"This paper argues that the compute spent on a Transformer layer should track how much that layer actually transforms its input representations. It proposes AdaDSF, which measures each layer's input-to-output cosine similarity on a calibration set, converts those similarities into per-layer token retention ratios, and uses a lightweight router to send only the most informative tokens through each full layer while the rest skip via residuals. A feature-preserving alignment loss then trains the sparse model to keep hidden states and output distributions close to the dense teacher. The reported result is that on GPT-NeoX-130M, Qwen2.5-0.5B, and Qwen2.5-1.5B, this reduces inference FLOPs to roughly 0.68–0.89x of dense while suffering less accuracy degradation than MoD, D-LLM, and DLO at matched sparsity. If correct, it offers a practical path to faster LLM inference from off-the-shelf checkpoints without redesigning the architecture.","feed_headline":"Cosine similarity routes LLM tokens, cutting FLOPs 20-30%","feed_subtitle":"Layer-level retention ratios from input-output similarity keep accuracy near dense on GPT-NeoX and Qwen2.5 models.","key_machinery":"The central object is the layer-wise cosine similarity s_i = CosSim(x_in^(i), x_out^(i)). It feeds a temperature-normalized softmax, deviation scaling, and sigmoid mapping that yields per-layer retention ratios r_i satisfying sum r_i = tL; these ratios set the number of tokens K_i = floor(r_i * s) each sparse layer's router keeps. The router is a lightweight MLP that generates a differentiable Top-K mask. The alignment objective combines output-distribution KL divergence (Eq. 10) with per-layer hidden-state softmax L2 losses (Eq. 9). The machinery's job is to convert a cheap, parameter-free similarity statistic into a data-driven compute budget.","core_discovery":"AdaDSF's central claim is that representation-transformation statistics—specifically the cosine similarity between a layer's input and output hidden states—provide a usable signal for allocating compute across depth. Layers whose input and output are more different get higher token retention ratios, under a fixed global budget; a trainable MLP router picks the Top-K tokens per layer. Together with hidden-state and output-distribution alignment against the dense model, this converts ordinary pre-trained LLMs into depth-sparse models with minimal architectural change, yielding reported PPL 18.9 on Wikitext103 at 80% retention (vs 17.9 dense) and average commonsense accuracy within about one to","pith_inferences":["The cosine-similarity proxy is only validated indirectly; a reader should check whether the same ratios emerge from an oracle that measures each layer's true contribution, such as ablating layers or measuring downstream loss change. If not, observed gains may be driven chiefly by the alignment objective rather than by the allocation itself.","The method's dependence on a calibration set means its allocation quality could degrade under distribution shift; an online or adaptive estimation of layer similarity is a natural testable extension.","Because experiments stop at 1.5B parameters, the scalability claim is untested at 7B+; whether similarity statistics remain stable at that scale is an open question.","The hidden-state alignment loss (softmax L2) is one particular choice; comparing it against cosine or KL-based intermediate alignment could shift the reported efficiency-accuracy frontier."],"forward_implications":["If similarity-driven allocation is right, any off-the-shelf dense LLM can be made depth-sparse with a light training pass, no full retraining or architecture change.","At 70–90% token retention, inference FLOPs drop to roughly 0.68–0.89x dense while language modeling PPL stays within about two points of dense on the tested models.","Under matched sparsity, the method claims smaller accuracy degradation than MoD, D-LLM, and DLO across six commonsense benchmarks, suggesting the allocation plus alignment objective is doing more work than fixed schedules.","The approach generalizes across model families (GPT-NeoX and Qwen2.5) and across retention ratios, a necessary condition for practical deployment.","Because retention ratios come from a calibration set rather than online adaptation, the method is cheap to apply and does not require task-specific fine-tuning."],"fun_headline_variants":["Cosine similarity sets token budgets per layer, slashing LLM inference costs","Similarity-driven token sparsity trims LLM compute while keeping accuracy","Layer I/O similarity allocates tokens to cut LLM inference FLOPs","AdaDSF: sparse tokens via layer similarity, near-dense accuracy","Similarity-based layer budgets slash LLM inference FLOPs"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that a layer's input-output cosine similarity measures how much compute that layer contributes, so giving more retained tokens to low-similarity layers improves accuracy; the paper does not independently validate this correlation, and if it fails, the allocation is just a fixed heuristic whose main benefit would come from the alignment loss.","fun_headline_variants_meta":{"raw":{"variants":["Cosine similarity sets token budgets per layer, slashing LLM inference costs","Similarity-driven token sparsity trims LLM compute while keeping accuracy","Layer I/O similarity allocates tokens to cut LLM inference FLOPs","AdaDSF: sparse tokens via layer similarity, near-dense accuracy","Similarity-based layer budgets slash LLM inference FLOPs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001162,"raw_usage":{"total_tokens":4640,"prompt_tokens":730,"completion_tokens":3910,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":3815}},"tokens_in":474,"tokens_out":3910,"duration_ms":29447,"temperature":1.0,"reasoning_tokens":3815,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T07:52:28.536318+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute per-layer cosine similarities on the same calibration set, then compare AdaDSF's allocation against an oracle that allocates tokens to maximize held-out accuracy, for example by grid-search or leave-one-layer-out importance. If the similarity-derived ratios do not correlate with oracle-optimal ratios across layers and models, the central allocation claim is unsupported. A simpler check: at fixed FLOPs, run AdaDSF with reversed ratios (more tokens to high-similarity layers) and see whether accuracy changes.","supporting_citations":[],"review_version":1}