{"id":"136ac1bf-f72f-4481-a9d5-6a4a3c6b437f","arxiv_id":"2607.07557","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"Activation-percentile-based per-layer sparsity allocation improves LLaMA-2-7B pruning by 15% perplexity over uniform Wanda, while gradient-based allocation fails worse than random.","lead":"PALS adjusts per-layer sparsity in one-shot LLM pruning using the 99th percentile of activation magnitudes, reducing LLaMA-2-7B perplexity from 12.92 to 10.96 at 50% sparsity. A smart generalist might read it for a cheap, drop-in improvement to LLM compression—though the benefit is architecture-dependent and absent on newer models.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The 9-run LLaMA-2 result is statistically sound, but the variance estimate only covers tie-breaking noise with a fixed calibration set, and all hyperparameter choices rest on single-run ablations using that same set — the method's benefit could be calibration-specific rather than architecture-specif","rationale":"The reader correctly identified the core concern: hyperparameters validated on single runs of the same model used for the main result, combined with poor cross-model generalization. I sharpen this by pointing out that the variance estimate itself is incomplete — the 9-run std of 0.59 only reflects tie-breaking noise with a fixed calibration set, so even the main statistical claim rests on a narrower variance estimate than it appears. The ablation tables inherit this same confound. However, this does not change the verdict. The LLaMA-2-7B result is still statistically significant for what it measures, the method is clearly described and cheap to adopt, and the authors are commendably honest about limitations. CONDITIONAL remains appropriate: the result is promising but needs calibration-robustness validation and multi-run ablations before being considered settled. The reader's assessment of novelty (5.0) and the three conditions for conditional acceptance are all well-placed. No adjustment needed.","tokens_in":10243,"tokens_out":3593,"duration_ms":260893,"concrete_test":"Re-run the α ∈ {0.03, 0.05, 0.07} ablation on LLaMA-2-7B with 5 different calibration draws from C4 (each 128 samples × 2048 tokens), 3 tie-breaking seeds per draw. Compute the per-α mean and std including calibration variance. If (a) the optimal α changes across calibration draws, or (b) the PALS-vs-Wanda gap at α=0.05 drops below 2× the combined std (i.e., the effect becomes marginal when calibration noise is included), the hyperparameter choices and possibly the main result are calibration-dependent rather than robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central statistical claim (t(14)=8.1, p<0.001) is correctly computed and well-powered for what it measures. But what it measures is narrow: the 9 PALS runs and 8 Wanda runs all use the same 128-sample calibration draw from C4, varying only the tie-breaking seed. The reported std of 0.59 therefore captures only tie-breaking variance, not calibration variance. Every ablation table (Tables 5–8) that justifies the choices of α=0.05, 99th percentile, and ±5% bounds uses single runs on this same fixed calibration set. If this particular calibration draw happens to produce activation percentile rankings that favor PALS's allocation scheme, the ablation tables would not reveal it — they all share the same confound. The cross-model results are consistent with this concern: PALS shows zero benefit on Mistral-7B and 0.3% on LLaMA-3-8B, which the authors attribute to architectural differences. An equally plausible explanation is that the hyperparameters are tuned to LLaMA-2-7B's activation distribution under this specific calibration set. Table 5 illustrates the sensitivity: α=0.01 gives 14.23 (worse than uniform Wanda at 12.92), a non-monotonicity the authors attribute to seed noise but cannot confirm without multi-run data. If even small changes in α can produce results worse than the baseline, the method operates in a sensitive regime where calibration-set dependence could matter substantially. The authors acknowledge this gap in §6.6 ('a proper calibration bootstrap is worth doing in future work') but do not quantify it, and the entire hyperparameter selection process depends on it.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"This paper proposes PALS (Percentile-Aware Layerwise Sparsity), a method that allocates per-layer sparsity ratios in one-shot LLM pruning based on the 99th percentile of activation magnitudes, bounded to ±5% around the target ratio. The method plugs into Wanda's existing pipeline with negligible overhead. On LLaMA-2-7B at 50% sparsity, PALS reduces WikiText-2 perplexity from 12.92 to 10.96 (mean over 9 runs, Welch's t(14)=8.1, p<0.001). The benefit is architecture-dependent: marginal on LLaMA-3-8B and absent on Mistral-7B. The paper also reports that gradient-based allocation produces results worse than random, a notable negative finding.","tokens_in":10471,"tokens_out":2334,"duration_ms":178372,"significance":"The paper addresses a clear gap in one-shot LLM pruning: existing methods (Wanda, SparseGPT) apply uniform per-layer sparsity, and the question of whether non-uniform allocation helps is natural and practically relevant. The primary LLaMA-2-7B result is well-powered (9 runs, proper Welch's t-test, calibration set separated from evaluation set). The gradient-failure finding (Table 8) is a genuinely interesting negative result that challenges a common assumption. The authors are commendably transparent about limitations, including the architecture-dependent gains and the single-run ablation tables. The method is simple, practical, and falsifiable.","major_comments":[{"comment":"§6.2, Table 5: All hyperparameter ablations (α, percentile, bound width) use single runs on a fixed calibration draw, while the primary result uses 9 runs. This is load-bearing because the specific choices of α=0.05, 99th percentile, and ±5% bounds are validated only by these single-run tables. Table 5 illustrates the problem: α=0.07 gives 10.12 PPL (single run) versus α=0.05's 10.96 (9-run mean), and α=0.01 gives 14.23 (worse than the 12.92 baseline). The authors attribute these to seed noise but cannot confirm without multi-run data. At minimum, the key settings bracketing the chosen α (e.g., α=0.03, 0.05, 0.07) should be re-run with multiple seeds to verify that the chosen operating point is not an artifact of the single calibration draw.","section":null},{"comment":"§6.6: The 9-run variance estimate (std=0.59) captures only tie-breaking noise with a fixed 128-sample calibration draw from C4. Calibration variance is not measured. The authors acknowledge this gap ('a proper calibration bootstrap is worth doing in future work') but it is central to the method's robustness claim. Since PALS's allocation depends entirely on per-layer activation percentiles from this calibration set, a sensitivity check across at least 2–3 different C4 draws would substantially strengthen the claim that the improvement is not calibration-specific. This is feasible within the paper's experimental scope.","section":null},{"comment":"§5.2, Table 2: PALS shows zero benefit on Mistral-7B (6.31 vs. 6.31) and only 0.3% on LLaMA-3-8B. The authors attribute this to architectural differences (§7.2), but an equally plausible explanation is that the hyperparameters are tuned to LLaMA-2-7B's activation distribution under the specific calibration set. Without either (a) multi-run evaluation on the other two models or (b) calibration sensitivity analysis on LLaMA-2, the paper cannot distinguish between 'architecture-specific benefit' and 'calibration-specific overfitting.' This matters for the paper's central practical claim that PALS is a drop-in improvement to Wanda.","section":null}],"minor_comments":[{"comment":"Figures 1 and 2 use the label 'HALO-v2' instead of 'PALS,' which appears to be a leftover from a previous version of the manuscript.","section":null},{"comment":"Table 7: The text states 'the 99th percentile (10.96) is not the best in this table,' but Table 7 does not include the 9-run mean for the 99th percentile—it lists 10.96, which is the 9-run value from Table 1. The other entries are single runs. The comparison is therefore between quantities with different variance estimates, which should be noted.","section":null},{"comment":"§6.5, Table 8: The gradient-failure result is striking but also based on single runs. A brief note confirming that the qualitative ordering (gradient worse than random worse than activation) is stable across at least 2 seeds would strengthen this finding.","section":null},{"comment":"§3.2, Eq. (3): The clip bounds in Eq. (4) are written as absolute values (s_target ± 0.05), but α=0.07 in Table 5 is labeled '(clipped).' It would help to clarify how often clipping occurs at the chosen α=0.05 setting, perhaps by reporting the fraction of layers at the bounds.","section":null},{"comment":"The abstract states 'PALS achieves 10.96 WikiText-2 perplexity versus 12.92 for uniform Wanda' without noting that this is specific to LLaMA-2-7B. Adding 'on LLaMA-2-7B' would prevent misreading.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper is honest to a fault—the authors flag most of the issues I'm raising themselves. The central LLaMA-2-7B result is statistically sound and the method is simple and practical. However, the combination of single-run ablations, fixed calibration set, and near-zero cross-model transfer means the paper cannot currently distinguish 'PALS helps when architectures have heterogeneous layers' from 'PALS is tuned to one calibration draw of one model.' A small set of additional experiments (multi-seed ablations at 2-3 α values, 2-3 calibration draws) would resolve this and likely bring the paper to acceptance. The 'HALO-v2' label in the figures suggests the method was renamed; the authors should verify consistency throughout."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. The referee correctly identifies that our single-run ablation tables and lack of calibration sensitivity analysis are the main weaknesses of the manuscript. We agree with all three major comments and will address them in revision.","responses":[{"response":"The referee is correct. The single-run ablation tables are insufficient to justify the chosen operating point, and the non-monotonicity at α=0.01 (14.23, worse than uniform) is exactly the kind of anomaly that requires multi-run verification. We will re-run α=0.01, 0.03, 0.05, and 0.07 with at least 5 seeds each (matching the primary result's methodology) and update Table 5 with means and standard deviations. If the non-monotonicity at α=0.01 persists under multi-run evaluation, we will discuss it as a genuine finding; if it washes out, we will note it was seed noise as we originally suspected. We will also apply the same multi-run treatment to the percentile ablation (Table 7) and bound-width ablation (Table 6) for the settings immediately adjacent to our chosen configuration.","revision_made":"yes","referee_comment":"§6.2, Table 5: All hyperparameter ablations use single runs on a fixed calibration draw, while the primary result uses 9 runs. The specific choices of α=0.05, 99th percentile, and ±5% bounds are validated only by these single-run tables. α=0.07 gives 10.12 PPL (single run) versus α=0.05's 10.96 (9-run mean), and α=0.01 gives 14.23 (worse than the 12.92 baseline). At minimum, the key settings bracketing the chosen α (e.g., α=0.03, 0.05, 0.07) should be re-run with multiple seeds."},{"response":"We agree that calibration sensitivity is central to the robustness claim and that our current indirect argument (order-statistic convergence over ~262K tokens) is not a substitute for empirical measurement. We will run PALS on LLaMA-2-7B with 3 independent C4 calibration draws (each 128 samples of 2048 tokens), with multiple seeds per draw, and report the resulting perplexity distribution. This will allow us to decompose variance into tie-breaking noise and calibration noise. We will update §6.6 with these results and revise the discussion accordingly. If calibration variance is non-trivial relative to the improvement margin, we will state this explicitly and discuss implications for practical deployment.","revision_made":"yes","referee_comment":"§6.6: The 9-run variance estimate (std=0.59) captures only tie-breaking noise with a fixed 128-sample calibration draw from C4. Calibration variance is not measured. Since PALS's allocation depends entirely on per-layer activation percentiles from this calibration set, a sensitivity check across at least 2–3 different C4 draws would substantially strengthen the claim that the improvement is not calibration-specific."},{"response":"This is a fair concern. The combination of single-run evaluation on LLaMA-3 and Mistral, single calibration draw, and single-run ablations means we cannot currently rule out calibration overfitting as an alternative explanation for the architecture-dependent results. We will address this on two fronts. First, the calibration sensitivity analysis on LLaMA-2 (from our response to the second comment) will directly test whether the LLaMA-2 improvement is calibration-specific. Second, we will run multi-seed evaluation (at least 5 seeds) on LLaMA-3-8B and Mistral-7B to determine whether the near-zero and exactly-zero improvements are stable or whether they reflect noise. We will also run at least one alternative calibration draw on LLaMA-3 and Mistral to check whether PALS produces different allocations. If the zero-gain result on Mistral is stable across seeds and calibration draws, the architecture-specific explanation is strengthened; if it shifts, we will revise our claims accordingly. We will also temper the 'drop-in improvement' framing in the abstract and conclusion to make clear that PALS should be validated per-model, consistent with the architecture-dependent results we already report.","revision_made":"yes","referee_comment":"§5.2, Table 2: PALS shows zero benefit on Mistral-7B (6.31 vs. 6.31) and only 0.3% on LLaMA-3-8B. Without either (a) multi-run evaluation on the other two models or (b) calibration sensitivity analysis on LLaMA-2, the paper cannot distinguish between 'architecture-specific benefit' and 'calibration-specific overfitting.' This matters for the paper's central practical claim that PALS is a drop-in improvement to Wanda."}],"tokens_in":10092,"tokens_out":1032,"duration_ms":191665,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"PALS is a simple, cheap modification to Wanda: use the 99th percentile of per-layer activation magnitudes to shift each layer's sparsity ratio within ±5% of the target. The main LLaMA-2-7B result is real — 12.92 → 10.96 WikiText-2 perplexity, 9 runs, Welch's t(14)=8.1, p<0.001. The method slots into Wanda's existing pipeline with negligible overhead. That's the headline, and it holds up.","headline":"PALS: activation-percentile layer sparsity for Wanda — solid LLaMA-2 result, honest about where it fails, but ablations underpowered","tokens_in":11078,"tokens_out":771,"would_cite":false,"duration_ms":37881,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Activation outliers, not gradients, guide better LLM pruning","keywords":["LLM pruning","layer-wise sparsity","activation outliers","one-shot pruning","Wanda","gradient-based pruning","model compression","transformer layers"],"falsifier":"If PALS with the same hyperparameters fails to improve over uniform Wanda on models beyond LLaMA-2-7B — as it already does on Mistral-7B — the method may be capturing a property specific to LLaMA-2's layer heterogeneity rather than a general principle of layer-adaptive pruning.","tokens_in":10280,"feed_emoji":"✂️","tokens_out":1460,"duration_ms":164295,"temperature":0.7,"pith_summary":"This paper argues that one-shot LLM pruning should not treat all transformer layers equally. Standard methods like Wanda and SparseGPT apply the same sparsity ratio to every layer, ignoring that some layers carry more critical information than others. PALS (Percentile-Aware Layerwise Sparsity) computes the 99th percentile of activation magnitudes for each layer as a proxy for importance, then shifts each layer's sparsity target up or down within a ±5% band around the global target. Layers with large activation outliers — known to be critical for model function — get pruned less; layers with uniformly small activations get pruned more. On LLaMA-2-7B at 50% sparsity, this reduces WikiText-2 perplexity from 12.92 to 10.96 (mean over 9 runs, p<0.001). The benefit is architecture-dependent: LLaMA-3-8B gains marginally and Mistral-7B gains nothing, which the authors attribute to newer models distributing computation more uniformly across layers. A secondary and arguably more provocative finding is that gradient-based layer importance metrics — the seemingly principled approach — produce sparsity allocations worse than random, suggesting that gradient magnitude does not predict the damage caused by removing a large fraction of weights.","feed_headline":"Activation outliers beat gradients for LLM layer pruning","feed_subtitle":"Varying per-layer sparsity by activation percentiles cuts LLaMA-2-7B perplexity 15% at 50% sparsity, while gradient-based allocation fails.","key_machinery":"PALS replaces uniform per-layer sparsity with a three-step mechanism: (1) compute the 99th percentile of absolute activations for each layer from a small calibration set, (2) standardize these scores and use them to shift each layer's sparsity target via a linear formula with a ±5% clip, (3) apply Wanda's existing weight scoring within each layer's adjusted budget. The entire procedure adds negligible cost because Wanda already collects the activations.","core_discovery":"The central object is the 99th percentile of activation magnitudes as a layer importance signal. The paper discovers that this tail statistic, when used to reallocate sparsity budgets across transformer layers within tight bounds (±5%), yields substantial perplexity improvements on LLaMA-2-7B while gradient-based importance metrics fail catastrophically. The failure of gradients is particularly striking: gradient-norm allocation produces 47.3 perplexity versus 12.9 for uniform allocation and 24.4 for random allocation, indicating that the relationship between infinitesimal loss sensitivity and discrete weight removal is fundamentally broken in pretrained LLMs.","pith_inferences":["If the benefit of PALS depends on layer heterogeneity, then larger or older models with more uneven layer utilization should benefit more, while well-trained or architecturally uniform models should not — this is testable by measuring activation percentile variance across layers as a predictor of PALS benefit.","The clustering of results at α≥0.07 (where bounds clip most layers to 45% or 55%) suggests that a simpler binary allocation scheme might work nearly as well, which would be cheaper to implement and easier to analyze.","The gradient failure could extend beyond sparsity allocation to other compression decisions (quantization bit allocation, rank selection in low-rank approximation) where gradient sensitivity is used as a proxy for the impact of discrete capacity reduction.","The connection between activation outliers and layer importance predicts that models where outlier dimensions are concentrated in specific layers will benefit more from non-uniform pruning than models where outliers are spread evenly."],"forward_implications":["If activation outliers are the right signal for layer importance in pruning, then models designed to have more uniform outlier distributions across layers (like Mistral) may be inherently harder to prune non-uniformly — a property that could become a design criterion for future architectures.","The failure of gradient-based allocation challenges the widespread assumption that gradient sensitivity is a good proxy for pruning damage, at least for large discrete weight removal in pretrained models.","The ±5% bound being critical suggests that information bottlenecks in even a single layer can cascade through the entire network, which has implications for any compression method that varies capacity across layers.","The U-shaped sparsity pattern (less pruning in early and late layers, more in middle layers) aligns with functional accounts of transformer layer roles and could inform other layer-adaptive interventions like quantization or distillation."],"fun_headline_variants":["Activation tails, not gradients, guide layerwise LLM sparsity","99th percentile activations beat gradient norms for pruning LLaMA-2","Gradient-based layer sparsity allocation fails; activation percentiles work","Tail activation statistics outperform gradients for LLM pruning budgets","Per-layer activation percentiles cut LLaMA-2 perplexity where gradients fail"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The specific hyperparameter choices (α=0.05, ±5% bounds, 99th percentile) are validated on the same LLaMA-2-7B results used to evaluate the method, and the alternatives that appear better in single-run ablations are dismissed as within run-to-run noise. The central LLaMA-2 result is well-powered, but the specific configuration that produces it is not independently validated, and the method shows no benefit on Mistral-7B and marginal benefit on LLaMA-3-8B.","fun_headline_variants_meta":{"raw":{"variants":["Activation tails, not gradients, guide layerwise LLM sparsity","99th percentile activations beat gradient norms for pruning LLaMA-2","Gradient-based layer sparsity allocation fails; activation percentiles work","Tail activation statistics outperform gradients for LLM pruning budgets","Per-layer activation percentiles cut LLaMA-2 perplexity where gradients fail"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":636,"prompt_tokens":545,"completion_tokens":91,"prompt_tokens_details":null},"tokens_in":545,"tokens_out":91,"duration_ms":85435,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T06:59:32.757309+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If PALS with the same hyperparameters fails to improve over uniform Wanda on models beyond LLaMA-2-7B — as it already does on Mistral-7B — the method may be capturing a property specific to LLaMA-2's layer heterogeneity rather than a general principle of layer-adaptive pruning.","supporting_citations":[],"review_version":1}