{"id":"676ca415-09a2-40a0-97f8-d244a88d025c","arxiv_id":"2607.08027","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Power transform, sign-preserving aggregation and 2% outlier removal let structured AFR match unstructured accuracy and beat prior structured pruners with ~1.57× speedup at 50% FFN sparsity.","lead":"The authors adapt Adaptive Feature Retention (AFR) pruning from unstructured to structured form for LLMs by adding power transformation, sign-preserving aggregation, and percentile outlier removal. The method keeps accuracy near unstructured AFR while delivering real 1.56–1.57× inference speedups on standard hardware.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Sign-preserving aggregation is the least-validated of the three fixes; its causal contribution to the accuracy claims is not isolated.","rationale":"The reader correctly isolates the signed-mean aggregation as the weakest assumption. The other two components have quantitative support (kurtosis/IQR saturation at 2% in Table 2; SNR argument and histograms for power transform). Sign consistency does not. Because the paper never isolates that step, the causal story that distinguishes the method from “just better score cleaning + power transform” remains untested. The concrete ablation above would settle the issue with a single controlled experiment already inside the paper’s evaluation protocol. Until that result is known, CONDITIONAL remains the right verdict: the empirical gains are large and consistent, yet one of the three claimed mechanisms lacks independent evidence. No stronger objection (e.g., data leakage, broken baselines, or non-reproducible speed-ups) appears in the manuscript, so the verdict does not need to move to REJECT or UNVERDICTED.","tokens_in":11101,"tokens_out":657,"duration_ms":40798,"concrete_test":"On Llama-3-8B at 20% and 50% FFN pruning, re-run the full pipeline twice: (A) signed-mean then abs (proposed) and (B) abs-then-mean (Eq. 2), both with identical 2% percentile cleaning and b=2.7 power transform on SNIP. Report the five-benchmark average for A vs B. If |A-B| < 1 point, the sign-preserving step is not load-bearing and the Section 4.2 justification collapses; if A exceeds B by ≥3 points, the assumption is empirically vindicated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim rests on the integrated pipeline (power transform + sign-preserving aggregation + 2% outlier removal) producing structured models whose accuracy is comparable to unstructured AFR and far above naïve averaging / LLM-Pruner / LoRAP / CFSP. Section 4.2 asserts that averaging signed scores then taking absolute value correctly ranks neurons by “optimization direction consistency”: consistent signs → large |mean| → keep; mixed signs → cancellation → prune. This is the only component whose correctness is not independently supported by distribution statistics (Table 2) or SNR analysis (Eq. 4 / Fig. 3). The paper never reports an ablation that replaces only the signed-mean step with the absolute-mean baseline of Eq. 2 while keeping power transform and outlier removal fixed. Consequently it is possible that the large gains over “AFR (naive avg.)” (Tables 3–4) are driven almost entirely by the other two fixes, and that the sign-consistency story is post-hoc. If that is the case, the mechanistic claim that underpins the method’s novelty is unsupported even though the end-to-end numbers remain real.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper adapts Adaptive Feature Retention (AFR), an unstructured pruning score that combines ReFer and SNIP via standardization (Eq. 1), to structured (neuron-level) pruning of LLM FFNs. It identifies three obstacles to naïve aggregation (Eq. 2): distribution mismatch between ReFer and SNIP, loss of sign information that is claimed to encode optimization-direction consistency, and outlier influence. The proposed pipeline applies 2% percentile outlier removal, sign-preserving (signed-mean then absolute-value) aggregation, power transformation of SNIP scores (Eq. 3, b=2.7), and final summation. Experiments on Llama-3-8B, Vicuna-v1.5-13B and LLaVA-v1.5-13B at 20% and 50% FFN pruning report large gains over naïve averaging (up to +21 points), competitive or superior accuracy versus LLM-Pruner, LoRAP and CFSP, near-parity with unstructured AFR, and measured 1.56–1.57× inference speedups.","tokens_in":11376,"tokens_out":1097,"duration_ms":9746,"significance":"If the empirical claims hold, the work supplies a practical structured-pruning recipe that recovers most of the accuracy of unstructured AFR while delivering real wall-clock speedups on standard dense kernels. The multi-model evaluation (including a VLM) and the systematic Appendix A ablation of the power exponent are useful contributions. The three-component diagnosis of why naïve AFR aggregation fails is also of methodological interest to the structured-pruning community. The paper does not claim theoretical novelty beyond the engineering combination; its value is therefore primarily empirical and engineering.","major_comments":[{"comment":"Section 4.2 asserts that the signed mean (then absolute value) correctly ranks neurons by “optimization direction consistency” and that mixed signs indicate internal cancellation of limited contribution. No independent validation of this mechanistic claim is provided. Critically, the manuscript never isolates the sign-preserving step: there is no ablation that replaces only the signed-mean aggregation with the absolute-mean baseline of Eq. 2 while keeping power transformation and 2% outlier removal fixed. Consequently it remains possible that the large gains over “AFR (naive avg.)” in Tables 3–4 are driven almost entirely by the other two components, leaving the central novelty claim of Section 4.2 unsupported even though the end-to-end numbers are real. An ablation table (or at least a single-column comparison) is required before the mechanistic story can be accepted.","section":null},{"comment":"Tables 3–5 report point estimates only; no multi-seed standard deviations, confidence intervals or calibration-set sensitivity are given. Given that AFR scores depend on a calibration set and that the power exponent and percentile thresholds are free parameters, the absence of error bars makes it impossible to judge whether the reported margins over LoRAP/CFSP (especially the smaller ones at 20%) are statistically reliable. At minimum, three independent calibration seeds for the main Llama-3-8B setting should be reported.","section":null}],"minor_comments":[{"comment":"Eq. (1) uses Z(·) for standardization but never states whether mean/std are computed over the full weight matrix, per layer, or per neuron; this affects reproducibility.","section":null},{"comment":"Figure 2 caption and surrounding text refer to “AFR pruning scores” while the body of Section 3.2 discusses ReFer outliers; clarify which score is plotted.","section":null},{"comment":"The integrated pipeline order in Section 4.4 (outlier removal → signed aggregation → power transform) differs from the order in which the three problems are introduced; a short justification of the chosen sequence would help.","section":null},{"comment":"Table 5 lacks the unstructured-AFR and naïve-averaging baselines that appear for the language-only models, making the LLaVA gains harder to contextualize.","section":null},{"comment":"Typographical inconsistencies: “Eq. equation 1”, “Eq. equation 2”, and mixed use of “SNIP” vs “SNIP” with trailing spaces appear throughout.","section":null}],"recommendation":"major_revision","confidential_remarks":"The base AFR construction is taken from two MIRU workshop papers by overlapping authors (Nitta et al. 2024/2025). The present manuscript is therefore an engineering extension rather than a wholly independent method; the novelty claim rests almost entirely on the three aggregation fixes. Once the missing sign-preserving ablation is supplied, the paper is a solid systems contribution suitable for a solid conference or journal, but the current version over-claims mechanistic insight relative to the evidence."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: they turn their own unstructured AFR into a structured FFN pruner that keeps accuracy close to the unstructured baseline and beats LLM-Pruner, LoRAP and CFSP by clear margins at 20% and 50%, with measured 1.56–1.57× speedups on Llama-3-8B and LLaVA-13B.\n\nWhat is actually new is the diagnosis of three concrete failure modes when you naively average AFR scores to neurons (distribution mismatch, sign loss, outliers) and the three corresponding fixes: power transform on SNIP (b≈2.7), signed-mean-then-abs aggregation, and 2% percentile clipping. The power-transform analysis (SNR argument + Fig. 3 + Appendix A ablation) and the outlier statistics (Table 2) are clean and reproducible in principle. Gains over naïve averaging are large (up to +21 points) and consistent across three models including a VLM; that is real engineering value for anyone already using AFR-style scores.\n\nThe soft spot the stress-test flags is real but not fatal. Section 4.2’s “optimization direction consistency” story for the signed mean is the least independently validated of the three pieces; they never isolate it while holding power transform and outlier removal fixed. So the mechanistic claim is weaker than the end-to-end tables. That said, the other two components are well-motivated, the hyper-parameter plateau around b=2.7 is flat, and the accuracy/speed numbers themselves do not depend on the story being perfect. Missing multi-seed error bars and code are ordinary limitations for this venue, not load-bearing flaws. Circularity is low: they inherit AFR but the aggregation steps are new and ablated.\n\nThis is for people who actually ship pruned LLMs or who work on structured pruning recipes. It is not a conceptual breakthrough, but it is a careful, usable extension with honest baselines. I would send it to referees; the empirical package is strong enough to deserve a proper review even if they ask for the missing ablation and error bars.","headline":"Solid engineering fix that closes most of the structured-vs-unstructured gap for AFR; the sign-consistency story is the softest piece but the end-to-end numbers still hold.","tokens_in":11982,"tokens_out":535,"would_cite":true,"duration_ms":5527,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Structured pruning of large language models can keep accuracy close to unstructured pruning when AFR scores are fixed by power transformation, sign-preserving aggregation, and outlier removal.","keywords":["structured pruning","large language models","Adaptive Feature Retention","power transformation","sign-preserving aggregation","outlier removal","inference speedup","feed-forward networks"],"falsifier":"On the same models and benchmarks, an ablation that keeps power transform and outlier removal but switches to absolute-value-before-average aggregation should not collapse accuracy relative to the full method; if that switch loses most of the reported gains over naive averaging, the sign-consistency claim is wrong.","tokens_in":12006,"feed_emoji":"✂️","tokens_out":926,"duration_ms":14984,"temperature":0.7,"pith_summary":"Large language models are expensive to run. Removing whole neurons (structured pruning) can speed them up on ordinary hardware, but it usually hurts accuracy more than removing individual weights (unstructured pruning). This paper claims that Adaptive Feature Retention (AFR), an unstructured scoring method that mixes feature-preservation and task-gradient signals, can be adapted to structured pruning if three scoring problems are fixed: mismatched score distributions, discarded sign information, and outlier-dominated averages. The authors combine a power transform on the gradient-based scores, averages that keep signs until after the mean, and 2% percentile outlier cleanup. On Llama-3-8B, Vicuna-v1.5-13B, and LLaVA-v1.5-13B they report accuracy near unstructured AFR, large gains over naive averaging, and measured inference speedups of about 1.56–1.57× at 50% feed-forward pruning.","feed_headline":"Structured LLM pruning matches unstructured accuracy at 1.57×","feed_subtitle":"Power transform, signed averages, and outlier cleanup adapt AFR for neuron cuts with real speedup.","key_machinery":"The integrated AFR structured-pruning pipeline: for each neuron, remove scores outside the 2nd–98th percentiles, average signed scores then take absolute value, apply a power transform (exponent b ≈ 2.7) to the neuron-level SNIP score, and sum with the neuron-level ReFer score to rank neurons for removal.","core_discovery":"The paper establishes that AFR can be made into an effective structured pruning criterion by integrating three fixes: power transformation of SNIP scores to align distributions, sign-preserving aggregation that treats consistent optimization direction within a neuron as importance, and percentile-based outlier removal. With that pipeline, structured FFN pruning on several large models yields accuracy comparable to unstructured AFR and better than prior structured methods, while delivering practical dense-matrix speedups.","pith_inferences":["If sign consistency is the real structural signal, similar signed aggregation may improve other gradient-based structured criteria beyond AFR.","Layer-wise adaptive rates (left as future work) could close more of the remaining gap to unstructured accuracy without giving up dense speedups.","The same distribution-mismatch diagnosis may apply whenever a wide-range feature score is added to a [0,1]-concentrated gradient score before neuron aggregation."],"forward_implications":["At 50% FFN structured pruning, models like Llama-3-8B and LLaVA-v1.5-13B can run about 1.56–1.57× faster on standard dense kernels without sparse libraries.","Naive averaging of AFR scores is not a fair structured baseline; the three fixes recover large accuracy gaps (reported up to ~21 points at 20% pruning).","Vision-language models can use full AFR on the language FFN and ReFer-only scoring on the vision tower where task gradients are undefined.","A single power exponent near 2.7 works across the reported language benchmarks with a flat plateau from about 2.5 to 3.0."],"fun_headline_variants":["Power transform and sign-preserving scores adapt AFR for structured LLM cuts","Structured AFR pruning matches unstructured accuracy with real speedups","Three AFR fixes enable structured pruning rivaling unstructured LLM accuracy","Sign-preserving aggregation turns AFR into effective structured neuron pruner","Power align, signed averages and outlier purge make AFR work for FFN cuts"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That averaging signed weight scores and then taking the absolute value correctly ranks a neuron’s importance by how consistent its weights’ optimization directions are; if mixed signs do not mean low contribution, the ranking fails.","fun_headline_variants_meta":{"raw":{"variants":["Power transform and sign-preserving scores adapt AFR for structured LLM cuts","Structured AFR pruning matches unstructured accuracy with real speedups","Three AFR fixes enable structured pruning rivaling unstructured LLM accuracy","Sign-preserving aggregation turns AFR into effective structured neuron pruner","Power align, signed averages and outlier purge make AFR work for FFN cuts"]},"model":"grok-4.5","effort":"low","cost_usd":0.008028,"raw_usage":{"total_tokens":1832,"prompt_tokens":693,"num_sources_used":0,"completion_tokens":89,"cost_in_usd_ticks":80280000,"prompt_tokens_details":{"text_tokens":693,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1050,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":693,"tokens_out":89,"duration_ms":10243,"temperature":1.0,"reasoning_tokens":1050,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T13:29:15.688336+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same models and benchmarks, an ablation that keeps power transform and outlier removal but switches to absolute-value-before-average aggregation should not collapse accuracy relative to the full method; if that switch loses most of the reported gains over naive averaging, the sign-consistency claim is wrong.","supporting_citations":[],"review_version":1}