{"id":"7a7f4792-cd53-4c62-98ea-d2fad749ea90","arxiv_id":"2506.02561","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Cus-Prun identifies and removes neurons that are irrelevant to a user's target language, domain, and task, producing specialized expert models without post-training.","lead":"The paper introduces Cus-Prun, a method that prunes a large general language model into a smaller expert model specialized along three dimensions: language, domain, and task, without any post-training. It matters because it could let users quickly create customized, smaller LLMs for specific needs while preserving useful general abilities.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 2 and Algorithm 1 define pruned neurons as an intersection of per-document bottom-σ sets; for σ=25% this may remove far less than 25% of parameters, so Table 1's equal-ratio comparisons may be unfair.","rationale":"The reader’s weakest assumption is corpus representativeness: small GPT-4o-generated corpora may not match the target distribution. That is a genuine concern, but it is secondary to the pruning-ratio question. The central claim depends on comparing Cus-Prun with baselines at the same pruning ratio; if the formal definition in Eq. 2 and Algorithm 1 produces a much smaller pruned set than σ requests, then the method is, as written, internally inconsistent with its own evaluation, and every headline comparison in Table 1 is potentially apples-to-oranges. I read the method in good faith: neuron-level output-perturbation scoring is a reasonable extension of prior interpretability work, the code is released, and the experiments span multiple model families and sizes. The issue is not that pruning per-dimension neurons is implausible; it is that the formal specification does not guarantee the advertised pruning fraction, and the paper nowhere reports actual pruned parameter counts. A single code-level check can settle this: count the neurons actually removed under the released implementation. Until that is run, the reported expert-vs-general performance at 25% pruning cannot be reliably interpreted. I therefore keep the verdict CONDITIONAL, but with the additional explicit condition that the authors demonstrate the true pruned fraction equals the reported ratio and, if necessary, rerun the comparisons with matched parameter budgets.","tokens_in":18976,"tokens_out":7551,"duration_ms":77172,"concrete_test":"Run the released code on Llama3-8B with σ=25% using the Section 4.2 corpus construction, and count the actual number of neurons in \\tilde N_L ∩ \\tilde N_{D∩T} (or read the parameter count after pruning). Compare the reduced parameter count to the dense model and to the 25% figure used for LLM-Pruner, SliceGPT, and ShortGPT in Table 1. If the true fraction differs from 25% by more than a few points, re-compute Table 1 with Cus-Prun pruned to exactly 25% by adding a budget-control mechanism; if Cus-Prun then falls below the baselines or requires more aggressive pruning, the central claim fails. Also inspect the code to determine whether the selection criterion is the Eq. 2 universal intersection or an averaged-impact ranking, and correct the formal description accordingly.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central quantitative claim is that Cus-Prun prunes 25% of parameters and still beats baselines (Section 4, Table 1; Section 4.5, Table 3). But the formal selection rule in Eq. 2 and Algorithm 1 (lines 13–17) keeps a neuron only if it is “Irrelevant to c, ∀ c ∈ C_L”, where per-document irrelevance is defined as being in the lowest-σ impact set. Taken literally, \\tilde N_L is an intersection of per-document bottom-25% sets; with the 50-document corpora of Section 3 and larger Section 4 corpora, this intersection should be very small unless per-document importance rankings are almost perfectly correlated. Eq. 5 then intersects \\tilde N_L, \\tilde N_D, and \\tilde N_T, shrinking the set further. Algorithm 1 contains no budget-adjustment step that forces the final set to equal the requested σ, and the paper never reports the actual number of pruned neurons or post-pruning parameter count for Cus-Prun. If Cus-Prun removes substantially less than 25%, Table 1 compares models of different sizes and the headline claim of consistently outperforming other pruning methods at the same ratio is unsupported. If the implementation instead ranks neurons by impact averaged over the corpus rather than by the Eq. 2 universal intersection, then the paper’s formal definition is not what was actually evaluated. Either way, the manuscript as written does not establish that Cus-Prun actually prunes the advertised 25%, 35%, or 45% of parameters.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Cus-Prun, a training-free structural pruning method that customizes a general LLM into an expert model along three dimensions (language, domain, task). For each selected dimension, it constructs a corpus, measures the output-perturbation impact of removing each neuron, marks neurons in the lowest-σ impact set as irrelevant per document, and then prunes the intersection of the per-dimension irrelevant-neuron sets. Experiments are reported on Llama3-8B, Mistral-12B, Llama2-13B, and Llama3-70B across multilingual, multidomain, and multitask benchmarks, together with one- and two-dimensional adaptive settings, claiming that Cus-Prun consistently outperforms LLM-Pruner, SliceGPT, and ShortGPT at a nominal 25% pruning ratio and remains effective up to 45% when combined with layer pruning.","tokens_in":19278,"tokens_out":5828,"duration_ms":56262,"significance":"The idea of identifying irrelevant neurons along user-specified dimensions and pruning their intersection is simple, practical, and aligns with a real need for compact, deployment-ready expert models without post-training. The paper's strengths include a clearly described algorithm, public code, experiments across multiple model families and sizes, and an honest limitation section. If the central quantitative claims were fully supported, this would be a useful contribution to the pruning literature. However, as written, the claimed improvements are not yet established: the formal pruning rule may not actually remove the advertised fraction of parameters, and the pruning corpora partly overlap with the evaluation benchmarks, so the headline comparisons in Table 1 and Table 3 are not verified.","major_comments":[{"comment":"The formal selection rule in Eq. (2) and Algorithm 1 (lines 13–17) defines a neuron as irrelevant only if it is in the lowest-σ impact set for every document in the corpus, and Algorithm 1 has no budget-enforcement step that makes the final pruned set equal to the requested σ. The paper never reports the actual number of pruned neurons or the post-pruning parameter count for any Cus-Prun model. If the intersection of per-document bottom-σ sets is much smaller than σ, then the equal-ratio comparisons in Table 1 and the 25%/35%/45% claims in Table 3 compare models of different actual sizes. If the implementation instead enforces the nominal ratio by ranking neurons on averaged impact, it does not match the formal definition. Please report actual post-pruning parameter counts, add a budget-adjustment step that guarantees the advertised sparsity, or re-run the comparisons at matched actual sparsity.","section":"§2.1, Eq. (2), Algorithm 1, Table 1, Table 3"},{"comment":"In the multilingual setting, the corpus C(D,T)_Exp is constructed from the English splits of the same datasets used for evaluation: GSM8K for MGSM, the English split of M3Exam for M3Exam, SQuAD for XQuAD, and XSum for XLSum. This is benchmark leakage: the pruning decisions are derived from material drawn from the target benchmarks' own distributions, so the reported expert-capability scores are not an independent evaluation of customization to a new use case. Additionally, hyperparameters are tuned on the validation set of XLSum and then applied to the other multilingual test sets. Please construct pruning corpora that do not overlap with evaluation datasets, or provide a clear argument that this overlap does not inflate the results.","section":"§4.2, Appendix A.3.1"},{"comment":"All reported results come from single runs with no error bars, no multiple seeds, and no variance estimates. Several claimed improvements are modest (e.g., Llama2-13B multidomain average: Cus-Prun 33.1 vs. SliceGPT 30.3 and ShortGPT 30.3), and in the multitask setting on Llama3-8B, LLM-Pruner achieves 54.7 while Cus-Prun achieves 52.2, an exception the text acknowledges. Without repeated runs or confidence intervals, the claim that Cus-Prun \"consistently outperforms\" other methods is not statistically supported. Please provide standard deviations or repeated-seed results, especially for the close comparisons.","section":"§4.1, Table 1, Figure 2, Figure 4"},{"comment":"The preliminary evaluation in Section 3 uses only 50 GPT-4o-generated documents per dimension, and the aggressive-pruning experiments in Section 4.5 combine layer pruning with Cus-Prun without separating the contributions of the two components. The paper reports no sensitivity analysis for corpus size or corpus generator, and it does not report how much of the 35% and 45% parameter reduction in Table 3 comes from layer pruning versus neuron pruning. As a result, the claim that Cus-Prun remains effective \"up to nearly half the parameters\" is hard to interpret. Please report the actual sparsity contributed by each component and include a corpus-size ablation.","section":"§3, §4.5, Table 3"}],"minor_comments":[{"comment":"In the sentence \"Cus-Prun largely preserves the performance of the dense model, retraining 92%, 83%, and 94%,\" the word \"retraining\" should be \"retaining.\"","section":"§3, Main Results"},{"comment":"The section heading \"MultilDomain Setting\" contains a typo and should read \"Multidomain Setting.\"","section":"§4.3 heading"},{"comment":"The abstract and conclusion state that Cus-Prun \"consistently outperforms\" other methods, but §4.4 explicitly reports an exception for LLM-Pruner on Llama3-8B in the multitask setting; the summary claims should be qualified accordingly.","section":"Abstract, §4.4"},{"comment":"The appendix contains a typo, \"conterfact,\" and the citation \"Amazon Summary (Wang et al., 2022;?)\" has a placeholder question mark that should be completed.","section":"Appendix A.3.3"},{"comment":"Several entries in Table 1 are visually corrupted by adjacent numbers running together (e.g., \"32.946.7\", \"0.061.876.0\", \"23.2.18.0\"), which makes the table difficult to read.","section":"Table 1"},{"comment":"The appendix tables use \"Expert Capability\" and \"Specific Capability\" inconsistently for the same column; the terminology should be unified.","section":"Tables 5–7"}],"recommendation":"major_revision","confidential_remarks":"The central concern is a reporting gap rather than an obviously fatal flaw: if the authors can report actual post-pruning parameter counts, fix the corpus overlap with evaluation datasets, and provide variance estimates, the method may be a solid contribution. I do not see any misconduct, but the paper as written does not yet substantiate its headline equal-ratio comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Charles, quick read of arXiv:2506.02561. Short version: the idea is sensible and the experiments are broad, but the paper's formal pruning rule and its stated pruning ratios don't obviously line up, and the \"consistently outperforms\" headline is too strong.\n\nWhat's actually new: the paper takes the language-specific neuron idea from Tang et al. and Zhao et al., extends it to domain and task dimensions, and prunes the intersection of the per-dimension irrelevant sets. That's a clean, practical recipe, and it produces expert models without post-training. The testbed is solid: four model families/sizes, several multilingual/domain/task benchmarks, and code is public. The method is simple enough to reproduce.\n\nThe soft spots are real. The formal definition in Eq. 2 and Algorithm 1 keeps a neuron only if it's in the bottom-sigma impact set for every document in the dimension corpus. For a 50-document corpus, that intersection should be tiny unless per-document rankings are almost perfectly correlated. The paper never reports the actual number of pruned neurons or post-pruning parameter count. If the implementation prunes a true 25%, it's not following the equations; if it follows the equations, Table 1 compares Cus-Prun at a lower pruning ratio than the baselines. Either way, the advertised comparison isn't established. This is easy to fix by reporting actual counts, but it's load-bearing.\n\nThe evaluation also lacks error bars or multiple seeds. The margins are often huge, so significance is likely, but we can't confirm. The multilingual setup uses English splits of the evaluation datasets (GSM8K for MGSM, SQuAD for XQuAD, XSum for XLSum) as the corpus, which is tuning on data very close to the test sets. And the abstract's \"consistently outperforms\" is contradicted by their own Table 1 for Llama3-8B multitask, where LLM-Pruner scores higher; they disclose this in Section 4.4, but the abstract overclaims. The GPT-4o-generated corpora in the preliminary study are a minor uncontrolled variable; real corpora elsewhere help.\n\nThe authors do include a limitations section and are transparent about the one exception, which counts in their favor.\n\nWho this is for: practitioners who want fast, task-specific pruning without retraining. The core idea is plausible and worth a serious look. But the pruning-ratio ambiguity needs to be resolved before the headline numbers can be trusted.\n\nMy recommendation: send it to review with major revision. Ask for actual pruned parameter counts, error bars, and a revision of the \"consistent\" claim. The paper deserves referee time.","headline":"A useful pruning recipe undermined by an unverified gap between its formal definition and its reported pruning ratios.","tokens_in":19833,"tokens_out":5186,"would_cite":false,"duration_ms":46218,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a large general language model can be pruned into a smaller expert model tuned to a specific language, domain, and task by removing only the neurons that are irrelevant across all three dimensions, with no…","keywords":["model pruning","expert models","customization","neuron importance","LLM compression","multilingual models","domain adaptation","no post-training"],"falsifier":"Generate two different 50-document corpora for the same language-domain-task profile using different prompt styles or sources, run Cus-Prun with the same pruning ratio on both, and compare the selected irrelevant-neuron sets and downstream test scores; if the sets or scores disagree substantially, the method's output is corpus-sensitive rather than a stable property of the target expertise.","tokens_in":18703,"feed_emoji":"✂️","tokens_out":9094,"duration_ms":79184,"temperature":0.7,"pith_summary":"Cus-Prun is a method for turning a large general-purpose language model into a smaller expert model without any post-training. The paper positions the target use case along three dimensions—language, domain, and task—and, for each dimension, identifies the neurons whose removal least changes model output on a small corpus. The expert model is built by pruning only the neurons that are irrelevant across every specified dimension, so neurons that any dimension depends on are kept. Across multilingual, multidomain, and multitask benchmarks, the paper reports that this consistently outperforms existing no-retraining pruning methods while preserving most general ability, on models from 8B to 70B parameters.","feed_headline":"Prune one LLM into many expert models with no retraining","feed_subtitle":"Cus-Prun removes only neurons irrelevant to the target language, domain, and task, keeping both expert and general skills.","key_machinery":"The machinery is per-neuron output-perturbation scoring. Each neuron $N_i^{(l)}$ is an individual row or column in a parameter matrix of an attention or feed-forward layer, and its relevance to a document $c$ is measured as the $L^2$ distance between the layer output with the neuron and the layer output without it. Neurons whose impact falls in the lowest $\\sigma\\%$ are declared irrelevant for that dimension. Corpora for each dimension are constructed by ablating the other dimensions, so the language corpus, for example, spans many domains and tasks in that language. The final expert model removes the intersection of the per-dimension irrelevant sets, and the paper adopts a parallel version of the sequential detection loop for efficiency.","core_discovery":"On the paper's own terms, the central discovery is that customized expertise can be carved out of a dense LLM by intersecting, rather than pooling, the sets of neurons that are irrelevant to each specified dimension. Given a requested expert model described by a language, a domain, and a task, the paper builds corpora that vary the unspecified dimensions, scores each neuron by the $L^2$ change in its layer's output when the neuron is removed, and flags the lowest $\\sigma\\%$ of neurons as irrelevant for that dimension. The pruned model is the original model with only the intersection of those per-dimension irrelevant sets removed, so neurons that any dimension depends on survive. The paper's experiments report that this preserves 83–94% of dense-model performance in three proof-of-concept scenarios and beats SliceGPT, LLM-Pruner, and ShortGPT by large margins on expert tasks while degrading general benchmarks much less than those baselines.","pith_inferences":["The intersection rule implies an inherent tradeoff the paper does not quantify: adding more dimensions shrinks the prunable set, so a three-dimensional expert model will likely be less compressed than a one-dimensional one; measuring that tradeoff would tell users when customization costs too much size reduction.","Because the paper relies on small corpora that can be manually collected or generated, a natural test is whether corpus quality and coverage, rather than the pruning rule itself, drive the gains; comparing neuron sets selected from different small corpora would settle this.","The paper leaves open whether pruned expert models can be post-trained; if they can, Cus-Prun could serve as a cheap initialization for further fine-tuning, and if not, its no-retraining property is the ceiling.","The same per-neuron scoring could be applied to other customization axes the paper names as limitations, such as query format or input structure, giving a direct extension to interaction-style expertise."],"forward_implications":["Users could produce a German-medical-QA model, a Chinese-e-commerce-sentiment model, or any similar language-domain-task combination from one general base model using small text corpora and no retraining.","Specialization granularity is adjustable: the same scoring works for one dimension, two dimensions, or all three, so a single pipeline covers language-only, language-domain, and full-profile customization requests.","Because pruning removes only neurons irrelevant to all chosen dimensions, general English capabilities are preserved much better than with layer removal or matrix approximation baselines; for example, Llama3-8B retains a 51.4 average general score versus at most 22.3 for baselines at 25% pruning.","The method extends beyond 25% pruning: combined with layer pruning, it keeps MMLU at 48.4 and expert capability at 50.6 on Llama3-8B when 45% of parameters are removed, where ShortGPT nearly collapses.","The approach transfers across model families and sizes, with reported gains on Llama3-8B, Mistral-12B, Llama2-13B, and Llama3-70B in multilingual, multidomain, and multitask settings."],"supporting_citations":[{"why":"Supplies the observation that neurons can be language-specific and provides the parallel neuron-detection implementation used in Algorithm 1.","marker":"(Zhao et al., 2024b)"},{"why":"Provides evidence that certain neurons are responsible for specific languages, the premise the paper extends to domains and tasks.","marker":"(Tang et al., 2024)"},{"why":"Offers related evidence that functional neurons exist per language and serves as one inspiration for per-dimension neuron identification.","marker":"(Liang et al., 2024)"},{"why":"LLM-Pruner is a structural-pruning baseline that Cus-Prun must beat without post-training.","marker":"(Ma et al., 2023)"},{"why":"SliceGPT is the matrix-approximation baseline used in the preliminary and main comparisons.","marker":"(Ashkboos et al., 2024)"},{"why":"ShortGPT is the layer-removal baseline, including the comparison at aggressive pruning ratios.","marker":"(Men et al., 2024)"},{"why":"Provides the Llama 3 8B and 70B base models used in the experiments.","marker":"(Dubey et al., 2024)"}],"fun_headline_variants":["Carve expert LLMs via targeted pruning, no retraining","Intersect irrelevant neurons to get expert LLMs","Targeted pruning: expert models for language, domain, task","No retraining: prune LLM into task-specific expert","Expert LLMs from one model: prune only irrelevant neurons"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a neuron judged unimportant by measuring output changes on a small corpus for each dimension—sometimes only 50 generated documents—will also be unimportant for the real target use case; if those corpora are unrepresentative, pruned models could lose exactly the capabilities the user needs.","fun_headline_variants_meta":{"raw":{"variants":["Carve expert LLMs via targeted pruning, no retraining","Intersect irrelevant neurons to get expert LLMs","Targeted pruning: expert models for language, domain, task","No retraining: prune LLM into task-specific expert","Expert LLMs from one model: prune only irrelevant neurons"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000591,"raw_usage":{"total_tokens":2756,"prompt_tokens":915,"completion_tokens":1841,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":1760}},"tokens_in":531,"tokens_out":1841,"duration_ms":11349,"temperature":1.0,"reasoning_tokens":1760,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:19:55.693269+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate two different 50-document corpora for the same language-domain-task profile using different prompt styles or sources, run Cus-Prun with the same pruning ratio on both, and compare the selected irrelevant-neuron sets and downstream test scores; if the sets or scores disagree substantially, the method's output is corpus-sensitive rather than a stable property of the target expertise.","supporting_citations":[{"cited_title":"Multilingual Knowledge Editing with Language-Agnostic Factual Neurons","cited_arxiv_id":"2406.16416","evidence_quote":"Offers related evidence that functional neurons exist per language and serves as one inspiration for per-dimension neuron identification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SliceGPT is the matrix-approximation baseline used in the preliminary and main comparisons."}],"review_version":1}