{"id":"85f4483b-b1b6-4510-8e1b-6acaa78c7777","arxiv_id":"2508.09471","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"EGGS-PTP adds a connectivity-preserving diagonal selection to RIA-style importance pruning, achieving slightly better perplexity under N:M sparsity.","lead":"A new pruning method uses ideas from expander graphs to keep large language models accurate after structured sparsity. It beats two existing pruning methods on perplexity and some zero-shot tasks across several LLaMA models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The expansion proof is a min-degree restatement, not an information-flow guarantee: it covers only subsets of size <B (B≤20, F in thousands), so the paper's central theoretical mechanism is unsupported; the empirical claim needs a mechanism-level ablation.","rationale":"The reader's weakest assumption is exactly the load-bearing concern: the proven expansion property is too weak to support the information-flow claim. Lemma 1 is mathematically correct but vacuous as an expander guarantee. For the input side, the proof only uses the fact that each input neuron has degree at least B, and then restricts attention to subsets of size less than B. With practical B values (1–20) and layer widths in the thousands, this covers only tiny sets and cannot ensure meaningful information flow. The output-side bound is similarly just a minimum-degree argument. Therefore the theoretical motivation does not distinguish EGGS-PTP from any method that gives each input neuron a few neighbors; the diagonal-selection structure is not shown to be the active mechanism. This does not invalidate the empirical results, but it means the central novelty of the paper—'expander-graph guided' pruning—is not established. The proposed ablation directly tests whether the diagonal structure matters beyond mere connectivity. Since the reader already issued CONDITIONAL and our stress test points to the same weakness, no verdict change is needed.","tokens_in":11418,"tokens_out":12393,"duration_ms":137310,"concrete_test":"Run a controlled ablation on LLaMA2-7B at 2:4 sparsity: (i) B=0, which reduces EGGS-PTP to the RIA baseline; (ii) EGGS-PTP as published; (iii) replace diagonal selection in the B connectivity blocks with a random perfect matching per M×M block, keeping all RIA-based selections and the same min-degree. If (iii) matches (ii) within ~0.1 perplexity, the specific expander/diagonal choice is not the active mechanism; if (iii) is clearly worse, structure matters. Also report the selected B values and the per-layer number of connectivity blocks for reproducibility.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Lemma 1's input-side expansion is a trivial consequence of min-degree, not an expander guarantee. Eq. (7) lower-bounds |Γ(T)| by B, the minimum degree of an input neuron, and the proof then restricts to |T|<B via c<B/F_{ℓ-1}. With F_{ℓ-1} in the thousands and B∈[1,20], the theorem covers only subsets smaller than B—for B=2, only singletons—so it cannot justify the paper's 'information flow' language. The output-side argument is also min-degree: every output has degree (F_{ℓ-1}/M)(M-N), which is a standard high-degree fact. Thus the theoretical section does not establish that the diagonal-selection structure preserves information flow; it only shows no input node is isolated. If the mechanism is not expansion, the method is a heuristic whose empirical gains may come from the low-RRI-block allocation or from the extra B hyperparameter rather than from expander structure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"EGGS-PTP proposes a post-training N:M structured pruning method for LLMs. The method computes RIA scores, permutes input channels via round-robin allocation, computes RRI row importance, partitions each pruning group into row-blocks, and then applies two strategies: a 'connectivity-aware' diagonal selection on the B blocks with lowest aggregated RRI, followed by RIA-based top-weight selection, while all other blocks use RIA-based selection. The paper claims that the resulting pruned linear layers are two-sided expanders (Lemma 1), which is said to preserve information flow. Experiments on LLaMA2-7B/13B, LLaMA3-8B, LLaMA-34B, and LLaMA3.2-1B report consistently lower Wikitext2 perplexity than Magnitude, Wanda, and RIA under 2:4 and 4:8 sparsity, as well as improved zero-shot accuracy on several benchmarks for LLaMA-34B, with a 1.64x runtime speedup.","tokens_in":11649,"tokens_out":6818,"duration_ms":75012,"significance":"If the empirical results hold, the paper offers a modest but consistent practical improvement over existing N:M post-training pruning methods across five model sizes and two sparsity patterns. The comparison to Wanda and RIA is useful, and the runtime analysis confirms the hardware benefit of the structured mask. However, the theoretical contribution is not established: Lemma 1 proves only a minimum-degree property in a negligible size regime, not an expander guarantee that can justify the 'information flow' claims. The paper provides no mechanism-level ablation linking the diagonal-selection structure to the observed gains. The value of the work is therefore as a heuristic with reproducible-looking gains, not as a validated graph-theoretic pruning principle.","major_comments":[{"comment":"The theorem is technically correct but is a restatement of minimum degree, not a meaningful expander guarantee. Eq. (7) follows only from the fact that every input neuron retains at least B connections; no property of the diagonal-selection pattern beyond degree is used. The proof then restricts to subsets of size smaller than B via c < B/F_{\\ell-1}. With F_{\\ell-1} in the thousands and B in [1,20], the input-side expansion is guaranteed only for subsets of size at most B-1; for 2:4 sparsity with B=2 this covers only singletons. The output side, Eq. (9), is the standard exact degree of an output node. Thus the lemma does not establish expansion for any set of realistic size, and the abstract/introduction claim that expansion 'ensures information flow' is unsupported. Please either prove a constant-fraction expansion property or revise the claims to describe a minimum-connectivity heurist","section":"Section 6, Lemma 1, Eq. (7)-(10)"},{"comment":"The causal role of the expander-inspired construction is not tested. The empirical gains over RIA could come from the added hyperparameter B, from the RRI-based row partitioning, or from the channel permutation, rather than from the diagonal selection per se. I request ablations that isolate the mechanism: (i) EGGS-PTP with B=0, i.e., importance-aware selection on all blocks; (ii) diagonal selection on randomly chosen low-RRI blocks; (iii) random diagonal selection instead of the magnitude-weighted quadrant selection; and (iv) direct measurement of graph properties after pruning, such as actual vertex expansion for subsets of various sizes, the number of corrupted input channels, and average retained degree per input/output neuron. Without such evidence, the paper's central theoretical narrative remains a conjecture.","section":"Sections 5.3-5.4, Algorithm 1"},{"comment":"The experimental protocol is under-specified for the free parameters. The value of alpha in Eq. (4) is never stated, and it is not reported whether alpha is taken from RIA or tuned. In addition, '1-fold cross-validation on the validation set' is ambiguous: the exact split between validation and test, and the relationship between the Wikitext2 perplexity test set and the validation set used to select B in [1,20], are not described. Since B can materially change perplexity, the comparison to fixed-hyperparameter baselines needs a clearer protocol, ideally with sensitivity analysis over B and alpha.","section":"Section 7.1, Eq. (4)"}],"minor_comments":[{"comment":"A pruning group has M columns, and the algorithm then divides it into M×M blocks. Clarify whether these blocks are disjoint row partitions of the pruning group and that the same M input columns appear in every block. This is important for verifying the claim that every input neuron retains B connections.","section":"Algorithm 1, line 8"},{"comment":"The numerical formatting is inconsistent and occasionally hard to read (e.g., '4 .88', '2400'). Use monospaced columns and align decimal points.","section":"Tables 1-2"},{"comment":"No standard deviations or repeated-run variability are reported. Several differences between EGGS-PTP and RIA are in the 0.1-0.5 perplexity range; reporting variance or at least a small number of seeds would make the 'consistently outperforms' claim more robust.","section":"Tables 3-4, Section 7.1"},{"comment":"The statement that pruning 'addresses overfitting' and improves generalization on BoolQ and RTE is presented as a conclusive explanation without supporting evidence. It would be safer to describe this as a possible interpretation.","section":"Section 7.3"},{"comment":"The paper cites RIA for the channel-permutation and RIA metric, which is appropriate, but it does not clearly state which components are novel relative to RIA and which are inherited. A short comparison table of RIA vs. EGGS-PTP would help.","section":"Related works"}],"recommendation":"major_revision","confidential_remarks":"The empirical results are consistently in the direction claimed, and the method respects the N:M hardware constraint, so there is a publishable heuristic here. However, the paper currently overclaims its theory: Lemma 1 is a minimum-degree restatement, and the absence of mechanism ablation means the central 'expander-guided' narrative is not evidenced. I would advise the editor to require a revision that either substantially strengthens the graph-theoretic analysis or repositions the paper as an empirically motivated pruning heuristic, with ablations to identify where the gains come from."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is RIA plus a connectivity heuristic, and it works a little better. The empirical tables are consistent across five LLaMA models and two sparsity patterns, always a few perplexity points below RIA. That is a real, if modest, result. The method is simple, hardware-compatible, and clearly specified. I believe the accuracy claim.\n\nWhat's actually new: the diagonal-selection step that forces each input and output neuron to keep at least one (actually B) connection in the low-importance blocks, plus partitioning rows by aggregated RRI and applying the connectivity-aware scheme only to the B lowest blocks. That is a sensible engineering combination and I don't see it in the cited prior work. The authors are honest that it builds on RIA.\n\nWhere it gets soft: the theory section. Lemma 1 is true but says much less than the abstract implies. The proof of input-side expansion uses the fact that every input has degree at least B, so for any T smaller than B the neighborhood has size at least B, which is larger than |T|. But with B in [1,20] and layer widths in the thousands, that only covers singleton or tiny sets. It is a min-degree restatement, not an expander guarantee in any useful sense. The output side is just the regular degree from N:M sparsity. So the 'information flow' language is not supported by the lemma. The mechanism could still be the low-RRI-block allocation or min-degree connectivity, but there is no ablation isolating it. The theoretical section should be reframed as intuition, or removed.\n\nAlso the experimental report is thin: no error bars, no code, no SparseGPT baseline, and B is tuned per model on the validation set while the baselines' hyperparameters (if any) are not described equally. That is fixable but it limits how strongly we can read the results. The zero-shot gains are mostly within noise and sometimes below dense, which is unsurprising.\n\nBottom line: useful paper for people working on structured pruning for LLM deployment. The method is cheap and consistently beats the nearest structured-PTP baselines. The theory needs a rewrite or a downgrade. I'd send it to review, with the expectation of a major revision focusing on mechanism-level ablations and a fuller experimental protocol.","headline":"A modest, credible N:M pruning heuristic that consistently beats RIA/Wanda, but the expander-graph theory is a dressed-up min-degree observation and should be treated as motivation, not proof.","tokens_in":12147,"tokens_out":1845,"would_cite":true,"duration_ms":17520,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EGGS-PTP claims that N:M pruning guided by expander-graph connectivity preserves LLM accuracy better than existing one-shot structured pruning methods.","keywords":["structured pruning","N:M sparsity","post-training pruning","large language models","expander graphs","bipartite graphs","information flow","LLM compression"],"falsifier":"Take LLaMA2-7B at 2:4 sparsity and replace EGGS-PTP's diagonal selection in the connectivity blocks with a random matching that still gives every input channel the same minimum degree, keeping RIA top-selection for the remaining weights. If perplexity stays at 10.32 or close to it, the expander-guided selection is not the active ingredient. A complementary check is to compute the spectral gap of the pruned layer adjacency matrices and see whether accuracy tracks expansion constants across layers.","tokens_in":11293,"feed_emoji":"✂️","tokens_out":7671,"duration_ms":73872,"temperature":0.7,"pith_summary":"EGGS-PTP is a post-training pruning method for large language models that removes weights in the N:M hardware pattern while trying to keep the surviving connections expander-like, so information can still flow through the pruned layer. The paper claims that a diagonal-selection step, which guarantees each input and output channel keeps at least one connection in the low-importance parts of each pruning group, is what preserves this flow, on top of an RIA importance metric that saves the most critical weights. Across LLaMA2, LLaMA3, and LLaMA-34B models, the method reports lower perplexity than magnitude pruning, Wanda, and RIA at both 2:4 and 4:8 sparsity, and zero-shot accuracy that sometimes beats the dense model. The reported gains over RIA are consistent but small, and the expansion theorem covers only very small subsets of channels.","feed_headline":"Expander-guided pruning beats existing LLM pruning at equal sparsity","feed_subtitle":"EGGS-PTP's diagonal selection keeps every channel connected, lowering perplexity on LLaMA models under N:M sparsity.","key_machinery":"The load-bearing mechanism is the bipartite-graph view of a linear layer together with a degree floor. In each low-importance $M \\times M$ block the algorithm splits the block into quadrants, sums weight magnitudes along the two diagonals of each quadrant, and keeps the diagonal pair with the larger total; this guarantees every input neuron retains at least $B$ surviving connections. Because each output neuron also keeps exactly $M-N$ connections per pruning group under the N:M mask, every small subset of input or output neurons has a neighborhood larger than itself -- the two-sided expansion property. On top of that floor, RIA-based selection picks the remaining highest-importance weights p","core_discovery":"The central claim is that a structured pruning mask for a linear layer can be built as a two-sided expander graph without abandoning the N:M sparsity that GPUs can accelerate. The paper models each linear layer as a bipartite graph whose adjacency submatrix is the weight matrix, then applies two complementary selections per pruning group: importance-aware selection keeps the top $M-N$ RIA-scored weights in each row, while connectivity-aware selection applies magnitude-based diagonal selection to the lowest-importance $M \\times M$ blocks so every input channel keeps at least one edge. The resulting mask is provably a two-sided $(c, a_I, a_O)$-expander for small channel subsets, and empiricall","pith_inferences":["Not tested in the paper: the expansion guarantee applies only to channel subsets smaller than $B$, a tiny fraction of a layer, so the empirical gains may come from the minimum-degree floor rather than from genuine spectral expansion.","A clean ablation would replace diagonal selection with any degree-preserving random mask while keeping RIA top-selection; if perplexity stays essentially unchanged, the expander-guided selection is not the active ingredient.","A natural extension is to measure the spectral gap of the pruned bipartite graphs and correlate it with layer-wise accuracy loss; the paper proves only a degree-based small-set property, not spectral expansion.","The authors themselves note that a dense 7B model can beat a pruned 13B model, implying the practical value of N:M pruning is mainly hardware cost savings, not accuracy recovery."],"forward_implications":["EGGS-PTP reports lower perplexity than RIA and Wanda on all tested LLaMA models under both 2:4 and 4:8 sparsity, with gains such as 10.32 vs 10.41 on LLaMA2-7B and 20.88 vs 21.47 on LLaMA3-8B.","Because the mask keeps the N:M structure and requires no retraining, it inherits the measured 1.64x inference speedup on LLaMA2-13B (31.82s vs 52.14s for the dense model).","On LLaMA-34B zero-shot benchmarks, the pruned model matches or exceeds the dense baseline on several tasks, including BoolQ and RTE under both sparsity patterns.","The optimal number of connectivity-aware blocks grows with model size, suggesting that larger models can afford more structural protection before importance-aware selection loses its edge."],"supporting_citations":[{"why":"Supplies the RIA importance metric and the round-robin channel permutation that EGGS-PTP builds on, and is the main baseline it must beat.","marker":"[30]"},{"why":"Provides the Wanda activation-based pruning baseline that EGGS-PTP compares against on perplexity and speed.","marker":"[23]"},{"why":"Defines two-sided expanders and supplies the property that small subsets of vertices stay well connected, used in Lemma 1.","marker":"[10]"},{"why":"Motivates using expander graph structure to keep information flow in deep networks despite pruning.","marker":"[20]"},{"why":"Connects graph expansion to pruning at initialization, motivating the expander-guided mask design.","marker":"[9]"},{"why":"Defines the N:M sparse tensor-core acceleration pattern that the pruning mask is designed to satisfy.","marker":"[19]"}],"fun_headline_variants":["Expander graphs make LLM pruning sharper than existing methods","LLM pruning guided by graph theory beats current techniques","Structured pruning with expander graphs: smaller, faster, better","EGGS-PTP: graph-based pruning that outperforms prior LLM pruning","Prune LLMs with expander graphs, keep accuracy, beat the pack"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that guaranteeing each channel keeps at least B connections -- rather than any measured spectral or information-flow property -- is what preserves accuracy; the paper's expansion proof holds only for channel subsets smaller than B and no experiment isolates the connectivity effect.","fun_headline_variants_meta":{"raw":{"variants":["Expander graphs make LLM pruning sharper than existing methods","LLM pruning guided by graph theory beats current techniques","Structured pruning with expander graphs: smaller, faster, better","EGGS-PTP: graph-based pruning that outperforms prior LLM pruning","Prune LLMs with expander graphs, keep accuracy, beat the pack"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000189,"raw_usage":{"total_tokens":1139,"prompt_tokens":680,"completion_tokens":459,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":424,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":424,"tokens_out":459,"duration_ms":6011,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:02:10.234821+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take LLaMA2-7B at 2:4 sparsity and replace EGGS-PTP's diagonal selection in the connectivity blocks with a random matching that still gives every input channel the same minimum degree, keeping RIA top-selection for the remaining weights. If perplexity stays at 10.32 or close to it, the expander-guided selection is not the active ingredient. A complementary check is to compute the spectral gap of the pruned layer adjacency matrices and see whether accuracy tracks expansion constants across layers.","supporting_citations":[{"cited_title":"Plug-and-play: An efficient post-training pruning method for large language models","cited_arxiv_id":null,"evidence_quote":"Supplies the RIA importance metric and the round-robin channel permutation that EGGS-PTP builds on, and is the main baseline it must beat."},{"cited_title":"Explicit two-sided unique-neighbor expanders","cited_arxiv_id":null,"evidence_quote":"Defines two-sided expanders and supplies the property that small subsets of vertices stay well connected, used in Lemma 1."},{"cited_title":"Deep expander networks: Efficient deep networks from graph theory","cited_arxiv_id":null,"evidence_quote":"Motivates using expander graph structure to keep information flow in deep networks despite pruning."},{"cited_title":"Revisiting pruning at initialization through the lens of ramanujan graph","cited_arxiv_id":null,"evidence_quote":"Connects graph expansion to pruning at initialization, motivating the expander-guided mask design."},{"cited_title":"Nvidia a100 tensor core gpu architecture","cited_arxiv_id":null,"evidence_quote":"Defines the N:M sparse tensor-core acceleration pattern that the pruning mask is designed to satisfy."}],"review_version":1}