{"id":"4719951a-9c13-4f36-a073-0fc755c09b4b","arxiv_id":"2508.12891","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ONG prunes neural network weights at initialization using NMF reconstruction error as an importance score, then strictly preserves the mask throughout training.","lead":"This paper introduces a one-shot pruning method that scores neural network weights by how poorly they are reconstructed by a low-rank NMF approximation, then trains only the kept weights. The reported accuracy is comparable to several baselines on CIFAR benchmarks, but the experiments are incomplete and compare methods at different sparsity levels.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tables I–II compare ONG to STR/DST at very different achieved sparsity, and at matched sparsity ONG trails GMP/DPF; the superiority claim is contradicted by the paper's own numbers.","rationale":"The reader's REJECT verdict is correct. The strongest claim is empirical, and its load-bearing condition is that accuracy is compared at the same sparsity level. Table I/II violate this for STR and DST, and among rows at the intended sparsity ONG is not better than GMP/DPF. So the evidence is not merely insufficient; it points against ONG. The reader's identified weakest assumption, that NMF reconstruction error on initial weights predicts post-training importance, is real but secondary: even a perfect NMF criterion would not rescue a comparison that does not hold sparsity fixed, and an ablation isolating NMF versus magnitude or random masks becomes meaningful only after matched-sparsity results exist. The internal inconsistencies (abstract promises ResNet34/18, undefined ONG-M/ONG-S, template placeholders) reinforce rejection but are not the single decisive check. I recommend no change to the reader's REJECT verdict.","tokens_in":10076,"tokens_out":5997,"duration_ms":62013,"concrete_test":"Re-run Tables I and II under a strict sparsity-matching protocol: for each goal (80/90/95%), tune every method so achieved global sparsity falls within ±0.1% of the goal (e.g., 94.9–95.1% for the 95% column), using the exact ONG-M and ONG-S configurations supplied in code, with at least 3 seeds and identical epochs/schedule; report top-1 accuracy as mean±std. If ONG then matches or exceeds GMP/DPF, the concern is resolved; if it falls below them as the current AS-matched rows suggest, the central claim fails. A simpler first check: recompute the 95% column with STR stopped at 95% sparsity instead of 99.3% and compare.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section V.A's central claim is that ONG is comparable or superior to established stable sparsifiers at 80/90/95% target sparsity. The reported tables do not support that claim. In Table I, the AS column shows STR at 99.17/99.26/99.30% and DST at 97.31/97.41/97.39% achieved sparsity, while ONG-M/ONG-S sit near 79.8/89.9/94.8%. Comparing accuracy at equal goal sparsity while achieved sparsity differs by 2.5–4.5 percentage points compares different problems; the large apparent margins over STR/DST are artifacts of this mismatch. Against methods that actually achieve the target sparsity, ONG does not win: CIFAR-10 at 80% has DPF 93.85 vs ONG-M 93.51; at 90% GMP 92.88 vs ONG-M 92.49; at 95% DPF 92.48 and GMP 92.07 vs ONG-M 91.45 and ONG-S 91.49. CIFAR-100 shows the same pattern: at 80%, DPF 70.31 and GMP 70.29 vs ONG-M 70.29; at 90%, DPF 69.91 vs ONG-M 69.25. Thus the paper's own tables contradict the 'comparable or superior' claim. This is the load-bearing empirical gap: the central claim requires matched-sparsity comparisons, and none of the favorable comparisons are matched. The undefined ONG-M/ONG-S variants and template hyperparameter placeholders further block independent verification, but the sparsity mismatch alone is decisive.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ONG (One-shot NMF-based Gradient Masking), a sparsification method that computes NMF reconstruction-error scores on the initial weight matrices, prunes once via per-layer thresholds controlled by a scaling factor γ, tunes γ by binary search to hit a user-specified global sparsity target, and then trains with strict gradient and weight masking so that the fixed sparse mask is preserved throughout training. Experiments are reported on CIFAR-10 and CIFAR-100 with ResNet56 inside the BIMP framework, and the claim is that ONG achieves accuracy comparable or superior to stable sparsification baselines (GMP, GSM, DPF, DNW, LC, STR, CS, DST) at 80%, 90%, and 95% target sparsity. The writing largely describes the method and tables; however, the paper's own numbers do not support the central comparison claim when sparsity levels are matched.","tokens_in":10396,"tokens_out":2850,"duration_ms":29708,"significance":"If the core claim were established, ONG would offer a simple one-shot alternative to iterative pruning with a clear sparsity-control mechanism and a novel scoring principle based on NMF reconstruction error. The paper's strengths are its explicit mask-construction algorithm (Algorithm 1 and 2), the automated γ-tuning procedure, and the insistence on hard sparsity preservation during training. These contributions are easy to state and, in principle, easy to verify. The weakness is empirical: the central comparative claim is not supported by the matched-sparsity evidence, the key assumption that NMF scores on initial random weights identify trainable subnetworks is not validated against even a random-mask baseline, and critical experimental details are left as unresolved template placeholders, so the results are not independently reproducible in their present form.","major_comments":[{"comment":"The claim that ONG is comparable or superior to established stable sparsifiers is contradicted by the paper's own matched-sparsity comparisons. In Table I, on CIFAR-10 at 80% sparsity, DPF achieves 93.85% accuracy versus 93.51% for ONG-M; at 90%, GMP achieves 92.88% versus 92.49% for ONG-M; at 95%, DPF achieves 92.48% and GMP 92.07% versus 91.45% (ONG-M) and 91.49% (ONG-S). In Table II on CIFAR-100, at 80% DPF and GMP achieve 70.31% and 70.29% versus 70.29% for ONG-M, and at 90% DPF achieves 69.91% versus 69.25% for ONG-M. The favorable comparisons to STR and DST are not matched in achieved sparsity: STR sits at 99.17–99.30% and DST at 97.31–97.41% while ONG sits near 79.80–94.80%, so the large reported margins are artifacts of comparing different problems rather than evidence of superiority.","section":"Section V.A, Tables I and II"},{"comment":"The text states that achieved sparsities closely matching targets 'verifies the consistency of the MAD-based masking mechanism,' but this match is guaranteed by construction by the binary-search γ-tuning procedure in Algorithm 2. The binary search explicitly minimizes the difference between achieved and target sparsity, so a close match is not an empirical verification of anything about the masking mechanism or the scoring method. This circular verification step should be removed or reframed as a check on the search implementation, not as evidence for the method's effectiveness.","section":"Section III.B and Section V.A"},{"comment":"The experiments are not reproducible because key hyperparameters and result definitions are left as template placeholders. Section IV.D contains unresolved items such as 'k=[e.g., 6]', 'γ search range [e.g., 0.01, 10.0]', 'Initial γ guess [e.g., 1.0 or 1.5]', and 'Random Seed: [Specify if you use a fixed seed...]'. Moreover, the table labels ONG-M and ONG-S are never defined in the text; the reader cannot tell whether these correspond to different thresholding types, different seeds, or different network variants. Without these details and without a clear definition of the two ONG variants, the reported accuracy numbers cannot be independently checked.","section":"Section IV.D and Table I"},{"comment":"The load-bearing assumption of the method is that NMF reconstruction error computed once on the initial random weights identifies weights that will remain important after training. The paper motivates this only by the assertion that high reconstruction error means 'unique information,' but it provides no theoretical argument and no control experiment to isolate this factor. In particular, there is no comparison of ONG against a random-mask baseline under the identical fixed-mask training protocol, nor against a magnitude-based one-shot scoring method under the same protocol. Such a comparison is necessary to establish that the NMF scoring, rather than the strong gradient/weight masking or the act of one-shot pruning itself, is responsible for the reported accuracy. Without this isolation, even the matched-sparsity results would not support the claim that NMF scores are a useful saliency signal.","section":"Section III.A and Section V"}],"minor_comments":[{"comment":"The name 'ONG' is expanded as 'One-shot NMF-based Gradient Masking' in the abstract, but Section V.A refers to 'Orthogonal NMF-based Gradient-masked pruning'; orthogonal NMF is not otherwise defined or used.","section":"Section V.A"},{"comment":"In the 80% column, the CS row lists achieved sparsity 89.56%, which appears to be a copy-paste error; the achieved sparsity for CS at the 80% target should presumably be near 79.89% like the other GMP-family baselines.","section":"Table II"},{"comment":"Equation (2) writes Score(w_ij) = |w_abs,ij - w_hat_abs,ij|, but the notation is introduced as W_abs and the score for w_ij; the subscripts should be aligned so that it is clear the score applies to the absolute value of the original element.","section":"Section III.A, Eq. (2)"},{"comment":"The list of baselines includes DPF and DNW in the results tables but the baseline list in Section IV.B does not mention DNW; a brief description or citation for DNW should be added.","section":"Section IV.B"},{"comment":"The NMF decomposition in Eq. (1) is stated as a minimization problem, but the paper does not specify which NMF algorithm, initialization, or convergence criterion is used; citing Lee and Seung [31] is not sufficient for reproducibility.","section":"Section IV.D"},{"comment":"The abstract claims 'maintaining structural integrity post-pruning,' but the experiments only measure unstructured global sparsity, accuracy, and FLOPs; no structural integrity metric or structured-pruning analysis is reported.","section":"Abstract and Section I"}],"recommendation":"reject","confidential_remarks":"The paper appears to be an unfinished draft: several hyperparameters and experimental-specification lines are literally template placeholders, and the two ONG variants in the results tables are never defined. The masking mechanism is straightforward and the γ-tuning construction is sound, but the central claim is contradicted by the matched-sparsity numbers in the authors' own tables. A revision could fix the empirical comparison by adding random-mask and magnitude-score baselines under identical training and by reporting exact hyperparameters, but the current manuscript does not meet the standard for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely new: using NMF reconstruction error as a per-weight importance score for one-shot pruning at initialization. Prior factorization work replaces weights with low-rank factors; ONG instead uses NMF only to decide which original weights to keep. That distinction is real, and the masking scheme (gradient masking plus weight re-masking) is clearly described and sound. The complexity analysis is honest, and integrating into the BIMP framework is a sensible way to get comparable baselines.\n\nThe problem is the evidence. The abstract promises ResNet34 and ResNet18 evaluations that do not appear anywhere in the text; only ResNet56 on CIFAR-10 and CIFAR-100 is reported. The central claim of \"comparable or superior\" performance is not supported by the tables. STR is compared at 99% achieved sparsity against ONG's 80/90/95%, and DST is at 97% — those are different problems, so the large margins over STR and DST are artifacts. Against methods that actually achieve the target sparsity, GMP and DPF beat ONG at most sparsity levels: at 80% ONG-M gets 93.51 vs DPF 93.85; at 90% 92.49 vs GMP 92.88; at 95% 91.45 vs DPF 92.48. The text's claim of \"consistently higher or competitive accuracy\" is not what the numbers show.\n\nThere are also smaller but telling issues: ONG-M and ONG-S are never defined; hyperparameters are full of template placeholders like \"[e.g., 6]\"; no seeds or error bars are reported; no code is provided. The binary-search gamma tuning forces achieved sparsity to match the target, so calling that match \"verification\" is circular — it is guaranteed by construction.\n\nWho is this for? Researchers working on pruning-at-initialization might find the NMF scoring idea worth a quick look, but this draft does not give them enough to evaluate it. The method could be salvageable, but the empirical case needs to be redone with matched-sparsity comparisons, defined variants, seeds, and code. As is, I would not accept it; if it came to me, I would send it back for major revision with those demands clearly stated.","headline":"The NMF-based scoring idea is new and plausible, but the empirical claims are contradicted by the paper's own tables, and the draft is not reproducible as written.","tokens_in":10939,"tokens_out":2978,"would_cite":false,"duration_ms":30983,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One-shot NMF pruning holds accuracy at 95% sparsity","keywords":["network pruning","one-shot pruning","non-negative matrix factorization","gradient masking","sparse training","model sparsification","ResNet56","CIFAR-10"],"falsifier":"Train two ResNet56 models on CIFAR-10 with the same target sparsity, one masked by ONG scores and one by randomly chosen masks of the same per-layer sparsity. If the random-mask model matches ONG's accuracy across 80, 90, and 95 percent sparsity, the NMF scores are not carrying the pruning signal. A weaker check: recompute ONG masks after randomly permuting the initial weights within each layer; if accuracy does not drop, the score values themselves are irrelevant.","tokens_in":1804,"feed_emoji":"✂️","tokens_out":4378,"duration_ms":91833,"temperature":0.7,"pith_summary":"The paper proposes ONG, a one-shot sparsification method: before any training, each weight is scored by how poorly it is reconstructed by a low-rank nonnegative factorization of its layer's absolute weight matrix, and the worst-reconstructed weights are kept while the rest are zeroed once. From then on, gradients and weights are masked so the zeroed positions stay zero throughout training. The claim is that this cheap, fixed-mask recipe reaches accuracy comparable to or better than stable sparsification baselines on CIFAR-10 and CIFAR-100 with ResNet56 at 80, 90, and 95 percent sparsity, while hitting the requested sparsity almost exactly. If true, it offers a simple, deterministic path to precisely controlled sparse models without iterative pruning cycles or retraining phases.","feed_headline":"One-shot NMF pruning holds accuracy at 95% sparsity","feed_subtitle":"A fixed mask picked before training matches or beats standard sparsifiers at 80 to 95 percent sparsity.","key_machinery":"The central mechanism is the NMF reconstruction-error score combined with strict gradient and weight masking. NMF decomposes the absolute value of each prunable weight matrix into a product of two nonnegative low-rank matrices, and the absolute difference between the original matrix and its reconstruction gives each weight an importance score: a high residual means the weight is poorly explained by the dominant low-rank factors, so it is considered structurally unique and worth keeping. A binary search over the threshold parameter gamma then sets per-layer cutoffs that achieve a desired global sparsity, and a fixed binary mask is applied once. During training, both gradients and weights are multiplied by this mask at every step, so the pruned positions stay exactly zero and the sparsity level cannot degrade.","core_discovery":"ONG's central discovery is that NMF reconstruction error on the initial weights can serve as a saliency signal for one-shot pruning, provided training strictly enforces the resulting mask. For every prunable weight matrix, ONG takes absolute values, factors them into two nonnegative low-rank matrices $F$ and $G$, and scores each weight by the per-element absolute difference between the original values and the reconstruction $FG$. Weights with large residual are treated as carrying unique information and are retained. A per-layer threshold, either mean plus gamma times standard deviation or median plus gamma times median absolute deviation, produces a binary mask, and a binary search over gamma matches a user-specified global sparsity. During training, every gradient is multiplied by the mask and every weight tensor is re-masked before the optimizer step. In the paper's experiments, ONG matches or exceeds the accuracy of stable sparsification baselines at target sparsities of 80, 90, and 95 percent on CIFAR-10 and at 80 and 90 percent on CIFAR-100 with ResNet56, while keeping achieved sparsity within about half a percent of the target.","pith_inferences":["If NMF residual scores really capture structural uniqueness rather than magnitude, the same scoring could be applied to activations or gradients to produce data-dependent masks; the paper does not test that extension.","Because the mask is frozen before training and never regrown, ONG is a natural test for the lottery-ticket question: comparing masks chosen from different random initializations would show whether the NMF scores are finding durable structure or just a lucky static subnetwork.","The strict masking could be combined with structured pruning by dropping channels whose masks are mostly zero, a post-processing step the paper lists as future direction but does not evaluate."],"forward_implications":["ONG-M and ONG-S match the best stable baselines at 80 and 90 percent sparsity on CIFAR-10 and stay within about one accuracy point of the top baseline at 95 percent sparsity.","At 95 percent sparsity on CIFAR-10, ONG-M reports 91.45 percent top-1 accuracy versus 69.99 percent for soft-threshold reparameterization and 88.55 percent for dynamic sparse training, a gap the paper attributes to the stability of a fixed mask.","Achieved sparsity lands close to the target, for example 79.80 percent versus an 80 percent goal and 94.80 percent versus a 95 percent goal, so the automated gamma tuning delivers precise sparsity control.","On CIFAR-100, ONG-M at 80 percent sparsity reports 70.29 percent accuracy, matching the best baseline, while the 90 percent result remains competitive.","Because masking is applied to both gradients and weights every step, the sparse structure is preserved exactly through training, with no regrowth of pruned weights."],"supporting_citations":[{"why":"Supplies the NMF decomposition whose per-element reconstruction error defines ONG's importance score.","marker":"[31]"},{"why":"Motivates pruning at initialization, the setting ONG targets with a single pre-training mask.","marker":"[21]"},{"why":"Provides a one-shot pruning baseline and the connection-sensitivity scoring idea that ONG contrasts with NMF residuals.","marker":"[22]"},{"why":"Provides the standardized comparison harness and training protocol that all compared methods run in.","marker":"[23]"},{"why":"The gradual magnitude pruning baseline is an iterative pruning-during-training method that ONG must match.","marker":"[15]"},{"why":"A dynamic pruning baseline used in the comparison tables.","marker":"[16]"},{"why":"A stable sparsification baseline; its large accuracy drop at high sparsity is a key comparison point.","marker":"[18]"},{"why":"A sparsity-inducing baseline used for comparison.","marker":"[19]"},{"why":"A dynamic sparse training baseline used for comparison.","marker":"[20]"},{"why":"A global sparse momentum baseline used for comparison.","marker":"[25]"}],"fun_headline_variants":["One-shot NMF pruning keeps accuracy at 95% sparsity","Prune once with NMF, mask gradients, keep accuracy","NMF residual scores pick weights for one-shot sparsity","Single-pass NMF pruning matches iterative sparsifiers","Mask once, train with mask: ONG achieves 95% sparsity"],"cache_read_input_tokens":12928,"weakest_assumption_plain":"The whole pipeline rests on the assumption that NMF reconstruction-error scores computed from the initial, untrained weights already tell which weights will matter after training; if the initial weights do not carry that information, the fixed mask cannot be saved by the later training.","fun_headline_variants_meta":{"raw":{"variants":["One-shot NMF pruning keeps accuracy at 95% sparsity","Prune once with NMF, mask gradients, keep accuracy","NMF residual scores pick weights for one-shot sparsity","Single-pass NMF pruning matches iterative sparsifiers","Mask once, train with mask: ONG achieves 95% sparsity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000563,"raw_usage":{"total_tokens":2678,"prompt_tokens":960,"completion_tokens":1718,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":1631}},"tokens_in":576,"tokens_out":1718,"duration_ms":14410,"temperature":1.0,"reasoning_tokens":1631,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:17:25.676527+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train two ResNet56 models on CIFAR-10 with the same target sparsity, one masked by ONG scores and one by randomly chosen masks of the same per-layer sparsity. If the random-mask model matches ONG's accuracy across 80, 90, and 95 percent sparsity, the NMF scores are not carrying the pruning signal. A weaker check: recompute ONG masks after randomly permuting the initial weights within each layer; if accuracy does not drop, the score values themselves are irrelevant.","supporting_citations":[{"cited_title":"SNIP: Single-shot network pruning based on connection sensitivity,","cited_arxiv_id":null,"evidence_quote":"Provides a one-shot pruning baseline and the connection-sensitivity scoring idea that ONG contrasts with NMF residuals."},{"cited_title":"How I learned to stop worrying and love retraining,","cited_arxiv_id":null,"evidence_quote":"Provides the standardized comparison harness and training protocol that all compared methods run in."},{"cited_title":"Dynamic model pruning with feedback,","cited_arxiv_id":null,"evidence_quote":"A dynamic pruning baseline used in the comparison tables."},{"cited_title":"Soft threshold weight reparameterization for pruning neural networks,","cited_arxiv_id":null,"evidence_quote":"A stable sparsification baseline; its large accuracy drop at high sparsity is a key comparison point."},{"cited_title":"Winning the lottery with continuous sparsification,","cited_arxiv_id":null,"evidence_quote":"A sparsity-inducing baseline used for comparison."},{"cited_title":"Global sparse momentum SGD for pruning very deep neural networks,","cited_arxiv_id":null,"evidence_quote":"A global sparse momentum baseline used for comparison."}],"review_version":2}