{"id":"a388b4e3-79a9-4dfa-9f6f-a7c9e0ee4b66","arxiv_id":"2507.04792","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Progressive Channel Pruning iteratively prunes a few channels from validation-selected layers, improving compressed-model accuracy on ImageNet and Office-31.","lead":"This paper proposes a progressive pruning method that repeatedly removes a small number of channels from a trained neural network, choosing the layers with the least accuracy loss on a validation set. It reports accuracy gains over existing pruning methods on ImageNet and on the Office-31 transfer-learning benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The selecting step's load-bearing assumption—that per-layer validation-accuracy drops before fine-tuning predict joint post-fine-tuning utility—is never tested; if it fails, PCP's greedy layer choice is unjustified.","rationale":"The reader's weakest assumption correctly identifies the load-bearing premise: pre-fine-tune validation accuracy is used to rank layers, and top-n selection is assumed to yield a good final structure. My concern is the same, and it is genuinely central: without this proxy, the attempting and selecting steps have no demonstrated value, and the claimed superiority over one-shot methods lacks a mechanism. The paper describes the algorithm clearly and reports consistent, if small, gains in the supervised tables, which is real evidence that the framework may work; however, those tables report only final accuracies and never test the selection criterion against an alternative (e.g., random selection or selection by reconstruction loss alone). The proposed test is expensive but feasible because the inner LASSO solver has public code and the VGG-16 prunable-layer count is small enough to enumerate candidate sets. I do not think the paper merits rejection: the method is plausible and the empirical trend is supportive. But the central claim is conditional on an untested empirical assumption, so the current CONDITIONAL verdict is appropriate. I would keep the verdict unchanged while requiring the authors to validate the proxy and, ideally, add the CP-with-pseudo-labels baseline for the transfer-learning claim.","tokens_in":18746,"tokens_out":5522,"duration_ms":67439,"concrete_test":"Re-run the VGG-16/ImageNet 5x experiment, but at iteration t=1 and t=6, after the attempting step record each layer's pre-fine-tune validation drop; then for a random subset of 6 of the 12 prunable layers, fine-tune each single-layer-pruned model for 2 epochs (matching the 10-epoch setting) and record its post-fine-tune validation accuracy. Compute Spearman correlation between the pre- and post-fine-tune drops. Also, at t=1 with n=4, enumerate all C(12,4)=495 candidate selected sets and compare the actual joint validation drop of the greedy top-4 set with the best set. If the correlation is <0.7 or the greedy set is not within 0.3% top-5 accuracy of the best set, the selecting step's proxy and independence assumptions fail, and the central claim needs re-scoping.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The core mechanism of PCP is the attempting-selecting-pruning loop (Sec. III-C, Algorithm 1). For each layer, Eq. (5)-(6) produce a temporarily pruned model, and the validation accuracy of that single-layer-pruned model is used to rank layers; the top-n are then jointly pruned in Eq. (7)-(8) and not fine-tuned until the whole run ends. This makes two untested approximations load-bearing: (1) validation accuracy before fine-tuning is a faithful proxy for post-fine-tuning accuracy (the paper's only support is the sentence 'Empirically, we find...' in Sec. III-C, citing [1],[39]); and (2) the drop from pruning a set of layers is well predicted by the marginal drops of individual layers, so top-n selection is near-optimal. Neither is validated anywhere in the paper. The experiments report only final fine-tuned accuracies, so the proxy itself is never checked; and because selected layers are pruned sequentially shallow-to-deep, interactions between simultaneously pruned layers are ignored. If either approximation fails, the selecting step reduces to a heuristic with no guarantee, and the claimed advantage over one-shot or layer-by-layer pruning could come from the extra fine-tuning or from pseudo-labels rather than from the selection mechanism. The transfer-learning results additionally lack a CP-with-pseudo-labels baseline, so part of the claimed gain is not isolated, but the more fundamental issue is the untested selection criterion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Progressive Channel Pruning (PCP), an iterative three-step attempting-selecting-pruning framework for CNN channel pruning. In each iteration, the method estimates per-layer validation-accuracy drops after pruning a small number of channels (attempting), greedily selects the top-n layers with the smallest drops (selecting), and then prunes those layers using an existing channel-pruning solver, specifically the LASSO/least-squares method of He et al. (pruning). The framework is extended to unsupervised domain adaptation by pruning a pre-trained DANN model, using pseudo-labelled target samples and variance-based spatial-location selection. Experiments on ImageNet with VGG-16, AlexNet, and ResNet-50 compare PCP with Filter Pruning, Channel Pruning, AMC, ThiNet, and NISP-B; experiments on Office-31 compare PCP with Channel Pruning on DANN backbones. The paper claims that PCP automatically determines per-layer channel counts, produces a series of compressed models at intermediate compression ratios, and outperforms existing channel pruning methods under both supervised and transfer learning settings.","tokens_in":18934,"tokens_out":6842,"duration_ms":69947,"significance":"If the empirical results hold, PCP provides a simple and base-pruner-agnostic way to automate per-layer channel allocation, reducing the human effort of hand-designing per-layer channel counts. The transfer-learning extension is novel in the pruning literature and practically relevant for deploying compressed models on resource-constrained devices. The algorithm is clearly described and easy to implement, and the paper honestly reports the additional pruning time. The main limitations are that the core selection criterion is justified only by an untested heuristic assumption, the transfer-learning baseline does not isolate the pseudo-label contribution, and the experimental comparisons lack statistical significance assessments. These issues are addressable with additional ablations and reporting, so the contribution is worth further consideration after the required revisions.","major_comments":[{"comment":"The selecting step's load-bearing assumption that validation accuracy before fine-tuning predicts post-fine-tuning accuracy is asserted in one sentence ('Empirically, we find...') but never validated in this manuscript. Moreover, the top-n greedy selection assumes that the accuracy drop of jointly pruning a set of layers is well approximated by the marginal drops of individually pruned layers, and this additivity is also untested. Since the experiments report only final fine-tuned accuracies, the reader cannot check whether the proxy holds, and since selected layers are pruned jointly in shallow-to-deep order (Eqs. (7)-(8)), interactions between simultaneously pruned layers are ignored. Please add an ablation that compares the greedy selection with random selection and with selecting the largest-drop layers, reporting both pre-fine-tuning and post-fine-tuning accuracies, and a check of whether the joint drop is consistent with the marginal drops.","section":"Sec. III-C, Algorithm 1, Eqs. (5)-(8)"},{"comment":"The comparison of PCP (w/ PL) with CP [1] does not isolate the effect of pseudo-labelling because the two methods differ in both the progressive selection mechanism and the use of pseudo-labelled target samples; there is no CP-with-pseudo-labels baseline. Consequently, large gains attributed to pseudo-labelling (e.g., ResNet-50 at 5x, A→W: 78.4 vs. 71.8 in Table VI) could be due to the self-training/pseudo-label component rather than the progressive pruning framework. Please add a CP + pseudo-labels baseline or an ablation in which PCP (w/o PL) is fine-tuned with pseudo-labels only at the fine-tuning stage.","section":"Sec. IV, Tables IV-VI"},{"comment":"All reported accuracies are from single runs with no standard deviations, confidence intervals, or significance tests. Several margins are small (e.g., VGG-16 at 2x: 90.0 vs. 89.9; 4x: 89.3 vs. 88.9 in Table I), so the claim that PCP 'consistently outperforms' existing methods is not statistically supported. Please report results over multiple seeds or at least include standard deviations and state whether differences are significant.","section":"Tables I-VI"},{"comment":"The claim that PCP is 'the first channel pruning framework that can automatically decide the network structure' is contradicted by the paper's own description of AMC [39], which uses reinforcement learning to decide the number of remaining channels per layer, and by NISP-B [54], which prunes channels based on global importance scores across layers. The novelty claim should be qualified (e.g., 'first progressive greedy framework that automatically decides per-layer channel counts for UDA') or the positioning should be revised.","section":"Sections I and II-A"},{"comment":"The two domain-adaptation-specific mechanisms are under-specified: the 'highly confident' threshold for assigning pseudo-labels and the 'empirically defined threshold' for spatial-location variance are never given numerical values, and no sensitivity analysis is provided. Since these thresholds are free parameters that can materially affect the transfer-learning results, please specify the values and report ablations over them.","section":"Sec. IV"}],"minor_comments":[{"comment":"The Index Terms contain the typo 'Pruining' and should be 'Pruning'.","section":"Index Terms"},{"comment":"The caption of Table I says 'IMAGE NET' and should be 'ImageNet'; the same misspelling appears in the heading of Section V-A.","section":"Table I and Sec. V-A"},{"comment":"The phrase 'optimal network structure' is too strong for a greedy heuristic and should be 'effective network structure' or 'good network structure'.","section":"Introduction and Sec. III-C"},{"comment":"The hyperparameter sensitivity discussion reports variations of 88.7-89.2 for top n and 88.8-89.2 for f_t, which supports 'not very sensitive,' but this should be accompanied by the number of runs used to compute those numbers.","section":"Sec. V-A"},{"comment":"The attempting step is described in Sec. III-C as 'can be performed in parallel' but is implemented as a sequential for-loop in Algorithm 1; since parallelism is mentioned as future work in Sec. VI, the wording in Sec. III-C should be aligned with the current implementation to avoid confusion.","section":"Sec. III-C and Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The reader's conditional verdict is fair. The core algorithm is plausible and worth publishing after the selection-proxy assumption is tested and the transfer-learning ablation is fixed. The novelty claim against AMC and NISP-B should be softened. I do not see evidence of circularity; the method is an empirical heuristic with held-out evaluation. The stress-test concern about the untested selection criterion is valid and should be addressed with a dedicated ablation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is the iterative attempting-selecting-pruning loop: at each step, the method prunes a few channels from each layer temporarily, measures validation accuracy drop, then greedily picks the top-n layers and prunes them together. That specific mechanism is not in the cited prior work, and the transfer-learning extension to DANN with pseudo-labeled target samples is also novel as far as I know. The paper is clearly written, the solver is the well-known LASSO/least-squares one from [1], and the results are consistently positive across VGG, AlexNet, ResNet on ImageNet and Office-31. The gains are small, but real in the reported tables.\n\nThe biggest soft spot is exactly what the stress-test flags: the selection step rests on an assumption that is never directly tested. The authors claim, with one sentence and two citations, that pre-fine-tuning validation accuracy predicts post-fine-tuning accuracy, and that top-n selection by individual layer drops approximates the joint drop. No ablation compares the greedy selection against random or uniform layer selection, and no experiment checks whether the proxy ranking actually matches the final fine-tuned ranking. The final accuracies are consistent with the proxy working, but they do not isolate it. If that proxy fails, the whole selecting step is a heuristic without a leg to stand on.\n\nThe second real issue is the transfer-learning comparison. PCP with pseudo-labels is compared against CP without pseudo-labels, and against PCP without pseudo-labels. There is no CP-with-pseudo-labels baseline, so part of the w/ PL gain could simply be self-training. That is an easy experiment to run and it should be added.\n\nMinor points: no error bars or statistical significance; some hyperparameters (top n, f_t, variance threshold) are set empirically with limited sensitivity analysis; no public code for PCP itself. The novelty claim of being \"the first to automatically decide network structure\" is too strong given AMC already searches structure with reinforcement learning. The specific progressive greedy is new, but that broader claim should be softened.\n\nFor the channel-pruning community this is a useful incremental contribution, not a breakthrough. It deserves a serious referee and, in my view, conditional acceptance after the missing baseline and a selection-proxy ablation are added.","headline":"A genuinely new iterative layer-selection mechanism for channel pruning, with consistent but small gains, held back by an untested selection proxy and a missing pseudo-label baseline in the transfer experiments.","tokens_in":19575,"tokens_out":2921,"would_cite":true,"duration_ms":34632,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that iteratively pruning a few channels from the least harmful layers—selected greedily by validation-accuracy drop before fine-tuning—produces compressed CNNs that beat one-shot channel pruning and automatically…","keywords":["progressive channel pruning","model compression","convolutional neural networks","greedy layer selection","unsupervised domain adaptation","domain adversarial neural network","pseudo-labeling","ImageNet"],"falsifier":"Train a small network where all possible per-layer channel allocations at a fixed FLOPs budget can be enumerated; run PCP to the same budget, fine-tune every allocation identically, and compare. If any non-greedy allocation that PCP never selected matches or beats PCP's post-fine-tuning accuracy, the greedy selection proxy is not reliable. A cheaper check is to compare PCP's per-iteration selections against selections made with a few hundred fine-tuning steps inserted between iterations.","tokens_in":18439,"feed_emoji":"✂️","tokens_out":5886,"duration_ms":55026,"temperature":0.7,"pith_summary":"This paper proposes Progressive Channel Pruning (PCP), a framework that compresses CNNs by iteratively removing a small number of channels from several layers at a time, rather than pruning each layer once in a fixed order. At every iteration the method tries pruning each candidate layer, measures the validation-accuracy drop, greedily selects the layers whose pruning hurts least, and then prunes exactly those layers. The same loop is extended to unsupervised domain adaptation by feeding a domain-adversarial network pseudo-labeled target samples during pruning. The authors report that PCP outperforms existing channel pruning methods on ImageNet for VGG-16, AlexNet, and ResNet-50, and that it preserves or improves DANN accuracy on Office-31 under compression.","feed_headline":"Progressive pruning beats one-shot channel pruning on ImageNet","feed_subtitle":"Picking the least harmful layers each round sets per-layer channel budgets automatically and keeps accuracy high.","key_machinery":"The attempting-selecting-pruning loop carries the argument: each iteration independently solves the LASSO channel-selection problem (Eq. (5)) and least-squares weight reconstruction (Eq. (6)) for every candidate layer to estimate its accuracy drop, then a greedy step selects the top-n layers with the smallest drops and prunes them in shallow-to-deep order via Eqs. (7)-(8), using validation accuracy before fine-tuning as a proxy for final accuracy. This loop is what converts a per-layer pruning primitive into an automatic, progressive global structure search.","core_discovery":"The central claim is that compression decisions should be made progressively and globally: at each iteration, temporarily prune each candidate layer, rank layers by the resulting validation-accuracy drop before fine-tuning, and prune only the top-n least harmful layers, repeating until the target FLOPs or parameter ratio is reached (Algorithm 1, Eqs. (5)-(8)). This replaces the common practice of pruning each layer once with a pre-designed per-layer channel budget, and also replaces per-layer one-shot pruning with a loop in which shallower layers are pruned before deeper ones so that deeper layers' LASSO-based weight reconstruction sees the updated input features. Because the per-layer attempts are independent, the framework can reuse any existing channel pruning primitive, and the authors instantiate it with the LASSO-plus-least-squares method of [1]. Under the transfer-learning extension, the same loop is run on a DANN model using both labeled source samples and pseudo-labeled target samples, plus spatial positions selected for informative response variance, which the paper argues reduces domain distribution mismatch during compression. The experimental claim is that this automatically determined structure yields higher post-fine-tuning accuracy than one-shot baselines and than the reinforcement-learning-based AMC at matched compression ratios.","pith_inferences":["The attempting-selecting loop is a generic search wrapper: the same validation-drop oracle could schedule quantization bit-widths, block- or head-pruning, or layer dropping, since it never depends on how the candidate compression is performed.","The pre-fine-tuning proxy is the main empirical risk; inserting cheap partial fine-tuning between iterations would test whether the greedy selections track final accuracy, without changing the framework's structure.","Pseudo-label noise in the UDA extension could be mitigated by weighting target samples by prediction confidence or entropy, an option the authors do not explore.","Because the per-layer attempts are independent and parallelizable, the framework's overhead (about 7-12% extra wall-clock time in the paper's measurements) should shrink further on multi-GPU setups, which the authors identify as future work."],"forward_implications":["PCP removes the need to hand-design per-layer channel budgets: the greedy loop decides how many channels survive in each layer, and the resulting distribution matches human-designed trends such as deeper layers retaining more channels.","A single run yields compressed models at every intermediate compression ratio, so a deployment that later needs a smaller model does not require a new pruning run.","At matched compression ratios, PCP reports higher top-1/top-5 accuracy than one-shot channel pruning on VGG-16, AlexNet, and ResNet-50 on ImageNet, and higher than AMC at 5x compression.","In the transfer setting, pruning the DANN model with pseudo-labeled target samples keeps accuracy at or above the uncompressed DANN model at 2x-5x compression across Office-31 tasks, whereas source-only pruning degrades it.","Any existing channel pruning method can be plugged into the attempting and pruning steps, so improvements in the underlying pruning primitive transfer directly to PCP."],"supporting_citations":[{"why":"provides the LASSO channel-selection and least-squares weight-reconstruction steps that PCP invokes in each attempt and prune operation.","marker":"[1]"},{"why":"defines the ThiNet baseline that PCP must beat on ResNet-50 and supplies the filter-level pruning comparison.","marker":"[2]"},{"why":"is the ImageNet benchmark where the supervised-learning experiments are run.","marker":"[3]"},{"why":"is the Office-31 benchmark used for the unsupervised domain adaptation experiments.","marker":"[4]"},{"why":"defines the Domain Adversarial Neural Network that PCP prunes and extends with pseudo-labels.","marker":"[5]"},{"why":"is the filter pruning baseline whose reported results are compared on VGG-16.","marker":"[24]"},{"why":"is the AMC reinforcement-learning baseline that also automates per-layer channel counts and is compared at 5x compression.","marker":"[39]"},{"why":"is the DANN training procedure used to obtain the initial model to be pruned in the transfer setting.","marker":"[47]"},{"why":"is the NISP baseline that prunes AlexNet globally and is compared on FLOPs and accuracy.","marker":"[54]"}],"fun_headline_variants":["Progressive pruning: keep accuracy by pruning least harmful layers","Iterative channel pruning beats one-shot methods on ImageNet","PCP: progressive channel pruning for smaller, faster CNNs","Prune progressively: select layers via accuracy drop ranking"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the accuracy of a pruned model measured before fine-tuning reliably predicts its accuracy after fine-tuning, so greedily choosing layers with the smallest pre-fine-tuning drop produces the best final compressed model.","fun_headline_variants_meta":{"raw":{"variants":["Progressive pruning: keep accuracy by pruning least harmful layers","Iterative channel pruning beats one-shot methods on ImageNet","PCP: progressive channel pruning for smaller, faster CNNs","Prune progressively: select layers via accuracy drop ranking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000628,"raw_usage":{"total_tokens":2944,"prompt_tokens":1026,"completion_tokens":1918,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":642,"completion_tokens_details":{"reasoning_tokens":1851}},"tokens_in":642,"tokens_out":1918,"duration_ms":14859,"temperature":1.0,"reasoning_tokens":1851,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:39:11.797022+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a small network where all possible per-layer channel allocations at a fixed FLOPs budget can be enumerated; run PCP to the same budget, fine-tune every allocation identically, and compare. If any non-greedy allocation that PCP never selected matches or beats PCP's post-fine-tuning accuracy, the greedy selection proxy is not reliable. A cheaper check is to compare PCP's per-iteration selections against selections made with a few hundred fine-tuning steps inserted between iterations.","supporting_citations":[{"cited_title":"Channel pruning for accelerating very deep neural networks,","cited_arxiv_id":null,"evidence_quote":"provides the LASSO channel-selection and least-squares weight-reconstruction steps that PCP invokes in each attempt and prune operation."},{"cited_title":"Thinet: A filter level pruning method for deep neural network compression,","cited_arxiv_id":null,"evidence_quote":"defines the ThiNet baseline that PCP must beat on ResNet-50 and supplies the filter-level pruning comparison."},{"cited_title":"Imagenet large scale visual recognition challenge,","cited_arxiv_id":null,"evidence_quote":"is the ImageNet benchmark where the supervised-learning experiments are run."},{"cited_title":"Adapting visual category models to new domains,","cited_arxiv_id":null,"evidence_quote":"is the Office-31 benchmark used for the unsupervised domain adaptation experiments."},{"cited_title":"Domain-adversarial training of neural networks,","cited_arxiv_id":null,"evidence_quote":"defines the Domain Adversarial Neural Network that PCP prunes and extends with pseudo-labels."},{"cited_title":"Pruning filters for efficient convnets,","cited_arxiv_id":null,"evidence_quote":"is the filter pruning baseline whose reported results are compared on VGG-16."},{"cited_title":"AMC: Automl for model compression and acceleration on mobile devices,","cited_arxiv_id":null,"evidence_quote":"is the AMC reinforcement-learning baseline that also automates per-layer channel counts and is compared at 5x compression."},{"cited_title":"Unsupervised domain adaptation by backpropagation,","cited_arxiv_id":null,"evidence_quote":"is the DANN training procedure used to obtain the initial model to be pruned in the transfer setting."},{"cited_title":"NISP: Pruning networks using neuron importance score propagation,","cited_arxiv_id":null,"evidence_quote":"is the NISP baseline that prunes AlexNet globally and is compared on FLOPs and accuracy."}],"review_version":1}