{"id":"1a56440e-3a72-448d-abd7-70fca6e23f07","arxiv_id":"2506.11469","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A graph-embedding search over discretized pruning rates, guided by contrastive learning, claims high compression on CIFAR-10 but lacks key validation.","lead":"The paper introduces SACP, which uses graph neural networks to rank channel pruning plans by similarity to the original network. The reported compression gains on CIFAR-10 are large, but missing ImageNet results, inconsistent numbers, and test-set selection weaken the evidence.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The ranking step in Eq. 10 is never validated against random selection, and the test-set-based final choice can produce reported gains even if the embedding similarity carries no signal.","rationale":"I agree with the reader's weakest-assumption analysis: the central claim depends on cosine similarity in Eq. 10 being a reliable proxy for post-pruning quality, and the paper never tests that proxy directly. My stress-test adds that the evaluation protocol makes this omission decisive. Because Algorithm 1 and Section 4.2 permit selecting the final model on test accuracy, the reported numbers can be inflated by selection among many retrained candidates even if the similarity score is pure noise. Thus the missing validation of Eq. 10 is not a minor gap; it is the difference between a structure-aware search and an arbitrary ranking followed by expensive test-set cherry-picking. The proposed random-ranking baseline isolates exactly the contribution of Eq. 10 while leaving the rest of the pipeline unchanged. If random ranking performs comparably, the central claim is refuted. If random ranking performs worse, the paper's mechanism would gain its first direct evidence. Either outcome is informative, and that is precisely why the absence of this experiment is the most load-bearing concern. The paper does not offer machine-checked proofs, released code, or parameter-free derivations that could substitute for this validation, so I concur with the reader's REJECT verdict.","tokens_in":12215,"tokens_out":6711,"duration_ms":68744,"concrete_test":"Run the full SACP pipeline on ResNet-18/CIFAR-10 with one change: after generating the constrained candidate pool, select top-m uniformly at random instead of by s(j), keep all fine-tuning and top-k steps identical, and choose the final model by validation accuracy, not test accuracy. Repeat with three random seeds and compare final test accuracy to the reported SACP result. If random selection matches SACP within seed noise, then Eq. 10 provides no measurable predictive signal and the central claim is unsupported.","verdict_should_be":"REJECT","load_bearing_attack":"The load-bearing step is Eq. 10 in Section 3.5: s(j) = cos(z(j), z_orig) is claimed to indicate structural preservation and therefore expected pruned-model quality. The paper provides no evidence for this causal chain. No correlation between s(j) and actual accuracy is reported; no ablation replaces similarity-based selection with random or low-similarity selection; and top-similarity candidates are never compared with bottom-similarity candidates at matched compression. The downstream pipeline in Algorithm 1 and Section 4.2 (top-m by similarity, top-k by validation accuracy, final model chosen by best test accuracy) means that the published results could be produced by retraining many candidates and selecting on test labels, with the embedding contributing no signal. Section 4.2 states: 'Final performance is reported based on the best test accuracy'; Algorithm 1 line 16 returns 'Final pruned model with best accuracy.' This also weakens the claim of a fully automated search. The decisive experiment is to hold candidate generation, fine-tuning, and validation-based selection fixed, and replace the s(j)-based top-m filter with random ranking, choosing the final model by validation rather than test accuracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SACP, a channel-pruning framework that represents a CNN as a DAG with per-channel binary masks, trains a GCN encoder using a contrastive loss on L1-pruned and randomly pruned variants, and then searches a discretized pruning-rate space by ranking candidates by cosine similarity between the candidate embedding and the original network embedding. The top-ranked candidates are lightly fine-tuned, the top-k are fully retrained, and the final model is selected by test accuracy. The reported experiments claim strong compression with competitive accuracy on VGG-16, ResNet-18, and ResNet-56 on CIFAR-10, and the abstract additionally claims ImageNet experiments.","tokens_in":12501,"tokens_out":5735,"duration_ms":54266,"significance":"If the similarity-ranking mechanism in Eq. (10) were validated, the framework would offer an attractive way to automate channel-rate search with structural information. The paper is transparent about its search space and spells out Algorithm 1, and the graph encoding is a reasonable way to couple topology with pruning state. However, the empirical support for the central mechanism is missing: there is no validation that cosine similarity to the original embedding ranks candidates by quality, no random baseline, and the final selection on test labels undermines the reported numbers. The absence of the claimed ImageNet results and the internal numerical inconsistencies further reduce confidence in the experimental claims.","major_comments":[{"comment":"The central ranking mechanism is unvalidated: the paper never reports a correlation between the similarity score s(j) and actual post-pruning accuracy, never compares top-similarity with low-similarity candidates at matched compression, and never replaces the similarity filter with a random baseline in an ablation. Without one of these tests, the claim that higher cosine similarity to the original embedding predicts better pruning quality is unsupported, and the downstream results could be produced by the retraining and selection pipeline alone.","section":"Section 3.5, Eq. (10)"},{"comment":"The final model is selected using test accuracy: Section 4.2 states 'Final performance is reported based on the best test accuracy' and Algorithm 1 returns the 'Final pruned model with best accuracy.' This makes the reported numbers best-of-k selections made on the test labels, so they are not unbiased estimates of pruning quality and do not support the claim of a fully automated, structure-aware search. The validation-based top-k selection should be followed by a single final evaluation on the test set, not selection by test accuracy.","section":"Section 4.2 / Algorithm 1"},{"comment":"The abstract and introduction state that experiments are conducted on both CIFAR-10 and ImageNet, but Section 4 contains no ImageNet results; all tables and ablations are on CIFAR-10. The abstract's claim of 'extensive experiments on benchmark datasets (CIFAR-10, ImageNet)' is therefore unsupported by the manuscript as written.","section":"Abstract / Section 4"},{"comment":"The reported ResNet-56 result drops from 94.58% baseline accuracy to 90.06%, a 4.52-point degradation, while the text says SACP 'caused only a small accuracy degradation.' This drop, at 78% FLOP reduction, is not small relative to the baselines in Table 3 and needs to be discussed as a trade-off, ideally with comparisons at matched FLOPs.","section":"Table 3 / Section 4.3"},{"comment":"The text reports the full SACP ResNet-18 accuracy as 91.54%, while Table 4 reports 93.41% for the same configuration; Table 4 also displays the SACP-1 row as '88.7391.4991.32' with no column separators. These inconsistencies make the experimental results difficult to interpret and must be corrected before the results can be assessed.","section":"Section 4.4 / Table 4"},{"comment":"The positive training samples are generated using L1-norm pruning, so the contrastive encoder may be learning a representation of L1-style channel importance rather than global structural information. The paper provides no matched-compression comparison against L1 pruning or against a similarity ranking based on random embeddings, so the claimed structure-aware advantage is not demonstrated.","section":"Section 3.4"}],"minor_comments":[{"comment":"\"KeywordsChannel\" is missing a space; it should be \"Keywords: Channel Pruning; ...\".","section":"Abstract / Keywords"},{"comment":"The method is described as \"unsupervised comparison learning\" in Section 1 and as \"supervised contrast learning\" in Section 3.4; the terminology should be made consistent.","section":"Sections 1 and 3.4"},{"comment":"The GAL-0.6 FLOPs entry \"78.30\" is missing units (M) and appears incomplete; the row formatting in the ResNet-56 block makes the columns difficult to read.","section":"Table 3"},{"comment":"The \"light fine-tuning\" stage for the top-m candidates is not specified; reporting the number of epochs, learning rate, and data splitting for this stage would improve reproducibility.","section":"Section 4.2 and Algorithm 1"},{"comment":"Reference [32] is formatted inconsistently as \"Cornell University - arXiv, Aug 2016\" rather than as a conference paper; the reference list should be normalized.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript's experimental protocol (final selection on test accuracy) means the headline numbers cannot be taken at face value, and the abstract promises ImageNet results that are absent from the experimental section. The central similarity-based ranking is never validated against a random baseline. If the authors redo the evaluation with validation-based selection and add the missing baseline and ImageNet comparisons, a resubmission could be considered, but the current manuscript does not support its claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a legitimate idea buried under an evaluation that doesn't support it. The novelty is modest—GCN-based pruning already exists (AGMC) and contrastive learning for architecture representation isn't new—but the specific trick of ranking a large pool of discretized pruning configurations by cosine similarity to the original model's embedding is at least a fresh combination. The pipeline is coherent, and the CIFAR-10 compression numbers are impressive if they hold. The ablations (SACP-1 and SACP-2) show the contrastive objective and the two-stage selection both contribute, which is real evidence that the components matter.\n\nThe soft spots are serious. The abstract and conclusion claim ImageNet results, but the full text contains no ImageNet experiments—no table, no numbers. That is a direct mismatch between claim and evidence. Second, the final model is chosen using test accuracy (Section 4.2 and Algorithm 1), so the reported numbers reflect best-of-k selection, not predictive power. Third, the load-bearing assumption—that cosine similarity between candidate and original embeddings predicts pruned-model accuracy—is never validated. There is no correlation analysis, no random ranking baseline, no comparison of top- vs bottom-similarity candidates. Without that, the search step could be arbitrary, and the results could come from retraining many candidates and picking the best. Fourth, internal inconsistencies (SACP accuracy 91.54% in Section 4.4 text vs 93.41% in Table 4, and ResNet-56's 4.5% drop described as \"small\") undermine trust. The citation pattern looks fine; the related work is adequately covered.\n\nI agree with the reader's reject verdict. The central mechanism is unproven and the evaluation protocol is biased. But the underlying idea has enough substance that a revision could turn this into a solid paper. It deserves peer review, not a desk reject, because the flaws are fixable and the problem is real. Recommend sending to referees with a strong request to redo the evaluation: add ImageNet results, validate the similarity ranking against random search, and select the final model on validation rather than test. I'd bring this to reading group as a cautionary example of how selection on test labels can inflate pruning results.","headline":"A fresh but unvalidated pruning pipeline whose reported results are undermined by test-set selection, missing ImageNet experiments, and an unproven similarity-ranking assumption.","tokens_in":13016,"tokens_out":2629,"would_cite":false,"duration_ms":24791,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that ranking pruned networks by cosine similarity of GCN embeddings picks configurations that keep accuracy, enabling automatic channel pruning with 85% FLOPs reduction.","keywords":["channel pruning","graph convolutional networks","structure-aware pruning","pruning rate search","contrastive learning","network compression","cosine similarity","model deployment"],"falsifier":"Rank-correlation test: sample, say, 100 candidate configurations from the search space, compute their cosine similarities to the original embedding, lightly fine-tune each, and compute the Spearman rank correlation between similarity and final validation accuracy; a correlation near zero would show the similarity signal does not carry the claimed information and the search reduces to random screening.","tokens_in":11996,"feed_emoji":"✂️","tokens_out":7134,"duration_ms":58908,"temperature":0.7,"pith_summary":"This paper proposes SACP, a method that treats a convolutional neural network as a graph and uses a graph convolutional network (GCN) to embed the original network and many pruned variants of it into a vector space. The paper's central claim is that cosine similarity between a candidate pruned network's embedding and the original network's embedding predicts how well that candidate will retain accuracy after pruning. With this ranking, SACP searches a discretized space of layer-wise pruning ratios, lightly fine-tuning the top-ranked candidates and fully retraining the best few. On CIFAR-10 it reports removing roughly 82–85% of FLOPs and over 92% of parameters from VGG-16 and ResNet-18 while keeping accuracy close to or above the unpruned baseline, and it claims better compression than existing baselines.","feed_headline":"Graph-search pruning cuts 85% of ResNet-18's FLOPs","feed_subtitle":"A GCN encodes network structure, and candidates are ranked by cosine similarity to the original model.","key_machinery":"The load-bearing machinery is the pair of graph representation and GCN encoder, together with the cosine similarity score $s^{(j)} = \\cos(z^{(j)}, z_{\\text{orig}})$ from Eq. (10). Each candidate pruned network becomes a directed acyclic graph whose nodes are layers and whose binary node features mark which channels are retained; the GCN encodes this into an embedding, and the contrastive loss (Eq. 8) with L1-norm-pruned positives and random-pruned negatives shapes the embedding space. The search space is the discrete set of per-layer pruning rates $r_l \\in \\{0, 0.1, \\ldots, 0.9\\}$ constrained by a global pruning threshold $R(r) \\ge \\tau$. This machinery lets the paper screen millions of candidates with one forward pass through the GCN before any fine-tuning.","core_discovery":"The discovery the paper asserts is that the embedding of a network graph, learned by contrastive training on L1-norm pruned variants (positive) versus randomly pruned variants (negative), carries enough structural information that a simple cosine similarity to the original network's embedding ranks pruning configurations by expected quality. Section 3.5 states this directly: a higher similarity score suggests the pruned model retains more of the original network's important features and is expected to perform better after pruning. The GCN encoder is trained on graphs whose node features are binary masks of retained channels, with layer topology encoded as edges, including residual skip connections. After training, the search step encodes the original model and a large pool of candidate configurations, selects the top-m by cosine similarity, fine-tunes them, then fully retrains the top-k by validation accuracy to pick the final pruned network.","pith_inferences":["One check the paper does not run is a direct correlation between similarity scores and post-fine-tuning accuracy; doing so on a random sample of candidates with the same global pruning threshold would show whether the ranking signal is real or whether the final accuracy comes mainly from the retraining stage.","Because the graph features are binary masks only, the encoder never sees weight magnitudes; if similarity ranking transfers across weight initializations or training epochs, the method could be used for early-exit or training-free pruning decisions, but the paper does not test that.","The contrastive supervision signal is implicitly tied to L1-norm pruning as the definition of 'good' pruning; replacing the positive-sample generator with another heuristic would shift what the embedding ranks as good, which is an untested design choice.","A natural extension is to apply the same graph-embedding search to layer-level pruning or to non-convolutional blocks (transformers), since the graph abstraction does not depend on convolution specifically; the paper leaves that unexplored."],"forward_implications":["If cosine similarity in the learned embedding space truly tracks pruning quality, then a single forward pass through the GCN can rank millions of candidate pruning configurations without retraining each one.","The same graph encoder can be reused across different backbone architectures (VGG-16, ResNet-18, ResNet-56) with only the graph structure changed, so pruning becomes a search over binary masks rather than a hand-designed per-layer rule.","The global pruning threshold $\\tau$ gives a practical control knob: a deployment team can set the minimum compression level and let the search find the best accuracy within that budget.","The two-stage selection (top-m light fine-tuning, then top-k full retraining) allows the search to rely on cheap signals for screening while reserving expensive training for the most promising candidates."],"supporting_citations":[{"why":"Supplies the graph convolutional network architecture used to encode network graphs into embeddings.","marker":"[38]"},{"why":"Provides the contrastive learning formulation (SimCLR) that the paper's contrastive loss is based on.","marker":"[44]"},{"why":"L1-norm filter pruning is used to generate positive training samples in the contrastive learning setup.","marker":"[32]"},{"why":"Establishes the idea of representing a neural network as a graph, which the paper adapts for pruning state representation.","marker":"[40]"},{"why":"A reinforcement-learning automatic pruning method that the paper cites as the main automatic-pruning alternative to its search-based approach.","marker":"[18]"}],"fun_headline_variants":["GCN-based pruning finds optimal channel cuts via graph similarity","Structure-aware pruning uses graph embeddings to rank channel configurations","Automated pruning searches graph embeddings to preserve structural importance","Graph-embedding search prunes channels by cosine similarity to original model","GCN ranks pruning configs by similarity to original network"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes, without measuring it, that a candidate pruned network's cosine similarity to the original network in the learned embedding space predicts how accurately that candidate will perform after fine-tuning.","fun_headline_variants_meta":{"raw":{"variants":["GCN-based pruning finds optimal channel cuts via graph similarity","Structure-aware pruning uses graph embeddings to rank channel configurations","Automated pruning searches graph embeddings to preserve structural importance","Graph-embedding search prunes channels by cosine similarity to original model","GCN ranks pruning configs by similarity to original network"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000507,"raw_usage":{"total_tokens":2449,"prompt_tokens":902,"completion_tokens":1547,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1465}},"tokens_in":518,"tokens_out":1547,"duration_ms":10772,"temperature":1.0,"reasoning_tokens":1465,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:03:54.831824+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rank-correlation test: sample, say, 100 candidate configurations from the search space, compute their cosine similarities to the original embedding, lightly fine-tune each, and compute the Spearman rank correlation between similarity and final validation accuracy; a correlation near zero would show the similarity signal does not carry the claimed information and the search reduces to random screening.","supporting_citations":[{"cited_title":"A simple framework for contrastive learning of visual representations","cited_arxiv_id":null,"evidence_quote":"Provides the contrastive learning formulation (SimCLR) that the paper's contrastive loss is based on."},{"cited_title":"Pruning filters for efficient convnets","cited_arxiv_id":null,"evidence_quote":"L1-norm filter pruning is used to generate positive training samples in the contrastive learning setup."},{"cited_title":"Graph structure of neural networks","cited_arxiv_id":null,"evidence_quote":"Establishes the idea of representing a neural network as a graph, which the paper adapts for pruning state representation."},{"cited_title":"Amc: Automl for model compression and acceleration on mobile devices","cited_arxiv_id":null,"evidence_quote":"A reinforcement-learning automatic pruning method that the paper cites as the main automatic-pruning alternative to its search-based approach."}],"review_version":1}