{"id":"70d464ba-86f7-4e60-b7d7-45cd9d8ba12f","arxiv_id":"2412.15750","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A greedy, ablation-based pruning method extracts a standalone task-specific subnetwork from GPT-2 Small, reducing parameters by up to 82.77% while keeping accuracy on three synthetic single-token tasks.","lead":"The authors present a method that prunes a large language model down to the set of components responsible for a single task, producing a smaller standalone model without any retraining. The work sits at the intersection of mechanistic interpretability and model compression, aiming for faster, more transparent single-task deployment.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No held-out evaluation: reported accuracies are measured on the validation set used for threshold selection, so the claimed task-preservation may not generalize.","rationale":"The paper's strongest claim is that a task-specific subset can be extracted with no additional training and a small amount of data. For this to hold, the pruned model must actually perform the task on new inputs. The algorithm's design (KL-based greedy pruning) and the manual circuit comparisons (RQ3) are interesting, but the evaluation protocol leaves the performance claim unverified: alpha is tuned on Dv and accuracy is reported on Dv. This is a fundamental evaluation validity problem. While the reader identified the KL-divergence proxy as the weakest assumption, that concern is partially acknowledged by the authors in Appendix D, and it does not by itself invalidate the mean-ablation results used in Table 1. In contrast, the absence of a held-out test affects every reported accuracy and directly undermines the empirical support for the central claim. The paper does provide code and reproducible experiments, which is credit-worthy, and the speedups are measured, but the accuracy numbers need independent validation. Thus the reader's CONDITIONAL verdict remains appropriate, with the added explicit condition of a held-out test.","tokens_in":17006,"tokens_out":6200,"duration_ms":54398,"concrete_test":"Create a held-out test set for each task by sampling new prompts from the same template distributions, disjoint from both the patching set and the validation set used for alpha selection. Re-run the pipeline (or simply apply the already-selected alpha values) and measure accuracy, parameter reduction, and inference latency on this test set. Compare to Table 3. If held-out accuracy drops significantly below the reported validation accuracy (e.g., more than a few points), the central claim of task preservation without training is not supported; if accuracy matches, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is that the paper never evaluates on a held-out test set. Algorithm 1 (Section 3) prunes components using the validation dataset Dv, and the thresholds alpha are selected from the accuracy-vs-size curves in RQ1 (Figures 2-7), which are computed on Dv. Table 1 then reports the accuracy of the resulting pruned models, and Table 3 in Appendix E reports extended results, but no independent test set is ever split off. Consequently, the headline numbers (e.g., 99.92% accuracy at 32.88% parameter reduction on acronyms) are in-sample estimates on the very data used to choose alpha. With only 150-250 samples per batch, the risk of overfitting to Dv is high. The averaging over five batches does not fix this if those batches are drawn from and evaluated on the same distribution used for selection. The central claim that the extracted subset 'properly performs a targeted task' in a standalone manner requires out-of-sample evidence; without it, the impressive accuracy could be an artifact of threshold tuning, and the claimed speedups, while plausible, are not tied to a verified task-preserving model.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method to automatically prune GPT-2 Small into a task-specific submodel (a 'circuit') by iterating over attention heads and optionally MLPs in reverse layer order, temporarily ablating each component, and permanently removing it if the increase in KL divergence between the ablated model and the original model's predictions on a validation set falls below a threshold α. The method requires no additional training and small datasets. The authors evaluate on three tasks whose circuits were manually identified in prior work: 3-letter acronym prediction, indirect object identification (IOI), and greater-than. They report parameter reductions up to 82.77%, inference speedups up to 88%, and accuracy that preserves or improves over the original model, along with low false-positive rates relative to the manually identified circuits. They also compare against a distillation baseline.","tokens_in":17241,"tokens_out":4941,"duration_ms":42226,"significance":"If the claims are validated, the work is a useful bridge between mechanistic interpretability and model compression: it is among the first to make extracted circuits usable as standalone, smaller, faster models without fine-tuning, and it provides a concrete algorithm with released code and data. The use of three tasks with known circuits is a strength, as is the honesty in reporting low true-positive rates relative to manual circuits. However, the significance is currently limited by evaluation gaps: no held-out test set, a pruning proxy (KL divergence) that the paper itself shows can diverge from task accuracy, and a relatively narrow experimental scope (GPT-2 Small, single-token prediction, synthetic templates). The core idea is promising and the direction is valuable, but the evidence as presented does not yet support the strong claims in the abstract.","major_comments":[{"comment":"The reported accuracies are computed on the same validation set Dv that was used to select the threshold α in RQ1 (Figures 2–7). No held-out test set is ever split off. Since α is chosen by inspecting accuracy-versus-size curves on Dv, the headline numbers (e.g., 99.92% accuracy at 32.88% parameter reduction on acronyms) are in-sample estimates. With batch sizes of only 150–250 samples, the risk of overfitting to Dv is non-negligible, and averaging over five batches drawn from the same distribution does not mitigate selection bias. The central claim that the extracted submodel properly performs the task in a standalone manner requires out-of-sample evidence; please report accuracies on a test set that is not used for threshold selection.","section":"Section 4, Table 1 and Appendix E, Table 3"},{"comment":"The pruning criterion is the increase in KL divergence between the ablated model and the original model, but the reported success metric is task accuracy. The paper itself documents a counterexample in Appendix D: for the greater-than task, with zero ablation and MLPs included, submodels around 50% size achieve 100% accuracy while having large KL divergence, and models at near-100% size reduction still reach 100% accuracy because the model degenerates to always predicting the starting year. This shows that KL divergence is not reliably correlated with task performance. Because every pruning decision in Algorithm 1 is made using this proxy, the method may remove task-critical components as long as the KL divergence stays low, or retain components that do not help accuracy. The paper acknowledges this phenomenon but leaves it out of scope; please adopt a task-appropriate metric (e.g., logit difference or direct accuracy) or provide evidence that KL-based pruning nonetheless recovers the correct submodel in this case.","section":"Algorithm 1 (Section 3) and Appendix D (Figure 7)"},{"comment":"The true-positive rates against manually identified circuits are low: 20–40% for acronyms, 25–57% for IOI and greater-than, while false-positive rates are low. The authors explain that many manually identified heads contribute small amounts and that different ablation schemes are used, but the claim that the pruned models are 'more interpretable' or 'focus on the circuit' is undercut when a large fraction of the known circuit is absent. Please provide a more direct analysis, for example by measuring how much task accuracy is recovered when the missing heads are re-added, or by quantifying the contribution of each missing head, so that the retained heads can be shown to be the functionally important ones.","section":"Section 4, RQ3 and Table 1"}],"minor_comments":[{"comment":"The loop bounds 'for layer ← [num layers(fθ), ...0]' and 'for head ← [num heads(fθ), ...0]' appear to include out-of-range indices if read literally; the intended bounds are likely num layers−1 and num heads−1. Also, in the MLP pruning branch, 'ablate mlp(g′θ, layer, head, ...)' uses 'head' from the previous loop, which is not a defined argument in that scope.","section":"Algorithm 1"},{"comment":"The y-axis label 'Model Size' is vague; please specify that it is the number of parameters (excluding embedding and unembedding matrices, as stated in the text) and consider a log-scale x-axis to make the threshold behavior clearer.","section":"Section 4, RQ1, Figure 2"},{"comment":"There is a typo: 'especifically' should be 'specifically'.","section":"Appendix A"},{"comment":"The distillation baseline is described as trained 'for a total of 20000 epochs'; please clarify whether this means 20,000 gradient update steps or 20,000 full passes over the (very small) training set, as the latter would be unusual.","section":"Section 4, RQ4"},{"comment":"The Δacc column shows positive improvements over the original model (e.g., +9.28% for Acronyms, +2.93% for IOI). Since pruning cannot add information, please explain how the pruned model can exceed the original model's accuracy on the same validation set, or specify whether the original model's accuracy on that set was imperfect.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable extension of the authors' prior mechanistic interpretability work on acronym circuits, but the evaluation gap is substantial. The absence of a held-out test set is the most serious issue and should be addressed before publication. The KL-divergence proxy concern is also load-bearing, as the authors themselves document a failure mode in the greater-than task. With those two points addressed, the paper could become a solid contribution to interpretability-oriented model compression; in its current form, the strong empirical claims are not yet supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline is that this is the first paper I know of that actually turns circuit identification into a standalone pruned submodel, not just a hook-based patch. That is a real contribution, and the size and speed numbers, if they hold up, are practically interesting.\n\nWhat is new: prior ACD work stops at identifying the circuit; this paper prunes the components and removes them from the graph. The description of how to physically delete attention heads and MLPs is clear and sensible, and the choice to prune nodes instead of edges is well motivated. Evaluating on three tasks with known circuits is the right testbed, and they ship code and data, which is good.\n\nThe biggest problem is evaluation: thresholds α are chosen from accuracy-vs-size curves computed on Dv, and then the same Dv is used to report the accuracies in Table 1. There is no held-out test set. With only 150–250 samples per batch, those numbers are in-sample estimates. The stress-test note is right; this needs a proper split before I would trust claims like 99.92% accuracy. Second, the KL divergence proxy is known to be unreliable for accuracy in the greater-than task (Appendix D, Figure 7), and the paper acknowledges this but leaves it to future work. That undercuts the \"properly performs the task\" part of the central claim. Third, the interpretability benefit is overstated: TPRs as low as 20% for acronyms mean the pruned model misses most of the manually identified circuit. The FPRs are low, but a submodel that drops 80% of the known heads is not obviously \"the circuit.\" The authors give plausible reasons, but the claim is softer than advertised.\n\nMinor points: the distillation baseline is weak but not a problem; speedups are plausible but measured on one GPU with short sequences; the pseudocode is sloppy but understandable. The citation pattern is fine, and the self-citation is appropriate given the prior work on acronym circuits.\n\nWho is this for: people working on LLM compression or interpretability. It deserves a serious referee, but the evaluation needs a real test split and a more honest treatment of the KL-versus-accuracy issue. I would send it to peer review, with a clear request for those revisions.","headline":"A genuinely new and practical idea—turning circuit identification into real standalone pruning—but the evaluation's missing held-out test and weak circuit recovery keep the central claims only partially supported.","tokens_in":17763,"tokens_out":2421,"would_cite":true,"duration_ms":22043,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pretrained LLM can be pruned without training into a standalone task-specific circuit that keeps accuracy while dropping up to 83% of parameters and running 88% faster.","keywords":["mechanistic interpretability","circuit extraction","structured pruning","task-specific submodels","KL divergence","activation patching","transformer interpretability","inference speedup"],"falsifier":"Run the paper's greater-than experiment with zero ablation and MLP pruning: the reported result is that a model pruned to roughly half size reaches 100% accuracy while its KL divergence from the original remains large. If a similar divergence between KL and task accuracy appears on any held-out task—or if a component whose ablation leaves KL nearly unchanged but destroys accuracy can be exhibited—then the KL gate is not selecting for task competence and the extracted submodel's reliability is not assured.","tokens_in":16827,"feed_emoji":"⚡","tokens_out":7948,"duration_ms":63005,"temperature":0.7,"pith_summary":"This paper tries to establish that a large language model can be carved down to the small set of internal components that actually perform one target task, producing a standalone submodel that runs without fine-tuning. The proposed method walks through the model's attention heads and MLPs from the last layer backwards, temporarily ablating each component and measuring how much the model's prediction distribution shifts; components whose removal barely changes the distribution are permanently pruned. On three GPT-2 Small tasks with known circuits—acronym prediction, indirect object identification, and greater-than comparison—the resulting submodels keep or improve accuracy while dropping up to 82.77% of parameters and cutting inference time by up to 88%. The paper also argues these submodels are more interpretable because they concentrate on task-relevant heads and largely avoid irrelevant ones, and that they beat a distillation baseline when only small datasets are available.","feed_headline":"Extract an LLM's task circuit: 83% fewer parameters, 88% faster","feed_subtitle":"A no-training pruning pass keeps single-task accuracy while shrinking a GPT-2 submodel and speeding its inference.","key_machinery":"Algorithm 1, a greedy backward ablation pass with a KL-divergence gate. Starting from the full model, it traverses layers from top to bottom and heads within each layer; for each candidate component it patches it (zero or mean ablation), computes ΔKL = KL(fθ ∥ g′θ) − KL(fθ ∥ gθ) on a validation set, and permanently prunes the component if ΔKL < α. The residual-stream formulation of transformers makes zero ablation equivalent to removing the component and mean ablation equivalent to replacing it with a bias term, so the pruned model is executable as a standard smaller network; pruning nodes instead of edges keeps the result compatible with parallel matrix-multiplication hardware.","core_discovery":"The central discovery is that the circuit responsible for a single-token task can be not just located but physically extracted: by greedily pruning nodes (individual attention heads and, optionally, MLPs) in reverse layer order and keeping only components whose ablation raises the KL divergence between the pruned model and the original model beyond a threshold α, the authors obtain a truly smaller network that still performs the task. Unlike prior circuit-discovery methods that patch edges via hooks and leave the model unchanged, this method removes the component's weight matrices outright (zero ablation) or replaces them with a constant mean bias (mean ablation), which is what yields actual size and speed gains. In their evaluation, the pruned GPT-2 Small models retain or exceed the original accuracy on acronym prediction, IOI, and greater-than, while removing up to 82.77% of parameters and reducing inference time by up to 88%; the false-positive rate against manually identified circuits is low, and a same-size distilled student trained on the same small datasets fails to reach comparable accuracy.","pith_inferences":["A task-specific accuracy metric (for example, logit difference or exact-match) could replace or augment KL divergence in the greedy gate; the paper's own greater-than results show KL and accuracy can diverge, so a hybrid gate might yield smaller submodels without the observed accuracy spikes.","The same backward ablation could be run separately for several tasks and the surviving component sets intersected or unioned, which would reveal shared versus task-specific circuitry and could enable multi-task submodels.","Because the method needs only forward passes, it should scale to larger models, but the sweep cost is roughly one forward pass per candidate component, so on very large models the ablation sweep rather than the data becomes the bottleneck.","The resulting task circuits could be stress-tested for shortcut learning: if a pruned submodel relies on a spurious template cue, its accuracy should collapse when the template distribution shifts, offering a practical check on whether the extracted circuit is the real mechanism."],"forward_implications":["Single-task deployment of an LLM no longer requires running the full model: the same pretrained weights, pruned by this method, can serve one task with a fraction of the parameters and latency.","Because pruning removes nodes rather than edges, the resulting submodel is executable with ordinary batching and matrix libraries, unlike earlier circuit-identification outputs that only slow the forward pass.","The extracted submodel is small enough to be inspected: low false-positive rates against manual circuits mean an analyst can apply mechanistic-interpretability tools to a handful of components rather than the whole network.","With only a few hundred curated samples, circuit extraction beats distillation at matching the teacher's task behavior, so the method is attractive when task-specific training data is scarce."],"supporting_citations":[{"why":"Supplies the KL-divergence metric for circuit discovery and the observation that hook-based patching slows the forward pass, motivating true pruning.","marker":"Conmy et al. 2023"},{"why":"Defines the residual-stream formulation of transformers that makes zero ablation equivalent to removing a component.","marker":"Elhage et al. 2021"},{"why":"Provides the IOI task, the manually identified circuit used as ground truth, and the mean-ablation scheme.","marker":"Wang et al. 2023"},{"why":"Provides the greater-than task and its manually identified circuit, used as a second ground truth.","marker":"Hanna, Liu, and Variengien 2023"},{"why":"Provides the acronym-prediction task and its manual circuit, the third ground truth.","marker":"García-Carrasco, Maté, and Carlos Trujillo 2024"},{"why":"Introduces activation patching, the intervention technique the pruning criterion is built on.","marker":"Meng et al. 2022"},{"why":"Justifies zero ablation as a gold-standard ablation for interpretability experiments.","marker":"McGrath et al. 2023"},{"why":"Supplies the distillation baseline that the method is compared against for task-specific compression.","marker":"Ba and Caruana 2014"}],"fun_headline_variants":["Extract just the task circuit from an LLM: 83% smaller, 88% faster","LLM pruning: keep only the task circuit, cut 83% params, speed up 88%","No-training circuit extraction: GPT-2 shrinks 83%, runs 88% faster","Task-specific LLM circuits: 83% fewer params, 88% faster inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that how much a component's removal shifts the model's full prediction distribution (measured by KL divergence) tells you how essential that component is to the task, and that this measurement remains trustworthy even after earlier components have already been pruned.","fun_headline_variants_meta":{"raw":{"variants":["Extract just the task circuit from an LLM: 83% smaller, 88% faster","LLM pruning: keep only the task circuit, cut 83% params, speed up 88%","No-training circuit extraction: GPT-2 shrinks 83%, runs 88% faster","Task-specific LLM circuits: 83% fewer params, 88% faster inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000741,"raw_usage":{"total_tokens":3324,"prompt_tokens":981,"completion_tokens":2343,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":2244}},"tokens_in":597,"tokens_out":2343,"duration_ms":14916,"temperature":1.0,"reasoning_tokens":2244,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:07:26.588408+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's greater-than experiment with zero ablation and MLP pruning: the reported result is that a model pruned to roughly half size reaches 100% accuracy while its KL divergence from the original remains large. If a similar divergence between KL and task accuracy appears on any held-out task—or if a component whose ablation leaves KL nearly unchanged but destroys accuracy can be exhibited—then the KL gate is not selecting for task competence and the extracted submodel's reliability is not assured.","supporting_citations":[],"review_version":1}