{"id":"70a55867-f6ab-4126-8203-36a4b7dd942d","arxiv_id":"2412.03915","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Saliency-guided training combined with PACT quantization keeps MNIST and CIFAR-10 accuracy near parity with a quantized baseline, while the claimed efficiency and interpretability gains are not directly measured.","lead":"This paper trains image classifiers with saliency-guided training plus low-precision PACT quantization, reporting accuracy on MNIST and CIFAR-10 close to a quantized baseline. The authors claim the models are more efficient and interpretable, but no efficiency measurement appears and the interpretability evidence is an indirect masking test.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Figure 3's masking-accuracy-drop evidence for interpretability is circular: SGT's D_KL term explicitly trains the model to behave that way, so it is not independent support for the interpretability claim.","rationale":"I agree with the reader's primary identification: the interpretability evidence in Section VIII-C is the least secure load-bearing component of the abstract's central claim. The SGT loss explicitly includes D_KL(f_theta(X) || f_theta(M_k(X))), so the masking-based accuracy-drop behavior is trained into the model and cannot serve as an independent validation of interpretability. The paper's own stated plan, to compare saliency maps of standard and quantized models, is never actually delivered, leaving the claim without the promised visual or quantitative evidence. The efficiency half is also unsupported by any direct measurement, but the interpretability concern is more fundamental because even the single experimental figure offered for it is confounded with the training objective. The accuracy-parity sub-claim (99.35% vs. 99.23% on MNIST; 76.12% vs. 75.55% on CIFAR-10) is at least directly measured and plausible, though it lacks error bars and seeds. The conditional verdict is therefore appropriate: the paper's honest core is the feasibility result, but the full central claim of 'significantly more efficient and interpretable' needs independent efficiency measurements, real saliency-map comparisons, and interpretability metrics that are not identical to the training loss. My read does not change the reader's verdict; it sharpens why the interpretability concern is load-bearing rather than merely cosmetic.","tokens_in":8628,"tokens_out":3853,"duration_ms":41568,"concrete_test":"Run deletion and insertion faithfulness curves on the test set for the SGT+PACT model and the PACT-only baseline, using Captum Input×Gradient saliency maps on at least 1000 images across 5 seeds, and compute the area under each curve. Separately apply the Adebayo et al. model-randomization sanity check to the SGT+PACT saliency maps. If the SGT+PACT model does not significantly outperform PACT-only on deletion/insertion AUC, or if its maps fail the sanity check, the interpretability claim should be removed from the abstract until direct saliency-map evidence is provided.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim requires that the SGT+PACT models are 'significantly more efficient and interpretable.' The efficiency half has no direct measurement anywhere in the paper: no latency, memory, FLOP/s, or energy numbers appear, and Section IX only asserts that 'resource consumption is limited.' The interpretability half rests entirely on Figure 3, which plots accuracy drop under feature masking. But Algorithm 2 and Section V train with the objective L = CE + lambda * D_KL(f_theta(X) || f_theta(M_k(X))), where M_k removes the k lowest-gradient features. The training procedure therefore directly optimizes for the relationship between masking and model output that Figure 3 then reports: a steeper accuracy drop when salient features are masked is a built-in consequence of the loss, not an independent measurement that the saliency maps are more meaningful or more aligned with human-judged relevance. The abstract also promises a comparison of saliency maps, but no saliency-map figure or quantitative map comparison is included. Thus the central claim's interpretability component is currently supported only by a metric that mirrors the training objective, and the efficiency component is supported only by an unquantified assertion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes combining Saliency-Guided Training (SGT) with PACT-based quantization-aware training to obtain deep neural network classifiers that are, according to the abstract, both resource-efficient and interpretable without compromising accuracy. The method trains a ResNet-20 model by masking the k lowest-gradient input features, adding a KL-divergence term that penalizes divergence between predictions on original and masked inputs, and using PACT to quantize activations and weights during training. Experiments are reported on MNIST and CIFAR-10. Table II reports accuracies of 99.35% and 76.12% for the proposed method versus 99.23% and 75.55% for a PACT-only baseline. Figure 3 shows accuracy drops under increasing input masking as evidence of interpretability, and Figures 1-2 show the evolution of the PACT clipping parameter. The conclusion claims that the quantized models maintain near-equivalent accuracy while resource consumption is limited and interpretability is enhanced.","tokens_in":8744,"tokens_out":2443,"duration_ms":25394,"significance":"If fully substantiated, the paper would address a real practical need: quantized models for edge deployment whose explanations remain trustworthy. The combination of SGT with PACT is a reasonable and relatively simple empirical proposal, and the accuracy numbers in Table II are consistent with the modest claim that adding SGT does not hurt accuracy relative to a PACT-only baseline. However, the paper's central contribution is empirical, and at present only the accuracy-parity sub-claim has direct supporting measurements. The efficiency claim is supported only by an unquantified assertion in Section IX, and the interpretability claim rests on a single masking experiment whose outcome is closely tied to the training objective itself. The paper provides no code, no variance/seed information, and no direct saliency-map comparison despite the abstract promising one. These gaps prevent the headline conclusion from being accepted as demonstrated.","major_comments":[{"comment":"The headline claim that the models are 'significantly more efficient' is not supported by any direct measurement. The paper reports no latency, memory footprint, FLOP/s, parameter count, energy consumption, or inference-speed comparison. Section IX only states that 'their resource consumption is limited,' which is not a quantitative result. To support the efficiency half of the central claim, the authors should report concrete efficiency metrics for the quantized models versus the full-precision and PACT baselines, with hardware and measurement details.","section":"Abstract and Section IX"},{"comment":"The interpretability evidence is circular with respect to the training objective. The loss in Algorithm 2 and Section V is L = CE + lambda * D_KL(f_theta(X) || f_theta(M_k(X))), where M_k masks the k lowest-gradient features. The training procedure explicitly optimizes the model so that masking low-saliency features does not change the output, which is exactly the behavior measured in Figure 3: a steeper accuracy drop when salient features are masked. This makes Figure 3 a check that the training objective was optimized, not an independent demonstration that the learned saliency maps are more meaningful or more aligned with human-judged relevance. The interpretability claim needs additional evidence, such as quantitative saliency-map quality metrics, localization benchmarks, or human evaluation, and the paper should directly compare saliency maps as the abstract promises.","section":"Section V, Algorithm 2, and Section VIII-C, Figure 3"},{"comment":"The accuracy comparison is under-specified. Table II reports only two columns, 'PACT' and 'Ours,' with no standard deviation, number of seeds, or test-set details, and the PACT baseline is not described (architecture, training hyperparameters, quantization bit widths, or whether it also uses SGT are not stated). There is also no full-precision baseline in the table, despite the text claiming that quantized models maintain near-equivalent accuracy to standard models. The authors should report mean and variance over multiple runs, describe the baseline precisely, and include the full-precision accuracy for both datasets.","section":"Table II and Section VIII-A"}],"minor_comments":[{"comment":"The text contains an unremoved editorial instruction: 'Here's a refined version of your text with improved flow and clarity:' followed by the rewritten paragraph. This is clearly an artifact of the writing process and must be removed.","section":"Section I"},{"comment":"Both algorithms contain a duplicated loop header 'for i = 1 to epochs do' with no matching indentation for the second occurrence. This makes the pseudocode ambiguous and should be corrected.","section":"Algorithm 1 and Algorithm 2"},{"comment":"The table caption reads 'Training Hyperparameters for Saliency-Guided Training with PACT Quantization and CIFAR-10 values,' which is grammatically incomplete and unclear. The caption should state clearly which hyperparameters apply to which dataset.","section":"Section VII, Table I"},{"comment":"Reference [24] is cited both as 'mixed-precision quantization [24]' in Section I and as the PACT paper by Choi et al. later in the same section, but the reference list entry is only for PACT. Please disambiguate the citations or add the correct reference for mixed-precision quantization.","section":"References"},{"comment":"The caption of Figure 3 says 'Accuracy drop comparison across different models on MNIST dataset,' but the text in Section VIII-C discusses 'models with lower bitwidths (higher quantization)' experiencing sharper accuracy drops. It is unclear which models and bitwidths are actually plotted, and whether the figure shows CIFAR-10 results anywhere. Please clarify the experimental setup and the set of curves in the figure.","section":"Section VIII-C"},{"comment":"The paper says the models are trained on 'ResNet-20' but gives no architecture details such as the number of parameters, the specific ResNet variant, or the input preprocessing. Adding these details would improve reproducibility.","section":"Section VII-A"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short empirical study whose central claims currently outrun its evidence. The accuracy comparison is plausible but lacks statistical grounding, and the two headline benefits—efficiency and interpretability—are respectively unmeasured and circularly argued. These are fixable within the manuscript's scope by adding concrete efficiency measurements, a non-circular interpretability evaluation, and proper experimental reporting. I also note that the manuscript appears hastily assembled, with leftover editorial text and duplicated pseudocode lines; this does not affect my technical assessment but should be corrected in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a small empirical paper that composes two published methods, SGT and PACT, and shows that the combination doesn't hurt accuracy on MNIST and CIFAR-10. That parity result is plausible and is the only part with direct evidence. The abstract's stronger claims—'significantly more efficient and interpretable'—are not supported by anything in the paper.\n\nWhat's new is the specific combination, and nothing else. Both components are off the shelf, and the application is straightforward. Credit where due: Table II gives clean numbers, and if those are real, the SGT penalty isn't costing accuracy under PACT quantization. That's a useful data point for practitioners.\n\nThe soft spots are substantial. Efficiency: there is no latency, memory, FLOP, or energy measurement anywhere. The conclusion simply asserts 'resource consumption is limited.' That's not an empirical claim, it's a hope. Interpretability: the only evidence is Figure 3, which shows a steeper accuracy drop under feature masking for the SGT-trained model. But the SGT loss explicitly trains the model to keep its output stable when low-gradient features are masked, so this curve is a built-in consequence of the training objective, not an independent demonstration that the saliency maps are more meaningful. The abstract promises a comparison of saliency maps, but no such figure appears. The central claim collapses to the accuracy-parity sub-claim, which by itself is modest.\n\nSmaller issues amplify the problem: no error bars or seeds, no code release, duplicated loop lines in both algorithms, and an unedited phrase in the introduction that looks like leftover LLM prose. These aren't fatal to the parity result, but they indicate the manuscript is rough.\n\nWho gets value from this? Someone specifically wondering whether adding an SGT-style KL penalty to a PACT quantized training loop will hurt accuracy. That's a narrow audience. The paper does not reorganize anything, and the efficiency/interpretability claims need real measurements and a non-circular evaluation before they can stand.\n\nMy recommendation: I would not send this to peer review as is. The novelty is thin and the headline claims are unsupported. I'd desk-reject with an invitation to resubmit after the authors either measure efficiency and compare actual saliency maps, or trim the claims down to the accuracy-parity result. If they do that, the short version could be a reasonable workshop paper or a data point in a larger study.","headline":"Near-parity accuracy holds up; the efficiency and interpretability claims don't.","tokens_in":9414,"tokens_out":3385,"would_cite":false,"duration_ms":31216,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Saliency-guided training and PACT quantization together preserve accuracy while making quantized models interpretable.","keywords":["saliency-guided training","quantization-aware training","PACT","model interpretability","neural network quantization","CNN classification","MNIST","CIFAR-10"],"falsifier":"Compare the accuracy-drop curve under random feature masks with the curve under gradient-ranked masks for the SGT+PACT model; if the two curves match, the masking test does not measure learned saliency and the interpretability claim is falsified.","tokens_in":8332,"feed_emoji":"🧠","tokens_out":5141,"duration_ms":44320,"temperature":0.7,"pith_summary":"The paper aims to establish that a neural network can be both quantized and interpretable without sacrificing classification accuracy. It combines saliency-guided training (SGT), which masks low-gradient input features during training, with PACT-based quantization-aware training, and tests this combination on MNIST and CIFAR-10 using a ResNet-20 architecture. The authors report that the combined approach achieves slightly higher accuracy than quantization alone (99.35% vs. 99.23% on MNIST; 76.12% vs. 75.55% on CIFAR-10) and produces saliency maps they argue are more focused. If true, this would make low-precision models for edge devices more trustworthy, since their decisions could be explained with the same quality as full-precision models.","feed_headline":"Saliency training keeps 4-bit models accurate and explainable","feed_subtitle":"Combining masking-based saliency training with PACT quantization preserves accuracy on MNIST and CIFAR-10.","key_machinery":"The central object is the combined loss $\\mathcal{L} = L(f_\\theta(X), y) + \\lambda D_{\\mathrm{KL}}(f_\\theta(X) \\| f_\\theta(\\tilde{X}))$, where $\\tilde{X}$ is the input with the bottom $k$ features removed according to their input-gradient magnitude, used together with PACT's learnable clipping parameter $\\alpha$ in a quantization-aware training loop. SGT provides the interpretability mechanism by forcing predictions to be invariant under removal of low-saliency features, while PACT provides the efficiency mechanism by quantizing activations and weights to 8 bits on MNIST and 4 bits on CIFAR-10. The dynamic optimization of $\\alpha$ is what lets the model tune how aggressively activations are clipped during training.","core_discovery":"The central claim is that saliency-guided training and quantization-aware training are complementary: SGT teaches the network to rely on a compact set of salient features by minimizing the KL divergence between outputs on the original input and a version with low-gradient features masked, while PACT learns a per-layer clipping level $\\alpha$ that keeps activations and weights at low precision. Combined, the method preserves classification accuracy (slightly improving over quantization alone) and yields models whose saliency maps are sharper. As evidence, the paper shows that when a larger fraction of low-gradient features is removed, the SGT-trained quantized model loses accuracy faster than a naively quantized model, which the authors interpret as the model having learned more salient features.","pith_inferences":["The steeper accuracy drop under masking may be partly an artifact of the training objective itself, since SGT explicitly trains the model to be sensitive to the masked features; without a control that trains with random masks, the interpretability claim is confounded.","The efficiency claim is stated from bitwidth reduction alone; actual gains in latency, energy, or memory on a target device remain to be demonstrated, and future work could measure them directly.","The same recipe could be tested on tasks with ground-truth object locations, such as detection or segmentation, where saliency quality can be scored against human annotations instead of gradient-based proxies.","Quantization noise and saliency masking may interact positively as a form of regularization; this suggests a testable hypothesis that the SGT regularizer acts as a stabilizer for low-bitwidth training."],"forward_implications":["If the combined approach works, low-precision models intended for phones and embedded devices can be trained with interpretability built in, so explanations do not have to be computed separately at deployment.","The reported accuracy parity suggests that saliency-guided regularization can offset some of the accuracy loss normally caused by aggressive quantization, implying the two techniques are compatible rather than competing.","The masking-ratio accuracy-drop test provides a cheap, training-time diagnostic for whether a quantized network relies on a focused set of features, which practitioners could adopt as a standard audit.","The dynamic $\\alpha$ schedule indicates that the clipping level can be tuned together with the saliency objective, potentially removing the need to hand-tune quantization ranges."],"supporting_citations":[{"why":"Supplies the saliency-guided training algorithm and the KL-divergence loss that the paper combines with PACT.","marker":"[19]"},{"why":"Supplies PACT, the parameterized clipping activation quantization scheme used to quantize the model.","marker":"[24]"},{"why":"Supplies the quantization-aware training technique that PACT extends and that the paper builds on.","marker":"[15]"},{"why":"Supplies the Captum library used to generate the saliency maps for the interpretability comparison.","marker":"[29]"},{"why":"Provides the MNIST dataset used for the main classification experiments.","marker":"[26]"},{"why":"Provides the CIFAR-10 dataset used for the main classification experiments.","marker":"[27]"}],"fun_headline_variants":["Saliency training preserves accuracy in quantized CNNs","Efficient interpretable CNNs via saliency and PACT quantization","Quantization plus saliency: accuracy and interpretability together","Saliency-guided PACT yields clear, compact neural nets","Combining saliency and quantization for sharper saliency maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The interpretability claim rests on the premise that a steeper accuracy drop when input features are masked is a valid measure of interpretability; the efficiency claim rests on the premise that lower bitwidths automatically mean lower resource use, since no latency or memory measurements are provided.","fun_headline_variants_meta":{"raw":{"variants":["Saliency training preserves accuracy in quantized CNNs","Efficient interpretable CNNs via saliency and PACT quantization","Quantization plus saliency: accuracy and interpretability together","Saliency-guided PACT yields clear, compact neural nets","Combining saliency and quantization for sharper saliency maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000268,"raw_usage":{"total_tokens":1611,"prompt_tokens":931,"completion_tokens":680,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":608}},"tokens_in":547,"tokens_out":680,"duration_ms":5808,"temperature":1.0,"reasoning_tokens":608,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:56:58.583197+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the accuracy-drop curve under random feature masks with the curve under gradient-ranked masks for the SGT+PACT model; if the two curves match, the masking test does not measure learned saliency and the interpretability claim is falsified.","supporting_citations":[{"cited_title":"& Feizi, S","cited_arxiv_id":null,"evidence_quote":"Supplies the saliency-guided training algorithm and the KL-divergence loss that the paper combines with PACT."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the quantization-aware training technique that PACT extends and that the paper builds on."},{"cited_title":"& Burges, C","cited_arxiv_id":null,"evidence_quote":"Provides the MNIST dataset used for the main classification experiments."},{"cited_title":"& Hinton, G","cited_arxiv_id":null,"evidence_quote":"Provides the CIFAR-10 dataset used for the main classification experiments."}],"review_version":1}