{"id":"eacfc3ab-3f15-4c64-a77d-38d3b27fafc8","arxiv_id":"2608.10203","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A kernel-matrix SVD followed by average pooling gives a controllable, high-compression representation of convolutional activations that preserves out-of-distribution and adversarial-attack detection performance.","lead":"This paper introduces a new way to shrink the internal activations of convolutional neural networks before feeding them to out-of-distribution and adversarial-attack detectors. The method compresses more aggressively than existing approaches while keeping detection accuracy comparable or better.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Concern: the SVD 'minimal information loss' guarantee in §4.2.2 covers reconstruction of B_hat, not the class/OoD discriminative structure DMD/MACS need; the experimental support is weakened by missing numeric AUCs/error bars and by test-informed layer/tuning selection.","rationale":"The reader's weakest assumption is the same area I would stress: SVD truncation plus average pooling is not guaranteed to preserve detector-relevant information. I sharpen it with the specific ResNet bound mismatch, where the maximum possible kernel dimension is far below the avgpooling dimension, and I tie it to evaluation-selection risks that the reader also noted (no numeric AUC tables, outcome-based MobileNet layer selection, unspecified tuning split). The method is plausible, and the code link plus multi-architecture/multi-detector experiments are real evidence, so this is not a rejection. The concern is that the central 'consistently better or comparable' claim rests on an empirical regularity that the paper does not yet make quantitatively verifiable, and the theoretical 'minimal information loss' wording overstates what the SVD guarantees. The max-κ parity test isolates the information-retention question from downstream GMM/dimensionality effects by comparing kernel at its least compressed setting against avgpooling. If the test shows parity, the concern is resolved and only reporting gaps remain; if it shows a drop, the paper should be revised to claim task-specific empirical benefit rather than minimal information loss. I therefore keep the reader's CONDITIONAL verdict unchanged.","tokens_in":19859,"tokens_out":15536,"duration_ms":170858,"concrete_test":"Run the ResNet/CIFAR-100 and ResNet/ImageNet1k DMD/MACS evaluations with kernel's κ fixed to its upper bound min(c_i h_k w_k + 1, c_o) on all selected layers, and report per-dataset AUC against avgpooling. For layers where this bound equals c_o, kernel should reproduce avgpooling up to an invertible linear map, which also checks the implementation; for ResNet layers the bound is well below c_o (e.g., 513 vs 2048 on l4.2.c.3). Parity at maximum κ would show the weight-SVD projection retains the detector-relevant subspace; a significant AUC drop would show the extra compression is not information-loss-free and that the reported advantages of kernel come from tuning/search choices rather than from information retention.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is Eq. (16): kernel trims the SVD of the reshaped weight matrix B_hat (§4.2.2), projects the unfolded activations with S_hat, and average-pools. The paper justifies this by Eckart-Young optimality and by phrases such as 'guaranteeing minimal information loss' (§4.1, §3.4.1). That guarantee concerns the Frobenius-norm reconstruction of B_hat, not the preservation of the class-conditional structure on which DMD's Mahalanobis feature association and MACS's GMM clustering operate. Because the projection basis is computed from layer weights alone, the directions most useful for separating in-distribution from OoD/AA samples may lie in the discarded singular subspace. This is not hypothetical: on ResNet bottleneck layers (Table 1), d+1 is much smaller than c_o (e.g., l4.2.c.3 has d+1=513 and c_o=2048), so even the maximum κ discards most of the avgpooling information. The experiments are the only support that this loss is harmless, but they report no per-dataset AUC tables or error bars (Figure 3 is plot-only), select MobileNet layers by empirically maximizing detection AUC (§5.1.2), and tune hyperparameters by maximizing Λ_all without stating whether this is done on validation or test (§5.2). The claim is therefore not yet supported at the strength stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new dimensionality reduction method, called \"kernel\", for convolutional layer activations in out-of-distribution (OoD) and adversarial attack (AA) detection. The method reshapes the convolution weights into a dense matrix (the W1 isomorphism), applies a trimmed SVD, uses the right singular vectors to project the unfolded activations, and then average-pools the result to obtain a corevector of dimension kappa <= min(c_i h_k w_k + 1, c_o). The authors extend DMD and MACS to accept arbitrary dimensionality reductions, and evaluate kernel against average pooling and the Toeplitz-SVD reduction on CIFAR-100 and ImageNet1k with four CNN architectures. They report that kernel achieves comparable or better detection AUC while using smaller corevectors and lower memory/compute footprints.","tokens_in":20184,"tokens_out":6312,"duration_ms":59554,"significance":"If the empirical claims hold, the contribution is practical and well-motivated: it provides a controllable compression method that preserves the information needed by activation-based detectors, and the extension of DMD and MACS to generic DR is a useful contribution in itself. The use of a dense weight-matrix isomorphism to avoid the quadratic growth of the Toeplitz operator is elegant, and the reported memory savings are substantial. The paper includes a source-code reference, which supports reproducibility. However, the current evidence does not yet fully support the headline claim of consistent superiority: the paper lacks numeric AUC tables, confidence intervals, and multiple-seed statistics, and the model-selection and tuning procedures are not documented as being restricted to validation data.","major_comments":[{"comment":"The central claim that kernel \"consistently perform[s] better than, or comparable to, the strongest alternative\" is not quantitatively supported by the manuscript. Figure 3 shows only scatter plots of Lambda_OoD and Lambda_AA, and the text does not provide per-dataset AUC values, confidence intervals, or multiple-seed statistics. Please add a table with the exact Lambda_OoD, Lambda_AA, and Lambda_all values for the best configuration of each method and model, together with standard deviations across at least three random seeds, and report statistical significance (e.g., paired tests) for the key kernel-vs-baseline comparisons.","section":"§5.2 and Figure 3"},{"comment":"The model-layer selection for MobileNet (\"empirically select 4 macro-blocks maximizing the detection methods' AUC\", §5.1.2) and the hyperparameter tuning (\"maximize Lambda_all\", §5.2) do not state whether these procedures are performed on validation or test data. If the test set is used for either layer selection or tuning, the reported results are optimistic and not valid as held-out evaluations. Please specify the exact data split used for all selection and tuning steps, and confirm that the final AUC numbers are computed on test samples that were never used for any model or hyperparameter choice.","section":"§5.1.2 and §5.2"},{"comment":"The phrase \"guaranteeing minimal information loss\" overstates the Eckart-Young property. The trimmed SVD in Eq. (16) minimizes the Frobenius-norm reconstruction error of B_hat, but the pipeline then discards P_hat and R_hat and applies average pooling to the projected activations. The guarantee does not cover preservation of the class-conditional or OoD-discriminative structure on which DMD's Mahalanobis distance and MACS's GMM clustering operate. Please revise the wording to state exactly what the SVD truncation guarantees, and discuss why the discarded singular subspaces are expected not to harm detection performance.","section":"§4.1 and §4.2.2, Eq. (16)"},{"comment":"The indexing formulas in Algorithm 1 are incorrect for non-square kernels. For a flat index channelo in {0,...,c_i h_k w_k - 1}, the correct decomposition is offset_w = channelo % w_k, offset_h = (channelo // w_k) % h_k, and channel_i = channelo // (w_k * h_k). The published line 3 uses (channelo // h_k) % h_k, which coincides with the correct mapping only when h_k = w_k. Since all layers in Table 1 have square kernels, this does not affect the reported experiments, but the method is claimed for general convolutional layers and the pseudocode must be fixed or qualified.","section":"Algorithm 1"}],"minor_comments":[{"comment":"DMD is evaluated only in its \"aware\" mode, which is stated in the text; the conclusions about DMD should nevertheless be explicitly framed as specific to this setting, since the aware mode uses OoD/AA samples from the same distribution as the test set for regressor training.","section":"§5.1.1"},{"comment":"The scatter plots in Figure 3 are dense and hard to read; consider marking the best configuration of each method with distinct colors or labels, and add a separate numeric table for the best configurations to complement the plots.","section":"Figure 3"},{"comment":"There is a typo in \"hence eliminating the quadratic scaling with the layer's input dimension present intoeplitz\": \"intoeplitz\" should be \"in toeplitz\".","section":"§4.1"},{"comment":"Please clarify why the search evaluates 50 combinations out of 10^L for avgpooling and 100^L for toeplitz and kernel; the relationship between the number of hyperparameters and the number of evaluated configurations is not immediately clear.","section":"§5.2"},{"comment":"The sentence \"The toeplitz's shortcoming lies in high-dimensional, highly sparse matrices\" is incomplete; specify that the sparse Toeplitz matrices lead to large memory usage and require large kappa values to retain information.","section":"§3.4.1"},{"comment":"The reference to \"han Liu and Arik (2020)\" appears with the lowercase initials; please use the correct author name in the reference list and citation.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the core idea is solid, but the evidence in the current version is not sufficient for acceptance. The missing numeric results and the unclear validation/tuning protocol are the main blockers. The SVD 'minimal information loss' claim and the Algorithm 1 indexing bug are also fixable but must be addressed. I would be willing to review a revised version with detailed AUC tables, error bars, and a clear statement of the tuning/selection protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know before you spend time on this. First, the core idea is simple and credible: rewrite the conv layer as a dense matrix via the Praggastis et al. isomorphism, take a trimmed SVD of that weight matrix, and average-pool the projected activations. This gives a controllable compression level that can be far smaller than channel-wise average pooling while staying dense and cheap. Second, the paper never reports actual AUC numbers. All you get are scatter plots and a claim that kernel 'matches or outperforms' the alternatives. That is the single biggest weakness.\n\nWhat is genuinely new: the kernel DR itself is a new combination, and the authors extend DMD and MACS to accept any DR, which is a fair contribution. The evaluation covers four architectures and two in-distribution datasets with standard OoD and attack benchmarks, and they include memory and runtime measurements. The math is straightforward and correct as far as it goes. The related work is well integrated; I did not spot citation problems.\n\nWhere the paper is soft: first, the absence of numeric AUC tables and error bars. The scatter plots in Figure 3 show many configurations, but no per-dataset numbers, no standard deviations, no multiple seeds. For a paper whose claim is 'comparable or better,' this is insufficient. Second, the MobileNet layer selection is explicitly described as empirically maximizing detection AUC. That is a selection on the outcome, and the text does not say whether it was done on validation or test. That can inflate results. Third, the phrase 'guaranteeing minimal information loss' is overbroad. The Eckart-Young guarantee applies to reconstructing the weight matrix B_hat, not to the downstream detector-relevant structure after you discard P_hat and R_hat and average-pool. The paper's own Table 1 shows that on ResNet bottleneck layers, even the maximum kappa discards most of the information available to avgpooling. The empirical results may still be right, but the theoretical claim as stated is not supported.\n\nThese are all soluble with a revision: add tables with AUC numbers, state explicitly which splits were used for layer selection and hyperparameter tuning, and tone down the information-loss claim to what the SVD actually guarantees.\n\nWho is this for: anyone doing activation-based OoD or adversarial detection on CNNs who cares about memory and compute. It is not a method that changes detection principles; it is a practical improvement. A serious referee should see it, but they should ask for the missing numbers. I would not desk-reject.","headline":"A credible kernel-based DR for activation analysis that needs numeric AUC tables and clearer split discipline before its headline claim is supported.","tokens_in":20751,"tokens_out":4453,"would_cite":true,"duration_ms":38394,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new dimensionality reduction compresses convolutional-layer activations to a user-chosen size while preserving or improving out-of-distribution and adversarial-attack detection.","keywords":["out-of-distribution detection","adversarial attack detection","dimensionality reduction","convolutional neural networks","intermediate activations","singular value decomposition","average pooling","corevectors"],"falsifier":"Run the kernel reduction with $\\kappa$ equal to the number of output channels and compare detection AUC with plain average pooling: if the rotated-then-pooled corevectors do not match or beat average pooling at that limit, the claimed information preservation fails. A sharper test replaces $\\hat{\\mathbf{S}}$ with a random projection of the same dimension; equal detection performance would show that the SVD ordering, not the compression alone, is doing the work.","tokens_in":19662,"feed_emoji":"🛡️","tokens_out":7615,"duration_ms":63948,"temperature":0.7,"pith_summary":"This paper tries to establish that convolutional-layer activations can be compressed far more aggressively than current practice without hurting, and sometimes helping, out-of-distribution (OoD) and adversarial-attack (AA) detection. It proposes a reduction it calls kernel: unroll the layer's kernels into a dense matrix, apply a trimmed singular value decomposition, project the unfolded input through the resulting matrix, and average-pool the projected activations. The resulting corevector has a user-chosen size $\\kappa \\le \\min(c_o, c_i h_k w_k + 1)$, so the compression level is controllable and its least-compressed setting matches channel-wise average pooling. On CIFAR-100 and ImageNet1k, across four CNN architectures and two detectors (DMD and MACS), the paper reports that kernel matches or beats the strongest alternative reduction while using less memory and computation. A sympathetic reader would care because activation-based detection is then cheaper, more scalable, and tunable to the model at hand.","feed_headline":"SVD-then-pool shrinks conv activations, keeps OoD and attack detection","feed_subtitle":"New reduction cuts memory and compute for activation-based detectors while matching or beating stronger alternatives.","key_machinery":"The carrying object is the dense convolution-isomorphism matrix $\\hat{\\mathbf{B}}$, with one row per output channel and columns that are the flattened input kernels plus a bias column, so its size depends on the kernel parameters rather than growing quadratically with the input spatial size as the Toeplitz unrolling does. A trimmed SVD gives $\\hat{\\mathbf{B}} \\approx \\hat{\\mathbf{P}} \\hat{\\mathbf{R}} \\hat{\\mathbf{S}}$; the rows of $\\hat{\\mathbf{S}}$ act as a new set of ordered kernels, and applying them to the unfolded input followed by average pooling yields the corevector of dimension $\\kappa \\le \\min(c_o, c_i h_k w_k + 1)$. The SVD provides the ordering that makes truncation loss-minimizing, and average pooling provides the spatial collapse, so the representation stays as compact as channel-wise pooling while remaining adjustable.","core_discovery":"The central claim is that the information a detector needs from a convolutional layer survives a two-stage squeeze: rotate the layer's kernels by the SVD of the dense unrolled kernel matrix $\\hat{\\mathbf{B}} = [\\hat{\\mathbf{w}} \\ \\mathbf{b}]$, keep only the top $\\kappa$ singular directions, and collapse each projected spatial map to its channel mean. The resulting corevector $\\boldsymbol{\\nu}_k \\in \\mathbb{R}^{\\kappa}$ preserves the per-channel low-level-feature content that DMD and MACS rely on, and the SVD ordering makes the loss controllable: reducing $\\kappa$ discards the least important directions first. The authors evaluate on CIFAR-100 and ImageNet1k with VGG, MobileNet, ResNet, and ConvNeXt, using OoD datasets and the BIM, PGD, FAB, Square, APGD, and APGD-with-TRADES attacks. They find that kernel consistently matches or outperforms the strongest alternative among average pooling and Toeplitz-SVD, while producing the smallest corevectors and the smallest SVD memory footprint.","pith_inferences":["A layer-wise rule that sets $\\kappa$ from the SVD energy spectrum, such as retaining 95% of the singular-value mass, would probably recover the tuned $\\kappa$ values without hyperparameter search, because the paper already shows the ordering carries the information.","The same unroll-project-pool recipe should transfer to other activation-based detectors and to non-convolutional layers expressible as affine maps, although the paper only tests convolutional layers.","When $\\kappa$ equals the number of output channels, the kernel reduction differs from average pooling only by an orthogonal rotation of channels; comparing the two at that limit would isolate whether the SVD rotation itself, rather than the tunable dimension, drives the gains.","Because the method compresses activations to their channel count with minimal loss, it could serve as a general feature-map compression layer for memory-limited inference beyond OoD and AA detection."],"forward_implications":["With the kernel reduction, both DMD and MACS match or beat the strongest alternative reduction on all tested OoD and AA benchmarks across all four architectures.","The smaller corevectors cut memory: for example, the stored SVD for ResNet drops from gigabytes with Toeplitz-SVD to 6.7 MB with the proposed method.","MACS, whose clustering degrades on high-dimensional corevectors, can use SVD-style control without suffering the large $\\kappa$ values that Toeplitz-SVD requires.","DMD's back-propagation-based peephole computation becomes faster because the corevectors it differentiates through are smaller.","Both detectors become DR-agnostic, so practitioners can choose a reduction by footprint and robustness instead of being tied to a fixed pooling scheme."],"supporting_citations":[{"why":"Defines the DMD detector whose Mahalanobis-distance scoring and average-pooling reduction the paper extends and compares against.","marker":"Lee et al. (2018)"},{"why":"Defines the MACS detector with its SVD-based Toeplitz reduction and unsupervised clustering that the paper extends and evaluates.","marker":"Capelli et al. (2025)"},{"why":"Supplies the dense convolution isomorphism (the W1 unrolling) on which the proposed kernel reduction is built.","marker":"Praggastis et al. (2022)"},{"why":"Provides the standardized Far-OoD benchmark splits used for CIFAR-100 and ImageNet1k out-of-distribution evaluation.","marker":"Zhang et al. (2024)"},{"why":"Provides the standardized adversarial-attack benchmark whose attacks generate the AA test samples.","marker":"Croce et al. (2021)"},{"why":"Supplies the empirical feature-association step that MACS uses to relate low-level features to class-level high-level features.","marker":"han Liu and Arik (2020)"}],"fun_headline_variants":["SVD-then-pool shrinks conv activations, keeps detection","New conv squeeze: best compression, same detection","Controllable loss in conv reduction for OoD and attack","Slim conv cores: smaller memory, equal detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the information DMD and MACS need for detection survives SVD truncation followed by average pooling; SVD guarantees faithful reconstruction of the kernel matrix, not preservation of detector-relevant signal.","fun_headline_variants_meta":{"raw":{"variants":["SVD-then-pool shrinks conv activations, keeps detection","New conv squeeze: best compression, same detection","Controllable loss in conv reduction for OoD and attack","Slim conv cores: smaller memory, equal detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1559,"prompt_tokens":993,"completion_tokens":566,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":609,"completion_tokens_details":{"reasoning_tokens":497}},"tokens_in":609,"tokens_out":566,"duration_ms":5886,"temperature":1.0,"reasoning_tokens":497,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:11:08.094634+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the kernel reduction with $\\kappa$ equal to the number of output channels and compare detection AUC with plain average pooling: if the rotated-then-pooled corevectors do not match or beat average pooling at that limit, the claimed information preservation fails. A sharper test replaces $\\hat{\\mathbf{S}}$ with a random projection of the same dimension; equal detection performance would show that the SVD ordering, not the compression alone, is doing the work.","supporting_citations":[],"review_version":1}