{"id":"84c4c6a0-27aa-47db-a19c-3854d23586cd","arxiv_id":"2607.22212","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DLM-SVDD alternately fits a large-margin ℓp-SVDD kernel boundary with Frank–Wolfe and updates CNN features with a softplus margin-violation loss, improving visual anomaly detection under imbalance.","lead":"This paper combines a convolutional neural network with a kernel-based decision boundary so that image features and the anomaly-detection boundary are learned together. It reports AUROC gains over fixed-feature baselines on several image benchmarks and studies faster kernel approximations for scaling.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparison in Table I is not controlled: 'Single-kernel [9]' vs 'DLM-SVDD (fixed CNN)' differ by ~0.09 AUROC on CIFAR-10 at 10%, so the claimed improvement over [9] may reflect pretrained-feature quality rather than joint boundary learning.","rationale":"The reader focused on the unproven asymptotic-regularity condition in Section III-F. That is a legitimate theoretical gap, but the central claim is empirical, and the more load-bearing weakness is the uncontrolled baseline. The convergence theorem is supporting analysis; even if asymptotic regularity is assumed, the headline result could still be true. Conversely, if the [9] baseline numbers come from a different feature extractor, the headline comparison itself is invalid. The paper deserves credit for the controlled Table V ablation, which supports the weaker claim that joint training improves over its own frozen backbone, and for the detailed kernel-approximation analysis. However, because Table I's key comparison is not tied to a same-feature rerun, the strong claim about [9] needs either a controlled reproduction or a clear statement of what features the [9] entries use. This does not change the reader's CONDITIONAL verdict, so I recommend UNCHANGED.","tokens_in":22253,"tokens_out":6605,"duration_ms":62120,"concrete_test":"Rerun the fixed-feature large-margin ℓp-SVDD baseline of [9] inside the DLM-SVDD codebase using the exact same ImageNet-pretrained ResNet-50 features, ℓ2 normalization, median-heuristic RBF bandwidth, 80/20 stratified splits, and 10 seeds as in Table V, with the same AUROC evaluation. Then compare that number to the Table I 'Single kernel [9]' entries and to Table V's 'fixed CNN' row. If the controlled rerun matches DLM-SVDD (fixed CNN) (~0.87 on CIFAR-10 at 10%) rather than [9]'s reported 0.958, the Table I comparison is confounded and the headline should be revised to use the controlled joint-vs-fixed delta. If it matches 0.958, the authors must identify the feature/protocol difference that makes their fixed-CNN variant 0.09 worse.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that DLM-SVDD 'consistently improves over the fixed-feature large-margin ℓp-SVDD baseline of [9]' and that the adaptive representation drives the gain. The evidence for this is Table I, but the fixed-feature baseline is not matched to the proposed system. In Table V, the authors' own 'DLM-SVDD (fixed CNN)' row—freezing the same ImageNet-pretrained ResNet-50 and training only the SVDD boundary—gets 0.867 AUROC on CIFAR-10 at ratio 10%. Yet the 'Single kernel [9] (ratio=10%)' row in Table I reports 0.958 on the same dataset and ratio. If both rows are fixed-feature large-margin ℓp-SVDD under the same protocol, they should be nearly identical; a 0.09 AUROC gap is too large to dismiss. Similarly, in Table II, 'Single-kernel ℓp-SVDD [9]' reaches 89.20 balanced accuracy on CIFAR-10-LT (ϱ=100), while 'DLM-SVDD (fixed CNN)' reaches only 59.50. The paper does not explain this gap. The most plausible reading is that the [9] entries were produced with a different feature extractor or evaluation protocol. If so, the headline improvement over [9] is confounded: the gain could come from the stronger pretrained features used by DLM-SVDD, not from the alternating feature-boundary learning. Table V still shows a positive joint-vs-frozen delta within DLM-SVDD, so the narrower claim 'joint learning helps' survives, but the specific claim about beating [9] is not yet supported.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DLM-SVDD, a deep anomaly-detection framework that alternates between solving a large-margin ℓ_p-SVDD dual boundary (via Frank–Wolfe on current CNN features) and updating the CNN backbone with a smooth margin-violation loss. It reports experiments on standard one-vs-rest benchmarks, long-tailed CIFAR/ImageNet variants, and a systematic study of seven kernel approximation backends. The central claim is that joint representation–boundary learning consistently improves over the fixed-feature ℓ_p-SVDD baseline [9] and achieves the best average rank among compared methods.","tokens_in":22717,"tokens_out":6116,"duration_ms":60337,"significance":"If the central claim holds, the paper makes a useful contribution: it connects an explicit large-margin kernel boundary to deep representation learning in a single alternating scheme, and it provides a practical scalability analysis of kernel approximations within that scheme. Strengths include the public code link, the controlled kernel-approximation study (Fig. 3, Table IV), the use of the Skillings–Mack rank test, and the internal fixed-CNN ablation (Table V). However, the main empirical claim is currently undercut by an uncontrolled comparison against the [9] baseline and by missing variance reporting; the optimization claim is also weakened by an unproven regularity assumption.","major_comments":[{"comment":"The headline claim 'consistently improves over the fixed-feature ℓ_p-SVDD baseline of [9]' is not a controlled comparison. Table I reports [9] at 0.958 AUROC on CIFAR-10 (ratio 10%), while the paper's own fixed-CNN DLM-SVDD variant, using the same ImageNet-pretrained ResNet-50 and training only the boundary, reaches 0.867 (Table V). Table II shows an even larger gap: 89.20 vs 59.50 balanced accuracy on CIFAR-10-LT (ϱ=100). Both rows are described as fixed-feature large-margin ℓ_p-SVDD. Unless [9] used a different feature extractor or protocol, these numbers are inconsistent. Rerun [9] with the same pretrained features, or restrict the claim to the within-method fixed-vs-joint improvement.","section":"Tables I, II, and V"},{"comment":"Eq. (7) writes L_ω as a sum of ψ^p terms, but Section IV-B states that p=1 is used in the softplus for all experiments, and Eq. (8) gives the gradient as σ(v), not pψ^{p-1}σ(v). Thus the CNN step actually minimizes a p=1 softplus margin loss while the boundary solver uses p=2. This is a mismatch between the stated ℓ_p objective and the implemented loss. Please correct Eqs. (7)–(8), state the p=1 choice explicitly, and justify it; the current text describes the method as optimizing the ℓ_p primal for general p>1 in the CNN update.","section":"Section III-C2 and Section IV-B"},{"comment":"The convergence statement (17) depends on 'assume asymptotic regularity, ||ω^{(t+1)}−ω^{(t)}||→0', which is neither proved nor implied by A1–A2 and the descent inequality (16). Adam with constant learning rate does not guarantee this, and the boundary re-estimation changes the objective each epoch. Please either prove this condition under explicit step-size/decay assumptions or present (17) clearly as a conditional statement. Figure 2 shows loss curves for two classes and does not verify the regularity condition.","section":"Section III-F"},{"comment":"Results are reported as ten-seed means without standard deviations or confidence intervals. For example, the CIFAR-10 ratio-75 gap between DLM-SVDD (0.984) and Single kernel [9] (0.972) is 0.012 AUROC; without variance, the 'consistent improvement' and rank claims are hard to evaluate. Report per-cell standard deviations/confidence intervals and, where possible, pairwise significance tests. The Skillings–Mack omnibus p-value only establishes that some rank difference exists.","section":"Tables I and II"},{"comment":"The common configuration p=2, ν=1.2 is selected on the validation split and then used for all reported test results. This can bias the comparison if baselines were not tuned to the same extent. Please specify the validation-based selection rule, report sensitivity of final AUROC to p and ν at least at one ratio, and state the tuning budget for each compared method.","section":"Section IV-B"}],"minor_comments":[{"comment":"Please specify which CIFAR classes and seeds are shown and add variability across runs; otherwise 'representative behavior' is unverifiable.","section":"Figure 2"},{"comment":"The caption says 'single seed per class', so the ±std is across the 10 classes, not across seeds. State this explicitly to avoid misinterpretation.","section":"Table IV caption"},{"comment":"ImageNet-LT results appear not to be averaged over ten seeds. Clarify the seed protocol and report variance for those rows.","section":"Section IV-D / Table II"},{"comment":"For RFF, the standard construction includes a normalization factor and often a paired sine term to obtain unbiased kernel estimates. Clarify whether the displayed map is the one used in all spectral backends and whether any normalization/scaling is applied.","section":"Eq. (14)"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern lands: the discrepancy between Table I's 'Single kernel [9]' and Table V's 'fixed CNN' is present inside the manuscript and is the main obstacle to accepting the headline improvement-over-[9] claim. The p-mismatch and the unproven asymptotic-regularity assumption are also real but likely fixable with rewriting and additional experiments. I recommend major revision with a request for matched baselines, variance reporting, and a corrected/qualified optimization statement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. The core idea is a sensible extension of the group's own fixed-feature ℓp-SVDD: alternate between fitting the large-margin boundary with Frank–Wolfe on the current deep features and updating a ResNet with a softplus margin-violation loss. The math in the derivation is mostly sound, and the paper is honest enough to include a frozen-CNN ablation. The problem is that the headline empirical claim — consistent improvement over the [9] baseline — is not supported by a controlled comparison. In Table I, 'Single kernel [9]' hits 0.958 AUROC on CIFAR-10 at 10% ratio, while the paper's own 'DLM-SVDD (fixed CNN)' in Table V gets 0.867 on the same dataset and ratio. Both are fixed-feature large-margin ℓp-SVDD; they should be nearly identical under the same protocol. Table II shows an even bigger gap on CIFAR-10-LT (89.20 vs 59.50). The most plausible explanation is that the [9] entries were produced with a different feature extractor or evaluation protocol. If so, the claimed gain over [9] is largely a feature-quality effect, not the joint training.\n\nWhat does survive? The within-pipeline comparison in Table V is controlled: joint training beats freezing the same ImageNet-pretrained ResNet-50 by +0.11 AUROC on CIFAR-10 at 10% ratio and similar elsewhere. That is meaningful evidence that the alternating scheme helps, even if the magnitude relative to [9] is overstated. The kernel-approximation study is also a solid piece of work: seven backends, complexity classes, Pareto frontier, and a sensible recommendation. That part reads like a real contribution.\n\nThe other soft spots are manageable. Tables I and II report ten-seed averages without error bars. The hyperparameters p=2 and ν=1.2 were selected on validation and then fixed. The CNN loss uses p=1 softplus while the boundary solver uses p=2; the paper acknowledges this, but it's an extra knob you'd want to see justified more thoroughly. The convergence proof in Section III-F depends on an 'asymptotic regularity' assumption (||ω(t+1)−ω(t)||→0) that is asserted rather than proven; the descent inequality alone doesn't give it. The authors show loss curves, but that's illustration, not proof.\n\nNet: this is a plausible method that deserves a serious referee, but the authors need to reconcile the baseline discrepancy, add error bars, and either match the [9] feature extractor or stop claiming a controlled improvement over [9]. The narrower claim — joint learning helps over a frozen backbone — is what the experiments actually support.\n\nI'd send it to review, with instructions to the authors to fix the protocol issue. I'd cite it once the baseline question is answered. Not sure I'd bring it to reading group — maybe when revising.","headline":"Joint deep feature learning with explicit ℓp-SVDD boundary is plausible, but the claimed consistent gains over the [9] baseline are not controlled; the frozen-CNN ablation is the real evidence.","tokens_in":23211,"tokens_out":5905,"would_cite":false,"duration_ms":47316,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that jointly learning a convolutional representation and an explicit large-margin ℓ_p-SVDD boundary, through alternating Frank-Wolfe boundary updates and a smooth margin-violation CNN loss, consistently outperforms fixed-fe","keywords":["anomaly detection","large-margin SVDD","ℓ_p norm","deep CNN","alternating optimization","Frank-Wolfe","kernel approximation","long-tailed recognition"],"falsifier":"Run the alternating scheme with the paper's fixed Adam learning rate and record squared parameter differences across epochs; if ||ω^(t+1) − ω^(t)|| does not tend to zero even as the loss decreases, then a premise of the stationarity proof is violated. A simpler check: if the CNN gradient norm plateaus above zero while the Frank-Wolfe gap goes to zero, the network block is not reaching the claimed stationarity.","tokens_in":22130,"feed_emoji":"🎯","tokens_out":4110,"duration_ms":39945,"temperature":0.7,"pith_summary":"The paper proposes DLM-SVDD, a method that trains a convolutional network and a large-margin ℓ_p-SVDD decision boundary together rather than separately. It alternates between solving the convex dual boundary problem with Frank-Wolfe and updating the network with a softplus margin-violation loss, so the representation adapts to the boundary and the boundary is re-fit to the new features. The authors claim this joint scheme consistently improves over the fixed-feature large-margin ℓ_p-SVDD baseline on CIFAR-10 and CIFAR-100 at every tested anomaly ratio, and that it performs strongly on long-tailed imbalance benchmarks. The practical point is that representation adaptation and explicit margin geometry can be combined in one stable training loop, and kernel approximations allow the method to scale.","feed_headline":"Joint feature and boundary training beats fixed-feature SVDD","feed_subtitle":"An alternating Frank-Wolfe plus CNN update improves anomaly detection on CIFAR and Fashion-MNIST, even with severe class imbalance.","key_machinery":"The alternating two-block optimization is the core mechanism: an α-step that solves the convex ℓ_p-SVDD dual via Frank-Wolfe with a closed-form linear subproblem, and a ω-step that backpropagates a smooth margin-violation loss through the CNN. The margin-violation gradient focuses updates on samples that cross the inner or outer boundary, and low-rank kernel backends (Nyström, random Fourier features, RPCholesky) make the kernel operations scalable.","core_discovery":"DLM-SVDD retains the exact large-margin ℓ_p-SVDD dual objective for the boundary while making the features trainable. At each epoch, features from the CNN define a kernel matrix, and a Frank-Wolfe solver recovers the optimal dual variables; then, with that boundary frozen, the CNN is updated to push normal samples inside the inner boundary and anomalous samples outside the outer boundary, using a softplus-regularized version of the slack penalties. The paper reports that this joint training lifts AUROC by up to about 0.117 over a frozen backbone and beats the fixed-feature baseline at all tested ratios, with the biggest gains under severe class imbalance.","pith_inferences":["The paper's own ImageNet-LT results hint that joint training can hurt when positive samples are extremely scarce; a hybrid that freezes the backbone for very small classes might combine the best of both.","Fixing the RBF bandwidth once from pretrained features avoids a feedback loop; a testable extension would be to adapt the bandwidth slowly or with a separate objective to get further gains.","Because the CNN loss only has gradient support near the boundary (via the softplus sigmoid), the method may be naturally insensitive to noisy labels far from the boundary—an untested but plausible side effect.","The convergence proof assumes successive CNN updates shrink to zero; in practice, using a decaying learning rate schedule would bring the algorithm in line with the theory."],"forward_implications":["If the claims hold, anomaly detectors can get both adaptive representations and explicit margin geometry without sacrificing either.","Joint training gives the largest gains under severe class imbalance, suggesting representation adaptation matters most when the normal class is poorly represented.","Kernel approximations such as Nyström and RPCholesky recover near-exact accuracy at a fraction of memory and time, making the approach practical on large datasets.","The alternating recipe of boundary re-fitting followed by margin-aware feature updates is a template that could extend to other kernel-based one-class objectives."],"fun_headline_variants":["Co-train CNN features and SVDD boundary for anomaly detection","Joint CNN-SVDD training raises AUROC by up to 0.117","Feature and boundary co-training beats fixed-feature SVDD","Deep margin SVDD with trainable features outperforms fixed kernel","DLM-SVDD: trainable features plus large-margin boundary"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The convergence guarantee rests on the assumption that consecutive CNN parameter updates shrink to zero during training; the paper assumes this 'asymptotic regularity' rather than proving it, and Adam with a fixed learning rate makes it nontrivial.","fun_headline_variants_meta":{"raw":{"variants":["Co-train CNN features and SVDD boundary for anomaly detection","Joint CNN-SVDD training raises AUROC by up to 0.117","Feature and boundary co-training beats fixed-feature SVDD","Deep margin SVDD with trainable features outperforms fixed kernel","DLM-SVDD: trainable features plus large-margin boundary"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001786,"raw_usage":{"total_tokens":6890,"prompt_tokens":771,"completion_tokens":6119,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":6028}},"tokens_in":515,"tokens_out":6119,"duration_ms":39452,"temperature":1.0,"reasoning_tokens":6028,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T05:26:45.544297+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the alternating scheme with the paper's fixed Adam learning rate and record squared parameter differences across epochs; if ||ω^(t+1) − ω^(t)|| does not tend to zero even as the loss decreases, then a premise of the stationarity proof is violated. A simpler check: if the CNN gradient norm plateaus above zero while the Frank-Wolfe gap goes to zero, the network block is not reaching the claimed stationarity.","supporting_citations":[],"review_version":1}