{"id":"4988be9d-db2b-42d3-bf41-f0bfdc2bfae2","arxiv_id":"2501.09187","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A vector-quantized autoencoder that dynamically allocates code resolution per image patch and uses learned normal budget priors to detect visual defects.","lead":"PVQAE is a defect detection model that assigns fine image codes to detailed regions and coarse codes to smooth regions, and learns normal code-allocation patterns from defect-free images. It reports strong accuracy on three industrial datasets, and its single-model design may reduce the cost of inspecting multiple product types.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim rests on an altered HVQ-Trans baseline and a nonstandard multi-object protocol; Table I does not yet establish superiority over the original method.","rationale":"The reader's weakest_assumption focuses on the localized-defect premise underlying the budget-prior transformer. That is a legitimate methodological limitation, but it is not the most load-bearing risk for the paper's headline claim of state-of-the-art performance. The headline claim is empirical and depends on fair, reproducible comparison. The paper weakens HVQ-Trans by removing its optimal-transport objective, uses a nonstandard multi-object protocol while likely borrowing standard per-class baseline scores, tunes λ on the test set, and gives no error bars or code. Any of these alone would warrant caution; together they prevent the SOTA claim from being verified. I do not see an internal inconsistency in the method itself, and the proposed mechanism is plausible, so the reader's CONDITIONAL verdict is appropriate. My concern does not move the verdict; it reinforces the requested revisions: standard per-class comparisons, original HVQ-Trans, held-out hyperparameter selection, multiple seeds, and code release.","tokens_in":11134,"tokens_out":3128,"duration_ms":35330,"concrete_test":"Rerun Table I using the official HVQ-Trans implementation with its optimal-transport objective intact under the paper's multi-object protocol on MVTecAD, and run PVQAE with λ selected on a held-out validation split instead of the test set; report image- and pixel-level AUROC means and standard deviations over at least 5 seeds for both methods, plus per-class results. If PVQAE still exceeds HVQ-Trans by the reported margin, the central claim survives; if the margin collapses, the SOTA claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim, stated in Section IV-B as state-of-the-art performance, is not reliably supported by the reported comparisons. Section IV-A says public implementations from [42] and [14] were used for baselines, 'excluding the optimal-transport objective from HVQ-Trans to focus on representation learning comparison.' The optimal-transport objective is a core component of HVQ-Trans, so removing it weakens the strongest reconstruction-based baseline; the comparison is therefore not against the method that actually achieved the published HVQ-Trans results. Additionally, all methods are trained and tested on multiple objects simultaneously, a nonstandard protocol for MVTecAD, while many baseline numbers are presumably taken from standard per-class evaluations; the table mixes protocols without per-class retraining evidence. Figure 5 selects the budget-loss weight λ by maximizing test-set AUROC, a test-set hyperparameter choice, and no error bars or multiple-seed variability are reported. No code is released. These issues do not prove the method is wrong, but they mean the evidence for 'superior performance' is not yet comparable or reproducible. The localized-defect premise identified by the reader is a real limitation, but S_recon provides a fallback; the evaluation comparability issue is more load-bearing for the SOTA claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PVQAE, a VQ-VAE-based framework for unsupervised visual defect detection. The model introduces a patch-aware dynamic code assignment scheme in which a dynamic routing module selects one of several code resolutions per image patch using a Gumbel-Softmax mechanism, guided by a budget loss based on DWT entropy. A Budget Prior Transformer learns normal code-budget sequences, and the final defect score combines a cross-entropy prior-mismatch term with the L2 reconstruction error. Experiments on MVTecAD, BTAD, and MTSD report image- and pixel-level AUROC against memory-based and reconstruction-based baselines, with ablations on the budget-loss schedule and on normal budget priors.","tokens_in":11328,"tokens_out":7237,"duration_ms":70410,"significance":"The core idea is coherent and potentially useful: learning a context-dependent code allocation instead of fixing the representation capacity per image is a reasonable way to address the expressiveness-compactness trade-off in VQ-based anomaly detection. The ablations on the loss schedule and on the use of class-conditioned priors are informative, and the method is evaluated on three public datasets. However, the paper's central state-of-the-art claim is not yet reliably supported because the strongest baseline is modified, the evaluation protocol is nonstandard and not fully controlled, the key hyperparameter is selected on the test set, and no uncertainty estimates are reported. If these issues are addressed, the method could be a meaningful contribution to multi-object unsupervised defect detection.","major_comments":[{"comment":"The state-of-the-art claim in Section IV-B rests on a comparison against a modified HVQ-Trans baseline: Section IV-A states that the public implementation was used 'excluding the optimal-transport objective from HVQ-Trans to focus on representation learning comparison.' Because the optimal-transport objective is a core component of HVQ-Trans's codebook assignment, removing it changes the method being compared. The authors should either run the unmodified HVQ-Trans under the same protocol or explicitly restrict the claim to the ablated baseline.","section":"Section IV-A, Table I"},{"comment":"The evaluation protocol is not controlled across baselines. The paper states that for fairness all methods were trained and tested on multiple objects simultaneously for MVTecAD, but standard MVTecAD evaluations are per-class, and the table does not identify which baseline numbers were produced under the multi-object retraining protocol and which, if any, were taken from published per-class results. Without retraining every baseline under identical conditions and reporting those conditions, the comparison in Table I cannot support the claimed superiority.","section":"Section IV-A, Section IV-B.1, Table I"},{"comment":"No error bars or multiple-seed results are reported, and the terminal budget-loss weight is selected by maximizing test-set AUROC on MVTecAD (Figure 5). Since this weight is a key hyperparameter of the method, choosing it on the test set makes the reported numbers optimistic and prevents assessment of whether the improvement is significant. The authors should use a validation split or report the full sensitivity curve with uncertainty estimates.","section":"Figure 5, Section IV-B"},{"comment":"Even under the authors' protocol, the reported numbers do not show uniform superiority over PatchCore: on Leather, PatchCore achieves 100/98.2 while PVQAE achieves 99.4/97.5, and on Bottle the image-level AUROC ties at 100. No mean AUROC across the 15 MVTecAD classes or statistical significance test is reported, so the blanket 'state-of-the-art performance' statement in Section IV-B is stronger than the table demonstrates.","section":"Table I"},{"comment":"The normal-budget-prior component assumes that defects are localized and that most regions of a defective image still look normal, so a transformer can predict a defective region's normal budget from its neighbors. The paper does not test the limits of this assumption; if a defect is large or appears in many scattered regions, S_prior will not fire and detection falls back entirely on S_recon. An experiment varying defect area would clarify the method's operating range.","section":"Section III-C.1"}],"minor_comments":[{"comment":"The Gumbel-Softmax equation is garbled; the denominator should be a sum over r of exp((g^r + delta^r)/tau), and the constraint sum_r b^r = 1 should be written as a separate equation.","section":"Section III-B.1"},{"comment":"The budget-loss equation is not typeset correctly, and the cost multiplier is defined as c = 2R-1 even though R was introduced as a set of resolutions; please clarify whether R here denotes the number of resolution levels.","section":"Section III-B.2"},{"comment":"The dataset name is written as both MTSD and MSTD in the same subsection; please standardize it.","section":"Section IV-A"},{"comment":"Reference [16] is the ImageNet classification paper and does not support the statement about 'minimal-volume spheres' for one-class classification; the intended reference appears to be missing or misnumbered.","section":"Section II-A"},{"comment":"The final score is called a pixel-wise product S = S_prior x S_recon, but S_prior is defined as a normalized scalar and S_recon as an L2 image; the dimensionality and alignment of the two terms should be specified.","section":"Section III-C.2"},{"comment":"The statement that public implementations from [42] and [14] were used is confusing because [42] is a dataset paper (Real-IAD), not an implementation; please correct the citation.","section":"Section IV-A"},{"comment":"The sentence 'It improves enhances the conventional VQ-VAE' contains a typo; please revise.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The core idea is worth pursuing, but the paper's main quantitative claim is not yet supported by the reported experiments. The modified HVQ-Trans baseline, the mixed evaluation protocol, and the test-set hyperparameter selection should be addressed before acceptance. I would also encourage the authors to release code or detailed configuration files, given the nonstandard protocol used in the comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a coherent VQ-VAE extension for unsupervised defect detection, and the entropy-guided dynamic code allocation plus the normal budget prior are genuinely new pieces not in HVQ-Trans or VQ-E. The SOTA claim in Section IV-B, however, is not yet supported: the comparison strips the optimal-transport objective from HVQ-Trans, runs everyone under a multi-object protocol that differs from standard MVTec practice, reports no error bars, and tunes lambda on the test set (Figure 5). None of that proves the method wrong, but it means the headline numbers aren't comparable to published baselines.\n\nWhat's good: the budget loss with context-dependent entropy cost is a sensible way to push the model toward economical code usage, and using the learned allocation pattern as an anomaly signal at test time is a nice twist. The single-model across classes story is practically appealing. The ablations on progressive scheduling and per-class priors are informative, even if small. Citation practice looks fine; self-citations are minor.\n\nSoft spots: the HVQ-Trans modification is load-bearing because the optimal-transport objective is core to that method's published performance. The multi-object training protocol is nonstandard; per-class results are reported, but it's unclear whether published baseline numbers were computed under the same protocol. No seeds, no code, and lambda chosen by test AUROC. The localized-defect premise is a genuine limitation, but the reconstruction term provides a fallback, so I'd call that minor-to-moderate. The paper also doesn't quantify memory or speed, which matters for the efficiency claim.\n\nVerdict: worth a serious referee, but the review should demand standard per-class comparisons, multiple seeds, held-out hyperparameter selection, and code release. If those come through, the contribution stands. As is, the paper is an interesting method with unfinished evidence.\n\nRecommendation: send to peer review with major revision requested.","headline":"A coherent VQ-VAE extension with genuinely new dynamic code allocation and a budget prior, but the SOTA claim rests on a modified HVQ-Trans baseline and a nonstandard protocol.","tokens_in":11896,"tokens_out":1629,"would_cite":true,"duration_ms":17418,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A patch-aware VQ-VAE that dynamically allocates code resolution by context richness, plus learned normal budget priors, is claimed to achieve state-of-the-art unsupervised defect detection on MVTecAD, BTAD, and MTSD.","keywords":["unsupervised defect detection","vector quantization","VQ-VAE","dynamic code allocation","normal budget prior","anomaly detection","MVTecAD","representation learning"],"falsifier":"Build or select a test set of defective images in which the defect covers more than half the image or appears as many scattered regions, and measure image-level AUROC with and without the S_prior term; if the paper's mechanism is the cause of the gain, the product score loses its advantage and performance falls toward reconstruction-only, exactly where the paper's localized-defect assumption is violated.","tokens_in":10886,"feed_emoji":"🔍","tokens_out":5027,"duration_ms":49962,"temperature":0.7,"pith_summary":"This paper argues that the tension in unsupervised defect detection between expressive and compact representations can be resolved by letting the model decide, per image region, how many latent codes to spend. It proposes a patch-aware VQ-VAE where a routing module chooses coarse or fine code resolutions according to context richness, guided by a budget loss that makes fine codes costly. The learned allocation patterns on normal data are stored as priors by a small transformer, and at test time a defect score multiplies reconstruction error by deviation from the predicted normal budget. The paper reports that this scheme outperforms memory-based and reconstruction-based baselines on three industrial datasets with a single model trained across all object classes. If correct, the contribution is a data-driven answer to codebook capacity selection, one of the main knobs in VQ-based anomaly detection.","feed_headline":"Defect detector learns where fine detail is normal to flag anomalies","feed_subtitle":"Patch-aware VQ-VAE spends coarse codes on plain surfaces, fine codes on intricate ones, and uses the learned pattern to catch defects…","key_machinery":"The core machinery is patch-aware dynamic code allocation: an encoder produces feature embeddings at several resolutions, and a Dynamic Routing Module (average pooling, MLP gating, Gumbel-Softmax) picks a resolution for each patch. A budget loss, whose cost is the inverse of the discrete wavelet entropy of each patch scaled by resolution, pushes the model toward economical codes. A frozen pass records the chosen resolution matrix as a sequence, a Budget Prior Transformer learns to predict each region's resolution from the rest, and the final defect score is the pixel-wise product of the cross-entropy between the dynamic budget and the predicted normal budget and the L2 reconstruction error.","core_discovery":"PVQAE claims that replacing fixed, uniform codebook allocation with a learned, context-dependent budget—fine codes for rich-detail regions, coarse codes for simple ones—and recording the resulting normal budgets as priors lets a single VQ-VAE-based model detect defects across many object classes at once. At inference, defects are scored by the product of how much a region's dynamically chosen resolution deviates from its predicted normal budget and how poorly that region reconstructs. On MVTecAD, BTAD, and MTSD the paper reports image- and pixel-level AUROC at or above memory-based (PaDiM, PatchCore) and reconstruction-based (VQ-E, HVQ-Trans) baselines, with ablations showing the linear budget-loss schedule and per-class priors contribute most of the gain.","pith_inferences":["A natural extension is to apply the same budget-prior mechanism to medical or satellite anomaly detection, where the 'defects are localized' assumption also often holds and where per-class priors could be learned from healthy anatomy rather than product textures.","Because the score is a product of a prior term and a reconstruction term, one could test replacing the pixel-space L2 reconstruction error with a feature-space distance to reduce sensitivity to lighting and sensor noise.","The paper does not explore how the budget prior degrades as defect area grows; a reader could measure the prior term's contribution separately on large-defect samples to map the boundary of the method's stated assumption.","The dynamic routing could be reused as a self-supervised pretraining signal: the predicted normal budget for a patch could supervise the router on unlabeled mixed-class data, potentially removing the need for class tokens at inference."],"forward_implications":["A single model can cover multiple product classes, removing the need to train a separate detector for each object type.","The normal budget prior acts as a second detection signal beyond reconstruction, so defects that happen to reconstruct well can still be caught by their unusual code-allocation pattern.","The progressive budget-learning schedule (linearly increasing the budget-loss weight) and per-class priors both improve AUROC over constant weighting and universal priors, as shown in the paper's ablations.","Resolution maps become a visible side output: they show where the model expects detail, which can localize defects and make detection decisions more interpretable.","The method keeps VQ's efficiency, spending few and coarse codes on smooth surfaces and reserving fine codes only for intricate normal regions."],"supporting_citations":[{"why":"HVQ-Trans is the hierarchical vector quantized transformer baseline with static multi-resolution codebooks that PVQAE compares against and extends.","marker":"[14]"},{"why":"VQ-GAN supplies the encoder, decoder, discriminator, and transformer-prior design that PVQAE builds on.","marker":"[35]"},{"why":"The Dynamic Routing Module for multi-resolution code assignment is directly inspired by this dynamic vector quantization work.","marker":"[40]"},{"why":"Gumbel-Softmax provides the differentiable approximation used to select resolution levels during training.","marker":"[41]"},{"why":"MVTecAD is the primary industrial defect detection benchmark on which image- and pixel-level AUROC are reported.","marker":"[43]"},{"why":"PatchCore is the strongest memory-based baseline and also supplies the evaluation protocol followed in the paper.","marker":"[13]"},{"why":"PaDiM is the memory-based patch-distribution baseline that PVQAE is compared against.","marker":"[12]"},{"why":"VQ-E is the standard vector quantization baseline for anomaly detection that motivates the need for dynamic code capacity.","marker":"[38]"},{"why":"The VQ-VAE objective and straight-through estimator are the foundation of the codebook learning procedure.","marker":"[39]"}],"fun_headline_variants":["Patch-aware VQ-VAE learns where fine detail is normal to spot defects","Context-aware code allocation boosts unsupervised defect detection","VQ-VAE adapts code resolution to reveal anomalies in industrial images","Learn normal detail budgets to catch defects with VQ-VAE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes defects are localized and that most regions of a defective image still look normal, so a transformer can predict each region's normal code budget from its neighbors; if a defect is large or scattered, the normal-budget prior stops flagging it and only reconstruction error remains.","fun_headline_variants_meta":{"raw":{"variants":["Patch-aware VQ-VAE learns where fine detail is normal to spot defects","Context-aware code allocation boosts unsupervised defect detection","VQ-VAE adapts code resolution to reveal anomalies in industrial images","Learn normal detail budgets to catch defects with VQ-VAE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1325,"prompt_tokens":816,"completion_tokens":509,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":437}},"tokens_in":432,"tokens_out":509,"duration_ms":5494,"temperature":1.0,"reasoning_tokens":437,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:10:37.474480+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build or select a test set of defective images in which the defect covers more than half the image or appears as many scattered regions, and measure image-level AUROC with and without the S_prior term; if the paper's mechanism is the cause of the gain, the product score loses its advantage and performance falls toward reconstruction-only, exactly where the paper's localized-defect assumption is violated.","supporting_citations":[{"cited_title":"Taming transformers for high- resolution image synthesis,","cited_arxiv_id":null,"evidence_quote":"VQ-GAN supplies the encoder, decoder, discriminator, and transformer-prior design that PVQAE builds on."},{"cited_title":"Hierarchi- cal vector quantized transformer for multi-class unsupervised anomaly detection,","cited_arxiv_id":null,"evidence_quote":"HVQ-Trans is the hierarchical vector quantized transformer baseline with static multi-resolution codebooks that PVQAE compares against and extends."},{"cited_title":"Towards accurate image coding: Improved autoregressive image generation with dynamic vector quantization,","cited_arxiv_id":null,"evidence_quote":"The Dynamic Routing Module for multi-resolution code assignment is directly inspired by this dynamic vector quantization work."},{"cited_title":"Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection,","cited_arxiv_id":null,"evidence_quote":"MVTecAD is the primary industrial defect detection benchmark on which image- and pixel-level AUROC are reported."},{"cited_title":"Padim: a patch distri- bution modeling framework for anomaly detection and localization,","cited_arxiv_id":null,"evidence_quote":"PaDiM is the memory-based patch-distribution baseline that PVQAE is compared against."},{"cited_title":"Patch- wise vector quantization for unsupervised medical anomaly detection,","cited_arxiv_id":null,"evidence_quote":"VQ-E is the standard vector quantization baseline for anomaly detection that motivates the need for dynamic code capacity."},{"cited_title":"Neural discrete representation learning,","cited_arxiv_id":null,"evidence_quote":"The VQ-VAE objective and straight-through estimator are the foundation of the codebook learning procedure."}],"review_version":1}