{"id":"5104746a-1eb2-4e98-865c-75ce390387de","arxiv_id":"2505.06881","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A handcrafted contrast-normalization preprocessing layer, NeuRN, gives mixed and often large changes in cross-digit-domain classification accuracy, with no aggregate or statistical support for the claimed improvement.","lead":"This paper proposes a new preprocessing layer, NeuRN, that replaces each image pixel with a value based on the local patch contrast, and tests whether adding this layer improves classification when models trained on one digit dataset are tested on another. The paper also applies the Needleman-Wunsch sequence alignment algorithm to compare deep neural network architectures and select a representative subset for experiments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that NeuRN broadly improves domain generalization is not established by Table 1: many architectures degrade on several tasks, no aggregate or statistical analysis is reported, and the preprocessing transform itself is underspecified.","rationale":"The reader's weakest_assumption focuses on the under-specification of the NeuRN transform, which is a genuine reproducibility problem: k, the scope of c, and the handling of zero-variance patches are never defined, so the method cannot be implemented from the text alone. However, the more load-bearing defect for the central claim is the evidentiary gap in Table 1. Even granting every unspecified implementation detail, the reported results do not demonstrate broad improvement: several architectures have negative mean deltas, large individual degradations are as prominent as large gains, no error bars or repeated runs are reported, and no aggregate significance test is provided. The 'n of 12 tasks improved' framing is insufficient because it ignores the magnitude of losses on the remaining tasks. The Needleman-Wunsch similarity contribution also shows inconsistent scoring parameters (a match score of 4 and gap penalty -1 in the second description versus 2 and -2 in the first), reinforcing the overall lack of rigor, though it is secondary to the central NeuRN claim. No independent support such as code, formal verification, or reproducible artifacts is provided. Since the reader already recommended REJECT, and my analysis strengthens that conclusion rather than changing it, the verdict should remain unchanged.","tokens_in":14534,"tokens_out":5294,"duration_ms":49260,"concrete_test":"Re-analyze Table 1 by computing, for each of the 14 architectures, the mean and median accuracy delta over the 12 transfer tasks, and run a paired Wilcoxon signed-rank test on the 168 model-task deltas (or a permutation test clustered by model). If the aggregate median delta is not significantly positive, or if more than a few architectures have non-positive median deltas, the headline claim that NeuRN broadly improves domain generalization is not supported by the paper's own data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that NeuRN 'demonstrates broad applicability across DNN solutions and improves performance' on source-only domain generalization. The evidence offered is Table 1, and that table does not internally support the claim. Per architecture, NeuRN improves some transfer tasks but degrades others; for example, DenseNet121 falls from 74.3 to 26.4 on M→U, ResNet50V2 falls from 78.7 to 31.4 on M→U, and VGG19 loses 17.8 points on U→M. The paper reports only 'n of 12 tasks improved' (e.g., 'DenseNet121 improves in 8 out of 12 tasks') without mean or median deltas, error bars, repeated seeds, or any paired significance test across the 12 tasks. Counting wins can mask large losses on the remaining tasks, and the claimed broad applicability requires showing that the typical effect is positive, not merely that more than half the tasks improve. Independently, the transform in Methods—Ia = 1/(c·sigma_pk), where c = max(sigma)—is underspecified: the patch size k is never stated, the scope of the maximum is not defined (image, batch, or dataset), and zero-variance patches make sigma_pk = 0, leaving Ia undefined with no epsilon. These are two separable problems: even if the transform were fully specified, the reported table would still not support the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes NeuRN, a preprocessing layer that maps each input image to a representation based on local standard deviations, motivated by divisive normalization and winner-take-all mechanisms in the visual cortex. The authors integrate NeuRN into 12 shortlisted DNNs plus two NAS-derived architectures and evaluate source-only domain generalization on MNIST, SVHN, USPS, and MNIST-M, reporting per-task accuracies in Table 1. As a second contribution, they adapt the Needleman-Wunsch sequence alignment algorithm to quantify architectural similarity among 44 DNNs and use this to select a representative subset of models. The paper claims that NeuRN broadly improves domain generalization across architectures, while acknowledging in the conclusion that tests on additional high-resolution datasets were not performed due to resource limits.","tokens_in":14767,"tokens_out":9522,"duration_ms":91834,"significance":"If the NeuRN claims were established, an architecture-agnostic preprocessing transform that improves source-only domain generalization would be a useful and inexpensive contribution, and the neuroscience connection would be of interest to the broader DNN robustness community. The N-W-based architecture similarity analysis is a reasonable idea for selecting diverse models, and the breadth of tested architectures (CNNs, ViT, NAS) is a strength. However, the headline claim is not supported by the reported evidence: the transform is underspecified and not reproducible, Table 1 contains large accuracy drops that are not analyzed, and no error bars, repeated seeds, or significance tests are provided. The paper is better read as an exploratory study than as a demonstration of broad applicability; in its current form the significance is limited.","major_comments":[{"comment":"The definition of Ia is not computable as written. The formula Ia = 1/(c·σpk) requires three unspecified quantities: the patch size k is never stated; the scope of c = max(σ) is not defined (per image, per batch, or per dataset); and no epsilon or floor is given for the case σpk = 0, which occurs in flat regions of digit images. Because NeuRN is the central proposed method, this underspecification prevents reproduction of Table 1 and makes the reported numbers unverifiable.","section":"Methods: Neural Response Normalization"},{"comment":"Table 1 does not support the claim of broad applicability. Several architectures suffer large losses: DenseNet121 drops from 74.3 to 26.4 on M→U, ResNet50V2 drops from 78.7 to 31.4 on M→U, Xception drops from 77.8 to 31.8 on MM→U, and VGG19 drops from 66.2 to 48.4 on U→M. The paper reports only the number of tasks improved for each model, never the mean or median delta, the spread of deltas, error bars, repeated seeds, or a paired test across the 12 tasks. Win counting can mask large negative effects, and the phrase 'significant improvement' is used without any statistical support. Establishing broad applicability requires showing that the typical effect is positive and that losses are not catastrophic, not merely that more than half of the tasks improve.","section":"Table 1 / Experiments & Results"},{"comment":"The appendix states that 'NeuRN consistently enhances model performance across a variety of domain transfer tasks' and that the enhancement 'does not disproportionately favor one model over another.' This is directly contradicted by Table 1, which shows large negative deltas for several models (for example, DenseNet121 M→U and ResNet50V2 M→U) and highly uneven gains across architectures. This internal inconsistency is a load-bearing problem because the functional-similarity analysis is offered as auxiliary evidence for the main claim.","section":"Appendix: DNNs' architectural and functional similarity"}],"minor_comments":[{"comment":"In the expression for σpk, the subscript is written as 'p,k' in the summation 'i,j∈p,k'; this appears to be a typo for 'i,j∈pk'.","section":"Methods: Neural Response Normalization"},{"comment":"The scoring parameters are specified inconsistently: the recurrence section describes match score s = 2 and gap penalty d = -2, while the later application says a match score of 4 and vertical/horizontal dissimilarity of -1; the manuscript should state which scoring scheme produced Figure 1 and the shortlist.","section":"Methods: Needleman-Wunsch algorithm"},{"comment":"Two references are incomplete: 'Zhang, Ragab, and Sagarna' has no conference or journal, and 'Veerabadran, Raina, and Sa' has no publication details; these must be completed or removed.","section":"References"},{"comment":"No random seeds or number of repetitions are reported, so the absence of error bars cannot be interpreted as stability; even a single-seed comparison should be accompanied by a statement of variance across seeds or a justification for not reporting it.","section":"Experiments & Results"},{"comment":"The phrase 'close to benchmark results' is undefined; the benchmark accuracy to which the bold-and-underlined entries are compared should be explicitly named.","section":"Table 1 caption"},{"comment":"The N-W-based shortlisting is not validated: the paper provides no quantitative evidence that the 12 selected models are representative of the 44-model pool, and no comparison with simpler selection criteria or alternative similarity measures.","section":"Experiments & Results: Similarity across DNN architectures"}],"recommendation":"reject","confidential_remarks":"The core NeuRN idea may be salvageable in a future submission with a fully specified transform, a much narrower empirical claim, and proper statistical analysis, but as submitted the central claim is undermined by the paper's own data and the method is not reproducible. The N-W model-selection contribution is also unevaluated. If the authors resubmit, the editor may wish to require code release, dataset splits, and a comparison of NeuRN against standard preprocessing baselines such as local contrast normalization or instance normalization."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take on arXiv:2505.06881. The paper is a wide empirical sweep of a cheap preprocessing layer for source-only domain generalization. The breadth is real—12 architectures, 12 transfer tasks across MNIST/SVHN/USPS/MNIST-M, including NAS-derived models—and the layer is attractively simple: no retraining, negligible compute, applied as a pre-processing step. That said, the headline claim that NeuRN 'demonstrates broad applicability' is not backed by the paper's own Table 1. Counting improved tasks (e.g., '8 of 12') masks some very large losses, and the method itself is underspecified.\n\nWhat is genuinely useful here: the per-task numbers for many architectures, and the N-W alignment idea for shortlisting representative architectures is a reasonable heuristic, though it's a direct adaptation of a known sequence-alignment method. The paper also cites LCN and LRN but never compares against them, so we don't know whether NeuRN is anything more than a re-statement of local contrast normalization.\n\nThe soft spots are in proportion to how central they are. First, the transform Ia = 1/(c·sigma_pk) is incomplete: the patch size k is never given; the scope of c=max(sigma) is not defined (per image, batch, dataset?); and there is no epsilon for zero-variance patches. In flat image regions sigma_pk is zero, so Ia is undefined or unbounded. This alone makes the reported numbers hard to reproduce. Second, the evidence: DenseNet121 falls from 74.3% to 26.4% on M→U, ResNet50V2 from 78.7% to 31.4%; VGG19 loses 17.8 points on U→M. There are no error bars, no repeated seeds, no paired significance test, and no aggregate mean or median delta. Counting wins is not a statistical argument. Third, the N-W scoring parameters are inconsistent between sections (match 2/mismatch -2 vs match 4/gap -1), and the similarity clusters are presented without validation.\n\nWho is this for? A reader looking for a quick preprocessing baseline for digit DG might find the table a useful starting point, but the paper's conclusions should not be taken as established. It deserves a serious referee: the claim is testable and the sweep is substantial, and a referee could force the missing specifications and baselines. I would send it out, but with the expectation of major revision or rejection.\n\nIn short: a solid empirical effort undermined by an under-specified method and an over-stated conclusion. Worth a reading-group discussion, but not something I'd build on as-is.","headline":"Large empirical sweep of a cheap normalization layer, but the headline claim of broad domain-generalization gains is not supported by the paper's own table, and the method is underspecified.","tokens_in":15343,"tokens_out":4185,"would_cite":false,"duration_ms":39897,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes NeuRN, a neuro-inspired preprocessing layer that normalizes local patch contrast, and claims it improves domain generalization in most CNN, ViT, and NAS-derived image classifiers it tested.","keywords":["domain generalization","neural response normalization","neuro-inspired deep learning","visual cortex","image classification","Needleman-Wunsch similarity","neural architecture search","cross-domain transfer"],"falsifier":"Run the NeuRN transform exactly as written on a single MNIST image with a uniform border: for a patch entirely inside the border, $\\sigma_{p_k}=0$, making $I_a=1/0$ undefined; since the paper specifies neither an epsilon for flat patches nor the scope of $c=\\max(\\sigma)$, the reported accuracies are not reproducible as stated. A consistency check that fixes these choices and reruns the Table 1 transfers would settle whether the gains are real.","tokens_in":14251,"feed_emoji":"🧠","tokens_out":8499,"duration_ms":77487,"temperature":0.7,"pith_summary":"NeuRN is a proposed preprocessing layer that converts each image into a local-contrast map by replacing every pixel with the reciprocal of the normalized standard deviation of the patch around it. The paper tries to establish that this neuro-inspired transform, inserted before any classifier, improves accuracy when a model trained on one digit dataset is tested on a different, unseen digit dataset, across CNNs, a Vision Transformer, and NAS-derived architectures. It also develops a Needleman-Wunsch sequence-alignment score that groups deep architectures by layer sequence and uses that score to pick a representative subset of models. If the central claim holds, domain generalization would be obtainable by a simple, architecture-agnostic data transform rather than by re-engineering each architecture.","feed_headline":"A neuro-inspired layer improves cross-domain accuracy in most models","feed_subtitle":"Added before training, the NeuRN layer lifts accuracy on unseen digit datasets in most of 12 CNN, ViT, and NAS models.","key_machinery":"The central object is the NeuRN preprocessing transform, a pixel-wise local standard-deviation normalization: each pixel of the input is replaced by $I_a = 1/(c\\,\\sigma_{p_k})$, where $\\sigma_{p_k}$ is the standard deviation of the $k\\times k$ patch around the pixel and $c=\\max(\\sigma)$. Conceptually this encodes local contrast and suppresses absolute intensity, which is what the paper means by a domain-agnostic representation; the design is inspired by divisive normalization and Winner-Takes-All behavior in visual cortex neurons. The second supporting mechanism is the Needleman-Wunsch sequence-alignment score, which linearizes each DNN into a string of layer types and aligns pairs of such strings with match scores and gap penalties to produce a similarity index; this index is used to cluster 44 architectures and choose the 12 models evaluated with NeuRN.","core_discovery":"On its own terms, the paper's central claim is that NeuRN produces a domain-agnostic feature representation, and that models trained on that representation generalize better to unseen target domains. For an input image $I \\in \\mathbb{R}^{W\\times H\\times C}$, NeuRN computes, for each pixel, the standard deviation $\\sigma_{p_k}$ of the $k\\times k$ patch centered at that pixel, and outputs $I_a = 1/(c\\,\\sigma_{p_k})$ with $c=\\max(\\sigma)$. Experiments in Table 1 compare fine-tuned DNNs with and without this preprocessing on MNIST, SVHN, USPS, and MNIST-M across 12 source-to-target transfers; representative reported gains include ShuffleNet on M to MM rising from 14.1% to 71.1% and SPOS on MM to S rising from 25.6% to 69.0%, while some transfers show drops. The paper also claims that Needleman-Wunsch alignment of layer sequences gives a meaningful architecture-similarity measure that clusters related models and supports shortlisting 12 representative architectures from 44.","pith_inferences":["Since the paper tests only digit datasets (MNIST, SVHN, USPS, MNIST-M), an extension I would draw is to test NeuRN on natural-image domain shifts such as ImageNet variants or medical imaging to see whether local-contrast normalization transfers to higher-resolution domains; the paper lists resource constraints as the current limitation.","Read as a contrast-normalization mechanism, NeuRN resembles global-contrast or style-normalization preprocessing that strips absolute intensity and color statistics; if that reading is right, it should be stackable with data augmentation and invariant-feature methods rather than a competitor to them.","The N-W architecture-similarity score could be used beyond model selection, for example to predict which architecture pairs transfer best or to weight ensemble members; the paper does not test those uses."],"forward_implications":["Adding NeuRN as a preprocessing layer raises source-only accuracy on most of the 12 digit-transfer tasks for most of the 12 architectures in Table 1, including large gains such as ShuffleNet on M to MM (14.1% to 71.1%) and SPOS on MM to S (25.6% to 69.0%).","Improvements appear across CNNs, a Vision Transformer, and NAS-derived models, supporting the paper's claim that the benefit is architecture-agnostic rather than tied to one network family.","In the appendix, NeuRN integration raises the average pairwise functional similarity of the models from about 0.7 to 0.8, which the paper interprets as NeuRN harmonizing model behavior.","The Needleman-Wunsch similarity measure, if valid, gives a reusable way to select a small representative set of DNNs from a large pool for benchmarking.","Not all transfers improve; some accuracies drop, so the paper itself frames the effect as needing further investigation."],"supporting_citations":[{"why":"Supplies the biological Winner-Takes-All and visual-cortex inspiration that NeuRN is designed from.","marker":"Iqbal et al. 2024"},{"why":"Provides the MNIST dataset used as a source or target domain in the transfer experiments.","marker":"LeCun, Cortes, and Burges 2010"},{"why":"Provides the SVHN dataset used as a real-world-context digit target domain.","marker":"Netzer et al. 2011"},{"why":"Provides the USPS dataset used as a grayscale scanned-mail target domain.","marker":"Hull 1994"},{"why":"Provides MNIST-M, the colored-background dataset used to test adaptation to texture and color shifts.","marker":"Ganin et al. 2015"},{"why":"Defines the Vision Transformer architecture that NeuRN is integrated with and evaluated on.","marker":"Dosovitskiy et al. 2021"},{"why":"Defines the SPOS neural architecture search approach whose derived model is benchmarked with NeuRN.","marker":"Guo et al. 2019"},{"why":"Defines the Autoformer transformer search architecture whose derived model is benchmarked with NeuRN.","marker":"Xu et al. 2021"},{"why":"Supplies the Needleman-Wunsch sequence alignment algorithm adapted to measure DNN architecture similarity.","marker":"Likic 2008"}],"fun_headline_variants":["NeuRN layer boosts cross-domain accuracy in most models","A neuro-inspired layer helps models generalize to unseen data","Simple normalization inspired by neurons improves domain generalization","NeuRN: a cortex-inspired layer for better cross-domain image classification","Most models improve on unseen domains with NeuRN preprocessing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the transform $I_a = 1/(c\\,\\sigma_{p_k})$ is an actually computable image representation: the paper does not state the patch size $k$, does not say whether $c=\\max(\\sigma)$ is taken over one image, a batch, or the whole dataset, and gives no epsilon for patches with zero variance, so on images with flat regions the operation as written divides by zero.","fun_headline_variants_meta":{"raw":{"variants":["NeuRN layer boosts cross-domain accuracy in most models","A neuro-inspired layer helps models generalize to unseen data","Simple normalization inspired by neurons improves domain generalization","NeuRN: a cortex-inspired layer for better cross-domain image classification","Most models improve on unseen domains with NeuRN preprocessing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000226,"raw_usage":{"total_tokens":1473,"prompt_tokens":957,"completion_tokens":516,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":439}},"tokens_in":573,"tokens_out":516,"duration_ms":5786,"temperature":1.0,"reasoning_tokens":439,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:31:19.990222+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the NeuRN transform exactly as written on a single MNIST image with a uniform border: for a patch entirely inside the border, $\\sigma_{p_k}=0$, making $I_a=1/0$ undefined; since the paper specifies neither an epsilon for flat patches nor the scope of $c=\\max(\\sigma)$, the reported accuracies are not reproducible as stated. A consistency check that fixes these choices and reruns the Table 1 transfers would settle whether the gains are real.","supporting_citations":[{"cited_title":"J.; Fishell, G.; and Honnuraiah, S","cited_arxiv_id":null,"evidence_quote":"Supplies the biological Winner-Takes-All and visual-cortex inspiration that NeuRN is designed from."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MNIST dataset used as a source or target domain in the transfer experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the SVHN dataset used as a real-world-context digit target domain."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the USPS dataset used as a grayscale scanned-mail target domain."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides MNIST-M, the colored-background dataset used to test adaptation to texture and color shifts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Vision Transformer architecture that NeuRN is integrated with and evaluated on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the SPOS neural architecture search approach whose derived model is benchmarked with NeuRN."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Needleman-Wunsch sequence alignment algorithm adapted to measure DNN architecture similarity."}],"review_version":1}