{"id":"361e9c89-a334-48bb-a4a4-104cb8d4aefa","arxiv_id":"2411.11110","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Mixing conventional and quadratic neurons via a genetic search gives competitive retinal vessel segmentation with a 0.84M-parameter model, and a trained hypernetwork predicts good neuron configurations without rerunning the search.","lead":"This paper mixes standard and quadratic neurons inside a U-Net and uses an evolutionary search, plus a learned predictor, to choose the best neuron type for each block in retinal blood vessel segmentation. On four public fundus datasets, the resulting 0.84M-parameter model matches or beats published models that are two to fifty times larger.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 8's single-run comparisons cannot support the claim that neuron-type diversity, rather than search luck, drives the reported gains; the F1 margins (0.0034–0.0069) are within plausible seed noise.","rationale":"The reader's weakest assumption is exactly the load-bearing concern: Table 8 compares the heterogeneous neuron mixture against homogeneous conventional and quadratic networks in single runs, with F1 differences of 0.0034–0.0069 and AUC differences of 0.0009–0.0022. These gaps are plausibly within training noise on a 20-image test set, especially without error bars or a significance test. If the gap collapses under repeated seeds, the paper's conceptual contribution — that neuron diversity, not architecture search, drives the improvement — is unsupported, and the remaining result is a reasonably good NAS outcome with a modest parameter count. The hypernetwork evaluation is even thinner: it tests only three interpolated points and reports no numeric comparison, so it cannot independently validate the neuron-type dimension. I agree with the reader that these issues warrant a conditional disposition rather than acceptance. I see no reason to move the verdict further: the reported metrics are competitive, the ablation direction is consistent with the claim, and the methodological gaps are addressable with multi-seed reporting and a released implementation. The paper also contains a minor internal inconsistency (the claimed 'over 1.8%' margin on CHASE is not supported by Table 4, where the largest F1 lead is about 0.007), but that is not load-bearing relative to Table 8. My recommendation is therefore UNCHANGED: the reader's CONDITIONAL verdict is the right call, and the concrete test above would settle whether the concern actually lands.","tokens_in":20860,"tokens_out":2717,"duration_ms":42723,"concrete_test":"Retrain the three configurations in Table 8 — conventional-only, quadratic-only, and the searched Conv+Qua mixture — under the exact protocol of Section 4.1 (focal loss, Lookahead/Adam, ReLinear initialization, 1000 epochs) with at least 10 independent seeds each. Compute per-image F1 and AUC on the DRIVE test set for every run, then apply a paired Wilcoxon signed-rank test (or bootstrap) between the mixture and each homogeneous variant. Declare the neuron-type effect supported only if the mixture is significantly better (p<0.05) in F1 or AUC and the mean improvement exceeds the bootstrap noise floor. Also report the three hypernetwork-predicted configurations (depth=2, channel=8; depth=3, channel=18; depth=4, channel=28) with the same multi-seed protocol and compare numerically to the joint-search and U-Net baselines.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in Section 1, is that searching neuron types outperforms any single universal neuron. The only direct evidence isolating the neuron-type effect is Table 8, where the searched heterogeneous network is compared with two homogeneous re-trainings on DRIVE. The reported advantages are small: F1 0.8334 vs 0.8265 (Conv) and 0.8300 (Qua), and AUC 0.9901 vs 0.9879 and 0.9892. Every number is a single run with no standard deviation, no seed count, and no significance test. DRIVE's test set has only 20 images, so per-image F1 and AUC vary considerably across training seeds; published vessel-segmentation work routinely reports seed variance of this magnitude. If the Conv+Qua edge is within run-to-run noise, then the neuron-type gene is decorative and the contribution reduces to a standard NAS search. The secondary premise that the hypernetwork generalizes from its training grid (channels 5–35 in steps of 5, depths 2–4) to held-out channels 8, 18, and 28 is tested at only three points in Figure 13 with no numeric table, so it cannot rescue the claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces \"neuron programming\": treating neuron type (conventional versus quadratic) as an additional searchable design dimension, complementary to architecture-level NAS. A genetic algorithm jointly searches architecture and neuron-type assignments within a U-shaped encoder–decoder on the DRIVE training set, yielding a 0.84M-parameter network that reports state-of-the-art DRIVE SE/F1/AUC and transfers to STARE, CHASE DB1, and HRF. A plug-and-play variant first searches the architecture and then optimizes neuron types, and a hypernetwork is trained to predict neuron configurations for unseen depth/channel settings. The paper claims that mixing neuron types outperforms single-neuron-type networks and that the hypernetwork predictions are comparable to full NAS results at a fraction of the cost.","tokens_in":21012,"tokens_out":2377,"duration_ms":25978,"significance":"If the claims hold, the paper adds a genuinely new and cheap design dimension to neural architecture search for medical image segmentation, with strong empirical results on four public benchmarks and a parameter count an order of magnitude below competing methods. The search is run only on DRIVE and the found network is tested on externally fixed splits of STARE, CHASE DB1, and HRF, which is a commendable evaluation design. The hypernetwork is evaluated on depth/channel combinations (8, 18, 28) that lie inside the range but off the 5-step training grid of channel sizes, so the generalization test is a genuine holdout rather than a re-reporting of training points. The central limiting factor is that the key isolation of the neuron-mixing effect (Table 8) rests on single runs with no variance reporting, and the hypernetwork comparison is reported only graphically.","major_comments":[{"comment":"The paper's central claim that neuronal diversity, rather than the architecture search itself, drives the gains rests on Table 8, where replacing the searched mixture with homogeneous conventional or quadratic neurons lowers F1 from 0.8334 to 0.8265 and 0.8300 and AUC from 0.9901 to 0.9879 and 0.9892. These margins (0.0034–0.0069 in F1) are each based on a single training run with no standard deviation, no seed count, and no significance test. Because DRIVE's test set has only 20 images and vessel-segmentation results are known to vary with initialization at this level, the reported differences are within plausible run-to-run noise; the paper needs repeated-seed results (means and variances, or paired tests) to support the assertion that the neuron-type gene is not decorative.","section":"§5, Table 8"},{"comment":"The claim that \"the hypernetwork-predicted network performs comparably to optimal networks derived through NAS and outperforms the original U-Net\" is supported only by three bar charts (depth/channel = 2/8, 3/18, 4/28) with no numeric table and no error bars. The reader cannot verify the magnitude of the claimed parity or the improvement over U-Net, nor can the holdout nature of channels 8, 18, and 28 be quantitatively assessed. A table reporting ACC/SE/SP/F1/AUC for the hypernetwork prediction, the NAS-searched network, and the U-Net baseline at these three settings should be added.","section":"§4.4, Figure 13"},{"comment":"The comparisons against prior state-of-the-art methods are reported as single point estimates for all methods, including the proposed one, with no variance or significance information. This is less serious than the Table 8 issue because the margins to the strongest competitors (e.g., DRIVE SE 0.8489 vs 0.8356 for FR-UNet; F1 0.8334 vs 0.8316) are again small and could overlap under seed variation. The authors should state whether the reported numbers for the proposed method are means over multiple runs and, at minimum, provide the number of seeds used for the main results.","section":"§4.2, Tables 4–7"}],"minor_comments":[{"comment":"In the focal loss equation, the exponent is written as omega (ω) but the prose two sentences later refers to it as w; the notation should be unified.","section":"§4.1, Eq. (2)"},{"comment":"The hypernetwork is trained on \"the top 10 architectures with the highest F1 scores\" per depth/channel group, but the total number of architectures per group, the F1 range of the selected subset, and the split of the resulting training data are not given; this information is needed to judge whether the hypernetwork is learning a meaningful mapping or memorizing a small sample.","section":"§3.3"},{"comment":"The frequency counts in Figure 7 are presented without the total number of evaluated architectures per depth, so the reader cannot tell whether a frequency of 271 out of 1000 means a robust preference or a minority pattern; please add denominators.","section":"§4.2, Figure 7"},{"comment":"The genetic algorithm parameters (population size 20, 50 generations, crossover 0.9, mutation 0.7, bit-flip 0.5) are given without any sensitivity or repeat-run analysis; since these are free parameters of the search, at least a brief discussion of their influence or a citation to prior GA-NAS usage would strengthen the reproducibility story.","section":"§4.1"},{"comment":"The paper contains many minor grammatical errors and awkward formulations (e.g., \"in a dedicate manner,\" \"towards a condensed neuron programming,\" \"the hypernetwork 'programs' in real-time and convenient manner\"). These do not affect the technical content but should be corrected in a revision.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The empirical core is relevant and the transfer design is a strength: the search is confined to DRIVE and the found network is evaluated on three additional public datasets. The main risk is statistical: the paper's central attribution of gains to neuron diversity rests on single-run comparisons in Table 8 with margins that could easily be seed noise. The hypernetwork section, which is a claimed contribution, currently lacks numeric results entirely. If the authors add repeated-seed evaluations with variances and a numeric hypernetwork comparison table, the paper could be acceptable; as it stands, the central claim is not yet quantitatively supported. Scope-wise the paper fits an applied medical-imaging venue more than a theory venue; the \"neuron programming\" concept is clearly presented as an extension of NAS, which is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate empirical paper that adds a searchable neuron-type gene (conventional vs quadratic) to genetic NAS for retinal vessel segmentation, plus a hypernetwork that predicts configurations from depth and channel. The headline result—a 0.84M-parameter U-Net with DRIVE F1 0.8334, AUC 0.9901—is competitive, and the experimental discipline is better than most in this area: search runs only on DRIVE, the found network transfers to three public datasets, and the authors report that plug-and-play underperforms joint search. That last point is a sign of honesty.\n\nWhat's actually new is the encoding of neuron type as part of the operation gene in the search space, plus the two-stage search and the hypernetwork predictor. The components (quadratic neurons, genetic NAS, hypernetworks) are prior art, but the combination as a programming layer is new, and the frequency analysis in Figure 7 tells a plausible qualitative story that quadratic neurons are favored when instance normalization is present.\n\nThe soft spot is exactly where the reader put it: Table 8. The claim that neuron diversity drives the gains rests on F1 differences of 0.0034 to 0.0069 over homogeneous re-trainings, all single runs, with no error bars and no significance test. DRIVE's test set is 20 images; those margins are within seed noise in this literature. If multi-seed reporting holds the margin, the claim is real; if not, the neuron-type gene is decorative. The hypernetwork evaluation is also thin: three interpolated points, no numeric table, and the architecture details are under-specified. There is a minor internal inconsistency: the text says a \"1.8%\" margin on CHASE, but Table 4 shows the F1 margin is about 0.7 points. No code is released.\n\nNone of these are fatal. The empirical core is defensible, the transfer results are genuinely informative, and the parameter efficiency is real. This paper needs revision, not rejection. I would send it to peer review, with reviewers asked to verify Table 8 with seeded runs and to require a numeric comparison for the hypernetwork predictions. Worth bringing to a reading group as a case study in how a well-meaning NAS paper can over-interpret small margins.","headline":"Solid empirical NAS paper whose central neuron-diversity claim rests on single-run ablations; send it to review, but ask for error bars and code.","tokens_in":21741,"tokens_out":2860,"would_cite":false,"duration_ms":27024,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that searching the neuron type at each network position jointly with the architecture, not just the architecture alone, yields better retinal vessel segmentation, with a 0.84M-parameter searched network achieving the…","keywords":["retinal vessel segmentation","neuron programming","quadratic neuron","neural architecture search","hypernetwork","genetic algorithm","neuronal diversity","U-Net"],"falsifier":"Repeat the three neuron-type configurations from Table 8 on DRIVE across at least 20 random seeds and test whether the mixed network's F1 and AUC are consistently above both single-type networks with a paired test; if not, the neuron-type gene is not the source of the reported gain. A second check: evaluate hypernetwork predictions at held-out channels 8, 18, and 28 and report F1 with variance against the top-10 searched networks at those channels.","tokens_in":20492,"feed_emoji":"👁","tokens_out":5854,"duration_ms":48896,"temperature":0.7,"pith_summary":"This paper proposes neuron programming: instead of fixing every neuron in a network to one type, it searches over neuron types (conventional inner-product neurons versus quadratic neurons) jointly with the architecture, using a genetic algorithm on a U-Net backbone. On DRIVE the resulting 0.84M-parameter network achieves the highest sensitivity (0.8489) and F1 (0.8334) among the compared methods, including much larger models, and transfers to STARE, CHASE DB1, and HRF with competitive F1. The authors argue that this shows no single universal neuron type suits all locations, and that treating neuron type as a searchable gene complements architecture-level NAS. They also build a hypernetwork trained on the best searched configurations to predict neuron types for unseen depth and channel settings, cutting search cost.","feed_headline":"Neuron diversity lifts retinal vessel segmentation","feed_subtitle":"A 0.84M-parameter network that jointly searches architecture and neuron type tops much larger models on DRIVE.","key_machinery":"The carrying mechanism is the genetic-algorithm search over a block-based encoding in which each node's operation gene includes kernel size, normalization, activation order, and neuron type (conventional or quadratic), giving 16 operation sequences per node. Quadratic neurons replace the inner product with a product of two inner products plus a Hadamard-square term, following Fan et al. (2018), and are trained with the ReLinear initialization of Fan et al. (2023) to keep higher-order terms stable. The hypernetwork is a small classifier taking network depth and channel size as input and predicting the operation sequence at each position, trained with categorical cross-entropy on the top-10 searched configurations per depth and channel group. What makes the argument run is that the neuron-type gene can be flipped while everything else stays fixed, letting the ablations in Table 8 isolate the contribution of neuronal diversity.","core_discovery":"The paper's central claim is that searching the neuron type of each position alongside the network architecture—rather than training a homogeneous network of one neuron kind—improves segmentation performance, and that this 'neuron programming' is complementary to architecture search. Concretely, the best joint search returns a U-shaped network that mixes conventional and quadratic neurons, where the quadratic neuron computes $y = \\sigma((w_1^\\top x + b_1)(w_2^\\top x + b_2) + w_3^\\top (x \\odot x) + b_3)$, and this mixture reaches DRIVE SE 0.8489, F1 0.8334, and AUC 0.9901, the best sensitivity and F1 among all compared models while using only 0.84M parameters. The same searched network transfers to STARE (F1 0.8692), CHASE DB1 (F1 0.8224), and HRF (F1 0.8123). The paper further claims that the hypernetwork-predicted networks perform comparably to NAS-found optima and better than the original U-Net, supporting the view that neuron-type assignment can be predicted rather than searched at deployment time.","pith_inferences":["An untested consequence is that neuron-type search may transfer to other dense prediction tasks, such as vessel segmentation in OCT angiography or coronary angiography, whenever the searched network is retrained from scratch, since the claim is about the neuron-type gene rather than DRIVE-specific weights.","The paper's frequency analysis suggests quadratic neurons are especially frequent in later blocks; a testable extension is to restrict the search space to later blocks only and measure whether most of the F1 gain remains.","The hypernetwork's interpolation claim is supported only at three held-out points without reported numbers; a stronger validation would compare predicted configurations against the full search distribution at those channels, with standard deviations."],"forward_implications":["A single searched network of 0.84M parameters outperforms or matches models with 5 to 43M parameters on four benchmark sets, so neuron-type search is a parameter-efficient route to high sensitivity on fine vessels.","Because the plug-and-play variant first searches architecture then neuron types, any existing well-performing architecture can be upgraded to heterogeneous neurons without re-running a full NAS.","The hypernetwork turns neuron programming into a one-shot prediction: given depth and channel settings, it outputs the whole configuration, eliminating search at deployment time.","The combination of focal loss and a searched mixture of neuron types keeps sensitivity high on tiny vessels, the clinically relevant failure mode for diabetic retinopathy screening."],"supporting_citations":[{"why":"Defines the quadratic neuron form used as the alternative neuron type in the search space.","marker":"Fan et al. (2018)"},{"why":"Provides the ReLinear initialization and expressivity analysis that make quadratic networks trainable.","marker":"Fan et al. (2023)"},{"why":"Supplies the theoretical result that heterogeneous networks can approximate certain functions with polynomial neurons while homogeneous ones need exponentially many.","marker":"Liao et al. (2024)"},{"why":"The U-Net backbone and baseline that neuron programming augments.","marker":"Ronneberger et al. (2015)"},{"why":"The DRIVE dataset used for the search and primary evaluation.","marker":"Staal et al. (2004)"},{"why":"Genetic U-Net, the GA-based NAS baseline that the search strategy extends and compares against.","marker":"Wei et al. (2021)"},{"why":"The hypernetwork concept reused for predicting neuron configurations.","marker":"Ha et al. (2016)"}],"fun_headline_variants":["Neuron programming squeezes vessel segmentation into 0.84M parameters","Quadratic neurons boost retinal vessel segmentation on DRIVE","Small net with mixed neuron types tops bigger vessel models","Neuron-level search outperforms architecture-only tuning","Predict neuron types, skip search, for fast vessel segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper treats the F1 differences in Table 8—0.8334 for the mixed network versus 0.8265 and 0.8300 for single-type networks—as evidence that neuron diversity itself carries the gain, but each number comes from a single run, so the load-bearing assumption is that those gaps exceed run-to-run noise.","fun_headline_variants_meta":{"raw":{"variants":["Neuron programming squeezes vessel segmentation into 0.84M parameters","Quadratic neurons boost retinal vessel segmentation on DRIVE","Small net with mixed neuron types tops bigger vessel models","Neuron-level search outperforms architecture-only tuning","Predict neuron types, skip search, for fast vessel segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000373,"raw_usage":{"total_tokens":2003,"prompt_tokens":966,"completion_tokens":1037,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":957}},"tokens_in":582,"tokens_out":1037,"duration_ms":118750,"temperature":1.0,"reasoning_tokens":957,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:56:02.097794+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the three neuron-type configurations from Table 8 on DRIVE across at least 20 random seeds and test whether the mixed network's F1 and AUC are consistently above both single-type networks with a paired test; if not, the neuron-type gene is not the source of the reported gain. A second check: evaluate hypernetwork predictions at held-out channels 8, 18, and 28 and report F1 with variance against the top-10 searched networks at those channels.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the quadratic neuron form used as the alternative neuron type in the search space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ReLinear initialization and expressivity analysis that make quadratic networks trainable."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the theoretical result that heterogeneous networks can approximate certain functions with polynomial neurons while homogeneous ones need exponentially many."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The U-Net backbone and baseline that neuron programming augments."},{"cited_title":"D., Niemeijer, M., Viergever, M","cited_arxiv_id":null,"evidence_quote":"The DRIVE dataset used for the search and primary evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Genetic U-Net, the GA-based NAS baseline that the search strategy extends and compares against."}],"review_version":1}