{"id":"5d082ccd-b554-46d2-ae56-3f387430ea9a","arxiv_id":"2506.13410","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"By learning neuron positions and computing weights from spatial distance, the paper builds O(n)-parameter MLPs and spiking networks that are competitive on MNIST and robust to pruning, but not more accurate than standard MLPs.","lead":"This paper proposes neural networks in which each neuron has a learnable position in space and the strength of a connection is the inverse of the distance between two neurons. Instead of storing a separate weight for every connection, the network stores only coordinates, so its memory footprint can grow linearly rather than quadratically with the number of neurons.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own MLP results are inconsistent with the 'competitive performance' claim; the SNN advantage rests on a single 14-neuron baseline, leaving the central empirical claim unsupported.","rationale":"The parameter-count reduction (Eq. 4 versus Eq. 3) is arithmetically correct as a count of stored scalars, so the O(n) claim itself is not in question. The vulnerability is the empirical claim. The paper's own Table 1 and Fig. 5 show spatial MLPs consistently below conventional MLPs with similar parameter counts, which is not consistent with the abstract's 'competitive' wording; the authors concede this in Section 4. The SNN result is a single positive data point against a 14-neuron baseline, and the pruning comparison at 80% compares a model with about 1.6M connections to one with about 11k, so the pruning-robustness claim is confounded by initial connectivity. The reader's conditional verdict is therefore appropriate; my stress-test identifies the same load-bearing assumption and does not move the verdict. A width-matched, parameter-matched low-rank baseline would settle whether the spatial parameterization offers any accuracy benefit, and at least one additional dataset is needed before the central empirical claim can be considered supported.","tokens_in":8776,"tokens_out":6892,"duration_ms":71045,"concrete_test":"Train a conventional MLP with 2,048 hidden neurons and low-rank weight matrices W1 = U V (U in R^{2048 x 4}, V in R^{4 x 784}), giving about 11k parameters, on MNIST under the same 300-epoch/Adam protocol; report unpruned test accuracy and accuracy after 80% magnitude pruning. Also run the analogous low-rank fully connected SNN with 2,048 hidden neurons. If these width- and parameter-matched baselines match or exceed the spatial models' 0.9217/0.9216 and pruned accuracies, the 'competitive' and 'pruning superiority' claims are not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that spatially embedded networks are 'competitive with conventional architectures' is not supported by the paper's own experiments. In Table 1, the 3D MLP (0.9217) is below MLP Baseline 14 (0.9429), a parameter-matched network with only 14 hidden neurons; Fig. 5 shows all spatial MLPs (3D through 32D) below their parameter-matched baselines. The authors acknowledge this in Section 4. The only favorable result is the 3D SNN (0.9216) against SNN Baseline 14 (0.9031), a network with 14 hidden neurons and roughly 11k weights, while the width-matched SNN Baseline 2048 (0.9525) outperforms it. Because the spatial model has 2,048 hidden neurons and about 1.6M connections, comparing it to a 14-neuron baseline conflates parameter count with capacity: the spatial model's pruning 'robustness' at 80% sparsity still leaves roughly 325k weights, whereas the baseline has only about 11k weights total. Thus the empirical half of the central claim rests on a single, confounded comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a neural architecture in which each neuron is assigned a position in Euclidean space and connection weights are computed as the inverse of the Euclidean distance between connected neurons. Instead of learning individual weights, the model optimizes neuron positions (plus a small number of auxiliary parameters such as layer distances and inhibition values). The authors argue that this reduces the parameter complexity from O(n^2) to O(n) and provides a biologically motivated inductive bias. They validate the approach on MNIST for both MLPs and fully connected SNNs, comparing against conventional baselines matched either by neuron count or by parameter count. They also report pruning experiments and an extension in which z-coordinates are learned, as well as embeddings in higher-dimensional spaces. The main claim is that spatially embedded networks are competitive with conventional architectures and maintain accuracy at high pruning rates.","tokens_in":8921,"tokens_out":4345,"duration_ms":45908,"significance":"The core idea is interesting and potentially useful: if a weight matrix can be replaced by a distance-dependent wiring rule, then a compact parameterization with a clear geometric interpretation becomes possible. The experiments are transparent, report multiple random seeds, and use an external benchmark, which is a strength. However, the current evidence does not establish the central claim of competitive performance: the paper's own MLP results consistently show the spatial models underperforming conventional baselines with comparable parameter counts but far fewer neurons, and the pruning advantage rests on comparisons that confound total parameter count with the number of effective connections. The significance of the work therefore depends on whether the claims can be appropriately reframed and whether the pruning robustness can be demonstrated under genuinely matched conditions.","major_comments":[{"comment":"The abstract claims that spatially embedded networks 'achieve a performance competitive with conventional architectures,' but the paper's own experiments contradict this for every MLP configuration tested. In Table 1, the 3D MLP reaches 0.9217 accuracy, well below MLP Baseline 14 with 0.9429 and MLP Baseline 2048 with 0.9745. Table 3 shows the Relaxed 3D MLP at 0.9337 versus MLP Baseline 17 at 0.9507, and Fig. 5 shows every higher-dimensional spatial MLP below its corresponding parameter-matched baseline. Section 4 explicitly concedes that spatially embedded MLPs 'were consistently outperformed by conventional MLPs.' The only MLP result that favors the spatial model is the single 95%-pruning point in Fig. 3a. The central empirical claim should either be substantially weakened or supported by additional evidence beyond this one point.","section":"Abstract and Section 3.1, Table 1, Fig. 5"},{"comment":"The pruning comparison conflates parameter count with capacity and with the number of effective connections. The 3D MLP has 2,048 hidden neurons and 1,626,112 weights, while MLP Baseline 14 has 14 hidden neurons and 11,116 weights. When 80% of the 3D MLP's weights are pruned, roughly 325,000 weights remain, which is about 30 times the baseline's entire network. The abstract's phrase 'outperforming traditional networks with the same number of parameters under similar conditions' is therefore not an accurate description of what Fig. 3 shows. The width-matched control, MLP Baseline 2048, outperforms the 3D MLP at most pruning levels. A fair assessment of pruning robustness would need to compare networks with matched numbers of remaining connections, or otherwise control for the large difference in network width.","section":"Section 3.2, Fig. 3 and Table 1"},{"comment":"Equation (1) defines the synaptic weight as the inverse of the Euclidean distance between connected neurons, but this quantity is undefined when the distance is zero. The paper does not specify how neuron positions are initialized, or what constraint, regularization, or projection step prevents two connected neurons from occupying the same position during optimization. Since positions are updated by gradient descent, a collision is not only possible but also not addressed by any stated mechanism. The model definition is therefore incomplete without a minimum-distance safeguard or an explicit justification for why collisions cannot occur.","section":"Section 2, Eq. (1)"}],"minor_comments":[{"comment":"The abstract says spatially embedded models 'outperform traditional networks with the same number of parameters under similar conditions,' but Section 4 states the opposite for MLPs. This inconsistency should be resolved by aligning the abstract with the actual results.","section":"Abstract and Section 4"},{"comment":"The parameter count in Eq. (4) includes positions for the input layer, but the text does not explain why input neurons need learnable positions. Clarifying the role of input positions would make the parameter-counting convention easier to interpret.","section":"Section 2, Eqs. (3) and (4)"},{"comment":"The learning rates differ between the spatial models (0.005) and the baselines (0.001). Since no learning-rate sweep is reported, it is unclear whether the performance gap could be reduced by more careful hyperparameter tuning of the spatial models.","section":"Section 3.1, Tables 1 and 2"},{"comment":"The paper describes pruning the longest connections as mimicking the brain's tendency to minimize wiring length, but removing long connections is not the same as actively minimizing total wiring length. The biological analogy would be more accurate if framed as a heuristic consistent with wiring minimization.","section":"Section 3.2"},{"comment":"The conclusion mentions 'Recurrent Neural Networks' as a possible extension in the introduction, but the experiments only cover MLPs and SNNs. The paper would benefit from stating this scope limitation explicitly in the relevant sections.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest about its limitations in Section 4, which partially mitigates the overclaiming in the abstract. The main risk is the pruning comparison in Section 3.2, which currently supports the paper's headline claim only under a confounded experimental setup. I believe the core idea is worth further development, and the manuscript could become publishable if the claims are reframed to match the evidence and the pruning experiments are redesigned to control for the number of effective connections. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere is my read. The actual new thing is the wiring rule as a weight parameterization: neurons embedded in Euclidean space, weights set to the inverse distance, and training optimized positions plus an inhibitory/excitatory sign per neuron. That is a real extension of the earlier spatial-embedding line from Achterberg, Stiso, Jacobs, and others, where positions were fixed and mainly regularized. The presentation is clean, and the O(n) parameter count for fixed embedding dimension is arithmetically correct, with the caveat that the number of connections and the inference cost stay O(n^2).\n\nCredit where it is due: the experiments are transparent, the numbers in Tables 1 and 2 and the figures are internally consistent, and the Discussion explicitly admits that the spatial MLPs underperform conventional MLPs. The SNN result against the 14-neuron baseline at least shows one positive data point. The z-coordinate relaxation and the higher-dimensional embedding ablations are sensible probes of the idea.\n\nThe soft spots are real, and they are mostly in the abstract's wording. 'Competitive with conventional architectures' is not supported by the paper's own tables: the 3D MLP gets 0.9217 versus 0.9429 for the parameter-matched 14-neuron MLP, and Figure 5 shows every spatial MLP below its parameter-matched baseline. The baseline itself is the issue: it matches total parameter count by shrinking the hidden layer to 14 neurons, so the comparison is parameter-matched but capacity-mismatched. The spatial model has 2,048 hidden neurons and about 1.6 million weights; the 14-neuron baseline has 14 hidden neurons and about 11,000 weights. At 80% pruning the spatial model still has roughly 325,000 nonzero weights, so 'same conditions' is misleading. The one truly interesting result is the 95% post-training pruning point, where the 3D MLP beats the width-matched 2,048-neuron baseline; that should have been the headline, not the weak parameter-count comparison.\n\nThere are a few smaller omissions: no code, no initialization details for the x,y coordinates, and only MNIST with one seed set. The citation pattern is fine; the biological motivation is heavily cited and the prior spatial-embedding work is acknowledged.\n\nBottom line: this is a paper a serious editor should send to peer review. The idea deserves scrutiny and the experiments can be fixed with matched-capacity baselines, width-matched comparisons, and at least one more dataset. I would take it to a reading group if your group works on parameter-efficient architectures, but I would not yet cite it as evidence for the competitive-performance claim.","headline":"A genuinely new weight-parameterization idea with honest reporting, but the paper's central 'competitive performance' claim is undermined by a parameter-matched baseline that matches counts while badly mismatching capacity.","tokens_in":9555,"tokens_out":2607,"would_cite":false,"duration_ms":29278,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training networks by moving neuron positions instead of learning weights cuts parameters from $O(n^2)$ to $O(n)$ and keeps MNIST accuracy past 80% pruning","keywords":["neuron position optimization","spatial embeddings","distance-dependent weights","parameter efficiency","spiking neural networks","network pruning","MNIST"],"falsifier":"Re-run the pruning experiments with a baseline that has the same 2,048 hidden neurons as the 3D MLP but whose per-connection weights are free and whose total trainable parameter count is forced to about 10,500, for instance by weight sharing across connections; if this width-matched free-weight baseline matches or outperforms the 3D MLP at every pruning level, the inverse-distance wiring rule is not what produces the observed pruning robustness.","tokens_in":8455,"feed_emoji":"🧠","tokens_out":14974,"duration_ms":134649,"temperature":0.7,"pith_summary":"The paper proposes training neural networks by optimizing neuron positions in Euclidean space rather than learning connection weights directly. Each synaptic weight is defined by the inverse-distance rule $w_{ij}=1/\\|p_i-p_j\\|_2$, so a full weight matrix is replaced by a few coordinates per neuron, cutting the parameter count from $O(n^2)$ to $O(n)$. The authors test this on MNIST with multi-layer perceptrons and spiking neural networks: the spiking version beats a conventional spiking baseline with the same parameter count, and both variants keep accuracy when more than 80% of connections are pruned. The MLP versions remain less accurate than conventional MLPs even at matched parameter counts, a gap the paper attributes to the coupling that distance-derived weights impose.","feed_headline":"Position-trained networks hold accuracy past 80% pruning","feed_subtitle":"Optimizing neuron coordinates instead of weight matrices cuts parameters from quadratic to linear on MNIST.","key_machinery":"The machinery is the wiring rule of Eq. (1), $w_{ij}=1/\\|p_i-p_j\\|_2$, combined with a fixed per-layer z-coordinate and a learnable continuous inhibition mask that assigns each neuron an excitatory or inhibitory sign. Because every weight in a fully connected layer is a function of only the two endpoint positions, backpropagation moves points instead of scalars; a layer with $n_l$ neurons is stored with $O(n_l)$ position parameters rather than $O(n_l n_{l+1})$ weights. A single step of backpropagation therefore updates all connections incident to a neuron at once, which is the parameter saving and also the main source of weight interdependence.","core_discovery":"The central discovery is that a distance-geometry wiring rule can carry most of the learning burden: instead of storing and updating an $O(n^2)$ weight matrix, the network stores $d-1$ coordinates per neuron plus a bias and an inhibition sign, computes weights as $w_{ij}=1/\\|p_i-p_j\\|_2$, and learns by moving points. On MNIST, spatially embedded MLPs and SNNs reach competitive accuracy, and both retain accuracy under magnitude-based pruning above 80% sparsity; at 95% post-training pruning the 3D MLP even beats a wide baseline with far more parameters. The same experiments show that relaxing the layer structure and raising embedding dimensionality improve accuracy, and that dense MLP accuracy stays below conventional MLPs, which the authors explain by the restricted set of weight configurations reachable through positions.","pith_inferences":["An editorial extension: the inverse-distance rule is one instance of a broader prior, monotone weight decay with spatial separation; testing other monotone parameterizations such as Gaussian or exponential kernels of distance would show whether the pruning robustness comes from the geometric prior or from the specific $1/d$ form.","An editorial extension: the overlap of learned z-distributions across layers suggests the rigid layer code is not essential, so a fully unconstrained position embedding plus a routing mechanism might improve accuracy further, at the cost of feedforward ordering.","An editorial extension: because each position update changes many weights, the effective number of independent degrees of freedom in a spatially embedded layer is far below its raw parameter count; measuring the rank of the Jacobian of the weight matrix with respect to positions would quantify that gap directly.","An editorial extension: the same position-to-weight map transfers naturally to graph neural networks, where node coordinates could be learned jointly with edge weights and the inverse-distance rule would supply an inductive bias aligned with spatially structured graphs."],"forward_implications":["A fully connected network can be deployed with weights reconstructed from stored positions, so connection memory falls from $O(n^2)$ to $O(n)$, a direct benefit for edge-device and real-time inference.","Magnitude-based pruning has a simple geometric reading: removing the smallest weights is the same as cutting the longest physical connections, so pruning can be specified in space rather than in weight space.","The framework composes with other compression techniques, such as quantization and pruning, and can be dropped into existing MLP and SNN architectures without changing the forward-pass structure.","Spiking versions outperform their parameter-matched conventional baseline, indicating that temporal spike timing can compensate for reduced weight flexibility; the same may hold for other temporally coded or recurrent models.","Allowing richer geometry such as learned z-coordinates and higher-dimensional embedding spaces monotonically improves MNIST accuracy, so the geometry itself is a tunable resource for trading parameters against expressiveness."],"supporting_citations":[{"why":"Supplies the MNIST dataset and the gradient-based learning paradigm used as the experimental benchmark.","marker":"[13]"},{"why":"Provides the optimization algorithm used to train all models.","marker":"[12]"},{"why":"Provides the leaky integrate-and-fire neuron model used in the spiking experiments.","marker":"[10]"},{"why":"Establishes that spatial embedding constrains neuronal network architectures, motivating the biologically inspired approach.","marker":"[16]"},{"why":"Introduces spatially embedded recurrent networks with fixed positions, which this paper extends by making positions learnable.","marker":"[1]"},{"why":"Grounds the wiring-length minimization principle used to justify pruning the longest connections.","marker":"[9]"},{"why":"Supports the core idea that optimizing wiring rules instead of individual weights can sustain learning.","marker":"[2]"},{"why":"Supplies the genomic bottleneck argument for why the brain encodes general wiring rules rather than detailed connections.","marker":"[15]"}],"fun_headline_variants":["Neural nets trained by moving neuron positions","Spatial wiring: fewer parameters, 80%+ pruning tolerance","Position optimization beats 80% pruning on MNIST","Distance-based weights cut parameters, not accuracy","Move neurons, not weights: train to 80% sparsity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that matching a conventional network only by total parameter count is a fair comparison: the 3D MLP has 2,048 hidden neurons and about 1.63 million effective connections but only 10,576 trainable parameters, while the parameter-matched MLP Baseline 14 has 14 hidden neurons and about 11,116 weights; if this premise is wrong, the claims of competitive performance and pruning superiority are not established.","fun_headline_variants_meta":{"raw":{"variants":["Neural nets trained by moving neuron positions","Spatial wiring: fewer parameters, 80%+ pruning tolerance","Position optimization beats 80% pruning on MNIST","Distance-based weights cut parameters, not accuracy","Move neurons, not weights: train to 80% sparsity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1334,"prompt_tokens":888,"completion_tokens":446,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":367}},"tokens_in":504,"tokens_out":446,"duration_ms":4896,"temperature":1.0,"reasoning_tokens":367,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:02:09.073447+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the pruning experiments with a baseline that has the same 2,048 hidden neurons as the 3D MLP but whose per-connection weights are free and whose total trainable parameter count is forced to about 10,500, for instance by weight sharing across connections; if this width-matched free-weight baseline matches or outperforms the 3D MLP at every pruning level, the inverse-distance wiring rule is not what produces the observed pruning robustness.","supporting_citations":[{"cited_title":"Trends in Cognitive Sciences22(12), 1127–1142 (2018)","cited_arxiv_id":null,"evidence_quote":"Establishes that spatial embedding constrains neuronal network architectures, motivating the biologically inspired approach."},{"cited_title":"The Journal of Neuroscience 14(4) (1994)","cited_arxiv_id":null,"evidence_quote":"Grounds the wiring-length minimization principle used to justify pruning the longest connections."},{"cited_title":"Nature Communications14(1) (2023)","cited_arxiv_id":null,"evidence_quote":"Supports the core idea that optimizing wiring rules instead of individual weights can sustain learning."}],"review_version":2}