{"id":"001843fe-94a4-44d7-9300-889540c30d67","arxiv_id":"2607.08406","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Single-parameter Monte Carlo mutation-selection trains deep networks and a simple Transformer on MNIST and Tiny Shakespeare without backpropagation.","lead":"A simple Monte Carlo rule—randomly mutate one network parameter and keep the change only if loss falls—trains deep fully connected nets and a small Transformer without gradients. It may enable hardware-friendly and physics-inspired training while exposing extreme redundancy in deep models.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Full-batch MNIST/Tiny-Shakespeare results with a single transferred hyper-parameter set do not yet establish practicality for deep nets in general.","rationale":"The Reader correctly isolates the weakest link: the leap from a narrow set of full-batch MNIST/Tiny-Shakespeare numbers to the claim that MCA is a practical training paradigm for deep networks. The paper itself supplies no counter-evidence that would neutralize this concern (no mini-batch ablations, no multi-seed statistics, no harder datasets). Consequently the CONDITIONAL verdict remains appropriate; the stress-test does not move it. The concrete mini-batch re-run is a minimal, decisive check that either shores up or falsifies the practicality premise without requiring new theory.","tokens_in":13978,"tokens_out":541,"duration_ms":5865,"concrete_test":"Re-train the 11-layer and 21-layer ReLU nets of Fig. 1(b) with mini-batches of size 256 (standard SGD regime) under the same δ/Δ, reporting mean±std test accuracy over 5 seeds after a wall-clock budget equal to 250 full-batch epochs; if accuracy falls below 95 % or variance exceeds 2 points, the practicality claim for deep nets is materially weakened.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that MCA “can practically train deep networks” rests on the premise that full-batch (or large-batch) training/test accuracy on MNIST and Tiny Shakespeare, obtained with one fixed hyper-parameter triple (δ=0.01, Δ=0.1, He-style init) transferred from a 5-layer net, is sufficient evidence of feasibility for deep networks in general. That premise is least secure in §2.1 and §2.5: (i) every deep fully-connected experiment uses the entire 60 k training set as a single batch and reports only a single hyper-parameter setting; (ii) the Transformer language-model result reaches only 0.47 test accuracy / 5.86 perplexity after 48 h on a 3-layer character-level model; (iii) no scaling curves, multi-seed statistics, or comparison against modern mini-batch BP baselines appear. If the acceptance rate of beneficial mutations collapses under realistic mini-batch noise or on harder data distributions, the “practical alternative” claim does not hold.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes that a minimal Monte Carlo mutation–selection rule (MCA)—randomly perturb one or more parameters and accept the change only if the empirical loss does not increase—can train deep neural networks without backpropagation. On a single GPU, the method is shown to train fully connected nets deeper than 20 layers, single-hidden-layer nets up to width 16 384, and a simple multi-layer Transformer on MNIST classification and Tiny Shakespeare character-level language modeling. Additional experiments demonstrate pure pruning training, discrete-weight support, Gaussian activations, and extreme sparsification. A theoretical section argues that MCA recovers gradient descent in the small-perturbation limit while remaining applicable to non-smooth, discrete, and hard-constrained settings where BP fails.","tokens_in":14245,"tokens_out":1104,"duration_ms":10504,"significance":"If the feasibility claim holds under broader conditions, the work supplies a concrete, implementable gradient-free alternative that sidesteps vanishing/exploding gradients and directly supports discrete weights, hard amplitude constraints, and pure pruning. The pure-pruning and extreme-sparsity results (remaining weights ~0.33 % while retaining >95 % test accuracy) are of independent interest for model compression and for studying redundancy. The Transformer language-modeling demonstration, though modest in scale, is a nontrivial proof-of-principle that the same rule can train attention-based sequence models. The theoretical comparison (local-limit recovery of gradient descent, escape from saddles via finite mutations, hard-constraint generalization bounds) is carefully framed and does not overclaim replacement of BP. These elements make the manuscript a useful contribution to gradient-free and physics-inspired training methods, provided the empirical scope is clarified.","major_comments":[{"comment":"§2.1 and the abstract claim that MCA “can practically train deep networks.” All deep fully-connected experiments use full-batch (60 k) or large-batch (30 k) training with a single hyper-parameter triple (δ=0.01, Δ=0.1, He-style init) transferred from a 5-layer net. No mini-batch noise, multi-seed statistics, or scaling curves on harder data are reported. Because acceptance of beneficial mutations is evaluated on the empirical loss of the current batch, the acceptance rate may collapse under realistic mini-batch variance; without that evidence the “practical alternative” claim remains under-supported for general deep-network practice.","section":null},{"comment":"§2.5 Transformer language-modeling result: after 48 h the 3-layer character-level model reaches test accuracy 0.47 / perplexity 5.86. The generated sample (Table 1) shows recognizable structure but also pseudo-words and local errors. The manuscript presents this as a “proof-of-principle,” yet the abstract and conclusion still list it among demonstrations that MCA trains “even a simple Transformer.” A clearer quantitative baseline (same architecture trained by BP under identical compute) and an explicit statement of the performance gap are needed so that the claim is not overstated.","section":null},{"comment":"§2.1 BP comparison is limited to a single full-batch Adam run on a 5-layer ReLU net (~98.19 % test). No modern mini-batch BP baselines, residual architectures, or learning-rate schedules appear for the deeper nets or for the Gaussian/Transformer cases. While the paper correctly states that extreme accuracy competition is not its focus, the absence of any competitive BP reference for depths >5 and for the Transformer leaves the relative practicality claim difficult to assess.","section":null}],"minor_comments":[{"comment":"§2.1: “10 000 Monte Carlo operations = 1 epoch” is non-standard; a short justification or conversion to wall-clock / FLOPs would help readers compare with BP epochs.","section":null},{"comment":"Figure 2(c) heat-maps are informative but lack a color-bar scale and an explicit statement of how “completely disconnected sites” are thresholded.","section":null},{"comment":"§3.2 Eq. (9): the proportionality constant for E[ΔW_acc] is left unspecified; a one-line derivation or citation would tighten the local-limit argument.","section":null},{"comment":"References [5] and [26] list 2026 publication years; verify that these are not future-dated placeholders.","section":null},{"comment":"CUDA Fortran implementation details (caching strategy, single vs double precision) are mentioned only briefly; a short appendix or code-availability statement would improve reproducibility.","section":null}],"recommendation":"major_revision","confidential_remarks":"The core algorithmic idea is a direct GPU-scale extension of the author’s earlier shallow-net MCA papers (arXiv:physics/0607046, arXiv:1704.06885). Novelty therefore rests on the deep-net / Transformer demonstrations rather than on a new algorithm. The journal should weigh whether the empirical scope (full-batch MNIST + modest Tiny Shakespeare) is sufficient for its standards; a major-revision request that forces clearer baselines and mini-batch evidence seems the proportionate path."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The punchline is simple: the author shows that the dumbest Monte Carlo rule—mutate one weight (or a channel), keep it if loss does not rise—can train fully connected nets past 20 layers, single-hidden-layer nets up to 16k units, and a small multi-layer Transformer on MNIST and Tiny Shakespeare, all without backprop, residuals, or (often) normalization.\n\nWhat is actually new relative to the author’s own earlier CPU/shallow MCA papers is the GPU scaling, pure-pruning training that leaves ~0.3% of weights while still classifying MNIST, the fact that Gaussian activations beat ReLU under this trainer, and the first Transformer runs. Those pieces are real and well documented. The pure-pruning heatmaps that strip background pixels while keeping digit structure are a nice, concrete observation. The short theory section is honest: MCA recovers gradient descent in the small-perturbation limit and stays applicable when gradients do not exist. Numbers are internally consistent (train to 100%, test ~97–98% on MNIST; Shakespeare test perplexity ~5.9 after 48 h).\n\nSoft spots, in proportion: almost everything is full-batch MNIST with one hyper-parameter triple transferred from a 5-layer net; BP baselines are sparse and also full-batch; no multi-seed stats or code; Transformer language modeling is only a proof-of-principle character model. Training is slower than BP, especially on the Transformer. The paper itself frames this as feasibility, not a drop-in replacement, so the stress-test concern about “practical for deep nets in general” is fair but does not sink the narrower claim that the algorithm works on the listed architectures.\n\nThis is for people who care about gradient-free training, discrete/sparse hardware, or neuromorphic/in-memory systems—not for someone looking for a new SOTA optimizer. Math and citations look solid; self-cites are the natural precursor work. I would send it to peer review. Worth a reading-group slot if the group is into alternative training or hardware-friendly nets; otherwise skim the pruning and Gaussian sections.","headline":"A clean feasibility demo: one-parameter Monte Carlo mutation trains deep FC nets, extreme pruning, and a small Transformer without gradients—useful, not yet a practical BP replacement.","tokens_in":14871,"tokens_out":537,"would_cite":false,"duration_ms":12699,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"The simplest Monte Carlo rule—mutate one parameter, keep it only if loss falls—can train deep nets, wide nets, and a small Transformer without any gradients.","keywords":["Monte Carlo method","deep neural networks","gradient-free optimization","unconventional transfer functions","pruning","discrete weights","Transformer"],"falsifier":"Train the same 21-layer fully connected net or the same three-layer Transformer under identical data and compute budgets with a modern optimizer; if MCA cannot reach comparable test accuracy or if its wall-clock time grows prohibitively with depth or sequence length, the practicality claim fails.","tokens_in":14836,"feed_emoji":"🎲","tokens_out":646,"duration_ms":6432,"temperature":0.7,"pith_summary":"Backpropagation trains almost every modern neural network, but it depends on gradients and therefore suffers from vanishing and exploding signals in deep stacks. This paper claims that a far simpler rule is already enough: pick a parameter at random, add a small random kick, and keep the change only if the loss does not rise. On a single GPU the method trains fully connected nets deeper than twenty layers, single-hidden-layer nets with more than sixteen thousand units, and a modest multi-layer Transformer on both MNIST digits and character-level Shakespeare. Because no derivatives are required, the same procedure works with pure pruning, discrete weights, and transfer functions such as the Gaussian that are hostile to gradient descent. The author presents the result as both a practical training alternative and a window onto how networks can self-organize by successive accepted mutations.","feed_headline":"Random mutations alone can train deep nets and a small Transformer","feed_subtitle":"No gradients, no residuals: accept a parameter kick only when loss falls, and deep networks still learn","key_machinery":"Monte Carlo mutation–optimization selection (MCA): randomly select one or more parameters, apply a finite perturbation (or set the weight to zero for pure pruning), and accept the change only when the empirical loss does not increase and the hard amplitude constraint is respected. The same rule is applied layer-wise with local-field caching so that only the affected forward slice is recomputed.","core_discovery":"A single-parameter Monte Carlo mutation–selection rule (randomly perturb a weight or bias, accept if loss does not increase, optionally subject to a hard magnitude bound) is sufficient to train deep fully connected networks, extremely wide single-hidden-layer networks, and a simple Transformer architecture to nontrivial accuracy on MNIST classification and Tiny Shakespeare language modeling, without backpropagation, residual connections, or (in many experiments) layer normalization.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Monte Carlo mutations train deep nets without gradients","Random parameter kicks train 20-layer nets and Transformers","Single-weight mutations train deep nets and a tiny Transformer","Gradient-free Monte Carlo trains wide nets without residuals","Mutate-and-accept rule trains deep nets and simple Transformers"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That full-batch or large-batch accuracy numbers on MNIST and Tiny Shakespeare, obtained with one fixed set of step-size and bound parameters, already show the method is a practical general alternative for deep networks.","fun_headline_variants_meta":{"raw":{"variants":["Monte Carlo mutations train deep nets without gradients","Random parameter kicks train 20-layer nets and Transformers","Single-weight mutations train deep nets and a tiny Transformer","Gradient-free Monte Carlo trains wide nets without residuals","Mutate-and-accept rule trains deep nets and simple Transformers"]},"model":"grok-4.5","effort":"low","cost_usd":0.004498,"raw_usage":{"total_tokens":1327,"prompt_tokens":770,"num_sources_used":0,"completion_tokens":64,"cost_in_usd_ticks":44980000,"prompt_tokens_details":{"text_tokens":770,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":493,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":770,"tokens_out":64,"duration_ms":4606,"temperature":1.0,"reasoning_tokens":493,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T07:56:21.573722+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the same 21-layer fully connected net or the same three-layer Transformer under identical data and compute budgets with a modern optimizer; if MCA cannot reach comparable test accuracy or if its wall-clock time grows prohibitively with depth or sequence length, the practicality claim fails.","supporting_citations":[],"review_version":1}