{"id":"d57d874b-5708-40de-b459-ec097e9369e0","arxiv_id":"2504.21662","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Combining convolutional channel grouping, channel-wise loss, chunked local updates, and last-layer inference reduces CIFAR10 test error for Forward-Forward networks, with lightweight models reaching about 19-24% error.","lead":"Researchers combined several existing improvements to the Forward-Forward neural network training method and report lower error on the CIFAR10 image benchmark, including a compact model with only 164,706 parameters. The work is relevant to running neural networks on low-power hardware because Forward-Forward training avoids storing activations for backpropagation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 20% improvement is not attributable: Table I's original CIFAR10 CNN uses 2 conv layers (32, 64 filters), while Table II's FF deep uses 6 conv layers (128 to 512 filters), batch norm, maxpool, and 300 epochs, conflating algorithm changes with scale.","rationale":"The reader's verdict is already CONDITIONAL, and this stress test reaches the same conclusion through a sharper route. The weakest point is not merely missing hyperparameters, but structural non-comparability: Table I's original CNN has two convolutional layers (32 and 64 filters) plus a 256x100 linear layer, whereas Table II's FF deep has six convolutional layers (128 to 512 filters), batch normalization, maxpool, and 300 epochs of training. Any comparison between these two configurations conflates capacity, normalization, and training budget with the algorithm changes. The paper's undocumented 27% attribution for channel grouping alone further confirms that no controlled decomposition of the improvement exists. The proposed concrete test isolates exactly the missing control: keep the improved architecture and training budget fixed, swap the improved FF components back to the original ones, and compare test error. This is fair to the authors because it grants the improved method full credit if the original-FF control fails to reach the improved accuracy. Given the reader already conditionalized on the release of code, ablations, and repeated runs, no verdict change is needed.","tokens_in":7630,"tokens_out":3791,"duration_ms":39922,"concrete_test":"Run one controlled CIFAR10 experiment: fix the FF deep architecture, data pipeline, 300-epoch multi-step schedule, and batch size; train with the original FF procedure (Eq. 1 loss, one-hot overlay for positive and negative samples, per-layer local updates, one-pass inference from the second-to-last layer), then report the test error. Compare against the 18.8% in Table II. If this original-FF control lands near 18.8%, the reported improvement is not due to the proposed techniques; if it stays near 40%, the techniques are the active ingredient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III reports the improved algorithm as four changes: convolutional group channels, channel-wise loss (Eq. 3), chunked local updates, and last-two-layer inference. The headline approximate 20% decrease in test error percentage (Conclusion) compares Table I's CIFAR10 CNN one-pass test error of 44.3% with Table II's FF deep one-pass test error of 18.8%. But the baselines are not controls: the Table I CNN is described as two convolutional layers of 32 and 64 filters plus a linear layer of 256x100, while FF deep is described as six convolutional layers of 128, 264, ..., 512 filters, each preceded by batch normalization and followed by ReLU, with three maxpool applications, trained for 300 epochs with a multi-step learning rate schedule. The 25.5 percentage-point gap therefore conflates parameter count (roughly an order of magnitude), depth, normalization, and training budget. The paper provides no ablation that isolates the FF-specific mechanisms, and its standalone attribution that convolutional channel grouping alone allows 27% test error (Section III) has no corresponding table or supporting run. Until a same-architecture baseline using the original goodness loss (Eq. 1), standard label overlay, per-layer local updates, and the same epoch and schedule budget is measured, the central claim that the proposed FF advancements cause the improvement remains unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reviews variants of the Forward-Forward algorithm and proposes an improved version that combines convolutional channel grouping, a channel-wise loss (Eq. 3), chunked local updates, and inference based on the last two layers. It reports MNIST and CIFAR10 results for the original and improved algorithms in Tables I and II, a family of lightweight CIFAR10 models in Tables III and IV, and a brief quantization experiment. The headline claim is an approximate 20% decrease in test error percentage relative to the original algorithm, with a one-pass CIFAR10 test error of 18.8% for the improved 'FF deep' model.","tokens_in":7952,"tokens_out":3906,"duration_ms":38225,"significance":"If the comparison were controlled, the result would be a useful demonstration that layer-wise and local-update training can scale to CIFAR10 with compact models, which is relevant for low-power hardware. The paper's useful contributions include integrating known FF techniques into one configuration, reporting parameter counts for lightweight FF models, and showing that quantization-aware training with 2-bit weights reaches 22.3% test error. However, the central numerical claim is currently not supported by a controlled experiment, and the lack of ablations, seeds, and code limits the value of the reported numbers until corrected.","major_comments":[{"comment":"The central claim of an approximate 20% improvement is not supported by the evidence because the 'original' and 'improved' models differ in architecture, depth, normalization, and training budget simultaneously. The original CIFAR10 CNN is described as two convolutional layers with 32 and 64 filters plus a linear layer, while FF deep uses six convolutional layers with 128 to 512 filters, batch norm, ReLU, maxpool, and is trained for 300 epochs with a multi-step learning-rate schedule. A same-architecture control trained with the original goodness loss in Eq. (1), standard label overlay, per-layer local updates, and the same epoch and schedule budget is required before the improvement can be attributed to the proposed FF mechanisms; without it, the 25.5-percentage-point gap in test error between Table I and Table II conflates algorithm changes with scale and training budget.","section":"Section III, Tables I and II"},{"comment":"The statement that convolutional channel grouping 'alone is responsible for allowing the model to obtain 27% in test errors' is not backed by any table, experimental setup, or run details. Since channel grouping is one of the four claimed improvements, a dedicated ablation, or at least an explicitly documented run with only channel grouping enabled, is needed to validate both this attribution and the combined claim.","section":"Section III, paragraph after Table II"},{"comment":"Model selection appears to be performed on the test set: the authors state that they used a 'tested minimum number of 160 filters at the final convolutional layer to generate test errors under 20% for all runs.' No train/validation/test split is described, no number of seeds is given, and no error bars are reported for Tables I, II, or IV. This makes it impossible to assess variance or overfitting; the authors should state the validation protocol, report mean and standard deviation over multiple seeds, and restrict any hyperparameter choice to a validation set rather than the test set.","section":"Section III, FF optimal paragraph"}],"minor_comments":[{"comment":"The affiliation text 'Institut for AI-Safety and Security' should read 'Institute for AI-Safety and Security.'","section":"Section I, author affiliations"},{"comment":"The description of the one-hot label overlay on the first pixels would benefit from specifying exactly how many pixels are replaced for MNIST and how the overlay is adapted for CIFAR10, since the label vector length differs between the datasets.","section":"Section II.A"},{"comment":"The text states that chunked local updates 'outperform the original individual layer-wise training' but does not provide a quantitative comparison or a direct reference for that specific claim; adding a citation or a small supporting experiment would improve precision.","section":"Section II.B.c"},{"comment":"The notation underlining numbers in the kernel-size row to indicate maxpool locations is easy to miss; a separate column or an explicit marker such as 'MP' would improve readability.","section":"Section III, Table III"},{"comment":"The quantized model is reported to reach 22.3% test error at 2-bit widths, but no architecture, training setup, or comparison with the full-precision model is given; adding these details would strengthen this exploratory result.","section":"Section III, quantization paragraph"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads more like a technical report than a full journal article, and the central comparison must be re-run as a controlled study before the claimed 20% improvement can be accepted. Releasing code and seeds would materially improve reproducibility. The topic is within scope for a machine-learning journal if the experimental rigor is increased."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is an incremental FF engineering report, not a new algorithm. The four techniques it bundles — convolutional channel grouping, channel-wise loss, chunked local updates, and last-two-layer inference — are all cited from prior work. What is genuinely new is the channel-width sweep that yields light CIFAR-10 models (164k–754k parameters, ~19–24% test error). Those tables are the useful part; the rest is a competent survey plus a flawed comparison.\n\nWhat the paper does well: the survey of FF variants in Section II is organized and accurate enough to orient a newcomer. The stack of improvements is clearly described, and the light-model family fills a practical gap for low-power hardware. The parameter/error numbers are concrete and the citation pattern is honest — the authors point to the sources of each component, which I appreciate.\n\nThe soft spots are real and load-bearing. The headline “approximate 20% decrease in test error percentage” compares Table I’s CIFAR-10 CNN (two conv layers, 32 and 64 filters, plus a linear layer) against Table II’s “FF deep” (six conv layers from 128 to 512 filters, batch norm, maxpool, 300 epochs, multi-step LR schedule). That is not an apples-to-apples control. The gap conflates depth, parameter scale, normalization, and training budget with the FF-specific changes. A same-scale baseline using the original goodness loss, standard label overlay, per-layer local updates, and the same schedule and budget would be needed to support the attribution.\n\nAlso, the claim that convolutional channel grouping alone gives 27% test error appears in Section III with no table, no run details, and no error bar. It should either be removed or fully documented. There is no code, no seeds, no repeated runs, and the “tested minimum number of 160 filters” selection is test-set-based model selection, so the reported errors may be optimistic. The 300-epoch deep model and the 50–70 epoch light models are not compared on a common budget; that is fine for a hardware-focused study, but it weakens any algorithmic comparison.\n\nWho this is for: FF practitioners, especially people who need low-parameter CIFAR-10 models or want a compact summary of current FF tricks. It deserves a serious referee because the engineering value is real and the light-model data could be useful, but the central claim needs a controlled baseline, code, and ablations before it can be taken as established. I would not cite the 20% improvement as an established result; I might use the light-model tables as a starting point for parameter-efficient FF experiments.","headline":"A useful engineering report on lightweight Forward-Forward models, but the headline 20% improvement is not established because the comparison baseline is confounded by architecture, scale, and training budget.","tokens_in":8478,"tokens_out":1659,"would_cite":false,"duration_ms":18751,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"An improved no-backprop training rule reaches 18.8% CIFAR-10 test error, a reported 20% improvement over the original algorithm.","keywords":["Forward-Forward algorithm","backpropagation-free learning","convolutional channel grouping","channel-wise loss","chunked local updates","CIFAR-10","lightweight neural networks","low-power hardware"],"falsifier":"Retrain the original Forward-Forward CNN on CIFAR-10 under exactly the same conditions as the improved model, including the same epoch count, learning-rate milestones, batch normalization, data preprocessing, and random seeds, then compare one-pass test error. If the original reaches 18.8 percent rather than 44.3 percent, the claimed improvement evaporates; if the improved model cannot reproduce 18.8 percent under those conditions, the reported result fails its own internal comparison.","tokens_in":7426,"feed_emoji":"🧠","tokens_out":10091,"duration_ms":95485,"temperature":0.7,"pith_summary":"The paper tries to show that the Forward-Forward algorithm, a training rule that updates layers from local goodness signals instead of backpropagated error, can be advanced to handle CIFAR-10 without losing its low-memory, flexible training. By combining convolutional channel grouping, a channel-wise loss, chunked local updates, and inference from the last two layers, the authors report roughly a 20 percent decrease in test error percentage over the original algorithm, with the best model at 18.8 percent one-pass test error. The same setup also produces a family of lightweight models with 164,706 to 754,386 parameters and test errors between 19.3 and 24.4 percent, which would make the approach relevant for low-capacity hardware. A sympathetic reader would take the central message to be that backpropagation-free local learning can be made competitive on a harder image benchmark while retaining its structural advantages.","feed_headline":"No-backprop network hits 18.8% CIFAR-10 error","feed_subtitle":"Four local-learning tweaks cut test error and shrink models to 165k parameters for low-power hardware.","key_machinery":"The load-bearing object is the convolutional group-channel construction. A convolutional layer's output $Y^l$ is subdivided into $J$ groups $\\hat{Y}^l_j$, one per class, and each group's mean squared activation $G^l_{n,j}$ serves as that class's goodness. The channel-wise loss $C^l = -\\frac{1}{N}\\sum_n \\log\\left(\\frac{\\exp((g^l_{pos})_n)}{\\sum_j \\exp(G^l_{n,j})}\\right)$ then turns goodness into a class posterior and drives competitive learning across the channel dimension. Chunked local updates carry this signal through blocks of layers, and last-two-layer inference uses the resulting hierarchical features. This mechanism is what lets each layer act as an independent classifier while preserving a forward flow of learned features.","core_discovery":"The central claim is that four modifications to the Forward-Forward algorithm work together to make it competitive on CIFAR-10. First, convolutional output channels are split into groups, one group per class, so each layer can produce a class-specific goodness score rather than a single scalar. Second, a channel-wise loss (equation 3) computes a softmax over those class goodnesses and drives the correct group to dominate, making each layer an independent classifier. Third, training proceeds by chunked local updates: blocks of consecutive layers are updated from local losses, and the blocks communicate learned features forward instead of being trained in isolation. Fourth, inference uses the concatenated activity vectors of only the last two layers. The paper reports a one-pass CIFAR-10 test error of 18.8 percent for the resulting FF deep model and a multi-pass error of 18.2 percent, compared with original CNN baselines of 44.3 and 48.3 percent, and describes the overall improvement as an approximate 20 percent decrease in test error percentage. Scaling down, the same techniques yield models with 164,706 to 754,386 parameters and test errors of 19.3 to 24.4 percent.","pith_inferences":["The channel-group design points to a class-incremental extension: new classes could be added as new channel groups while freezing old groups, a test the paper does not run.","Because no activations are stored for a backward pass, chunked local updates imply training memory scales with the largest block rather than depth, so the same approach could train much deeper networks on fixed hardware; the paper does not state this corollary.","A direct lesion experiment, removing or ablating the channel group associated with one class and measuring that class's accuracy, would test whether the reported gains come from the group-channel mechanism specifically or from the other combined techniques.","The parameter-error frontier reported for the light models suggests a practical design rule: around 160 filters in the final layer and four learning-rate milestones keep test error under 20 percent, a recipe that could transfer to other datasets."],"forward_implications":["If the reported results hold, Forward-Forward training can reach single-pass CIFAR-10 accuracy around 18.8 percent without ever computing a backpropagation error signal.","The channel-group construction means each class is associated with a dedicated set of filters per layer, which suggests a route to class-specific feature visualisation and to adding new classes by adding new channel groups.","Chunked local updates allow 50 to 70 epochs to converge and keep training memory bounded by the largest block, which is the property that makes low-capacity hardware implementation plausible.","The lightweight model family gives an explicit accuracy-parameter trade-off, from 24.4 percent error at 164,706 parameters to 18.8 percent at 4.1 million parameters, and a 2-bit quantized variant reports 22.3 percent error."],"supporting_citations":[{"why":"Defines the original Forward-Forward goodness objective and the one-pass and multi-pass inference procedures that serve as the baseline.","marker":"[1]"},{"why":"Supplies the deeper convolutional topology and the observation that initial layers can hurt inference, motivating the last-two-layer inference choice.","marker":"[4]"},{"why":"Introduces convolutional channel grouping and the channel-wise loss function that are the core of the improved algorithm.","marker":"[7]"},{"why":"Provides loss-function variants and random feedback block-training ideas that inform the improved training routine.","marker":"[2]"},{"why":"Supports the local-update training architectures, including chunked local updates used during training.","marker":"[8]"},{"why":"Supplies the CIFAR-10 dataset on which all reported error percentages are measured.","marker":"[13]"},{"why":"Supplies the MNIST dataset used for the original-algorithm baseline results.","marker":"[3]"}],"fun_headline_variants":["Forward-Forward reaches 18.8% CIFAR-10 error via local learning","No-backprop network hits 18.8% CIFAR-10 error with 'chunky' updates","CIFAR-10: Forward-Forward drops to 18.8% error using four tweaks","Improved Forward-Forward achieves 18.8% CIFAR-10 error without backprop"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Table I original-algorithm runs are a fairly tuned, apples-to-apples control; if those baselines lacked the training budget, normalization, learning-rate schedule, or data preprocessing that the improved runs received, the reported 20 percent decrease would not be attributable to the algorithm changes.","fun_headline_variants_meta":{"raw":{"variants":["Forward-Forward reaches 18.8% CIFAR-10 error via local learning","No-backprop network hits 18.8% CIFAR-10 error with 'chunky' updates","CIFAR-10: Forward-Forward drops to 18.8% error using four tweaks","Improved Forward-Forward achieves 18.8% CIFAR-10 error without backprop"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000441,"raw_usage":{"total_tokens":2229,"prompt_tokens":933,"completion_tokens":1296,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":1193}},"tokens_in":549,"tokens_out":1296,"duration_ms":11792,"temperature":1.0,"reasoning_tokens":1193,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:56:56.880217+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the original Forward-Forward CNN on CIFAR-10 under exactly the same conditions as the improved model, including the same epoch count, learning-rate milestones, batch normalization, data preprocessing, and random seeds, then compare one-pass test error. If the original reaches 18.8 percent rather than 44.3 percent, the claimed improvement evaporates; if the improved model cannot reproduce 18.8 percent under those conditions, the reported result fails its own internal comparison.","supporting_citations":[{"cited_title":"Convolutional channel-wise competitive learning for the forward-forward algorithm","cited_arxiv_id":null,"evidence_quote":"Introduces convolutional channel grouping and the channel-wise loss function that are the core of the improved algorithm."}],"review_version":1}