{"id":"49ea0f98-c8c8-4b8e-bcf4-4f22d26590c2","arxiv_id":"2501.03176","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"SFF trains convolutional networks layer-by-layer with auxiliary class-goodness layers and block-wise local backpropagation, reaching backprop-comparable accuracy on small benchmarks in the reported runs.","lead":"This paper introduces Scalable Forward-Forward (SFF), a training method that updates network layers with local objectives instead of backpropagating through the whole network. Experiments on CIFAR-10/100 and Imagenette report accuracy comparable to backpropagation on modern CNNs, though the paper's own tables show slower training and higher memory use in some settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed BP-comparable accuracy is not yet established: SFF is evaluated by averaging all layer goodness scores while BP is scored by its final head, and BP is run without the layer normalization SFF adds; a controlled layer-ensemble/normalized BP baseline is needed.","rationale":"Reader's weakest assumption is on point. Section 4.4 and Table 2 show that SFF never uses a classifier head; its prediction is the argmax over the sum of per-block goodness vectors, while BP's prediction is the argmax of a single softmax head. These are different decision rules, and there is no reason to expect them to be equivalent. Since ensemble averaging is a known accuracy booster, the correct control is to give BP the same layer-ensemble treatment. Similarly, the architecture differs because SFF adds layer norm after each block; normalization often improves convolutional training, so without a BP+SFF-equivalent-normalization baseline the method comparison is confounded. Neither control is present in the paper. The speed mismatch is a separate, self-admitted inconsistency: Section 5 states SFF is slower than BP in all settings, while the abstract says 'similar training speed.' That should be corrected regardless of the accuracy analysis. But the fair-comparison concern is the most load-bearing because it undermines the central claim that SFF achieves BP-comparable or better accuracy. This is fixable with experiments, so CONDITIONAL remains the right verdict; the claims should be reframed until the controlled baselines are run.","tokens_in":9369,"tokens_out":6779,"duration_ms":65915,"concrete_test":"Retrain the BP baselines with layer normalization inserted at exactly the block boundaries where SFF inserts it, then add a lightweight linear probe (or the same auxiliary 1x1 convolution) at each boundary and average probe predictions as in Section 4.4, exactly matching SFF's layer-ensemble evaluation. Report both (a) BP with layer norm + layer ensemble and (b) BP without layer norm but with layer ensemble. If either matches or exceeds SFF's reported accuracy on Table 2 rows (especially CIFAR-100 CNNB and Imagenette ResNet18), then the SFF gain is not attributable to the local FF objective.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The core empirical claim—SFF is comparable to or better than backpropagation—rests on Table 2, but the comparison is not apples-to-apples. SFF is evaluated with the layer-ensemble procedure of Section 4.4, which averages goodness vectors from every block and selects the max; BP is evaluated only with its final classifier head. A layer-ensemble is a multi-predictor aggregation (effectively a committee of local classifiers), and such ensembles usually beat a single head even when the local training rule is weaker. The reported gaps (e.g., +4.85 on CIFAR-100 CNNB, +1.40 on Imagenette ResNet18) could therefore be largely an artifact of the evaluation protocol. The confound is compounded by Section 3, which inserts layer normalization after every trainable SFF layer/block while the BP baseline 'omits the additional normalization layers.' Thus the comparison changes both the inference procedure and the architecture, so the accuracy advantage cannot be attributed to SFF's update rule. The paper also concedes in Section 5 that SFF is slower than BP in every setting, contradicting the abstract's 'similar training speed,' but the fair-comparison issue is the more decisive threat to the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Scalable Forward-Forward (SFF), a layerwise training method for convolutional networks. SFF replaces the class-channel partitioning of the Channel-wise Competitive Learning (CwC) method with small auxiliary convolutional layers that produce per-class goodness tensors, and trains each layer or block with a margin objective based on the log-sum-exp of class goodness values. For deeper models, gradients are allowed to flow within blocks but not between blocks. The authors evaluate SFF against backpropagation (BP) and a modified CwC baseline on CIFAR-10, CIFAR-100, and Imagenette using CNN, CNNB, ResNet18, and MobileNetV3 architectures, reporting accuracy, peak VRAM, runtime, and parameter counts, plus small-data (1000-sample) and ImageNet-pretraining transfer experiments.","tokens_in":9651,"tokens_out":8596,"duration_ms":72574,"significance":"If the empirical comparison were controlled, SFF would be a useful contribution: the auxiliary-convolution idea is simple and removes the hard channel-group constraints of CwC, the within-block hybrid is a practical compromise for modern architectures, and the paper ships a public implementation and reports memory, time, and parameter counts. The layerwise objective and one-pass goodness computation are interesting. However, the current evidence for the central claims is weakened by comparison-protocol and architectural confounds, and several abstract-level claims are contradicted by the paper's own tables. The contribution is promising but needs revision.","major_comments":[{"comment":"The headline accuracy comparison is not controlled: SFF is evaluated by averaging goodness vectors from all layers (Section 4.4), whereas BP is evaluated only by its final classifier head. A layer ensemble is a multi-classifier aggregation and can outperform a single head even when the local training rule is weaker; the gaps in Table 2 (e.g., +4.85 on CIFAR-100 CNNB, +1.40 on Imagenette ResNet18) could therefore reflect the evaluation protocol rather than SFF. Please add a BP baseline evaluated with an equivalent layer ensemble (e.g., averaged softmax/logits of all layers) and an SFF evaluation using only its final goodness layer, and report both.","section":"§4.4 and Table 2"},{"comment":"The SFF models include layer normalization after every trainable layer or block, while the BP baseline omits the additional normalization layers. This changes both the architecture and the optimization landscape, so any SFF accuracy improvement cannot be attributed solely to the update rule. Please train BP with the same normalization layers (and the same hyperparameter search) to isolate the effect of the training algorithm.","section":"Section 3"},{"comment":"The speed claim is internally inconsistent: the abstract says the hybrid design maintains 'a similar training speed,' but Table 2 reports SFF time ratios from 1.2x to 7.4x slower than BP in every setting, and Section 5 explicitly states 'In all experimental settings, SFF is slower than BP.' Please correct the abstract and conclusions to state the actual slowdown, or provide a controlled speed comparison under matched conditions.","section":"Abstract and Section 5"},{"comment":"The claim of 'significantly lower memory allocation' is contradicted by Table 2 on CIFAR-100: SFF uses 883.78 MB versus 561.85 MB for BP on the CNN and 702.08 MB versus 561.12 MB on CNNB, i.e., 1.25--1.57x more memory. The memory advantage only appears on some architectures (e.g., Imagenette). Please qualify the memory claim and discuss the dependence on block size and auxiliary-layer overhead.","section":"Introduction and Table 2"},{"comment":"The abstract states that the method 'eliminates the need for backpropagation,' but the competitive results on ResNet18 and MobileNetV3 are obtained with a hybrid design that allows backpropagation within each block (Section 4.3, Table 1). This is a legitimate design, but it should be stated prominently in the abstract and introduction; otherwise the central 'no backpropagation' claim is overstated.","section":"Abstract and Section 4.3"},{"comment":"The CwC-mod baseline is not a faithful representation of CwC, and its complete failure on CIFAR-100 (1.00 ± 0.00 for both layer and head accuracy) does not establish that CwC cannot handle large numbers of classes. The paper notes that the original CwC required superclasses for CIFAR-100, but the modified version removes CFSE blocks and ILT. Please validate the modified baseline against the original CwC configuration or report the original numbers, and temper the comparative claim accordingly.","section":"Section 5 and Table 2"}],"minor_comments":[{"comment":"The sentence 'The only tunable parameter in this approach is the kernel size of the auxiliary convolutional layer' contradicts Section 3, which describes hyperparameter searches over two learning rates and weight decay. Please clarify that this refers to architectural hyperparameters, not optimization hyperparameters.","section":"Section 4"},{"comment":"The parameter counts are not directly comparable because SFF includes auxiliary layers but does not include a standard classifier head, whereas BP includes the head. Please state the head parameter count separately or use a matched architecture for the parameter comparison.","section":"Table 2"},{"comment":"Step 3 says to compute 'positive and negative goodness factors,' but the note says the negative factor is not used in the loss for the presented results. This is confusing; please clarify whether the negative factor is used in any variant or remove it from the description.","section":"Section 4.3"},{"comment":"There are typographical artifacts in the text, such as 'CIF AR-10' and 'CIF AR-100' in several places and a Unicode ligature in the GitHub URL. These should be corrected.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's main claims are plausible but the empirical evaluation needs controlled baselines before the accuracy and efficiency claims can be accepted. I would encourage the editor to require the layer-ensemble/normalized BP baseline and a corrected handling of the speed and memory claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper is worth reading, and it deserves a careful referee, but the core empirical claim—comparable to backprop—is not yet supported. The evaluation protocol and the abstract's speed/memory claims both need fixing.\n\nWhat's actually new: SFF replaces CwC's hard channel-to-class assignment with small 1x1 convolutions that produce a per-class goodness tensor, and it uses block-wise local backprop to scale to ResNet18 and MobileNetV3. That's a clean, practical idea, and the public code should make reproduction easy. The transfer-learning result (ImageNet-pretrained weights fine-tuned with FF) is a good addition.\n\nWhat I like: the method runs on modern architectures without global backprop, and the raw numbers on CNNB (81.38 on CIFAR-10, 55.34 on CIFAR-100) are in the right ballpark. The authors also state in Section 5 that SFF is slower than BP in every setting, which makes the abstract's 'similar training speed' puzzling.\n\nThe soft spots, in order of importance:\n\n1. The 'new loss' is not new. The CwC loss in Section 2.3.2 is -log(softmax(g+)) = -g+ + logsumexp(G). The equation in Section 4.2 is -(g+ - logsumexp(G)), which is the same function. The real novelty is the auxiliary convolution, not the loss. That claim should be corrected.\n\n2. The accuracy comparison is confounded, as you noted. SFF is evaluated by averaging goodness vectors from all layers; BP is scored by its final head. That is a committee of classifiers versus a single head. In addition, BP is run without the layer-normalization layers that SFF inserts. Both changes can easily account for a few points. The fix is straightforward: add a BP baseline with the same layer ensemble and the same normalization, and compare like with like.\n\n3. The memory claim is not universal. On CIFAR-100 CNN, SFF peaks at 884 MB versus BP's 562 MB. The 'significantly lower memory' claim only holds on the ImageNet models, and even then it is modest on ResNet18. The paper's own data contradict the abstract.\n\n4. The small-data experiment is a single dataset with three seeds. The 9-point gap on Imagenette is suggestive, but I would want more runs and a properly tuned BP baseline before taking it seriously.\n\nBottom line: the method is plausible and the direction is right, but the paper overstates its case. Send it to referees, and ask for a fair comparison protocol and a corrected abstract. If the accuracy advantage survives a layer-ensemble BP baseline, this will be a solid contribution.","headline":"SFF is a promising FF variant, but its headline accuracy, speed, and memory claims are undermined by an unfair evaluation protocol and an abstract that contradicts its own table.","tokens_in":10154,"tokens_out":6095,"would_cite":false,"duration_ms":53362,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The Scalable Forward-Forward algorithm trains deep convolutional networks without global backpropagation, matching standard backpropagation in accuracy and, in its block-wise hybrid form, sometimes exceeding it while using less memory.","keywords":["forward-forward algorithm","backpropagation-free training","layerwise learning","convolutional neural networks","local goodness","block-wise training","transfer learning","small-data learning"],"falsifier":"Train the backpropagation baseline on CIFAR-100 CNNB and Imagenette ResNet18 using SFF's architecture (including its added layer-normalization layers), evaluate it with the same layer-ensemble goodness averaging used for SFF, and compare against SFF; if backpropagation then matches or exceeds SFF, the claimed superiority is an artifact of the evaluation protocol.","tokens_in":9156,"feed_emoji":"🧠","tokens_out":8822,"duration_ms":74245,"temperature":0.7,"pith_summary":"The paper proposes the Scalable Forward-Forward (SFF) algorithm, a way to train deep convolutional networks without global backpropagation: each layer or block is updated from a local loss computed in a single forward pass, so error gradients never flow across layer boundaries. The author's central claim is that SFF reaches accuracy comparable to standard backpropagation on CIFAR-10, CIFAR-100, and Imagenette with modern architectures, and that a block-wise hybrid—backpropagation inside residual blocks but not between them—tends to outperform the backpropagation baseline while using less memory. The reason this matters is that layer-local training is more modular, more memory-efficient, and compatible with non-differentiable components, which would make SFF a practical drop-in alternative to backpropagation rather than a research curiosity. The paper further reports that SFF benefits from ImageNet pretraining done with backpropagation and outperforms backpropagation when only 1,000 training samples are available.","feed_headline":"Forward-Forward training matches backprop on deep CNNs","feed_subtitle":"A layerwise goodness loss trains ResNet18 and MobileNetV3 without global backpropagation, often using less memory.","key_machinery":"The load-bearing mechanism is a per-block goodness readout. After each trainable layer or block, a small auxiliary convolutional layer (kernel size $1\\times 1$ or slightly larger) maps the activations to a tensor of shape [batch size, number of classes, height, width]; pooling over space gives a per-class goodness vector. The local loss is $L_{\\mathrm{SFF}} = -\\mathbb{E}\\big[g_{\\mathrm{pos}} - \\log \\sum_{j=1}^{J} \\exp(g_j)\\big]$, a log-sum-exp margin that compares the ground-truth goodness to all class goodnesses in a single pass and removes the need for explicit negative samples. For deeper networks, backpropagation is allowed inside each block—such as a residual block—but never between blocks, while layer normalization is applied after each trainable layer to stabilize the local objectives. This combination carries the argument: the auxiliary convolutions replace CwC's channel partitioning, the smoothed margin makes the one-pass loss trainable, and the block-wise restriction preserves FF's memory and modularity benefits.","core_discovery":"The central claim is that the Forward-Forward recipe—separately maximizing a layer's 'goodness' on correct inputs and minimizing it on incorrect ones—can be scaled to large convolutional models without the channel-partitioning apparatus of earlier variants. SFF replaces hard class-to-channel assignments with small auxiliary convolutional layers that read each layer's activations and produce a per-class goodness tensor in one forward pass; the local objective is a log-sum-exp margin that pushes the ground-truth class's goodness above the smoothed aggregate of all classes. On Imagenette with ResNet18, the paper reports 83.24% layer-ensemble accuracy for SFF against 81.84% classifier-head accuracy for backpropagation, and on CIFAR-100's CNNB model 55.34% against 50.49%; on MobileNetV3/Imagenette, SFF trails at 77.06% versus 80.44%, so the overall claim is a mix of small wins, small losses, and one large win. Because SFF has no classifier head of its own, its reported accuracy averages the goodness vectors from all layers, while backpropagation is evaluated through its final head; the paper presents this as the natural head-free evaluation for SFF.","pith_inferences":["The headline comparisons are not apples-to-apples: SFF is scored by layer-ensemble goodness while backpropagation is scored by its classifier head. A natural test the paper does not run is to give the backpropagation baseline the same layer-ensemble readout; if BP also improves, the reported advantages on CNNB and ResNet18 would need to be reinterpreted.","The log-sum-exp margin makes each layer behave like a small softmax classifier over goodness values, so SFF can be viewed as a form of local learning with per-layer losses; this suggests connections to other local-learning methods and raises the question of whether the auxiliary layers' overhead could be reduced by sharing filters or pruning class-specific channels.","The transfer-learning results open the reverse experiment: pretrain with SFF and fine-tune with backpropagation. If the feature hierarchies are truly compatible, this should also work, and it would test whether SFF's benefit is a property of the objective or just of the starting point.","If the low-data advantage persists on medical or remote-sensing benchmarks, layerwise goodness training could become a practical default for small datasets, since it appears to regularize without explicit data augmentation."],"forward_implications":["SFF gives a concrete recipe for training modern convolutional networks without global backpropagation, with measured peak-memory reductions of roughly 280 MB on ResNet18/Imagenette and 530 MB on MobileNetV3/Imagenette.","Because gradients stay inside blocks, SFF can accommodate non-differentiable or black-box components and allows blocks to be trained independently, supporting modular or parallel training.","Weights pretrained with backpropagation can be fine-tuned with SFF, so existing ImageNet-pretrained models remain usable under forward-only training.","The small-data experiments (1,000 training samples) suggest SFF generalizes better than backpropagation in data-scarce settings, with a 9-percentage-point gain on Imagenette/MobileNetV3.","The hybrid block-wise variant is the configuration that matches or beats backpropagation, while pure layerwise FF remains competitive only on the simpler CNN; the paper attributes the efficiency advantage to having many small blocks, as in MobileNetV3."],"supporting_citations":[{"why":"Supplies the ResNet18 architecture used as the deep-model test bed; its residual blocks define the block-wise hybrid training units.","marker":"[1]"},{"why":"Supplies the MobileNetV3 architecture, whose small inverted-residual blocks give SFF its largest efficiency gains.","marker":"[2]"},{"why":"Provides the Channel-wise Competitive Learning baseline and the one-forward-pass positive/negative goodness idea that SFF replaces with auxiliary convolutional layers.","marker":"[3]"},{"why":"Introduces the Forward-Forward algorithm and the goodness concept that SFF scales to modern convolutional architectures.","marker":"[4]"},{"why":"Supplies the CIFAR-10 and CIFAR-100 datasets used for the main small- and medium-scale classification comparisons.","marker":"[11]"},{"why":"Supplies the Imagenette dataset used for the deep ResNet18 and MobileNetV3 comparisons, transfer-learning checks, and small-data experiments.","marker":"[12]"}],"fun_headline_variants":["Forward-forward scales to CNNs, rivals backprop accuracy","Backprop-free Forward-Forward trains modern CNNs","Layerwise goodness loss scales forward-forward to deep nets","Forward-forward without backprop rivals backprop on CNNs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that layer-ensemble goodness accuracy for SFF and classifier-head accuracy for backpropagation are fair equivalents; if backpropagation were evaluated with the same layer-ensemble readout (or with the same added layer normalization), the reported SFF advantages could disappear.","fun_headline_variants_meta":{"raw":{"variants":["Forward-forward scales to CNNs, rivals backprop accuracy","Backprop-free Forward-Forward trains modern CNNs","Layerwise goodness loss scales forward-forward to deep nets","Forward-forward without backprop rivals backprop on CNNs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00073,"raw_usage":{"total_tokens":3261,"prompt_tokens":930,"completion_tokens":2331,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":2266}},"tokens_in":546,"tokens_out":2331,"duration_ms":18364,"temperature":1.0,"reasoning_tokens":2266,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:52:39.331793+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the backpropagation baseline on CIFAR-100 CNNB and Imagenette ResNet18 using SFF's architecture (including its added layer-normalization layers), evaluate it with the same layer-ensemble goodness averaging used for SFF, and compare against SFF; if backpropagation then matches or exceeds SFF, the claimed superiority is an artifact of the evaluation protocol.","supporting_citations":[{"cited_title":"De ep residual learning for image recognition, 2015","cited_arxiv_id":null,"evidence_quote":"Supplies the ResNet18 architecture used as the deep-model test bed; its residual blocks define the block-wise hybrid training units."},{"cited_title":"Searching for mobilenetv3","cited_arxiv_id":null,"evidence_quote":"Supplies the MobileNetV3 architecture, whose small inverted-residual blocks give SFF its largest efficiency gains."},{"cited_title":"Convolutional channel-wise competitive le arning for the forward-forward algorithm","cited_arxiv_id":null,"evidence_quote":"Provides the Channel-wise Competitive Learning baseline and the one-forward-pass positive/negative goodness idea that SFF replaces with auxiliary convolutional layers."},{"cited_title":"Learning multi ple layers of features from tiny images","cited_arxiv_id":null,"evidence_quote":"Supplies the CIFAR-10 and CIFAR-100 datasets used for the main small- and medium-scale classification comparisons."},{"cited_title":"Imagenette: A smaller subset of 10 easil y classified classes from imagenet, March 2019","cited_arxiv_id":null,"evidence_quote":"Supplies the Imagenette dataset used for the deep ResNet18 and MobileNetV3 comparisons, transfer-learning checks, and small-data experiments."}],"review_version":1}