{"id":"9b87fdf2-1b9d-408d-9243-77cb98a921bf","arxiv_id":"1909.01311","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DRTP trains hidden layers using only fixed random projections of the one-hot label, removing feedback pathways and update locking at a modest accuracy cost versus backpropagation.","lead":"DRTP is a training rule that replaces error backpropagation with fixed random projections of the one-hot class label, so each layer can be updated locally and immediately during the forward pass. The paper shows this trades some accuracy for large reductions in memory and compute, which helps low-power edge and neuromorphic chips.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 90-degree alignment proof is not valid as printed because it labels an orthogonal projection as the identity; the conclusion is recoverable, but the proof's assumptions also do not cover the tanh/minibatch networks actually evaluated.","rationale":"The reader's conditional verdict is reasonable. The proof gap they identified is real: the paper explicitly writes a product times its pseudo-inverse as the identity, which is false for the rank-one matrices that arise in the lemma. However, the gap is not fatal to the theorem: using the rank-one structure, strict positivity of the projected quadratic form follows from s_WK^T e > 0, a fact the proof already establishes. So the concern is primarily that the proof is presented incorrectly and that the theoretical guarantee is proven only for a highly idealized setting. The empirical results in Tables 1 and 2, with error bars and open-source code, provide independent support for the practical claim that DRTP trains hidden layers on MNIST and CIFAR-10 without a feedback pathway. The remaining risk is the transfer from the proof's assumptions (linear hidden layers, zero initialization, single example) to the evaluated settings (tanh, He initialization, minibatches), which is supported only by angle measurements. Because the proof is repairable and the experimental evidence is substantial, I would not move the verdict from conditional; the main recommendation is to correct the projection/identity conflation and either extend the proof or explicitly temper the 'always' claim in the experimental regime.","tokens_in":25091,"tokens_out":19262,"duration_ms":165919,"concrete_test":"Re-derive the Alignment step of Supplementary Note 1 using the lemma's explicit rank-one form: for k in [1,K-1], P = ∏_{i=k+1}^K W_i^t = -γ^t s_WK^t (B_k^T y*)^T, so P P^+ is the orthogonal projector onto span{s_WK^t}. Verify that e^T P P^+ e = (s_WK^{tT} e)^2 / ||s_WK^t||^2, which is strictly positive because the theorem proof establishes s_WK^{tT} e > 0. If this verification succeeds, the proof is repairable and the remaining risk is only the transfer to tanh, He-initialized, minibatch networks; if it fails, the theoretical alignment guarantee collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central theoretical claim is that DRTP modulatory signals are always within 90 degrees of BP modulatory signals. As printed, Supplementary Note 1 does not establish this: in the final Alignment step, the product of forward matrices times its Moore-Penrose pseudo-inverse is labeled I, but it is an orthogonal projection, not the identity. Under the lemma's zero-initialized, single-example construction, the product P = ∏_{i=k+1}^K W_i^t is rank-one with range span{s_WK^t}, so P P^+ projects onto that one-dimensional subspace. The strict positivity e^T P P^+ e > 0 does follow from the earlier proven fact s_WK^{tT} e > 0, since e^T P P^+ e = (s_WK^{tT} e)^2 / ||s_WK^t||^2, but the paper does not supply this argument. Separately, the proof covers only linear hidden layers, zero-initialized weights, and a single training example, while the headline experiments use tanh hidden units, He-initialized weights, and minibatches. The 'always within 90 degrees' claim in that experimental regime therefore rests entirely on the empirical angle measurements in Figures 2 and 3 and Supplementary Figures S1 and S2. This is a correctable proof gap and a scope-transfer gap, not a contradiction of the empirical results.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Direct Random Target Projection (DRTP), a learning rule for feedforward classifiers that replaces backpropagated error signals with fixed random projections of the one-hot target vector. Because the target vector is known at the start of the forward pass, DRTP removes the feedback pathway and the update-locking constraint of backpropagation. The paper gives a theoretical alignment argument in Supplementary Note 1 for networks with linear hidden layers, zero-initialized weights, and a single training example, and it supports the method with experiments on synthetic regression and classification tasks, MNIST, and CIFAR-10, comparing BP, FA, DFA, sDFA, and DRTP over 10 runs with reported standard deviations. Public code is provided.","tokens_in":25284,"tokens_out":6191,"duration_ms":62604,"significance":"If the central claim holds, DRTP is a significant contribution: it is a simple, purely feedforward alternative to backpropagation that addresses both weight transport and update locking, with demonstrated accuracy on MNIST and CIFAR-10 and a plausible hardware-cost argument supported by an independent implementation in [33]. The empirical evaluation is careful by the standards of the field: fixed train/test protocols, 10 trials, standard deviations, grid-selected learning rates, and publicly available code. The theoretical guarantee is not yet presented in a fully valid form as printed, but the gap is localized and correctable, and the empirical angle measurements provide substantial supporting evidence for the transfer of the idealized result to the actual tanh/minibatch experiments.","major_comments":[{"comment":"The displayed derivation in the Alignment step is not valid as printed. The product of the forward matrices and its Moore-Penrose pseudo-inverse is labeled as the identity, but it is an orthogonal projection, not the identity. Strict positivity of the dot product can still be recovered from the earlier parts of the proof: for the rank-one matrix P = α s_WK^t (B_k^T y*)^T, we have e^T P P^+ e = (s_WK^{tT} e)^2 / ||s_WK^t||^2, and the positivity of s_WK^{tT} e follows from the previously shown sign agreement between s_WK^t and the error vector. This argument is absent, so the proof as written does not establish the claimed strict positivity; the gap is correctable but must be repaired in the final version.","section":"Supplementary Note 1, Alignment step"},{"comment":"The theorem establishing the within-90-degree alignment is explicitly restricted to linear hidden layers, zero-initialized weights, and a single training example, while the headline classification experiments use He-initialized tanh networks trained with minibatches (Section 4, Tables 1 and 2). The paper should state clearly that the formal guarantee does not cover the evaluated setting and that the transfer to tanh/minibatch training rests on the empirical angle measurements in Figures 2, 3, S1, and S2. This is a scope-transfer gap rather than a contradiction of the empirical results, but it affects the strength of the central claim as worded.","section":"Section 2.3 and Section 4"}],"minor_comments":[{"comment":"The sentence \"the forward weights are drawn from a He uniform distribution and are zero-initialized for feedback-alignment-based algorithms\" is ambiguous; please clarify which weight matrices are zero-initialized in the FA/DFA/sDFA/DRTP experiments.","section":"Section 4, synthetic regression and classification"},{"comment":"The product notation ∏_{i=k+1}^K W_i^t is used for noncommuting matrices without specifying the multiplication order; please define it explicitly (for example, as W_K^t W_{K-1}^t ... W_{k+1}^t or the reverse).","section":"Supplementary Note 1, network dynamics"},{"comment":"The captions could be clearer about which curves correspond to which algorithm; in particular, Supplementary Figure 3 plots three algorithms but the caption refers to them collectively as feedback-alignment-based.","section":"Figure 4 and Supplementary Figure 3 captions"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an arXiv version of a paper already accepted at Frontiers in Neuroscience, and the core empirical contribution appears sound. The main issue is the invalid identification of a projection with the identity in the supplementary proof; this is correctable and does not undermine the experimental findings. I recommend major revision to fix the proof and to qualify the theoretical claim's scope, but I see no grounds for rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DRTP is worth knowing about. It replaces the propagated error with a fixed random projection of the one-hot label, which is a genuine algorithmic step beyond DFA and FA: for classification with sigmoid/softmax outputs, the label encodes the error sign, so each hidden layer can update from information already available in the forward pass. That observation in Section 2.2 is simple and correct, and the update rule really does solve both weight transport and update locking at almost zero feedback cost.\n\nThe empirical side is better than average for this literature: ten runs per configuration, standard deviations, fixed protocols, grid-searched learning rates per algorithm, and public PyTorch code. MNIST/CIFAR-10 results are clearly positioned as a tradeoff for edge hardware, not as an accuracy breakthrough. The discussion of the convolutional bottleneck is honest, and the comparison to sDFA is done carefully. I also checked the citation pattern: the prior art on FA/DFA, and [13] specifically, is credited properly; the reference to the authors' own hardware implementation [33] is an independent chip implementation, so self-citation is not a problem there.\n\nThe soft spot is the theory. The proof in Supplementary Note 1 is presented as a guarantee that DRTP updates are always within 90 degrees of BP. As printed, the final Alignment step is wrong: the product of forward matrices times its pseudo-inverse is not the identity but an orthogonal projection. The conclusion can be recovered — because under the lemma's zero-initialized, single-example construction the relevant product is rank-one with range spanned by the error-aligned vector, the quadratic form is positive — but the paper doesn't supply that argument. And even after repair, the guarantee covers linear hidden layers, zero-initialized weights, and a single training example. The tanh, He-initialized, minibatch networks in the headline experiments are outside its scope; in that regime the 90-degree claim rests on the empirical angle plots, not on the theorem. Both are correctable issues, and they don't contradict the empirical results.\n\nWho gets value: neuromorphic and edge-computing hardware people deciding whether DRTP is cheap enough to implement, and researchers working on feedback-alignment theory. I'd bring it to reading group. It deserves proper peer review — the proof needs fixing, but the algorithm and experiments justify referee time.","headline":"Simple, useful training rule with honest empirical work; the proof of 90-degree alignment has a fixable gap that should be requested before publication.","tokens_in":25904,"tokens_out":2291,"would_cite":true,"duration_ms":21822,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fixed random label projections train deep networks without backpropagation or feedback pathways.","keywords":["direct random target projection","feedback alignment","weight transport","update locking","biologically plausible learning","edge computing","feedforward training"],"falsifier":"Train a deeper fully-connected tanh network on a classification dataset with DRTP while measuring the angle between DRTP modulatory signals and backpropagation gradients; if the angle exceeds 90 degrees for any hidden layer while accuracy is still improving, or if the network fails to learn despite the angle claim, the central claim is falsified. More directly, evaluate the alignment identity \\(e^T(\\prod W_i)(\\prod W_i)^+e\\) in a single-example linear-hidden-layer setup; if it is non-positive at any update step, the proof's conclusion fails.","tokens_in":24791,"feed_emoji":"🧠","tokens_out":2581,"duration_ms":20596,"temperature":0.7,"pith_summary":"This paper proposes that supervised classification can train deep feedforward networks using only the one-hot labels and fixed random projection matrices, eliminating the backpropagation feedback pathway entirely. The authors introduce the direct random target projection (DRTP) algorithm, which replaces propagated error signals with fixed random projections of the target label. They aim to show this works because, for classification outputs bounded between 0 and 1, the error sign is known in advance from the label, serving as a proxy for the true gradient direction. If true, this would make neural network training cheaper, faster, and more biologically plausible, enabling low-cost adaptive edge devices.","feed_headline":"Fixed random label projections train deep networks without backpropagation or feedback…","feed_subtitle":"A feedforward learning rule uses one-hot labels and random projection matrices to train hidden layers at low cost.","key_machinery":"The central object is the direct random target projection (DRTP) update rule, which sets each hidden layer's estimated gradient to \\(\\delta y_k = B_k^T y^*\\), where \\(B_k\\) is a fixed random connectivity matrix and \\(y^*\\) is the one-hot target. The insight is that replacing the error vector with the target is equivalent to a shift-and-rescale of the error sign, so no feedback pathway is needed. The proof of 90-degree alignment relies on a lemma showing that with zero-initialized weights and a single training example, each weight matrix and hidden activation becomes a scalar multiple of a random target projection, which then lets the DRTP modulatory signal be expressed as a negative scalar multiple of a Moore-Penrose pseudo-inverse times the error.","core_discovery":"The central claim is that for classification networks with sigmoid or softmax outputs and cross-entropy loss, the one-hot target encoding y* can replace the error signal in feedback alignment-based training, and the resulting modulatory signals remain aligned within 90 degrees of the backpropagation gradient directions. This makes it possible to train hidden layers of feedforward networks using only layer-local updates computed from a label-dependent random vector selection, with no dedicated feedback pathway. The paper proves this alignment for networks with linear hidden layers trained on a single example, and demonstrates empirically that DRTP trains fully-connected tanh networks on MNIST and CIFAR-10, reaching accuracies between backpropagation and shallow learning.","pith_inferences":["The proof relies on the alignment term \\(e^T(\\prod W_i)(\\prod W_i)^+e\\), which the paper labels as \\(e^Te\\), but the product of a matrix and its pseudo-inverse is an orthogonal projection, not the identity. Strict positivity of the alignment is therefore not established by the algebra as written, though the conclusion may still hold for other reasons.","The empirical success on tanh networks suggests the 90-degree alignment may hold more broadly than the linear-hidden-layer proof, but the proof itself does not cover nonlinear activations or minibatch training.","The authors note that DRTP drives tanh activations toward saturation, which stops learning; this could be exploited deliberately to produce networks with binarized activations for efficient inference.","Since the algorithmic benefit is clearest for fully-connected layers and the paper observes a bottleneck effect in convolutional layers, an interesting extension would be combining DRTP for fully-connected layers with local-error methods for convolutional layers."],"forward_implications":["Layers can be updated immediately during the forward pass, eliminating update locking and reducing memory buffering overhead.","The weight transport problem disappears because no symmetric feedback weights are needed at all.","DRTP is computationally cheaper than direct feedback alignment, since projecting the one-hot target is just selecting a precomputed random vector per layer per class.","On MNIST and CIFAR-10 fully-connected networks, DRTP outperforms shallow learning and approaches feedback alignment accuracy, making it a candidate for adaptive edge computing.","The purely feedforward update rule can be formulated as a three-factor synaptic plasticity rule, connecting to biologically plausible learning."],"supporting_citations":[{"why":"Provides the original feedback alignment proof of 90-degree alignment that DRTP extends.","marker":"[13]"},{"why":"Direct feedback alignment, the baseline algorithm DRTP derives from and is compared against.","marker":"[15]"},{"why":"Backpropagation, the algorithm whose gradient directions DRTP must align with.","marker":"[4]"}],"fun_headline_variants":["No backprop needed: Random label projections train deep nets","Feedforward training via fixed random label projections","DRTP: Train deep nets without feedback using random labels","Random target projections replace error signals in deep learning","Fixed random labels enable low-cost feedforward deep net training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The mathematical guarantee that DRTP updates stay within 90 degrees of backpropagation is proven only for a network with linear hidden layers, weights initialized to zero, and a single training example, and even that proof contains an unproven step where a matrix times its pseudo-inverse is treated as the identity; the paper assumes this result transfers to tanh networks trained on minibatches, supported only by measured angles.","fun_headline_variants_meta":{"raw":{"variants":["No backprop needed: Random label projections train deep nets","Feedforward training via fixed random label projections","DRTP: Train deep nets without feedback using random labels","Random target projections replace error signals in deep learning","Fixed random labels enable low-cost feedforward deep net training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1409,"prompt_tokens":840,"completion_tokens":569,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":456,"completion_tokens_details":{"reasoning_tokens":493}},"tokens_in":456,"tokens_out":569,"duration_ms":5409,"temperature":1.0,"reasoning_tokens":493,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:21:46.362508+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a deeper fully-connected tanh network on a classification dataset with DRTP while measuring the angle between DRTP modulatory signals and backpropagation gradients; if the angle exceeds 90 degrees for any hidden layer while accuracy is still improving, or if the network fails to learn despite the angle claim, the central claim is falsified. More directly, evaluate the alignment identity \\(e^T(\\prod W_i)(\\prod W_i)^+e\\) in a single-example linear-hidden-layer setup; if it is non-positive at any update step, the proof's conclusion fails.","supporting_citations":[{"cited_title":"Random synaptic feedback weigh ts support error backpropagation for deep learning,","cited_arxiv_id":null,"evidence_quote":"Provides the original feedback alignment proof of 90-degree alignment that DRTP extends."},{"cited_title":"Direct feedback alignment provides learn ing in deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Direct feedback alignment, the baseline algorithm DRTP derives from and is compared against."},{"cited_title":"Learning repre sentations by back-propagating errors,","cited_arxiv_id":null,"evidence_quote":"Backpropagation, the algorithm whose gradient directions DRTP must align with."}],"review_version":1}