{"id":"25236442-cb6a-4245-be00-910ff748a418","arxiv_id":"2505.15840","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A top-down feedback module for spiking transformers improves temporal information flow, reduces temporal vanishing gradients, and reaches 86.83% top-1 accuracy on ImageNet.","lead":"This paper adds a top-down feedback pathway to spiking transformers, where later processing stages receive information about earlier, higher-level states, and reports state-of-the-art accuracy on ImageNet for spiking neural networks. A smart generalist might read it to see whether biologically inspired temporal feedback actually closes the accuracy gap between spiking and conventional neural networks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ImageNet gains may be a fine-tuning artifact: Table 1 compares TDFormer fine-tuned for 15 epochs against a QKFormer baseline not re-run under the same fine-tuning protocol.","rationale":"The reader's verdict was CONDITIONAL, and I agree that the paper should not be accepted as-is. However, I do not think the theorem about nonzero feedback gradients is the single most load-bearing concern. That theorem is used to explain why TDAC helps, but even if the proof is incomplete, the empirical accuracy gains could still stand on their own. The more serious issue is that the headline ImageNet comparison is not an apples-to-apples control: TDFormer is fine-tuned for 15 epochs from a pretrained QKFormer checkpoint, while the baseline numbers in Table 1 are the published QKFormer results, not a QKFormer baseline fine-tuned under the same protocol. Continued fine-tuning of a strong pretrained checkpoint can itself produce nontrivial accuracy gains, so the reported +1.15 to +1.18 points may not be caused by TDAC at all. This is a concrete, testable confound, and it directly affects the paper's central claim of state-of-the-art performance. The missing code and checkpoints further prevent independent verification, but the matched-baseline control is the logical first check. The theory concern is real but secondary; the reader's weakest_assumption focuses on it, so my agreement is partial rather than full. I would keep the verdict CONDITIONAL: release the fine-tuned baseline results, the exact pretrained checkpoint, and ideally code, before the SOTA claim can be credited to the feedback mechanism.","tokens_in":21499,"tokens_out":4110,"duration_ms":43142,"concrete_test":"Re-run the three ImageNet rows of Table 1 with TDAC removed but with the same pretrained QKFormer checkpoint and the same 15-epoch fine-tuning recipe (AdamW, base learning rate 2e-5, cosine schedule, batch size 57, same augmentation), reporting top-1 accuracy at 224x224, 288x288, and 384x384 resolutions. If fine-tuned QKFormer reaches approximately 86.8% at 384x384, or if the TDFormer gain shrinks to within a few tenths of a point at the other resolutions, the claimed SOTA improvement is not attributable to the feedback structure.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing empirical claim is the ImageNet state-of-the-art accuracy of 86.83%. Appendix A.1 states that for QKFormer, the authors 'fine-tuned the pretrained network with a base learning rate of 2e-5 for 15 epochs' because direct training on ImageNet is expensive. Table 1 then compares these fine-tuned TDFormer models (85.37, 86.29, 86.83) against QKFormer rows (84.22, 85.20, 85.65) that appear to be the published baseline numbers, not QKFormer re-run under the identical 15-epoch fine-tuning recipe. No 'QKFormer(ours)' row is reported for ImageNet, although Table 2 does include '(ours)' baselines for the smaller datasets. This creates a confound: the +1.15 to +1.18 accuracy gains could come wholly or partly from continued training of a pretrained checkpoint with a favorable low learning rate and cosine schedule, rather than from the TDAC feedback pathway. The causal attribution of the headline gain to top-down feedback is therefore not established. A secondary concern is that Theorem 4.3's nonzero feedback-gradient claim is asserted after Eq. (108) without specifying the feedback function phi or proving that d(phi)/dS is nonzero inside the threshold interval; however, that affects the explanatory story, whereas the missing matched baseline affects the headline accuracy claim itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TDFormer, a modification of three transformer-based spiking neural network backbones (SpikformerV1, Spike-driven TransformerV1, and QKFormer) that adds a top-down feedback pathway called TDAC. The pathway consists of a control module (CM), which mixes bottom-up and top-down signals to modulate attention, and a processing module (PM), which generates the feedback signal from higher-level representations. The authors report consistent accuracy improvements on CIFAR-10, CIFAR-100, CIFAR10-DVS, DVS128 Gesture, and ImageNet, where they claim a new state of the art of 86.83% top-1 accuracy with minimal parameter and energy overhead. The paper also presents two theoretical contributions: a variance bound for the PM attention output that motivates a clamp operation, and a theorem claiming that the feedback term alleviates temporal vanishing gradients in SNNs.","tokens_in":21746,"tokens_out":6829,"duration_ms":64158,"significance":"If the reported ImageNet result is trustworthy, the paper makes a significant empirical contribution: it introduces a structurally new mechanism for temporal information flow in transformer-based SNNs, demonstrates accuracy close to ANN-level on ImageNet, and supports the main claims with ablations across CM/PM variants, multiple backbones, multiple seeds, corruption robustness experiments, and energy-consumption estimates. The theoretical analysis is less central but could add value if made rigorous. However, the headline ImageNet claim currently rests on an unmatched fine-tuning comparison, and the vanishing-gradient theorem is asserted rather than proved for the actual feedback modules. These issues are load-bearing for the paper's main claims.","major_comments":[{"comment":"The ImageNet comparison is not matched. Appendix A.1 states that for QKFormer the authors 'fine-tuned the pretrained network with a base learning rate of 2e-5 for 15 epochs' to obtain the TDFormer results, yet Table 1 reports QKFormer baseline accuracies (84.22, 85.20, 85.65) that appear to be the published numbers, with no 'QKFormer(ours)' row re-run under the same 15-epoch fine-tuning recipe. The reported +1.15 to +1.18 gains therefore conflate the effect of the TDAC module with the effect of continued training of a pretrained checkpoint at a low learning rate. The central SOTA claim of 86.83% is not established unless the QKFormer baseline is re-run under the identical fine-tuning protocol, or TDFormer is compared against QKFormer trained from scratch under the original protocol.","section":"Appendix A.1, Table 1"},{"comment":"The vanishing-gradient theorem is not supported as stated. The proof assumes a differentiable feedback function φ_θ, but the manuscript never specifies φ_θ analytically or proves that ∂φ_θ(S_l(t))/∂S_l(t) is nonzero inside the firing-threshold interval; the sentence after Eq. (108) merely asserts this. In addition, there is a factor inconsistency: inside the interval 1/2ϑ < H < 3/2ϑ the rectangular surrogate gives ∂S/∂H = 1/ϑ, so combining Eqs. (106) and (107) yields ϵ_l(t)_jj = (1/ϑ) · ∂φ_θ/∂S_l(t), not ∂φ_θ/∂S_l(t) as written in Eq. (108). The theorem and its proof need to be corrected, and the nonzero-derivative condition must either be proved for the actual PM/CM modules or explicitly stated as an assumption with empirical verification.","section":"Theorem 4.3, Appendix B.3, Eq. (108)"},{"comment":"The variance analysis that motivates the clamp operation and the hyperparameter recommendation b=0, a∈[1,2] relies on strong, partly unstated assumptions: asymptotic independence of X and M (Lemma B.3), b≈0, a≥1, small firing rate f, and the approximation μ≈0 with exponential tail terms dropped. The errors in these approximations are not controlled, so the recommendation is not rigorously grounded. Please state these assumptions in the main text and provide empirical validation, for example by reporting measured firing rates and the variance of M against the predicted bounds.","section":"Proposition B.4, Appendix B.1"}],"minor_comments":[{"comment":"The text says 'increasing the model size by merely 0.02 million parameters and 0.59 millijoules of power consumption,' but the first TDFormer row versus QKFormer shows +0.59 million parameters and +0.02 mJ; the two quantities are swapped.","section":"Section 5.1, Table 1"},{"comment":"The text refers to 'the exception of Spikformer-4-384 (V1) at time step 6,' but Table 2 only shows time steps 2 and 4; the exception appears in Table 5 for CIFAR-100/T=6, so the cross-reference should be corrected.","section":"Section 5.2, Table 2"},{"comment":"Section 5.3 says the robustness evaluation uses CIFAR-10C with 15 corruption types, while Appendix A.2 and the caption of Table 7 state 19 corruption types; the number should be consistent.","section":"Section 5.3, Appendix A.2"},{"comment":"The notation for shapes is confusing: S_bu is given shape T×H×W×C while H_n and S_td have shape T×N×C, even though the control module concatenates these along the channel dimension; the manuscript should clarify that N = H·W (or otherwise define the reshaping explicitly).","section":"Equations (10)-(14)"},{"comment":"The derivation of Eq. (107) is missing the factor 1/ϑ from ∂S/∂H when simplifying the feedback term; this is related to the major comment on Eq. (108) and should be fixed consistently.","section":"Appendix B.3, Eq. (107)"}],"recommendation":"major_revision","confidential_remarks":"The paper's headline is a state-of-the-art ImageNet result, but the unmatched fine-tuning baseline is a serious confound. I would prioritize asking for a matched QKFormer baseline under the identical 15-epoch fine-tuning protocol before considering acceptance. The theoretical vanishing-gradient claim is also not yet rigorous, but it is secondary to the empirical claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the take on arXiv:2505.15840. The architecture is honest work: a top-down pathway (CM + PM) that routes previous-step spike signals back into the attention computation. The gains are consistent across backbones and datasets, and the overhead numbers are presented cleanly. That part deserves a serious look.\n\nThe problems are in two claims. First, the headline ImageNet state-of-the-art of 86.83% may be a fine-tuning artifact. Appendix A.1 says they fine-tuned a pretrained QKFormer for 15 epochs at 2e-5 to get TDFormer. But the QKFormer rows in Table 1 are the published numbers, not re-run under that fine-tuning protocol. There's no 'ours' baseline for ImageNet, even though Table 2 includes those for CIFAR. So the +1.15 to +1.18 gains could come partly from continued training of a pretrained checkpoint rather than from the feedback module. That undermines the causal attribution in the main table.\n\nSecond, Theorem 4.3 claims the feedback term makes the temporal sensitivity nonzero inside the threshold region and potentially larger than 1 - 1/tau. But the feedback function phi is never specified, and the claim that its derivative can't be zero is asserted after Eq. (108), not proven. If the learned feedback path happens to have near-zero derivative in that region, the argument collapses. This is the explanatory story more than the empirical result, but it's presented as a theorem.\n\nWhat's good: the CM/PM decomposition is a new combination, not just a new name; the ablations over four PM variants and three CM variants give a decent picture of the design space; the repeated-seed results in Table 5 are above the field's usual bar; and the robustness tables, while mixed, are a useful addition. The variance analysis in Proposition B.4 assumes Bernoulli firing and asymptotic channel independence—stated, but not tested. That's a minor issue compared to the unmatched baseline. Citation pattern is fine; the related work on top-down feedback in ANNs is acknowledged.\n\nBottom line: for someone building spiking transformers, the module is worth trying and the paper should be engaged with seriously. But the SOTA claim needs matched baselines and the gradient theorem needs either a proof or an empirical check of the feedback derivative. I'd send it to review with those requests, not desk-reject it.","headline":"A clean top-down feedback module for spiking transformers with consistent small gains, but the ImageNet SOTA number is built on an unmatched fine-tuning baseline and the vanishing-gradient theorem is asserted rather than proved.","tokens_in":22324,"tokens_out":2116,"would_cite":false,"duration_ms":20678,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A top-down feedback pathway lifts spiking transformer accuracy to 86.83% on ImageNet.","keywords":["spiking neural networks","transformer","top-down feedback","temporal gradient vanishing","spike-based self-attention","ImageNet classification","LIF neuron","mutual information across time steps"],"falsifier":"Train TDFormer and its baseline to convergence, then freeze the model and evaluate $\\partial\\varphi_\\theta(S_l(t))/\\partial S_l(t)$ for all membrane potentials in the band $\\vartheta/2 < H^l_j(t) < 3\\vartheta/2$ across layers and time steps. If the mean absolute value of that derivative is close to zero, or below $1-1/\\tau$, in the trained network, then the claimed vanishing-gradient relief does not hold for the learned feedback path. A complementary check is to compare the temporal-gradient norm through BPTT with the feedback term included versus ablated, at matched accuracy.","tokens_in":21232,"feed_emoji":"⚡","tokens_out":10292,"duration_ms":93846,"temperature":0.7,"pith_summary":"TDFormer argues that existing transformer-based spiking neural networks are temporally impoverished: each time step runs like a separate subnetwork whose only link to the past is the membrane potential, so features across time carry little mutual information and temporal gradients fade. The paper's proposed fix is a top-down attention-controlled feedback pathway that carries high-order representations from a later time step back to modulate an earlier one through a learned differentiable function $\\varphi_\\theta(S_l(t))$. On the forward pass this raises mutual information across time steps; on the backward pass the paper proves that the same feedback term makes the per-step temporal sensitivity $\\epsilon^l(t)_{jj}$ nonzero inside the firing-threshold interval, where a conventional LIF neuron has sensitivity zero. Empirically, the model reaches 86.83% top-1 accuracy on ImageNet at four time steps, with minimal added parameters and power over its QKFormer baseline, which the paper reports as the best SNN number on that benchmark. The claim matters because it points to a structural, biologically inspired fix for temporal representation in SNNs rather than a training-only workaround.","feed_headline":"Feedback path lifts spiking transformer to 86.83%","feed_subtitle":"A top-down loop between time steps sharpens temporal features and eases vanishing gradients at tiny energy cost.","key_machinery":"The load-bearing machinery is the TDAC top-down pathway, made of a control module (CM) and a processing module (PM), wrapped around a transformer backbone built on leaky integrate-and-fire (LIF) neurons. CM concatenates the bottom-up spikes of the current stage with the top-down spikes of the previous stage to form the key, while queries and values come from bottom-up features, so attention is controlled by higher-order temporal context. PM computes a spatial attention map $M_{\\rm spatial}=\\operatorname{clamp}(\\sum_c W_c X_{t,n,c}, b, a)$, multiplies it into the spiking feature map, and re-spikes, producing the feedback signal $S^{(n)}_{td}$ and the additive differentiable term $\\varphi_\\theta(S_l(t))$ in the membrane dynamics. Proposition 4.1 bounds the variance of the PM output $X \\odot M_{\\rm spatial}$ under a Bernoulli firing-rate model, and Proposition B.4 gives mean and variance approximations that justify the recommended setting $b=0$, $a\\in[1,2]$. The whole design is expressed as a sequence of $N$ sub-networks, each with its own classification loss, so the feedback path doubles as a temporal hierarchy.","core_discovery":"On its own terms, the paper's central discovery is that adding a top-down feedback term to the membrane potential update changes both what an SNN represents over time and how it learns over time. In the forward direction, the control module CM folds the previous stage's top-down spikes into the key computation of spike-based self-attention, and the processing module PM produces a spatially gated, clamped spiking feature map, so later high-order representations steer earlier low-order processing. In the backward direction, the paper proves (Theorem 4.3) that with a rectangular surrogate gradient the per-step sensitivity of a conventional leaky integrate-and-fire (LIF) neuron is $\\epsilon^l(t)_{jj}=0$ when the membrane potential lies in the interval $(\\vartheta/2, 3\\vartheta/2)$ and $1-1/\\tau$ otherwise, while with the feedback path it is $\\partial\\varphi_\\theta(S_l(t))/\\partial S_l(t)$ in that interval, which the paper argues is nonzero and can exceed $1-1/\\tau$. That is the mechanism claimed to alleviate temporal vanishing gradients. The measured consequence is consistent gains over three spiking transformer baselines on CIFAR-10, CIFAR-100, CIFAR10-DVS, DVS128 Gesture, and ImageNet, with the ImageNet result at 86.83%.","pith_inferences":["Editorial: the same control-module pattern could be tested on CNN-based SNNs by replacing the attention-based key fusion with a convolutional feedback mixer; the paper lists transformer-only generality as a limitation, so this is a natural next experiment.","Editorial: if the gradient mechanism is real, it should appear as a larger temporal gradient norm early in training for TDFormer than for its baseline; measuring that norm at matched loss would separate the gradient benefit from the mutual-information benefit.","Editorial: the variance bound assumes independent, low-rate Bernoulli spikes, so under correlated or high-rate firing the recommended clamp setting $b=0$, $a\\in[1,2]$ may need re-tuning; the paper does not report that stress test.","Editorial: because the feedback term is additive in the membrane update and differentiable, the same construction could be dropped into other time-rolled architectures, such as recurrent or equilibrium models, as a generic temporal-context gate."],"forward_implications":["If the central claim is right, transformer-based SNNs can reach ANN-level accuracy on ImageNet (86.83% at four time steps), narrowing the gap to non-spiking vision transformers on the benchmark.","The temporal-gradient result implies that training deeper or longer-time-step SNNs should suffer less from gradient decay along time, making the time dimension a more usable resource for network design.","The feedback path's low overhead means the accuracy gain does not come at the cost of the energy advantage that motivates SNNs.","Raising mutual information between time steps suggests the model integrates temporal context rather than just averaging spike rates, which should improve tasks where ordering or accumulation matters.","Weighted supervision of every sub-network output gives earlier stages their own learning signal, consistent with the claim that temporal receptive field grows along the feedback hierarchy."],"supporting_citations":[{"why":"defines the Spikformer architecture and spike-based self-attention that TDFormer uses as one backbone and baseline.","marker":"[2]"},{"why":"provides the QKFormer/HST baseline, the ImageNet accuracy and power numbers TDFormer improves on, and the training recipe it fine-tunes.","marker":"[3]"},{"why":"introduces the spike-driven transformer and SDSA attention used as another TDFormer backbone.","marker":"[5]"},{"why":"supplies the SDSA2 attention variant and Meta-SpikeFormer configurations that appear in comparisons.","marker":"[6]"},{"why":"cited as prior evidence that temporal gradients through membrane potentials vanish in LIF networks.","marker":"[8]"},{"why":"cited with [8] for the vanishing temporal gradient problem and for the surrogate-gradient setting used in Theorem 4.3's proof.","marker":"[9]"},{"why":"supplies the rectangular surrogate gradient assumption that Theorem 4.3 adopts.","marker":"[12]"},{"why":"the appendix credits this work for the temporal-gradient derivation that the feedback analysis extends.","marker":"[40]"}],"fun_headline_variants":["Top-down feedback sharpens spiking transformer to 86.83%","Brain-inspired feedback loop boosts spiking transformer accuracy","Top-down attention control lifts spiking transformer on ImageNet","Feedback path cuts temporal gradient loss in spiking transformers","Hierarchical top-down spikes improve SNN temporal learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof that feedback removes temporal vanishing gradients assumes the learned feedback function $\\varphi_\\theta$ has a derivative with respect to the spike that is never zero inside the firing-threshold band, yet the paper never specifies $\\varphi_\\theta$ or proves that nonzero property.","fun_headline_variants_meta":{"raw":{"variants":["Top-down feedback sharpens spiking transformer to 86.83%","Brain-inspired feedback loop boosts spiking transformer accuracy","Top-down attention control lifts spiking transformer on ImageNet","Feedback path cuts temporal gradient loss in spiking transformers","Hierarchical top-down spikes improve SNN temporal learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000813,"raw_usage":{"total_tokens":3611,"prompt_tokens":1042,"completion_tokens":2569,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":2489}},"tokens_in":658,"tokens_out":2569,"duration_ms":17946,"temperature":1.0,"reasoning_tokens":2489,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:40:30.621944+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train TDFormer and its baseline to convergence, then freeze the model and evaluate $\\partial\\varphi_\\theta(S_l(t))/\\partial S_l(t)$ for all membrane potentials in the band $\\vartheta/2 < H^l_j(t) < 3\\vartheta/2$ across layers and time steps. If the mean absolute value of that derivative is close to zero, or below $1-1/\\tau$, in the trained network, then the claimed vanishing-gradient relief does not hold for the learned feedback path. A complementary check is to compare the temporal-gradient norm through BPTT with the feedback term included versus ablated, at matched accuracy.","supporting_citations":[{"cited_title":"Spike- driven transformer","cited_arxiv_id":null,"evidence_quote":"introduces the spike-driven transformer and SDSA attention used as another TDFormer backbone."},{"cited_title":"Spike-driven transformer v2: Meta spiking neural network architecture inspiring the design of next-generation neuromorphic chips","cited_arxiv_id":null,"evidence_quote":"supplies the SDSA2 attention variant and Meta-SpikeFormer configurations that appear in comparisons."},{"cited_title":"Training high-performance low-latency spiking neural networks by differentiation on spike representation","cited_arxiv_id":null,"evidence_quote":"the appendix credits this work for the temporal-gradient derivation that the feedback analysis extends."}],"review_version":1}