{"id":"be515cb5-d6c2-4553-bf57-526098c5313d","arxiv_id":"2509.09977","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ISTASTrack fuses RGB and event features through bidirectional ISTA-unrolled adapters between an ANN ViT and an SNN SpikingFormer, reporting state-of-the-art benchmark scores.","lead":"This paper presents a hybrid tracker that combines a regular vision transformer for RGB frames and a spiking transformer for event-camera data, joined by new ISTA-based adapters. It reports top results on four RGB-event tracking benchmarks with lower estimated compute than a dual-ANN version.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main results (Tab. I) do not match the full-model ablation (Tab. IV); VisEvent SR differs by 0.84, larger than the claimed SOTA margin, so the headline SOTA is unreproducible.","rationale":"The reader's weakest assumption highlighted the fairness of the benchmark comparison and the selection of N and T on the test sets. Those are valid concerns, but the more fundamental and concrete problem is that the paper's own main results table is not reproducible from the described method and ablations. The reported SOTA margins over BAT are very small (e.g., VisEvent SR +0.2), so any internal inconsistency of comparable size is decisive. Table IV's 'Bidirectional adapter' row should be the full model, yet Table I reports higher numbers on VisEvent, COESOT, and FELT. On VisEvent, the gap (0.84 SR) is four times the SOTA margin. Table VI suggests T=1 may have been used for the main table, contradicting the stated default T=3. This is not a question of baseline tuning or protocol; it is an internal mismatch that prevents verification of the central claim. If the authors can explain the discrepancy (e.g., a different checkpoint, a typo, or an unreported configuration), the paper may still be salvageable; as written, the SOTA claim is unsupported. Therefore the verdict should move from CONDITIONAL to UNVERDICTED pending clarification.","tokens_in":23274,"tokens_out":11951,"duration_ms":124440,"concrete_test":"Run the public code with the stated defaults (N=4, T=3, 60 epochs) on VisEvent and FE240hz, reproduce Tab. IV and Tab. I. If VisEvent SR is ~66.5 (matching Tab. IV) rather than 67.3, then Tab. I's SOTA number is not reproducible; if it is ~67.3, then Tab. IV is missing some component and must be corrected. Also check whether the reported VisEvent/COESOT/FELT numbers in Tab. I actually use T=1; if so, the T=3 default and energy trade-off analysis in Sec. IV-F need revision.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim is that ISTASTrack achieves state-of-the-art accuracy. The supporting evidence is Table I, but the numbers are internally inconsistent with the paper's own ablations. The full model (bidirectional ISTA adapters + TDA, N=4, T=3) is reported in Tab. IV as: FE240hz 64.59/92.83, VisEvent 66.46/84.34, COESOT 75.74/86.82, FELT 54.93/65.75 (SR/PR). Table I reports the same metric pairs as 64.7/92.2, 67.3/84.6, 75.7/87.1, 55.2/65.8. On VisEvent the SR gap (0.84) is larger than the reported advantage over the runner-up BAT (0.2); on COESOT and FE240hz the PR gaps are 0.28 and 0.63. These are not rounding errors. Moreover, Tab. VI shows that T=1 gives VisEvent 67.26/84.61 and FELT 55.18/65.75, close to Tab. I, while Tab. IV uses T=3; FE240hz PR 92.2 matches neither T=1 (91.8) nor T=3 (92.83). Thus Table I cannot be mapped to any configuration in the ablations, and the default T=3 statement in Sec. IV-A contradicts the data. Without knowing which configuration produced Table I, the SOTA claim is untestable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ISTASTrack, a hybrid ANN-SNN tracker for RGB-event visual object tracking. The architecture uses a ViT-based ANN branch for RGB and a SpikingFormer-based SNN branch for event streams, with bidirectional ISTA adapters inserted at multiple transformer layers to transfer features between branches. A temporal downsampling attention (TDA) module aggregates multi-step SNN features into single-step ANN representations. The authors report state-of-the-art results on FE240hz, VisEvent, COESOT, and FELT, along with energy-efficiency estimates showing lower estimated energy than a dual-ANN counterpart. The derivation of the adapter from sparse representation theory via ISTA unfolding is a central methodological claim. The paper includes extensive ablations over adapter direction, depth, placement, and time steps, and the code is publicly released.","tokens_in":23807,"tokens_out":3156,"duration_ms":36642,"significance":"If the results are reproducible, the contribution is notable: it is among the first transformer-based ANN-SNN hybrids for RGB-event tracking, and the proposed adapter is lightweight (0.32M parameters), which is a practically relevant property. The paper gives explicit credit to reproducible artifacts: the code is public, the comparisons cover many recent baselines, and the ablations are fairly comprehensive. The energy-estimation methodology follows a widely used convention for SNN cost. However, the strength of the claim rests on the consistency and fairness of the reported benchmark numbers, and on whether the ISTA derivation is genuinely enforced by the training objective. The main value would be as a practical hybrid architecture with competitive accuracy and low estimated cost, rather than as a strict validation of the sparse-coding model.","major_comments":[{"comment":"The headline SOTA numbers in Table I are internally inconsistent with the full-model ablation in Table IV. Table I reports ISTASTrack as FE240hz 64.7/92.2, VisEvent 67.3/84.6, COESOT 75.7/87.1, FELT 55.2/65.8 (SR/PR). Table IV reports the bidirectional-adapter model (the full model, with N=4 and T=3 according to Section IV-A) as FE240hz 64.59/92.83, VisEvent 66.46/84.34, COESOT 75.74/86.82, FELT 54.93/65.75. On VisEvent the SR gap is 0.84, larger than the reported advantage over BAT (0.2); on FE240hz and COESOT the PR gaps are 0.63 and 0.28. These are not rounding errors. Table VI also does not resolve the discrepancy: T=1 gives VisEvent 67.26/84.61 and FELT 55.18/65.75, close to Table I, while Table IV uses T=3. The FE240hz PR 92.2 matches neither T=1 (91.8) nor T=3 (92.83). The authors must identify which configuration produced Table I and report the exact same numbers in the ablation","section":"Table I vs. Table IV"},{"comment":"The benchmark comparison mixes retrained baselines with published numbers from other papers. The text states that nearly all networks are retrained on the four datasets using the authors' settings, while CrossEI and SNNPTrack are reported as published. Since the authors' training protocol may not match each baseline's best configuration, and since no per-method hyperparameter tuning or validation protocol is described, the small margins over BAT (e.g., 0.2 SR on VisEvent) cannot be interpreted as meaningful improvements. The authors should either use each baseline's official code/checkpoints under its own protocol, or report variances and a common tuning budget. This is especially important because the claimed SOTA margins are small.","section":"Sec. IV-C.1, Table I"},{"comment":"The sparse-coding derivation is not enforced by the training objective. Eq. (4) defines a LASSO problem for the sparse code, but the adapters in Eq. (7) are trained by backpropagation through the tracking loss in Eq. (12); there is no reconstruction term, no sparsity penalty, and no constraint that the learned dictionaries satisfy the ISTA fixed-point equations. Moreover, Eq. (6) explicitly treats P and D^T as independent matrices, which breaks the ISTA update's reliance on D^T. As a result, the adapters behave as learned residual cross-modal modules, and the claim that the method is 'derived from sparse representation theory' and 'interpretable' overstates the link. The authors should either enforce the model-based objective (e.g., add a sparse-coding auxiliary loss or a reconstruction term) or reframe the ISTA connection as architectural motivation only.","section":"Sec. III-C.2, Eq. (4) and Eq. (12)"},{"comment":"The hyperparameters N (number of adapter layers) and T (number of time steps) are selected on the same test datasets used to report SOTA results. Section IV-A fixes N=4 and T=3 based on the ablations in Tables V and VI, but those ablations are evaluated on FE240hz, VisEvent, COESOT, and FELT themselves, not on a held-out validation split. Selecting hyperparameters on the test set inflates the reported accuracy and makes the SOTA claim optimistic. The authors should use a validation split or nested evaluation, and report mean and variance over multiple runs, especially given the small margins in Table I.","section":"Sec. IV-A, Sec. IV-E"}],"minor_comments":[{"comment":"The heading reads 'Overview of ISTATrack'; should be 'ISTASTrack' to match the paper title.","section":"Sec. III-A heading"},{"comment":"The sentence 'with a threshold θ=θ' is circular and likely a typo; it should define the threshold parameter, e.g., θ>0.","section":"Eq. (6)"},{"comment":"The notation D_T for the TDA module conflicts with the dictionary notation D (e.g., D_I, D_E). Using the same letter for a temporal downsampling operator and for dictionaries is confusing; rename one of them.","section":"Eq. (7) and Sec. III-D"},{"comment":"The 'Experimental Settings' subsection has no body text; it jumps directly to 'Datasets'. Either fill in the settings or merge the heading with the previous section.","section":"Sec. IV-A"},{"comment":"The label 'FEL T' in the radar chart appears to be a typo; should be 'FELT'.","section":"Fig. 7"},{"comment":"In Sec. III-C.3, the text cites [14] for the inspiration from bidirectional adapters, but [14] is 'Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining,' which is unrelated to multimodal tracking. The relevant reference appears to be [45] (BAT). Please correct the citation.","section":"References [14] and [45]"}],"recommendation":"major_revision","confidential_remarks":"The internal inconsistency between Table I and Table IV is the most serious issue. Before this paper can be considered for publication, the authors must reconcile the reported numbers and disclose the exact configuration used for the headline comparisons. I would also want the editors to verify whether the ISTA-derived adapter claim is supported by the actual training objective, as the current formulation does not optimize the stated sparse-coding problem. The paper has value as a hybrid architecture study, but the empirical claims need to be placed on solid, reproducible ground."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this if you care about event tracking or hybrid ANN-SNN design; otherwise skip. The genuinely new thing is a two-branch tracker: a ViT for RGB, a SpikingFormer for event data, glued by bidirectional ISTA adapters unrolled from sparse coding, plus a TDA module that pools across SNN time steps. That combination is new relative to MMHT and SNNPTrack, and the ablations support the design choices: bidirectional beats unidirectional, TDA helps, early-layer adapters help. The energy numbers are the real selling point—halving MACs for roughly matched accuracy against the ANN-ANN variant.\n\nNow the soft spots, and one of them is large. Table I, the headline SOTA table, does not match the paper's own full-model ablation in Table IV. VisEvent SR is 67.3 vs 66.46, a 0.84 gap four times the reported margin over BAT (0.2). FE240hz PR is 92.2 vs 92.83; COESOT PR is 87.1 vs 86.82. Table VI doesn't explain it: VisEvent T=1 gives 67.26/84.61, close to Table I, but FELT and FE240hz don't line up with any T. No configuration in the ablations reproduces Table I, so the SOTA claim is currently untestable. That needs to be fixed before this paper is reliable.\n\nThe sparse-coding story is also oversold. The shared-code assumption in Eq. (2) is asserted, and the unrolled ISTA layers are trained end-to-end with no loss enforcing the sparse reconstruction, so the adapters behave as learned residual projections. That's not a fatal flaw—unfolding is a valid design idiom—but calling the result 'interpretable' is a stretch.\n\nThe comparison protocol is mixed (retrained baselines plus published numbers for CrossEI and SNNPTrack), margins over the runner-up are thin on VisEvent and COESOT, hyperparameters N and T are chosen by ablations on the same test sets, and no variance is reported. The T=3 default is also hard to defend: on VisEvent and FELT the hybrid is better at T=1.\n\nTo their credit, the authors state the main limitation (SNN state resets each frame, so long-term temporal memory is unused) in the conclusion, and the code is public.\n\nBottom line: the architecture and efficiency analysis are worth a serious referee, but the reporting inconsistency in the central table is not minor. I'd send it to review with a demand for corrected, variance-aware results and a cleaner statement of which configuration produces the headline numbers. Not desk-reject material.","headline":"Plausible hybrid ANN-SNN tracker with a useful energy story, but the headline SOTA table does not match the paper's own ablations, so the central claim is currently unreproducible.","tokens_in":24200,"tokens_out":4355,"would_cite":false,"duration_ms":45499,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hybrid spiking-transformer tracker with sparse-coding adapters sets new state-of-the-art results across four RGB-event tracking benchmarks.","keywords":["RGB-event tracking","spiking neural network","hybrid ANN-SNN","iterative shrinkage-thresholding","algorithm unfolding","transformer tracker","sparse representation","temporal fusion"],"falsifier":"Retrain CrossEI and SNNPTrack under the exact protocol used for the other baselines (same data splits, loss, epochs) and compare success rate on COESOT or VisEvent: if either matches or exceeds ISTASTrack's score, the claimed state of the art is an artifact of protocol mixing rather than the adapter design.","tokens_in":23239,"feed_emoji":"🎯","tokens_out":4005,"duration_ms":42532,"temperature":0.7,"pith_summary":"The paper proposes ISTASTrack, a tracker that pairs a vision transformer on RGB frames with a spiking transformer on event-camera data. It claims this is the first transformer-based ANN-SNN hybrid for RGB-Event tracking, and that the key to making the two very different feature types work together is a set of 'ISTA adapters' obtained by unrolling the iterative shrinkage-thresholding algorithm for sparse coding. The adapters let each branch read and refine the other's features in both directions, at both attention and MLP stages, while a temporal downsampling attention module compresses the SNN's multi-step outputs into the ANN's single-step format. On FE240hz, VisEvent, COESOT, and FELT, ISTASTrack reports the best success-rate and precision scores among compared trackers, alongside an estimated energy cost roughly 35-40 percent lower than an equivalent ANN-ANN variant. The paper's broader point is that treating cross-modal fusion as a sparse-coding problem gives an interpretable, lightweight route to bridging ANN and SNN representations.","feed_headline":"Hybrid spiking tracker tops four RGB-event benchmarks","feed_subtitle":"ISTA sparse-coding adapters fuse RGB and event branches bidirectionally, winning four benchmarks at roughly 40% lower energy.","key_machinery":"The ISTA adapter: a module formed by unrolling one iteration of the iterative shrinkage-thresholding algorithm, with soft-thresholding using learnable thresholds and dictionary and transpose-dictionary linear layers. Four such adapters are inserted per encoder layer—two directions (RGB-to-event, event-to-RGB) at both the MSA and MLP stages—so the sparse code is progressively refined as features move through the transformer. The TDA module uses adaptive pooling and a sigmoid weighting over the T time steps to collapse multi-step SNN sparse codes to a single-step representation.","core_discovery":"The central claim is that bidirectional feature interaction between an ANN transformer and an SNN transformer, formulated as a shared sparse-code problem and solved by unrolled ISTA steps, lets a hybrid network match and exceed the accuracy of ANN-only trackers while cutting estimated energy use. The authors argue that because RGB and event features are assumed to share the same sparse code, the ISTA adapter can project each branch into the other's latent space, and iterate this projection across the first four encoder layers. The temporal downsampling attention module then aligns the event branch's T time steps with the RGB branch's single step before synthesis. Under this design, the 8-lay","pith_inferences":["The shared-sparse-code assumption, if it holds generally, makes ISTA adapters a candidate template for fusing ANN and SNN features in other event-based tasks such as detection, action recognition, or deblurring, where similar hybrid branches already exist.","Because energy estimates rely on 45nm 32-bit MAC/AC costs and assumed firing rates, the efficiency margin may change on real neuromorphic hardware; a direct chip-level measurement would be needed to confirm it.","A testable extension: with SNN states allowed to persist across frames—the paper names this as a limitation—the TDA module could attend over history, potentially improving long-term tracking on FELT further.","The largest reported gains over the strongest baseline are around 0.2-0.4 success-rate points; whether that margin survives identical training protocols for the two baselines whose published scores are used is an open empirical question the paper does not settle."],"forward_implications":["If the central claim holds, transformer-based ANN-SNN hybrids are a viable route to state-of-the-art RGB-event tracking, not just a low-energy compromise.","The bidirectional, stage-wise adapter design outperforms unidirectional prompt-based fusion on the tested benchmarks.","Early-layer interaction (first 4 layers) beats late fusion; adding adapters beyond four layers can hurt accuracy.","The SNN branch's 8 layers and 3 time steps deliver accuracy comparable to a 12-layer ANN branch, at estimated energy savings of roughly 40 percent.","A lightweight adapter (0.32M parameters, about 2 percent of the total) is sufficient to bridge the ANN-SNN gap, suggesting the fusion mechanism rather than extra capacity drives the gains."],"fun_headline_variants":["ISTA adapter fuses RGB and event branches for top tracking","Hybrid ANN-SNN tracker wins four benchmarks at lower energy","Sparse-coding ISTA bridge makes spiking tracker beat ANN-only","Bidirectional ISTA adapter merges RGB and spikes for SOTA tracking","Event-RGB tracker with ISTA adapter tops four benchmarks efficiently"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The comparison is fair: most baselines are retrained under the authors' protocol, but two are taken from their published papers, and the best hyperparameters (N=4, T=3) are chosen on the same test sets; if those protocol differences are not equivalent, the state-of-the-art margins are not meaningful.","fun_headline_variants_meta":{"raw":{"variants":["ISTA adapter fuses RGB and event branches for top tracking","Hybrid ANN-SNN tracker wins four benchmarks at lower energy","Sparse-coding ISTA bridge makes spiking tracker beat ANN-only","Bidirectional ISTA adapter merges RGB and spikes for SOTA tracking","Event-RGB tracker with ISTA adapter tops four benchmarks efficiently"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000516,"raw_usage":{"total_tokens":2385,"prompt_tokens":834,"completion_tokens":1551,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":1462}},"tokens_in":578,"tokens_out":1551,"duration_ms":11193,"temperature":1.0,"reasoning_tokens":1462,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T18:22:24.210285+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain CrossEI and SNNPTrack under the exact protocol used for the other baselines (same data splits, loss, epochs) and compare success rate on COESOT or VisEvent: if either matches or exceeds ISTASTrack's score, the claimed state of the art is an artifact of protocol mixing rather than the adapter design.","supporting_citations":[],"review_version":1}