{"id":"d114b447-575e-402b-8279-dd28ed0058fd","arxiv_id":"2412.02890","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"EvRT-DETR, a frozen RT-DETR with lightweight ConvLSTM adapters in its latent space, sets new state-of-the-art mAP on Gen1 (52.7) and 1Mpx (50.1) event-camera benchmarks.","lead":"Event cameras produce sparse brightness-change streams that most image detectors cannot use. This paper trains a standard RT-DETR detector on simple event histograms, then inserts small recurrent memory blocks into its frozen network; the resulting EvRT-DETR outperforms specialized event-camera detectors on Gen1 and 1Mpx, suggesting mainstream vision models can be adapted to event data rather than replaced.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The +1.4 mAP 1Mpx SOTA margin is smaller than the paper's own measured protocol sensitivities: bilinear-versus-nearest interpolation changes RT-DETR-B base mAP by 2.9 points, and the 5-to-10-frame clip change adds 0.3 points, so cross-method comparability of the headline number is not established.","rationale":"The reader identified the 50ms/10-bin stacked histogram representation as the weakest assumption. I agree that representation is load-bearing, but the sharper, more concrete risk is cross-method comparability of the headline numbers. The paper's own ablations show protocol sensitivities (2.9 mAP from interpolation choice, 0.3 mAP from clip length) that are as large as or larger than the reported 1Mpx SOTA margin of +1.4. This is internal evidence that the headline margin may not survive a standardized evaluation. The paper is otherwise well-executed: the two-stage freeze-and-adapt design is clearly specified, the YOLOX generalization provides independent support that the mechanism transfers, and the code is released. The concern is not that the method is unsound—it is that the numerical SOTA claim, especially on 1Mpx, has not been shown robust to protocol differences or seed variance. The CONDITIONAL verdict is appropriate, and I would keep it, with the condition being an independent replication on a standardized pipeline plus multi-seed runs.","tokens_in":18403,"tokens_out":2853,"duration_ms":23244,"concrete_test":"Run EvRT-DETR-B and, where public checkpoints exist, RVT-B, S5-ViT-B, ASTMNet, AEC, and ERGO-12 on the same 1Mpx test split with one standardized pipeline: identical 50ms/10-bin stacked histogram, identical bilinear 640x360 resize from 1280x720, identical Prophesee Automotive Dataset Toolbox version, and identical COCO evaluator. If the 1Mpx margin over the best prior method shrinks below ~1.0 mAP, the SOTA claim is protocol-dependent. In parallel, train EvRT-DETR-B on 1Mpx with 3 different seeds and report mean and std of mAP; if the std is comparable to or larger than the +1.4 margin, the headline advantage is not statistically established.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is a SOTA claim on Gen1 (52.7 vs 50.4) and 1Mpx (50.1 vs 48.7). The most load-bearing concern is the comparability of the 1Mpx evaluation. Supplementary E.1 reports that RT-DETR-B base mAP swings from 45.2 (bilinear) to 42.3 (nearest) to 43.1 (bicubic): a 2.9 mAP difference driven purely by the interpolation method used to downsample 1280x720 to 640x360. Prior methods (RVT, S5-ViT-B, ASTMNet, AEC, ERGO-12) each use their own preprocessing, resolutions, and evaluation pipelines, so the 50.1 mAP is directly comparable only if the exact frame-construction and evaluation protocols are identical. The paper states the 1Mpx clip length was increased from RVT's 5 to 10 frames (Supplementary E.2), improving 1Mpx by 0.3 mAP (49.8 to 50.1); that tuning is performed on the same benchmark that produces the headline margin, and 0.3 is 21% of the reported +1.4 advantage. Additionally, the Gen1 headline is largely driven by the strong single-frame RT-DETR-B baseline (47.6, already above RVT-B's 47.2); the temporal module adds +5.1, but the general claim about 'mainstream detectors can replace specialized EBC methods' rests on the base detector being competitive, not only on the adaptation. The absence of error bars or seed variance leaves open the possibility that the reported margin is within training noise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces I2EvDet, a two-stage framework for adapting frame-based object detectors to event-based camera data. In the first stage, an RT-DETR detector is trained on 20-channel stacked 2D histograms of 50 ms event windows. In the second stage, the detector is frozen and lightweight ConvLSTM modules are inserted at the three encoder scales, with residual fusion, to add temporal memory. The resulting EvRT-DETR models are evaluated on Gen1 and 1Mpx and are reported to reach 52.7 and 50.1 mAP, respectively, surpassing prior specialized EBC methods. The paper also reports ablations on module placement, hidden size, kernel size, augmentation, interpolation, clip length, and generalization to YOLOX.","tokens_in":18785,"tokens_out":4010,"duration_ms":39463,"significance":"If the headline results hold, the paper makes a useful contribution by showing that a mainstream detector can be adapted to event data with minimal architectural changes, and the code release and detailed ablations support reproducibility. The strongest aspects are the breadth of ablations, the YOLOX generalization experiments, and the parameter-efficiency analysis. The main caveat is that the 1Mpx SOTA margin is small relative to the paper's own measured sensitivities in preprocessing and hyperparameter choices, and no uncertainty estimates are provided.","major_comments":[{"comment":"The choice of 10-frame clips for 1Mpx was made by tuning on the same benchmark used for the headline claim; it yields +0.3 mAP (49.8 to 50.1), which is about 21% of the reported +1.4 margin. Please report results with a held-out validation split for hyperparameter selection, or provide multi-seed runs and confidence intervals, so that readers can assess whether the margin is distinguishable from protocol variation and training noise.","section":"§4.1 and Supplementary E.2, Tables 1 and 11"},{"comment":"The interpolation method used to downsample 1Mpx changes the base RT-DETR-B mAP by 2.9 points (bilinear 45.2 vs. nearest 42.3 vs. bicubic 43.1), which is larger than the reported 1Mpx SOTA margin of +1.4. Since prior methods such as RVT, S5-ViT-B, ASTMNet, SAST-CB, and ERGO-12 use their own preprocessing pipelines, the 50.1 mAP result is directly comparable only if the exact frame construction and evaluation protocols are identical; please document the protocols of the prior comparisons or re-evaluate them under a common protocol.","section":"§4.1 and Supplementary E.1, Table 10"},{"comment":"All results appear to be from single training runs without seeds or variance estimates. Given that the headline Gen1 margin is +2.3 and the 1Mpx margin is +1.4, please report standard deviation over at least three seeds for the principal comparisons, or otherwise justify that the reported differences exceed run-to-run variation.","section":"Tables 1, 3, 4, 7-12"}],"minor_comments":[{"comment":"RT-DETR-B Gen1 mAP is listed as 47.5 in Table 12, whereas Tables 1, 3, and 7 report 47.6; please reconcile the inconsistency.","section":"Table 12"},{"comment":"The text says W_down is integrated within the RNN, but the equations present it as a separate projection; please align the notation with the actual implementation.","section":"Equations (1)-(4)"},{"comment":"The heading describes 'interpolation methods on base detector performance for the 1Mpx dataset downsampling'; please clarify that the comparison is for the downsample interpolation only and not for other preprocessing steps.","section":"Table 10"},{"comment":"The marker for ERGO-12 reports a 100 ms runtime, but Table 1 shows a converted value; please state the conversion method in the caption so that the reader can interpret the time axis.","section":"Figure 1"},{"comment":"The end-to-end training experiments are mentioned but not quantified; either report the numbers or state explicitly that they are omitted for brevity.","section":"Supplement A.5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of a computer vision journal, and the authors are transparent about their ablations and limitations, which is commendable. The main risk is that the 1Mpx SOTA claim depends on protocol choices that the paper itself shows to be sensitive; I would encourage the editor to require multi-seed runs and a common-protocol comparison before accepting the headline numbers. There are no concerns about citation patterns or novelty disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the take: the core trick — train a mainstream detector on 50 ms stacked histograms, freeze it, bolt on ConvLSTM adapters — is real and works. On Gen1 the +2.3 mAP over the prior best is credible, and the YOLOX generalization plus the placement ablations are honest, useful work. But the 1Mpx claim (+1.4) is not as solid as the abstract implies. The paper's own supplementary shows that changing the downsampling interpolant moves the base detector by 2.9 mAP (Table 10), and extending the clip length from 5 to 10 frames adds 0.3 (Table 11). That 0.3 is 21% of the reported margin, and the protocol differences across prior methods (resolution, evaluation details) are not fully controlled. No error bars or multi-seed runs are reported anywhere. So the 1Mpx SOTA should be read as 'likely competitive, not established.'\n\nWhat is actually new is the demonstration that a frozen RT-DETR on plain stacked histograms already reaches RVT territory on Gen1 (47.6 vs 47.2). The two-stage latent-space adaptation with recurrent memory is a sensible transfer of LoRA/adapter ideas to temporal vision, and the ablation showing low-level features benefit most is a nice, non-obvious insight. The paper is clearly written and the method is simple enough to reproduce. The code link is a plus.\n\nThe main soft spot is evaluation protocol. Clip length is tuned on the same 1Mpx benchmark used for the headline claim, and the interpolation sensitivity means the comparison with prior work rests on choices that are not standardized across the field. The runtime numbers for prior methods are also converted via FLOPS/throughput approximations — rough, but the authors do flag it. One minor criticism: the 'static objects disappear' story is motivating, but the paper doesn't quantify how much of the temporal gain comes from that specific failure mode versus generic temporal smoothing.\n\nWho is this for? Anyone working in event-based detection or temporal adaptation of mainstream detectors. It deserves a serious referee, not a desk reject. The reviewer should ask for seed variance, a per-baseline protocol table, and either a fixed-protocol comparison or a direct reimplementation of the strongest baseline. I would not yet cite the 1Mpx number as ground truth, but the Gen1 result and the I2EvDet framework are worth engaging with. Conditional accept with revisions is the right outcome.","headline":"Solid two-stage recipe and a genuine Gen1 SOTA; the 1Mpx margin is within protocol noise.","tokens_in":689,"tokens_out":887,"would_cite":true,"duration_ms":68071,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Frozen image detector plus tiny memory beats event-camera specialists.","keywords":["event-based vision","object detection","RT-DETR","latent space adaptation","ConvLSTM","temporal memory","Gen1 dataset","1Mpx dataset"],"falsifier":"Find or construct a long stationary-sequence clip from Gen1 or 1Mpx where the vehicle stays still for more than ten seconds with dense traffic, and run EvRT-DETR: if its detections of cars and pedestrians drop to near-zero once the recurrent memory window (about 10 frames) expires, the paper's central claim that ConvLSTM memory maintains object persistence during event silence is wrong.","tokens_in":18178,"feed_emoji":"⚡","tokens_out":7070,"duration_ms":65157,"temperature":0.7,"pith_summary":"Event-based cameras produce sparse asynchronous streams, and object detection on them has typically required bespoke representations and architectures. This paper claims that a mainstream image detector, RT-DETR, trained on nothing more than 50 ms stacks of event histograms, already matches specialized event-camera methods, and that adding small recurrent memory modules to the frozen detector's encoder features makes it the new state of the art on both standard automotive benchmarks, Gen1 (52.7 mAP, +2.3) and 1Mpx (50.1 mAP, +1.4). The framework, called I2EvDet, keeps the base detector frozen and trains only lightweight ConvLSTM adapters inserted at three feature scales, so the improvement comes with a few extra milliseconds of inference time. For a reader, the point is that event vision may not need custom architectures: a general recipe for turning any image detector into a temporal one could transfer to other asynchronous visual domains.","feed_headline":"Adapting a frozen image detector beats event-camera specialists","feed_subtitle":"Adding small recurrent memory modules to a frozen RT-DETR encoder lifts Gen1 mAP to 52.7 and 1Mpx to 50.1.","key_machinery":"The load-bearing component is the Temporal Adaptation Module: a ConvLSTM unit (a convolutional recurrent network) with down/up projection matrices, placed at each of the three encoder scales E3, E4, E5 of a frozen RT-DETR and combined with the spatial features through a residual addition with learnable scaling (ReZero-style). Together with the input representation—a Stacked 2D Histogram that packs events from a 50 ms window into a 20-channel image (ten 5 ms bins × two polarities)—this module is what lets the image detector ingest temporal context without retraining its spatial weights. The authors find the lowest feature scale contributes most, and a 4× reduction of the recurrent hidden dimension costs only 0.6 mAP while cutting trainable parameters to 2.3 M, a LoRA-like regime.","core_discovery":"The central discovery is that the latent representation space of a frozen image detector is a sufficient substrate for temporal event reasoning. After training RT-DETR on stacked 2D histograms of event data (20 channels: 2 polarities × 10 five-millisecond bins), the paper freezes its backbone and encoder and inserts three ConvLSTM modules, one per encoder feature scale, coupled through residual connections with learned scaling. These adapters give the model a memory that keeps detecting objects after they stop generating events—when the camera is stationary, event streams go quiet and frame-only detectors fail—and this accounts for the full gain, from 47.6 to 52.7 mAP on Gen1 and from 45.2 to 50.1 mAP on 1Mpx. The paper also reports that the same recipe improves YOLOX detectors by 4.4–6.4 mAP, indicating the mechanism is not specific to transformers.","pith_inferences":["If the simple 50 ms histogram is really enough, then even lighter detectors (mobile backbones, knowledge-distilled variants) should inherit the same +5 mAP gain from the adapters; that is a cheap experiment the paper did not run.","The 1Mpx result is sensitive to the downsampling interpolant (nearest vs bilinear changes base mAP by 2.9 points), suggesting the preprocessing pipeline, not the architecture, may set the ceiling; a testable extension is to learn the downsampling or representation jointly with the adapters.","The same frozen-detector-plus-recurrent-adapter recipe could transfer to other asynchronous sensors (tactile, audio, radar-like streams) wherever data arrive as sparse timed events rather than dense frames.","One could replace ConvLSTM with a state-space or attention-based memory to test whether recurrence is essential for object persistence or just sufficient."],"forward_implications":["If the claim holds, event-based object detection no longer needs specialized architectures: a frozen mainstream detector plus small recurrent adapters is enough to set new benchmarks.","The I2EvDet recipe generalizes across detector families: applying it to YOLOX yields consistent gains of 4.4–6.4 mAP on Gen1, not just to RT-DETR.","The temporal memory directly addresses the 'static object disappears when camera stops' failure mode of event cameras, as shown in the stop-and-go automotive sequences.","The framework is parameter-efficient in a LoRA-like way: using hidden dimension 64 (2.3 M trainable parameters) still beats all prior methods, so deployment cost is low."],"supporting_citations":[{"why":"Defines the RT-DETR architecture (with ResNet-18/50 backbones) that the paper freezes and adapts; provides the baseline detector.","marker":"[38]"},{"why":"Supplies the frame construction parameters (50 ms windows, 10 bins) and the RVT recurrent baseline whose ConvLSTM choice the adaptation borrows.","marker":"[12]"},{"why":"ERGO-12 is the prior state-of-the-art on Gen1 that EvRT-DETR surpasses by +2.3 mAP, setting the benchmark to beat.","marker":"[41]"},{"why":"Introduces the 1Mpx/Gen4 dataset, the RED baseline, and the Prophesee evaluation toolbox used for all reported metrics.","marker":"[28]"},{"why":"Introduces the Gen1 dataset with its sparse manual annotations, one of the two standard testbeds.","marker":"[6]"},{"why":"Defines the ConvLSTM cell that constitutes the temporal adaptation module inserted in the frozen encoder.","marker":"[33]"},{"why":"AEC baseline and prior work on adaptive event conversion that the paper compares against and cites for representation-choice and interpolation considerations.","marker":"[25]"},{"why":"LoRA, the parameter-efficient adaptation technique that motivates the low-rank, minimal-capacity design of the temporal adapters.","marker":"[18]"}],"fun_headline_variants":["Frozen image detector learns event vision with tiny memory boost","Small adapters turn frozen RT-DETR into top event-camera detector","Memory modules unlock event-based detection in frozen image networks","Event cameras: frozen image detector + 3 ConvLSTMs beats specialists","RT-DETR frozen, then given memory, tops event-detection benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire result rests on the premise that a fixed 50 ms stacked histogram—ten 5 ms bins, resized and padded to RT-DETR's input shape—preserves enough of the event stream's information for automotive detection; the interpolation experiments on 1Mpx show that even the downsampling filter can move base mAP by 2.9 points, so if a richer temporal representation (time surfaces, voxel grids) were necessary for harder scenes, the reported gains would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Frozen image detector learns event vision with tiny memory boost","Small adapters turn frozen RT-DETR into top event-camera detector","Memory modules unlock event-based detection in frozen image networks","Event cameras: frozen image detector + 3 ConvLSTMs beats specialists","RT-DETR frozen, then given memory, tops event-detection benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000451,"raw_usage":{"total_tokens":2314,"prompt_tokens":1030,"completion_tokens":1284,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":1192}},"tokens_in":646,"tokens_out":1284,"duration_ms":10966,"temperature":1.0,"reasoning_tokens":1192,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:59:46.614319+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find or construct a long stationary-sequence clip from Gen1 or 1Mpx where the vehicle stays still for more than ten seconds with dense traffic, and run EvRT-DETR: if its detections of cars and pedestrians drop to near-zero once the recurrent memory window (about 10 frames) expires, the paper's central claim that ConvLSTM memory maintains object persistence during event silence is wrong.","supporting_citations":[{"cited_title":"DETRs beat YOLOs on Real-time object detection","cited_arxiv_id":null,"evidence_quote":"Defines the RT-DETR architecture (with ResNet-18/50 backbones) that the paper freezes and adapts; provides the baseline detector."},{"cited_title":"Recurrent vi- sion transformers for object detection with event cameras","cited_arxiv_id":null,"evidence_quote":"Supplies the frame construction parameters (50 ms windows, 10 bins) and the RVT recurrent baseline whose ConvLSTM choice the adaptation borrows."},{"cited_title":"From chaos comes order: Ordering event representations for object recognition and detection","cited_arxiv_id":null,"evidence_quote":"ERGO-12 is the prior state-of-the-art on Gen1 that EvRT-DETR surpasses by +2.3 mAP, setting the benchmark to beat."},{"cited_title":"Learning to detect objects with a 1 megapixel event camera","cited_arxiv_id":null,"evidence_quote":"Introduces the 1Mpx/Gen4 dataset, the RED baseline, and the Prophesee evaluation toolbox used for all reported metrics."},{"cited_title":"Convolutional LSTM network: A machine learning approach for precipitation nowcasting","cited_arxiv_id":null,"evidence_quote":"Defines the ConvLSTM cell that constitutes the temporal adaptation module inserted in the frozen encoder."},{"cited_title":"Better and faster: Adaptive event conversion for event-based object detection","cited_arxiv_id":null,"evidence_quote":"AEC baseline and prior work on adaptive event conversion that the paper compares against and cites for representation-choice and interpolation considerations."},{"cited_title":"Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen","cited_arxiv_id":null,"evidence_quote":"LoRA, the parameter-efficient adaptation technique that motivates the low-rank, minimal-capacity design of the temporal adapters."}],"review_version":1}