{"id":"960dc253-4b6d-487e-9559-2a5ebf89ee29","arxiv_id":"2608.01948","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Event ActivityNet converts 3,263 untrimmed ActivityNet videos into simulated event voxels with temporal annotations and captions, enabling event-based recognition and causal online localization at 106.94 hours of scale.","lead":"The authors built a simulated-event video benchmark from 3,263 untrimmed ActivityNet videos, with matched 5-bin and 9-bin event-voxel representations plus temporal annotations and captions. It lets researchers train and test event-based action recognition and online temporal action localization on long videos without collecting native event-camera data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Load-bearing gap: simulated-voxel transfer to native event cameras is validated only for trimmed recognition, not for the causal online TAL task that is a core benchmark offering.","rationale":"The reader's weakest assumption identifies the same load-bearing gap: native-event validity is tested only for recognition, not for online TAL. The paper is internally consistent and unusually transparent about this limitation (Sec. E, App. B.4), so I do not read it as a defect in construction or a fraud concern. It is a genuine external-validity risk for the central claim that the benchmark supports long-horizon event modeling, because the signature online-TAL protocol depends most on temporal event structure that the simulation explicitly does not reproduce. I considered the seconds-to-voxel timing approximation as an alternative concern; it is real but narrower and can be checked directly from source-video PTS metadata. The native online-TAL transfer gap is more consequential: it determines whether the benchmark's main task protocol has any demonstrated connection to real event cameras. I therefore keep the reader's CONDITIONAL verdict, conditioned on native online-TAL validation or a sharpened scope statement.","tokens_in":29405,"tokens_out":5560,"duration_ms":71074,"concrete_test":"Collect or assemble a native-event untrimmed dataset with instance-level temporal annotations—e.g., 10–20 hours of DVS/event-camera recordings covering a subset of Event ActivityNet classes with human-annotated action boundaries—and run the matched online TAL protocol from Sec. 5.2 under three budgets: EA-pretrain + native fine-tuning, joint scratch, and target-only. Report Avg. mAP at tIoU {0.3,0.4,0.5,0.6,0.7} and the supervision-budget ordering. If EA pretraining does not consistently beat target-only and joint scratch on native online TAL, the central transfer claim is limited to recognition, and the benchmark's online-TAL protocol should be labeled an unvalidated proxy rather than a general event-camera benchmark.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Event ActivityNet's headline value is a scalable simulated-event benchmark for long-horizon event modeling, with causal online TAL as a signature task (Sec. 3.1, 5.2). The only native-camera validation is Table 5, which covers trimmed recognition on HARDVS and SeAct. The online TAL protocol is evaluated entirely on discrete voxels generated from decoded frame order with a fixed contrast threshold c=0.2 and no per-frame presentation timestamps (App. B.4, Eq. S6). Simulation therefore lacks native asynchronous statistics, microsecond timing, and sensor noise; Sec. E acknowledges this. The load-bearing point is not that the simulated data are internally wrong, but that the paper's most consequential transfer claim—staged pretraining beats target-only and joint-from-scratch—is established only for recognition, whereas the benchmark's online-TAL conclusions are the part that most depends on event-stream temporal structure. If the transfer ordering reverses or disappears on native causal streaming data, the benchmark's utility for real event-camera online TAL is unverified despite the recognition transfer result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Event ActivityNet, a simulated-event benchmark derived from ActivityNet v1.3 and ActivityNet Captions, with 3,263 untrimmed videos, 200 action classes, 106.94 hours of video, matched 5-bin and 9-bin event-voxel representations, temporal action annotations, and timestamped captions. It defines three tasks: annotated-segment action recognition, auxiliary event–language alignment, and causal online temporal action localization (TAL). Voxels are generated directly from decoded RGB frames using a fixed contrast threshold, with rational-rate metadata for approximate time mapping and a center-frame LPIPS reconstruction audit. Reference baselines include adaptive event framing (AEF-Split&Merge), contrastive prompt/caption alignment, and SimOn/MATR online TAL with RGB-only, event-only, and RGB–event fused inputs. Main empirical results are: recognition Top-1 improves from 52.25 to 66.42 across progressive scales, online TAL avg mAP improves from 21.7 to 29.0, 5-bin vs 9-bin granularity has small effects, and staged Event ActivityNet pretraining outperforms target-only and joint-from-scratch training on HARDVS and SeAct recognition across 10/50/100% supervision budgets. The paper explicitly disclaims native-camera equivalence and repeatedly states that native evaluation remains necessary for deployment-oriented conclusions.","tokens_in":29595,"tokens_out":9800,"duration_ms":110940,"significance":"If the results hold, the benchmark fills a genuine gap: large-scale untrimmed event-based action understanding with temporal annotations, captions, and a causal online TAL protocol. The evaluation design is careful in several respects: 3-run means, fixed schedules, final-checkpoint reporting without early stopping, matched modality rows sharing the same SimOn head, public-dataset sanity checks, and a held-out generator comparison. The paper is also transparent about limitations in Sec. E and Appendix B.4/B.9. I found no circularity in the main evaluation: AEF-SM thresholds are calibrated on the training split and evaluated on the fixed validation split, and 5/9-bin comparisons vary only temporal grouping. The main risk is external validity: transfer to native event cameras is demonstrated only for trimmed recognition, not for the online TAL task that is a signature offering. This is a scope limitation rather than an internal inconsistency, but it needs to be reflected precisely in the paper's claims.","major_comments":[{"comment":"The abstract states that staged Event ActivityNet pretraining followed by native-event fine-tuning 'consistently outperforms target-only and joint-from-scratch training across multiple supervision budgets.' As written this suggests a general result, but Table 5 only supports it for trimmed recognition on HARDVS and SeAct. Causal online TAL—a core task defined in §3.1—is never validated on native event data, and §E explicitly disclaims native equivalence. Please restrict the transfer claim to recognition, or provide native online-TAL evidence. The current wording overstates the empirical support, even though the limitation is acknowledged elsewhere.","section":"Abstract; §5.3, Table 5"},{"comment":"The recognition protocol says 'we select the target ActivityNet v1.3 action instance' but never defines how the target is chosen or how multi-instance videos are handled. Is one annotated segment used per video, or are all action instances used as recognition samples? This changes the effective training-set size and directly affects the meaning of every recognition number in Tables 2, 4, and Figure 3. Please specify the selection rule and report the number of recognition samples per split.","section":"§3.1"},{"comment":"The Event ActivityNet pretraining protocol for the native-transfer experiments is not specified. Which nested scale (diagnostic, Small, Medium, or Large) is used, for how many epochs, and with which objective (with or without caption alignment)? Since §5.3 shows large performance differences across scales, the transfer gains in Table 5 cannot be reproduced or interpreted without this information. Please report the exact pretraining protocol, or state that it is fixed across all native datasets and supervision budgets.","section":"§5.3, Table 5"}],"minor_comments":[{"comment":"The pipeline diagram labels the LPIPS check as a 'High-quality Data Guarantee,' but Table S7 shows that zero videos exceed the δ=0.30 threshold and the text describes δ as a soft audit flag rather than an exclusion rule. Suggest rewording to avoid implying a stricter quality gate.","section":"§3.2, Fig. 2"},{"comment":"The paragraph reports recognition rising 'from 52.25 to 66.42' but does not explicitly state which scale corresponds to the lower value. Since the figure labels include 10h, Small, Medium, and Large, please clarify whether 52.25 is the diagnostic scale or the Small scale.","section":"§5.3, Fig. 3"},{"comment":"The deterministic RNG reset (seed 42 per simulator call) means that equal-shaped videos reuse the same pseudorandom perturbations for base noise, initial residual, and hot pixels. This is disclosed in Sec. E, but because it is a data-generation property, it should also be stated in the main construction section with a note on the expected impact on cross-video comparisons.","section":"§B.4, Sec. E"},{"comment":"The streaming-step sweep rows clearly specify Δ∈{0.25,0.5,1.0}s, but the subsequent encoder/fusion rows do not state which default step is used. Please add the default step to the table or the caption.","section":"Table 3"},{"comment":"The seconds-to-voxel mapping is explicitly approximate for variable-frame-rate streams. This is properly acknowledged; a one-sentence note in the main text that annotation misalignment can affect online TAL absolute numbers would help readers weigh the reported mAP values.","section":"§5.2, Eq. S6"}],"recommendation":"major_revision","confidential_remarks":"This is a resource/benchmark paper with a defensible central claim. The main issues are wording overreach in the transfer claim and missing protocol specifications (recognition sampling rule and EA pretraining setup), both of which are fixable without changing the core benchmark. I would not reject; after the authors clarify these points and tighten the abstract, the paper would be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The first thing to know: this is a solid dataset paper, not a modeling paper. The contribution is Event ActivityNet, 3,263 untrimmed videos derived from ActivityNet, with matched 5/9-bin voxel representations, temporal annotations, captions, and fixed protocols for segment recognition and causal online TAL. That fills a real gap — every existing event-action benchmark is trimmed clips. The release structure (nested scales, manifests, checksums, HDF5 validation) is above field norm, and the baselines are carefully controlled: fixed schedules, final-checkpoint reporting, 3-run means, matched modality comparisons.\n\nThe paper is unusually honest about what it is not. Section E and B.4 explicitly say the voxels do not reproduce native sensor noise, microsecond timing, or per-frame PTS, and that the seconds-to-voxel mapping is approximate for variable frame rate. The LPIPS diagnostic is openly a soft audit, not a guarantee. That honesty is real credit.\n\nThe soft spot is the one the stress-test flags: the only native-camera validation is recognition transfer (Table 5), while the online TAL protocol — the signature task — is evaluated entirely on simulated voxels. The staged-pretraining transfer claim is about recognition, and the paper does not overclaim it for TAL. So the gap is not a logical flaw; it is an unverified extension. A reader wanting to deploy on native event cameras for online TAL should treat that as open, as the authors do. Minor issue: the per-call seed reset (App B.4) makes the noise deterministic across equal-shaped calls; they disclose it, but it means the stochastic component is really a fixed artifact, which slightly weakens the diversity of the simulated data. Likewise the caption alignment is weak (37.5% of retained matches below 0.3 tIoU), but they handle it with ablations and downweighting.\n\nOverall: the central claim — this is a scalable, auditable simulated-event testbed for untrimmed action understanding — holds up. It deserves a serious referee. I'd send it to review and bring it to reading group if anyone works on event-based or online TAL.","headline":"A serious, honestly scoped simulated-event benchmark for untrimmed action understanding; the native-camera transfer gap is real but the paper says so itself.","tokens_in":30222,"tokens_out":1414,"would_cite":true,"duration_ms":15612,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Event ActivityNet establishes a scalable simulated-event benchmark for untrimmed action understanding, and shows that staged pretraining on it transfers to native event recognition.","keywords":["event-based vision","action recognition","temporal action localization","untrimmed video benchmark","simulated event voxels","online TAL","video-to-voxel simulation","transfer learning"],"falsifier":"Re-run the staged-pretraining versus target-only comparison from Table 5 on a native untrimmed event-action dataset with temporal action annotations: if simulated-event pretraining no longer beats target-only training (or hurts), or if the online-TAL conclusions from simulated streams reverse on native streams, the central transfer claim fails. A more targeted check: find a video where the released LPIPS audit passes but the per-frame timestamp mapping misaligns annotated boundaries, which would expose the approximate frame-index-to-seconds assumption.","tokens_in":29219,"feed_emoji":"⚡","tokens_out":6954,"duration_ms":65520,"temperature":0.7,"pith_summary":"The paper introduces Event ActivityNet, a simulated-event testbed that converts 3,263 human-annotated untrimmed videos (200 action classes, 106.94 hours) into matched 5-bin and 9-bin event-voxel representations, reusing ActivityNet's temporal action annotations and timestamped captions as supervision. It establishes reference protocols and baselines for annotated-segment action recognition, auxiliary event–language alignment, and causal online temporal action localization, demonstrating that progressive scaling raises recognition Top-1 from 52.25 to 66.42 and online localization average mAP from 21.7 to 29.0. Its central empirical claim is that staged pretraining on these simulated event voxels, followed by fine-tuning on native event data, consistently outperforms both target-only training and joint-from-scratch training across 10%, 50%, and 100% supervision budgets. The authors explicitly state that the simulation does not reproduce native sensor noise, microsecond timestamps, or high dynamic range, and that native-camera evaluation remains essential for deployment-oriented conclusions.","feed_headline":"Simulated event streams cover 106 hours of untrimmed action","feed_subtitle":"Pretraining on the simulated voxels beats native-only training on real event data, by up to 7.7 points.","key_machinery":"The load-bearing object is the event-voxel tensor: each X_q in R^(B×H×W) quantizes signed log-intensity changes between consecutive decoded grayscale frames into B temporal bins using fixed contrast thresholds c+=c-=0.2, with no frame-rate conversion, interpolation, or continuous raw-event list. This direct video-to-voxel construction is what lets the benchmark reuse ActivityNet's human annotations at scale. Around it, three mechanisms carry the experiments: AEF-Split&Merge, a nonparametric adaptive windowing procedure that splits and merges voxel windows by activity-rate and polarity-histogram dissimilarity; a contrastive objective that aligns event embeddings to frozen class prompts and te","core_discovery":"The central claim is that long-horizon event-based action understanding can be studied at scale through discrete event voxels generated directly from non-interpolated RGB frames, without expensive native event-camera recordings or intermediate high-frame-rate video. Each voxel tensor accumulates adjacent-frame log-intensity changes into B temporal bins (B=5 or 9) at a fixed contrast threshold c=0.2, and the dataset augments these tensors with per-video rational frame-rate metadata for approximate time mapping, inherited human action boundaries, timestamped captions, and a soft reconstruction-based quality audit at annotated action centers. On this testbed, reference models show steady gains","pith_inferences":["A natural extension the paper leaves implicit is testing whether staged simulated-event pretraining also transfers to native causal online temporal action localization, not just recognition; the paper's own limitation statement highlights that this transfer direction is untested.","The benchmark's curation enriches for videos whose captions mention motion or whose first frames are dark, and the paper reports a source-to-subset class-frequency correlation around r=0.64; downstream users should probe whether scale and transfer conclusions hold on the full ActivityNet distribution.","Because the simulator resets its random seed per call, equal-shaped calls reuse the same pseudorandom perturbation patterns; this artifact could affect experiments that rely on stochastic variation, and a sensitivity analysis over seeds would clarify its impact.","If the transfer advantage persists on larger native datasets, simulated-event pretraining could lower the annotation cost barrier for event-camera action understanding, since the pretraining corpus is derived from freely available RGB video with existing human labels."],"forward_implications":["If correct, long-horizon event-based action understanding can be evaluated and developed at scale without native event recordings, making controlled comparisons of temporal discretization and streaming protocols possible on 3,263 untrimmed videos.","Staged pretraining on these simulated event voxels is a viable recipe for native event recognition: it beats target-only and joint-from-scratch training at every supervision budget tested (10%, 50%, 100%), with gains of 2.61 to 7.65 Top-1 points.","Increasing training data on this benchmark yields consistent improvements: recognition Top-1 rises from 52.25 to 66.42 and online TAL average mAP from 21.7 to 29.0 as scale grows from about 10 to 106.94 hours.","Voxel granularity (5-bin vs 9-bin) changes online localization metrics by up to 1.8 mAP@30 while leaving recognition nearly unchanged, so temporal discretization choices are consequential for streaming tasks.","Causal RGB–event fusion via cross-attention outperforms matched RGB-only and event-only online TAL, indicating that event cues add measurable value under strict prefix-only inference."],"fun_headline_variants":["Simulated events beat native data for action understanding","106-hour simulated event benchmark for untrimmed actions","Pretraining on simulated events improves action localization","Event ActivityNet: large-scale simulated event benchmark","RGB-derived event voxels power 106-hour action benchmark"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The benchmark's value depends on the assumption that discrete event voxels simulated from RGB frames at a fixed contrast threshold preserve enough native event-stream structure that conclusions, especially the pretraining-transfer and online-localization results, carry over to real event cameras; the paper explicitly disclaims native equivalence and tests transfer only for recognition.","fun_headline_variants_meta":{"raw":{"variants":["Simulated events beat native data for action understanding","106-hour simulated event benchmark for untrimmed actions","Pretraining on simulated events improves action localization","Event ActivityNet: large-scale simulated event benchmark","RGB-derived event voxels power 106-hour action benchmark"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000175,"raw_usage":{"total_tokens":1156,"prompt_tokens":808,"completion_tokens":348,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":275}},"tokens_in":552,"tokens_out":348,"duration_ms":4156,"temperature":1.0,"reasoning_tokens":275,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T17:52:58.788283+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the staged-pretraining versus target-only comparison from Table 5 on a native untrimmed event-action dataset with temporal action annotations: if simulated-event pretraining no longer beats target-only training (or hurts), or if the online-TAL conclusions from simulated streams reverse on native streams, the central transfer claim fails. A more targeted check: find a video where the released LPIPS audit passes but the per-frame timestamp mapping misaligns annotated boundaries, which would expose the approximate frame-index-to-seconds assumption.","supporting_citations":[],"review_version":1}