{"id":"27e67d0f-c9da-4010-ba07-c8cde04ba921","arxiv_id":"2608.04865","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"EventKitchen provides a 5.5-hour egocentric stereo event camera dataset of unscripted cooking in 13 kitchens, with 10,762 action segments and 13,482 bounding boxes for event-based action recognition, detection, and depth estimation.","lead":"This paper introduces EventKitchen, a dataset of natural cooking activities recorded with a wearable stereo event camera system in 13 kitchens. It offers 5.5 hours of synchronized event, RGB, depth, and IMU data with action, object, and depth annotations for multiple vision tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benchmark's ground truth hinges on projecting RGB-depth annotations onto event cameras without event-domain validation; sub-2px calibration error does not rule out systematic label bias.","rationale":"The paper is a solid dataset contribution: it is genuinely real-world and egocentric, releases calibration matrices, preprocessing code, and baselines for three tasks, and the kitchen-level split is a sensible evaluation protocol. The strongest claim, however, is only as strong as the ground truth. The authors' own Section 3.3 acknowledges that annotators do not label event data directly because it is too challenging; this is exactly why the projection must be independently checked. The reported intrinsic and extrinsic errors are a necessary but not sufficient condition for label correctness. My proposed event-domain re-labeling is a direct, low-cost audit. The supplementary note that the IMU is not fully calibrated and is therefore excluded from the main paper is a real inconsistency with the abstract's 'synchronized RGB, depth, and IMU data' claim, but IMU is not used by any baseline or task, so it is secondary to the central dataset claim. Since the concern is about validation rather than an observed failure, the reader's CONDITIONAL verdict remains appropriate; a small validation study would either clear the pipeline or expose a bias that would need correction before the benchmark is fully endorsed.","tokens_in":20400,"tokens_out":7934,"duration_ms":85797,"concrete_test":"Take about 200 annotated D-RGB frames stratified across sessions, kitchens, and the 12 object classes. At each annotation timestamp, reconstruct left and right event images with E2VID, have two annotators label boxes directly on those reconstructions, and compute per-class IoU and AP50 between the projected boxes and event-domain human boxes. For depth, additionally compare projected depth against manual stereo correspondences on a rigid calibration object placed at 0.5-1.5 m. If AP50 for fork/spoon or median IoU falls below about 0.5, the projection pipeline is biased and the labels require re-validation before the benchmark is fully trusted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3's annotation pipeline is the load-bearing link between human annotations and the event-domain labels used by all baselines. Boxes are drawn on D-RGB frames, unprojected through RealSense depth, and re-projected into left/right event views; depth ground truth is rectified through the same projection. The reported <2 px reprojection errors are computed from checkerboard calibration, not from annotated objects, and do not bound the full-pipeline error. Depth noise (<2% at 2 m, roughly 40 mm) plus an axis-aligned D-RGB box that inevitably includes background or occluding 3D points can systematically enlarge or shift the projected box, especially for the small fork/spoon classes that already obtain near-zero AP. Temporal alignment between the 15 fps depth/D-RGB stream and asynchronous events is asserted via ROS timestamps but not validated, and a few-ms offset matters for 50 ms detection windows. Figure 5 compares left/right class distributions, not geometric accuracy. No human labels were collected directly on event frames or event reconstructions, so systematic label bias would be invisible in the reported results.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces EventKitchen, a 5.5-hour stereo event camera dataset of unscripted cooking activities, recorded with a wearable helmet equipped with two Prophesee Gen4 event cameras, two CMOS RGB cameras, an Intel RealSense depth camera, and an IMU. Ten participants were recorded in 13 kitchens. Human annotations of 10,762 action segments and 13,482 bounding boxes are made on D-RGB frames and projected into the left and right event camera views via calibration matrices; depth maps are also rectified into event camera coordinates. The authors train baseline models for event-based action recognition (TSM, Swin), object detection (YOLOv10, RVT, EvRT-DETR), and stereo depth estimation (SE-CFF, FoundationStereo), reporting low performance as evidence of task difficulty, and claim that EventKitchen is the first large-scale, real-world, egocentric, stereo event camera benchmark supporting multiple tasks.","tokens_in":20620,"tokens_out":4339,"duration_ms":46536,"significance":"If the ground truth is reliable, EventKitchen is a valuable community resource: it fills a genuine gap in human-centric daily-life event vision, uses real unscripted activities, combines stereo events with RGB-D and IMU data, provides annotations for three tasks, and includes reproducible baseline configurations and a public toolkit. The low baseline numbers are informative for the field. However, the benchmark's value hinges on the accuracy of projecting RGB-D annotations and depth maps into the event domain; without independent event-domain validation, the risk of systematic label bias is substantial. The manuscript is strongest in its detailed recording setup, transparent participant statistics, and the public release of data and tools.","major_comments":[{"comment":"The central load-bearing step is the projection of bounding boxes and depth maps from the D-RGB/depth domain into the left and right event camera views, but this projection is never validated on event-domain data. The reported <2 px reprojection errors come from checkerboard calibration, not from annotated objects, and do not bound the full-pipeline error: depth noise (<2% at 2 m, roughly 40 mm), axis-aligned 3D boxes that inevitably include background or occluding points, and temporal misalignment can systematically enlarge or shift projected boxes. This is especially relevant for the fork and spoon classes, which obtain near-zero AP and could reflect label bias rather than genuine task difficulty. The authors should provide an independent validation, e.g., human labels collected directly on event reconstructions for a subset of frames, and report agreement against the projected boxes.","section":"Section 3.3, Fig. 3"},{"comment":"The manuscript contains a material contradiction about the IMU. The main paper lists '5.5 hours of 6-axis IMU data at 200 fps' and shows an IMU column in Fig. 1, while the supplementary material states that the IMU is physically present but 'excluded from the main paper because its calibration relative to the camera module has not been fully validated' and the supplementary Fig. 1 describes a 9-axis IMU. As written, a user cannot know whether IMU data are part of the released dataset or whether they are synchronized and calibrated. This inconsistency must be resolved in the main text and the released metadata.","section":"Section 3.1, Section 3.4, Supplementary Section 1"},{"comment":"Temporal alignment between the 15 fps D-RGB/depth stream and the asynchronous event streams is asserted via ROS timestamps but is not validated. A few milliseconds of offset is significant for the 50 ms detection windows used in Section 4.2 and for fast actions such as stirring, so the authors should report a concrete synchronization validation, for example by cross-correlating motion onset between D-RGB frames and event rates, or by showing that projected boxes track moving objects in event reconstructions with bounded lag.","section":"Section 3.1, 'Synchronization'"}],"minor_comments":[{"comment":"The number of depth maps is inconsistent: Section 3.4 reports 297,547 raw depth maps while Table 1 reports 297,457. Please correct the discrepancy.","section":"Section 3.4 vs. Table 1"},{"comment":"The caption states that the figures 'demonstrate our projection pipeline is not biased' because the left and right cameras have similar numbers of bounding boxes per class, but equal class counts do not demonstrate geometric accuracy; this wording should be softened or replaced with a geometric validation.","section":"Fig. 5 caption"},{"comment":"The metric AP05 (IoU threshold 0.05) is used without definition or justification; please define it and explain its role, since it is not a standard COCO metric.","section":"Section 4.2"},{"comment":"The implementation list contains the typo 'FundationStereo' for 'FoundationStereo'; please fix this in the final version.","section":"Supplementary Section 3"},{"comment":"The comparison of verb frequency rankings with EPIC-KITCHENS is somewhat opaque because 'Rinse' and 'Soap' are counted separately in EventKitchen but merged into 'Wash' for EPIC-KITCHENS; a short clarifying note in the table caption or text would help.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong and useful dataset contribution, and I would be willing to accept it after the authors provide an independent validation of the cross-modal projection pipeline, resolve the IMU inconsistency, and add a temporal synchronization check. The main risk is not novelty but the integrity of the ground truth on which all three benchmark tasks depend."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"EventKitchen is worth knowing about. It is the first real-world, egocentric, stereo event dataset for unscripted daily activities, and that gap was real. The paper puts it together cleanly: 5.5 hours from 10 participants in 13 kitchens, 110 sequences, 10,762 action segments and 13,482 boxes, plus synchronized RGB, depth, and IMU. The baselines across action recognition, object detection, and stereo depth are honestly run with public models and fixed seeds. Compared to N-EPIC-Kitchens, which is synthetic, and the scripted lab datasets, this is a clear step forward. The low baseline numbers are framed as benchmark difficulty, which is fair.\n\nThe soft spot is exactly where your stress test points. The bounding boxes and depth ground truth are human-annotated on D-RGB frames and then projected into event views through depth-based 3D unprojection. The reported <2 px calibration errors come from checkerboards, not from annotated objects, and depth noise plus an axis-aligned box can systematically enlarge or shift the projected label, especially for small classes like fork and spoon that already sit near zero AP. Temporal sync between the 15 fps depth stream and asynchronous events is asserted via ROS timestamps but not validated, and that matters for 50 ms detection windows. This is not a dealbreaker for the dataset as a resource, but it is central for benchmark credibility. I would ask for a small set of human labels drawn directly on event reconstructions, or at least a geometric check on object boundaries, plus a discussion of where projected labels could fail.\n\nTwo smaller inconsistencies: Table 1 says 297,457 depth maps while Section 3.4 says 297,547; and the abstract lists synchronized IMU data while the supplement says the IMU is excluded because its calibration is not fully validated. Those need harmonizing.\n\nMy bottom line: this is a solid dataset contribution that deserves a serious referee. I would not desk reject. I would ask for the validation study and the consistency fixes before accepting. The central claim — that this is the first real stereo egocentric event dataset for natural human activities with multi-task annotations — holds up. The projection pipeline is the main thing to verify, and it is verifiable.","headline":"A genuinely useful real-world egocentric stereo event dataset for daily human activities, held back mainly by a cross-modal annotation pipeline that needs independent validation in the event domain.","tokens_in":21068,"tokens_out":2999,"would_cite":true,"duration_ms":33428,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EventKitchen is presented as the first large-scale stereo event camera dataset of unscripted human cooking, egocentrically recorded with synchronized RGB, depth, and IMU, and labeled for action recognition, object detection, and depth…","keywords":["event cameras","neuromorphic vision","egocentric vision","stereo depth estimation","action recognition","object detection","multi-modal dataset","EventKitchen"],"falsifier":"Select a random sample of event frames, have independent annotators draw bounding boxes directly on temporally aggregated event images, and compare those boxes with the dataset's projected labels by IoU and center distance; if the disagreement for small or fast objects is much larger than the reported sub-2-pixel reprojection error, the projection-based ground truth is systematically biased.","tokens_in":20285,"feed_emoji":"🍳","tokens_out":8008,"duration_ms":77463,"temperature":0.7,"pith_summary":"The paper introduces EventKitchen, a benchmark dataset claimed to be the first large-scale stereo event camera dataset recorded in a real-world, egocentric, unscripted daily-life setting. Ten participants wore a helmet with stereo event cameras, stereo RGB cameras, a depth camera, and an IMU while cooking in 13 kitchens, yielding 5.5 hours of synchronized streams. Human annotators labeled 10,762 action segments and 13,482 bounding boxes, and the authors provide 297,547 ground-truth depth maps plus calibration matrices. Baseline models for action recognition, object detection, and stereo depth estimation all perform far below their performance on automotive event datasets, which the authors take as evidence that EventKitchen poses genuinely new challenges for neuromorphic vision beyond autonomous driving.","feed_headline":"5.5 hours of stereo events capture unscripted cooking","feed_subtitle":"Ten cooks, 13 kitchens, and labels for action, objects, and depth make a benchmark for event vision beyond driving.","key_machinery":"The carrying mechanism is the multi-modal recording rig and, especially, the cross-modal label projection pipeline. Human annotations are made on RGB frames from the depth camera, where objects are easy to see; each annotated bounding box is lifted into 3D using the aligned depth map, then projected into both event cameras using intrinsic and extrinsic calibration matrices with reported reprojection error below two pixels. Depth maps are likewise projected and rectified into the event stereo frame, so ground truth for two independent event cameras is produced from a single annotation pass in the RGB-depth domain.","core_discovery":"The central claim is that EventKitchen fills a gap: no existing event camera dataset combines real-world capture, stereo event cameras, an egocentric viewpoint, unscripted natural actions, and support for multiple tasks in one benchmark. The dataset consists of 110 sequences from 10 participants across 13 kitchens, with stereo Prophesee Gen4 streams at 1280x720 resolution, synchronized CMOS RGB, depth, and IMU data, plus human annotations of 268 action classes and 12 kitchenware object classes. Baselines show top-1 action accuracy of 24.69% (Swin), object detection AP of 16.2% (YOLOv10), and stereo depth RMSE of 84.91 mm (SE-CFF), which the authors interpret as evidence that the benchmark is complex and that models specialized for automotive or scripted event data do not transfer well.","pith_inferences":["If the projection pipeline is sound, the same annotate-once-in-RGB-depth, project-to-events recipe could be reused to create multi-task event labels for other egocentric settings without requiring annotators to label sparse event streams directly.","The especially low detection performance on forks and spoons may partly reflect projection-induced label noise on small, fast, occluded objects rather than detector weakness alone; a direct annotation study on event frames could separate these effects.","Because the IMU stream is not yet calibrated to the camera module, as the supplementary material states, a natural extension is to complete that calibration and add egomotion estimation as a fourth benchmarked task.","The long-tail action distribution and unseen-kitchen split make EventKitchen a plausible testbed for few-shot learning and domain generalization in event vision, though the paper does not itself run those experiments."],"forward_implications":["Event-based action recognition can be studied on unscripted, long-tail actions in unseen kitchens; the low top-1 accuracy of strong frame-based baselines quantifies the open gap.","Object detection baselines trained for automotive events transfer poorly to egocentric kitchens, and small, fast-moving kitchenware such as forks and spoons is nearly undetected, so new event-specific detectors are needed.","Stereo depth estimation is feasible in near-field indoor scenes: SE-CFF reaches RMSE below the ground-truth scatter, while a zero-shot RGB foundation model fails on reconstructed event images.","The verb-frequency ranking of EventKitchen closely tracks an established unscripted egocentric cooking benchmark, supporting the paper's claim that the collected actions are natural rather than scripted.","The kitchen-level train/test split, with all test kitchens unseen during training, makes EventKitchen a testbed for generalization rather than memorization."],"supporting_citations":[{"why":"Supplies the unscripted egocentric cooking benchmark whose verb list and action-label format are adopted, and whose verb-frequency ranking is used to argue EventKitchen is natural.","marker":"[11]"},{"why":"Supplies the calibration method that reconstructs event streams into grayscale frames so standard calibration can be applied, enabling the cross-modal label projection.","marker":"[42]"},{"why":"Supplies the VIA annotation tool used to create the human action-segment and bounding-box labels.","marker":"[16]"},{"why":"Provides the SE-CFF event stereo depth baseline, trained from scratch and evaluated on EventKitchen.","marker":"[43]"},{"why":"Provides the RVT event-based object detector baseline, pretrained on automotive data and benchmarked on EventKitchen.","marker":"[24]"},{"why":"Provides the EvRT-DETR event-based detector baseline used to show transfer difficulty.","marker":"[63]"},{"why":"Provides the YOLOv10 frame-based detector baseline whose large performance drop from COCO to EventKitchen supports the difficulty claim.","marker":"[65]"},{"why":"Supplies the automotive 1Mpx event dataset used to pretrain the detection baselines and as the reference for their performance drop.","marker":"[46]"},{"why":"Supplies the event-to-video reconstruction method used to feed reconstructed frames to the zero-shot RGB stereo baseline.","marker":"[54]"}],"fun_headline_variants":["Unscripted cooking from 10 cooks in 13 kitchens","Stereo event camera dataset: egocentric kitchen tasks","Natural cooking, labeled: 10,762 actions in stereo","EventKitchen: 5.5 hours of unscripted human activity","Beyond driving: event vision for real cooking scenes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that labels made on RGB-depth frames and projected into event views through calibration are correct, so if depth noise, calibration error, or timestamp misalignment is larger than reported, every event-domain ground-truth label could be systematically wrong, and the paper gives no independent check of projected labels directly on event frames.","fun_headline_variants_meta":{"raw":{"variants":["Unscripted cooking from 10 cooks in 13 kitchens","Stereo event camera dataset: egocentric kitchen tasks","Natural cooking, labeled: 10,762 actions in stereo","EventKitchen: 5.5 hours of unscripted human activity","Beyond driving: event vision for real cooking scenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000624,"raw_usage":{"total_tokens":2893,"prompt_tokens":955,"completion_tokens":1938,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":1854}},"tokens_in":571,"tokens_out":1938,"duration_ms":14242,"temperature":1.0,"reasoning_tokens":1854,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:37:16.406596+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Select a random sample of event frames, have independent annotators draw bounding boxes directly on temporally aggregated event images, and compare those boxes with the dataset's projected labels by IoU and center distance; if the disagreement for small or fast objects is much larger than the reported sub-2-pixel reprojection error, the projection-based ground truth is systematically biased.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the RVT event-based object detector baseline, pretrained on automotive data and benchmarked on EventKitchen."}],"review_version":1}