{"id":"b7432904-8466-44ef-bd95-c336c8986784","arxiv_id":"2505.19169","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new pipeline for egocentric event-based 3D hand mesh reconstruction uses a hand segmentation module to suppress background events and outperforms prior methods on the synthetic N-HOT3D dataset.","lead":"EventEgoHands is the first method to reconstruct 3D hand meshes from egocentric event camera streams, filtering out background events caused by the wearer's motion. It reports large gains over prior event-based hand methods on a new synthetic dataset, N-HOT3D, supporting future AR/VR and low-light hand tracking.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Mask supervision and evaluation rely on HOT3D ground-truth projections; if the mask is trained on clean projections but evaluated on noisy v2e event frames, the segmentation gain may be overstated, and no held-out mask IoU or real-data check is reported.","rationale":"The reader identified the synthetic-data transfer concern as the weakest assumption, and I agree. The additional concern about mask supervision quality is a concrete, testable sub-instance of that transfer problem. The paper provides no error bars, no held-out mask IoU, and no real-data validation, so the central claim remains conditional on the synthetic-to-real transfer holding. The conditional verdict is appropriate; the concrete test would either strengthen or weaken that condition.","tokens_in":9775,"tokens_out":1212,"duration_ms":8125,"concrete_test":"Compute the hand segmentation mask IoU on the N-HOT3D held-out test set using the ground-truth projections, and then correlate mask accuracy with downstream R-AUC/MPJPE on test samples stratified by background-event density and hand motion. Additionally, run the trained model on a small set of real egocentric event sequences with manually labeled hand regions (or use a pre-trained RGB hand segmenter as a weak labeler) and report whether the segmentation module still removes background events effectively and whether the reconstruction metrics hold.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that filtering background events with a learned hand mask enables accurate egocentric event-based hand mesh reconstruction on N-HOT3D. The load-bearing premise is that the mask supervision, obtained by projecting HOT3D 3D mesh ground truth into 2D, provides a clean and complete signal for the U-Net to learn from, and that this learned mask then transfers to noisy v2e event frames at inference. However, the paper neither analyzes the effect of the projection parameters on mask quality (e.g., occlusion, truncation, self-occlusion, and mesh alignment errors) nor provides a quantitative mask evaluation on real event data. If the mask training relies on a strong signal that is unavailable or misaligned in deployment (for instance, if v2e event frames exhibit temporal misalignment with RGB frames, or if HOT3D mesh annotations are not perfectly registered to the video frames after distortion correction), then the 0.450 vs 0.392 R-AUC improvement attributed to the Hand Segmentation Module could be overstated. Furthermore, since the evaluation is entirely on synthetic v2e data, the failure mode that the mask is intended to address (dynamic background events) may be substantially different in real event streams, where camera motion and sensor noise are less structured. The paper does not report the segmentation IoU on the held-out test set, nor does it quantify the fraction of background events removed, making it impossible to verify the hypothesized mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EventEgoHands, the first method for event-based egocentric 3D hand mesh reconstruction. The method consists of two modules: a Hand Segmentation Module, which uses a U-Net on LNES event frames to predict a binary hand mask that filters out background events, and a Hand Reconstruction Module, which uses PointNet++ with Cross-Attention to regress MANO parameters for both hands from the filtered event cloud. To train and evaluate the method, the authors create N-HOT3D, a large synthetic dataset of 447,704 samples generated by applying the v2e event simulator to the HOT3D egocentric dataset. Experiments on N-HOT3D report substantial improvements over prior event-based hand reconstruction baselines: R-AUC increases from 0.243/0.251 (EventHands/Ev2Hands) to 0.450, MPJPE decreases from 105.80/106.75 mm to 59.51 mm, and MPVPE from 65.23/69.66 mm to 42.92 mm. Ablations show that removing the segmentation module reduces R-AUC to 0.392 and increases MPJPE to 66.24 mm, while replacing Cross-Attention with Self-Attention gives intermediate results.","tokens_in":10094,"tokens_out":6365,"duration_ms":57157,"significance":"If the reported results are robust, the paper makes a valuable contribution: it is the first to address egocentric event-based hand mesh reconstruction, and the proposed segmentation-based filtering is a simple and intuitively effective mechanism for suppressing camera-wearer-induced background events. The N-HOT3D dataset, though synthetic, is large and could be useful for future research. The improvements over adapted baselines are large and the ablation supports the central role of the segmentation module. However, the evidence is currently confined to a single synthetic dataset with no real-event-camera validation, no held-out mask quality metrics, and incomplete baseline adaptation details. These gaps limit the strength of the central claim that the method handles dynamic egocentric scenes in practice.","major_comments":[{"comment":"The dataset split is described inconsistently. Section 4.1 states that N-HOT3D is 'divided into 347,854 samples for training and 99,850 samples for evaluation,' while Section 4.2 states a split of approximately 278K (62%) for training, 70K (16%) for validation, and 99K (22%) for testing. These numbers do not match: the first split has no validation set, and the second split uses far fewer training samples. The discrepancy is load-bearing for reproducibility because the reported test results must be tied to a specific train/validation/test partition, and hyperparameter selection (Supplementary Tables 1 and 2) is presumably performed on a validation set. Please clarify the exact split and which split is used for the results in Table 1.","section":"§4.1 and §4.2"},{"comment":"The entire evaluation, including the stability of the segmentation module, rests on the synthetic N-HOT3D dataset generated with v2e. The central claim is that the Hand Segmentation Module mitigates background events caused by egocentric camera motion, but v2e's event generation model may not reproduce the statistics of real event sensors (e.g., DAVIS346) or the actual distribution of ego-motion-induced background events. The mask supervision itself is obtained by projecting HOT3D mesh ground truth onto event frames that are generated from distortion-corrected videos, so training and test are perfectly aligned with the same simulation pipeline. Without any evaluation on real event camera data (even qualitative), or at least a domain-gap analysis comparing event noise and background event statistics, the reported gains (e.g., R-AUC 0.450 vs. 0.392 without segmentation) may not transfer to physical event cameras. This is a load-bearing limitation for the practical claim of the paper.","section":"§4.1, §5.1, Table 1"},{"comment":"No quantitative evaluation of the Hand Segmentation Module on the held-out test set is reported. The main paper reports only reconstruction metrics (R-AUC, MPJPE, MPVPE) and the ablation of removing the module. The supplementary Table 1 reports IoU only for hyperparameter selection of loss weights, not for the final model on the test set, and it is unclear whether that IoU is computed on a validation or test split. Because the segmentation module is a core contribution and its mechanism (filtering background events) is the explanatory hypothesis for the reconstruction improvement, the paper should report test-set mask IoU (and, ideally, the fraction of background events removed) to verify that the module indeed learns clean hand masks and that the improvement is not an artifact of, e.g., simply reducing the number of points in the cloud.","section":"§5.1 and Supplementary Table 1"},{"comment":"The baseline adaptations are not fully specified, which makes the comparison potentially unfair. For Ev2Hands, the authors state that they 'first trained on the annotated Ev2Hands-S dataset and then fine-tuned it with N-HOT3D,' but they do not describe the fine-tuning objective, learning rate schedule, or how the absence of per-event left/right/background labels in N-HOT3D is handled during fine-tuning. Since Ev2Hands originally uses per-event classification as part of its pipeline, removing that supervision may significantly degrade it; the paper should specify the exact fine-tuning protocol. For EventHands, training separate left-hand and right-hand models is mentioned but the details (data cropping, joint outputs) are omitted. These omissions prevent readers from assessing whether the reported baseline numbers are reasonable.","section":"§4.3"}],"minor_comments":[{"comment":"The number of epochs for both modules is fixed at 10, but no early stopping criterion or validation-based model selection is described; please clarify how the final model is chosen.","section":"§4.2"},{"comment":"The fixed point-cloud size N=2048 is mentioned in the supplementary, but the main paper does not specify how the filtered event cloud of size N_f is converted to N=2048 points (e.g., random subsampling, padding, or selection of the most recent events). This matters for interpreting the computational cost reduction and the segmentation filter's effect.","section":"Supplementary §B.2"},{"comment":"The phrase 'while independently providing the raw event and hand mask' is confusing; it likely means that the event stream and mask are generated from the RGB frame and MANO annotations, but the wording should be clarified.","section":"Figure 3 caption"},{"comment":"The reference to the DAVIS346 datasheet is written as 'iniV ation' with a stray space; please correct.","section":"Reference [20]"},{"comment":"All results are reported as single runs without error bars or multiple seeds. Given the large observed margins this is a minor concern, but adding variance estimates would strengthen the claims.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and underexplored problem, and the core idea of segmentation-based background filtering for egocentric event-based hand reconstruction is sensible. The main risk is the complete reliance on a single synthetic dataset and the absence of any real-event-camera validation; this is a standard expectation for event-vision papers, especially when the proposed method is intended to handle real-world ego-motion noise. The dataset split inconsistency and missing mask metrics reinforce the sense that the experimental section needs another round of careful revision. I would encourage the editor to invite a revision and explicitly request (a) a consistent and detailed dataset split, (b) held-out mask evaluation and, if at all feasible, at least a small real-data qualitative experiment, and (c) a complete description of baseline adaptation. If the authors can provide these, the paper could become a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the key thing you should know: this is a clean, well-executed application paper. It is the first to formulate egocentric event-based 3D hand mesh reconstruction, and it introduces a hand segmentation module that filters ego-motion background events. The held-out evaluation on N-HOT3D shows consistent gains over adapted baselines, and the ablation isolating the segmentation module is credible. The architecture recombines known pieces (LNES, U-Net, Event Cloud, PointNet++, cross-attention), so the novelty is the task and the empirical demonstration, not the machinery.\n\nWhat is good: the problem is well motivated; event cameras genuinely struggle with ego-motion background events, and the segmentation module is a sensible fix. The dataset creation is careful: they use HOT3D MANO annotations, correct for distortion, and simulate with v2e. The ablations and baseline adaptations show awareness of the pitfalls, and the results are consistent across R-AUC, MPJPE, and MPVPE.\n\nWhere it is soft: everything is validated on synthetic v2e data. No real event camera recording, no code or data release, and results are single-run without error bars. The stress-test note worries that the mask supervision from projected MANO meshes might not transfer cleanly to real v2e noise; that is a fair external-validity caveat, but it is not an internal flaw. The mask is trained and evaluated on the same synthetic domain, so the 0.450-vs-0.392 segmentation gain is internally consistent. What is missing is a test-set mask IoU or a count of background events removed; the supplementary reports IoU only for hyperparameter selection. That would have made the mechanism more transparent. The baselines also required non-trivial adaptation, particularly Ev2Hands, which needed fine-tuning on the original annotated dataset; this could bias comparisons, though the qualitative figures look believable.\n\nBottom line: if the goal is a first step toward egocentric event-based hand mesh reconstruction, this deserves a serious review. The synthetic-only evidence should be called out as the key limitation, and the authors should be pushed to release data/code or add a real-camera sanity check. I would accept for peer review with a request for those additions.","headline":"First egocentric event-based hand mesh method with a segmentation module that clearly helps on their synthetic benchmark — but all evidence is v2e-simulated, so the real-world claim is unproven.","tokens_in":10639,"tokens_out":2495,"would_cite":true,"duration_ms":25037,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EventEgoHands: masking background events makes egocentric event-based 3D hand mesh reconstruction feasible.","keywords":["3D hand mesh reconstruction","event camera","egocentric vision","hand segmentation","point cloud reconstruction","MANO model","N-HOT3D dataset","event simulation"],"falsifier":"Record real egocentric event streams with an event camera and synchronized ground-truth hand meshes (for instance from motion-capture gloves or multi-view RGB), run the published EventEgoHands pipeline on them, and compare MPJPE to the reported 59.51 mm. If real-data error stays near the ~106 mm baseline level, the simulated-to-real transfer premise is the point of failure.","tokens_in":9585,"feed_emoji":"🖐️","tokens_out":8066,"duration_ms":66802,"temperature":0.7,"pith_summary":"EventEgoHands is offered as the first method to reconstruct a 3D hand mesh from an egocentric event camera, and its central claim is that background events from the wearer's motion, not a lack of hand signal, are what broke prior event-based hand tracking. The paper introduces a Hand Segmentation Module that predicts a hand mask from event frames, filters the event point cloud to the hand region, and then reconstructs both hands with a point-cloud network. On the synthetic N-HOT3D dataset, this raises R-AUC from 0.243/0.251 for two prior event-based methods to 0.450 and cuts mean joint error from about 106 mm to 59.51 mm and vertex error from about 65–70 mm to 42.92 mm. The practical stake is a route to hand tracking in low light and fast ego-motion for AR/VR and human-computer interaction, though all evidence comes from simulation.","feed_headline":"Masking background events cuts egocentric hand-mesh error by 45 mm","feed_subtitle":"Filtering out camera-motion noise lets the first event-only egocentric hand-mesh method beat prior baselines.","key_machinery":"The load-bearing mechanism is a two-stage event filter: the Hand Segmentation Module takes T=3 Locally-Normalised Event Surfaces (a frame representation that weights events by recency within a time window) and a U-Net predicts a binary hand mask; the mask is then applied to the Event Cloud representation (raw events as 5D points with position, timestamp, and two polarity channels) to keep only hand events, downsampled to at most N=2048 points. The filtered cloud feeds the Hand Reconstruction Module, whose PointNet++ encoder extracts point features and cross-attention lets the left- and right-hand branches exchange information before a decoder outputs MANO parameters. The segmentation does two jobs at once: it removes the ego-motion background events that break prior methods, and it caps the point count so the cost scales with 2048 events rather than the full stream.","core_discovery":"The central discovery argued in the paper is that the dominant obstacle to egocentric event-camera hand reconstruction is not hand motion but background events created by the wearer's own movement, and that a learned mask can remove that obstacle. EventEgoHands first predicts a hand-region mask from three LNES event frames with a U-Net, then discards all event-cloud points outside the mask before a PointNet++ encoder with cross-attention between the two hand branches regresses MANO pose and shape parameters. On N-HOT3D, a synthetic dataset of 447,704 samples generated from HOT3D via v2e, the method reports R-AUC of 0.450 versus 0.243 and 0.251 for EventHands and Ev2Hands, MPJPE of 59.51 mm versus 105.80 and 106.75 mm, and MPVPE of 42.92 mm versus 65.23 and 69.66 mm. Ablations attribute the largest share of the gain to the Hand Segmentation Module: removing it drops R-AUC to 0.392 and raises MPJPE to 66.24 mm.","pith_inferences":["The paper does not test against real event hardware; the natural next check is whether the mask's relative gain survives on real streams with v2e-style noise removed, and absolute errors likely rise if it does not.","Since mask supervision comes from projected MANO ground truth, the design inherits MANO's limitations; the reported failure on object occlusion follows directly, and a hand-object interaction term is the obvious extension.","Because the segmentation ablation accounts for the largest performance gap, a cheaper non-learned baseline (e.g., fixed central region or motion-magnitude thresholding) would clarify how much of the gain is learned semantics versus coarse background rejection.","The same mask-then-reconstruct pattern is a plausible recipe for other event-based egocentric tasks degraded by ego-motion clutter, such as full-body pose or object tracking, but the paper presents no evidence for that transfer."],"forward_implications":["If the reported numbers hold, event-based egocentric hand mesh reconstruction can be done without RGB or depth input: the full pipeline uses only event frames and event clouds.","The mask-first design means computational cost scales with the masked event budget (N=2048) rather than the total event count, which makes real-time wearable use more plausible.","The roughly 43% reduction in mean joint error over EventHands and Ev2Hands indicates that background events, rather than hand-motion ambiguity, were the dominant error source in prior egocentric settings.","Because the method outputs MANO parameters directly, the reconstructed mesh can be used in AR/VR avatars or robotic teleoperation without an additional fitting step."],"supporting_citations":[{"why":"v2e simulator converts HOT3D RGB frames into the synthetic event streams that make up N-HOT3D; the entire training and test signal depends on it.","marker":"[14]"},{"why":"HOT3D supplies the source egocentric RGB video, MANO annotations, and camera parameters used to build N-HOT3D and its ground-truth masks.","marker":"[15]"},{"why":"Ev2Hands provides the Event Cloud point representation, the reconstruction architecture the Hand Reconstruction Module is inspired by, and the main point-cloud baseline.","marker":"[13]"},{"why":"EventHands supplies the LNES frame representation and serves as the frame-based baseline compared against in Table 1.","marker":"[11]"},{"why":"MANO is the parametric hand model whose pose and shape parameters are regressed to produce joints and a 778-vertex mesh.","marker":"[16]"},{"why":"U-Net is the backbone of the Hand Segmentation Module that predicts the hand mask from event frames.","marker":"[17]"},{"why":"PointNet++ is the point-cloud backbone of the Hand Reconstruction Module that extracts features from the filtered event cloud.","marker":"[18]"},{"why":"Dice loss, combined with BCE in the mask loss, supervises hand segmentation.","marker":"[19]"}],"fun_headline_variants":["Masking background events cuts egocentric hand-mesh error by 45 mm","First event-only egocentric hand mesh: 43% lower error via mask","Background-noise mask slashes hand-mesh MPJPE by 43%","EventEgoHands: 43% better hand mesh by masking camera motion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that v2e-simulated event streams from HOT3D RGB video faithfully reproduce real egocentric event-camera statistics, noise included, so that gains measured on N-HOT3D will transfer to physical event cameras.","fun_headline_variants_meta":{"raw":{"variants":["Masking background events cuts egocentric hand-mesh error by 45 mm","First event-only egocentric hand mesh: 43% lower error via mask","Background-noise mask slashes hand-mesh MPJPE by 43%","EventEgoHands: 43% better hand mesh by masking camera motion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00022,"raw_usage":{"total_tokens":1452,"prompt_tokens":957,"completion_tokens":495,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":411}},"tokens_in":573,"tokens_out":495,"duration_ms":3258,"temperature":1.0,"reasoning_tokens":411,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:19:11.384104+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record real egocentric event streams with an event camera and synchronized ground-truth hand meshes (for instance from motion-capture gloves or multi-view RGB), run the published EventEgoHands pipeline on them, and compare MPJPE to the reported 59.51 mm. If real-data error stays near the ~106 mm baseline level, the simulated-to-real transfer premise is the point of failure.","supporting_citations":[{"cited_title":"Real-time hand tracking under occlusion from an egocentric rgb-d sensor,","cited_arxiv_id":null,"evidence_quote":"v2e simulator converts HOT3D RGB frames into the synthetic event streams that make up N-HOT3D; the entire training and test signal depends on it."},{"cited_title":"Event-Based Vi- sion: A Survey,","cited_arxiv_id":null,"evidence_quote":"HOT3D supplies the source egocentric RGB video, MANO annotations, and camera parameters used to build N-HOT3D and its ground-truth masks."},{"cited_title":"HandDAGT: a denoising adaptive graph transformer for 3D hand pose estimation,","cited_arxiv_id":null,"evidence_quote":"Ev2Hands provides the Event Cloud point representation, the reconstruction architecture the Hand Reconstruction Module is inspired by, and the main point-cloud baseline."},{"cited_title":"Reconstructing hands in 3D with transformers,","cited_arxiv_id":null,"evidence_quote":"EventHands supplies the LNES frame representation and serves as the frame-based baseline compared against in Table 1."},{"cited_title":"Secrets of event-based optical flow, depth and ego-motion estimation by contrast maximiza- tion,","cited_arxiv_id":null,"evidence_quote":"MANO is the parametric hand model whose pose and shape parameters are regressed to produce joints and a 778-vertex mesh."},{"cited_title":"Event-based se- mantic segmentation with posterior attention,","cited_arxiv_id":null,"evidence_quote":"U-Net is the backbone of the Hand Segmentation Module that predicts the hand mask from event frames."},{"cited_title":"EventEgo3D: 3D human motion capture from egocentric event streams,","cited_arxiv_id":null,"evidence_quote":"PointNet++ is the point-cloud backbone of the Hand Reconstruction Module that extracts features from the filtered event cloud."},{"cited_title":"EventHPE: event-based 3D human pose and shape es- timation,","cited_arxiv_id":null,"evidence_quote":"Dice loss, combined with BCE in the mask loss, supervises hand segmentation."}],"review_version":1}