{"id":"1e1c1578-0b51-4fc8-b431-db670cc50ce5","arxiv_id":"2607.25710","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A STGCN-GRU-BiLSTM model detects the frame of ground impact in falls from 3D skeleton data, reaching a reported 97.5% accuracy on a relabeled UP-Fall subset.","lead":"This paper trains a spatio-temporal graph network combined with GRU and bidirectional LSTM layers to detect the exact moment of ground impact in falls recorded as 3D body skeletons. The authors report over 97 percent accuracy on a relabeled version of the UP-Fall dataset and release the improved skeleton dataset publicly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 97.50% accuracy is not a valid estimate of impact-detection performance because Section 4.2's test split includes two sequences from every training subject, allowing subject-identity features to drive frame-level predictions; a subject-disjoint evaluation is required before the…","rationale":"The strongest claim requires that high accuracy reflects detection of ground-contact frames in unseen subjects, so the first necessary condition is that test subjects are not in the training set. Section 4.2 violates or at best fails to establish this condition by selecting two fall sequences from each subject for the test set. Since all reported results come from this protocol, every accuracy number in Tables 7, 8, 10, and 11 is suspect as an estimate of generalization. I agree with the reader's overall REJECT verdict, but I weight the subject-overlap split as the more directly falsifying problem because it is evident from the text without needing to adjudicate whether the SMV-threshold (beta equals 2g) labels match true ground contact. The label question is important and the reader's weakest_assumption identifies it correctly; however, even a perfect label set would not make the current split valid. A subject-disjoint rerun is inexpensive and would settle the question. The UMAFall cross-dataset tables are also under-explained, since no skeleton-extraction pipeline for that dataset is described; I treat this as a further reason not to accept the generalization claim, though the UP-Fall split alone is already sufficient to block the central claim.","tokens_in":24663,"tokens_out":6035,"duration_ms":53539,"concrete_test":"Retrain the proposed STGCN-GRU-BiLSTM and all baselines on the improved UP-Fall dataset using leave-one-subject-out cross-validation, or any split that keeps all sequences of each subject in a single fold, using the same semi-automatic labels and hyperparameters. Report mean and standard deviation of frame-level accuracy, precision, recall, F1, and MCC across folds. If the subject-disjoint accuracy is materially below 97.50%, for example by more than 3 percentage points, the current headline accuracy is attributable to subject leakage and the central claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 describes the UP-Fall evaluation as an 80/10/10 split and then states that 'we selected two fall sequences from each subject for inclusion in the test set.' This means every test subject also has sequences available for training. For skeleton-based models, subject-specific body proportions, habitual joint angles, and recording setup are strong confounds; a model can achieve high frame-level accuracy by recognizing the person or the sequence rather than by localizing the actual ground-contact frame. Because the paper reports no subject-disjoint split, no error bars, and no per-subject results, the 97.50% accuracy in Table 7 and the per-fall-type numbers in Table 8 cannot be interpreted as evidence that the model detects impacts in unseen individuals. This concern is independent of label quality: even if the SMV-threshold plus visual-inspection labels were perfectly accurate, the current split would still invalidate the reported generalization estimate. The paper itself concedes in Section 3.5 that discrepancies between semi-automatic labeling and actual impact events can occur, but the train/test overlap alone is sufficient to block the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a frame-level impact-detection architecture, STGCN-GRU-BiLSTM, that takes MediaPipe BlazePose 3D skeletons from RGB video and classifies each frame as impact or non-impact. The authors introduce an 'improved' version of the UP-Fall dataset with labels obtained by a semi-automatic accelerometer SMV threshold (β = 2g) plus visual inspection, and they report 97.50% accuracy on this dataset, per-fall-type accuracies between 86% and 97.50%, and 94–97% accuracy in cross-dataset experiments on UMAFall. The paper also includes an ablation study, preprocessing comparisons, an occlusion robustness experiment, a joint-attention analysis, and a computational-cost table.","tokens_in":24876,"tokens_out":6370,"duration_ms":55073,"significance":"If the quantitative claims were valid, this would be a practically useful contribution: precise impact-moment localization from skeleton data could reduce false alarms in elderly fall monitoring, and the released improved dataset could support further research. The paper deserves credit for making the dataset public and for comparing several baselines under a common protocol, and the occlusion and computational analyses are informative. However, the evaluation protocol has load-bearing flaws that currently invalidate the headline accuracy numbers, and the cross-dataset setup is not described at a level that would allow reproduction or interpretation.","major_comments":[{"comment":"The test split is not subject-disjoint. The paper states that 'we selected two fall sequences from each subject for inclusion in the test set' after an 80/10/10 split, which means sequences from the same subjects also appear in the training set. For skeleton-based models, subject identity (body proportions, habitual joint angles, recording setup) is a strong confound, so the reported 97.50% accuracy in Table 7 and the per-fall-type numbers in Table 8 cannot be interpreted as generalization to unseen individuals. A subject-disjoint split with per-subject results and error bars is required before the central claim can be accepted.","section":"Section 4.2, Training and Testing Strategy"},{"comment":"The ground-truth impact labels are generated by an accelerometer SMV exceeding a fixed threshold β = 2g, followed by visual inspection by the authors. No independent validation of this threshold against true physical ground contact is provided, and the paper itself acknowledges that 'discrepancies between the semi-automatic labeling and the actual impact event can be identified.' The reported accuracies therefore measure agreement with a self-defined label, not necessarily detection of actual ground contact. The authors should validate the labeling procedure (e.g., against manual annotation by multiple raters or an independent sensor modality), report inter-annotator agreement, or explicitly restrict the claims to the defined label.","section":"Section 3.5, Eq. (2)"},{"comment":"The UMAFall cross-dataset evaluation is not sufficiently described. UMAFall is an inertial-sensor dataset, and the paper does not explain how camera images or video were obtained and synchronized with the sensor data, how MediaPipe skeletons were extracted from UMAFall data, or how ground-truth impact labels were defined for the UMAFall test frames. Without this information, the 94–97% accuracies in Tables 9 and 11 cannot be reproduced, and the claim of cross-dataset generalization is not verifiable.","section":"Sections 4.6 and 4.7, Cross-dataset experiments on UMAFall"},{"comment":"Several rows in Table 8 contain identical metric values across different models, which is implausible. For example, in the FKnee+Standing block, LSTM and BiLSTM share identical Precision, F1 Score, Specificity, Recall, AUC ROC, and MCC, while in the FH+Standing block, CNN and STGCN share identical Accuracy, Precision, Specificity, and MCC. These repetitions suggest reporting errors and undermine the comparative claim that the proposed method outperforms the state of the art in each fall scenario. The authors should verify and correct the table, and ideally provide error bars over multiple runs.","section":"Table 8, Individual fall scenario performance"}],"minor_comments":[{"comment":"The terms 'Manual Labelization' and 'Semi-Automatic Labelization' should be replaced with 'Manual Labeling' and 'Semi-Automatic Labeling'.","section":"Section 3.4 and 3.5"},{"comment":"In Eq. (4), the symbol V is used in the term Γ_µ ⊗ V, but V was previously defined as the set of graph vertices; the intended operand appears to be the input feature tensor, not the vertex set.","section":"Section 3.7, Eq. (4)"},{"comment":"The description of the 80/10/10 split should state explicitly whether the split is performed at the subject level or the sequence level; the current text is self-contradictory because selecting two sequences per subject for the test set conflicts with a random split.","section":"Section 4.2"},{"comment":"The abbreviation 'FSitting' is used in the table but is not defined in the caption; the corresponding activity is listed as 'FE' in Table 4.","section":"Table 8 caption"},{"comment":"The section reports testing with fixed-length inputs of 60, 90, and 180 frames, but Table 6 lists a sequence length of 100 frames; the paper should explain how variable-length inputs are handled by the BiLSTM layer.","section":"Section 4.10"},{"comment":"No code, random seeds, or number of repeated runs are reported; the authors should provide the implementation and report mean and standard deviation over multiple seeds.","section":"General"},{"comment":"Figure 2 is very dense and difficult to read at the current resolution; it should be enlarged or simplified for clarity.","section":"Figure 2"},{"comment":"The abstract states 'accuracy exceeding 90%' while the body reports 97.50%; consider reporting the headline accuracy directly in the abstract.","section":"Abstract"},{"comment":"There are minor typos: 'sleleton' appears in Section 3.1.3 and 'However, While manual labeling' appears in Section 3.4.","section":"Section 3.1.3 and 3.4"}],"recommendation":"major_revision","confidential_remarks":"The idea of frame-level impact detection from 3D skeletons is interesting and the released dataset has potential value, but the current evaluation is not credible: the test set is not subject-disjoint, the label definition is unvalidated, the UMAFall cross-dataset protocol is underspecified, and Table 8 contains clearly duplicated values across models. A major revision that re-runs all experiments under a subject-disjoint protocol, validates or redefines the labels, documents the UMAFall pipeline, and corrects the tables would be needed before the paper can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the released 3D-skeleton version of UP-Fall is a useful resource and the task framing—frame-level impact detection rather than fall/no-fall—is genuinely underexplored. But the headline 97.5% accuracy is not a valid estimate of performance on unseen people. Section 4.2 says the test set contains two fall sequences from every subject, meaning each test subject also has training sequences. For skeleton-based models, subject identity is a strong confound, so the reported numbers can reflect person recognition, not impact localization. The stress-test note is right; this alone blocks the central claim.\n\nWhat the paper does well: the dataset release is concrete and reproducible, the ablation is standard, and the attention visualization gives a plausible story about which joints matter. The authors also acknowledge label discrepancies in Section 3.5, which is honest.\n\nThe other soft spots are serious but secondary. There is no code and no error bars; every number appears to be a single run. Several rows repeat across tables in ways that look like copy-paste errors—for instance, the CNN and STGCN rows for FH+Standing in Table 8 share the same precision and MCC, and the CNN rows for FB and FH in Table 9 are identical. The UMAFall cross-dataset evaluation is missing its key methodological detail: UMAFall is an inertial-sensor dataset without a camera stream, so where do the 3D skeletons come from, and how are impact frames labelled there? Without that description, the 94–97% cross-dataset numbers are uninterpretable. The SMV-threshold labeling is a reasonable idea, but it defines 'impact' by an accelerometer threshold plus visual inspection; that is a legitimate source for weak labels, not a gold standard.\n\nIf the authors release code, redo the split to be subject-disjoint, report standard deviations, and spell out the UMAFall pipeline, the dataset alone could merit publication. As it stands, I would not trust the accuracy claims. For peer review, I would still send it to referees rather than desk-reject, because the resource is real and the evaluation flaws are fixable; the revision bar should be high. I would bring it to reading group mostly as a case study in dataset-split leakage.","headline":"Useful dataset and honest task framing, but the subject-overlapping test split invalidates the headline accuracy claim.","tokens_in":25444,"tokens_out":3706,"would_cite":false,"duration_ms":31367,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the ground-contact frame inside a fall can be recovered from a 3D skeleton graph, and that its STGCN-GRU-BiLSTM detector reaches 97.50% accuracy.","keywords":["impact detection","fall detection","3D skeleton","graph convolutional network","GRU","BiLSTM","UP-Fall dataset","cross-dataset generalization"],"falsifier":"Re-collect a small set of falls with synchronized force-plate or floor-vibration contact timestamps, apply the paper's exact preprocessing and labeling rules, and compare the SMV-generated labels with the physical contact frame. If the threshold labels disagree with physical contact in a substantial fraction of falls, the reported 97.5% would measure label consistency rather than real impact detection; alternatively, retrain the same architecture on independently annotated labels and check whether the accuracy survives.","tokens_in":24433,"feed_emoji":"🦴","tokens_out":10917,"duration_ms":92954,"temperature":0.7,"pith_summary":"This paper is trying to establish that the precise impact moment in a fall—the frame where the body actually contacts the ground—can be detected from 3D joint skeleton data by treating the skeleton as a graph and reading the sequence in both time directions. Most fall detectors classify whole events and trigger during the descent or on near-falls; the proposed architecture, a Spatio-Temporal Graph Convolutional Network combined with Gated Recurrent Unit and Bidirectional Long Short-Term Memory layers (STGCN-GRU-BiLSTM), labels individual frames instead. Trained on the authors' improved 3D-skeleton version of the UP-Fall dataset, it reaches 97.50% accuracy overall, 96.50% for backward falls, and 97.50% for falls while sitting, with 94–97% accuracy in cross-dataset tests on UMAFall. If these results hold, camera-based monitoring could report the actual moment of ground contact rather than the occurrence of a fall-like motion, which is what caregivers need to filter false alarms and dispatch help immediately.","feed_headline":"Skeleton graph model pinpoints ground-contact frame in 97.5% of falls","feed_subtitle":"Frame-level impact detection beats fall-only classifiers and shifts alerts from near-falls to true ground contact.","key_machinery":"The load-bearing object is the human body-joint graph $G=(A,V,E)$, built from 33 three-dimensional joints extracted by MediaPipe BlazePose and processed through a two-hop adjacency matrix. Spatial features are gathered by graph convolution over this graph, then temporal features are extracted by three temporal convolutions, and the sequence is passed through GRU layers followed by a bidirectional LSTM so each frame is classified with both past and future context. An attention vector multiplies the adjacency matrix to up-weight joints that are likely initial contact points; the paper identifies head, right shoulder, left elbow, right wrist, left hip, and right knee as the joints with consistently high attention. The same mechanism also carries the semi-automatic labeling: a Signal Magnitude Vector from synchronized accelerometer data is thresholded at $\\beta=2g$ to mark impact frames, and each candidate is visually checked.","core_discovery":"The central claim is that re-framing fall analysis as frame-level impact detection, and modeling the body as a graph with bidirectional temporal context, yields a detector that can identify the exact frame of ground contact from skeleton sequences. The authors report that their STGCN-GRU-BiLSTM model outperforms the STGCN baseline by 5.34 accuracy points and reaches 97.50% accuracy, 98.11% precision, and 97.26% F1-score on the improved UP-Fall dataset. On individual fall scenarios it achieves 96.50% for backward falls while standing and 97.50% for falls while attempting to sit; in cross-dataset evaluation it holds 94–97% accuracy on UMAFall scenarios. The paper also claims that the preprocessing pipeline—cropping, background removal, MediaPipe BlazePose skeleton extraction, and semi-automatic accelerometer-based labeling—improves every compared algorithm, with the STGCN baseline rising from 72% to 87.43% accuracy. The same architectural components are shown through an ablation to contribute progressively: STGCN alone 92.16%, with ConvLSTM 93.00%, with GRU 93.50%, and the full model 97.50%.","pith_inferences":["Because the ground-truth labels are generated from an accelerometer threshold and then visually verified, the true quantity being learned is probably 'high-acceleration frame that coincides with visible ground contact'; in deployment, falls with low impact force (slow collapses, soft surfaces) may fall below the 2g threshold and be systematically missed even if the skeleton signal contains the con","The occlusion results suggest a practical design rule the paper does not state: confidence should be gated on which body parts are visible, since upper-body-visible occlusion keeps accuracy at 95.2% while lower-body-only visibility drops it to 76.6%.","The attention ranking of head, shoulder, elbow, wrist, hip, and knee yields a testable prediction for injury prevention—real-world fall injuries should cluster at these joints—which could be checked against clinical records independently of model accuracy.","Because the falls are simulated by healthy young adults, an obvious next test is whether the same skeleton features and the 2g label threshold survive falls by older adults with slower descent and lower impact; the paper lists real-elderly validation as future work."],"forward_implications":["If the accuracy figures hold, a camera-only system can issue a fall alert at the ground-contact frame rather than during the descent, which is the timing needed to suppress alerts caused by near-falls and stumbles.","The improved UP-Fall dataset with frame-level impact labels becomes a reusable benchmark for comparing impact-localization methods, not just fall-event classifiers.","Backward falls and seated falls are the scenarios with the largest reported gains, implying the model's joint-attention and bidirectional context capture the movement patterns specific to those fall types; a deployed detector should therefore be more reliable in two situations where elderly falls are common.","The cross-dataset results on UMAFall suggest the learned impact signature transfers across camera and sensor setups when the same preprocessing is applied, which is a necessary condition for practical deployment beyond a single lab.","The BiLSTM feature analysis indicates that falls of different speeds form separable clusters, so impact detection could be extended to report fall direction or velocity from the same features."],"supporting_citations":[{"why":"supplies the UP-Fall multimodal dataset from which the improved 3D skeletons are derived.","marker":"[59]"},{"why":"provides MediaPipe BlazePose, the 33-joint pose estimator that produces the 3D skeleton input.","marker":"[51]"},{"why":"gives the Signal Magnitude Vector equation and threshold idea used to create the semi-automatic impact labels.","marker":"[55]"},{"why":"provides UMAFall, the independent dataset used for cross-dataset generalization tests.","marker":"[60]"},{"why":"is the STGCN baseline that the proposed model extends with GRU and BiLSTM layers.","marker":"[57]"},{"why":"supplies GrabCut background removal, which the authors show increases usable pose-estimation frames.","marker":"[49]"},{"why":"is the BiLSTM-based fall detection approach used as a comparison and as justification for bidirectional temporal modeling.","marker":"[40]"},{"why":"provides the CNN baseline used in the fall-scenario and cross-dataset performance comparisons.","marker":"[63]"},{"why":"provides the LSTM fall-detection baseline used in the ablation and generalization tables.","marker":"[64]"},{"why":"is the authors' earlier multisensor impact-detection study that first targeted impact localization rather than general fall classification.","marker":"[38]"}],"fun_headline_variants":["Graph model hits 97.5% in pinpointing fall impact frame","97.5% impact-frame detection with skeleton graph + RNN","Skeleton graph detects exact fall impact frame at 97.5% accuracy","Pinpointing impact frame: skeleton graph + RNN achieves 97.5%","Graph-based model hits 97.5% on fall impact frame detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that an impact frame is correctly defined by a single combined accelerometer reading (Signal Magnitude Vector) exceeding $\\beta=2g$, with the authors' visual check as confirmation; if that threshold does not match true physical ground contact, the reported accuracies measure agreement with a self-created label, not actual impact detection.","fun_headline_variants_meta":{"raw":{"variants":["Graph model hits 97.5% in pinpointing fall impact frame","97.5% impact-frame detection with skeleton graph + RNN","Skeleton graph detects exact fall impact frame at 97.5% accuracy","Pinpointing impact frame: skeleton graph + RNN achieves 97.5%","Graph-based model hits 97.5% on fall impact frame detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000593,"raw_usage":{"total_tokens":2810,"prompt_tokens":1006,"completion_tokens":1804,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":1705}},"tokens_in":622,"tokens_out":1804,"duration_ms":10724,"temperature":1.0,"reasoning_tokens":1705,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:24:48.975144+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-collect a small set of falls with synchronized force-plate or floor-vibration contact timestamps, apply the paper's exact preprocessing and labeling rules, and compare the SMV-generated labels with the physical contact frame. If the threshold labels disagree with physical contact in a substantial fraction of falls, the reported 97.5% would measure label consistency rather than real impact detection; alternatively, retrain the same architecture on independently annotated labels and check whether the accuracy survives.","supporting_citations":[{"cited_title":"Sensors19(9), 1988 (2019)","cited_arxiv_id":null,"evidence_quote":"supplies the UP-Fall multimodal dataset from which the improved 3D skeletons are derived."},{"cited_title":"IEEE Sensors Letters4(6), 1–4 (2020)","cited_arxiv_id":null,"evidence_quote":"gives the Signal Magnitude Vector equation and threshold idea used to create the semi-automatic impact labels."},{"cited_title":"Procedia Computer Science 110, 32–39 (2017)","cited_arxiv_id":null,"evidence_quote":"provides UMAFall, the independent dataset used for cross-dataset generalization tests."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"is the STGCN baseline that the proposed model extends with GRU and BiLSTM layers."},{"cited_title":"Mathematics11(8), 1965 (2023)","cited_arxiv_id":null,"evidence_quote":"supplies GrabCut background removal, which the authors show increases usable pose-estimation frames."},{"cited_title":"37 Biomedical Signal Processing and Control71, 103242 (2022)","cited_arxiv_id":null,"evidence_quote":"is the BiLSTM-based fall detection approach used as a comparison and as justification for bidirectional temporal modeling."},{"cited_title":"Multimedia Tools and Applications83(6), 18091–18118 (2024)","cited_arxiv_id":null,"evidence_quote":"provides the CNN baseline used in the fall-scenario and cross-dataset performance comparisons."},{"cited_title":"In: WAMWB@ MobileHCI, pp","cited_arxiv_id":null,"evidence_quote":"provides the LSTM fall-detection baseline used in the ablation and generalization tables."},{"cited_title":"In: 2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP), pp","cited_arxiv_id":null,"evidence_quote":"is the authors' earlier multisensor impact-detection study that first targeted impact localization rather than general fall classification."}],"review_version":2}