{"id":"ea769b49-70ff-419c-9f74-5c97a8f32593","arxiv_id":"2608.13197","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"On real-world fall data, a symbolic representation augmented with physical impact descriptors degrades least under data scarcity and domain shift, while top simulated-data methods collapse.","lead":"Falls are hard to collect in the real world, so many detection models are trained on laboratory simulations and may look better than they perform. This study compares five motion representations under realistic data scarcity and finds that a simple symbolic representation with physical impact markers transfers best from simulated to real falls, while high-performing kernel and foundation models degrade sharply.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The physical-prior mechanism is not identified: Table A2 varies bin coarseness and threshold provenance together, and the winning 3-level token was selected on the FARSEEING transfer target, so the dataset-invariance claim and FallLM's headline advantage are not independently tested.","rationale":"The paper is a systematic comparison of motion representations for wearable fall detection under data scarcity and domain shift, with a two-part central claim: (1) in-domain accuracy poorly predicts real-world robustness, and (2) representations anchored to dataset-invariant physical quantities transfer more gracefully. Part (1) is robustly supported by Table 3: Mantis, MiniRocket, WEASEL, and QUANT all exceed F1 = 0.95 on FallAllD and drop to 0.37–0.62 on FARSEEING, and the streaming protocol with subject-wise splits is a sound methodological choice. Part (2) is the fragile core. The supporting evidence is the FallLM row of Table 3, the MrSQM contrast (a second SAX method without impact tokens loses precision from 0.91 to 0.42 under transfer), and the Appendix A.1 ablation. The MrSQM and SAX-only comparisons do support the claim that the impact token carries the transfer signal. However, the 3-level versus 12-bin comparison — the only evidence that the operative property is physical grounding rather than coarseness — varies coarseness and threshold provenance simultaneously, so the mechanism is not identified. The selection circularity (the 3-level token chosen on the FARSEEING transfer target) further means the headline 9.5% figure is not an independent estimate. Both factors are correctable: a four-way ablation disentangling coarseness from provenance, and an evaluation on a second real-world dataset not used in selection. The paper's own limitation statement ('Our analysis rests on a single clinically-verified real-world dataset') is accurate and weighs against the breadth of the cross-dataset claims but is not itself a flaw. A minor inconsistency: the abstract credits the interval-based representation with 'the strongest absolute real-world performance,' while Table 3 gives FallLM the highest real F1 (0.67 versus 0.62) and Table 2 gives Mantis the highest in-domain F1 (0.83 versus 0.82); the abstract should be aligned with the tables. Because the broad empirical finding stands and the specific claim is plausible, transparently presented, and testable, the reader's CONDITIONAL verdict remains appropriate; my concern sharpens the conditions under which acceptance is warranted.","tokens_in":14218,"tokens_out":15577,"duration_ms":132243,"concrete_test":"Run a four-way version of the Appendix A.1 ablation (FallAllD to FARSEEING), fixing the impact-token configuration before any FARSEEING evaluation: (i) 3-level thresholds from [10] (current FallLM); (ii) 3-level thresholds fitted to FallAllD training peak magnitudes; (iii) 12-bin thresholds defined a priori from [10]'s biomechanical ranges; (iv) 12-bin thresholds fitted to FallAllD (current). If configuration (ii) matches (i), coarseness — not physical grounding — explains transfer, and the Section 6 mechanism claim must be revised; likewise if (iii) collapses alongside (iv). Only if (i) transfers while (ii) and (iii) degrade is the 'dataset-invariant physical quantity' explanation supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central mechanism claim — 'representations anchored to dataset-invariant physical quantities transfer more gracefully than those modelling dataset-specific signal statistics' (Section 6; also Section 3.4) — rests on exactly one controlled comparison: the impact-token ablation in Appendix A.1 (Table A2). That comparison does not identify the proposed mechanism. The winning 3-level token differs from the losing 12-bin token in two ways at once: bin count (coarseness) and threshold provenance (literature values from [10] versus bins fitted to FallAllD's training distribution). A coarse token would be expected to survive distribution shift even without physical grounding, while a fine token fitted to the training population would be expected to miss shifted peaks even if its boundaries were physically motivated; Table A2 cannot distinguish 'dataset-invariant physical quantity' from 'coarse quantization is shift-robust'. Section 6's claim that 'what governs transfer is not model simplicity but the type of prior' is exactly the dichotomy the data leave unresolved. Compounding this, the 3-level configuration was selected using the same FARSEEING transfer result that is then reported as the headline finding: Section 3.4 states the token 'transferred best to FARSEEING (see Appendix A.1)', so Table 3's 9.5% drop is a target-selected estimate, not an independent test of the physical-prior hypothesis. This critique does not touch the broad empirical finding that simulated performance overstates real-world robustness — Table 3 shows four methods collapsing from simulated F1 of 0.95–0.98 to real F1 of 0.24–0.50 — but it does jeopardize the distinctive explanatory claim and the FallLM-specific deployability recommendation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper systematically evaluates four families of motion representations—interval-based (QUANT), kernel-based (MiniRocket), symbolic (WEASEL, MrSQM, FallLM), and foundation models (Mantis)—for wearable fall detection under real-world data scarcity. Using a streaming event-detection protocol with subject-wise splits, the authors compare methods on FallAllD (simulated falls) and FARSEEING (clinically verified real-world falls) via in-domain cross-validation, controlled data-scarcity experiments, and zero-shot cross-dataset transfer. The main empirical finding is that high simulated performance does not transfer to real-world data: methods with F1>0.95 on FallAllD drop to 0.37–0.62 on FARSEEING. The authors introduce FallLM, a lightweight SAX-based representation augmented with physically grounded impact tokens (absolute and relative), which achieves the smallest relative F1 drop under transfer (9.5%) and retains detection sensitivity with as few as two training falls. They argue that robustness tracks the type of representational prior, with dataset-invariant physical quantities transferring more gracefully than dataset-specific signal statistics.","tokens_in":14588,"tokens_out":2722,"duration_ms":26072,"significance":"If its central claims hold, this paper makes a valuable contribution to wearable fall detection by providing a principled, reproducible comparison of representation families under realistic deployment constraints. The streaming evaluation protocol, subject-wise splits, and cross-dataset transfer design are methodologically sound and address a real gap in the literature. The paper also ships code, supports token-level interpretability, and offers a concrete, falsifiable claim: coarse, physically grounded impact tokens transfer better than fine-grained, data-fitted magnitude tokens. The scarcity analysis is a useful addition. However, the load-bearing mechanism claim—that physical grounding, rather than coarse quantization or model simplicity, drives transfer robustness—is not independently tested because the winning 3-level token configuration was selected on the FARSEEING transfer metric itself.","major_comments":[{"comment":"The headline transfer result is target-selected. Section 3.4 states that the 3-level impact token 'transferred best to FARSEEING (see Appendix A.1)', and Table A2 indeed compares the 3-level configuration against SAX-only and a 12-bin variant. Since the 3-level token was chosen after observing its FARSEEING transfer F1, the reported 9.5% degradation in Table 3 is not an independent test of the physical-prior hypothesis. This inflates the apparent advantage of FallLM and weakens the central claim that dataset-invariant physical quantities transfer more gracefully. A prespecified configuration, a nested model-selection procedure, or explicit reporting of selection bias would be needed to make this claim solid.","section":"Section 3.4, Appendix A.1, Table A2"},{"comment":"The ablation confounds two factors: coarseness of the magnitude encoding and provenance of the thresholds. The 3-level token uses only three bins with literature thresholds (1.8g, 2.5g), while the 12-bin token uses twelve bins fitted to FallAllD's training distribution. A coarse token would be expected to survive distribution shift even without physical grounding, and a fine token fitted to the training population would be expected to lose recall on shifted data. Table A2 cannot distinguish 'dataset-invariant physical quantity' from 'coarse quantization is shift-robust'. The claim in Section 6 that 'what governs transfer is not model simplicity but the type of prior' requires ablations that vary one factor at a time, e.g., a 3-level token fitted to FallAllD versus the literature-threshold 3-level token, or a 12-bin token with literature-derived boundaries.","section":"Appendix A.1, Table A2"},{"comment":"The paper's main empirical conclusion—'in-domain accuracy is a poor predictor of real-world robustness'—is based on a single simulated dataset (FallAllD) and a single real-world dataset (FARSEEING). The authors acknowledge this limitation in their final paragraph, but the claim is stated in general terms throughout the abstract and discussion. Since FARSEEING is the only clinically verified real-world corpus used, and the L5 subset comprises 150 falls, the generality of the simulation-to-reality gap remains uncertain. A second real-world dataset or a sensitivity analysis varying the choice of real-world test corpus would materially strengthen the claim. As written, the conclusion is an existence proof for this pair of datasets rather than a general law.","section":"Section 6 / Limitations (final paragraph)"}],"minor_comments":[{"comment":"Displayed text contains 'T able' (Tables 1, 2, 3, 4 and others), which appears to be a LaTeX rendering artifact; please fix the formatting.","section":"Several table captions"},{"comment":"The scarcity curves report mean performance over five random subsets but no error bands or per-subset variability. Given the extreme 1% condition (two falls), reporting standard deviations or individual run values would help the reader assess the reliability of the claim that FallLM 'is the only method to achieve substantial detection'.","section":"Section 5.2 / Figure 3"},{"comment":"The FARSEEING alphabet-size sweep shows n_bins=4 performing notably worse (F1=0.518) than all other settings, yet n_bins=5 is declared the default based on interpretability. The text says this sweep was 'not used to select n_bins', but the reader may wonder whether the paper's conclusions would change if n_bins were allowed to vary; a brief interpretive sentence on why n_bins=4's drop does not affect the representation-family comparison would improve clarity.","section":"Appendix A.2, Table A4"},{"comment":"The description of the relative impact token ('peak acceleration relative to the magnitude distribution within the window') is vague. Please specify the exact formula or algorithm used to compute the three relative levels, as this affects reproducibility.","section":"Section 3.4 / Table A1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution with a reproducible evaluation pipeline and clearly stated limitations. The main blocker is the selection-on-the-transfer-target issue: the 3-level impact token was chosen using FARSEEING transfer performance, and the ablation confounds coarseness with threshold provenance. These issues are fixable within the manuscript's scope by rerunning the transfer comparison with a prespecified configuration or by adding orthogonal ablations. The code release and the framing of cross-dataset evaluation as a primary selection criterion are strengths worth preserving."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the broad finding holds up, the headline mechanism does not. The paper is worth a serious referee.\n\nFirst, credit. The paper does something the fall detection literature needs: it compares four representation families under the authors' streaming protocol, with subject-wise splits, controlled scarcity, and cross-dataset transfer from FallAllD to FARSEEING. Code is released. Table 3 is the center of gravity: simulated F1s of 0.95–0.98 for MiniRocket, Mantis, WEASEL, and QUANT collapse to 0.37–0.62 on real falls. That is a clean, reproducible caution against simulated-only evaluation.\n\nNow the soft spots. The paper's distinctive claim is that a SAX representation augmented with absolute impact thresholds in g transfers better because the prior is physically dataset-invariant. That claim rests almost entirely on Appendix A.1, and the stress-test note is correct: the winning 3-level token differs from the 12-bin token in both coarseness and threshold provenance. A coarse token fitted to nothing would plausibly survive domain shift regardless of physical grounding. Section 6 says what governs transfer is 'the type of prior', but the data only show that this particular coarse token beats this particular fine token. And because the 3-level token was selected after observing its FARSEEING transfer advantage, Table 3's 9.5% drop is a target-selected number, not an independent confirmation. The paper is honest about the selection, but it still means the central mechanism is not tested.\n\nSecond, the whole real-world comparison sits on 150 FARSEEING falls from one sensor placement. The authors state this limitation, and it doesn't sink the broad conclusion, but it should cap how strongly anyone recommends FallLM specifically.\n\nThird, the abstract says the interval-based representation achieves the strongest absolute real-world performance. In Table 2, Mantis has the highest F1; in Table 3, FallLM does. The sentence doesn't match either table. It is a fixable wording problem, but it is confusing.\n\nThe citation pattern looks fine. The authors build on their own prior streaming protocol and cite the relevant aeon, ROCKET, SAX, FARSEEING, and FallAllD work. I don't see a red flag.\n\nBottom line: this is a useful benchmark, the simulation-to-reality gap is real, and the interpretability analysis is a nice bonus. The FallLM-specific deployability recommendation should be presented as exploratory until the ablation disentangles coarseness from physical grounding and the configuration choice is not made on the test target. I would send it to peer review with a request to fix those issues and to align the abstract with the tables.","headline":"A genuinely useful fall-detection benchmark with a credible simulation-to-reality warning, but the physically-grounded-transfer claim is selected on the test target and should not be treated as established.","tokens_in":15126,"tokens_out":3113,"would_cite":true,"duration_ms":28158,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that in-domain accuracy is a poor predictor of real-world robustness, and that anchoring motion representations to dataset-invariant physical quantities—specifically impact magnitude in units of g—is what lets a fall…","keywords":["fall detection","wearable accelerometer","motion representation","real-world data scarcity","cross-dataset transfer","symbolic aggregate approximation","impact descriptors","streaming evaluation"],"falsifier":"Train FallLM on FallAllD and evaluate it on a second clinically verified real-world fall dataset recorded at a different sensor placement, such as wrist or thigh; if the cross-dataset F1 advantage over QUANT and Mantis shrinks or reverses, the claim that the g-anchored thresholds are dataset-invariant is falsified. A more immediate check is to re-run the Appendix A.1 ablation while holding out FARSEEING during token selection; if a token configuration chosen without peeking no longer beats SAX-only under transfer, the reported robustness is partly an artifact of selection.","tokens_in":13977,"feed_emoji":"📉","tokens_out":10004,"duration_ms":77281,"temperature":0.7,"pith_summary":"Falls are hard to detect in practice because real-world falls are extremely rare: collecting 100 of them takes an estimated 100,000 days of monitoring, so most models train on simulated falls and fail when deployed. This paper systematically compares four motion-representation families—interval, kernel, symbolic, and foundation-model—under subject-wise cross-validation, controlled data scarcity, and simulated-to-real transfer on accelerometer data. It finds that the methods with the best simulated performance, kernel and foundation models, degrade most under domain shift, while FallLM, a lightweight symbolic representation that augments SAX tokens with physically grounded impact descriptors, loses only 9.5% of its F1 score in transfer and still detects falls after training on as few as two real examples. The central lesson is that representation choice, not model capacity, governs deployability, and that in-domain accuracy alone is a poor guide to real-world robustness.","feed_headline":"Impact magnitude in g transfers where fall-detection models fail","feed_subtitle":"A symbolic detector keyed to physical impact levels drops only 9.5% F1 from simulated to real falls; top benchmarks drop 35-62%.","key_machinery":"The load-bearing mechanism is FallLM's impact token: the peak acceleration magnitude in each window is discretized into three levels—low (<1.8g), medium (1.8–2.5g), high (>2.5g)—following biomechanical ranges, plus a relative impact token comparing the peak to the window's own magnitude distribution. Because these thresholds are expressed in gravitational units, their meaning is invariant across datasets and sensor placements; an ablation shows this coarse invariant encoding transfers far better than either no magnitude cue or a 12-bin magnitude fitted to the training data. These tokens are appended to a three-symbol SAX (Symbolic Aggregate Approximation) representation of the 3-second fall window, vectorized as TF-IDF n-grams, and classified by logistic regression, making every decision reducible to human-readable motion motifs such as 'b d c impact_med'.","core_discovery":"The paper's central claim is that robustness to the simulated-to-real gap in fall detection is determined not by model capacity but by the type of prior a motion representation encodes. Representations that learn dataset-specific signal statistics—MiniRocket's random kernels, Mantis's transformer embeddings, WEASEL's SFA dictionaries—reach F1 above 0.95 on simulated FallAllD falls but fall to 0.37–0.50 on real FARSEEING falls, with the biggest losses in recall. FallLM, which encodes each 3-second window as three SAX symbols plus an absolute impact token thresholded at 1.8g and 2.5g and a relative impact token, transfers with the smallest F1 drop (9.5%, from 0.74 simulated to 0.67 real) and is the only method whose precision improves across domains. The same token gives measurable detection (mean F1 ≈ 0.35) when trained on just two real falls, while the best abundant-data methods collapse under extreme scarcity. The paper is explicit that FallLM is a proof-of-principle rather than a deployable detector: it is imprecise in-domain (F1 0.64) and biased toward high-impact falls, so the conclusion is about what makes representations transfer, not that this particular model is ready for clinical use.","pith_inferences":["We infer the principle generalizes beyond falls: any wearable event-detection task with scarce real labels could anchor features to physical units that are invariant across domains, such as g-force thresholds or joint-angle ranges.","We infer that a two-stage detector—keeping the invariant impact anchor for recall and adding a learned low-impact-sensitive stage—would address the two complementary failure modes the paper identifies.","We infer that the dataset-invariance claim for the 1.8g/2.5g boundaries is itself testable: re-running the transfer experiment on another real-world fall corpus with a different sensor placement (wrist, thigh, chest) would either support or bound the claim.","We infer that calibrating the impact thresholds on a small labelled real-world subset could further narrow the simulation-to-reality gap, since the paper deliberately studies only zero-shot transfer."],"forward_implications":["Cross-dataset evaluation should become a standard reporting requirement: selecting a fall detector on simulated performance alone chooses among the least deployable options.","Under extreme data scarcity, representation choice matters more than model capacity: FallLM reaches mean F1 around 0.35 from two real training falls, while the other methods stay near or at zero.","Representations anchored to dataset-invariant physical quantities, such as impact magnitude in g, transfer more gracefully than representations that model dataset-specific signal statistics.","A deployable detector will need to combine impact-anchored robustness with sensitivity to low-impact falls, because FallLM's transfer advantage comes with in-domain precision loss and a high-impact bias."],"supporting_citations":[{"why":"FARSEEING real-world fall repository supplies the clinically verified falls used for cross-validation, scarcity analysis, and transfer testing.","marker":"[11]"},{"why":"FallAllD supplies the simulated falls used for training in the cross-dataset transfer experiment.","marker":"[22]"},{"why":"Supplies the biomechanical impact ranges (1.8g and 2.5g) that define FallLM's absolute impact token.","marker":"[10]"},{"why":"Provides the streaming evaluation protocol with tolerance intervals and debounce used for realistic detection.","marker":"[1]"},{"why":"Supports the magnitude aggregation, windowing, and training-on-real-falls choices for the detection pipeline.","marker":"[20]"},{"why":"MiniRocket is the representative kernel-based baseline in the comparison.","marker":"[6]"},{"why":"QUANT is the representative interval-based baseline.","marker":"[7]"},{"why":"Mantis is the representative foundation-model baseline.","marker":"[8]"},{"why":"MrSQM is the SAX-based symbolic baseline used to separate the effect of SAX from the effect of impact tokens.","marker":"[18]"},{"why":"WEASEL is the SFA-based symbolic baseline whose transfer failure illustrates dataset-specific statistics.","marker":"[23]"}],"fun_headline_variants":["Physical impact tokens beat deep models on real falls","Symbolic fall detector drops only 9.5% F1 in real world","Real-world fall detection: simple symbolic wins, deep models fail","Under real data scarcity, physical impact cues transfer best","Deep models crumble on real falls; simple physical tokens hold"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusions rest on FARSEEING being representative of real-world falls and on the 1.8g and 2.5g impact thresholds being truly dataset-invariant; moreover, the 3-level token was chosen via its transfer performance on FARSEEING, so the headline transfer result is not a fully independent test.","fun_headline_variants_meta":{"raw":{"variants":["Physical impact tokens beat deep models on real falls","Symbolic fall detector drops only 9.5% F1 in real world","Real-world fall detection: simple symbolic wins, deep models fail","Under real data scarcity, physical impact cues transfer best","Deep models crumble on real falls; simple physical tokens hold"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000687,"raw_usage":{"total_tokens":3194,"prompt_tokens":1103,"completion_tokens":2091,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":719,"completion_tokens_details":{"reasoning_tokens":2006}},"tokens_in":719,"tokens_out":2091,"duration_ms":14174,"temperature":1.0,"reasoning_tokens":2006,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:36:57.255517+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train FallLM on FallAllD and evaluate it on a second clinically verified real-world fall dataset recorded at a different sensor placement, such as wrist or thigh; if the cross-dataset F1 advantage over QUANT and Mantis shrinks or reverses, the claim that the g-anchored thresholds are dataset-invariant is falsified. A more immediate check is to re-run the Appendix A.1 ablation while holding out FARSEEING during token selection; if a token configuration chosen without peeking no longer beats SAX-only under transfer, the reported robustness is partly an artifact of selection.","supporting_citations":[{"cited_title":"European review of aging and physical activity13, 1–7 (2016)","cited_arxiv_id":null,"evidence_quote":"FARSEEING real-world fall repository supplies the clinically verified falls used for cross-validation, scarcity analysis, and transfer testing."},{"cited_title":"IEEE Sensors Journal21(2), 1849–1858 (2020)","cited_arxiv_id":null,"evidence_quote":"FallAllD supplies the simulated falls used for training in the cross-dataset transfer experiment."},{"cited_title":"Signals2(1), 1–12 (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the biomechanical impact ranges (1.8g and 2.5g) that define FallLM's absolute impact token."},{"cited_title":"arXiv preprint arXiv:2509.11789 (2025)","cited_arxiv_id":null,"evidence_quote":"Provides the streaming evaluation protocol with tolerance intervals and debounce used for realistic detection."},{"cited_title":"Sensors20(22), 6479 (2020)","cited_arxiv_id":null,"evidence_quote":"Supports the magnitude aggregation, windowing, and training-on-real-falls choices for the detection pipeline."},{"cited_title":"In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining","cited_arxiv_id":null,"evidence_quote":"MiniRocket is the representative kernel-based baseline in the comparison."},{"cited_title":"Data Mining and Knowledge Discovery pp","cited_arxiv_id":null,"evidence_quote":"QUANT is the representative interval-based baseline."},{"cited_title":"In: International Workshop on Advanced Analytics and Learning on Temporal Data","cited_arxiv_id":null,"evidence_quote":"MrSQM is the SAX-based symbolic baseline used to separate the effect of SAX from the effect of impact tokens."},{"cited_title":"Machine Learning 112(12), 4763–4788 (2023)","cited_arxiv_id":null,"evidence_quote":"WEASEL is the SFA-based symbolic baseline whose transfer failure illustrates dataset-specific statistics."}],"review_version":1}