{"id":"6359ca2c-6d42-42e2-b830-26c53a8fc995","arxiv_id":"1908.07416","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An LSTM autoencoder reconstruction error, combined across X, Y, Z axes with inverse-training-error weights, distinguishes normal from abnormal gait with per-sequence AUC 0.969.","lead":"The paper proposes a gait quality score built from LSTM autoencoders trained on healthy walking skeletons. The score separates normal from simulated abnormal gaits on a treadmill dataset, beating two reimplemented baseline methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim rests on the unvalidated assumption that reconstruction error on normal-only training generalizes to new subjects; with only four test subjects and no per-subject analysis, the reported EER may reflect subject identity rather than gait abnormality.","rationale":"The reader identified the core load-bearing assumption: that an autoencoder trained only on normal gaits produces higher reconstruction error for abnormal gaits in unseen subjects. I agree, and I sharpen the concern by pointing out that the paper's pooled EER/AUC is computed from only four normal test sequences, and the absence of per-subject results means the reported separation could be inflated by subject-specific confounds. The proposed concrete test—per-subject AUC under the original split—would directly reveal whether the index works consistently across the four test subjects. This does not change the verdict: CONDITIONAL remains appropriate because the method is clearly described but its central generalization claim is under-supported.","tokens_in":7433,"tokens_out":8177,"duration_ms":77519,"concrete_test":"Using the original training split and the weighted-sum-plus-dropout index, compute and report the per-sequence AUC separately for each of the four test subjects (one normal vs. eight abnormal sequences per subject). If any subject's AUC is near 0.5, or if the per-subject AUCs vary widely, the pooled EER of 0.056 does not demonstrate that the reconstruction-error index generalizes across new subjects.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 states the load-bearing premise: 'An input of abnormal gait is expected to provide a bad reconstruction compared with normal gait.' The three LSTMs are trained exclusively on normal gaits from five subjects, and the test set contains four subjects, each contributing one normal and eight abnormal sequences. The reported per-sequence EER of 0.056 in Table 2 is therefore computed from only four normal sequences; the ROC curve is a step function and the EER threshold is determined by at most a handful of points. More importantly, the paper provides no per-subject results, so the pooled AUC of 0.969 could be driven by one or two subjects. Because the autoencoder has seen only five subjects' skeletons, reconstruction error may encode subject-specific joint-coordinate distributions (body proportions, distance to camera) rather than a general model of normal gait. If a new subject's normal sequences reconstruct poorly for subject-identity reasons, the index is not a reliable abnormality measure outside this experiment. The abstract's claim of outperforming recent works depends entirely on this unverified generalization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a gait-index estimator based on three LSTM autoencoders, one per coordinate axis, trained on normal gait skeleton sequences from five subjects. The reconstruction MSE of each autoencoder serves as a weak per-segment index; the three indices are combined with weights inversely proportional to training-set errors, and a per-sequence index is obtained by averaging segment indices. On a test set of four subjects (one normal and eight abnormal gaits each), the paper reports an EER of 0.056 per sequence, which it compares with reimplemented HMM and one-class SVM baselines and claims to outperform recent works. The central claim is empirical, not derivational, and rests on the assumption that reconstruction error generalizes from the training subjects to unseen subjects.","tokens_in":7593,"tokens_out":6663,"duration_ms":62294,"significance":"If the result holds, the method is a useful unsupervised alternative to hand-crafted gait features: it is simple, fully specified (LSTM equations, joint selection, weighting), and reports standard metrics (AUC, EER, accuracy, F1). The three-axis decomposition and inverse-error weighting are clearly described, and the experimental protocol (subject split, overlap parameters, segment lengths) is explicit enough to reproduce. However, the significance is conditional: the per-sequence result is based on only four normal test sequences, no per-subject or confidence-interval analysis is given, and the baselines are reimplementations rather than the original published methods. These gaps prevent the reported superiority from being established at the level claimed in the abstract.","major_comments":[{"comment":"The per-sequence EER of 0.056 reported in Table 2 is computed from only four normal sequences (one per test subject) and 32 abnormal sequences. The ROC curve is therefore a step function whose EER threshold is determined by roughly two to three sequences, so the difference from the baseline EER of 0.139 is within the sampling noise of this test design. Please report per-subject ROC curves, bootstrap confidence intervals, or a leave-one-subject-out evaluation to support the comparative claim.","section":"Section 3.1 and Table 2"},{"comment":"The load-bearing premise, stated as 'An input of abnormal gait is expected to provide a bad reconstruction compared with normal gait', is not directly verified for unseen subjects. Because the autoencoder is trained only on normal gaits from five subjects, a new subject's normal sequences may produce high reconstruction errors due to body proportions, joint-angle biases, or camera distance rather than gait abnormality. The paper provides no per-subject analysis, so the pooled AUC of 0.969 could be driven by subject identity. Please provide per-subject index distributions and demonstrate that, for each test subject, normal sequences are separated from abnormal ones.","section":"Section 3.2"},{"comment":"The comparison with related methods is not a comparison with the original published works. The paper states that it reimplemented [3] as a one-class SVM and [7] as an HMM, whereas the original [3] used a binary SVM with a temporal trigger and supervised training. The abstract's claim of outperforming 'some recent works' is therefore an overstatement if these baselines are not the published methods. Please either reimplement the original methods or explicitly frame the results as 'our method vs. our reimplemented one-class/HMM baselines', and temper the conclusion accordingly.","section":"Section 3.3"}],"minor_comments":[{"comment":"The abstract describes the dataset as 'nearly one hundred thousand skeletons'; this refers to the number of frames, not the number of independent subjects (nine), which is the sample size that limits the conclusions. Please rephrase to avoid overstatement.","section":"Abstract and Section 3.1"},{"comment":"The notation e^{-1}_k is confusing; writing 1/e_k would be clearer and would avoid confusion with matrix inversion.","section":"Equation (7)"},{"comment":"The normalization step ('the data range along each axis is normalized by scaling it into [0, 1]') does not state whether the scaling is per sequence or computed globally over the training set. This affects whether subject-specific distance information is removed and should be clarified for reproducibility.","section":"Section 2.1"},{"comment":"The t-SNE visualizations of LSTM weights are difficult to quantify, and the statement that dropout 'seemed to improve' the mapping is not supported numerically. A comparison of weight statistics (e.g., mean and variance of gate weights) before and after dropout would be more convincing.","section":"Section 3.2 and Figure 6"},{"comment":"The manuscript does not report hyperparameters such as the optimizer, learning rate, weight initialization, or mini-batch size. Please include them for reproducibility.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is already published as a conference paper, and the journal version should either substantially extend the evaluation or clearly frame the contribution as a preliminary study. The confidence in the comparative claim is limited by the small number of subjects and by the baseline reimplementation choices. The method itself is reasonable and clearly described, so a major revision with additional per-subject analysis and a more careful baseline framing is feasible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent and clearly written application of a standard idea. Three axis-wise LSTM autoencoders are trained on normal gait only; reconstruction error is the weak index; the three errors are combined with weights inversely proportional to training MSE. That weighted sum plus dropout gives an EER of 0.056 per sequence, versus 0.139 for a reimplemented one-class SVM and 0.250 for an HMM. The core idea is not new — autoencoder reconstruction error as an anomaly score is textbook — but the specific combination of axis-wise decomposition and inverse-training-error weighting for skeleton gait index is a reasonable twist, and the paper is honest about what it does. What it does well: the method is simple to reproduce from the description, the training details are concrete (T=12, 256 hidden units, 100 epochs, dropout 0.5), and the paper reports per-segment and per-sequence results with AUC, EER, precision, specificity, sensitivity, accuracy, and F1. The observations that the Z-axis model is weaker and that weighting and temporal smoothing help are internally coherent. Comparing against their own earlier HMM work is a legitimate baseline choice, not a citation problem. Soft spots, in proportion: the evaluation is the whole ballgame and it is thin. Nine subjects total, four in the test set. The per-sequence EER of 0.056 is computed from four normal sequences and 32 abnormal sequences, so the ROC is essentially a step function and one or two subjects could be driving the pooled AUC. There are no error bars, no per-subject breakdown, and no released code or data. The load-bearing assumption — that an autoencoder trained on normal gaits from five subjects will produce higher reconstruction error on abnormal gaits from unseen subjects — is not independently checked. Because each axis is normalized to [0,1], distance-to-camera is partly mitigated, but subject-specific joint-coordinate patterns remain a plausible confound. The stress-test concern is real rather than manufactured; the missing per-subject table is the natural fix. Baseline reimplementation uncertainty is a minor issue for a conference paper. Bottom line: for someone in rehabilitation or fall-risk screening who wants a cheap gait abnormality score from a single depth camera, this is a useful existence proof and worth engaging with. If it were submitted today it would deserve peer review, but with a request for a larger evaluation and per-subject reporting. My verdict would be conditional acceptance, not a clean accept.","headline":"A modest, clearly written application of LSTM autoencoders to gait scoring, with a plausible result undermined by a very small test set and no per-subject analysis.","tokens_in":708,"tokens_out":1803,"would_cite":false,"duration_ms":33308,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A gait index built entirely from LSTM auto-encoder reconstruction errors separates normal from abnormal walking with an equal-error rate of 0.056 per sequence.","keywords":["gait index estimation","skeleton-based gait analysis","LSTM autoencoder","abnormal gait detection","reconstruction error","unsupervised anomaly detection","Kinect skeleton","equal error rate"],"falsifier":"Run the same protocol on a held-out cohort with several subtle abnormal gaits (for instance, a 2-centimeter sole lift), and compute per-sequence weighted reconstruction indices; if the ROC AUC for separating normal from abnormal is not near the reported 0.969, or if mildly abnormal gaits overlap normal indices, the central assumption is falsified.","tokens_in":7197,"feed_emoji":"🚶","tokens_out":7733,"duration_ms":71523,"temperature":0.7,"pith_summary":"This paper proposes a way to turn a skeleton video into a single number that measures how abnormal a person's gait is. The system is a stack of three LSTM auto-encoders, one for each spatial axis, trained only on normal walking sequences. Each auto-encoder reconstructs short segments of the selected 17 joints, and its mean-squared error is treated as a weak gait index; the three weak indices are combined by a weighted sum, then averaged over the whole 1200-frame sequence. On a dataset of nearly one hundred thousand skeletons, the resulting index separates normal from abnormal walking with an equal-error rate of 0.056 per sequence, against 0.139 for a one-class SVM baseline and 0.250 for an HMM baseline. The paper's point is that unsupervised reconstruction error, with no hand-crafted gait features, can serve as a competitive gait quality score.","feed_headline":"Gait abnormality score from LSTM autoencoders hits 5.6% error","feed_subtitle":"Normal-only training flags abnormal walking from skeleton streams, beating SVM and HMM baselines.","key_machinery":"The central object is the LSTM auto-encoder with peephole connections: an encoder LSTM compresses a temporal sequence of joint coordinates into a latent representation, and a decoder LSTM reconstructs the input in reverse order from that representation. The mean-squared error between input and reconstruction is the weak gait index, under the assumption that normal patterns are reconstructed better than abnormal ones. Two auxiliary mechanisms carry the argument: the axis-wise decomposition (17 selected joints, three separate auto-encoders for X/Y/Z), and the weighted combination $w_k = e_k^{-1} / \\sum_k e_k$ in which a model with smaller training MSE contributes more. A dropout layer on the encoder input slightly improves the score, and averaging per-segment indices over long sequences converts the noisy weak indices into the final gait index.","core_discovery":"On the paper's own terms, the discovery is that the reconstruction error of an LSTM auto-encoder trained exclusively on normal gait is a usable gait index. An abnormal gait is expected to lie outside the learned distribution of normal motion and therefore to be reconstructed poorly, producing a higher index. The authors realize this with three independently trained peephole-LSTM auto-encoders processing the X, Y, and Z coordinate streams separately; per-axis MSEs are weak indices, and the final index is the weighted sum $w_k = e_k^{-1} / \\sum_k e_k$ over axes. Averaging per-segment indices over a trial yields an AUC of 0.969 and an EER of 0.056, lower than the reimplemented HMM and one-class SVM baselines. The conclusion is that an automatic, unsupervised index can outperform recent hand-crafted gait analysis methods on this task.","pith_inferences":["The same reconstruction-error recipe could transfer to other movement-quality domains, such as rehabilitation exercise form or fall risk screening, because it only requires a stream of body joints and a definition of normal motion; this extension is not tested in the paper.","The weak Z-axis index hints that depth-channel noise or treadmill positioning dominates reconstruction error; a reader might expect a joint model or per-axis adaptive weighting to improve robustness.","With only four subjects in the test set, the reported 0.056 EER is likely optimistic for unseen populations; a realistic deployment would need validation across body shapes, speeds, and camera viewpoints."],"forward_implications":["Abnormal-gait screening can be run without collecting or labeling abnormal samples at training time, since only normal sequences are needed.","The per-segment design supports online or streaming assessment, with a smoothed index obtained by averaging over a longer walk.","Because the recurrent auto-encoders accept variable-length inputs, the same pipeline can adapt to different walking speeds or camera frame rates.","The comparison suggests that a purely reconstruction-based, unsupervised index can beat hand-crafted-feature baselines on the abnormal-gait detection task."],"supporting_citations":[{"why":"Supplies the peephole LSTM cell used in both encoder and decoder.","marker":"[4]"},{"why":"Supplies the Kinect-based 3D skeleton extraction that produces the 25-joint input.","marker":"[12]"},{"why":"Provides the one-class SVM silhouette baseline reimplemented for comparison.","marker":"[3]"},{"why":"Provides the HMM skeleton baseline reimplemented for comparison, the closest prior approach.","marker":"[7]"},{"why":"Motivates placing a dropout layer at the encoder input, which improves the index.","marker":"[17]"},{"why":"Supplies the dropout technique used to reduce overfitting in the recurrent model.","marker":"[13]"}],"fun_headline_variants":["LSTM autoencoder reconstruction error flags abnormal gait","Gait abnormality from LSTM autoencoder: 5.6% EER","LSTM autoencoder scores gait: outperforms HMM, SVM","Unsupervised gait index from LSTM reconstruction error","Gait index via LSTM autoencoder: AUC 0.969"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an auto-encoder trained only on normal gait will reconstruct abnormal gait poorly; if an abnormal walking pattern is close enough to normal motion in the skeleton stream, its reconstruction error will look normal and the index will not flag it.","fun_headline_variants_meta":{"raw":{"variants":["LSTM autoencoder reconstruction error flags abnormal gait","Gait abnormality from LSTM autoencoder: 5.6% EER","LSTM autoencoder scores gait: outperforms HMM, SVM","Unsupervised gait index from LSTM reconstruction error","Gait index via LSTM autoencoder: AUC 0.969"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000395,"raw_usage":{"total_tokens":2019,"prompt_tokens":841,"completion_tokens":1178,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":1090}},"tokens_in":457,"tokens_out":1178,"duration_ms":8394,"temperature":1.0,"reasoning_tokens":1090,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:48:01.606739+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same protocol on a held-out cohort with several subtle abnormal gaits (for instance, a 2-centimeter sole lift), and compute per-sequence weighted reconstruction indices; if the ROC AUC for separating normal from abnormal is not near the reported 0.969, or if mildly abnormal gaits overlap normal indices, the central assumption is falsified.","supporting_citations":[{"cited_title":"Gers, Nicol N","cited_arxiv_id":null,"evidence_quote":"Supplies the peephole LSTM cell used in both encoder and decoder."},{"cited_title":"Shotton, A","cited_arxiv_id":null,"evidence_quote":"Supplies the Kinect-based 3D skeleton extraction that produces the 25-joint input."},{"cited_title":"Automatic detection of abnormal gait","cited_arxiv_id":null,"evidence_quote":"Provides the one-class SVM silhouette baseline reimplemented for comparison."},{"cited_title":"Skeleton-based abnormal gait detection","cited_arxiv_id":null,"evidence_quote":"Provides the HMM skeleton baseline reimplemented for comparison, the closest prior approach."},{"cited_title":"Dropout: A simple way to prevent neural networks from overfitting","cited_arxiv_id":null,"evidence_quote":"Supplies the dropout technique used to reduce overfitting in the recurrent model."}],"review_version":1}