{"id":"f4df78c8-13cc-46a9-947a-89b8449201d7","arxiv_id":"2507.00050","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"SEZ-HARN recognizes unseen activities from IMU data and generates skeleton videos as explanations, but the videos are class-conditional renderings rather than faithful explanations of the decision.","lead":"This paper adds skeleton-video generation to a zero-shot wearable activity recognizer, so the system both predicts an activity and draws a stick-figure video of it. The predictions match earlier black-box models on three of four datasets, but the videos visualize the predicted activity rather than demonstrate the sensor evidence behind the decision.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The explainability claim rests on an untested faithfulness assumption: Eq. (5) trains a decoder to render class skeletons, and the reported TSA/PSA/OA, DFD, and user-study metrics only measure super-class legibility, so a class-template generator would pass all of them.","rationale":"The reader's rejection hinges on this same point, and my reading confirms it. The recognition accuracy results in Table 3 may be genuine, and the paper is honest about borrowing baseline numbers and about dataset limitations, but the headline contribution is explainability. Eq. (5) is where the explanation is trained, and it is constructed so that the class semantic vector can dominate the output: \\bar{h}_v is directly supervised toward h_c, the skeleton of class c. The evaluation metrics never test counterfactual sensitivity to the IMU input. TSA/PSA/OA check super-class membership of a DTW match; DFD checks realism relative to a reference skeleton; the user study checks whether humans recognize the activity. A trivial baseline that always outputs the skeleton for the predicted class would score well on all of them. Thus the paper's central claim collapses unless faithfulness is demonstrated. The proposed ablation would settle it: it directly measures whether the IMU evidence changes the explanation. This is not a disagreement with external consensus; it is an internal gap between the claim that the model explains its decision-making process and the metrics used to operationalize that claim. If the ablation showed strong IMU dependence, the paper could be revised toward conditional acceptance; as written, the reader's REJECT verdict is appropriate.","tokens_in":13178,"tokens_out":4704,"duration_ms":51977,"concrete_test":"Run a controlled ablation on PAMAP2: for the same unseen IMU samples, generate explanations with (a) the full decoder, (b) a decoder conditioned only on the predicted class semantic vector v_c with the IMU branch zeroed, and (c) a decoder conditioned only on the IMU feature f_{xs} with the semantic branch zeroed. Compute per-sample DTW/DFD to seen-class skeletons and rerun the user study on all three conditions. If (a) and (b) are statistically indistinguishable while (a) and (c) differ, the skeleton is a class rendering and the explainability claim fails; if (a) tracks (c), the IMU evidence is doing real explanatory work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SEZ-HARN explains its decision-making process by generating skeleton videos. For that claim to hold, the generated skeleton must depend on the specific IMU evidence that drove the prediction, not merely render the predicted class. The paper never establishes this. In Eq. (5), the reconstruction loss LR = ||\\bar{h}_f - h_c||^2 + ||\\bar{h}_v - h_c||^2 trains the decoder to reproduce h_c, the skeleton extracted from a video of the true seen class c, with \\bar{h}_v conditioned on the class semantic vector v_{ys}. Thus the decoder learns to output class-typical skeletons. At inference, the manuscript does not specify precisely what conditions the decoder, but the evaluation never isolates the contribution of the IMU feature f_{xs} to the generated video. The TSA/PSA/OA metrics in Sec. 4.4 compare the generated skeleton to seen-class skeletons via DTW and check only whether the nearest seen class lies in the same super-class as the target or predicted class. DFD in Sec. 4.5 compares realism to a matching seen-class skeleton. The user study in Sec. 4.6 asks participants to identify the super-class. Every reported measure is consistent with a decoder that ignores the IMU input and outputs a class template for the predicted class. Legibility is not faithfulness. The self-explainability claim is therefore unsupported; without it, the paper's novelty reduces to adding a skeleton renderer to TEZARNet.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes SEZ-HARN, a zero-shot human activity recognition model that takes IMU time series as input, builds a semantic space from auxiliary activity videos, and generates skeleton videos as explanations for its predictions. The training loss combines alignment, classification, and skeleton reconstruction terms, and the paper reports unseen-class accuracy on PAMAP2, DaLiAc, UTD-MHAD, and MHEALTH, together with DTW- and DFD-based explanation metrics and a user study. The authors claim that this is the first IMU-based ZS-HAR framework with self-explainability through skeleton-based activity videos.","tokens_in":13512,"tokens_out":8470,"duration_ms":76703,"significance":"If the explanation faithfulness claim were supported, SEZ-HARN would be a useful contribution to zero-shot HAR and to XAI for wearable sensing: the skeleton-video output is an intuitive interface for lay users, and the authors state that code and collected video data are publicly available, which is a reproducibility strength. The user study is a reasonable complement to automatic metrics. However, the current evidence only supports that the model can render class-typical, human-recognizable skeletons; it does not support the claim that these skeletons explain the model's decision. The central novelty of the paper is therefore not yet demonstrated.","major_comments":[{"comment":"The self-explainability claim is not established. The reconstruction loss in Eq. (5) trains the decoder to reproduce the seen-class skeleton h_c using both the IMU feature f_xs and the class semantic vector v_ys, and the TSA/PSA/OA metrics in Sec. 4.4 only test whether the nearest seen-class skeleton lies in the same super-class as the target or predicted class. A decoder that ignores f_xs and renders a class template for the predicted class would pass all reported explanation metrics, including the DFD and user-study evaluations. This concern is reinforced by Sec. 5, which lists 'improving the explanation mechanism to highlight salient motion patterns that influence model decisions' as future work. Please add a counterfactual test that fixes the class conditioning and perturbs or replaces f_xs and measures the resulting change in the generated skeleton, or reframe the contribution as class-conditional skeleton visualization rather than self-explainability.","section":"Sec. 3, Eq. (5); Sec. 4.4; Sec. 5"},{"comment":"The accuracy comparison is weakened by mixing numbers borrowed from prior papers for MLCLM and TEZARNet with a self-implemented VbZSL, and by omitting error bars or significance tests. The abstract's claim of accuracy 'within 3%' of the best black-box model on PAMAP2 is also contradicted by the table, where the gap to TEZARNet is 58.27 - 55.20 = 3.07 percentage points. Please re-evaluate all baselines under identical fold splits and report mean and standard deviation across folds, or soften the accuracy claims accordingly.","section":"Sec. 4.3, Table 3"},{"comment":"The realism (DFD) and user-study metrics measure whether the generated videos look like natural skeletons and can be recognized by human viewers, not whether they are faithful to the model's decision-making process. Because the decoder is trained to output class-typical skeletons, high DFD quality and high user accuracy in identifying super-classes are expected from a class-conditional renderer. To support the 'self-explainable' label, the paper needs an evaluation that ties the generated skeleton to the specific IMU input, or a carefully stated weaker claim.","section":"Sec. 4.5 and Sec. 4.6"}],"minor_comments":[{"comment":"The dataset name is written as 'HTD-MHAD' in the abstract and as 'DiLiAc' in the conclusion; both should be 'UTD-MHAD' and 'DaLiAc', respectively.","section":"Abstract and Sec. 6"},{"comment":"The phrase 'Dynamic Time Wrapping' should be 'Dynamic Time Warping'.","section":"Sec. 4.4"},{"comment":"The Discrete Fréchet Distance formula appears garbled; please rewrite it with explicit norms and quantifiers so that the computation is unambiguous.","section":"Eq. (10)"},{"comment":"The column header 'F olds' and the dataset label 'MHEAL TH' contain typos; also, the dashes for MLCLM on three datasets in Table 3 should be explained (e.g., not reported in the original paper) rather than left as dashes.","section":"Table 1 and Table 3"},{"comment":"The text refers to 'Kinetic-400' (should be Kinetics-400) and to an 'NVIDIA GeForce RTX 2040', which is not a standard GPU model.","section":"Sec. 4.2"},{"comment":"The inference procedure does not state explicitly whether the decoder is conditioned on f_xs, on the predicted class semantic vector, or on both; please specify this in the text and figure.","section":"Sec. 3 and Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the explanation evaluation is circular with respect to the class conditioning in Eq. (5). I believe the paper can be strengthened by adding counterfactual faithfulness experiments and a proper same-protocol baseline comparison; if the authors choose to keep the current claims without those experiments, the contribution would not meet the bar for this journal. Please also verify that the GitHub and video-data links are functional, since the reproducibility claims rely on them."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: the paper adds a skeleton-video decoder to TEZARNet and calls the output an explanation of the model's decision. That framing doesn't survive contact with the evaluation. The decoder is trained to reproduce the skeleton of the class whose semantic vector is supplied (Eq. 5), so the videos are class renderings. The TSA/PSA/OA metrics, the Frechet distances, and the user study all only check whether the rendering is a recognizable instance of the predicted super-class. A decoder that ignores the IMU input and always outputs the predicted class's template would pass them all. Faithfulness is never measured.\n\nWhat's genuinely new: the DTW/DFD evaluation metrics for explanation quality in ZS-HAR are not in the prior work, and the user study is a reasonable first step. The authors ship code and collected video data, which is more than most papers in this area do. The recognition results are plausible and consistent with the authors' own TEZARNet baseline, and the paper is honest that performance is on par rather than better.\n\nWhere it's soft: the load-bearing problem is the one above. The abstract says 'within 3%' of the best black-box model on PAMAP2, but Table 3 puts the gap at 3.07 points, a small overstatement. Baseline numbers are borrowed from prior papers without error bars, and VbZSL is a reimplementation, so the accuracy comparisons have unknown variance. The explanation metrics validate legibility, not faithfulness, and the paper never attempts an intervention to show the IMU signal affects the output.\n\nBottom line: for someone working on explainable zero-shot HAR, this is a useful data point and a cautionary example of how easy it is to oversell a class-conditional generator as an explanation. But the central claim, as stated, is not supported. I'd send it to review because the contribution is real and the flaw is fixable — either by an ablation that perturbs the IMU input, or by reframing the videos as visualizations rather than explanations of the decision. I wouldn't cite it in my own work until that's resolved.","headline":"Skeleton-video explanations are class renderings, not faithful explanations; the recognition part is a plausible TEZARNet extension.","tokens_in":14055,"tokens_out":2531,"would_cite":false,"duration_ms":27827,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SEZ-HARN recognizes activities never seen in training and, for each prediction, renders a skeleton video that explains what it thinks it saw.","keywords":["Human Activity Recognition","Zero-shot Learning","Inertial Measurement Unit data","Explainable AI","Self-explainable models","Skeleton video generation","Video semantic embeddings","Wearable sensors"],"falsifier":"Scramble the time order of an unseen-class IMU window that SEZ-HARN classified correctly, regenerate the explanation, and compare it with the skeleton produced from the intact window; if the skeleton is nearly unchanged whenever the predicted class is unchanged, the video renders the class rather than the sensor evidence, and the self-explainability claim fails. The paper's TSA, PSA, and OA metrics would not catch this, because the seen-class skeletons they match against are the same class-typical skeletons the decoder was trained to reproduce, so high alignment partly measures self-consistency.","tokens_in":12976,"feed_emoji":"🦴","tokens_out":26312,"duration_ms":232162,"temperature":0.7,"pith_summary":"SEZ-HARN tries to solve two problems in wearable-sensor activity recognition at once: models trained on labeled data fail on activities they have never seen, and the models that work give no reason a caregiver can check. The paper claims that a single network can recognize never-seen activities from inertial sensor data and, for each prediction, render an animated skeleton video of the activity it believes it recognized, making it the first IMU-based zero-shot recognizer whose explanation is a human-viewable video rather than a heatmap or saliency map. The competitive claim is that this explainability costs almost no accuracy: on the PAMAP2 benchmark the zero-shot accuracy is 55.20 percent against 58.27 percent for the best black-box comparator, with parity or better on the other three datasets. The paper further claims the generated skeletons are realistic and understandable, and supports that with new distance-based metrics and a 53-person user study. If these claims hold, healthcare and assisted-living systems could flag novel activities and let a lay observer verify the flag by eye.","feed_headline":"SEZ-HARN names unseen activities, shows its work as skeleton video","feed_subtitle":"First IMU activity model to explain zero-shot predictions as video, within 3 points of the best black-box on PAMAP2.","key_machinery":"The mechanism that carries the argument is the three-term objective $L = L_M + \\lambda L_C + \\alpha L_R$. Matching loss $L_M = \\|f_{x_s} - v_{y_s}\\|^2$ pulls the Bi-LSTM's IMU embedding toward the semantic vector of the true class; classification loss $L_C$ is a softmax negative log-likelihood over seen classes; and reconstruction loss $L_R = \\|\\bar{h}_f - h_c\\|^2 + \\|\\bar{h}_v - h_c\\|^2$ forces the skeleton decoder to regenerate the class skeleton $h_c$ from both the IMU features and the class vector. Since $h_c$ is derived from video of the class through BlazePose (25 keypoints reduced to 12), this final term is what lets the decoder later turn the semantic vector of a never-seen class into a skeleton video at inference. Supporting components are the Bi-LSTM IMU encoder, the I3D video encoder that averages per-video features into class semantic vectors, and the decoder of a bidirectional recurrent skeleton autoencoder; on the evaluation side, Dynamic Time Warping with Mahalanobis distance defines the super-class alignment metrics TSA (Target Super-class Alignment), PSA (Predicted Super-class Alignment), and OA (Overall Alignment), while Discrete Fréchet Distance scores the realism of the rendered skeletons.","core_discovery":"The central claim, stated on the authors' terms, is that self-explainability can be built into an IMU-based zero-shot activity recognizer instead of bolted on afterwards. SEZ-HARN embeds a window of inertial readings with a Bi-LSTM encoder, aligns that embedding with class semantic vectors built by a pre-trained video encoder (I3D) from example videos of each activity, and classifies an unseen sample by whichever class vector its embedding is closest to. The same network, through a skeleton decoder, then renders the predicted class as a moving skeleton video, and the decoder is not an add-on: the reconstruction loss in Eq. (5) trains it to regenerate a class's skeleton from both the IMU features and the class semantic vector, so the explanation is produced by the same representation that drove the classification. The empirical claim is that this does not cost accuracy—55.20 percent average per-class accuracy on PAMAP2 versus 58.27 percent for TEZARNet, the strongest black-box baseline, and parity or better on DaLiAc, UTD-MHAD, and MHEALTH—while the generated videos meet the paper's new alignment and realism criteria and were identified correctly by participants in a user study.","pith_inferences":["A stricter test than the paper's own alignment metrics would fix the predicted class and vary the IMU signal, checking whether the decoded skeleton changes with the sensor evidence; near-unchanged skeletons would mean the videos illustrate classes rather than explain decisions.","The user study and realism scores would keep passing even if the skeleton were a pure class rendering, which is exactly why the invariance test described above is the decisive experiment for the self-explainability claim.","The same recipe—align a non-visual sensor embedding with video-derived class vectors, then decode the winning vector into a class exemplar—could carry self-explanatory zero-shot recognition into other modalities such as radar, audio, or ambient sensors.","The paper's own DTW alignment score could be repurposed as a per-instance confidence estimate: a prediction whose explanation fails to match any seen class in its super-class is likely a misclassification, giving deployments a cheap way to flag outputs that need human review."],"forward_implications":["A caregiver-facing monitor could name a never-seen activity and simultaneously show a skeleton animation of it, so a human can compare the model's claim against the patient's actual movement instead of trusting an opaque score.","Because the explanation comes from the same semantic space used for classification, zero-shot systems no longer need a separate post-hoc explanation step, removing the possibility that the explanation describes a different model than the one that predicted.","Video-derived semantic vectors would do double duty in any deployment: they carry the knowledge transfer to unseen classes and anchor the skeleton renderer, so recognition and explanation rise and fall together.","The new evaluation tools (TSA, PSA, and OA based on Dynamic Time Warping, plus Discrete Fréchet Distance) give subsequent self-explaining recognizers a way to measure whether a generated explanation is aligned with the prediction and realistic, rather than merely plausible.","The PAMAP2 gap of about three percentage points below TEZARNet, together with parity or better on the other three datasets, suggests that on current benchmarks self-explainability does not exact a systematic accuracy toll."],"supporting_citations":[{"why":"VbZSL, the zero-shot HAR model that introduced video-embedding semantic spaces; SEZ-HARN extends this idea and must beat this baseline on all four datasets.","marker":"[11]"},{"why":"TEZARNet, the strongest black-box comparator; its 58.27 percent PAMAP2 score defines the headline claim of accuracy within about 3 points.","marker":"[23]"},{"why":"The bidirectional recurrent autoencoder whose decoder is reused as SEZ-HARN's skeleton generator, making the explanation mechanism possible.","marker":"[35]"},{"why":"BlazePose, which extracts the 25 (reduced to 12) skeleton keypoints used as the reconstruction target $h_c$.","marker":"[36]"},{"why":"I3D, the pre-trained video encoder that produces the class semantic vectors from auxiliary activity videos.","marker":"[38]"},{"why":"Supplies the Dynamic Time Warping with Mahalanobis distance behind the new TSA, PSA, and OA explanation-alignment metrics.","marker":"[43]"},{"why":"Supplies the Discrete Fréchet Distance used to score how realistic the generated skeleton videos are.","marker":"[45]"},{"why":"PAMAP2, the benchmark dataset behind the headline result that zero-shot accuracy stays within about 3 points of the best black-box model.","marker":"[6]"},{"why":"DaLiAc, where SEZ-HARN's 76.41 percent matches TEZARNet's 76.10 percent, supporting the claim that explainability costs no accuracy.","marker":"[4]"},{"why":"MHEALTH, where SEZ-HARN's 46.67 percent beats TEZARNet's 40.40 percent, the strongest evidence that self-explainability does not hurt recognition.","marker":"[18]"}],"fun_headline_variants":["Zero-shot HAR model explains via skeleton videos","SEZ-HARN: self-explaining activity recognition for unseen actions","IMU model recognizes new activities and renders skeleton demos","Self-explainable zero-shot HAR with video explanations","SEZ-HARN: zero-shot recognition with built-in skeleton video explanations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The explanation claim rests on one assumption: the skeleton video regenerated from the predicted class's description reflects the sensor readings that produced the prediction, rather than being a generic illustration of that activity label.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot HAR model explains via skeleton videos","SEZ-HARN: self-explaining activity recognition for unseen actions","IMU model recognizes new activities and renders skeleton demos","Self-explainable zero-shot HAR with video explanations","SEZ-HARN: zero-shot recognition with built-in skeleton video explanations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000419,"raw_usage":{"total_tokens":2201,"prompt_tokens":1033,"completion_tokens":1168,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":1086}},"tokens_in":649,"tokens_out":1168,"duration_ms":11422,"temperature":1.0,"reasoning_tokens":1086,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:57:46.376130+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Scramble the time order of an unseen-class IMU window that SEZ-HARN classified correctly, regenerate the explanation, and compare it with the skeleton produced from the intact window; if the skeleton is nearly unchanged whenever the predicted class is unchanged, the video renders the class rather than the sensor evidence, and the self-explainability claim fails. The paper's TSA, PSA, and OA metrics would not catch this, because the seen-class skeletons they match against are the same class-typical skeletons the decoder was trained to reproduce, so high alignment partly measures self-consistency.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"VbZSL, the zero-shot HAR model that introduced video-embedding semantic spaces; SEZ-HARN extends this idea and must beat this baseline on all four datasets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TEZARNet, the strongest black-box comparator; its 58.27 percent PAMAP2 score defines the headline claim of accuracy within about 3 points."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The bidirectional recurrent autoencoder whose decoder is reused as SEZ-HARN's skeleton generator, making the explanation mechanism possible."},{"cited_title":"Carreira, A","cited_arxiv_id":null,"evidence_quote":"I3D, the pre-trained video encoder that produces the class semantic vectors from auxiliary activity videos."},{"cited_title":"Switonski, H","cited_arxiv_id":null,"evidence_quote":"Supplies the Dynamic Time Warping with Mahalanobis distance behind the new TSA, PSA, and OA explanation-alignment metrics."},{"cited_title":"Devogele, L","cited_arxiv_id":null,"evidence_quote":"Supplies the Discrete Fréchet Distance used to score how realistic the generated skeleton videos are."},{"cited_title":"Reiss, D","cited_arxiv_id":null,"evidence_quote":"PAMAP2, the benchmark dataset behind the headline result that zero-shot accuracy stays within about 3 points of the best black-box model."},{"cited_title":"Leutheuser, D","cited_arxiv_id":null,"evidence_quote":"DaLiAc, where SEZ-HARN's 76.41 percent matches TEZARNet's 76.10 percent, supporting the claim that explainability costs no accuracy."},{"cited_title":"Banos, R","cited_arxiv_id":null,"evidence_quote":"MHEALTH, where SEZ-HARN's 46.67 percent beats TEZARNet's 40.40 percent, the strongest evidence that self-explainability does not hurt recognition."}],"review_version":1}