{"id":"5dacf857-25f8-450b-8997-c548bd49099d","arxiv_id":"2607.27635","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A transformer with AttentiveCAT yields class-specific, time-step importance scores for wearable health data, beats deep-learning baselines, and beats random time-step selection in masking tests.","lead":"HealthCAT adds a class-activation interpretation layer to an encoder-only transformer and produces time-step-level importance scores for predicting health outcomes from wearable sensor data. On two datasets it beats three deep-learning baselines in F1 and accuracy, and the time steps it flags retain more predictive signal under masking than randomly chosen ones.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Uncalibrated GENEActiv intensity thresholds may render the MVPA/LPA/SED features and all downstream time-step interpretations invalid in this Pacific Island adolescent cohort.","rationale":"The reader's weakest assumption exactly identifies the most load-bearing threat to the paper's claimed contribution. The central value proposition is not just that AttentiveCAT scores are class-specific, but that they reveal 'when and how physical activity patterns occur' in a health-relevant way. That promise is broken if the underlying features misclassify activity intensity. The threshold issue is concrete, directly affects the PA dataset's features and every downstream attribution, and is not mentioned in the limitations. Other criticisms (statistical underspecification of the masking trials, code unavailability, confounded Transformer comparison) are valid but are either more easily addressed or less fundamental to the interpretability claim. A recalibration test would settle whether the domain-adapted visualizations are artifacts.","tokens_in":16236,"tokens_out":14180,"duration_ms":158651,"concrete_test":"Obtain raw GENEActiv data for a subsample of Pacific Island adolescents (or published calibration data) and derive population-specific cut-points via ROC against a reference measure (e.g., indirect calorimetry or validated age-specific cut-points). Recompute the hourly MVPA/LPA/SED features for the PA dataset using these new thresholds and rerun the full HealthCAT pipeline (training, masking, visualization). If the AttentiveCAT vs. random masking advantage disappears or the time-of-day importance maps shift substantially, the interpretation is not robust to threshold choice.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section III-B adopts fixed 3-second MVPA/LPA and 60-second SED bout thresholds and intensity cut-points from [50]/[51] without recalibrating them for Pacific Island adolescents. Those cut-points were calibrated in other child/adolescent populations, and body-composition or activity-pattern differences could shift true MVPA/SED boundaries. Since the model's multivariate time series are bout counts based on these thresholds, every AttentiveCAT importance score and the time-of-day visualizations in Section III-D are computed on features that may not correspond to actual physical activity intensity. The central interpretive claim—'when and how physical activity patterns occur'—then rests on threshold artifacts, and the paper neither acknowledges nor controls for this in the reported results.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HealthCAT, an Encoder-only Transformer combined with AttentiveCAT class-activation scores, to produce class-specific, time-step-level importance scores for wearable sensor time series. The framework is evaluated on two real-world datasets: a physical activity dataset of 206 Pacific Island adolescents (predicting healthy weight status) and the DREAMT sleep dataset of 100 adults (predicting AHI-based sleep health). The authors report that HealthCAT outperforms vanilla Transformer, GRU, and LSTM baselines by up to 17% in F1 score, and that AttentiveCAT-guided masking retains more predictive information than random masking. They also present hour-of-day visualizations of MVPA/LPA/SED contributions for two example participants.","tokens_in":16471,"tokens_out":6794,"duration_ms":71785,"significance":"If the central claims are correct, the paper provides a practical step beyond aggregated wearable summaries: class-specific, time-step-level interpretations that can be mapped onto domain-relevant cycles. The use of two real-world datasets and a quantitative masking protocol are strengths, and the domain-adapted visualization is a useful template for health researchers. However, the current evaluation is mostly internal: the masking experiments demonstrate that the attribution scores track the model's own decisions, not that the identified time steps correspond to externally meaningful behavioral or clinical phenomena. The claim of being the 'first validated time-step-level interpretation' is therefore stronger than the evidence supports. The main predictive comparison is also confounded because the architectural difference between HealthCAT and the vanilla Transformer baseline is not specified.","major_comments":[{"comment":"The paper never specifies how HealthCAT differs architecturally from the vanilla Transformer baseline. The only new component described, AttentiveCAT, is a post hoc scoring mechanism (Eqs. 3–5) and cannot alter training or predictions. Yet HealthCAT improves F1 by 0.063 on PA and 0.074 on DREAMT over the Transformer. HealthCAT's classification head uses the concatenated final-layer representation h̄ = [h^L_1;…;h^L_w]; if the baseline uses a [CLS]-style head or different pooling, that is an architectural difference unrelated to the proposed interpretability layer. Preprocessing differences are also unspecified. Without a controlled variant, RQ1's superiority claim is confounded and not reproducible.","section":"§III-C / Table II"},{"comment":"The masking experiment tests internal model fidelity, not external behavioural relevance. Showing that steps ranked by AttentiveCAT retain more predictive value than random steps only establishes that the scores track the model's own decision process; it does not validate that these time steps are the ones that matter for the health outcome, since no external ground truth is available. The abstract's phrase 'validated time-step-level interpretability' and 'predictively informative' therefore overstates the evidence. In addition, the protocol is ambiguous: 'zeroing out ... bottom 25%, 50%, and 75%' is inconsistent with the non-monotonic HealthCAT results (e.g., 75% masking F1 0.7578 > 25% masking F1 0.7277 in Table III). Clarify whether masking keeps top-k or removes bottom-k, and discuss ranking stability.","section":"§III-E / Table III"},{"comment":"Equation (4) is not well-defined as written. α^l_t is described only as head-averaged self-attention weights, but it is not stated whether it is a scalar, a vector over tokens, or a matrix; CAT^{c,l}_t is a d-dimensional vector. The product α^l_t · CAT^{c,l}_t therefore lacks a defined dimension. The original AttentiveCAT (ref. [20]) sums over token positions weighted by attention. Please provide the explicit summation over source tokens and define all indices.","section":"§III-C, Eq. (4)"},{"comment":"The paper reports AttentiveCAT-guided masking for GRU and LSTM baselines, but AttentiveCAT as defined in Eqs. (3)–(5) requires Transformer self-attention weights. No description is given of how CAT or attention scores are computed for recurrent models. If gradients with respect to GRU/LSTM attention weights were used, the implementation is nontrivial and must be specified. As it stands, the cross-model rows of Table III cannot be reproduced or interpreted.","section":"§III-E / Table III"},{"comment":"The intensity and bout thresholds for MVPA/LPA/SED are taken from [50]/[51] without recalibration for this cohort of Pacific Island adolescents (10–16 years). Because the multivariate features are bout counts derived from these thresholds, and because the domain-adapted visualizations in Fig. 2 are built from those features, a systematic threshold mismatch would propagate into both predictions and interpretations. The paper neither acknowledges this risk nor provides a sensitivity analysis (e.g., varying cut-points or bout lengths). This is important for the central interpretive claim about 'when and how physical activity patterns occur.'","section":"§III-B / §IV-C"}],"minor_comments":[{"comment":"Appendix A states 'AdamW as optimiser,' while Section III-E states Rectified Adam (RAdam). This inconsistency must be resolved for reproducibility.","section":"Appendix A vs §III-E"},{"comment":"References [50] and [51] are the same paper (Phillips, Parfitt, and Rowlands 2013); one duplicate should be removed or renumbered.","section":"References"},{"comment":"The statistical reporting is slightly incorrect: a paired t-test with 100 trials has df = 99, not 100 (e.g., t(100)=2.016). Also, no multiple-comparison correction is applied across the 12 model-masking comparisons, and several comparisons are not significant; the abstract's 'across all masking conditions (p<0.05)' is too broad.","section":"§IV-B"},{"comment":"The caption does not define the color scale, the bucket width ('10 minutes duration within an hour' appears in text but not the caption), or the units of the importance scores. Add axis legends and a colorbar.","section":"Fig. 2"},{"comment":"The paper says 'The source code will be published on GitHub' but provides no repository link. Add a link or state an anonymized availability policy for review.","section":"Availability"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The central idea is reasonable and the datasets are real, but the manuscript currently overclaims validation. The most serious issue is that HealthCAT's performance advantage over the vanilla Transformer is not explained; if it comes from an architectural difference such as the classification head or preprocessing, the predictive comparison needs to be reframed. The masking evaluation is a useful self-consistency check but not external evidence of behavioral relevance. The threshold-calibration concern is also legitimate and should be addressed at least as a sensitivity analysis. I recommend major revision rather than rejection because the interpretability problem is important and the limitations appear addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, know this: the paper imports AttentiveCAT from NLP into wearable health time series, wraps it in a clean three-stage framework, and evaluates the resulting time-step importance scores with a masking-based faithfulness test. That is a legitimate, useful contribution. It is not a paradigm shift, but it moves a known method into a domain where fine-grained temporal explanations are genuinely needed.\n\nWhat the paper does well: the preprocessing is clearly described; the two real-world datasets (PA and DREAMT) are appropriate; the masking protocol is a sensible way to check whether the scores point at time steps the model actually relies on; and the domain-adapted visualization showing contributions by time-of-day and intensity level is a nice practical touch. The authors also acknowledge the small cohort and the lack of contextual variables in their limitations. The writing is honest and mostly careful.\n\nThe soft spots, in order of importance. The biggest issue is the predictive gain over the vanilla Transformer. AttentiveCAT is a post hoc scoring layer; it does not change the forward pass or the training objective. So if HealthCAT uses the same encoder as the Transformer baseline, the two should give identical predictions. Table II shows a gap of about 0.06 F1 on both datasets. The paper claims this is due to 'sensor modality aware preprocessing and Encoder-only structure,' but the baseline presumably uses the same preprocessing and structure. The authors need to specify exactly what differs — different hyperparameters, a different training scheme, or a modified architecture. Without that, the headline improvement over baselines is confounded.\n\nSecond, the masking evaluation is a self-consistency check, not external validation. Showing that top-ranked time steps retain more predictive information than random time steps tells you the scores reflect the model's decision process. It does not tell you those time steps are the ones that actually matter for the physiological outcome. That is fine as a faithfulness test, but the paper should not oversell it as evidence about 'when and how physical activity patterns occur' in the world.\n\nThird, the stress-test concern about intensity thresholds is legitimate: the MVPA/LPA/SED cut-points come from a calibration study in children and are applied without revalidation to Pacific Island adolescents. If body composition or activity patterns shift the boundaries, the hourly bout features — and every attribution built on them — inherit that bias. This doesn't sink the paper, but it should be stated as a limitation rather than hidden.\n\nMinor points: the abstract says 'significantly more predictive value than random selection across all masking conditions,' which holds for the HealthCAT rows but not for all baseline-model comparisons in the tables. The reference list has [50] and [51] as the same paper. And the t-statistic is reported with df=100 rather than 99.\n\nWho should read it: people working on explainable AI for time series or wearable health informatics. It is a solid application paper with a useful evaluation protocol, worth serious review. I would ask the authors to resolve the predictive-gap issue and recalibrate or at least discuss the thresholds before publication.","headline":"Useful application of a known interpretability method to wearable sensor time series, but the unexplained predictive gap over its own transformer backbone is the main weakness.","tokens_in":16889,"tokens_out":4552,"would_cite":true,"duration_ms":44369,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HealthCAT pairs a transformer's health predictions with time-step-level explanations that survive aggressive masking.","keywords":["interpretable deep learning","wearable sensors","transformer","time-step attribution","class activation","health prediction","masking evaluation","time-series interpretability"],"falsifier":"Recalibrate the intensity cut-points and bout-length thresholds on a sample of the target adolescent population, then rerun the masking experiments; if the AttentiveCAT-vs-random advantage disappears, the claim that the identified time steps are predictively informative is an artifact of the chosen thresholds.","tokens_in":16165,"feed_emoji":"⌚","tokens_out":9284,"duration_ms":82594,"temperature":0.7,"pith_summary":"The paper proposes HealthCAT, a framework that predicts health indicators from wrist-worn sensor data and, in the same pass, assigns each minute or hour of the recording a class-specific importance score. Its central claim is that these time-step-level scores are genuinely informative—not just plausible-looking—because masking out the low-scoring time steps hurts predictions far less than masking random ones. On two real-world datasets, adolescent physical-activity and adult sleep, the model beat LSTM, GRU, and plain Transformer baselines by up to 17% in F1 and 12% in accuracy. If the claim holds, HealthCAT would let researchers see when, during the day, a person's behaviour pushes a prediction toward a healthy or unhealthy outcome, moving wearable analytics beyond aggregated daily totals to individual-level temporal patterns.","feed_headline":"AI model shows which wearables minutes drive a health prediction","feed_subtitle":"HealthCAT beats deep learning baselines, and the minutes it flags stay predictive even when the rest is masked.","key_machinery":"The central object is the AttentiveCAT score: for each encoder layer l and time step t, it is the product of the layer's head-averaged attention weights α^l_t and the class activation token CAT^l_t, where CAT^l_t is the element-wise product of the hidden representation h^l_t with the gradient of the class logit with respect to h^l_t. Summed over layers and pooled over hidden dimensions, each time step receives one scalar class-specific score. This combines two signals: the gradient tells how much the model's decision changes with that time step's representation, while the attention weights tell how much the model listens to that time step. The scores are then projected onto domain features (","core_discovery":"HealthCAT integrates an Encoder-only Transformer with an Attentive Class Activation Token (AttentiveCAT) that produces, per time step and per class, an importance score derived from the hidden representation, the gradient of the class logit with respect to it, and the head-averaged self-attention weights. The paper reports that on the physical-activity dataset (206 adolescents, healthy-weight status) and the sleep dataset (100 adults, apnea-hypopnea index), the model outperformed three deep-learning baselines, and in cumulative masking experiments the time steps selected by AttentiveCAT retained significantly more predictive value than randomly selected time steps across all masking conditio","pith_inferences":["The masking comparison is against random selection, not against ground-truth behavioural relevance; a clinician-labelled study would be the natural next step to test whether the highlighted minutes are ones a human expert would also flag.","Because the features inherit fixed activity-intensity thresholds borrowed from other populations, a sensitivity analysis that recalibrates those thresholds for the target group would reveal whether the interpretations are threshold-dependent or stable.","If these scores generalise, a natural application is just-in-time interventions: the model could point to the hours of the day where a person's routine has the most leverage for a healthier outcome."],"forward_implications":["The masking protocol provides a reusable, quantitative template for validating any future temporal-interpretability method, not just this one.","Individual-level analytics: a clinician or coach can inspect, for a specific person, which hours of moderate, light, or sedentary activity support the predicted health category.","The framework is flexible across temporal resolutions—hourly features for physical activity, minute-level for sleep—so the same code path can be applied to other wearable sensors and health indicators.","The contrastive participant examples suggest that temporal placement of activity matters: people with similar daily totals can show very different importance patterns, and HealthCAT makes that difference visible."],"fun_headline_variants":["Wearable AI pinpoints the minutes that drive health predictions","Transformer model shows which sensor moments matter most","HealthCAT: the exact time steps behind health predictions","For wearables, it's not just averages—it's the timing","HealthCAT beats baselines and shows which time steps matter"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole interpretation stack assumes that the intensity cut-points and bout-length thresholds adopted from other populations correctly classify physical activity intensity in Pacific Island adolescents; if those thresholds are off, the time-step scores and the masking results would describe threshold artifacts rather than true behaviour.","fun_headline_variants_meta":{"raw":{"variants":["Wearable AI pinpoints the minutes that drive health predictions","Transformer model shows which sensor moments matter most","HealthCAT: the exact time steps behind health predictions","For wearables, it's not just averages—it's the timing","HealthCAT beats baselines and shows which time steps matter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000775,"raw_usage":{"total_tokens":3293,"prompt_tokens":797,"completion_tokens":2496,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":2425}},"tokens_in":541,"tokens_out":2496,"duration_ms":17156,"temperature":1.0,"reasoning_tokens":2425,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T04:19:40.364748+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recalibrate the intensity cut-points and bout-length thresholds on a sample of the target adolescent population, then rerun the masking experiments; if the AttentiveCAT-vs-random advantage disappears, the claim that the identified time steps are predictively informative is an artifact of the chosen thresholds.","supporting_citations":[],"review_version":1}