{"id":"3c44f2eb-e3b2-4303-8e3c-f2e77b90c6cc","arxiv_id":"2502.09919","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A multimodal Transformer with cross-attention and multi-scale attention forecasts blood glucose from CGM and activity data, beating a CNN-LSTM baseline on AI-READI by about 10% in RMSE.","lead":"AttenGluco, a Transformer that combines CGM glucose readings with smartwatch activity data, reports lower blood glucose forecasting errors than an LSTM baseline on the AI-READI dataset. The paper is relevant to anyone tracking deep learning for diabetes management, though it compares against only one baseline and has methodological gaps.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table III shows AttenGluco is worse at PH=5 in three of four cohorts, contradicting 'consistently outperformed'; the headline ~10% gain may be pooled or concentrated at long horizons.","rationale":"The reader's weakest assumption focuses on feature sufficiency and window length, which is a plausible external-validity concern. The more immediately damaging problem is internal: Table III directly contradicts the claim of consistent improvement. The headline 10% improvement is claimed from the isolated-subject experiment (Table I), but no table gives the per-horizon breakdown for that scenario, so a reader cannot tell whether the improvement is a pooled average or a 60-min-only result. At PH=5, the paper's own cohort-wise numbers show the baseline is better by up to 9.6% (Pre-T2DM). This inconsistency must be resolved before accepting the central empirical claim. The proposed test is straightforward: recompute per-horizon and pooled RMSE from the released code on a clean chronological split. If the short-horizon deficit persists, the abstract's 'about 10% improvement' should be restated as a long-horizon-only result and the conclusion about consistent improvement should be corrected. The reader's conditional verdict remains appropriate because the concern is resolvable with additional analysis and clarification, not a fundamental invalidation of the method.","tokens_in":11206,"tokens_out":6381,"duration_ms":67247,"concrete_test":"Rerun the PH experiment using the released code with a time-ordered 85/15 split per subject, and report RMSE separately for PH=5, 30, and 60 minutes, together with the pooled RMSE used for the headline. Also state explicitly whether Tables I and II report a pooled average or a fixed PH. If the pooled advantage vanishes or reverses at PH=5, the claim of consistent outperformance is unsupported and the headline improvement must be restricted to long-horizon forecasting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table III is the load-bearing weak point for the central claim. It reports RMSE by prediction horizon in the cohort-wise fine-tuning scenario: at PH=5 min, AttenGluco is worse than the baseline for Healthy (7.63 vs. 7.35), Pre-T2DM (8.70 vs. 7.94), and Oral (9.33 vs. 9.15), and only marginally better for Insulin (11.94 vs. 12.11). The conclusion nevertheless states that AttenGluco 'consistently outperformed the baseline model' across PHs. If the headline ~10% RMSE improvement in Tables I and II is a pooled average over PHs, or is reported only at 60 min, then the model's advantage may be entirely concentrated at longer horizons while it regresses at the short horizon that is most clinically relevant for alerts. The paper never states which PH or pooling rule produces the headline numbers, and it gives no per-horizon breakdown for the isolated-subject scenario that is the basis of the abstract claim. Because the 5-min deficit is internal to the paper's own table, this is a correctness risk, not a matter of interpretation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AttenGluco, a Transformer-based framework for blood glucose forecasting that combines CGM data with two activity features (walking steps and walking intervals) using cross-attention and multi-scale attention. The authors evaluate on the AI-READI dataset across four subject cohorts (healthy, pre-T2DM, oral medication, insulin) in three scenarios: isolated subject training, cohort-wise fine-tuning, and a forgetting analysis. The headline claim is that AttenGluco outperforms a multimodal CNN-LSTM baseline by about 10% in RMSE and 15% in MAE, with stronger gains in the cohort-wise fine-tuning scenario.","tokens_in":11406,"tokens_out":2370,"duration_ms":26323,"significance":"If the reported gains hold, AttenGluco would be a useful contribution to multimodal glucose forecasting, especially for long-horizon prediction. The paper is the first to report blood glucose forecasting results on the AI-READI dataset, and the authors provide a public codebase. The isolated-subject experiment uses a clean 85/15 train-test split per subject, which supports the main claim in a non-circular way. The inclusion of per-cohort results, a prediction-horizon breakdown, and a forgetting analysis are commendable. However, the significance is tempered by internal contradictions in the horizon-level results, a circular evaluation protocol in the cohort-wise fine-tuning scenario, and the absence of uncertainty quantification and ablations.","major_comments":[{"comment":"Table III directly contradicts the Conclusion's claim that AttenGluco 'consistently outperformed the baseline model' across prediction horizons. At PH=5 min, AttenGluco has higher RMSE than the baseline for Healthy (7.63 vs. 7.35), Pre-T2DM (8.70 vs. 7.94), and Oral (9.33 vs. 9.15), and is only marginally better for Insulin (11.94 vs. 12.11). The paper never states which prediction horizon or pooling rule produces the headline ~10% RMSE improvement in Tables I and II, and no per-horizon breakdown is provided for the isolated-subject scenario. The authors must specify the pooling rule, report per-horizon results for all scenarios, and either correct the 'consistently outperformed' claim or provide a granular analysis that resolves the discrepancy.","section":"Table III and Conclusion"},{"comment":"The cohort-wise fine-tuning evaluation is circular: the text states that 'each subject serving as both training and testing data.' Reporting RMSE on subjects whose data were used for training inflates the apparent improvement and does not measure generalization. This affects Table II and Fig. 3, and the statement that 'performance progressively improves' as more subjects are added. The authors should either add a held-out test set for this scenario, or clearly reframe Table II and Fig. 3 as training-fit diagnostics rather than predictive performance. The current framing overstates the benefit of the fine-tuning protocol.","section":"Section IV-B2, Table II"},{"comment":"The paper states that each model undergoes five independent training runs, but no standard deviation, confidence interval, or statistical significance test is reported for any metric. Without error bars, the reported differences (e.g., 18.04 vs. 16.05 in Table I) could be within run-to-run variability. Reporting means over five runs is insufficient; the authors should report variance and, where possible, paired significance tests, especially for the Table III PH=5 results where AttenGluco's deficit may or may not be meaningful.","section":"Section IV-B, Tables I-III"},{"comment":"The paper attributes the performance gains to the cross-attention and multi-scale attention mechanisms, but no ablation study is provided. Replacing each attention component with alternatives, removing the activity features, or varying window length and downsampling factors are necessary to verify that the proposed architecture components are responsible for the improvements. Additionally, the input representation uses only walking steps and walking intervals even though the AI-READI dataset provides heart rate and stress index signals; the paper should justify this choice and ideally report sensitivity to the input feature set and window length (400 minutes).","section":"Section III, Section IV-B"}],"minor_comments":[{"comment":"The phrase 'a baseline model consisting if a 1D-CNN and LSTM' contains a typo ('if' should be 'of').","section":"Section IV-B"},{"comment":"The algorithm description says 'Multi-scale atention block' — 'atention' should be 'attention'.","section":"Algorithm 1"},{"comment":"The text describing the second cross-attention branch says XWI is used as 'both the key and the query'; based on Eq. (3), it should be 'key and value' with XG as the query. Please clarify.","section":"Section III-B, Eq. (3)"},{"comment":"The cohort naming is inconsistent: 'pre T2DM', 'pre-T2DM', and 'Pre-T2DM' are used interchangeably. Please standardize.","section":"Section IV-A"},{"comment":"The forgetting analysis in Fig. 4 is described only briefly; the exact evaluation protocol (which subjects are tested, which cohorts are trained, and what the error bars represent, if any) should be stated in the caption or text.","section":"Figure 4"},{"comment":"The paper does not state how the error metrics (RMSE, MAE, correlation) in Tables I and II are aggregated across subjects and across prediction horizons. Please specify the aggregation procedure explicitly.","section":"Section IV-B"}],"recommendation":"major_revision","confidential_remarks":"The core isolated-subject result appears sound under a clean split, but the manuscript's own Table III undermines the 'consistently outperformed' claim, and the cohort-wise fine-tuning evaluation is circular. These issues are fixable with reanalysis and clarification, so I do not recommend rejection. The authors should be encouraged to provide per-horizon and per-subject error distributions, and to add ablations for the attention mechanisms, as the current evidence does not yet isolate the source of the improvement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what you should know: this is a modest but real empirical contribution — a transformer that fuses CGM and two activity features and beats a CNN-LSTM baseline by ~10% RMSE on a clean per-subject split — but it has an internal inconsistency in its per-horizon table that the authors don't flag. The headline 'consistently outperformed' is not true at the 5-minute prediction horizon.\n\nWhat's new: it's the first glucose-forecasting model evaluated on the AI-READI dataset, and it ships code. The cross-attention + multi-scale attention architecture is a reasonable combination of existing ideas; there's no new theory, but it's a sensible design for irregularly sampled multimodal data. The isolated-subject experiment (85/15 split) is clean and supports the abstract's ~10% RMSE claim.\n\nWhere it's soft: Table III shows AttenGluco with higher RMSE than baseline at PH=5 for Healthy (7.63 vs 7.35), Pre-T2DM (8.70 vs 7.94), and Oral (9.33 vs 9.15), only marginally better for Insulin. The conclusion says 'consistently outperformed' — that's false within their own table. The paper never says which PH or pooling rule generates the headline numbers, so the claimed benefit may be concentrated at 30-60 min. That's a correctness risk.\n\nSecond, the cohort-wise fine-tuning protocol has each subject serving as both training and testing data (they state this explicitly), so Table II and Fig 4 are not valid performance evaluations — they're an optimistic upper bound. The forgetting analysis inherits the same problem.\n\nThird, there are no ablations, no error bars, no significance tests. The gains might come from model capacity rather than the attention mechanisms. The activity inputs are limited to walking steps and intervals; AI-READI also has heart rate and stress index, which they ignore. The 400-minute window isn't tested for sensitivity.\n\nBottom line: it's a serious engineering effort with a real dataset and one clean experiment. It deserves peer review, but the authors need to correct the per-horizon reporting, fix the fine-tuning protocol, and add ablations and variance. I'd send it to referees with major revisions requested.","headline":"Modest but real results on a new dataset, undermined by an internal contradiction in the per-horizon table.","tokens_in":11980,"tokens_out":3689,"would_cite":false,"duration_ms":34627,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AttenGluco forecasts blood glucose up to 60 minutes ahead with roughly 10% lower RMSE and 15% lower MAE than a multimodal LSTM baseline across all four AI-READI cohorts.","keywords":["blood glucose forecasting","transformer","cross-attention","multi-scale attention","continuous glucose monitoring","AI-READI dataset","multimodal time series","type 2 diabetes"],"falsifier":"Retrain AttenGluco with the input window set to 200, 400, and 600 minutes, and with heart-rate and stress-index streams added; if the RMSE advantage over the LSTM baseline shrinks, disappears, or reverses, the claimed benefit depends on window coverage and omitted activity channels rather than on the attention mechanism itself.","tokens_in":11016,"feed_emoji":"📉","tokens_out":6141,"duration_ms":60333,"temperature":0.7,"pith_summary":"AttenGluco is a transformer-based forecasting model that predicts blood glucose levels up to an hour ahead by combining continuous glucose monitor readings with two walking-activity signals: steps and intervals between walks. The paper's central claim is that its dual attention design, cross-attention to fuse differently sampled signals and multi-scale attention to catch long-range dependencies, reduces forecast error by about 10% in RMSE and 15% in MAE compared with a multimodal LSTM baseline across healthy, prediabetic, oral-medication, and insulin-treated cohorts in the AI-READI dataset. The paper also reports that the gain grows under cohort-wise fine-tuning, reaching about 12% RMSE reduction, and that AttenGluco degrades more gracefully when new cohorts are introduced. If the claim holds, the architecture supplies a concrete recipe for long-horizon glucose forecasting from wearable data.","feed_headline":"Transformer cuts blood glucose forecast error by 10-15%","feed_subtitle":"AttenGluco fuses CGM and walking-activity signals, beating LSTM baselines on the AI-READI cohort at 5-60 minute horizons.","key_machinery":"The identifying mechanism is a hybrid attention stack: cross-attention plus multi-scale attention. Cross-attention uses the glucose embedding as the query and each activity embedding as key and value, letting the model align irregularly sampled activity signals with glucose without explicit resampling. Multi-scale attention then repeats self-attention at temporal downsampling factors of 1, 2, and 4, upsampling and summing the branch outputs so that fine-grained details and long-range structure live in one representation. The transformer is encoder-only: it replaces the decoder with a linear head that outputs m future CGM values, each corresponding to one 5-minute interval.","core_discovery":"On the paper's own terms, AttenGluco is the first blood-glucose forecasting model evaluated on the AI-READI dataset and is designed to solve two problems that hamper existing predictors: mismatched sampling rates between CGM and activity sensors, and the difficulty of retaining long-term temporal context. The model takes the last 400 minutes of CGM, walking steps, and walking intervals, embeds them, and runs a two-branch cross-attention in which glucose queries each activity stream; the fused representation then passes through three parallel self-attention branches operating at downsampling factors 1, 2, and 4, whose outputs are summed. Across isolated-subject, cohort-wise fine-tuning, and continual-learning scenarios, AttenGluco reports lower RMSE and MAE and higher correlation than a 1D-CNN plus LSTM baseline in every cohort, with RMSE reductions of roughly 8 to 13 percent depending on scenario and cohort. The model's most notable advantage appears at the 60-minute horizon, where the baseline's error grows sharply while AttenGluco's RMSE stays comparatively stable.","pith_inferences":["The paper documents forgetting but does not test mitigation; a natural extension is to add replay or regularization during cohort transitions and see whether AttenGluco can retain earlier cohorts while gaining new ones.","Because AI-READI also contains heart-rate-derived stress scores, adding that stream to the two activity features is a direct test of whether the two-feature activity representation captures all the relevant physical-activity influence.","The comparison is made against a single LSTM baseline, so testing against other transformer forecasters would clarify whether the gain comes from the hybrid attention design or from transformer scale in general.","The choice of downsampling factors 1, 2, and 4 suggests a connection to dilated or hierarchical time-series models; an ablation varying these factors would quantify how much accuracy is attributable to the specific scales chosen."],"forward_implications":["If the reported gains generalize, 60-minute-ahead glucose forecasts become usable for proactive hypo-and-hyperglycemia alerts, because AttenGluco's long-horizon RMSE degrades far less than the baseline's.","The cohort-wise fine-tuning results imply that adding more subjects from a cohort continuously reduces AttenGluco's test error, so larger training sets should push accuracy further.","The forgetting analysis shows that both models lose accuracy on earlier cohorts when fine-tuned on new ones, but AttenGluco maintains lower absolute error throughout, making it a better starting point for continual-learning fixes.","The architecture offers a template for fusing any irregularly sampled physiological streams, not just glucose and activity, because cross-attention handles differing sampling rates without explicit resampling."],"supporting_citations":[{"why":"Supplies the stacked-LSTM personalized glucose formulation that motivates the recurrent baseline and the approach AttenGluco is compared against.","marker":"[4]"},{"why":"Provides the scaled dot-product attention and multi-head formulation underlying both the cross-attention and multi-scale attention blocks.","marker":"[11]"},{"why":"Documents the AI-READI cohort design and sensor protocol, establishing the dataset's suitability for cohort-wise glucose forecasting.","marker":"[14]"},{"why":"Is the specific Flagship dataset release from which CGM and activity data are extracted.","marker":"[15]"},{"why":"Represents the transformer-based glucose forecaster that lacks cross-attention and multi-scale attention, framing the gap AttenGluco targets.","marker":"[29]"},{"why":"Introduces the multi-scale refining transformer idea that the multi-scale attention block adapts with downsampling factors 1, 2, and 4.","marker":"[31]"},{"why":"Defines catastrophic forgetting, the phenomenon the continual-learning and forgetting analysis is designed to quantify.","marker":"[34]"}],"fun_headline_variants":["Multimodal transformer fuses CGM and activity for glucose forecast","Cross-attention model beats LSTM on long-term glucose prediction","AttenGluco: stable 60-min glucose forecasts with transformer","AI-READI glucose forecasting: transformer beats LSTM by 15% MAE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that two walking-related activity features (steps and walking intervals) plus a 400-minute input window capture enough of the physical-activity influence on glucose that the remaining accuracy gap over the LSTM baseline can be attributed to the attention architecture; if activity acts through other channels such as heart rate or stress, the reported gains may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Multimodal transformer fuses CGM and activity for glucose forecast","Cross-attention model beats LSTM on long-term glucose prediction","AttenGluco: stable 60-min glucose forecasts with transformer","AI-READI glucose forecasting: transformer beats LSTM by 15% MAE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000727,"raw_usage":{"total_tokens":3336,"prompt_tokens":1103,"completion_tokens":2233,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":719,"completion_tokens_details":{"reasoning_tokens":2155}},"tokens_in":719,"tokens_out":2233,"duration_ms":18071,"temperature":1.0,"reasoning_tokens":2155,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T20:01:51.290940+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain AttenGluco with the input window set to 200, 400, and 600 minutes, and with heart-rate and stress-index streams added; if the RMSE advantage over the LSTM baseline shrinks, disappears, or reverses, the claimed benefit depends on window coverage and omitted activity channels rather than on the attention mechanism itself.","supporting_citations":[{"cited_title":"Deep multitask learning by stacked long short-term memory for predicting personalized blood glucose concentration,","cited_arxiv_id":null,"evidence_quote":"Supplies the stacked-LSTM personalized glucose formulation that motivates the recurrent baseline and the approach AttenGluco is compared against."},{"cited_title":"AI-READI: rethinking data collection, preparation and sharing for propelling AI-based discoveries in diabetes research and beyond,","cited_arxiv_id":null,"evidence_quote":"Documents the AI-READI cohort design and sensor protocol, establishing the dataset's suitability for cohort-wise glucose forecasting."},{"cited_title":"Gluformer: Transformer-based personalized glucose forecasting with uncertainty quantification,","cited_arxiv_id":null,"evidence_quote":"Represents the transformer-based glucose forecaster that lacks cross-attention and multi-scale attention, framing the gap AttenGluco targets."},{"cited_title":"Overcoming catastrophic forgetting in neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines catastrophic forgetting, the phenomenon the continual-learning and forgetting analysis is designed to quantify."}],"review_version":1}