{"id":"91df6fb9-c20d-490c-9e55-2ea88890d043","arxiv_id":"2411.11094","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A CNN-GRU model predicts blood glucose from PPG with reported MAE 2.96 mg/dL, but the tiny and possibly leaked dataset undermines the result.","lead":"A hybrid CNN-GRU deep learning network was applied to estimate blood glucose from photoplethysmography (PPG) signals, reporting a low mean absolute error of 2.96 mg/dL on a small public dataset. The evaluation is compromised by a likely data-leakage flaw in how the train and test sets were constructed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported accuracy is not credible as a generalization estimate because Section III.B.3 augments the data before Section III.B.5 splits it, and the split is not participant-disjoint; test samples can contain noisy copies of training signals.","rationale":"The strongest claim is the test-set accuracy. For that claim to hold, the test partition must consist of genuinely unseen PPG-glucose pairs. The paper's own methods section (III.B.3 and III.B.5) describes Gaussian-noise augmentation before shuffling and splitting, which directly creates a leakage path: an augmented copy of a training signal can be drawn into the test set, carrying the same label and nearly the same waveform. The model then only needs to recognize a noised version of a seen signal, so the metrics stop measuring generalization. Independent of augmentation, the dataset has only 23 participants; without subject-level grouping, random splits place the same person's recordings in both training and test, a known source of optimistic bias in physiological signal modeling. The paper provides no code, seed, or participant identifiers to rule out either leakage path. A further internal inconsistency reinforces the concern: the reported 10-fold cross-validation means (MAE about 1.77 mg/dL, RMSE about 2.63 mg/dL) do not match the abstract's headline MAE 2.96 and RMSE 3.94 mg/dL, so the provenance of the headline numbers is unclear. The conclusion's explicit limitation about the narrow 88-187 mg/dL glucose range is honest and should be weighed in the authors' favor, but a limitation on the dataset's applicability is not a substitute for a valid test set. If a participant-disjoint rerun with post-split augmentation still yields MAE near 3 mg/dL and R² near 0.97, the method would be genuinely impressive. If not, the current REJECT is justified. I therefore find no reason to change the reader's verdict.","tokens_in":5569,"tokens_out":3909,"duration_ms":51223,"concrete_test":"Re-run the entire pipeline with a participant-disjoint split: group the original 67 signals by participant, perform augmentation only after the split on training participants only, and report MAE, RMSE, R², and Clarke zone percentages on held-out participants. Also compare train and test augmented signals by correlation or nearest-neighbor distance; if any test sample matches a training sample derived from the same original waveform above the injected noise level, the current leaked split is confirmed. If held-out-participant metrics degrade materially (for example, MAE above 10 mg/dL or Clarke zone A below 90%), the paper's headline numbers are leakage artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the test-set MAE of 2.96 mg/dL, RMSE of 3.94 mg/dL, R² of 0.97, and 100% Clarke zone A. For these numbers to measure generalization, the test set must contain neither augmented copies of training signals nor recordings from participants whose other signals were used in training. The pipeline order in Section III.B places augmentation (adding Gaussian noise to increase 67 signals to 269) before shuffling and splitting, so a noisy variant of one original signal can appear on both sides of the split. Moreover, with 67 recordings from only 23 participants, a random split will typically place multiple recordings from the same participant in both training and test, allowing the model to exploit per-subject patterns rather than learn a glucose-to-PPG mapping. The paper reports neither a participant-independent split nor code, seeds, or participant IDs to verify disjointness. An additional internal inconsistency supports the concern: the average of the reported 10-fold cross-validation MAE values is about 1.77 mg/dL and RMSE about 2.63 mg/dL, both notably different from the abstract's 2.96 and 3.94 mg/dL, so it is unclear which split produced the headline numbers. Without an independent test partition, R²=0.97 and 100% zone A are fragile: even mild leakage can produce near-perfect regression because augmented copies share the same underlying waveform and label. The conclusion's honest note about the narrow 88-187 mg/dL glucose range is a strength, but a limitation on clinical applicability does not repair the evaluation protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hybrid CNN-GRU network for non-invasive blood glucose estimation from photoplethysmography (PPG) signals. Using the public Mazandaran dataset v2 (67 signals from 23 participants), the authors preprocess the PPG signals, augment the data by adding Gaussian noise (increasing 67 signals to 269), normalize, shuffle, and split into training, validation, and test sets. The model combines two parallel CNN branches and a GRU branch, with their outputs merged for a final regression layer. The manuscript reports a test MAE of 2.96 mg/dL, MAPE of 2.40%, RMSE of 3.94 mg/dL, R² of 0.97, and 100% of predictions in Clarke Error Grid zone A, and claims superiority over several prior PPG-based methods. A 10-fold cross-validation is also reported in Table III.","tokens_in":6039,"tokens_out":2992,"duration_ms":29738,"significance":"If the reported results were valid, the proposed method would represent a substantial improvement over existing PPG-based glucose estimation approaches (prior MAEs of 4.7–8.9 mg/dL in Table II) and would be of considerable practical interest for non-invasive monitoring. The paper's strengths include its use of a publicly available dataset and its explicit acknowledgment in the conclusion that the dataset's narrow glucose range (88–187 mg/dL) limits clinical applicability. However, the significance is critically undermined by the evaluation protocol: the central results are not reliable estimates of generalization, and the lack of code, seeds, or participant identifiers prevents independent verification. As presented, the performance claims cannot support the paper's conclusions.","major_comments":[{"comment":"Data augmentation is performed before the dataset is shuffled and split. Because augmentation adds Gaussian noise to the original 67 signals to create 269 signals, the test set can contain augmented copies of training signals. The reported test MAE of 2.96 mg/dL, RMSE of 3.94 mg/dL, and R² of 0.97 are therefore not necessarily measures of out-of-sample performance, and the central claim of superior accuracy is not supported.","section":"III.B.3 and III.B.5"},{"comment":"The dataset consists of only 67 signals from 23 participants, yet the split is performed on individual signals rather than on participants. With a random shuffle-and-split protocol, the test set will almost certainly contain multiple recordings from participants whose other recordings appear in the training set, allowing the model to exploit per-subject characteristics rather than learning a general PPG-to-glucose mapping. The paper does not report a participant-independent split or provide participant identifiers to rule out this leakage.","section":"III.A and III.B.5"},{"comment":"The headline results (MAE 2.96 mg/dL, RMSE 3.94 mg/dL) are inconsistent with the reported 10-fold cross-validation results: the average MAE across folds in Table III is approximately 1.77 mg/dL and the average RMSE approximately 2.63 mg/dL. The paper does not explain which split produced the abstract and Table II metrics, nor why the test-set errors are larger than the cross-validation errors. This discrepancy leaves the provenance of the key numbers unclear and further undermines their reliability.","section":"Abstract and Table III"},{"comment":"The 100% Clarke Error Grid zone A result is claimed for 'test samples', but given the leakage described in the two comments above, this result is not a valid indicator of clinical safety. Even without leakage, the narrow glucose range (88–187 mg/dL) would make zone A classification easier than in a population with a wider glucose distribution; the conclusion's acknowledgment of this limitation is commendable but does not rescue the reported figure.","section":"Section IV and Figure 7"}],"minor_comments":[{"comment":"The manuscript contains numerous grammatical errors and inconsistencies in capitalization (e.g., 'convolution neural network' should be 'convolutional neural network', 'Exploiting' after a comma, inconsistent use of 'BGL' and 'blood glucose level'). These should be corrected.","section":"Abstract and Throughout"},{"comment":"The sentence 'the raw PPG signal goes through pre-processing steps... as shown in' is incomplete; no figure or equation number is provided. Either reference Figure 2 here or complete the sentence.","section":"III.B.1"},{"comment":"The text says 'Table IV shows the 10-fold cross-validation metrics results,' but the table is labeled Table III. The table cross-reference should be corrected.","section":"Section IV, "},{"comment":"Reference [5] has uneven spacing in the author initials ('C. -Y . Liao') and reference [11] is a GitHub repository; the citation should include the date of access and a version or commit, if available.","section":"References"}],"recommendation":"reject","confidential_remarks":"The data-leakage concern is not a matter of interpretation: the pipeline order in Section III.B explicitly places augmentation before splitting, and with only 23 participants a signal-level split guarantees subject overlap. This is a fundamental flaw in the evaluation, and the discrepancy between the headline metrics and the cross-validation average in Table III further suggests that the reported test numbers are not stable or readily reproducible. Fixing the protocol would require rerunning all experiments with participant-disjoint, post-split augmentation, which would change the reported results and likely the conclusions. Given the absence of code, seeds, and participant IDs, I cannot recommend this manuscript for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper's headline numbers are not credible as generalization estimates. The methodology augments the 67 signals to 269 by adding Gaussian noise before shuffling and splitting, so augmented copies of training signals can land in the test set. With only 23 participants, random splits also mix the same person's recordings across train and test. The reported MAE 2.96, RMSE 3.94, R² 0.97, and 100% Clarke zone A are therefore not out-of-sample. The stress-test note holds up on reading; the pipeline order in Section III.B is unambiguous.\n\nGive credit where it is due: the paper is clearly written, uses a public dataset, honestly states the glucose range limitation, and positions itself against LRCN and other prior work. The architecture is a modest variant—parallel CNN branches and GRU instead of LSTM—so the novelty is incremental, not a breakthrough.\n\nThe soft spots go beyond the leakage. The average of the reported 10-fold MAE values is about 1.77 mg/dL and RMSE about 2.63 mg/dL, both well below the abstract's 2.96 and 3.94. The paper never explains which split produced the headline numbers. No code, seeds, or participant IDs are provided to verify disjointness. The narrow 88–187 mg/dL glucose range limits clinical relevance, a point the authors acknowledge.\n\nThis is not a paper to cite or build on as it stands. The central claim fails because the evaluation protocol cannot support it. That said, the authors are not sloppy thinkers; they just made a load-bearing methodological error that is fixable. A proper subject-independent split with augmentation applied only after splitting would make the experiment meaningful, and a resubmission with corrected evaluation could be worth a look.\n\nMy recommendation: desk reject this version. If the authors redo the evaluation and report both subject-independent and augmentation-safe results, the paper would deserve peer review.","headline":"Augmentation-before-split and non-subject-disjoint splits invalidate the claimed generalization performance.","tokens_in":6453,"tokens_out":2747,"would_cite":false,"duration_ms":84085,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a hybrid CNN-GRU network that maps photoplethysmogram (PPG) signals to blood glucose with a mean absolute error of 2.96 mg/dL, placing all test predictions in the clinically safe Clarke zone A.","keywords":["photoplethysmography","non-invasive glucose monitoring","CNN-GRU","deep learning","blood glucose regression","Clarke Error Grid","time series"],"falsifier":"Re-run the exact pipeline but split by participant (all 67 original signals from each of the 23 people kept together, with augmentation applied only to the training partition after the split), then compare test MAE against the reported 2.96 mg/dL. If the error rises substantially or zone-A coverage drops below 100%, the reported generalization claim is an artifact of duplicated signals across the split.","tokens_in":5399,"feed_emoji":"🩸","tokens_out":6156,"duration_ms":52494,"temperature":0.7,"pith_summary":"The paper is trying to establish that blood glucose can be read directly from a photoplethysmogram (PPG) pulse signal by a hybrid deep network, without needles and without hand-crafted features. It combines two parallel convolutional branches with a gated recurrent unit branch, trains on the public Mazandaran dataset, and reports a mean absolute error of 2.96 mg/dL, an R² of 0.97, and 100% of test predictions in the clinically reliable zone A of the Clarke Error Grid. If these numbers reflect genuine generalization, the method would offer a painless, low-cost route to continuous glucose monitoring that outperforms earlier PPG-based methods. The authors themselves note the dataset's glucose range is narrow, so further validation on wider ranges is needed before clinical use.","feed_headline":"PPG pulse wave plus CNN-GRU pinpoints glucose to 2.96 mg/dL","feed_subtitle":"No pricking needed: an optical pulse signal and a CNN-GRU network hit clinically safe glucose estimates.","key_machinery":"The load-bearing component is the multi-branch hybrid block: two parallel 1D CNN branches with different kernel sizes extract local pulse-waveform features at different scales, while a GRU branch processes the same signal sequence to capture temporal dependencies. Each branch is flattened and passed through three identical fully connected layers; the branches are concatenated and fed to a final single-unit regression layer. Preprocessing consists of band-pass filtering (0.5–8 Hz), downsampling from 2175 Hz to 30 Hz, Gaussian-noise augmentation, normalization, and a shuffle-split into train, validation, and test sets.","core_discovery":"The central discovery claimed is that a hybrid architecture of one-dimensional convolutional layers and gated recurrent units can learn the physiological relationship between the optical pulse waveform and blood glucose directly from raw signals, outperforming both feature-engineering methods and a CNN-LSTM baseline. On the test partition of the Mazandaran dataset, the model achieves MAE 2.96 mg/dL, MAPE 2.40%, RMSE 3.94 mg/dL, and R² 0.97, with all predictions falling in Clarke Error Grid zone A (within ±20% of the reference, or both values below 70 mg/dL). The paper presents this as evidence that a simple PPG sensor plus a trained neural network can replace fingertip pricking for glucose estimation.","pith_inferences":["The reported metrics are likely optimistic because Gaussian-noise augmentation was applied before the shuffle/split, so the test set may contain near-duplicates of training signals; a participant-disjoint split is the fair test of generalization.","The CNN-GRU advantage over the CNN-LSTM baseline is attributed to GRU's gating, but the comparison also changes kernel and architecture details, so the source of improvement is not isolated.","A natural extension is to validate on a second PPG dataset or on signals with motion artifacts to see whether the learned mapping captures glucose physiology or merely dataset-specific patterns.","Since the model outputs a continuous glucose value, the same regression design could be tested for hypoglycemia detection if a dataset with readings below 70 mg/dL becomes available."],"forward_implications":["A working direct PPG-to-glucose mapping would let glucose be tracked continuously from an optical wristband or fingertip sensor, eliminating finger pricking.","The reported zone-A coverage (100%) implies that on this dataset no test prediction would have led to a clinically unsafe treatment decision under Clarke Error Grid criteria.","Because no hand-crafted features are used, the same architecture could be retrained on other optical signals or other physiological targets without redesigning the feature set.","The narrow glucose range of the data (88–187 mg/dL) limits the claim to that range; the authors state that a wider-range dataset is required before use in care units."],"supporting_citations":[{"why":"Supplies the 67 PPG signals and paired glucose readings from 23 participants that the model is trained and evaluated on.","marker":"[9]"},{"why":"Provides the CNN-based feature-engineering baseline (MAE 8.9 mg/dL) that the proposed method claims to outperform.","marker":"[3]"},{"why":"Provides the machine-learning baseline using intrinsic mode functions (MAE 7.05 mg/dL) used for comparison.","marker":"[4]"},{"why":"Provides the LRCN CNN-LSTM baseline that motivates the hybrid recurrent design and gives a second comparison point.","marker":"[5]"},{"why":"Defines the Clarke Error Grid zones used to claim clinical safety of the predictions.","marker":"[10]"},{"why":"Inspires the multi-scale convolutional branch design (Google's LeNet/Inception structure) used in the model's CNN blocks.","marker":"[7]"},{"why":"Supports the choice of GRU over LSTM by providing evidence that multi-layer GRU can match or beat LSTM with faster training.","marker":"[8]"}],"fun_headline_variants":["CNN-GRU learns glucose from pulse waves, MAE 2.96 mg/dL","Painless glucose tracking: PPG signal feeds hybrid deep net, 2.96 mg/dL error","Deep learning reads glucose from PPG without pricks, hits 2.96 mg/dL MAE","Hybrid CNN-GRU on optical pulse signal predicts glucose at 2.96 mg/dL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The single load-bearing premise is that the test set is genuinely unseen: no augmented copies of training signals and no signals from the same participants appear in training and test after the shuffle-split, which the paper's order of operations (augment before split) makes doubtful.","fun_headline_variants_meta":{"raw":{"variants":["CNN-GRU learns glucose from pulse waves, MAE 2.96 mg/dL","Painless glucose tracking: PPG signal feeds hybrid deep net, 2.96 mg/dL error","Deep learning reads glucose from PPG without pricks, hits 2.96 mg/dL MAE","Hybrid CNN-GRU on optical pulse signal predicts glucose at 2.96 mg/dL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000173,"raw_usage":{"total_tokens":1259,"prompt_tokens":904,"completion_tokens":355,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":259}},"tokens_in":520,"tokens_out":355,"duration_ms":3764,"temperature":1.0,"reasoning_tokens":259,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:54:25.764978+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the exact pipeline but split by participant (all 67 original signals from each of the 23 people kept together, with augmentation applied only to the training partition after the split), then compare test MAE against the reported 2.96 mg/dL. If the error rises substantially or zone-A coverage drops below 100%, the reported generalization claim is an artifact of duplicated signals across the split.","supporting_citations":[{"cited_title":"The dataset of photoplethysmography signals collected from a pulse sensor to measure blood glucose level","cited_arxiv_id":null,"evidence_quote":"Supplies the 67 PPG signals and paired glucose readings from 23 participants that the model is trained and evaluated on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CNN-based feature-engineering baseline (MAE 8.9 mg/dL) that the proposed method claims to outperform."},{"cited_title":"”EMD-Based Noninvasive Blood Glucose Estimation from PPG Signals Using Machine Learning Algorithms” 2024 Applied Sciences 14, no","cited_arxiv_id":null,"evidence_quote":"Provides the machine-learning baseline using intrinsic mode functions (MAE 7.05 mg/dL) used for comparison."}],"review_version":1}