{"id":"913a0222-9393-4517-a92e-011dc1a8559d","arxiv_id":"1908.10226","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A multi-task Gaussian process plus a dilated convolutional network reconstructs daily hormone levels from sparse samples, and an oracle-style sampling rule is shown to help, all on synthetic cycles.","lead":"This paper tries to reconstruct a woman's daily reproductive hormone levels from just a few blood samples per week, using a Gaussian process to fill in a plausible curve and a convolutional network to correct it with population-level patterns. The best results also assume the sampling days are chosen with knowledge of the true hormone curve, so the headline numbers are optimistic.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ED sampling evaluation protocol is unspecified: Eq. (2)-(3) require true hormone values, and if the Table 1 schedules are computed from the 10 test individuals, the reported sparse-sampling advantage is an oracle artefact; the released code check can settle this.","rationale":"The reader identified synthetic validity as the weakest assumption; that concern is real but bears on external validity. The ED-protocol ambiguity is internal and directly controls the paper's strongest numerical claim: the superiority of B-MGP-DCNN with Expected Distance sampling at low budgets. The paper deserves credit for releasing code and for a derivation (Eq. 3) that is internally consistent, but the ambiguity in Sections 3.2 and 4.1 makes Table 1 unverifiable as written. If the ED schedule used test true curves, the 0.061 number is not a reconstruction result; it is an oracle result. Even if the schedule used training data, the non-monotonic MSE across budgets (e.g., ED 0.050 at |ti|=70 vs 0.037 at |ti|=35) and the absence of error bars mean the comparative claims need more than a single 10-individual test split. The proposed code check settles the more severe issue; if it passes, the remaining conditions from the reader (real-data validation, non-oracle approximation, error bars) still justify a conditional verdict. I therefore keep the verdict CONDITIONAL, but the primary condition should be explicitly checking the ED sampling split, which is sharper and more actionable than the reader's oracle concern because it can be settled with the released implementation.","tokens_in":17764,"tokens_out":9036,"duration_ms":98668,"concrete_test":"Run the released code (github.com/iurteaga/hmc) for the B-MGP-DCNN (ED) condition at budgets |ti|=10 and |ti|=35 in two modes: (a) compute the ED schedule from the 50 training plus 10 validation individuals' full true curves and apply the same normalized days to the 10 test individuals; (b) compute the ED schedule from the 10 test individuals' true curves. Compare the resulting test MSE against the random-sampling column of Table 1. If mode (a) does not reproduce the reported 0.061 and 0.037 values, or the ED-vs-random advantage disappears, the main claim depends on test-label leakage; if mode (a) does reproduce Table 1, the protocol concern is resolved. In either case, report the chosen ED days and add repeated-seed error bars.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing uncertainty is the evaluation protocol for the Expected Distance (ED) sampling schedules in Section 4.2. Equation (2) chooses t* by maximizing a sum over individuals of Psi_i^h(t|D), and Eq. (3) defines Psi_i^h(t|D) using the true hormone level y_i(t). The paper never states whether this sum is over the Itrain=50/validation individuals (a legitimate population-level design) or over the Itest=10 test individuals (oracle leakage). Section 4.1 specifies the train/test split for MGP and DCNN training but is silent on how the ED days are selected for Table 1. If the test individuals' true curves were used, the headline advantage of B-MGP-DCNN (ED) at |ti|=10 (0.061 vs 0.302 for random B-MGP-DCNN) and the claim that |ti|=35 ED (0.037) beats |ti|=70 (0.050) are not attainable in a deployment where y_i(t) is unknown. If the schedule was computed from training data only, the comparison is fair; but as written the protocol is ambiguous, and the numerical claim cannot be evaluated. Section 5's statement that the ED function is computed at the cohort level does not resolve the ambiguity, because the cohort used for evaluation is not identified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an end-to-end framework for reconstructing and forecasting five reproductive hormone levels from sparse, irregular samples. The pipeline first fits an individual multi-task Gaussian process (MGP) over the observed hormone measurements, then maps sampled posterior streams through a population-level non-causal dilated convolutional network (DCNN) trained to minimize squared error against the true hormone curves. The authors also propose an Expected Distance (ED) acquisition function, derived from the MGP posterior, to greedily choose sampling days under a fixed budget, and they evaluate the method on a synthetic dataset of 60 hormone cycles generated from the Clark et al. mechanistic model with cycle length and ovulation day matched to Clue app users. Results in Table 1 indicate that a blockwise MGP plus DCNN with ED sampling outperforms baselines at most budgets, with the strongest claims being that ED sampling with 35 measurements (0.037 MSE) beats every-day (70-measurement) sampling with other methods, and that ED sampling at 10 measurements (0.061 MSE) beats all non-ED baselines at any budget below 35.","tokens_in":18047,"tokens_out":7337,"duration_ms":73901,"significance":"If the reported numbers are obtained without information leakage, the paper offers a practical method for sparse hormone monitoring: the combination of multi-task GPs with a dilated CNN is natural, the code is publicly released, and the ED sampling framework is a sensible way to reduce measurement burden. The paper also demonstrates the value of combining generative probabilistic models with deep sequence models for irregularly sampled physiological data. However, the central claims hinge on the ED evaluation protocol and on the fidelity of the synthetic simulator; these need to be clarified and strengthened.","major_comments":[{"comment":"The ED acquisition function in Eq. (3) uses the true hormone values y_i(t), but the paper never states whether the cohort I in Eq. (2) is the training/validation cohort (Itrain=50) or the test cohort (Itest=10). If the test individuals' true curves were used to select the ED days, the headlined advantages (e.g., 0.061 vs 0.302 at |ti|=10, and 0.037 at |ti|=35 vs 0.050 at |ti|=70) are oracle results and not attainable when y_i(t) is unknown. Please state the exact cohort used, and if it includes test individuals, rerun the evaluation with ED schedules computed from training data only.","section":"Section 4.2 / Table 1 and Section 3.2 / Eq. (3)"},{"comment":"The text acknowledges that 'performance results are volatile' due to randomness in training and optimization, yet Table 1 reports only single MSE values with no error bars, confidence intervals, or repeated-run statistics. Differences that are central to the paper, such as 0.037 vs 0.050 at |ti|=35 vs |ti|=70, are small relative to the acknowledged volatility, so the claim that ED sampling at 35 days outperforms every-day sampling is not statistically supported without variability estimates. Please report mean ± standard deviation (or similar) over multiple random seeds and sample draws for the key models and budgets.","section":"Section 4.2"},{"comment":"The ED schedule is said to be 'computed at the cohort level with respect to a normalized cycle length' with individual days readjusted to each woman's cycle length, but this procedure is never described in enough detail to reproduce it, and it is unclear whether Table 1 used this normalized-schedule transfer or used the true cycle-specific values directly. Please specify the exact algorithm used in the experiments, and if the cohort-level normalized schedule is the intended deployment mechanism, validate it by comparing its reconstruction error against the oracle ED schedule and random sampling.","section":"Section 5"},{"comment":"The evaluation is entirely on synthetic data from the Clark et al. model. While the authors clearly state this limitation, the abstract's unqualified 'accurate' and the clinical framing risk overclaiming; please add an explicit statement that all MSE numbers are relative to the simulator, and discuss the expected transfer to real hormonal data.","section":"Section 2"}],"minor_comments":[{"comment":"The sentence 'We propose to use a a non-causal dilated convolutional architecture' contains a duplicated article 'a a'.","section":"Section 3.2"},{"comment":"The phrase 'the-best performing approach' should be 'the best-performing approach'.","section":"Section 4.2"},{"comment":"The caption contains 'blue doted line', which should be 'blue dotted line'.","section":"Figure 2 caption"},{"comment":"The header 'Test-set overal MSE for P' contains a typo: 'overal' should be 'overall'.","section":"Table 7 header"},{"comment":"The reference 'Vassena et al.' spells the author's name as 'Valrie', which should be 'Valérie'.","section":"References"},{"comment":"The MGP kernel parameters are learned per individual, but the optimization procedure (e.g., initialization, gradient-based optimizer, number of restarts) is not described; please provide these details for reproducibility.","section":"Section 4.1"},{"comment":"Many entries in the per-hormone tables are non-monotonic in the sampling budget (e.g., Independent GPs at |ti|=70 are worse than at |ti|=35), which reinforces the need for repeated-run statistics; consider noting this in the text.","section":"Appendix Tables 2-8"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is potentially interesting for MLHC, and the public code is a plus. However, the ED sampling protocol ambiguity is serious: if the schedules in Table 1 were computed with test-set true hormone values, the paper's central message would be overstated and might warrant rejection rather than revision. I recommend asking the authors to (1) state the cohort used for ED schedule selection, (2) rerun experiments with training-data-only schedules if necessary, and (3) provide variability estimates for the key comparisons. I would also check the released code to verify the protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a competent, clearly-written ML4H paper that deserves a referee, but its strongest quantitative claim—that optimal sparse sampling beats dense sampling—rests on an evaluation protocol the paper never fully specifies. The combination of individual multi-task GPs with a population-level non-causal dilated CNN is a genuine, if incremental, novelty, and using MGP posterior samples as training inputs for the network is a nice touch. The ED acquisition function is also a reasonable idea, and the closed-form expression is useful.\n\nThe paper does several things well: the synthetic data generation is grounded in a real-world cycle-length distribution from Clue, the baselines are sensible, and the authors are honest about volatility. The code and data are public.\n\nNow the soft spots. First, the ED evaluation protocol is ambiguous. Equations (2) and (3) define the acquisition function in terms of the true hormone values y_i(t), and the paper never states whether the schedules used to produce Table 1 are computed from the 50 training individuals or from the 10 test individuals. If the latter, the headline advantage of B-MGP-DCNN (ED) at |ti|=10 (0.061 vs 0.302) is an oracle artefact, and the claim that optimal sampling at |ti|=35 beats every-day sampling would be unsupported. Section 5's statement that ED is computed at the cohort level with respect to a normalized cycle length suggests the schedule is meant to be transferable, but the evaluation section does not confirm which cohort was used. This needs to be checked against the released code; it is fixable in revision but load-bearing.\n\nSecond, the paper reports no error bars or repeated-run statistics despite acknowledging that results are volatile. The non-monotonic MSE across budgets (e.g., B-MGP-DCNN at 0.120 for |ti|=15 versus 0.189 for |ti|=25) suggests the point estimates are not stable. A handful of seeds would go a long way.\n\nThird, all evaluation is against a simulator. The simulator is grounded in real cycle characteristics, but the 'accurate' label in the abstract is only relative to that model. Real hormone dynamics almost certainly include more noise and inter-individual variability. This is a minor limitation for an ML4H methods paper, but should be stated more carefully.\n\nFor the record, the central reconstruction pipeline is not circular; the ED criterion is the only place where true values enter, and its use in defining the objective is defensible. The concern is solely about how it is evaluated.\n\nWho is the paper for? Researchers working on sparse multivariate time-series, particularly in healthcare. It would benefit from revision but is worth engaging with. I would accept it for peer review and ask for clarification of the ED protocol, error bars, and a more measured abstract.","headline":"A useful methods paper for sparse hormone reconstruction, but the headline sparse-sampling gains rest on an ambiguous, possibly oracle, evaluation protocol and synthetic data; worth refereeing with requests for clarification.","tokens_in":18630,"tokens_out":5630,"would_cite":false,"duration_ms":49797,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A carefully timed weekly blood-draw schedule can reconstruct and forecast all five daily reproductive hormone levels better than daily-sampled baselines.","keywords":["multi-task Gaussian processes","dilated convolutional networks","menstrual cycle modeling","reproductive hormone reconstruction","sparse irregular sampling","expected distance sampling","hormone forecasting","probabilistic time series"],"falsifier":"The clincher is a direct transfer test: collect daily hormone measurements from a small cohort of real women, run the same B-MGP-DCNN with ED sampling at budgets of 10 and 35, and compare the MSE and the chosen sampling days to the simulator-based results. If real-data error is substantially higher or the optimal days shift, the simulator-fidelity assumption fails.","tokens_in":17509,"feed_emoji":"🩸","tokens_out":12139,"duration_ms":113415,"temperature":0.7,"pith_summary":"The paper is trying to establish that a woman's full daily hormone profile over the menstrual cycle—estrogen, progesterone, inhibin, follicle-stimulating hormone, and luteinizing hormone—can be reconstructed and forecast from only a handful of blood draws, and that the calendar days chosen for those draws matter more than the number of draws. The proposed framework first fits a personalized multi-task Gaussian process to sparse, irregular measurements, producing a probabilistic curve for all five hormones at once, and then trains a dilated convolutional network across a population to correct those curves. A greedy \"expected distance\" rule picks the next sampling day where the model is most uncertain and most wrong, and the paper's headline result is that ten optimally chosen measurements over two cycles achieve lower reconstruction error (0.061 MSE) than any non-optimal baseline at any larger budget tested. Because no large gold-standard dataset of daily hormone measurements exists, training and evaluation use synthetic cycles produced by a mechanistic model and anchored to real-world cycle-length and ovulation-day statistics, so the \"accurate\" claim is measured against that simulator.","feed_headline":"Ten well-timed blood draws can reconstruct the full hormone cycle","feed_subtitle":"A Gaussian-process-plus-CNN model with greedy sampling beats daily-sampling baselines on simulated cycles.","key_machinery":"The load-bearing mechanism is a two-stage model with three components. First, a multi-task Gaussian process for each individual uses a covariance that is a Kronecker product of a hormone-correlation matrix and a periodic time kernel, $\\mathbf{K}_{h,h'} \\otimes k(t,t')$, so observations of one hormone inform predictions of the others and the posterior is a full distribution over time. Second, a non-causal dilated convolutional network—convolution filters that see past, present, and future with exponentially growing receptive fields—is trained on $S = 100$ sampled trajectories from those posteriors across a population; the network corrects systematic MGP errors without needing densely sampled individuals. Third, an Expected Distance acquisition function selects the next measurement day by maximizing the expected absolute deviation between the true hormone level and the MGP posterior, balancing exploitation where the posterior is confident and exploration where it is uncertain. The sampling days chosen by this rule, not the raw number of measurements, are what drive the reported accuracy gains.","core_discovery":"The central claim is that an end-to-end reconstruction pipeline—a per-individual multi-task Gaussian process followed by a population-level non-causal dilated convolutional network—plus an expected-distance sampling rule can reconstruct and forecast all five reproductive hormones accurately from very sparse measurements. In the paper's evaluation, the combined model with optimal sampling reaches an overall test MSE of 0.061 with only ten measurements across two cycles (roughly weekly sampling), and 0.037 at every-other-day sampling; both beat the same model with random sampling and all baselines, including daily-measured alternatives in the lower-budget regime. The paper attributes the gain to three jointly necessary ingredients: learning cross-hormone correlations in the Gaussian process, training the network on full posterior sample streams rather than point estimates, and choosing sampling times by expected distance to the true curve. The authors also report that separating the sharply peaked hormones (LH, FSH) from the smoother ones (E, P, Ih) into different kernel blocks improves accuracy.","pith_inferences":["The expected-distance rule itself is not tied to the simulator—it only needs a posterior and a target—so if real daily hormone data ever become available, the same machinery could relearn the optimal sampling days from measured curves.","The result suggests a general lesson for cyclic physiological signals: where phase is partly predictable, timing of sparse samples can matter more than sample count; the same design could be tried for cortisol, testosterone, or other hormones with known cycle structure.","The improved accuracy from separating peaked and smooth hormone groups is a testable modeling hypothesis for other multi-task GP applications: biomarkers with different curve shapes may need separate time kernels even when their levels are correlated.","A prospective study could assign sampling days by the expected-distance rule, measure only those days, and validate against daily lab values in a small cohort; that would resolve simulator fidelity without waiting for a large gold-standard dataset."],"forward_implications":["Weekly blood collection (about ten timed samples over two cycles) could be enough for personalized reconstruction and forecasting of all five hormone curves, making continuous monitoring practical outside the lab.","A population-level optimal sampling schedule can be computed once and then shifted to each woman's own cycle length and ovulation day, so no per-woman optimization is needed at collection time.","Because the same serum sample gives all five hormones, the framework's accuracy gains come without increasing per-visit cost or invasiveness.","Because training uses only 50 simulated individuals, the approach does not require a large collection of densely measured real hormone datasets to reproduce."],"supporting_citations":[{"why":"Mechanistic model of hormonal control of the menstrual cycle that generates the simulated hormone curves used as ground truth.","marker":"Clark et al. (2003)"},{"why":"Real-world tracker dataset providing the empirical cycle-length and ovulation-day distribution used to anchor the synthetic population.","marker":"Clue (2019)"},{"why":"Source of the known variability in cycle length and ovulation timing that grounds the individual-level modeling goal and the cohort age restriction.","marker":"Treloar et al. (1967)"},{"why":"Earlier single-hormone GP modeling of the menstrual cycle that the framework extends and whose peak-sampling findings motivate the ED rule.","marker":"Urteaga et al. (2017)"},{"why":"Multi-task GP formulation with a free-form inter-task kernel used to capture hormone correlations while keeping training data requirements small.","marker":"Bonilla et al. (2008)"},{"why":"The empirical comparison that justifies choosing dilated convolutional networks over recurrent networks for sequence reconstruction.","marker":"Bai et al. (2018)"},{"why":"Provides the standard GP formulation and kernel machinery the MGP posterior and the closed-form Expected Distance expression rely on.","marker":"Rasmussen and Williams (2006)"},{"why":"Bayesian-optimization tutorial that motivates the Expected Distance acquisition function as an exploration-exploitation sampling rule.","marker":"Frazier (2018)"}],"fun_headline_variants":["Ten smart draws reconstruct daily hormone levels","Optimal sampling: 10 draws beat daily blood tests","AI maps hormone cycles from sparse blood samples","Ten well-timed draws predict full hormone curve","Greedy sampling + deep net: hormone cycle from 10 draws"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole evaluation assumes that synthetic cycles generated by a mechanistic simulator—with cycle length and ovulation day matched to real tracker users—behave like real women's daily hormone curves; if real curves differ in timing, peak shape, noise, or cross-hormone coupling, the reported accuracies and optimal sampling days do not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Ten smart draws reconstruct daily hormone levels","Optimal sampling: 10 draws beat daily blood tests","AI maps hormone cycles from sparse blood samples","Ten well-timed draws predict full hormone curve","Greedy sampling + deep net: hormone cycle from 10 draws"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1353,"prompt_tokens":903,"completion_tokens":450,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":375}},"tokens_in":519,"tokens_out":450,"duration_ms":4942,"temperature":1.0,"reasoning_tokens":375,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:50:15.702890+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The clincher is a direct transfer test: collect daily hormone measurements from a small cohort of real women, run the same B-MGP-DCNN with ED sampling at budgets of 10 and 35, and compare the MSE and the chosen sampling days to the simulator-based results. If real-data error is substantially higher or the optimal days shift, the simulator-fidelity assumption fails.","supporting_citations":[],"review_version":1}