Pith. sign in

REVIEW 3 major objections 4 minor 59 references

A pre-deployment digital twin can match a retrospective oracle for mobile-health algorithm testing.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 07:30 UTC pith:W2P3T2P3

load-bearing objection A serious, carefully evaluated mHealth digital-twin paper; the γ reporting inconsistency is real but the headline claims largely survive at other γ values, so this needs revision, not rejection. the 3 major comments →

arxiv 2607.21403 v1 pith:W2P3T2P3 submitted 2026-07-23 cs.LG stat.ME

A Diffusion-Model Subpopulation Digital Twin for Mobile Health Deployment: A Case Study on the HeartSteps Intervention

Ziping Xu , Yuyi Chang , Chenshun Ni , Nithin Sugavanam , Asim H. Gazi , Pedja Klasnja , Emre Ertin , Susan A. Murphy This is my paper
classification cs.LG stat.ME
keywords digital twinjust-in-time adaptive interventionmobile healthdiffusion modeltemporal consistencyinference-time calibrationHeartStepssubpopulation simulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to establish that a digital twin of an upcoming mobile-health subpopulation can be built entirely from data available before the deployment runs, and that this twin is faithful enough to test candidate online learning algorithms. It claims a three-stage pipeline — pre-training a temporally consistent diffusion model on a large observational cohort, fine-tuning on small prior intervention trials, and inference-time calibration from a domain-scientist forecast — produces a simulator that reproduces the target cohort's daily activity profile, temporal bout structure, and between-participant heterogeneity. On a replay of the HeartSteps v2–v4 deployments, the calibrated twin at the v4 hand-off is the only non-oracle simulator faithful on all three axes at once, bringing the daily-profile Wasserstein to 8.34 against a real-data floor of 8.30 and matching the v4-oracle within one standard error on every metric, using no v4 data. A sympathetic reader would care because committing to an algorithm design before enrollment is the prerequisite for reproducible, low-burden adaptive interventions.

Core claim

The central claim is that a fully pre-deployment JITAI-Twin — a conditional diffusion model fine-tuned on prior HeartSteps deployments and calibrated at inference time toward a language-model-forecast target profile — reproduces a held-out target deployment's temporal structure, between-participant heterogeneity, and marginal daily distribution together, which the paper shows no simpler baseline does. At the v4 hand-off the calibrated v2/v3 twin sits at the real-data floor on daily-profile Wasserstein (8.34 vs 8.30), posts the panel's best run-length Wasserstein (0.22), and preserves heterogeneity ratios (bu-acf 0.28, bu-run 0.31) that retrieval and parametric baselines collapse toward near

What carries the argument

The load-bearing object is a temporally consistent conditional time-series diffusion model: a denoising network that generates a whole 168-hour week of hourly step counts at once, conditioned on a frozen text-embedding of each participant's baseline survey and the week's intervention sequence. Temporal consistency is enforced architecturally by forward-only state-space kernels and left-padded convolutions, so the generated hour never depends on future actions — a requirement for the twin to serve as a valid environment for online learning algorithms. The three-stage training scheme pre-trains the backbone on a large observational cohort with no intervention, fine-tunes only 1.6% of parameter

Load-bearing premise

The calibrated twin's success rests on the pre-deployment forecast of the target population's mean step profile being roughly right; the paper tests that forecast on one target population using a language model standing in for a domain scientist, and with the forecast wrong the calibration would pull the twin toward the wrong level and shape.

What would settle it

Run the full pre-deployment pipeline on a target population whose true per-hour profile is known in advance but give the calibration an intentionally wrong forecast (for example, tell the language model the target is more active in the morning when it is actually active in the evening): if the calibrated twin's daily-profile Wasserstein and per-hour MAE degrade to the uncalibrated level — or track the wrong forecast — the inference-time calibration is doing nothing more than matching the guess, and the twin's fidelity is contingent on forecast accuracy rather than on the model.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Candidate online algorithm designs can be ranked on realistic simulated subpopulations before anyone is enrolled, letting researchers commit to a configuration without mid-deployment improvisation.
  • The twin's advantage over resampling baselines is largest exactly when consecutive deployments differ, so it targets the 'data impoverishment' setting where prior-deployment data are stale.
  • Because between-participant heterogeneity is preserved, decisions about how much to personalize or pool across participants can be informed by simulation — choices that every baseline simulator erases.
  • The calibration mechanism generalizes beyond the per-hour mean to any approximately linear target, including action-dependent targets that let a scientist inject an expected intervention effect directly into the sampler.
  • The paper's own limitation is explicit: only the data-to-model direction of the twin loop is demonstrated; the model-to-decision direction remains to be tested prospectively.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The language-model forecast — produced without naming the target deployment or using any post-deployment results — suggests that the calibration step could in principle be fully automated from a written protocol and prior-trial statistics, but whether that holds for a human domain scientist or across other behavioral domains is untested.
  • A testable transfer check: apply the same pipeline to a different JITAI program (e.g., oral self-care or substance use) where the target population and diurnal behavior differ more radically; the architecture makes no program-specific assumptions beyond hourly outcomes.
  • The near-null learned intervention effect (−0.02% weekly steps at observed send rates) is stated as a property of the HeartSteps program, but it also means the fine-tuning stage may be adapting baseline behavior rather than learning intervention response — a synthetic experiment with a programmed known effect could separate the two.
  • The trust parameter γ appears with different values (1.0, 1.5, 0.9) across sections, so calibration strength apparently needs per-hand-off adjustment; a principled, data-driven way to set γ from the prior data would be a natural next step.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes JITAI-Twins, a three-stage pipeline for building digital twins of target subpopulations in mobile-health JITAIs: (1) pre-training a temporally consistent conditional time-series diffusion model on large observational data (All of Us), (2) parameter-efficient fine-tuning on prior HeartSteps deployments (v2/v3), and (3) inference-time reward-tilted calibration toward a pre-deployment target profile elicited from a domain scientist or, in the evaluation, an LLM. The method is evaluated by replaying the HeartSteps v2/v3 and v4 deployments as pretend-prospective hand-offs, comparing against KNN, per-state rejection sampling, linear SEM, and a GRU baseline. The central claim is that, at the v4 hand-off, the calibrated v2/v3-fine-tuned twin is the only non-oracle simulator faithful on daily-profile distribution, temporal bout structure, and between-participant heterogeneity, coming within one standard error of the v4 oracle on all columns.

Significance. If the central claim holds, this would be a practically valuable tool: an mHealth team could compare online learning algorithms on a simulator before a new deployment, using only pre-deployment knowledge. The evaluation design has many genuine strengths: the daily-profile Wasserstein is footprint-matched to control sample size; standard errors are participant bootstraps; oracle rows and split-half reference rows are included; run-length metrics are checked at two activity thresholds; and Appendix J gives a stage-by-stage credit-attribution ablation. The temporally consistent (forward-only) architectural constraint is a clear and useful contribution for simulation-based evaluation of online algorithms. The authors also state limitations candidly (single distribution shift, no prospective model-to-decision test, no causal guarantee). However, the pre-deployment claim currently rests on inconsistent reporting of the calibration trust parameter and on checkpoint-selection choices that are not fully prospective; these issues are load-bearing and need to be resolved before the claim is established.

major comments (3)
  1. [§4.1, App. G, App. H.2, App. J] The calibration trust parameter γ is the single knob that produces the calibrated rows of Table 2, yet the manuscript reports three different values. Section 4.1 says γ=1.0 is a fixed default; Appendix G's Figure A9 caption and Appendix H.2 use γ=1.5 for the v4 calibration; Appendix J uses γ=0.9 for its ablation and states that Table 2's calibrated row uses 'the γ=1.5 default of Section 3.4.' These statements are mutually inconsistent. Because calibration delivers the headline v4 gains (run-length W 0.80→0.22; daily W 8.43→8.34), the reader cannot tell whether those rows reflect a fixed pre-specified default or a favorable post hoc selection. Please state exactly which γ produced each Table 2 row, show the γ=1.0 row explicitly, and provide a sensitivity sweep over γ for all Table 2 metrics at both hand-offs. If γ was tuned on held-out target metrics, the pre-deployment claim must be rest
  2. [App. D, App. E] The pretend-prospective replay appears to use target-data information in model selection. Appendix D states that the pretrain-only twin reported in Table 2 is the 70,000-iteration checkpoint 'which generates better for the held-out HeartSteps cohorts,' while fine-tuning initializes from the final 150,000-iteration checkpoint. At the first hand-off, v2/v3 is the upcoming target, so selecting the checkpoint on v2/v3 fidelity violates the prospective protocol. Appendix E reports early fine-tuning checkpoints (3,000 iterations for the v2/v3-fine-tuned twin used at the v4 hand-off) but gives no selection rule; if that iteration count was chosen using v4 metrics, the second hand-off is also not fully prospective. Please clarify the selection rule for every reported checkpoint and, if necessary, re-run the evaluation using only pre-deployment criteria (e.g., a fixed training schedule or validat
  3. [Table A6, §4.7, App. H] The calibrated pipeline's contribution is tied to the accuracy of the pre-deployment target µ⋆, and the paper validates µ⋆ on a single target population (v4) with an LLM standing in for a domain scientist. Table A6 does show that the headline daily Wasserstein is similar for several target sources, which is reassuring, but the main text does not quantify how sensitive the Table 2 metrics are to perturbations of the LLM forecast itself (as opposed to alternative target sources). Since the pre-deployment claim depends on a forecast that is elicited with a prompt and an LLM, please add a perturbation analysis around the realized µ⋆ (e.g., scaling or smoothing the shape, varying the LLM temperature/draws) and report the resulting ranges for the Table 2 columns. This would also address whether the 'within one standard error of the oracle' conclusion is stable to forecast error.
minor comments (4)
  1. [§4.4] The sentence 'the only non-oracle simulator faithful on all three axes at once' is a strong qualitative claim. Consider stating the criterion explicitly (e.g., which columns must be within a given tolerance of the real-data reference or of the oracle) or supporting it with pairwise significance tests, rather than relying on visual inspection of overlapping error bars.
  2. [Table A6, §4.7, App. G] The calibration comparison table uses a held-out test split with per-participant matching for the non-distributional columns, while Table 2 uses full-cohort scores. This is explained in the caption, but the main text sometimes quotes numbers from Table A6 (e.g., MAE 53 vs 48) without reminding the reader that they are not directly comparable to Table 2's ACF-MAE and run-length columns. Please align terminology or add a sentence in §4.7 noting the different scoring protocol.
  3. [App. H.2] Appendix H.2 says an 'earlier version' of the prompt returned the prior-trial shape and mislocated v4's activity, and that the final protocol removes the anchor and takes the daily level from prior data. This is an important methodological detail; it should be summarized in the main text (Section 4.7) so readers know the LLM protocol was refined during development, not fixed a priori in its final form.
  4. [Table 2 note] The bold rule — mark any entry whose ±1 SE overlaps the column's best non-reference entry — is unusual and makes many entries bold. It would be clearer to bold only the best entry per column, or to mark significant differences from the best explicitly.

Circularity Check

0 steps flagged

No significant circularity: the calibration target is a genuine pre-deployment forecast, and the headline temporal-structure and heterogeneity metrics are not directly controlled by the calibration step.

full rationale

The only quantity the calibration step directly sets is the per-hour mean: Section 3.4 defines the tilt so that the sampler 'approximately shifts the mean of s by γv' with v = μ* − μ_base. The paper explicitly flags the per-hour-of-day mean MAE as the metric that 'the calibration target directly optimizes' (Appendix I) and uses it to evaluate the LLM-generated forecast, not as an independent fidelity claim for the twin. The headline results — daily-profile Wasserstein, run-length Wasserstein, and the between-participant heterogeneity ratios — are not calibration targets. Appendix G/Table A6 shows that a naive post-hoc mean shift can degrade the daily-profile Wasserstein (8.43→9.96) and run-length Wasserstein (0.74→2.64), so the calibrated twin's improvements on these metrics are empirical rather than forced by construction. The target μ* is produced from pre-deployment knowables (prior-trial statistics plus the upcoming site's eligibility criteria), not from the held-out v4 data, and the paper evaluates the LLM forecast separately against v4. The self-citations (Gazi et al.) supply definitions and context rather than the load-bearing derivation; the core sampling framework (Moitra et al.) and architecture backbone (TimeWeaver) are external. The inconsistent reporting of γ (1.0 in Section 4.1, 1.5 in Appendices G/H.2, 0.9 in Appendix J) is a reproducibility and robustness concern, but it does not make any predicted quantity equivalent to its input by construction. No circular step meeting the quoted-evidence standard was found.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

No new physical entities are introduced. The method relies on several fitted/arbitrary constants (γ, iteration counts, thresholds, compression breakpoint) and on assumptions about survey embedding semantics, transfer across subpopulations, and the LLM-as-expert substitution. None of these are independently evidenced beyond the reported single-case replay.

free parameters (5)
  • γ (calibration trust parameter) = 1.0 (§4.1), 1.5 (App G/H), 0.9 (App J)
    Controls the strength of the reward-tilted calibration; reported inconsistently across sections, suggesting it is tuned per setting rather than a fixed default.
  • Fine-tuning iteration count = 3,000 (v2/v3 twin), 15,000 (v2/v3 oracle)
    Early checkpoints are selected based on validation performance; the v2/v3 oracle is explicitly chosen for its between-participant heterogeneity, an oracle-type selection.
  • Log-compression breakpoint a = 600 steps/hour
    Chosen transform parameter in the piecewise log-compression (Appendix B.6); not theoretically forced.
  • Wear/artifact thresholds = 200 steps/day non-wear; 45,000 steps/hour artifact
    Arbitrary preprocessing thresholds used to mask the diffusion loss; affect all reported metrics.
  • Activity threshold τ for run-length = 100 steps/hour
    HeartSteps convention, but still a hand-chosen binarization threshold; robustness to 250 steps/hour is reported, but the main table uses 100.
axioms (5)
  • domain assumption The frozen T5 text encoder preserves the semantic content of harmonized baseline surveys across different schemas.
    Appendix C builds the participant embedding through natural-language summaries and T5; if this embedding loses cross-study information, conditioning is degraded.
  • domain assumption Temporal and between-participant behavioral dynamics learned from prior deployments transfer to the target subpopulation.
    The v4 hand-off relies on v2/v3 fine-tuning transferring to a different population/geography/era; the paper acknowledges this is a single-case test.
  • domain assumption An LLM (GPT-5.4-mini) can serve as a valid proxy for domain-scientist expertise in generating the calibration target µ⋆.
    Section 4.7 evaluates the LLM forecast only on one target (v4); the entire calibration pipeline depends on this input.
  • domain assumption Forward-only S4 kernels and left-padded 1-D convolutions are sufficient to guarantee conditional independence from future actions in the full reverse-diffusion chain.
    The temporal-consistency claim (St+1 ⊥ At+2,... | Ht, At) is argued architecturally (Appendix B.4) but not empirically verified on the trained model.
  • standard math The reward-tilted sampling algorithm of Moitra et al. (2026) approximately implements a mean shift by γv in the model's operating space.
    Appendix F derives the tilted reverse step from Algorithm 1 of Moitra et al.; the approximation is taken from that cited work.

pith-pipeline@v1.3.0-alltime-deepseek · 37335 in / 14236 out tokens · 134090 ms · 2026-08-01T07:30:51.641742+00:00 · methodology

0 comments
read the original abstract

Mobile-health interventions increasingly use online learning and decision making algorithms to personalize when to nudge users toward healthier behavior, but a poorly designed algorithm can burden and disengage participants. New algorithm design decisions should therefore be vetted against realistic simulated users before each real-life deployment. We propose a method to develop ``JITAI-Twins'': digital twins of a target subpopulation for comparing candidate online algorithms before a just-in-time adaptive intervention (JITAI) deployment. The method builds on a conditional time-series diffusion model that is temporally consistent (future actions do not affect the generated past), and it supports repeated updating from three sources of information, in three steps: pre-training on a large observational dataset, fine-tuning on small prior intervention deployments in related populations, and inference-time calibration to the next target population from domain-scientist expertise. We validate the twin at each pre-deployment stage of the long-running HeartSteps series (v2 through v4) of physical-activity suggestion intervention deployments, treating each successive deployment as an upcoming study. The proposed method reproduces the target subpopulation's temporal and between-participant structure better than simpler simulators. These results suggest that our twin can be used to simulate a target deployment before it runs, the prerequisite for testing and informing online algorithm design decisions.

Figures

Figures reproduced from arXiv: 2607.21403 by Asim H. Gazi, Chenshun Ni, Emre Ertin, Nithin Sugavanam, Pedja Klasnja, Susan A. Murphy, Yuyi Chang, Ziping Xu.

Figure 1
Figure 1. Figure 1: The JITAI-Twin pipeline: pre-train on All of Us, fine-tune on prior HeartSteps deployments, then calibrate the sampler to the target deployment at inference time from domain knowledge or an LLM forecast. The fine-tuning data are small-scale intervention data from prior deployments, not data from the target population. architecture (Narasimhan et al., 2024), on a large observational dataset of people monito… view at source ↗
Figure 2
Figure 2. Figure 2: Mean and standard deviation of step-count by hour of day across the HeartSteps deployments and the All of Us (AoU) pre-training cohort. All of Us (Ramirez et al., 2022) is a national observational cohort run by the NIH: hundreds of thousands of volunteers across the United States contribute health data, and a large subset wears commercial activity trackers in daily life without receiving any intervention. … view at source ↗
Figure 3
Figure 3. Figure 3: The HeartSteps system: an activity-suggestion notification (left) and the step￾count graph in the HeartSteps app (right) [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Two-step prompt for producing the target profile µ ⋆ before the HeartSteps v4 deployment runs. Step 1 elicits a qualitative account of how the new population differs from the prior trials; Step 2 turns it into a normalized target profile for the sampler. Bracketed italics mark the study-specific fields, replaced per deployment. profile, the mean trajectory that the calibrated twin should match (e.g., the p… view at source ↗
Figure 5
Figure 5. Figure 5: shows the distributional picture against the baselines for the second hand-off, using the v4-fine-tuned twin (the oracle row of Table 2b): it tracks v4’s active-hour distribution and long upper tail (panel a) and follows the bimodal diurnal shape in the per-hour mean (panel b), its main miss being the exact-zero spike, which a continuous sampler cannot emit. The baselines fail differently: PSRS over-produc… view at source ↗
Figure 6
Figure 6. Figure 6: Demographic conditioning on v4. (a) Each point is one demographic variable, binarized into two groups over the whole cohort; axes are the between-group difference in mean daily steps (real x, twin y). All six variables agree in sign. (b) Two fields the twin matches in direction and magnitude: stress and employment. 4.6. What the twin learned about the intervention. Demographic conditioning is one half of w… view at source ↗
Figure 7
Figure 7. Figure 7: Producing and applying the calibration target. (a,b) The protocol-only LLM forecast (red) of the target diurnal shape at the two hand-offs, against the true target (black) and the prior deployment (AoU for the first hand-off, v2/v3 for the second hand-off); the daily level is taken from the prior. (c,d) Calibrating the v2/v3 twin toward the v4 target moves it onto real v4: it matches the marginal step dist… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 2 canonical work pages

  1. [1]

    Frontiers of Information Technology & Electronic Engineering , volume=

    Diffusion models for time-series applications: a survey , author=. Frontiers of Information Technology & Electronic Engineering , volume=. 2024 , publisher=

    2024

  2. [2]

    International conference on machine learning , pages=

    Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  3. [3]

    arXiv preprint arXiv:2403.02682 , year=

    Time weaver: A conditional time series generation model , author=. arXiv preprint arXiv:2403.02682 , year=

    arXiv

  4. [4]

    Journal of personalized medicine , volume=

    Survey of transfer learning approaches in the machine learning of digital health sensing data , author=. Journal of personalized medicine , volume=. 2023 , publisher=

    2023

  5. [5]

    2018 , publisher=

    Improving language understanding by generative pre-training , author=. 2018 , publisher=

    2018

  6. [6]

    Bert: Pre-training of deep bidirectional transformers for language understanding , author=. Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) , pages=. 2019 , doi=

    2019

  7. [7]

    Personalized

    Liao, Peng and Greenewald, Kristjan and Klasnja, Predrag and Murphy, Susan , journal=. Personalized. 2020 , publisher=

    2020

  8. [8]

    Efficacy of contextually tailored suggestions for physical activity: a micro-randomized optimization trial of

    Klasnja, Predrag and Smith, Shawna and Seewald, Nicholas J and Lee, Andy and Hall, Kelly and Luers, Brook and Hekler, Eric B and Murphy, Susan A , journal=. Efficacy of contextually tailored suggestions for physical activity: a micro-randomized optimization trial of. 2019 , publisher=

    2019

  9. [9]

    arXiv preprint arXiv:2509.13499 , year=

    Reproducible workflow for online AI in digital health , author=. arXiv preprint arXiv:2509.13499 , year=

    arXiv

  10. [10]

    Annals of Behavioral Medicine , volume=

    Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support , author=. Annals of Behavioral Medicine , volume=. 2018 , publisher=

    2018

  11. [11]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  12. [12]

    arXiv preprint arXiv:2208.09399 , year=

    Diffusion-based time series imputation and forecasting with structured state space models , author=. arXiv preprint arXiv:2208.09399 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2408.15076 , year=

    Miwaves reinforcement learning algorithm , author=. arXiv preprint arXiv:2408.15076 , year=

    Pith/arXiv arXiv

  14. [14]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    A deployed online reinforcement learning algorithm in an oral health clinical trial , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=. 2025 , doi=

    2025

  15. [15]

    Advances in Neural Information Processing Systems , volume=

    Improved variance-aware confidence sets for linear bandits and linear mixture mdp , author=. Advances in Neural Information Processing Systems , volume=

  16. [16]

    International Conference on Artificial Intelligence and Statistics , pages=

    Offline policy selection under uncertainty , author=. International Conference on Artificial Intelligence and Statistics , pages=. 2022 , organization=

    2022

  17. [17]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Offline evaluation of online reinforcement learning algorithms , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=. 2016 , doi=

    2016

  18. [18]

    arXiv preprint arXiv:2211.07614 , year=

    Towards data-driven offline simulations for online reinforcement learning , author=. arXiv preprint arXiv:2211.07614 , year=

    Pith/arXiv arXiv

  19. [19]

    arXiv preprint arXiv:2205.08716 , year=

    No more pesky hyperparameters: Offline hyperparameter tuning for RL , author=. arXiv preprint arXiv:2205.08716 , year=

    Pith/arXiv arXiv

  20. [20]

    Modeling engagement with a digital behavior change intervention (

    De La Torre, Steven A and El Mistiri, Mohamed and Hekler, Eric and Klasnja, Predrag and Marlin, Benjamin and Pavel, Misha and Spruijt-Metz, Donna and Rivera, Daniel E , journal=. Modeling engagement with a digital behavior change intervention (. 2024 , publisher=

    2024

  21. [21]

    Operations Research , volume=

    Nonstationary bandits with habituation and recovery dynamics , author=. Operations Research , volume=. 2020 , publisher=

    2020

  22. [22]

    Journal of the American Medical Informatics Association , volume=

    Synthetic seismocardiogram generation using a transformer-based neural network , author=. Journal of the American Medical Informatics Association , volume=. 2023 , publisher=

    2023

  23. [23]

    arXiv preprint arXiv:2411.18822 , year=

    RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data , author=. arXiv preprint arXiv:2411.18822 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2410.13638 , year=

    Scaling Wearable Foundation Models , author=. arXiv preprint arXiv:2410.13638 , year=

    Pith/arXiv arXiv

  25. [25]

    Gazi, A. H. and Gao, D. and Ghosh, S. and Xu, Z. and Trella, A. and Klasnja, P. and Murphy, S. A. , title =. Journal of Medical Internet Research , year =. doi:10.2196/72830 , url =

    work page doi:10.2196/72830

  26. [26]

    arXiv preprint arXiv:2310.17848 , url=

    Boosting data analytics with synthetic volume expansion , author=. arXiv preprint arXiv:2310.17848 , url=. 2023 , doi=

    Pith/arXiv arXiv 2023

  27. [27]

    arXiv preprint arXiv:2405.02358 , year=

    A survey of time series foundation models: Generalizing time series representation with large language model , author=. arXiv preprint arXiv:2405.02358 , year=

    arXiv

  28. [28]

    arXiv preprint arXiv:2602.16570 , year=

    Steering diffusion models with quadratic rewards: A fine-grained analysis , author=. arXiv preprint arXiv:2602.16570 , year=

    arXiv

  29. [29]

    International Conference on Learning Representations , year=

    DiffWave: A Versatile Diffusion Model for Audio Synthesis , author=. International Conference on Learning Representations , year=

  30. [30]

    Advances in Neural Information Processing Systems , volume=

    Denoising diffusion probabilistic models , author=. Advances in Neural Information Processing Systems , volume=

  31. [31]

    Journal of Machine Learning Research , volume=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. Journal of Machine Learning Research , volume=

  32. [32]

    2024 , doi =

    Foundational Research Gaps and Future Directions for Digital Twins , publisher =. 2024 , doi =

    2024

  33. [33]

    and Sacks, Michael S

    Niederer, Steven A. and Sacks, Michael S. and Girolami, Mark and Willcox, Karen , title =. Nature Computational Science , volume =. 2021 , doi =

    2021

  34. [34]

    and Barto, Andrew G

    Sutton, Richard S. and Barto, Andrew G. , title =

  35. [35]

    Bandit Algorithms , publisher =

    Lattimore, Tor and Szepesv. Bandit Algorithms , publisher =. 2020 , doi=

    2020

  36. [36]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , title =

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , title =. International Conference on Learning Representations (ICLR) , year =

  37. [37]

    Ramirez, Andrea H and Sulieman, Lina and Schlueter, David J and Halvorson, Alese and Qian, Jun and Ratsimbazafy, Francis and Loperena, Roxana and Mayo, Kelsey and Basford, Melissa and Deflaux, Nicole and others , journal =. The. 2022 , publisher =

    2022

  38. [38]

    and O'Hagan, Anthony , title =

    Kennedy, Marc C. and O'Hagan, Anthony , title =. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume =. 2001 , doi =

    2001

  39. [39]

    and Liu, Yuan , title =

    Wang, Yuchen and Su, Zhongzheng and Guo, Shuo and Dai, Minghao and Luan, Tom H. and Liu, Yuan , title =. arXiv preprint arXiv:2401.13699 , year =

    Pith/arXiv arXiv

  40. [40]

    and Zhang, Kelly W

    Trella, Anna L. and Zhang, Kelly W. and Nahum-Shani, Inbal and Shetty, Vivek and Doshi-Velez, Finale and Murphy, Susan A. , title =. Algorithms , volume =. 2022 , doi =

    2022

  41. [41]

    and Zhang, Kelly W

    Trella, Anna L. and Zhang, Kelly W. and Carpenter, Stephanie M. and Elashoff, David and Shetty, Vivek and Doshi-Velez, Finale and Dempsey, Walter and Murphy, Susan A. , title =. arXiv preprint arXiv:2406.13127 , year =

    Pith/arXiv arXiv

  42. [42]

    and Ghosh, Susobhan and Guo, Yongyi and Gao, Daiqi and Xu, Ziping and Nahum-Shani, Inbal and Murphy, Susan A

    Gazi, Asim H. and Ghosh, Susobhan and Guo, Yongyi and Gao, Daiqi and Xu, Ziping and Nahum-Shani, Inbal and Murphy, Susan A. , journal =. doi:10.31234/osf.io/47w3e_v2 , year =

    work page doi:10.31234/osf.io/47w3e_v2

  43. [43]

    and Gullapalli, Bhanu Teja and Gao, Daiqi and Marlin, Benjamin M

    Gazi, Asim H. and Gullapalli, Bhanu Teja and Gao, Daiqi and Marlin, Benjamin M. and Shetty, Vivek and Murphy, Susan A. , journal=. 2025 , doi=

    2025

  44. [44]

    and Gazi, Asim H

    Gullapalli, Bhanu Teja and Shetty, Vivek and Trella, Anna L. and Gazi, Asim H. and Murphy, Susan A. , journal=. 2026 , doi=

    2026

  45. [45]

    , journal=

    Chasalow, Kyla and Wu, Skyler and Murphy, Susan A. , journal=. 2025 , doi=

    2025

  46. [46]

    Biometrika , volume=

    On the likelihood that one unknown probability exceeds another in view of the evidence of two samples , author=. Biometrika , volume=. 1933 , doi=

    1933

  47. [47]

    and Van Roy, Benjamin and Kazerouni, Abbas and Osband, Ian and Wen, Zheng , journal=

    Russo, Daniel J. and Van Roy, Benjamin and Kazerouni, Abbas and Osband, Ian and Wen, Zheng , journal=. A Tutorial on. 2018 , doi=

    2018

  48. [48]

    arXiv preprint arXiv:2601.15353 , year=

    Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions , author=. arXiv preprint arXiv:2601.15353 , year=. 2601.15353 , archivePrefix=

    Pith/arXiv arXiv

  49. [49]

    PLoS Medicine , volume=

    The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review , author=. PLoS Medicine , volume=. 2013 , publisher=

    2013

  50. [50]

    Science Translational Medicine , volume=

    The emerging field of mobile health , author=. Science Translational Medicine , volume=. 2015 , publisher=

    2015

  51. [51]

    Learning phrase representations using

    Cho, Kyunghyun and van Merri. Learning phrase representations using. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=. 2014 , doi=

    2014

  52. [52]

    Mobile health technology evaluation: the

    Kumar, Santosh and Nilsen, Wendy J and Abernethy, Amy and Atienza, Audie and Patrick, Kevin and Pavel, Misha and Riley, William T and Shar, Albert and Spring, Bonnie and Spruijt-Metz, Donna and others , journal=. Mobile health technology evaluation: the. 2013 , publisher=

    2013

  53. [53]

    2013 , publisher=

    Statistical Power Analysis for the Behavioral Sciences , author=. 2013 , publisher=

    2013

  54. [54]

    and Yom-Tov, Elad and Keller, Punam A

    Lauffenburger, Julie C. and Yom-Tov, Elad and Keller, Punam A. and McDonnell, Marie E. and Crum, Katherine L. and Bhatkhande, Gauri and Sears, Ellen S. and Hanken, Kaitlin and Bessette, Lily G. and Fontanet, Constance P. and Haff, Nancy and Vine, Seanna and Choudhry, Niteesh K. , journal=. The impact of using reinforcement learning to personalize communic...

    2024

  55. [55]

    Journal of Behavioral Medicine , volume =

    Can the artificial intelligence technique of reinforcement learning use continuously-monitored digital data to optimize treatment for weight loss? , author =. Journal of Behavioral Medicine , volume =. 2019 , doi =

    2019

  56. [56]

    JAMA Internal Medicine , volume =

    Patient-centered pain care using artificial intelligence and mobile health tools: a randomized comparative effectiveness trial , author =. JAMA Internal Medicine , volume =. 2022 , doi =

    2022

  57. [57]

    JMIR mHealth and uHealth , volume =

    Evaluating machine learning-based automated personalized daily step goals delivered through a mobile phone app: randomized controlled trial , author =. JMIR mHealth and uHealth , volume =. 2018 , doi =

    2018

  58. [58]

    International Journal of Environmental Research and Public Health , volume =

    Reinforcement learning to send reminders at right moments in smartphone exercise application: a feasibility study , author =. International Journal of Environmental Research and Public Health , volume =. 2021 , doi =

    2021

  59. [59]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Field study in deploying restless multi-armed bandits: Assisting non-profits in improving maternal and child health , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2022 , doi =

    2022