{"id":"1bf9512f-919f-45f4-8ee3-cec3149ee37f","arxiv_id":"2507.19491","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Daily self-rated sleep quality in 29 adults correlates weakly with Oura-derived changes in REM sleep, heart rate, bedtime, and working memory scores, and the alignment differs across three user groups.","lead":"A 29-person field study compared daily sleep-quality self-reports with Oura ring data and a working memory task. It finds small but significant links between self-assessments and changes in REM sleep, heart rate, bedtime, and N-back scores, with large individual differences.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'three groups' and differential information-gain claims rest on in-sample GMM clustering with n chosen to maximize R^2; no out-of-sample validation is reported, so the paper's central novelty is unsupported as it stands.","rationale":"The reader's conditional verdict is appropriate, and the weak overall correlations are likely robust; even substantial noise in Oura's REM estimates would tend to attenuate the already-weak effects, not manufacture them. However, the reader's weakest assumption (Oura REM validity) is not the most load-bearing concern. The subgroup analysis in Section 4.2 is a clear case of in-sample model selection: the number of groups is chosen by R^2, and the GMM and group-specific regressions are fit and evaluated on the same participants. The central novelty of 'three groups' with different sensitivities, and the resulting claim of differential information gain, would not survive honest out-of-sample evaluation if the clustering is noise-driven. The paper itself flags this limitation in Section 5.1. A participant-level cross-validation test would settle whether the grouping generalizes. If it does not, the paper should remove or heavily qualify the group and information-gain conclusions. This keeps the overall verdict conditional but identifies the specific analysis that must be validated.","tokens_in":23031,"tokens_out":8742,"duration_ms":87584,"concrete_test":"Run a participant-level cross-validation of the full subgroup pipeline on the released data: split participants into training and held-out sets, fit the GMM on training participants only, assign held-out participants to the nearest mixture component, fit the Table 3 mixed model within each training group, and compute held-out R^2 (squared correlation of observed vs predicted self-assessment) for n=2,3,4. If n=3 no longer has the highest held-out R^2, or if the group-level held-out R^2 values converge to the overall ~0.15, the three-group / information-gain claim is an artifact of in-sample selection. A simpler alternative is to bootstrap participants and re-run the GMM plus per-group models, reporting the distribution of the R^2 differences.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 4.2 reports the subgroup analysis: a Gaussian mixture model is fit on sleep features, the Table 3 mixed-effects model is refit separately within each GMM-derived group, and the number of groups n=2,3,4 is compared by R^2_marginal/conditional (0.157/0.177, 0.185/0.208, 0.163/0.190). The authors select n=3 because it yields the highest R^2. All of these values are in-sample: the GMM and the group-specific regressions are fit and evaluated on the same participants. No cross-validation, bootstrap, or correction for selecting the number of groups is performed. The paper's central novelty—that 'self-report sensitivity towards sleep markers differs among participants' and that 'sleep trackers provide more information gain for some users than others'—rests entirely on these in-sample R^2 differences. The group sizes are small (5, 11, 13), and the group-specific coefficients in Table 7 have large standard errors; for example, Group 1's N-Back coefficient is 0.348 (SE 0.409) and Group 3's HR coefficient is 0.009 (SE 0.011). The authors themselves concede in Section 5.1 that 'there are indications that the currently identified number of 3 groups comes from data limitations.' If the group structure does not generalize, the differential information-gain conclusion is unsupported, and the contribution is reduced to the dataset plus weak overall correlations.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports an in-the-wild study of 29 university students who wore an Oura ring for 4–8 weeks, answered a daily relative sleep-quality self-assessment, and completed a 3-back working-memory task. The authors compute Spearman correlations between sleep features and self-assessment/N-back scores, then fit mixed-effects, ordinal, and Bayesian models to identify predictors of self-assessment and N-back performance. They further divide participants into subgroups via a Gaussian mixture model and claim that self-report sensitivity toward sleep markers differs across three identified groups. The main reported findings are that differences in REM sleep duration, nocturnal heart rate, bedtimes, and N-back scores predict sleep self-assessment, and that REM sleep and self-assessment predict N-back scores. The dataset is promised to be made publicly available.","tokens_in":23322,"tokens_out":2929,"duration_ms":29022,"significance":"If the findings hold, the paper would provide a useful longitudinal dataset and a modest confirmation that consumer wearables partially align with subjective sleep experience, with the additional observation that this alignment may be heterogeneous across users. The data-sharing commitment and the within-subject, in-the-wild design are strengths. However, the central novelty—the three-group differential-information-gain claim—rests on an in-sample model-selection procedure without out-of-sample validation, and the abstract's wording ('highly predict') substantially overstates the reported effect sizes (R²_marginal = 0.15 for self-assessment; rho = 0.08 for the N-back–self-report association). The subgroup analysis is exploratory, and the authors themselves concede in Section 5.1 that the number of three groups may be an artifact of data limitations. The paper's contribution is therefore best viewed as a descriptive, exploratory study rather than a confirmatory demonstration of distinct user archetypes; with appropriate reanalysis or tempering of claims, it could be a worthwhile addition to the HCI/sleep-tracking literature.","major_comments":[{"comment":"The number of GMM subgroups is selected by comparing in-sample R² values (0.157/0.177, 0.185/0.208, 0.163/0.190 for n=2, 3, 4) and choosing n=3 because it gives the highest R². These values come from models fit and evaluated on the same participants, with no cross-validation, bootstrap, or correction for model selection. Since the three-group claim is the paper's central novelty ('self-report sensitivity towards sleep markers differs among participants'), this conclusion is not supported as it stands. The group sizes are small (5, 11, 13), and several coefficients in Table 7 have large standard errors (e.g., Group 1 N-Back coefficient 0.348, SE 0.409). The authors' own admission in Section 5.1 that 'there are indications that the currently identified number of 3 groups comes from data limitations' underscores that this result is fragile. I recommend adding out-of-sample validation such as repeated K-fold cross-validation or bootstrap stability of group assignments, or explicitly reframing the subgroup analysis as exploratory and removing the three-group claim from the abstract.","section":"Section 4.2, Table 7"},{"comment":"The abstract claims that the identified features 'highly predict sleep self-assessment in significance and effect size,' but Table 3 reports R²_marginal = 0.15 and R²_conditional = 0.18, and Table 2 shows a Spearman correlation of only 0.08 between N-back score and self-assessment. These are weak-to-modest in-sample associations, which is also acknowledged in the body text ('weak correlations'). The language in the abstract and conclusion should be calibrated to the observed effect sizes, or the authors should provide out-of-sample predictive metrics (e.g., cross-validated R² or RMSE) to justify a stronger claim.","section":"Abstract and Section 4.2, Table 3"},{"comment":"The mixed-effects models are built by iteratively removing features to minimize VIF and AIC on the same dataset used to evaluate them, and the reported p-values and R² are not adjusted for this selection. This inflates the apparent significance and effect sizes. A notable discrepancy: Table 2 reports no significant raw correlation between REM sleep duration and N-back score (rho = 0.036, p = 0.162), yet Table 4 reports REM sleep duration and its difference as significant predictors with R²_marginal = 0.46. This pattern is consistent with selection overfitting. The authors should either prespecify their predictors, use a proper post-selection inference procedure, or clearly label these models as exploratory and report the number of candidate models considered.","section":"Section 4.2, Tables 3 and 4"}],"minor_comments":[{"comment":"Please clarify whether T_max and T_min in Eq. (1) are computed per participant, per day, or globally across the whole dataset; the interpretation of the composite N-back score depends on this choice.","section":"Section 3.3, Eq. (1)"},{"comment":"The Likert responses are described as '1) Much better ... 5) Much worse' but are later mapped symmetrically to +2x, +x, 0, −x, −2x. Please state explicitly where this mapping is applied and whether the arbitrary scale factor x affects any reported coefficients or R² values.","section":"Section 3.2 and Section 4.2"},{"comment":"The right-hand column header reads 'Pearson correlation of N-Back score,' but the text states that Spearman's rank correlation is used because of skewness. Please correct this inconsistency.","section":"Table 2"},{"comment":"The term 'Consecutive Day-Pairs' is used in Table 1 but is not formally defined until Section 3.5; moving the definition earlier would improve readability.","section":"Section 3.4, Table 1"},{"comment":"References [28] and [29] appear to be the same paper (Kainec et al., 2024) listed twice; please deduplicate.","section":"References"},{"comment":"The group color coding (blue, beige, red) may be difficult to distinguish for color-blind readers; consider adding hatching or direct labels.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The paper's own limitation statement in Section 5.1 concedes that the three-group count may reflect data limitations, yet the abstract presents the three groups as a solid finding. This internal concession supports a major-revision rather than outright rejection, because the issue is fixable either by stronger validation or by reframing the subgroup analysis as exploratory. The public data release and the careful within-subject design are genuine strengths. The main risk is that the paper's framing overstates both the predictive strength and the generality of the group structure; the authors should be encouraged to align the claims with the effect sizes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is an honest, carefully run in-the-wild study with a genuinely reusable dataset, but the abstract oversells the strength of the findings and the subgroup claim is the weakest link.\n\nWhat is actually new: the combination of daily relative sleep self-reports (compared to the previous night), Oura sleep-stage data, and N-back working memory scores in one study, with the data made public. The design choices are mostly good. Participants reported before seeing Oura data, the relative scale is a sensible way to avoid floor and ceiling effects, N-back learning days were removed, and the authors checked robustness by dropping the participants with most and least data. The correlations in Table 2 are Bonferroni-corrected, and the ordinal and Bayesian models broadly confirm the mixed-effects results. The limitations section is candid, covering self-bias, learning effects, gaps in participation, and the small sample.\n\nNow the soft spots, in proportion. First, the abstract says the predictors 'highly predict' sleep self-assessment. The actual numbers are marginal R-squared of 0.15 and rho of 0.08 for the N-back to self-report link. That is weak to modest, not high. Second, the 'three groups' finding, which is presented as a central novelty, rests on fitting a Gaussian mixture model and then choosing the number of groups by comparing in-sample R-squared values. There is no cross-validation, bootstrap, or correction for the model selection. The group sizes are 5, 11, and 13, and several group-specific coefficients have large standard errors. The authors themselves concede in Section 5.1 that the number 3 may come from data limitations. So the claim that sleep trackers provide different information gain for different user groups is plausible but not established by the analysis as it stands. Third, the main models select features on the same data via VIF and AIC, and the reported p-values do not account for that selection. Fourth, the Oura ring's REM estimates are treated as ground truth for within-person day-to-day changes; the cited validation studies are aggregate, and per-participant accuracy is not confirmed. Since REM is the strongest predictor in several models, this is a real measurement caveat, though not a fatal one.\n\nThe central qualitative finding, that perceived sleep quality aligns only weakly and variably with wearable data, is probably robust. The subgroup heterogeneity is the part that needs more work.\n\nRecommendation: send this to peer review. A serious referee should push for a softened abstract, out-of-sample validation or corrected inference for the subgroup count, and a more cautious framing of the information-gain conclusion. With those revisions, the dataset and the weak-correlation finding would be a solid contribution to HCI and personal informatics.","headline":"A useful public dataset and a mostly sound weak-correlation story, but the 'three subgroups' novelty is built on in-sample model selection and needs out-of-sample validation before it can carry the paper.","tokens_in":23871,"tokens_out":2120,"would_cite":true,"duration_ms":23725,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Day-to-day changes in REM sleep, overnight heart rate, bedtimes, and N-back scores are the strongest predictors of how people rate their sleep, while REM sleep and the self-rating together best predict working-memory performance.","keywords":["wearable sleep tracking","sleep self-assessment","REM sleep","working memory","N-back task","ecological momentary assessment","Oura ring","in-the-wild study"],"falsifier":"Run overnight polysomnography in parallel with the ring for a subset of these participants and check whether night-to-night changes in ring-derived REM duration match polysomnography-derived changes. If the ring's REM differences do not track the reference measurements, the paper's REM-based conclusions lose their foundation; alternatively, a pre-registered replication that swaps the order so the N-back test comes before the sleep-rating question would test whether the self-rating-to-N-back link survives.","tokens_in":22832,"feed_emoji":"😴","tokens_out":8260,"duration_ms":77262,"temperature":0.7,"pith_summary":"This paper asks whether a consumer sleep tracker tells users something they do not already feel about their sleep. Across 4–8 weeks, 29 participants rated each night's sleep against the previous night, wore an Oura ring, and took a daily 3-back working-memory test. The paper argues that day-to-day changes in REM (rapid eye movement) sleep duration, overnight heart rate, bedtimes, and N-back scores are the strongest predictors of how people rate their sleep, while REM sleep from the current and prior night plus the self-rating best predict working-memory performance. The associations are real but modest: the model explains $R^2 = 0.15$ of the variance in self-ratings, and the direct correlation between self-rating and N-back score is $\\rho = 0.08$. The study also finds that users split into three groups by which sleep signals their self-reports track, so the informational value of a sleep tracker is not the same for everyone.","feed_headline":"REM sleep and bedtimes predict self-rated sleep quality","feed_subtitle":"In 29 in-the-wild users, ring data explains 15% of sleep-quality ratings; REM also predicts working-memory scores.","key_machinery":"The central mechanism is the day-pair difference design: every sleep feature is converted into a change from the previous night, matched to a five-point comparative self-rating, so the outcome is a first derivative of sleep quality rather than a level. This is paired with reverse regression—modeling the subjective rating as the outcome rather than the sensor data—with participant as a random effect, and cross-checked by an ordinal cumulative link model and a Bayesian regression. The N-back score itself is a composite of accuracy and response time, $S = C + \\frac{T_{\\max}-T}{T_{\\max}-T_{\\min}}$, with correctness and time each scaled to $[0,1]$, analyzed only within participants because N-back is not treated as comparable between individuals. Gaussian mixture clustering on the sleep features then splits participants into three groups to test whether the same predictors hold for everyone.","core_discovery":"On the paper's own terms, the discovery is that subjective sleep quality is not a single internal feeling but a mixture of signals, and a wearable's sensor stream captures part of that mixture while missing the rest. Using a daily comparative question ('How was your sleep compared to yesterday?') avoids the floor and ceiling effects that plague absolute scales. Mixed-effects, ordinal, and Bayesian models converge on the same feature set: a night rated as better tends to be one with more REM sleep than the previous night, a lower average overnight heart rate, a later bedtime start and end, and a higher N-back score; together these account for $R^2 = 0.15$ of the variance in self-ratings (0.18 with participant random effects). For working memory, the strongest predictors are absolute REM duration, the change in REM duration from the prior night (negative coefficient), and the participant's self-rating, with marginal $R^2 = 0.46$. The authors conclude that sensor readings and self-reports carry partly non-overlapping information, and that grouping participants by their sleep-feature profiles reveals three subgroups with different sensitivities to these markers.","pith_inferences":["A natural next test the paper leaves implicit: whether the self-rating-to-N-back link is causal or an artifact of question order and expectation, since participants rated sleep before the test and the correlation is only $\\rho = 0.08$; a version with randomized feedback or reversed order would separate these.","The three subgroups' different sensitivities to bedtime start versus bedtime end hint at chronotype differences; a larger study with morning-eveningness questionnaires could make the grouping interpretable rather than purely data-driven.","If the ring's REM stage estimates are validated per-participant against polysomnography, the same modeling pipeline could be rerun on those validated days, which would tell whether the REM findings survive measurement error.","The public dataset invites direct replication with other wearables or clinical-grade devices, and the day-pair difference format could be reused as a standard template for comparative sleep self-reports."],"forward_implications":["If correct, sleep trackers offer genuine but partial information gain: self-ratings do not fully capture what sensors measure, and sensors do not fully capture what self-ratings capture.","REM sleep acts over at least two nights: both absolute REM duration and the change from the prior night enter the working-memory model, so single-night summaries miss part of the effect.","Because self-assessment itself predicts N-back performance beyond REM measures, asking users how they slept adds predictive information that the ring alone does not provide.","The three user groups imply that a one-size-fits-all wearable interface will over-serve some users and under-serve others; designs that identify which signals a user tracks could tailor feedback.","For HCI, translating sleep data into predicted cognitive readiness may be the actionable format that keeps users engaged with trackers."],"supporting_citations":[{"why":"This validates Oura nocturnal heart rate and heart rate variability against electrocardiography, supporting the physiological features used in the model.","marker":"[12]"},{"why":"This provides multi-night validation of the ring's sleep tracking in adolescents against actigraphy and polysomnography.","marker":"[13]"},{"why":"This validates the smart ring against medical-grade actigraphy in everyday settings, supporting the in-the-wild use in this study.","marker":"[38]"},{"why":"This validates the Oura Gen3 sleep staging algorithm against ambulatory polysomnography, which is the basis for trusting the REM estimates.","marker":"[65]"},{"why":"This supplies the N-back task settings used here and evidence linking sleep to working-memory performance.","marker":"[33]"},{"why":"This documents a mismatch between measured and perceived sleep quality with other devices, which this study revisits with the Oura ring.","marker":"[4]"},{"why":"This contributes a comparable in-the-wild Oura dataset, which this study extends with working-memory scores and comparative self-reports.","marker":"[1]"},{"why":"This provides the pseudo-R-squared method used to compare model fit across the identified user subgroups.","marker":"[43]"},{"why":"This provides the reverse-regression approach used to model self-assessments from sleep markers.","marker":"[81]"},{"why":"This relates N-back performance to sleep duration and efficiency, motivating the choice of working memory as the cognitive outcome.","marker":"[40]"}],"fun_headline_variants":["REM, bedtime, and heart rate predict sleep self-ratings","Ring data explains 15% of perceived sleep quality","Sleep perception: REM and bedtime matter most","Wearable sleep data predicts self-rated quality","Sensor data meets self-reported sleep quality"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Oura ring's nightly sleep-stage estimates, especially REM, are accurate enough to track real within-person day-to-day changes; if the ring's REM numbers are noisy or biased for some participants, the reported links between REM differences and both self-rated sleep quality and working-memory scores would be distorted.","fun_headline_variants_meta":{"raw":{"variants":["REM, bedtime, and heart rate predict sleep self-ratings","Ring data explains 15% of perceived sleep quality","Sleep perception: REM and bedtime matter most","Wearable sleep data predicts self-rated quality","Sensor data meets self-reported sleep quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1459,"prompt_tokens":937,"completion_tokens":522,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":449}},"tokens_in":553,"tokens_out":522,"duration_ms":5994,"temperature":1.0,"reasoning_tokens":449,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:07:12.590520+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run overnight polysomnography in parallel with the ring for a subset of these participants and check whether night-to-night changes in ring-derived REM duration match polysomnography-derived changes. If the ring's REM differences do not track the reference measurements, the paper's REM-based conclusions lose their foundation; alternatively, a pre-registered replication that swaps the order so the N-back test comes before the sleep-rating question would test whether the self-rating-to-N-back link survives.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This provides multi-night validation of the ring's sleep tracking in adolescents against actigraphy and polysomnography."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This validates the smart ring against medical-grade actigraphy in everyday settings, supporting the in-the-wild use in this study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This validates the Oura Gen3 sleep staging algorithm against ambulatory polysomnography, which is the basis for trusting the REM estimates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This supplies the N-back task settings used here and evidence linking sleep to working-memory performance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This documents a mismatch between measured and perceived sleep quality with other devices, which this study revisits with the Oura ring."},{"cited_title":"BiHeartS: Bilateral Heart Rate from multiple devices and body positions for Sleep measurement Dataset","cited_arxiv_id":"2308.06811","evidence_quote":"This contributes a comparable in-the-wild Oura dataset, which this study extends with working-memory scores and comparative self-reports."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This provides the pseudo-R-squared method used to compare model fit across the identified user subgroups."},{"cited_title":"van Dijk, Willem van Rhenen, Jaap M","cited_arxiv_id":null,"evidence_quote":"This provides the reverse-regression approach used to model self-assessments from sleep markers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This relates N-back performance to sleep duration and efficiency, motivating the choice of working memory as the cognitive outcome."}],"review_version":1}