{"id":"798a6f46-2977-48c2-a430-21f79136c2c1","arxiv_id":"2502.09947","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A two-stage pipeline encodes daily home-activity text with MiniLM, clusters the embeddings, and applies PageRank to derive per-patient behavior vectors.","lead":"This paper turns daily movement data from dementia patients into compact behavior summaries by encoding activity text with a language model and ranking latent states with PageRank. It aims to help clinicians spot behavior patterns and personalize care, but the current evidence is preliminary.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Clinical validation of the deep-state vectors is internally inconsistent: Appendix A.7 reports p-values and effect sizes that contradict its own text, so the central claim that PageRank vectors identify clinically similar patients is unsupported.","rationale":"Appendix A.7 provides the only quantitative evidence that the PageRank vectors capture clinically meaningful deeper semantics. The central claim in Section 3 specifically points to this comparison: 'This revealed that the clinical differences between similar groups were indeed smaller in features like age, change in ADAS-Cog score.' Because the appendix's statistics are internally inconsistent, the claim has no valid empirical support, independent of whether t-SNE distances are meaningful. In particular, the text misidentifies which features are significant and interprets nonsignificant p-values as evidence of small differences, while the reported effect sizes for the most-similar group are large. This is not a matter of disagreement with a consensus; it is an internal correctness failure. The reanalysis proposed above would be feasible in principle if the 50 participant vectors and clinical scores were released, and it would settle whether the clinical signal survives proper testing. The reader's t-SNE concern is real but secondary: replacing t-SNE with a metric-preserving projection would be a valuable robustness check, but it would not repair the broken validation. The code and data are withheld per Appendix A.1, so the paper cannot currently be checked or reproduced, and no baseline comparison is provided. Under these conditions the REJECT verdict should stand.","tokens_in":10304,"tokens_out":6146,"duration_ms":59215,"concrete_test":"Reconstruct the 50 participant PageRank vectors from the published pipeline and recompute Appendix A.7: for each participant, find the 3 most and 3 least similar participants using the same distance metric on the 5-dimensional vectors; test each clinical feature (MMSE, ADAS-Cog, age, ΔMMSE, ΔADAS-Cog, HADS) with an appropriate paired or mixed-effect model that accounts for repeated use of the same participants, report exact p-values with Benjamini-Hochberg correction, and report Cohen's d as mean difference divided by pooled standard deviation. If the corrected analysis does not show that most-similar pairs have smaller clinical differences than least-similar pairs for age and ΔADAS-Cog, or if the effect sizes remain large, the central claim is unsupported. The same recomputation should be run on a held-out subset of participants.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central clinical claim rests entirely on Appendix A.7, and that appendix fails on its own terms. The text states that in the least-similar comparison the significant features are HADS-Anxiety, Age, and change in ADAS-Cog, but Table 3 shows significance only for HADS-Anxiety (p=0.0069), Age (p=4.59e-05), and ΔMMSE (p=0.0224); ΔADAS-Cog has p=0.4426. More importantly, the claimed finding that clinical differences between similar groups were indeed smaller is contradicted by Table 4: for the most similar patients, effect sizes are not small (ADAS-Cog d=2.34, HADS-Anxiety d=2.72, HADS-Depression d=1.25). No test statistic, sample-size calculation, or multiple-comparison correction is reported, so nonsignificant p-values cannot be read as evidence of similarity. Finally, the five latent states, 30-day window, and t-SNE/cluster parameters were selected on the same test set via silhouette scores, making the supporting experiment circular. Even if t-SNE distances were meaningful, this validation does not establish the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage self-supervised learning framework for analyzing daily movement data from 134 people living with dementia. In the first stage, raw sensor data are downsampled to 20-minute intervals, converted into text strings, and encoded with a fine-tuned all-MiniLM-L12-v2 language model to produce 384-dimensional embeddings. The second stage projects these embeddings with t-SNE, clusters them with K-means to identify latent states, constructs a transition matrix between clusters, and applies PageRank to obtain a per-participant low-dimensional vector. The authors use these PageRank vectors to identify the three most and least similar participants to each participant and compare clinical characteristics (MMSE, ADAS-Cog, HADS, age, etc.) between the groups. The central claim is that the PageRank vectors quantitatively assess deeper latent-state semantics and that the resulting participant similarity reflects clinically meaningful differences.","tokens_in":10584,"tokens_out":5331,"duration_ms":49335,"significance":"If the method worked as claimed, it would offer a novel, interpretable approach to summarizing high-resolution home-sensor data and linking daily behavior patterns to clinical outcomes in dementia. The paper addresses a relevant problem, uses a substantial real-world dataset, and combines text-encoder representation learning with PageRank in a way that is interesting and potentially applicable. However, the current manuscript does not establish the central claim. The mathematical definition of the transition matrix in Eq. (1) is degenerate, the clinical validation in Appendix A.7 is internally inconsistent, and key model parameters are selected on the test set via silhouette scores without held-out validation. The paper's only quantitative support for its central claim is the flawed Appendix A.7, so the significance of the proposed approach remains unsubstantiated.","major_comments":[{"comment":"The transition matrix defined in Eq. (1) is mathematically degenerate: the numerator and denominator are identical sums with dummy indices renamed, so T_{ij} = 1 whenever at least one pair of points between clusters i and j is within the threshold, and is undefined (0/0) otherwise. Consequently, the transition matrix carries no information about relative transition frequencies, and the PageRank computation in Eq. (2) operates on a matrix that is effectively uniform or undefined. This invalidates the central methodological step of the paper, as the PageRank vectors are derived entirely from this matrix. The threshold parameter is also never reported, and no sensitivity analysis is provided.","section":"A.4.2, Eq. (1)"},{"comment":"The clinical validation is internally inconsistent. The text states that in the least-similar comparison the significant features are HADS-Anxiety, Age, and change in ADAS-Cog, but Table 3 reports p = 0.4426 for ΔADAS-Cog and p = 0.0224 for ΔMMSE. The effect sizes in Table 5 are implausible for n = 50 (e.g., Age d = -24.86, ΔMMSE d = -23.02), and Table 4 shows large effect sizes for the most-similar group (ADAS-Cog d = 2.34, HADS-Anxiety d = 2.72), directly contradicting the claim that clinical differences between similar groups are smaller. No test statistic, sample-size calculation, or multiple-comparison correction is reported, so nonsignificant p-values cannot be interpreted as evidence of similarity. Since Appendix A.7 is the only quantitative support for the central claim, that claim is unsupported.","section":"Appendix A.7, Tables 2-5"},{"comment":"The number of daily latent states (K = 5), the 30-day positive-sampling window for triplet selection, the t-SNE hyperparameters, and the patient-level cluster count (K = 6) are all selected using silhouette scores computed on the same test set that is later used for the similarity analysis and clinical comparison. This circular selection procedure means the reported latent-state structure and the PageRank vectors are partly artifacts of fitting choices made on the test data. The manuscript provides no held-out validation, no baseline comparison (e.g., against random embeddings or non-embedding features), and no sensitivity analysis showing that the conclusions are robust to the chosen parameters.","section":"Section 3 and Appendices A.3, A.5, A.8"},{"comment":"The statement that from the deep vector 'we can easily identify the disease type, age, MMSE, ADAS-Cog scores ... for the three patients most and least similar to any given participant' is an overclaim. The reported analysis only compares group means between most/least similar participants; it does not perform any predictive task, does not report cross-validated performance, and does not provide confidence intervals for the similarity-based assignments. The paper's own characterization as 'preliminary results' and 'initial results' is inconsistent with the strength of this claim.","section":"Section 3"}],"minor_comments":[{"comment":"The silhouette scores for the 30-day window are identical (0.554) for K = 4, 5, and 6, so the choice of K = 5 as 'optimal' is not uniquely justified by the reported table.","section":"Table 1"},{"comment":"The procedure for selecting the 50-participant test set from the 134 available participants is not described in terms of inclusion/exclusion criteria; potential selection bias and its effect on generalizability should be discussed.","section":"Appendix A.2"},{"comment":"The code availability statement says notebooks 'will be released after review,' which is not sufficient for reproducibility, and the Sentence-Transformers library is incorrectly attributed to Pedregosa et al. (2011) rather than Reimers and Gurevych.","section":"Appendix A.1"},{"comment":"There are several typos and stylistic inconsistencies, including 'Viualizing data using t-SNE' in the van der Maaten and Hinton reference, 'pariticipants' in the Appendix A.7 heading, inconsistent capitalization of 'Adas-Cog', and the abstract's 'bi-dimensionalized' phrasing.","section":"Throughout"},{"comment":"The fine-tuning procedure is underspecified: the number of epochs, convergence criteria, and which layers of the language model are updated are not stated.","section":"Appendix A.3"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early preprint that is not ready for journal publication. The mathematical error in Eq. (1) and the internally inconsistent statistics in Appendix A.7 are load-bearing and cannot be fixed with minor revisions. The silouette-based parameter selection on the test set is a further circularity that would require a fundamentally different evaluation design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take on arXiv:2502.09947. The paper extends Capstick et al. (2024) by adding a PageRank stage over t-SNE embeddings and a per-participant similarity analysis. The idea is sound: summarize daily movement data into a compact five-dimensional vector and use it to compare patients. The dataset is real and substantial: 134 dementia patients, ~66k patient-days, with cognitive scores. The fine-tuning approach using triplet loss and cluster-based sample selection is reasonable.\n\nWhat the paper does well: it is clearly written, states assumptions, and discusses limitations (e.g., data sensitivity, code release after review). The ablation in Table 1 comparing window sizes and models is a nice touch.\n\nThe soft spots are serious. The clinical validation in Appendix A.7 is internally inconsistent. The text says that in the least-similar comparison, HADS-Anxiety, Age, and change in ADAS-Cog were significant, but Table 3 shows ΔADAS-Cog with p=0.44 (not significant) and instead ΔMMSE with p=0.022. More importantly, Table 4 shows effect sizes for most-similar patients are large (e.g., ADAS-Cog d=2.34, HADS-Anxiety d=2.72), which directly contradicts the claim that 'clinical differences between similar groups were indeed smaller.' There is no multiple-comparison correction, no sample-size calculation, and no test statistic for the effect sizes. The hyperparameters (cluster counts, 30-day window, t-SNE settings, PageRank threshold) are selected on the same test set via silhouette scores, making the downstream analysis circular. No baseline comparison is reported. The transition matrix in Eq. (1) relies on Euclidean distances between t-SNE points, which are non-metric and seed-dependent; the threshold is unreported. Finally, code and data are not available, so nothing can be independently reproduced.\n\nThese are not minor quibbles; they cut to the central claim. The paper as it stands does not support the conclusion that PageRank vectors capture clinically relevant movement dynamics. That said, the approach is plausible and the authors are honest about the preliminary nature. A revision with corrected statistics, proper held-out validation, and released code could change my mind.\n\nMy recommendation: I would not send this to peer review in its current form. The statistical inconsistencies are too serious, and the circular validation is a red flag. The authors need to fix the analysis first. If a colleague works on remote health monitoring, it's worth a skim for the method, but I wouldn't cite it.\n\nBest.","headline":"A promising pipeline undermined by invalid clinical validation: the paper's own tables contradict its central claim.","tokens_in":11106,"tokens_out":3819,"would_cite":false,"duration_ms":38354,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a five-component PageRank vector computed from text-encoded daily home activity carries clinically relevant dementia signatures, so similar vectors mark clinically similar patients.","keywords":["dementia","home activity monitoring","time series representation learning","language model encoding","PageRank","t-SNE","latent state analysis","personalized care"],"falsifier":"Recompute each participant's five-dimensional PageRank vector from the same text embeddings but with a different t-SNE random seed, or build the transition graph directly from the original 384-dimensional embeddings, and check whether the most-similar-patient clinical differences remain smaller than the least-similar differences. If the association vanishes under either change, the claimed clinical signal is an artifact of the projection geometry.","tokens_in":10107,"feed_emoji":"🧠","tokens_out":6889,"duration_ms":70821,"temperature":0.7,"pith_summary":"Daily activity records from passive home sensors are noisy, high-dimensional, and hard to compare across patients. This paper claims that a two-stage encoding—turning each day's movement into text, embedding that text with a fine-tuned language model, then projecting and clustering the embeddings and ranking the resulting latent states with PageRank—compresses each participant into a five-number vector that still carries clinically meaningful information. On a test set of fifty people with dementia, participants with the most similar vectors were closer in age and in year-over-year change of cognitive-test scores than participants with the least similar vectors. If the claim holds, a purely behavioral signature from sensor data could help identify comparable cases and support personalized dementia care.","feed_headline":"Home sensor data becomes a five-number dementia signature","feed_subtitle":"If correct, passive in-home sensors could rank dementia patients by behavioral similarity and support personalized care.","key_machinery":"The load-bearing mechanism is a two-stage encoding chain: daily sensor records are downsampled to 20-minute intervals and written as text strings; a fine-tuned all-MiniLM-L12-v2 sentence transformer embeds each day into a 384-dimensional vector; t-SNE projects those vectors to 2D; k-means identifies five latent states; and a transition matrix is built by counting, for each pair of clusters, how many points lie within a Euclidean distance threshold. PageRank is then run on that transition matrix, and the stationary distribution becomes the participant's five-component deep vector. The transition matrix of Eq. (1) is where raw movement dynamics are converted into a Markov-chain structure, and the PageRank vector is what makes different participants comparable.","core_discovery":"The paper's central claim is that the five-dimensional PageRank vector derived from a participant's daily home-activity trajectories is a deep behavioral signature, not just a compression artifact. The authors argue that after language-model encoding and t-SNE projection, the cluster-to-cluster transition graph of a participant's movements can be ranked by PageRank, and the resulting $(1,5)$ vector lets them retrieve the three most and least similar patients for any given participant; the retrieved groups differ less in clinical traits such as age and rate of ADAS-Cog change. They further report that clustering these vectors across fifty participants yields groups that separate along MMSE and ADAS-Cog scores, with the expected inverse relationship between the two cognitive measures. In the paper's framing, the method allows quantitative assessment of the deeper semantics represented by the vector clusters, or latent states.","pith_inferences":["The paper does not test whether the result survives a change of projection; recomputing the transition graph directly from the original 384-dimensional embeddings would separate a real behavioral signal from a t-SNE artifact.","The unreported distance threshold and the seed-dependence of t-SNE mean the five-number vector's run-to-run stability is unknown, and measuring that variance would tell whether the similarity ranking is reproducible.","Because most clinical-feature comparisons did not reach statistical significance, the practical promise depends on effect size and replication in a larger cohort rather than on the current p-values.","The five latent states could be given behavioral names, such as 'highly mobile', 'room-bound', or 'nocturnal wandering', by linking cluster centroids back to the original room sequences, giving clinicians directly interpretable state semantics."],"forward_implications":["The five-component PageRank vector gives a compact, comparable summary of a patient's movement dynamics, so clinicians could rank patients by behavioral similarity without reviewing raw sensor streams.","If the clinical similarity result survives validation, passive home sensing could function as a remote, low-burden source of information about dementia progression.","Clustering these vectors by cognitive scores suggests that activity-state dynamics reflect cognitive state, which could yield early-warning signals for decline.","The same two-stage pipeline should transfer to other irregular, discrete time-series domains where text encoders and graph ranking can be applied.","The paper notes the pipeline could be extended to a generative model for producing sensitive medical datasets, supporting data augmentation and alignment."],"supporting_citations":[{"why":"Establishes the text-encoder representation of daily movement data that this paper extends with a second modelling stage.","marker":"Capstick et al. (2024)"},{"why":"Provides the PageRank algorithm and random-walk interpretation used to rank latent states.","marker":"Page et al. (1999)"},{"why":"Supplies the t-SNE method that projects embeddings into the 2D space where transitions are measured.","marker":"van der Maaten and Hinton (2008)"},{"why":"Introduces the all-MiniLM-L12-v2 sentence-transformer model used as the encoder.","marker":"Muennighoff et al. (2023)"},{"why":"Supplies the MMSE instrument whose scores anchor the clinical similarity comparisons.","marker":"Kurlowicz and Wallace (1999)"},{"why":"Supplies the ADAS-Cog instrument used for the cognitive-score change comparisons.","marker":"Kueper et al."}],"fun_headline_variants":["Home sensors distill daily movement into five-number dementia score","Five-number dementia signature emerges from home activity data","Self-supervised encoding plus PageRank yields five-number behavior vector","Patient movement becomes a five-number dementia fingerprint via PageRank","Two-stage model converts home activity to a five-dim dementia signature"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach depends on treating Euclidean distances between points in a two-dimensional t-SNE projection as meaningful movement transitions between latent states, even though t-SNE distances are non-metric and projection-dependent and the distance threshold used in the transition matrix is not reported.","fun_headline_variants_meta":{"raw":{"variants":["Home sensors distill daily movement into five-number dementia score","Five-number dementia signature emerges from home activity data","Self-supervised encoding plus PageRank yields five-number behavior vector","Patient movement becomes a five-number dementia fingerprint via PageRank","Two-stage model converts home activity to a five-dim dementia signature"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000702,"raw_usage":{"total_tokens":3102,"prompt_tokens":816,"completion_tokens":2286,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":2216}},"tokens_in":432,"tokens_out":2286,"duration_ms":17190,"temperature":1.0,"reasoning_tokens":2216,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T19:57:41.336837+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute each participant's five-dimensional PageRank vector from the same text embeddings but with a different t-SNE random seed, or build the transition graph directly from the original 384-dimensional embeddings, and check whether the most-similar-patient clinical differences remain smaller than the least-similar differences. If the association vanishes under either change, the claimed clinical signal is an artifact of the projection geometry.","supporting_citations":[{"cited_title":"Representation Learning of Daily Movement Data Using Text Encoders","cited_arxiv_id":"2405.04494","evidence_quote":"Establishes the text-encoder representation of daily movement data that this paper extends with a second modelling stage."},{"cited_title":"Motwani, and T","cited_arxiv_id":null,"evidence_quote":"Provides the PageRank algorithm and random-walk interpretation used to rank latent states."},{"cited_title":"Viualizing data using t- SNE","cited_arxiv_id":null,"evidence_quote":"Supplies the t-SNE method that projects embeddings into the 2D space where transitions are measured."}],"review_version":1}