REVIEW 4 major objections 3 minor 23 references
Identification of Patterns of Cognitive Impairment for Early Detection of Dementia
T0 review · 4 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Cognitive test scores can reveal a person's dementia route before symptoms appear.
desk verdict Exploratory clustering, but the early-detection claim is not supported: the subtype labels are post-hoc, the counts don't add up, and there's no external validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is a two-step discovery pipeline. Step one is ensemble wrapper feature selection: seven classifiers each pick the cognitive-test attributes that best separate normal from mildly impaired people, and the union of their choices (33 of 68 attributes) is kept so no single classifier biases the result. Step two is t-SNE, a dimensionality-reduction technique that places the high-dimensional test vectors on a two-dimensional map while preserving local proximity, followed by morphological reconstruction—an image-segmentation routine that grows regions outward from one manually chosen seed pixel per visible cluster. The t-SNE projection is what turns "people with similar test profiles" into "nearby points"; the segmentation is what turns the map into clusters; and the statistical comparison of normal versus impaired test scores within each cluster is what assigns each cluster a cognitive-domain profile.
What would settle it
Re-run the same 24,000-person pipeline with several t-SNE perplexity settings (for example 5, 30, and 100) and with automated cluster seeding instead of hand-picked pixels; if the three clusters do not reappear with largely the same memberships, the subtype assignment is a projection artifact rather than a property of the test scores.
Extended reading notes
Core claim
The paper's central claim is that patterns of cognitive impairment can be learned from a mixed population of normal and mildly impaired people before any subtype labels are assigned, and that these patterns correspond to clinically accepted variants of mild cognitive impairment. The procedure first uses an ensemble of seven classifiers to select 33 test features that best separate normal from mildly impaired people, then projects those features to two dimensions with t-SNE, and finally extracts clusters from the map with a region-growing morphological segmentation. Within each cluster the distributions of normal and impaired people are similar, and the features that differ most between them define the cluster's profile: one cluster spans all cognitive domains and is labeled amnestic multi-domain mild cognitive impairment, while the other two show concentration in attention, executive, and visuospatial domains and are labeled non-amnestic multi-domain. Because each cluster contains both normal and impaired people, the authors conclude that a person currently diagnosed as normal can be matched to the closest cluster and thereby to the most likely future route of cognitive decline.
Load-bearing premise
The scheme's entire subtype assignment depends on the assumption that the clusters visible on the two-dimensional t-SNE map correspond to real, distinct patterns of cognitive decline and are not artifacts of the projection or of where the seed pixels were manually placed.
Editorial extensions
If this is right
- A person who is currently classified as normal can be assigned to the cognitive subtype whose trajectory they are most likely to follow.
- Each cluster supports a shorter, personalized battery of tests, which lowers cost and fatigue in repeated check-ups.
- Because normal and impaired people within a cluster look alike on most features, the scheme treats decline as a graded continuum, not a binary switch.
- The three cluster profiles give clinicians concrete domains to monitor for each person during follow-up.
Reading between the lines
- The paper leaves untested whether the three clusters are stable under the t-SNE settings or the choice of seed pixels; re-running the procedure with automated cluster-seeding and multiple perplexities would show whether the subgroups are a property of the data or of the visualization.
- A longitudinal extension the paper does not report would be to follow members of each cluster over time and check whether those at the normal end convert to the mild-cognitive-impairment subtype the cluster predicts.
- The feature-selection stage was built to separate normal from impaired people, so the clusters are shaped by diagnostic status; an alternative would be to cluster the mildly impaired subgroup alone and ask whether the same three profiles appear, which would test whether the subtypes exist independently of the healthy population.
- If the route is truly individual-specific, then repeated testing should show domain-specific rates of decline that differ across clusters; this could be checked against existing longitudinal cohorts without new data collection.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a two-step pipeline for identifying patterns of cognitive impairment from baseline NACC data on 24,000 subjects (12,000 cognitively normal and 12,000 with mild cognitive impairment). First, an ensemble wrapper feature-selection procedure using seven classifiers selects 33 features that discriminate CN from MCI. Second, t-SNE is applied to these features, and three clusters are extracted from the 2-D embedding via morphological reconstruction with manually chosen marker pixels. The authors characterize each cluster by computing Mann-Whitney U tests and Cohen's d effect sizes between CN and MCI subjects within each cluster, leading to labels such as amnestic multi-domain MCI (C1) and non-amnestic multi-domain MCI (C2 and C3). The paper claims these clusters correspond to clinically accepted MCI subtypes and that normal subjects can be assigned to a likely route of cognitive impairment, enabling personalized short test batteries for periodic follow-up.
Significance. The clinical motivation is strong: early detection of dementia and personalized test batteries could reduce patient burden and improve screening. The use of a large, well-known dataset (NACC) is a positive feature, and the idea of combining feature selection with clustering to assign pre-symptomatic individuals to a trajectory is, in principle, valuable. However, the current manuscript does not provide the evidence needed to support its central claims. The cluster structure is derived from a subjective, under-specified t-SNE/marker procedure, the subtype labels are based on the same features used to build the clusters, and no external validation or longitudinal follow-up is presented. The paper also contains a numerical inconsistency in the reported cluster sizes. If these issues were addressed with independent validation, the approach could be of interest, but as it stands the central claims are unsupported.
major comments (4)
- [§II-C, Fig. 1] The three clusters are extracted from a t-SNE projection, but no t-SNE hyperparameters (perplexity, learning rate, number of iterations, initialization) are reported, and the marker image consists of 'a single pixel chosen in the interior of each cluster' by visual inspection. This procedure cannot distinguish genuine population structure from t-SNE artifacts, and it presupposes the number of clusters rather than discovering it. The authors must report the t-SNE parameters, perform a sensitivity analysis, and ideally replace the manual marker-based segmentation with an objective clustering algorithm (with model selection) applied in the original feature space.
- [Table II] The reported cluster totals are C1 = 1,895, C2 = 7,112, and C3 = 8,629, which sum to 17,636, whereas the study population is stated as 24,000 subjects in the abstract and Section II-A. The discrepancy of 6,364 unassigned subjects is not explained. Either a substantial subset of the data was excluded from the clustering, or the counts are erroneous; in either case, the cluster characterization cannot be taken at face value.
- [Section III, Tables I and III] The subtype labels (amnestic multi-domain, non-amnestic multi-domain) are assigned based on Cohen's d effect sizes computed on the same features that were selected using CN-vs-MCI labels and on the same clusters being labeled. This is circular: the features and clusters are not independent of the labeling process. There is no external reference, such as clinical MCI subtype diagnoses, independent neuropsychological criteria, or neuropathological endpoints, to support the claim that 'these patterns have been shown to correspond to clinically accepted variants of MCI.' A direct comparison to an independent set of subtype labels or to longitudinal conversion outcomes is required.
- [Section III, Conclusions] The central claim that a normal person in a cluster can be assigned to the 'most likely route of cognitive impairment' is not supported by the data presented. The study is cross-sectional and uses only baseline assessments; no longitudinal follow-up of the normal participants is provided to show that they subsequently develop the predicted MCI subtype. The inference that normals in a cluster share the same trajectory as MCIs in that cluster is an unstated assumption, not a demonstrated result.
minor comments (3)
- [Section III] There is a typo in the phrase 'This plot has been gnerated' — it should be 'generated.'
- [Section II-B] The phrase 'has been shown to be be unbiased' contains a duplicated word 'be'; it should be 'has been shown to be unbiased.'
- [Section III] The manuscript states that a normal subject can be 'associated with the closest MCI sub-type,' but the distance metric and the space in which this association is computed (t-SNE embedding or original feature space) are not defined. The method for assigning normals to clusters should be specified explicitly.
Circularity Check
No circular derivation; the predictive claim is an unvalidated extrapolation rather than a fitted input renamed as prediction.
full rationale
The derivation chain is not circular. Features are selected with an ensemble wrapper using CN-versus-MCI labels, but the clusters are then obtained by unsupervised t-SNE projection followed by morphological reconstruction on the combined CN+MCI sample. The subtype labels (amnestic multi-domain, non-amnestic multi-domain) are post-hoc interpretations based on Cohen's d effect sizes computed within each cluster; they are not pre-specified outcomes used to fit the clusters. The abstract's claim that learned clusters can identify a likely route for normal people is a conceptual extrapolation (nearest-cluster assignment) and is not validated with longitudinal follow-up or held-out data, but this is a validation gap, not a circular reduction. No equation is shown where an output equals an input by construction, and no fitted parameter is relabeled as a prediction. The paper contains no load-bearing self-citations; reference [3] is not authored by the present authors. The Table II sum inconsistency (17,636 vs. 24,000) and the manual marker-pixel selection for morphological reconstruction are methodological or reporting concerns, but they do not make the derivation self-referential. Overall, the central clustering result has independent empirical content, even though its predictive utility for pre-symptomatic individuals remains unsupported.
Assumptions & free parameters
free parameters (4)
- t-SNE perplexity =
not reported
- t-SNE learning rate and number of iterations =
not reported
- Manual marker pixel for morphological reconstruction =
not reported
- Number of clusters =
3
assumptions (4)
- domain assumption t-SNE preserves the cluster structure relevant to cognitive impairment
- ad hoc to paper Features selected by CN-vs-MCI classification are appropriate for revealing MCI subtypes
- ad hoc to paper Normals in a cluster follow the same impairment route as MCIs in that cluster
- domain assumption NACC cognitive test scores are complete or missing at random
Cite this review
Pith. "Pith review of Identification of Patterns of Cognitive Impairment for Early Detection of Dementia." pith.science (2026). https://pith.science/paper/5MTIDS4R
@misc{pith2026250523109,
author = {Pith},
title = {Pith review of: Identification of Patterns of Cognitive Impairment for Early Detection of Dementia},
year = {2026},
howpublished = {\url{https://pith.science/paper/5MTIDS4R}},
note = {Machine review of arXiv:2505.23109}
}
read the original abstract
Early detection of dementia is crucial to devise effective interventions. Comprehensive cognitive tests, while being the most accurate means of diagnosis, are long and tedious, thus limiting their applicability to a large population, especially when periodic assessments are needed. The problem is compounded by the fact that people have differing patterns of cognitive impairment as they progress to different forms of dementia. This paper presents a novel scheme by which individual-specific patterns of impairment can be identified and used to devise personalized tests for periodic follow-up. Patterns of cognitive impairment are initially learned from a population cluster of combined normals and MCIs, using a set of standardized cognitive tests. Impairment patterns in the population are identified using a 2-step procedure involving an ensemble wrapper feature selection followed by cluster identification and analysis. These patterns have been shown to correspond to clinically accepted variants of MCI, a prodrome of dementia. The learned clusters of patterns can subsequently be used to identify the most likely route of cognitive impairment, even for pre-symptomatic and apparently normal people. Baseline data of 24,000 subjects from the NACC database was used for the study.
Figures
Reference graph
Works this paper leans on
-
[1]
Mild cognitive impairment can be detected by multiple assessments in a single day,
D. Darby et al., “Mild cognitive impairment can be detected by multiple assessments in a single day,” Neurology, vol. 59, no. 7, pp. 1042–1046, 2002
work page 2002
-
[2]
Mild cognitive impairment as a diagnostic entity,
R. C. Petersen, “Mild cognitive impairment as a diagnostic entity,” J. Intern. Med., vol. 256, no. 3, pp. 183–194, Sep. 2004
work page 2004
-
[3]
Mild cognitive decline: Concept, types, presentation, and management,
A. A. Subramanyam et al. , “Mild cognitive decline: Concept, types, presentation, and management,” J. Geriatr. Ment. Health , vol. 3, no. 1, p. 10, 2016
work page 2016
-
[4]
Predicting MCI status from multimodal language data using cascaded classifiers,
K. C. Fraser et al. , “Predicting MCI status from multimodal language data using cascaded classifiers,” Front. Aging Neurosci., vol. 11, p. 205, 2019
work page 2019
-
[5]
M. J. Kang et al., “Prediction of cognitive impairment via deep learning trained with multi-center neuropsychological test data,” BMC Medical Informatics and Decision Making , vol. 19, no. 1, p. 231, Nov. 2019
work page 2019
-
[6]
E. C. Edmonds et al., “Susceptibility of the conventional criteria for mild cognitive impairment to false-positive diagnostic errors.” Alzheimer’s & Dementia, vol. 11, pp. 415–24, Apr 2015
work page 2015
-
[7]
Are empirically-derived subtypes of mild cognitive impairment consistent with conventional subtypes?
L. R. Clark et al., “Are empirically-derived subtypes of mild cognitive impairment consistent with conventional subtypes?” Journal of the International Neuropsychological Society , vol. 19, no. 6, pp. 635–645, 2013
work page 2013
-
[8]
L. Delano-Wood et al. , “Heterogeneity in mild cognitive impairment: Differences in neuropsychological profile and associated white matter lesion pathology,” Journal of the International Neuropsychological So- ciety, vol. 15, no. 6, pp. 906–914, 2009
work page 2009
Show all 23 references
-
[9]
The heterogeneity of mild cognitive impairment: A neuropsychological analysis
D. J. Libon et al., “The heterogeneity of mild cognitive impairment: A neuropsychological analysis.” J. Int. Neuropsychol., vol. 16, pp. 84–93, Jan 2010
2010
-
[10]
Latent classes of mild cognitive impairment are associated with clinical outcomes and neuropathology: Analysis of data from the national alzheimer’s coordinating center,
J. J. Hanfelt et al. , “Latent classes of mild cognitive impairment are associated with clinical outcomes and neuropathology: Analysis of data from the national alzheimer’s coordinating center,” Neurobiology of Disease, vol. 117, pp. 62–71, Sep. 2018
2018
-
[11]
Statistically derived subtypes and associations with cerebrospinal fluid and genetic biomarkers in mild cognitive impairment: A latent profile analysis
J. S. Eppig et al., “Statistically derived subtypes and associations with cerebrospinal fluid and genetic biomarkers in mild cognitive impairment: A latent profile analysis.” J. Int. Neuropsychol. Soc. , vol. 23, pp. 564– 576, Aug 2017
2017
-
[12]
B. McGuinness et al., “Predicting conversion to dementia in a memory clinic: A standard clinical approach compared with an empirically defined clustering method (latent profile analysis) for mild cognitive impairment subtyping,” Alzheimer’s & Dementia, vol. 1, no. 4, pp. 447– ...
2015
-
[13]
Modeling the heterogeneity in risk of progression to alzheimer’s disease across cognitive profiles in mild cognitive impair- ment,
C. Tatsuoka et al., “Modeling the heterogeneity in risk of progression to alzheimer’s disease across cognitive profiles in mild cognitive impair- ment,” Alzheimer’s Res. Ther., vol. 5, no. 2, pp. 14–14, Mar. 2013
2013
-
[14]
The National Alzheimer’s Coordinating Center (NACC) database: The uniform data set,
D. L. Beekly et al. , “The National Alzheimer’s Coordinating Center (NACC) database: The uniform data set,” Alzheimer. Dis. Assoc. Disord., vol. 21, no. 3, pp. 249–258, 2007
2007
-
[15]
The Alzheimer’s Disease Centers’ Uniform Data Set (UDS): The neuropsychologic test battery,
S. Weintraub et al. , “The Alzheimer’s Disease Centers’ Uniform Data Set (UDS): The neuropsychologic test battery,” Alzheimer Dis. Assoc. Disord., vol. 23, no. 2, pp. 91–101, 2009
2009
-
[16]
Version 3 of the Alzheimer Disease Centers’ Neuropsychological Test Battery in the Uniform Data Set,
——, “Version 3 of the Alzheimer Disease Centers’ Neuropsychological Test Battery in the Uniform Data Set,” Alzheimer. Dis. Assoc. Disord. , vol. 32, no. 1, pp. 10–17, 2018
2018
-
[17]
A review of feature selection techniques in bioinfor- matics,
Y . Saeys et al. , “A review of feature selection techniques in bioinfor- matics,” Bioinformatics, vol. 23, no. 19, pp. 2507–2517, Aug. 2007
2007
-
[18]
Interpreting distortions in dimensionality reduction by superimposing neighbourhood graphs,
B. Colange et al., “Interpreting distortions in dimensionality reduction by superimposing neighbourhood graphs,” 2019 IEEE Visualization Conf. (VIS), Oct 2019
2019
-
[19]
Reducing dimensionality in a database of sleep EEG arousals,
A. Diego et al., “Reducing dimensionality in a database of sleep EEG arousals,” Expert Syst. Appl. , vol. 38, no. 6, pp. 7746–7754, Jun. 2011
2011
-
[20]
Visualizing data using t-sne,
L. V . D. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of Machine Learning Research , vol. 9, no. Nov, pp. 2579–2605, 2008
2008
-
[21]
Morphological grayscale reconstruction: Definition, effi- cient algorithm and applications in image analysis,
L. Vincent, “Morphological grayscale reconstruction: Definition, effi- cient algorithm and applications in image analysis,” IEEE Conf. Com- puter Vision and Pattern Recognition , pp. 633–635, 1992
1992
-
[22]
D. J. Sheskin, Handbook of parametric and nonparametric statistical procedures. Chapman and Hall/CRC, 2003
2003
-
[23]
A power primer
J. Cohen, “A power primer.” Psychol. Bull., vol. 112, no. 1, p. 155, 1992
1992
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.