REVIEW 3 major objections 4 minor 53 references
The influence of diversity on the measurement of functional impairment: An international validation of the Amsterdam IADL Questionnaire in 8 countries
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Across 3,571 people in eight Western countries, the Amsterdam IADL Questionnaire shows no indication of clinically relevant item bias for age, gender, education, or culture.
desk verdict Solid large-sample DIF validation of the A-IADL-Q-SV, but the 'no meaningful bias' conclusion rests on a cutoff just above all observed effects, and the paper lacks the sensitivity analysis to make that claim stick. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is differential item functioning (DIF) analysis on item-level responses, using ordinal logistic regression with nested model comparisons, alongside a McFadden pseudo-$R^2$ effect size cutoff of .035 (moderate .035–.070, large > .070) to separate statistical from practically meaningful bias. Monte Carlo simulations under a no-DIF null generated empirical effect-size thresholds, and the authors examined whether DIF materially changed IRT-based $T$-scores, which were calibrated to a mean of 50 with a standard deviation of 10 in a Dutch memory-clinic population.
What would settle it
Re-run the same ordinal logistic regression DIF analysis on the A-IADL-Q-SV in a sample that includes people with fewer than six years of formal education and in non-Western countries; if multiple items exceed $\Delta R^2 = .035$ and correcting for DIF shifts mean $T$-scores by more than 5 points (half a standard deviation), the no-bias conclusion would be contradicted.
Extended reading notes
Core claim
The paper's central claim is that clinically relevant item bias in the Amsterdam IADL Questionnaire is absent: no indication was found that age, gender, education, or national culture distorts the measurement of functional impairment in a way that would change conclusions drawn from the $T$-score. The evidence is that $\Delta R^2$ effect sizes never exceeded .034 across all items and comparisons in the short version, and the four items with meaningful DIF in the original version (Spanish 'using the washing machine', 'making appointments', 'playing card and board games'; French 'functioning adequately at work') shifted mean scores by no more than 0.16 points on the $T$-scale. The paper therefore argues that the A-IADL-Q-SV $T$-scores need no correction for these diversity factors and supports the instrument as an outcome measure for international dementia research and trials.
Load-bearing premise
The conclusion of no clinically relevant bias rests on a pre-set $\Delta R^2$ cutoff of .035 for meaningful differential item functioning and on trait estimates that come from a Dutch memory-clinic calibration; if that cutoff is too lenient or those trait estimates do not generalize, under-detection of item bias is possible.
Editorial extensions
If this is right
- The short version's $T$-scores can be compared across the eight countries and across age, gender, and education groups without DIF-based adjustment.
- The findings support using the A-IADL-Q as an outcome measure in multinational clinical trials where functional impairment is a key endpoint.
- The four country-flagged items in the original version have a negligible effect on total scores, so historical data from the long version remain usable.
- Correlations with cognitive and functional measures were similar to the original Dutch validation, reinforcing the instrument's construct validity in new settings.
- The cross-cultural adaptation process (forward and backward translation, expert review, cognitive interviews) likely contributed to the small number of biased items.
Reading between the lines
- The paper does not claim that the .035 threshold is a law of nature; its own simulations produced effect-size thresholds up to .018, so a stricter cutoff would flag more items. A user who cares about fine-grained score differences should check whether lowering the threshold in these data changes any clinical decision.
- The conclusion applies to Western countries with mostly well-educated participants; it does not extend to non-Western cultures, ethnic or racial groups, or people with very little formal education, where other IADL instruments have shown response bias.
- Because endorsement gaps were largest for technology-related activities (computers, ATMs) and household tasks, the item parameters could drift over time as technology adoption and gender roles change; a future replication using the same DIF procedure would reveal whether the no-bias result is stable across decades.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates whether the Amsterdam IADL Questionnaire (A-IADL-Q) and its short version (A-IADL-Q-SV) exhibit item bias due to country, age, gender, and education in a sample of 3,571 participants from eight Western countries. Differential item functioning (DIF) is assessed by ordinal logistic regression, using a McFadden ΔR2 threshold of .035 for practically meaningful DIF, supplemented by Monte Carlo simulations. The authors report SV effect sizes in the range .000–.034, below the threshold, while four items in the original version show moderate DIF for nationality with negligible impact on T-scores. They conclude that there is no indication of clinically relevant bias and that A-IADL-Q scores are comparable across these diversity factors.
Significance. If the conclusion holds, the study provides valuable evidence for the cross-cultural validity of a functional outcome measure used in dementia research, supporting the use of the A-IADL-Q-SV in international trials. The study has notable strengths: a large multi-country sample, a standard DIF methodology, the use of Monte Carlo simulations to estimate null thresholds, and explicit effect-size criteria. However, the central no-bias claim depends on a single cutoff, and the reported analysis does not establish robustness to alternative thresholds. The manuscript also contains an inconsistency in the reported number of DIF comparisons. These issues are addressable and should be resolved before the claim is accepted.
major comments (3)
- [Results 3.2 / Methods 2.2] The conclusion that the A-IADL-Q-SV has no practically meaningful item bias is operationalized entirely by the fixed McFadden ΔR2 ≥ .035 cutoff in Methods 2.2. In Results 3.2, the empirical SV ΔR2 values range from .000 to .034, so the main finding is that no item exceeded a threshold set only 0.001 above the largest observed effect. The Monte Carlo simulations produced 99th-percentile null thresholds up to .018, and the text states that lowering the threshold would lead to more items being flagged. The authors do not report how many items would be flagged, for which comparisons, or whether the cumulative effect of those items would shift T-scores by a clinically meaningful amount. Because the empirical distribution is packed against the chosen cutoff, the no-bias conclusion is not invariant to a defensible alternative criterion such as the simulation-based threshold. Please provide the number of items flagged using the simulation-based threshold and an impact analysis for the SV analogous to the original-version analysis in Section 3.2.
- [Results 3.2 / Table 1] The reported counts of analyzed item-factor combinations are inconsistent. For the SV, 272 out of 300 comparisons is consistent with 30 items × 10 comparisons (7 countries plus age, gender, and education). For the original version, 437 out of 490 comparisons is consistent with 70 items × 7 country comparisons only, yet the text in Section 3.2 reports effects for age, gender, and education for the original version ('The effects for age, gender and education were again small'). Please clarify which factors were analyzed for each version and provide the total number of comparisons per version so that the denominators can be verified.
- [Methods 2.1.1 / Results 3.2] The IRT-based trait estimates used for matching in the DIF analysis were calibrated in a Dutch memory-clinic population. DIF detection depends on this anchor metric; if the latent trait metric is not invariant across national samples or across the diagnostic spectrum included here, DIF could be under- or over-detected. The manuscript does not explicitly discuss this as a limitation of the cross-cultural no-bias claim. Please discuss whether the Dutch calibration could affect the generalizability of the conclusion, and if feasible, perform a sensitivity analysis with an alternative anchor metric, such as recalibration on the combined sample or fixed-item anchoring.
minor comments (4)
- [Abstract] The abstract reports an effect-size range of '0–0.03' for the SV, while Results 3.2 reports '.000–.034'; please align these values.
- [Discussion, paragraph 6] The sentence 'We used DIF, which is a powerful procedure to detect variance in measurement between groups on an item level and was possible as a result of the IRT scoring method' contains an ungrammatical phrase; 'and was possible' should read 'and which was possible.'
- [Results 3.1] The text states 'In Mediterranean countries, it seemed people used computers less often than in Northern European countries and America,' but Serbia, which is included in the analysis, is not typically considered Mediterranean; please rephrase to 'Southern and Eastern European countries' or list the specific countries.
- [Methods 2.2] The sentence 'OLR has previously been shown to be superior the Mantel Haenszel procedure' is missing the preposition 'to'; it should read 'superior to the Mantel-Haenszel procedure.'
Circularity Check
No significant circularity: the DIF conclusion is an empirical finding from multinational data with a literature-based cutoff, not a reduction of outputs to inputs.
full rationale
This is an empirical validation study. The central claim—that the A-IADL-Q shows no practically meaningful item bias across country, age, gender, and education—is estimated from observed multinational data using ordinal logistic regression implemented in the lordif package, not derived from a fitted parameter that already encodes the conclusion. The McFadden ΔR2 = .035 cutoff is adopted from published DIF effect-size recommendations (refs 43–45) and is not calibrated to the current dataset; the Monte Carlo simulations are used only to characterize null variability, and the authors explicitly concede in the Limitations that lowering the threshold would flag more items, which undercuts any suggestion that the threshold was chosen to force the result. The IRT T-scores calibrated in earlier Dutch work are an input measurement model, but the DIF analysis is exactly an independent check of that model's invariance across groups; no equation in the paper makes the empirical ΔR2 values equal the threshold or makes the T-score impact minimal by construction. Self-citations (refs 29–33) concern questionnaire development, validation, and the short-version rationale; they do not themselves supply the multinational DIF estimates or the conclusion of absent bias. The robustness concern about the cutoff is a statistical/correctness issue, not circularity. No exhibited reduction exists, so the appropriate finding is no significant circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption The A-IADL-Q measures a unidimensional latent trait (IADL functioning) and satisfies IRT assumptions of local independence and monotonicity.
- domain assumption Country of residence is a valid proxy for cultural differences.
- domain assumption The latent trait estimates from the Dutch-calibrated IRT model are applicable to all groups before DIF correction.
- domain assumption A McFadden pseudo-R2 change of .035 is the correct threshold for practically meaningful DIF.
Cite this review
Pith. "Pith review of The influence of diversity on the measurement of functional impairment: An international validation of the Amsterdam IADL Questionnaire in 8 countries." pith.science (2026). https://pith.science/paper/XLT7FGL3
@misc{pith2026190901915,
author = {Pith},
title = {Pith review of: The influence of diversity on the measurement of functional impairment: An international validation of the Amsterdam IADL Questionnaire in 8 countries},
year = {2026},
howpublished = {\url{https://pith.science/paper/XLT7FGL3}},
note = {Machine review of arXiv:1909.01915}
}
read the original abstract
INTRODUCTION: To understand the potential influence of diversity on the measurement of functional impairment in dementia, we aimed to investigate possible bias caused by age, gender, education, and cultural differences. METHODS: 3,571 individuals (67.1 {\pm} 9.5 years old, 44.7% female) from the Netherlands, Spain, France, United States, United Kingdom, Greece, Serbia and Finland were included. Functional impairment was measured using the Amsterdam IADL Questionnaire. Item bias was assessed using differential item functioning (DIF) analysis. RESULTS: There were some differences in activity endorsement. A few items showed statistically significant DIF. However, there was no evidence of meaningful item bias: effect sizes were low ({\Delta}R2 range 0-0.03). Impact on total scores was minimal. DISCUSSION: The results imply a limited bias for age, gender, education and culture in the measurement of functional impairment. This study provides an important step in recognizing the potential influence of diversity on primary outcomes in dementia research.
Reference graph
Works this paper leans on
-
[1]
American Psychiatric Association, Diagnostic and statistical manual of mental disorders: DSM-
-
[2]
Lawton, M.P. and E.M. Brody, Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist, 1969. 9(3): p. 179-86
work page 1969
-
[3]
Peres, K., et al., Natural history of decline in instrumental activities of daily living performance over the 10 years preceding the clinical diagnosis of dementia: a prospective population-based study. J Am Geriatr Soc, 2008. 56(1): p. 37-44
work page 2008
-
[4]
Giebel, C.M., C. Sutcliffe, and D. Challis, Activities of daily living and qu ality of life across different stages of dementia: a UK study. Aging Ment Health, 2015. 19(1): p. 63-71
work page 2015
-
[5]
2013, Arlington, VA: American Psychiatric Association
5th ed. 2013, Arlington, VA: American Psychiatric Association
work page 2013
-
[6]
Luck, T., et al., A hierarchy of predictors for dementia -free survival in old -age: results of the AgeCoDe study. Acta Psychiatr Scand, 2014. 129(1): p. 63-72
work page 2014
-
[7]
Weintraub, S., et al., Measuring cognition and function in the preclinical stage of Alzheimer's disease. Alzheimers Dement (N Y), 2018. 4: p. 64-75
work page 2018
-
[8]
2018, Silver Spring, MD: Center for Drug Evaluation and Research
Food and Drug Administration, Draft guidance for industry: Alzheimer's disease: developing drugs for the treatment of early stage disease . 2018, Silver Spring, MD: Center for Drug Evaluation and Research
work page 2018
Show all 53 references
-
[9]
Ethn Health, 2000
Truscott, D.J., Cross-cultural ranking of IADL skills. Ethn Health, 2000. 5(1): p. 67-78
2000
-
[10]
Aging Clin Exp Res, 2003
Nikula, S., et al., Are IADLs comparable acro ss countries? Sociodemographic associates of harmonized IADL measures. Aging Clin Exp Res, 2003. 15(6): p. 451-459
2003
-
[11]
Health Qual Life Outcomes, 2012
Vergara, I., et al., Validation of the Spanish version of the Lawton IADL Scale for its application in elderly people. Health Qual Life Outcomes, 2012. 10: p. 130
2012
-
[12]
Dement Geriatr Cogn Disord, 2008
Lechowski, L., et al., Patterns of loss of abilities in instrumental activities of daily living in Alzheimer's disease: the REAL cohort study. Dement Geriatr Cogn Disord, 2008. 25(1): p. 46 - 53
2008
-
[13]
Sikkes, S.A.M. and J. De Rotrou, A qualitative review of instrumental activities of daily living in dementia: what's cooking? Neurodegener Dis Manag, 2014. 4(5): p. 393-400
2014
-
[14]
Negash, and R
Chin, A.L., S. Negash, and R. Hamilton, Diversity and disparity in dementia: the impact of ethnoracial differences in Alzheimer disease. Alzheimer Dis Assoc Disord, 2011. 25(3): p. 187- 95
2011
-
[15]
Alzheimers Dement, 2018
Babulal, G.M., et al., Perspectives on ethnic and racial disparities in Alzheimer's disease and related dementias: Update and areas of immediate need. Alzheimers Dement, 2018
2018
-
[16]
Dement Neuropsychol, 2014
Parra, M.A., Overcoming barriers in cognitive assessment of Alzheimer's disease. Dement Neuropsychol, 2014. 8(2): p. 95-98
2014
-
[17]
Psychol Aging, 2009
Mungas, D., et al., Age and education effects on relationships of cognitive test scores with brain structure in demographically diverse older persons. Psychol Aging, 2009. 24(1): p. 116-28
2009
-
[18]
Santo, and F
Epstein, J., R.M. Santo, and F. Guillemin, A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. J Clin Epidemiol, 2015. 68(4): p. 435-41
2015
-
[19]
van der Flier, W.M. and P. Scheltens, Amsterdam Dementia Cohort: Performing Research to Optimize Care. J Alzheimers Dis, 2018. 62(3): p. 1091-1111
2018
-
[20]
Lancet Psychiatry,
Ritchie, C.W., et al., Development of interventions for the secondary prevention of Alzheimer's dementia: the European Prevention of Alzheimer's Dementia (EPAD) project. Lancet Psychiatry,
-
[21]
BMJ Open, 2018
Solomon, A., et al., European Prevention of Alzheimer's Dementia Longitudinal Cohort Study (EPAD LCS): study protocol. BMJ Open, 2018. 8(12)
2018
-
[22]
Curr Alzheimer Res, 2018
Facal, D., et al., Assessing everyday activities across the dementia spectrum with the Amsterdam IADL Questionnaire. Curr Alzheimer Res, 2018. 15(13): p. 1261-1266
2018
-
[23]
J Nutr Health Aging, 2016
Facal, D., et al., Characterizing Magnitude and Selectivity of Attrition in a Study of Mild Cognitive Impairment. J Nutr Health Aging, 2016. 20(7): p. 722-8. Page 24 of 25
2016
-
[24]
Alzheimers Dement (N Y), 2016
Molinuevo, J.L., et al., The ALFA project: A research platform to identify early pathophysiological features of Alzheimer's disease. Alzheimers Dement (N Y), 2016. 2(2): p. 82- 92
2016
-
[25]
Lancet Neurol, 2018
Dubois, B., et al., Cognitive and neuroimaging features and brain beta -amyloidosis in individuals at risk of Alzheimer's disease (INSIGHT-preAD): a longitudinal observational study. Lancet Neurol, 2018. 17(4): p. 335-346
2018
-
[26]
Alzheimers Dement, 2017
Lee, A., et al., The Butler Alzheimer's Prevention Registry: recruitment and interim outcome. Alzheimers Dement, 2017. 13(7): p. 622-623
2017
-
[27]
Capturing declining daily activity performance in a technologically-advancing older populati on: UK cultural validation of the Amsterdam IADL Questionnaire
Stringer, G., et al. Capturing declining daily activity performance in a technologically-advancing older populati on: UK cultural validation of the Amsterdam IADL Questionnaire . in British Society of Gerontology 45th Annual Conference. 2016. Stirling
2016
-
[28]
PeerJ, 2017
Petrovic, J., et al., Slower EEG alpha generation, synchronization and "flow" -possible biomarkers of cognitive imp airment and neuropathology of minor stroke. PeerJ, 2017. 5: p. e3839
2017
-
[29]
J Neurol Neurosurg Psychiatry, 2009
Sikkes, S.A.M., et al., A systematic review of Instrumental Activities of Daily Living scales in dementia: room for improvement. J Neurol Neurosurg Psychiatry, 2009. 80(1): p. 7-12
2009
-
[30]
Alzheimers Dement, 2012
Sikkes, S.A.M., et al., A new informant-based questionnaire for instrumental activities of daily living in dementia. Alzheimers Dement, 2012. 8(6): p. 536-43
2012
-
[31]
Neuroepidemiology, 2013
Sikkes, S.A.M., et al., Validation of the Amsterdam IADL Questionnaire(c), a new tool to measure instrumental activities of daily living in dementia. Neuroepidemiology, 2013. 41(1): p. 35-41
2013
-
[32]
J Geriatr Psychiatry Neurol, 2013
Sikkes, S.A.M., et al., Assessment of instrumental activities of daily living in dementia: diagnostic value of the Amsterdam Instrumental Activities of Daily Living Questionnaire. J Geriatr Psychiatry Neurol, 2013. 26(4): p. 244-50
2013
-
[33]
Alzheimers Dement, 2015
Koster, N., et al., The sensitivity to change over time of the Amsterdam IADL Questionnaire((c)). Alzheimers Dement, 2015. 11(10): p. 1231-40
2015
-
[34]
Alzheimers Dement (Amst), 2017
Jutten, R.J., et al., Detecting functional decline from normal aging to dementia: Development and validation of a short version of the Amsterdam IADL Questionnaire. Alzheimers Dement (Amst), 2017. 8: p. 26-35
2017
-
[35]
Spine (Phila Pa 1976), 2000
Beaton, D.E., et al., Guidelines for the process of cross -cultural adaptation of self -report measures. Spine (Phila Pa 1976), 2000. 25(24): p. 3186-91
1976
-
[36]
Reise, S.P. and N.G. Waller, Item response theory and clinical measurement. Annu Rev Clin Psychol, 2009. 5: p. 27-48
2009
-
[37]
Mini-mental state
Folstein, M.F., S.E. Folstein, and P.R. M cHugh, "Mini-mental state": A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 1975. 12(3): p. 189-198
1975
-
[38]
Aging Neuropsychology and Cognition, 1996
Huppert, F.A., et al., Psychometric properties of the CAMCOG and its efficacy in the diagnosis of dementia. Aging Neuropsychology and Cognition, 1996. 3(3): p. 201-214
1996
-
[39]
Int Psychogeriatr, 1997
Morris, J.C., Clinical dementia rating: a reliable and valid diagnostic and staging measure for dementia of the Alzheimer type. Int Psychogeriatr, 1997. 9 Suppl 1: p. 173-6
1997
-
[40]
Yesavage, J.A. and J.I. Sheikh, Geriatric Depression Scale (GDS) - Recent evidence and development of a shorter version. Clinical Gerontologist, 1986. 5(1-2): p. 165-173
1986
-
[41]
Gibbons, and P.K
Choi, S.W., L.E. Gibbons, and P.K. Crane, lordif: An R package for det ecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. Journal of Statistical Software, 2011. 39(8)
2011
-
[42]
Herrera, A.N. and J. Gomez, Influence of equal or unequal comparison group sample sizes on the detection of differential item functioning using the Mantel-Haenszel and logistic regression techniques. Quality & Quantity, 2008. 42(6): p. 739-755
2008
-
[43]
2011, Cambridge: Cambridge University Press
De Vet, H.C.W., et al., Measurement in Medicine . 2011, Cambridge: Cambridge University Press. Page 25 of 25
2011
-
[44]
Jodoin, M.G. and M.J. Gierl, Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 2001. 14(4): p. 329-349
2001
-
[45]
an d B.D
Gelin, M.N. an d B.D. Zumbo, Differential item functioning results may change depending on how an item is scored: An illustration with the Center for Epidemiologic Studies Depression Scale. Educational and Psychological Measurement, 2003. 63(1): p. 65-74
2003
-
[46]
PLoS One, 2019
Rouquette, A., et al., Differential Item Functioning (DIF) in composite health measurement scale: Recommendations for characterizing DIF with meaningful consequences within the Rasch model framework. PLoS One, 2019. 14(4): p. e0215073
2019
-
[47]
2016, IBM Corp.: Armonk, NY
IBM Corp., IBM SPSS Statistics for Windows. 2016, IBM Corp.: Armonk, NY
2016
-
[48]
R Core Team, R: A language and environment for statistical computing. 2019
2019
-
[49]
Spector, and B.M
Fleishman, J.A., W.D. Spector, and B.M. Altman, Impact of differential item functioning on age and gender differences in functional disability. J Gerontol B Psychol Sci Soc Sci, 2002. 57(5): p. S275-84
2002
-
[50]
Med Care, 2004
Tennant, A., et al., Assessing and adjusting for cross-cultural validity of impairment and activity limitation scales through differential item functioning within the fra mework of the Rasch model: the PRO-ESOR project. Med Care, 2004. 42(1 Suppl): p. I37-48
2004
-
[51]
J Alzheimers Dis, 2017
Berezuk, C., et al., Functional Reserve: Experience Participating in Instrumental Activities of Daily Living is Associated with Gender and Functional Independence in Mild Cognitive Impairment. J Alzheimers Dis, 2017. 58(2): p. 425-434
2017
-
[52]
Domingue, and E
Sheehan, C., B.W. Domingue, and E. Crimmins, Cohort Trends in the Gender Distribution of Household Tasks in the United States and the Implications for Understanding Disability. J Aging Health, 2019. 31(10): p. 1748-1769
2019
-
[53]
J Clin Epidemiol, 2007
Niti, M., et al., Item response bias was present in instrumental activity of daily living scale in Asian older adults. J Clin Epidemiol, 2007. 60(4): p. 366-74. Dubbelman et al. (2020) – Supplementary Material Page 1 of 3 Supplementary Material Cross-cultural adaptation To dat...
2020
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.