Comparing Detection of Alzheimer’s and Vascular Disease–Related Cognitive Impairment With Brief Cognitive Screens

Dementia, one of the leading causes of disability and death in older adults, is associated with substantial medical, social, and economic burden (1, 2). The estimated number of people currently living with dementia (47 million) is expected to increase to 75 million by 2030 (3). In 2022 alone, the expected health care costs associated with dementia are expected to exceed $300 billion (4). Although there are many types of dementia, Alzheimer’s disease (AD) and cerebrovascular disease (VaD) are the two most common etiologies and account for most of the dementia-related economic burden (5, 6). Prompt diagnosis of these dementias is key in providing opportunities for quality-of-life interventions and treatments that may attenuate cognitive decline and reduce socioeconomic burden (7). The gold standard for diagnosis of dementia remains a comprehensive neurocognitive evaluation (8); however, cognitive screens in various medical settings often represent the first and best method for early detection of cognitive decline and dementia. Despite this fact, cognitive screens are underused and dementia remains underdiagnosed in medical settings (4, 9, 10).

Many cognitive screening tasks have been developed to efficiently and accurately identify abnormal cognitive aging. One of the most widely used and best validated screens is the Mini-Mental State Examination (MMSE) (11, 12). Despite its clinical utility, it has been criticized for its limited coverage of executive functions, language, and visuocognitive abilities (13–15). It has also been criticized for a restricted scoring range with ceiling effects (14, 16). Finally, the MMSE is copyrighted intellectual property that cannot be used without financial cost (17).

In response to these and other limitations, the Modified Mini-Mental State Examination (3MS) was developed (18). The 3MS includes additional items covering executive functions, memory, and language abilities and uses an expanded scoring system to better capture a range of performances. Notably, the 3MS is free for clinical use (unlike the MMSE) and does not require costs associated with mandatory training/certification, unlike other popular screen alternatives (e.g., the Montreal Cognitive Assessment) (19). The clinical utility of the 3MS is supported by multiple studies indicating improved reliability over the MMSE across a range of clinical populations and settings (13, 20–23).

Given the practical and clinical advantages, some have argued for widespread substitution of the 3MS for the MMSE (15). However, research has been mixed regarding the differential sensitivity of the 3MS and MMSE in detecting dementia. Studies employing samples with heterogenous dementing etiologies have found the 3MS to be more sensitive not only to dementia (i.e., major neurocognitive disorder) but also to mild cognitive impairment (i.e., mild neurocognitive disorder) (13, 15, 18, 20, 24). In contrast, these screens have demonstrated relatively comparable sensitivities when detecting dementia due to probable and possible AD (23). Differences with respect to dementia etiology (i.e., mixed dementias vs. AD) across these study samples may be a driving force in their discrepant findings.

To date, no study has compared the classification accuracies of the 3MS and MMSE in detecting cognitive decline from the two most common neurodegenerative disease processes in a single sample. The primary objective of the study was to compare the differential utility of these measures in predicting an eventual diagnosis of AD or VaD. We hypothesized that 3MS classification accuracy would exceed that of the MMSE and that this advantage would be most pronounced in detecting VaD.

Methods

Participants

Participants consisted of 98 male veterans referred for comprehensive neuropsychological assessment due to concern for cognitive decline. Sixty-four adults met criteria for a mild or major neurocognitive disorder due to objective cognitive impairment and the extent of functional decline. Of these, cases were subdivided into two groups designated by predicted etiology: cognitive impairment due to AD (N=20) or VaD (N=44). Healthy adults in the comparison group (HC; N=34) were those with cognitive complaints who did not meet criteria for any cognitive disorder.

Procedures

All assessments took place at a southeastern Department of Veteran Affairs hospital from March 2, 2017, to December 21, 2017. All study procedures were approved by the local institutional review board and the research and development board of the Department of Veteran Affairs. Informed consent was waived due to the retrospective nature of the study.

Evaluations for clinical purposes were completed by licensed psychologists who were board certified or eligible for board certification in neuropsychology, with frequent involvement of a neuropsychology trainee under direct supervision (e.g., graduate students, interns, and postdoctoral fellows). All participants underwent neuropsychological evaluations consisting of a comprehensive neuropsychological test battery as well as clinical interview with the patient and, if available, collateral informants such as family. Prior to the evaluation, veterans completed the 3MS (with derived MMSE). Diagnoses and etiological conclusions were made by the licensed provider at the time of the evaluation with integration of medical record review, neuroimaging, laboratory work, neuropsychological test performance, and a clinical interview conducted with the patient and, if available, collateral informants, such as family members. Diagnoses were made in accordance with criteria put forth by the DSM-5 (25). For this study, cases were retrospectively selected from archival clinical data based on diagnosis after comprehensive neuropsychological evaluation.

Measures

The MMSE is a standardized cognitive screen that assesses orientation, attention, working memory, verbal memory, language, and visuoconstruction. Administration is brief (5–10 minutes), and there is empirical support for its sensitivity and specificity of identifying individuals with pathological cognitive impairment (11, 26). The 3MS is an enhanced version of the MMSE that was designed to be more standardized in its assessment, more granular in scoring, and to cover a broader variety of cognitive domains (18). It includes an expanded scoring system with four additional items: additional orientation questions, verbal fluency, verbal reasoning, and delayed recall (with semantic and multiple-choice cues).

Statistical Analysis

Descriptive statistics were run for demographic variables. Nonparametric statistics were employed for correlations (e.g., Spearman’s rho) and group comparisons of cognitive screen performance due to violations of parametric assumptions. Specifically, Kruskal-Wallis tests were run with 3MS and MMSE performance as the between-subjects factor. Mann-Whitney U tests were conducted for post hoc comparisons. Effect sizes were calculated with eta squared. Logistic regression models used cognitive screen performance (3MS, MSSE) as separate, individual predictors of group status (HC vs. cognitive impairment [CI]; HC vs. VaD; HC vs. AD). Area under the curve (AUC) for receiver operating characteristic (ROC) values were calculated in order to quantify and compare the relative discriminability of the cognitive screens. Discriminability was classified as acceptable (0.70≤AUC≤0.79), excellent (0.80≤AUC≤0.89), or outstanding (AUC≥0.90) (27).

Results

Table 1 summarizes descriptive statistics and univariate group comparisons for cognitive screen performance.

Kruskal-Wallis tests revealed 3MS and MMSE performance differed significantly across the three groups with large effect sizes (η²>0.30). Post hoc analyses revealed that HC performed significantly better than VaD across the 3MS and MMSE. Both HC and VaD performed significantly better than AD on both cognitive screens.

Ages of participants ranged from 45 to 95 and differed significantly across groups (F=18.77, p<0.001). The healthy control (HC) group (M=69.0, SD=9.4) was significantly younger than the VaD group (M=75.4, SD=8.8), and both HC and VaD groups were younger than the AD group (M=84.1, SD=7.5). Age is not thought to reflect a confounding variable in the current study. Evidence suggests the traditional pass/fail cut-score of the 3MS has equivalent, if not better, classification accuracy than the 3MS age-adjusted scoring when distinguishing healthy elderly adults from those with dementia or cognitive impairment (CI) (28). The HC (M=13.1, SD=2.1), VaD (M=13.1, SD=2.4), and AD (M=12.3, SD=3.9) groups were equivalent with respect to education.

Table 2 presents descriptive correlations for cognitive screen performance and demographic variables. As would be expected given the degree of test item overlap, the 3MS and MMSE were strongly correlated but not redundant (r=0.87). The 3MS and MMSE showed equivalent relationships with both age and education.

Table 3 presents logistic regression and ROC statistics for the 3MS and MMSE across all group comparisons of interest (HC vs. CI; HC vs. VaD; HC vs. AD). Both cognitive screens were significant predictors (p<0.001) of group status across all group comparisons. Table 4 shows classification accuracy results (e.g., hit rate, sensitivity, specificity) across 3MS and MMSE cutoffs for group comparisons of interest.

Overall Classification Accuracy (HC vs. CI)

The 3MS showed outstanding discriminability (AUC=0.93), whereas the MMSE demonstrated excellent discrimination (AUC=0.86) of HC and CI. Moreover, the 3MS showed a better balance of hit rate, sensitivity, and specificity than the MMSE across a range of cutoffs.

Classification Accuracy×Etiology (HC vs. AD; HC vs. VaD)

Overall, the 3MS and MMSE showed comparable outstanding discrimination (AUCs> 0.90) of HC and AD. Classification accuracies for both measures demonstrated strong overall accuracy (hit rate> 0.90), sensitivity, and specificity (i.e., exceeding 0.95) at multiple cutoffs.

In contrast, the 3MS outperformed the MMSE when distinguishing HC from VaD. Specifically, the outstanding discriminability of the 3MS (AUC=0.92) exceeded the excellent discriminability of the MMSE (AUC=0.83). Additionally, the 3MS demonstrated a better balance of hit rate, sensitivity, and specificity than the MMSE across a range of cutoffs.

Discussion

Findings support the central hypothesis that the 3MS would demonstrate superior detection of cognitive impairment compared to the MMSE. Moreover, the extent of this enhanced detection was influenced by the underlying etiology of cognitive impairment. Consistent with our hypothesis, the enhanced classification accuracy of the 3MS was most pronounced when discriminating healthy adults from adults with cognitive impairment due to VaD. In contrast, the 3MS and MMSE showed relatively comparable discrimination of healthy adults and those with cognitive impairment due to AD. Taken together, findings support the incremental utility of the 3MS over the traditional MMSE as a screening measure for the two most common dementia etiologies.

The outstanding discriminability of the 3MS exceeded the MMSE’s excellent classification accuracy when distinguishing healthy adults from those with mild or major neurocognitive disorder due to AD or VaD. This finding is consistent with the large body of research showing that the 3MS is more sensitive than the MMSE in detecting dementias (i.e., major neurocognitive disorders) in samples of mixed etiologies (13, 18, 20, 24). Similarly, the 3MS has shown better detection of mild cognitive impairment (i.e., mild neurocognitive disorder) stemming from a variety of etiologies (15). Although modest, the increased discriminability of the 3MS in this study is meaningful when one considers the medical and societal costs associated with delayed and missed diagnosis of dementia.

The enhanced psychometric properties of the 3MS in detecting cognitive impairment are likely caused by a combination of factors. First, the addition of items and expanded scoring allows for greater variability in performance, increasing reliability potential. Indeed, the 3MS has shown increased reliability over the MMSE across a range of settings and clinical populations (13, 20–23). Beyond this pure psychometric advantage, research suggests the content of the additional items have afforded the measures increased discriminant validity. Specifically, the 3MS has shown stronger relationships with key cognitive abilities (e.g., memory, language) among individuals with mild cognitive impairment (15). Last, the incremental utility of the 3MS appears dependent on etiology of cognitive impairment.

Findings add to the literature by establishing that the incremental clinical utility of the 3MS varies across the two most common dementia etiologies. Consistent with prior research, both measures demonstrated comparable outstanding detection of individuals with AD (23). Moreover, research suggests that the verbal fluency item is the sole addition to the 3MS that contributes to its marginal advantage in detecting mild forms of AD (19). With AD progression comes significant global cognitive impairment that can be equally detectable by both measures. In contrast, VaD is more commonly associated with comparatively subtler cognitive impairments. In the context of cerebrovascular disease, the 3MS showed outstanding detection that exceeded the excellent discriminability of the MMSE. This finding is consistent with research showing that the MMSE has reduced detection of mild cognitive decline (14, 20), other subcortical dementia processes (e.g., Parkinson’s disease) (19), and cerebrovascular insult (e.g., right hemisphere stroke) (22). Moreover, the items added to the 3MS tap constructs that are particularly sensitive to cerebrovascular decline (e.g., processing speed, executive functioning) (29, 30). Specifically, verbal fluency requires processing speed and aspects of executive functioning, whereas verbal reasoning taps unique aspects of executive functioning. Finally, the addition of category and multiple-choice cues to memory items may permit the differentiation of retrieval versus consolidation deficits that are more characteristic of VaD versus AD memory profiles (31).

Several limitations should be considered when interpreting these results. First, this was a study of primarily male veterans from a single Veterans’ Affairs Hospital in the Southeastern United States. As such, the potential impact of other demographic factors (e.g., nonveteran status, sex) is unknown. Next, differential diagnoses were made clinically in the context of a full neuropsychological evaluation; however, comorbid neurodegenerative processes appear to be the rule rather than the exception (32–34). A portion of our AD group likely had neuropathic features of VaD processes and vice versa. Additionally, definite diagnosis requires neuropathologic confirmation of disease. Last, our study used a modest sample size. Despite this limitation, it is important to note that ROC results were strikingly similar to those obtained from studies employing significantly larger samples (23, 28).

The current study included mixed samples of individuals with mild and major neurocognitive disorders (i.e., mild cognitive impairment and dementia). Although this decision reduced homogeneity within groups, it maximized ecological validity. Specifically, practitioners treating elderly adults with cognitive complaints are often unable to distinguish mild cognitive impairment from mild dementia on the basis of interview alone. As such, research investigating cognitive screen cutoffs that combine these groups mirrors the clinical environment that practitioners face in their daily work.

Conclusions

Overall, results support the adoption of the 3MS over the MMSE. The added cost associated with the 3MS (i.e., approximately 5–10 minutes of additional administration time) pales in comparison to its incremental utility. The 3MS is particularly better than the MMSE at detecting VaD, the second most common dementing etiology. These classification enhancements have considerable medical, societal, and financial implications. Early and accurate detection of AD and VaD allows for targeted intervention to promote health behaviors, attempt to slow disease progression, and foster patient and caregiver preparedness and psychological coping. Moreover, early detection is critical to research that furthers understanding of these conditions and the future development of interventions targeting disease progression. Replication with larger, ethnically diverse veteran and civilian populations is strongly recommended. Additionally, future studies could examine item-level data to explore relationships to various cognitive abilities and relative contribution to enhance detection of VaD.