Errors Produced on the Mini-Mental State Examination and Neuropsychological Test Performance in Alzheimer's Disease, Ischemic Vascular Dementia, and Parkinson's Disease

The Mini-Mental State Examination (MMSE)¹ is the most frequently administered brief cognitive screening measure for the identification of dementia. Traditionally, a summary score at or below 23 is the point where actual cognitive impairment might be present.² However, recent reports have suggested that scores at or below 26 provide a better tradeoff between sensitivity (the ability to correctly identify those individuals who are cognitively impaired) and specificity (the ability to correctly identify those individuals who are cognitively intact).^3,4

Over the years, the construct validity of the MMSE has been studied. Several studies have compared MMSE test performance between patients with Alzheimer's disease (AD) and normal control subjects and found that the normal control subjects performed significantly better on all MMSE items except for two language items (repetition of the phrase “no ifs, ands, or buts” and naming a watch and pencil/pen).^5,6 Indeed, these researchers concluded that these items are not useful in making the diagnosis of dementia. Factor-analytic studies conducted on the MMSE^7,8 commonly produce a two-factor solution, yet the items loading on each factor vary among the studies.^8,9 However, in a recently published study with more than 8,000 participants, a five-factor solution was reported.⁹ Additional research has found the MMSE to have low sensitivity for detecting changes in cognitive functioning when documenting disease course or changes related to therapy.¹⁰

Evidence has emerged over the past 25 years suggesting that patients with AD show differential impairment on tests of declarative memory and semantic knowledge, whereas patients with dementia syndromes such as ischemic vascular dementia associated with periventricular and deep white matter alterations (IVD) and dementia due to Parkinson's disease (PD) produce greater impairment on tests of executive control and motor/visuoconstruction.^11–13 However, despite the popularity of the MMSE, relatively few studies have examined its ability, beyond its summary score, to differentiate between individuals with dementia. Specifically, there is a paucity of research examining the criterion-related validity of the MMSE (i.e., performance differences among patients with different types of dementia). For example, Van Gorp et al.¹⁴ administered the MMSE to patients with AD, patients with vascular dementia, and normal control subjects (NC), and found the two dementia groups did not differ from each other on the summary MMSE score. By contrast, Brandt et al.¹⁵ compared patients with AD and Huntington's disease (HD) on the MMSE and found the HD patients had more difficulty performing the serial subtraction task, whereas the AD patients had more difficulty recalling three words and were more disoriented to the current date.

Given the lack of research examining performance differences on the MMSE among patients with different types of dementia, the purpose of the present study was to quantify the errors produced on the MMSE among patients with various dementia diagnoses (AD, IVD, and PD). In the present research, we derived MMSE indices related to temporal orientation, physical orientation, declarative memory, language, working memory, and motor/constructional functions. We expected that, consistent with past research,¹⁶ the three dementia groups would not differ from each other in overall level of impairment as measured by the MMSE summary score. Our first prediction, however, was that, also consistent with previous research,^14,15,17 participants with AD would show differential impairment on MMSE indices that measure orientation, declarative memory, and language. Our second prediction was that participants with IVD and PD would show differential impairment on MMSE indices related to working memory and motor/constructional functions.

METHODS

Participants

The participants in this study were patients from the Crozer Chester Medical Center's Alexander Silberman Geriatric Assessment Program. An interdisciplinary team including a social worker, geriatrician, neurologist, psychiatrist, and neuropsychologist examined each patient. Neuroimaging (CT or MRI) and laboratory studies were obtained for all participants. Neuropsychological assessments in conjunction with structured clinical interviews were also conducted for each participant. Using this information, a clinical diagnosis of dementia was determined by the interdisciplinary team. On the basis of the team diagnosis, 65 patients received diagnoses of probable Alzheimer's disease according to NINCDS-ADRDA criteria¹⁸ and 63 patients received diagnoses of probable/possible ischemic vascular dementia (IVD) according to the California Criteria of Chui.¹⁷ Patients with cortical infarcts (6 AD, 6 IVD) were excluded from this study.

Patients with PD (n=19) were initially evaluated and followed at the Crozer Chester Medical Center Parkinson's Disease and Movement Disorders clinic. The diagnosis of dementia secondary to Parkinson's disease was made by a neurologist (N.L.), based on the presence of cognitive impairment as well as the presence of three of the four hallmark features of Parkinson's disease (rigidity/postural instability, bradykinesia, resting tremor, and an obvious and sustained response to levodopa or dopamine agonists). Each of the Parkinson's disease patients was assessed with the Unified Parkinson's Disease Rating Scale¹⁸ and was taking antiparkinsonian medication at the time of neuropsychological testing.

There were no between-group differences in overall level of dementia as assessed with the MMSE, age, education, or level of depression as assessed by the Geriatric Depression Scale¹⁹ (Table 1). Data regarding participants' medical history were collected during the clinical interview. Individuals with a history of seizure disorder, thyroid or B₁₂ deficiency, closed head injury, substance abuse (including alcohol abuse), major depression, or other serious psychiatric disorders were excluded.

The Mini-Mental State Examination

The MMSE was administered by doctoral-level clinical psychology graduate students according to standard instructions. In our clinical experience we have noted that dementia patients often cannot either establish or maintain set for the serial sevens task. Therefore, in the attention/concentration section of the MMSE we omitted the serial sevens task and administered only the “world” backwards task.

In addition to the calculation of the total MMSE score (range 0–30), we derived seven indices extrapolated directly from the original MMSE test items. Although the possible range varies among the individual indices, higher scores indicate better performance across all indices.

Temporal Orientation Index (range 0–5):

Consisted of the five items that assess orientation to time (year, season, date, day, and month).

Physical Orientation Index (range 0–5):

Consisted of the five items that assess orientation to place (state, county, town, hospital, and floor).

Total Orientation Index (range 0–10):

Constructed by summing the Temporal and Physical Orientation indices.

Language Index (range 0–4):

Consisted of those items purported to assess language abilities (naming of a watch and pen, repeating the phrase “no ifs, ands, or buts,” and following the written command to “close your eyes”).

Declarative Memory Index (range 0–3):

Noted how many words patients recalled after a delay.

Working Memory Index (range 0–8):

Consisted of spelling “world” backwards and carrying out the three-step command (“take this paper in your right hand, fold it in half, and place it on the floor”). The three-step command was included in the Working Memory Index as opposed to the Language Index because it correlated more strongly with performance on the “world” backwards task (P=0.090) than with the Language Index tasks of naming, repeating, and following a written command (P=0.320).

Motor/Constructional Index (range 0–2):

Consisted of the two items on the MMSE that require a motor response: copying the intersecting pentagons and producing a written sentence to command.

The errors patients make on these two MMSE test items can be quite diverse. Thus, in addition to the seven indices described above, we created two new subindices that tallied the number of errors produced when participants were asked to copy the intersecting pentagons and to produce a sentence. A high score signifies an increased number of errors and increasingly impaired test performance. These indices were scored independently by two raters (A.L.J., S.A.C.). Discrepancies in rating a participant were resolved by group consensus (A.L.J., S.A.C., D.J.L.). These additional subindices were created to measure the breadth of errors produced by dementia patients on these two test items.

Copy of Intersecting Pentagons Subindex (range 0–8):

Listed in Appendix A are eight errors used in assessing how well patients were able to copy the intersecting pentagons. For this subindex, we attempted to develop operational definitions for a wide variety of motor and perseverative behaviors. For example, hyperkinetic and/or interminable perseverations were scored according to criteria described by Lamar et al.²⁰ Examples of patient responses illustrating these errors are provided in Figure 1.

Sentence Production Subindex (range 0–8):

Listed in Appendix A are eight errors used in assessing how well patients were able to produce a sentence to command. For this subindex, we again attempted to develop concrete operational definitions for a wide variety of motor, language-related, and perseverative behaviors. For example, when asked to produce a sentence, some patients in our sample wrote the phrase “close your eyes.” We felt that such behavior was an example of a recurrent perseveration as defined by Sandson and Albert,²¹ and it was therefore scored as an error.

Neuropsychological Assessment

All participants were administered the following neuropsychological tests:

Executive Control:

Assessed with the Boston Revision^22,23 of the Wechsler Memory Scale–Mental Control subtest (WMS-MC).²⁴ In addition to the three tasks that comprise the standard WMS-MC subtest (counting backwards from 20 to 1, reciting the alphabet, and adding serial threes), the Boston Revision of the subtest includes four additional tasks: reciting the months of the year forward and backward, an alphabet rhyming task that requires patients to identify letters that rhyme with the word “key,” and an alphabet visualization task that requires patients to identify all block-printed letters that contain curved lines. Patients were allowed to work as long as necessary on these tasks provided they were working meaningfully.

The dependent variable derived from this test consisted of an accuracy index (AcI) derived from three non-automatized tasks from the WMS-MC (months backward, alphabet rhyming, and alphabet visualization). This accuracy index was based on the following algorithm: AcI={1–[(false positives+misses)/# possible correct]}×100. This algorithm yielded a percentage score ranging from 0 to 100, such that patients obtaining a score of 100% correctly identified all targets and made no false positive responses or misses. Composite scores assessing performance on the non-automatized mental control tasks were calculated by averaging the AcI for all respective tasks for each patient.

Executive systems functioning was also assessed with tests of letter word list generation (WLG).²⁵ On the letter WLG test, patients were given 60 seconds to generate words, excluding proper nouns, beginning with a specified letter (F, A, or S). The dependent variable was the number of words generated for each individual letter.

Language/Semantic Functioning:

Assessed with the 60-item version of the Boston Naming Test²⁶ (BNT). The dependent variable derived from the BNT was the number of correct responses.

Declarative Memory:

Assessed with the nine-word dementia version²⁷ (CVLT-9) of the California Verbal Learning Test²⁸ (CVLT). For the present research two CVLT indices were analyzed: Immediate Free Recall, assessed by tallying the total number of words recalled from list A, trials 1–5 and the Recognition Discriminability Index, which takes into account hits, omissions, and false positive responses. These indices were chosen because previous research with patients with AD and IVD has shown that these indices load on separate factors and therefore apparently assess different aspects of declarative memory.²⁸

Data Analysis

Three sets of analyses were conducted for this study. The first consisted of three sets of chi-square analyses to assess for between-group differences across all of the individual MMSE test items (AD vs. IVD, AD vs. PD, and IVD vs. PD; significance set at P<0.05). In cases where a task contributed more than one point to the total MMSE score, the participants' performance on that task was coded as being either correct or incorrect.

The second set of analyses consisted of nonparametric Kruskal-Wallis one-way analyses of variance (ANOVAs) performed to assess for between-group differences among the three dementia groups on the seven MMSE indices and two MMSE subindices. These analyses were followed by pairwise Mann-Whitney U-tests (significance set at P<0.05).

The third set of analyses consisted of a series of nonparametric Spearman correlations between the nine MMSE subscales and neuropsychological test performance. Despite the nonparametric nature of these correlations, significance was set at P<0.001. This was done for several reasons, including the large number of analyses, the modest relationship of the correlation coefficients produced, and the need to minimize the likelihood of making a type I error.

RESULTS

Between-Group Analyses: MMSE Individual Test Items

Between-group results for the individual MMSE test items indicate a significantly greater proportion of AD participants, compared with IVD patients, obtained correct scores when asked to spell “world” backwards (χ²=12.22, df=1, P=0.001), execute a three-step command (χ²=4.14, df=1, P=0.042), and produce a written sentence (χ²=7.61, df=1, P=0.006). Similarly, a significantly greater proportion of AD than PD participants obtained correct scores when asked to spell “world” backwards (χ²=9.39, df=1, P=0.002) and copy the intersecting pentagons (χ²=8.66, df=1, P=0.003). By contrast, a significantly greater proportion of IVD and PD participants performed better than the AD participants when asked for the day of the week (AD vs. IVD, χ²=10.26, df=1, P=0.001; AD vs. PD, χ²=13.25, P=0.000). The only significant between-group finding for the IVD and PD groups was for the copy of the intersecting pentagons (χ²=4.02, df=1, P=0.045), where a greater proportion of the IVD than the PD participants obtained correct scores. These results are displayed in Table 2.

Between-Group Analyses: MMSE Indices and Subindices

Kruskal-Wallis one-way ANOVAs revealed significant between-group differences on the following indices and subindices: Temporal Orientation (χ²=8.66, df=2, P=0.013), Total Orientation (χ²=8.02, df=2, P=0.018), Declarative Memory (χ²=9.95, df=2, P=0.007), Working Memory (χ²=14.69, df=2, P=0.001), Motor/Constructional (χ²=12.78, df=2, P=0.002), Sentence Production Subindex (χ²=17.83, df=2, P=0.000), and Intersecting Pentagons Subindex (χ²=18.17, df=2, P=0.000). Mann-Whitney U-tests were used to measure differences among the dementia groups on these five MMSE indices and two subindices (Table 3).

When participants with AD and IVD were compared, the AD group obtained significantly poorer scores on Temporal Orientation (z=–2.21, P=0.027) and Declarative Memory (z=–2.09, P=0.037). By contrast, patients with IVD obtained poorer scores on Motor/Constructional (z=–2.58, P<0.001) Working Memory (z=–3.50, P<0.001), Intersecting Pentagons (z=–3.04, P=0.002), and Sentence Production (z=–3.90, P<0.001).

Similar findings were obtained when AD and PD participants were compared: the AD group obtained significantly poorer scores on Temporal Orientation (z=–2.56, P<0.010) and Total Orientation (z=–2.71, P=0.007), whereas patients with PD obtained poorer scores on Working Memory (z=–2.78, P<0.005), Motor/Constructional (z=–3.22, P=0.001), Intersecting Pentagons (z=–3.83, P<0.001), and Sentence Production (z=–2.90, P=0.004). No differences between IVD and PD groups were found for any scale. However, only a comparatively small number of PD patients were assessed, and it is possible that additional or larger differences might have emerged if more PD patients had been included in the study.

Temporal Discrepancy Analyses

Given the between-group differences among participants with AD and IVD/PD on the Temporal Orientation Index (AD performing significantly worse than IVD and PD), additional analyses were conducted for the participants' responses to the current year, month, and season to quantify the disparity between the patients' responses and the actual date of their examination. For example, if the evaluation took place in June 1998 and the patient said it was April 1992, scores of 2 and 7 were assigned, respectively, for the month and year discrepancy scores. If the size of the discrepancy in months depended on whether one counted backward to the previous correct month or forward to the next one (e.g., patient says it is April when it is actually September; the score could be 5 or 7), the patient was given the benefit of the doubt and the lower score was assigned, representing the shortest distance between the two responses.

Once these discrepancy scores were calculated for the actual year, month, and season, follow-up analyses were conducted only with those individuals who provided an incorrect response. The decision to include only participants who provided an incorrect response was made to guard against restriction of range and the creation of distributions that might been very skewed. Limiting the analyses to patients who incorrectly stated the year, month, or season reduced the sample sizes for the IVD and PD groups, which were then collapsed into a single group and compared with the incorrectly responding AD participants. These analyses found that the AD group gave answers of greater temporal disparity than the IVD/PD group with respect to the current year (AD, n=26, IVD/PD, n=21; z=–2.10, P=0.036) and month (AD, n=23, IVD/PD, n=21; z=–2.06, P=0.040).

Correlational Analyses

Finally, nonparametric correlational analyses (Table 4) were conducted between the MMSE summary score, the seven MMSE indices and two MMSE subindices, and the neuropsychological measures described above (WMS-MC, FAS, BNT, and CVLT-9). We found significant correlations between the MMSE summary score and most neuropsychological measures, such that a higher MMSE summary score was associated with better neuropsychological test performance (see Table 4).

A different profile emerged when neuropsychological test performance was correlated with the MMSE indices and subindices (see Table 4). For example, as participants obtained better scores on the Temporal Orientation Index, they recalled more words from the CVLT-9 Immediate Free Recall test trials and obtained a higher score on the CVLT-9 Recognition Discriminability Index. Similarly, as participants obtained better scores on the Physical Orientation Index, they also recalled more words from the CVLT-9 trials and obtained better scores on the Boston Naming Test. Better scores on the Total Orientation Index were also positively correlated with more words recalled from the CVLT-9 trials and better scores on the CVLT-9 recognition index. Not surprisingly, significant positive correlations were also obtained between our MMSE Declarative Memory Index and the CVLT-9 recall trials and recognition index.

Different relationships were found when we examined correlations of the MMSE Motor/Constructional Index, Working Memory Index, Sentence Production Subindex, and Intersecting Pentagons Subindex with neuropsychological test performance (see Table 4). Better scores on the Motor/Constructional Index were accompanied by greater output on tests of letter WLG. Better scores on the Working Memory Index were accompanied by better scores on the non-automatized tasks from the WMS-MC test and a higher score on the CVLT-9 Recognition Discriminability Index. Finally, we found that as participants produced sentences and figures containing fewer errors on the Sentence Production and Intersecting Pentagons subindices, output on tests of WLG increased.

DISCUSSION

Clinical practice with respect to the evaluation of the dementias has undergone significant change over recent years. Because of the exigencies of managed care, more information must be obtained in less time from fewer tests. Of even greater importance is the emerging literature suggesting that large numbers of patients diagnosed with AD in life actually present with evidence of multiple neuropathologies that can be reasonably associated with dementia upon autopsy,²⁹ suggesting that the diagnostic labels that we currently use may not predict the presence of a single type of neuropathology. Thus the evaluation of the dementias, in addition to their differential diagnosis, is increasingly challenging and complex. For these reasons, in the present study we sought to maximize the clinical utility of the MMSE by quantifying the errors produced on the MMSE among patients with different dementia syndromes. Indices that assessed orientation, memory, language, working memory, and motor/constructional function were derived directly from the corpus of the MMSE. In addition, we also developed two subindices designed to provide a better assessment of how well patients were able to copy the intersecting pentagons and produce a written sentence.

As was found in past research,^14–16 our patient groups did not differ on the MMSE summary score. However, consistent with our first prediction, participants with AD showed differential impairment on the MMSE indices measuring orientation and memory when compared to participants with IVD and PD. Inconsistent with our first prediction, we did not find a significant difference between our patient groups on the MMSE Language Index. However, as noted above, prior studies have also failed to show between-group differences on the MMSE repetition and object naming test items.^5,6 Furthermore, it has been suggested¹⁰ that the MMSE repetition item included in our Language Index may not be measuring the same construct as test items such as carrying out the three-step command and object naming. This implies that the Language Index used in the present research may not be a pure measurement of language functioning and may instead represent more heterogeneous cognitive functions.

Our second prediction was that participants with IVD and PD would show differential impairment on the MMSE indices of working memory and motor/constructional functions when compared with AD patients. Consistent with this prediction, we found that the IVD and PD groups performed significantly worse than the AD patients on the MMSE indices assessing working memory and motor/constructional functions. These findings are consistent with a substantial body of literature that has shown that these patients are particularly disadvantaged on measures that assess a wide range of executive control and visuoconstructional skills.^14,15 However, the number of PD patients assessed in the current study was small compared with the other patient groups. Therefore our findings must be interpreted with caution. Also, patients with PD were taking antiparkinsonian medication at the time of their assessment, and this could have differentially affected their test performance.

On the basis of the between-group differences for the Temporal Orientation Index, further analyses were conducted to examine the disparity between patient responses and the true temporal orientation answer. We found that the participants with AD responded with answers of greater temporal disparity than participants with IVD and PD, suggesting that patients with AD were more disoriented both quantitatively and qualitatively. To our knowledge, such a finding has never been reported. These results suggest that temporal disparity may be a construct separate from simple temporal disorientation. Clearly, this finding warrants further study.

To address issues of criterion-related validity, we correlated several traditional neuropsychological measures with our MMSE indices and subindices. Several interesting findings emerged. First, strong and consistent relationships were found between our MMSE orientation indices and measures that assess immediate free recall and delayed recognition memory from the CVLT-9 and the Boston Naming Test, such that relatively intact scores on the MMSE orientation indices (Temporal Orientation Index, Physical Orientation Index) were related to better performance on tests of memory and naming. Second, our MMSE Working Memory and Motor/Constructional indices were highly correlated with output from tests of letter word generation and tests that assess patients' capacity to mentally manipulate non-automatized mental sets. In sum, the between-group and correlational analyses reported above corroborate the differential patterns of neuropsychological strengths and weaknesses often associated with AD on the one hand, and IVD and PD on the other hand. Again, to our knowledge, this is the first time such findings have been reported for the MMSE.

On the basis of these data one might question if the MMSE provides a comprehensive cognitive assessment. The answer to this is, of course, no. We found the MMSE summary score to be significantly correlated with virtually all of our neuropsychological measures. Therefore, we believe that the best use of the MMSE summary score is to provide an estimate of global cognitive impairment. Although the MMSE indices we have constructed yielded significant between-group differences, the absolute between-group differences among the three patient populations were modest. Therefore, an analysis of our various MMSE indices along with the MMSE summary score cannot substitute for a comprehensive neurocognitive assessment.

Nonetheless, an analysis of the indices described above can assist professionals in deriving important information regarding the etiology of a patient's dementing illness. As already noted, an emerging literature suggests that many patients who have received diagnoses of dementia actually present with a wide array of neuropathology, and that these neuropathological alterations can be distributed throughout cortical as well as subcortical regions of the brain.²⁹ If this is truly so, greater attention to the distribution of errors on the MMSE can result in the articulation of important clinical issues, perhaps earlier in the course of the evaluation process, and may generate more targeted referral questions. Other researchers have reported interesting results based on modifications of standard MMSE test items. For example, in addition to asking patients to spell “world” backwards, Leopold and Borson³⁰ also required patients to spell “world” in alphabetical order. As compared to a diagnosis of dementia derived from formal neuropsychological assessment, this modification achieved a positive discrimination rate of 95 percent. Thus, novel ways of evaluating MMSE test performance yield useful information regarding patients' cognitive status.

One application of the data reported above might be in clinical situations where the MMSE is used as a standard part of neuropsychiatric follow-up care. In this context, the MMSE summary score should be analyzed independently with respect to performance on the various indices and subindices. Even though the MMSE summary score might suggest the presence of a mild dementia, differential impairment on the orientation test questions (particularly if the responses are very disparate), or on the motor/construction and working memory test items, may offer greater specificity as to the etiology of a dementing illness. If this paradigm is applied to a long-term care setting, changes in the distributions of the errors produced on the MMSE, particularly when the summary score remains stable, could signal the presence of newly developing medical problems.

ACKNOWLEDGMENTS

The authors thank Richard N. Jones, Sc.D., for his insightful comments on an earlier version of this manuscript and John Caruso for his assistance with computer graphics. A portion of this work was presented at the 20th annual meeting of the National Academy of Neuropsychology, Orlando, FL, October 2000.

TABLE 1. Demographic characteristics

Enlarge table

TABLE 2. Chi-square comparisons yielding significant differences between dementia groups on individual MMSE items

Enlarge table

TABLE 3. Performance of dementia groups on MMSE indices and subindices

Enlarge table

TABLE 4. Correlations of MMSE summary, index, and subindex scores with neuropsychological test scores

Enlarge table