Agreement and conversion formula between mini-mental state examination and montreal cognitive assessment in an outpatient sample

Abstract

AIM

To explore the agreement between the mini-mental state examination (MMSE) and montreal cognitive assessment (MoCA) within community dwelling older patients attending an old age psychiatry service and to derive and test a conversion formula between the two scales.

METHODS

Prospective study of consecutive patients attending outpatient services. Both tests were administered by the same researcher on the same day in random order.

RESULTS

The total sample (n = 135) was randomly divided into two groups. One to derive a conversion rule (n = 70), and a second (n = 65) in which this rule was tested. The agreement (Pearson’s r) of MMSE and MoCA was 0.86 (P < 0.001), and Lin’s concordance correlation coefficient (CCC) was 0.57 (95%CI: 0.45-0.66). In the second sample MoCA scores were converted to MMSE scores according to a conversion rule from the first sample which achieved agreement with the original MMSE scores of 0.89 (Pearson’s r, P < 0.001) and CCC of 0.88 (95%CI: 0.82-0.92).

CONCLUSION

Although the two scales overlap considerably, the agreement is modest. The conversion rule derived herein demonstrated promising accuracy and warrants further testing in other populations.

Key Words: Mini mental state examination; Montreal cognitive assessment; Cognition; Equation; Assessment; Old age psychiatry

Core tip: In this study we examined the agreement between mini-mental state examination and montreal cognitive assessment in an older population attending mental health service outpatients. Although both scales assess the same construct (cognition) the agreement between them was modest. Further we delivered a conversion rule which can allow conversion of scores between these scales. The converted scores had a high agreement with original ratings. Finally, this new conversion rule was superior to three previously suggested equating rules.

INTRODUCTION

The mini-mental state examination (MMSE)[1] and montreal cognitive assessment (MoCA)[2] are cognitive screening tests that are widely used in both everyday clinical practice and research. However, some evidence suggests that the MMSE is less sensitive for detecting milder cognitive deficits compared to the MoCA, while other studies indicate that it’s relative insensitivity to visuospatial and executive deficits impact limit it’s suitability in particular populations, e.g., vascular cognitive impairment or Parkinson’s disease[3-6]. Comparison of these two tests in specific populations such as patients with Parkinson’s disease[7,8] brain metastases[9] or sub-arachnoid haemorrhage[10] indicate that the MoCA is more suitable because it can detect mild forms of cognitive impairment and especially where this includes executive dysfunction. Similarly, population based studies of mild cognitive impairment (MCI) indicate that the MoCA is more sensitive than the MMSE in detecting mild forms of cognitive impairment[11]. However, to our knowledge no specific comparison of those two tests has been conducted in a general psychogeriatric population where cognitive testing is routine practice.

Furthermore, clinical trials vary in their use of these two scales which makes comparisons between studies and meta-analyses difficult. Equating methodologies can facilitate comparison between studies using different scales to measure the same construct. Previous studies have developed conversion rules for the MMSE and MoCA using either equipercentile equating and/or log-linear smoothing methods. These studies relate to specific populations; Roalf et al[12] studied a selected population with Alzheimer’s disease (AD), MCI and cognitively intact participants, van Steenoven et al[8] studied a population with Parkinson’s disease, and Trzepacz et al[13] studied a selected population of AD, MCI and participants deemed cognitively intact. However, given that these studies used specific and biased populations there is a lack of data in general elderly psychiatric patients. In addition, only one of these conversion rules (van Steenoven et al[8]) has been subject to further examination[7] which although conducted in a similar population only found moderate agreement [Pearson correlation coefficient 0.66 (95%CI: 0.56-0.75)].

Given that both scales are widely used in clinical settings, as well as in clinical trials and cohort studies, a rule to facilitate conversions and comparison of data from different centres and different clinical trials which have used these instruments would have utility.

Therefore, the aims of the present study were threefold (1) to estimate the level of agreement between MMSE and MoCA within an old age psychiatry population; (2) to derive a conversion formula for the two scales and test it in a random population of similar setting; and (3) to compare the new conversion formula with those described in previous studies.

MATERIALS AND METHODS

Subjects and design

This is an observational cross sectional study of performance using two screening cognitive scales in consecutive patients attending an old age psychiatry outpatient clinic and Day Hospital. The “single group design” method was used in this study reflecting that the same population was assessed with the two cognitive tests (MoCA and MMSE).

Procedures

All assessments were conducted by the same psychiatrist who was trained in the use of MoCA and MMSE (McCarthy G). Both tests were administered on the same day with a maximum of 3 h time gap to avoid boredom and/or learning effects. The tests were administered with no particular order (randomly).

Clinical assessments

Demographics: Demographic data (gender, age) were collected from medical records (files and hospital computerised database). In addition, information about years of education was collected from patients and relatives.

Diagnosis: ICD-10 psychiatric diagnoses were collected from the files and collapsed to main ICD-10 F categories. Where multiple diagnoses were evident the most predominant was chosen.

Cognitive assessments: (1) MoCA[2]. The MoCA assesses visuospatial/executive function, naming, memory, attention, concentration, language, abstract thinking, recall memory and orientation. It is scored on a 30-point scale. Higher scores indicate better cognitive performance. Administration typically takes about 12-15 min. Its psychometric properties have been investigated in many studies and it has been found to be superior to the MMSE for the detection of MCI[14]. In addition unlikely to MMSE it takes to account the education level; and (2) MMSE[1]. The MMSE comprises 11 questions assessing orientation to time and place, attention, immediate and short-term recall, language and visuospatial abilities. It is a brief cognitive screening instrument that takes less than ten minutes for administration. Over the past 40 years it has been the most widely used tool in clinical and research settings for brief assessment of cognitive status in elderly individuals. It’s psychometric properties have been thoroughly reviewed and indicate moderate-to-high levels of reliability and good evidence of criterion and construct validity[15]. It has a total score of 30, with higher scores indicating better cognitive performance. Disadvantages of the MMSE include a ceiling effect, the influence of education especially for the serials sevens component[15,16], and a documented learning effect[17].

Ethics

The procedures and rationale for the study were explained to all patients but because many patients had cognitive impairment at entry into the study it was presumed that many might not be capable of giving informed written consent. Because of the non-invasive nature of the study, Sligo Regional Ethics Committee approved an approach to establishing consent by virtue of augmenting patient assent with proxy consent from next of kin (where possible) or a responsible caregiver for all participants in accordance with the Helsinki Guidelines for Medical research involving human subjects.

Statistical analysis

Statistical analyses were conducted using the R “equate” package[18]. Z scores were used to compare MMSE and MoCA scores because although they are from the same sample they follow different distributions. The overall agreement between the two scales was assessed using Pearson’s product-moment correlation coefficient (r). However this estimation has been criticised by Bland et al[19] as misleading and therefore the concordance correlation coefficient (CCC) was also calculated[20]. The CCC measures agreement by assessing how well the relationship between the measurements is represented by a line through the origin at an angle of 45 degrees (as would be generated if the two measurements generated identical results).

To convert MoCA scores to MMSE (and vice versa), we generated an equating table to link the two scales. The conversions were extracted from a random population of the studied group and then tested in the remaining sub-population. Given that both scales have the same lower and higher scores but with different difficulty we used the Circle‐Arc Method[21]. However, we also applied other methods of equating models (linear, mean and equipercentile) and compared the standard errors of each model after bootstrapping (Figure 1).

Figure 1 Graphical representation of standard errors after bootstrapping of different equating methods.

By doing this we made the following assumptions: (1) that both scales measure the same latent construct (cognition); (2) that the two scales are not free of errors but the errors are small (both scales must have high reliability); and (3) that the ratings have been conducted by experts and the conversion rule will apply again in measurements that have been performed by experts.

Although both scales are continuous they are discretized continuous meaning that for a person A the score in MoCa (or MMSE) will be for example 11 and never 11.2 and thus the delivered score in MMSE was converted to the nearest integer. Finally, in the second sample we evaluated the conversion methods suggested by (1) Roalf et al[12]; (2) van Steenoven et al[8]; and (3) Trzepacz et al[13] using Pearson’s r and CCC to measure the agreement between the original scores and the converted scores.

RESULTS

The total sample (n = 135) was randomly divided in two groups; one which was used to derive the equating table (called experimental sample, n = 70) and a second evaluation sample (n = 65) in which the derived conversion rule was tested.

Descriptive statistics

Table 1 shows demographic data as well as the MoCA and MMSE scores in the two samples. The two samples did not significantly differ regarding gender distribution (χ² = 0.084, df: 1, P = 0.772), age (t = 1.25, df: 133, P = 0.214), MoCA scores (t = 0.406, df: 133, P = 0.686), MMSE scores (t = 0.31, df: 133, P = 0.976) and years of education (t = 1.29, df: 133, P = 0.200). In addition, Table 2 shows the principal diagnoses in the two samples in percentages. A comparison of the two samples in terms of diagnoses did not identify significant difference (χ² = 0.644, df: 3, P = 0.886). However, as shown in Table 1 the total MoCA scores were significantly lower than the total MMSE scores in both samples (For experimental sample: n = 70, z scores: -4.77, -8.19 respectively for MMSE and MoCA; P < 0.001; for the evaluation sample, n = 65, z-scores: -4.35, -7.88; P < 0.001).

Table 1 Demographic characteristics and cognitive test scores of the two samples.

	Experimental n = 70range (min-max)		Evaluation n = 65range (min-max)
Males	21 (30%)		21 (32.3%)
Age	77.36 (SD: 7.06)	62-89	78.83 (SD: 6.6)	66-91
MoCA	19.03 (SD: 6.35)	4-29	18.57 (SD: 6.78)	4-30
MMSE	24.47 (SD: 4.87)	9-30	24.45 (SD: 4.71)	8-30
Years of education	10.71 (SD: 2.47)	6-18	10.20 (SD: 2.15)	7-17

MoCA: Montreal cognitive assessment; MMSE: Mini-mental state examination.

Table 2 Main diagnoses in the two samples.

Diagnoses	Experimental sample			Evaluation sample			Total
	n (%)	MMSE mean (SD)	MoCA mean (SD)	n (%)	MMSE mean (SD)	MoCA mean (SD)
Organic, including symptomatic, mental disorders (F00-F09)	32 (45.7)	22.31 (4.95)	15.93 (5.68)	32 (49.2)	21.75 (4.61)	13.91 (5.11)	64
Schizophrenia, schizotypal and delusional disorders (F20-F29)	4 (5.7)	23.5 (3.7)	18.25 (4.43)	2 (3.1)	26 (5.66)	20 (7.07)	6
Mood (affective) disorders (F30-F39)	18 (25.7)	26.06 (4.42)	21.61 (6.29)	17 (26.2)	26.17 (3.46)	22.29 (5.47)	35
Neurotic, stress-related and somatoform disorders (F40-F48)	16 (22.9)	27.25 (3.49)	22.5 (5.33)	14 (21.5)	28.28 (1.85)	24.5 (3.69)	30

MoCA: Montreal cognitive assessment; MMSE: Mini-mental state examination.

Agreement of the two scales in the experimental sample

The Pearson’s product-moment correlation coefficient for MoCA and MMSE was 0.86 (P < 0.001) which indicates very good agreement. However, the more conservative CCC was 0.57 (95%CI: 0.45-0.66), indicating a lower agreement between the two scales. Figure 2 depicts a scatterplot including a fitted linear line and a cubic. As evident from the scatterplot, the scores do not fit well to a linear model.

Figure 2 Linear Quadratic and Cubic relationship of montreal cognitive assessment and mini-mental state examination scores.

Linking the two scales (MoCA and MMSE)

The “circle-arc” method was used. Table 3 shows the conversion table. Also other equating methods were used but as expected the “circle-arc” had the least standard errors and less biases compared to the others. Figure 1 shows the standard error of the different methods after bootstrapping.

Table 3 Conversion table.

MoCA scores	MMSE scores
1	3
2	6
3	8
4	10
5	11
6	13
7	14
8	15
9	16
10	17
11	19
12	19
13	20
14	21
15	22
16	23
17	24
18	25
19	25
20	26
21	27
22	27
23	28
24	28
25	29
26	29
27	30
28	30
29	30
30	30

MoCA: Montreal cognitive assessment; MMSE: Mini-mental state examination.

Evaluation of the derived conversion

In the 2^nd sample (evaluation sample) we converted MoCA scores to MMSE scores according to the above table and then examined the agreement between the converted MMSE scores and the originals. The Pearson’s product-moment correlation coefficient was 0.89 (n = 65, P < 0.001) and the Lin’s CCC was 0.88 (95%CI: 0.82-0.92). Thus the converted MMSE scores from MoCA have a high level of agreement with the actual MMSE scores.

Evaluation of the other methods suggested

With the Roalf et al[12]’s method, the Pearson’s product-moment correlation coefficient was equal to 0.88 (n = 65, P < 0.001) and the CCC equal to 0.86 (CI: 0.79-0.81). With the van Steenoven et al[8]’s method the agreement between the converted and the actual MMSE scores was high with the Pearson’s product-moment correlation coefficient of 0.86 (n = 65, P < 0.001) and the CCC of 0.84 (CI: 0.76-0.90). Finally, using the method suggested by Trzepacz et al[13] the Pearson’s product-moment correlation coefficient was 0.85 (n = 65, P < 0.001) and the CCC was 0.82 (CI: 0.72-0.88). All three previously described conversion rules were inferior to that derived herein.

DISCUSSION

It is often assumed that because the MoCA and MMSE measure the same general construct (cognition) that they can be used interchangeably. However, they each emphasise different aspects of cognition and, as our results demonstrate, their agreement is modest. For instance, the MMSE allocates more points for orientation (10 out of 30) compared to only 6 out of 30 in the MoCA, while the MoCA places greater emphasis on visuospatial domains (5 out of 30) compared to only 1 point out of 30 with the MMSE. As a consequence, it is not surprising that these two tests do not have a linear relationship (Figure 2). Furthermore because performance is more difficult in visuospatial and executive domains than orientation, scores in MoCA were significantly lower compared to scores in MMSE. In addition, although both tests are used as continuous scales (ranging from 0 to 30) in fact neither is a true ratio scale such that a score of 10 does not indicate half the cognitive ability of a score of 20. Similarly, both scales include arbitrary anchor points (e.g., a score of 0 does not mean that someone has no cognitive function at all).

Our second aim was to derive an equating rule to allow for accurate conversion of scores between the two scales. This has important utility for standardising multiple assessments of patients who are assessed using different scales over time. However, most importantly this conversion rule can allow for comparisons between multiple centers in clinical trials which use MoCA or MMSE alternately and can be used for pooling data from different studies to facilitate meta-analyses. Our conversion rule compared very favourably with those described in previous studies in terms of a better (higher agreement). We examined this issue using both Pearson’s correlation coefficients as well as the CCC which provides a more conservative method as, in comparison to Pearson’s correlation coefficient, it emphasises level of actual agreement over the general pattern of relationship[19]. Of note, the new method described herein performed better than previous methods by both measures of agreement (Pearson’s r or CCC). One explanation for these findings is that our sample is more representative of a general old age population in comparison to the three previous studies in which the samples were more restricted. However, one of the assumptions for equating methods is that the equating relationship is group invariant and as such does not change across the groups[22], if the sample or sampling method influenced the converted scores the conversion rule is not valid.

Although the sample can influence some psychometric values, the most likely explanation for the higher agreement is the equating method that we used as it provides a better fit for our data. The circle-arc which we used does not requires the estimated equating transformation to be linear. It constrains the end points in the two pre-specified end points and a middle point determined from the data and it is thus robust even for small samples[21]. In addition, the circle-arc method produces the most accurate results for different sample sizes compared to other methods like equipercentile with smoothing, linear equating, and mean equating[23]. Therefore, it is likely that the greater accuracy of the conversion rule described herein relates substantially to the methods that were used in its development rather than to the sampling method. These observations are further supported by Armstrong et al[7] who found a moderate agreement between the converted and actual scores when they applied the conversion rule suggested by van Steenoven et al[8], even though they tested the rule in a similar sample to that in which the rule was originally derived (i.e., patients with Parkinson’s disease). However, when the two scales or tests are different in content, reliability, or intended population, it is expected that the scales will be less equivalent to some degree[24], but this is not the case for the MoCA and MMSE as they both have high reliability, assumes that measure the same construct, (cognition) and are used in populations with possible cognitive deficits.

In conclusion, we found that the MMSE and MoCA have moderate agreement when used to assess general cognitive function reflecting their different emphasis into particular neuropsychological domains. Further, we found that their relationship is non-linear such that non-linear methods of equating should be used to compare performance on these scales. Finally, we derived a conversion rule which performed well in comparison to previously suggested methods and which merits further assessment in other larger and clinically diverse samples.

COMMENTS

Background

Mini-mental state examination (MMSE) and montreal cognitive assessment (MoCA) are widely used assessments of cognition in older people populations.

Research frontiers

Given the different tests of cognition the challenge is how to interpret them to a common “language”.

Innovations and breakthroughs

This study applies advance and robust techniques to overcome the above challenges.

Applications

The authors have derived an equation rule to convert the scores from MoCA to MMSE which can be used to pull together data from different studies.

Terminology

Equation models can be used to transform the scores from one scale or instruments to another.

Peer-review

The manuscript is a generally well-written and interesting paper. The topic is important because the rate of dementias increases around the world.