Normative Data for the Symbol Digit Modalities Test in Older White Australians and Americans, African-Americans, and Hispanic/Latinos

Ryan, Joanne; Woods, Robyn L.; Britt, Carlene J.; Murray, Anne M.; Shah, Raj C.; Reid, Christopher M.; Wolfe, Rory; Nelson, Mark R.; Orchard, Suzanne G.; Lockery, Jessica E.; Trevaks, Ruth E.; Storey, Elsdon; behalf of the ASPREE Investigator Group on

doi:10.3233/ADR-200194

Normative Data for the Symbol Digit Modalities Test in Older White Australians and Americans, African-Americans, and Hispanic/Latinos

Article type: Research Article

Authors: Ryan, Joanne^{a; *} | Woods, Robyn L.^a | Britt, Carlene J.^a | Murray, Anne M.^b | Shah, Raj C.^c | Reid, Christopher M.^{a; d} | Wolfe, Rory^a | Nelson, Mark R.^{a; e} | Orchard, Suzanne G.^a | Lockery, Jessica E.^a | Trevaks, Ruth E.^a | Storey, Elsdon^a | behalf of the ASPREE Investigator Group on¹

Affiliations: [a] School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia | [b] Berman Center for Outcomes and Clinical Research, Hennepin Health Research Institute and Division of Geriatrics, Department of Medicine, Hennepin HealthCare and University of Minnesota, MN, USA | [c] Department of Family Medicine and Rush Alzheimer’s Disease Center, Rush University Medical Center, Chicago, IL, USA | [d] School of Public Health, Curtin University, Perth, Western Australia, Australia | [e] Menzies Institute for Medical Research, University of Tasmania, Hobart, Australia

Correspondence: [*] Correspondence to: Joanne Ryan, ASPREE, School of Public Health & Preventive Medicine, Monash University, 99 Commercial Road, Melbourne, 3004 Victoria, Australia. Tel.: +61 399030200; E-mail: joanne.ryan@monash.edu.

Note: [1] ASPREE Investigator Group listed on http://www.aspree.org

Keywords: Aging, cognition, impairment, normative data, Symbol Digit Modalities Test

DOI: 10.3233/ADR-200194

Journal: Journal of Alzheimer's Disease Reports, vol. 4, no. 1, pp. 313-323, 2020

Accepted 7 July 2020

Published: 04 August 2020

Get PDF

Supplementary Materials:

Supplementary Material

Abstract

Background:

Processing speed, which can be assessed using the Symbol Digit Modalities Test (SDMT), is central to many brain functions. Processing speed declines with advanced age but substantial impairments are indicative of brain injury or disease.

Objective:

The purpose of this study was to provide SDMT normative data for older community-dwelling individuals in the U.S. and Australia.

Methods:

The ASPREE trial recruited 19,114 relatively healthy older men and women in Australia and the U.S. from the general community. All participants were without a diagnosis of dementia and with a Modified Mini-Mental State examination score of 78 or more at enrolment. The SDMT was administered at baseline as part of a neuropsychological test battery.

Results:

The median age of participants was 74 years (range 65–99), and 56% were women. The median years of education was 12. Ethno-racial differences in SDMT performance were observed and normative data were thus presented separately for 16,289 white Australians, 1,082 white Americans, 891 African-Americans, and 316 Hispanic/Latinos. There were consistent positive associations found between SDMT and education level, and negative associations between SDMT and age. Mean scores for women were consistently higher than men with the exception of Hispanic/Latinos aged ≥70 years.

Conclusion:

This study provides comprehensive SDMT normative data for whites (Australian and U.S.), Hispanic/Latinos, and African-Americans, according to gender, age, and education level. These norms can be used clinically as reference standards to screen for cognitive impairments in older individuals.

INTRODUCTION

Processing speed is defined as how fast information can be processed and generally assessed as the time needed to carry out a cognitive task. It is central to many brain functions and one of the most basic capacities underlying performance on neurocognitive assessment [1]. It reflects the speed at which an individual can identify, integrate, and understand information, and react or respond to this information [2, 3].

Driven by age-related changes in brain structure and function [4], processing speed tends to slow with advancing age [5, 6], and can impact the daily lives of older individuals [7]. More substantial decline, however, is considered a sign of brain injury or disease [8–14]. Impaired processing speed is observed with traumatic brain injury [10] and various diseases including multiple sclerosis [8], stroke [14], Huntington’s disease [13], Parkinson’s disease [11], and Alzheimer’s disease [9, 12]. Slowed processing speed is used as a marker of brain structural integrity associated with white matter signal abnormalities [15–17].

The Symbol Digit Modalities Test (SDMT) is a commonly used test to assess psychomotor speed, which measures processing speed as well as motor speed [1]. It is a paper-pencil measure which requires an individual to substitute digits for abstract symbols using a reference key [18, 19]. Performance is also affected by attention, visual scanning and tracking, and working memory [20, 21]. The SDMT has excellent psychometric properties, with high reliability [22] and validity [23]. Test performance has been shown to be highly sensitive to a range of neuropsychiatric disorders and neurologic conditions [24–27]. Performance on the SDMT is a significant predictor of conversion from cognitively normal to mild cognitive impairment (MCI) [28]. The brevity and ease of administration of the SDMT, together with its unambiguous scoring, make the SDMT a test of choice in many neuropsychological batteries and these factors have contributed to its widespread use.

Reliable normative data for neuropsychological tests is essential to their utility as effective screening tools in clinical practice. It enables the clinician to assess whether an individual’s performance falls within the ‘normal’ range or not, the latter case being an indication of more serious cognitive impairments warranting further investigation. To be accurate, however, such normative data must consider the diversity of the population and factors that can influence test performance in the absence of any cognitive deficits.

The SDMT test manuals report normative data for individuals with various disorders [18, 19]. Subsequent studies have generated normative data for non-patient samples [26, 29, 30]. Across these studies there is good evidence that, when socio-demographic characteristics are considered, test performance is influenced by age and education level, and possibly gender, although findings have not always been consistent [31–33]. Ethnic and cultural differences have also been shown to influence SDMT performance, even after accounting for variations in other demographic factors [34–36]. For example, a matched sample of Russian and American adults found that cultural differences in time attitudes were associated with differences in performance on timed neuropsychological tests, including the SDMT [34]. Another study comparing Non-Latino whites and African-American in the U.S. also found differences in SDMT performance which were not explained by sociodemographic factors [36]. Indeed, it is now well recognized that ethnicity and more broadly culture have an influence on neuropsychological test performance, including psychomotor speed [37–40].

There is a need for high-quality normative data in older community-dwelling individuals which considers not only cultural (country) and ethic differences, but the influence of age, education, and gender. The need for such data is increasingly pertinent given the aging population [41] and the absence of existing normative data from large community samples of older individuals.

The primary purpose of this study is to provide SDMT normative data for older community-dwelling individuals, considering age, gender, and education. Given the importance of cultural differences across racial and ethnic groups, all data are presented separately for Australians and U.S. whites, African-Americans, and Hispanic/Latinos.

METHODS

Participants

This analysis uses baseline data gathered from the ASPREE study cohort. ASPREE (ASPirin in Reducing Events in the Elderly) was a double-blind, placebo-controlled trial of low-dose aspirin in initially healthy older individuals [42]. Full details regarding the study design, including participant recruitment and visit procedures, have been described in detail previously [43, 44]. In brief, from 2010 to 2014 community-dwelling individuals aged 70 years and over were recruited in the U.S. through clinical-based mailing lists, electronic medical screening, and media advertisements; and through partnership with over 2000 general practitioners (GPs) in Australia [45]. In the U.S., the age limit was lowered to 65 years for African-Americans and Hispanic/Latinos reflecting their higher burden of diseases for which aspirin was indicated (e.g., aspirin) and overall survival disadvantage [46]. The criteria for eligibility to the study included being free of physical disability and established cardiovascular disease, and being expected to survive for at least five years (the duration of the trial). Ineligible individuals had a self-report or physician diagnosis of dementia, or a low score on the Modified Mini-Mental State examination (3MS, <78/100) [47].

The ASPREE study followed the Code of Federal Regulations as it relates to areas of clinical research. It was conducted in accordance with the Declaration of Helsinki 1964 as revised in 2008, the National Health & Medical Research Council (NHMRC) Guidelines on Human Experimentation, the federal patient privacy (HIPAA) law and ICH-GCP guidelines, and the International Conference of Harmonisation Guidelines for Good Clinical Practice. All participants in ASPREE provided written informed consent.

Procedure and instruments

An alternative form of the written version of Symbol Digit Modalities Test (SDMT), Form C, was used. This has been validated previously [48, 49]. It was administered by trained and accredited staff as part of the baseline assessment which also included detailed questionnaires, clinical examinations and physical measures. Almost all participants completed all cognitive testing in English (n = 19,057); however, a small number (n = 57) reported Spanish as their primary language, and they were thus given the verbal test instructions in Spanish.

The SDMT is a paper-pencil measure which involves a substitution task using a coding key with nine different abstract symbols, each paired with a numeral. Below the key, a series of these symbols was presented and the participant was asked to write down the corresponding number for each symbol. Participants initially completed a practice session of 10 items, during which time staff corrected any errors made by explaining the substitution of the symbol with a numeral with reference to the key. Participants then undertook their test which consisted of completing as many of 110 items as they could in 90 s. The number of correct substitutions/responses within this time was recorded as their score.

The SDMT has confirmed content and construct validity [20] and good test-retest reliability in healthy adults [50]. To help ensure consistency from all staff members, administration of the SDMT was strictly monitored. In the U.S., staff were trained by qualified Coordinating Center staff supervised by geriatrician study investigators (J.W., A.M). In Australia, staff were trained by a neurologist study investigator (E.S). Staff were re-accredited annually in both countries.

Statistical analysis

Table 1

Characteristics of participants overall and by country

Characteristic	All participants		Australia		U.S.
	N ¹	%	N	%	N	%
All participants	18 578		16 289		2 289
Ethno-racial groups
white Australian	16 289	87.7	16 289	100
white U.S.	1 082	5.8			1 082	47.3
African American U.S.	891	4.8			891	38.9
Hispanic/Latino U.S.	316	1.7	316	13.8
Gender
Female	10 512	56.7	8 989	55.2	1 523	66.5
Male	8 066	43.4	7 300	44.8	766	33.5
Age, y
65–69²	535	2.9	535	23.4
70–74	10 274	55.3	9 403	57.7	871	38.1
75–79	4 893	26.3	4 327	26.6	566	24.7
80–85+	2 876	15.5	2 559	15.7	317	13.9
Education level, y
<9	2 914	15.7	2 829	17.4	85	3.7
9–11	5 518	29.7	5 409	33.2	109	4.8
12	2 246	12.1	1 764	10.8	482	21.1
13–15	3 160	17.0	2 486	15.3	674	29.5
16	1 712	9.2	1 297	8.0	415	18.1
≥17	3 028	16.3	2 504	15.4	524	22.9

N. number. ¹The ASPREE cohort comprised 19,114 individuals. Not included in these tables are individuals who did not provide their education level or race/ethnicity (n = 2) or did not undertake the SDMT (n = 83), individuals whose primary language was Spanish (n = 57) and individuals in another ethno-racial group (n = 394). ²African American and Hispanic/Latino U.S. participants only, given eligibility criteria for inclusion in ASPREE.

Mean (SD) scores on the SDMT were calculated across the major ethno-racial groups; whites (separately for participants in Australia and the U.S.), African-Americans, and Hispanic/Latinos. The numbers of individuals in other ethno-racial groups including Asians, Aboriginal and Torres Strait Islanders, and American Indians, was too small to permit normative data to be generated. This was also the case for the small number of individuals who completed the test in Spanish, and they were thus excluded from the current normative data.

To determine whether age, education, and gender were associated with SDMT scores, multivariable linear regression was used, stratifying by ethno-racial group. Age was treated as a continuous variable but the number of years of education was categorized, given that the data was collected this way. Standard model assumptions including homogeneity of variance and approximately normal distribution of the residuals were confirmed as valid. A quadratic age term was added to the models to account for possible non-linear association between age and SDMT scores. Standard regression model assumptions of homogeneity of variance and approximately normal distribution of the residuals were examined.

Ethno-racial specific normative data were generated according to age (65–69 years for U.S. ethnic minorities only, 70–74 years, 75–79 years, 80+ years), gender (men/women), and education categories (which ranged from a self-report of less than 9 years to more than 16 years). For the U.S. participants, categories were collapsed to ensure that no individual cell contained fewer than 20 participants. Stata version 15.0 (StataCorp, College Station, USA) was used for all statistical analyses.

RESULTS

This study sample comprised 97.2% of participants from ASPREE, including 16,289 whites in Australia and 2,289 participants in the U.S. (47.3% white, 38.9% African-American, and 13.8% Hispanic/Latinos). Table 1 provides the overall characteristics of the cohort and separately for Australian and U.S. participants. Individuals ranged in age from 65 to 99 years (median 74, mean 75), and more than half were women. The number of years of completed education varied. In Australia, the highest proportion of participants reported 9 to 11 years of education, whereas in the U.S. it was 13 to 15 years. Full details regarding baseline health characteristics of the ASPREE participants has been published previously [44].

SDMT scores across the entire cohort were approximately normally distributed (Supplementary Figure 1). The summary statistics for SDMT according to the demographic characteristics of the study participants are shown in Table 2. Overall the mean SDMT score was 36.8 (SD 10.2), the mean varied from 31.5 in the oldest age group (≥80 years) and the lowest education level (<9 years), to 41.0 among individuals with ≥17 years of education.

Table 2

Summary statistics for the Symbol Digit Modalities Test (SDMT) according to participant’s characteristics

Characteristic	Mean	S.D.	Range
All participants	36.8	10.2	1 to 97
Ethno-racial groups
white Australians	36.7	10.0	2 to 97
white U.S.	40.9	9.6	3 to 66
African American U.S.	33.9	11.4	2 to 86
Hispanic/Latino U.S.	34.9	10.7	1 to 71
Gender
Female	37.8	10.2	1 to 96
Male	35.4	9.9	2 to 97
Age, y
65–69	37.2	11.2	9 to 86
70–74	38.9	9.7	4 to 90
75–79	35.5	9.8	6 to 97
80–85+	31.5	9.8	1 to 88
Education level, y
<9	31.5	9.9	1 to 90
9–11	35.6	9.8	2 to 97
12	36.6	10.0	4 to 96
13–15	38.1	9.8	7 to 86
16	39.6	9.4	3 to 81
≥17	41.0	9.3	4 to 81

S.D., standard deviation.

The results of the multivariable linear regression model for SDMT, including all demographic variables together in the one model are shown in Supplementary Table 1. Age, education level, and gender were associated with cognitive performance. Significant differences in SDMT performance were also observed across the four ethno-racial groups; with white U.S. participants performing significantly better than white Australians, and both minority groups with lower performance. Given the likely cultural differences across these groups that influence performance, all subsequent analysis and normative data was stratified by ethno-racial group.

Consistent across all ethno-racial groups was a highly significant association between each of increasing age and fewer years of education, and lower performance on the SDMT (Tables 3 and 4). The effect sizes for age were fairly consistent across ethno-racial groups, ranging from –0.58 for Hispanic/Latinos to –0.70 for African-Americans. U.S. whites had the smallest effect size for education level (3.81, compared to 7.09 and 7.61 for Africa-American and Hispanic/Latinos respectively).

Table 3

Multivariable linear regression model of the association between demographic characteristics and Symbol Digit Modalities Test scores among Australian whites (n = 16,289)

	β (S.E.)	p
Age, y	–0.65 (0.02)	<0.0001
Male	Reference
Female	2.87 (0.15)	<0.0001
Education Level (y):
<9	Reference
9–11	3.56 (0.21)	<0.0001
1	4.95 (0.28)	<0.0001
13–15	6.01 (0.25)	<0.0001
16	7.00 (0.31)	<0.0001
≥17	8.73 (0.25)	<0.0001

β, beta-coefficient from the multivariable linear regression model; S.E., standard error.

Table 4

Multivariable linear regression model of the association between demographic characteristics and Symbol Digit Modalities Test scores among U.S. participants

	β (S.E.)	P
Whites (n = 1 082)
Age, y	–0.60 (0.06)	<0.0001
Male	Reference
Female	2.84 (0.59)	<0.0001
Education level, y
≤12	Reference
>12	3.81 (0.70)	<0.0001
African-Americans (n = 891)
Age, y	–0.70 (0.06)	<0.0001
Male	Reference
Female	4.27 (0.72)	<0.0001
Education level, y
≤12	Reference
>12	7.09 (0.73)	<0.0001
Hispanic/Latinos (n = 316)
Age, y	–0.58 (0.11)	<0.0001
Male	Reference
Female	1.99 (1.11)	0.08
Education level, y
≤12	Reference
>12	7.66 (1.11)	<0.0001

β, beta-coefficient from the multivariable linear regression model; S.E., standard error.

Table 5

Age, education, and gender specific reference values for SDMT in Australian whites (n = 16,289)

	Education<9 y		Education 9–11 y		Education 12 y		Education 13–15 y		Education 16 y		Education≥17 y
Age, y	Male	Female	Male	Female	Male	Female	Male	Female	Male	Female	Male	Female
70–74	31.9 (8.8)	35.9 (9.5)	35.6 (8.9)	39.5 (9.3)	36.6 (9.2)	40.9 (9.4)	38.6 (9.0)	41.7 (9.2)	40.9 (8.6)	41.7 (8.7)	42.2 (9.0)	43.0 (8.7)
n = 664	n = 774	n = 1 319	n = 1 785	n = 443	n = 558	n = 670	n = 780	n = 368	n = 437	n = 888	n = 717
75–79	28.9 (9.0)	32.5 (9.9)	32.5 (9.3)	36.0 (9.3)	34.5 (9.2)	37.0 (9.8)	35.8 (8.8)	36.9 (9.4)	36.7 (9.2)	38.2 (9.4)	39.2 (8.2)	40.0 (9.1)
n = 353	n = 462	n = 604	n = 890	n = 198	n = 267	n = 253	n = 390	n = 128	n = 195	n = 323	n = 264
80+	24.9 (7.7)	28.5 (9.6)	28.7 (8.3)	31.5 (9.9)	31.1 (8.4)	33.5 (9.7)	32.2 (8.3)	34.6 (10.0)	33.5 (8.4)	34.7 (8.3)	34.8 (8.8)	36.3 (9.5)
n = 238	n= 338	n = 287	n = 524	n = 122	n = 176	n = 184	n = 209	n = 83	n = 86	n = 175	n = 137

S.D., standard deviation; N, number; SDMT, Symbol Digit Modalities Test.

Among white participants from Australia and U.S., women performed significantly better than men. This association was consistent, although considerably stronger for African-Americans, but no significant difference between Hispanic/Latino men and women was found.

Normative data for SDMT in Australian whites is shown in Table 5, separately for men and women across six levels of education and three age groups. Women had consistently higher mean scores than men across all sub-categories. SDMT scores increased with more years of education and the pattern was very similar regardless of the age group. Percentile tables are also presented in Supplementary Table 2.

SDMT normative data for the U.S. participants is shown in Table 6. Data are presented across gender categories and two age and education groups, separately for white, African-American, and Hispanic/Latino participants. Women outperformed men across all of the sub-categories with the exception of Hispanic/Latinos with over 12 years of education, where men had higher mean scores. Similar to the Australian participants, mean SDMT scores were lower in the highest age group, and with ≤12 years of education. These patterns can also be seen in the percentile tables shown in Supplementary Table 3.

DISCUSSION

This study presents normative data for performance on the SDMT in a large sample of older community-dwelling individuals across four different cultural groups; separately for Australian and U.S. whites, as well as African-Americans and Hispanic/Latinos. Age, education level, and gender were all independently associated with SDMT performance, although some differences were observed across ethno-racial groups.

The SDMT is a multifactorial task that provides a valid measure of psychomotor speed, but also relies on other neuropsychological domains including attention, working memory [20, 21], and executive function [51]. Poor performance on the SDMT is not indicative of specific neurological conditions [23] but is highly sensitive to the presence of cognitive impairment. It is thus useful for clinical screening; however, high-quality normative data are required if the results of the test are to be used in a meaningful way.

Normative SDMT data are already available for individuals across a number of countries, including Italy, North America, Argentina, Brazil, Greece, Netherlands, Spain, France, Denmark, China, and Australia. However much of the data has been generated from small studies, commonly involving 200 individuals or less [11, 26, 31–33, 52–56], and this has prohibited the generation of demographic-specific norms. Furthermore, the studies that are available have generally focused on specific patient samples (e.g., multiple sclerosis [57–59]), or younger adults [26, 52, 56, 60, 61]. There has been a lack of large scale normative SDMT data for older individuals, and particularly that which has considered variation according to key demographic characteristics that are known to influence cognitive performance [33].

Table 6

Age, education, and gender specific reference values for SDMT in U.S. whites (n = 1,082), African Americans (n = 891) and Hispanic/Latinos (n = 316)

	Education 0–12 y		Education>12 y
	Male	Female	Male	Female
U.S. White
70–74	38.7 (8.0), n = 28	41.0 (8.9), n = 56	42.6 (8.6), n = 131	44.8 (8.6), n = 271
75–85+	31.4 (8.7), n = 30	37.5 (9.5), n = 88	37.8 (9.8), n = 155	40.5 (9.4), n = 323
U.S. African American
65–69	27.6 (9.3), n = 54	34.3 (10.8), n = 64	36.7 (11.4), n = 100	41.1 (10.6), n = 182
70–85+	25.2 (9.3), n = 60	27.6 (10.9), n = 105	31.7 (9.5), n = 87	34.5 (10.4), n = 239
U.S. Hispanic/Latino
65–69	31.9 (9.5), n = 28	35.1 (10.3), n = 47	40.9 (8.9), n = 23	43.5 (7.9), n = 37
70–85+	29.4 (10.0), n = 37	31.2 (10.6), n = 79	37.6 (7.2), n = 33	36.2 (10.9), n = 32

S.D., standard deviation; N, number; SDMT, Symbol Digit Modalities Test.

The largest prior SDMT normative data was from a nationally representative sample of 14,456 Australians aged from 15 to 100 years, which included 1,449 individuals aged 70 years or more [29]. Consistent with the findings of the current study, they reported that women performed better than men, with a very similar effect size to the current sample of white Australians (β: 3.09 versus 2.87). Likewise, similar negative linear associations between SDMT and age were found, as well as positive associations with education, although the latter association was not linear. While the mean SDMT scores in comparable groups appear remarkably alike, direct comparisons of the normative data across studies is difficult given that Kiely et al. were limited by their smaller sample size of older adults and thus only stratified education into two groups (≥12 years versus <12), compared to the six groups reported here for white Australians.

The importance of generating culturally specific normative data is well recognized, as differences in knowledge, attitudes, and beliefs across countries and ethno-racial groups can influence neuropsychological test performance [62, 63], including on the SDMT [34]. The findings of our study further add to this body of literature. In epidemiological studies of cognitive ageing using mixed samples (multiple countries and/or ethno-racial groups), careful consideration should thus be given as to whether it is more appropriate to stratify all analyses. Acknowledging the need for normative data for U.S. ethnic minorities, Gonzalez et al. conducted a nationally representative sample of 2,670 African Americans, 1,297 Caribbean Blacks, and 570 non-Latino Whites in the U.S. [36]. The mean age of participants was 42 years, and only 520 were over 65 years. They found that performance on a modified version of the SDMT was associated with age, education, and gender, with relatively consistent associations across the three ethnic groups. Another study of 3,977 healthy adults from 11 Latin American Spanish speaking countries reported country-specific norms according to age and education level. Gender-specific data was not provided given it was not strongly predictive of SDMT performance in most of the 11 countries [64]. Interestingly, this aligns with findings from the current study, where gender was not associated with SDMT performance in Hispanic/Latinos, but was in the other ethno-racial groups. The exact reason for this is unclear, but could be due to socio-cultural factors.

Other studies have reported mixed findings in terms of possible gender differences in SDMT performance [26, 29, 33, 36], which may be influenced by the age of the participants. A study of white adults across a wide age range found gender differences in the 20–24 years and 40–44 years age groups, but not in the 2,551 individuals in the oldest age group (60–64 years) [32]. In contrast, a study of 324 Mandarin speaking adults in China found that gender was associated with SDMT performance among the oldest age group (mean age of 69 years), but not in the younger age groups (mean age of 36 and 53 years). It may also be that the effects of gender are smaller than those of other demographic factors [64, 65], and thus only observed in studies that are sufficiently large [29, 53, 60]. Indeed, in this study the effect size for gender was considerably smaller than that for education level.

Age is thought to be an important factor influencing performance on the SDMT [58], but only once individuals reach a certain age in later adulthood. The prior community studies that have focused specifically on older individuals, including 1780 French aged 70 years and over [53]; 354 Spanish aged 50–90 [66]; and 151 Danish aged 64–83 [54], have reported significant negative associations between age and SDMT scores. This aligns with the findings of the current study, which were consistent across ethno-racial groups. In contrast, studies of younger adults have generally failed to find an association between age and test performance [26, 32, 56, 67]. This supports the notion that decline in psychomotor speed may only start to occur in later adulthood, driven by changes in brain structure and function in later life Kerchner et al. [4, 68].

Education has consistently been shown to influence performance on a number of neuropsychological tests and most previous studies do indeed support an effect of education on SDMT performance [29, 30, 33, 36, 52, 53, 64–66, 69, 70]. Despite this, not all studies have reported normative data according to educational attainment [26, 32, 54]. Education level is often used as a marker of cognitive reserve, which is the ability to tolerate age-related changes in the brain without developing clinical symptoms [71]. Individuals with lower education may not have had the same level of experiences, and thus not the same opportunities to acquire specific abilities which may contribute to poorer performance on the SDMT. Education level may also be a proxy measure of other socioeconomic factors that could impact brain development and functioning, for example poverty and early-life nutrition, and exposure to smoking and alcohol in utero. Higher-educated individuals generally perform better on a range of cognitive tests, which may reflect a rich environment or better intellectual functioning [72, 73]. The potential for bias in the cognitive assessments and the specific culture in which they have been developed [38, 74], must also be recognized as a potential explanation for the findings.

Differences in the recruitment procedure between Australia (through general practitioners) and the U.S. (through clinical-based mailing lists, electronic medical screening and media advertisements) is unlikely to be the sole explanation for the differences in education level between older white participants in the two countries [45]. Indeed, differences in education level for older individuals of this generation has been reported previously [75].

Limitations and strengths

Some features of the ASPREE sample will limit the generalizability of this normative data. Under representation of other ethnic/racial groups within Australia (including indigenous persons and Asians), meant that normative data could not be generated for these peoples. Recruitment to the ASPREE trial required individuals to be relatively healthy, without cardiovascular disease or other major health conditions likely to limit survival to less than five years. Participants were without a diagnosis of dementia at recruitment, and with a modified Mini-Mental State Examination score above 77, meaning those with severe cognitive impairments were not included. However, performance on other cognitive tests was not part of the inclusion criteria, and it is therefore possible that there is an admixture of mild cognitive disorders among the participants [76]. Additional limitations include the number of different staff at different sites administering the assessment. To counter for this, staff underwent extensive initial training, including accreditation by a neuropsychologist, and were monitored for quality control and retrained on an annual basis. Finally, if there were individuals with severe visual impairment in the population, this would have had a negative impact on their performance on the SDMT, and this has not been considered in the analysis.

The inclusion of different ethno-racial groups combined with standardized test application in a large sample is a strength of this study. By presenting data according to gender, as well as age and education groups, this provides a broader representation of the older population and will permit clinicians to use appropriate reference standards that align with the demographic characteristics of their patients. Indeed, this study provides SDMT normative data from the largest study of older white Australians, white U.S., African-American and Hispanic/Latinos.

Conclusion

The SDMT is a relatively quick test that is easy to administer and score, and provides a useful measure of psychomotor speed across different participant groups, including clinical samples. This study conducted in the largest older population to date, shows that gender, age, and education level are all independent factors that influence performance on the SDMT, but the strength of associations varies across ethno-racial groups. The current study provides an important contribution to the field and will facilitate further application of the SDMT in the clinical and research setting. By providing normative data for the SDMT according to key demographic characteristics and separately for several ethno-racial groups, this will provide a useful resource to screen for older individuals with low or atypical cognitive function. These findings highlight the importance of using appropriate normative data which are relevant to the population being examined. It also emphasizes the importance of adjusting for these socio-demographic factors in epidemiological studies focused on cognitive aging.

CONFLICT OF INTEREST

The authors have no conflict of interest to report.

ACKNOWLEDGMENTS

A. G. Bayer provided aspirin and matching placebo. The authors acknowledge the dedicated and skilled staff in Australia and the Unites States for the conduct of the trial. The authors also are most grateful to the ASPREE participants, who so willingly volunteered for this study, and the general practitioners and medical clinics who support the participants in the ASPREE study. Trial Registration: International Standard Randomized Controlled Trial Number Register (ISRCTN83772183) and clinicaltrials.gov (NCT01038583).

This work was supported by the National Institute on Aging and the National Cancer Institute at the National Institutes of Health [grant number U01AG029824]; the National Health and Medical Research Council (NHMRC) of Australia [grant numbers 334047 and 1127060]; Monash University (Australia); the Victorian Cancer Agency (Australia). JR is funded by a NHMRC Dementia Research Leader Fellowship [APP1135727] and CMR by a NHMRC Principal Research Fellowship [APP1136372].

SUPPLEMENTARY MATERIAL

[1] The supplementary material is available in the electronic version of this article: https://dx.doi.org/10.3233/ADR-200194.

REFERENCES

[1]	Ebaid D , Crewther SG , MacCalman K , Brown A , Crewther DP ((2017) ) Cognitive Processing speed across the lifespan: Beyond the influence of motor speed. Front Aging Neurosci 9: , 62.
[2]	Birren JE , Fisher LM ((1995) ) Aging and speed of behavior: Possible consequences for psychological functioning. Annu Rev Psychol 46: , 329–353.
[3]	Magistro D , Takeuchi H , Nejad KK , Taki Y , Sekiguchi A , Nouchi R , Kotozaki Y , Nakagawa S , Miyauchi CM , Iizuka K , Yokoyama R , Shinada T , Yamamoto Y , Hanawa S , Araki T , Hashizume H , Sassa Y , Kawashima R ((2015) ) The relationship between processing speed and regional white matter volume in healthy young people. PLoS One 10: , e0136386.
[4]	Kerchner GA , Racine CA , Hale S , Wilheim R , Laluz V , Miller BL , Kramer JH ((2012) ) Cognitive processing speed in older adults: Relationship with white matter integrity. PLoS One 7: , e50425.
[5]	Cerella J , Hale S ((1994) ) The rise and fall in information-processing rates over the life span. Acta Psychol (Amst) 86: , 109–197.
[6]	Jenkins L , Myerson J , Joerding JA , Hale S ((2000) ) Converging evidence that visuospatial cognition is more age-sensitive than verbal cognition. Psychol Aging 15: , 157–175.
[7]	Edwards JD , Bart E , O’Connor ML , Cissell G ((2010) ) Ten years down the road: Predictors of driving cessation. Gerontologist 50: , 393–399.
[8]	Benedict RH , DeLuca J , Phillips G , LaRocca N , Hudson LD , Rudick R ((2017) ) Validity of the Symbol Digit Modalities Test as a cognition performance outcome measure for multiple sclerosis. Mult Scler 23: , 721–733.
[9]	Duke Han S , Nguyen CP , Stricker NH , Nation DA ((2017) ) Detectable neuropsychological differences in early preclinical Alzheimer’s disease: A meta-analysis. Neuropsychol Rev 27: , 305–325.
[10]	Dymowski AR , Owens JA , Ponsford JL , Willmott C ((2015) ) Speed of processing and strategic control of attention after traumatic brain injury. J Clin Exp Neuropsychol 37: , 1024–1035.
[11]	Konstantopoulos K , Vogazianos P , Doskas T ((2016) ) Normative data of the Montreal Cognitive Assessment in the Greek population and parkinsonian dementia. Arch Clin Neuropsychol 31: , 246–253.
[12]	Kuate-Tegueu C , Avila-Funes JA , Simo N , Le Goff M , Amieva H , Dartigues JF , Tabue-Teguo M ((2017) ) Association of gait speed, psychomotor speed, and dementia. J Alzheimers Dis 60: , 585–592.
[13]	Maroof DA , Gross AL , Brandt J ((2011) ) Modeling longitudinal change in motor and cognitive processing speed in presymptomatic Huntington’s disease. J Clin Exp Neuropsychol 33: , 901–909.
[14]	Su CY , Wuang YP , Lin YH , Su JH ((2015) ) The role of processing speed in post-stroke cognitive dysfunction. Arch Clin Neuropsychol 30: , 148–160.
[15]	Jacobs HI , Leritz EC , Williams VJ , Van Boxtel MP , van der Elst W , Jolles J , Verhey FR , McGlinchey RE , Milberg WP , Salat DH ((2013) ) Association between white matter microstructure, executive functions, and processing speed in older adults: The impact of vascular health. Hum Brain Mapp 34: , 77–95.
[16]	Jokinen H , Ryberg C , Kalska H , Ylikoski R , Rostrup E , Stegmann MB , Waldemar G , Madureira S , Ferro JM , van Straaten EC , Scheltens P , Barkhof F , Fazekas F , Schmidt R , Carlucci G , Pantoni L , Inzitari D , Erkinjuntti T ((2007) ) Corpus callosum atrophy is associated with mental slowing and executive deficits in subjects with age-related white matter hyperintensities: The LADIS Study. J Neurol Neurosurg Psychiatry 78: , 491–496.
[17]	Lu PH , Lee GJ , Tishler TA , Meghpara M , Thompson PM , Bartzokis G ((2013) ) Myelin breakdown mediates age-related slowing in cognitive processing speed in healthy elderly men. Brain Cogn 81: , 131–138.
[18]	Smith A ((1968) ) The Symbol-Digit Modalities Test: A neuropsychologic test of learning and other cerebral disorders. Learning Disorders, Special Child Publications, In, Helmuth J , ed., Seattle, pp. 83–91.
[19]	Smith A ((1982) ) Symbol Digits Modalities Test, Western Psychological Services, Los Angeles.
[20]	Lezak M , Howieson D , Loring D ((2004) ) Neuropsychological Assessment, Oxford University Press, New York.
[21]	Schear JM , Sato SD ((1989) ) Effects of visual acuity and visual motor speed and dexterity on cognitive test performance. Arch Clin Neuropsychol 4: , 25–32.
[22]	Hinton-Bayre A , Geffen G ((2005) ) Comparability, reliability, and practice effects on alternate forms of the Digit Symbol Substitution and Symbol Digit Modalities tests. Psychol Assess 17: , 237–241.
[23]	Strauss E , Sherman EM , Spreen O ((2006) ) A compendium of neuropsychological tests: Administration, norms, and commentary,Oxford University Press, USA..
[24]	Elahipanah A , Christensen BK , Reingold EM ((2011) ) What can eye movements tell us about Symbol Digit substitution by patients with schizophrenia? . Schizophr Res 127: , 137–143.
[25]	Langdon DW , Amato MP , Boringa J , Brochet B , Foley F , Fredrikson S , Hamalainen P , Hartung HP , Krupp L , Penner IK , Reder AT , Benedict RH ((2012) ) Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult Scler 18: , 891–898.
[26]	Sheridan LK , Fitzgerald HE , Adams KM , Nigg JT , Martel MM , Puttler LI , Wong MM , Zucker RA ((2006) ) Normative Symbol Digit Modalities Test performance in a community-based sample. Arch Clin Neuropsychol 21: , 23–28.
[27]	Waldron-Perrine B , Kisser JE , Brody A , Haacke EM , Dawood R , Millis S , Levy P ((2018) ) MRI and neuropsychological correlates in African Americans with hypertension and left ventricular hypertrophy. Am J Hypertens 31: , 865–868.
[28]	Cherbuin N , Sachdev P , Anstey KJ ((2010) ) Neuropsychological predictors of transition from healthy cognitive aging to mild cognitive impairment: The PATH through life study. Am J Geriatr Psychiatry 18: , 723–733.
[29]	Kiely KM , Butterworth P , Watson N , Wooden M ((2014) ) The Symbol Digit Modalities Test: Normative data from a large nationally representative sample of Australians. Arch Clin Neuropsychol 29: , 767–775.
[30]	Nocentini U , Giordano A , Di Vincenzo S , Panella M , Pasqualetti P ((2006) ) The Symbol Digit Modalities Test - Oral version: Italian normative data. Funct Neurol 21: , 93–96.
[31]	Hsieh SL , Tori CD ((2007) ) Normative data on cross-cultural neuropsychological tests obtained from Mandarin-speaking adults across the life span. Arch Clin Neuropsychol 22: , 283–296.
[32]	Jorm AF , Anstey KJ , Christensen H , Rodgers B ((2004) ) Gender differences in cognitive abilities: The mediating role of health state and health habits. Intelligence 32: , 7–23.
[33]	Vanotti S , Cores EV , Eizaguirre B , Angeles M , Rey R , Villa A , Caceres F ((2015) ) Normatization of the symbol digit modalities test-oral version in a Latin American country. Appl Neuropsychol Adult 22: , 46–53.
[34]	Agranovich AV , Panter AT , Puente AE , Touradji P ((2011) ) The culture of time in neuropsychological assessment: Exploring the effects of culture-specific time attitudes on timed test performance in Russian and American samples. J Int Neuropsychol Soc 17: , 692–701.
[35]	Cores EV , Vanotti S , Eizaguirre B , Fiorentini L , Garcea O , Benedict RH , Cáceres F ((2015) ) The effect of culture on two information-processing speed tests. Appl Neuropsychol Adult 22: , 241–245.
[36]	Gonzalez HM , Whitfield KE , West BT , Williams DR , Lichtenberg PA , Jackson JS ((2007) ) Modified-Symbol Digit Modalities Test for African Americans, Caribbean Black Americans, and non-Latino Whites: Nationally representative normative data from the National Survey of American Life. Arch Clin Neuropsychol 22: , 605–613.
[37]	Era P , Berg S , Schroll M ((1995) ) Psychomotor speed and physical activity in 75-year-old residents in three Nordic localities. Aging Clin Exp Res 7: , 195–204.
[38]	Fernández AL , Abe J ((2018) ) Bias in cross-cultural neuropsychological testing: Problems and possible solutions. Culture Brain 6: , 1–35.
[39]	Fernández AL , Marcopulos BA ((2008) ) A comparison of normative data for the Trail Making Test from several countries: Equivalence of norms and considerations for interpretation. Scand J Psychol 49: , 239–246.
[40]	Ojeda N , Aretouli E , Peña J , Schretlen DJ ((2016) ) Age differences in cognitive performance: A study of cultural differences in historical context. J Neuropsychol 10: , 104–115.
[41]	Kontis V , Bennett JE , Mathers CD , Li G , Foreman K , Ezzati M ((2017) ) Future life expectancy in 35 industrialised countries: Projections with a Bayesian model ensemble. Lancet 389: , 1323–1335.
[42]	McNeil JJ , Woods RL , Nelson MR , Reid CM , Kirpach B , Wolfe R , Storey E , Shah RC , Lockery JE , Tonkin AM , Newman AB , Williamson JD , Margolis KL , Ernst ME , Abhayaratna WP , Stocks N , Fitzgerald SM , Orchard SG , Trevaks RE , Beilin LJ , Donnan GA , Gibbs P , Johnston CI , Ryan J , Radziszewska B , Grimm R , Murray AM ((2018) ) Effect of aspirin on disability-free survival in the healthy elderly. N Engl J Med 379: , 1499–1508.
[43]	ASPREE IG ((2013) ) Study design of ASPirin in Reducing Events in the Elderly (ASPREE): A randomized, controlled trial. Contemp Clin Trials 36: , 555–564.
[44]	McNeil JJ , Woods RL , Nelson MR , Murray AM , Reid CM , Kirpach B , Storey E , Shah RC , Wolfe RS , Tonkin AM , Newman AB , Williamson JD , Lockery JE , Margolis KL , Ernst ME , Abhayaratna WP , Stocks N , Fitzgerald SM , Trevaks RE , Orchard SG , Beilin LJ , Donnan GA , Gibbs P , Johnston CI , Grimm RH ; ASPREE Investigator Group ((2017) ) Baseline characteristics of participants in the ASPREE (ASPirin in Reducing Events in the Elderly) study. J Gerontol A Biol Sci Med Sci 72: , 1586–1593.
[45]	Lockery JE , Collyer TA , Abhayaratna WP , Fitzgerald SM , McNeil JJ , Nelson MR , Orchard SG , Reid C , Stocks NP , Trevaks RE , Woods R ((2019) ) Recruiting general practice patients for large clinical trials: Lessons from the Aspirin in Reducing Events in the Elderly (ASPREE) study. Med J Aust 210: , 168–173.
[46]	Hyattsville M , National Center for Health Statistics, with special feature on socioeconomic status and health,Health, http://www.cdc.gov/nchs/data/hus/hus11.pdf#024, Accessed 16 May 2020.
[47]	Ryan J , Woods RL , Britt C , Murray AM , Shah RC , Reid CM , Kirpach B , Wolfe RS , Nelson MR , Lockery JE , Orchard SG , Trevaks RE , McNeil JJ , Storey E ((2019) ) Normative performance of healthy older individuals on the Modified Mini-Mental State (3MS) examination according to ethno-racial group, gender, age, and education level. Clin Neuropsychol 33: , 779–797.
[48]	Hinton-Bayre AD , Geffen G , McFarland K ((1997) ) Mild head injury and speed of information processing: A prospective study of professional rugby league players. J Clin Exp Neuropsychol 19: , 275–289.
[49]	Hinton-Bayre AD , Geffen G , McFarland K ((2005) ) Comparability, reliability, and practice effects on alternate forms of the Digit Symbol Substitution and Symbol Digit Modalities Tests. Psychol Assess 17: , 237–241.
[50]	Amodio P , Wenin H , Del Piccolo F , Mapelli D , Montagnese S , Pellegrini A , Musto C , Gatta A , Umilta C ((2002) ) Variability of trail making test, symbol digit test and line trait test in normal people. A normative study taking into account age-dependent decline and sociobiological variables. Aging Clin Exp Res 14: , 117–131.
[51]	Llinas-Regla J , Vilalta-Franch J , Lopez-Pousa S , Calvo-Perxas L , Torrents Rodas D , Garre-Olmo J ((2017) ) The Trail Making Test. Assessment 24: , 183–196.
[52]	Goretti B , Niccolai C , Hakiki B , Sturchio A , Falautano M , Minacapelli E , Martinelli V , Incerti C , Nocentini U , Murgia M , Fenu G , Cocco E , Marrosu MG , Garofalo E , Ambra FI , Maddestra M , Consalvo M , Viterbo RG , Trojano M , Losignore NA , Zimatore GB , Pietrolongo E , Lugaresi A , Langdon D , Portaccio E , Amato MP ((2014) ) The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS): Normative values with gender, age and education corrections in the Italian population. BMC Neurol 14: , 171.
[53]	Lechevallier-Michel N , Fabrigoule C , Lafont S , Letenneur L , Dartigues JF ((2004) ) [Normative data for the MMSE, the Benton visual retention test, the Isaacs’s set test, the digit symbol substitution test and the Zazzo’s cancellation task in subjects over the age 70: Results from the PAQUID Study]. Rev Neurol (Paris) 160: , 1059–1070.
[54]	Nielsen H , Lolk A , Kragh-Sorensen P ((1995) ) Normative data for eight neuropsychological tests, gathered from a random sample of Danes aged 64–83. Nordisk Psykologi 47: , 241–255.
[55]	Spedo CT , Frndak SE , Marques VD , Foss MP , Pereira DA , Carvalho Lde F , Guerreiro CT , Conde RM , Fusco T , Pereira AJ , Gaino SB , Garcia RB , Benedict RH , Barreira AA ((2015) ) Cross-cultural adaptation, reliability, and validity of the BICAMS in Brazil. Clin Neuropsychol 29: , 836–846.
[56]	Tamayo F , Casals-Coll M , Sanchez-Benavides G , Quintana M , Manero RM , Rognoni T , Calvo L , Palomo R , Aranciva F , Pena-Casanova J ((2012) ) [Spanish normative studies in a young adult population (NEURONORMA young adults Project): Norms for the verbal span, visuospatial span, Letter-Number Sequencing, Trail Making Test and Symbol Digit Modalities Test]. Neurologia 27: , 319–329.
[57]	Berrigan LI , Fisk JD , Walker LA , Wojtowicz M , Rees LM , Freedman MS , Marrie RA ((2014) ) Reliability of regression-based normative data for the oral symbol digit modalities test: An evaluation of demographic influences, construct validity, and impairment classification rates in multiple sclerosis samples. Clin Neuropsychol 28: , 281–299.
[58]	Burggraaff J , Knol DL , Uitdehaag BMJ ((2017) ) Regression-based norms for the Symbol Digit Modalities Test in the Dutch population: Improving detection of cognitive impairment in multiple sclerosis?. Eur Neurol 77: , 246–252.
[59]	Drake AS , Weinstock-Guttman B , Morrow SA , Hojnacki D , Munschauer FE , Benedict RH ((2010) ) Psychometrics and normative data for the Multiple Sclerosis Functional Composite: Replacing the PASAT with the Symbol Digit Modalities Test. Mult Scler 16: , 228–237.
[60]	Merritt VC , Meyer JE , Cadden MH , Roman CA , Ukueberuwa DM , Shapiro MD , Arnett PA ((2017) ) Normative data for a comprehensive neuropsychological test battery used in the assessment of sports-related concussion. Arch Clin Neuropsychol 32: , 168–183.
[61]	Morrow SA ((2013) ) Normative data for the Stroop color word test for a North American population. Can J Neurol Sci 40: , 842–847.
[62]	Ardila A ((2005) ) Cultural values underlying psychometric cognitive testing. Neuropsychol Rev 15: , 185–195.
[63]	O’Bryant SE , Humphreys JD , Bauer L , McCaffrey RJ , Hilsabeck RC ((2007) ) The influence of ethnicity on Symbol Digit Modalities Test performance: An analysis of a multi-ethnic college and hepatitis C patient sample. Appl Neuropsychol 14: , 183–188.
[64]	Arango-Lasprilla JC , Rivera D , Rodriguez G , Garza MT , Galarza-Del-Angel J , Rodriguez W , Velazquez-Cardoso J , Aguayo A , Schebela S , Weil C , Longoni M , Aliaga A , Ocampo-Barba N , Saracho CP , Panyavin I , Esenarro L , Martinez C , Garcia de la Cadena C , Perrin PB ((2015) ) Symbol Digit Modalities Test: Normative data for the Latin American Spanish speaking adult population. Neurorehabilitation 37: , 625–638.
[65]	Cancela JM , Ayan C , Varela S ((2012) ) Symbol Digit Modalities Test: Normative values for Spanish home care residents: A pilot study. Actas Esp Psiquiatr 40: , 299–303.
[66]	Pena-Casanova J , Quinones-Ubeda S , Quintana-Aparicio M , Aguilar M , Badenes D , Molinuevo JL , Torner L , Robles A , Barquero MS , Villanueva C , Antunez C , Martinez-Parra C , Frank-Garcia A , Sanz A , Fernandez M , Alfonso V , Sol JM , Blesa R ((2009) ) Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for verbal span, visuospatial span, letter and number sequencing, trail making test, and symbol digit modalities test. Arch Clin Neuropsychol 24: , 321–341.
[67]	Amato MP , Portaccio E , Goretti B , Zipoli V , Ricchiuti L , De Caro MF , Patti F , Vecchio R , Sorbi S , Trojano M ((2006) ) The Rao’s Brief Repeatable Battery and Stroop Test: Normative values with age, education and gender corrections in an Italian population. Mult Scler 12: , 787–793.
[68]	Karlamangla AS , Lachman ME , Han W , Huang M , Greendale GA ((2017) ) Evidence for cognitive aging in midlife women: Study of women’s health across the nation. PLoS One 12: , e0169008.
[69]	O’Bryant SE , Edwards M , Johnson L , Hall J , Gamboa A , O’Jile J ((2018) ) Texas Mexican American adult normative studies: Normative data for commonly used clinical neuropsychological measures for English- and Spanish-speakers. Dev Neuropsychol 43: , 1–26.
[70]	Vogel A , Stokholm J , Jorgensen K ((2013) ) Performances on Symbol Digit Modalities Test, Color Trails Test, and modified Stroop test in a healthy, elderly Danish sample. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn 20: , 370–382.
[71]	Meng X , D’Arcy C ((2012) ) Education and dementia in the context of the cognitive reserve hypothesis: A systematic review with meta-analyses and qualitative analyses. PLoS One 7: , e38268.
[72]	Stern Y ((2012) ) Cognitive reserve in ageing and Alzheimer’s disease. Lancet Neurol 11: , 1006–1012.
[73]	Ritchie SJ , Bates TC , Der G , Starr JM , Deary IJ ((2013) ) Education is associated with higher later life IQ scores, but not with faster cognitive processing speed. Psychol Aging 28: , 515–521.
[74]	Aiken Morgan AT , Marsiske M , Dzierzewski JM , Jones RN , Whitfield KE , Johnson KE , Cresci MK ((2010) ) Race-related cognitive test bias in the active study: A mimic model approach. {Exp Aging Res 36: , 426–452.
[75]	Tunny G ((2006) ) Educational attainment in Australia, https://treasury.gov.au/publication/economic-roundup-autumn-2006/educational-attainment-in-australia.
[76]	Ryan J , Woods RL , Murray AM , Shah RC , Britt CJ , Reid CM , Wolfe R , Nelson MR , Lockery JE , Orchard SG , Trevaks RE , Chong TJ , McNeil JJ , Storey E ((2020) ) Normative performance of older individuals on the Hopkins Verbal Learning Test-Revised (HVLT-R) according to ethno-racial group, gender, age and education level. Clin Neuropsychol. 10.1080/13854046.2020.1730444.