Reference Data for Attentional, Executive, Linguistic, and Visual Processing Tests Obtained from Cognitively Healthy Individuals with Normal Alzheimer’s Disease Cerebrospinal Fluid Biomarker Levels
Abstract
Background:
Conventional neuropsychological norms likely include cognitively unimpaired (CU) individuals with preclinical Alzheimer’s disease (AD) pathology (amyloid-β, tau, and neurodegeneration) since they are based on cohorts without AD biomarkers data. Due to this limitation, population-based norms would lack sensitivity for detecting subtle cognitive decline due to AD, the transitional stage between healthy cognition and mild cognitive impairment. We have recently published norms for memory tests in individuals with normal cerebrospinal fluid (CSF) AD biomarker levels.
Objective:
The aim of the present study was to provide further AD biomarker-based cognitive references covering attentional, executive function, linguistic, and visual processing tests.
Methods:
We analyzed 248 CU individuals aged between 50–70 years old with normal CSF Aβ, p-tau, and neurodegeneration (t-tau) biomarker levels. The tests included were the Trail Making Test (TMT), Semantic Fluency Test, Digit and Symbol Span, Coding, Matrix Reasoning, Judgement of Line Orientation and Visual Puzzles. Normative data were developed based on regression models adjusted for age, education, and sex when needed. We present equations to calculate z-scores, the corresponding normative percentile tables, and online calculators.
Results:
Age, education, and sex were associated with performance in all tests, except education for the TMT-A, and sex for the TMT-B, Coding, and Semantic Fluency. Cut-offs derived from the current biomarker-based reference data were higher and more sensitive than standard norms.
Conclusion:
We developed reference data obtained from individuals with evidence of non-pathologic AD biomarker levels that may improve the objective characterization of subtle cognitive decline in preclinical AD.
INTRODUCTION
The biological presence of the Alzheimer’s disease (AD) is defined by amyloid-β (Aβ) and tau pathology. The early impact of Aβ and tau in cognition has been reported even in the cognitively healthy population [1, 2]. The effects of Aβ and tau have been mainly associated with a decline in memory function, but recent evidence suggested that beyond memory, other cognitive processes such as executive functioning are affected at early stages of preclinical AD [3]. A comprehensive assessment of cognitive function across cognitive domains, and not just of memory function, might help to delineate better the subtle cognitive decline associated with subjacent Aβ deposition, and aggregated tau at the preclinical stage of AD.
In 2018, the research criteria of the National Institute on Aging-Alzheimer’s Association (NIA-AA) defined subtle cognitive decline as the stage of transitional decline between healthy cognition and mild cognitive impairment (MCI) [4]. The characterization of subtle cognitive decline in preclinical AD aims to advance clinical diagnosis at earlier asymptomatic stages, and to improve sensitivity in clinical trials targeting cognitive function. The NIA-AA provided a classification system based on the status of the three AT(N) biomarker types that describe the biological profile along the AD continuum, Aβ (A), p-tau (T), and non-specific neurodegeneration ([N]) [4]. Thus, the AT(N) system allowed the classification of individuals with normal AD biomarkers, individuals within the AD continuum, or individuals with a non-AD pathologic change, providing a research framework for delineating the earliest relationship between biological changes and the decline of cognitive function at preclinical stages.
Normative data for clinical neuropsychological testing are conventionally obtained from cognitively unimpaired (CU) individuals. Thus, cognitive impairment could be identified by comparing individual performances with normal range while adjusting by sociodemographic factors, such as age, education, and sex. Nevertheless, this canonical procedure might yield some deflation in normative cognitive scores according to the presence of subtle cognitive decline in CU individuals due to incipient abnormal levels of AT(N) biomarkers indicating underlying AD pathology.
Since conventional neuropsychological norms might lack enough sensitivity for detecting subtle cognitive decline in preclinical stages, robust normative procedures typically account for the reliable performance of clinically stable populations (including only individuals who maintain a status of healthy cognitive performance on longitudinal follow-up and excluding those that latter on develop MCI/dementia). Robust normative approaches based on longitudinal assessment demonstrated an increase in the probability of labeling cognitive performance as below average in comparison to conventional norms [5–7], by establishing higher thresholds for what is considered normal performance or decline according to these robust procedures [8, 9].
An alternative approach is to rely on biomarker data for constructing robust references sensitive enough to identify subtle decline in cognitive performance. The presence of Aβ positivity in CU individuals ranging from 50 to 70 years has an estimated prevalence of between 10.4% and 23.1% [10], and the presence of either Aβ, tau pathology, or neurodegeneration in CU individuals older than 65 years has an estimated prevalence of up to 44% [11]. In fact, the effect of aging in the cognitive performance of older adults is attenuated when AD biomarker status is accounted for in the analysis [12]. Accordingly, excluding individuals with signs of altered Aβ, p-tau, neurodegeneration (neurofilament light) biomarker levels, cerebrovascular pathology, or uncontrolled systemic medical illness, results in more rigorous standards for detecting cognitive decline than those of conventional norms [13, 14]. Therefore, accounting for cerebrospinal fluid (CSF) AD biomarkers for the adjustment of normal performance in cognitive testing may enhance the sensitivity of normative neuropsychological data beyond standard published norms for detecting subtle cognitive decline in individuals not meeting yet MCI criteria [15, 16].
Previous research identified that the use of norms excluding individuals with positive Aβ biomarker levels, as criteria for a more robust normative procedure, allowed the identification of individuals at risk of dementia and increased the predictive accuracy of dementia progression using memory measures [16]. Thus, Bos et al. proposed a “normal or impaired” classification model based on three groups combining the classification of conventional norms and Aβ negative norms (i.e., Group 1: normal with both norms; Group 2: impaired only with robust Aβ negative norms, Group 3: impaired with both norms [16]). Such classification model provides a complementary interpretational framework for characterizing subtle cognitive decline. Although in Bos et al. [16] it was only necessary to adjust for AD biomarkers in memory measures, other evidence pointed out that removing preclinical AD participants from normative samples also results in higher means and less variability in visuospatial ability and executive function [17]. For this reason, we considered relevant to explore the usefulness of AD biomarker-based normative data in the assessment of other several cognitive domains apart from memory processes.
We have recently published norms for the verbal memory tests Free and Cued Selective Reminding Test and the WMS-IV Logical Memory Test in a sample of CU individuals with normal AD CSF biomarker levels. The use of such biomarker-based norms in combination to standard references may be useful for the identification of subtle impairments in memory function [15]. These biomarker-based normative data have demonstrated to capture cognitive decline in individuals with evidence of positive Aβ and tau status in a 3-year follow-up within the ALFA+study, while conventional norms failed to do so (in preparation).
In the present work, as a robust normative procedure we only used data of CU individuals with normal (non-pathological) levels of AD biomarkers. We excluded from our analyses individuals with positive status of underlying AD pathology, defined by CSF Aβ (A), p-tau (T), t-tau ([N]) biomarkers, with the aim to provide a more sensitive normative reference for cognitive evaluation. The AD biomarker-based normative procedure is a complementary interpretational framework for the neuropsychological assessment of subtle cognitive decline in preclinical AD.
METHODS
Participants
We included 248 Spanish participants that completed the first visit (2016–2019) of the ongoing longitudinal ALFA+(ALzheimer and FAmilies) study. ALFA+is a research cohort of middle-aged CU individuals, most of them with family history of AD (153 participants [61.7%] had at least one parent diagnosed with AD before the age of 75). ALFA+participants have been thoroughly characterized with clinical interviews, lifestyle and risk factors questionnaires, cognitive testing, CSF biomarkers, and neuroimaging procedures, including magnetic resonance imaging (MRI), and positron emission tomography (PET). All tests and procedures are repeated every 3 years with the aim to identify the earliest pathophysiological changes in the preclinical AD continuum [18].
ALFA+ inclusion criteria were: 1) participants who had previously participated in the 45–65/FPM2012 study (ALFA parent cohort [18]); 2) range of 45–65 years-old at the moment of the inclusion in the 45–65/FPM2012 study; 3) long-term commitment to the study: inclusion and follow-up visits and agreement to undergo all tests and procedures (e.g., MRI and lumbar puncture). ALFA+ exclusion criteria included: 1) cognitive impairment (Clinical Dementia Rating [CDR] > 0, Mini-Mental State Examination [MMSE] < 27, semantic fluency < 12); 2) significant systemic illness or unstable medical condition which could lead to difficulty complying with the protocol; 3) contraindication to any test or procedure; 4) suspected family history of monogenic AD.
AD biomarker status definition
CSF analyses defined Aβ, p-tau, and t-tau status. CSF collection and processing have been described previously [19]. CSF p-tau and t-tau were measured using the electrochemiluminescence Elecsys® Phospho-Tau (181P) CSF and Total-Tau CSF immunoassays, respectively, on a fully automated cobas e601 module (Roche Diagnostics International Ltd.). CSF Aβ42 and Aβ40 were measured with the exploratory Roche NeuroToolKit immunoassays (Roche Diagnostics International Ltd, Rotkreuz, Switzerland) on a cobas e601 module. Measurements were performed at the Clinical Neurochemistry Laboratory, Sahlgrenska University Hospital, Mölndal, Sweden. Aβ status was defined using the cutoff of 0.071 for the ratio Aβ42/40, tau status was defined using the cutoff of 24 pg/ml for p-tau and neurodegeneration status was defined using the cutoff of 300 pg/ml for t-tau as previously described [19].
Cognitive measures
We included standard neuropsychological tests for the clinical assessment of attentional (WAIS-IV: Digit Span; WMS-IV: Symbol Span; TMT-A), executive (TMT-B; WAIS-IV: Coding; WAIS-IV: Matrix reasoning), linguistic (Semantic Fluency Test), and visual processing (WAIS-IV: Visual Puzzles; RBANS: Judgement of line orientation).
WAIS-IV: Digit Span
The Digit Span subtest used is included in the WAIS-IV [20]. This test measures attentional processing, specifically executive attention understood as the relationship between working memory and attentional control. It consists of three parts: forward, backward, and sequencing. The forward part requires the participant to repeat, in the same order, the numbers recited by the examiner. The backward part requires the participant to repeat, in reverse order, the numbers recited by the examiner. The sequencing part requires the participant to repeat, in ascending order, the numbers recited by the examiner. The test consists of 8 items for each part. Each item has two attempts that are scored with 1 or 0. The administration of each part is suspended if the participant obtains a score of 0 in both attempts of the same item. The maximum score is 16 points for each part, with a maximum score of 48 for the total test. The main variables of the test are forward span (0–9), backward span (0–8), and the total (0–48). We present normative data and equations to calculate z-scores for the forward span, backward span, as well as the total score.
WMS-IV: Symbol Span
The Symbol Span subtest used is included in the Spanish version of the Wechsler Memory Scale-IV [21]. This test is used to measure visual working memory. The participant briefly observes a series of abstract symbols. The examiner then removes these from his view and asks the participant to sequentially identify them, following the order in which he has seen them, from a larger series of symbols. The first two items are scored with 0 or 1 point, while the remaining items (from 3 to 26) are scored with 0, 1, or 2. The main variable of interest is the total sum of the scores obtained in the items from 1 to 26 (0–50). We present normative data and equations to calculate z-scores for the total score.
Trail Making Test (TMT)
The Trail Making Test [22] consists of two parts, A and B. Part A is used to assess selective attention and it consists of the connection, by means of drawing a line, of 25 numbers randomly distributed on a sheet. The participant must consecutively join the numbers following their natural order from 1 to 25. Part B is used to assess cognitive flexibility and it consists of the alternating joining of numbers and letters following the natural order of the numbers and the order of the alphabet. The main variables of interest are measured in seconds for Part A (completion time) and Part B (completion time). We present normative data and equations to calculate z-scores for the completion times of Part A and Part B.
WAIS-IV: Coding
The Coding subtest used is included in WAIS-IV [20]. This test assesses processing speed. The participant must reproduce the symbol that corresponds to each number according to the model presented on the same sheet, where there are numbers from 1 to 9 and the symbol that corresponds to each of the numbers. The participant tries to complete as many items as possible for two minutes. The main variable of interest is the total score (0–135). We present normative data and equations to calculate z-scores for the total score.
WAIS-IV: Matrix reasoning
The Matrix reasoning subtest used is included in the WAIS-IV [20]. It captures fluid intelligence, general visual intelligence, spatial aptitude, and classification. The participant must look at an incomplete matrix or series and select, from among five options, the one that best completes the matrix or series. The test has 26 items, and a maximum time of 30 seconds is allowed for each response, which will be scored with 0 or 1. After three consecutive scores of 0, the administration of the test is suspended. The main variable of interest is the total score (0–26). We present normative data and equations to calculate z-scores for the total score.
Semantic Fluency Test
The Semantic Fluency Test [23] assesses semantic verbal fluency. The test requires the participant to say the maximum number of words within the semantic category “animals” in one minute. The main variable of interest is the total score, composed by the number of correct productions. We present normative data and equations to calculate z-scores for the total score.
WAIS-IV: Visual Puzzles
The Visual Puzzles subtest used is included in the WAIS-IV [20]. It captures non-verbal reasoning. Within a time limit (20 or 30 s, depending on the complexity), the participant must select the three pieces that allow the reconstruction of the presented puzzle. Each item is scored with 0 or 1 and the administration is suspended after 3 consecutive scores of 0. The test consists of 26 items. The main variable of interest is the total score (0–26). We present normative data and equations to calculate z-scores for the total score.
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Judgment of line orientation.
The Judgment of line orientation subtest is included in the RBANS [24]. It captures visuospatial processing. The participant must estimate spatial relationships between line segments by matching them to a series of sample lines. Ten items are offered with two segments to match for each of them. The main variable of interest is the total score (0–20). We present normative data and equations to calculate z-scores for the total score.
Development of normative data
Normative data were developed following multiple linear regression-based methods. The procedure is as follows: 1) Centering the age of the participants by subtracting the mean group age from each individual’s chronological age. 2) Constructing a set of multiple regression models (one for each cognitive score of interest), with the cognitive score as dependent variable and age-centered, education (with 4 category levels [elementary = 0, secondary = 1, graduate = 2, postgraduate = 3]), and sex (male = 0; female = 1) as predictors. In this study we defined education categories as the highest completed education level. The number of years of formal education may overlap among levels depending on the education system and the number of additional years attended from an uncompleted educational program. A backward stepwise method was used, with a criterion of p < 0.1 for the beta coefficient to maintain a predictor in the model. 3) Using the constant and the coefficients obtained to calculate predicted scores following Equation 1 below.
(1)
4) Calculating the residuals between each possible value of the cognitive score and each possible expected score (using the relevant predictors for each variable) by subtracting them. Then, the residuals were converted to a z-score by dividing them by the standard deviation of the unstandardized residuals of the regression model. Clinicians may use the equations with the coefficients provided in the results to calculate the z-scores associated with the specific raw scores of a patient. 5) To simplify the use of the normative data, we provide tables for percentiles 1, 2, 5, 10, 15, 20, 30, 40, 50, 60, 70, 80, 90, 95, and 98. In each table, the theoretical raw scores associated with each percentile value are shown. Since age accounted for a relevant effect in all cognitive variables, age groups were collapsed considering the distribution of the percentiles along the age range to reduce the number of tables presented. 6) To facilitate the use of biomarker-based cognitive norms, we provide online calculators for expected scores and actual z-scores and percentiles associated.
Comparison of normative frameworks
To explore the sensitivity of the current biomarker-based cognitive references we compared the number of lower scores in our sample by using both the biomarker-based references provided in this work and the available Spanish data from the NEURONORMA project and the WAIS-IV and WMS-IV norming studies. We assessed the proportion of lower scores in: Digit Span (forward and backward), Symbol Span, TMT (A and B), Coding, Matrix Reasoning, Semantic Fluency, and Visual Puzzles. Sociodemographic-adjusted scaled scores equal or below to 5 (equivalent to percentile 5) were considered as impaired. Individuals were classified as having or not a lower score (impaired or normal) using this definition. The McNemar test for related proportions (adjusted for continuity) was used to analyze the distribution of individuals labeled as impaired between both references. The Fleiss’ Kappa interrater correlation coefficient for categorical variables was used to test the agreement for the labeling of the same individuals between biomarker-based and standard references.
RESULTS
Demographic, presence of the APOE ɛ4 allele, inclusion cognitive outcomes, and biomarker data of the participants included in this study are presented in Table 1. Descriptive data of the neuropsychological assessment is presented in Table 2. Results of multiple linear regression analyses according to the estimated coefficient (beta) value for each variable and related p-value are presented in Table 3. Equations used to calculate z-scores accounting for relevant sociodemographic factors are presented in Table 4. Normative tables with the calculations developed and raw scores equivalence to percentiles are available as Supplementary Tables (1–20).
Table 1
Participants characteristics (n = 248)
| Mean (SD) | Range | Count (%) | ||
| Demographic | ||||
| Age | 60.5 (4.52) | 50–70 | ||
| Sex (females) | 153 (61.7%) | |||
| Education | ||||
| Elementary | 25 (10.1%) | |||
| Secondary | 109 (44.0 %) | |||
| Graduate | 73 (29.4%) | |||
| Postgraduate | 41 (16.5 %) | |||
| APOE ɛ4 carrier | 105 (42.3 %) | |||
| CSF Biomarker | ||||
| Aβ40 (ng/mL) | 16.8 (4.74) | 4.1–31.1 | ||
| Aβ42 (pg/mL) | 1474 (514) | 364–3595 | ||
| Aβ42/40 | 0.0866 (0.0086) | 0.0710–0.116 | ||
| p-tau181 (pg/mL) | 13.9 (4.2) | 7.9–23.6 | ||
| t-tau (pg/mL) | 175 (48) | 79.9–299 | ||
| Inclusion cognitive data | ||||
| MMSE | 29.2 (0.9) | 27–30 |
Education was coded as follows: Elementary equals to finished elementary school (range of formal effective education 8–11 years); Secondary equals to finished secondary studies (range of formal effective education 9–14 years); Graduate equals to a university or superior degree (range of formal effective education 14–18 years); Postgraduate equals to Master or PhD (range of formal effective education 15–20 years). MMSE, Mini-Mental State Examination. The cutoffs of CSF biomarkers to classify individuals in the AT(N) triple-negative group were > 0.071 for the Aβ42/40 ratio, <24 pg/ml for p-tau181 and < 300 pg/ml for total-tau [19]. Data only included individuals classified in the AT(N) triple-negative group.
Table 2
Descriptive data of the neuropsychological assessment
| Mean (SD) | Range | |
| WAIS-IV: Digit Span forward | 5.77 (1.12) | 3–9 |
| WAIS-IV: Digit Span backward | 4.63 (1.29) | 2–8 |
| WAIS-IV: Digit Span total | 25.33 (5.44) | 12–45 |
| WMS-IV: Symbol Span | 19.68 (6.16) | 5–35 |
| TMT-A | 35.71 (10.00) | 16–69 |
| TMT-B | 79.98 (29.79) | 32–232 |
| WAIS-IV: Coding | 66.67 (14.03) | 36–120 |
| WAIS-IV: Matrix Reasoning | 17.09 (4.14) | 6–25 |
| Semantic Fluency Test | 23.06 (5.18) | 13–38 |
| WAIS-IV: Visual Puzzles | 13.83 (3.98) | 5–23 |
| RBANS: Judgment of line orientation | 17.44 (2.46) | 9–20 |
WAIS-IV, Wechsler Adult Intelligence Scale –IV; WMS-IV, Wechsler Memory Scale –IV; TMT, Trail Making Test; RBANS, Repeatable Battery for the Assessment of Neuropsychological Status.
Table 3
Results of the multiple linear regression analyses
| Outcome | Constant | Predictors | Beta | p |
| WAIS-IV: Digit Span forward | 6.2211 | |||
| Sex | –0.7243 | <0.001 *** | ||
| WAIS-IV: Digit Span backward | 4.67255 | |||
| Edu | 0.20451 | 0.0238 * | ||
| Sex | –0.58233 | <0.001 *** | ||
| WAIS-IV: Digit Span total | 26.24519 | |||
| Age | –0.17414 | 0.0169 * | ||
| Edu | 0.80929 | 0.0299 * | ||
| Sex | –3.48183 | <0.001 *** | ||
| WMS-IV: Symbol Span | 20.26840 | |||
| Age | –0.34090 | <0.001 *** | ||
| Edu | 0.86530 | 0.041 * | ||
| Sex | –3.13710 | <0.001 *** | ||
| TMT-A | 33.77380 | |||
| Age | 0.59410 | <0.001 *** | ||
| Sex | 3.19660 | 0.012 * | ||
| TMT-B | 91.73320 | |||
| Age | 1.72490 | <0.001 *** | ||
| Edu | –7.61760 | <0.001 *** | ||
| WAIS-IV: Coding | 60.25810 | |||
| Age | –0.77660 | <0.001 *** | ||
| Edu | 4.20890 | <0.001 *** | ||
| WAIS-IV: Matrix Reasoning | 15.89589 | |||
| Age | –0.21038 | <0.001 *** | ||
| Edu | 1.34383 | <0.001 *** | ||
| Sex | –1.37351 | <0.001 ** | ||
| Semantic Fluency Test | 20.89791 | |||
| Age | –0.13718 | 0.05385 . | ||
| Edu | 1.42148 | <0.001 *** | ||
| WAIS-IV: Visual Puzzles | 14.15703 | |||
| Age | –0.27828 | <0.001 *** | ||
| Edu | 0.75432 | 0.00375 ** | ||
| Sex | –2.39266 | <0.001 *** | ||
| RBANS: Judgment of line orientation | 17.73402 | |||
| Age | –0.05749 | 0.067972· | ||
| Edu | 0.59889 | <0.001 *** | ||
| Sex | –1.96645 | <0.001 *** |
WAIS-IV, Wechsler Adult Intelligence Scale – IV; WMS-IV, Wechsler Memory Scale – IV; TMT, Trail Making Test; RBANS, Repeatable Battery for the Assessment of Neuropsychological Status. Education was entered as: Elementary = 0; Secondary = 1; Graduate = 2; Postgraduate = 3. Sex was entered as: Male = 0; Female = 1. Age was centered to 60.5. Significance codes: ‘***’ <0.001 ‘**’ <0.01 ‘*’ <0.05 ‘·’ <0.1.
Table 4
Z-scores calculation formula
| WAIS -IV: Digit Span forward | (Raw Score – [6.2211 + Sex * –0.7243])/1.06586 |
| WAIS-IV: Digit Span backward | (Raw Score – [4.67255 + Education * 0.20451 + Sex * –0.58233])/1.239101 |
| WAIS-IV: Digit Span total | (Raw Score – [26.24519 + Age * –0.17414 + Education * 0.80929 + Sex * –3.48183])/5.045744 |
| WMS-IV: Symbol Span | (Raw Score – [20.2684 + Age * –0.3409 + Education * 0.8653 + Sex * –3.1371])/5.708696 |
| TMT-A | (Raw Score – [33.7738 + Age * 0.5941 + Sex * 3.1966])/9.558886 |
| TMT-B | (Raw Score – [91.7332 + Age * 1.7249 + Education * –7.6176])/27.74928 |
| WAIS-IV: Coding | (Raw Score – [60.2581 + Age * –0.7766 + Education * 4.2089])/12.94024 |
| WAIS-IV: Matrix reasoning | (Raw Score – [15.89589 + Age * –0.21038 + Education * 1.34383 + Sex * –1.37351])/3.746918 |
| Semantic Fluency Test | (Raw Score – [20.89791 + Age * –0.13718 + Education * 1.42148])/4.973485 |
| WAIS-IV: Visual Puzzles | (Raw Score – [14.15703 + Age * –0.27828 + Education * 0.75432 + Sex * –2.39266])/3.510158 |
| RBANS: Judgment of line orientation | (Raw Score – [17.73402 + Age * –0.05749 + Education * 0.59889 + Sex * –1.96645])/2.175314 |
Education should be entered as: Elementary = 0; Secondary = 1; Graduate = 2; Postgraduate = 3. Sex should be entered as: Male = 0; Female = 1. Age should be centered to 60.5. Elementary education equals to finished elementary school (range of formal effective education 8–11 years); Secondary equals to finished secondary studies (range of formal effective education 9–14 years); Graduate equals to a university or superior degree (range of formal effective education 14–18 years); Postgraduate equals to Master or PhD (range of formal effective education 15–20 years).
We assessed the proportion of impaired scores when using the biomarker-based in contrast to the standard references. Using biomarker-based references, the percentage of at least one or more lower scores among the 9 measures considered was 44.76%. Using the standard references, the percentage was 21.77%. The McNemar test for related proportions was statistically significant when comparing the distribution of impaired individuals between both references (McNemar X2 (df = 1, n = 248) = 52.155, p = <0.001). The Fleiss’ Kappa coefficient revealed low-intermediate agreement between datasets for the labeling (k = 0.470, z = 7.36, p = <0.001). To illustrate the different behavior of both norms we further provide a single test example, we assessed the proportion of impaired scores when using biomarker-based as compared to WMS-IV Spanish references in the Symbol Span. Using the current biomarker-based references, the percentage of lower scores for this single measure was 5.65%, while using the WMS-IV this percentage was 1.61%. The McNemar test for related proportions was statistically significant (McNemar X2 (df = 1, n = 248) = 6.75, p = 0.009). The Fleiss’ Kappa coefficient revealed low agreement between datasets for the labeling using this single measure (k = 0.308, z = 4.83, p = <0.001).
To facilitate the use of the reference data presented here and the previously published norms for the Free and Cued Selective Reminding Test and the WMS-IV Logical Memory Task developed with the same sample and approach [15], we provide online interactive tables with automatic calculators in the Supplementary Material.
DISCUSSION
In this study we provided regression-based normative data obtained from a sample of CU individuals (aged between 50 and 70 years) without underlying AD pathology, assessed by CSF determination of core AD biomarkers (AT[N]). Normative data is presented for the following tests: the Digit Span of the WAIS-IV, the Symbol Span of the WMS-IV, the TMT, the Coding of the WAIS-IV, the Matrix reasoning of the WAIS-IV, the Semantic Fluency Test, the Visual Puzzles of the WAIS-IV, and the Judgement of line orientation of the RBANS. The current biomarker-based cognitive references aim to provide an interpretational framework for the characterization of subtle cognitive decline in preclinical AD.
Given the potential limitations of conventional neuropsychological norms in identifying subtle cognitive decline, robust normative procedures are commonly employed to ensure the inclusion of clinically stable populations. These procedures involve selecting individuals who consistently demonstrate healthy cognitive performance over longitudinal follow-ups, while excluding those who later develop MCI or dementia. Such robust normative procedures aimed to increase the sensitivity of norms for the detection of MCI [5, 8]. However, robust normative procedures are not limited to clinical stability over longitudinal follow-up, nor to the detection of MCI. Recent biomarker-based normative approaches have expanded beyond these parameters by excluding data obtained from individuals with underlying AD pathology. Such norms render higher cut-offs and more sensitive standards than those provided by conventional norms aiming to detect the subtle cognitive decline that precedes MCI [12, 13]. By incorporating such innovative methodologies, researchers can enhance the accuracy and sensitivity of cognitive assessment tools, thereby facilitating the early identification of individuals at risk of dementia and potentially enabling interventions at preclinical stages.
The present work may help in the characterization of subtle cognitive decline. As defined by the NIA-AA in 2018, subtle cognitive decline is the stage of transitional decline before MCI [4]. This criterion proposed that this stage can be documented through subjective reports of cognitive decline or objectively by a longitudinal follow-up. Since most individuals with subjective reports of decline do not progress to dementia [25], objective cognitive measures may be more predictive of future dementia [16]. A direct approach is to use cross-sectional performance to predict clinical progression. The use of the common demographically adjusted cut-offs of< -1.5 SD below the mean, or percentile < 5, using a group of CU individuals without biological evidence of AD pathology as a reference has already been reported to improve the accuracy prediction of dementia progression [16]. By combining conventional and biomarker-based normative data, it is possible to identify those CU individuals who perform in the normal range according to standard normative data (without using negative-biomarker CU as reference) but fall in the impaired range according to normative data based on AD biomarkers (Group 2 classification according to Bos et al. [16]). We propose that cognitive scores showing these discrepancies could serve as indicators of subtle cognitive decline.
We assessed the sensitivity of the biomarker-based norms provided in this work in comparison to the conventional published references. We demonstrated that the proportion of lower scores (individuals labeled as impaired) significantly differed between the datasets, being higher for the biomarker-based norms. In addition, there was a low-intermediate agreement for impaired labeling between the norms. Considering the classification discrepancies between references and the fit of biomarker-based approaches for the characterization of preclinical AD, results suggest that the presented norms work as intended and indeed were more sensitive to subtle cognitive changes than the standard references.
As it has been previously reported, removing individuals with positive status of amyloid-β and/or tau pathology in normative data, yields higher reference scores than those of standard procedures [15–17]. As a result, the sensitivity was enhanced but the specificity of the norms decreased. To explore the characterization of subtle cognitive decline, the norms provided here do not aim to replace previously published ones. We suggest using these norms as a complementary tool (e.g., not in isolation but rather in combination with published conventional norms) for an interpretative framework in characterizing subtle cognitive decline. For reference, we provide hypothetical examples of the use of the normative data provided in this work in comparison with standard norms (see the Supplementary Material).
We found a relevant effect of age on the performance of all tasks, except for the Digit Span forward and backward parts. Education affected the performance for all tasks except for two variables within the attentional domain, the Digit Span forward, and the TMT-A. Education was positively associated with cognitive performance, except for the TMT-B, in which the association was negative, as expected (higher education associated with lower completion times). Sex affected the performance in all tasks except for two tasks within the executive domain, the TMT-B, and the WAIS-IV: Coding, and for the task in the linguistic domain, the Semantic Fluency Test. Males outperformed women on the TMT-A, WAIS-IV: Digit Span, WMS-IV: Symbol Span, WAIS-IV: Matrix reasoning, WAIS-IV: Visual Puzzles, RBANS: Judgment of line orientation, indicating that males tend to have higher scores than females. These associations between sex and cognitive performance showing better scores in men are consistent with previous studies in the studied cognitive domains [26–29], but are in contrast with our previously published normative data from the same sample in verbal episodic memory, in which women consistently outperformed men [15].
The influence of sociodemographic variables on cognitive function is well known [30]. However, available normative data typically presents adjustments for age, but adjustments for education are not always included, and including them for sex is even less frequent. It is of high relevance to adjust cognitive norms for sex, since not accounting for sex differences could lead to misdiagnoses [31]. Following this consideration, and in contrast with available normative data, the present study included adjustments for education for the WMS-IV: Symbol Span; the WAIS-IV: Coding; and the WAIS-IV: Matrix reasoning, and additional adjustments for sex for the WAIS-IV: Digit Span; the WMS-IV: Symbol Span; the TMT-A; the WAIS-IV: Matrix reasoning; and the WAIS-IV: Visual Puzzles.
The main limitation of our work resides in the partial applicability of the normative data provided. Individuals tested were aged from 50 to 70 years-old, thus constraining the age range of application. Nevertheless, the proposal of normative data based on AD biomarkers is especially useful in this age range, since the estimated prevalence of subjective cognitive decline is 25% for individuals older than 60 years [32]. The range of application is also constrained by the fact that our sample is mainly composed of individuals with at least elementary studies. Therefore, there is an important gap in the lower levels of education and the presented norms are not directly applicable to individuals with lower educational levels than those of the present investigation. Considering sample characteristics, the percentage of APOE ɛ4 carriers is higher in our sample than in the general population, and arguably, this may yield some bias in our norms. Nevertheless, although APOE ɛ4 might be associated with lower performance in this age range, we believe that we are controlling this effect by including AD biomarkers, since the effect of APOE ɛ4 on cognition is thought to be mediated by the presence of amyloid-β and tau pathology [33]. We also acknowledge that we used a highly sensitive cut-off for the CSF biomarker levels [19], therefore using more liberal cut-offs or different measurements such as PET imaging to define the reference group could result in different distributions.
In conclusion, we developed socio-demographically fully adjusted reference data obtained from individuals with evidence of normal (non-pathological) (AT[N]) biomarker levels in CSF. In combination with conventional norms, the biomarker-based cognitive references presented here may improve the objective characterization of subtle cognitive decline in preclinical AD.
ACKNOWLEDGMENTS
This publication is part of the ALFA study (ALzheimer and FAmilies). The authors would like to express their most sincere gratitude to the ALFA project participants and relatives without whom this research would have not been possible. Collaborators of the ALFA study are: Müge Akinci, Annabella Beteta, Raffaele Cacciaglia, Irene Cumplido, Carme Deulofeu, Ruth Dominguez, María Emilio, Carles Falcon, Ana Fernández-Arcos, Sherezade Fuentes, Patricia Genius-Serra, Laura Hernandez, Gema Huesa, Jordi Huguet, Tania Menchón, Grégory Operto, Eleni Palpatzis, Albina Polo, Sandra Pradas, Blanca Rodríguez-Fernández, Aleix Sala-Vila, Gemma Salvadó, Mahnaz Shekari, Anna Soteras, Laura Stankeviciute, Núria Tort-Colet, Marc Vilanova, and Natalia Vilor-Tejedor.
The authors thank Roche Diagnostics International Ltd for providing the kits to measure CSF biomarkers, and the laboratory technicians at the Clinical Neurochemistry Lab in Mölndal, Sweden, who performed the analyses. COBAS, COBAS E, and ELECSYS are trademarks of Roche. The Roche NeuroToolKit is a panel of robust exploratory prototype assays used for research purposes only and not approved for clinical use.
FUNDING
The research leading to these results has received funding from “la Caixa” Foundation (ID 100010434), under agreement LCF/PR/GN17/50300004, the Alzheimer’s Association, and an international anonymous charity foundation through the TriBEKa Imaging Platform project (TriBEKa-17-519007). Additional support has been received from the Universities and Research Secretariat, Ministry of Business and Knowledge of the Catalan Government under the grant no. 2021 SGR 00913. DL-M is supported by Instituto de Salud Carlos III through the project PI19/00117 (Co-funded by European Regional Development Fund/European Social Fund “A way to make Europe”/“Investing in your future”). MS-C receives funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (Grant agreement No. 948677), the Instituto de Salud Carlos III (PI19/00155, PI22/00456), and from the ERC under the EU’s ‘la Caixa’ Foundation (ID 100010434) and from the EU’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant (no. 847648, LCF/BQ/PR21/11840004). OG-R is supported by the Spanish Ministry of Science and Innovation –State Research Agency (IJC2020-043417-I/MCIN/AEI/10.13039/501100011033) and the European Union laquoNextGenerationEUraquo/PRTR. Henrik Zetterberg is a Wallenberg Scholar supported by grants from the Swedish Research Council (#2022-01018 and #2019-02397), the European Union’s Horizon Europe research and innovation programme under grant agreement No 101053962, Swedish State Support for Clinical Research (#ALFGBG-71320), the Alzheimer Drug Discovery Foundation (ADDF), USA (#201809-2016862), the AD Strategic Fund and the Alzheimer’s Association (#ADSF-21-831376-C, #ADSF-21-831381-C, and #ADSF-21-831377-C), the Bluefield Project, the Olav Thon Foundation, the Erling-Persson Family Foundation, Stiftelsen för Gamla Tjänarinnor, Hjärnfonden, Sweden (#FO2022-0270), the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860197 (MIRIADE), the European Union Joint Programme –Neurodegenerative Disease Research (JPND2021-00694), and the UK Dementia Research Institute at UCL (UKDRI-1003). GS-B has received funding from the Ministerio de Ciencia e Innovacion, Spanish Research Agency, PID2020-119556RA-I00.
CONFLICT OF INTEREST
Gonzalo Sánchez-Benavides worked as a consultant for Roche Farma, S.A. Marc Suárez-Calvet has served as a consultant and at advisory boards for Roche Diagnostics International Ltd and has given lectures in symposia sponsored by Roche Diagnostics, S.L.U, Roche Farma, S.A and Roche Sistemas de Diagnósticos, Sociedade Unipessoal, Lda. Henrik Zetterberg has served at scientific advisory boards and/or as a consultant for Abbvie, Acumen, Alector, Alzinova, ALZPath, Annexon, Apellis, Artery Therapeutics, AZTherapies, CogRx, Denali, Eisai, Nervgen, Novo Nordisk, Optoceutics, Passage Bio, Pinteon Therapeutics, Prothena, Red Abbey Labs, reMYND, Roche, Samumed, Siemens Healthineers, Triplet Therapeutics, and Wave, has given lectures in symposia sponsored by Cellectricon, Fujirebio, Alzecure, Biogen, and Roche, and is a co-founder of Brain Biomarker Solutions in Gothenburg AB (BBS), which is a part of the GU Ventures Incubator Program (outside submitted work).
Gonzalo Sánchez-Benavides, Henrik Zetterberg, and José Luis Molinuevo are Editorial Board Members of this journal but were not involved in the peer-review process nor had access to any information regarding its peer-review.
All other authors have no conflict of interest to report.
DATA AVAILABILITY
The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
SUPPLEMENTARY MATERIAL
[1] The supplementary material is available in the electronic version of this article: https://dx.doi.org/10.3233/JAD-230290.
The online calculator is available via google docs at the following link: https://docs.google.com/spreadsheets/d/1rr2Kvs6JrbcXHmKkPC0NYLBBSPBl82yocVVnLjgsPpc/edit?usp=sharing
REFERENCES
[1] | Insel PS , Donohue MC , Sperling R , Hansson O , Mattsson-Carlgren N ((2020) ) The A4 study: β-amyloid and cognition in 4432 cognitively unimpaired adults. Ann Clin Transl Neurol 7: , 776–785. |
[2] | Tort-Merino A , Olives J , León M , Peñaloza C , Valech N , Santos-Santos MA , Càmara E , Grönholm-Nyman P , Martínez-Lage P , Fortea J , Molinuevo JL , Sánchez-Valle R , Laine M , Rodríguez-Fornells A , Rami L ((2019) ) Tau protein is associated with longitudinal memory decline in cognitively healthy subjects with normal Alzheimer’s disease cerebrospinal fluid biomarker levels. J Alzheimers Dis 70: , 211–225. |
[3] | Tideman P , Stomrud E , Leuzy A , Mattsson-Carlgren N , Palmqvist S , Hansson O ((2022) ) Association of β-amyloid accumulation with executive function in adults with unimpaired cognition. Neurology 98: , E1525–E1533. |
[4] | Jack CR , Bennett DA , Blennow K , Carrillo MC , Dunn B , Haeberlein SB , Holtzman DM , Jagust W , Jessen F , Karlawish J , Liu E , Molinuevo JL , Montine T , Phelps C , Rankin KP , Rowe CC , Scheltens P , Siemers E , Snyder HM , Sperling R , Elliott C , Masliah E , Ryan L , Silverberg N ((2018) ) NIA-AA Research Framework: Toward a biological definition of Alzheimer’s disease. Alzheimers Dement 14: , 535–562. |
[5] | Kaser AN , Kaplan DM , Goette W , Kiselica AM ((2023) ) The impact of conventional versus robust norming on cognitive characterization and clinical classification of MCI and dementia. J Neuropsychol 17: , 108–124. |
[6] | Sliwinski M , Lipton RB , Buschke H , Stewart W ((1996) ) The effects of preclinical dementia on estimates of normal cognitive functioning in aging. J Gerontol B Psychol Sci Soc Sci 51B: , P217–P225. |
[7] | Goodwill AM , Campbell S , Henderson VW , Gorelik A , Dennerstein L , McClung M , Szoeke C ((2019) ) Robust norms for neuropsychological tests of verbal episodic memory in Australian women. Neuropsychology 33: , 581–595. |
[8] | Kramer AO , Casaletto KB , Umlauf A , Staffaroni AM , Fox E , You M , Kramer JH ((2020) ) Robust normative standards for the California Verbal Learning Test (CVLT) ages 60–89: A tool for early detection of memory impairment. Clin Neuropsychol 34: , 384–405. |
[9] | Koscik RL , La Rue A , Jonaitis EM , Okonkwo OC , Johnson SC , Bendlin BB , Hermann BP , Sager MA ((2014) ) Emergence of mild cognitive impairment in late middle-aged adults in the Wisconsin Registry for Alzheimer’s Prevention. Dement Geriatr Cogn Disord 38: , 16–30. |
[10] | Jansen WJ , Ossenkoppele R , Knol DL , Tijms BM , Scheltens P , Verhey FRJ , Visser PJ , Aalten P , Aarsland D , Alcolea D , Alexander M , Almdahl IS , Arnold SE , Baldeiras I , Barthel H , Van Berckel BNM , Bibeau K , Blennow K , Brooks DJ , Van Buchem MA , Camus V , Cavedo E , Chen K , Chetelat G , Cohen AD , Drzezga A , Engelborghs S , Fagan AM , Fladby T , Fleisher AS , Van Der Flier WM , Ford L , Forster S , Fortea J , Foskett N , Frederiksen KS , Freund-Levi Y , Frisoni GB , Froelich L , Gabryelewicz T , Gill KD , Gkatzima O , Gomez-Tortosa E , Gordon MF , Grimmer T , Hampel H , Hausner L , Hellwig S , Herukka SK , Hildebrandt H , Ishihara L , Ivanoiu A , Jagust WJ , Johannsen P , Kandimalla R , Kapaki E , Klimkowicz-Mrowiec A , Klunk WE , Kohler S , Koglin N , Kornhuber J , Kramberger MG , Van Laere K , Landau SM , Lee DY , De Leon M , Lisetti V , Lleo A , Madsen K , Maier W , Marcusson J , Mattsson N , De Mendonca A , Meulenbroek O , Meyer PT , Mintun MA , Mok V , Molinuevo JL , Mollergard HM , Morris JC , Mroczko B , Van Der Mussele S , Na DL , Newberg A , Nordberg A , Nordlund A , Novak GP , Paraskevas GP , Parnetti L , Perera G , Peters O , Popp J , Prabhakar S , Rabinovici GD , Ramakers IHGB , Rami L , De Oliveira CR , Rinne JO , Rodrigue KM , Rodriguez-Rodriguez E , Roe CM , Rot U , Rowe CC , Ruther E , Sabri O , Sanchez-Juan P , Santana I , Sarazin M , Schroder J , Schutte C , Seo SW , Soetewey F , Soininen H , Spiru L , Struyfs H , Teunissen CE , Tsolaki M , Vandenberghe R , Verbeek MM , Villemagne VL , Vos SJB , Van Waalwijk Van Doorn LJC , Waldemar G , Wallin A , Wallin AK , Wiltfang J , Wolk DA , Zboch M , Zetterberg H ((2015) ) Prevalence of cerebral amyloid pathology in persons without dementia: A meta-analysis. JAMA 313: , 1924–1938. |
[11] | Jack CR , Wiste HJ , Weigand SD , Therneau TM , Knopman DS , Lowe V , Vemuri P , Mielke MM , Roberts RO , Machulda MM , Senjem ML , Gunter JL , Rocca WA , Petersen RC ((2017) ) Age-specific and sex-specific prevalence of cerebral β-amyloidosis, tauopathy, and neurodegeneration in cognitively unimpaired individuals aged 50–95 years: A cross-sectional study. Lancet Neurol 16: , 435–444. |
[12] | Harrington KD , Aschenbrenner AJ , Maruff P , Masters CL , Fagan AM , Benzinger TLS , Gordon BA , Cruchaga C , Morris JC , Hassenstab J ((2021) ) Undetected neurodegenerative disease biases estimates of cognitive change in older adults. Psychol Sci 32: , 849–860. |
[13] | Borland E , Stomrud E , Van Westen D , Hansson O , Palmqvist S ((2020) ) The age-related effect on cognitive performance in cognitively healthy elderly is mainly caused by underlying AD pathology or cerebrovascular lesions: Implications for cutoffs regarding cognitive impairment. Alzheimers Res Ther 12: , 30. |
[14] | Harrington KD , Lim YY , Ames D , Hassenstab J , Rainey-Smith S , Robertson J , Salvado O , Masters CL , Maruff P ((2017) ) Using robust normative data to investigate the neuropsychology of cognitive aging. Arch Clin Neuropsychol 32: , 142–154. |
[15] | Brugulat-Serrat A , Cañas-Martínez A , Canals-Gispert L , Marne P , Gramunt N , Milà-Alomà M , Suárez-Calvet M , Arenaza-Urquijo EM , Grau-Rivera O , González-De-Echávarri JM , Minguillon C , Fauria K , Kollmorgen G , Suridjan I , Zetterberg H , Blennow K , Gispert JD , Molinuevo JL , Sánchez-Benavides G ((2021) ) Enhancing the sensitivity of memory tests: Reference data for the Free and Cued Selective Reminding Test and the Logical Memory Task from cognitively healthy subjects with normal Alzheimer’s disease cerebrospinal fluid biomarker levels. J Alzheimers Dis 84: , 119–128. |
[16] | Bos I , Vos SJB , Jansen WJ , Vandenberghe R , Gabel S , Estanga A , Ecay-Torres M , Tomassen J , den Braber A , Lleó A , Sala I , Wallin A , Kettunen P , Molinuevo JL , Rami L , Chetelat G , de la Sayette V , Tsolaki M , Freund-Levi Y , Johannsen P , Novak GP , Ramakers I , Verhey FR , Visser PJ ((2018) ) Amyloid-β, tau, and cognition in cognitively normal older individuals: Examining the necessity to adjust for biomarker status in normative data. Front Aging Neurosci 10: , 193. |
[17] | Hassenstab J , Chasse R , Grabow P , Benzinger TLS , Fagan AM , Xiong C , Jasielec M , Grant E , Morris JC ((2016) ) Certified normal: Alzheimer’s disease biomarkers and normative estimates of cognitive functioning. Neurobiol Aging 43: , 23–33. |
[18] | Molinuevo JL , Gramunt N , Gispert JD , Fauria K , Esteller M , Minguillon C , Sánchez-Benavides G , Huesa G , Morán S , Dal-Ré R , Camí J ((2016) ) The ALFA project: A research platform to identify early pathophysiological features of Alzheimer’s disease. Alzheimers Dement (N Y) 2: , 82–92. |
[19] | Milà-Alomà M , Salvadó G , Gispert JD , Vilor-Tejedor N , Grau-Rivera O , Sala-Vila A , Sánchez-Benavides G , Arenaza-Urquijo EM , Crous-Bou M , González-de-Echávarri JM , Minguillon C , Fauria K , Simon M , Kollmorgen G , Zetterberg H , Blennow K , Suárez-Calvet M , Molinuevo JL ((2020) ) Amyloid beta, tau, synaptic, neurodegeneration, and glial biomarkers in the preclinical stage of the Alzheimer’s continuum. Alzheimers Dement 16: , 1358–1371. |
[20] | Wechsler D (2012) Escala de Inteligencia Wechsler para adultos IV (Spanish version), Pearson, Madrid. |
[21] | Wechsler D (2013) Wechsler Memory Scale-IV. Spanish edition, Pearson Clinical & Talent Assessment, Madrid. |
[22] | Partington J , Leiter R ((1949) ) Partington’s pathways test. Psychol Serv Cent Bull 1: , 9–20. |
[23] | Peña-Casanova J , Quiñones-Úbeda S , Gramunt-Fombuena N , Quintana-Aparicio M , Aguilar M , Badenes D , Cerulla N , Molinuevo JL , Ruiz E , Robles A , Barquero MS , Antúnez C , Martínez-Parra C , Frank-García A , Fernández M , Alfonso V , Sol JM , Blesa R ((2009) ) Spanish multicenter normative studies (NEURONORMA project): Norms for verbal fluency tests. Arch Clin Neuropsychol 24: , 395–411. |
[24] | Randolph C , Tierney MC , Mohr E , Chase TN ((1998) ) The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Preliminary clinical validity. J Clin Exp Neuropsychol 20: , 310–319. |
[25] | Jessen F , Amariglio RE , Buckley RF , van der Flier WM , Han Y , Molinuevo JL , Rabin L , Rentz DM , Rodriguez-Gomez O , Saykin AJ , Sikkes SAM , Smart CM , Wolfsgruber S , Wagner M ((2020) ) The characterisation of subjective cognitive decline. Lancet Neurol 19: , 271–278. |
[26] | Cavaco S , Goncalves A , Pinto C , Almeida E , Gomes F , Moreira I , Fernandes J , Teixeira-Pinto A ((2013) ) Trail making test: Regression-based norms for the Portuguese population. Arch Clin Neuropsychol 28: , 189–198. |
[27] | Daseking M , Petermann F , Waldmann HC ((2017) ) Sex differences in cognitive abilities: Analyses for the German WAIS-IV. Pers Individ Dif 114: , 145–150. |
[28] | Lynn R , Irwing P ((2008) ) Sex differences in mental arithmetic, digit span, and g defined as working memory capacity. Intelligence 36: , 226–235. |
[29] | Peña-Casanova J , Quintana-Aparicio M , Quiñones-Úbeda S , Aguilar M , Molinuevo JL , Serradell M , Robles A , Barquero MS , Villanueva C , Antúnez C , Martínez-Parra C , Frank-García A , Aguilar MD , Fernández M , Alfonso V , Sol JM , Blesa R ((2009) ) Spanish multicenter normative studies (NEURONORMA project): Norms for the visual object and space perception battery-abbreviated, and judgment of line orientation. Arch Clin Neuropsychol 24: , 355–370. |
[30] | Freitas S , Simões MR , Alves L , Santana I ((2012) ) Montreal Cognitive Assessment: Influence of sociodemographic and health variables. Arch Clin Neuropsychol 27: , 165–175. |
[31] | Sundermann EE , Maki P , Biegon A , Lipton RB , Mielke MM , Machulda M , Bondi MW ((2019) ) Sex-specific norms for verbal memory tests may improve diagnostic accuracy of amnestic MCI. Neurology 93: , E1881–E1889. |
[32] | Röhr S , Pabst A , Riedel-Heller SG , Jessen F , Turana Y , Handajani YS , Brayne C , Matthews FE , Stephan BCM , Lipton RB , Katz MJ , Wang C , Guerchet M , Preux PM , Mbelesso P , Ritchie K , Ancelin ML , Carrière I , Guaita A , Davin A , Vaccaro R , Kim KW , Han JW , Suh SW , Shahar S , Din NC , Vanoh D , van Boxtel M , Köhler S , Ganguli M , Jacobsen EP , Snitz BE , Anstey KJ , Cherbuin N , Kumagai S , Chen S , Narazaki K , Ng TP , Gao Q , Gwee X , Brodaty H , Kochan NA , Trollor J , Lobo A , López-Antón R , Santabárbara J , Crawford JD , Lipnicki DM , Sachdev PS ((2020) ) Estimating prevalence of subjective cognitive decline in and across international cohort studies of aging: A COSMIC study. Alzheimers Res Ther 12: , 167. |
[33] | O’Donoghue MC , Murphy SE , Zamboni G , Nobre AC , Mackay CE ((2018) ) APOE genotype and cognition in healthy individuals at risk of Alzheimer’s disease: A review. Cortex 104: , 103–123. |
