In-Out-Test: A New Paradigm for Sorting the Wheat from the Chaff in Prodromal Alzheimer’s Disease

INTRODUCTION

Mild cognitive impairment (MCI) is defined as a decline in cognitive functioning that is greater than would be expected for a patient’s age and educational background and that goes beyond normal changes seen in aging [1]. Various cognitive domains can be affected including learning and memory, attention, executive function, language, and visuospatial skills. MCI is viewed as a transitional state between normal aging and dementia; the annual progression rate to dementia is 10% in clinical settings and 5% in community settings [2]. Conversely, reversion rates of 23% and 10% are seen from population and clinical studies, respectively, implying this is a dynamic diagnosis.

There is some evidence to suggest a nutritional intervention may be beneficial in this early stage of disease [3]. Furthermore, with other disease-modifying therapies likely to become available in the next years, it is important to be able to identify which patients with MCI will go on to develop Alzheimer’s disease (AD). Currently cerebrospinal fluid (CSF) biomarkers are relied upon, but patients may be unwilling to undergo a lumbar puncture or facilities for analyzing samples may not be available. Thus, there is a great need for a neuropsychological assessment that could detect such patients with accuracy.

Characteristic of AD and prodromal AD (MCI due to AD) is amnestic syndrome of hippocampal type (HA) which is a recall deficit that does not significantly improve with cueing or recognition procedures, after effective encoding of information [4]. Other neuropsychological deficits and subtle changes in the activities of daily living (ADL). However, it is widely accepted that AD is characterized by the presence of HA.

The incorporation of biomarkers has been instrumental in the early diagnosis of AD posing a challenge to the Neuropsychologist to design sensitive and accurate tests to predict whether a patient consulting with memory complaints will go on to develop AD and subsequent dementia [5]. Many investigations have looked at how neuroimaging, changes in the CSF biomarkers, and serum and blood markers may predict the risk of conversion from MCI to dementia. Thus, cerebral atrophy, hypometabolism, and amyloid burden have been suggested as neuroimaging hallmarks, and decreased levels of amyloid-β₄₂ (Aβ₄₂) in CSF, in parallel with changes in the ratio total tau (t-tau) to Aβ₄₂ and phosphorylated tau protein (p-tau) to Aβ₄₂ as biochemical indicators. However, these values are not sufficiently accurate, and there is still high controversy regarding the criteria to establish in terms of CSF biochemistry, the onset and longitudinal changes of CSF biomarkers [6–10].

Obviously, further characterization and deeper studies are required. For instance, it has been thought for a long time that changes in CSF biomarkers precede the symptoms of MCI [11]. Thus, it seems that classical symptoms-based diagnostic criteria of AD lack of sufficient sensitivity for detecting “hiding clinical signs” at prodromal and preclinical stages of the disease. In addition, current data show a wide variation on the cut-off of the levels of Aβ₄₂, t-tau, and p-tau making it difficult to determine the reliability of these biomarkers.

Recently, the complexity of standard clinical diagnostic tests [12–14] have been increased in the attempt to increase sensitivity and specificity in the early diagnosis of AD. In line with this, the results of the SOMI (Stages of Objective Memory Impairment) are promising, by showing the ability of the Free and Cued Selective Reminding Test (FCSRT) to retrospectively describe the temporal unfolding of declining episodic verbal memory in predementia and AD [15].

Other paradigms have been proposed to evaluate episodic memory. Rentz et al. designed an associative encoding task that comprised faces paired with fictional first names. The authors describe a pattern on functional MRI activation during the encoding which is differentially altered in the early stages of AD compared with normal aging. Recently, Rentz et al. found that performance on face-name retrieval was associated with Aβ burden in cognitively normal older individuals [16]. These results open a promising line of investigation in individuals with normal cognition. However, the studies involved only individuals with higher education and this can limit the external validity of the results. In addition, as AD progresses, tests that are sensitive at the preclinical stage may be challenging to the subject at the MCI stage. For this reason, shorter versions of these tests have been developed by others [17].

Different mechanisms might interfere with setting the threshold for detecting amnesia and other cognitive deficits by conventional neuropsychological tests in the early stages of AD. Those mechanisms show paradoxical features for degenerative processes, such as hypertrophy and hyperactivity, which may reflect compensated cognitive performances. In degenerative processes, the loss of both neurons and neural connections is preceded by different aspects, including nuclear and body cell hypertrophy, higher amount of dendrites, and increased dendritic size and length in both the hippocampus and neocortex due to the action of neurotrophins binding to receptor p75 [18]. Paradoxical changes are also found in functional neuroimaging studies [5] and Aβ burden measured by PET [7]. In patients with MCI, who later converted to dementia, there was an increased neuronal activity in both, functional MRI [19–25] and magneto-encephalography in areas other than the medial temporal lobe (MTL), particularly in frontal and parietal lobes [26–28]. The increased activity might start in the hippocampus of the same hemisphere in very early stages of MCI, when these structures have only partial lesions [10, 29, 30]. A recent meta-analysis study has reported that both patients with AD and those at significant risk showed statistically significant consistent activation differences in the MTL relative to controls during the episodic memory [31]. Furthermore, it has been observed that cerebral hyperactivation acts as a compensatory mechanism activated in response to neuronal damage at the onset of AD. In this order of ideas, a longitudinal study comparing individuals carrying the apolipoprotein E4 isoforms gene (APOE ɛ4) to non-carriers, demonstrated that the subjects carrying the APOE ɛ4 gene had higher cerebral activity that decreased within the following five years, a phenomenon that was accompanied by medial temporal atrophy and decreased cognition [32].

The fact that most available tests of memory and executive function, taken individually, do not accurately distinguish patients with prodromal AD from patients with subjective cognitive impairment (SCI) or non-AD MCI, supports the idea that early MTL lesions may be largely overcome by other brain areas in an exceptional manner. Indeed, these other brain areas may play a significant role in non-mnemonic cognitive functions, independent of MTL activities in healthy people. Taking into account these observations, it could be suggested that networks involved in executive functions may take the command of memory during prodromal AD so consequently, dual tasks might be good candidate tests to distinguish converters from non-converters. With this in mind, it was hypothesized that memory deficits underlying prodromal AD might be unveiled if simultaneous executive tasks are used to engage such neuronal networks which support memory encoding in MTL. Despite many investigations of memory performance in a dual task context, to the best of our knowledge, no reliable strategy has been successfully established so far. Most attempts have included immediate and working memory in dual tasks [33] with the absence of significant differences between MCI and healthy elderly people when performance is adjusted to individual abilities [34]. Considering that short- and long-term episodic memory are more relevant than immediate and working memory in AD, both for diagnosing dementia and predicting conversion to dementia, a new paradigm has been developed. The In-out-test was designed to assess episodic memory along with a simultaneous executive task thereby interfering encoding by non-mnemonic cognitive functions [35]. In contrast to tests that evaluate memory after effective encoding of information (FSCRT), using the HA paradigm, the In-out-test makes an inference at the moment of encoding through simultaneous performance of an executive task.

A pilot study was conducted investigating 110 subjects using successive assessments over three years. Conversion to dementia based on clinical signs, impaired ADLs, and Mini-Mental State Examination (MMSE) score were measured. Conversion to dementia from MCI or SCI was predicted using the In-out-test with a sensitivity and specificity of 0.90, and 0.94, respectively, and with a positive predictive value (PPV) of 0.90 and a negative predictive value (NPV) of 0.94 [35].

The objective of the present study is to evaluate the validity of the In-out-test in identifying which patients with MCI have prodromal AD and will likely go to develop a dementia. To test this, the study compared the outcomes of the In-out-test with the outcomes of the FCSRT and also evaluated the degree of correlation between the In-out-test and the FCSRT with the CSF-biomarkers of AD.

METHODS

In-out-test

This is a new paradigm that assesses episodic memory after the patient performs a categorization task and memorizes 6 words simultaneously. The time taken to complete this test was 5 to 12 minutes. Three parts compose the In-out-test: Learning of the Organized Series (LOS), Memory of the Organized Series (MOS), and Random Memory (RAM). A participant performing the In-out-test views an image of six rows each made of two squares. One square of the row contains a drawing of a house and the other is black (Fig. 1). Every time the evaluator says the name of an object the participant repeats the name while pointing to either the house or the black square to indicate the usual location of the object, inside or outside the home. The task is repeated for each object from the bottom to the top until the participant gets the 6 names and locations correctly. The participant is then asked to remember and say the names of the objects in the given sequence while pointing to the corresponding square. This learning task is repeated up to ten times, or until the participant reaches 3 total consecutive ordered series, or after 5 attempts if the participant does not remember any word. After a distraction period (2.5 to 3 minutes) by a non-related task, the participant is asked to resume the In-out-test learning part by saying in the right sequence the name of the objects. In the case that the participant is not able to reproduce the complete series, he/she is asked to recall as much words as possible without taking into account the order. Scoring: the total items remembered in LOS is multiplied per 0.20 (maximum possible punctuation: 12); MOS is multiplied per 0.5 (maximum possible punctuation: 3); RAM is multiplied per 0.5 (maximum possible punctuation: 3); one extra-point is given when the participant reaches at least one complete series in LOS and another extra-point if he/she reaches three consecutive series. The maximal score is 20 points resulting of the following sum: 12+3+3+1+1.

Fig.1

Work template of In-out-test.

RESULTS

The sociodemographic characteristics and functional features of the total sample are shown in Table 1. There were no significant differences in age (U = 29.0; p = 0.10), education (U = 37.5; p = 0.30), anxiety symptoms (U = 49.5 p = 0.86), or depressive symptoms (U = 32.00; p = 0.16). Neuropsychological performances in CH (n = 16) and SCI (n = 16) were statistically similar and thus were joined into the NPsych group (n = 32). No statistically significant differences between the three groups were observed for formal education (F = 1.84; p = 0.1700), anxiety symptoms (F = 0.16; p = 0.8470), and depression symptoms (F = 0.20; p = 0.8180). Conversely, significant differences were detected for age (F = 6.46; p = 0.0040). Post-hoc LSD test revealed significant differences between NPsych and MCI (p = 0.001) and between NPsych and dementia group (p = 0.001).

The internal consistency of the In-out-test showed a Cronbach Alpha of r = 0.81 for the total scale, the index being smaller if the total score was excluded (α= 0.61). The reliability between evaluators was k = 0.94. ICC for the test-retest reliability was r = 0.57 (p = 0.575). The ICC for the evaluation of the concurrent validity between the In-out-test and the FCSRT was r = –0.87 (p = 0.001).

The three groups exhibited varying performances on the three different parts of the In-out-test. Thus, ANOVA analyses rendered significant differences for total score (F = 107.192; p = 0.01), LOS (F = 94.227; p = 0.01), RAM (F = 76.612; p = 0.01), and MOS (F = 47.097; p = 0.01).

The concordance between the In-out-test and levels of CSF biomarkers in NPsych, MCI, dementia, and total sample are shown in Tables 2 and 3. Also, the concordances between the p-tau/Aβ₄₂ ratio for different groups and the different cut off values for the In-out-test were calculated. Thus, a high degree of correlation between In-out-test total score with CSF biomarkers (Table 2) for healthy (Aβ₄₂ r = 0.73, p = 0.01 and p-tau/ Aβ₄₂ r = 0.75, p = 0.01); MCI (Aβ₄₂ r = 0.64, p = 0.01 and p-tau, r = 0.77, p = 0.01); DAT (Aβ₄₂ r = –0.71, p = 0.01), as well as for the total sample (Aβ₄₂ r = 0.67, p = 0.01 and p-tau r = 0.70, p = 0.01). In addition, we could detect significant relationships between learning tasks and p-tau (Table 3). Table 4 shows the correlation between FCSRT with CSF biomarkers. Further, we observed that these relationships were not linear, but rather organized in ranges (Fig. 2). Indeed, a high correlation with p-tau up to 50 pg/ml (r = 0.762, p = 0.01) and above 80 pg/ml (r = –0.740, p = 0.01), was observed, but not in the range 50 to 80 pg/ml (r = 0.30, p = 0.01).

Finally, the diagnosis of AD using the 6 words recalled (RAM6) together with the p-tau/ Aβ₄₂, taking into account the predetermined cut-off (0.12) by our laboratory (Table 5), was compared. Individual performance on the six words in a random way (RAM6) showed the highest concordance (k = 0.71) (Table 5). Those using the FCSRT had a concordance of k = 0.49. When diagnosing prodromal AD, the concordance for the In-out-test was a k = 0.69 (Table 6) and for the FCSRT a k = 0.44. In addition, sensitivity, specificity, predictive values and likelihood ratios were also calculated from the tables of contingence (Tables 5 and 6).

Fig.2

Relationship between the learning task of In-out-test and p-tau levels in CSF.

DISCUSSION

The present study introduces a novel paradigm to assess the risk of conversion to dementia, the In-out-test. Unlike other tests using the HA paradigm, such as the FCSRT, which assesses memory after effective encoding of information, the In-out-test provokes an interference during encoding by means of a simultaneous executive task. This paradigm is proposed based on the compensatory functional networks which are believed to be widespread in early stages of AD. If parts of those networks that are usually involved in other functions take the command of memory during the earliest stages of AD, as proposed in the new compensatory brain theories, it can be predicted that these networks would be less efficient in these persons than those with the same memory complaints, but not suffering the disease and, therefore, do not need these auxiliary networks. According to this model the memory deficit in prodromal AD can be unmasked when those compensatory networks are saturated by an executive activity without being able to compensate the already damaged mnemonic activity.

Even though neuropsychological tests oriented to detect HA can distinguish cognitively healthy, MCI, and dementia groups these tests lack the ability to separate early AD from non-progressive MCI consistently. To overcome the limitations for diagnosing prodromal AD, some authors have increased the complexity of the original tests [13]. Other studies have used different models to quantify amnesia. Grober et al. propose that the FCSRT free recall is the best measure for detecting prevalent dementia and predicting future dementia, and that the “total recall impairment” component of the test discriminates between AD and non-AD dementia [61]. In a recent study, researchers from this same group used the sum of both scores to retrospectively track the progression of patients already diagnosed with dementia [15]. In this and other previous studies that use the hippocampal memory paradigm [62], it is shown that free recall can be an early marker of later impairment. These data can be useful in clinical trials where memory performance is assessed by means of such a paradigm.

The authors of the Memory Binding Test propose a model that amplifies the numbers of items to memorize as they consider the FCSRT is limited by the presence of a “ceiling effect” [12, 13]. Despite the fact that the conventional tests have acceptable sensitivity and specificity, their low predictive ability makes them unsuitable for clinical settings. For example, in a study assessing conversion to dementia, Auriacombe et al. report a PPV of 7 for FCSRT [63]. Clinically, it means that only 7% of the patients with positive results will have dementia in the future and more than 90% with positive results will not have dementia. In a recent study, Derby et al. report a PPV of 30 [64].

The objective of this study was to evaluate the validity of the In-out-test for identifying prodromal AD. First, performance on the In-out-test and that on the FCSRT were compared. FCSRT was chosen as several studies have demonstrated its reliability for the diagnose of AD in addition to biomarkers [4]. It was of particular interest to administer the In-out test with the approach of naturalistic studies to increase the possibility to extrapolate results to future evaluations in clinical setting.

Preliminary results suggest that the In-out-test is not affected by the formal education and does not require reading and writing skill for its proper execution. However, the present study included reading and writing skills as participation criteria.

Fig.3

Relationship between the learning task of In-out-test and p-tau levels in CSF.

The In-out-test accomplishes the basic principles for construct, criterion validity, and classic psychometrics, as well as reliability, with high correlation indexes. Based on such results, the In-out-test can be considered as a valid, reliable, and easily applicable tool for the diagnosis of AD. The results showed high correlations between the three parts of the In-out-test (LOS, MOS, and RAM) with FCSRT across the total sample. There were significant differences between cognitively healthy, MCI, and patients with dementia. Furthermore, there was no influence of age or formal education in NPsych, although differences were detected in the total sample. This result might be related to the increased age of the participants in the dementia group and this, in turn, may be related to fewer years of compulsory education in that group. Participants were recruited consecutively in the ongoing PROA project, which is working on homogenizing the ratio of age and years of education in the future. In agreement, no differences were found in the pilot study [35].

In the second part of this study the relationship of the In-out-test performance with levels of CSF biomarker was analyzed. ICC showed a high concordance between two different measures. Thus, the results obtained in the In-out-test were in good agreement with the standardized values of CSF biomarkers, specifically in p-tau for the MCI and Aβ₄₂ for NPsych and dementia. Therefore p-tau value can be inferred with high reliability in MCI patients from performance on the In-out test. Likewise, the Aβ₄₂ value can be inferred in cognitively healthy and dementia groups (Tables 2 and 3). Conversely, concordance of FCSRT performance and CSF biomarkers was very poor (ICC <0.3) both for the total sample and within the different groups (Table 4). Cohen’s Kappa was used to diagnose AD and prodromal AD. Furthermore, the In-out-test was a better predictor than FCSRT, with good agreement in the first and moderate for FCSRT both for the diagnosis of AD and for prodromal AD.

There was a significant coefficient correlation between the In-out-test performance and p-tau levels (Fig. 2) and, notably it was also observed that correlations varied depending on p-tau ranges (Fig. 3). Thus, a statistically significant positive correlation was found between the In-out-test to p-tau in NPsych being 50 pg/ml the cut off level. There was a gap between 50 pg/ml to 80 pg/ml without any significant correlation, which is followed by a strong negative correlation beyond 80 pg/ml (Fig. 3). This is particularly interesting since the cut off level of 50 pg/ml is in agreement with some previous reports. Thus, both Mogekar [11] and, independently, Höltta [65] reported that patients who converted to MCI or dementia exhibited a mean p-tau level of 49 pg/ml. The positive relationship between p-tau and the In-out-test up to 50 pg/ml seems paradoxical, as the performance improved in parallel with a rise of p-tau levels as a marker of neurodegeneration (p-tau) rises as well. In agreement, in a longitudinal study by Seppälä and colleagues [66], a slight increase of p-tau in healthy and stable MCI participants along a three years follow-up was reported. They also showed that changes in biomarkers occur at different paces in each stage of the AD continuum. The increase of p-tau reaches its maximum in dementia patients, to a lesser degree in MCI, and are lowest in healthy controls [66]. The authors found a positive correlation between the results in the In-out-test and p-tau up to 50 pg/ml. Up to 50 pg/ml the low speed in neuronal loss in healthy individuals, might allow the activation of compensatory mechanisms as shown by Rao et al. [32]. The limit of 50 pg/ml is also in agreement with the upper limit in p-tau levels in the control group in a study on TREM2 and AD (49 pg/ml) [67]. The low correlation in the 50–80 pg/ml gap might be a consequence of the accelerated neuronal loss in MCI and AD patients (intermingled in this range) with a high interindividual variability in the remaining compensatory mechanisms typical of aging and others due to AD. Finally, it can be speculated that the total loss of compensatory mechanisms makes performance on the In-out-test only dependent on the neuronal loss as measured by p-tau levels (>80 pg/ml).

Although the results of this study suggest that the In-out-test might be useful for accurate clinical decisions, there were of course some limitations worthy of discussion.

Firstly, the sample size was rather small in particular with respect to the CSF biomarkers. Although a sample size calculation informed that there were sufficient participants in each group for the In-out test. Secondly, there were some variations in the composition of the groups: there were more women than men, the individuals in the dementia group were older (but this is to be expected), and the level of education of participants was below. Finally, some participants were taking antidepressants and anxiolytics although they did not have severe depression.