Reproducibility of Alzheimer’s Disease Cerebrospinal Fluid-Biomarker Measurements under Clinical Routine Conditions

INTRODUCTION

The diagnosis of Alzheimer’s disease (AD) is based on different pillars. In addition to medical history, clinical examination, neuropsychological testing, and imaging methods, analysis of the cerebrospinal fluid (CSF) plays an important role in the biomarker-supported differential dementia diagnosis. According to different international [1] and national guidelines (e.g., German “S3-Leitline Demenz” [2]) for the clinical diagnosis of Alzheimer’s disease as well as the recommendations of the “National Institute on Aging and the Alzheimer’s Association working group” [3], increased CSF levels of total-Tau (tTau) and phospho(181)Tau (pTau), and low CSF Amyloid-β₄₂ (Aβ₄₂) can be considered valid AD biomarkers and have been recommended to support an accurate diagnosis.

A non-invasive way to measure the amyloid burden in the central nervous system (CNS) display nuclear medicine imaging methods for in vivo detection of amyloid plaque density. The ¹⁸F-labelled amyloid tracer Florbetaben shows high sensitivity and specificity for the detection of cerebral amyloid plaques in correlation to histopathology [4]. Therefore, amyloid PET imaging might be a valuable addition to common CSF-Aβ diagnosis.

In addition to Aβ₄₂, the analysis of the shorter Aβ-variant Aβ₄₀ seems to be useful. Aβ₄₀ is the most abundant C-terminal Aβ-variant in CSF, followed by Aβ₃₈ and Aβ₄₂ [5]. In CSF, Aβ₄₀ correlates well with the total Aβ concentration [6] and thus can serve as a surrogate-marker for total CSF Aβ. In patients with either unusually low or high overall Aβ, the Aβ₄₂ to Aβ₄₀ concentration ratio (Aβ_42/40) can be a helpful tool to normalize the Aβ₄₂ levels to Aβ₄₀ (as a proxy of total Aβ) and thereby improve the diagnostic accuracy [7–9].

CSF Aβ_42/40 was reported to correlate more closely with tracer retention on Amyloid-PET imaging than Aβ₄₂ alone [10] and appears to be less prone to variances resulting from pre-analytical sample handling [9]. Furthermore, in our hands, Aβ_42/40 turned out to provide higher inter-assay and between lot reproducibility than CSF Aβ₄₂ alone, when assessed with a multiplex chemiluminescence immunoassay (Aβ V-PLEX, Mesoscale) [6].

Several studies and meta-analyses described that the core AD CSF biomarkers tTau, pTau, and Aβ₄₂ provide good differentiation between AD and controls as well as mild cognitive impairment (MCI) due to AD and stable MCI. Consequently, their use in clinical practice and clinical research was clearly recommended by these authors [11–13]. However, it appears that the appraisal regarding the clinical use of these biomarkers, in particular in patients with MCI, is quite controversial: For example, in their 2014 Cochrane review, Ritchie et al. did not recommend CSF Aβ levels as an accurate test of AD pathology in patients with MCI [14, 15]. It is important to note that the published biomarker studies presumably have been performed in general according to highly standardized and well controlled operating procedures and with very carefully selected clinical cohorts. Therefore, it is not obvious to what extent these optimized study conditions reflect the situation in everyday clinical routine in memory clinics or at general practitioners (GPs). Although CSF biomarkers are recommended for clinical research, McKhann et al. did not support the use of CSF biomarkers for routine diagnostics [3].

To address this question, we have compared parallel CSF biomarker measurements and their interpretations executed on a fee for service basis by two different certified clinical laboratories. To this end, parallel aliquots of 114 CSF samples were sent to two independent laboratories for the simultaneous determination and interpretation of t-Tau, p-Tau, Aβ₄₂, and Aβ_42/40 according to the respective lab-specific standard operating procedures (SOPs) and cut-off points. The measured biomarker levels and corresponding diagnostic classifications obtained from the two different sites for each one of the 114 subjects were compared and the degree of agreement between the different sites in term of the diagnostic interpretations was assessed. An important question to be addressed was whether Aβ_42/40 would show a higher level of agreement between the two sites than Aβ₄₂ alone in term of diagnostic classifications.

A small number of patients from this cohort underwent an additional amyloid-PET diagnostic to measure the central amyloid deposition as a marker of amyloid pathology. We have also compared Aβ₄₂ and Aβ_42/40 classifications from both centers with the amyloid-load as measured by amyloid-PET.

MATERIALS AND METHODS

RESULTS

We compared the results from parallel measurements and diagnostic classifications of the CSF levels of tTau, pTau, Aβ₄₂, and Aβ_42/40 obtained from two different certified diagnostic clinical laboratories under everyday clinical conditions. In total, n = 114 subjects were included in this study. Bland-Altman method comparisons suggested systematic center differences regarding Aβ₄₂ and Aβ_42/40, but not Aβ₄₀, tTau, and pTau (Fig. 1). This can be explained by the use of different assay kits at the two sites for measuring CSF Aβ₄₂ (IBL versus Fujirebio).

Fig.1

Bland-Altman plots for each biomarker. The mean value of the two biomarker measurements was plotted on the X-axis. On the Y-axis the difference between both centers (center 1 – center 2) was pictured. This presentation facilitates the direct comparison between two different centers. While no systematic differences were observed regarding the measurements for tTau, pTau, and Aβ₄₀, the measured Aβ₄₂ levels and, as a result Aβ_42/40, were generally higher in center 1.

The CSF levels of tTau, pTau, Aβ₄₀, Aβ₄₂, and Aβ_42/40 as well as the corresponding cut-off values are summarized in Table 2 and illustrated in Figs. 2 and 3. The mean concentration of CSF Aβ₄₂ was on average 233.4 pg/ml higher in laboratory 1 (1034.0±421.0 pg/ml) compared to laboratory 2 (800.6±398.3 pg/ml) (Fig. 2D). The cut-off point used by laboratory 2 (620 pg/ml) was 170 pg/ml higher than the one used in laboratory 1 (450 pg/ml). Thus, a higher number of pathological Aβ₄₂ samples was found in laboratory 2. For Aβ_42/40, identical cut-off points were used (0.05 Aβ₄₂/Aβ₄₀) at the two sites. The mean of the calculated Aβ_42/40 ratio was 0.028 higher in laboratory 1 (0.086±0.034 Aβ₄₂/Aβ₄₀) compared to laboratory 2 (0.058±0.018 Aβ₄₂/Aβ₄₀) (Fig. 2E). A substantially higher discrepancy at higher Aβ_42/40 (indicating neurochemically healthy subjects) could be seen in the Bland-Altman-Analysis (Fig. 1E).

Fig.2

Measured concentrations of tTau, pTau, Aβ₄₀, and Aβ₄₂ in both centers presented as scatter blots with the corresponding cut-off points. For Aβ_42/40, both centers had the same cut-off points (0.05 Aβ₄₂/Aβ₄₀). Interestingly, in center 2, two different subgroups, most likely AD patients and controls, can be seen, which are separated by the cut-off of 0.05 (E). In contrast, no such separations can be seen in center 1. Significant differences are indicated as **p < 0.01 and ***p < 0.001.

Fig.3

Correlation between CSF-biomarker measurements in the two different centers. Cut-off points of each center are indicated as red horizontal and vertical lines. Patients with discordantly interpreted biomarkers can be found in the upper left square for tTau (A) and pTau (B), whereas for Aβ, discordant biomarkers are mainly in the lower right square (C: Aβ₄₂ and D: Aβ_42/40). All biomarkers correlated between both centers: R² = 0.7129, p < 0.001 (tTau); R² = 0.7914, p < 0.001 (pTau); R² = 0.5078, p < 0.001 (Aβ₄₂); R² = 0.5739, p < 0.001 (Aβ₄₀); R² = 0.4308, p < 0.001 (Aβ_42/40).

Both tTau and pTau measurements of the two centers were strongly correlated (tTau: R² = 0.7129, p < 0.0001; pTau: R² = 0.7914, p < 0.0001) (Fig. 3A, B). However, the diagnostic interpretations (normal CSF level versus pathological level) obtained from the two different laboratories were discordant in 29.6% of the cases with regard to tTau and in 25.0% of the patients with regard to pTau. The Aβ₄₂ and Aβ₄₀ measurements of the two different laboratories also showed statistically significant correlations: R² = 0.5078 (p < 0.0001) for Aβ₄₂ and R² = 0.4308 (p < 0.0001) for Aβ_42/40, respectively (Fig. 3D, E). CSF Aβ₄₀ correlated with R² = 0.5739 (p < 0.0001) between both centers (Fig. 3C). Although the cut-off points were different in both centers, this could not explain the different interpretations completely: parallel measured CSF aliquots were significantly different for pTau, Aβ₄₀, Aβ₄₂, and Aβ_42/40. Only tTau measurements were similar in both centers (Fig. 2, Table 2).

The diagnostic interpretations of the Aβ₄₂ CSF levels (normal versus pathological range) obtained by the two different sites were discrepant in 31.5% of the cases. The corresponding Cohen’s kappa (κ), as a measure of inter-rater agreement, was 0.170 indicating a poor agreement between the two different clinical laboratories [16]. In contrast, the diagnostic classifications of the subjects according to the Aβ_42/40 ratio were discordant in only 16.8% of all cases, with κ= 0.589 indicating a moderate agreement. For tTau and pTau, calculated values of Cohen’s kappa were 0.449 and 0.528 (moderate agreement), respectively.

For a subset of the subjects included in this study, additional in vivo amyloid PET imaging data was available. Seven out of ten patients showed an increased amyloid-load within the CNS. Table 3 summarizes the degree, to which the diagnostic interpretations of the CSF Aβ₄₂ levels and Aβ_42/40 obtained from the two different sites were in line with the observations from amyloid PET imaging. On average, in only 55% of the cases both, CSF biomarkers and amyloid imaging suggested concordantly either absence or presence of brain amyloid deposition.

DISCUSSION

In this study, we report on the comparison of parallel CSF-biomarker measurements in a clinical cohort and their diagnostic interpretations executed under everyday routine conditions in two different clinical laboratories on a fee for service basis. In a sample of more than 100 patients, tTau, pTau, Aβ₄₂, and Aβ_42/40 were measured in parallel in two different laboratories according to the respective SOPs and the measured concentrations were classified according to the respective laboratory-specific cut-off points.

We observed substantial discrepancies between the two centers—with regard to the reported CSF Aβ₄₂ concentrations and the Aβ_42/40 ratios, which can be easily explained by the use of different assay kits for measuring Aβ₄₂. More importantly, the diagnostic interpretations, provided by the different centers, were often discordant, not only for CSF Aβ₄₂ but also for tTau and pTau. The classifications (normal range versus pathological range) made by the two centers were discrepant in 25.0% of the cases with regard to pTau, in 29.6% of the cases for tTau and in 31.5% of the cases regarding Aβ₄₂. Considerable inter-laboratory variability in the immunological measurements of tTau, pTau, and Aβ peptides in CSF has been reported previously, also in carefully conducted multicenter quality survey studies [17, 18]. In particular, the reliable and reproducible determination of CSF Aβ₄₂ is known to be technically challenging, and consequently efforts on standardization, including, for example, the development of certified reference materials and reference measurements procedures, have been launched [19].

In the samples studied here, the diagnostic classifications of the Aβ_42/40 concentration ratios showed substantially less discordance between the different centers than those of the CSF Aβ₄₂ levels. This observation further supports Aβ_42/40 to represent the better neurochemical biomarker of pathological cerebral Aβ accumulation than Aβ₄₂ alone [9, 20, 21].

The incorporation of additional diagnostic techniques is particularly important in the case of patients with atypical CSF-constellations or clinical appearance. PET-imaging is expensive and extensive, but might lead to an improved diagnostic accuracy in some individual cases.

In the present cohort, the agreement between CSF biomarkers and Amyloid-PET imaging was moderate. Overall, 55% of the CSF biomarker interpretations were in line with the Amyloid-PET results, whereby the agreement between the different centers and biomarker modalities (Aβ₄₂ versus Aβ_42/40) were varying. However, there was only a small number of imaging results available. Mattsson et al. showed that the correlation between CSF Aβ₄₂ and amyloid load, measured with Florbetapir, depends on the diagnosis and clinical stadium [22]. In cognitively healthy subjects, 21% of all cases showed discordant results between Aβ₄₂ and Amyloid-PET with mainly pathologically decreased Aβ₄₂ but normal Amyloid-PET results. In patients with AD, only 6% discordant cases were observed. Hence, the authors concluded that decreased Aβ₄₂ in CSF without Amyloid-PET abnormalities corresponded to early stages of AD, whereas Amyloid imaging changes to pathological in progressed stages [22]. The high discrepancy between the Amyloid-PET results and the measured CSF-biomarkers we observed here could be due to diagnostically inconclusive patients. In this cohort Amyloid-PET was not routinely performed, thus mainly clinical atypical or unclear patients underwent an additional Amyloid-PET.

As Amyloid-PET might not yet be available at many diagnostic sites, most centers will most probably make the diagnosis of AD supported by CSF-biomarkers. This study clearly reveals the importance of improving the possibilities to interpret CSF markers in subjects in order to reduce false negative and particularly false positive AD diagnoses, an issue of enormous importance to subjects suspected to suffer from AD.

In view of the here observed discordance in the diagnostic interpretation of CSF-biomarker data of up to 31.5% between the two centers, there is a great need for multicenter efforts to define valid cut-off points and to harmonize biomarker data interpretation. Different efforts on biomarker harmonization studies have been done so far. For example, a quality control program has been established, the Alzheimer’s Association QC program for CSF biomarkers (http://www.neurochem.gu.se/TheAlzAssQCprogram) [23, 24]. In addition, Bjerke et al. studied certified reference materials for the comparison between different assay techniques, which might be a useful tool for quality controls [25]. Besides, more and more efforts are made on the implementation of fully automated analyzers for CSF measurements [19, 26–28]. Recent data suggest the superiority of these fully automated electrochemiluminescence immunoassay, such as the novel Elecsys β-amyloid (1–42) assay by Roche Diagnostics, over classical ELISAs [29]. Besides Roche Diagnostics, Fujirebio (Lumipulse G600II) and Euroimmun (EUROIMMUN RA Analyzer 15) also provide fully automated immunoassays. All of these systems are supposed to provide a minimal lot-to-lot variation and thus might lead to a higher agreement between different centers and measurements. Further studies on fully validated systems are still pending to evaluate the benefit of automated immunoassays.

Nevertheless, Mattsson et al. could clearly demonstrate that the CSF biomarker variability is too high to calculate general cut-off values using the classical immunoassay based assays [28]. Experiences with fully automated immunoassays to determine general cut-off values are still insufficient. Therefore the determination of center-specific cut-off points, which need to be adapted within certain periods, is of high importance [30].

With a prevalence of estimated 5.4 million affected AD persons in the USA in 2016 [31], there is a substantial need for reliable CSF diagnostics. Thus, there is a big need for an improved inter-laboratory standardization. Since this study did not focus on the reason for the observed discrepancies (e.g., transport, SOP, methods, reference values), further research on the identification of influencing factors and consecutively optimization is required.