Strategies Associated with Retaining Participants in the Longitudinal National Alzheimer’s Coordinating Center Uniform Data Set Study

INTRODUCTION

Longitudinal cohort studies play an essential role in advancing our understanding of the onset and progression of Alzheimer’s disease (AD) and related dementias (ADRD). Participant attrition, however, poses a challenge for ADRD researchers as it reduces statistical power to detect effects, increases the potential for bias in estimated treatment effects, and limits the generalizability of study findings [1]. In some instances, threats to internal validity may occur through differential loss to follow-up or nonignorable missingness [2, 3], which can then lead to incorrect inferences of observed estimates. Given investments in ADRD cohort studies, identifying effective strategies to improve participant retention is of paramount importance.

Considerable efforts have been made over the years to examine the effectiveness of various retention methods. A previous systematic review [4] of several established retention strategies demonstrated that offering greater amounts of participant incentives (cash or gift) and employing various modes of contact such as reminder letters improves retention. The effectiveness of some approaches was, however, observed to vary depending on the age of the cohort. Some studies have suggested that offering social support to families of participants, particularly those with dementia, could improve retention in AD cohorts [5–8]. Indeed, the evidence shows that family members become more involved in decisions about medical care over the course of the study period as patients progress with their disease [9]. This suggests that optimal communication about study procedures and support resources, in addition to more intensive follow-up and contact, become increasingly important for retention. A more recent systematic review and meta-analysis [10] of 143 longitudinal studies and 95 different retention strategies found that employing the barrier-reducing strategy of offering alternative methods of data collection to participants was the strongest predictor of retention. Notably, the authors also observed that the greater number of retention strategies used was not associated with improved retention, suggesting that a more parsimonious selection of retention strategies, perhaps based on participant barriers and characteristics, can be both cost-effective and impactful.

With regard to characteristics of participants, there is perhaps no greater need than to improve the retention of racial and ethnic groups who have been historically underrepresented in ADRD research [11]. Of the limited studies in older racial and ethnic groups, findings underscore the importance of maintaining strong and active community relationships [12], fostering cultural competence among study personnel who actively engage with culturally diverse participants and offering social support to families of participants with dementia as well as accessible locations for study assessments [13–15]. The available evidence, however, is scarce and often lacking the methodological rigor to draw firm conclusions into best strategies [16]. More work is needed to offer additional insight.

Improving retention using effective strategies in AD studies is a national and public health priority. We have previously reported a positive association between the number of employed retention tactics at Alzheimer’s Disease Research Centers (ADRCs) and participant follow-up rates using longitudinal data from the National Alzheimer’s Coordinating Center Uniform Data Set (NACC-UDS) [17]. In the present study, we expand upon these findings to: 1) evaluate the associations of 12 domain-specific retention strategies with participant retention and 2) explore whether the association of a select group of strategies hypothesized to improve participation of special populations varies by race and ethnicity or by diagnostic group.

MATERIALS AND METHODS

RESULTS

Study participants had an overall mean age of 70 years at enrollment and were 79% NH White (n = 4,515), 6% Hispanic/Latino (n = 335), 11% Black (n = 651), and 4% Asian (n = 214). Figure 1 depicts the distribution of race and ethnicity across the 24 study sites. Over 65% of Asian participants came from 3 study centers. Approximately 60% of Hispanic/Latino participants came from 4 centers and 40% of Black participants came from 3 centers.

Fig. 1

Histogram depicting the proportion of individuals from our study sample (n = 5,715) distributed across the 24 Alzheimer’s disease research centers by race and ethnicity. NH, Non-Hispanic.

We observed differences in the distribution of participant characteristics by race and ethnicity (Table 1). Black participants were more often female and less often married. More than half of all Hispanic/Latino participants had attained a high school education or less (52%) and, compared to the other racial and ethnic groups, included a greater proportion diagnosed with dementia. Black, Hispanic/Latino, and Asian participants less frequently enrolled with a spousal study partner when compared to their NH White counterparts. Retention at the first follow-up visit also varied appreciably by race and ethnicity; Hispanic/Latino and Black participants exhibited the lowest retention rates (62% and 63%, respectively) and NH Whites had the highest rate (74%).

Associations between strategies and participant retention are shown in Table 2. In multivariable models, individuals at the highest tertile strategy scores had significantly greater odds of retention when compared to those at the lowest tertile scores for strategies that involved personnel characteristics (adjusted odds ratio [aOR] = 1.61, 95% confidence interval [CI]: 1.05–2.47) and study description (aOR = 1.55, 95% CI: 1.03–2.35). The association involving personnel characteristics had an approximately linear trend across tertiles of scores. Sensitivity analyses showed that no single tactic drove this association. For the strategy involving study identification, we observed that individuals at the highest tertile of scores were less likely to be retained when compared to those at the lowest tertile (aOR = 0.59, 95% CI: 0.37–0.95). No other significant associations were observed.

Stratified analyses by race and ethnicity are shown in Table 3. Across each ethnoracial group, associations between scores of selected strategies and participant retention were generally positive, albeit not reaching statistical significance for most effect estimates. Although the associations appeared to be stronger for Asians compared to other groups, we did not find evidence for effect modification by race and ethnicity (all p for interactions > 0.05). Sensitivity analyses were conducted in which multivariable models serially excluded the 3 study centers with the highest proportion of Asian participants (≥15%) and showed that effect estimates did not attenuate or appreciably change (not shown). When we considered effect modification by diagnostic group (Table 4), stratum-specific multivariable models showed generally positive associations. We found no evidence of statistical interaction by diagnostic group (all p for interactions > 0.05).

In exploratory analyses that assessed associations of tactics with participant retention using CART, we found that having off-site locations, a phone tree, a diverse staff, and an active blog best partitioned the data on retention (Fig. 2). When we added the variables for race and ethnicity and diagnostic groups separately to the CART model, off-site locations and phone tree tactics consistently partitioned the data before these variables. In sensitivity analyses using multivariable models that adjusted for the same variables as in the main effect models, we found that off-site locations and phone tree tactics were positively associated with participant retention and did not vary significantly by race and ethnicity or diagnostic groups (not shown).

Fig. 2

Classification and regression tree (CART) results using 76 retention tactics. The data were partitioned on being retained (yes/no) during the two-year follow-up period. The numerators indicate the number of individuals retained or not retained, and the denominators indicate the total number of individuals in the subpopulation.

DISCUSSION

In a sample of 5,715 participants of the NACC-UDS study enrolled across 24 ADRCs, we found that specific strategies focused on study personnel and study description were associated with a higher odds of participant retention. These associations were independent of potential confounding factors and did not vary significantly across racial and ethnic or diagnostic groups. Overall, these findings are consistent with previous data [4, 10] and suggest that, in the setting of limited resources, some retention strategies may be more valuable than others.

The study personnel strategy includes tactics such as maintaining a diverse staff, engaging staff in regular retention trainings, and ensuring that specific staff members work with specific participants over time. The lack of effect modification by race and ethnicity or diagnostic group may suggest that all participants stand to gain in terms of increased retention through implementation of this strategy. Indeed, staff who understand and appreciate the target populations’ cultural practices, beliefs and values is among the most pivotal features of a successful retention strategy [23]. These results are also consistent with previous studies reporting improvements in retention of older underrepresented racial and ethnic participants by consistently using the same interviewers or field staff over time [24, 25]. The current results suggest this tactic may be valuable for retaining all participants.

Our data showed that implementing a strategy that enhances study description, communication of study requirements and details, was associated with improved participant retention. Establishing expectations and helping participants understand and plan for study requirements/burden during recruitment and the informed consent process may be essential for maintaining participant satisfaction. In fact, every ADRC indicated that they employed some of these tactics. Greater variance (and therefore greater opportunity to contribute to the observed effect) was observed in the use of family conferences during recruitment and study participation, using decision aids to facilitate consenting, and incorporating video tools to explain the overall study or specific procedures [17]. Ensuring understanding of the study goals and requirements, perhaps especially after enrollment may be key. For example, in a large longitudinal study of Black women, successful retention strategies involved frequent contact about study procedures with individuals who were more likely to be poor and mobile [26].

We found no evidence for effect modification of associations between selected strategies and participant retention according to diagnostic groups. This was somewhat surprising for the strategy focused on study benefits, given that we expected participants with dementia to differentially benefit from tactics that offer social support services and make study test results such as routine laboratories, neuropsychological measures, and biomarker results more accessible to them and their families. A possible explanation for the lack of interaction is that these benefits were viewed as universally important to participants, regardless of their diagnostic category [27]. Alternatively, some of the tactics with the greatest potential for effect modification (e.g., access to a clinical care or a social worker) were employed by a very small number of ADRCs. It is also possible that, for this population of participants diagnosed with MCI or dementia at baseline, other strategies and tactics were more important for retention. More research is needed to understand the optimal means of achieving successful retention across levels of disease severity.

In exploratory analyses, CART identified holding study visits at off-site locations, presumably ones more convenient to participants, as the tactic most effective (i.e., highest on the regression tree) at separating retained from lost participants. After that, using a phone tree to maintain contact with participants, having a diverse staff and offering an active blog for participants (a study identity tactic) best separated the data. A potential strength in using CART is that it accounted for any potential interactions between the 76 tactics tested. Though exploratory, these results may suggest that the most practical approaches to retention—making visits convenient and ensuring modes of contact—are of greater value than unique tactics within other strategies.

There are several limitations in the present study. Quantification of retention efforts was at the center level, rather than the participant level. Thus, interpreting the value of observed associations to retaining individual participants must be done with caution. The timing, duration, and intensity of exposure to retention tactics at each center were not ascertained. Despite the longitudinal nature of the study, the timing of exposure would be necessary to establish a causal relationship. Additionally, it is possible that differences in duration and intensity of specific strategies could potentially impact participant retention. For example, building and sustaining trusting relationships in the community takes time and considerable investment [28]; thus, a more established strategy of community involvement might lead to better participant engagement than a nascent one. Our survey was limited to binary indications for use of strategies; these types of effects could not be assessed. We only assessed a subset of retention strategies, selected a priori, for potential interaction effects with subgroup categorizations. We did this in an effort to reduce the risk of multiplicity, but it is conceivable that interactions could have existed for other retention strategies. Assessment of potential effect modification by race and ethnicity was also limited by disproportionate contribution of specific racial and ethnic groups by a small number of centers. To explore potential bias resulting from this skew, we conducted sensitivity analyses whereby centers with high proportions of specific racial and ethnic groups were serially omitted in stratified models that assessed interaction by race and ethnicity. Our findings showed consistency in the stratum-specific effect estimates. Another potential limitation was our approach to quantifying strategy exposure using count-based summary scores that gave equal weighting to tactics within each strategy. It is possible that specific tactics within strategies may have carried higher weight and were diluted in the scoring.

Some of the observed associations were counterintuitive, such as greater use of study identification strategy being associated with poorer retention. Included in this strategy are tactics that attempt to create a “study identity” for participants in an effort to retain them; for example, use of a center logo or moniker that is easily recognizable, maintaining a social media presence via Facebook, Twitter, YouTube, or blog, and publishing electronic or print newsletters for participants to read. Maintaining a study website and distributing electronic or print newsletters, were performed by every or nearly ever center [17] and therefore were unlikely to contribute to the unexpected relationship. The remaining tactics included maintaining a blog and social media channels and engaging in live webinars or other remote opportunities for participants to give feedback and have questions answered. These types of activities were increased and may have become essential during the COVID-19 pandemic, which occurred after the collection of the study data. It is also possible that participants may have preferred other more personalized forms of contact to connect with a study [29, 30] and maintain high participant interest. Another possibility is that some participants may not have easy access to online materials and thereby lack opportunities to fully engage through digital communication [31]. Alternatively, since many ADRCs now incorporate a wide variety of potentially burdensome ancillary procedures such as imaging, lumbar puncture, and fibroblast donation for induced pluripotent stem cell generation, we cannot rule out that study identity may become synonymous with research burden for some participants (and lead to dropout). Elucidating why such associations occurred will indeed require further study.

In summary, our study found that the most effective retention strategies were those that focus on improving study personnel through trainings and communicating study description irrespective of race and ethnicity or diagnostic groups. Identifying evidence-based retention strategies can inform national guidelines/policy to improve the conduct of AD studies and help to allocate limited resources more efficiently.