A Virtual Cohort Study of Individuals at Genetic Risk for Parkinson’s Disease: Study Protocol and Design
Abstract
Background:
The rise of direct-to-consumer genetic testing has enabled many to learn of their possible increased risk for rare diseases, some of which may be suitable for gene-targeted therapies. However, recruiting a large and representative population for rare diseases or genetically defined sub-populations of common diseases is slow, difficult, and expensive.
Objective:
To assess the feasibility of recruiting and retaining a cohort of individuals who carry a genetic mutation linked to Parkinson’s disease (G2019S variant of LRRK2); to characterize this cohort relative to the characteristics of traditional, in-person studies; and to evaluate this model’s ability to create an engaged study cohort interested in future clinical trials of gene-directed therapies.
Methods:
This single-site,3-year national longitudinal observational study will recruit between 250 to 350 LRRK2 carriers without Parkinson’s disease and approximately 50 with the condition. Participants must have undergone genetic testing by the personal genetics company, 23andMe, Inc., have knowledge of their carrier status, and consent to be contacted for research studies. All participants undergo standardized assessments, including video-based cognitive and motor examination, and complete patient-reported outcomes on an annual basis.
Results:
263 individuals living in 33 states have enrolled. The cohort has a mean (SD) age of 56.0 (15.9) years, 59% are female, and 76% are of Ashkenazi Jewish descent. 233 have completed the baseline visit: 47 with self-reported Parkinson’s disease and 186 without.
Conclusions:
This study establishes a promising model for developing a geographically dispersed and well-characterized cohort ready for participation in future clinical trials of gene-directed therapies.
INTRODUCTION
Since its completion in 2003, the Human Genome Project has led to two modern phenomena—direct-to-consumer genetic testing and a rapid expansion in development of gene-directed therapies [1–3]. More than 26 million individuals have purchased direct-to-consumer genetic tests, and the number is increasing exponentially [4]. As a result, many individuals are now aware of whether they carry known genetic markers that place them at risk, for rare and common diseases [5]. Genetic information can inform the identification of appropriate drug targets for disease therapies. Drugs that target pathways supported by genetic information are twice as likely to progress from early stages of development to approval [6].
One challenge to investigating emerging gene-directed therapies is identifying and characterizing the genetically defined populations. By their nature, these genetically defined sub-populations are smaller than the aggregate, undifferentiated disease population. As a result, identifying and recruiting a sufficiently sized population, particularly for large phase 3 trials, is challenging. The advent of direct-to-consumer genetic testing enables the identification of a potentially large number of individuals with the genetic marker of interest. Potential participants identified in this manner are likely to be widely geographically dispersed, requiring a move beyond traditional, brick-and-mortar site-based research studies. Virtual clinical studies can be conducted from a single site, can recruit from pre-identified geographically dispersed genetic sub-populations, and reduce geography- and disability-based barriers to participation [7–9]. While this approach has garnered much attention for investigating drugs [10–12], it may be even more readily adapted to lower risk observational studies [13].
Parkinson’s disease is the fastest growing neurological disorder in the world [14]. In 1997, Polymeropoulos and colleagues identified the first known genetic cause for the disease [15]. Since then, numerous genetic variants have been identified, and gene-directed therapies are emerging rapidly [16]. In 2004, the first genetic mutations in the leucine-rich repeat kinase 2 (LRRK2) gene were identified as causes of Parkinson’s disease, and LRRK2 mutations are the most common autosomal dominant genetic causes of the disease [17]. One to four percent of individuals with Parkinson’s disease carry the most common LRRK2 mutation, G2019S [18]. The penetrance of LRRK2 mutations is incomplete and increases with age; for LRRK2 G2019S carriers the risk is 28% at 59 years and increases to 74% at 79 years [18]. 23andMe, Inc., a personal genetics company, has identified over 3,000 LRRK2 G2019S carriers, located in almost every state in the U.S. This cohort could augment current studies of the disease and expand our understanding of the natural history of LRRK2 Parkinson’s disease.
Our virtual cohort study will enable broad clinical phenotyping of a group at high risk for Parkinson’s disease and can help accelerate the development of new treatments for the disease.
METHODS
Study design
This is a 36-month remote observational cohort study of individuals with the G2019S variant of LRRK2 (both with and without Parkinson’s disease according to self-report). Participants are asked to complete four annual virtual research visits from their home or preferred location using a HIPAA-compliant web-based video platform. Each virtual visit includes collection of a series of participant-reported outcomes in addition to study team-conducted assessments.
Study objectives
The study has three primary aims. The first is to assess the ability to recruit and retain a national cohort of LRRK2 G2019S carriers with and without Parkinson’s disease from the genetic database of 23andMe into a virtual cohort study. The second is to characterize prospectively this large cohort and to compare its demographic and clinical characteristics to those of traditionally assessed cohorts. The third is to assess the value of this study model for creating a cohort ready for future clinical trials.
Setting
The study is a collaboration between the University of Rochester and 23andMe and is funded by the National Institute of Neurological Disorders and Stroke. The virtual visits are conducted by investigators and coordinators from the University of Rochester. The study was reviewed and approved by the University of Rochester’s Research Subjects Review Board (STUDY00003703).
Participants
Individuals who have pursued genetic testing through 23andMe, elected to know they are carriers of the LRRK2 G2019S mutation, and have previously provided 23andMe with their consent to be contacted regarding research opportunities are eligible to participate. Within the Parkinson’s disease report, 23andMe only reports on the presence of the LRRK2 G2019S mutation and does not report on the presence of other LRRK2 mutations. Additionally, participants must be at least 18 years old, have viewed their Parkinson’s disease report through 23andMe more than 30 days prior to contact (to allow time to process the information), reside in the U.S., be fluent in English, have access to an internet-enabled device that will support video conferencing, and be willing and able to provide informed consent. Up to 400 individuals will participate in the study. Approximately 50 will have Parkinson’s disease (manifest), and 250 to 350 will not (non-manifest) according to self-report.
Procedures
To ensure the inclusion of older adults and increase the odds of phenoconversion to manifest disease during the study period, enrollment is stratified by age (<60 and ≥60). We are using disproportional stratified sampling with the goal of enrolling approximately equal numbers of individuals who are under the age of 60 and over the age of 60 among LRRK2 manifest and non-manifest carriers.
23andMe emails eligible individuals an invitation to participate in the research study, and directs them to a 23andMe-hosted website to learn more about the study. Eligible individuals are emailed up to three times. Interested individuals can click on a secure personalized link that will direct them to a form in the Research Electronic Data Capture system (REDCap; Nashville, TN) where they can provide their email, phone number, and consent to be contacted by the study team at the University of Rochester. A study coordinator then emails or calls interested participants to provide a study overview and assess their willingness to participate. Prospective participants are emailed a link to an IRB-approved electronic consent document in REDCap and scheduled for a test video visit. If needed, a web camera is mailed to the prospective participant at no cost to the participant. An informational handout explaining the relationship between the LRRK2 G2019S variant and Parkinson’s disease is included with the electronic consent document. This handout also connects participants with free genetic counseling offered through Indiana University should they desire more information or guidance. After providing consent, the study participant completes surveys in REDCap. Included is a modified (to reduce participant burden) environmental risk factors questionnaire that covers use of tobacco, caffeinated coffee, green tea, and black tea, and exposures to pesticides or solvents—all of which have been linked to Parkinson’s disease [19, 20].
During the test video visit, using Zoom video conferencing software (San Jose, CA), the coordinator reviews the study with the participant, addresses any questions, and ensures that the participant understands the study details. The coordinator also assists the participant with any technical difficulties using Zoom on their computer, tablet, or smartphone device and establishes a suitable set-up for future video visits. In addition, the coordinator collects medications, a health history, and a family history of Parkinson’s disease. Participants who have provided informed consent and completed the test video visit are considered enrolled. At the baseline visit and annual visits thereafter, participants complete additional surveys and undergo standardized cognitive and physical assessments. As part of our retention efforts, we have started distributing newsletters to participants and will be hosting regular webinars for participants.
Outcome measures
As outlined in Table 1, each annual visit involves the completion of several surveys that assess Parkinson’s disease symptoms, including non-motor features (e.g., depression, anxiety). A trained investigator administers a modification of the standard motor examination of the Movement Disorders Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS) [21]. The modification excludes assessments of rigidity and postural stability, which cannot be performed remotely. Previous studies have demonstrated the feasibility and reliability of similar remote assessments for Parkinson’s disease [22–24]. Investigators are asked to rely on history where necessary to determine the presence of postural instability and the Hoehn & Yahr stage.
Table 1
Study outcomes
| Study Team-Completed Assessments | |
| Assessment | Description |
| Movement Disorder Society – Unified Parkinson Disease Rating Scale (MDS-UPDRS) Parts Ia, III, IV [21–24] | Most commonly used Parkinson’s disease rating scale assessing non-motor aspects of daily living, motor features, and motor complications. Balance and rigidity tests are excluded with remote administration. |
| Montreal Cognitive Assessment [25, 58] | Assesses visuo-spatial and executive function, language, short-term memory, attention, abstraction, and orientation. |
| Stroop Color and Word Test [26, 59] | A measure of cognitive flexibility adapted for digital administration on computers, tablets, and smartphones where participants respond to stimuli using their keyboard or screen. |
| Schwab &England Activities of Daily Living Scale [60] | An assessment of the participant’s level of independence in activities of daily living. |
| Clinician global impression of severity and change [61, 62] | A scale for the global assessment of the participant compared to the investigator’s past experience with others who have the same diagnosis. |
| Expert-Assessed Parkinson’s Disease Status | Participant’s most likely diagnosis determined primarily through expert clinical determination and secondarily using modified Movement Disorders Society clinical diagnostic criteria [27], UK PD Brain Bank Criteria [28], and NIH PD criteria [29]. |
| Participant-Completed Assessments | |
| Assessment | Description |
| Parkinson’s Disease Symptoms and Diagnosis History | A survey capturing the presence of any Parkinson’s disease symptoms, response to any Parkinson’s disease medications, and details regarding any history of diagnosis of Parkinson’s disease. |
| REM Sleep Behavior Disorder Screening Questionnaire [63] | A questionnaire for detecting the presence of REM Sleep Behavior Disorder. |
| MDS-UPDRS Parts Ib and II [22] | An assessment of motor and non-motor experiences of daily living in Parkinson’s disease. |
| Patient global impression of severity and change | A scale assessing the overall severity of symptoms and their change. |
| Scales for Outcomes in Parkinson’s Disease – Autonomic [64] | A scale for the assessment of autonomic symptoms in Parkinson’s disease. |
| Epworth Sleepiness Scale [65] | An assessment of excessive daytime sleepiness in eight different situations encountered commonly in daily life. |
| Parkinson’s Disease Questionnaire – 39 [66] | A measure of how Parkinson’s disease has affected the participant’s health and overall quality of life. |
| Beck Depression Inventory – II [36] | A scale for the assessment of depressive symptoms. |
| Parkinson Anxiety Scale [67] | A scale for the assessment of anxiety in Parkinson’s disease. |
| University of Pennsylvania Smell Identification Test [68] | An assessment for olfactory loss with normative data. |
The study team also administers two cognitive assessments at each annual visit. The first, the Montreal Cognitive Assessment, has previously been studied for remote administration [25]. Participants are mailed or emailed the portions of the Montreal Cognitive Assessment that require a motor response in advance of the visit, are instructed by a coordinator to complete the tasks during the video visit, and are scored by displaying their completed items to their web camera. Additionally, participants complete the Stroop Color and Word Test, adapted for digital administration. Participants are emailed personalized links to a Java-based version of the test [26] where they are presented with stimuli (two minutes for each of the three rounds) and respond by pressing the correct response using their computer’s arrow keys or their device’s screen. At present, no validated external normative data exists for this modified digital administration of the Stroop.
Participants (including those who already have been diagnosed with Parkinson’s disease) are further assessed for features typical or atypical (e.g., exposure to anti-psychotic medication) of Parkinson’s disease—including the Movement Disorder Society Clinical Diagnostic Criteria for Parkinson’s disease [27], the UK Parkinson’s Disease Society Brain Bank Criteria [28], and the NIH Diagnostic Criteria for Parkinson’s disease [29]—to determine if the disease is present clinically. Items that cannot be adequately assessed remotely are excluded: rigidity and supranuclear gaze palsy, which are common to all three sets of criteria; assessment of cerebellar oculomotor abnormalities, graphesthesia, stereognosis, pyramidal weakness, and pathologic hyperreflexia for the Movement Disorder Society Clinical Diagnostic Criteria; and assessment of the Babinski sign for the UK Parkinson’s Disease Society Brain Bank Criteria. Investigators are blinded to self-report of Parkinson’s disease diagnosis. Additionally, at the baseline and Year 3 visits, participants are mailed and asked to complete and return the University of Pennsylvania Smell Identification Test. The 40-item scratch-and-sniff test assesses olfaction, which may be diminished early in the course of Parkinson’s disease [30, 31].
After each visit, participants complete a satisfaction survey developed based on earlier telehealth studies [32, 33]. The survey assesses overall satisfaction with the telehealth study visit as well as specific satisfaction with the technical quality, convenience, and comfort of the visit. The survey also determines interest in future research participation—separately assessing for interest in observational studies, symptomatic treatment trials, disease prevention trials, and disease-modifying trials—both with and without virtual visits. Recognizing the quickly evolving LRRK2 research landscape, our intention is to adapt the questions in response to the development of specific interventions of interest.
Assessment of prodromal disease
We have incorporated into our study and are able to assess all elements of the 2015 MDS Prodromal Parkinson’s Disease Criteria to identify those without self-reported Parkinson’s disease that have probable prodromal Parkinson’s disease [34, 35]. These criteria assign a likelihood ratio to a variety of participant- and investigator-reported measures (e.g., exposure to environmental risk factors, non-motor symptoms of Parkinson’s, motor function, prior diagnostic testing). The product of these likelihood ratios provides a total likelihood ratio which can be combined with their age-based prior probability of prodromal Parkinson’s to determine whether they meet criteria for probable prodromal Parkinson’s disease. The ability to identify individuals with prodromal disease is critical to establishing a clinical-trial-ready cohort for evaluating the impact of potential disease-modifying therapies.
Safety oversight
While this observational study is associated with minimal risk, we have developed a safety monitoring plan that takes into account the remote nature of the study and will assess the safety of participants throughout. The participant’s location is collected at the start of each visit to ensure that emergency services can be correctly directed should the need arise. For urgent medical concerns (e.g., chest pain) that arise during a virtual visit, either the participant, their care partner (if present), or a study team member will contact the local emergency medical response system, and the study team member will remain in video contact with the participant until emergency personnel arrive. If a participant has a high score (above 14) or indicates suicidal thoughts on the Beck Depression Inventory- II [36], an automatic notification is triggered prompting evaluation by an investigator.
If an investigator determines that a non-manifest participant has signs or symptoms consistent with parkinsonism or another neurological disease, the investigator uses his/her judgment in determining whether to suggest that the participant follow up with a local healthcare provider. At the conclusion of each remote visit, the investigator documents any negative events, such as a fall during the study visit or compromise of confidentiality. These events are reported to the study’s principal investigator or co-principal investigator.
Data sharing
In the study’s electronic informed consent form, participants are asked whether they would like their physician(s) to receive a summary of their research evaluation. If desired, the summary, which includes select test results but not an interpretation or diagnosis, is provided after the baseline and annual visits thereafter. Results of the modified MDS-UPDRS motor examination, REM Sleep Behavior Disorder Screening Questionnaire, Epworth Sleepiness Scale, Beck Depression Inventory-II, and Montreal Cognitive Assessment are included in this summary along with the study team’s contact information for any follow-up questions. Such information is also shared with the participant if requested. As indicated in the consent form, data from the study (except for personal contact information) are shared with 23andMe, employees of which are members of the study team and have previously collaborated with the study team at the University of Rochester [37].
Statistical analysis
We will determine the number and percentage of contactable non-manifest and manifest LRRK2 G2019S carriers that enroll in the study. Enrollment of at least 250 (approximately 10%) of 2,514 currently contactable non-manifest and 50 (approximately 18%) of 272 currently contactable manifest LRRK2 G2019S carriers in the 23andMe database will be considered successful recruitment for this virtual cohort study. With successful recruitment, our cohort will be comparably-sized to other traditionally-established LRRK2 carrier cohorts [38–41].
We will also determine the percentages of manifest and non-manifest participants who complete the 36-month study and will consider at least 80% who complete follow-up as successful retention during the study. We will define completers as those who do not withdraw from the study, are not lost to follow up, and complete at least three of the four virtual research visits, including the Year 3 visit. In order to characterize feasibility, we will also provide descriptive statistics for the mean number of virtual research visits completed and the proportion of scheduled research assessments completed at each visit.
We will use descriptive statistics to characterize the baseline characteristics of participants. We will explore differences in baseline characteristics between the non-manifest, prodromal, and manifest groups in our cohort using analysis of variance or chi-square tests as appropriate.
Due to differences in recruitment, we anticipate that our cohort will differ from traditional cohorts with respect to demographics and disease characteristics. We will determine the proportion of participants who have previously participated in LRRK2 or Parkinson’s research. We will use descriptive statistics to characterize baseline characteristics and compare these to corresponding figures from published reports of other established LRRK2 cohorts [38–41]. We will use t-tests or chi-square tests as appropriate to compare our non-manifest and manifest LRRK2 cohorts to these other established cohorts. Changes in mean scores over time between our cohort (manifest and non-manifest) and other established cohorts will be compared using t-tests at each available time point common to the studies being compared.
Additionally, we will determine the proportion of self-described non-manifest carriers who have clinically probable Parkinson’s disease and the proportion who have probable prodromal Parkinson’s disease at each visit. ‘Clinically probable’ will be determined primarily through expert clinical determination and secondarily using modified Movement Disorder Society’s clinical diagnostic criteria, UK Parkinson’s Disease Brain Bank criteria, and NIH Parkinson’s Disease criteria. Probable prodromal Parkinson’s disease will be defined as a probability of disease exceeding the 80% threshold, calculated in accordance with the 2015 Movement Disorder Society’s Prodromal Parkinson’s Disease Criteria [34]. While we do not capture each item in the updated 2019 Movement Disorder Society’s Prodromal Parkinson’s Disease Criteria, we will also calculate the probability of prodromal Parkinson’s disease in accordance with these criteria [42]. We will evaluate concordance between self-reported Parkinson’s disease status and remote expert assessment at each visit using Cohen’s kappa coefficient. We will determine the proportion of non-manifest carriers who develop manifest Parkinson’s disease, as determined separately by self-report and remote expert assessment and the proportion who develop probable prodromal Parkinson’s disease, as determined by different thresholds on the Prodromal Parkinson’s Disease Criteria, during the course of the study.
Finally, we will assess the value of this research model. We will consider at least 80% of participants reporting being satisfied or very satisfied with the virtual study and at least 80% of participants being willing to participate in other virtual research studies as indicative of a well-received acceptance of virtual study designs. We will also consider at least 50% of non-manifest participants being willing to participate in a Parkinson’s disease prevention trial as indicative of a successful model for this non-manifest group.
Sample size considerations
We anticipate enrolling at least 250 non-manifest LRRK2 G2019S carriers and 50 manifest carriers. A sample size of 250 non-manifest participants will enable us to estimate the percentage of completers with a 95% confidence interval width of 9.9%, and a sample size of 50 manifest participants will enable us to estimate the percentage of completers with a 95% confidence interval width of 23.7%, assuming that the true (unknown) percentages are close to 80%. A sample size of 250 non-manifest participants will enable us to estimate the percentage who have clinically probable Parkinson’s disease at baseline with a 95% confidence interval width of 7.4%, assuming that the true (unknown) percentage is close to 10%. Data from The Michael J. Fox Foundation’s LRRK2 cohort indicated a standard deviation of 11.6 years for mean age of Parkinson’s disease onset [40]. We anticipate a similar standard deviation in our study cohort. Atotal sample size of 566 participants (50 manifest LRRK2 carriers enrolled in this study and 516 manifest LRRK2 carriers in the MJFF LRRK2 Consortium) will provide 83% power to detect a 5-year difference in mean age of Parkinson’s disease onset between the two groups using a t-test with a significance level of 5% (two-tailed). Analogous t-tests comparing 250 non-manifest LRRK2 carriers enrolled in this study to 208 non-manifest LRRK2 carriers from the Parkinson’s Progression Markers Initiative will provide 80% power to detect a 1-point difference in mean baseline MDS-UPDRS Part III scores (assuming a pooled standard deviation of 3.8 points) and 82% power to detect a 1.6-point difference in mean baseline SCOPA-AUT total scores (assuming a pooled standard deviation of 5.9 points) between the two groups [43].
RESULTS
Between June 28, 2019 and May 5, 2020, 3,761 individuals were invited by 23andMe to participate in this study. A total of 9,909 email invitations were delivered and 38.7% of these emails were opened. A total of 351 (9.3%) individuals provided consent to be contacted by a University of Rochester study team member. Of these, 302 (86%) were successfully contacted and completed prescreening. A total of 22 individuals declined to participate; 10 found the study too burdensome, 6 were too busy, 4 had privacy concerns, 2 were uncomfortable with the reminder of their LRRK2 G2019S carrier status, and 1 sought an interventional trial. In the first ten months of recruitment, 263 individuals from 33 states and Washington, D.C. (Fig. 1) have enrolled in this single-site study, yielding an enrollment rate of approximately 26 per month. The cohort with completed baseline visits (n = 233) includes 47 individuals with self-reported Parkinson’s disease and 186 without, is 59% female, and has a mean (SD) age of 56.0 (15.9). Fifty-eight percent of non-manifest carriers have a family history of Parkinson’s disease and 77% are Ashkenazi Jewish.
Fig.1
Distribution of current study participants, by state.
Virtual visits are widely accepted by our participants. Of the 233 participants who have completed a baseline visit, 97% reported they were “Satisfied” or “Very Satisfied” with the study overall, and satisfaction ratings of the technical quality, convenience, and comfort of the video visits were comparably high—95%, 99%, and 99%, respectively. Beyond this study, our participants report strong interest in future Parkinson’s disease clinical trials. Eighty-four percent are willing to participate in a clinical trial evaluating a “treatment for the symptoms of Parkinson’s disease,” 86% in a clinical trial evaluating a “treatment to slow the progression of Parkinson’s disease,” and 94% in a clinical trial evaluating a “treatment to prevent the development of Parkinson’s disease.”
DISCUSSION
Virtual cohort studies offer the potential to connect the rapidly increasing number of individuals with known genetic mutations to researchers seeking to advance knowledge and evaluate new gene-directed therapies. This study in Parkinson’s disease, in collaboration with one of the largest direct-to-consumer genetic companies, will help evaluate that promise.
Enrollment has been rapid and nationwide. Normally, such an effort would require multiple sites, review by numerous institutional review boards, training of dozens of investigators, years of time, and considerable cost. A virtual cohort study bypasses such requirements and brings research studies to the participants directly, among other advantages (Table 2). The virtual nature of our study makes our study team nimble and affords the opportunity to quickly adapt our surveys and assessments in response to a changing research landscape. Moreover, by replacing site-based recruitment with remote nationwide recruitment, we may be able to recruit a more representative sample of the population.
Table 2
Comparison of virtual clinical studies to traditional site-based studies
| Characteristic | Traditional study | Virtual study |
| Focus | Participants, investigators, sites | Participants |
| Geographic reach | Sites | Internet access |
| Sites | Many | One |
| Institutional Review Boards | Many | One |
| Time to initiate study | Long | Medium |
| Investigators | Many | Few |
| Rating Variance | High | Low |
| Participant Comfort | Low | High |
| Participant Convenience | Low | High |
| Cost | High | Moderate |
Several traditional in-person cohort studies have sought to study this genetically defined sub-population of Parkinson’s disease. The traditional time-consuming process for identifying LRRK2 carriers requires first screening large numbers of individuals with Parkinson’s disease who present to the clinic for a LRRK2 mutation and second, screening relatives of the identified probands. While such studies have recruited hundreds of individuals, they may not represent the broader population of those who carry the genetic mutation (Table 3). In the Parkinson’s Progression Markers Initiative, 86% of non-manifest carriers had a family history of Parkinson’s disease [43]. Enrollment of LRRK2 carriers was enriched through focused recruitment of individuals of Ashkenazi Jewish descent [44], a population which has a high prevalence of the G2019S mutation [45]. Similarly, in the Ashkenazi Jewish LRRK2 Consortium cohort, 100% of non-manifest carriers were Ashkenazi Jewish and 100% had a family history of Parkinson’s disease [38]. Whereas in this study, thus far 77% of non-manifest carriers are Ashkenazi Jewish and 58% have a family history of Parkinson’s disease. Engagement and comprehensive clinical characterization of this geographically dispersed population coupled with targeted assessment of clinical trial interest may help generate a clinical trial-ready cohort. Exceeding our expectations, nearly 95% of our participants are interested in participating in a Parkinson’s disease prevention trial. Beyond generating interest, longitudinal characterization of our large cohort of non-manifest carriers will help to fill key knowledge gaps, such as the identification of protective and risk factors for phenoconversion to manifest Parkinson’s disease, that will ultimately inform the design of clinical trials.
Table 3
Characteristics of LRRK2 mutation carriers
| Mutation carriers without Parkinson’s disease | Mutation carriers with Parkinson’s disease | ||||||
| 23andMe Research Cohort | Ashkenazi Jewish LRRK2 Consortium [38] | Parkinson’s Progression Markers Initiative [43] | LRRK2 Cohort Consortium [41] | 23andMe Research Cohort | Ashkenazi Jewish LRRK2 Consortium [39] | LRRK2 Cohort Consortium [40] | |
| Number | 2514 | 134 | 208 | 342 | 272 | 97 | 516 |
| Mean age (SD) | 48.0 (16.6) | 49.5 (16.8) | 61.6 (7.6) | 51.9 (15.6) | 71.0 (8.9) | 68.6 (8.8) | 65.0 (11.5) |
| Women (%) | 51.9 | 56 | 58 | 58 | 47.1 | 51 | 48 |
| Ashkenazi Jewish (%) | 38.9 | 100 | NA | NA | 66.9 | 100 | NA |
| College | NA | NA; Median years | 83 | 71 | NA | 75 | 61 |
| education (%) | 16 | ||||||
| Number ofsites | 1 | 3 | 34 | 20* | 1 | 3 | 20 |
NA, Not available. *By design, the LRRK2 Cohort Consortium included the sites from the Ashkenazi Jewish LRRK2 Consortium and thus, information on more than 70% of the reported manifest carriers in the LRRK2 Cohort Consortium had been previously published.
This study also reinforces the importance of data privacy procedures with virtual research studies, ensuring that the overall benefits of current and future data use are maximized, while the risks—including data breaches—are minimized. In an increasingly interconnected and digital world, concerns regarding data privacy and regulation of data sharing are growing. While the Health Insurance Portability and Accountability Act (HIPAA) provides guidance on de-identifying health information, there is a risk of re-identification from other data sets, particularly as more personal information becomes available online [46, 47]. Ensuring adequate data privacy and transparency regarding data sharing may require additional expertise and deliberations outside the traditional IRB review, including patients and experts in data privacy and societal ethics [47, 48]. To this end, we have included patients and an expert in electronic consent methods on our steering committee and use secure HIPAA-compliant systems for the collection of participant information and the conduct of study visits.
In this study, all participants are aware of their LRRK2 carrier status prior to study enrollment. However, participants without a self-reported diagnosis of Parkinson’s disease may have never undergone an evaluation for Parkinson’s disease and be unaware of their clinical status. Other investigators have examined understanding of Parkinson’s disease and genetic testing, desire for genetic testing, and reasons for or against genetic testing among Ashkenazi Jewish individuals with Parkinson’s disease and their non-manifest relatives [49]. Similar work is needed to inform our understanding of participant desire for disclosure of their clinical status. Our approach is to explicitly ask participants if they would like their physician(s) to receive a summary of their research evaluation; study coordinators email a summary directly to participants only if requested. Participants are interested in receiving their research results; 43.5% have asked for their study results to be sent to their primary care provider, neurologist, or both and 31% have requested that their results be shared with them directly.
We do not provide diagnostic status, rather results that may inform the need for further neurological evaluation. If an investigator determines that a participant without self-reported Parkinson’s disease has signs or symptoms consistent with parkinsonism, they are instructed to inform the participant and depending on the individual situation, may recommend evaluation by their primary care provider or neurologist. We do not provide referrals or coordinate clinical evaluation but we will follow-up on the results of any clinical evaluation at subsequent visits. With this approach, we are attempting to balance the ethical principles of autonomy (a participant’s right to receive their individual study results) and non-maleficence (concern regarding potential negative psychological effects of disclosure) [50]. Studies examining the psychological effects of clinical status disclosure to individuals at genetic risk for Parkinson’s disease are needed.
There are significant limitations to the study. Only individuals who have undergone genetic testing at 23andMe, know their status, and are willing to participate in a virtual study are able to participate. These individuals differ from the general population. In one survey of nearly 1000 consumers of direct-to-consumer genetic testing, participants were largely white (86%), highly educated (over half have at least a college degree), wealthy (45% had a household income above $100,000 annually), and healthy (over 60% rated their health as “very good” or “excellent”) [51]. That said, adoption of direct-to-consumer genetic testing is increasing rapidly, is accessible to many, and is now available at common convenience, pharmacy, and grocery stores where almost all U.S. consumers shop. One future approach may be to take a combined approach recruiting from direct-to-consumer genetic testing companies, industry-sponsored programs that provide free genetic testing for rare diseases [52], and clinics. The need for video conferencing could also reduce access to research for those on the other side of the “digital divide.”
Although over 3000 LRRK2 G2019S carriers who received testing through 23andMe have been invited to this virtual study, fewer than 10% have expressed interest in participating. The low response rate raises concern for potential bias as responders may differ from non-responders in key ways. Importantly, after we have completed recruitment we plan to examine variables (such as age, ethnicity, education level, socioeconomic status, and location) that may be predictors of enrollment. We have speculated as to possible explanations for this low response rate. One possibility is unease or reservations surrounding the required use of technology, for which we offer extensive technical support, and with the implications of being a LRRK2 G2019S carrier. By approaching only individuals more than 90 days after they have reviewed their 23andMe report, and by providing additional information about LRRK2 and Parkinson’s disease, we aim to reduce this unease. Also, the target cohort is primarily non-manifest participants. As such, many individuals may have no personal connection to Parkinson’s disease motivating them to participate. Our recruitment materials specifically highlight the importance of non-manifest participants in research in an attempt to increase interest. Lastly, the burden of a three-year study may deter some participants, especially for those that are employed full-time or have more advanced Parkinson’s disease. We combat this by allowing participants to complete all required surveys and questionnaires at their convenience and minimizing time spent in research visits.
There are aspects of a typical in-person research study that require modification for remote administration. Certain hands-on elements of the physical exam (e.g., muscle tone and postural stability) are challenging to assess remotely, and certain procedures (e.g., lumbar puncture) or imaging tests require evaluation in medical or research centers. However, as more care is moving toward the home, so too can clinical research (e.g. in-home research nursing, in-home phlebotomy) [53]. Doing so makes participation in research more convenient for participants and may enhance recruitment and retention. We are not collecting biological samples or assessing digital markers; however, we are committed to exploring optional additional assessments for a sub-cohort. Finally, the personal connection between study participants and researchers may differ between in-person and virtual visits. Previous studies have found high rates of satisfaction with virtual research visits [37, 54], as has been seen with participants in this study and with virtual medical care [55–57]. Moreover, virtual visits shift the dynamic between participants and researchers by bringing research into the home and enable a more holistic view of the participant.
Virtual cohort studies are poised to take advantage of the rise in direct-to-consumer genetic testing and the availability of gene-directed therapies. This large, national study will help determine the value of such an approach in defining the natural history of genetically-defined diseases and creating a cohort ready for future clinical trials.
CONFLICT OF INTEREST
Helen Rowbotham, Marie Luff, Paul Cannon, and members of the 23andMe Research Team are employed by and hold stock or stock options in 23andMe, Inc.
ACKNOWLEDGMENTS
We would like to thank the research participants and employees of 23andMe for making this work possible. The following members of the 23andMe Research Team contributed to this study: Michelle Agee, Stella Aslibekyan, Adam Auton, Robert K. Bell, Katarzyna Bryc, Sarah K. Clark, Sarah L. Elson, Kipper Fletez-Brant, Pierre Fontanillas, Nicholas A. Furlotte, Pooja M. Gandhi, Karl Heilbron, Barry Hicks, David A. Hinds, Karen E. Huber, Ethan M. Jewett, Yunxuan Jiang, Aaron Kleinman, Keng-Han Lin, Nadia K. Litterman, Jennifer C. McCreight, Matthew H. McIntyre, Kimberly F. McManus, Joanna L. Mountain, Sahar V. Mozaffari, Priyanka Nandakumar, Elizabeth S. Noblin, Carrie A.M. Northover, Jared O’Connell, Steven J. Pitts, G. David Poznik, J. Fah Sathirapongsasuti, Anjali J. Shastri, Janie F. Shelton, Suyash Shringarpure, Chao Tian, Joyce Y. Tung, Robert J. Tunney, Vladimir Vacic, Xin Wang, Amir S. Zare.
Research reported in this publication was supported by the National Institute of Neurological Disorders and Stroke of the National Institutes of Health under Award Number P50NS108676. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funder had no role in the design of the study and collection, analysis, interpretation of data, or in writing of the manuscript.
REFERENCES
[1] | High KA , Roncarolo MG ((2019) ) Gene therapy. N Engl J Med 381: , 455–464. |
[2] | Covolo L , Rubinelli S , Ceretti E , Gelatti U ((2015) ) Internet-based direct-to-consumer genetic testing: A systematic review. J Med Internet Res 17: , e279. |
[3] | Collins FS , Morgan M , Patrinos A ((2003) ) The Human Genome Project: Lessons from large-scale biology. Science 300: , 286–290. |
[4] | RegaladoA, More than 26 million people have taken an at-home ancestry test, Elizabeth Bramson-Boudreau, https://www.technologyreview.com/s/612880/more-than-26-million-people-have-taken-an-at-home-ancestry-test/, February 11, 2019. |
[5] | Klein C , Westenberger A ((2012) ) Genetics of Parkinson’s disease. Cold Spring Harb Perspect Med 2: , a008888. |
[6] | Nelson MR , Tipney H , Painter JL , Shen J , Nicoletti P , Shen Y , Floratos A , Sham PC , Li MJ , Wang J , Cardon LR , Whittaker JC , Sanseau P ((2015) ) The support of human genetic evidence for approved drug indications. Nat Genet 47: , 856. |
[7] | Hirsch IB , Martinez J , Dorsey ER , Finken G , Fleming A , Gropp C , Home P , Kaufer DI , Papapetropoulos S ((2017) ) Incorporating site-less clinical trials into drug development: A framework for action. Clin Ther 39: , 1064–1076. |
[8] | Dorsey ER , Venuto C , Venkataraman V , Harris DA , Kieburtz K ((2015) ) Novel methods and technologies for 21st-century clinical trials: A review. JAMA Neurol 72: , 582–588. |
[9] | ((2019) ) Virtual Clinical Trials: Challenges and Opportunities: Proceedings of a Workshop, ShoreC, KhandekarE, AlperJ, eds. The National Academies Collection: Reports funded by National Institutes of Health, Washington, DC. |
[10] | Wicks P , Vaughan TE , Massagli MP , Heywood J ((2011) ) Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nat Biotechnol 29: , 411–414. |
[11] | Orri M , Lipset CH , Jacobs BP , Costello AJ , Cummings SR ((2014) ) Web-based trial to evaluate the efficacy and safety of tolterodine ER 4 mg in participants with overactive bladder: REMOTE trial. Contemp Clin Trials 38: , 190–197. |
[12] | Bedlack RS , Wicks P , Vaughan T , Opie A , Blum R , Dios A , Sadri-Vakili G ((2019) ) Lunasin does not slow ALS progression: Results of an open-label, single-center, hybrid-virtual 12-month trial. Amyotroph Lateral Scler Frontotemporal Degener 20: , 285–293. |
[13] | Tanner CM , Meng CC , Ravina B , Lang A , Kurlan R , Marek K , Oakes D , Seibyl J , Flagg E , Gauger L , Guest DD , Goetz CG , Kieburtz K , DiEuliis D , Fahn S , Elliott RA , Shoulson I ((2014) ) A practical approach to remote longitudinal follow-up of Parkinson’s disease: The FOUND study. Mov Disord 29: , 743–749. |
[14] | Collaborators GBDN ((2019) ) Global, regional, and national burden of neurological disorders, 1990-2016: A systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol 18: , 459–480. |
[15] | Polymeropoulos MH , Lavedan C , Leroy E , Ide SE , Dehejia A , Dutra A , Pike B , Root H , Rubenstein J , Boyer R , Stenroos ES , Chandrasekharappa S , Athanassiadou A , Papapetropoulos T , Johnson WG , Lazzarini AM , Duvoisin RC , Di Iorio G , Golbe LI , Nussbaum RL ((1997) ) Mutation in the alpha-synuclein gene identified in families with Parkinson’s disease. Science 276: , 2045–2047. |
[16] | Singleton AB , Farrer MJ , Bonifati V ((2013) ) The genetics of Parkinson’s disease: Progress and therapeutic implications. Mov Disord 28: , 14–23. |
[17] | Zimprich A , Biskup S , Leitner P , Lichtner P , Farrer M , Lincoln S , Kachergus J , Hulihan M , Uitti RJ , Calne DB , Stoessl AJ , Pfeiffer RF , Patenge N , Carbajal IC , Vieregge P , Asmus F , Muller-Myhsok B , Dickson DW , Meitinger T , Strom TM , Wszolek ZK , Gasser T ((2004) ) Mutations in LRRK2 cause autosomal-dominant parkinsonism with pleomorphic pathology. Neuron 44: , 601–607. |
[18] | Healy DG , Falchi M , O’Sullivan SS , Bonifati V , Durr A , Bressman S , Brice A , Aasly J , Zabetian CP , Goldwurm S , Ferreira JJ , Tolosa E , Kay DM , Klein C , Williams DR , Marras C , Lang AE , Wszolek ZK , Berciano J , Schapira AH , Lynch T , Bhatia KP , Gasser T , Lees AJ , Wood NW , International LRRK2 Consortium ((2008) ) Phenotype, genotype, and worldwide genetic penetrance of LRRK2-associated Parkinson’s disease: A case-control study. Lancet Neurol 7: , 583–590. |
[19] | Tanner CM , Kamel F , Ross GW , Hoppin JA , Goldman SM , Korell M , Marras C , Bhudhikanok GS , Kasten M , Chade AR , Comyns K , Richards MB , Meng C , Priestley B , Fernandez HH , Cambi F , Umbach DM , Blair A , Sandler DP , Langston JW ((2011) ) Rotenone, paraquat, and Parkinson’s disease. Environ Health Perspect 119: , 866–872. |
[20] | Ascherio A , Chen H , Weisskopf MG , O’Reilly E , McCullough ML , Calle EE , Schwarzschild MA , Thun MJ ((2006) ) Pesticide exposure and risk for Parkinson’s disease. Ann Neurol 60: , 197–203. |
[21] | Goetz CG , Tilley BC , Shaftman SR , Stebbins GT , Fahn S , Martinez-Martin P , Poewe W , Sampaio C , Stern MB , Dodel R , Dubois B , Holloway R , Jankovic J , Kulisevsky J , Lang AE , Lees A , Leurgans S , LeWitt PA , Nyenhuis D , Olanow CW , Rascol O , Schrag A , Teresi JA , van Hilten JJ , LaPelle N , Movement Disorder Society UPDRS Revision Task Force ((2008) ) Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): Scale presentation and clinimetric testing results. Mov Disord 23: , 2129–2170. |
[22] | Cubo E , Gabriel-Galan JM , Martinez JS , Alcubilla CR , Yang C , Arconada OF , Perez NM ((2012) ) Comparison of office-based versus home Web-based clinical assessments for Parkinson’s disease. Mov Disord 27: , 308–311. |
[23] | Dorsey ER , Deuel LM , Voss TS , Finnigan K , George BP , Eason S , Miller D , Reminick JI , Appler A , Polanowicz J , Viti L , Smith S , Joseph A , Biglan KM ((2010) ) Increasing access to specialty care: A pilot, randomized controlled trial of telemedicine for Parkinson’s disease. Mov Disord 25: , 1652–1659. |
[24] | Stillerova T , Liddle J , Gustafsson L , Lamont R , Silburn P ((2016) ) Remotely assessing symptoms of Parkinson’s disease using videoconferencing: A feasibility study. Neurol Res Int 2016: , 4802570. |
[25] | Abdolahi A , Bull MT , Darwin KC , Venkataraman V , Grana MJ , Dorsey ER , Biglan KM ((2016) ) A feasibility study of conducting the Montreal Cognitive Assessment remotely in individuals with movement disorders. Health Informatics J 22: , 304–311. |
[26] | von Bastian CC , Locher A , Ruflin M ((2013) ) Tatool: A Java-based open-source programming framework for psychological studies. Behav Res Methods 45: , 108–115. |
[27] | Postuma RB , Berg D , Stern M , Poewe W , Olanow CW , Oertel W , Obeso J , Marek K , Litvan I , Lang AE , Halliday G , Goetz CG , Gasser T , Dubois B , Chan P , Bloem BR , Adler CH , Deuschl G ((2015) ) MDS clinical diagnostic criteria for Parkinson’s disease. Mov Disord 30: , 1591–1601. |
[28] | Hughes AJ , Daniel SE , Kilford L , Lees AJ ((1992) ) Accuracy of clinical diagnosis of idiopathic Parkinson’s disease: A clinico-pathological study of 100 cases. J Neurol Neurosurg Psychiatry 55: , 181–184. |
[29] | Gelb DJ , Oliver E , Gilman S ((1999) ) Diagnostic criteria for Parkinson disease. Arch Neurol 56: , 33–39. |
[30] | Haehner A , Hummel T , Hummel C , Sommer U , Junghanns S , Reichmann H ((2007) ) Olfactory loss may be a first sign of idiopathic Parkinson’s disease. Mov Disord 22: , 839–842. |
[31] | Haehner A , Boesveldt S , Berendse HW , Mackay-Sim A , Fleischmann J , Silburn PA , Johnston AN , Mellick GD , Herting B , Reichmann H , Hummel T ((2009) ) Prevalence of smell loss in Parkinson’s disease–a multicenter study. Parkinsonism Relat Disord 15: , 490–494. |
[32] | Beck CA , Beran DB , Biglan KM , Boyd CM , Dorsey ER , Schmidt PN , Simone R , Willis AW , Galifianakis NB , Katz M , Tanner CM , Dodenhoff K , Aldred J , Carter J , Fraser A , Jimenez-Shahed J , Hunter C , Spindler M , Reichwein S , Mari Z , Dunlop B , Morgan JC , McLane D , Hickey P , Gauger L , Richard IH , Mejia NI , Bwala G , Nance M , Shih LC , Singer C , Vargas-Parra S , Zadikoff C , Okon N , Feigin A , Ayan J , Vaughan C , Pahwa R , Dhall R , Hassan A , DeMello S , Riggare SS , Wicks P , Achey MA , Elson MJ , Goldenthal S , Keenan HT , Korn R , Schwarz H , Sharma S , Stevenson EA , Zhu W , Connect.Parkinson Investigators ((2017) ) National randomized controlled trial of virtual house calls for Parkinson disease. Neurology 89: , 1152–1161. |
[33] | Korn RE , Wagle Shukla A , Katz M , Keenan HT , Goldenthal S , Auinger P , Zhu W , Dodge M , Rizer K , Achey MA , Byrd E , Barbano R , Richard I , Andrzejewski KL , Schwarz HB , Dorsey ER , Biglan KM , Kang G , Kanchana S , Rodriguez R , Tanner CM , Galifianakis NB ((2017) ) Virtual visits for Parkinson disease: A multicenter noncontrolled cohort. Neurol Clin Pract 7: , 283–295. |
[34] | Berg D , Postuma RB , Adler CH , Bloem BR , Chan P , Dubois B , Gasser T , Goetz CG , Halliday G , Joseph L , Lang AE , Liepelt-Scarfone I , Litvan I , Marek K , Obeso J , Oertel W , Olanow CW , Poewe W , Stern M , Deuschl G ((2015) ) MDS research criteria for prodromal Parkinson’s disease. Mov Disord 30: , 1600–1611. |
[35] | Mirelman A , Saunders-Pullman R , Alcalay RN , Shustak S , Thaler A , Gurevich T , Raymond D , Mejia-Santana H , Orbe Reilly M , Ozelius L , Clark L , Gana-Weisz M , Bar-Shira A , Orr-Utreger A , Bressman SB , Marder K , Giladi N , AJ LRRK2 Consortium ((2018) ) Application of the Movement Disorder Society prodromal criteria in healthy G2019S-LRRK2 carriers. Mov Disord 33: , 966–973. |
[36] | Leentjens AF , Verhey FR , Luijckx GJ , Troost J ((2000) ) The validity of the Beck Depression Inventory as a screening and diagnostic instrument for depression in patients with Parkinson’s disease. Mov Disord 15: , 1221–1224. |
[37] | Dorsey ER , Darwin KC , Mohammed S , Donohue S , Tethal A , Achey MA , Ward S , Caughey E , Conley ED , Eriksson N , Ravina B ((2015) ) Virtual research visits and direct-to-consumer genetic testing in Parkinson’s disease. Digit Health 1: , 2055207615592998. |
[38] | Mirelman A , Alcalay RN , Saunders-Pullman R , Yasinovsky K , Thaler A , Gurevich T , Mejia-Santana H , Raymond D , Gana-Weisz M , Bar-Shira A , Ozelius L , Clark L , Orr-Urtreger A , Bressman S , Marder K , Giladi N , LRRK2 AJ consortium ((2015) ) Nonmotor symptoms in healthy Ashkenazi Jewish carriers of the G2019S mutation in the LRRK2 gene. Mov Disord 30: , 981–986. |
[39] | Alcalay RN , Mirelman A , Saunders-Pullman R , Tang MX , Mejia Santana H , Raymond D , Roos E , Orbe-Reilly M , Gurevich T , Bar Shira A , Gana Weisz M , Yasinovsky K , Zalis M , Thaler A , Deik A , Barrett MJ , Cabassa J , Groves M , Hunt AL , Lubarr N , San Luciano M , Miravite J , Palmese C , Sachdev R , Sarva H , Severt L , Shanker V , Swan MC , Soto-Valencia J , Johannes B , Ortega R , Fahn S , Cote L , Waters C , Mazzoni P , Ford B , Louis E , Levy O , Rosado L , Ruiz D , Dorovski T , Pauciulo M , Nichols W , Orr-Urtreger A , Ozelius L , Clark L , Giladi N , Bressman S , Marder KS ((2013) ) Parkinson disease phenotype in Ashkenazi Jews with and without LRRK2 G2019S mutations. Mov Disord 28: , 1966–1971. |
[40] | Marras C , Alcalay RN , Caspell-Garcia C , Coffey C , Chan P , Duda JE , Facheris MF , Fernandez-Santiago R , Ruiz-Martinez J , Mestre T , Saunders-Pullman R , Pont-Sunyer C , Tolosa E , Waro B , LRRK2 Cohort Consortium ((2016) ) Motor and nonmotor heterogeneity of LRRK2-related and idiopathic Parkinson’s disease. Mov Disord 31: , 1192–1202. |
[41] | Pont-Sunyer C , Tolosa E , Caspell-Garcia C , Coffey C , Alcalay RN , Chan P , Duda JE , Facheris M , Fernandez-Santiago R , Marek K , Lomena F , Marras C , Mondragon E , Saunders-Pullman R , Waro B , LRRK2 Cohort Consortium ((2017) ) The prodromal phase of leucine-rich repeat kinase 2-associated Parkinson disease: Clinical and imaging Studies. Mov Disord 32: , 726–738. |
[42] | Heinzel S , Berg D , Gasser T , Chen H , Yao C , Postuma RB , MDS Task Force on the Definition of Parkinson’s Disease ((2019) ) Update of the MDS research criteria for prodromal Parkinson’s disease. Mov Disord 34: , 1464–1470. |
[43] | Simuni T , Uribe L , Cho HR , Caspell-Garcia C , Coffey CS , Siderowf A , Trojanowski JQ , Shaw LM , Seibyl J , Singleton A , Toga AW , Galasko D , Foroud T , Tosun D , Poston K , Weintraub D , Mollenhauer B , Tanner CM , Kieburtz K , Chahine LM , Reimer A , Hutten SJ , Bressman S , Marek K , PPMI Investigators ((2020) ) Clinical and dopamine transporter imaging characteristics of non-manifest LRRK2 and GBA mutation carriers in the Parkinson’s Progression Markers Initiative (PPMI): A cross-sectional study. Lancet Neurol 19: , 71–80. |
[44] | Foroud T , Smith D , Jackson J , Verbrugge J , Halter C , Wetherill L , Sims K , Xin W , Arnedo V , Lasch S , Marek K , Parkinson’s Progression Markers Initiative ((2015) ) Novel recruitment strategy to enrich for LRRK2 mutation carriers. Mol Genet Genomic Med 3: , 404–412. |
[45] | Clark LN , Wang Y , Karlins E , Saito L , Mejia-Santana H , Harris J , Louis ED , Cote LJ , Andrews H , Fahn S , Waters C , Ford B , Frucht S , Ottman R , Marder K ((2006) ) Frequency of LRRK2 mutations in early- and late-onset Parkinson disease. Neurology 67: , 1786–1791. |
[46] | Sweeney L , Yoo JS , Perovich L , Boronow KE , Brown P , Brody JG ((2017) ) Re-identification risks in HIPAA Safe Harbor data: A study of data from one environmental health study. Technol Sci 2017: , 2017082801. |
[47] | Cohen IG , Mello MM ((2019) ) Big data, big tech, and protecting patient privacy. JAMA 322: , 1141–1142. |
[48] | Parasidis E , Pike E , McGraw D ((2019) ) A Belmont report for health data. N Engl J Med 380: , 1493–1495. |
[49] | Gupte M , Alcalay RN , Mejia-Santana H , Raymond D , Saunders-Pullman R , Roos E , Orbe-Reily M , Tang MX , Mirelman A , Ozelius L , Orr-Urtreger A , Clark L , Giladi N , Bressman S , Marder K ((2015) ) Interest in genetic testing in Ashkenazi Jewish Parkinson’s disease patients and their unaffected relatives. J Genet Couns 24: , 238–246. |
[50] | Pont-Sunyer C , Bressman S , Raymond D , Glickman A , Tolosa E , Saunders-Pullman R ((2015) ) Disclosure of research results in genetic studies of Parkinson’s disease caused by LRRK2 mutations. Mov Disord 30: , 904–908. |
[51] | Gollust SE , Gray SW , Carere DA , Koenig BA , Lehmann LS , Mc GA , Sharp RR , Spector-Bagdady K , Wang NA , Green RC , Roberts JS , PGen Study Group ((2017) ) Consumer perspectives on access to direct-to-consumer genetic testing: Role of demographic factors and the testing experience. Milbank Q 95: , 291–318. |
[52] | Invitae, Invitae Corporation, https://www.invitae.com/en/sponsored-testing/. |
[53] | Dorsey ER , Glidden AM , Holloway MR , Birbeck GL , Schwamm LH ((2018) ) Teleneurology and mobile technologies: The future of neurological care. Nat Rev Neurol 14: , 285. |
[54] | Dorsey ER , Wagner JD , Bull MT , Rizzieri A , Grischkan J , Achey MA , Sherer T , Chowdhury S , Meunier C , Cappelletti L , Rocker C , Richard IH , Schwarz H , Kang G , Ahmad SH , Biemiller RA , Biglan KM ((2015) ) Feasibility of virtual research visits in Fox Trial Finder. J Parkinsons Dis 5: , 505–515. |
[55] | Mair F , Whitten P ((2000) ) Systematic review of studies of patient satisfaction with telemedicine. BMJ 320: , 1517–1520. |
[56] | Agha Z , Schapira RM , Laud PW , McNutt G , Roter DL ((2009) ) Patient satisfaction with physician-patient communication during telemedicine. Telemed J E Health 15: , 830–839. |
[57] | Wilkinson JR , Spindler M , Wood SM , Marcus SC , Weintraub D , Morley JF , Stineman MG , Duda JE ((2016) ) High patient satisfaction with telehealth in Parkinson disease: A randomized controlled study. Neurol Clin Pract 6: , 241–251. |
[58] | Nasreddine ZS , Phillips NA , Bedirian V , Charbonneau S , Whitehead V , Collin I , Cummings JL , Chertkow H ((2005) ) The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53: , 695–699. |
[59] | Van der Elst W , Van Boxtel MP , Van Breukelen GJ , Jolles J ((2006) ) The Stroop color-word test: Influence of age, sex, and education; and normative data for a large sample across the adult age range. Assessment 13: , 62–79. |
[60] | McRae C , Diem G , Vo A , O’Brien C , Seeberger L ((2002) ) Reliability of measurements of patient health status: A comparison of physician, patient, and caregiver ratings. Parkinsonism Relat Disord 8: , 187–192. |
[61] | Busner J , Targum SD ((2007) ) The clinical global impressions scale: Applying a research tool in clinical practice. Psychiatry (Edgmont) 4: , 28–37. |
[62] | Guy W , National Institute of Mental Health (U.S.). Psychopharmacology Research Branch., Early Clinical Drug Evaluation Program. (1976) ECDEU assessment manual for psychopharmacology, U. S. Dept. of Health, Education, and Welfare, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, National Institute of Mental Health, Psychopharmacology Research Branch, Division of Extramural Research Programs, Rockville, MD. |
[63] | Stiasny-Kolster K , Mayer G , Schafer S , Moller JC , Heinzel-Gutenbrunner M , Oertel WH ((2007) ) The REM sleep behavior disorder screening questionnaire–a new diagnostic instrument. Mov Disord 22: , 2386–2393. |
[64] | Rodriguez-Blazquez C , Forjaz MJ , Frades-Payo B , de Pedro-Cuesta J , Martinez-Martin P , Longitudinal Parkinson’s Disease Patient Study, Estudio Longitudinal de Pacients con Enfermedad da Parkinson Group ((2010) ) Independent validation of the scales for outcomes in Parkinson’s disease-autonomic (SCOPA-AUT). Eur J Neurol 17: , 194–201. |
[65] | Spira AP , Beaudreau SA , Stone KL , Kezirian EJ , Lui LY , Redline S , Ancoli-Israel S , Ensrud K , Stewart A , Osteoporotic Fractures in Men Study ((2012) ) Reliability and validity of the Pittsburgh Sleep Quality Index and the Epworth Sleepiness Scale in older men. J Gerontol A Biol Sci Med Sci 67: , 433–439. |
[66] | Martinez-Martin P , Jeukens-Visser M , Lyons KE , Rodriguez-Blazquez C , Selai C , Siderowf A , Welsh M , Poewe W , Rascol O , Sampaio C , Stebbins GT , Goetz CG , Schrag A ((2011) ) Health-related quality-of-life scales in Parkinson’s disease: Critique and recommendations. Mov Disord 26: , 2371–2380. |
[67] | Leentjens AF , Dujardin K , Pontone GM , Starkstein SE , Weintraub D , Martinez-Martin P ((2014) ) The Parkinson Anxiety Scale (PAS): Development and validation of a new anxiety scale. Mov Disord 29: , 1035–1043. |
[68] | Doty RL , Bromley SM , Stern MB ((1995) ) Olfactory testing as an aid in the diagnosis of Parkinson’s disease: Development of optimal discrimination criteria. Neurodegeneration 4: , 93–97. |
