Neurocognitive Driving Rehabilitation in Virtual Environments (NeuroDRIVE): A pilot clinical trial for chronic traumatic brain injury
Abstract
BACKGROUND:
Virtual reality (VR) technology may provide an effective means to integrate cognitive and functional approaches to TBI rehabilitation. However, little is known about the effectiveness of VR rehabilitation for TBI-related cognitive deficits. In response to these clinical and research gaps, we developed Neurocognitive Driving Rehabilitation in Virtual Environments (NeuroDRIVE), an intervention designed to improve cognitive performance, driving safety, and neurobehavioral symptoms.
OBJECTIVE:
This pilot clinical trial was conducted to examine feasibility and preliminary efficacy of NeuroDRIVE for rehabilitation of chronic TBI.
METHODS:
Eleven participants who received the intervention were compared to six wait-listed participants on driving abilities, cognitive performance, and neurobehavioral symptoms.
RESULTS:
The NeuroDRIVE intervention was associated with significant improvements in working memory and visual search/selective attention— two cognitive skills that represented a primary focus of the intervention. By comparison, no significant changes were observed in untrained cognitive areas, neurobehavioral symptoms, or driving skills.
CONCLUSIONS:
Results suggest that immersive virtual environments can provide a valuable and engaging means to achieve some cognitive rehabilitation goals, particularly when these goals are closely matched to the VR training exercises. However, additional research is needed to augment our understanding of rehabilitation for driving skills, cognitive performance, and neurobehavioral symptoms in chronic TBI.
1Introduction
Each year, emergency departments treat approximately 2.5 million traumatic brain injuries (TBIs) (Langlois, Rutland-Brown, & Thomas, 2006; Marin, Weaver, Yealy, & Mannix, 2014). TBI can affect a wide range of brain systems, resulting in sensorimotor deficits (e.g., coordination, visual perception), cognitive deficits (e.g., memory, attention), emotional dysregulation (e.g., irritability, depression), and somatic symptoms (e.g., headache, fatigue) (Ponsford et al., 2014). These TBI-related impairments can have significant life consequences. Studies conducted across a wide range of neurological and psychiatric conditions show that neuropsychological abilities are strongly associated with functional skills and employment outcomes (Andelic, Stevens, Sigurdardottir, Arango-Lasprilla, & Roe, 2012; Dawson, Schwartz, Winocur, & Stuss, 2007; Drake, Gray, Yoder, Pramuka, & Llewellyn, 2000; Kalechstein, Newton, & van Gorp, 2003; Leahy & Lam, 1998; Wehman, Targett, West, & Kregel, 2005). For example, challenges in attention and concentration could result in distractibility and errors in work settings, and deficits in executive functions could lead to poor organization and problems with setting and achieving occupational goals. As many as 3.2–5.3 million people in the US are living with TBI-related disability (Selassie et al., 2008).
Rehabilitation has been shown to improve outcomes for those experiencing chronic effects of TBI (Cicerone, 2002; Cicerone et al., 2011; Cooper et al., 2017; Kennedy et al., 2008; Rohling, Faust, Beverly, & Demakis, 2009). Previously-validated rehabilitation approaches for TBI include both ‘cognitive’ and ‘functional’ approaches. ‘Cognitive’ methods of rehabilitation are focused on improving performance on individual cognitive tasks, with the hope that these gains will generalize to functional activities (Cicerone, 2002; Giles, 2010). Restorative cognitive training approaches have been shown to improve cognitive functioning across multiple conditions such as schizophrenia, traumatic brain injury, and normal aging (Ball, Edwards, & Ross, 2007; Fisher, Holland, Merzenich, & Vinogradov, 2009; Lebowitz, Dams-O’Connor, & Cantor, 2012; Lovell & Solomon, 2011; Smith et al., 2009). Some of the most promising results to date have been demonstrated for training of attention and working memory, which have been shown to correspond to changes in functional brain activity (Kim et al., 2009). Evidence suggests that the format of therapist-guided rehabilitation is able to harness some of the well-established benefits of the therapeutic relationship, and may be preferable to computer-guided training (Cooper et al., 2017). While there is some evidence indicating that benefits of cognitive remediation extend to untrained tasks, a number of studies have shown that improvements in performance on individual cognitive tasks tend to generalize very weakly, if at all, to other cognitive tasks and functional abilities (Devos et al., 2009; Owen et al., 2010). This weak transfer of training might be attributable to low levels of correspondence between the cognitive and sensorimotor demands of rehabilitation tasks and those encountered during challenging real-world situations.
In contrast to methods of rehabilitation that rely upon generalization of cognitive benefits to functional outcomes, ‘functional’ methods of rehabilitation focus on improving performance on real-life activities through direct practice of those activities (Fadyl & McPherson, 2009; Giles, 2010; Vanderploeg et al., 2008). This approach requires effective targeting of specific functional tasks that are relevant to each patient and may be limited by the physical environments available within the treatment setting (e.g., a simulated home environment used to practice activities of daily living). However, the basic functional tasks that are often emphasized in functional rehabilitation (e.g., self-care, food preparation) may not be sufficiently challenging to address more subtle or ‘higher order’ cognitive and functional deficits that many mild to moderate TBI patients experience in the long-term phase of recovery (Giles, 2010).
Virtual reality (VR) technology may provide an effective means to integrate cognitive and functional approaches to TBI rehabilitation (Imhoff, Lavallière, Teasdale, & Fait, 2016; Lew, Rosen, Thomander, & Poole, 2009). The guiding concept for VR rehabilitation is to provide an immersive, engaging, and realistic environment in which to practice cognitive, sensorimotor, and functional skills. VR scenarios can simulate a wide range of real or imagined tasks and environments. While VR may not be necessary for tasks that are easily recreated in existing therapy environments, it is particularly well-suited for tasks that are challenging or dangerous to recreate within real-world treatment environments, such as driving an automobile (Imhoff et al., 2016; Lew et al., 2009).
Driving is one of the most universal, cognitively challenging, and potentially-dangerous activities of everyday life. Safe driving requires continuous synchronization of processes like reaction time, visuo-spatial skills, attention, executive function, and planning (Lundqvist, 2001; Lundqvist & Rönnberg, 2001; Spiers & Maguire, 2007). Whereas it would be obviously unsafe to place an impaired patient into many real-world driving situations, VR allows for safe assessment and rehabilitation of driving-relevant skills at the true limits of the individual’s current capabilities. Individuals with TBI are at elevated risk for motor vehicle accidents and other driving difficulties (Bivona et al., 2012; Bottari, Lamothe, Gosselin, Gélinas, & Ptito, 2012; Classen et al., 2011; Cyr et al., 2009; Formisano et al., 2005; Lundqvist & Rönnberg, 2001). Many individuals with severe TBI never return to driving (Novack, Alderson, Bush, Meythaler, & Canupp, 2000; Ponsford et al., 2014), and an estimated 63% of those with severe TBI who do return to driving are involved in motor vehicle accidents (Bivona et al., 2012). Available evidence suggests that deficits in attention and visual search may underlie these driving impairments. While most of this research has been conducted with moderate-to-severe TBI populations, these issues are not exclusive to severe forms of TBI. Individuals recovering from mild TBI have also been found to exhibit slower detection of driving hazards in simulated driving experiments (Preece, Horswill, & Geffen, 2010, 2011) and to be at increased risk for real-world motor vehicle accidents (Schneider & Gouvier, 2005).
Previous results suggest that VR driving rehabilitation can be effective for improving driving skills among those with moderate-to-severe TBI (Cox et al., 2010). However, these findings have not been replicated or validated for those with symptomatic mild TBI. Additionally, little is known about the effectiveness of VR rehabilitation programs for TBI-related cognitive deficits (Imhoff et al., 2016). In response to these clinical and research gaps, we developed an intervention known as Neurocognitive Driving Rehabilitation in Virtual Environments (NeuroDRIVE), which was designed to improve cognitive performance and overall driving safety by providing integrated training in these skills. In contrast to intervention approaches that are geared toward more severely impaired individuals, NeuroDRIVE was designed for use with a wide range of TBI patients (i.e., mild, moderate, or severe TBI) who are seeking treatment in these areas and have the capability to engage in the driving process. This pilot clinical trial examined feasibility and preliminary efficacy of NeuroDRIVE for improving VR driving performance, cognitive performance, and symptom outcomes among those with chronic TBI. Given the focus of the intervention, effects on attention and working memory were of particular interest. Additionally, we have provided the NeuroDRIVE treatment manual as a supplementary document to facilitate continued development of VR rehabilitation for those with TBI.
2Methods
Institutional review boards of the National Institutes of Health Clinical Center and the Uniformed Services University of the Health Sciences approved this study. This study is registered on clinicaltrials.gov (NCT02411227). All participants provided written informed consent prior to participation.
2.1Participants
Male and female civilians and military personnel in the Washington D.C. metropolitan area with a history of TBI (greater than 12 weeks prior to enrollment; M = 15.78 years, SD = 12.79 years) were recruited for this study. History of TBI was evaluated using the Ohio State University Traumatic Brain Injury Identification Method (OSU TBI-ID; Corrigan & Bogner, 2007). TBI severity was classified according to DOD/VA guidelines (Department of Veterans Affairs, 2009). Participants were categorized as having moderate/severe TBI if they had a head injury with loss of consciousness (LOC) >30 minutes, alteration of consciousness (AOC) or post traumatic amnesia (PTA) >24 hours, or radiological findings of intracranial abnormalities secondary to head trauma. Participants were categorized as having mild TBI if they did not meet criteria for moderate-to-severe TBI but had a head injury with positive LOC (<30 minutes), AOC (<24 hours), or PTA (<24 hours).
Participants were included in the study if they were 1) in possession of a valid US driver’s license, 2) 18 years of age or older, 3) able to effectively manipulate the steering wheel and the gas/brake pedals of the simulator without adaptive equipment, and 4) able to read, write, and speak English. Participants were excluded from the study if they had 1) a history of penetrating brain injury, 2) a history of serious medical condition other than TBI that could affect cognitive or motor abilities, 3) a history of severe motion sickness and/or vertigo, or 4) other medical or psychological instability that could create difficulty fulfilling the study requirements. Because magnetic resonance imaging (MRI) was performed as part of the baseline and follow-up assessments (results to be reported separately), participants with risk of injury from the magnet and pregnant women were excluded from the study. For the mild TBI subgroup, participants were also excluded if they had a Neurobehavioral Symptom Inventory (NBSI) score <16 to ensure that this sub-sample represented the chronic/symptomatic population that is relevant to clinical rehabilitation settings.
Information about participant flow is presented in Fig. 1. As shown, 34 individuals met all inclusion/exclusion criteria and were enrolled in the study. One participant was withdrawn for MRI-related vertigo, three participants were withdrawn after enrollment due to simulation sickness (i.e., VR-induced motion sickness) which prevented completion of the baseline assessment, and three participants dropped out before completing baseline assessment procedures. Twenty-seven participants with a history of TBI completed the baseline assessment. Seventeen participants were randomly assigned to the Intervention group, four of whom dropped out or were lost to follow-up, and two of whom were withdrawn due to simulation sickness. Ten participants were randomly assigned to the Wait List group, three of whom dropped out or were lost to follow-up, while one who was withdrawn due to simulation sickness. Seventeen total participants completed the majority of the baseline and follow-up assessment procedures, including eleven participants in the intervention group and six in the wait listed group. Characteristics of participants who completed the study are shown in Table 1.
Fig.1
Participant Flow Chart.
Table 1
Demographic and Clinical Characteristics of Study Population
| Characteristic | Intervention | Wait List | P |
| Total n | 11 | 6 | – |
| Age, years (SD) | 49.73 (9.05) | 56.17 (7.13) | 0.43 |
| # Male (%) | 7 (63.6%) | 4 (66.7%) | 0.90 |
| Race/Ethnicity | 0.26 | ||
| # Caucasian (%) | 6 (54.5%) | 5 (83.3%) | |
| # African-American (%) | 3 (27.3%) | 0 (0.0%) | |
| # Hispanic/Latino (%) | 1 (9.1%) | 0 (0.0%) | |
| # Other (%) | 1 (9.1%) | 1 (16.7%) | |
| Injury Severity | 0.37 | ||
| # Mild (%) | 7 (63.6%) | 2 (33.3%) | |
| # Moderate/Severe (%) | 1 (9.1%) | 2 (33.3%) | |
| # Severe (%) | 3 (27.3%) | 2 (33.3%) | |
| Cause of injury | 0.61 | ||
| # Falls (%) | 5 (45.4%) | 2 (33.3%) | |
| # Motor vehicle accident (%) | 5 (45.4%) | 3 (50.0%) | |
| # Pedestrian-motor vehicle accident (%) | 1 (9.1%) | 1 (16.7%) | |
| Estimated Verbal IQ (SD)† | 105.40 (11.82) | 116.75 (3.50) | 0.02 |
| Glasgow Outcome Scale-Extended | 6.10 (1.10) | 5.20 (1.10) | 0.20 |
†Comparison n = 14.
2.2Study design
After pre-screening, eligible participants were randomized into either the Intervention or Wait List study groups and they completed a medical evaluation and the baseline assessment. The baseline assessment included a comprehensive battery of neuropsychological tests and self-report questionnaires and the VR Driving Assessment, as well as a battery of MRI measurements (to be reported separately). Beginning 4 weeks later, participants in the Intervention group received six 90-minute sessions of VR Driving Rehabilitation (NeuroDRIVE) over a period of approximately 4 weeks. Participants then completed a follow-up assessment that included the same outcome measures as the initial baseline assessment. Participants in the Wait List group completed the follow-up assessment approximately 10 weeks following the pre-assessment, before receiving six sessions of NeuroDRIVE.
2.3Virtual reality driving simulator
VR assessments and interventions were completed using the General Simulation Driver Guidance System (MBFARR, LLC; Moncrief, Behensky, Harkins, & Fuller, 2015; see Fig. 2). This simulator consisted of an 8-foot circular frame supporting a curved screen and a driving console analogous to that found in a typical automobile. Three digital video projectors displayed a 180-degree field of view, including simulated side- and rear-view mirrors. The driving console had turn signals, gas and brake pedals, a steering wheel, digital dashboard, and a seat belt. Participants sat in the console and operated the steering wheel and pedals while responding to the virtual environment projected onto the screen and auditory stimuli from connected speakers. The examiner configured and initiated VR scenarios from a nearby computer, provided instructions to the participant, and solicited verbal responses or answered questions as needed. Additionally, during each assessment or intervention session, research staff followed a protocol to gradually acclimate the participant to the VR environment, evaluate potential symptoms of simulation sickness, and manage any symptoms that occurred.
Fig.2
T3 VR Driving Simulator.
2.4NeuroDRIVE intervention
The NeuroDRIVE intervention consisted of six 90-minute sessions (9 hours total) conducted over a four-week period. Intervention sessions were designed to allow participants to practice driving skills as well as cognitive skills. Both the examiner and the automated system provided guidance and feedback throughout the session to assist the participant with improving their performance. A typical intervention session included the following key elements: 1) a brief review of training and progress thus far (or an introduction, for the first session); 2) practice of component cognitive skills such as dual processing, working memory, and response inhibition through the use of standardized cognitive driving scenarios; 3) practice of composite driving skills such as following the rules of the road and being vigilant for road hazards while simultaneously performing working memory or visual attention tasks; and 4) finishing with an open-ended race-track course to promote engagement in the process and to allow participants to safely “test the limits” of their skills in a simulated environment. Early sessions focused on fundamental, basic skills, and later sessions incorporated more complex, higher-order skills. Difficulty was also modulated by the examiner in each activity to match the skill level of the participant. Additional intervention details are provided in the Supplementary Document, “NeuroDRIVE Intervention Manual”.
2.5Measures
2.5.1Neuropsychological assessment
A battery of neuropsychological and self-report measures was selected from measures previously defined by a US interagency panel as TBI Common Data Elements. All measurements were performed by trained psychometrists under the supervision of a licensed clinical neuropsychologist. The evaluation was completed in approximately four hours plus breaks. Cognitive performance measures were used to assess working memory (WAIS-IV Digit Span: Wechsler, 2008), visual search/selective attention (Trail Making Test Part A: Reitan, 1992; WAIS-IV Symbol Search, WAIS-IV Coding: Wechsler, 2008), set shifting (Trail Making Test Part B: Reitan, 1992), verbal fluency (Controlled Oral Word Association Test, Letters & Animals: Benton, Hamsher, & Sivan, 1994), learning/memory (California Verbal Learning Test-II: Delis, Kramer, Kaplan, & Ober, 2000), and fine motor dexterity (Grooved Pegboard: Ruff & Parker, 1993). Self-report measures were used to assess overall neurobehavioral symptoms (Neurobehavioral Symptom Inventory: Cicerone & Kalmar, 1995), post-traumatic stress (PTSD Checklist-Civilian: Weathers, Litz, Herman, Huska, & Keane, 1994), depression (Beck Depression Inventory-II: Beck, Steer, & Brown, 1996), sleep and fatigue (Epworth Sleepiness Scale: Johns, 1991; Fatigue Severity Scale: Krupp, LaRocca, Muir-Nash, & Steinberg, 1989), overall physical and mental health (SF-36v2 Health Survey: Ware, 1993) and quality of life (Satisfaction with Life Scale: Diener, Emmons, Larsen, & Griffin, 1985). Additional measures were administered to characterize the participant sample in terms of estimated verbal intelligence (Test of Premorbid Functioning: Pearson, 2009) and global functional outcome (Glasgow Outcome Scale-Extended: Wilson, Pettigrew, & Teasdale, 1998).
2.5.2Virtual reality driving assessment
For the VR Driving Assessment, participants completed a series of automated software scenarios using the driving simulator. VR scenarios were designed to measure sensory-motor abilities (e.g., braking and steering in response to driving stimuli), cognitive abilities (e.g., attention to and memory of road stimuli), and holistic driving skills (e.g., ability to follow rules of the road). All scenarios employed a driving-relevant context in which the participant responded to stimuli by executing driving maneuvers. VR scenarios provided automated instructions to the participant and collected driving-relevant data such as use of the steering wheel, brakes, and turn signals. Some scenarios also required the participant to provide verbal responses to the examiner. Driving ability was quantified using two different scores: VR Operational Composite and VR Tactical Composite. The VR Operational Composite score was derived from 10 different Operational subtests, including structured scenarios designed to assess individual cognitive, visual and motor skills within a driving-relevant virtual environment (see Table 2). The VR Tactical Composite score was derived from 15 different variables related to speed control, steering, braking, and judgement that were collected during a simulated four-mile drive with traffic (see Table 3). Throughout the tactical scenario, an automated voice provided navigation instructions, similar to the verbal guidance provided by a GPS unit. Composite scores were computed as the average of the individual test/variable z-scores, converted based on normative data into a standard score with a mean of 100 and a standard deviation of 15. The normative sample consisted of 448 healthy participants recruited from Virginia Department of Motor Vehicles locations (Cox, 2014) and was 27% female, aged 43.52 ± 12.82 years with 15 ± 3.27 years of education. Test-retest reliability has been estimated at r = 0.80 for the Tactical Composite Score and r = 0.49 for the Operational Composite Score (Cox, 2014). In order to minimize potential practice effects (e.g., remembering the location of specific road challenges), alternate versions of the VR Tactical scenario were used for the baseline and follow-up assessments; these alternate forms were designed to provide comparable numbers and types of different driving challenges. See Tables 2 and 3 for additional information about VR Driving Assessment scenarios.
Table 2
Virtual Reality Driving Simulator Operational Score Components
| Domain | Tasks | Task Description |
| Vision | Visual Acuity | Read progressively smaller letters, down to 20/30. |
| Contrast Sensitivity | Read progressively lighter letters, down to 2.35% black letters on a white background. | |
| Dynamic Vision | Read rapidly scrolling letters across a digital road sign, down to 700 milliseconds. | |
| Peripheral Vision | Detect vehicles presented from 55 to 70 degrees peripherally. | |
| Motor | Arm/Hand Coordination | Steer around all potholes. |
| Foot/Leg Coordination | Brake to forward vehicle. | |
| Cognitive | Rapid Information Processing Speed | Identify rapidly-presented road signs while in moving vehicle, down to 16ms. |
| Divided Attention | Identify rapidly-presented road signs and vehicles, with distractors, down to 16ms. | |
| Dual Processing | Steer around potholes and brake for forward vehicle. | |
| Response Inhibition | Steer around potholes (unfilled only) and brake for forward vehicle. | |
| Working Memory | Recall traffic signs, steer around unfilled potholes and braking for forward vehicle. |
Table 3
Virtual Reality Driving Simulator Tactical Score Components
| Domain | Variables |
| Speed Control | Tailgating |
| Speeding (5 mph over posted speed) | |
| Reckless Driving (20 mph over posted speed) | |
| Difficulty Maintaining Stable Speed | |
| Steering | Roadway Departure Resets |
| Driving Across Midline | |
| Swerving | |
| Driving Off-Road | |
| Braking & | Rolling Stops |
| Accidents | Deceleration Smoothness |
| High Speed Collisions | |
| Low Speed Collisions | |
| Judgment | Driving Too Slowly (20mph below posted speed) |
| Number of Lane Changes | |
| Missed Turns |
2.6Statistical analysis
All statistical analyses were performed using SPSS v 24.0. Chi-square analyses and independent-samples t-tests were performed to compare participant group characteristics. A series of ANCOVAs were performed to evaluate the effect of the NeuroDRIVE intervention on driving abilities, cognitive abilities, neurobehavioral symptoms, symptoms of post-traumatic stress, and measures of overall health/wellbeing. For each measure, an ANCOVA was conducted with intervention group (Intervention vs. Wait List) as the independent variable, the baseline assessment score as a covariate (to control for potential practice effects), and the follow-up assessment score as the dependent variable. Results were considered significant if p < 0.05. Missing data were excluded pairwise for all analyses.
3Results
3.1Feasibility and participant dropout
Most participants completed study procedures without incident; however, as described previously, 4/31 participants (12.9%) were withdrawn due to simulation sickness elicited by the VR assessment, and an additional 2/17 participants randomized to the intervention group (11.8%) were withdrawn due simulation sickness elicited by the VR intervention. Four of 15 (26.6%) of the remaining participants in the intervention group dropped out for other reasons, which is similar to the 3/10 (30.0%) drop-out rate in the wait list group.
3.2VR driving performance
Summarized results of ANCOVA analyses are presented in Table 4. As shown, change in VR tactical and VR operational composite scores from baseline to follow-up did not differ significantly between the intervention and wait list groups (p > 0.05; see Fig. 3). Within the intervention group, overall VR driving scores (Tactical and Operational) fell within the ‘average’ range at both baseline and follow-up time points. Within the wait list group, VR tactical driving performance also fell within the ‘average’ range at both time points. However, in contrast to the intervention group, the wait list group demonstrated overall ‘impaired’ VR Operational driving performance at baseline and ‘low average’ performance at follow-up.
Fig.3
Effects of NeuroDRIVE intervention on VR driving performance.
Table 4
Baseline and Follow-Up Measurements for Intervention and Wait List Groups
| Intervention Group n = 11 | Wait List Group n = 6 | Analytic n | Partial η2 | |||||||
| Outcome measure | PRE | POST | PRE | POST | ||||||
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | |||
| VR Tactical Driving Quotient | 100.88 | 9.91 | 102.13 | 4.36 | 91.25 | 16.88 | 104.50 | 5.80 | 12 | 0.08 |
| VR Operational Driving Quotient | 92.09 | 22.28 | 101.27 | 34.68 | 67.33 | 29.51 | 83.83 | 44.14 | 17 | 0.02 |
| WAIS-IV Digit Span (SS) | 10.30 | 2.98 | 11.10 | 3.38 | 11.60 | 2.51 | 11.00 | 3.46 | 15 | 0.51** |
| WAIS-IV Symbol Search (SS) | 10.60 | 1.51 | 12.70 | 2.11 | 9.20 | 3.90 | 9.00 | 1.87 | 15 | 0.42* |
| WAIS-IV Coding (SS) | 10.90 | 1.66 | 11.20 | 1.48 | 10.20 | 1.79 | 10.60 | 2.19 | 15 | 0.00 |
| Trail Making Test Part A (t) | 52.70 | 12.89 | 52.40 | 9.67 | 44.80 | 11.12 | 44.20 | 15.34 | 15 | 0.03 |
| Trail Making Test Part B (t) | 53.70 | 8.65 | 57.10 | 8.33 | 41.20 | 16.16 | 46.00 | 9.97 | 15 | 0.09 |
| Letter Fluency (t) | 50.44 | 9.94 | 53.56 | 10.24 | 46.20 | 11.52 | 49.00 | 8.57 | 14 | 0.01 |
| Animal Fluency (t) | 51.67 | 9.43 | 51.33 | 6.32 | 37.00 | 18.57 | 43.00 | 14.76 | 14 | 0.00 |
| CVLT-II Trials 1-5 Total Recall (t) | 51.91 | 13.35 | 49.64 | 11.02 | 39.80 | 11.30 | 44.60 | 13.87 | 16 | 0.11 |
| CVLT-II Short Delay Free Recall (z) | – 0.05 | 1.33 | 0.23 | 1.46 | – 1.10 | 1.71 | – 0.60 | 1.64 | 16 | 0.00 |
| CVLT-II Long Delay Free Recall (z) | – 0.05 | 1.25 | 0.09 | 1.32 | – 1.00 | 2.03 | – 1.00 | 1.94 | 16 | 0.03 |
| Grooved Pegboard Dominant (t) | 53.90 | 8.41 | 51.70 | 7.78 | 37.20 | 13.50 | 45.60 | 10.36 | 15 | 0.06 |
| Grooved Pegboard Nondominant (t) | 51.00 | 10.81 | 51.80 | 8.69 | 29.20 | 17.28 | 32.80 | 18.02 | 15 | 0.02 |
| Neurobehavioral Symptom Inventory Total | 25.90 | 12.20 | 16.30 | 8.29 | 30.50 | 10.61 | 9.50 | 6.36 | 12 | 0.12 |
| PTSD Checklist-Civilian Total | 34.91 | 12.21 | 28.00 | 9.55 | 27.00 | 7.75 | 24.80 | 6.10 | 16 | 0.00 |
| Beck Depression Inventory-II Total | 11.55 | 6.64 | 7.55 | 4.63 | 9.00 | 8.31 | 5.80 | 3.77 | 16 | 0.02 |
| Satisfaction with Life Scale Total | 23.60 | 7.73 | 26.00 | 7.29 | 27.20 | 10.26 | 24.60 | 6.19 | 15 | 0.09 |
| SF-36 Physical Component Total | 52.41 | 6.77 | 55.67 | 4.52 | 45.35 | 5.85 | 43.72 | 7.90 | 15 | 0.39* |
| SF-36 Mental Component Total | 44.89 | 11.06 | 46.25 | 12.23 | 53.41 | 12.41 | 55.69 | 6.81 | 15 | 0.05 |
| Epworth Sleepiness Scale Total | 7.20 | 5.03 | 7.50 | 3.98 | 5.20 | 3.11 | 4.20 | 2.17 | 15 | 0.19 |
| Fatigue Severity Scale Total | 33.30 | 16.85 | 29.00 | 14.27 | 26.40 | 8.38 | 24.80 | 11.50 | 15 | 0.00 |
*p < 0.05. **p < 0.01.
3.3Cognitive performance
Relative to the wait list group, the intervention group demonstrated significantly greater improvement in working memory (WAIS-IV Digit Span, partial η2 = 0.51, p = 0.004) and visual search/selective attention (WAIS-IV Symbol Search, partial η2 = 0.42, p = 0.01) from baseline to follow-up assessment visits (see Table 4). As shown in Fig. 4, Digit Span performance improved for 6/10 (60.0%) in the intervention group (mean SS change = 0.8) as compared to 1/5 (20.0%) in the waitlisted group (mean scaled score [SS] change = – 0.6). Similarly, as shown in Fig. 5, Symbol Search performance improved for 9/10 (90.0%) receiving intervention (mean SS change = 2.1), as compared to 3/5 (60.0%) on the wait list (mean SS change = – 0.2). Change on other measures of cognitive performance (including WAIS-IV Coding, Trail Making Test A/B, Letter & Animal Fluency, California Verbal Learning Test-II, and Grooved Pegboard) did not differ significantly between the intervention and wait listgroups.
Fig.4
Effects of NeuroDRIVE intervention on working memory performance.
Fig.5
Effects of NeuroDRIVE intervention on visual search performance.
3.4Neurobehavioral symptoms and quality of life
Comparison of self-report measures and rating scales demonstrated significantly greater improvement in physical functioning (SF-36 Physical Component, partial η2 = 0.39, p < 0.05) within the intervention group relative to the wait list group (see Table 4). As shown in Fig. 6, SF-36 Physical Component score improved among 6/10 (60.0%) participants receiving intervention (mean change = 3.07), as compared to 2/5 (40.0%) on the wait list (mean change = – 1.63). On the other hand, change in mental functioning (SF-36 Mental Component) did not significantly differ between groups (partial η2 = 0.05, p > 0.05); although a greater proportion improved in the intervention group (7/10 [70.0%]) relative to the wait list group (2/5 [40%]), the wait list group demonstrated superior average improvement overall (wait list M = 2.28 vs intervention M = 1.36). Change on other measures of neurobehavioral symptoms and global outcomes (including Neurobehavioral Symptom Inventory, PTSD Checklist-Civilian, Beck Depression Inventory-II, Satisfaction with Life Scale, Epworth Sleepiness Scale, and Fatigue Severity Scale) did not significantly differ between the intervention and wait list groups (p > 0.05).
Fig.6
Effects of NeuroDRIVE intervention on health-related quality of life.
4Discussion
This study was conducted as a pilot investigation of NeuroDRIVE, a novel rehabilitative intervention for TBI involving clinician-guided cognitive and driving exercises conducted within a VR environment. Eleven participants who received the intervention were compared to six wait-listed participants on driving abilities, cognitive performance, neurobehavioral symptoms, and quality of life. The results of this study provide an illustration of both the promises and the challenges of using NeuroDRIVE, and VR (more generally) in cognitive and functional rehabilitation of chronic TBI.
VR-based rehabilitation offers a number of potential advantages, such as higher levels of safety (compared to driving rehabilitation conducted in the real world) as well as enhanced engagement and immersion (compared to conventional forms of cognitive rehabilitation). However, VR also presents potential barriers to routine clinical use, including significant investments of time and resources for technology management and issues related to tolerability of VR interventions. Specifically, simulation sickness is a well-known issue, but little is known about its impact on the provision of rehabilitation. In this study, 12.9% of participants were unable to tolerate the initial VR assessment, and another 11.8% of those remaining were unable to tolerate the more extensive VR intervention procedures. These rates of VR sickness are consistent with the 7–20% incidence suggested in previous literature (Brooks et al., 2010). Continued improvements in VR technology and procedures for use may help to reduce the incidence of VR sickness. Until that time, this issue is likely to persist as a barrier to VR rehabilitation for a subset of clinical patients and research participants.
For those individuals with chronic TBI who are able to tolerate VR, the results of this study provide encouraging preliminary support for the use of this technology for cognitive rehabilitation. Participation in the NeuroDRIVE intervention was associated with sizable and statistically-significant improvements in working memory and visual search/selective attention— two cognitive skills that represented a primary focus of the intervention. By comparison, no significant changes were observed in untrained areas such as verbal fluency, verbal learning/memory or fine motor performance. These results suggest that immersive virtual environments (such as those used in the NeuroDRIVE intervention) can provide a valuable and engaging means to achieve cognitive rehabilitation goals, particularly when these goals are closely matched to the VR training exercises. For research purposes, the intervention exercises used in this study were standardized across participants; however, greater levels of cognitive improvement may be attained in clinical practice by tailoring the VR rehabilitation activities to the areas of greatest need for individual patients, and by providing a greater quantity of intervention overall.
Contrary to our expectations based on previous research, the NeuroDRIVE intervention did not produce significant improvements in driving skills (relative to the control group) as measured by the VR driving assessment. This finding may be due to a number of factors related to our chosen approach to driving rehabilitation, as well as to some limitations in our study sample. Potentially, the NeuroDRIVE intervention method of providing simultaneous training of cognitive and driving skills may have interfered with our ability to produce a positive impact on driving skills; more focused driving training may be optimal to achieve benefits in this complex functional skill. Additionally, compared to a more acute population, the chronic TBI population examined in this study may be less receptive to (or less in need of) training in a functional skill like driving. Just as elderly drivers may adopt strategies to manage driving challenges and reduce the risk of accidents (Langford & Koppel, 2006), those with more remote TBIs may have already functionally adapted to injury-related changes in cognitive, sensory, or motor function, and may be less likely to benefit from additional training. Consistent with this possibility, our intervention group demonstrated ‘average’-level driving performance even before receiving the intervention. Future studies of VR driving rehabilitation may benefit from a focus on those individuals who show measurable driving impairments prior to beginning the intervention. Of note, our control group also demonstrated significantly higher estimated preinjury intellectual functioning (high average range overall) than our intervention group (average range overall). This difference may have also influenced our participants’ relative receptivity to intervention and could have also enhanced practice effects within the control group, potentially reducing the measured effect size of treatment.
While relatively few clinical studies have focused on VR approaches to cognitive and functional rehabilitation, a sizeable body of literature has shown that VR can be useful for treatment of mental health conditions such as phobias (Parsons & Rizzo, 2008; Walshe, Lewis, Kim, O’Sullivan, & Wiederhold, 2003) and post-traumatic stress (Gerardi, Cukor, Difede, Rizzo, & Rothbaum, 2010; Rizzo, Reger, Gahm, Difede, & Rothbaum, 2009). As expected within a population that has suffered traumatic injuries, our sample exhibited relatively high levels of post-traumatic stress overall. In many individual cases, our participants’ TBIs were sustained in a motor vehicle accident. Not uncommonly, these participants anecdotally reported apprehension, avoidance, or hypervigilance related to driving. Targeted treatment of post-traumatic stress was not a focus of this pilot study, however, exposure and/or relaxation techniques would be relatively straightforward to integrate into future VR treatment protocols and may increase its clinical value for many TBI patients.
This study has a number of limitations that are important to consider in interpretation of our results. Most notably, the overall sample size was small, and may have contributed to uneven randomization and a limited ability to detect significant differences between the intervention and control groups. The drop-out rate in this study (35–40%) also introduces the possibility that factors related to selective attrition could have impacted results. Future studies with larger sample sizes are warranted to replicate and extend these findings. Additionally, stratified randomization methods may reduce the likelihood of pre-treatment differences between groups.
In summary, the results of this study provide an encouraging early view of the clinical potential of NeuroDRIVE and related VR-based approaches to rehabilitation of cognitive and functional deficits in TBI. Additional research, including larger-scale studies directly comparing NeuroDRIVE with other active treatment modalities, should be conducted to validate and extend these findings. Continued development should be performed to further increase the feasibility and effectiveness of this technology and the underlying rehabilitation techniques in preparation for use in real-world clinical environments. Additional research is also needed to examine potential neural mechanisms of intervention effects, and to examine risk factors for unsafe driving after TBI.
Conflict of interest
The authors report no conflicts of interest in the conduct of this research.
Acknowledgments
This study was sponsored by the Center for Neuroscience and Regenerative Medicine, with additional institutional support provided by the Uniformed Services University of the Health Sciences and the Defense and Veterans Brain Injury Center.
The views and opinions presented in this manuscript are those of the authors, and do not necessarily represent the position of USUHS, the Department of Defense, the Department of the Navy, or the United States government.
Supplementary material
[1] The supplementary material is available in the electronic version of this article: http://dx.doi.org/10.3233/NRE-192718.
References
1 | Andelic N. , Stevens L.F. , Sigurdardottir S. , Arango-Lasprilla J.C. , & Roe C. ((2012) ). Associations between disability and employment 1 year after traumatic brain injury in a working age population. Brain Injury, 26: (3), 261–269. |
2 | Ball K. , Edwards J.D. , & Ross L.A. ((2007) ). The impact of speed of processing training on cognitive and everyday functions. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 62: (Special_Issue_1), 19–31. |
3 | Beck A.T. , Steer R.A. , & Brown G.K. ((1996) ). Beck depression inventory manual (2nd ed.). San Antonio, TX: Psychological Corporation. |
4 | Benton L. , Hamsher K. , & Sivan A. ((1994) ). Controlled oral word association test. Multilingual aphasia examination, 3. |
5 | Bivona U. , D’Ippolito M. , Giustini M. , Vignally P. , Longo E. , Taggi F. , & Formisano R. ((2012) ). Return to driving after severe traumatic brain injury: Increased risk of traffic accidents and personal responsibility. J Head Trauma Rehabil, 27: (3), 210–215. doi:10.1097/HTR.0b013e31822178a9 |
6 | Bottari C. , Lamothe M.-P. , Gosselin N. , Gélinas I. , & Ptito A. ((2012) ). Driving difficulties and adaptive strategies: The perception of individuals having sustained a mild traumatic brain injury, Rehabilitation Research and Practice, (2012) . |
7 | Brooks J.O. , Goodenough R.R. , Crisler M.C. , Klein N.D. , Alley R.L. , Koon B.L. , Logan W.C. , Ogle J.H. , Tyrell R.A. , & Wills R.F. ((2010) ). Simulator sickness during driving simulation studies. Accident Analysis & Prevention, 42: (3), 788–796. |
8 | Cicerone K.D. ((2002) ). Remediation of “working attention” in mild traumatic brain injury. Brain Inj, 16: (3), 185–195. doi:10.1080/02699050110103959 |
9 | Cicerone K.D. , & Kalmar K. ((1995) ). Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury. The Journal of Head Trauma Rehabilitation, 10: (3), 1–17. |
10 | Cicerone K.D. , Langenbahn D.M. , Braden C. , Malec J.F. , Kalmar K. , Fraas M. , Felicetti T. , Laatsch L. , Harley P. , Bergquist T. , Azulay J. , Cantor J. , & Bergquist T. ((2011) ). Evidence-based cognitive rehabilitation: Updated review of the literature from 2003 through 2008. Archives of Physical Medicine and Rehabilitation, 92: (4), 519–530. |
11 | Classen S. , Levy C. , Meyer D.L. , Bewernitz M. , Lanford D.N. , & Mann W.C. ((2011) ). Simulated driving performance of combat veterans with mild traumatic brain injury and posttraumatic stress disorder: A pilot study. American Journal of Occupational Therapy, 65: (4), 419–427. |
12 | Cooper D.B. , Bowles A.O. , Kennedy J.E. , Curtiss G. , French L.M. , Tate D.F. , & Vanderploeg R.D. ((2017) ). Cognitive rehabilitation for military service members with mild traumatic brain injury: A randomized clinical trial. The Journal of Head Trauma Rehabilitation, 32: (3), E1–E15. |
13 | Corrigan J.D. , & Bogner J. ((2007) ). Initial reliability and validity of the Ohio State University TBI Identification Method. J Head Trauma Rehabil, 22: (6), 318–329. doi:10.1097/01.HTR.0000300227.67748.77 00001199-200711000-00002 [pii]. |
14 | Cox D.J. ((2014) ). Virtual Reality Driving Simulation in Virginia Department of Motor Vehicles: Executive Summary. |
15 | Cox D.J. , Davis M. , Singh H. , Barbour B. , Nidiffer F.D. , Trudel T. , Mourant R. , & Moncrief R. ((2010) ). Driving rehabilitation for military personnel recovering from traumatic brain injury using virtual reality driving simulation: A feasibility study. Military Medicine, 175: (6), 411–416. |
16 | Cyr A.A. , Stinchcombe A. , Gagnon S. , Marshall S. , Hing M.M. , & Finestone H. ((2009) ). Driving difficulties of brain-injured drivers in reaction to high-crash-risk simulated road events: A question of impaired divided attention? J Clin Exp Neuropsychol, 31: (4), 472–482. doi:901408923 [pii]10.1080/13803390802255627 |
17 | Dawson D.R. , Schwartz M.L. , Winocur G. , & Stuss D.T. ((2007) ). Return to productivity following traumatic brain injury: Cognitive, psychological, physical, spiritual, and environmental correlates. Disability and Rehabilitation, 29: (4), 301–313. |
18 | Delis D.C. , Kramer J.H. , Kaplan E. , & Ober B.A. ((2000) ). California Verbal Learning Test - Second Edition. San Antonio, TX: Psychological Corporation. |
19 | Department of Veterans Affairs D.o.D. ((2009) ). VA/DoD clinical practice guideline for management of concussion/mild traumatic brain injury. In. Washington, DC: Department of Veteran Affairs, Department of Defense. |
20 | Devos H. , Akinwuntan A.E. , Nieuwboer A. , Tant M. , Truijen S. , De Wit L. , Kiekens C. , & De Weerdt W. ((2009) ). Comparison of the effect of two driving retraining programs on on-road performance after stroke. Neurorehabil Neural Repair, 23: (7), 699–705. doi:1545968309334208 [pii] 10.1177/1545968309334208 |
21 | Diener E. , Emmons R.A. , Larsen R.J. , & Griffin S. ((1985) ). The satisfaction with life scale. Journal of Personality Assessment, 49: (1), 71–75. |
22 | Drake A.I. , Gray N. , Yoder S. , Pramuka M. , & Llewellyn M. ((2000) ). Factors predicting return to work following mild traumatic brain injury: A discriminant analysis. J Head Trauma Rehabil, 15: (5), 1103–1112. |
23 | Fadyl J.K. , & McPherson K.M. ((2009) ). Approaches to vocational rehabilitation after traumatic brain injury: A review of the evidence. J Head Trauma Rehabil, 24: (3), 195–212. doi:10.1097/HTR.0b013e3181a0d458 00001199-200905000-00006 [pii] |
24 | Fisher M. , Holland C. , Merzenich M.M. , & Vinogradov S. ((2009) ). Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia. American Journal of Psychiatry, 166: (7), 805–811. |
25 | Formisano R. , Bivona U. , Brunelli S. , Giustini M. , Longo E. , & Taggi F. ((2005) ). A preliminary investigation of road traffic accident rate after severe brain injury. Brain Inj, 19: (3), 159–163. |
26 | Gerardi M. , Cukor J. , Difede J. , Rizzo A. , & Rothbaum B.O. ((2010) ). Virtual reality exposure therapy for post-traumatic stress disorder and other anxiety disorders. Current Psychiatry Reports, 12: (4), 298–305. |
27 | Giles G.M. ((2010) ). Cognitive versus functional approaches to rehabilitation after traumatic brain injury: Commentary on a randomized controlled trial. Am J Occup Ther, 64: (1), 182–185. |
28 | Imhoff S. , Lavallière M. , Teasdale N. , & Fait P. ((2016) ). Driving assessment and rehabilitation using a driving simulator in individuals with traumatic brain injury: A scoping review. NeuroRehabilitation, 39: (2), 239–251. |
29 | Johns M.W. ((1991) ). A new method for measuring daytime sleepiness: The Epworth sleepiness scale. sleep, 14: (6), 540–545. |
30 | Kalechstein A.D. , Newton T.F. , & van Gorp W.G. ((2003) ). Neurocognitive functioning is associated with employment status: A quantitative review. J Clin Exp Neuropsychol, 25: (8), 1186–1191. doi:10.1076/jcen.25.8.1186.16723 |
31 | Kennedy M.R. T. , Coelho C. , Turkstra L. , Ylvisaker M. , Moore Sohlberg M. , Yorkston K. , Chiou H. , & Kan P.F. ((2008) ). Intervention for executive functions after traumatic brain injury: A systematic review, meta-analysis and clinical recommendations. Neuropsychological Rehabilitation, 18: (3), 257–299. |
32 | Kim Y.H. , Yoo W.K. , Ko M.H. , Park C.H. , Kim S.T. , & Na D.L. ((2009) ). Plasticity of the attentional network after brain injury and cognitive rehabilitation. Neurorehabil Neural Repair, 23: (5), 468–477. doi:1545968308328728 [pii] 10.1177/1545968308328728 |
33 | Krupp L.B. , LaRocca N.G. , Muir-Nash J. , & Steinberg A.D. ((1989) ). The fatigue severity scale: Application to patients with multiple sclerosis and systemic lupus erythematosus. Archives of Neurology, 46: (10), 1121–1123. |
34 | Langford J. , & Koppel S. ((2006) ). Epidemiology of older driver crashes– identifying older driver risk factors and exposure patterns. Transportation Research Part F: Traffic Psychology and Behaviour, 9: (5), 309–321. |
35 | Langlois J.A. , Rutland-Brown W. , & Thomas K.E. ((2006) ). Traumatic brain injury in the United States emergency department visits, hospitalizations, and deaths. Retrieved from Atlanta, GA: Centers for Disease Control and Prevention. |
36 | Leahy B.J. , & Lam C.S. ((1998) ). Neuropsychological testing and functional outcome for individuals with traumatic brain injury. Brain Inj, 12: (12), 1025–1035. |
37 | Lebowitz M.S. , Dams-O’Connor K. , & Cantor J.B. ((2012) ). Feasibility of computerized brain plasticity-based cognitive training after traumatic brain injury. Journal of Rehabilitation Research and Development, 49: (10), 1547–1556. |
38 | Lew H.L. , Rosen P.N. , Thomander D. , & Poole J.H. ((2009) ). The potential utility of driving simulators in the cognitive rehabilitation of combat-returnees with traumatic brain injury. J Head Trauma Rehabil, 24: (1), 51–56. doi:10.1097/HTR.0b013e3181956fe3 00001199-200901000-00007 [pii] |
39 | Lovell M.R. , & Solomon G.S. ((2011) ). Psychometric data for the NFL neuropsychological test battery. Applied Neuropsychology, 18: (3), 197–209. |
40 | Lundqvist A. ((2001) ). Neuropsychological aspects of driving characteristics. Brain Injury, 15: (11), 981–994. |
41 | Lundqvist A. , & Rönnberg J. ((2001) ). Driving problems and adaptive driving behaviour after brain injury: A qualitative assessment. Neuropsychological Rehabilitation, 11: (2), 171–185. |
42 | Marin J.R. , Weaver M.D. , Yealy D.M. , & Mannix R.C. ((2014) ). Trends in visits for traumatic brain injury to emergency departments in the United States. JAMA, 311: (18), 1917–1919. |
43 | Moncrief R.L. , Behensky M.L. , Harkins T.G. , & Fuller B.A. ((2015) ) Driving assessment and training method and apparatus. In Google Patents. |
44 | Novack T.A. , Alderson A.L. , Bush B.A. , Meythaler J.M. , & Canupp K. ((2000) ). Cognitive and functional recovery at 6 and 12 months post-TBI. Brain Injury, 14: (11), 987–996. |
45 | Owen A.M. , Hampshire A. , Grahn J.A. , Stenton R. , Dajani S. , Burns A.S. , Howard R.J. , & Ballard C.G. ((2010) ). Putting brain training to the test. Nature, 465: (7299), 775–778. doi:nature09042 [pii] 10.1038/nature09042 |
46 | Parsons T.D. , & Rizzo A.A. ((2008) ). Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: A meta-analysis. Journal of Behavior Therapy and Experimental Psychiatry, 39: (3), 250–261. |
47 | Pearson N. ((2009) ). Advanced clinical solutions for WAIS-IV and WMS-IV: Administration and scoring manual. The Psychological Corporation, San Antonio. |
48 | Ponsford J.L. , Downing M.G. , Olver J. , Ponsford M. , Acher R. , Carty M. , & Spitz G. ((2014) ). Longitudinal follow-up of patients with traumatic brain injury: Outcome at two, five, and ten years post-injury. Journal of Neurotrauma, 31: (1), 64–77. |
49 | Preece M.H. , Horswill M.S. , & Geffen G.M. ((2010) ). Driving after concussion: The acute effect of mild traumatic brain injury on drivers’ hazard perception. Neuropsychology, 24: (4), 493–503. doi:2010-13425-008 [pii] 10.1037/a0018903 |
50 | Preece M.H. , Horswill M.S. , & Geffen G.M. ((2011) ). Assessment of drivers’ ability to anticipate traffic hazards after traumatic brain injury. J Neurol Neurosurg Psychiatry, 82: (4), 447–451. doi:jnnp.2010.215228 [pii] 10.1136/jnnp.2010.215228 |
51 | Reitan R.M. ((1992) ). Trail Making Test: Manual for administration and scoring: Reitan Neuropsychology Laboratory. |
52 | Rizzo A. , Reger G. , Gahm G. , Difede J. , & Rothbaum B.O. ((2009) ). Virtual reality exposure therapy for combat-related PTSD. In Post-traumatic stress disorder (pp. 375–399): Springer. |
53 | Rohling M.L. , Faust M.E. , Beverly B. , & Demakis G. ((2009) ). Effectiveness of cognitive rehabilitation following acquired brain injury: A meta-analytic re-examination of Cicerone et al.’s (2000, 2005) systematic reviews. Neuropsychology, 23: (1), 20–39. doi:2008-19137-007 [pii] 10.1037/a0013659 |
54 | Ruff R.M. , & Parker S.B. ((1993) ). Gender- and age-specific changes in motor speed and eye-hand coordination in adults: Normative values for the Finger Tapping and Grooved Pegboard Tests. Percept Mot Skills, 76: (3 Pt 2), 1219–1230. |
55 | Schneider J.J. , & Gouvier W.D. ((2005) ). Utility of the UFOV test with mild traumatic brain injury. Appl Neuropsychol, 12: (3), 138–142. doi:10.1207/S15324826AN1203_3 |
56 | Selassie A.W. , Zaloshnja E. , Langlois J.A. , Miller T. , Jones P. , & Steiner C. ((2008) ). Incidence of long-term disability following traumatic brain injury hospitalization, United States, 2003. J Head Trauma Rehabil, 23: (2), 123–131. doi:10.1097/01.HTR.0000314531.30401.39 00001199-200803000-00007 [pii] |
57 | Smith G.E. , Housen P. , Yaffe K. , Ruff R. , Kennison R.F. , Mahncke H.W. , & Zelinski E.M. ((2009) ). A cognitive training program based on principles of brain plasticity: Results from the Improvement in Memory with Plasticity-based Adaptive Cognitive Training (IMPACT) Study. Journal of the American Geriatrics Society, 57: (4), 594–603. |
58 | Spiers H.J. , & Maguire E.A. ((2007) ). Neural substrates of driving behaviour. Neuroimage, 36: (1), 245–255. doi:S1053-8119(07)00163-2 [pii] 10.1016/j.neuroimage.2007.02.032 |
59 | Vanderploeg R.D. , Schwab K. , Walker W.C. , Fraser J.A. , Sigford B.J. , Date E.S. , Scott S.G. , Curtiss G. , Salazar A.M. , & Warden D.L. ((2008) ). Rehabilitation of traumatic brain injury in active duty military personnel and veterans: Defense and Veterans Brain Injury Center randomized controlled trial of two rehabilitation approaches. Arch Phys Med Rehabil, 89: (12), 2227–2238. doi:S0003-9993(08)01485-8 [pii] 10.1016/j.apmr.2008.06.015 |
60 | Walshe D.G. , Lewis E.J. , Kim S.I. , O’Sullivan K. , & Wiederhold B.K. ((2003) ). Exploring the use of computer games and virtual reality in exposure therapy for fear of driving following a motor vehicle accident. CyberPsychology & Behavior, 6: (3), 329–334. |
61 | Ware J.E. ((1993) ). Scoring the SF-36. SF-36. Health Survey: Manual and Interpretation Guide. |
62 | Weathers F. , Litz B. , Herman D. , Huska J. , & Keane T. ((1994) ). The PTSD checklist-civilian version (PCL-C). Boston, MA: National Center for PTSD. |
63 | Wechsler D. ((2008) ). Wechsler Adult Intelligence Scale - Fourth Edition: Administration and Scoring Manual. San Antonio, TX: Pearson, Inc. |
64 | Wehman P. , Targett P. , West M. , & Kregel J. ((2005) ). Productive work and employment for persons with traumatic brain injury: What have we learned after 20 years? J Head Trauma Rehabil, 20: (2), 115–127. doi:00001199-200503000-00001 [pii] |
65 | Wilson J.L. , Pettigrew L.E. , & Teasdale G.M. ((1998) ). Structured interviews for the Glasgow Outcome Scale and the extended Glasgow Outcome Scale: Guidelines for their use. Journal of Neurotrauma, 15: (8), 573–585. |
