Lessons learned from conducting a pragmatic, randomized, crossover trial on robot-assisted gait training in children with cerebral palsy (PeLoGAIT)

1.Introduction

Cerebral palsy (CP) is one of the most common physical disabilities, which affects about 2–3 children per 1,000 live births in Western countries [1]. It involves multiple dimensions of health. Children with CP are often restricted regarding independent mobility and daily walking activity, with one in three not being able to walk [2]. Thus, improving gait function in these children is often one of the main goals of the families and the rehabilitation team [3, 4]. Pediatric neurorehabilitation addresses this goal with various impairment-based approaches (i.e., focusing on the body structures and functions of the International Classification of Functioning, Disability and Health: Children and Youth Version (ICF-CY)) and more task-specific approaches (i.e., focusing on the activity component of the ICF-CY) [5]. A meta-analysis showed a substantial effect of different task-specific gait training interventions on walking speed in children with CP [5]. Additionally, Zwicker et al. found positive results on various parameters of the body function (e.g., step length, cadence, and gait speed) and the activity component of the ICF-CY (e.g., Gross Motor Function Measure-88 (GMFM-88) Dimension D and E) when summarizing the evidence of several systematic reviews on different kinds of treadmill training in children with neuromotor disorders [6].

New robotic rehabilitation technologies have revolutionized the neurorehabilitation field during the last 15 years and have increasingly been implemented in the clinical setting [7]. They facilitate prolonged active training duration with more repetitions in a consistent movement pattern and provide an attractive and entertaining training opportunity due to the implementation of serious games. Further, the training parameters are automatically recorded and allow for a better comparison of training intensity across different centers. One of these rehabilitation robots is the driven gait orthosis Lokomat® (Hocoma AG, Volketswil, Switzerland), which enables robot-assisted gait training (RAGT) for children starting at an early age of approximately five years [8]. The Lokomat automates gait therapy on a treadmill by two actuated leg orthoses which are available in pediatric and adult sizes [8, 9]. These orthoses are individually adapted to the patient’s legs and are attached with cuffs, while the patient is secured using a dynamic bodyweight support system with a harness providing partial bodyweight support. The Lokomat offers many possibilities for variations of training parameters, such as leg joints’ range of motion, gait speed, amount of bodyweight support, and guidance force [10]. Such modern technology raises high expectations in parents of children with CP regarding gains in their child’s walking abilities and the development of a gait pattern comparable to that of non-disabled children [3]. However, it is often a challenge for the therapy team to meet these expectations appropriately, as scientific evidence in this area shows large deficits. Although studies have provided evidence on the Lokomat’s feasibility and safety in pediatric populations [8, 11], information on the optimal frequency and duration of such interventions as well as the intensity and dosage of the training parameters is sparse. A recent systematic review concluded that no specific delivery modes for RAGT in children exist and that evidence of its effectiveness is still weak and inconsistent [12]. A primary reason for the weak evidence is that most published results were derived from uncontrolled single case studies or case series [8, 13, 14, 15, 16, 17] and only a few randomized controlled trials (RCT) have been published [18, 19, 20, 21, 22]. However, performing an RCT in rehabilitation is challenging. Most of these RCTs showed some methodological limitations. Limitations included the randomization procedure [18, 19], inconsistencies between inclusion criteria and included patient sample [18], a large dropout rate [19], and severe statistical flaws [20, 21].

When we started our RCT in 2009, only the results of the first uncontrolled single case studies and case series studies were available. As we were interested in the effects of RAGT as it is performed in our rehabilitation center, we applied a pragmatic trial design [23]. This study design evaluates the effects of an intervention under the usual everyday real-life conditions in which it is applied, by keeping aspects such as eligibility criteria of the included patient population, outcome measures, flexibility of intervention, or therapists’ expertise as close as possible to everyday life conditions [24]. Accordingly, we chose a 5-week period of RAGT, as this conforms to the duration and frequency of RAGT as it is usually provided in the outpatient setting of our center. We further chose to apply a crossover design, which enables each child to receive both treatments in a random order. As each patient can act as his or her control, and variation within a person is usually less than between patients, the treatment effect can be estimated more precisely. With that, it requires only about half the number of patients needed in a parallel group design [25]. As pediatric neurorehabilitation is generally faced with low case numbers, a reduction in sample size is a favorable factor in this research field. We, therefore, accepted possible disadvantages of the crossover design like carry-over or period effects and longer study duration for the participants and addressed these statistically.

Thus, this research project aimed to investigate the effectiveness of RAGT on improvements of functional gait parameters in ambulatory children with CP in a pragmatic, randomized crossover trial. We hypothesized that a 5-week period of RAGT on the Lokomat in an outpatient setting would be superior concerning the improvement in gait function when compared to a 5-week phase of standard treatment in children with a bilateral spastic CP within a crossover design with a randomized treatment sequence.

After a study duration of more than eight years (compared to planned four years), we decided to stop the trial mainly due to severe recruitment difficulties which we could not overcome despite several measures that were taken in the course of the trial. In this article, we report on the results of the 16 patients we included until study termination. Further, we address the lessons learned from our study, on the one hand from the content side regarding the Lokomat intervention, and on the other hand regarding methodological choices that potentially influenced study enrollment.

2.Methods

The study was designed in accordance with the Declaration of Helsinki [26] and the extensions of the CONSORT (Consolidated Standards of Reporting Trials) statement for non-pharmacologic treatment interventions [27] and pragmatic trials [28]. It was approved by the Ethics Committee Zurich and registered on ClinicalTrial.gov (NCT00887848).

Written informed consent and assent was obtained from all the legal guardians and the child by the study coordinator (C.A.) before participation.

2.3Randomization

Children were randomized to two different pre-specified sequences of interventions. The two interventions were RAGT as treatment (T) on the Lokomat on an outpatient basis and usual care as control (C) in the home environment of the child. A child could be randomized to a T/C sequence (TC-group) or a C/T/C sequence (CTC-group). Randomization into the two groups was performed using a minimization method with a random factor of 0.9. Minimization facilitates balancing even small groups in terms of selected patient factors at all stages of a trial. We included the factors severity of impairment (GMFCS-levels II and III/IV), age (6–11 years and 12–18 years) and Botulinum Toxin A-treatment in the preceding six months (present and absent). Concealed allocation was ensured by storing the independently generated randomization list off-site and requesting information about children’s group allocation by phone or e-mail.

Figure 1.

Overview of the outcome measures and the measurement time points per group. Children randomized to the CTC-group with the intervention sequence usual care/robot-assisted gait training (RAGT)/usual care were measured at four different time points within 16 weeks. Children randomized to the TC-group underwent the intervention sequence RAGT/usual care and were measured at three different time points within 11 weeks. Differences regarding intervention time and numbers of measurement time points were deliberately chosen to reduce the burden on children and families as much as possible.

2.5Outcome measures

2.5.1Primary outcome measure

We selected the percentage score of the dimension E of the GMFM-88 (GMFM E) as the primary outcome measure. The GMFM E assesses walking, running, and jumping [29] and consists of 24 items, which are scored on a 4-point ordinal scale. The test was performed without walking aids, regardless of the children’s GMFCS-level. A GMFM-certified therapist instructed and rated the child. The psychometric properties of the total GMFM-88 are well explored in children with CP [30]. Concerning the GMFM E, there is limited positive evidence for the responsiveness [31].

2.5.2Secondary outcome measures

We used the percentage score of the dimension D of the GMFM-88 (GMFM D) to assess the standing ability. The GMFM D consists of 13 items, which were tested and scored in the same way as described for GMFM E. There is also limited positive evidence for its responsiveness [31], while reliability seems to be high [32].

Further, a 6-minute walking test (6MinWT) was performed on a 30-meter long corridor with poles at each end [33]. Evidence for its test-retest reliability is moderate, the minimum detectable change (MDC) is 61.9, 64.0, and 47.4 m for children with a GMFCS level I, II, and III, respectively [34]. Its responsiveness has not been established in children with CP so far [31].

Gait speed was assessed with the 10-meter walking test at self-selected (10MWTss) and fast (10MWTfast) speeds. Children walked on a 14-meter long track with orthoses and walking aids usually used in daily life. Each test was performed twice and the time needed for the middle 10 meters was measured with a stopwatch. Evidence for test-retest reliability of the 10MWTfast is moderate in children with CP [31] and MDCs of 1.7, 4.3, and 17.7s were reported for GMFCS level I, II, and III, respectively [34].

Further, measures of body function (range of motion, modified Ashworth scale, and manual muscle tests of the lower extremities) were evaluated for descriptive purposes [35, 36]. All measurements were performed by trained physiotherapists who were experienced in working with children and who followed standardized written instructions. Assessors were not blinded regarding children’s assessment time point.

Figure 2.

Overview of the study protocol and the statistical analyses. CTC1, baseline assessment in CTC-group; TC1, baseline assessment in TC-group; CTC2, intermediate assessment in CTC-group; TC2, intermediate assessment in TC-group; CTC3, end assessment in CTC-group; TC3, end assessment in TC-group; CTC4, follow-up assessment in CTC-group; ΔC1, change during usual care in CTC-group; ΔC2, change during usual care in TC-group; ΔC3, change during follow-up in CTC-group; ΔT1, change during robot-assisted gait training in TC-group; ΔT2, change during robot-assisted gait training in CTC-group.

3.Results

Due to severe recruitment difficulties and the already prolonged study duration, the study coordinator and the sponsor (H.v.H.) unanimously decided to close the trial early. Until trial termination, 16 children and adolescents with a bilateral spastic CP completed the study, without any dropouts. Descriptive characteristics of the study participants are presented in Table 1.

Three children belonging to the TC-group had received Botulinum Toxin-A treatment into muscles of the lower extremities within six months before their study participation.

Regarding the ICF-CY body function level, the median modified Ashworth score of all children was 0 with an interquartile range (IQR) of 1. With a median of 3 (IQR: 1), children’s muscle strength was moderate. The sagittal range of motion values of the lower extremities are listed in Table 1.

Participants’ treatment adherence was high. Except for one child, who performed 10 RAGT sessions, all others completed at least 14 of 15 sessions. Descriptive parameters regarding dosage and intensity of the RAGT sessions are listed in Table 2.

Figure 3.

Individual effects of robot-assisted gait training (RAGT) and usual care on Gross Motor Function Measure dimension E (GMFM E) change scores. The order of the bars reflects the chronological sequence of the treatment. GMFCS, Gross Motor Function Classification Level.

In addition to the RAGT sessions, children attended on average 125 min ± 148 min (0–540 min) of physiotherapy during the T-intervention, and four children also continued with their hippotherapy sessions (41 min ± 85 min; 0–300 min). During the C-intervention, physiotherapy added up to 282 min ± 256 min (0–1’020 min) and seven children attended hippotherapy (63 min ± 82 min; 0–225 min).

4.Discussion

We investigated the effectiveness of RAGT on improving walking abilities in ambulatory children with bilateral spastic CP. For this, we used a crossover design with randomized treatment sequences. We stopped the trial early, i.e., after more than eight years, when only 16 of the intended 30 patients (34 including dropouts) finished the protocol, due to recruitment difficulties.

We hypothesized that a 5-week phase of RAGT would be superior regarding the improvement of walking abilities when compared to standard treatment with conventional physiotherapy for the same duration. The included participants covered a broad spectrum of our inclusion criteria regarding age, GMFCS-levels, and walking abilities. After about half of the intended number of patients finished the protocol, we could not confirm this hypothesis. While the lower power may have hampered statistically significant changes, the effect sizes of the primary and secondary outcome measures were not promising.

Existing evidence on the effectiveness of RAGT is weak and inconsistent [12]. Only four RCTs on RAGT with the Lokomat in children with CP have been published. Druzbicki et al. found no effects on kinematic and temporospatial gait parameters after four weeks with 20 sessions of RAGT in children with GMFCS levels II and III [19]. Peri et al. compared different protocols of gait training interventions with RAGT and task-oriented physiotherapy in 44 children with GMFCS level I–III [22]. While spatiotemporal and kinematic gait parameters did not change in any of the protocols, GMFM E scores improved significantly after 40 RAGT sessions over ten weeks, as well as after 40 physiotherapy sessions with standardized gait targeting exercises for the same period. Improvements (RAGT +3.5%, gait exercises +3.2%) exceeded the minimal important clinical difference in both groups. The GMFM D also improved considerably in both groups (RAGT +3%, gait exercises +5%), while the 6MinWT increased only in the RAGT group (+21 m). However, these results did not reach statistical significance, and no differences between the intervention protocols were present. Surprisingly, the protocols with mixed interventions (a combination of 20 RAGT and 20 conventional gait therapy sessions over 4 or 10 weeks) did not reveal any significant changes at all.

In contrast to the studies described above, the two RCTs conducted by Wallard et al. (both seem to include a similar sample of 30 children with spastic CP, GMFCS-level II) found significant improvements in diverse kinematic parameters on the lower and upper extremities as well as in GMFM D and E scores after 20 sessions of RAGT [20, 21]. However, these two studies prompt various questions regarding the randomization and concealment of the allocation sequence procedure, the statistical analyses, and the standardization of the therapeutic interventions. Further, the clinical relevance of significant improvements in kinematic parameters of only a few degrees is at least questionable. The lower-level design studies on RAGT in children, which did not include a control group, yielded inconsistent results regarding improvement in gait speed, gait endurance, and gross motor function [12].

The results of our study imply that RAGT on an outpatient basis showed no effects on the investigated walking abilities in ambulatory children with bilateral spastic CP. Besides the statistically insignificant results, the effect sizes, except those of the 6MinWT, did not indicate any positive results either. Taking our findings and experiences conducting the trial over the past years into consideration, we have several points to address. Some aspects refer to a conceptual level regarding developing insights into the contents of a rehabilitation process based on the approach of the ICF-CY. Other aspects allow more profound methodological insights into working with a pragmatic trial design in general as well as in this particular patient group and setting.

Half of the children in our study had previous RAGT experience and completed their second or third RAGT phase. Because of our limited sample size, we could not investigate whether this previous RAGT experience influenced the outcomes. However, Schroeder et al. found that repetition of RAGT negatively correlates with improvement in GMFM scores [40].

Further, evidence exists indicating that the development of gross motor function in children with CP may vary in countries with different health services systems [41, 42]. In Switzerland, as in many other western European countries and Canada, children with CP are embedded in a timely and constant rehabilitation regime. Thus, children in our study already received a relatively large amount of therapy before their study participation. Our study results could indicate that in such situations, RAGT is not intensive or active enough to promote further improvement in these children.

Concerning the RAGT intervention itself, we tried to follow a pragmatic approach as much as possible. As methods and principles of RAGT application and the accompanying software changed over the last eight years, we always aimed to apply the current state-of-the-art training strategies when performing RAGT. For example, we continually implemented the latest Lokomat software and serious games and adjusted the training parameters individually to promote active participation in each child. In regular practice, we do not opt for an isolated Lokomat training in our center but always combine it with physiotherapy and sports therapy lessons, focusing on the same individualized goals within a multidisciplinary rehabilitation approach. These goals are defined in advance together with the children and parents to cover the areas that are most relevant to each family and thus mostly concern the activity and participation component of the ICF-CY.

For the trial, we selected the GMFM E as our primary outcome, because the GMFM was considered the gold standard for the evaluation of gross motor function in children with CP. However, in retrospect, it was not the most optimal choice for several reasons. Children with a GMFCS-level III and IV will be able to score only a few points at most. A substantial change in this group of children is not to be expected as the test was performed barefoot and without walking aids (according to standard practice in our center). Furthermore, a pragmatic study approach should reflect the personalized rehabilitation goals as best as possible. Thus, an individualized measurement instrument should be used as a primary outcome. Such measures will ensure that intervention studies examine results relevant to both the child and family as well as the rehabilitation team. For this purpose, for example, the Goal Attainment Scale (GAS) or the Canadian Occupational Performance Measure (COPM) may be a promising option, as the measures quantify the achievement of goals in a standardized way, while these goals can be individually set for each child [43, 44, 45, 46]. Both tools could be a feasible option as outcome measures in pediatric rehabilitation research [47]: In a recent study, 69% of the children with CP who had performed a clinical RAGT intervention had achieved their goals formulated with the GAS [48]. The COPM was successfully applied as an outcome measure in the RAGT study of Schroeder et al., where patient-selected participation goals showed clinically meaningful improvements [17]. Furthermore, the COPM score improvements in this study – unlike the GMFM changes – were not correlated with age, GMFCS-level, or the number of previous RAGT periods.

Our training protocol prescribed three RAGT sessions per week throughout five weeks – a frequency and duration that our team had considered clinically adequate and feasible on an outpatient basis. A 4–5 week period of 15–20 RAGT sessions also conforms to what is usually financed in children with CP by the health insurances in Switzerland. However, Peri et al. had adopted 40 therapy sessions within ten weeks in their groups with significant and clinically meaningful improvements, while their group with a mixed protocol of 20 RAGT and 20 physiotherapy sessions over four weeks did not improve [22]. The four-week RAGT intervention of Druzbicki et al. did not result in a positive effect either [19]. The duration of the applied RAGT period could be a crucial factor for the potential of improvement in these children.

A crucial methodological lesson learned from this trial involves the recruitment of study children. Although we took various actions to increase the number of participating children (prolonged trial duration, broadened eligibility criteria, expansion from a mono- to a bi-center trial), we had to recognize that achieving the calculated sample size within the next one to three years would have been unattainable. Due to our intention to explore the effectiveness of RAGT as a single intervention compared to a usual care approach (when we started our study, it was standard practice to compare single therapy interventions with one another), we decided not to conduct this study in our regular inpatient setting. In our rehabilitation center, children stay in the inpatient setting for many weeks and attend various therapy sessions and school, as it is the only rehabilitation center specifically for children in Switzerland. Offering only RAGT over several weeks instead of a whole rehabilitation treatment package would not have been acceptable under these conditions, neither for the families nor for the treatment team. Although we also treat children on an outpatient basis in our center, this fact, in retrospect, turned out to be one of the reasons that we were only able to recruit half of the anticipated children. Most families preferred to arrange the RAGT period during their child’s school holidays, to limit the burden on the child as much as possible. The geographically decentralized location of our center also brought considerable travel expenses for many families. These issues reduced the number of participants from the anticipated ten children to maximally two to three per year. We expanded the mono- to a bi-center trial after five years, enlisting the support of the Pediatric Therapy Center of the Rehabilitation center in Rheinfelden, the only other center with a pediatric Lokomat in Switzerland. Unfortunately, despite the local study coordinator’s high motivation and effort, no single child could be recruited in this second center.

To avoid such problems, recruitment should focus as much as possible on the patient population that is referred to the center for the intervention under study, thus representing everyday clinical practice. In this way, logistical challenges, planning efforts, and resources of participating and executing parties can be kept within a better manageable and foreseeable range. Moreover, the research question can be answered within the context it originated from and not in a more artificial situation.

Unfortunately, discontinuation and nonpublication of RCTs conducted in children have a high prevalence, with difficulties in patient recruitment being the most commonly mentioned reason [49]. Nevertheless, it is also crucial to publish results of such a trial for several reasons. In addition to providing at least an indication regarding the effectiveness of the intervention studied, researchers can share valuable information about the problems they were faced with when designing and conducting their study and the lessons that have been learned [50].

In general, it should be considered that design choices towards a more pragmatic or explanatory study design may affect validity, generalizability, precision, and feasibility [51]. Currently, the PRECIS-2 tool, which was developed to support researchers in designing trials, would be a helpful instrument to make informed decisions about the trials’ position on the pragmatic-explanatory continuum [52].

We opted for a crossover design because it was regarded as the optimal study design for small patient groups when the study protocol was established. Recognizing the lessons mentioned above as well as the present disadvantages of this design (longer trial duration, potential carry-over and period effects, higher drop-out rates, sophisticated statistical analyses) [53], other study designs, as n-of-1 trials, single case methodology designs, or observational designs, could be considered as possible alternatives to the chosen crossover design.

5.Conclusions

The results of our study indicate that 15 sessions of RAGT within five weeks are not effective in improving walking abilities in ambulatory children with CP. Our study adds to the limited evidence of RAGT in children with CP. However, the results of the current study must always be interpreted acknowledging that we could only include half of the targeted number of participants. We applied RAGT as a single intervention compared to usual care, in line with the methodological point of view when designing the study. Our results must therefore also be interpreted in this light. Currently, clinical recommendations are to embed its application in a holistic treatment with other therapies [10].

An inpatient approach, offering the possibility of higher RAGT dosages as an add-on therapy element within an individualized holistic treatment package – as it is common practice in our center – may be a more promising, although still challenging strategy to investigate the effect on walking abilities in children with CP living in Switzerland. Further high-level design studies are on the way [54], and until the publication of their results, we have to monitor each child carefully during a RAGT intervention to determine whether benefits are present that justify the efforts and costs of this highly specialized treatment approach.