Stride Velocity 95th Centile: Insights into Gaining Regulatory Qualification of the First Wearable-Derived Digital Endpoint for use in Duchenne Muscular Dystrophy Trials

Ambulation parameters

ActiMyo and the new-generation model (Syde^®) use magneto-inertial sensors to quantify movement [15], and were specifically designed for real-world use in patients with movement disorders (Fig. 1) [16]. Although the EMA clarifies that other devices worn at the ankle may be used to capture SV95C [13], devices must meet the quality standards outlined in the EMA qualification to ensure that the data collected meet regulatory standards. Furthermore, changes to the sensor mechanism that may impact the clinical measurement properties would need to be supported by bridging data in a justified sensitive model [13]. ActiMyo is currently the only validated device suitable for capturing SV95C in clinical trials [17]. For these reasons, other devices are not discussed.

Fig. 1

Wearable magneto-inertial sensors: The third-generation ActiMyo (a and b) is succeeded by the new-generation device (Syde; c and d) which will be used in patients from 2021. Ambulant patients can wear a sensor strapped to each ankle, or one on the wrist and one on the ankle; sensors capture data throughout the day. The devices are transferred to a docking station overnight and the encrypted, anonymized data can be sent via a secure web cloud or stored on the internal USB drive [13]. The new-generation device (c and d) was redesigned with input from patients and healthcare professionals; it maintains the same level of sensor performance and measurement precision as its predecessor (a and b) but is one-third of the size and half the weight. The data recording station has also been modified to enable data capture for up to two weeks without internet connection or battery charging. Image a © PhotoRoom.

Both devices include a high-precision triaxial accelerometer, gyrometer, magnetometer, and a barometer recording data above 100 Hz, capturing linear acceleration, angular velocity, magnetic field of movement (in all directions), and barometric altitude [13]. The sensors were selected and individually factory-calibrated to compensate for repeatable errors (including thermal variations). Dedicated algorithms precisely and accurately capture the trajectory of the ankle, enabling high-fidelity 3D stride reconstruction, from which multiple stride parameters are derived (Fig. 2).

Fig. 2

Stride trajectory captured by ActiMyo. Computed trajectory of: a) two individual strides captured during ambulation; b) a circuit with stairs.

Three wearable-derived ambulation variables were evaluated (stride length, stride speed, and distance walked/recorded per hour) [18]. Variables were selected because of their clinical relevance in patients with DMD [19] and because they were robustly measurable in ambulant patients [18]. For each variable, different centiles were studied to optimize the longitudinal sensitivity to change. SV95C, reflecting the fastest strides taken, was selected for qualification based on measurement accuracy, reliability, sensitivity to change, and clinical relevance [18].

Unmet need

Timed-function tests and rating scales are predominantly used to assess motor function in NMDs, including the 6-minute walk test (6MWT), North Star Ambulatory Assessment (NSAA), and 4-stair climb [20]. In spinal muscular atrophy (SMA), the Hammersmith Functional Motor Scale Expanded and Motor Function Measure are commonly used [21, 22]. Performance on these types of measures may be influenced by patient motivation levels [1, 18], verbal encouragement during the test [23], and the learning effect [18, 23]. Inter-test variability is often high [2, 24] and subjective rating scales may be prone to rater bias [24, 25].

EMA guidelines recommend evaluating strength as well as motor function in clinical studies for muscular dystrophies [26]. Handgrip has been proposed as a feasible and reliable measure of strength in SMA [27] and DMD [28], particularly when expressed as a percentage of predicted values for age [29], but the clinical significance of strength change remains unclear [27–29].

As part of the qualification process for SV95C, it was important to clearly describe the limitations of currently used methods for assessing motor function to provide a clear rationale for seeking qualification. In the context of NMDs, the limitations of these study endpoints can lead to variability in study results and influence clinical development programs for new treatments. For example, in DMD, phase IIb and III studies of ataluren failed to show significant improvements versus placebo in the 6MWT [30, 31], but benefits were observed in subgroups of patients with a baseline 6-minute walking distance (6MWD) between 300 m and 400 m [31] and in patients with a baseline 6MWD of less than 350 m [30]. Similarly, eteplirsen and drisapersen showed similar mixed results in the 6MWT [32–35], although improvements in physical functioning were reported for all three treatments in long-term studies [36–39] and in patient and caregiver testimonials [39–41]. The lack of consistent efficacy data may account for discordance between the EMA and FDA; ataluren was conditionally approved by the EMA [42] but not by the FDA [43], while eteplirsen was conditionally approved by the FDA [33] but not by the EMA [44]. Drisapersen was discontinued owing to safety concerns [45]. Natural history studies have confirmed that the trajectory of longitudinal changes in the 6MWT differs depending on patient age, steroid use, and baseline 6MWD [46], although one prognostic model estimated that only 28% of variation in the 6MWT can be explained by these factors [47]. Crucially, episodic clinic-administered tests such as the 6MWT may not provide an accurate reflection of real-world ambulation ability [17].

NMDs are debilitating diseases –even small improvements in function could have a meaningful impact on the patient, but clinic-based tests may not be sensitive enough to detect small functional changes, particularly if compensatory muscle strategies are adopted in the early stages of the disease [48].

Wearables enable continuous and objective data capture from the home environment, offering a potential solution to problems associated with clinic-based testing including rater bias, performance variability due to patient motivation or verbal encouragement during the test, and the learning effect. However, identifying a wearable device with the analytical and technical validity to support regulatory decision-making was a key challenge in the development of SV95C.

Analytical and technical validity

The qualification process focuses on how the technology will provide clinically meaningful data, rather than the device technical specifications [49]. It is therefore important to consider early in the process what falls under the EMA remit. Device factors that could influence the reliability of the data, for example, the accuracy, precision, and clinical validity of the measure, are key considerations for qualification.

Multiple factors influence sensor accuracy, including the stability, linearity, calibration [50], noise characteristics [51], positioning, and orientation of sensor(s) [52], as well as the algorithms and methodologies used for deriving gait parameters [53, 54].

Inertial sensors (accelerometers, magnetometers, and gyroscopes) are increasingly used for ambulation assessment [52], but are reliant on machine-learning methods for activity recognition, which have variable accuracy [52]. Algorithms are typically tested in a controlled environment [55] and real-world activities may be wrongly interpreted (e.g., cycling as strides) [56]. Sensors do not directly measure position, but instead capture kinetic data (e.g., acceleration), and the calculated measurement values may ‘drift’ from the true values over time, reducing measurement accuracy [57]. Inertial sensors are prone to small errors in acceleration which are cumulative and result in 3D position drift over time [58]; magnetometers are susceptible to magnetic distortions, particularly indoors [59]; and ambient temperature fluctuations can cause gyroscope measurement drift [60].

Inertial measurement units often use fixed-threshold-based detectors to overcome drift, but these methods fail with atypical strides [56]. While SV95C may be captured by any ‘valid and suitable wearable device’ [13], device selection was driven by the need for validated measurement accuracy of the atypical strides characteristic of patients with NMDs.

ActiMyo stride detection is derived from the 3D trajectory of the ankle, providing a 98% stride detection success rate over 2000 atypical strides, including small or side steps [56]. The algorithms were based on an approach developed for military operations using sensors anchored to the shoe [61]. However, trajectory reconstruction is influenced by shoe rigidity and duration of contact between the foot and the ground (e.g., running vs walking) [61]. A novel patented dynamic realignment algorithm was developed for use with an ankle strap [61]. Ankle placement allows more precise trajectory reconstruction than shoe placement because stride impact is absorbed by the leg, increasing the signal to noise ratio and avoiding sensor saturation (and the resulting error accumulation) which can occur if a sensor measures a value that is larger than its dynamic range.

Movement in patients with neuromuscular diseases may be of a low amplitude, speed and acceleration –ActiMyo sensors are designed for long-term measurement stability with good signal to noise ratio, allowing accurate quantification of movement even at slow speeds [62]. Trajectory reliability and repeatability are achieved with a novel patented algorithm based on physical modeling paired with high-performance sensors. Unlike most activity trackers, ActiMyo sensors are calibrated using a rotation table and temperature-compensated to minimize sensor error and negate the need for individual a priori patient calibration [13]. Proper compensation of magnetic field variations, with multiple sequences of inertial sensor integration, permits high-fidelity 3D trajectory reconstruction and reduces magnetic field interference [13].

In healthy volunteers, ambulation parameters were captured with similar levels of accuracy to motion capture, with a peak-to-peak discrepancy rate below 2.5% [13].

Sensitive magneto-inertial sensors may capture multiple ambulation variables, but identifying the right parameters, as determined by the characteristics of the patient population, is vital for deriving meaningful motor function assessments.

Clinical meaningfulness

Step count is frequently used for real-world activity monitoring in NMDs [63]. Step count is easily captured using a pedometer, but measurement accuracy is compromised in patients with walking difficulties and at slow walking speeds [64].

Of the real-world ambulation parameters assessed, SV95C was the most sensitive indicator of clinical change in patients with DMD. After 6 months, in patients aged 7 years and above with baseline 6MWT below 450 m, stride length 95th centile declined by 2.4% (p < 0.05), median stride velocity by 4.7% (p < 0.05), and SV95C by 8.5% (p < 0.01) [13]. In contrast, stride length increases with age and height in healthy volunteers, while SV95C remains stable over a one-year period [65]. During the qualification process, the EMA also requested evidence showing that SV95C is sensitive to positive improvements in function. This was a key point of contention because it is particularly challenging to measure positive functional changes in a disease for which there is no effective disease-modifying treatment available. Similarly, any improvement observed in a clinical trial setting would be invalid unless the experimental treatment has since been shown to be effective.

Corticosteroid treatment is associated with short-term improvements in motor function, followed by a phase of gradual decline in DMD [66]. We have subsequently shown that SV95C is sensitive to the positive and negative functional changes associated with steroid treatment; in steroid-naïve patients, steroid initiation resulted in an 11.6% increase in SV95C after 6 months, while age-matched patients receiving stable steroid treatment experienced a 6.8% decline [67].

To demonstrate clinical meaningfulness, results must correlate to some degree with previously validated measures. All variables had moderate but significant correlations with the 6MWT and NSAA [13] (Table 1). However, a limiting factor in the qualification process was the use of the 6MWT as the gold standard reference point. It is inherently difficult to prove that a change in SV95C that is not detectable with the 6MWT is anything more than signal noise when there are no other, more sensitive measures available to validate the findings.

A robust assessment of the minimal clinically important difference (MCID) ensures clinical studies are adequately powered to demonstrate meaningful benefits [68]. The MCID for SV95C was calculated using the distribution-based methods described by McDonald et al. [20] as 0.1 m/s (a relative 6.24% decline), corresponding to a 36 m difference in 6MWT [13] (30 m is the accepted 6MWT MCID in DMD trials [20]). Studies are ongoing to confirm the validity of the MCID [13].

Large sample sizes are required for studies employing the 6MWT as a primary endpoint. In the phase III ataluren study for DMD, an estimated 105 patients per treatment arm were needed to show a 30 m difference in 6MWT at 48 weeks with 85% power [31]. For SV95C, 14 patients per arm with DMD aged 7 years or above (with 6MWT baseline < 450 m) provides sufficient statistical power (80%) to demonstrate significant stabilization of motor function with treatment at 6 months (vs natural history) [13] indicating the improved sensitivity and reliability of SV95C versus the 6MWT. NMDs are typically rare diseases [69] and patient enrollment for studies is challenging [70]; therefore, the ability to conduct smaller, shorter clinical studies using SV95C confers a significant advantage over traditional endpoints.

Usability and compliance

Compliance is an important consideration for wearables. Data collection may be compromised if the device is difficult to use or uncomfortable to wear, or if patients avoid wearing it in public or on certain days (e.g., weekends). In patients with NMDs, muscle weakness might preclude compliance if the device is heavy. Position on the body can also influence compliance with wearables in children [71]. The ActiMyo ankle strap enables the user to change footwear during the day and conceal the device under clothing. The newer device is smaller and lighter than the previous model (Fig. 1), which could improve future compliance. The risks of poor compliance are mitigated through patient, caregiver, and investigator training, and the provision of clear instructions (using videos and instruction manuals).

Wearable device usage can decline over time [72]. Therefore, patient compliance was a key consideration in determining the duration of recorded data for calculating SV95C. A total of 180 hours was optimal, with good compliance and low measurement variability; 90% of patients recorded 180 hours over 1 month [13]. Mean SV95C variability was 6.38% at 50 hours of recording (standard deviation [SD], 2.60%); this reduced to 4.41% (SD, 2.33%) at 180 hours [13]. Data-processing software flags a warning if step count is abnormally low or sensors fail [13]. Data are deemed missing if under 50 hours are recorded over 1 month [13]. Evaluating the possible causes of measurement variability and mathematically defining the optimum recording period were contributing factors for success in the qualification process for SV95C.

Data privacy

Regulation of ownership and responsibility for wearable-generated data are lacking [73], and wearables may be vulnerable to security breaches if security is not prioritized in development. Wearer concerns regarding data privacy could also reduce compliance. A risk analysis was conducted to identify, address, and mitigate risks relating to data integrity, availability, and confidentiality [13]. The EMA deemed the risk level acceptable for several reasons: ActiMyo records motion, not global positioning [13]; no personal information is associated with data capture [13]; motion data are stored in a binary format interpretable only with extraction software and communication channels are encrypted [13]; and data are identified using a unique patient identification code linked to the patient’s personal information at the clinical center and accessible only to the researcher [13].

Impact of SV95C on DMD clinical studies

SV95C is the first regulatory-approved, digital COA qualified for use as a secondary endpoint in pivotal DMD studies. This landmark qualification enables researchers to continuously track real-world patient ambulation in clinical studies, potentially transforming the development of new treatments in an area of significant unmet need.

As highlighted by the EMA, SV95C has potential as a primary endpoint in DMD trials with further supportive data [13]. Strengthening the correlation between SV95C and functional tests with longer follow-up and additional patients, expanding normative data, and further justification of the MCID are recommended [13]. Research is ongoing to support primary endpoint qualification.

As a primary endpoint, the sensitivity of SV95C should allow shorter studies enrolling fewer patients, reduce patient exposure to experimental treatments, and lower patient burden by enabling less frequent, shorter clinic visits than traditional measures. Ultimately, this work will help bring effective treatments to patients with DMD, and potentially other NMDs, sooner than previously possible. Confounding factors and the associated uncertainty of traditional study endpoints have hindered development of DMD treatments; SV95C overcomes these limitations, providing a robust rationale for adoption.

Broader applications of wearable magneto-inertial sensors

Magneto-inertial sensors offer the opportunity to provide a gold-standard measure of disease progression for use in clinical development and clinical practice, allowing continuous movement capture and assessment of natural functional ability under free-living conditions in the real world. Sensors are being used to evaluate other important functional outcomes, including falls, stair-climbing ability, and other ambulation parameters [13], and could provide insights into the quality of movement as well as the quantity, potentially offering more granularity than traditional motor function assessments. Indeed, in the final qualification opinion, the EMA strongly encouraged further research into other outcomes, including quality of walking [13]. Several variables for upper-limb function have been identified using sensors strapped to the wrist, which could expand use to non-ambulant patients [62].

Continuous capture and quantification of movement could yield insights into many diseases. Work is ongoing to identify disease-specific measures in spinal muscular atrophy [21, 22], facioscapulohumeral muscular dystrophy [74], centronuclear myopathy [75], and amyotrophic lateral sclerosis. However, an integrated value of physical activity, combining lower- and upper-limb movements in patients with NMDs remains to be validated.

Beyond the neuromuscular field, ActiMyo is currently being evaluated in Angelman syndrome, Parkinson’s disease, and multiple sclerosis. The ability to quantify physical activity has broad application in a range of contexts. The 6MWT is a prognostic indicator in heart failure [76] and chronic obstructive pulmonary disorder [77]. Physical activity is an indicator of quality of life in elderly patients [78], patients with cancer receiving chemotherapy [79], and patients with eye diseases such as glaucoma [80]. Continuous activity tracking is also valuable in patients recovering from bone fractures [81].

Digital tools could improve clinical research and patient outcomes

The accuracy of the data derived from commercial- or consumer-grade devices may vary depending on the device used [82], the placement of the device [83], and the mobility of the wearer [84]. These concerns regarding accuracy and the lack of regulatory support have limited the incorporation of wearables into clinical trials [85, 86].

However, the development of clinically validated, high-performance wearable technologies provides an exceptional opportunity to improve the efficiency of clinical and real-world studies and could yield insights into disease progression to inform clinical practice in the future. Other promising efforts are ongoing, for example, the MOBILISE-D initiative aims to implement and validate a panel of regulatory-grade, real-world digital mobility outcomes for patients with mobility difficulties [87]. Although a global mobility assessment would have broader utility than SV95C, developing a validated measure across multiple diseases presents complex challenges. Disease course and clinical characteristics differ considerably across different patient populations [88] and the current EMA qualification process may need to evolve to accommodate a measure developed across multiple diseases using a common approach.

Another promising effort is the KineDMD study, which is using high-resolution wearable sensors to track full-skeleton movements and derive a complete picture of full-body behavioral capacity over a 12-month period in patients with DMD [89]. The data collected will be used to develop novel clinical endpoints in the future [90].

Beyond the assessment of motor function, wearables can now quantify a range of different clinical parameters, from vital signs [91] to sleep [92], which could yield deeper insights into many other diseases in the future.

Development of these types of digital COAs could not only permit shorter, smaller studies, but also enable earlier go/no-go decisions on experimental treatments to reduce the risk of late-stage study failures and bring effective treatments to patients faster and more affordably than previously possible.

However, the process of developing and validating a digital endpoint is by no means easy or quick. We strongly advocate scientific rigor and comprehensive analytical, technical, and clinical validation of digital technologies used in clinical studies. This requires significant investment and resource, and is dependent upon successful collaborations between data scientists, device and algorithm engineers, clinical drug development teams, and regulators.

The qualification process itself is lengthy, with a duration of at least 160 days for qualification advice and 250 days for the qualification opinion [93]. The EMA encourages applicants to engage with them early in the process and to seek qualification from the FDA in parallel [94, 95], enabling support of global clinical development programs. It is worth noting that there are fundamental differences between the EMA and FDA programs; the EMA is primarily focused on the metric properties of the outcome, whereas the FDA is primarily interested in the clinical validity.

As the first successful applicants to complete the EMA process, we did encounter some initial difficulties. This is unsurprising given how rapidly technology has evolved over the last few years. The EMA is continuing to evolve their process and has since published new guidance to support applicants in the qualification process [49].

We hope our work provides a rationale for increased investment in the clinical validation of wearables. While the returns may not be immediate, in the future, the volume and breadth of data captured could generate actionable insights to improve patient outcomes across the care continuum, for example, enabling the discovery of novel diagnostic, predictive, and prognostic biomarkers (Fig. 3). Wearables could enable patients living in remote areas to participate in clinical studies, and could reveal insights into the real-world patient experience, enabling regulators, health technology assessment bodies, and payers to make informed decisions on the safety, efficacy, and value of new medicines. In the advent of genetic therapies for inherited NMDs, digital tools will be invaluable for assessing the durability of single-dose treatments. For antisense therapies requiring repeat dosing, digital tools could enable early detection of non-responders to minimize patient exposure to costly ineffective treatments. For patients with NMDs, use of digital tools could provide a higher standard of care, access to value-based treatments with fewer clinic visits, and less need for onerous and repetitive performance assessments.

Fig. 3

Potential impact of digital data on patient care in the future. Data collected from digital tools will enable the generation of actionable insights that could improve patient outcomes across the care continuum.