Post-hoc Nonparametric Analysis of Forced Vital Capacity in the COMET Trial Demonstrates Superiority of Avalglucosidase Alfa vs Alglucosidase Alfa

INTRODUCTION

Pompe disease is a rare, progressive neuromuscular disorder in which deficiency of lysosomal acid α-glucosidase (GAA), an enzyme that breaks down glycogen, leads to glycogen accumulation in the lysosomes and progressive muscle weakness [1, 2]. Patients with late-onset Pompe disease (LOPD) present with symptoms anytime from childhood to late adulthood with varying degrees of skeletal muscle dysfunction that result in mobility and respiratory problems, often leading to the need for respiratory and ambulatory support [1, 2].

Since 2006, enzyme-replacement therapy (ERT) with alglucosidase alfa (Myozyme/Lumizyme, Sanofi, Cambridge, MA, USA) has been the standard of care for all forms of Pompe disease. Among patients with LOPD, however, variable treatment response and disease progression are observable in some patients receiving ERT, including worsening of clinical symptoms after several years of treatment [3]. Avalglucosidase alfa (Nexviazyme/Nexviadyme, Sanofi, Cambridge, MA, USA) [4] is a recombinant human enzyme replacement therapy designed specifically for enhanced mannose-6-phosphate (M6P)-receptor targeting and enzyme uptake in the cells of target tissues, with the aim of increasing the clinical efficacy achieved with alglucosidase alfa [5]. Avalglucosidase alfa has approximately 15-fold greater M6P content compared with alglucosidase alfa [6, 7].

In the Phase 3, randomized, double-blind COMET trial (NCT02782741) of 100 participants with LOPD at 55 study sites in 20 countries, greater improvement in upright forced vital capacity (FVC) % predicted was observed with avalglucosidase alfa (AVA) compared with alglucosidase alfa (ALGLU) [5]. While statistical non-inferiority of AVA vs ALGLU for improvement in upright FVC % predicted was achieved, statistical superiority was narrowly missed (P = 0.063; 95% CI: –0.13–4.99). One participant in the AVA treatment arm had severe chronic obstructive pulmonary disease (COPD) and extremely low upright FVC % predicted at baseline, which declined further throughout the trial.

In rare disease clinical trials, sample size is often limited, and participant heterogeneity may give rise to unique confounding issues. Thus, the effect of an extreme outlier on overall study outcomes can be considerable, and additional statistical methods that account for outlier effects may enhance interpretation of study results. We examined the statistical impact of outlying FVC values from the participant with COPD in the COMET trial. Our findings illustrate the unique confounding influence of patient heterogeneity in rare disease clinical research.

METHODS & RESULTS

The COMET study protocol was reviewed and approved by appropriate ethics committees or institutional review boards and done in accordance with the Declaration of Helsinki and the International Council for Harmonisation guidelines for Good Clinical Practice. Written informed consent was obtained from participants before any study-related procedures [5].

The pre-specified analysis plan for the COMET trial called for a mixed model for repeated measures (MMRM) analysis of efficacy endpoints reporting least-squares (LS) mean and standard error (SE). An intention-to-treat approach was used to include data from all participants to address the issue of missing data (assuming data are missing at random). This predefined MMRM estimated a treatment difference for upright FVC% predicted (primary outcome measure) of 2.43% (95% CI –0.13–4.99) favoring AVA and demonstrating statistical non-inferiority of AVA vs ALGU (P = 0.0074), but narrowly missing statistical superiority (P = 0.063) (Table 1). This treatment difference can be regarded as clinically meaningful based on a minimal clinical difference of 2–6% that has been established for a restrictive lung disease of different etiology [8] and used in other LOPD studies [9, 10].

When the observed changes from baseline to Week 49 in upright FVC% predicted are represented as a cumulative probability function, a clear right shift of the AVA curve compared with the ALGLU curve shows systematically greater benefit of AVA regardless the level of change from baseline (Fig. 1A). Approximately 70% of AVA participants showed improvement of upright FVC % predicted (i.e., had a positive absolute change from baseline) compared with 47% of ALGLU participants. Since gradual decline in respiratory function is part of the natural progression of LOPD [11], any improvement of respiratory function can be considered a positive response to the therapy.

Fig. 1

Panel A: Cumulative distribution function of upright FVC% predicted change from baseline at Week 49 in all COMET trial participants. Panel B: Longitudinal values from pulmonary function tests in a single study participant with known comorbidity of chronic obstructive pulmonary disease and marked reduction of both forced vital capacity (FVC) and forced expiratory volume in the first second (FEV₁).

Figure 1A also shows that one participant treated with AVA had an extremely large decline in upright FVC% predicted from the baseline value (depicted at the extreme left of the blue curve) compared with the other study participants. The change in FVC in this participant was –24.12% predicted as compared with a mean change (SD) of +3.02% (6.83%) predicted in the AVA-treated group, corresponding to a large Z-score of -3.97. Analysis of data from the ALGLU-treated group yielded a maximal Z-score that was much lower at 2.42. This outlier participant had a history of COPD and asthma, with upright FVC of 46.5% predicted at baseline and then declining continuously during the trial (Figure 1B). In particular, this participant also had severe limitation of maximal expiratory airflow, consistent with presence of Global Initiative for Chronic Obstructive Lung Disease (GOLD) stage III COPD [12, 13], with continuous worsening over time based on serial measurements of FEV₁ and FEV₁/FVC during the trial. This patient did not experience any serious adverse events or adverse events of special interest.

When the outlier participant was excluded from the MMRM analysis, the estimated treatment difference between groups changed from 2.43% to 2.98% (95% CI: 0.65, 5.30) predicted with a statistically significant P value of 0.013 (Table 1). In addition, a non-parametric Wald-type rank test was performed to enable analysis of all available data based on the rank repeated measures analysis of covariance model using SAS codes based on Fan and Zhang’s codes [14]. This nonparametric test achieved a statistically significant P value of 0.019 (Table 1) using all data, including the outlier participant.

DISCUSSION

These additional analyses of data from the COMET study illustrate the effect that an extreme outlier may have on outcome data in rare disease studies, when only a small population of participants can be studied. Although the pre-specified MMRM failed to demonstrate statistical superiority of AVA vs ALGLU with respect to change in respiratory function, superiority of AVA vs ALGLU was graphically evident from the cumulative distribution function of change in upright FVC % predicted from baseline to end of study at Week 49. A clear separation of treatment effect was noted throughout the entire range of change in respiratory function, with the exception of the single outlier participant with COPD who was included in the AVA group.

The data presented here extend the observations in the COMET study by demonstrating a clinically significant improvement of FVC on re-analysis of the trial data using two methods: (1) repeat of the primary MMRM analysis after removing the outlier participant; and (2) non-parametric analysis using all data, including the outlier, equivalent to the approach that was used in the primary MMRM analysis. These analyses confirm that the pre-specified MMRM estimate of the mean treatment effect during AVA therapy may have been disproportionately affected by the presence of a single extreme outlier. It should be noted that, while the outlier value in the COMET study precluded the detection of a statistically significant treatment benefit, it is theoretically possible that outliers may imply a statistically significant effect of a therapy that is of limited or no benefit.

Four additional participants in the COMET trial reported a prior history of obstructive lung disease (e. g. asthma and/or COPD). In three of these participants, analysis of individual spirometry data did not support the reported diagnosis as objective evidence of airflow obstruction was absent at baseline, and only minimal change in airflow was demonstrated during subsequent visits. In the single remaining participant, baseline spirometry showed borderline airflow limitation. However, airflow remained stable during the study with changes that are within the known error of measurement. Collectively, these data suggest that these additional participants with reported history of either asthma and/or COPD are unlikely to have influenced the observed changes in FVC during the COMET study.

The data reported here have direct implications for the design of clinical trials in rare diseases where clinical manifestations are heterogeneous and sample sizes may be small. For this reason, inclusion criteria in rare disease trials are often relatively broad in order to facilitate recruitment. However, given the small sample size characteristic of rare diseases, the study design should prevent inclusion of participants with severe manifestations of confounding disorders that may directly affect the main outcomes of the trial (e.g., COPD in a study where respiratory function is the primary endpoint, as in the COMET study). In addition, the statistical analysis plan should include efforts to detect outlier values coupled with options for alternate analytic techniques (e.g., non-parametric tests) that are less affected by outlier values.

In summary, the superiority of AVA vs ALGLU for improving upright FVC % predicted in patients with LOPD was demonstrated in two post-hoc analyses to address the impact of an extreme outlier. The impact of outlier values can be substantial in rare disease trials, and additional analyses may be necessary to fully capture the study results. We consider it both prudent and acceptable to account for foreseeable, often comorbidity-related issues when pre-specifying study methodology. In this sense, exclusion of patients with severe COPD should be considered for future trials in LOPD.

ARTICLE INFORMATION