Measurement properties of patient-reported outcome measures used in rehabilitation of adults with chronic musculoskeletal pain: A mapping review

1.Introduction

Rehabilitation in the field of chronic musculoskeletal pain (CMP) is based theoretically on the biopsychosocial model [1]. The main aim is to enable the patient to deal better with pain and pain-related disabilities in order to improve daily functioning and participation. The selection of specific treatment modules is preceded by the assessment of relevant factors that maintain chronic pain and associated disabilities. Subsequently, when conducting a rehabilitation programme, proper measurement of factors maintaining pain and disability considered relevant, as well as of outcomes, is extremely important [1]. The purposes of such measurements can be diagnostic, including setting treatment goals (focusing on problems that need to be targeted to achieve the patient’s treatment goals), prognostic (predicting outcomes), or evaluative (evaluation of treatment goals) [2]. The measurement tools used could be a part of history taking, physical examination, neurophysiological testing, or imaging, and could include biomarker measurements, performance tests, or a wide array of patient-reported outcome measures (PROMs). PROMs are identified through self-completed questionnaires, with the goal of enabling patients to rate their own perceived physical, psychological and/or social functioning, participation, and/or quality of life [1, 3]. The use of PROMs, as part of the overall assessment, continues to expand beyond clinical research, in recognition of its potential for daily care by supporting patient-centred approaches and contributing to shared decision-making.

Choosing from the large variety of available PROMs, identifying the most appropriate measure(s) for predefined diagnostic, prognostic or, most relevant, evaluative purposes within the population being treated, is a real challenge for clinical practice [4]. In the field of pain rehabilitation, several attempts and initiatives have been described for developing a minimum set of tools. This set should be used in clinical practice to support informed clinical decision-making, for prognosis, to monitor treatment progress, to evaluate outcomes, and also to facilitate the comparison and pooling of data for scientific purposes. The Dutch Dataset Pain Rehabilitation (DDPR), in addition to others like the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) and Outcome Measures in Rheumatoid Arthritis Clinical Trial (OMERACT), is an example of such a set and has been chosen as the core outcome set of PROMs for clinical practices providing interdisciplinary multimodal pain programmes in the Netherlands [1]. The set has been implemented in part of Dutch chronic pain rehabilitation facilities but there are, however, important drawbacks to this implementation, such as the lack of a systematic overview of the research literature on the measurement properties of the tools. This also applies to PROMs that health care professionals use in addition to the DDPR. An important first step to aid further research and eventually for adopting tools in daily practice is an overview of which measurement properties have already been studied, and which have not.

The Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) advocates a scientifically sound taxonomy, which groups measurement properties for health measurement tools, including PROMs, into the domains of reliability, validity, and responsiveness. The COSMIN taxonomy is a recommended resource in the identification of knowledge gaps in clinimetric evaluations of PROMs [5]. An additional characteristic is the interpretability of the outcome scores themselves. Within the taxonomy of the COSMIN, interpretability is considered the assignment of clinical or commonly understood connotations to a measure’s quantitative collected outcome scores or change in scores [2]. As such, interpretability cannot be considered a measurement property like reliability, validity, and responsiveness because it does not refer to the clinimetric quality of a measure itself. However, interpretability stands for the translation from quantitative scores into a clinically meaningful message to the patient and caregiver, which clearly demonstrates its great importance for clinical practice [2]. Unfortunately, interpretability often receives little attention in science or clinical practice and, therefore, may be an insufficiently studied aspect of the measurement tools, along with measurement properties [2].

As indicated above, a mapping review is a significant first step in increasing the evidence about and application of measurement tools. The goal of a mapping review is to identify and categorize existing literature to define research priorities and make informed decisions for systematic reviews as well as primary studies. The research field is described through a systematic search in a broad field and presentation of the results in a user-friendly format, often a visual figure or graph [6, 7, 8]. The scope for this mapping review is defined as follows:

2.Materials and methods

A mapping review was conducted to provide a broad overview of existing studies on measurement properties for commonly used PROMs in clinical pain rehabilitation programmes in the Netherlands.

3.Results

A total of 24,476 records were identified, of which 24,402 were obtained through database searching and 74 through other sources (reference tracking and reviewers’ personal libraries). After deduplication, 13,957 records were screened based on title and abstract (Fig. 1). Consequently, 106 full-text records were assessed for eligibility, of which 40 were included in the quantitative synthesis. Thirty-six full-text publications and four (conference) abstracts were distinguished. Finally, 35 unique studies were identified for inclusion.

Figure 1.

The flowchart representing the identification, screening, and eligibility process.

The study populations of the majority of studies consisted of groups of patients with different diagnoses: chronic primary musculoskeletal pain (e.g., low back, neck, shoulder, and knee pain), chronic widespread pain (e.g., fibromyalgia), chronic post-traumatic pain (e.g., whiplash injury), and/or chronic secondary musculoskeletal pain (e.g., arthritis). In addition, studies were conducted in different settings (hospitals, rehabilitation centres, pain centres, primary care, or national surveys), or included patients from several settings. The mean reported duration of pain varied from 3 to 162 months. The average age of all populations was 46.5 years (SD 6.4, range 32.1–68.6).

Results for measurement properties were found for the HADS, PCS, PDI, PSC, PIPS, PSEQ (including a short form PSEQ-2 item version), SF-12 (including a derivative SF6D12), and SCL-90 (including SCL-90-R). No studies, in chronic pain populations, were found for the CIS, IPQ, and UCL.

Thirty-one studies included PROMs in the language of original development (English (n= 27), Dutch (n= 2), Swedish (n= 2)). In addition, one study evaluated a PROM (PDI) in both the original (English) and translated versions (Dutch), and eight studies investigated PROMs translated into Dutch. One included article reported the results of a Delphi study for consensus on the most appropriate PROM to assess psychosocial risk factors (PSEQ and PCS) in patients with chronic pain. These results were regarded as an outcome for the domain of validity (content validity). Four articles included results for two PROMs: HADS/PDI, PCS/PSEQ, SF-12/PDI, and SF-12/SCL-90.

The numbers of included studies for each PROM in the original version and translated version (Dutch) are presented in Table 2. The study characteristics of the included records for each PROM are presented in Supplement 2.

3.2Overview of measurement properties of translated PROMs

Of the PROMs translated into Dutch, cross-cultural validity was only reported on for the HADS and PDI. Most measurement properties were studied for the Dutch PDI, with the exception of face, criterion and structural validities. For the PCS, no studies were found for the domain reliability, and in the domain of validity, only structural validity was examined. In the domain reliability, internal consistency was tested for HADS, PIPS, and PSEQ, with reliability as well for the PSEQ. In the domain validity, structural validity was reported for all three and hypotheses testing for the PSEQ and PIPS. Only interpretability was studied for HADS, but

Table 3

Investigated measurement properties presented in included studies for each PROM

	PROM	Author	Year	Reliability	Internal consistancy	Measurement error	Content validity	Criterion validity	Structural validity	Hypotheses testing	Cross cultural validaty	Responsiveness	Interpretability	Other
Dutch	HADS	Giusti [32]	2020		√				√CFA		√		√	√EFA
language	PCS	van Damme [33]	2000										√RV
PROMs	PCS	van Damme [34]	2002						√CFA
	PCS	Pulles [35]	2020									√	√
	PDI	Soer [36, 37]	2011,									√	√
			2012
	PDI	Soer [17]	2013	√	√	√				√	√
	PDI	Soer [38]	2015										√
	PSC*	Beurskens [39]	1996									√
	PSC	Beurskens [18]	1999									√		√FE
	PIPS	Trompetter [21]	2014		√				√CFA	√				√EFA∗
	PSEQ	van der Maas [25]	2012	√	√				√CFA	√				√EFA∗
Non-dutch	HADS	Pallant [40]	2005		√				√CFA					√EFA
language	HADS	Rusu [41]	2016					√CV
PROMs	PCS	Osman [42]	2000		√			√CV	√CFA	√			√RV
	PCS	George [43]	2010	√		√		√CV∗				√
	PCS	Prime [44]	2012							√∗
	PCS	Sleijser [45]	2019				√
	PDI	Pollard [16]	1984							√
	PDI	Tait [46]	1987		√				√∗CFA
	PDI	Jerome [47]	1991							√
	PDI	Millard [48]	1991							√
	PDI	Strong [49]	1994		√			√CV		√
	PDI	Crighton [50]	2014					√CV		√
	PDI	Soer [38]	2015										√RV
	PDI	Morris [51]	2015									√
	PDI	Rusu [41]	2016					√CV
	PDI	McKillop [52]	2018							√
	PIPS	Wicksell [20]	2008		√			√CV		√				√EFA∗
	PIPS	Wicksell [53]	2010		√			√CV	√CFA	√				√EFA∗
	PSEQ	Nicholas [24]	2007	√	√					√
	PSEQ	Maughan [54]	2010								√
	PSEQ	Nicholas∗∗ [55]	2015	√	√				√CFA	√		√		√EFA
	PSEQ	Costa [56]	2017	√
	PSEQ	Sleijser [45]	2019				√
	SF-12	Luo∗∗ [57]	2012		√					√		√	√∗
	SF-12	Morris [51]	2015									√
	SF-12	Tawiah [58]	2018							√
	SF-12	Tawiah [59]	2019							√
	SF-12	Kroenke∗∗ [60]	2019							√		√
	SCL-90-R	Kinney [61]	1991							√
	SCL-90-R	Bernstein [62]	1994						√CFA	√
	SCL-90-R	Peebles [63]	2001							√
	SCL-90	Kroenke∗∗ [64]	2019							√		√

*Interpretation of the reviewers based on ambiguous information. Abbreviations: HADS: Hospital Anxiety and Depression Scale; PCS: Pain Catastrophizing Scale; PDI: Pain Disability Index; PIPS: Psychological Inflexibility in Pain Scale; PSEQ: Pain Self-Efficacy Questionnaire; PSC: Patient Specific Complaints; SCL-90: Symptom Checklist-90; SF-12: 12-Item Short Form Health Survey; CV: Concurrent Validity; RV: reference values; FE: feasibility; CFA: Confirmatory Factor Analysis; EFA: Explorative Factor Analysis. ∗∗study of derivative or subscale of original PROM.

Figure 2.

Data on measurement properties mapped in ascending order of the number of studies presented for each PROM (PROMs in bold and italicized = Dutch, bold = English, normal font = Swedish). The shapes of the COSMIN domains are according to the taxonomy developed by COSMIN [5]. Red indicates no studies, orange one study, and green two or more studies, blank does not apply (cross-cultural validation). Abbreviations: HADS: Hospital Anxiety and Depression Scale; PCS: Pain Catastrophizing Scale; PDI: Pain Disability Index; PSC: Patient Specific Complaints; CIS: Checklist Individual Strength; PIPS: Psychological Inflexibility in Pain Scale; IPQ: Illness Perception Questionnaire; PSEQ: Pain Self-Efficacy Questionnaire; SF-12: 12-Item Short Form Health Survey; SCL-90: Symptom Checklist-90; UCL: Utrecht Coping List.

not responsiveness; for PIPS and PSEQ, responsiveness and interpretability were not studied. No studies on the Dutch SF-12 and SCL-90 were found in populations of patients with chronic pain.

4.Discussion

The objective of this study was to map out the literature available on the reliability, validity, responsiveness, and interpretability of PROMs commonly used in Dutch pain rehabilitation, including measurement tools proposed by the DDPR. The results show many knowledge gaps, although more studies were found for the PROMs in their original languages than for translated versions. Information has been published for all domains, but not all measurement properties, of both the original and Dutch version of the PDI and the original versions of the PCS and SF-12. For the original Dutch CIS, IPQ, and UCL, as well as for the Dutch translations of the SF-12 and SCL-90, no information was found on any measurement property. Overall, for the other tools, aspects of validity were most frequently reported, followed by reliability, responsiveness, and interpretability.

The results of this study complement a previous evidence review for a population with CMP evaluating measurement properties of PROMs for pain severity and pain-related functional impairment not included in the current review. For only three of the 13 multi-item tools (Oswestry Disability Index (ODI); Roland-Morris Disability Questionnaire (RMDQ); SF-36 Bodily Pain Scale (SF-36 BPS)), were data found for all COSMIN domains [65]. Interestingly, the PROMs investigated in the present study had already been developed in the ’70s, ’80s, or ’90s, apart from the PSEQ (2007) and PIPS (2008). Since their development, PROMs have been widely used both clinically and in research, and are nowadays recommended as core outcome measures by the DDPR. The limited number of clinimetric studies performed in the CMP population in recent decades, despite the prominent role of these questionnaires, clearly points out a shortcoming in prior and current practices. At the same time, the paucity of research found by this review suggests that both researchers and funders have insufficiently acknowledged the importance of clinimetric research. An additional illustration of this point is the fact that a selective group of researchers is responsible for the majority of studies concerning specific PROMs, such as the PDI, which is unfavourable considering research biases.

The finding that responsiveness has received less attention than reliability and validity may be explained by the challenges in executing and interpreting research on this, including the necessity of multiple repeated measurements, as well as the fact that not all PROMs have an evaluative purpose. Interpretability, of high importance as it gives clinical meaning to (change in) scores, has been unjustifiably ignored. The importance of interpretation of scores was underlined during the implementation of the DDPR, as the lack of information on interpretability was a frequently mentioned reason for the PROMs not being feasible or relevant to use in practice [66]. It is also striking that only a single study reported on the content validity of a measure, while COSMIN considers it the most important measurement property. This is because the content of a measurement instrument should be relevant, comprehensive, and comprehensible with respect to the construct of interest and to the target population [5].

A major strength of this review is that it was performed completely within the theoretical framework of the COSMIN taxonomy, which is based on international consensus [5]. However, as the extraction of measurement properties was determined by the authors’ reporting instead of reviewer assessment, the allocation may not entirely be in accordance with the taxonomy’s definition. This limitation is a particular concern for PROMs developed before the publication of the taxonomy, and where the terminology used in a study is ambiguous. Content validity and pilot testing (not included in this review), for example, can involve overlapping aspects, and factor analysis can be part of both construct validity and field testing (not included) [2]. Similarly, potentially relevant studies may not have been identified due to discrepancies between the terms in our comprehensive search strategy and the terminology of measurement properties used by authors of such studies, resulting in selection bias. This may also explain the finding that nine studies from reviewers’ personal libraries were not identified by our search strategy.

Another strength was the broad range of PROMs examined, covering many aspects of the biopsychosocial model. They were chosen pragmatically, emerging from the DDPR and the clinical experience of the research team. Nevertheless, a limitation of the selection’s being made by a narrow group is that other relevant questionnaires and corresponding studies may have been missed. The Global Perceived Effect (GPE), Numerical Rating Scale (NRS), and Visual Analogue Scale (VAS), although part of the DDPR, were considered nonspecific response scales: they are used to measure a variety of constructs and, correspondingly, adopt diverse question wordings, anchors at either end of the scale, time spans, and presentations. Given the limited comparability of the different variants, single-item GPE, NRS, and VAS scales were not included in this review. Last, the objective was restricted to the language of the original PROMs and their Dutch translations. Therefore, it is not possible to generalize our results to other languages and cultures, although similar trends, or even fewer studies, could be expected.

For all PROMs, more research is needed to fill the knowledge gaps about their measurement properties, with particular attention to content validity and interpretability. At the same time, for the Dutch versions of the PDI, PCS, and PSC as well as for the original language versions of the PDI, PCS, PSEQ, PIPS, SF-12, and SCL-90, sufficient data seem available to perform systematic reviews to describe their measurement properties, synthesize the data, and perform quality assessments of studies. Specific attention is required as to whether the measurement properties of investigated tools are in line with their intended purposes (diagnostic, predictive, evaluative). The focus of further work could be extended to translations other than into Dutch and to other PROMs. Several research teams performing (replication) studies would likely produce more valid and generalizable conclusions. While the COSMIN taxonomy and the other COSMIN tools are recommended for future studies, their applicability or otherwise to the specific field of rehabilitation medicine should be considered. For instance, the lack of confirmed clinimetrically sound comparator measurement instruments can make testing hypotheses imprecise. Moreover, adequate sample sizes, as indicated in the COSMIN study design checklist, can be challenging to accomplish within the setting of CMP, especially if one aims to examine measurement properties in subpopulations.

While awaiting the results of future clinimetric studies, researchers need to be critical when using PROMs and transparently acknowledge any limitations. Policy- and decision-makers, including health insurance companies, should not overestimate the impact of either individual or group-based PROM outcomes and be careful when drawing conclusions from them. Likewise, health care professionals in rehabilitation should be aware of the limited evidence of PROMs’ clinimetric qualities. Given the heterogeneity of the CMP population and the complexity of rehabilitation interventions, it is in any case of the utmost importance to guide the rehabilitation trajectory, based on combinations of measurement instruments within the biopsychosocial model, together with the clinical expertise of the interdisciplinary team and the patient’s perspective. However, this necessity to integrate outcomes of multiple PROMs covering the total biopsychosocial spectrum when assessing CMP patients, can lead to extensive sets of questionnaires to be completed by the patient, not only in the diagnostic phase but also during and after treatment.

Given the associated burden of this for patients, item banks and computer adaptive testing (such as the Patient-Reported Outcomes Measurement Information System (PROMIS)) that have been validated could be considered as alternatives to traditional patient-reported outcome measurements. For the sake of comparability, a collective approach based on expert consensus between representatives of health care professionals, researchers, and patients is desirable, and an important focus for future work.

5.Conclusion

The studies included in this mapping review demonstrate a paucity of evidence for the reliability, validity, responsiveness, and interpretability of most PROMs frequently used in rehabilitation of patients with CMP in the Netherlands. The main implication is that these PROMs need to be used and interpreted with caution in daily practice.

Ethical approval

Not applicable.

Funding

This work was supported by the Centre for Integral Rehabilitation (CIR). The public partner, Maastricht University, is responsible for the study design, data collection and analysis, decision to publish, and preparation of the manuscript.

Informed consent

Not applicable.

Author contributions

Study design and conceptualizations: S.R., B.L., Search strategy: K.J., I.T., B.L., Databases searching: I.T. B.L., data screening K.A., I.T., S.R., B.L., Data extraction: K.A., I.T., B.L., Data synthesis and interpretation: K.A., B.C.,S.R., B.L., Manuscript drafting B.L., Writing review K.A. B.C., K.J., S.R., B.L., Editing review and visualization K.A., Supervision S.R., B.L. All authors read and approved the final version of the manuscript.

Supplementary data

The supplementary files are available to download from http://dx.doi.org/10.3233/BMR-220133.