A strategy to test questionnaires at a national statistical office

Abstract

There are many methods (for example, expert reviews, cognitive interviews, and experiments) to test questionnaires and other data collection instruments. In practice, all surveys cannot be tested with all methods; there has to be a balance with regard to survey importance, consequences of data errors, available resources, and costs. Statistics Sweden has developed a strategy how to test questionnaires in different surveys, taking the abovementioned factors into account. This strategy is based on a small set of survey characteristics which mainly are taken from a database with classifications for many surveys. Based on how a survey is classified in the chosen characteristics, the strategy proposes different levels of testing. These levels vary in both ambition and the methods included. The strategy has, since implemented, increased the amount of testing but in a differentiated way, based on risk and resources.