Explaining political participation: A comparison of real and falsified survey data
Abstract
This paper examines differences between real survey data and data falsified by interviewers. Previous studies show that there are only small differences between real and falsified data which implies that falsifying interviewers are able to (re-)produce realistic frequency distributions. The question this paper aims to answer is whether they are also able to produce multivariate results in accordance with the assumptions of established social science approaches. As an example for a realistic theory-driven data analysis, real and falsified data are compared in terms of the identified determinants of political participation. I use an experimental data set with data partly collected in real interviews and partly by interviewers being instructed to falsify; that is, to fill in the questionnaire based on little information about the respondent. The questionnaire measures twelve political activities, based on which I calculate an index for political participation. There are differences in the models between the real and the falsified data: The explained variances are higher in the regression models of the falsified data. There are some variables significant in both data sets and some that are significant only in the real or in the falsified data. These differences can be explained by our theoretical assumptions.
References
[1] | Mangione TW., , Fowler FJ., and Louis TA., Question characteristics and interviewer effects, Journal of Official Statistics 8: (3) ((1992) ), 293-307. |
[2] | Bredl S., , Storfinger N., and Menold N., , A literature review of methods to detect fabricated survey data. in: Interviewers' Deviations in Surveys, Winker P., , Menold N., and Porst R., eds, Frankfurt: Peter Lang (2013) , pp. 3-24. |
[3] | AAPOR. Interviewer Falsification in Survey Research: Current Best Methods for Prevention, Detection and Repair of Its Effects. (2003) [cited 2015 May 15]. Available from: http://www.aapor.org/pdfs/falsification.pdf. |
[4] | Sodeur W., Entscheidungsspielräume von Interviewern bei der Wahrscheinlichkeitsauswahl, Methoden Daten Analysen 1: (2) ((2007) ), 107-130. |
[5] | Winker P., Assuring the quality of survey data: Incentives, detection and documentation of deviant behavior, Statistical Journal of the IAOS 32: (3) ((2016) ), 295-303. doi: 10.3233/SJI-161012. |
[6] | Menold N., , Winker P., , Storfinger N., and Kemper CJ., , A method for ex-post-identification of falsifications in survey data. in: Interviewers` Deviations in Surveys, Winker P., , Menold N., and Porst R., eds, Frankfurt: Peter Lang (2013) , pp. 25-48. |
[7] | Menold N., and Kemper CJ., How do real and falsified data differ? Psychology of survey response as a source of falsification indicators in face-to-face surveys, International Journal of Public Opinion Research 26: (1) ((2014) ), 41-65. doi: 10.1093/ijpor/edt017. |
[8] | Krosnick JA., and Alwin D., An evaluation of a cognitive theory of response-order effects in survey measurement, Public Opinion Quarterly 51: (2) ((1987) ), 201-219. |
[9] | Reuband K-H., Interviews, die keine sind. ``Erfolge'' und ``Mißerfolge'' beim Fälschen von Interviews, Kölner Zeitschrift für Soziologie und Sozialpsychologie 42: (4) ((1990) ), 706-733. |
[10] | Schnell R., Der Einfluss gefälschter Interviews auf Survey-Ergebnisse, Zeitschrift für Soziologie 20: (1) ((1991) ), 25-35. |
[11] | Schraepler J-P., and Wagner GG., Identification, Characteristics and Impact of Faked Interviews in Surveys, An analysis by means of genuine fakes in the raw data of SOEP. IZA Discussion Paper No. 969, Forschungsinstitut zur Zukunft der Arbeit, (2003) . |
[12] | Koch A., Gefälschte Interviews: Ergebnisse der Interviewerkontrolle beim ALLBUS 1994, ZUMA-Nachrichten 36: ((1995) ), 89-105. |
[13] | Krejsa EA., , Davis MC., and Hill JM., Evaluation of the Quality Assurance Falsification Interview used in the Census 2000 Dress Rehearsal, Proceedings of the American Statistical Association (Survey Research Methods Section) (1999) , 635-640. |
[14] | Li J., , Brick JM., , Tran B., and Singer P., Using Statistical Models for Sample Design of a Reinterview Program, Proceedings of the American Statistical Association (Survey Research Methods Section) (2009) , 4681-4695. |
[15] | Kuriakose N., and Robbins M., Don't get duped: Fraud through duplication in public opinion surveys, Statistical Journal of the IAOS (2016) , 1-11. |
[16] | Koczela S., , Furlong C., , McCarthy J., and Mushtaq A., Curbstoning and beyond: Confronting data fabrication in survey research, Statistical Journal of the IAOS 31: ((2015) ), 413-422. |
[17] | Landrock U., and Menold N., Validation of theoretical assumptions with real and falsified survey data, Statistical Journal of the IAOS 32: (3) ((2016) ), 305-312. doi: 10.3233/SJI-161020. |
[18] | Winker P., , Kruse K-W., , Menold N., and Landrock U., Interviewer effects in real and falsified interviews: Results from a large scale experiment, Statistical Journal of the IAOS 31: (3) ((2015) ), 423-434. doi: 10.3233/SJI-150908. |
[19] | Kemper CJ., and Menold N., Nuisance or remedy? The utility of stylistic responding as an indicator of data fabrication in surveys, Methodology. (2014) . Advanced online publication. doi: 10.1027/1614-2241/a000078. |
[20] | Barnes SH., , Kaase M., , Allerbeck KR., , Farah BG., , Heunks F., , Inglehart R., , Jennings MK., , Klingemann D., , Marsh A., and Rosenmayr L., eds. Political Action. Mass Participation in Five Western Democracies, Beverly Hills (CA): Sage, (1979) . |
[21] | Kaase M., , Marsh A., and Political A., , A Theoretical Perspective. in: Political Action. Mass Participation in Five Western Democracies, Barnes SH., and Kaase M., eds, Beverly Hills (CA): Sage (1979) , pp. 27-56. |
[22] | Marsh A., and Kaase M., , Measuring Political Action. in: Political Action. Mass Participation in Five Western Democracies 27-56, Barnes SH., and Kaase M., eds, Beverly Hills (CA): Sage (1979) , pp. 57-96. |
[23] | Koch A., , Kurz K., , Mahr-George H., and Wasmer M., Konzeption und Durchführung der ``Allgemeinen Bevölkerungsumfrage der Sozialwissenschaften'' (ALLBUS) (1998) . Mannheim: ZUMA-Arbeitsbericht 1999/02. 1999 [homepage on the Internet, cited 2014 Sep 30] Available from: http://nbn-resolving.de/urn:nbn:de:0168-ssoar-200413. |
[24] | van Deth JW., , Soziale und politische Beteiligung: Alternativen, Ergänzungen oder Zwillinge? in: Politische Partizipation in der Bundesrepublik Deutschland. Empirische Befunde und theoretische Erklärungen, Koch A., , Schmidt P., and Wasmer M., eds, Opladen: Leske und Budrich (2001) , pp. 196-219. |
[25] | Lüdemann C., , Politische Partizipation, Anreize und Ressourcen. Ein Test verschiedener Handlungsmodelle und Anschlusstheorien am ALLBUS 1998. in: Politische Partizipation in der Bundesrepublik Deutschland. Empirische Befunde und theoretische Erklärungen, Koch A., , Schmidt P., and Wasmer M., eds, Opladen: Leske und Budrich (2001) , pp. 196-219. |
[26] | Opp KD., and Finkel SE., , Politischer Protest, Rationalität und Lebensstile. Eine empirische Überprüfung alternativer Erklärungsmodelle. in: Politische Partizipation in der Bundesrepublik Deutschland. Empirische Befunde und theoretische Erklärungen, Koch A., , Schmidt P., and Wasmer M., eds, Opladen: Leske und Budrich (2001) , pp. 73-108. |
[27] | Steinbrecher M., Politische Partizipation in Deutschland, Baden-Baden: Nomos, (2004) . |
[28] | Westle B., , Politische Partizipation und Geschlecht. in: Politische Partizipation in der Bundesrepublik Deutschland. Empirische Befunde und theoretische Erklärungen, Koch A., , Schmidt P., and Wasmer M., eds, Opladen: Leske und Budrich (2001) , pp. 132-168. |
[29] | Schulz W., , Politische Mobilisierung durch Mediennutzung? Beziehungen zwischen Kommunikationsverhalten, politischer Kompetenz und Partizipationsbereitschaft. in: Politische Partizipation in der Bundesrepublik Deutschland. Empirische Befunde und theoretische Erklärungen, Koch A., , Schmidt P., and Wasmer M., eds, Opladen: Leske und Budrich (2001) , pp. 170-194. |
[30] | Wasmer M., , Scholz E., and Blohm M., Konzeption und Durchführung der ``Allgemeinen Bevölkerungsumfrage der Sozialwissenschaften'' (ALLBUS) 2008. GESIS-Technical Reports (2010) ; 2010/04. |
