Making sense of pollsters’ errors.

An analysis of the 2014 second-order European election predictions

Pollsters have been recently accused of delivering poor electoral predictions. Maybe this allegation is not entirely correct, as Will Jennings and Cristopher Wlezien argue in a quantitative analysis published in Nature Human Behavior. Yet some substantial and politically important blunders are undeniable, and, for commentators and the public at large, it is irrelevant if those errors are outliers or indicative of a deteriorating quality of pollsters’ work. Can we learn something from those errors, and make sense of them using well-established theories of electoral behaviour? This is the question that we have addressed in a recent article that will be published in the Journal of Elections, Public Opinion & Parties. Our answer is largely affirmative.

jepop

Predicting electoral results is not a straightforward process that simply requires computing frequencies from standard questions. A series of corrections, adjustments and use of post-stratification weights, developed and fine-tuned through several rounds of elections, are needed in order to produce plausible guesses in regard to actual voting behaviours. These improvements have been guided not only by sound statistical know-how but also by the conscious application of deep understanding of the electoral process. In a word, by ‘theory’. We argue that, due to the swift transformations of political systems, and the recent periods of social unrest, we have failed to update this deep knowledge of our changing environment, thus producing less efficient corrections and, ultimately, less precise predictions.

We test this idea of the need for a theory-driven tuning of pollsters’ analyses by checking all the predictions made for the 2014 European election in the 28 EU member states. In that event, the theory of second-order election argues that incumbent parties should lose ground, especially if in the middle of the electoral cycle, while small, new and Eurosceptic parties should succeed in the ballot. If pollsters had failed to include this election-specific knowledge, then they would have had overestimated the former parties and underestimated the latter. Using several measures of the accuracy of those party predictions, our analysis confirms this expectation. Making (theoretical) sense of those errors is reassuring for political science, since it reflects the need for a disciplined and systematic investigation of social dynamics. Yet it is also a challenge, especially if the unparalleled turbulent times in which we live continuously shake the consolidated knowledge of those same dynamics.

Dataset and codebook