“I took forward to the inquest” said a member of the twitter commentariat reflecting on the volatility in British polling and the wide divergence we saw from different pollsters predicting anything from an 11% Conservative win to a 3% Labour victory on June 8th.
YouGov however broke the mould during the campaign earning a reputation as something of a maverick, producing poll after poll consistently showing a close race. With the benefit of hindsight we can now see they were pretty close as the graphic indicates. So how did they do it?
YouGov gathered data from 50,000 interviews conducted each week during the campaign. This was supported by additional polling data on previous voting intention, demographic information and likelihood to turnout which they built into a live model of the electorate. It certainly raised eyebrows, showing Labour competing in seats neither they nor the Conservatives seemed to think Corbyn’s party had a chance of taking.
YouGov’s numbers were underpinned by a prediction of a rise in the ratio of the youth and student vote suggesting it would be the highest turnout of 18-24 year olds since 1992. These figures have been born out by an early analysis of the voting public, with a turnout of over 66% among 18-24 year olds, up significantly from 43% in 2015 according to analysis by the Independent. Crucially for Labour, they won the support of 63% of this growing youth vote at the ballot box.
Other polling aggregators such as electoral calculus used a more traditional uniform swing model enriched with demographic data to provide an average of current polls and show a rough approximation. Their predictions of a Conservative majority in the high 60s apparently exaggerating support for Theresa May’s party on a national scale. Those pollsters relying heavily on phone based surveys seem to have fared the worst in failing to reflect the rise of youth vote and in some cases revising down turnout projections for the 18-24 group.
The kind of modelling undertaken by YouGov in place of these traditional polling techniques represents an attempt not just to poll a representative sample but to use that information to create a simulation of electorate. It has picked up momentum in recent years and has become something of a holy grail for British election aficionados looking enviously across the Atlantic to the likes of Nate Silver and FiveThirtyEight’s probability model which came to fame after correctly predicting the 2012 US Presidential Election result. To sound a note of caution, these simulations are only as good as the information submitted to them. Silver’s model failed to predict the US presidential election in 2016 which potentially presents a vulnerability in such simulations when elections are close.
There has been a fair amount of healthy scepticism around the polls and simulations this time around due in part to failures to accurately predict the 2015 General Election and Brexit referendum outcomes. This has been reflected in traditional news media which have as a rule invested in far more qualitative on the ground reporting and exit polling than in previous elections. This in itself could mark a huge sea-change for the polling industry which has traditionally relied on the income from print and news media to sustain their business model.
While there are certainly moves away from a reliance on polling when reporting and understanding elections, the YouGov study demonstrates that when done well it remains highly relevant and it will be one to watch for the next General Election, whenever that may be.