I still need some time to process all of this so this is a very rough first attempt/stream of consciousness thing to get down in writing some thoughts and feelings. As such this isn’t so much an argument as an attempt to lay out the stuff I’m trying to unpack in the hope that over time Election 2016 may make more sense.

That was a big reason why I wanted to write this piece here. Because of just about everyone I know Ian’s got this stuff more right than wrong and has a very sound grasp too on what should come next. So I’m grateful to him for letting me use some of his space to air my thoughts aloud.

But before I get into the substance of what I think may have happened, and why, I want to say this: I am sorry. I am sorry for the confidence and cockiness. For the data-driven self-assuredness. For the ‘don’t bedwet’ rants I threw at people on Twitter. For being brash and arrogant where I should have been calm and scientific. I’ll try to do better next time.

OK, LET’S DO THIS!

I’m shocked and thrown by what happened. Things that I counted on turned out not to be the case. Some of the smartest people in politics (David Plouffe) thought that Hillary would walk it. But there were warning signs (for example Nate Silver on the danger of the popular vote/electoral college mismatch; YouGov’s own Freddie Sayers’s concerns about Hillary’s grip, or lack thereof, in the rustbelt) found in electorate volatility, turnout questions and enthusiasm numbers to name but a few.

Basically, Nate Silver’s central lesson to the whole of the commentariat for some months has been ‘low probability events happen people!’ and yet the more information folks like me had the more likely we were to dismiss this warning!

So why did it happen? My rough take thus far (and granted I am still in full post-election exhaustion/recovery mode) is this:

  • Driving through Wisconsin on Monday my friend Mark and I discussed how lucky the Democrats had been in gradually losing their hold in the Rustbelt at a slower rate than they were gaining strength in the Sunbelt. Obviously, the reverse was the case.
  • White working class voters backed Trump in turbo-charged numbers in rural and exurban areas. So there were hidden Trump voters.
  • The political fallout from Globalisation is even worse than I had thought (and I thought I understood it pretty well)
  • Far from being the “nothingburger” me and many Democratic strategists thought, the Comey revelations may well have harmed Clinton in material ways in key electoral college battlegrounds.
  • It looks like Clinton may have gotten as much as 5 or 6 million votes FEWER than Obama four years ago. This wasn’t mere chance. In Lackawanna County, PA, home to the white, blue collar, rustbelt legend that is Scranton, the Democrats went from winning by 27 points in 2012 to winning by 4 points in 2016
  • So, in a nutshell… Democrats turned out in smaller numbers, GOPers turned out in larger numbers and the so-called Obama Coalition (college educated whites, ethnic minorities and millennials) turned out to be aptly named after all – they turned out for Obama both times and voted super-heavy Democrat both times but did not turn out or stay as super loyal to Clinton this time.
  • So what does this all mean?

    Let’s talk polling then politics.

    PROBLEMS WITH POLLING
  • Polls got the national result roughly right (by the time California finishes counting Clinton should be ahead by over one point)
  • Battleground polling was poor (partly this was to do with aggregators and the ‘you put garbage in, you get garbage out’ problem as a bunch of pop-up robodial pollsters does not a real body of data make)
  • But battleground polling was also more limited this cycle than previously. Just take Ohio. This time we had 4 Ohio polls to look at on election eve from the PREVIOUS TWO WEEKS! as opposed to the same amount four years ago from the previous few days.
  • That said, there were also mistakes made by really, really good pollsters like my old favourite phone pollster PPP who, like most of us, predicted a different electorate than the one that actually showed up.
  • How different? Well, it was meant to be younger, more ethnically diverse and more female. But it wasn’t just that more older white men showed up, it was also that within those groups Trump did better than pollsters thought. With the important caveat that we still need county-level results, some reports indicate that Hillary carried Hispanics by six points less than Obama did in 2012 at 65% to 71%.
  • Polling has a problem getting the turnout filter right for low income voters. Previous involvement in elections does not seem to be a good test when populist forces are in play.
  • It may be that to a certain extent progressive voters are inclined to overstate their likeliness to a) vote and b) vote for the progressive choice.
  • At the same time it may also be that conservative voters are inclined to understate their likeliness to a) vote and b) vote for the conservative choice.
  • This may be particularly true of phone pollsters (interviewer effect?) which is why online may well have a valuable contribution to make to the future.
  • We should also look to the success of some modelling efforts. Indeed YouGov’s own US model consistently warned of Pennsylvania’s weakness for Clinton based on socio-economic and demographic fundamentals. It is worth exploring how such models can play a bigger role in future.
  • We should also remember that some data people like @jameskanag also warned of the chance of exactly this scenario BEFORE the election (kudos)
  • Lastly (for now!) we should think about the key drivers of voter decision-making a lot more (psychology, influencers, emotions, hopes and fears and so on) and be more cautious around issues and policies as drivers of voter decision-making. Ian’s work in this area is going to be more important than ever.
  • EXPLAINING THE ELECTION

    Elections, like human nature, are multiply determined. It is tempting to explain the election result away with just one argument, just one factor. But that is wrong. Consider the following (very) incomplete list:

  • Clinton lost Wisconsin by 27,000 votes. 300,000 voters were turned away from the polls on Tuesday in the wake of the state’s Republican-driven voter participation restriction efforts
  • African-American turnout was down – but there are likely to be multiple reasons for this including the fall in enthusiasm from Obama to Clinton as well as the fact that 2016 was the first election in 50 years in which key provisions of the Voting Rights Act were no longer in force
  • What’s more, the GOP deliberately moved against early vote in areas of African American voter concentration. Take North Carolina as an example where in some counties locations went from 16 strong in 2012 to just one in 2016
  • Clinton lost non-college educated white women in battleground states like: Arizona, Colorado, Florida, Michigan, Minnesota, Nevada, New Hampshire, New Mexico, North Carolina, Ohio, Pennsylvania and Wisconsin. So some kind of gender/class disconnect occurred.
  • The Hispanic vote did not support Clinton to the extent they were expected to
  • Also let’s note that Jill Stein’s Green voters in Florida outweigh Clinton’s margin of defeat there. So to those progressives who thought Clinton and Trump were the same: good luck with the reality of your hypothesis.
  • Finally, the overall margin of victory for Trump over Clinton in the Rustbelt states was roughly the size of an American football stadium (100,000). But we should also remember that it looks like Clinton lost over one million Obama voters in the Rustbelt.
  • Change just a couple of points on that list and you have President Clinton! So I’d advise against embracing any single explanation of the result.

    CANDIDATE, MEDIA, AND CAMPAIGN

    For starters I need to get this off my chest: I think Hillary Clinton suffered more as a candidate because of forces beyond her control than because of any actual errors she made. She was more trusted on the economy and on national security. And she won all three debates by lopsided margins. She put on a great convention (with a great poll bounce) and presided over the largest, best resourced, most professional presidential campaign she could have hoped for.

    But… there’s still a problem. And that problem may be the candidate. However unfairly it may well be that voters felt that 30+ years in frontline politics were enough. Add to that the anti-establishment mood of voters everywhere of late and the problems that Hillary Clinton has always had connecting on an emotional level with voters and we begin to see the makings of a case that she was not the ideal candidate to excite and unite the Obama coalition.

    This is likely not her fault. In fact, a large responsibility must be placed on the media who decided roughly on the following:

    Trump university/missing charity money/sexual harassment/Anti-Defamation League condemnations/not knowing the nuclear triad/the wall/Putin/demanding protection money from Japan and South Korea/undermining NATO = email server mistakes.

    Moral equivalence may make for great ratings but it makes for poor morality. Truth does not equal balance and sometimes in fact requires a trade-off, and if it does that trade-off should be in favour of truth both balance. Instead what happened was multiple days of even the New York Times leading on pneumonia or emails instead of issues and the stakes. This may very well have cost Dems support at the polls. After all, five to six million Democrats didn’t just stay home on Tuesday by accident…

    Did Clinton connect with voters? Emotionally? And might this be more important than we had previously thought. This is fruitful ground for future research.

    Now let’s turn to the Clinton campaign itself. Doubtless the internet is full of ‘what happened to the Clinton groundgame?’ and ‘So much for data and analytics’ and various other anti-value hottakes. My own view is that these two functions of the Clinton campaign (in common with most if not all of its functions) performed well. “But how could that be if they lost?”, comes your question. Easy: field and data is in the service of the candidate. Do not mistake technical abilities for strategy. And do not rely on technical ability to counter a weak relationship between candidate and voter. I am pretty sure that field and data led to turnout amongst low propensity target Democratic voters to be higher (indeed potentially significantly higher) than would otherwise have been the case. Take early voting among Hispanic Floridians – it doubled in 2016 compared to 2012. That was not just natural political gravity, that was, at least in part, good data informing good field. That is why I say they did their job well.

    None of which is to say however that there aren’t lessons to be learned. Where there is a concern on the analytics side it is in not preparing for a scenario whereby with but two tweaks of the demographic dial meaning minority turnout goes down and white turnout goes up…

    In future there should be more of a ‘trust but verify’ attitude to data and more understanding of margin of error and the danger of X factors. This goes for pollsters as much as commentators. But we should not throw out data as worthless or declare polling over. Instead we should learn our lessons to make things better.

    FINALLY…

    That’s probably enough for now. And I haven’t even opened up the Pandora’s Box of why no campaigning in Wisconsin, some campaigning in Michigan and massive campaigning in Pennsylvania all seemed to produce the same narrow result…

    But the last thing I’ll say is this: for all the analysis even I have engaged in here on the election probably our biggest mistake was the extent to which the entire conversation about the election became about all of us playing 270towin pundit when perhaps we should have focused more on what was at stake in this election and on the choice before voters. That’s a lesson I’m going to be reflecting on a lot over the next four years.