Having been responsible for the YouGov’s Labour leadership polling over the last 18 months, Election Data has asked me to shed some light on how YouGov is consistently able to accurately reflect the membership in these niche elections. As with all survey research, the most important part of any polling is making sure that your sample has the right proportions of people from across demographic groups. At its most basic level, if you surveyed only men, you would likely get very different responses to a survey question than if you only surveyed women so you ensure that you have the right number of both in your sample at the outset.

For YouGov’s Labour leadership polls, they use a number of important demographics :

  • Age
  • Gender
  • Social Grade
  • Region
  • Vote in the 2016 Labour leadership election
  • Membership length

When you look at the full tables, you will note that there are significant differences amongst some of these groups; members who joined before and after Corbyn’s leadership are, for example, very different in their strength of support for Jeremy. This is why it is so important to get the relative sizes of these groups right for each of the bullet points above. If they’re wrong, the overall sample will be wrong and your results will not be accurate of the membership as a whole.

So how does YouGov go about getting the numbers right for each of these groups? Generally, there are three routes to take:

  1. Beg or borrow official party breakdowns
  2. Read the newspapers which often have detailed breakdowns of the membership
  3. Internal YouGov analysis

Routes 1 and 2 above are self-explanatory so I will not focus on them but I think it’s important to briefly outline how YouGov goes about analysing its internal data. YouGov has a panel of over 800,000 UK adults of which a significant number are Labour members. Of the 800,000 UK adults on YouGov’s panel, there will be some skews which make it, at a total level, unrepresentative of the UK population. We call these skews in the data and the 800,000 people will likely skew younger, better off etc. So, once YouGov knows how it’s overall panel skews, it can then look at the Labour members on its panel and adjust based on likely skews in the data based on those that are known to exist in the panel as a whole. This approach was undertaken for all three Labour leadership elections at YouGov and the table outlines just how accurate it proves to be:

Year Average error
2010 0%
2015 1%
2016 0%

Every time I would release a Labour membership survey at YouGov, depending on the results of the poll, either one group or another would seek to claim it unrepresentative, wrong and “never going to happen”. In response, I simply point to the above table and humbly suggest that the reason you doubt the results is simply because you wish them to be untrue.

One thought on “How YouGov get Labour membership polls right”

Comments are closed.