Viewpoint 49,3 Polling, politics and the press

Deborah Mattinson

Twenty years ago I arrived on the political scene armed with my toolbox of market research techniques and set out to persuade a sceptical Labour Party of the value of listening to members of the public. Since then, like all main political parties, Labour has commissioned regular quantitative and qualitative research, and has learned that, used properly, these techniques can provide a unique insight into the views of different voting groups.

Labour’s original scepticism came, of course, from the anxiety that research might be used (or abused) to sell a particular point of view: that rather than providing insight, it could peddle prejudice. In fact, as any pollster knows, research designed to sell an idea rather than listen and properly evaluate the reaction will lead to flawed conclusions. For political parties, this would be a short-term policy. Use poor research to sell the unsellable and you lose elections.

Methodological rigour is vital. However, for the media, eager both to adopt modern political polling techniques and find an exciting news story, there is no such brake. Over the years, Opinion Leader Research has provided both qualitative and quantitative polling data to print and broadcast journalists. I understand the tension between creating a good story and telling it like it is (sometimes less sensational and more balanced). I’m now concerned that, increasingly, polling is being used by the media to create headlines rather than uncover facts. Look at two very different examples.

First, an ICM poll for the Guardian, published last November, contrasted voters’ views of Gordon Brown and David Cameron with the headline ‘Brown feels the Cameron effect’. ICM is, rightly, a highly respected polling company and I have no quarrel with the methodology used. However, for a poll comparing two potential prime ministers to be of any real value it is important to know that the attributes on which they were tested were the same attributes by which voters judge the job.

In reports of the poll in the Guardian and several follow-up stories, over which ICM had no control, maximum weight was given to ‘having a pleasant personality’ – not on any PM job description I’ve ever seen – while ‘making the right decisions when the going gets tough’, a much more salient attribute, gained little prominence – there Cameron trailed behind Brown by 7%. Other polling also shows Brown leading in key areas, including a 30% advantage over Cameron on successful management of the economy, while a poll by think-tank Opinion Leader Forum reveals that, among the electorate that really matters – vital swing voters – Brown had a massive 27% lead as preferred prime minister.

Another example was a Newsnight ‘people meter’ session. Its top-line spin – that Home Secretary John Reid could be a front runner – was reported as a fundamental truth, seized gleefully by those whose cause it helped, and used to cast doubt on those who fared less well. But its method – a designed-for- TV hybrid between media pulse analysis and a focus group – is controversial to say the least.

As practitioners, we know the focus group is a technique whereby a small group of people (usually six to eight) is convened and a discussion is moderated by a market research professional. The ideal group structure is as homogenous as possible to enable the group dynamics to work well. The aim is for all group members to speak out, truthfully. The public may not be familiar with this definition.

Described as a ‘focus group’, the piece featured 30 or so individuals drawn from a wide range of backgrounds. The large, heterogeneous meeting was impossible to moderate fairly, with only the self-assured and outspoken making contributions, with the effect – well known by qualitative researchers – of leading the whole session into ‘groupthink’. This was compounded by highly selective probing and the repeated use of shows of hands – a secret ballot on the choices given would at least have had the virtue of giving the quiet dissenters an out.

Further bias was introduced in the dial test. This is a tried-and-tested approach in commercial market research, especially for ad testing. Each participant is given a dial to turn up or down, to indicate simple approval or disapproval in response to material shown. It is crucial to be clear about what message we are testing.

The Newsnight report confused measuring reactions to the content of the speech excerpts and reactions to the character of each politician shown. While John Reid is seen making a speech with a populist anti-terror message, Gordon Brown and David Miliband are both shown answering sensitive questions about the Labour Party leadership. Are the voters responding to the man or the message?

Yet this work was reported as if it provided real insight, despite the fact that it contradicted much of the then current published poll data (ICM the same week gave Brown 46%, a 28% lead over his nearest competitor – Reid). Other possible candidates (Alan Johnson and Milburn) scored just 3% each. Brown’s lead at the time was consistent and unchanged month by month – one thing I’ve learnt over the years is just how long it takes to change people’s views for good or bad – although that makes less exciting copy.

Accurate and insightful market research is contingent not just on reliable fieldwork but also on thoughtful and experienced analysis that sets the findings in a proper context and interprets them accordingly. But, too often, analysis is carried out with an eye on the headline not an eye on the truth.

In the US, especially in broadcast media, there are blurry lines between aggressive push polling and political reporting. I believe we must resist this; it is up to us as practitioners to protect the integrity of what we do. Journalists can’t be allowed to have it both ways: sacrificing rigorous methodology or rigorous analysis will always mean sacrificing the facts. If that is allowed to happen, the sceptics will have been proved right in the end.

Deborah Mattinson is Joint Chief
Executive, Opinion Leader Research.

International Journal of Market Research 49(3), 2007

Would you like to respond to this Viewpoint? Or perhaps you have an idea for another? Responses and new submissions are welcome. They should be emailed to the IJMR, where they'll be considered for publication.

What's New - Membership - Company Partner Service - Members' Area - Code/Guidelines - Qualifications - Training - Awards
Events - Networking - Publications - Media Info - Market Research - Search - Site Map - A-Z Directory - Contact Us - Home