Viewpoint: Checks and balances

David V.L. Smith
Incepta Marketing Intelligence

There has been the emergence of a 'new' category of market research that we could describe as the 'development of reasoned, robust arguments based on a firm understanding of customers'. This requires researchers to provide, in a transparent, accountable way, evidence that is robust and rigorous, but also responsive to the growing calls for researchers to: draw on multiple sources of (often) imperfect evidence; go beyond the 'literal' consumer survey evidence; make various creative interventions to provide fresh insights and new perspectives; and also weave customer knowledge together with other types of market, financial and organisational information. This has left us with a challenge in the way in which we now evaluate the 'quality' of this more 'holistic' offer.

The market research industry has an array of textbooks that provide clear accounts of how 'classic' qualitative and quantitative market research 'works'. In addition, we have various industry guidelines that help define what constitutes good practice with regard to different types of market research methodology. Furthermore, we have a rigorous set of quality assurance standards that help regulate our data collection, and related, processes. Moreover, we have the ESOMAR/Market Research Society Code of Conduct to help maintain standards and alert the different parties involved in the research process as to their responsibilities and appropriate conduct. Taken together, this provides the users of our 'new'-look data with a guide to how they can establish what constitutes a 'reasoned, robust, evidence-based argument'. However, notwithstanding the above quality infrastructure, there remains a shortfall in what the end user really needs to know about the quality of the evidence they are using to make informed judgments. The arrival of this new 'category' of more eclectic market research takes us into an entirely new 'quality paradigm'. We are edging towards a point where, very soon, what we have in place to evaluate the quality of what we do as market researchers is going to look very thin and outdated.

So, in my view, we should now set up an industry-wide working party to reexamine just how well the existing Code of Conduct, our good practice guidelines and the various other quality assurance standards we have in place, are working to educate clients about the quality of the survey evidence they are using for decision making. Would it be possible to amalgamate an up-to-date, honest account of how, in practice, we now evaluate our 'new'-look consumer evidence with the essence of the existing Code of Conduct and our various other industry guidelines, and standards, on good practice? Should we not integrate these documents to create an easy-to-understand, accessible account of the key concepts and principles that the users of our data need to know to establish its robustness? Clearly, this needs a considerable amount of development, but such a document – let us give it the working title of a 'Charter' – would seem invaluable in helping decision-makers make sense of the customer knowledge upon which they are now relying. This must be written in an engaging, conceptual way that senior decision-makers, the media (and the general public) will easily understand.

The Charter could take the form of a series of 'telling' questions that decision-makers should ask their suppliers about the evidence they are providing. These 'killer' questions – which need to be pitched at a conceptual level – would enable the decision-maker to establish, for example, whether the research design that generated the evidence was fit-to-purpose in the context of the decision. It would allow the decision-maker to get a feel for the degree to which the dialogue that took place between the interviewer and the respondent did justice to what the respondent knew, and wanted to say, on this subject. And, of course, there will be various other hard-hitting questions to get to grips with the rigour with which the data analyst integrated the different sources of evidence and arrived at their final interpretation of the central storyline.

Of course, there is a massive amount of devil in the detail. But, hopefully, in this short piece, readers will get an idea of the form such a Charter could take, and the role it would play. In essence, we are talking about a punchy, easy-to-understand document that embraces: the authority of the Code of Conduct; the spirit of the various industry good practice guidelines we already have in place; and some of the detail of our quality assurance documentation.

In sum, we need to rethink the issue of how we now determine what constitutes quality in a demanding environment in which market researchers are increasingly being called upon to be transparent, accountable, rigorous and robust with their recommendations, but are also constantly being asked to be creative, lateral in their thinking and imaginative in the pursuit of consumer 'insights'. In my opinion, we must respond to this new reality with the creation of some form of Charter or Manifesto. If the market research industry does not take action along the lines I am suggesting we could drift into a very dangerous 'anything goes' approach to evidence-based decision making. So we owe it to the next generation of market researchers to get on top of this issue now.

International Journal of Market Research 48(1), 2006

Would you like to respond to this Viewpoint? Or perhaps you have an idea for another? Responses and new submissions are welcome. They should be emailed to the IJMR, where they'll be considered for publication.

What's New - Membership - Company Partner Service - Members' Area - Code/Guidelines - Qualifications - Training - Awards
Events - Networking - Publications - Media Info - Market Research - Search - Site Map - A-Z Directory - Contact Us - Home