In the first issue of 2017 we published an obituary written
by Bill Blyth for Martin Collins, who died late last year. As a tribute to Collin’s very extensive
contribution over many years to the development and application of research
methodology, my Landmark Paper selection for this quarter is a paper where he
applies Andrew Ehrenberg’s principles for data reduction to the outputs from a
typical market research survey.
In his Introduction to the paper, Collins draws
particular attention to the sub-title, ‘Analysing statistical data’, of
Ehrenberg’s original book on this subject, ‘Data reduction’, the aim being to
teach people to ‘see patterns and relationships that exist in numerical data
and to reduce these to summaries that can be readily interpreted, used and
Collins reiterates the six core rules for the Ehrenberg
approach, illustrated with examples of best practice, demonstrating how this
leads to easily identifiable patterns in data. As Collins says latter on in the
paper, Ehrenberg’s rules are ‘straightforward, but not, sadly, common sense’ – a comment that is as
relevant today as when the book was first published in 1975 (or since the
advent of complex statistical data), especially in the current era of big data.
Commenting on the initial example in his paper, Collins concludes that: ‘In
terms of detail, almost nothing has been lost; in terms of communication,
everything has been gained’.
The focus of the paper is a very detailed application of
Ehrenberg’s rules to a survey conducted by the then SCPR (now NatCen) on public
perceptions of bias in TV new coverage in the UK – a topic that is also as
relevant today as when the paper was written.
The final model produced from the
data clearly identified the key underlying factor, thus enabling comparisons to
be easily made between sub-groups, and for other key issues to be discerned.
The model also identified the image for news coverage of the then new,
anti-establishment, Channel 4.
Collins concludes with some key lessons for analysts when
applying Ehrenberg’s principles to survey data.
Firstly, Collins advises that an analyst needs to be sure of
the patterns in the data before attempting to communicate findings to others; secondly,
think simple - such as using percentages and cross-tabulations to identify key
insights; thirdly, experience and intuition count – so-called objective based
insights maybe biased by assumptions, whereas subjective views based on
experience can provide insights of more value to decision makers; fourthly, don’t
ignore the actual questions asked in the survey as the context for delivering
insights; finally, ensure that any limitations caused by survey design
are not ignored.
In my role as IJMR editor, I frequently see instances where
these basic rules have not been applied when analysing, or presenting, data in
papers submitted to IJMR, as I’m sure many of our peer reviewers will attest.
Collins, for example, was often exasperated by
the poor analysis and presentation of data in some of the many papers he
reviewed over the years for IJMR, and this exasperation showed in his reviews!
In the era of big data, where survey data may
be combined with other sources, or analysts are faced with deriving and
communicating the analysis from millions of individual records, the principles
derived by Ehrenberg and applied in this Landmark Paper by Collins, need to be
front of mind if researchers are to provide decision makers with clear,
meaningful and actionable insights.