Matching and prediction on the principles of biological classification[Download PDF] William A Belson (MRS Gold Medal winner, 1961) In this article Dr Belson describes a technique for matching population samples. This depends upon the combination of empirically developed predictors to give the best available predictive, or matching, composite. The underlying principle is quite distinct from that inherent in the multiple correlation method.
Test Marketing: an examination of sales patterns found in 44 recent tests[Download PDF] E J Davis (MRS Gold Medal winner, 1966) In this paper, which won the 1966 MRS Gold Medal, 44 test market results were analysed to see if general patterns in the sales curves could be found. All the cases came from retail audit data from continuous test panels of stores. Two thirds were grocery products, the remainder from chemist panels. Distribution in the chemists is typically much higher than among the grocers. The distribution curve (proportion of stores handling) reaches its shoulder for most products by the third 4-weekly check, after which the increase slows. At this shoulder, on average, 80% of chemists were handling products but only 35% of grocers. Build-up of consumer purchases was analysed in three ways: build up to peak sales; run-down from peak sales to stable level and the ratio between peak sales and the stable level. This is calculated as peak sales minus stable sales divided by peak sales (the D ratio). The average drop is about 40% from peak level. It is concluded that a brand launch will fail to meet its long-term (stable) target unless it achieves at least twice that target level at some point in its early life. The sample is too small and the test periods studies mostly too short to generalise about the consistency of these D ratios (or Drop factors). However, a test market product from the BMRB archives, which had very varied peak and stable sales levels between eight areas, was reanalysed to calculate D ratios. These were found to group much more tightly (range between .63 and .74) than the sales levels, suggesting that D may be a useful statistic for predicting test markets.
Some techniques and interesting results in discrimination testing[Download PDF] C Greenhalgh (MRS Gold Medal winner, 1968) This paper won the 1968 MRS Gold Medal, and describes and illustrates discrimination testing (ensuring that a change in product formulation, e.g. to use cheaper raw materials, is indistinguishable from the current formulation to brand users). The test was used on large samples of ordinary users. The method used was triangular testing: each tester is given three samples, two of which are identical, and asked to pick out the one that is different. In the first test, 40% correctly chose the odd one out, significantly more than guesswork would give (33%). The proportion who can discriminate is 40-33 divided by .67 (the proportion available after eliminating the correct guessers), i.e. 10%. (the sample size had been set to desired confidence limits). Three further triangular tests tested respectively 50/50, 75/25 and 67/33 mixtures of the new ingredient, all against the original formula. The 50/50 version was identified (above guesswork) by no-one. The other two failed to reduce the discrimination below 7%. It was decided to adopt the 50/50 mixture. The paper goes on to discuss another series of more complex tests, in which testers were asked to choose between more alternatives. This enables samples to be reduced, but introduces other sensitivities. Concerns about effects of artificiality and complexity in biasing test results (discussed) led to the same product being tested `real-life' (a split sample using normally and re-interviewed later). Could they distinguish the sample from what they normally bought? Results (confirmed in a replicated study) showed little discrimination. A further test in which respondents were told that they would be re-interviewed about the product increased discrimination, confirming that an artificial test format does bias answers. Triangular testers who could not discriminate unexpectedly had similar preferences to those who could. Possible explanations for this anomalous result are discussed.
Some observations on, and developments in, the analysis of multivariate survey data[Download PDF] D Johnston and J Inglis (MRS Gold Medal winner, 1970) This important paper takes a critical look at two commonly used multivariate analysis tools - principal component analysis and cluster analysis - which were at the time becoming feasible with the advent of computers, and even easier since through PC power. Uncritical use of these methods may fail to take account of features in the data set (and every survey has some unique characteristics) which may jeopardise the validity of the whole analysis. Under principal components, the problems considered are: 1) the effect on the final result of some statistical and non-statistical properties of the data; 2) discrepancies between `factor loadings' and `factor estimation weights', the importance of such discrepancies and some circumstances under which they occur; 3) the value of popular analytic rotation procedures as means of bridging the gap between the mathematics and psychological interpretability. Under cluster analysis, the problems are: 1) the measurement and importance of the extent to which populations can be considered as discrete segments or clusters; 2) the effect of the nature and number of variables used as classification criteria for the segments (a new method to aid selection of variables is suggested); 3) the use of alternative statistics in measuring between-person overall similarity or difference; 4) the selection of starting points or `trial characteristics' for segments (a new method is suggested and compared with existing ones). In both cases, the underlying theme is that these methods will produce an apparent result even if the data put into the analysis are meaningless or break the assumptions on which they depend, and if this happens the apparent patterns may well be an illusion. Yet the pressure for ever simpler understanding at ever greater speed makes this more and more likely to happen. This paper was published in the Journal of the Market Research Society 12 (2), April 1970, and republished as part of the `Milestones in Research' series celebrating the 50th anniversary of the MRS. An updating comment by the authors is included.
Evaluating community preferences[Download PDF] G Hoinville (MRS Gold Medal winner, 1972) The Priority Evaluator research method discussed in this paper was first described in a paper by the author at a Weekend Seminar held by The Market Research Society in February 1970. That paper was then revised and extended to its present form in September 1970. The paper was published in Environment and Planning in 1971 and subsequently received The Market Research Society Gold Medal award for 1972. The paper describes the early developments in the research approach which was devised and applied by Social and Community Planning Research during its initial eighteen months. Since then the work has been considerably extended and several major research projects have been undertaken which incorporate the basic approach. The underlying aim of the method was to extend traditional attitude survey measurements by allowing respondents to understand and respond to the concept of trade-off preferences. As the work has progressed so a number of modifications have been made. The analytical component of the research method has also been more extensively developed with the aid of the much greater volume of data available from the later studies. The author would like to extend his thanks to all those who have helped in the development of the research method and particularly to Roger Jowell who was jointly responsible in developing the basic approach.
Pretesting press advertising campaigns[Download PDF] C Greenhalgh and H A Smith (MRS Gold Medal winner, 1973) This paper, which won the 1973 MRS Gold Medal, discusses how to pre-test press advertisements for evaluation (i.e. deciding whether to run the campaign or not). Three problems in such testing are identified and discussed in detail: tests involve showing a single exposure, whereas in real life campaigns readers will see multiple exposures; tests inevitably invite unrealistically high attention levels compared to what the ads will achieve in reality and pre-test experiments must be monadic, with matched samples, yet it can be difficult to match on variables which really matter. A test designed to overcome these difficulties is described. It compared the use of colour versus black-and-white in a women's magazine campaign (colour gets more attention but is more expensive to run, so fewer exposures would be afforded). First, a large sample (2000+) of housewives was contacted to provide a subset with the required qualifications (including brand images). Later (without forewarning) the potential testers were invited to take part in a test of Woman's Own. Matched clusters were formed from those 420 who responded, thus ensuring that they were matched on the distributions of all 60 brand image dimensions. The two matched subsamples were sent six weeks' issues of Woman's Own, containing respectively the colour and the mono ads. A final personal interview repeated the brand image questions, and followed with readership and page traffic questions, reading and noting, and other questions about the ads. Results are discussed in detail: the matching produced far better results on image dimensions than a random procedure, and at least as good on magazine readership; brand image improvements pre-post were modest, but with variations and colour did not seem to attract greater attention, but once attended to, did induce more extensive reading and more favourable opinions about the ads (though not about the brands).
Linguistic coding: a new solution to an old problem[Download PDF] C D P McDonald (MRS Gold Medal winner, 1974) The method described in this article was first put forward in a paper by McDonald & Blyth published in the 1971 Thomson Gold Medal Awards 'How to Handle Soft Data - A Linguistic Approach' (1971). The judges in their foreword, while commending it for originality, expressed some doubts about the practical applications of the proposed technique. Since then (i.e. during 1972) the technique has in fact been used for five client commissions, four of which involved open-ended interviewing on a large scale and in the process we have been able to refine and improve considerably the operation and cost-effectiveness of the technique. It seems timely therefore to write about it again, both to give the lie to the judges' comment and to indicate, hopefully, how useful this new approach can be in tackling a very old research problem.
The trade-off model and its extensions[Download PDF] Dick Westwood, David Beazley and Tony Lunn (MRS Gold Medal winner, 1975) This 1975 paper marked the launch onto the European market research scene of both the trade-off method of estimating consumer values and also Micro-Behavioural Modelling (MBM) more generally. Both approaches aroused controversy at the time. But they were widely regarded as innovatory, and the paper was awarded the Society's Gold Medal. Revisiting the paper 21 years on was an encouraging experience. Along with the two associated publications in European Research (see references), it provides a cogent exposition of the authors' early work on trade-off and remains a useful introduction to the MBM approach.
Sequential analysis in market research[Download PDF] K Gorton, E J Anderson and R Tudor (MRS Gold Medal winner, 1976) This article, based on the 1976 MRS Gold Medal winner, discusses sequential analysis (analysing research results as they are obtained rather than after data collection is completed). It is especially applicable to dichotomies. Sequential theory is largely based on Wald's sequential probability ratio test. Two main applications occur: when it is important to determine which alternative (version A or version B) is preferred, and when the concern is only whether A is preferred, other alternative being of less interest. For the first of these, symmetrical designs are used. Control lines on charts show the outer and inner boundaries in each direction (A or B preferred, no difference). As results come in, the path is drawn on the chart and the survey can be stopped when one outer boundary is crossed. In selecting the most appropriate chart one must consider the level of significance required, power of the test (probability of obtaining a significant result at a chosen level), probability of a preference and economy. For the second, a range of validation studies are described and discussed. Applications are discussed, such as in-company research. It is argued that the method is adaptable to multi-choice situations. Various cautions about the use of the method are noted.
The effect of clustering on costs and sampling errors of random samples[Download PDF] Paul Harris (MRS Silver Medal winner, 1979) This paper first reviews briefly the theory associated with clustering used in the design of random samples. A further section deals with some particular intra-class correlation coefficients calculated from various town surveys, where different degrees of clustering were employed. These calculated coefficients are summarised and an attempt made to assess their effect when used on national samples. Costs for such national studies are estimated and some compromise sampling solutions suggested. The effect of changing the number of interviews in each cluster is investigated.
Interviewer variability[Download PDF] Martin Collins (MRS Silver Medal winner, 1981) This paper presents a review of the problem of interviewer variability in survey data. Results of recent experiments, together with instances of the problem recorded by other researchers, are used to suggest likely causes of interviewer variability and to indicate ways in which the problem can be tackled in survey design and administration.
Pricing research techniques[Download PDF] Chris Blamires (MRS Silver Medal winner, 1982) This paper seeks to review current pricing research techniques from the (hopefully) unbiased viewpoint of the user. Techniques are, for the purposes of the review, divided into direct techniques which simulate the point-of-sale situation (and which include recent applications of 'trade-off' analysis) and indirect techniques - the more 'psychological' measures of which Granger-Gabor is one example. The advantages and limitations of these techniques are assessed and a new methodology outlined which, it is suggested, provides improvements in several key areas by making use of elements of both.
The use of interaction coding and follow-up interviews to investigate comprehension of survey questions[Download PDF] Wendy Sykes and Jean Morton-Williams (MRS Silver Medal winner, 1984) This paper was based on findings from a research programme, funded by the ESRC and instituted at the Survey Methods Centre at SCPR under the directorship of the late Professor Gerald Hoinville. Building on the work of researchers in a number of diverse fields, the programme of research sought to 'lay bare' aspects of the survey process which are normally concealed: namely the interactions that take place between interviewers and respondents in the field. The main tool which was developed to assist in this was a classification of interviewer and respondent behaviour, applied in a systematic way to tape-recordings of interviews taking place in the field. For the purposes of this paper, we selected codes which we felt were indicative of problems with the administration or answering of survey questions (e.g. requests for clarification, questions misread), and attempted to identify the items in a survey which seemed persistently to create difficulties of one kind or another. Clarification of the nature of these difficulties was sought using an approach developed in the UK by Bill Belson. Semi-structured interviews were carried out with respondents to the original survey exploring - for each 'problem' item -their understanding of the question, the process by which they had arrived at their answer, their motivation to respond and their views as to the 'accuracy' of their response. The methods we employed now form part of a well-recognised battery of approaches which fall under the loose rubric of cognitive techniques. At the time - in the UK at least - we were breaking new ground and this is all too evident in our rather laborious introductory sections. Nowadays a brief reference to behaviour coding and retrospective 'think aloud' cognitive interviews would do away with most of the first few pages! Whatever the intrinsic value of the contents of this paper, we believe that it contributed to the development of current interest in, and application of more deeply probing methods for developing and testing survey questions. Probably the most famous of all surveys - the Census - is currently being rehauled using ~uch methods. Our re-reading of the paper reminds us only too vividly of the sense of excitement (and occasional despair) with which we approached the whole project. Most remarkable to recall is the opportunity which was given to us to undertake a labour intensive, exploratory piece of work. And to follow our interest without commitment to a specific output or 'useful' application beyond the reporting of our findings.
Time use, technology and the future of work[Download PDF] Jonathan Gershuny (MRS Silver Medal winner, 1986) This paper, which won the 1986 MRS Gold Medal, reviews preliminary results from a study of changing time-use patterns over the previous half-century, as a basis for speculation about future activity patterns. A particular interest is the growing application of IT (information technology). The limited sources available for time use in the past are reviewed. Time spent on various tasks (in minutes per average day) is compared between 1961 and 1983. Five areas stand out and are discussed: reduction in time spent on paid work; increase in time devoted to education and medical consumption; large decline in routine housework; increased time spent in out-of-home leisure activities; growth in shopping time. It is argued that current technology has now met saturation in satisfying current needs and markets. But the way is open for new technology, especially IT, to fuel the development of new habits of living and therefore new markets.
Analysing qualitative data[Download PDF] Steve Griggs (MRS Silver Medal winner, 1987) This paper is the 1987 MRS Gold Medal winner, and discusses methods of analysing qualitative research data, arguing that they should be open and public (i.e. any researcher should be following the same procedures be able to arrive at the same conclusions). The author is troubled about the lack of concern among qualitative researchers, and in the literature, about analysis: they talk instead only about interpretation, which is what follows analysis. a key preliminary is to develop an appropriate conceptual framework and a set of research questions to answer, before approaching analysis. Without these, the researcher is reduced to dredging the data: this may reveal patterns, but it is impossible to know whether they are random or have meaning. Three basic methods are described and discussed: data reduction, data display and drawing and verifying conclusions. Within each of these, a number of approaches are described, with some examples from actual projects. Analysing qualitative data closely mirrors (or should do) quantitative data analysis. The results should always be transparent and open to question. These standards should not be compromised by time and money pressures.
What have we learned from researching AIDS?[Download PDF] John Samuels and Simon Orton (MRS Silver Medal winner, 1988) This paper is a case history setting out what we have learned from a major programme of research into public knowledge, attitudes and behaviour in relation to the disease AIDS, with the emphasis on research methodology. The research was commissioned by the British Government in relation to its programme of public education and publicity. The paper is in seven sections: Section 1 discusses the background, presenting data on the incidence and spread of the disease in Britain and projections for the future. The publicity campaign objectives are given together with a general outline of the scale of the research programme to date in which 4,500 people, including a large sample of the 'high risk' groups, (just under 1,000 homosexuals and over 700 young people aged 13-17) have been interviewed with a 40 minute questionnaire. Section 2 describes the sample design for the general public and homosexuals, outlining the methodology in detail and the efforts made to ensure matching of samples between waves of interviewing. The success of the methodological approach adopted is described together with some less successful aspects. Section 3 is devoted to the questionnaire design, outlining the problems faced in relation to obtaining extremely sensitive and explicit information (e.g. on sexual behaviour). The ways in which these problems were tackled are explained and a comment is made on the quality of the data. Section 4 covers the reactions of interviewers to working on a project obtaining such sensitive information. Section 5 explains how the project was extended from the general public 18-64 to include young people 13-17 and how the questionnaires and sampling methodology were adapted. Section 6 gives some results to demonstrate changes in knowledge, attitudes and behaviour over the period investigated. Section 7 comprises four concluding observations about (i) the role of the research in strategic planning, (ii) the opportunities for deeper and more sophisticated analyses, (iii) the likely future evolution of the research programme and (iv) a plea for survey researchers working on AIDS projects to share their experiences openly as time is not on our side.
Small is beautiful but difficult[Download PDF] David Smith (MRS Silver Medal winner, 1990) In this paper it is argued that the solution to the problem of how to conduct effective research for small businesses lies not just in improved techniques, but also in redefining the market researcher's role. It is argued that market research should become a more integrated part of the business development process. In this way, market research will be better placed to demonstrate to small business clients that effective market research requires a range of higher order investigative and analytical craft skills that are not generally recognised by the wider business community. It is argued that it is these craft skills that give market research its power. It is this craft dimension that distinguishes true research from head counting and number crunching. To illustrate this point the paper draws on three quite varied examples of small business problems requiring research; a decision whether or not to open a book shop, the question of whether to invest in machinery to produce specialist ethnic cheese products for the UK market and a decision on the marketing opportunities for a new portable toilet. In short, this paper's message is that in order to unleash the potential of the market research craft, ways must be found of ensuring that market researchers work in close proximity to small business clients. It is important for small businesses to obtain a feel for the workings - the subtleties of the market research craft - and not see market research as a black box technique that generates an answer.
Microdata From the 1991 Census of Population in Britain: Applications in Marketing Research[Download PDF] Catherine Marsh (MRS Silver Medal winner, 1991) Samples of anonymised census records (SARs) are to be released for the 1991 census in Britain. At present the proposals only cover England, Wales and Scotland, but representations are also being made to the Census Office in Northern Ireland to grant a similar request with respect to the 1991 Northern Irish census. In this article, after some background to the request that census microdata be released for re-analysis, an outline is given of the two files to be released, and the methods that have been adopted to protect the data from risk of disclosure. The purchasing arrangements and timetable are explained. Then follows discussion of the different ways in which this microdata could be used, and various applications which market researchers might find of benefit are suggested.
Preventing Childhood Diseases: The Need for Action[Download PDF] Martin van Staveren and Jenny Turtle (MRS Silver Medal winner, 1992) Case history describing a research project carried out by BMRB for the Department of Health. The research was designed to find out why, when safe and reliable vaccines were available, many children remain unprotected by immunisation. The proportion of children immunised against whooping cough, measles and polio varied widely between health districts. The study explored possible reasons for the variation, distinguishing between social and demographic factors and the management of the immunisation programmes. Multiple regression methods were used to identify the relative importance of different factors. The findings showed a strong link between low levels of uptake and social disadvantaged inner city areas. Controlling for these differences, it was possible to identify those differences in practice and management which are associated with high uptake rates: as a result, a number of practical recommendations could be made to the Department of Health. The methodology used in this study is applicable to other areas of social research where the subject of study is strongly linked to population differences. This paper was first presented at the 44th ESOMAR Congress, Luxembourg 1991.
Advertising Effects: More than Short Term[Download PDF] Simon Broadbent (MRS Silver Medal winner, 1993/94) It is argued that it now seems old-fashioned to assume that a brand has a constant share base, independent of its advertising (and other marketing activities). The assumption can be tested, and more often than not movements are found in the base. The paper reports on two attempts to quantify longer term advertising effects.
Good Information - Generals Can't Do Without it. Why Do CEOs Think They Can?[Download PDF] David Cowan (MRS Silver Medal winner, 1994/95) Argues that market research should be given the same importance as military intelligence in the army. This would imply a market research presence from the top level downwards in companies. Other departments all have their own agenda, and a Chief Executive is not short of conflicting advice when a company is in crisis. He needs, and lacks, an independent and objective intelligence source. The servant relationship of market research can lead to two problems: stand-alone use of research within narrowly-defined boundaries, and research which is focused on the brand only, neglecting the wider sector, distribution etc.: examples are given where these practices led to wrong conclusions and disaster. Questions that research should be engaged in (strategic questions) never get asked. If research is given the chance, results can be sensational (the Taco Bell example). Three reasons why research rarely makes a major contribution to growth: the ideas of Strategic Management contain little to do with customers or consumers; although marketers may claim to do this they rarely do; research is tactical only and too low in the hierarchy. Examples of this quoted from management textbooks. Two things are needed: managing directors who understand the central importance of understanding the consumer, and a qualified person to help him (skills needed by this person are sketched).
Messages from the spiral of silence: developing more accurate marketing information in a more uncertain political climate[Download PDF] John Turner and Nick Sparrow (MRS Silver Medal winner, 1995/96) The failure of the polls to forecast the eventual outcome of the 1992 election has created a period of uncertainty for all political parties. This paper attempts to explore this failure and has uncovered the existence of a spiral of silence which has significant implications for party tactics and strategy in the run-up to the next election. The current uncertainty partly emanates from the continued publication of polls which do not adjust for the effect of the spiral of silence. This has led the media to grossly exaggerate Labour's electoral advantage in terms of overall standing. Furthermore it has significantly distorted the messages from the polls in relation to individual areas of policy. The authors explain how closer scrutiny of the messages from the spiral of silence should lead politicians to derive more meaningful messages from the polls for inputs into overall marketing strategies. Drawing on specific examples which include an analysis of the 1993 Conservative Party Conference initiatives on crime, Labour's 1994 Conference and the handling by both main parties of the 1992 General Election, the paper shows how adjustments for the spiral of silence would have led both main parties to adopt somewhat different marketing strategies than they did.
Research in new fields[Download PDF] Graham Mytton (MRS Silver Medal winner, 1996/97) This paper reviews the function of audience research for a world broadcasting service and traces its history since 1936. The work of the current department -International Broadcasting Audience Research (IBAR) - is described and some of the problems and complexities of conducting surveys in different countries and cultures discussed. The end of the Cold War and economic and political liberalisation in many countries have opened up many areas previously closed or very restricted to market research and IBAR's role in pioneering both quantitative survey and qualitative research in new areas is described. The paper illustrates that this has not been without its specific problems. A discussion of harmonisation of research data points to the difficulties - some of which may never be fully dealt with. The author suggests that all kinds of social and market research rely on making generalisations about human behaviour and all generations are, in their way, distortions of reality. The enigma of research is that it seeks to simplify in order to help understanding. However co-operation is well-established and a working party set up under the aegis of the European Broadcasting Union believes that harmonisation in the measurement of audiences for international radio broadcasters is possible. The paper also addresses research on sensitive subjects and looks at the BBC as a brand.
The very different methods used to conduct telephone surveys of the public[Download PDF] Humphrey Taylor (MRS Silver Medal winner, 1997/98) This review of the methods used by 83 leading marketing research firms in 17 countries shows enormous differences in the ways they design and conduct telephone surveys of the public to obtain information about the population. While the biggest differences are between countries, there are also many differences within countries. Indeed, no two firms, of the 8 surveyed, use identical methods. The primary purpose of this review is to provide comparative information and stimulate more discussion about different methodologies, not to criticise the methods used. However, some of these are clearly open to criticism and are hard to defend. Furthermore, the survey suggests that many of these firms have not given much thought to the possible weaknesses of the methods they use or whether they should improve them. Overall, the most striking finding was the complete absence of consensus on almost all aspects of sampling and weighting. Specifically, four out of ten firms describe their methods as 'quota sampling' and 45% as 'probability sampling with weighting'. Half use random digit dialling, half do not. No one method of selecting the individual within the household is
Public Transport: The Role of Mystery Shopping in Investment Decisions[Download PDF] Justin Gutmann and Alan Wilson (MRS Silver Medal winner, 1999) This paper looks at London Underground's use of mystery shopping as an input to investment decisions relating to the improvement of the travel environment. The paper starts by briefly reviewing the literature on mystery shopping, it then considers London Underground and the procedures involved in its mystery shopping activity. The paper discusses the role of mystery shopping scores in London Underground's Value of Improvements Model. This Model helps to determine priorities for the many improvements that can be made to the travel environment. Finally the paper looks at the applicability of London Underground's approach to other service organisations.
Brands - Dead or Alive?[Download PDF] Terry Hanby (MRS Silver Medal winner, 2000) This paper examines the classical conception of brands before discussing changing views in modern and postmodern brand literature. The implications of brands as holisitc entities with many characteristics of living beings lead into a discourse on metaphor and how this relates to both brands and organisations. The author reviews the deep structures (root metaphors) that underpin the two main conceptions of brands that have dominated theory and practice over the last fifty years. It is argued that companies need to be aware of their organisational metaphors in order to optimise their approach to managing brands. Metaphor consonance is important, not only within an organisation but also between client and service companies.
Does internet research work?[Download PDF] Humphrey Taylor (MRS Silver Medal winner, 2001) Internet research is revolutionary, but how well does it work? This article considers the question, and includes a comparison between Internet and telephone research. The Internet is different in several ways: sampling is not probability but `volunteer' or `convenience'; it is a visual medium; it captures the unedited voice of the respondent (open-ended replies have been found to be fuller and richer); it may be more effective when addressing sensitive issues; scales may elicit different response patterns; online surveys may generate more `don't knows'; raw online data substantially underrepresent some groups. There is no list of e-mail addresses of the Internet population, and unsolicited e-mail is frowned upon. Weighting is a major issue. Systems of `propensity' weighting are being developed (propensity to be on line and to reply to surveys). These involve attitudinal, behavioural and demographic variables, the weights being derived by comparisons with other (non-Internet) survey data. Some biases, however, will never be removed by weighting. As in other methods, there are many different types of on-line surveys, and generalisation is dangerous. The main cost is not data collection, but the investment in hardware, software and people needed to draw the samples, send the e-mails and update the database. The advantages of the Internet are that it makes possible: huge samples, accessibility of tiny sub-groups, richer verbatim replies, ability to show lists, still and moving images, and do everything possible on CATI or CAPI, and all this incredibly fast and at affordable cost. Qualitative possibilities are discussed briefly. A comparison of some parallel telephone and Internet surveys in the U.S. indicated where some biases exist and how they might be corrected, and showed the value of `propensity weights' (illustrated).
The Relationship Between Customer and Supplier Perceptions of the Manufacturer's Market Orientation and It's Business Performance[Download PDF] Fred Langerak (MRS Silver Medal winner, 2002) This study is designed to test the relationship between the manufacturer's downstream (upstream) market orientation and the manufacturer's business performance, using self-reports and customer (supplier) reports of the manufacturer's downstream (upstream) market-oriented behaviours. The findings from a sample of 72 matched sets of suppliers, manufacturers and customers in the Netherlands reveal that the manufacturer's downstream (upstream) market orientation has a positive effect on its business performance, regardless of whether self-reports or customer (supplier) reports are used. The results also reveal that customer reports and self-reports of the manufacturer's downstream market-oriented behaviours are consistent. However, the results show a discrepancy between supplier reports and self-reports of the manufacturer's upstream market-oriented efforts. Of interest to marketers is to what extent this discrepancy affects the manufacturer's long-term business performance.
Structuring and Measuring the Size of Business Markets[Download PDF] Phyllis Macfarlane (MRS Silver Medal winner, 2003) A review of business or `industrial' markets research and how it differs from consumer research. Covers: a brief historical background to business research; how to define business markets; business market structures; business universes, sampling and statistics; collecting usage information; response rates and dealing with non-response; standard error calculations.
Developments in outputs from the 2001 Census[Download PDF] Barry Leventhal (MRS Silver Medal winner, 2004) The 2001 Census is starting to provide market researchers with updated information on the size and structure of the UK population. The objective of this paper is to identify the most important methodological changes and developments in the 2001 package that will be relevant when using results from this unique source. Section 2 of the paper gives an overview of the Census operation and goes on to discuss the top-line results and why they have led to revision of the mid-year estimates series from 1982 to 2000. Section 3 highlights a number of innovations in Census outputs and discusses their implications for users of the data.
How much can we predict?[Download PDF] Ben Page (MRS Silver Medal winner, 2005) This paper argues that in considering survey results, researchers need to be more sensitive to the impact of place and demography on responses. By looking at what one might expect for a given type of area, or a given type of respondent, we can reach more intelligent conclusions about our results.
Cluster sampling: a false economy?[Download PDF] Andrew Zelin and Roger Stubbs (MRS Silver Medal winner, 2006) For convenience and to save on fieldwork costs, many random samples involve an element of clustering. This paper seeks to explain how clustering of a sample can have a detrimental effect on its statistical reliability, reducing effective sample size, and how precision can be improved more effectively by increasing the number of clusters rather than increasing the number of respondents per cluster. There is increased pressure among agencies to release results as quickly and inexpensively as possible. In response to this, this paper takes both a methodological and practical ROI-based approach to illustrate that reducing the number of clusters in order to get costs as low as possible for a given sample size may often turn out to be a false economy.
Developing reliable online polls[Download PDF] Nick Sparrow (MRS Silver Medal winner, 2007) Based on their success at predicting the outcome of elections, opinion polls are used by the media, government and the political parties to measure public attitudes to a very wide range of other issues, helping to shape policy proposals and inform debate. Despite their importance within the political process, the media, political parties and pressure groups nevertheless want feedback from opinion polls quickly and cheaply. Large-scale random probability surveys may provide the
The implicit and explicit role of ad memory in ad persuasion: rethinking the hidden persuaders[Download PDF] Alastair Goode (MRS Silver Medal winner, 2008) In 1957 Vance Packard wrote The Hidden Persuaders arguing how ads could persuade at a sub-conscious level. However, since Freud first popularised the concept of the ‘sub-conscious’, psychologists have been advancing the understanding into what the systems underlying sub-conscious processing are and the extent to which it affects behaviour. Cognitive psychologists have focused much of their effort on exploring the differences between ‘explicit’ memory (the conscious recollection of events) as opposed to ‘implicit’ memory (a ‘subconscious’ memory that affects behaviour without the necessity of awareness of prior exposure). Using current knowledge about implicit memory, this paper provides a testable psychological mechanism by which advertising can persuade sub-consciously. A case study is presented that illustrates how ads work at a ‘subconscious’ level and how this understanding led to insight into why creative ads often fail in conventional qualitative research.
Squatting at the digital campfire - researching the open source software community[Download PDF] John Cromie and Michael Ewing (MRS Silver Medal winner, 2009) This paper describes an internet-mediated netnography of the open source software (OSS) community. A brief history of OSS is presented, along with a discussion of the defining characteristics of the phenomenon. A theoretical rationale for the method is then offered and several unique features detailed. The evolution of the methodology in practice is described and salient lessons highlighted. In addition to gathering a large volume of rich data as intended, early phases of the implementation of this method produced a number of unanticipated but significant findings. The paper concludes by summarising the key methodological considerations for conducting a phenomenology of a true online community.
Essence: the structure and dynamics of the brand[Download PDF] Chris Barnham (MRS Silver Medal winner, 2010) The concept of ‘brand essence’ is relatively well established in marketing circles. It has come to the fore as a way for marketers to better understand their brands and also as a benchmark to evaluate brand activities. In some quarters, however, the concept has encountered more resistance. It is seen by many in the creative community as something that oversimplifies the marketing process and limits the power of the brand. The main argument of this paper is that brand essence has been fundamentally misunderstood. This has resulted in a number of negative consequences for the branding process. However, this paper will also show how the concept still has much to offer marketing professionals. A new, and more relevant, interpretation of brand essence is put forward in this paper, which recognises the intrinsically relational and dynamic aspects of the concept. As such, it creates a new platform upon which we can build our understanding of brands.
Statistical alchemy - the misuse of factor scores in linear regression[Download PDF] Cataldo Zuccaro (MRS Silver Medal winner, 2011) Linear regression and factor analysis are probably the most employed statistical techniques in market research. During the last several decades these two techniques have been employed jointly by market researchers in modelling a wide spectrum of behavioural and psychological phenomena. More specifically, market researchers have employed factor scores as predictor (independent) variables to model the ‘variability’ of a variety of constructs (latent variables). Many of these studies can be classified as ‘modified psychometric investigations’ of the link between ‘supposedly latent structures’ and a wide variety of manifest and latent dependent (criterion) variables. Unfortunately, this standard market research practice is inappropriate and can lead to faulty analyses and recommendations; in addition, it is still employed today by academicians and professional market researchers. The objective of this paper is to demonstrate, on both mathematical and ontological grounds, the inappropriateness of the exercise and to recommend more robust practices in attempting to model the relationships between a set of latent variables measured through factor structures.
Associative networks: a new approach to market segmentation[Download PDF] Céline Brandt, Charles Pahud de Mortanges, Christian Bluemelhuber and Allard C.R. van Riel (MRS Silver Medal winner, 2012) This paper aims to expand the domain of brand image perception measurement by providing a method for eliciting brand associative networks and comparing it with traditional brand image measurement methods. This paper then argues that these networks may differ from one individual to another, depending on the cultural background and/or the experience with the brand. Accordingly, the authors introduce a methodology of clustering consumers with similar perceptions into distinct segments, which can be targeted differently. Using picture analysis and metaphor-based elicitation techniques, Lipton’s Ice Tea brand associations are extracted and utilised as an input for the creation of 160 individual associative networks.These networks are first aggregated to measure the brand reputation and subsequently clustered into six segments. This paper provides clear arguments for using associative networks as the preferred method to capture the complete brand image. The paper discusses implications of perceptual segmentation for image management, brand positioning, perceptual competition analysis and brand communication.
Predicting elections: a 'Wisdom of Crowds' approach[Download PDF] Martin Boon (MRS Silver Medal winner, 2013) Opinion polls are the currency of politics. They are used by media organisations to evaluate the performance of governments, and by governments and political parties to test the policies that shape manifestos and reform agendas. But opinion polls all rely on one thing – asking people how they themselves intend to vote – and, too often, classical opinion research techniques fail to confront the issues that underpin inaccuracy. In the UK and in many other countries around the world, their performance over the past 20 years has ranged from excellent to disastrous. The ‘Wisdom of Crowds’ concept turns conventional predictions on their head. It assumes that any crowd that conforms to a core set of principles is capable of delivering a more accurate prediction than the smartest people within it. This paper tests this proposition within the context of actual elections in the UK, showing that the Wisdom of Crowds approach used by ICM Research at the 2010 general election would have produced the most accurate final pre-election prediction. It also shows that a Wisdom approach to regular vote intention tracking produces an interesting complement to classically conducted vote intention polls. Or, if one were to be bold, a competitor to them.
Let their fingers do the talking? Using the Implicit Association Test in market research[Download PDF] Aiden P. Gregg, James Klymowsky, Dominic Owens and Alex Perryman (MRS Silver Medal winner, 2014) [This is a digital first article – it has been published online before it appears in print] Self-report methodologies - such as surveys and interviews - elicit responses that are vulnerable to a number of standard biases. These biases include social desirability, self-deception and a lack of self-insight. However, indirect measures, such as the Implicit Association Test (IAT), offer a potential means of bypassing such biases. Here, we evaluate the scope for using the IAT in market research, drawing on recent empirical findings. We conclude that the IAT meets several desirable criteria: it yields consistent results, possesses predictive power, offers unique advantages, is relevant to commercial issues and poses no insuperable challenges to adoption.
The importance of rank for shorter, smarter surveys[Download PDF] Kyle Findlay, Jan Hofmeyr and Alice Louw (MRS Silver Medal winner, 2015) The traditional market research paradigm believes that, the more data you measure, the more potential for insight the data will hold. However, this paper takes the counter-intuitive standpoint that'‘less is more'. Drawing on the authors' familiarity with neuroscience and behavioural economics, as well as five years' worth of panel behavioural data in three categories and data from 2,769 studies across 1.9 million respondents, the paper argues that, just as it is important to ask the right questions in a survey, it is similarly important to measure 'just enough' but not too much information about brands. We show that measuring too much data is unnecessary and can even be detrimental to the richness of your data. Readers should take away practical guidelines for creating shorter, smarter surveys that still maintain the integrity of their data and perhaps even improve it.