Research Buyers Guide

The authoritative guide to research suppliers. Go

Research Jobfinder

The No.1 jobs resource for research and insight professionals. Go

The definitive source of research news and opinion. Go

Fair Data

The Fair Data mark helps consumers recognise who they can trust. Go

International Journal of Market Research (IJMR)

The world authority on research methodologies and techniques for professionals and academics. Go

Geodemographics Knowledge Base (GKB)

For people interested in the application of geodemographics and geo-spatial analysis. Go



[Digital first] means papers that have not yet appeared in print. See all

Peter Mouncey Blog

Integration: turning data into knowledge


This is the big challenge for so many organisations, in the public and private sectors, as the stream some of us lived with in the 1980s became a river as we moved into the 21st century, and is now a raging torrent with the advent of social media and mobile technologies. It’s been called ‘infobesity’ for good reason.

Wrestling with just one of the tributaries provides data analysts (or are they now data scientists?) with enough headaches, but integrating these tributaries into one flow of data, and making sense of it all, represents the supreme challenge. 

A recent IBM international survey of 4,000 senior executives (‘The Customer Activated Enterprise’, IBM 2013) identified that whilst 82% saw ‘integrate information across the enterprise’ as a very important need within their organisation, only 24% felt this was being effectively achieved – a massive gap of 58%!

So, the new report from the Institute of Practitioners in Advertising in the UK, ‘Data Integration Explained’ (see for details), is very timely. It is written by Steve Wilcox, and Edited by Ken Baker – who both have a wealth of experience in this field – and draws on the integrations that underpin the IPA TouchPoints Channel Planner dataset. 

What they describe is of course much more complex than simply developing matchkeys to combine data about a customer from various sources to create a ‘single customer view’ as is common in direct marketing. 

Coming from the IPA, an emphasis on media planning is to be expected, but a chapter covering wider examples, under the heading of big data, widens the value of the report.

Overall, the report seeks to provide readers with a practical guide to the key methodologies, and how to avoid some of the pitfalls. At its heart is an overview of the eight main data integration techniques, including clearly written summaries of the main pros and cons for each of method. 

The methods start with the most simplistic, cross analysing a single demographic group in each independent survey, and finishes with hub surveys for multi-media fusions (as in TouchPoints). This is followed by a more in-depth description of the main modelling techniques and fusion techniques. Writing a brief and selecting an appropriate integrator are covered, with comprehensive checklists for initiating an integration project, plus a guide to evaluating the outputs. 

Thorough planning and post evaluation are vital to success. A summary is included of how the TouchPoints Channel Planner is created, using 10 sources, including the IPA TouchPoints Hub Survey at its core. 

Finally, the report contains a very useful bibliography, which underlines the role played over the years by IJMR papers on this field of research, including ‘Data fusion: an appraisal and experimental evaluation’, Ken Baker, Paul Harris & John O’Brien (JMRS Vol. 30, No.1, 1997) based on the fusion of two TGI datasets funded by the Market Research Development Fund and conducted in 1989. 

Re-reading this paper, and comparing it with the TouchPoints Channel Planner, demonstrates how far integration methods have evolved over the intervening years.

Despite this being a very practically orientated guide, as Baker states, ‘Data integration is a complex subject’, and I’m tempted to warn ‘don’t try this at home’! 

The report is packed with good advice, and is extremely helpful in providing a layman’s guide to the topic, but then hand it over to your favourite data scientist (who, of course, will be a Fox – see earlier blog).

Comments (0)

No Comments Added

Leave a reply

Please enter the 3 red symbols only