The Future of Insights: Where are all the Humans?
03 April 2024

Report from the RANZ and APRC conference 2024

The Future of Insights: Where are all the Humans?

Debrah Harding, Managing Director, MRS

On 18^th March 2024 the Research Association New Zealand (RANZ) held its conference in partnership with the Asia Pacific Research Committee (APRC) which represents the national research associations across the Asia Pacific region. The conference theme was The Future of Insights: Where are all the Humans?, and the event was expertly chaired by RANZ Board member Geoff Lowe.

The event kicked off with a keynote speech from Nik Samoylov, Founder at Conjointly, questioning the validity and reliability of synthetic data in research.  Nik’s argument was based upon the misuse of synthetic data which can result in bias, lack of disclosure, sloppy analysis and over-reliance.  Nik questioned the use of phrases such as ‘digital twins’ for describing synthetic data, arguing that research methods must be reliable and valid, and highlighting that synthetic data results can vary depending on issues such as prompts used to create the data.  Nik noted that whilst there are some benefits with the use of synthetic data, such as idea generation and placebo effects when actual results are not applicable, synthetic data should be used with care and only for limited purposes.

I followed Nik’s session with a presentation on AI and Research: Ethics vs the Machine setting out why ethics are essential when using AI in research.  The presentation highlighted some of society’s concerns about AI technology but also some of the early research which indicates that AI technologies can be beneficial as research tools. The presentation concluded with the work MRS has done in establishing ethical guidance in this area (see: MRS Guidance on using AI and related technologies).

Last of the morning’s main stage speakers was Chris Barry, CEO of The Evolved Group who presented a session, Travels of a Research Brief in the new AI Insights Economy.  Within this presentation Chris focused on the disruptive change of AI and related technologies, identifying this as the ‘Kodak’ moment of disruption for research, and the need for the research sector to start scenario planning if less research is needed in the value supply chain and the research briefs stop coming.  Chris stressed the need for the sector to redefine the areas where research and insight can add value; highlighting that nothing is going to overcome for the need for teamwork, including collaborative human-AI workflows.

After the morning break the conference divided into two streams.  First up I selected Vince Galvin, Chief Methodologist from Stats NZ, who detailed how AI and Machine Learning is being used for official statistics in the public sector in New Zealand including estimating the likely behaviour of citizens.  Vince also looked to the future for other uses, particularly Generative AI tools such as Large Language Models for responsive prompting in surveys and how the tools might be used to supplement Nationally Representative approaches.

Next was Jonathan Pickup, Group Client Director at Kantar and Yasmin Handrich, Group Account Director at Kantar who presented the very entertaining, The Great Insights Showdown – Team Human vs Team Machine.  This session, which won the People’s Choice award at the conference, compared a project using the standard quantitative research process with the same project using AI to generate the results. 

Whilst the ‘human’ approach demonstrated local knowledge, understanding of subtleties and context with coherent routing and filtering, the AI did produce some nice touches although the questions were simple, somewhat clumsy and sometimes felt very US-centric.  Moreover, the synthetic data used to create the sample had a tendency towards positive bias and AI produced hallucinations within the data sets, which increased each time the LLM was prompted about the results. The speakers’ view was that current AI tools are about three years away from being smart enough to produce the nuances needed to be valuable research and insight tools. Even then, practitioners will still be needed to be the ‘human insights consultants’ which make sense of the data.

The last stream session before lunch was Petra Baker, Insights Manager from Research First, who presented a valuable reminder about the limitations of AI, with a session focused on the decolonisation and indigenisation perspective.  This session, which won Best Paper, highlighted cultures which are actively reclaiming and centring indigenous ways of knowing and being, based around oral traditions which are very different from the Western university/academia approach.  Petra stressed that “…all knowledge is not taught in the same school…” and that AI is like a school that teaches from a narrow curriculum and does not recognise other curriculums such as those of indigenous cultures and peoples.  Whilst technology businesses are a seeking more non-Western sources of data to enrich their AI tools, how this is achieved ethically with consent and in consultation with indigenous groups remains a significant challenge.

For a change of pace after lunch, Winifred Henderson, an Educator and Community Development Manager for Dementia NZ, focused on the need for humans to protect their health in the evolving human insights industry, particularly with over 200 causes of dementia. 

Kevin Miniter, from Consider.it, took a different angle and looked at how AI can be used to harness environmental “voices’ to better inform human opinions.  Kevin focused specifically on a number of environmental locations within New Zealand which have been given legal person status, ‘personhood’, and how AI has been used to provide a voice for these ‘persons’ as part of the environmental fight for greater protections.

The last stream session I attended was Andrew Cannon, the Executive Director of the Global Research Business Network (GRBN), who presented the 2024 results from the GRBN’s Trust Survey and how companies providing AI tools were significantly less trusted than market research companies. With only 17% trusting AI companies compared to 30% trusting market research companies, with concerns about AI driven largely by security issues. Andrew highlighted that whilst AI can increase trust in market research by removing human errors, unconscious bias and enhancing data quality, there is also the potential for the use of AI in research to decrease trust by misrepresenting complex human behaviours, introducing bias and enhancing privacy concerns. 

As the conference moved back to the main stage, Kathryn Topp, CEO and Founder at Yabble, presented a counterpoint to the opening session with Synthetic data: the future of data driven decision making. Kathryn made the case for the use of synthetic data, stating that disruption from within is the most powerful form of innovation; and that AI-generated data has arrived and will be a huge part of the future of the sector.  From Kathryn’s perspective synthetic data, forming ‘large knowledge lakes’, eliminates privacy concerns, addresses sample quality issues, is scaleable, cost effective and provides rich, accurate and robust insights. 

The final main stage presentation was a great PechaKucha session with four fabulous speakers giving diverse presentations on topics such as loneliness and neurodiversity.

The day closed with Geoff Lowe chairing a panel with me, Nik Samoylov, Kathryn Topp and Vince Galvin bringing together the themes of the day: battling out the issues surrounding the pros and cons of AI, synthetic data and the like.  The conclusion from the panel, and indeed the day, was that AI is already here and it is here to stay, but humans will be needed to provide the value of good quality research and insight; success will be for those that can combine AI and the human in a meaningful, ethical and accessible way.