Survey Data Quality: Controls, Detection, Validation and Removal

Drawing on three new guidance documents, the session will cover the full quality process from pre-survey planning and in-survey checks through to post-fieldwork validation, documentation, representativeness review and transparent reporting. It will explore how to use multiple indicators, rather than single pass/fail checks, to make balanced, defensible decisions in a research environment shaped by disengagement, technical fraud, bot activity and AI-generated responses.

Why Attend?

• Take an end-to-end view of quality: Understand how pre-survey decisions, survey design, technical checks, in-survey controls and post-survey validation all contribute to the integrity of the final dataset.
• Move beyond single indicators: Learn why checks such as speeding, straight-lining, trap questions, open-ended quality and consistency tests should be interpreted in context and combined before exclusion decisions are made.
• Apply a practical validation workflow: See how the post-survey checklist supports intake, duplicate removal, automated flags, technical and metadata review, response pattern assessment, open-ended review, composite scoring, documentation and reporting.
• Avoid the risks of over-removal: Explore how overly aggressive exclusion can reduce representativeness, introduce bias, weaken statistical power and create unnecessary cost.
• Report decisions transparently: Understand what to document, how to disclose removal criteria and why sensitivity analysis matters when explaining the impact of quality decisions.

What Will Be Covered?

• Why poor-quality survey data can distort findings, waste resources and reduce confidence in research outcomes
• The three-part quality process: pre-survey, in-survey and post-survey validation
• Practical checks for duplicates, speeding, open-ended response quality, response patterns, honesty and attention checks, and answer consistency
• When to use real-time removal, when to review post-fieldwork, and how to balance efficiency with sample integrity
• How to create a composite quality score and classify responses as clear excludes, borderline cases for review or clear includes
• The importance of documenting thresholds, exclusion decisions, sample loss, representativeness checks and sensitivity analysis
• The Dos and Don’ts for clients when managing data quality challenges

The webinar will also highlight practical lessons from MRS-sponsored research, including how reviewer judgements can differ, why open-ended checks require care, and why the impact of removing flagged records should be tested rather than assumed.

Who Should Attend?

• Research practitioners responsible for survey design, fieldwork, analysis or reporting
• Data quality, data operations, sampling and fieldwork teams
• Sample providers, panels and technology partners involved in fraud detection or participant validation
• Research buyers, procurement teams and client-side insight professionals who need confidence in data quality decisions
• Anyone seeking a practical, transparent and proportionate approach to survey data validation and removal

Speakers

• Chair: Debrah Harding, Managing Director, MRS
  
  
• Presenters and panellists:
  
  
  • Jo Bygrave, Director Data Quality, NielsenIQ and member of the MRS data quality working group
  • Greg Matheson, Co-CEO, Quest Mindshare and Chair of the CRIC Online Research Committee
  • Chris Stevens, consultant and member of the MRS data quality working group
  • John Tabone, CEO, Canadian Research Insights Council (CRIC)

Format

• 60 minutes
• CPD-accredited
• Free for all