Since forming in March 2021, the MRS Representation in Research group has been talking to research practitioners across the UK about the issue of participant representation in UK research projects. More specifically, how we can improve representation of groups that are often underrepresented. Over the course of this process, we have developed the following FAQs.

Except for references to MRS Code of Conduct, this FAQ resource is advisory. It aims to offer clarification and resources to help research practitioners improve representation within their research projects. Occasionally the FAQs may link to external sites with relevant resources. In these instances, MRS is not responsible for any such content.

Our understanding of issues of equality, diversity and inclusion will continue to grow, and we may update and change some of the following advice particularly with regards to terminology as this is constantly evolving. The FAQs will remain under regular review. If you have any feedback on these FAQs or any questions that you would like to see included that do not currently feature in this guide, please email these to rebecca@cobalt-sky.com.


Supporting information

Designing research that is inclusive can be dependent on many factors: what groups are you trying to reach that may have barriers to your research that you are trying to address, what sort of research are you conducting and through what methodology, the topic of your research, what market are you conducting your research in etc. In some cases, research practitioners may need to go into depth of detail and some of the FAQs in this guide will be specific to certain research projects and not others. However, when thinking about Inclusive Research, there is a “principles based” approach that can be applied to all projects.

Clifford Lewis, an academic and expert in conducting research with marginalised and/or potentially vulnerable groups, has offered the following key principles of Inclusive Research:

Inclusive research approaches are designed with the aim of making participants feel comfortable, able, and empowered to participate despite their unique circumstances. The following principles are put forward:

  1. The research is designed by researchers who understand their own bias: We are all a product of society and are raised believing that characteristics of the majority (race, ability, sexual and/or gender orientation, linguistic ability, etc) is the norm. Our biases force us to think and act in certain ways partly influenced by our conscious or unconscious prejudices, and by limited understanding of diverse identities that depart from the majority. Reflecting on the biases we hold about diverse identities can help make us become more consciously aware of our predispositions, enabling us to be vigilant about how they might influence our thinking, the design of research, how data is analyzed, and the inferences drawn from it. The aim is to identify your blind stops and take action to reduce them.
  2. The research should avoid making assumptions about the participants: Both while working on a project’s design, and once the project and any data collection instruments (discussion protocols, questionnaires, etc) have been designed, it is important to step back and ask what assumptions have been made about the participants. That is what assumptions are we making about the participants and their ability to contribute based on how the project has been designed. This can range from assumptions about the participant’s gender to assumptions about their language capabilities, and physical abilities. Assumptions may also be made about the participant’s ability and comfort in visiting a place for in-person interviews. The aim is to ensure a nationally representative population would be able to participate in the study.
  3. The project is explicitly designed to ensure diversity at all stages from recruitment to the design of research instruments: Inclusive research tries to seek people from varied backgrounds and is constructed in a way that demonstrates respect for diverse individuals and avoids use of an aggregated classification of ‘other’. This is relevant in how recruitment is conducted – that is: were all efforts made to include diversity? It is also relevant in the language used when designing research instruments – that is, does the language make participants comfortable, shows respect, and avoids othering people for being diverse. The aim is to ensure the diversity is actively considered and respected in all stages of the research.
  4. The research should be designed in consultation with those who have a lived experience: Without a shared lived experience, it can be easy to ignore some components of that experience. The best way to ensure the project considers diverse identities and how those identities impact the individual’s lifestyle is to consult with people from diverse backgrounds. Co-creating the project, or getting feedback on it, the language used, and the instruments applied can provide valuable input in making the project sensitive to different lived experiences.

The MRS Representation in Research group has been established to explore the participant pools that currently engage in UK research projects and whether they accurately reflect the wider population that UK research seeks to represent.

Our profession is underpinned by statistical and social science principles, enabling us to gather information and make verifiable claims about the populations we research. This can only be achieved if we include accurate and robust proportions for key sub-samples of the target audiences we claim to represent. To be considered valid, either quantitatively or qualitatively, any given study should carefully consider, recruit, and reflect the distribution of the target population of interest.  

Our aim is to ensure that research claiming to be nationally representative (“Nat Rep”) of the UK population or a clearly defined sub-sample of the population, e.g., adults aged 16 years and over, takes into consideration its diversity and is inclusive of all groups within the population. More specifically, our aim is that research declared as Nat Rep accurately reflects nationwide representation across ethnicity, gender identity, sexual orientation, and physical disability and/or mental health conditions.

The perspectives of those in minority groups can be underrepresented, meaning they may not be widely heard or understood, and thus do not help inform decision-making that impacts them. Moreover, even when they are heard, the narrative is often in danger of being focused on subgroups or formulated without a clear understanding or respect of their uniqueness. As such, this can lead to misunderstandings and the perpetuation of restrictive, and potentially harmful, stereotypes. Our sector plays an important role in creating visibility of the experiences, needs and perspectives of these groups. Not only is this powerful in and of itself, but it also has positive commercial and policy implications for research buyers, enabling them to understand and serve diverse audiences better.  

Traditionally when the term “Nat Rep” is used for research carried out in the UK, characteristics that are included as standard are age, gender, region, and social grade. 

MRS’ current best practice recommendation is that the characteristics required to be genuinely representative of the current UK population, and therefore truly representative, are the following: age, gender identity, region, social grade, ethnicity, sexual orientation, and physical disability and/or mental health conditions. The release of the 2021 census data will provide national data on all these characteristics.

When research projects are undertaken with a Nat Rep sample requirement, all stakeholders should have complete clarity and transparency on which characteristics are to be used within the specific research methodology being deployed to define samples as nationally representative. The characteristics used to define Nat Rep must be reported as required by the MRS Code of Conduct reporting requirements.

Samples claiming to be nationally representative can be obtained via either random (or probability) or non-probability sampling methodologies. Whilst random probability sampling relies on random participant selection, non-probability sampling is carried out by setting quotas that reflect the population to ensure the resultant sample is representative. 

MRS' Inclusion Pledge is an important initiative calling on leaders in the UK research profession to demonstrate their commitment towards creating safer and more representative workplaces. The Inclusion Pledge is driven by the MRS Diversity Inclusion and Equality (DI&E) Council, who undertake projects and activities to help guide our sector on issues of representation, equality and accountability within the sector itself. 

The MRS Representation in Research group has been established to explore the participant pools that currently engage in UK research projects and whether they accurately reflect the wider population groups that UK research seeks to represent. Unlike the DI&E Council, and the Inclusion Pledge, this topic is focused solely on participants of research, as opposed to internal sector representation. 

If you would like to get in touch regarding the Inclusion Pledge, or any other issue around diversity, inclusion and equality within the UK research sector please get in touch with MRS here.

If you would like to get in touch regarding issues around representation of participants of UK research, please get in touch with the Representation in Research group here.

All MRS Members and Company Partners are obliged to comply with the MRS Code of Conduct, and should ensure they are up to date with changes and requirements. 

There are several elements of the MRS Code of Conduct that are particularly relevant to representation in research, in particular rules 9, 11 and 28 plus the reporting rules 57, 58 and 59. The 2022 update to the MRS Code will include explicit reference to the transparency requirements specifying which characteristics have been used when providing or reporting ‘Nationally Representative’, ‘City Representative’ or other specified sub-samples of the population.

There is a distinction between the MRS Code of Conduct and best practice guides or advice. Whilst the MRS Code is binding on MRS members and MRS Company Partners, best practices or advice are considered guidance.  

Except for references to the MRS Code of Conduct considerations, this FAQ resource is advisory, and aims to offer clarification and resources in improving the representation of research participants. It will remain under regular review to ensure it is as current and relevant as possible. 

The Representation in Research group was formed by MRS in response to strong demand for information, advice, and investigation over whether the current understanding of Nat Rep is fit for purpose. 

Earlier in the year a piece of Qualitative research was undertaken on behalf of this group to explore how agencies, panel providers and clients are approaching this topic. The findings confirmed the demand for a more inclusive definition of Nat Rep across all cohorts whilst also helping the group to recognise and identify barriers and challenges that the sector will need support to overcome. An executive summary of this qualitative research can be found here

In January 2022 the group commenced a follow-up research project seeking to quantify the demand for amending the definition of “Nat Rep” amongst research practitioners. The findings of this report will be discussed at an MRS webinar, details for which are available here. One of the headline findings from the report was that out of 50 MRS company partners companies surveyed, 80% stated that they thought it would be beneficial to expand the definition of Nat Rep to include ethnicity, confirming the demand to re-examine the current approach to Nat Rep.

Professionals need to be adopting best practice now, clients are already contacting MRS to ask for guidance and training in this area. MRS Senior Client Council, which comprises some of the most senior research clients in the UK, is following progress closely.

We recommend that details around representation in any research project is built into briefs, proposals, and costing conversations as standard. Best practice dictates that when research projects are commissioned with a Nationally Representative (also known as Nat Rep) sample requirement, for example, that all parties have complete clarity on which characteristics are included in this definition. To demonstrate commitment to this, the best way is to ensure complete transparency on which characteristics are used to define any sample as Nationally Representative, together with acceptable levels for quotas and weighting parameters and full clarification regarding metrics and methodologies. This will also ensure that MRS Company Partners and MRS members are adhering to the MRS Code of Conduct update coming into effect in 2022 which will require clear inclusion of which characteristics have been used when providing or reporting ‘Nationally Representative’, ‘City Representative’ or other specified sub-samples of the population.

Do not forget that representation is not just about quotas – it’s about the ‘opportunity to respond’. Demonstrate that you have awareness of this at every stage; research design, participant recruitment, participant facing documents and sample sources should all be reviewed with accessibility and inclusion in mind. For help with Key Principles of Inclusive Research please see What are the key principles of Inclusive Research?

Whether you are conducting research in-house or purchasing through a supplier, there are a number of approaches you can take to ensure representativeness.

  1. Leverage the sampling and methodology guidelines built on behalf of the Senior Client Council, available here. These documents provide client-side practitioners with some guidance on inclusive sampling and data collection, as well as an overview of how different methodologies can be utilised to make research more inclusive and accessible
  2. When working with suppliers:
  3. Aim to work with suppliers who have signed the MRS Inclusion Pledge. More information on the Pledge, what it is and who has signed up to it already, can be found on the MRS website here. (If your preferred suppliers have not yet signed up to the MRS Inclusion Pledge why not ask them to sign up!)
  4. Ask your suppliers to conform to the guidelines issued by MRS which provide the appropriate legal and ethical best practice on the following:
  1. Increase your focus on including D&I within your roster reviews and research tenders/request for proposals, both as business and also within the adopted research methods

For any further information on Diversity & Inclusion within research please refer to the section on the MRS website here.

The responsibility for defining the characteristics for Nat Rep samples is shared between clients, agencies and suppliers and should be discussed during the project design stage.

The MRS Code  of Conduct has two rules which are relevant to this issue:

Rule 11. Members must take reasonable steps to design projects to the specification and/or quality standards agreed with clients.

Rule 28. Members must take reasonable action when undertaking data collection to ensure all of the following: 

  1. a) that data collection processes are fit for purpose and clients have been advised accordingly; 
  2. b) that the design and content of data collection processes are appropriate for the audience being analysed; 

Please note there are XX clauses in Rule 28. For the topic of representation, clauses a) and b) are the most relevant; refer to the full Code for the comprehensive list of rules and clauses.

MRS is updating the Code of Conduct to incorporate transparency requirements about Nat Rep samples, including the requirement to report sampling parameters demonstrating the application of rules 58 and 59 for samples:

Rule 58. Members must ensure that data include sufficient technical information to enable reasonable assessment of the validity of results. 

Rule 59. Members must ensure that reports include sufficient information to enable reasonable assessment of the validity of results.

This MRS Code of Conduct will require clear inclusion of which characteristics have been used when providing or reporting ‘Nationally Representative’, ‘City Representative’ samples, or other specified sub-samples of the population. This update is due to come into effect in 2022.

The above are examples of MRS Code of Conduct considerations, and these are mandatory for MRS members and MRS Company Partners. MRS’ best practice recommendation is that the characteristics required for genuine representation of the current UK population, and therefore truly Nat Rep, are the following: age, gender identity, region, social grade, ethnicity, sexual orientation and physical disability and/or mental health conditions.

Research practitioners should design research to the specification agreed with clients. When clients request a Nat Rep sample, suppliers (field suppliers, panel companies, or full-service agencies) should discuss with the client what characteristics will meet requirements, bearing in mind and advising the client of all relevant considerations as indicated above.

Best practice and other resources

 

MRS has produced Best Practice Guides for asking for Ethnicity, Sexual Orientation, Physical disability and/or mental health conditions and Gender Identity.

There is no one size fits all question and response approach to designing such questions and collecting data on these characteristics, and the MRS guides give several potential examples, alongside considerations that should be made when asking these questions. Practitioners can use the examples given in the guides, or their own, provided there is adherence to MRS Code of Conduct (covered within in the guides).

MRS guidance will develop as understanding and circumstances change.

MRS is initially focussing only on the UK. For other markets, it is important to understand what questions are either legally or socially acceptable to ask. This is an area which is continuously developing: we recommend consulting with experts in the field and/or the relevant national associations in the countries within which you are carrying out the research. To find the relevant national association you can use this GRBN link (this visual includes hyperlinks to individual national associations built into each colour block of the regional federations) or the ESOMAR association search which is available here.  

The Global Research Business Network (GRBN) has initiated a global project on diversity and inclusion supported by MRS as one of the founders of the GRBN.   The results from the GRBN’s 2021 Diversity, Equality and Inclusion in the Market Research Sector research are available here. Going forward it is hoped that this initiative will result in more global resources being available. 

When trying to determine the level of acceptability of asking for extended gender or sexual orientation, it may also be useful to use resources such as the ILGA’s (International Lesbian, Gay, Bisexual, Trans, and intersex Association) assessment of the global legislation upon same-sex acts within specific countries. A link to this can be found here. Alternatively, the Human Dignity Trust have a similar resource here.

There is currently no specific MRS guidance on analysing and interpreting these characteristics.  MRS is producing new guidance, and these will evolve as understanding of the issues develops.  In 2022 the MRS is producing a Special Issue of its academic journal, IJMR, which will specifically look at inclusion and researching vulnerable groups, and this will include methodological considerations and approaches.

The Commission on Race and Ethnic Disparities Report (also known as the Sewell Report) contains some useful points to consider when undertaking research with ethnic minority groups, as does the “Leaving no one behind. How can we be more inclusive in our data?” Recommendations Report from the Inclusive Data Taskforce, released in July 2021. Representation in Research group member Sania Haq, a specialist in intercultural research, has produced this report which contains four core areas for practitioners to focus on.

Clifford Lewis and Nina Reynolds offer some tips on conducting research with LGBTQIA+ communities in a Research Live article which can be found here. The full IJMR paper referenced in the article “Considerations for conducting sensitive research with the LGBTQIA+ communities” can be found here.

Commercial impact/ROI

 

Addressing the issue of diversity and inclusivity in sampling is not something which is motivated by commercial advantage. Undertaking this initiative reflects the desire to deliver more representative samples of the population. MRS best practice recommendation is that all relevant characteristics are required for Nat Rep samples to be inclusive of the population e.g., age, gender identity, social grade, region, ethnicity, physical disability and/or mental health conditions and sexual orientation. This is not linked to whether a commercial advantage has been proven. 


Although commercial advantage is not a requirement, the Representation in Research group has undertaken a project that will investigate whether there is a commercial or insight benefit to a more inclusive version of Nat Rep compared to the traditional understanding of Nat Rep. More details about this project and target timings can be found here.

In terms of commercial advantage for end clients, there have been many reports detailing the potential spending power of minority groups. More than a billion people, equating to 15% of the world’s population, have physical disabilities (WHO, 2020), making them the biggest minority group in the world. Globally they hold a disposable income of approximately £1 trillion (PurplePound, 2018). Pre-Covid-19, in the UK alone failure to appropriately cater to customers with physical disabilities in the marketplace witnessed monthly losses of £163 million for restaurants, pubs and clubs, £267 million for high street shops, and a mammoth £501 million for supermarkets (PurplePound, 2019). Other minority groups are likely being kept equally on the fringe of insight and decision making, and brands are missing an opportunity by not making sure that the opinions and needs of those groups are being heard.

GDPR/DPA

 

Within the UK GDPR some personal data is categorised as ‘special category data’ and is subject to additional requirements when being collected.

Personal data categorised as special category data is data on:

  • religious or philosophical beliefs
  • health
  • racial or ethnic origin
  • trade union membership
  • political beliefs
  • sex life or sexual orientation
  • genetic data
  • biometric data (when used for the purpose of uniquely identifying a natural person)

The presumption is that special category data needs to be treated with greater care because collecting and using it is more likely to interfere with these fundamental rights or open someone up to discrimination. This is part of the risk-based approach of the UK GDPR.

Researchers collecting and processing special category data as well as personal data will need to have a legal basis for all categories of data being processed.  When processing special category data practitioners must have a lawful basis under Article 6 of the GDPR in addition to meeting a special condition under Article 9 of the GDPR but these grounds do not have to be linked. The MRS GDPR in Brief No.10 - Collection of Ethnic Data and Other Special Category Data provides guidance on what you need to do to meet these requirements and checklists to help to identify what you need to do to collect sensitive special category data in accordance with GDPR.

It is important to note that there is no restriction on processing any special category data as long as the requirements of Article 6 and Article 9 are being met and practitioners fully document what and how they collect special category data

Question format and Design (screeners, questionnaires, and discussion guides)

 

Based on the most recent reliable information, in 2011, 7.7% of the population (4.2 million people) of England and Wales has a main language other than English.  1.3% of the population reported that they could not speak English well and 0.3% reported that they could not speak English at all.

7.4% of the population (380,000) of Scotland has a main language other than English.

3.1% of the population (55,000) of Northern Ireland has a main language other than English.

(Sources: 2011 Census.  www.ons.gov.uk for England and Wales, www.scotlandscensus.gov.uk for Scotland and www.ninis2.nisra.gov.uk for Northern Ireland:

As with most questions in this area, looking at more granularities can be useful. For example, after White British, people from a Black Caribbean background were the ethnic group most likely to speak English as their main language whereas people from a Bangladeshi ethnic background were the group most likely to not speak English well, or at all. Intersectionality should also be considered: - for example, 3 in 5 of those who identified as not speaking English well were female (Pakistani and Bangladeshi women were 5 times more likely than their male counterparts to speak no English at all).

Research practitioners may need to make provisions for non-English speakers, and when doing so may need to landscape intersectionality considerations to be able to make decisions on exactly which other languages should be included.

Depending on how the initial contact with the participant is made and the method of data collection, various initiatives can be employed to ensure non-English speakers can complete the research, including providing online and postal research information in several languages, and utilising multilingual interviewers for telephone or in-person interviewing.

Care should be taken when designing data collection questions for sensitive data. The MRS Code of Conduct states that special care must be taken when the nature of a project is sensitive or the circumstances under which the data is collected might cause a participant to become upset or disturbed (rule 33).

There is no single way in which inclusivity questions can be asked or answered. The MRS Code of Conduct sets out the considerations when collecting data, for example rule 28: 

 

Rule 28. Members must take reasonable action when undertaking data collection to ensure all of the following: 

  1. a) that data collection processes are fit for purpose and clients have been advised accordingly; 
  2. b) that the design and content of data collection processes are appropriate for the audience being analysed; 
  3. c) that participants are able to provide information in a way that reflects the view they want to express, including don’t know/prefer not to say; 
  4. d) that participants are not led toward a particular point of view; 
  5. e) that responses and/or data collected are capable of being interpreted in an unambiguous way; 
  6. f) that any potential use of the personal data is revealed; 
  7. g) that personal data collected and/or processed is limited to what is relevant; and 
  8. h) that personal data is stored and transmitted by secure means and only accessible to authorised individuals. 

 

Clauses a) to e) are particularly relevant when collecting sensitive data.  It is therefore important to consider design issues such as:

 

Potentially providing response options in alphabetical order: For example, not having the Male option at the top of a gender response list. Alphabetised answer lists may mitigate any perceived superiority stigma which people may subconsciously have for the response first given. Due consideration should be given to the actual or potential effects of any given order on data collection.

Provide options to self-describe: this allows participants to define their identity in their own words as this could be a sensitive topic: For example offering an open ended option to the question below:  

Which sexual orientation category best represents you?

  • Bisexual
  • Pansexual
  • Straight/Heterosexual
  • Gay man
  • Gay Woman / Lesbian
  • Prefer to self-describe ____________________
  • Prefer not to say

Allows participants to describe themselves without narrowing them to a predefined list. This also ensures that Rule 28 clause c), d) and e) is being met.

Always include a “Prefer not to say” option:  this ensures that participants don’t have to answer questions they don’t feel comfortable with.  This also ensures that Rule 28 clause c) is being met.

Remind participants that they have the right to withdraw or terminate any data collection: this should provide participants reassurance that they are in control of the process.  This also is an MRS Code requirement (rule 34).

Include a clear introduction to sensitive questions: explain why the information is being requested, how it will be used, and convey to participants the steps that will be taken to ensure the privacy and confidentiality of their responses

Consider the language that is being used: use language that supports different identities as opposed to one that is based on the binary (for example heterosexual – not-heterosexual should be avoided; the ‘not’ acts to position LGBT people as not part of the mainstream.). Consider moving away from using “Other/Alternative/Another way” as these can suggest a difference from the “norm”.  

For interviewer-administered interviews: Consider providing a letter (A, B, C, etc.) to label each discreet answer and ask the participant read out a letter(s) rather than the answer(s)

CAPI interviewing:  Consider handing the CAPI unit to the participant to enter the answer(s) to sensitive questions themselves instead of giving a verbal response

You can find MRS Best Practice guides which provide examples of ways to ask these questions here.

Any organisation producing documents or materials should have measures in place to ensure that Inclusion and Diversity is given sufficient consideration when such materials are being produced. 

Qualitative screeners and quantitative questionnaires effectively act as ‘gatekeepers’ for any research being conducted, and it is at this stage that practitioners managing projects have the opportunity to actively review who is being included / excluded, intentionally or unintentionally, in any proposed research design. 

It is important that such issues are considered from the very outset of any project to ensure that the broadest possible cross-section of participants can fully take part in research, regardless of their personal characteristics or living situation, e.g., living with a disability, low English language proficiency, technology proficiency, etc.

Ideally practitioners undertaking research should have designated experts who are well-versed in producing research materials and who keep up to date with any developments in this area. This should be further augmented by training programs for all research staff so that everyone understands the issues at play when addressing this topic.

MRS provides inclusion in research training:

MRS is also developing a new module for the Recruiter Accreditation Scheme (RAS) covering diversity and inclusion recruitment.  The RAS training materials are freely available, and the new module will be available from the RAS website later in 2022.

This is a challenging area, as some approaches to questionnaire design (such as gamification) can be in direct opposition to the requirements of people with physical disabilities. Furthermore, there are many kinds of physical disabilities and mental health conditions, and there is no single approach to questionnaire design which is appropriate for all. There are several principles however that might be useful to research practitioners to consider, depending on the applicability and the research methods being used:

  • Provide alternative text for in-survey pictures
  • Enable auto-subtitling for videos.
  • Standard advice for good survey design becomes more important when considering people with physical disabilities and/or mental health conditions:
    • keep questions short and direct
    • make sure that questions are easy to read in terms of font size and colour contrast
    • make sure responses are easy to identify and select
    • keep interview lengths short.
  • If the views of a subset of people with physical disabilities and/or mental health conditions is particularly important, design the survey around their needs, rather than expecting them to complete a questionnaire of standard design. This might include switching data collection method or survey approach.
  • If the survey topic is one on which people with specific physical disabilities and/or mental health conditions might reasonably be expected to have different views or experiences than others, this should always be referenced in reporting/recommendation, even if (especially if) the survey has not be designed to accommodate their needs.

In addition to the above Christine Hemphill from Open Inclusion offers the following advice:

“The main over-riding advice would be to consider all of the research journey for participants with different impairments, needs or adaptive approaches. 

Consider how people learn about the research, the screener and selection process, how you communicate with them about the research, the actual engagement approach you are using for the research, and then having sufficient knowledge of the context of their responses to make sure the analysis and insight reflects their meaning. 

Take each part of the journey and consider how to make it work well for someone who moves, senses, thinks or feels significantly differently. Otherwise, they may be disabled from fully, safely or confidently completing the research. If you don’t have much experience in disability-inclusive research (even if you do and you are doing something a new) ask those with the lived experience of disability to help. 

Start with the basics. Check that all technologies being used are accessible (ask providers if they can confirm that they are at minimum WCAG 2.1 AA compliant and preferably also user tested with disabled people). For in person research ensure that spaces have step free access, accessible washrooms and a quiet space for anyone who may need it. Provide choice / options around communication. Offer captions as standard, sign language as required and make sure all people are easily visible and well-lit for those who lip read. Then make all information easy and understandable. This helps everyone, from those who learn differently to those who are operating in a second language, or who are tired or stressed. 

There is a lot to consider. You won’t get it all right from the start. It’s a journey. Continually consider how participants (and researchers) differ and work to understand what is causing barriers or friction so that you can find ways around them. Keep asking and learning and you will keep progressing.“

Quotas, Targets, and Weighting

 

Quotas for inclusive ‘Nat Rep’

For our research to truly represent the national population, i.e. be ‘Nat Rep’, we must ensure that our achieved sample reflects its distribution. Historically, this has required quotas on age, gender, social grade and region, with these being interlocked where possible.

There are however other characteristics which should be considered when defining the population e.g., physical disability and/or mental health conditions, sexual orientation, gender identity and ethnicity. These characteristics now account for a considerable percentage of the UK population and should be included within quota requests to ensure a more inclusive and nationally representative sampling approach.

The current challenges involved in achieving robust samples of minority groups mean that having quotas at a total level for each minority group is seen as an adequate starting point, i.e. quotas do not need to be interlocked with other variables. Given that there are often considerable demographic nuances within minority groups, in future it is hoped that the research sector will be able to evolve this stance as sample accessibility improves.

Context about population data sources

The Office for National Statistics (ONS) has been collecting data on minority groups for many years, meaning that there are established datasets to help inform research design and sample structure.

There are, however, gaps in these datasets and this should be taken into consideration when selecting sources for sample quota purposes. More specifically:

  • Capturing in-depth data on minority groups in the UK can be time/cost intensive. As such, it is infrequent and so the most effective sources may be a few years old
  • There are inconsistencies in how and which variables are captured across population datasets

The Census

The Census provides the most comprehensive and wide-ranging data on minority groups across health, disability, sexual orientation, gender identity and ethnicity. As such, it is the most effective source to use for setting quotas for inclusive ‘Nat Rep’ research.

Detailed information is available at a total level for each sub-group, allowing correct proportions within samples to be reflective of the national population. For example, ensuring that X% of your overall sample is from X minority group. As mentioned above, quotas for minority groups at this overall level (i.e. not interlocking) are currently considered sufficient for achieving an inclusive ‘Nat Rep’ sample.

To achieve a more nuanced or advanced distribution of minority groups within samples, there is additional information from the Census to inform this and/or set interlocking quotas. For example, making sure samples for a given minority group is reflective of specific distributions across key variables, such as age and location in the UK.

Limitations of the Census

The Census is infrequent (every 10 years) and so data is not as up to date as other population sources. For example, the ONS’s Mid-year Population Estimates are often used for traditional ‘Nat Rep’ quotas, given that they are produced on a yearly basis. However, the main focus of the dataset is on tracking changes in age, sex and location, with little relevant information available about minority groups.

There are also inconsistencies in the Census. Most notably that the England and Wales Census is conducted together, while other countries in the UK do so separately. You will therefore need to refer to the dataset for the relevant UK nation(s).

Census 2021 data is coming soon…

Phase 1 of the release will be available as of Summer 2022. This will include data by age and gender, with in-depth data on health, disability, sexual orientation, gender identity and ethnicity to follow.

Recruitment, Sampling, and Screening

 

At the planning and design stage consideration should be given to why additional demographic data is being requested and what it is going to be used for. 

One of the key principles of the Data Protection Act 2018 and the UK GDPR is the data minimisation.  This requires that personal data being processed is:

  • Adequate – sufficient to properly fulfil the stated purpose
  • Relevant – has a rational link to that purpose
  • Limited to what is necessary – data is limited to what is needed for that purpose

Collecting data such as ethnicity, sexual orientation, physical disabilities and/or mental health conditions, and extended gender for samples to be national representative meets the data minimisation requirements of adequacy and relevance.  However, there is also a need to limit data collected to what is necessary for the purposes of any given project, so you need to consider the level of details that is collected.   The detail required should be appropriate, proportionate, and balanced.  Whilst collecting inclusive data is important, participants also should be protected from harm and have the right to a private life. 

There may be some projects where having samples which are nationally representative against the extended characteristics of ethnicity, sexual orientation, physical disabilities and/or mental health conditions is not needed due to the nature of the exercise, or the topics being researched.  If this is the case, and noting the data minimisation requirements, the full set of additional characteristics would not need to be asked.  Equally it might be determined that some apply e.g., ethnicity and health conditions, and in such cases only those characteristics required would need to be asked.

Miscellaneous/Other

 

Technology can play an important role in removing barriers to research, particularly for disabled people. Research practitioners can start by understanding the different technologies available to support people with either physical disabilities and/or mental health conditions e.g., visual and audio technologies to help with sensory impairments, AI and digital technologies for mobility, cognitive, intellectual, and learning impairments. The use of QR codes and accessible documents via word, pdf, etc. can allow disabled persons to use the accessible systems installed on their smart devices or computers.

In the future, digital home systems like Alexa and Google could bring revolutionary new ways of being able to allow disabled people to participate in research. Experience has shown that many disabled people have found their worlds were opened more widely with the invention of computers and in particular smart technology. As such, understanding the best ways to use this (within the context of disability) will permit better representation and participation in research.

Christine Hemphill from Open Inclusion offers the following advice:

“Technology can be very flexible both in its design and in its use to make it accessible and usable to people with a very wide range of disabilities to both conduct and engage with research.” 

Before digital enablement, someone with sight loss would find it hard to fill in a paper survey. Someone with a speech impediment or hearing loss may find it hard to participate in a focus group discussion and a wheelchair user may not be able to get into a research space or use the facilities. On the other side of the table, disabled people may have found it difficult to work in research organisations that didn’t have inclusive and accessible practices, internal and external engagement approaches.

Technology can support the full engagement of these individuals in the research, either as participants or practitioners. It reduces the reliance on participants who can travel more easily, removes the obstacle of inaccessible research spaces and allows support services like sign language interpreters and captioning to be easily booked and brought in virtually from a potentially much wider geographic range. 

As long as varying participant needs are considered and catered to in the design of the research, disabled people can engage safely, confidently and valuably, adapting the technologies as they need to do so. As researchers we need to ensure that we don’t accidentally diminish the representation or equivalent input and influence of disabled people in our insights. To do this we need to think about the technologies that we are using. Is a survey built on an accessible platform and in such a way that blind screen reader user or someone who magnifies the content significantly can engage with? Did we check it with someone who uses these technologies as a native user would?  For the focus group did we hire a STTR reporter to allow live captioning of the session or use an AI based capability to do so? If the person is a sign language user, did we hire a sign language interpreter for the online session? Did we consider how someone with a speech impediment may be able to communicate by writing ideas down rapidly using predictive text then have someone supporting them who can speak on their behalf from the notes? 

If you are unsure what technology solutions work for your participants or make it more difficult, just ask them. 

This all takes a bit more time and care in the research design and planning stage. The tech is simply the enabler. It can be highly adaptive and provide a better experience to participants with and without disabilities when well-designed. For example, a screen reader friendly survey with consistent heading structure makes it easy to scan through, in-room captioning for a focus group session makes it easy for anyone who lost their place for a moment or who didn’t quite catch that last word.   

In short, tech is good if we ensure that it has been designed with accessibility in mind for all who may need or wish to use it. It is much more adaptable to individual needs than the physical environment.”

 

Get the latest MRS news

Our newsletters cover the latest MRS events, policy updates and research news.