Understanding Why High-Quality Research Needs High-Quality Participants

Conducting research into the experience of your users, customers or employees is vital for UX designers. It's an opportunity to enter the mind of your target audience and understand their needs, wants, and motivations.

Yet research is only as good as the participants you recruit – and in an age of increased fraudulent behavior, it's more important than ever to ensure you're working with high-quality participants.

But why does high-quality research demand high-quality participants? Isn't it enough to get a large sample size and hope the majority of responses are genuine?

Unfortunately, it's not that simple. Read on as we explore the reasons why high-quality participants are essential for high-quality research.

Why You Need High-Quality Data

There's no rule that says 'you must source quality data!' In fact, lots of companies attempt to get away with low-quality participants and data. It's cheaper, easier, and faster to do so.

The problem is, research is only as good as the data it produces – and if that data is of poor quality, the entire research project is undermined. 

Instead, you should view the data you collect as an incredibly volatile yet valuable commodity. It's your opportunity to understand your target audience on a deeper level; it allows you to uncover their pains, their motivations, and the underlying reasons for their behavior.

With those insights, you have the power to improve their experience, make better product decisions and deliver real business results. It's the foundation that your entire UX strategy is built on, so it's worth investing time and resources into getting it right.

Use poor data, however, and you'll end up with:

• Incomplete insights. You'll only have part of the picture, so you won't be able to see the full picture and make well-informed decisions.
• Incorrect assumptions. Without accurate data, your assumptions about your users will be inaccurate, leading to poor decision-making.
• Wasted time and resources. If you have to go back and redo research because the data was of poor quality, you've wasted valuable time and money.

Quality data is essential for quality insights – it's as simple as that. Respondent.io is your sure-fire way to access only the most qualified participants at the click of a button (but we’ll get to that later on.)

What Do We Mean By 'High-Quality Participants'?

At first, it may seem insensitive to refer to your participants as 'low-quality' or 'high-quality'. Keep in mind, however, that these distinctions are made based on the data they provide, not on the participants themselves – and these are necessary distinctions to make.

So, what do we mean by 'high-quality participants'? In short, high-quality participants are those who provide you with the most helpful data possible; data that is accurate, relevant, and actionable.

There are a few key characteristics that separate high-quality participants from their low-quality counterparts. Let's take a look at a few of the most important ones.

Willing and able to provide honest feedback.

While we'd like to assume that every participant has our best interests at heart, that's not always the case. ‘Bad participants’ may withhold important information or give inaccurate responses in order to please researchers (perhaps because they want to be seen as more favorable or competent). 

On the other hand, they might be in it for the reward incentive and speed through the testing with random or made-up responses. This is especially common in online research, where it's easy to remain anonymous. 

High-quality participants are those who are willing and able to provide honest feedback – even if it's not what we want to hear. They understand the importance of accurate data and are committed to providing it.

Able to articulate thoughts and feelings clearly.

For qualitative research, in particular, it's crucial to be able to understand what participants are thinking and feeling. This type of data is often more valuable than quantitative data; it can provide insights that numbers alone cannot. 

However, not all participants are equally articulate. Some may have difficulty expressing themselves, either because they're shy or because they don't have the vocabulary to describe their thoughts and feelings. This can make it difficult to understand what they're trying to communicate.

You'll find that high-quality participants in this area are the people who have a vested interest in the success of your product or service, and so will take the time to provide feedback that is clear and helpful.

Representative of your target audience. 

Honest feedback and clear articulation are only helpful coming from the right people; in other words, people who are representative of your target audience. 

It can be tempting to use friends, family, or employees as research participants, but this often leads to what's called confirmation bias or friendship bias. Because these people know and care about you, they're more likely to give positive feedback that isn't necessarily accurate. 

Another common pitfall is to use people who are readily available, rather than those who are actually representative of your target audience. This might mean using people from your same office or neighborhood, even if they don't fit the profile of your typical customer. 

The most useful data can only be collected from people who are representative of your current or desired customer base – so be sure to keep this in mind when recruiting participants.

Free of malicious or negligent intentions.

Unfortunately, not everyone is interested in helping you – some people may have malicious or negligent intentions. 

For example, a competitor might pose as a research participant in order to collect information about your product or service. Someone could deliberately provide false information in an attempt to sabotage your research. 

These situations might seem fictitious, but the world's a competitive place. Be sure to screen participants carefully to ensure that they're not trying to game the system.

Of course, not all malicious or negligent behavior is intentional. Some people may simply be careless with the information they provide, leading to inaccurate data. 

This is why it's important to verify the information that participants provide, either through follow-up questions or by checking it against other sources. 

Aside from these general traits, the best participants are:  

• The embodiment of the people you are trying to understand, from their demographics (age, location, education, income, etc.) to their psychographics (lifestyle, values, personality, interests, etc.).
• Able to provide insights that are actionable and useful for your business. 
• Motivated to help you, whether because they believe in your product or service, or because they want to see the industry as a whole improve. 
• Easy to work with and communicate with; in other words, a low-maintenance participant who will make your life easy.
• Reliable, meaning they will show up on time and follow through on their commitments.

We know what you're thinking – there are a lot of boxes to check. And finding participants who meet all of these criteria can seem daunting, if not impossible. 

But don't despair. While it might take some extra effort, it is possible to find high-quality research participants. 

Respondent.io is designed for researchers who seek only the best. With our platform, you can access a pre-screened, global panel of millions of participants who have been vetted for quality. 

Potential Outcomes of Poor-Quality Participants

It's clear that sourcing high-quality participants has many benefits, such as improved data quality, study validity, and participant engagement. But what if you move forward with lower-quality participants? What negative outcomes should you expect?

Poor data quality

Perhaps most obviously, lower-quality participants will produce poorer data. This is due to a number of factors, including:

• Lack of understanding or poor comprehension of study tasks and materials
• Inability to articulate their thoughts and feelings clearly
• Unwillingness or inability to follow instructions

These problems compound one another, leading to data that is inaccurate, difficult to interpret, and ultimately not useful for informing design decisions. Even if you're able to make some sense of the data, it will be more difficult and time-consuming than if you had high-quality participants.

Study validity

Another potential issue with lower-quality participants is that they can jeopardize the validity of your study. Poor data quality (as discussed above) can lead to invalid conclusions, and participants who do not represent your target audience can also invalidate your results.

For example, imagine you're conducting a study to understand the needs of new parents. If you recruit participants who are not actually parents (or who are not new parents), then your data will be meaningless in terms of understanding the needs of this group.

In addition, if your study tasks require specific skills or knowledge that lower-quality participants lack, this can also weaken the validity of your findings. For example, if you're studying the usability of a financial planning tool, but your participants can't read or do basic math, then you won't be able to get accurate data about the tool's usability.

Finally, keep in mind that even if your study is technically valid (i.e., the data are accurate and representative), it may still be perceived as invalid by others if you recruit lower-quality participants. This is because people are often skeptical of research that relies on self-reported data, and they may be less likely to trust your findings if they know that your participants were not carefully screened.

Participant engagement

In addition to producing poorer data and jeopardizing study validity, lower-quality participants are also more likely to disengage with the study tasks. This can lead to a number of problems, including:

• Incomplete data: If participants disengage before finishing all of the tasks, you'll be left with incomplete data that can't be used.
• Frustration: If participants are unable to complete the tasks or don't understand what they're supposed to do, they may become frustrated and give up altogether. This not only wastes their time, but it also wastes your time (and money) as you recruit and train new participants.
• Boredom: Even if participants are able to complete the tasks, they may become bored and disengaged if the tasks are too easy or repetitive. This can lead to poorer data quality (as discussed above) as well as participants who give up early and don't complete all of the tasks.

It's important to note that even high-quality participants may disengage if the tasks are not interesting or relevant to them. However, lower-quality participants are more likely to have this problem, and it can be compounded by their other issues (such as poor comprehension or unwillingness to follow instructions).

Time and money

Finally, consider the impact that lower-quality participants can have on your schedule and budget – because like it or not, they can actually end up costing you more time and money in the long run.

This is a common pitfall for researchers who are new to conducting user research. They assume that using random participants means cutting costs; after all, you aren't providing any incentives and you're not spending any time screening or recruiting participants.

However, you'll likely need to recruit more participants to compensate for the poor data quality, and you may need to spend more time training and orienting them before each study session. 

If you use lower-quality participants too often, you'll start to develop a reputation for conducting research with subpar participants, which can make it more difficult to recruit high-quality participants in the future. It's simply not worth it to save a few dollars upfront if it ends up costing you more time and money in the long run.

How to Ensure You're Working With High-Quality Participants

With your ideal participant in mind, how can you make sure that you're actually working with high-quality individuals? It may take some time and effort, but the outcome is well worth the grunt work. Here are a few helpful tips you can follow:

1. Define what "high-quality" means for your study.

As we mentioned earlier, the measure of quality is based on what you're looking for in a participant. A study on web design might want participants who frequently use the internet and are familiar with different types of websites, while a study on a new app might want participants who don't have any experience with the app so they can give unbiased feedback.

• Answer these questions to help you determine your ideal participant:
• What age group/s accurately reflect your target market?
• Are you looking for participants with a certain level of education or income?
• What interests/needs must they have to be able to relate to your product/service?
• Are there any other specific requirements that are necessary for your research?

Keep in mind, too, that quality trumps quantity. It's better to have fewer, high-quality participants than a large pool of unqualified individuals – otherwise, you'll just be wasting your time (and money).

2. Use a screening questionnaire.

Questionnaires will help you weed out individuals who do not fit your specific criteria. Be sure to include pertinent questions that will give you an accurate idea of whether someone is suited for your study. 

For example, if you're conducting market research for a new type of toothpaste, you'll want to ask questions about oral hygiene habits and what types of products they currently use.

Here is a list of example questions you could use to screen participants for a web design study:

• How frequently do you use the internet?
• What types of websites do you typically visit? (e.g., social media, retail, news, etc.)
• How would you rate your level of computer literacy? (e.g., beginner, intermediate, expert)
• Have you ever taken part in a usability study before? If so, please describe the experience.
• Are you comfortable providing feedback on web design elements?
• Do you have any visual impairments that might affect your ability to participate in this study?

Researchers can also run the risk of being too specific when defining their target participant. In this case, it's important to remember that you can always adjust your questionnaire – or even your entire study – if you find that you're not getting the quality (or quantity) of responses that you need.

Guide: Build the perfect screener survey for an unmoderated research study

3. Offer an incentive. 

In order to attract high-quality participants, you may need to offer an incentive for their time and trouble. This doesn't mean you need to break the bank, but offering a small token of appreciation will show that you're serious about your research and value their input.

Some common incentives include:

• Monetary compensation (e.g., $10-$20 for completing a 20-minute survey)
• A chance to win a prize (e.g., a $100 gift card)
• A coupon or discount code for your product or service

The majority of user research incentives are monetary, falling between $60 and $150 per hour of participation. Some companies are better suited to coupons, however, especially if they're targeting a specific demographic like students or seniors.

Just be sure that your incentive is appropriate for your target market and won't result in any ethical concerns. Here's a handy calculator that you can use to calculate the appropriate research incentive for your target audience.

4. Set clear expectations.

When you're recruiting participants, it's important to be clear about what your study entails. This includes the time commitment, what they'll be asked to do, and any other relevant details. 

Make sure you clear up these details from the get-go:

• How long will the study take? 
• What format will it take? (e.g., in-person, online, via phone)
• What type of tasks will they be asked to complete? 
• What is the compensation for their time?
• Are there any risks or discomforts involved? 
• What are the confidentiality and privacy protections in place?

If you're upfront about the expectations, you'll be more likely to attract individuals who are actually interested and willing to participate.

5. Follow the ESOMAR guidelines.

ESOMAR is the world's largest market research association, and they have a set of guidelines that help ensure ethical and responsible research practices. When recruiting participants, be sure to adhere to these guidelines in order to maintain a high level of quality.

Some of the key points to keep in mind include:

• Informed consent: informing participants about the study before they agree to take part
• Data protection and confidentiality: ensuring participants' data is kept safe and secure
• Deception: only deceiving participants if it's absolutely necessary for the study

By following these guidelines, you can be sure that your research will be of the highest quality. You'll also avoid any legal or ethical issues that could arise from improper recruiting practices.

6. Actively avoid bias.

Bias is a major problem in all forms of research, and it can have a serious impact on the quality of your data. There are many different types of bias, but some of the most common include self-selection bias, response bias, and recall bias. 

There are steps you can take during the recruitment phase to avoid bias:

• Use a recruitment script to make sure all participants are given the same information
• Use a diverse pool of recruitment sources to reach a wide range of potential participants
• Use quotas to ensure that your sample is representative of the population you're studying

By taking these steps, you can be sure that your data will be as unbiased as possible. This, in turn, will lead to higher-quality research results.

And of course, the most useful tip of them all – work with a reputable recruiting agency. Respondent.io makes it easy to screen and recruit participants that fit your specific research needs. We've got a community of participants who are ready and willing to take part in your study, so you can focus on conducting quality research.

Conclusion

High-quality research demands high-quality participants – there's simply no way around it. By ensuring you're working with the right people, you can be confident in the data you collect and the insights you gain from it.

Interested in working with us at Respondent.io? We’d love to hear from you. Publish your project with us today and recruit from a high-quality pool of over 100k verified professionals!

Recommend Resources:

• The Best UX Research Tools of 2022
• The Best Unmoderated Usability Testing Tools of 2022
• How to Recruit Participants for Qualitative Research (2022 Edition)
• Unmoderated Usability Testing 101: How To Get Started
• How To Get Survey Participants