Conducting research into the experience of your users, customers or employees is vital for UX designers. It's an opportunity to enter the mind of your target audience and understand their needs, wants, and motivations.
Yet research is only as good as the participants you recruit – and in an age of increased fraudulent behavior, it's more important than ever to ensure you're working with high-quality participants.
But why does high-quality research demand high-quality participants? Isn't it enough to get a large sample size and hope the majority of responses are genuine?
Unfortunately, it's not that simple. Read on as we explore the reasons why high-quality participants are essential for high-quality research.
There's no rule that says 'you must source quality data!' In fact, lots of companies attempt to get away with low-quality participants and data. It's cheaper, easier, and faster to do so.
The problem is, research is only as good as the data it produces – and if that data is of poor quality, the entire research project is undermined.
Instead, you should view the data you collect as an incredibly volatile yet valuable commodity. It's your opportunity to understand your target audience on a deeper level; it allows you to uncover their pains, their motivations, and the underlying reasons for their behavior.
With those insights, you have the power to improve their experience, make better product decisions and deliver real business results. It's the foundation that your entire UX strategy is built on, so it's worth investing time and resources into getting it right.
Use poor data, however, and you'll end up with:
Quality data is essential for quality insights – it's as simple as that. Respondent.io is your sure-fire way to access only the most qualified participants at the click of a button (but we’ll get to that later on.)
At first, it may seem insensitive to refer to your participants as 'low-quality' or 'high-quality'. Keep in mind, however, that these distinctions are made based on the data they provide, not on the participants themselves – and these are necessary distinctions to make.
So, what do we mean by 'high-quality participants'? In short, high-quality participants are those who provide you with the most helpful data possible; data that is accurate, relevant, and actionable.
There are a few key characteristics that separate high-quality participants from their low-quality counterparts. Let's take a look at a few of the most important ones.
While we'd like to assume that every participant has our best interests at heart, that's not always the case. ‘Bad participants’ may withhold important information or give inaccurate responses in order to please researchers (perhaps because they want to be seen as more favorable or competent).
On the other hand, they might be in it for the reward incentive and speed through the testing with random or made-up responses. This is especially common in online research, where it's easy to remain anonymous.
High-quality participants are those who are willing and able to provide honest feedback – even if it's not what we want to hear. They understand the importance of accurate data and are committed to providing it.
For qualitative research, in particular, it's crucial to be able to understand what participants are thinking and feeling. This type of data is often more valuable than quantitative data; it can provide insights that numbers alone cannot.
However, not all participants are equally articulate. Some may have difficulty expressing themselves, either because they're shy or because they don't have the vocabulary to describe their thoughts and feelings. This can make it difficult to understand what they're trying to communicate.
You'll find that high-quality participants in this area are the people who have a vested interest in the success of your product or service, and so will take the time to provide feedback that is clear and helpful.
Honest feedback and clear articulation are only helpful coming from the right people; in other words, people who are representative of your target audience.
It can be tempting to use friends, family, or employees as research participants, but this often leads to what's called confirmation bias or friendship bias. Because these people know and care about you, they're more likely to give positive feedback that isn't necessarily accurate.
Another common pitfall is to use people who are readily available, rather than those who are actually representative of your target audience. This might mean using people from your same office or neighborhood, even if they don't fit the profile of your typical customer.
The most useful data can only be collected from people who are representative of your current or desired customer base – so be sure to keep this in mind when recruiting participants.
Unfortunately, not everyone is interested in helping you – some people may have malicious or negligent intentions.
For example, a competitor might pose as a research participant in order to collect information about your product or service. Someone could deliberately provide false information in an attempt to sabotage your research.
These situations might seem fictitious, but the world's a competitive place. Be sure to screen participants carefully to ensure that they're not trying to game the system.
Of course, not all malicious or negligent behavior is intentional. Some people may simply be careless with the information they provide, leading to inaccurate data.
This is why it's important to verify the information that participants provide, either through follow-up questions or by checking it against other sources.
Aside from these general traits, the best participants are:
We know what you're thinking – there are a lot of boxes to check. And finding participants who meet all of these criteria can seem daunting, if not impossible.
But don't despair. While it might take some extra effort, it is possible to find high-quality research participants.
Respondent.io is designed for researchers who seek only the best. With our platform, you can access a pre-screened, global panel of millions of participants who have been vetted for quality.
It's clear that sourcing high-quality participants has many benefits, such as improved data quality, study validity, and participant engagement. But what if you move forward with lower-quality participants? What negative outcomes should you expect?
Perhaps most obviously, lower-quality participants will produce poorer data. This is due to a number of factors, including:
These problems compound one another, leading to data that is inaccurate, difficult to interpret, and ultimately not useful for informing design decisions. Even if you're able to make some sense of the data, it will be more difficult and time-consuming than if you had high-quality participants.
Another potential issue with lower-quality participants is that they can jeopardize the validity of your study. Poor data quality (as discussed above) can lead to invalid conclusions, and participants who do not represent your target audience can also invalidate your results.
For example, imagine you're conducting a study to understand the needs of new parents. If you recruit participants who are not actually parents (or who are not new parents), then your data will be meaningless in terms of understanding the needs of this group.
In addition, if your study tasks require specific skills or knowledge that lower-quality participants lack, this can also weaken the validity of your findings. For example, if you're studying the usability of a financial planning tool, but your participants can't read or do basic math, then you won't be able to get accurate data about the tool's usability.
Finally, keep in mind that even if your study is technically valid (i.e., the data are accurate and representative), it may still be perceived as invalid by others if you recruit lower-quality participants. This is because people are often skeptical of research that relies on self-reported data, and they may be less likely to trust your findings if they know that your participants were not carefully screened.
In addition to producing poorer data and jeopardizing study validity, lower-quality participants are also more likely to disengage with the study tasks. This can lead to a number of problems, including:
It's important to note that even high-quality participants may disengage if the tasks are not interesting or relevant to them. However, lower-quality participants are more likely to have this problem, and it can be compounded by their other issues (such as poor comprehension or unwillingness to follow instructions).
Finally, consider the impact that lower-quality participants can have on your schedule and budget – because like it or not, they can actually end up costing you more time and money in the long run.
This is a common pitfall for researchers who are new to conducting user research. They assume that using random participants means cutting costs; after all, you aren't providing any incentives and you're not spending any time screening or recruiting participants.
However, you'll likely need to recruit more participants to compensate for the poor data quality, and you may need to spend more time training and orienting them before each study session.
If you use lower-quality participants too often, you'll start to develop a reputation for conducting research with subpar participants, which can make it more difficult to recruit high-quality participants in the future. It's simply not worth it to save a few dollars upfront if it ends up costing you more time and money in the long run.
With your ideal participant in mind, how can you make sure that you're actually working with high-quality individuals? It may take some time and effort, but the outcome is well worth the grunt work. Here are a few helpful tips you can follow:
As we mentioned earlier, the measure of quality is based on what you're looking for in a participant. A study on web design might want participants who frequently use the internet and are familiar with different types of websites, while a study on a new app might want participants who don't have any experience with the app so they can give unbiased feedback.
Keep in mind, too, that quality trumps quantity. It's better to have fewer, high-quality participants than a large pool of unqualified individuals – otherwise, you'll just be wasting your time (and money).
Questionnaires will help you weed out individuals who do not fit your specific criteria. Be sure to include pertinent questions that will give you an accurate idea of whether someone is suited for your study.
For example, if you're conducting market research for a new type of toothpaste, you'll want to ask questions about oral hygiene habits and what types of products they currently use.
Here is a list of example questions you could use to screen participants for a web design study:
Researchers can also run the risk of being too specific when defining their target participant. In this case, it's important to remember that you can always adjust your questionnaire – or even your entire study – if you find that you're not getting the quality (or quantity) of responses that you need.
Guide: Build the perfect screener survey for an unmoderated research study
In order to attract high-quality participants, you may need to offer an incentive for their time and trouble. This doesn't mean you need to break the bank, but offering a small token of appreciation will show that you're serious about your research and value their input.
Some common incentives include:
The majority of user research incentives are monetary, falling between $60 and $150 per hour of participation. Some companies are better suited to coupons, however, especially if they're targeting a specific demographic like students or seniors.
Just be sure that your incentive is appropriate for your target market and won't result in any ethical concerns. Here's a handy calculator that you can use to calculate the appropriate research incentive for your target audience.
When you're recruiting participants, it's important to be clear about what your study entails. This includes the time commitment, what they'll be asked to do, and any other relevant details.
Make sure you clear up these details from the get-go:
If you're upfront about the expectations, you'll be more likely to attract individuals who are actually interested and willing to participate.
ESOMAR is the world's largest market research association, and they have a set of guidelines that help ensure ethical and responsible research practices. When recruiting participants, be sure to adhere to these guidelines in order to maintain a high level of quality.
Some of the key points to keep in mind include:
By following these guidelines, you can be sure that your research will be of the highest quality. You'll also avoid any legal or ethical issues that could arise from improper recruiting practices.
Bias is a major problem in all forms of research, and it can have a serious impact on the quality of your data. There are many different types of bias, but some of the most common include self-selection bias, response bias, and recall bias.
There are steps you can take during the recruitment phase to avoid bias:
By taking these steps, you can be sure that your data will be as unbiased as possible. This, in turn, will lead to higher-quality research results.
And of course, the most useful tip of them all – work with a reputable recruiting agency. Respondent.io makes it easy to screen and recruit participants that fit your specific research needs. We've got a community of participants who are ready and willing to take part in your study, so you can focus on conducting quality research.
High-quality research demands high-quality participants – there's simply no way around it. By ensuring you're working with the right people, you can be confident in the data you collect and the insights you gain from it.
Interested in working with us at Respondent.io? We’d love to hear from you. Publish your project with us today and recruit from a high-quality pool of over 100k verified professionals!
Recommend Resources: