A foundational skill for researchers is knowing how to screen participants for research. Build the perfect screener survey with these 7 high-impact tips, shared by an expert researcher with over a decade of experience in conducting research.
Respondent recently hosted John Whaley, the Director of Research and Testing at Instrument for a webinar on the topic of creating great screener surveys. Instrument is a digital innovation and experience company serving Fortune 500 clients such as Nike and Google that designs and builds websites, applications, navigation, logos, and other creative work. John and his team of strategists and researchers are responsible for primary research. They design research studies, including screeners, to deliver reliable data to clients, enabling them to improve customer experience, retain users, and expand the company's reach into new markets.
In this article, we summarize the webinar to highlight the actionable advice that was shared to help our researcher community create effective screeners. Learn how John and his team collaborate with stakeholders and clients to develop screener questions, explore screener FAQS, and learn how to create your own screeners for user research.
You need to find the right people. Each client has a specific current customer and prospective customer, a place they want to go. We need to take the entire world of people and get it down to 8 or 10 or 15 that we want to interview. - John Whaley
The purpose of screeners
Screener surveys serve the purpose of identifying the most relevant, articulate and insightful research participants from a (hopefully large) pool of potential research participants, based on demographics, job titles, and/or experiences. The type of screener a particular research project needs depends on the research objectives, methodology and size of the participant pool. A screener for recruiting focus group participants can be very different from the screener needed for an unmoderated study.
It is essential to whittle down the participant pool to select only the best-fitting study participants. John uses the analogy of a funnel, where recruitment involves casting a wide net and slowly narrowing down the pool until there are just a few left.
In the webinar, John noted that clients placed a premium on diversity in participants selected for research. He elaborated that diversity in personas and demographics is increasingly important for researchers, when they seek to fill out a slate of research participants. A research team may want to speak with 5 current and prospective customers, each cohort made up of males and females, at most one student, at most one retiree, and at least one CEO. To select with such precision without compromising on quality, screener design is key.
7 Essential Tips for Screener Survey Design
We’ve curated these 7 vital snippets from the webinar with John, to help you create the perfect screener survey for your research project.
Note: The below videos are all 1 min. or shorter and help you catch up on the essentials, even if you were unable to join the webinar.
- Treat the screener as the start of a relationship
Remember that screeners are the beginning of a relationship with the participant. Question one is the first thing they see from you.
- Meet your participants where they are
Create introductory questions to meet the survey-taker where they are, and bring them where you need them. John says, “Start them cold.” Make no assumptions about any prior knowledge they may have regarding your product or study.
- Start with a white piece of paper. Design from scratch
Templates and reused screener questions are useful, but always start with a clean slate, so that you can put yourself in the shoes of an ideal participant and anticipate their thought processes as you draft the research screener experience. By empathizing with the participant and molding questions that have proved effective in the past, you can craft a screener that is hews closely to first principles.
- Include all stakeholders in the design process
Collaborate with strategists, clients, project managers, etc. as you build the screener. Work collectively on a Google Doc or GitHub, or another iterative content repository where everyone can make changes and comments, and have access to version control, before programming it into your participant recruitment tool or research tool of choice.
- Make sure the screener is the appropriate length
Respect the time of your prospective participants, but do not hold back from being absolutely sure that they fit the demographic, psychographic and experience profiles that you seek. Refrain from being so concise that you don’t collect all the data you need from the study.
You could collect demographic information before the start of the screener to eliminate extra questions.
Make use of skip logic and branching features offered by platforms like Respondent, it allows you to build questions and follow-ups that can tease out nuances, depth of knowledge and articulation by conditional flows to maximize the effectiveness of your screener. John noted that "We tend to program last, It’s the best way to QA our work”. The screener eliminates or accepts potential participants when programmed right.
- Test internally before shipping
Does your screener work as expected? Test every conditional branch and skip logic segment thoroughly before you get potential participants to take the screener, because the last thing you want is for a broken screener that leads to disgruntled candidates, low-quality participants or both.
Excerpts from the Q and A with John Whaley
John answered questions from the webinar guests and the Respondent moderator. Here are some of the most insightful bits from that part of the webinar.
Q: Do you reuse screener questions?
A: Only basic questions like demographics can be reused which can be stored on a document. To design a custom screener, start from a ‘blank piece of paper.’
Q: What do “S1” and “Q1” mean in the example screener? (This was a reference to the example screener that was shared in the full video).
A: “S” stands for screener questions. “S1”= Screener question number 1. “Q1” = The first question from the actual study. No research questions are asked when answering a screener.
Q: How do you prevent fakers from sneaking into your study? (Remember, participants get paid, so some miscreants may have an incentive to lie.)
A: Ask trick questions to make sure the people are not pretending to be who they say they are. For example, on a B2B screener seeking an Android developer, you can ask them to select software they regularly use and include a fake program. If they select it, your logic design will eliminate them.
Q: Can ChatGPT create screeners for us?
A: They can be a decent start, but a strategist and researcher must use their knowledge of the specific context to iterate upon the generated screener draft.
Want more screener support?
We recently developed 2 guides for researchers.
Grab your FREE guide to unmoderated B2B and B2C Screeners
With screeners now addressed, you may be thinking about how to recruit research participants.
How to recruit research study participants
Researchers need to recruit from a pool of participants that match the research requirements of their client or company. Recruiting, vetting, and managing research participant potential is hard work. The Respondent Platform, with 2.6 Million+ vetted participants and counting, handles participant recruitment and management so researchers can focus on their research. Check out a recent blog for more information about how researchers can recruit study participants using a recruitment platform, networking, social media, and more.