To hear John Whaley tell it, he “kind of fell into” the market research industry after graduating from college with degrees in English and Psychology. Seventeen years later he has wiped the dust off and is now the Director of Research and Testing at Instrument, fully integrated digital agency that works with the biggest names in business.
Everyone has to start somewhere, and Whaley is no exception. His first job was with YouGov, a market research firm based in the UK. He worked mostly on quantitative research, which accounted for about 75% of the company’s projects. YouGov has a research participant panel numbering in the millions, so finding the right participants for each project wasn’t hard with the right screening survey. During his 10 years at YouGov, Whaley learned the ins and outs of business research, and honed his ability to design good surveys and screeners—to use data to narrow a respondent pool to exactly those whose input he wanted.
“YouGov is where I cut my teeth and learned how business research actually works,” Whaley says. “It’s where the methods became real.”
In 2018, Whaley took a job at Instrument, which had a well-established strategy discipline but no dedicated primary researchers. In fact, when he was hired, his job title was “Strategist”, simply because it was the title that most closely approximated his skillset. His task: found and grow an internal research discipline that could stand beside the designers, strategists, developers, and producers whose roots already grew deep.
Instrument was founded in Portland, Oregon in 2005, and saw a sustained expansion during the pandemic as companies jockeyed to establish and improve their digital presence. Companies call on Instrument for web and app design, strategy, as well as brand, product, and creative work. Its client list reads like a who’s who of the corporate world: Google, Nike, Twitter, Marriott, Netflix and Levi’s, to name a few. Though the agency’s work is primarily business-to-companies (B2C), they do a fair amount of business-to-business (B2B) projects, as well.
To manage this clientele Instrument’s, now a virtual-first organization with 400 employees, operates as a set of independent-but-interconnected teams, each delivering to its own set of clients. Whaley’s research and testing team, however, is centralized, and works with each of the client teams. Currently the “RaTs”—a self-imposed nickname referring to the fact that they do Research And Testing– have worked on engagements for more than 30 clients over the last four years. Some 70-80% of those projects involve qualitative research, while the remaining 20-30% are quantitative.
“There’s no replacing the qualitative research that we need because a lot of our research questions revolve around the ‘Why?’ and ‘How?’” Whaley said. “We need that depth in a majority of our studies to understand how we can improve that little 5-10% at the end.”
Unlike YouGov, Instrument doesn’t have its own participant panel that its researchers can turn to, so when Whaley joined, one of his first tasks was to determine where research participants would come from. Because the research for the projects Instrument conducts routinely rely on quick-turn, one-on-one interviews, Whaley had to find a way to locate high quality respondents quickly. He needed to make sure that the research team didn’t get in the way of expedient project delivery.
“[At Instrument],” we’re creating new designs, we’re creating new products, we’re creating new experiences. We need to build them, test them, iterate and keep going,” Whaley says. “So we as the researchers can’t slow things down too much.”
Whaley’s research at Instrument occasionally requires trickier B2B professional recruits, like developers and IT decision makers at Enterprise-sized companies. To locate them, Whaley turned to some of the existing platforms for research participants.
“We tested out a few platforms and quickly realized that Respondent was getting us good people fast,” he said. “That’s frankly all we need.”
Respondent worked so well for Whaley and the team at Instrument, in fact, that their use shot up exponentially since the first year. In the second year alone, Whaley’s team used Respondent 13 times more than it had the first year. The ease and speed of finding exactly the right participants for its projects was clutch for a team that places a premium on getting it right the first time.
It’s easy to see why recruiting quickly and easily is so crucial to Instrument’s process—first and foremost, the research and testing team strives to embed itself seamlessly into Instrument’s client work.
“What we don’t want to be is the brakes,” Whaley said. “We’re always in the middle of a creative process, making our clients a… new navigation structure, or an entirely new brand, or whatever the project calls for. The researcher’s job is to fit inside of that creative flow without being like, ‘Hold up, we have to run some interviews.’”
To accomplish this, Whaley dispatches team members to client-facing teams for anywhere from a few weeks to many months. The researcher takes ownership of the entire research process: defining goals and methodology, writing the screener, screening and recruiting participants, writing the moderation guide, and helping to prepare the prototype, and integrating with the entire team throughout. At the end of the engagement, insights are packaged inside the deliverables offered to the client - whether they are strategic, creative, or fully developed code - everything is nested into one cohesive offering.
The process has proven to be effective. Clients respond well to it, and all Instrument’s disciplines appreciate knowing that their work has been user-vetted. And as the research team has grown, the share of projects it can touch has expanded along with it.
Instrument’s Research & Testing team might be called a differentiator for Instrument, as many agencies don’t employ in-house researchers; the research world can therefore feel very detached from the people who are building products and brands. Whaley has proven to be a proselytizer of sorts for the role of research, educating more company roles on the power it can provide and showing what they can learn from it.
Summed up in one slogan, research like Whaley’s is about giving the end user the loudest voice in the room. When strategists, creatives, and executives sit down to make decisions, end users aren’t literally represented—unless research has skillfully recorded and amplified their voices.
“[Through research], the person who will eventually use your [product] is able to express themselves. They’re there, they have a seat at the table,” Whaley said. “That is our job: to bring them into the metaphorical conference room.”
After 17 years in the business, Whaley finds that he lives his life as a moderator. In his day-to-day he catches himself trying to ask only open-ended questions and letting other people do the talking. He maintains a broad curiosity about people in general. These days, most screens in his home (outside of 9-5) are dark due to the birth of his first child, so he stays culturally plugged in by listening to podcasts. This both keeps his brain busy and circles back around to his work, too—podcast hosts are essentially moderators.
Asked to offer advice to other Respondent users, Whaley had this to say:
And finally, make sure your screening questions are top-notch.
“Even inside of a good sample pool, your participants are still only as good as your screener questions. Good participants don’t screen themselves.”
Words to live by for a researcher.