UX Diary Study: 5 Essential Tips For Conducing Better Studies

Everything you need to know for conducting a user research diary study, the best method for producing powerful, accurate, useful insights.

Jose Gallegos
Sep 20, 2022

In the realm of user experience, there are many ways to go about understanding how users interact with a product or service. We can observe users in a lab setting or track their interactions through analytics tools. But there’s another method that is often overlooked: The diary study.

Just as the name suggests, a diary study involves participants keeping a diary of their thoughts and experiences while using a product or service. It's almost like a personal journal, but with specific prompts to help researchers understand how users interact with a product.

Diary studies can be a great way to get insights into user behavior that wouldn’t be possible through other methods. They provide insights into real behavior in real time as well as over time. They can also be more engaging for participants and help build a rapport between researchers and participants.

Let's take a closer look at how you can conduct a successful diary study for in-depth insights into user behavior.

What is a UX Diary Study Exactly?

Also referred to as a camera study, a diary study is a qualitative research method where participants are asked to document their behaviors and experiences and their thoughts and feelings about those behaviors and experiences over time. 

Diary studies are an excellent research method to explore use of a product or service. Unlike focus groups or interviews, a key benefit of diary studies is that they allow researchers to in reality track user behavior and attitudes over a specified period of time versus a participant attempting to think back and recall their behavior and attitudes or be requested to artificially complete tasks.

In the same way that a personal diary gives deep insights into a person's thoughts and feelings, diary studies can provide valuable insights into how users interact with a product or service and how they feel about those interactions. 

By asking participants to document their actions and thoughts in a journal-like format, researchers can gain an understanding of how users interact with a product on a daily basis, as well as pinpoint any problems or frustrations they may experience.

There are different methods researchers can use to gather data from diary study participants:

  • Open-ended: Participants are asked to document their thoughts and feelings about using a product or service in an open-ended format. This allows researchers to gain a broad understanding of how participants use and feel about the product.
  • Questionnaires or Question Prompts: Participants are asked to answer specific questions about their use of the product or service, which can help researchers identify areas of concern. Often participants are asked to perform specific tasks or actions and post their actions, reactions and assessments of those tasks. 
  • Check-in calls: Researchers contact participants at regular intervals to ask them about their diary entries, and to get feedback on how they're using the product or service.
  • Review, Respond and Re-question: If diary studies are done electronically, either by video diary or written, researchers can review daily entries and respond with additional probes and questions specific to those posts.

While some might view the diary study as an unstructured way to conduct research, done correctly, a diary study can provide valuable insights into authentic user behavior and attitudes. 

Who Should Use a Diary Study?

Diary studies are highly useful tools, but they won't suit every need. They're best for understanding how people use a product on their own, in their natural environment. If you want to understand how users interact with your product in a lab setting, or if you need precise quantitative data, diary studies may not be the best option.

The main uses for these studies include:

  • Observing the habits of your user base
  • Discovering pain points in the user experience, 
  • Exploring users’ attitudes towards your product while using it
  • Understanding the situations in which your product is used
  • Gaining insights into the journey of a user, from discovery to abandonment
  • Spotting trends in user behavior
  • Exploring usability, affinity for and ideas for improvement as features are built and added to a platform in-progress, prior to launch

Examples of when a dairy study is an appropriate research method are:

  • Imagine you are trying to decide on a product update for your food delivery app. You could use a diary study to see whether users are neglecting a certain feature, or whether they're struggling with a certain part of the process.
  • You’re a web designer working on a new e-commerce site. Your research goal is to understand how users interact with the site's search bar and navigation menus. You want to learn what types of searches users conduct, where they click first, and how they interact with the results. You'll also want to know the average time users spend on the site.

When You Should Use a Different UX Method

Keep in mind that a diary study is designed to collect qualitative and longitudinal (over time) information and insights. If you're looking for something more specific or if you need to quantify your results, consider a more appropriate research method.

Some factors that could lead you to choose an alternative to a diary study include:

  • The product is used in a lab setting, rather than in the user's natural environment
  • The product is used for a short period of time or is only used once
  • The product is used by a very small number of people
  • You need precise quantitative data
  • The product is used by very different types of users

An example of when a diary study is not the best research method is: You are redesigning a registration process for a new website. You're interested only in understanding how the sign-up process works for a limited number of users.

For this example, you might want to use a different UX method as It is only focused on one element of the website, and not the entire user journey – so a diary study would be unnecessary.

Conducting a Successful Diary Study

Diary studies are perhaps one of the most participant-friendly forms of user research. Unlike other research methods that require in-person sessions or long surveys, diary studies can be conducted through a simple questionnaire or open-ended form that users complete independently.

The benefits of diary studies are clear: they allow researchers to capture authentic user behavior in their natural environment, over an extended time period. 

However, there's still the potential to waste time on this method if you don't plan it out effectively. So, where should you begin?

1. Understand Your Research Question and Scope

This is a crucial step to cover, and it will also help you decide which type of UX test is the best method for your needs. Ask the following questions to clarify your research question and your scope:

  • What is your research goal?
  • What are you trying to learn from users?
  • What behaviors do you want to observe?
  • Who are the users you want to study?
  • What type of data do you need?
  • What's your budget?

After answering each question carefully, you'll be able to develop a plan for your diary study. 

2. Decide On Your Study Duration

Diary studies can go for days, weeks, months even – whichever length works best for your goal or research question and for your budget. 

Of course, the longer the duration of your diary study the more expensive it becomes—higher incentives will need to be paid to participants, more time will need to be spent managing participants, adding questions, prompts and tasks. 

3. Choose Your Format

Your next step is to decide the format of your diary study. There are two main types of diary studies:

  • Questionnaire-based: This type uses a structured questionnaire that users complete on their own time. The questions are usually a mix of open-ended and closed, and they usually focus on specific behaviors or goals.
  • Open-ended form: This type is a simple form that users complete on their own time. It's less structured than a questionnaire, and it allows users to write about anything they want related to their experience using the product.

Both formats have their pros and cons, so you'll need to decide which is best for your research question and scope. For example, if you want to focus on a specific behaviors, a questionnaire-based study is the better option. But if you're interested in getting a broad picture of user behavior and the thoughts and emotions behind those behaviors, an open-ended form is likely the better choice.

Another factor to consider is the collection platform. Do you want to use paper forms or electronic (online) forms? A pen-and-paper approach is more cost-effective, but you won’t always be able to guarantee legibility and it will be difficult and time-consuming to make sure participants are engaged and active in their entries. 

On the other hand, while more costly, electronic collection has many benefits: 

  • It can shorten the time it takes you to collate and analyze responses.
  • You can review responses throughout the study, easily assuring participants are engaged and posting.
  • Since you can review diary posts you can dig deeper with additional questions and probes based on the content of the posts.
  • You can be confident that your study is progressing successfully because you can see and analyze the posts ongoing versus having to rely on phone call, text or email updates from your participants.

     

4. Create Your Questionnaire or Form

Your third step involves crafting the diary itself. This can be done in a few ways:

  • If you're using a questionnaire-based study, you'll need to write the questions yourself. Make sure they're clear and concise, and that they cover all the behaviors or goals you want to observe.
  • If you're using an open-ended form, you'll need to come up with a list of topics for users to write about. These could be general topics like "How was your overall experience?" or specific questions related to your research goals.
  • You may also choose to have no prompts at all, and simply ask that users write about their experience with the product. This option can be less structured, but it can also produce more varied and interesting results.

Creating a Questionnaire-Based Study

If you are writing a question-based diary study, remember that your questions should never be too prescriptive or leading. You want users to answer them honestly, without feeling like they're being led towards a specific response.

Here are some tips for writing effective questions:

  • Be clear and concise.
  • Make sure the scope of your questions covers all the behaviors or goals you want to observe.
  • Primarily use open-ended questions
  • Avoid leading or suggestive questions.
  • Make sure the questions are easy to understand based on the background of your participant base.

Here are some example questions that might be used in a diary study: 

  • Describe the tasks you completed today.
  • Which, if any, of the tasks were difficult to complete? 
  • Describe what caused the difficulties for each task that was difficult to complete.
  • What task(s), if any, did you feel you should be able to complete on the platform but were unable to? Why?

The questions are clear and concise. They cover not only what tasks were attempted and completed but also spotlight tasks that were difficult to complete and why as well as what couldn’t be completed.  

Creating an Open-Ended Form

With an open-ended form, your goal is to gain as many insights as possible from users. These can be used to answer general questions about user behavior or to get a deeper understanding of specific behaviors or goals.

Here are some tips for creating an effective open-ended form:

  • Make sure the form is easy to complete and understand.
  • Include a variety of topics for users to write about.
  • Avoid leading questions or topics.
  • Provide enough structure to facilitate, but not direct, the user’s answers.

For instance, this topic might be included in an open-ended form:

"What are your thoughts on the visual design of the website, consider different pages/parts of the site, likes, dislikes and why?"

While the question is non-leading and allows users to write about their thoughts it also encourages participants to offer enough information to be valuable.

5.  Choose Your Logging Style

Before handing the diary study out to users, you’ll need to decide exactly how you want them to post. You’ve got a few options here. 

Option A: Interval-Contingent Protocol

An interval-contingent protocol asks users to complete diary entries at fixed time intervals, rather than after specific events. For example, you might ask participants to complete an entry every morning, afternoon, and evening. This type of protocol is generally used for studies that are less time-sensitive, or when the researcher wants general or broad insights.

Option B: Signal-Contingent Protocol

You might opt for a signal approach, which asks users to complete diary entries only after specific events. If you send a text message to a participant’s phone, for example, you could ask them to post in their diary in response to the text. Signal-contingent protocols are generally used when researchers want responses to specific questions or explore specific user reactions or behaviors.

Option 3: Event-Contingent Protocol

The third option is the event-contingent protocol in which users complete diary entries after specific events, such as using a new feature or visiting a particular website. You would consider this option if you want to learn about users’ experiences in a specific context.

Once you’ve decided on the type of protocol, it’s important to create clear instructions for participants. Tell them exactly when and how they should complete diary entries. This will help to ensure that your data is accurate and reliable.

6. Distribute the Questionnaire or Form

You've created your questionnaire or form, so now it's time to distribute it to your users. There are a few ways to do this:

  • Email the questionnaire or form to users.
  • Post the questionnaire or form on a website.
  • Send the questionnaire or form as a link in an email.
  • Put the questionnaire or form in a physical location, like a waiting room.

When distributing your questionnaire or form, make sure to include clear instructions on how to complete it. Also, be sure to remind users that they can take their time in completing the form, and that there is no rush – the last thing you want is hurried answers that aren’t accurate.

How To Recruit Your Participants For UX Diary Study

Let's step back for a minute and think about who you would like to participate in your diary study. The people you select will have a big impact on the success of your study – from the quality of the information you collect to how much work you have to do in the follow-up phase.

Ideally, your participants should be:

  • Representative of your target audience. Aim to recruit participants who match your target user profile in terms of demographics (age, gender, occupation, etc.), as well as in terms of their technology experience and how they use your product.
  • Engaged users. Find people who are already using your product – they're more likely to be willing to spend time recording their thoughts and feelings about using the product.
  • Willing to provide feedback. Make sure your participants are open to providing feedback and want to help improve the product. This is especially important if you're looking for comments on specific design issues.
  • Available for follow-up. You'll need to contact your participants both during and after the study is over to collect their diary data and get feedback on the process. Choose people who are easy to reach and willing to answer your questions.

Now that you know who you're looking for, how do you go about recruiting them? One way is to post a recruitment message on your company's website or blog, or on social media sites like Twitter and Facebook. You can also use traditional research recruiters. 

Another option is to reach out to your existing user base. If you have a mailing list, send out an email inviting people to participate. You can even try contacting people who have recently contacted your support team or left comments on your blog or social media pages.

Recruiting platforms are a newer option. Respondent is a recruiting platform where millions of professionals are pre-vetted and qualified. If your UX research requires specific users from specific professions or at certain professional levels, consider Respondent. And Respondent doesn’t only make the recruit easy, it makes reminders and incentive payments turnkey

Diary Studies: Well Worth the Effort

For a successful diary study, it is important to think about the goals of the study and the types of information you hope to gain. Be sure to create a study protocol that outlines the tasks and questions that will be asked of participants, as well as the format in which the diaries will be collected and analyzed. Stay in contact with your participants to make sure they’re engaged and making entries and at the end of the study you’ll have the authentic and rich insights you need to improve your user experience and that would be almost impossible to obtain through other research methods.

 

Recommend Resources:

 

Similar posts