Qualitative and quantitative research methods — advantages and disadvantages

There are many methods for conducting UX research, and depending on the type of data being collected, they can be divided into qualitative or quantitative research. What are the differences between these two types of research? When should you use each one? You will read about this in this article!

UX Research Methods: Quantitative and Qualitative

In all phases of the design process, it is crucial to keep the user and their needs in mind. But how can we know what our users’ needs are? To find out, the best way is to conduct thorough user research and gather a sufficient amount of data. Below, I will explain what qualitative and quantitative research methods involve, what are the advantages and disadvantages of each, and when to use them.

Qualitative research

Qualitative research focuses on a small number of users to better understand the needs of the respondents. The purpose of qualitative research is considered to be to identify a problem, analyze a phenomenon, or seek solutions to specific existing problems. This type of research answers “why?” or “how?” questions. If you want to find out why users behave in a certain way, or for what reason they have an unsatisfactory experience with a particular product, listen to their opinions! This will give you valuable knowledge and help you understand what causes their dissatisfaction. And this is where qualitative research methods will help.

The most popular qualitative research methods.

In-depth interviews

Interviews with users are the most well-known method of UX research. It involves researchers conducting individual conversations with respondents based on previously established research questions and gathering information about a given service, product, or application. During such interviews, questions are mainly formulated using phrases such as “why?” and “how?” to deepen specific issues. This method allows for the collection of a large amount of qualitative data, which is the responses given by the users themselves, and their subsequent analysis. What will you gain from them? Above all, you will gain a better understanding of the needs, motivations, and behaviors of your users. Remember — before you decide to arrange an interview with a user, first prepare a conversation script.

Focus Groups

This method involves studying a small group of people (from a few to several dozen individuals) and involves a discussion in the presence of a moderator. During the study, the moderator asks respondents questions that should provide information about the opinions, beliefs, and emotions that accompany the topic being researched. Focus groups are extremely valuable — by understanding the user’s point of view (e.g. how they perceive your product or what they value most in it), you will be able to create or improve your service/product/application and make it more interesting and attractive to your potential audience.

Moderated/Unmoderated Usability Testing

These tests are conducted to evaluate a particular product, such as a website, on current or future users. Participants are asked to perform a task that has been defined by the UX researcher. During the task, the researcher observes the user and analyzes metrics (such as time to complete the task).

Field Studies

When designing a new service, we often isolate ourselves from the world and forget who we are creating it for. However, the user and their needs should be our top priority. Field studies are nothing more than research in the user’s natural environment. They are a great method of observation, during which UX researchers assign specific tasks to participants. Through field studies, you can better understand your future customers and create a product that meets their needs.

Advantages of Qualitative Research

• Qualitative research allows for a more flexible approach compared to quantitative research. If during interviews, the researcher realizes that they are not getting useful information, they can quickly change or adjust the questions to get the desired responses and data.
• Qualitative research can capture changing attitudes in the target group, such as consumers of a product, service, workplace attitudes, or even marketing strategy.
• Qualitative research, unlike quantitative research, is not easily translated into numbers. Qualitative research provides a lot of information about human behavior, desires, personality traits, and emotions. These data will be useful for designing a new product or service that is as user-friendly as possible. Numbers alone would not be able to reveal this information.

The pillar of qualitative research is primarily observation. Qualitative research, such as in the form of interviews, relies mainly on individual opinions and respondents’ points of view. However, there is a certain risk that the respondent may not be one hundred percent honest with the researcher and will answer the questions as they should and not as they feel. And that is why observation plays a key role here, rather than the answers themselves. Remember to properly define the group of respondents before doing the survey and (as much as possible) select people who will put their sincerity on the line.

Quantitative research

Quantitative research answers the questions “how much?” and “how much?”. As you may have guessed, these questions alone suggest that for quantitative research we will need a much larger group of respondents to collect enough data. Quantitative research is mainly used to determine the scale of a given problem, to test previously stated hypotheses, and to find answers or solutions to a problem. What is particularly important, this type of research provides us with hard data and statistics, so the results are presented in the form of graphs and figures. If you need to get answers to one or two questions, conducting a quantitative test among users will be the best choice.

Most popular quantitative research methods

Surveys (NPS survey)

This technique involves sending surveys to users of a given product. With this method, you can verify users’ opinions by asking them a simple question: “On a scale of 1 to 5, how satisfied are you with our product?”

A/B testing

This is a way to conduct experiments on real users of a product. You can use this method to choose the most suitable version of an application or website that performs best from a comparative analysis perspective. In the test, compare two versions of a selected element (such as an interface) and then measure the results of each, taking into account indicators such as user engagement or conversion.

Eye-tracking

Using special systems and tools, we can track the movement of a user’s eyes, from which we can create a so-called heat map that shows the user’s focus on a particular element based on mouse movements and clicks. By using this method, you can find out which elements attract the user’s attention the most and which ones the least.

Web analytics

This is the measurement of traffic and actions on a website that provides you with information about users and insights into how users behave while using your website. Data can be divided into two types: technical (such as time spent on a particular page) and user data (such as demographic data). Analytical data will help you monitor product performance or identify problems.

Advantages of quantitative research

• Quantitative research does not require participants to provide personal data. The most important requirement is that the participants match the profile appropriate for the test being conducted and provide valuable information — nothing more is needed.
• In quantitative research, statistics and numerical data are important. Because it is conducted on a larger group, it allows for the expansion of knowledge about users and increases the credibility of the study. If we conduct research for a larger sample, the chances of obtaining generalized and averaged conclusions increase.
• Thanks to quantitative research methods, we can acquire enormous amounts of data much faster than when using other research methods. This, in turn, allows for the rapid accumulation of data and the extraction of averages from them.

As you already know, quantitative research is mainly used to confirm a certain assumption or hypothesis. If the study is carefully designed and methodologically conducted correctly on an appropriate number of respondents, the results can be generalized to the overall user/customer population.

Disadvantages of quantitative research

• Unfortunately, surveys often contain closed-ended questions, so the respondent does not have the space to explain their choice (for example, their rating). A lack of complete answers contributes to a shortage of data, which in turn can lead to errors in product design.
• Quantitative research helps to understand whether and how often a particular problem occurs. However, it does not tell you the cause of the problem, as it only points out the frequency of a particular behavior or phenomenon. Moreover, it does not suggest any solutions.
• UX researchers face the challenge of statistical processing and interpreting the results after collecting data from the entire group of respondents. Depending on the data collection method chosen during the research design, the time required to interpret the results will vary.

When to use which method?

Let’s look at a specific example below.

Let’s say we want to find out how satisfied users are with a particular mobile application. I will show you how both approaches can answer this question.

If I take a quantitative approach, I can conduct an online survey among 400 customers and ask them the question: “On a scale of 1 to 5, how satisfied are you with application X?”. Based on the data collected, I will conduct a statistical analysis and conclude, which could sound something like: “Customers rate their satisfaction with product X on average at 4.6/5”.

However, if I use a qualitative approach, I can conduct individual interviews with 10 customers and ask them open-ended questions such as: “How satisfied are you with product X?”, and “What can be done to increase satisfaction with using product X?”, “What can be done to improve application X?”. Then, I will prepare a transcript of the interviews and try to find patterns or recurring issues that were identified by the majority of respondents, for example, that customers would like the application interface to be more intuitive.

In summary, every research method has both advantages and disadvantages. The ability to effectively and meaningfully collect data requires an awareness of the limitations and strengths of each method. The key to success is asking yourself what you want to achieve through research and what your goal is. Only then, knowing what you are aiming for, will you decide which research method to use.