Learning session 20: Quantitative Research Methods

🚁 Topic

Quantitative research methods’ main points are to gather insights and create solutions to users’ problems on a daily basis.
It involves methods that help you to know your users and to get pieces of information from them by various methods. These methods have an important place in UX design, as it helps to find what it’s wrong with your product and what it’s the best solutions that you can do. Here is what I learned today.

🤓 What I learned

Instead of gathering insights, we collect numbers describing some user experience aspects when conducting quantitative UX research.

These figures are known as UX metrics. Analytics, A/B testing, surveys, and quantitative usability testing are all widely known quantitative UX methodologies.

If you’re trying to determine the priority or scale of a problem, you might want to use a quantitative methodology. For example, what percentage of my users are affected by this particular issue?

If you want to compare alternative design options and use data to inform your decision, you could conduct a quantitative study, particularly A/B testing.

Your requirements and available resources should guide your quantitative research method selection. These methods can be adapted to meet your specific needs and produce qualitative data.

A/B and multivariate testing

Designers create two versions of the same interface and test them on different groups of users to determine which one performs best. Typically, only one variable is different, such as a CTA or a navigation bar.

Multivariate testing is used when you want to test multiple design elements, such as typefaces, button placement, and alternative icons.

What factors contribute to the effectiveness of A/B testing?

- Choose a variable to test, like a CTA or other UI elements that can be improved.

- Define a goal like choosing one metric to analyze possible outcomes

- Split your audience randomly and equally; avoid splitting your audience based on their age or gender.

Eye tracking

Eye tracking is a research method that requires specialized equipment to track users’ eyes as they scan an interface, and this method is costly.

Its goal is to determine which elements of the page draw attention and which go unnoticed.

Aside from the high cost of the equipment, one of the significant drawbacks of this method is that researchers frequently ask users to think aloud while navigating a page. It could lead to dangerous misunderstandings. Because of the think-aloud task, users may spend more time on a page explaining what they see and looking at some content for longer than they would in real life.

Numerous factors can have an impact on an eye-tracking study because it requires users to:

• To sit straight in the chair
• To not move around excessively
• To not drink from a cup
• To not lean forward or back in the chair

Surveys

A survey is a low-cost, highly customizable user research tool to gather information about participants’ preferences, behaviors, characteristics, and thoughts on a topic. Surveys are a series of questions that can be placed on a live website, within the body of an email, or after user interviews.

The ease of conducting a survey can be deceiving, resulting in bad surveys and useless data that do not provide insights into users’ problems.

Surveys help in better understanding users and reduce the risk of developing unfunctional solutions. On the other hand, surveys are based on what users say and may not accurately reflect or explain user behavior. They should never be used in place of usability studies and interviews.

What factors will contribute to the effectiveness of the surveys?

• Define the goal — It’s essential to know your goal, what I want to know from users, and why the surveys are the best method to do it.
• Logically structured questions — To assist users while navigating the form, group related questions together and separate them from other groups.
• Include open-ended questions — Open-ended questions evoke more information from users. Include multiple-choice questions to make the form completion process more straightforward.
• Be short — Long forms are tedious and too time-consuming for users

SUS scores

The System Usability Scale is the most affordable and time-efficient tool for measuring usability. It has ten questions with five response options ranging from Strongly Agree to Strongly Disagree.

It can assess any product or service, including hardware, software, websites, mobile devices, and applications.One of the main drawbacks of using SUS scores is that it has a complicated scoring system that is difficult to understand.

The final score reflects how well or poorly your product is usable. The average score is 68, and anything below this number indicates that users are having serious problems when interacting with your product. System usability scale can be used as the first step of usability evaluation but isn’t the only reliable source.

Web analytics

Analytic data shows how people use a live product. Analytics data provides solid facts about a user’s experience and permits you to compare what users do to what they say.

Using this quantitative method persuades data-driven stakeholders and provides more data to do additional user research.

An example of web analytics, it’s Google Analytics which is the most popular tool with a free version. If used correctly, you can get unique insights by exploring:

Standard page view data — you can see which pages users visited and left your product, the paths they took, and the device they used.

Behavior flow report — It allows you to examine user behavior at a specific flow point or determine which steps in your scenario fail the most frequently.

Goals — You can set different goals for the same website, such as sign up, purchase accomplishment, or email subscription, and track the steps users take to achieve those goals.

Time tracking: — This method can determine how much time users spend achieving a goal or which actions take the longest to complete or process.

Usability benchmarking

Usability benchmarking is a method of assessing a product’s usability using metrics gathered from quantitative usability testing.Quantitative usability testing requires users to complete tasks in a system.

The only distinction is that researchers collect metrics such as time on task or success rate in quantitative usability testing. Moderators observe and pay attention to usability issues that users encounter while performing a task during qualitative usability testing.

Usability testing can be conducted in person or remotely, with or without a moderator, depending on your objectives and budget.

The data gathered during usability testing can be used for:

• Monitoring the evolution of a product or service over time
• Comparing the usability of your product to that of a competitor, an industry benchmark, or a stakeholder-defined goal
• Demonstrating the added value of your and your team’s UX work

Tree testing

It’s a method that evaluates the product’s information architecture and the findability of content in a product.

Participants are given a text-only version of the site’s hierarchy and a set of tasks to complete. The goal is to see if users can find what they need based on the structure of the site when all other UI elements are removed.

Tree testing is an excellent tool for:

• To evaluate findability, labeling, and information architecture
• To validate the ideas before designing them
• To get insight into users’ mental models

Desirability studies

A desirability study is a method of user research for determining aesthetic appeal and visual design directions that speak to users and create the right brand image.

Typically, participants are shown product images or asked to use prototypes or live products. You can also use different versions of the same interface to find the best solution.

They are then instructed to describe the product using adjectives or descriptive phrases from a list. For example, you can discover that a significant number of users find the blue palette trustworthy and reliable on banking websites.

Desirability studies are an excellent tool for determining the best design direction and resolving disagreements among team members.

This method has the disadvantage of being unable to predict real-world user behavior. During the study, you can only learn what users say; you can’t learn how they’d act or feel while interacting with a product.

🤺 What challenged me

One challenge I faced while learning about quantitative research methods was understanding the different types of data. There are two main types of data: primary and secondary. Each type of data has its own set of advantages and disadvantages. Another challenge I faced was understanding how to select the appropriate research method based on the research question. There are many different research methods, and each has its own strengths and weaknesses. Selecting the right method is essential to conducting a successful research project.

Thank you for coming this far. Any feedback or critique is appreciated.❤️