Usability Testing – What, Why, When, Who and How

What is usability testing? It’s an attempt to evaluate a product’s usability, or ease of use, by seeing if its design fits the user’s natural expectations in an efficient, effective, and satisfying manner. 

The user’s comments and feelings about the overall experience of using a particular product validate and dictate its design. This could be a teapot or a saltshaker, of course, but in digital marketing, we’re usually talking about a website or mobile application. It’s critical that the website or mobile app appeals to a person’s natural inclinations regarding how to use it. 

Usability testing measures ensure that this is accomplished, and it’s easier than you think.

Why Is Usability Testing Done?

Usability testing, also referred to as user testing, is the process by which the intuitiveness of a design is assessed. Any successful company will routinely test usability during production to make sure a design is used the way it’s intended. For a website, we’re referring to navigation menus, layout, button placement, features, content, and more.

People expect a great user experience (UX) from websites and mobile apps. We call this a user-friendly or human-centered design. Things such as confusing navigation, broken links, unclear instructions, and obstruction of information are the opposite of user-friendliness, and can sometimes all but guarantee that a visitor will exit your website out of frustration, never to return. 

That person might complain to their friends or post on social media about how Company X’s website was simply “unusable.” The converse is also true. Provide an excellent experience for the user, and they will buy your product, return in the future and sing your praises to their friends.

UX testing helps prevent the former and ensure the latter. The famous $300 million button is a great example of this. In short, usability = conversions = profit.

Additional, oft-overlooked benefits of usability testing include:

  • Improved efficiency (more user-friendly internal systems help employees work faster and smarter)
  • Reduced development costs (you’ll be able to focus on the essential features and can fix flaws early in the process without wasting time later)
  • Reduced support costs (less time answering users’ questions, fewer returns, etc.)

When Is Usability Testing Done?

It’s a common misconception that once you do one round of usability testing, you can stop, or that a “best practices” approach obviates the need for testing. ABT stands for Always Be Testing. Usability testing should be performed on a regular schedule, especially during different phases of development. 

Design trends come and go, and users’ expectations evolve with these trends, so this is not a set-and-forget process. Similarly, there is no one-size-fits-all user-friendly design. What works for Facebook may not – and likely will not – work for you.

Who Conducts Usability Testing? And Who Are the Users?

Ideally, you’d want someone with some UX experience facilitating the test, but it isn’t difficult to do on your own. All you need is the product (we’ll continue with a website as an example), a video recorder (screen recording software for a website), a microphone, and a list of tasks. Thus, designers, developers, project managers, and CEOs can all conduct usability tests.

Testing can be a completely internal process, in which in-house employees do it, or you can hire users to come in and perform it.  websites offer remote, unmoderated testing.

How Do I Do It?

1. Identify users

It helps to have users similar to the real users of the product, but this is not absolutely necessary. Most human brains are similar in terms of performing tasks. Also, you need only five users.

2. Design tasks

Think of specific actions you want to test. These are referred to as tasks. An entire blog post could be dedicated to this concept alone because writing tasks is probably the most essential aspect of your study. 

Preface tasks with specific scenarios to set the scene, making it feel real for the user and emulating a true circumstance. For example, for the Fish Hippie website, we asked users to imagine they were shopping for a shirt to keep them cool in the summer heat because we wanted to see if people were easily able to find the new line of performance shirts. 

One user commented, “That’s fitting since it is really hot outside right now, so I would probably look for that anyway.” Tasks should be specific enough to eliminate any possible ambiguity, yet general enough to allow the natural journey to unfold, which is the entire point of usability testing. 

Avoid being suggestive or asking leading questions in your tasks. This introduces bias and defeats the purpose of observing how a user naturally solves the problem. The task sequence regarding the shirt example above looked like this:

  • a. With the summer heat rising, you want to find a shirt that will keep you cool. Shop to find a shirt you like that will keep you cool.
  • b. Is it available in your desired color and size?
  • c. What were the important factors in choosing which shirt to get?

3. Do the testing

Gather your materials, set up your testing location in a private room, get snacks, and invite observers – especially managers and executives – to watch live testing sessions. They usually walk away fascinated at the issues found by having one person attempt to use their product because it’s never as smooth as they expect.

4. Analyze the results

UX testing data should be analyzed and summarized to present the information on users’ comments and the problems they encountered. Use  a simple bulleted list for this. If you have many issues that need fixing, you may want to prioritize them.

A Word on Websites That Offer Remote, Unmoderated User Testing …

If you need a quick usability test, consider – a website on which you provide tasks and pay to get a video back from a tester completing them – is fine. But for any other case, in-person testing is a better alternative. Here’s why:

  • Body language ­– For web-based services, you’re usually not going to be able to see the tester’s face, so you can’t see facial expressions such as a surprised reaction when the user suddenly encounters something unexpected. This happens often.
  • Probing questions – Perhaps the most important reason in-person testing is better is the ability to talk to the user. You’re able to ask probing questions, which usually give you more significant insights than the user’s unsolicited flow of thoughts. For example, if a user is surprised, you can ask, “Was there something that happened that you didn’t expect there?” – which is usually followed by an answer such as, “Yea! I thought when I clicked this button, X would happen, but Y happened instead!” 
    Remote users might sit in silence while scrolling and hovering, forgetting to constantly think out loud, which is common. In-person testing allows you to ensure there is a constant stream of thought and the ability to ask questions such as:
    What are you thinking now?
    What made you do that?
    What are you looking at?
    What are you doing now?
  • Q&A – In-person testing allows for questions and answers from both parties. I’ve also found that simply being in the same room and periodically saying “OK” and “That’s very helpful” cause the user to think out loud much more than they otherwise might in an unmoderated environment sitting at home.
  • Observers – Watching a live testing session can be a rewarding and eye-opening experience for someone who is unfamiliar with the process and who assumes the product will definitely work the way it’s intended. Remote, unmoderated testing doesn’t allow for observation.
  • Time – Testers for sites such as have the incentive to spend as little time as possible doing a test. This could prevent them from caring as much and flying through the process. Tests on these sites are usually only about 15 minutes long.
    You can argue that these users eventually become expert testers after doing it so much, which affects the authenticity of your data. Real consumers will not be expert testers.
  • Deeper data – You’re able to gain more meaningful insights from in-person usability testing, such as the psychological motivation behind specific behaviors, instead of the strict interaction with the product in isolation. By solely using remote, unmoderated testing, you’re limiting yourself to discovering only top-level issues.

Usability testing is extremely important; it directly fuels profits. It should be done on a regular basis, can be done by anyone, and is relatively easy to do. Opt for in-person, facilitated testing over remote, unmoderated testing for better testing, more meaningful insights, and more impactful results.

Need your website usability-tested? (Yes, you do!) Want us to handle the heavy lifting? We can help with the details, including when to conduct usability testing. Contact us today and ask how we can make your website or mobile app more user-friendly.


Nick McNeill

Interactive Director

Nick uses his talents in computer science and graphic design to grow the online presence of brands such as Santee Cooper, Southern Tide, frogg toggs, Farmers Telephone Cooperative and Blue Force Gear. A serial marathon runner, his steely determination shines through every brand he grows. As lead user interface designer of the GuestDesk software suite for online hotel reservations, he watched it explode from $1 million in reservations to over $400 million annually.

Read more posts from Nick