User testing: the miracle recipe to convince anyone

· usability, user testing, and convincing

25.10.2014: updated to include the questions from the comments

Every month, someone tells me: “I’d like to run a user test, but I can’t!”

Recently, two things convinced me to write this article for you:

So here is my approach for convincing anyone of the importance of usability testing.

It comes in three steps:

  • How I run a test
  • What the client gets out of it
  • And I answer all the questions that follow.

Nothing complicated. Let’s go.

What is a user test?

I always start by explaining what a user test is, and how it differs from an acceptance test, a unit test or a focus group.

A user test is quite simple:

  • Take 5 potential users of the platform, service, software or app you’re building.
  • Have them use the application (or the mockups/prototypes that show it).
  • Ask them to carry out real tasks.
  • And (just) watch how they use it, and learn from their mistakes.

Usability testing isn’t asking someone whether they like the product, but whether they can use it.

I often add that if you think you already know the answer to that question, I don’t think you know what you’re talking about.

What’s in it for the client

Then I explain what user testing brings to the person I’m trying to convince. I try to make clear what I will get out of the tests, and what the benefit is for the client.

  • The test shows whether what we’ve designed so far makes sense. We validate all the assumptions made during design. We find out what works and what doesn’t.
  • We get a view of users’ mental model (how they talk, how they picture the things we put in front of them), which helps design the next features better.
  • We reduce as much as possible the risk that the product we ship doesn’t fit!
  • If the test goes well, we can show with numbers that the product is usable by its target audience and that it’s “good”. That should reassure the whole hierarchy before launch!

Yes, but…

At this point, the person I’m talking to always comes up with one or more discouraging arguments. Here are the ones I hear most often, and the answer I find most fitting.

The application isn’t finished… it’s only a prototype / mockups

  • It will show where to put the effort next.
  • It lets us test before we build.
  • We’re grown-up enough to tell what comes from the mockup and what comes from the concept.

We already know what users want

  • We’re not trying to find out what people want, but whether they can use what we offer them.
  • Even if we were, asking them is rarely the right option (use the Henry Ford or Steve Jobs arguments).

Users haven’t been trained – it’s a specialist product

  • Absolutely. That said, we can tell whether a problem comes from not knowing the product or from usability issues.
  • And seeing what they try first gives us an idea of how they think.

We want to test with more users, 5 isn’t enough

Studies show that testing with 5 users reveals about 80% of an interface’s problems. It will take us a day, and we’ll have plenty to chew on.

Yes, but we don’t have enough time

Come on, a test takes four days at most. Plenty of time. And if you take part in the tests with me, we can get it down to two.

And we’ll never have the budget to act on the feedback

That’s tricky. The big question is: do you want an application full of features nobody wants or can use, or a product people understand and can use?

But everyone is different

  • Yes and no. Even if everyone has a different education and culture, everyone has fingers of a certain size, and roughly the same memory.
  • For business applications, people often share a vocabulary and a way of thinking. Designing for what they have in common is the whole challenge.

But you’re the expert, you should know what will be usable

  • My whole design is built on assumptions: who the users are, what they do, what they know. Like any assumption, they need validating at some point. Do you want to validate them BEFORE the product goes on sale, or after? I’d say the sooner the better.
  • You do test a product before release to make sure there are no bugs, don’t you? Well, user tests check there’s no gap between the proposed design and what users expect.

You’ve already done an expert review, isn’t that enough?

  • They’re complementary methods (obviously). Nothing replaces contact with real people. Once the software is in a user’s hands, you’ll see things no review could reveal.
  • On the technical side: there are code reviews and unit tests to validate code. But they never replace functional tests and demos. Usability tests are the same.

What next?

There you go, you now have my secret weapon. It may not convince everyone, but never mind. You have to start somewhere.

And if they’re convinced and you now have to take the plunge, take an hour to read Steve Krug’s famous Rocket Surgery Made Easy. This great book gives you a ready-to-use method to run your first tests.

What about you: have you tried another approach to convince people? Have you heard other “Yes, but…”?

Many thanks to my two reviewers, Guillaume Berry and Thomas di Luccio