Your First Usability Test: Step by Step with Five Users

A small farm just outside town sells vegetable boxes on subscription. The website is new, the team is proud of it, and yet the phone keeps ringing. Almost every call is about the same thing: “How do I pause my delivery while I’m on vacation?” The feature exists. It lives in the customer account under “Manage subscription,” then “Delivery schedule,” then all the way at the bottom. To the team, that’s logical. To customers, apparently not.

This is exactly what usability testing is for. You sit a few people down in front of your website, give them a task like “You’re going away for two weeks, make sure no box arrives while you’re gone,” and watch. After three people, you usually know where things break. After five, you have a pretty clear picture.

Many teams shy away from testing because it sounds like labs, one-way mirrors and big budgets. It doesn’t have to. This article walks you through your first test with five people, step by step: from the goal to the tasks and participants all the way to a first analysis on the same day. You don’t need any prior knowledge, just half a day and the willingness to sit quietly while someone struggles with your design.

What a usability test is, and what it isn’t

Usability describes how easily, quickly and satisfyingly something can be used. A usability test checks exactly that: real people complete typical tasks with a product while you observe. Kate Moran of the Nielsen Norman Group (NN/g), a well-known UX research and consulting firm, names three building blocks: a facilitator who runs the session, realistic tasks, and participants who resemble your actual audience as closely as possible.

It’s just as important to know what a usability test is not. It isn’t an opinion poll. Whether someone likes the colors barely matters here. What counts is what people do, not what they say about your design. And it isn’t a test of the participant. If someone can’t complete a task, the person hasn’t failed. The design has.

Most tests use the think-aloud method. Participants say out loud whatever is going through their mind as they work: “I’m looking for vacation now … this says delivery schedule, probably not that …”. That way you learn not only where someone gets stuck but also why. Jakob Nielsen, co-founder of NN/g, calls thinking aloud perhaps the single most valuable usability engineering method. His reasoning: it’s cheap, needs no special equipment, works at every stage of a project, and produces useful findings even when the session isn’t run perfectly.

You also don’t have to wait until your website is finished. You can test anything that lets someone play through a task: a live website, a clickable prototype from a design tool, or even sketches on paper. The earlier you test, the cheaper the changes.

Why five people are usually enough

The number five goes back to Jakob Nielsen. Based on a large number of projects, he found that a single participant uncovers about 31 percent of usability problems on average. Each additional person finds some of the same problems and some new ones. Mathematically, five people get you to roughly 85 percent of the problems. Beyond that, extra participants mostly show you what you already know.

That leads to Nielsen’s actual recommendation, which often gets lost: instead of one big round with fifteen people, run three small rounds with five each and improve the design in between. The second round shows whether your changes work and reveals problems that bigger ones were hiding before.

A few caveats apply. The five refers to qualitative tests, the kind meant to find problems. If you want to measure metrics, such as a precise success rate, you need many more participants. And if you have several very different audiences, test each one separately. Nielsen recommends three to four people per group when testing two groups, and three per group for three or more.

Steve Krug, author of the classic “Don’t Make Me Think,” goes even further in his book “Rocket Surgery Made Easy.” He suggests testing with just three people one morning a month, discussing what you saw over lunch, and fixing the most serious problems right away. For your first test, five people is a good target. If only three show up, that’s no reason to cancel.

Step 1: Decide what you want to find out

A test without a goal produces a long list of small issues and no answers. Before you plan anything, write down in one or two sentences what you want to learn. Good goals focus on specific flows, not the entire website.

Before: “We want to know whether the website is user-friendly.”

After: “We want to know whether customers can pause a delivery and adjust next week’s box on their own, without calling us.”

The second version tells you right away which areas to test and how you’ll recognize success. It also helps you say no: when someone on the team wants to squeeze in the recipe page and the new newsletter, you can point to the goal. A first test with two or three clear questions is worth more than one that covers a little bit of everything.

Good hints about where to start are often right in front of you: questions customer service hears again and again, drop-offs at a particular point in checkout, or important features that hardly anyone uses.

Step 2: Write realistic tasks

Tasks are the heart of the test. Small wording mistakes can skew the entire result, either by giving away the answer or by sending participants in the wrong direction. A few rules help. Many of them are summarized in seven tips by Jeff Sauro of MeasuringU, a research firm specializing in UX measurement.

Describe the goal, not the steps

A task says what someone wants to achieve, not where to click. If you spell out the path, you’re only testing whether people can follow instructions.

Before: “Go to your account, open ‘Manage subscription’ and pause the delivery.”

After: “You’re on vacation from August 3 to August 17. Make sure no vegetable box ends up on your doorstep during that time.”

Use your audience’s words, not the interface’s

If your website says “Delivery schedule,” that phrase must not appear in the task. Otherwise participants simply scan for the matching label, and you never learn whether they would have found the feature without help. Phrase tasks the way people talk in everyday life. Nobody says “I’d like to adjust my delivery schedule.” They say “I don’t want the box to show up while I’m away.”

Give brief context and all the details needed

A sentence of background helps people put themselves in the situation. Too much backstory, on the other hand, confuses them. Also provide every detail the task requires, such as dates, quantities or a test address. Otherwise participants make something up or stop to ask, and both cost time.

Make sure there’s a clear end

You need to know exactly when a task is complete. “Have a look around the website” has no end. “Find out whether this week’s box contains potatoes” does. Sauro recommends defining a correct answer for each task in advance. That makes the analysis much easier later.

Avoid loaded words

“Try out our handy pause feature” reveals not only that a pause feature exists but also that you think it’s handy. Neutral wording keeps participants from feeling they should confirm your opinion.

How many tasks?

For a 45 to 60 minute session, five to seven tasks is a common rule of thumb. For your first test, three to five is often enough. Plan too few rather than too many: thinking aloud is tiring, and concentration drops noticeably after about 45 minutes. For the farm example, the tasks might look like this:

1. “You’d like to try the vegetable box. Order a small box for delivery to this address.” (address on a card)
2. “You’re on vacation from August 3 to August 17. Make sure no box arrives during that time.”
3. “You don’t like fennel. Find out whether you can swap it out for your next delivery.”
4. “You want to know where the tomatoes in your box come from. Find out.”

Step 3: Find the right participants

The best tasks won’t help much if the wrong people work on them. Participants should match your audience, in this case people who would order groceries online and live in the delivery area. Colleagues from your own team are out: they know the product, the jargon and often the debates behind every design decision.

Friends and family are a fallback, to be used with care. They usually don’t want to hurt your feelings and are therefore more forgiving of your design. Better options include:

• Existing customers you invite to a short session through your newsletter or website
• People who showed interest in the product but didn’t order
• Friends of friends who fit the audience but have no connection to the team
• Specialized recruiting services, if the budget allows

A short screening questionnaire helps you filter. Two or three questions are enough: How do you buy groceries today? Have you ever used a subscription for groceries or meal kits? Roughly where do you live? Aim for a bit of variety, for example in age or experience with online shopping.

Also plan a small thank-you, such as a gift card. People are giving you an hour of their time, and an incentive makes it much easier to get commitments. Invite one more person than you need. Someone almost always cancels at short notice.

Step 4: Prepare a script and do a dry run

A test script is your cheat sheet for the session. It makes sure every participant gets the same information and that you don’t forget anything when nerves kick in. A simple outline for one hour looks like this:

1. Welcome (5 minutes): Explain what this is about. One sentence matters most: “We’re testing the website, not you. You can’t do anything wrong. If something doesn’t work, that’s exactly what we want to find out.”
2. Consent (2 minutes): If you want to record the screen or the conversation, get consent first and explain what the recording will be used for and when it will be deleted. Recordings contain personal data, so if in doubt, check with whoever handles data protection in your organization.
3. Explain thinking aloud (3 minutes): Ask the person to say everything they notice, what they’re looking for and what they expect. Many people find this odd at first. A short practice task on an unrelated website helps them warm up.
4. A few opening questions (5 minutes): For example: “How do you usually buy vegetables?” This relaxes people and gives you context for your observations.
5. Tasks (30 minutes): Hand out each task separately, ideally on a card or as text, so the person can reread it.
6. Wrap-up (10 minutes): Open questions like “What felt especially easy or hard?” If you want a benchmark, this is where a standardized questionnaire such as the System Usability Scale fits in. Then: thanks and the incentive.

Run through the whole session once before the first real appointment, with someone who doesn’t know the project. This dry run almost always surfaces something: a task that’s easy to misread, a prototype with a missing link, or a schedule that’s far too tight. Better now than halfway through session three.

Think about the setup, too. Is the prototype open on the device you’ll test on? Is the recording running? Is the test account loaded with sample data so the person actually finds a subscription to pause? Details like these decide whether you find usability problems or just flaws in your test setup.

Step 5: Moderate without helping

Now it gets real, and the hardest part begins: doing nothing. Jakob Nielsen sums up the facilitator’s role as mostly keeping quiet and letting the participant do the talking. That sounds easy. It isn’t. When someone scrolls past the right menu item for the third time, you’ll be itching to point at the screen.

Hold back. Any help changes the result, because in real life nobody sits next to your customers either. And the moments when someone gets stuck are the most valuable of the entire test.

Three techniques for neutral follow-up questions

You don’t have to stay completely silent, though. Kara Pernice of NN/g describes three techniques for following up without steering:

• Echo: Repeat the participant’s last words with a slightly questioning tone. If someone says “Hmm, this is kind of weird …,” a calm “Weird?” is enough.
• Boomerang: When the participant asks you something, send the question back. To “Do I have to register now?” you reply “What do you think?” or “What would you normally do?”
• Columbo: Named after the seemingly absent-minded TV detective. Start a sentence and leave it open: “You’re wondering whether …”. The person fills in what they’re thinking.

If the participant goes quiet, gently remind them to think aloud: “What’s going through your mind right now?” Avoid questions that point in a direction, like “Did you see the button in the top right?” or “That wasn’t very clear, was it?”

When someone is truly stuck

Sometimes a participant can’t move forward at all and becomes visibly uneasy. Then it’s fine to end the task: “Thanks, that’s really helpful already. Let’s move on to the next one.” Mark the task as not completed. If you do have to help so the following tasks can work, note that too. Nielsen advises recording moments where you may have influenced the participant so you can weigh them carefully during analysis.

Step 6: Observe and take notes

If at all possible, work in pairs: one person moderates, the other takes notes. Anyone who moderates, observes and types at the same time misses half of it. Other team members are welcome to watch, ideally from another room or via video, so the participant doesn’t feel watched. Nielsen points out that stakeholders often become more committed to usability work once they’ve seen someone struggle with their own eyes.

For notes, a simple table per participant is enough, with columns for task, completed yes or no, what happened, and notable quotes. Above all, write down what you see and hear, not what you conclude from it:

Observation: “First looks under ‘Delivery’ for a vacation option, then under ‘FAQ.’ Says: ‘I would have just called.’ Gave up after four minutes.”

Interpretation: “The pause feature isn’t where people expect it.”

Both matter, but keep them apart. Interpretations go in a separate column or into the analysis. That way it stays clear what a finding is based on, and you’re more likely to notice when you’re only seeing what you expected to see.

Step 7: Analyze on the same day

The best time for a first analysis is right after the last session, while impressions are fresh. That’s the idea behind Krug’s lunch debrief. Sit down together and go through the tasks one by one.

A simple overview helps a lot: fill in a table showing which participant completed which task. For the farm example, the result might look like this: all five manage to order. Only one person manages to pause a delivery, and only after several minutes. When swapping out the fennel, three people look in the product description instead of the customer account. Now it’s clear where to start.

Next, collect the problems that came up for several people and ask for each one: how many were affected, and how severe is it? A problem that stops all five people from ordering is more urgent than a typo everyone notices. For a detailed look at grouping problems and rating their severity, see the article How to Extract Valuable Feedback from Usability Tests.

For a first test, a short list is enough: the three to five most important problems, each with an observation that backs it up and an idea for a fix. You don’t need a long report. In the farm example, the fix might simply be a clearly visible “Pause delivery” link on the account home page, labeled with exactly the words participants used.

And then: improve and test again. Only the second round shows whether your fix works. It also often reveals problems nobody could have noticed before, because everyone was already failing one step earlier.

In person or remote?

Your first test can take place in a meeting room or over a video call. In a moderated remote test, the participant shares their screen and you moderate live, just as you would in person. That saves travel and makes it easier to find people from your real audience. In person, on the other hand, you see more: facial expressions, hesitation, the hand already reaching for the phone.

There are also unmoderated tests, where an online tool like Maze presents the tasks and records the session while the participant works alone. That’s handy for larger numbers but requires especially clear tasks, since nobody can ask questions. Sauro therefore recommends writing tasks for unmoderated tests in more detail. For your very first test, a moderated session is the better choice. You’ll learn the most, including about moderating itself.

Conclusion

Back to the farm and its vegetable boxes. One morning, five customers, four well-written tasks: that’s all it takes to find out why the phone keeps ringing. The test takes less time than a week of calls, and it delivers something no internal discussion can: the perspective of people seeing the website for the first time.

The most important step is the first one. Your first test won’t be perfect. Maybe a task gives too much away, maybe you help once too soon. That’s fine. As Nielsen writes, thinking aloud usually produces useful results even in a bumpy session, as long as you don’t steer participants. With every round you’ll get more confident, and your product will get better.

Checklist

• The test goal fits in one or two sentences and focuses on specific flows.
• Three to five tasks describe goals rather than click paths and avoid words from the interface.
• Each task has brief context, all required details and a clear end.
• Five participants from the target audience are invited, plus one backup. Nobody from your own team.
• An incentive is arranged.
• The script covers welcome, consent, explaining think-aloud, tasks and wrap-up.
• A dry run has taken place, and the setup and test data are ready.
• Roles are assigned: one person moderates, one takes notes.
• Nobody helps during the test. Follow-up questions are neutral (echo, boomerang, Columbo).
• Notes keep observation and interpretation apart.
• The first analysis happens on the same day and ends with a short list of the most important problems.
• A second round of testing after the improvements is planned.

Sources

• Kate Moran: Usability (User) Testing 101. Nielsen Norman Group, 2019
• Jakob Nielsen: Why You Only Need to Test with 5 Users. Nielsen Norman Group, 2000
• Jakob Nielsen: Thinking Aloud: The #1 Usability Tool. Nielsen Norman Group, 2012
• Kara Pernice: Talking with Participants During a Usability Test. Nielsen Norman Group, 2014
• Jeff Sauro: Seven Tips for Writing Usability Task Scenarios. MeasuringU, 2013
• dscout People Nerds: Usability Task Writing
• Creative Bloq: Steve Krug on DIY usability testing
• Steve Krug: Rocket Surgery Made Easy. New Riders, 2010