One stop shop for free articles & web content

88 articles on writing web content and intranet content

Web site usability testing: recommended procedures

Usability testing of web sites is:

• an essential element of quality assurance
• a true test of how people actually use a web site
• easy when you know how
• extremely cheap when you do it yourself.

A usability test is:

• a series of individuals using a site under gentle guidance from a facilitator
• a test of whether outsiders can successfully use a web site
• not a focus group
• not a meeting.

top of page

Definitions

Facilitator: the person who is guiding the user through the test, and taking notes.
Observer: someone else observing the test, often in a separate room.
Owner: the owner of the web site.
Web site development team: everyone involved with developing the web site — including the strategy group, designers, programmers, stakeholders, etc.
Usability: the extent to which the intended user can meet his or her goals using the system being tested.
User: person who uses the web site in a usability test.
You: the person most involved in organising and facilitating the test. (You may or may not belong to the web site development team.)

top of page

When to test

Testing little and often is far more valuable and cost effective than doing one whopping big usability test of an entire site when it is almost finished. ("Don't Make me Think" Steve Krug, ISBN 0-7897-2310-7, p. 146-147.)

Suitable times for testing:

• at the web site's conception (start by testing a printed mockup of the home page)
• before planning a redevelopment
• repeatedly during (re)development, as critical pages or sections are prepared
• when traffic analysis shows an anomaly
• when the owner requires hard information about a page or site.

top of page

What and why are you testing?

Know what you hope to discover each time you test.

With all tests you want to discover whether the user:

• gets the point of the page(s)
• understands the navigation system
• can guess where to find things.

In a general test you want to know:

• how do users interact with the web site you are testing?
• what is difficult for people to do?
• where do they get lost?
• what makes sense to them?
• what makes them feel distrustful or insecure?
• what do they like and what do they hate?

In a specific test you might want to know, for example:

• can the user accomplish a key task?
• can the user find something specific?

top of page

How long is a usability test?

Tests range from 5 minutes (for a single page design) to 1 hour (for a general response to a whole site or new design).

Most people (including you) will be tired after 1 hour's testing.

Users should be outsiders

Finding users need not be a problem. Ask anyone who is:

• not involved with the web site in any way
• completely new to the web site (so don't ask the same person twice)
• somewhat familiar with the Web in general.

Students. Neighbours. Volunteers. Friends. People in the same building but a different company.

(In a full-blown, traditional usability test, a usability consultant would select members of the web site's target audience. This requires great effort, and the rewards are comparatively small.)

top of page

Where to test

Ideally, run tests in the user's home or work place. Benefits:

• the user feels more relaxed
• the user doesn't need to learn new systems
• you get to see how the web site works on different computers, browsers and modems.

If it's not practical to test on the user's home territory, test at your place of work. Essential: a private room, two chairs, a clipboard and a computer.

Are you connected to the Internet by cable, ADSL? Then arrange for your test computer to use a dial-up modem. That's how 90% of users probably access your web site.

(Optional: a webcam or camcorder enables observers to watch the computer screen and hear the user's comments from another room. Problem is, with a web cam, the test immediately becomes a big deal. This is counterproductive, because the ideal is to test little and often.)

top of page

Prepare a script

A script is part of your own quality assurance system: it helps ensure that you follow procedures, and that you are asking each user to do the same task.

Items on the script:

1. Introduce yourself
2. Reassure, establish rapport: 'Please think aloud, you cannot make mistakes, we are testing the site (not you).'
3. Clarify: purpose of test, confidentiality issues.
4. Start test: open-ended questions, 'what if?', 'tell me more', 'yes?' 'uh huh...' 'mm...'
5. State task or tasks, e.g.: 'What might you click on if you wanted to find out about a home loan?'
6. Allow users to try to accomplish the task in their own way.
7. End the test: say thank you, reply to previously unanswered questions, give payment or a gift if appropriate.

Plan how you will take notes

What is your preferred style of note taking? What would make it easier?

• a column for comments on your script?
• blank paper for a mindmap?
• printouts of key pages?
• thumbnails of key pages?

To save time you can use abbreviations when noting occasions when the user hesitates, looks worried, misunderstands, looks frustrated or gives up.

top of page

Test the script

It's an excellent idea to test your script with a colleague acting as the user.

One problem with usability testing is that outsiders may be too polite and eager to please. You can get useful feedback about your own facilitation style from a colleague.

After testing, ask your colleague about the experience. For example, did the colleague feel hustled or hurried or controlled by you? Did the test run smoothly? Did you set tasks in the right order, or the right way? How could you improve your manner?

top of page

Now start the test

Then turn on the computer and show the first web page you're testing. Start the test.

Notice the user's behaviour, and note every occasion the user:

• hesitates, worries, or is forced to think
• misunderstands something
• gets frustrated or annoyed
• gives up.

top of page

Facilitation style: bite your lip

You have an agenda, certainly: but you are only an observer. Watch the user do what comes naturally. Don't help.

The whole point of the test is to see what users do alone, without you helping.

Much of the time you'll just be probing, and encouraging the user to say what they're doing and thinking.

Use everyday language, not in-house jargon, unless you are testing an intranet.

Keep calm: you want the user to find faults! Don't take it personally.

Suspend judgement: don't even think about solving the problems at this stage.

Keep encouraging the user to think aloud.

Repeat: don't help! Don't think for the user. Explain that you can't answer questions during the test, because it's a test of how people use the site without you. But you'll answer them later if you can.

Enjoy the job.

Write everything down. If a task is part of the test, note the time it takes each user.

Don't interfere or ask leading questions.

Check your original task list as you go.

top of page

Report on test

Write a 1-2 page report simply noting each problem the user found. Do it immediately while the test is fresh in your mind.

Optional: mark each problem as serious, less serious, or preference. ('Preference' means matters of opinion, such as whether the colours are pretty. Preferences don't affect usability.)

Recommend solutions only if you are required to do so. Clearly differentiate your recommendations from your observations.

Give your report to the appropriate person or group: e.g. the owner or web development team. Don't delay.

Decide on action

Now you may need to meet with the web development team. Discuss the problems, and decide which ones are easy and cheap to fix, and which need further investigation.

Carry on testing

You'll get better if you keep practising! Remember, for cost-effective results, test little and often.

But just because you are experienced, don't be tempted to speed up the test or put words into the users' mouths. They are the experts on their own experience: not you.

The image "Boy holding whitebait " is © Kirby Wright: email: kirbyw@onq.co.nz.

QWC home     contact     Contented.com
2012 Rachel McAlpine PO Box 19184 Wellington 6149 New Zealand