Artigo Especial #3: Why You Need to Test Your Web Site with Real Users

I recently attended the NN/g User Experience World Tour (see my review if you want to know more). I learned much about making Web sites more usable, but the most common theme was the importance of usability testing. If you think usability isn’t important, a recent test of shopping sites showed that almost half the tasks set to the test shoppers weren’t completed successfully ­ which equates to a lot of frustrated consumers and lost sales when scaled up to the global level!

- Assessing the usability of your site

To assess how usable your Web site is, you can choose a variety of techniques. I hope to convince you that user testing is one of the most important:

· User surveys and focus groups are marketing tools, to find out what a selected group of users remember about a Web site (rather than how they actually felt while they were using it).
Naturally, the "feel" of a Web site is an important element of branding and corporate image. A good feeling can increase trust in the site, so we should not dismiss it. 
· User feedback solicits the views of a self-selected user group. Experience shows that, usually:
· Visitors who hate the site leave promptly without responding,
· Visitors with minor problems: broken links, typos etc.
Often provide feedback, as do those who love the site,
· The majority probably think the site is OK and don’t bother to tell you this. Therefore feedback isn’t a reflection of the overall spectrum of user experience, though it can highlight specific problems.
· Webmasters may rely on mechanical aids such as log files and automated testing services (like www.usablenet.com). Log files tell you which pages are most visited, longest studied, first/last visited etc., but not whether users found what they wanted, or why they left. Automated services advise on predictable problems: broken links, and missing ALT tags for images, but are inevitably unable to spot complex things like difficulty in locating products, meaningless graphics, or poor results from search pages.
· Heuristic inspections are conducted by a usability expert or a team, systematically examining a representative selection of pages. Most people use checklists to evaluate against, though experience and instinct are equally important. The inspection report describes, at least, the most important flaws in the site, and may recommend improvements. However, they can cause problems - the creative and technical people in the Web team often resent the intervention of a know-it-all consultant! For more details of these and other methods, try the Information and Design or Usability 1st sites. Thus, marketing people, webmasters and consultants can help with usability, but with drawbacks. Let’s look at that word again, "usability." The start of the word is the most important ­ the bit that’s to do with the users. It is the user’s experience of trying to do something with a site that is the critical thing to get right, and this can only be done effectively by testing with real users.

- How user testing works

A small number of users (five are usually enough ­ see Jakob Neilsen’s Alertbox) are asked, one by one, to perform set tasks using the site. Their behavior is observed (not directed!) by a knowledgeable person sitting in the same room, who takes notes on what they do and say. Sometimes, a video link or two-way mirror allows other staff to watch without disturbing the test.

The users are reasonably representative of the target audience for the site. If there are different target audiences, each must be tested separately. Testing can take place at almost any stage of development ­ in general, the earlier the better. Simple tests of early prototypes allow problems to be fixed, and then the next iteration of the site can be tested. This is a much more efficient use of resources than completing the site and doing one big test at the end.

Summary: Why you need to test with real users

Consultants, marketing and Web development people have their own biased views of what users want or need. But only users can really tell you whether your site meets their requirements.

However experienced the usability consultant who advised you on your site, there will always be some aspects that real users will find problematic. Caroline Jarrett, who’s conducted many tests, says "In a test, at least one user will surprise you by using the site in an unanticipated way." Testing also has a useful side-effect: it can act as a neutral agent to convince your boss, your designers, the marketing team, and the developers of the importance of good usability. While anyone can legitimately question the opinion of the professionals, it takes real nerve to suggest that the intended users of the site don’t know what they need!

- Organizing a user test

I suggest that you choose a Web site (neither your own, nor one you are very familiar with) to practice these steps on, before looking at your own site, with which it’s more difficult to be objective.

1. What is the purpose of this site? Ask yourself what it’s trying to do: is it a mixture of expression (of a brand, community or personality), creating relationships with visitors, providing information, and perhaps selling and/or delivering a product or service?

Note: Different people will often have different ideas of the site’s purpose.

2. Who are the users of this site? Ask yourself who will visit and what they will want to do; what’s their level of knowledge of the subject; will they have visited the site before, etc. Also, think about how quickly they’ll need results, whether for business or pleasure, and will they be doing specific tasks or browsing. You may need to consider Web literacy, visual or motor impairments, gender, cultural and age differences. Come up with a profile of one or more representative users ­ not every visitor, but enough to test realistically.

3. What should be tested? Taking the results of 1 and 2, devise specific tasks. Begin with a simple one (like finding the street address of the organization), then choose a few realistic tasks. Write a plain description of the required result, not instructions on what to do. For example, "order a bouquet of flowers costing about $35 for delivery on Saturday," or "find out the prices of one-week self-catering holidays in June in Orlando" (not: "click on the search button").

4. Who should test this site? If the profile comes up with several exclusive types of user, a test must be devised for each. But assuming you have a single user type, imagine how you could recruit people. Sources might be small ads, customers, friends, friends of friends, family, clubs or associations, work colleagues, or internal staff from an intranet. You may consider recruitment or market research agencies, but the response rate is usually better if there is a personal connection between the recruits and the test organizer.

5. Where should the test be run? You must think about practical details of the venue (hardware, software, seating, internet connection, credit card numbers if requiring a purchase, provision of incentives/rewards, refreshments, toilet facilities etc.) There are companies offering test suites for hire, you can use a meeting room, or you may need to improvise.

6. What should the observer say to the tester? Devise a simple script:

· Greet the tester and explain why he/she is there; 
· Ask for permission to record if you will be doing so;
· Set the tasks one after the other (hint: write task cards to give to the tester during the test);
· Afterwards, ask for general reactions and any questions not covered in the test. Review what you have done so far, and ensure that the proposed test will find out what you want to know. So, are you ready to test?

Hold on: what happens if it’s impossible to finish the tasks in a reasonable time, or you forgot some vital part of the preparation? Ah yes ­ a rehearsal would be an excellent idea!

7. How do I run a rehearsal? Find a friend or colleague. Do the whole thing as you would with a test user. Check the timing (tasks shouldn’t take longer than an hour), and make any adjustments necessary.

- Running the test

It’s crunch time ­ you’ve done your planning, preparation, and practicing, so it’s time to get out there and do it for real! Give yourself a whole day, or better yet, two half days to run the tests ­ it’s difficult to remain fresh when you repeat the test for the fifth time. As well as time for the tasks, you need to allow for welcoming and debriefing, plus a short break to reset the computer and tidy up ready for the next test.

Some points to note:

· Try to keep to the set tasks. Discourage testers from inventing their own, but note what they wanted to do. 
· Be relaxed and reassuring ­ testers often feel they, not the site, are being tested. Be interested in what they say and do, take notes, and ask them to repeat any comment or action that wasn’t clear.
· Do not question users too much, or you’ll unconsciously direct them or break their concentration. If you need to ask questions, don’t put blame on the tester, say "how did this page make X difficult for you?" rather than "why did you find X difficult?"
· Listen!

- Writing up the results

Observers' notes must be consolidated into a report detailing the problems encountered by testers, making comments on why they occurred, and perhaps how they could be solved. You'll need to interpret some of the findings using your own understanding of usability and the particular characteristics of individual testers. 
For example, testers less familiar with the site, or the Web in general, often have different problems than more savvy testers. 
As well as describing what went wrong for the users, the report can draw attention to similar places in the site where the same problems might be anticipated. On a positive note, it should recordany favourable comments made during the test or the debriefing.

- Making use of the results

It's no good doing user tests if nothing happens as a result, so do all you can to make sure that action is taken on the results. You may be able to help by prioritizing items according to how long they'll take to fix and how much they'll improve the site. As with most such things, the 80/20 rule applies.

- Conclusion

The ultimate purpose of your Web site should be to help your visitors find the information, product, or service that they want, quickly and painlessly. Professionals of all kinds find it difficult to put themselves in the shoes of ordinary users, so it is essential to ask users what their problems are. Although running usability tests will cost a few thousand dollars, an unusable commercial site will lose far more than that by driving away visitors. So, go for it!.

[voltar]