You Say Focus Group, I Say Usability (continued...)

"So," said the project manager. "The focus groups didn't care one way or the other about the copy. It didn't test that well, which is so weird, because I always thought people loved that."

"You have to remember," I said. "Those are focus groups, not usability tests."

I can't tell you how many times this issue arises in one way or another. People have learned the word "usability" and they like to throw it around or use it interchangable with "focus groups," despite the fact that the two things are as different as designing advertising and designing web sites.

Interactive groups that are housed within ad agencies are particularly susceptible to confusing the two issues, probably because they're used to doing focus groups for their advertising, and so assume that they can just throw a web site up in front of a bunch of people and learn what they want to know. So, for the usbility-challenged, a quick primer on the difference between focus testing and usability testing:

If you're observing a group of people's reactions as they are being shown static images of a web site devoid of any actual scenario or explanation of how someone might use something, you are in a focus group

If you are in a room where people are asked how they feel about a particular color, image or the location of an item, you are most likely in a focus group If someone gives you a scenario to act out on a web site or a set of tasks to complete and then asks you open-ended questions about the scenario or task, you are in a usability test, If someone walks you step-by-step through a web site and then asks what you think about it, you are probably in a poorly constructed usability test

This is not to say that nothing good can come out of focus groups. On the contrary, if you're working within an ad agency and they've budgeted for focus groups instead of usability, there are ways to learn interesting IA-related items, but the opportunities for any sort of deep learning are limited.

I had a project recently where the clients had tested an art direction in focus groups before I arrived on the project. To their immense credit, the focus group participants hated the design because the Art Director hadn't happened to include any navigation on the screen. So, even though I personally didn't need a focus group to tell me that navigation is good, it did hit the point home to the Art Director.

The point is, if you're going to subject your designs to focus groups, just remember not to call it usability, and don't expect to actually learn anything about how people are going to use your site. Take the results with a grain of salt. Holding up an image of a screen and asking people how they feel about it isn't going to reveal any deep mysteries of the universe.

- Hana Schank, 10.04