Jakob Nielsen

Jakob Nielsen is a pioneer of heuristic evaluation. He is currently principal of the Nielsen Norman Consultancy Group and the author of numerous articles and books, including his recent book, Designing Web Usability (New Riders Publishing). He is well known for his regular sound bites on usability which for many years have appeared at useit.com. In this interview Jakob talks about heuristic evaluation, why he developed the technique, and how it can be applied to the web.

JP: Jakob, why did you create heuristic evaluation? 

JN: It is part of a larger mission I was on in the mid-’80s, which was to simplify usability engineering, to get more people using what I call ‘discount usability engineering.’ The idea was to come up with several simplifi ed methods that would be very easy and fast to use. Heuristic evaluation can be used for any design project or any stage in the design process, without budgetary constraints. To succeed it had to be fast, cheap, and useful.

JP: How can it be adapted for the web? 

JN: I think it applies just as much to the web, actually if anything more, because a typical website will have tens of thousands of pages. A big one may have hundreds of thousands of pages, much too much to be assessed using traditional usability evaluation methods such as user testing. User testing is good for testing the homepage or the main navigation system. But if you look at the individual pages, there is no way that you can really test them. Even with the discount approach, which would involve fi ve users, it would still be hard to test all the pages. So all you are left with is the notion of doing a heuristic evaluation, where you just have a few people look at the majority of pages and judge them according to the heuristics. Now the heuristics are somewhat different, because people behave differently on the web. They are more ruthless about getting a very quick glance at what is on a page and if they don’t understand it then leaving it. Typically application users work a little harder at learning an application. The basic heuristics that I developed a long time ago are universal, so they apply to the web as well. But as well as these global heuristics that are always true, for example ‘consistency,’ there can be specialized heuristics that apply to particular systems. But most evaluators use the general heuristics because the web is still evolving and we are still in the process of determining what the web-specific heuristics should be.

JP: So how do you advise designers to go about evaluating a really large website? 

JN: Well, you cannot actually test every page. Also, there is another problem: developing a large website is incredibly collaborative and involves a lot of different people. There may be a central team in charge of things like the homepage, the overall appearance, and the overall navigation system. But when it comes to making a product page, it is the product-marketing manager of, say, Kentucky who is in charge of that. The division in Kentucky knows about the product line and the people back at headquarters have no clue about the details. That’s why they have to do their own evaluations in that department. The big thing right now is that this is not being done, developers are not evaluating enough. That’s one of the reasons I want to push the heuristic evaluation method even further to get it out to all the website contributors. The uptake of usability methods has dramatically improved from fi ve years ago, when many companies didn’t have a clue, but the need today is still great because of the phenomenal development of the web.

JP: When should you start doing heuristic evaluation? 

JN: You should start quite early, maybe not quite as early as testing a very rough mockup, but as soon as there is a slightly more substantial prototype. For example, if you are building a website that might eventually have ten thousand pages, it would be appropriate to do a heuristic evaluation of, say, the fi rst ten to twenty pages. By doing this you would catch quite a lot of usability problems.

JP: How do you combine user testing and heuristic evaluation? 

JN: I suggest a sandwich model where you layer them on top of each other. Do some early user testing of two or three drawings. Develop the ideas somewhat, then do a heuristic evaluation. Then evolve the design further, do some user tests, evolve it and do heuristic evaluation, and so on. When the design is nearing completion, heuristic evaluation is very useful particularly for a very large design.

JP: So, do you have a story to tell us about your consulting experiences, something that opened your eyes or amused you? 

JN: Well, my most interesting project started when I received an email from a cofounder of a large company who wanted my opinion on a new idea. We met and he explained his idea and because I know a lot about usability, including research studies, I could warn him that it wouldn’t work—it was doomed. This was very satisfying and seems like the true role for a usability consultant. I think usability consultants should have this level of insight. It is not enough to just clean up after somebody makes the mistake of starting the wrong project or produces a poor design. We really should help defi ne which projects should be done in the fi rst place. Our role is to help identify options for really improving people’s lives, for developing products that are considerably more effi cient, easier or faster to learn, or whatever the criteria are. That is the ultimate goal of our entire field.

JP: Have there been any changes in the way heuristic evaluation and discount usability methods are used or perceived? 

JN: I have changed my preferred approach to heuristic evaluation from emphasizing a small set of general heuristics to emphasizing a large set of highly specifi c usability guidelines. I did this because there CINTERVIEW06.indd 2 INTERVIEW06.indd 2 18/06/11 10:44 AM 8/06/11 10:44 AM INTERVIEW WITH JAKOB NIELSEN 3 are now millions of people who perform user interface design without knowing anything about general HCI principles, and it’s diffi cult for these people to apply general heuristics correctly. One of my earliest research results for heuristic evaluation is that the method works best with experienced evaluators who have a deep understanding of usability. All very well, but when a team doesn’t have experienced usability professionals on staff, what should it do? That’s where the specifi c guidelines come into play. When you tell people, for example, that search should be represented on a website by a type-in box on every page and that the box should be at least 27 characters wide, then you are giving them evaluation criteria that anybody can apply without knowing the theory. For example, my group developed a set of 75 usability guidelines for the design of the public relations area of a corporate website. There are guidelines for everything from the way to present press releases online to how to show the PR department’s contact information. These guidelines are based on our own user testing of a broad range of journalists working in newspapers, magazines, and broadcast media in several different countries, so we know that the guidelines represent the needs and wants of the target audience. In real life, PR information is placed on a website by the PR department, and they don’t have time to conduct their own user testing with journalists. Neither does the typical PR professional have any educational background in HCI. Thus, I don’t think that the broad list of general heuristics would do them much good, but we know from several examples that a company’s PR pages get much better when the PR group has evaluated it with the 75 detailed guidelines.

JP: What about changes regarding usability in general? 

JN: The general trend has been toward hugely increased investment in usability around the world. I don’t think anybody has the real numbers, but I would not be surprised if the amount of resources allocated to usability increased by a thousand percent or more from 1995 to 2005. Of course, this is not nearly enough, because during the same ten years, the number of websites increased by 87,372%. In other words, we are falling behind by a factor of about 87. This is why discount usability engineering is more important than ever.

JP: And how do you think the web will develop? What will we see next, what do you expect the future to bring? 

JN: I hope we will abandon the page metaphor and reach back to the earlier days of hypertext. There are other ideas that would help people navigate the web better. The web is really an ‘article-reading’ interface. My website useit.com, for example, is mainly articles, but for many other things people need a different interface, the current interface just does not work. I hope we will evolve a more interesting, useful interface that I’ll call the ‘Internet desktop,’ which would have a control panel for your own environment, or another metaphor would be ‘your personal secretary.’ Instead of the old goal where the computer spits out more information, the goal would be for the computer to protect you from too much information. You shouldn’t have to actually go and read all those webpages. You should have something that would help you prioritize your time so you would get the most out of the web. But, pragmatically speaking, these are not going to come any time soon. My prediction has been that Explorer Version 8 will be the first good web browser and that is still my prediction. The more short-term prediction is really that designers will take much more responsibility for content and usability of the web. We need to write webpages so that people can read them. For instance, we need headlines that make sense. Even something as simple as a headline is a user interface, because it’s now being used interactively, not as in a magazine where you just look at it. So writing the headline, writing the content, designing the navigation are jobs for the individual website designers. In combination, such decisions are really defi ning the user experience of the network economy. That’s why we really have an obligation, every one of us, because we are building the new world and if the new world turns out to be miserable, we have only ourselves to blame, not Bill Gates. We’ve got to design the web for the way users behave.

JP: Finally, can heuristic evaluation be used to evaluate mobile systems and games? JN: I only have direct experience from mobile devices, and heuristic evaluation certainly works very well in this domain. You can identify a lot of issues with a phone or other mobile user experience by using exactly the same heuristics as you would for any other platform. However, you have to interpret the heuristics in the context of the smaller screen, which changes their relative importance. For example, say that a user selects a headline from a list of news stories on a mobile device, the next screen will usually be the full text of the story. You might expect that the standard heuristic “visibility of system status” should imply that the system should provide feedback by repeating the selected headline on top of the story. And that’s indeed what I would recommend for a Web page. But on a small screen, it’s better to devote more of the space to new information and assume that the user can remember the headline from the previous screen. This doesn’t violate the heuristic “recognition rather than recall” because you don’t need to use the exact wording of the headline for anything while reading the typical news story. If the headline were in fact used on the next screen, then it should be repeated, in order to minimize the user’s memory load. Games are a different matter. I haven’t done such a project, so I don’t know for a fact, but I suspect that traditional heuristic evaluation might help on the limited question of evaluating the controls of a game. Games are no fun if you can’t fi gure out how to play them, For example, the heuristic for “consistency and standards”, would indicate that if there’s a certain way to pick up guns, it should be the same for all forms of guns. Similarly if there’s a certain button on the controller that’s used to shoot the gun in all other games then our game should use the same button. However, I don’t think that the standard heuristics would be very helpful in evaluating the gameplay quality of a game. It’s hard to say, for example, whether a game like Civilization should have fewer raging barbarians, or how much more food should be grown on a tile if you irrigate it. It’s possible that one could discover a different set of heuristics to help make such decisions, maybe by studying how successful games designers make their trade-offs. ■