Chapter Introduction | Web Resources | Assignment
Comments | Teaching Materials
Imagine you have designed a new shared web space intended for advertising second-hand
goods. How would you find out whether householders would be able to use it to find what
they wanted and whether it was a reliable and effective service? What evaluation methods
would you employ? In this chapter we describe two very different approaches—usability
testing and field studies. The first you would use to evaluate its effectiveness, ease of use, and satisfaction, and the second you would use to assess how it would support householders finding desired items. The first, usability testing, takes place mainly in a controlled environment; the second, a field study, examines how a product is used in natural home environments.
Typically, the performance of users interacting with a product is measured for set tasks
in a usability test. This allows the system’s effectiveness to be assessed and usability problems to be identified, e.g. problems printing a document or finding information. Usability testing varies in terms of the amount of control over the study. At one end are experiments,
typically carried out in laboratory settings, that test user performance (e.g. reading) on one
or two variables (e.g. font size, resolution) on a number of participants. At the other end
are more opportunistic studies, that investigate a subset of usability and user experience
goals, e.g. satisfaction, fun, frustration, for a handful of users. Between these two extremes
are controlled tasks that examine such things as the ease of searching for and locating
Recently, it has become more common for field studies to be conducted in natural
environments, such as homes, offices, and outdoors. This is especially so for the evaluation
of new technologies, such as mobile devices. Observation, diaries, and interviews are commonly used methods. Importantly, they take place in the setting where the product is used.
The main aims of this chapter are to:
Explain how to do usability testing through examples.
Outline the basics of experimental design.
Discuss the methods used in usability testing.
Discuss the role of field studies in evaluation.