This in-depth activity provides useful experience in using these approaches. As you do the usability testing remember that you need to control the testing environment so that each participant has a similar experience.

A preliminary step in doing this is to work on developing a typical task, or two tasks that you expect anyone using the system needs to do. An obvious example is to book a ticket. Some tasks can only be achieved by following a set order of steps, others can be achieved in a variety of different ways. Then write the task down being careful to make sure that you provide a clear description of the task.

In larger evaluation studies you would be wise to carry out a pilot test with participants who are not going to participate in the main study, so that you can test out the task, your description of it and the test set-up and environment – for example, if you were going to video record the person and log their keystrokes, you need to make sure that the equipment that you will use to capture this data is working properly. Another thing to think about is whether you need to give the participants some activities to get them used to using the system before you conduct the usability test.

In addition to working out a task you need to decide who your participants will be. Who is a “typical user” of this system. Sometimes there are clearly a set of people with similar expertise, experience and demographic make-up who will use the system – for example an accounting system that is designed for trained accountants. Often, as in the case of this system, the potential participant group is quite broad and variable, so how will you select typical participants? Is there such a thing as a typical participant? Almost always you will need to develop an informed consent form. You will also need to pay attention to how you introduce each participant to the task and how you control the environment in which they will be working. Many of us do our usability testing in lab-like conditions that we create by using a small room; few of us have access to a fully-fledged usability laboratory. The activity suggests that you should observe each participant, noting any errors that they make, and the paths that they take, etc., and, if possible, timing them.

This activity is going to be more difficult than you may expect, so try out the method you will use to do this before you work with one of your participants. Before collecting your data think about how you will analyze it. There will be too few participants to do much analysis but think about how you will present your findings and what you would do if you were performing a larger study. When thinking about the in-the-wild study that you will do, focus on the differences between usability testing and in-the-wild studies. What could you learn from an in the wild studies that you won’t know from usability testing.

Based on your knowledge of the requirements for this application, develop a standard task (for instance, booking two seats for a particular performance). Depending on whether there is a list of concerts or performers, our task could be to select two of the least expensive seats to attend a Taylor Swift Concert. The sequence of events could be similar to the following depending on how you designed your prototype:

• Select type of concert of artist from those listed.

• Select preferred date of the concert

• Select preferred time of the performance

• Look for the least expensive seats

• Select two seats next to each other

• Confirm your seat selection

• Decide how you will pay for your tickets (eg which credit card you will use or Apple Pay, etc.)

• Provide credit card payment details

• Select to receive confirmation of your booking by text message.

Our participants are three close friends (two women in their 20s and one man in his 60s), so we won’t prepare a formal informed consent form for them to read and sign. However, we will explain to the three participants that they are participating in a trial activity that will produce data for us to learn how to do product evaluations but the data will not be published or recorded. We will destroy the data after the activity. To be sure that we cover all the issues mentioned in an informed consent form we will use the form contained in the Interaction Design book for guidance but we will not ask our friends to sign it.

It should only take our friends 10-15 minutes to make the booking but we will time each of them. Because one friend lives in Canada and the other two live in the USA we will be especially careful to check if they have problems entering dates into the booking form. There shouldn’t be a problem adding their credit card payment advice but we will also check that carefully. The system is not implemented so we cannot check whether they get text or email confirmations but we will ask them how important this feature is for them.

Since the application is not actually implemented, it is difficult to study it in typical settings of use. However, imagine that you are planning a controlled usability study and an in-the-wild study in a natural setting. How would you do it? What kinds of things would you need to take into account? What sort of data would you collect, and how would you analyze it?

For the controlled usability study we will arrange to do it in a quiet corner of a living room. We will close the blinds so that the participants don’t get distracted by passers-by or other activities going on outside. We will close the door and put a notice on the door so that no one interrupts the study. In addition to timing how long each participant takes to complete the task, we will also note any problems that they encounter and any comments that they make while doing the task. Since this is an informal study we will not video record the participants as it might be disconcerting for them. The data would consist of some numerical data (time to complete the task, number of problems and errors each participant makes, comments about the way the product works and the participants’ reactions to using it. Data from three participants is too little to draw any conclusions about how a population of participants might react to the product, but it might be useful for identifying problems with the current design and evaluation methodology – i.e., it can serve as a pilot study. It will inform us about any problems with the prototype and obvious glitches in our procedure.

The main benefits of a usability study is that it produces reliable data from a controlled testing environment but it would not tell us much about how the participants would use the prototype in their natural environment. In contrast, an in-the-wild study would provide more information about how the participants use the product in natural situations, for example, making a booking while riding the bus to class or in the pub in the evening, but it would not tell us so much about the specific problems that the participants experience with the product or provide quantitative data.