After working on iPhone applications, I have come up with several methodologies to administer, monitor and analyze iPhone apps. I plan on sharing my findings and musing over the next few months, but wanted to start by sharing some colleagues’ recent articles about the subject.
An article from 24 April 2010 by Liz McMahon
Usability experts familiar with testing websites may find they are faced with some new challenges the first time they carry out a usability test of an iPhone app. The tried and tested methods that worked before will need to be adapted – so here are Pancentric’s top tips on what to expect.
When testing a website, the tester is interacting with a static laptop, which the moderator and observers can easily see, either directly or via a camera. People holding iPhones have a tendency to move them around quite a bit, especially expressive types who gesticulate while talking. If you are recording the session you will sometimes need to reposition the device so that it is in shot – try to be as unobtrusive as possible, and don’t interrupt their flow.
Watch your language
Web terminology is well-defined and understood by the majority of web users – link, click, refresh, etc. But in the world of apps, the language to describe the interactions (tapping, sliding) is new and unfamiliar. Be sure that the tester understands the terminology you are using. Remember, the iPhone being relatively new, testers may be unfamiliar with the terminology and may be more reluctant than web users to admit they can’t complete a task. Reassure your testers by using their language, even if it isn’t the term you would ordinarily use.
Put your testers at ease
Its relatively easy when testing a website to create a ‘natural’ environment where the tester feels comfortable, and not as if they are being studied in a lab. When it comes to testing an app however, testers will likely be asked to hold the device in a particular manner so that it can be captured by the camera. This may feel unnatural. You will need to reinforce the message that each tester’s opinion is important.
Spend time interviewing the testers
Interviewing website testers about their online habits can be predictable, with people typically citing EBay, Facebook, Hotmail, BBC News etc as their favourite / most visited sites. Although some of the same names crop up again in their app incarnations (particularly Facebook, EBay and email), asking testers about their most-used apps can offer great insight into their personality and technical expertise. The wealth of apps available on the market tailored to almost every need means that people will download and use a very individual range of apps.
Remember this is a new field for most people
Most web users today feel comfortable using the web to purchase items and conduct online banking, but paying by credit card via an app is an issue for iPhone users who are not convinced of its security. Talking to app users is much like talking to web users 6 or 7 years ago – many are suspicious of security and distrustful of the unfamiliar names behind the products.
During website usability tests, testers will typically spend at least a few seconds on a web page. This, and the time in between clicking a link to a new page loading, can provide valuable breathing space for the moderator, and an opportunity for the tester to offer more opinions. Apps however can be navigated very quickly, often without time for the testers to fully “think aloud” while completing a task. You’ll need to be very aware of what is happening on the screen – if you can get somebody else to take notes, so much the better. Using a system such as Morae which allows you to mark observations quickly is recommended.
Just do it
As with a website, a usability test on an app can be carried out quickly and cheaply at a testers’ desk, or full-scale in a specialist lab; either way it’s always a valuable exercise. My final observation is that testing an app can be cheaper and quicker than testing a website. This is because testing a website usually involves exploring many user journeys and tasks and tests generally last 60 – 90 minutes, whereas an iPhone app typically has far fewer interactions than a website, leading to shorter and more defined tasks for usability testing. Tests are therefore much shorter, meaning more tests can be carried out in a single day than a typical website test.
Another article discussing how to choose a good company to do the testing (software, stability & usability testing)