Harry Brignull from 90 Percent of everything has previously written about cheap ways to make usability testing “sleds” for your mobile devices.
I think these findings have been “found” before, but its good to reassure the researchers that our assumptions are still valid.
- 5 Second Usability Tests: Ratings of website usability after only 5 seconds are the same as those after 10 minutes.
- Unmoderated Usability Data is Mostly Reliable: Data from remote usability test takers is rather similar to lab based studies except for task-times which differ more substantially.
- Cheaters: Around 10% of paid usability testers will cheat on your test by rushing through the questions just to receive the honorarium.
- The Geometric Mean works better than the median for reporting the best middle task time for sample sizes less than 25.
- Usability accounts for at least 30% of customer loyalty: Net Promoter Scores correlate highly with scores from the System Usability Scale (SUS).
- Users Self-Reporting Problems: Users are able to find and report around 50% of the problems usability professionals find. Just asking users to report what problems they encountered, how severe they are and potential fixes can be a cheap and effective complement to other usability activities.
- Survey respondents prefer the left-side of the rating scale. The way you order your response options matters. People generally lean toward responses that are on the left-side. If you have more favorable responses (e.g. Strongly Agree) on the left you’ll get a slightly inflated score.
- Asking users to rate task-ease during a task lowers ratings: If you give users only five seconds to complete a task they will rate the task as much more difficult than those who are given no time limit. Contrast this finding with the 5 seconds tests results which shows that user attitudes about usability are different at the task vs. whole website level.
- Making survey questions more extreme will generate more disagreement: Scores will be higher if questions are all extremely negatively worded and scores will be lower if all the questions are extremely positively worded.
- Usability problems are almost 10-times more common on business applications than on websites
As usual, our usability leaders have shown a light on the shortcomings of the latest user interfaces.
I don’t necessarily take these reviews as an admonition of all that is gestural, but as cautionary tales for designers moving forward.
Jakob Nielsen, Kinect Gestural UI: First Impressions
This PPT touches on many of the key points of how designing for mobile provides key opportunities to not make the same mistakes we made in the early 90s with the web. Namely: design to what mobile does well: Places, context, temporal and spatial relationships.
I am excited to go to Boulder and hope to attend her workshop.
Over the course of the last month, I did four focus groups exploring mental models and gathering conceptual feedback on how users wanted a company to present communication. This next month I will be doing something similar for another company where we will not only gather user expectations, but also determine if there is even a need for the product or an expectation of how that product would be presented.
I find that in the focus groups, more than any other type of usability research that marketing and market research, specifically are integrated into the questions, the responses and the analysis. Continue reading