Top Ten Usability Findings of 2010

I think these findings have been “found” before, but its good to reassure the researchers that our assumptions are still valid.

Keep these, posted by Jeff Sauro on measuring usability, in mind as you design your next research project.

  1. 5 Second Usability Tests: Ratings of website usability after only 5 seconds are the same as those after 10 minutes.
  2. Unmoderated Usability Data is Mostly Reliable: Data from remote usability test takers is rather similar to lab based studies except for task-times which differ more substantially.
  3. Cheaters: Around 10% of paid usability testers will cheat on your test by rushing through the questions just to receive the honorarium.
  4. The Geometric Mean works better than the median for reporting the best middle task time for sample sizes less than 25.
  5. Usability accounts for at least 30% of customer loyalty: Net Promoter Scores correlate highly with scores from the System Usability Scale (SUS).
  6. Users Self-Reporting Problems: Users are able to find and report around 50% of the problems usability professionals find. Just asking users to report what problems they encountered, how severe they are and potential fixes can be a cheap and effective complement to other usability activities.
  7. Survey respondents prefer the left-side of the rating scale. The way you order your response options matters. People generally lean toward responses that are on the left-side. If you have more favorable responses (e.g. Strongly Agree) on the left you’ll get a slightly inflated score.
  8. Asking users to rate task-ease during a task lowers ratings: If you give users only five seconds to complete a task they will rate the task as much more difficult than those who are given no time limit. Contrast this finding with the 5 seconds tests results which shows that user attitudes about usability are different at the task vs. whole website level.
  9. Making survey questions more extreme will generate more disagreement: Scores will be higher if questions are all extremely negatively worded and scores will be lower if all the questions are extremely positively worded.
  10. Usability problems are almost 10-times more common on business applications than on websites

Social Statistics Overstate Findings

This latest ESP study from Cornell brings into focus an issue that we statisticians and behavioral analysts have been aware of for years; considering a finding “statistically significant” if there is less than 5% probability that the finding could have been found by chance does not take into account ALL the findings that could have happened by chance. Therefore, we get lots of results that are found later to be, at best, inaccurate, but more often, totally false (I’m looking at you, Wakefield!).

Continue reading

Human-Computer-Human Interactions

Something about that strikes me. I like the fact that it points out the interfaces and blockades we have with each other as we depend more and more on the devices, be they mobile or not, that we carry around with us everyday. using them instead of, to augment or to avoid interactions with other humans.

we interface with the machine. the machine interfaces often with another machine (or two) and then interfaces with another human. the new internet age game of telephone. what gets lost is… well…the humanness of the interaction.