While working at Intel Security, working with colleagues we developed and tested an economical and expedient alternative method for testing usability.
Usability testing has long been considered a gold standard in evaluating the ease of use of software and websites-producing metrics to benchmark the experience and identifying areas for improvement. However, logistical complexities and costs can make frequent usability testing infeasible. Alternatives to usability testing include various forms of expert reviews that identify usability problems but fail to provide task performance metrics.
This case study describes a method by which multiple teams of trained evaluators generated task usability ratings and compared them to metrics collected from an independently run usability test on three software products. Although inter-rater reliability ranged from modest to strong and the correlation between actual and predicted metrics did establish fair concurrent validity, opportunities for improved reliability and validity were identified. By establishing clear guidelines, this method can provide a useful usability rating for a range of products across multiple platforms, without costing significant time or money.
Video Summary Abstract
We presented the findings at CHI EA in 2016. the abstract is located at: ACM Digital Library
The biggest tool I use for my job trying to figure out what people need and how they use products is my power of observation.
Sometimes it is hard to figure out the nuances of people’s behavior and what they say their wants are. Ideo tried to improve the shopping cart but those are, sadly, not being incorporated in our trips to the stores.
But sometimes, the truth and the need is right in front of your face.
These shoppers aren’t dumb, the user is never wrong! What I see here is that people find it easier to leave the baskets on the ground. It all makes sense to me. Think about how annoying it would be if you had to carry your belongings through the airport.
We have had roller cases for quite some time; why wouldn’t we apply this to our consumer brick & norther shopping? Target Shoppers using their baskets “wrong”
The Windows 8 Blog has finally started giving us some real information about how Windows 8 will work and deal with more complicated issues.
Some of the main complaints users had with all the Windows versions in the past was how files were copied, managed and renamed. It looks like the people on the Windows 8 team at Microsoft are trying some user focused solutions.
I highly recommend taking a look at the blog and getting involved in the conversation. each post seems to reflect that they are actually reading our comments and trying to respond.
For a while it was looking like we were going to have to wait until the Build Conference on September 13 in Anaheim.
One more (I can’t resist) step toward full body interfaces. Great idea for many reasons and situations and as an alternative to keyboards, mouse, joystick, etc. MojoKid writes
“When it comes to controlling your favorite PC title, you’ve got a few options. There’s a mouse. There’s a keyboard. There’s a control pad and the joystick. Now, there’s one more option apparently. Keith McMillen Instruments (KMI) announced today the SoftStep KeyWorx multi-touch foot controller, the world’s first foot controlled digital interface. Available for Mac and Windows, this controller sits on the floor. The company claims that it has multiple uses for gamers, video editors, programmers, data entry professionals, disabled people, repetitive stress syndrome sufferers, etc. It’s both pressure and location sensitive, USB-powered, and contains ten fully customizable keys that remember up to 100 sets of commands for repetitive tasks.”