Practical Usability Rating by Experts (PURE): A Pragmatic Approach for Scoring Product Usability

While working at Intel Security, working with colleagues we developed and tested an economical and expedient alternative method for testing usability.

Usability testing has long been considered a gold standard in evaluating the ease of use of software and websites-producing metrics to benchmark the experience and identifying areas for improvement. However, logistical complexities and costs can make frequent usability testing infeasible. Alternatives to usability testing include various forms of expert reviews that identify usability problems but fail to provide task performance metrics.

This case study describes a method by which multiple teams of trained evaluators generated task usability ratings and compared them to metrics collected from an independently run usability test on three software products. Although inter-rater reliability ranged from modest to strong and the correlation between actual and predicted metrics did establish fair concurrent validity, opportunities for improved reliability and validity were identified. By establishing clear guidelines, this method can provide a useful usability rating for a range of products across multiple platforms, without costing significant time or money.

Video Summary

Abstract
We presented the findings at CHI EA in 2016. the abstract is located at: ACM Digital Library

Further Innovation
Here is an article Christian Rohrer wrote with an update to the methodology based on his continuing work Quantifying and Comparing Ease of Use Without Breaking the Bank

Daily UX Stories

Check out the Daily UserZen!

I have been curating my UX Twitter feed for years. (a second feed is in the works) I feel that the stories, suggestions and news that come each day are relevant and interesting. Many years ago I also found the Paper.li service which puts together the feed into whatever layout I like automatically each day.

Not only is it a nice way to view the content, but also highlights different Twitter feeds each day and does some linking back to each company or professional.

UX Career Paths and Job Satisfaction

I made the change to a usability specialist career over 8 years ago and have never looked back. I have been extremely happy with my jobs, my coworkers, the topics and tasks I do everyday are

This article on UX Magazine by Patrick Malecek details job satisfaction by UX practitioners and I found it extremely interesting.

Survey research conducted in the fall of 2010 investigating job satisfaction among usability practitioners and how satisfaction is impacted by employer type, organizational placement, and other factors. Continue reading

Design and Usability Conferences 2011

I am hoping to make it to at least a few of these conferences this year.

SXSW South by Southwest Interactive

March 11-15, 2011, Austin, TX

Conference $675 ($1150 for all access)

Flight about $325

Hotel going fast! About 160 per night ($800)

Plus food stipend

http://sxsw.com/interactive

Computer Human Interaction 2011

May 7-12, 2011, Vancouver, BC

conference cost $760 ($1360 late and for non-members)

flight about $600,

hotel 239 CAD per night ($1440)

plus food stipend

http://chi2011.org/index.html

UPA 2011 , Usability Professionals Association

June 21- 24, 2011, Atlanta Georgia

conference cost $695 ($895 for non-members)

flight about $500,

hotel 173 per night ($692)

plus food stipend

https://www.usabilityprofessionals.org/conference/2011/index.html

Continue reading

Why Don’t Usability Problems Get Fixed?

There are various reasons why usability problems exist in the first place—some simple and some complex. Identifying problems and recommending solutions is not always enough. Unfortunately, the same factors that cause problems in the first place also hinder their getting fixed. The following are some of the most common reasons why usability problems don’t get fixed.

  • Lack of Resources

o   No One Has the Skills to Fix Them
o   There Is a Lack of Time, Money, or Resources

  • Technical Limitations

o   Technical Limitations Make Changes Difficult
o   Vendor Software Is Difficult to Change Continue reading