Top Ten Usability Findings of 2010

I think these findings have been “found” before, but its good to reassure the researchers that our assumptions are still valid.

Keep these, posted by Jeff Sauro on measuring usability, in mind as you design your next research project.

  1. 5 Second Usability Tests: Ratings of website usability after only 5 seconds are the same as those after 10 minutes.
  2. Unmoderated Usability Data is Mostly Reliable: Data from remote usability test takers is rather similar to lab based studies except for task-times which differ more substantially.
  3. Cheaters: Around 10% of paid usability testers will cheat on your test by rushing through the questions just to receive the honorarium.
  4. The Geometric Mean works better than the median for reporting the best middle task time for sample sizes less than 25.
  5. Usability accounts for at least 30% of customer loyalty: Net Promoter Scores correlate highly with scores from the System Usability Scale (SUS).
  6. Users Self-Reporting Problems: Users are able to find and report around 50% of the problems usability professionals find. Just asking users to report what problems they encountered, how severe they are and potential fixes can be a cheap and effective complement to other usability activities.
  7. Survey respondents prefer the left-side of the rating scale. The way you order your response options matters. People generally lean toward responses that are on the left-side. If you have more favorable responses (e.g. Strongly Agree) on the left you’ll get a slightly inflated score.
  8. Asking users to rate task-ease during a task lowers ratings: If you give users only five seconds to complete a task they will rate the task as much more difficult than those who are given no time limit. Contrast this finding with the 5 seconds tests results which shows that user attitudes about usability are different at the task vs. whole website level.
  9. Making survey questions more extreme will generate more disagreement: Scores will be higher if questions are all extremely negatively worded and scores will be lower if all the questions are extremely positively worded.
  10. Usability problems are almost 10-times more common on business applications than on websites

Nobody Reads Privacy Policies

Facebook is not the only site that tracks, shares and sells your data. You allow it everytime you click through that agreement. When was the last time you actually read through the whole agreement before saying yes? Never? Well, apparently you are not alone.

According to a study done by SelectOut, most people do not read them. There is a reason why: They are too long. Take a look at the infographic below to see what the averages are for about 1000 of the top sites.

The longest privacy policies among the top 1,000 websites would take around 45 minutes to read. The average policy takes around 10 minutes to read.

And while most of the websites (72%) allow users to opt out of tracking mechanisms, around 40% require their users to take a few extra clicks to the Network Advertising Initiative’s website to opt out.

Should privacy policies and terms of service be short and sweet enough for users to actually read them, or do you think that would increase tracking opt-outs enough that it would hurt the companies in question?

Design Your Own Mobile Application With These Nine Easy Steps

I especially like the idea of mocking up in HTML5 for prototyping and doing informal research by observation and interviewing in the bar. Great ideas, Patrick!

Posted by Patrick Neeman on UsabilityCounts

2011 is the year of mobile.

This year is the tipping point that’s really going to turn the World Wide Web into a “platform doesn’t matter” medium. If you’re doing User Experience and you don’t have a mobile app in your portfolio, you’d better get cracking.

We’ve been working on a few applications as prototypes (iPad, iPhone, and Android), and I’m at version 1.1 of Pick An Excuse (finally iOS 3 compatible). I’ve made a few mistakes along the way but have found that designing apps for mobile devices isn’t that difficult.

If you have even an inkling of an idea, you should work on your own application — just for the experience.

It’s All about Context, Baby

I do all my best user research at the local bars here in San Francisco, California. It’s a fairly connected crowd, and there’s always someone that has a device out. They always have an opinion about their phone, because people identify with their devices more so than other technology. Think about it, when’s the last time anyone talked about what television brand they own?

Watching people use devices at Tony Nik’s is one of my favorite pastimes. They send instant messages, answer emails, and check on local restaurants, all while having a Dark and Stormy. Fascinating.


It’s because of the context. They aren’t sitting at home eating Bon Bon’s while using the phone, they’re at a bar, at a restaurant, at work, or somewhere else. The phone may not be the primary focus of their attention.

The fun thing about it is that all I have to do is ask a few questions, and they’ll tell me everything they love about their device — without having to buy them a drink.

Keep It Simple

There’s nothing worse than downloading an application that’s overly complicated. I was asked to download an application that did some really cool things, but it required quite a few clicks to get around. It was obvious the developers weren’t optimizing for the user experience and was stuffing as much functionality in the app as possible. Sometimes the best applications do one thing really well.

My advice: come up with something that’s silly or fun and that doesn’t have any profit potential whatsoever. My application is a wonderful conversation starter that compels me to grab people’s iPhones and download the application (and I’ll be doing the same for Android too!).

Additionally, set a budget for how much you want to spend. There’s nothing that makes you want to limit the feature set more than spending your own money.

Make It Social

It’s a phone!

It’s for talking (or texting) to people!

It’s a social device!

It’s connected to the Internet, dammit!

The Pick An Excuse application is extremely social. You can send excuses to friends, share on Facebook, tweet excuses, and send them as texts. Most of the best mobile applications have some kind of social component (Foursquare, Facebook) that allows for some kind of integration and/or mashup.

There’s also a ton of services out there that have almost open APIs that allow you to pull in interesting data. For example, Yelp! allows you to pull in reviews as long as you credit them. Foursquare has a bunch of APIs. There are many ways you can create an application that pulls data from different sources, make it fun, and have it seem complete and polished.

Adjust the User Experience

There’s nothing worse than pulling up a website, and you can’t read the small type on a mobile device.

You’re designing for a device that’s not much bigger than the size of a business card. Additionally, Most of the applications that should be designed for it need to be able to be used with one hand. There’s always the other hand holding the device, so designing for simple interactions is they key. Swiping is good, typing is not.

Surfing the web or using an application is a completely different experience. Certain things don’t work. For example, you can’t really hover over things, so you don’t have to design for those interactions. Lots and lots of typing is a pain in the neck on most devices too, so you have to simplify purchasing to the exact information you need. Large websites are a bad idea, because too many clicks mean pages that will never be visited.

The design has to be flexible too (Oh my god, we’re back to multiple browsers! At least most of them are using WebKit), but it’s mostly around the width of the device to make sure things line up.

Don’t Reinvent the Wheel

One of the troubling things I find about a lot of mobile applications is the need to design a “different” user experience. Apple has provided this wonderful library of features, icons and user experience guidelines, and yet I see applications that have icons in a different location and custom actions that aren’t nearly as easy as they should be. Let’s be honest, Apple knows better.

Unless you’re designing a game that requires an immersive experience, using a paradigm that’s different from all other applications means the user is going to have to relearn your application.

That’s not necessarily a good thing.

Apple’s gone to the trouble of producing top notch guidelines that almost guarantee a good user experience. The closer you get to the standards, the better chance you have getting the application approved on the first try (I did). That’s very important in keeping the cost down.

Why reinvent the wheel (and piss off Steve Jobs in the process)?

Mobile Necessarily Doesn’t Have to Be Native

One of the big assumptions about developing for mobile is that the applications have to be in the iPhone application store to gain traction. It helps, but it’s not true. As the mobile revolution takes place, you’ll see more and more websites that have a separate display for mobile devices.

Granted, most of them aren’t very good now (go to MSNBC now to see an example), but as time goes on, people will get better at designing for mobile.

One of the next applications I’ve been working on is actually a mix of web and native. We’re using the WebKit container to display the content and native functionality for some of the features. It creates this interesting hybrid where you have the flexibility of the web with some of the speed of a native applications. For example, we’re caching many of the images and the CSS.

Even though it’s technically a web page, it looks like a iPhone application.

And that’s cool.

Mock It Up in HTML (HTML5 Even!)

The best thing about the device is that it can connect to the Internet. That means you can mock up the user experience with simple HTML pages, and test it on your phone. Apple has produced a kick-ass guide for developing for the iPhone, and you can replicate much of the experience as you would have with a native application in HTML. You can come up with the idea yourself in rough form, and then show it to a developer when done and say, “Hey, this is what I want built.” There’s nothing better than a visual example.

Mock it up.

Test it with friends.


Because it’s a mockup, it doesn’t have to be perfect, but you have a pretty good idea if it’s going to work or not. That way, you can work out some of the ideas before you get to development, and that will save you time and, most importantly, money.

Get a Technical Architect

Here’s my story:

The worst thing about new technology is there are all these small details you have to consider before implementation. I ran into that with the first application I designed, Pick An Excuse.

The implementation went well. I gave them pixel perfect mockups, and for the most part, the implementation went pretty smooth.

And then…

The application was designed for iOS 4, which cut out half of the market. There wasn’t a single iOS 4 feature. A bunch of my friends wanted to download the application, but couldn’t. The application didn’t have a single feature in it that required iOS 4, yet I had to pay my offshore firm to downgrade it.

This was because I didn’t discuss what I wanted exactly at the very beginning. It would have been nice if that was a question I was asked during development, but it wasn’t.

The solution: get a strong technical architect that you can discuss every detail with. Devil is in the details especially in the mobile world. I have one now, and we discuss everything about the experience. He’s great, because he fights for the user too.

Have Fun with It

Remember, it’s your passion project (and your money).

Have fun with it. Experiment and play, because there’s nothing better than pulling out your phone out at a bar.