Usability & Analysis Tools

Some images, text  and additional information was originally posted by Tom Walker. Tom works with a UK based specialist in Epson original ink cartridges and other printing supplies where usability and conversion rates are under constant examination. You can read more of his writing about design on their blog.

It was posted as an article on Sprye studios at:

Throughout my career of providing usability services to companies, I have found that different situations, applications and challenges, require different tools that will work best for what the client needs.

The more options I come in contact with, the more I realize that one of the best things I can provide to my clients is a clear recommendation of what works best for their particular situation.

There are new additions to those tools, as well as some old reliable tools (with recent software upgrades,) listed below. I invite visitors to comment on each of them about when you may have used them, what you have used them for and suggestions of best practices for collection and analysis of the information that will best define a great user experience.

Usability & Feedback Tools

1. Silverback (free for 30 days then $49.95)


Before Silverback, people used to film user reactions to their sites with camcorders during testing, a slow an arduous process involving hours of video editing, not to mention intimidation of test subjects. Silverback, a fantastic app for Mac OSX (with iSight or equivalent) lets you film how a user responds to your site and track their clicks too. You can set chapter markers in the video, when something interesting happens, simply by pressing the “+” button on the Apple remote.

2. Usabilla (free for 5 pages)


Test your web page at any stage in the design process with Usabilla. Simply upload the URL or a picture from your hard drive, choose from predefined test questions or create your own, and invite people to participate in the study by emailing them a link, embedding a widget in your website, or simply pressing the Twitter or Facebook button.

Once you’ve done that, sit back and wait for the data to start pouring in. Usabilla tracks where participants click on your web page, recording the results of different questions in different colors. Participants can also add notes to clicks on the page, which you can view easily.

3. Clixpy ($5 for 100 recorded sessions)


Clixpy is a usability tool which tracks everything from mouse movements, clicks, scrolling and form inputs on your actual site. It’s essentially a screen recorder, installed by simply inserting a few lines of JavaScript into your site’s HTML. It records exactly how individual users interact with your site, allowing you to play back the videos when you like.

4. Crazy Egg ($9 for basic package)

Crazy Egg

Enter your site’s URL and the number of visits you want to track into Crazy Egg. It will generate a small chunk of JavaScript code to be inserted into your site’s HTML. Once inserted, Crazy Egg creates beautifully designed heatmaps, showing where users are clicking on your site, and confetti, showing where users are clicking based on different search terms, browsers etc. Crazy Egg tabulates all these results too, making them easier to digest.

5. Five Second Test (free basic package)

Five Second Test

The Five Second Test comes in two flavors: Memory Test and Click Test. The Memory Test gives participants a meager five seconds to look at your web page or design before attempting to recall specific elements. The Click Test gives users an equally scant five seconds to click on an element that you identify. Simply upload your page or design, choose which test you wish to run as well as certain parameters, and invite friends, colleagues and randoms to take part in your test with a unique link.

6. Userfly (free basic package)


Similar to Clixpy, but with a more attractive interface, Userfly records everything that each visitor to your site does, listing each recorded visit for you to replay individually. As well as listing how many user sessions are captured, it clearly shows next to each one how many pages were viewed in each session and for how long. Simply select a session and it pops up in a new window. Press play to gain a valuable insight into how that specific user interacted with your site.

7. Morae ($1,495)


Morae is a far more comprehensive tool that the others in this list and can be used to gather feedback on more than just websites, but that’s to be expected considering its rather hefty price tag. $1,495 buys you a bundle consisting of Morae Recorder, Observer and Manager, which can be installed on up to three different computers.

Using cameras, Recorder records how a specific user interacts with your site. Other people can watch this live on their computers, making notes if necessary, using Observer. Finally, video, audio and computer data, as well as notes are synced up and saved as an RDG file, which is then viewed, analyzed and shared in Manager.

8. Feedback Army (from $10)

Feedback Army

If you want to get feedback on a certain aspect of your site quickly and inexpensively, you need the Feedback Army. $10, paid with your credit card or Paypal account, buys you 10 responses to any questions you submit. There’s no guarantee how quickly you’ll see a response, but usually it only takes an hour or so. The more questions you ask, the longer it takes to receive replies.

9. Kampyle (free basic package)


Kampyle is a highly effective tool for getting feedback on your site. Once installed, an attractive green “Feedback” tab is added to the side of your chosen web page. When visitors click on it, a survey pops up for them to complete. You can manage all your feedback in a really intuitive way on the Kampyle website.

10. Ethnio (first 20 recruits are free)


Ethnio isn’t a usability tool in itself, but it’s a great way of recruiting people to take part in usability research and so is worthy of a place in this list. Once signed up to Ethnio, you can create a screener, which is added as a pop-up to your site, to find suitable respondents for research. Once somebody signs up to take part, you’re alerted and can choose to email them or just call them there and then. It’s all done remotely so you don’t need to meet any strangers face-to-face.

11. Open Hallway ($49 per month)

Open Hallway

Accessible from anywhere, Open Hallway is an entirely browser based tool which tests sites for usability with a minimum of fuss. It couldn’t be simpler. You just sign up, enter the URL of the site you wish to test, issue instructions to the tester and click to generate a link, which can be emailed to potential participants. The tester then clicks the link, reads the instructions and presses “Start” to begin a recording of their session. The session is finally uploaded for you to view, complete with screen-captured video and voice recording.

12. Concept Feedback (free – premium concepts are $9.99) UPDATE APRIL 13, 2016: This site is now closed down

Concept Feedback

Concept Feedback let’s you post design concepts that people can then review and give you feedback on. This is a great tool if you’re looking to improve your current site design or are working on a redesign. Getting impartial feedback from members can help improve your design tremendously.

Conversion Tools

13. Google Website Optimizer (free)

Google Website Optimizer

Google Website Optimizer lets you test anything from headings, layouts, images, text, buttons and much more. It splits visitors into groups, presenting each one with a different combination of these elements to see which one performs the best. Known as multivariate testing, this is a great way to be sure that your site design is as effective as it could and should be.

14. Site Tuners (free consultation)

Site Tuners

Google are not the only ones providing effective multivariate testing. Site Tuning offers a similar service, only with packages that can be designed to suit individuals. Besides multivariate testing, Site Tuners provides conversion consulting, whereby experts quickly review your site, suggesting changes to be made to landing pages and the best elements to test. You can also access free documents and resources for info.

15. Webtrends Optimize (call for quote)

Webtrends Optimize

Webtrends Optimize offers various tests, from A/B tests, which involve splitting visitors into two groups, testing one variable at a time, to multivariate testing and more sophisticated full factorial and fractional factorial tests, which can give you results without the wait. Optimize is really slick and comes with great support.

16. Omniture Test&Target (call for quote)

Omniture Test&Target

In addition to standard A/B and multivariate testing, Test&Target lets you test adverts to understand their impact on conversion rates and user experience. Furthermore, it lets you segment your visitors according to criteria including URL, geography and visitor behavior. Once segmented, you can change ads, offers and more based on each segments’ unique interests.

17. Google Site Search (free)

Google Site Search

Adding an internal Google Site Search feature to your site is one of the easiest ways to increase conversion rates. Don’t ask visitors what they want, let them tell you every time they Search. Not only will the usability of your site improve, by analyzing the search results you’ll be able to see which elements of your site are most in demand and which are the hardest to find.

18. ClickTale (free basic package)


Like Userfyl and Clixpy, ClickTale lets you record and view how each visitor navigates their way around your site. Where ClickTale really comes into its own, however, is with its Form Analytics function, which gives you exactly the information you need to create the most user-friendly forms, boosting conversion rates. Find out which fields take too long to complete, which generate errors and which fields cause visitors to abandon the form altogether.

19. Google Analytics (free)

Google Analytics

There are various companies offering incredibly detailed website analytics, but for most people, Google Analytics is perfectly sufficient. Not only does Google Analytics produce detailed reports, including a wealth of information about where your visitors have come from, what they click on once they get to your site and which pages are most popular, it monitors these reports constantly, alerting you to any sudden changes. You can set yourself conversion goals and keep tabs on your success.

20. Feng-Gui (free)


Upload an image of a web page to Feng-Gui and it will create a heatmap showing which aspects of the design will get the most attention. What’s unusual about this tool is that, rather than using human researchers, it simulates human vision using algorithms. You can find out the ideal places to put buttons and banners to get them noticed.

What You Get From a Heuristic Evaluation

originally posted by Dana Chisnell on UX Magazine at


What You Really Get From a Heuristic Evaluation

Every user experience researcher I know gets requests to do heuristic evaluations. But it isn’t always clear that the requester actually knows what is involved in doing a heuristic evaluation. That happens. If I had a dollar for every time someone called asking for a focus group when what they ended up needing was a usability test, I could take a very nice holiday on Aruba.

They’ve heard the buzzword. They might have heard that Jakob Neilsen had something to do with the method, so that adds something to the appeal (though I’m not sure what). And they know that someone who they hope is a “usability expert” can just tell them what’s wrong. Right now.

Some clients who have asked me to do heuristic evaluations have picked up the term somewhere but often are not clear on the details. Typically, they have mapped “heuristic evaluation” to “usability audit,” or something like that. It’s close enough to start a conversation.

Unfortunately, the request usually suggests that a heuristic evaluation can substitute for usability tests. I chat with the person, starting by talking about what a heuristic evaluation is, what you get out of it, and how it compares to what you find out in a usability test.

How do you do a heuristic evaluation?

Let’s talk about what a “classic” heuristic evaluation is. When Jakob Nielsen and Rolf Molich published the method in 1990, these two really smart guys were trying to distill some of the basic principles that make a user interface usable to its audience. They came up with 10 “accepted usability principles” (heuristics) that, when multiple evaluators applied them to any UI, should reveal gaps in the design of the UI that could cause problems for users.

Armed with the Nielsen checklist of accepted usability principles—heuristics—someone who had never seen the target UI before, and who was not necessarily knowledgeable about the domain, should be able to determine whether any UI complied with these 10 commandments of usable UI design. If three or four or five people sat down for an hour or two and inspected an interface separately, they could come up with piles of problems. Then they could compare their lists, normalize the issues, and then hand a list off to the engineers to go fix.

What do you get out of a heuristic evaluation?

Let’s say that the person who called me the other day was asking for a review in the form of a heuristic evaluation to resolve a conflict on the team. The conflict on this team was about the page flow: What should the order of steps in the process be? The same as site X, or the same as site Y? Should the up-sell be at the beginning or the end of the purchase process? “Could you please review the UI and just tell us what to do because we don’t have time and money to do a usability test?”

Several of the Nielsen heuristics might apply. Some probably don’t. For example, did the success of the page flow require users to remember things from step to step (recognition rather than recall)? Were there any shortcuts for return customers (flexibility and efficiency of use)? Where might users get bogged down, distracted, or lost (aesthetic and minimalist design)? By applying these heuristics, what have we found out?

The flow might require people to remember something from one step to another. The way the heuristic is written, requiring this of users is always bad. But it might not be.

The flow might not have shortcuts for expert users. The way the heuristic is written, not having shortcuts is bad. But it might not be.

There may be places in the flow that slow people down. The way the heuristic is written, you always want users to be able to do tasks quickly. But you might not.

And I don’t think we have resolved the conflict on the team.

When applying what I call “checklist usability” in a heuristic evaluation to learn what the flaws and frustrations of a design might be, the outcome is a determination of whether the UI complies with the heuristics. It is an inspection, not an evaluation. It is not about the user experience. It’s not even about task performance, which is what the underlying question was in the team’s conflict: Will users do better with this flow versus that flow? If we interrupt them, will they still complete a purchase? Any inspection method that claims to answer those kinds of questions is just guessing.

A team may learn about some design flaws, but the frustrations could remain stubbornly hidden—unless the reviewer has already observed many, many users trying to reach goals using this site or process, or something very like it in the same domain. Even then, there’s a huge risk that a single inspector or even a small group of inspectors—who are applying very general guidelines, are not actually using the design as part of the inspection, and are not like the users—will miss flaws that will be task-stoppers. Worse, they may identify things that don’t comply with the heuristics that should not be changed.

How does heuristic evaluation compare to usability testing?

Heuristic evaluation was codified around 1990, at a time when it was expensive to get access to users. It was common for people to have to be trained to use the technology being evaluated before they could sit down in a usability lab to perform some tasks. The whole concept of even having an interface for end-users was pretty new. Conventions were just settling into place.

Usability testing has been around since at least the 1980s, but began to be widely practiced about the same time Nielsen and Molich published their heuristic evaluation method. While usability testing probably needs some updating as a method, the basic process still works well. It is pretty inexpensive to get access to users. UIs to technology are everywhere. For most of the applications of technology that I test, users don’t need special training.

Heuristic evaluation may help a team know whether their UI complies with someone else’s guidelines. But observing people using a design in a usability test gives a team primary data for making design decisions for their users using their design—especially in a world evolved far beyond command line entry and simple GUIs to options like touchscreens, social media, and ubiquitous connectivity. Separately and in combination, these and other design decisions present subtle, complex problems of usability. For me, observing people using a design will always trump an inspection or audit for getting solid evidence to determine a design direction. There is nothing like that “ah ha!” moment when a user does something unexpected to shed light on how well a design works.

Tristream Best Practices

Just because the information is provided to decision makers, doesn’t mean that they will make the right choices.

The suggested process to avoid that:

Translate business requirements and user requirements into features, and then prioritize them.

Two key points:

  • The best teams develop a features specification, which flows from the business requirements document(s) and the user requirements document(s).
  • The best teams prioritize the features based on rankings from three sources: business, users, and IT. The input is typically in the form of numeric ranking; in order of importance from business and users, and in order of doablility from IT.

more at the Tristream Blog

A Moment in Time

How is it that so much time can go by without even noticing? There was a time when I focused on my online identity multiple times during each day. I was posting news, photos, videos and observations all the time. Quite often I would be sure that the blog was updated with everything I was looking at and all the latest cool gadgetry.

Somehow, with the birth of my daughter and starting a large contract with Kaiser Permanente, I have moved away from this largely cathartic activity. I cannot let this happen and will be rejuvenating my blog life in upcoming months.

Please test your site for usability

It is amazing to me, as a usability researcher, that there are still companies that do not test their sites for usability.

While I know that hiring professional usability researchers may be out of the budget for some smaller companies, it is possible (and highly recommended) that you they at least put their designs in front of a few users and fix the basics.

Steve Krug, who wrote, Don’t Make Me Think”, still standard reading for designers, web developers & usability professionals, has written another book.  It is essentially a companion book to his first, very successful book on usability . While the first book focused on design, this new book is all about testing.

Rocket Surgery Made Easy“Rocket Surgery Made Easy” explains just how easy it is to do usability testing on your own.

I highly recommend it for those who think that usability testing is too difficult or too expensive and so end up putting your website out there with all the horrible problems that make the internet difficult to navigate and information hard to find.

While this book does lay out all the things that designers and web developers would need to do the testing themselves, I am not concerned that we usability  specialists will lose much work, as there are still plenty of more difficult usability issues, often related to enterprise websites as well as internal applications that will require larger tests as well as in depth studies that Steve Krug does not cover in this book.

There is a companion website for the book with additional downloads and videos.