Date: October 5, 2011
Speaker: Thomas S. Tullis — Fidelity Investments
Usability labs and similar settings have served us well for 30+ years of research and practical study of human interfaces. But it’s time to move beyond that and leverage the power of the web.
In the early days of computers, when users tended to be much more uniform, usability labs and their associated smaller numbers of study participants worked well. The internet, web, and proliferation of mobile devices have changed things. It’s hard to imagine someone who wouldn’t be a candidate user for the applications that many of us design and develop these days. We also know much more about what works and doesn’t work for users. Now, instead of being used to uncover major design issues, usability testing is often used to identify small design improvements that can make a big impact to a company’s bottom line.
Thomas will present case studies and discuss the pros and cons of some of the latest trends in usability testing, from “crowd-sourcing” to automated techniques for metric collection.
About the Speaker
Thomas S. Tullis., Ph.D., is Vice President of Usability & User Insight at Fidelity Investments. He joined Fidelity in 1993 and was instrumental in the development of the company’s usability team and a state-of-the-art Usability Lab. He received his B.A. from Rice University, M.A. in Experimental Psychology from New Mexico State University and Ph.D. in Engineering Psychology from Rice University.
With more than 30 years of experience in human-computer interface studies, Thomas has published over 35 papers and holds eight U.S. patents. Prior to Fidelity, he held positions at Canon Information Systems, McDonnell Douglas, Unisys Corporation, and Bell Laboratories. He is co-author (with Bill Albert) of the book Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics, and co-author (with Bill Albert and Donna Tedesco) of the new book Beyond the Usability Lab: Conducting Large-scale User Experience Studies. He is also an Adjunct Professor at Bentley University, where he teaches in the graduate program in Human Factors in Information Design.
Oct. 5 Meeting Notes
taken by Andrew Sweany:
Great presentation last by Tom Tullis from Fidelity.
Some of the key takeaways I captured include:
- Interesting story about the value of using large scale online studies to make rather small design decisions (category label of Retirement & Guidance vs. Guidance & Retirement – the later was the winner
-
Large online studies are good because users of technology are becoming more diverse
-
polleverywhere.com is a fun little service – allows you to set up text based voting on multiple choice questions and see the results real time online. Tom posed several questions to the audience using it during his presentation
-
The most surprising was a test of 4 different designs where variables were text only and different font/background colors, and images and text with different colors. Eye tracking heat map revealed very few users looked at the image at all and the most looked at was simple dark text on a white background (with a bolded box border)
-
Mr Tullis likes usertesting.com for self moderated testing. Costs between $35 – $70 per participant, and the clips captured are the users screen and audio ‘think out loud’ type comments
-
The most interesting remote unmoderated example was around Amazon’s Mechanical Turk where users work on Human Intelligence Tasks (HITs), usually for 5 or 10 cents at a pop. Basically the HITs appear to be like performing a single usability task.
-
A new remote service called Loop 11 can be used for iPad testing
-
At Fidelity they typically do an online study first to test 4 or 5 different designs, then follow up with a more traditional lab study to dive into the details, understand the ‘why’, etc.
-
Have found that users (in his example, Fidelity employees) typically perform worse in the lab and generally give tasks/sites lower ratings than they do when performing the tasks at their desks
Thank you Andrew, for helping us document this meeting!
There is a video of this event. You must be a paid member and be signed in to see the video.
Sign-In / Register
There is a video of this event. You must be a paid member of CHIFOO to see this video.
Join as a Member!
Why It’s Time To Move Beyond the Usability Lab from CHIFOO Communications on Vimeo.