Monday, September 8, 2008
User-Reported Usability Testing
A while ago I heard about UserTesting.com, and then this post in Usability News alerted me to the presence of Userfocus. Both companies offer unmoderated, "crowdsourced" usability testing in which users spend a bit of time with a website and then offer their opinions. Whereas UserTesting.com seems a bit more structured--users are screened ahead of time and paid for answering pre-defined questions, Userfocus seems more open-ended and open to more users. In other words, UserTesting.com delivers more feedback but also a lot of noise, while UserTesting.com will yield less but more useful feedback.
Users are notoriously-bad at self-evaluation, so my first reaction to these services is that they probably are a good first pass but do not substitute for more traditional, moderated usability testing. In some respects, these services seem more like crowdsourced heuristic evaluations, which typically involve "experts" in user interface design rating an interface against a standard set of metrics. Of course, UserTesting.com also provides recordings of each user's screen actions and the "think-out-loud" things that users say, which is a useful check on the self-reported findings.
In any case, these services seem to be filling an important niche of providing cheap and fast usability testing. While not perfect, this may be the only usability testing that many startups and small businesses can afford, and certainly some usability testing is better than no usability testing.
Image from here.