However beautiful the strategy, you should occasionally look at the results. (Winston Churchill)

And isn’t that what usability testing is all about? 

Of course, there are other ways to get that information as well, especially these days. For one thing, you can simply put it out there and monitor the heck out of it. 

A kind of old-fashioned way to do that is Voice of the Customer – popping up surveys, including “give feedback” links, and so on. These days, you can even monitor social media. It’s all good stuff, but there’s also no shortage of possible drawbacks. I already wrote a post going over those in great detail, so I won’t repeat all that here.

You could also take a look at web analytics. (I wrote about that, to some degree, in another post.) The main thing with analytics, though, is that, though you might have tons of data showing exactly where users went and what they did as they actually used your system, you totally lack why they did so. And that can be pretty darn important.

It’s the same problem with A/B testing. Once again, you get reams of real data, all tied directly to the bottom line, and in this case, totally actionable. You are, however, relying totally on correlation – which, in my opinion, means relying totally on conjecture. We know that A performed better, but why? How can we tell with just the numbers? What were users thinking when they went with A?

And then there’s usability testing. It’s the prime way to get feedback before release, but it also provides super-rich data that can be used to completely understand your users, identify and understand their issues, and then come up with legitimate solutions to address those issues. 

That said, if you are getting some kind of feedback, you are way ahead in the game. What really worries me are those companies out there who basically don’t do any of this.

By the way, when Churchill was talking about strategy and results he was actually not talking about usability testing (I know, hard to believe). I would imagine he was talking about something like war plans, or economic strategy, or political campaigns. Unfortunately, we may never know. There is actually no evidence that Churchill ever said that (and I’m not sure who did). But it certainly does sound like something he would have said.

Bet you had no idea that the Daleks were a secret weapon during WWII

(or that Churchill looked anything like that)