Wednesday, April 18, 2018

Please forgive the long letter; I didn't have time to write a short one. (Blaise Pascal)

Huh! I always thought it was Voltaire.

Anyway, one thing I’ve noticed over the years that this is a very common problem with new usability engineers and user researchers. And what that means is both too many pages (in reports, of course – not letters) as well as too many words on that page.

Now, I do have a hunch of where this comes from. And that would basically be school. Think about it. Throughout everyone’s academic career, we are usually awarded for over-delivering. It’s basically the standard way to show how smart you are and how much work you’ve done. Hey, nobody’s awarded gold stars for 16-pt type, wide margins, and something slightly under the mandated page count, right?

I guess I’m kind of lucky in that I happened to round out my academic career with a graduate degree in tech writing. The difference between what I learned in that program and what I learned as an undergrad English major could not have been more stark. 

In fact, it was only as a grad student that I first learned the basics of the rhetorical situation – audience, purpose, and context. So, who is this for? What are you trying to accomplish with it? How is it being delivered? How will it be processed? How likely is it to be accepted?

In other words, in the real world, the point of writing is not to impress the teacher and get a good grade. It’s to get things done – to impart information, to offer suggestions, to come to an agreement …

And what’s an effective way to do that? How about a PPT where you can get through every slide in an hour? A presentation where the amount of information on each page is not a distraction to what you are saying? Something where an audience member might remember 3 main points 10 minutes after the presentation is over? 

To tell you the truth, this is advice that is not just for junior team members. I just sat through a presentation where the team may have gotten through 10 slides of a 40-page presentation.  

And it’s definitely not easy either. It takes a brave (and experienced and humble) soul to take all of that great work they did and distill it down to something that their audience can actually relate to, process, and value. It’s truly something every UE and UX researcher needs to remember though – it’s not about you!


Yup, that's him alright!

Monday, April 2, 2018

It depends (every usability engineer ever)

I’ve often joked that I want this on my tombstone … Even though I feel a little conflicted by it.

On the one hand, I see my offering this statement as a good thing. It means people are coming to me with questions. So, basically, I’ve gained their trust. Yay!

On the other hand, though, there are some things that make me cringe a little when I have to say this. Probably the main thing is that my clients seem to be thinking in black and white terms. Alternatively, they may also simply be wanting a thumbs up / thumbs down from yours truly.

One of the things I feel like I am always trying to get across to my clients is that the world is a complex place, and a lot of it is very situational. In particular, I like to get across that usability issues are often contextual in nature.

For example, who is this system for? A technical audience? Well, they may be just fine with it. Your average Joe or Jane? No way in heck!

In addition … What is the user’s goal? How motivated are they? Where will they be doing this? Have they seen stuff like this before? 

What are your goals? Are those conflict in with the users’ in any way? How should we prioritize? Who is making the decisions?

How hard is to the make that change? Do you have time to do that? Is it worth effort? What are the goals of your project?

Overall, how is this decision being applied? In just this particular situation? Across the board?

And on and on and on.

You know, maybe “it depends” is just a good way to begin the conversation. Maybe a better way to word that simply would be, “Tell me more.”


Ha ha!  Very clever, funny developer guys!

Thursday, March 29, 2018

There are three kinds of lies: lies, damned lies, and statistics. (Benjamin Disraeli)

I have a mixed relationship with statistics. 

On the one hand, I know their an important part of our history. I also know there some of us out there who still do a bang-up job with them (Jeff Sauro and Jim Lewis come immediately to mind), Finally, I also realize that stats definitely have their use.

I myself, however, am much more qualitative. I generally like to leave stats to things where they are a much better fit – surveys, web analytics, A/B testing, big data ...  I’m more than happy with my rich, why-based qualitative data.

That said, I do use stats.  Because of the qualitative nature of my data and the much smaller number of participants I typically have, though, my stats tend to be much more on the lighter side.  For example, if I’ve got my typical 10 users, I think it’s good for my audience to know if it was 2 users who had a certain problem versus 9 of them.  

I also typically eliminate any 1-offs. I find they typically draw too much attention to themselves by their mere inclusion.  That also has the added benefit of making my report easier to write, and easier to read as well. 

Also might do error rates.  Once again, though, I try to keep it simple.  If the user aces a particular task, that’s a 100.  If they veer off somewhere but eventually get back on track, or if they complete everything but miss 1 small part, that’s a 50.  If they give up and look to me (and even then complete it with some help from me), that’s a 0.  I add ‘em all up, and then generally use simple academic grade scale – 90 and above is an A, 80 to 90 a B, and so on.  I just want to give my audience a rough idea of how the system performed.  (BTW, time on task – because of think aloud – means absolutely nothing to me.)

Finally, I also like to do a SUS. Note, though, that scores for individual tests mean almost nothing on their own.  I do keep a running total of all tests we run however. And that let’s me compare 1 particular score to all the rest, to an average for the type of system or user group, or for previous tests of the same thing. Very helpful.

So, yes, I do use stats.  But only in the simplest, most direct ways that pretty much anyone – business types, developers, writers, graphic designers … could understand.  


I have no idea why he’s on a phone case, but there you have it

Tuesday, March 20, 2018

Do the right thing. (Digital Equipment Corporation)

My first job out of grad school was at Digital Equipment Corp., or DEC. For those of you who haven’t been around as long as I have, I like to tell people that it was the number two computer manufacturer in the world at that time, and was something of the Google of its day.

Google, in fact, has a similar motto, “Don’t be evil.” Well, DEC’s is a little more positive, of course. But it's still the same general idea.

Now, we might contrast those two that with the motto of Facebook. Their motto was originally “Move fast and break things” (they’ve since come up with some bland official pablum). 

Well, I guess they've managed to do both, haven't they? They moved very fast, and they seem to have broken many things – the ability to go more than a few minutes without looking at your phone, any sense of personal privacy on the Internet, journalistic objectivity, the democratic process, civil discourse …

I think this is particularly ironic as Facebook is a social media company – a company who’s purported purpose is to bring people together. Of course, there is also their business model. And that revolves around targeted marketing – basically delivering people to large corporations so they can extract your hard-earned dollars (and yen, and euros, and yuan) from you in the most efficient manner possible. You know, maybe their motto ought to be, “Profits before people.”

So, what does all this mean for the average usability engineer out there? Well, there was a time when our main job was to make things easier to use, for users. In essence, we made the world a friendlier, nicer place. These days, though, it seems mostly to be to sell, sell, sell. That moral high ground we used to occupy is steadily eroding. 

So, what can we do about it? Well, when your company, or project team, or colleagues, appear to be blindly headed down that path of moving fast and breaking things, perhaps you can be the still, small voice of conscience. In other words, don’t be afraid to raise your hand. Ask them about the implications of what they’re proposing. Remind them that there is more to the world than profits, that a handsome paycheck does not absolve you of all moral obligations, that there is still a possibility for UX to make the world a better place.


The only wrong thing former DEC CEO Ken Olson did was to bet against the PC  :^(

Wednesday, March 7, 2018

Agile is twenty cement mixers pouring feverishly while somebody looks in the Yellow Pages under "architect." (Roger Belveal)

I have incredibly mixed feelings about Agile

On the one hand, I know people are making it work with usability. I have even done something like that myself.

On the other hand, though, I know it was developed without usability in mind. Now, you can make it work with usability, but you really do have to make a separate, concerted effort.  It is not baked in.

Out-of-the-box Agile does not guarantee usability. This worries me, as I am often struck by the zealotry of a lot of Agile advocates.  In fact, there seems to be a certain fundamentalism within Agile that implies that you have to do Agile only in its purest form.

So, what does Agile have instead? Well, there are the user stories. And they are a great way to turn requirements into something that ties in real users trying to do real tasks with real goals in mind. So, that does loop the user in when it comes to creation and design at least.

But how about the evaluation part? Yes, Agile does a great job making a tweak here, a tweak there, then rushing it out to market to see if it “moves the needle” with actual users. A/B testing is an even more sophisticated version of that.

Note, though, that this is totally correlational. You know that something worked better, but you have no idea why. Design, then, becomes a simple shot in the dark. Throw it on the wall and see if it sticks. Run it up the flagpole and see who salutes. I was hoping we had come a little further than that in all these years. Sigh …

Now, there’s plenty out there about how to make usability work within Agile, and it does make a lot of sense (and, like I said before, I’ve done it myself). I do really worry, though, about the people who aren’t super-sophisticated about UX who hear about Agile, can’t wait to jump on the bandwagon, and want to “do it right.” Unfortunately, that last bit often means not even thinking about something that took years to finally become adopted in IT and which has proven its value many times over. For me, it almost feels like starting from scratch in a way. Maybe I’m na├»ve, but I honestly thought I’d never have to make those arguments again.


In addition to being a kick-ass UX professional,
Roger Belveal is also a pretty talented artist
(these are his iPhones made out of concrete and steel)

Friday, February 16, 2018

However beautiful the strategy, you should occasionally look at the results. (Winston Churchill)

And isn’t that what usability testing is all about? 

Of course, there are other ways to get that information as well, especially these days. For one thing, you can simply put it out there and monitor the heck out of it. 

A kind of old-fashioned way to do that is Voice of the Customer – popping up surveys, including “give feedback” links, and so on. These days, you can even monitor social media. It’s all good stuff, but there’s also no shortage of possible drawbacks. I already wrote a post going over those in great detail, so I won’t repeat all that here.

You could also take a look at web analytics. (I wrote about that, to some degree, in another post.) The main thing with analytics, though, is that, though you might have tons of data showing exactly where users went and what they did as they actually used your system, you totally lack why they did so. And that can be pretty darn important.

It’s the same problem with A/B testing. Once again, you get reams of real data, all tied directly to the bottom line, and in this case, totally actionable. You are, however, relying totally on correlation – which, in my opinion, means relying totally on conjecture. We know that A performed better, but why? How can we tell with just the numbers? What were users thinking when they went with A?

And then there’s usability testing. It’s the prime way to get feedback before release, but it also provides super-rich data that can be used to completely understand your users, identify and understand their issues, and then come up with legitimate solutions to address those issues. 

That said, if you are getting some kind of feedback, you are way ahead in the game. What really worries me are those companies out there who basically don’t do any of this.

By the way, when Churchill was talking about strategy and results he was actually not talking about usability testing (I know, hard to believe). I would imagine he was talking about something like war plans, or economic strategy, or political campaigns. Unfortunately, we may never know. There is actually no evidence that Churchill ever said that (and I’m not sure who did). But it certainly does sound like something he would have said.


Bet you had no idea that the Daleks were a secret weapon during WWII
(or that Churchill looked anything like that)

Thursday, February 15, 2018

Mobile is a magnifying glass for your usability problems. (Josh Brewer)

There are designers and developers and UX researchers out there whose whole world seems to be totally encompassed within the mobile space. Now, that is definitely a fact of life these days. At the same time, though, that’s also a tad unfortunate as well.

For some of these folks, mobile is a brave new world, where they are intrepid explorers creating their own rules. Now, mobile definitely is terra nova. It is not, however, terra incognita.

For someone who’s been doing this as long as I have, mobile is really just the latest whatever. All the things that we learned about green screens, CLIs, GUIs, the Internet … they pretty much all still apply. Yes, there are new things like gestures, but affordances, mental models, proximity, hierarchy, information density, legibility … all that great stuff has not somehow magically disappeared overnight.

Reinventing the wheel is not the point with mobile. In fact, designers need to think, not about being innovative so much, but more about handling the old principles within the additional constraints that mobile involves. 

And what are those? Well, the main one by far is simply the huge difference in screen size. And what does that mean relative to UX? More than anything, I think it takes the old KISS formula and makes it absolutely paramount. Complexity that you might have been able to get away with on a nice big screen is simply going to blow up in your face on mobile.

Less central – but certainly not unimportant – issues include awkward input, new environmental contexts, and several others. There’s also one thing I’ve noticed which probably doesn’t need to be an issue at all. For some reason, some designers took the new context of mobile and decided to just ditch the idea of affordances altogether. 

What do I mean by that? I just seem to see missing pieces on every mobile test I run – or as simply a user myself. In fact, just this morning, I was testing out a prototype for a test I’m running next week on an iPhone …

First, though, I had to clean the screen up a bit, and get rid of some old icons on the home screen. Now, how to do that? Why press and hold those icons, of course (I somehow remembered that from the last time I had to test an iPhone). That puts you into delete mode, where the icons wiggle and have litte x’s in their top-left corners. Okay, this I can figure out – an x is a pretty darn clear affordance. But what do I do to return to normal mode? I would imagine I would press a wiggly icon again. Nope, you press the Home button (had to really think about that one).

Now I can type in the URL of my new prototype. I do that, bring it up, then wonder how I can put an icon for it on my home screen. Oh, it’s the little box icon with an arrow in it (had to ask an iPhone user for that). What do I click now (interestingly, the iPhone user couldn’t help me there). Oh, that little bar of icons on the bottom – looks like I can slide it over (had to ask an iPhone expert for that one). Now, why didn’t they signal that somehow? You know, like with a >, or a little bit of the next icon, or practically anything? 

And all this is before I even get to start in on that prototype. And after that, I then get to try it all over again on an Android, which I least am familiar with, but which also has its own way of doing things, and its own set of affordances to ignore as well. Ugh, it’s gonna be a long day …


Josh was once the principle designer at Twitter