I have to admit that I still read Mini-Microsoft regularly (and occasionally sneak in a comment) even though I’m no longer a Microsoft employee. Mostly a combination of schadenfreude and getting insights on how to lead, manage, and reward software projects (mostly how not to do them.)
Today I read Hamilton Verissimo’s story that someone posted on Mini on why he left Microsoft. I found the following two snippets especially interesting:
For PMs, like me, some manager pushes idiot time-consuming exercises like scenario validation…two months to produce collateral that is bound to be useless in six months, since everything is likely to change.
I’m a big believer in scenario-based design and believe that, when done properly, is the best way to make a good design great. Consequently, I found this slam disturbing.
BTW: I’m not impressed by the “bound to be useless in six months, since everything is likely to change” claim. That’s a classic technologist argument to weasel out of something you don’t want to do. (Disagree? Try this exercise: The iPhone was released in June 2007—well more than 6 months. Identify the 10 top original iPhone scenarios that have completely changed since then. Should be easy to do if this claim were true. How about 3? OK, give me just one.) But I think what he’s really saying is that giant specs aren’t effective, which I completely agree with.
But then Hamilton goes on to describe the challenges he faced:
One thing that really frustrated me was that those random suggestions come from intuition, instead of actual scenarios/facts/data, and commonly show how disconnected MS employees are from the real world. In my case, as I worked in the developer division, it demonstrated how people there were disconnected from how developers work, and what they value. I had to constantly remind them that we should strive for simplicity since developers don’t have the time to become expert on our product, since it would be another tool in their toolbox.
Here, Hamilton has just made an excellent case for why scenario-based design is so important. We have a natural tendency to design for ourselves—to make decisions based on what we think is best given our own personal preferences and context. Scenario-based design, when properly done, forces us to make design decisions based on our target users’ point of view, by considering their personal characteristics, goals, and context. (Isn’t this what that “idiot”scenario validation is supposed to do?) Scenarios are a model of the facts and data we have about our customers. And designing from the target users’ point of view makes a huge difference!
So, what’s going on here? Does Hamilton support scenario-based design or does he think it’s a waste of time? I’d love to ask him in person, but in the meantime I’ll have to speculate. The key words above were “when properly done.” My observation while at Microsoft was that the typical scenario-based design effort was a waste of time. The “scenarios” were actually just thinly-veiled feature advertisements, that did nothing but support the existing plan and inevitably ended by claiming a thrilled user. Such “scenarios” are just a process tax and don’t lead to better design.
While it’s good to have a design process, mindlessly following a design process without understanding the point behind it is just a waste of time. If your team is just going through the motions without making things obviously better, you’re doing it wrong!
If you do only one thing: Make sure everyone understands the goals behind each step of your design process. (A bit of training is a good thing!) Constantly evaluate what your process is buying your team and your product. If the benefit of a step isn’t worth the cost, find out why. Either fix the step or get rid of it.
It’s fun to work with beginning designers because they make fundamental mistakes so predictably. Perhaps the most enduring beginner mistake is to design for yourself. The key to user-centered design is to realize that you are designing for someone else and that all your decisions should be made from their point of view, not yours.
It’s so typical for a beginning designer to say “I would never do that.” If you then remind them that they are designing for their target users not themselves, they’ll often reply “OK then, my mom would never do that!” As if that somehow makes their case stronger.
While we all want to please our moms, designing for our mothers is pointless (unless your mom really is the target user.) Despite the tremendous number of practitioners, there’s a reason why “mom-centered design” isn’t an established discipline.
You aren’t the target user, and neither is your mom. Imagine your mom walking into the cockpit of a jet aircraft and saying “All these dials and levers are really complicated—I would never use this.” True, but unless your mom is a professional pilot, it hardly matters. More typical examples are just as pointless, just less obviously so.
Another problem with mom-centered design is that it’s disingenuous. You are still designing for yourself or for your assumptions about your target user—you’re just using mom as a proxy.
While I expect beginners to design for themselves, experienced designers often do it too. This is such an easy trap to fall into.
In the discussion How not to design for user experience: the Ferrari example, the poster asserts “When the use of something as intuitive as a steering wheel takes a nine-minute video for just a brief overview, you’ve got a problem.” Most of the people leaving comments agreed. But this is a design blog and the commenters are all designers, and they are thinking about this design purely from their own point of view. If Michael Schumacher felt this way, then I would be concerned.
Yes, the Ferrari Formula 1 steering wheel is complex, but it’s designed for professional race car drivers whose livelihood totally depends upon winning races. They need the features—saving a second is a big deal. They have the motivation. They can spare the nine minutes to learn how to use it.
Different users have different characteristics. Here are some of them:
Details like motivation make a big difference with respect to what users can or are willing to do. Having a clear understanding of these characteristics enables you to make better decisions on behalf of your target users—something mom-centered design can never do.
If you do only one thing:
Remember that user-centered design is all about designing for your real target users. Not you. Not your assumptions. Definitely not your mom.
It’s usually pretty clear when you are not done with a design, but how do you know when you are done? The need for releasing at the right time is clear. Releasing a product in a timely manner is crucial to success—the least successful products are those that release outside their window of opportunity—so your design completion criteria must be release-friendly. On the other hand, the days when you could release a half-baked “good enough” product successfully into a competitive market are long past.
One way to weasel out of declaring your product finished is to release it unfinished as a “Beta.” Doing so lowers customer expectations and provides an obvious excuse when those lowered expectations aren’t met (“What did you expect? It’s a Beta.”) Some companies have a habit of labeling their products as Beta seemingly indefinitely (called “forever Beta.”) I’ll let you draw your own conclusion about the long-term viability of this approach.
There is no universally accepted list of criteria, but here is what I like to use:
(BTW: This list is in order of fundamentals—most fundamental first. And while the entire list can be applied as release criteria, appropriate subsets can be used as exit criteria for preliminary phases.)
I expected some readers will think this bar is too low, others too high. If there is a reasonable balance between the two, I’ll consider that a success. For those who think this bar is too high, I’d like to explore a few ideas.
While it should be difficult to find obvious problems or improvements in a quality product, that’s far from trying to be perfect. As Voltaire once noted, “The best is the enemy of the good” and this classic battle often rears into release criterion discussions. People will say “We don’t have time to be perfect!” Absolutely true—nobody does. But that’s irrelevant because perfection isn’t a goal. Don’t let the inability to achieve perfection be an excuse to lower your quality bar.
I’ve seen teams waste an incredible amount of time during the final project phases trying to address feedback by making changes that fall in the “different but not better” category. I believe this is a symptom of lacking confidence, which in turn is likely due to a weak design process. If you consider several design alternatives during the design phase and use user-centered design techniques as a decision making framework, you should have enough insight and confidence to recognize “different, but not better” design changes late in the process. Don’t waste your time with these—they will consume a lot of time but get you nowhere.
Some project managers live in fear—terrified that their career will be harmed if they don’t incorporate all feedback, especially feedback coming from management. This fear is not without reason. In some team and country cultures, what the boss says goes—regardless.
When I was at Microsoft, everyone was terrified of executive design reviews. One particular senior executive had a reputation for being obsessed with minor details that had nothing to do with goals of the design review (he had a unique talent for finding text his mother wouldn’t understand). As a result, people preparing for these exec reviews spend a significant portion of their time crafting their presentation to avoid getting his feedback. Not a healthy sign.
Managers can really help here by explicitly communicating how the team is expected to handle feedback so that everyone is on the same page. During reviews, executives should clearly distinguish mandatory tasks from unimportant, off-hand remarks. While it’s important to take all feedback seriously and perform due diligence, nobody should worry about their career being harmed for trying to do the right thing.
If you do only one thing:
Set criteria to determine when you are done with a project. Strive for high quality in a release-friendly manner. Avoid wasting time on changes that make the product different, but not better. Managers should set clear expectations on how feedback should be handled. Executives need to be clear on what feedback people really need to follow up on.
“For every complex problem, there is a solution that is simple, neat, and wrong” —HL Mencken
At the typical design review, there will be a variety of feedback. Some of it will be obviously brilliant and right on target. Some of it will be OK and require some mulling over. Then there will be some that is just plain bizarre. You are amazed that this otherwise intelligent person is giving you this feedback, and you’re even more amazed at how adamant they are in presenting it.
Over the years, I’ve found that all feedback is ultimately good, but some is just poorly presented. Even the poorly presented feedback is good because you will eventually discover that there is a valid problem behind it. Chances are, the reason the feedback is poorly presented is that the person is identifying a solution—often a really poor solution—to an unstated problem instead of giving you the problem directly.
I like to call this situation UX Design Jeopardy! because it resembles the TV game show. Just as in Jeopardy! you have to respond in the form of a question, in UX Design Jeopardy! you are given a design problem in the form of a poor solution.
This problem happens because sometimes people find it easier to articulate ideas in terms of solutions instead of problems. This is especially true for non-designers. It might be because they don’t even know the problem or even realize the problem—rather they’re just thinking in terms of the solution that is simple, neat, and wrong. “Make the logo bigger” may be a poor solution, where “this design doesn’t reflect our brand” may be a valid problem.
People tend to fall in love with the first idea that comes into their heads. While this problem is well known during the design phase, it happens during feedback and other phases as well. And it’s way to easy to focus on how the solution solves the problem without thinking through the larger consequences.
Adding to the problem is that culturally, we tend to believe that people who offer solutions are more sophisticated than those who identify problems. Anybody can find problems, right? And finally, people who give feedback in terms of solutions might be showing off a little—they want everyone know that they too know how to design stuff.
The problem is that any debate over an obviously bad solution is a waste of time. Worse, such debates can get rather emotional. The person giving the feedback is thinking “Why doesn’t this person get it? Why can’t they just accept my feedback and move on?” But the person receiving this feedback is thinking “Why are they so focused on this idiotic idea? Why can’t they just let it go?” Once you get past this, the emotional charge may make the rest of the feedback process less productive.
The solution is to recognize when you are in this situation as quickly as possible. Instead of debating the merits of a bad solution, get right to the problem by asking:
“What specific problem are you trying to solve?”
That usually that does the trick. Now you can understand the real problem, document it, and move on.
However, this question might not help if the person just can’t articulate the problem. In this case, a good approach is to ask the person provide examples:
“I’d like to understand your feedback better. Can you give me specific examples of where the product would benefit from this solution?”
“Can you give me examples of other good experiences that benefit from this solution?”
This helps because people can often express their ideas easier using examples. Hopefully, you can then use those examples to determine the real problem and get back on track.
Asking the magic question helps only after the fact. A better approach is to establish design review ground rules. I’ve found ground rules for effective design reviews to be extremely helpful and they do not go without saying. Here are some relevant ground rules I like to use.
Determine goals. The first step is to determine your goals. What help do you need? What would you like to get out of the meeting? Do you want high-level feedback or low-level feedback? The clearer your goals are, the more likely you will achieve them. Without clear goals, you will get random feedback.
Respect the presenter’s goals If they ask for high-level feedback, don’t critique the layout, fonts, or colors. If they ask for detailed feedback, don’t critique their scenarios or value propositions. If you have feedback that you believe is helpful but outside their goals, consider tactfully sending it via email after the meeting.
Don’t redesign Your goal is to help the presenters, not do their design work for them. These are design reviews, not design sessions. This is not a design committee. Assume that the presenters are fully capable of incorporating your feedback. Please do not offer your version of the design unless the presenters ask you to. Doing so otherwise is rude.
If you do only one thing:
Remember that there is no bad feedback only poorly presented feedback, which is often presented in the form of a solution. To win UX Design Jeopardy!, get the discussion back on track quickly by recognizing the situation and asking the magic question.
Once you learn how to play UX Design Jeopardy!, you’ll quickly learn that feedback you would have dismissed in the past has some hidden insight waiting to be revealed. The result of winning this game is more effective design reviews and, ultimately, better user experiences.