Have you ever wondered what is necessary to make sure that an iPod can be used intuitively even by new users or that a phone does not break when you accidently sit on it? Every great product is not just the result of a great designer but also of great user and product testing experts.
YouTube, Microsoft, Nokia and Apple are four companies that are well known for their innovative products and the focus they put on the user. Yet little is actually known about what these companies do to really ensure that their customers have a remarkable customer experience.
I would like to share with you some practices that companies apply when they are aiming to design and test new products that I have collected in the last months.
YouTube: Why do users watch videos online?
YouTube/Google is not sharing much corporate information, but in a recent blog post [1] they have shared some insights into their user testing process. YouTube has identified two distinct groups of users, one who just wants to watch movies and one who wants to connect with other people online. Here are mockups of what these different user groups expect from YouTube:
The design for each user group is different as they summarized in their blog post:
So what exactly is user research like at YouTube? Sometimes it means letting users design their ideal experience. For example, last year we used a method called FIDO (first utilized by Fidelity Investments) where we cut out different elements of various video sites, stuck them on magnets, and had users arrange their ideal organization of the elements (see below for an example). Other times we use a more standard research method called a usability study, which entails seeing whether a user can or can’t complete certain standard site tasks in a usability lab.
One of the most important findings has to do with the difference between the large group of users who are on YouTube simply to watch videos and a smaller, but very important, group of more engaged users — often uploaders. The latter group will, unsurprisingly, care about details like how to make communication with their audience easier and more effective, how to grow their audience, and even how to make money on YouTube. The former, on the other hand, want as simple of an interface as possible: "Just let me watch the video, please!"
Microsoft: Inventing a new game of play for Halo 3
The designers of Bungie Studios, the company that is developing Halo 3 [2] for Microsoft, are facing a tough challenge. They need to create an experience that is challenging enough to thrill the 15 million existing hardcore fans of Halo— yet appealing enough to lure in millions of new players. In their quest to make the video game a success the company is able to gather a lot of data that can be used to derive conclusion about the actual game play.
The lab also records video footage of every testing session and hyperlinks these clips to the individual progress reports. If the design team wonders why players are having trouble in a particular area, they can just pull up a few test games to see what’s going wrong.
Take what happened last March: A report noted an unusual number of "suicides" among players piloting the alien Wraith tank in an upper level. After watching dozens of archived test games, Griesemer spotted the problem. The players were firing the tank’s gun when its turret was pointed toward the ground, attempting to wipe out nearby attackers. But the explosion ended up also killing (and frustrating) the player.
To prevent this, Griesemer reprogrammed the tank so that the turret couldn’t be lowered beyond a certain point. The Wraith suicides stopped.
Read more about this in Wired Magazin [3] or download a presentation [4] from about tracking player behavior in computer games by Ramon Romero at the Game Developer’s Conference. (Direct link to presentation, [5] 14 MB)
Apple: Why you can’t innovate like Apple
Ask somebody to name an innovative company and you will most likely hear Apple. One reason why there is so much mystic around Apple is that the company doesn’t talk about the process of developing radical new products.
Sometimes though, some individuals set out and try to collect all information that is available and put together a coherent picture that explains how things play together. Alain Breillatt has done this for Apple and summarized what he could find about Apple’s development process. Alain has written up an article presenting his perspective of Apples design philosophy. One of them is the 10 to 3 to 1 principle:
10 to 3 to 1. Take the pixel-perfect approach and pile on top of it the requirement that Apple designers expect to design 10 different mockups of any new feature under consideration. And these are not just crappy mockups; they all represent different, but really good, implementations that are faithful to the product specifications.
Then, by using specified criteria, they narrow these 10 ideas down to three options, which the team spends months further developing…until they finally narrow down to the one final concept that truly represents their best work for production.
This approach is intended to offer enormous latitude for creativity that breaks past restrictions. But it also means they inherently plan to throw away 90% of the work they do. I don’t know many organizations for which this would be an acceptable ratio. Your CFO would probably declare, “All I see is money going down the drain.” This is a major reason why I say you can’t innovate like Apple.
You can find the full article here. [6]
Nokia: Breaking phones on purpose
How can you ensure that mobile phones are not breaking when customers use them? By breaking them early in the development process and ensuring that you are using designs that ensure that they withstand the daily wear out.
The idea is to simulate years’ worth of real-life product use in just a few days — individual tests last anywhere from a few hours to three weeks or longer — by pressing buttons, sliding sliders, actuating hinges, heating, cooling, wetting, drying, dropping, whacking, shaking, rubbing, bending, and generally defacing the phones in every way imaginable.
Once a phone finally breaks, they look for obvious reasons — cracked plastic, broken springs, and the like — but if that initial analysis fails, they’ve got a well-equipped lab on site complete with a scanning electron microscope and CT scanner for taking microscopic looks at failed components in both two and three dimensions; from here, they can find broken connections on chips, incorrectly-fabricated materials, and the occasional ant eye magnified a couple thousand times (it’s posted on the wall in the lab, and yes, it’s scary).
I have embedded two videos, this one from the Nokia Test Center in Southwood, UK
This one is produced by Engadget [7] and shows their development facility in San Diego:
Microsoft: The Story of the Ribbon
This has already been posted here on the Customer Experience Labs [8] but I still think that this is a valuable case study that shows how a product was iteratively developed and how continuous user testing beginning in early stages is done.
The Story of the Ribbon is a presentation by Jensen Harris from 2008. He tells the story of the development of the new Microsoft Office 2007 user interface. There are great insights on how Microsoft solved the challenge of “menu clutter”, various methods to collect user insights and how to setup an iterative prototyping process to create a product that radically improves the users experience when working with a Word processor or Excel sheet.
You can watch the talk online [9] or download it [10]. Powerpoint slides are available too [11]. Highly recommended.