Focus Groups and the XBOX360

That it could have been 15 Million units. :wink:

The barometer for success is ultimately profit. But profit is limited by those that can and will buy the product, so designs role is to help maximize the adoption rate of the target market.

The Wired article was interesting because there was a lot of emphasis on the networking aspect of future gaming (and commerce) and yet I don’t see that coming through semantically.

they might be refering to the future wireless technology (ironically because they’re “wired magazine” right?).

but eventually it’ll happen.cell companies are already investing a huge amount on wireless gaming and media, which is mainly a downloading problem and has little to do with design, maybe just graphics or rather the programming or software that runs it.

i read an article on yahoo where gates admits that the future of gaming is on wireless and cell companies and microsoft works with partners like samsung on that idea.

but he forgot to mention that there’re new platforms being developed specifically for that purpose (guess where?) as we speak and will eventually knock microsoft out of the wireless game. he just said he wants to retire in ten years!

well - ok bill your wish is granted, but what about those billions of people who will be heart broken?

Afterall, we are assuming the product should sell based on its power and performance on screen. Use design in an “anti-design” manner?

not exactly the same thing but MS is making a point of the 360 being "customizable via different faceplates & shells

To sum up, with the PS3 and 360 being released close together it will be a crap shoot who wins. Personally I’m betting on them splitting the market share 49/49 with 2% buying a Nintendo. That parity will continue until either someone blinks, or someone takes a big risk.

sony already has the advantage and i do not see that changing…despite xbox coming out of the gates 1st it is only partially backwards compatible while the ps3 will be backwards compatible to the ps1 and the ninty revolution will play ninty’s whole library of games via downloadable content…

Agreed – users can’t tell us what to design, but they can express what they want, and we in the design community have to think hard to understand the real needs that are behind the “top of the head” suggestions. Far too much focus group activities are silly attempts to take bullshit comments literally, expecting people to have some god-like insight into the future.

But I have to think that gaming is about the worst topic to focus group – hardly anyone can talk cogently about gaming at all (dude!), much less talk about what they want from gaming in the future.

Outside of corporate design groups, how connected are we as designers when it comes to decoding user research findings? We’ve all been there [right?], how should we react to perimeters that are bestowed upon us after such testing? Is it out of our realm to suggest and fight agianst such findings? I actually have a few different takes on this subject, yet I 've yet to validate what direction/stand is appropriate regarding producing the in house “desired” concept vs. that of the proposed market “driven” direction. Any thoughts here? Are clients and their market/user test the end all be all?

Edit

A little off-topic, but yesterday I realized why they named it the 360 instead of the XBOX2–they’re competing with the Playstation “3.” ie. “360” is better than “2” or “3.”

This is exactly what MS did with Windows XP. Windows “98” and “2000” were literally showing their age, so MS decided to bail on the year concept.

Likewise, I wouldn’t expect Apple to ever introduce an OS XI–they’ll milk the X brand and continue to use animal names for major releases. Interesting…

Plus everything has to have an ‘X’ in it doesn’t it?

The “X” branding craze goes back to the 1980s. Old industrial companies like US Steel and the rail freight firm (original name escapes me) changed their names to USX and CSX. I remember people saying at the time how dated those brands would sound once the X fad was played out. Funny, 20 years later and everyone is still branding things with an X. Some fads have legs, apparently.

Great article on the evils of focus groups:

Who says that the design of this new xbox was directed by anyfocus groups? Can anyone validate this? This thread is full of hot air by the way.

Edit

I have no idea where that analysis of this discussion comes from, but it seems spurious and based some emotional knee-jerk on your part, rather than a careful reading of who wrote what. The issue is not “feedback” vs. “no feedback” - the issue is whether or not focus groups specifically are effective tools in obtaining meaningful/representative/actionable feedback.

Here’s a few–just google “XBOX 360” and “focus group” for more examples:

The sleek white machine with the subtle curves was from Apple, Sony or some other company known for hardware design. > The Japanese consumers in the focus group were convinced of it. And they were sure it wasn’t a Microsoft product.



Microsoft guru J Allard estimates > 20,000 people will have touched the new Xbox before the first one is sold. All to ensure that nothing was left to chance. > From the choice of Xbox 360 as name to its sleek white look, Microsoft consulted focus groups and design experts from around the world.

As someone who was involved in the research behind the new Xbox, I thought I might add a few thoughts:

Focus groups, done correctly, can be very useful tools for design. They get a bum rap because most people take what they hear from focus groups at face value. Someone earlier in this thread noted, correctly, that a focus group’s usefulness had to do with digging deeper into comments, understanding the underlying issues, analyzing these issues, then asking even more probing questions.

Our objective on the 360 was to be as in-touch as possible with the people we were designing for. This doesn’t mean they “design the product”, it simply means they inform our approach, just as any other input would; the design decisions are still ours to make. To design without this input reduces your chances of creating a product that resonates with its intended audience, which after all, is the point.

In my experience, the “most people” include those in the suite. The lesson is to agree beforehand on what the test is for and how the results will be used. Before even selecting the technique, the core team should agree on their goals.

I’ve avoided running focus-groups, but have had success with NGT (Nominal Group Technique) with subject matter experts.

From what I’ve heard, Apple credits their innovation (beginning with the iMac) to ditching the focus groups alltogether and concentrating on individual use and market differentiation: Leading, not following.

Ultimately, the 360’s apparent following vs. leading is what I think is really bugging the designers here, who see so much wasted opportunity.

Ultimately, the 360’s apparent following vs. leading is what I think is really bugging the designers here, who see so much wasted opportunity.

After re-reading all of these threads, I am still left wondering exactly what this “wasted opportunity” might be.

Since the initial, predictable potshots were taken about the XBOX 360’s aesthetics in another forum, we then got the counter-assertion that hardware aesthetics were likely not the driving factor in the target audiences’ purchase decisions.

OK.

Then we got the assertion that focus groups, groupthink and corporate timidity were to blame for its lousy aesthetics - or was it perhaps for its relative lack of (again, hardware) differentiation vis-a-vis its competitors? And yet, we have now heard from someone who worked on the project who asserts that focus groups were not used by the design team to “design the product.” (Having worked with this person myself, and knowing that he is one of the finest in the field at what he does, I for one, believe him.)

But what ARE the “missed opportunities” arising from the particular design of the hardware?

  1. Has the intended audience been alienated or under-served? No one knows yet.
  2. Is the actual gaming experience sucky - or mediocre? Owners of the new XBOX have yet to weigh in. All of the negative criticism here has been in response to IMAGERY OF HARDWARE. What happened to “design is about the whole experience?” Is this now just rhetoric?
  3. Has Microsoft’s desired financial and marketshare growth taken place or not? We won’t likely know until the end of the year.
  4. Has the Microsoft brand suffered or been enhanced?
    And even if it has, how could anyone -Interbrand notwithstanding - prove it anyway?
  5. And again, is the actual design of the hardware mission-critical to any of these?

We also heard from someone (who did not have the courage to sign-in) accuse this thread of being full of hot-air. Maybe s/he was partially right.

There is too much here that sounds too much like people in a badly moderated focus group, reacting to a sketch of something they haven’t seen before. Isn’t this just a little bit ironic? And finally it is indeed admirable to try to create alternative XBOX designs, as has been happpening elsewhere on these boards, but I can’t help but think all of this needless huffiness is about something else. Perhaps something more base. Like that others got to work on this juicy project, while the rest of us did not.

Someone please disprove this last bit…please.

It’s a box…who cares

CS

You make some great points. As you said (in point 5) the aesthetic design is not critical to the sale (other than it maintain a tight footpint for the Japanese market). IMO that is where the missed opportunity might lie. This was a chance to create an iconic piece that enhanced the overal experience by creating something that had an intrinsic value to it. Something that made the user proud to display it if they chose to vs tuck it in a cabinet. The designers could go out on a limb, because as you stated, the visual would not affect sales (though this could be argued)

Of course to play devil’s advocate to my own point, perhaps the iconic design is in the understadedness of the product. It would have been easy I supose to make something flashy. Instead the designers kept it minimal and functional letting the software and capabilities be the star. I admire that thinking if that was what was in place.

I can’t help but think about the iPod a bit though. It’s success comes from its abilities, but the iconic design is a bonus that makes it more of a premium experience. From the white headphones to the proud little stand that displays it so nicely on my shelf. It’s minimmalist, but its a showoff.

As far as project envy… it looks like not many of us where willing to put up or even shut up. I can’t speak to other peoples motives, but for myself … of course I am a little envious, not being on such a juicy, high profile project. Who wouldn’t be? I wouldn’t attempt to deny it. I try my best to acknowledge that and keep that feeling separate from my biased judgement of the design, which itself is based only on photos at this point. Projects like this are exciting and rare, and I was eager to see what I might have done on it as well as what some of the other nay sayers might show.

Cordy, are you asking if its fair to criticize? As designers, I’d say it’s our duty to! This product in particular because it’s iconic, and we all know something about the market, and something about research, innovation and design.

What motivated me to start this thread was simple:

With this design, apparently MS desires for the XBOX brand to be confused with that of their rival. This is intuitively a doomed strategy. So my question is: was it truly intentional, or was it the result of following the focus-group feedback?

Prove that wrong: Give some examples of “me too” products that successfully converted the market.

First off, Yo, thank you for your honesty. It is only from acknowledging our own biases and (heretofore) hidden motivations that we can begin to make grounded, useful criticism. And in this case I think that some of the aesthetic criticism of the hardware is justified. (I also confess to having similar motivations myself in criticising corporations with whom I would like to work.)

cg, of course it is fair to criticize. And you have done a great job in leading the charge in it. But it is not fair -as others here have done - to criticize unfairly. What I mean is that since - as you point out - we all know something about the market, about research, about innovation and design, don’t you think we - of all people - should withold conclusion-making, judgement and especially a post-mortem on process until one of us has ACTUALLY USED THE PRODUCT?

As far as “me too” products successfully “converting” a market, I am not totally clear on what you mean by “converting,” nor what, if any, metrics there are for it -so I am going to assume that it means to win marketshare from competitors. “Me too” products happen all the time and can be more successful than an “original.” (One of my pet peeves is the phrase, “good design is good business.” It is sadly not. And there are loads of case studies that prove it.)

  1. What immediately jumps to mind is MS’s own history of Windows OS vs. the Mac OS. Remember when the joke used to be that Windows 97 was actually Mac 89? Microsoft won the OS war not because it was the most “innovative,” it clearly wasn’t. (In fact WIndows just cherry-picked any decent feature from the Mac for years.) Windows became successful because of its hardware beachhead through its initial willingness to play nicely with others like IBM, something that despite all I love about Apple, it continually proves itself incapable of doing. Windows’ victory was also a lot less about design strategy and a lot more about the reality of cost, distribution, etc.

In the case of XBOX, perhaps this M.O. is intentional on the part of MS or perhaps it is now just its instinctive behaviour as a company. But they have proven in the past that “me too” - combined with lots of money - works. But I will grant that the rules are different for behemoths like MS - so the heavy lifting of innovation, design and strategy are indeed still likely the better course for most other corporations.

  1. The Germans are masters of the “me too” approach. They didn’t “invent” modern day automotive engineering (eg the assembly line, or features like the car radio, gps) the way the British or the Americans did, but they are now demonstrably much better at it. The thing that makes me scratch my head in wonder is that most German cars are more desireable, have impeccably managed platform portfolios and enjoy higher margins than competitors - all claims that American design strategists try to make to their potential clients - and yet on the design side, the German automakers do little to no consumer research or consumer-oriented innovation. Yes, from a process and multi-disciplinary standpoint, they are dinosaurs - engineering in one silo, design in another, marketing in another, purchasing and finance in another. But they do dinosaur exceedingly well.

While I am the last one to advocate living in a world where things are designed by default, maybe the XBOX is not as “icnonic” to consumers as it is to designers? Maybe consumers just consider it as more replaceable clutter?