market research

Design Discussions general design discussion
1

Dear all,

I’d like to know from other industrial designers what kind of good market research they’ve seen. Unfortunately I have only ever seen pretty awful content which is flawed when you dig into the details of how it’s been conducted. Has anyone ever had a good experience? :confused:

I think there’s a lot to say for the words of henry ford when he talked about his aversion to market research. I feel that good designers are good empaths and so a lot of the time maybe market research is not worth bothering with. Unfortunately in my experience, market research is carried out by marketeers who are not very competent. Most times it feels that they are simply doing it so that once the product is released and subsequently fails, they have something to blame. Surely this can’t be right?

However I’m not completely cynical about it. I recently received a very good piece on the differences between our customer needs in Germany and the UK and it was very concise. Unfortunately it’s often the case that the research has been delivered directly to a marketeer who has then re-hashed the findings to present 2nd or 3rd hand the findings which again doesn’t help. Lets face it - as a designer under time pressure, sometimes it’s difficult to get the contacts internationally to be able to understand the different needs between people in Brazil, France, North America and Australia - yet a marketing brief is often the same for each country. :angry:

Does anyone feel some recognition in what I’m saying? It would be great to hear other views on market research and any other positive market research experiences you’ve all had.

thanks!
ed

2

Research for design is very difficult. I think the way we do it fits our needs, lots of guerilla interviews, getting designers on site and immersed, concepting on the fly and then digging some more. I think this kind of first hand research is more valuable. The more experiential the better… the more removed it is the less usable it tends to be.

What I have encountered more than poor research is thorough research that is not action oriented. This tends to be done by market research type of people with lots of statistical analysis but no real meat to it to the point where one can infer from it almost anything. I’ve also encountered just really shoddy or hasty market analysis…

3

Well…

Quantitative market research is only as good as the design of the study. You can go out there and get hundreds of surveys filled out and spin your wheels forever trying to interpret the data. Or you can design an experiment carefully, in a manner that controls for unknown/uninteresting factors, and you’ll converge to useful results quicker and with greater reliability. I’d suggest looking at cog-psy or behavioral economics research for some pretty cool designed experiments. Unfortunately good experiment design is very very rare outside academia.

Why do it if you can just observe and arrive at good conclusions through empathy/genius penetrating insight? I think it’s most powerful in areas where our internal biases make us blind to design choices. Where the designer is likely to be just as blind to the effect of the feature as the end user. So not so much in matters of aesthetics, but more in matters of interaction and decision making. Things like the selection of default settings. One study showed that changing the default response to the question “Would you like to be an organ donor?” from “no” to “yes” (while still leaving he form-filler the option to change it) resulted in much higher percentage of positive responses. Good design goes beyond making a usable interface (a well laid out form) to one that actually accounts for the nature of decision making (the phrasing and options of the questions).

4

So, does this prove, or disprove the power of surveys and research? For me, this seems to prove that any data is able to be manipulated to result in the answer you’re looking for.

5

Yes, data can manipulated. Or, data is not manipulated. Or, there is everything in between.

Haven’t you heard of “lies, damned lies, and statistics”?

6

That particular study is a well known and well repeated one. These biases exist and are not a matter of data manipulation. I don’t blame you for being skeptical - they can often appear counter intuitive or even flat out dumb. But they’re as well established as anything can be in the social sciences.

If you’re asking if studies in general can be manipulated to conform to preformed conclusions, of course. Can a data wizard draw unwarranted conclusions from ambiguous data and present it as convincing to a layperson? Sure. But a well designed study followed by intellectually honest analysis can yield some pretty useful results.

7

Yes, I’m very skeptical. But not cynical. Design research is a tool that I have found to be used, more often than not, to persuade people into following a gut instinct. In most Design research, it is conducted over the course of a few days, rarely over the course of weeks, and more often than not, over a few hours (go to the mall and ask some questions).

Correlation does not equal causation.

8

Well I can’t really argue if that is standard practice. But I have seen it bring value to the process when done right.

I guess the best analogy I can think of is engineers doing the ID or human factors stuff when the company doesn’t value it enough to hire a pro. I (an engineer) do some basic human factors for surgical instruments for example. Well-intentioned as these efforts are, I know they’re amateurish at best. Now if tomorrow surgeons start bitching about a whole host of things that make it cumbersome to use, I wouldn’t want people saying “We tried factoring in human factors (even put our most awesome engineer on it) and it still didn’t help.” Maybe the problem is we didn’t do it well enough.

9

I read ideo’s ten faces of innovation recently and think there are some very pertinent insights inside the covers of that book. The whole idea of an anthropologist approach is very insightful and could bring amazing credibility to the process… I was discussing this with the director of r&d the other day and he said “how would you go about getting the data you want?” I said that i wanted to do some observational research rather than sending out questionnaires… Which will already affect our sample - due to whoever can be bothered to answer them. Unfortunately, before I knew it some marketing idiot jumped in and said " we don’t do observational research because the findings are not hard". Fighting that battle - when faced with such idiocy is a very difficult one in a large company.

10

Hi all,

Why shouldn’t observational data be “hard” ?

It might be, that I have easier market access than most designers, as our company caters only to a small
(like 1.000?) row of industrial clients. But, if we set off to develop something new, we allways gather the data,
that is needed, in the field. Which means, that we meet with prospective customers and watch them
do what they do.

You often find, that people are not very accurate in their description of what they are doing and how they
are doing it. I`d strongly advocate, that a designer is able to find the “hard facts”, that are needed, while
watching.

If you happen to work on a project with a large prospective buyer base my guess is, that it would be very
useful to study the use of competitive products, not in a laboratory, but in the market.

Example:

I bought a digital NIKON during summer, that I like very well. If they sent me a survey, I’d feel compelled
to answer it. They’d probably “learn”, they built a nice camera, but that I felt, the digital interface was crap.
But if they wisely decided to study the interaction of some random customers with that camera, they’d
probably really learn something:

A: the digital interface is to complicated and follows a logic, that the user doesn’t = “crap”
B: most users are not as advanced in fotography as the professional tool they bought = users are morons.
C: They could learn to build a more moron friendly interface, like Sony does.

Probably they wouldn’t arrive at C with survey data alone.

And to come back to the T.O’s first posting: No, during my days in a design agency I never received a
“design brief”, that included enough, or surprising information. That idiocy was part of my decision
to move into management, as I felt I could do better.

And I hope, I do.

mo-i

11

  1. Designers are not empaths or mind readers - even good designers. It’s like saying the work is 99% inspiration. Our skill is transforming information about a problem into outcomes that satisfy and delight customers and end users. Crappy information → crappy outcomes.

  2. Market research is not Design research.

Market research, the way I was taught and think about it, is research aimed at identifying market opportunities and important trends that can be leveraged by a company. Not concept level stuff. Do new trends align well with the company’s capabilities? Can a company sell a product in category “X?” Will customers accept an offering from that company, in that category, as credible? How many customers does the company estimate have a need for products in category “X?” Of those customers, how many can the company realistically capture? What value propositions is the company competing against in category “X”? Market scaling. Etc. Very little to do with the specific details of the product, and often quantitative. Very early phase zero stuff. Useful for a company’s marketers to sell the need to develop product in a particular space to their upper management and secure PD funding. Depending on how the work is done, it can produce somewhat more detailed findings, but not a lot that will help us make product level design decisions. That requires fieldwork, in person engagement with end users, and tools to help us get to why people do what they do so we can respond appropriately - e.g. design research.