To me, it’s like trying to study the natural behaviour of an animal in captivity.
I do see some value of it. I think this value is true in cases where the test topic is very specific, such as replacement of a particular part in a machine, because the frequency of such activity is low to begin with.
A lot of companies use focus group methology as their main source of observing human behaviour which they use as a reference for their product designs. Personally, I think I’d expect this more from an engineer, since scientists don’t like random variables in their experiments.
So what do you think about it? How do you think things can be done, if you have all the resources in the world and if you need to be realistic.
Firstly, the term “focus group” has become ambiguous. Let’s say it is safe to assume that it is a way of gathering information from a group of appropriate individuals (e.g. customers, end-users) around a product, service or concept.
But beyond that loose definition, there’s a lot of room for variability. For example some focus groups just SHOW the product, while others let the participants actually USE the product. Some emphasize opinion/preference questions, while others emphasize personal experieces or anecdotes.
As a designer, it’s important to be savvy about the established pros/cons of a variety of research techniques for both discovery and validation. Your work hangs in the balance, and wrongly performed research can kill good projects.
Designers frequently need to educate marketing on the role of focus groups over other techniques like Contextual Inquiry. It’s vital to clarify the goals and methods of user discovery research over the goals and methods of user or market validation research.
I recommend reading up on the subject. “Contextual Design” is one of my favorite references. But also take a look at what the top design research firms have to say about product design research methods.
I’m referring to situation where you put a group of people in a room, give them some tools and tell them to perform some tricks, then observe from behind the mirror. Sometimes they get treated with sodas and chips.
To my knowledge, many “top” design firms don’t place quantifiable research as their main influence in their designs. They participate in the daily events of the users and observe from real situations as opposed to staged ones. Even for “interviews”, they get answers from conservations in which the users take the initiative to express their experiences as opposed to being asked with questions. Therefore the observations are made under the most natural and true conditions.
In my opinion, you can have millions of video tapes of people trying to operate a product in a controlled environment but you will never know if the observations truely reflect the reactions of users when they are in the real context of using the product.
Focus Groups are an overused, misunderstood market research tool that many marketers confuse with design research.
Most design researchers will tell you not to use them unless there is a benefit from group interaction (I’ve personally found Nominal Group Technique to be a useful form of focus-grouping with subject-matter experts.)
It’s our job as designers to know the values of these various tools and to educate those who would use them improperly.
I encourage marketers to use open-ended phone interviews for initial concept screening instead of surveys or focus groups. Then move on to NGT and qualitative field research.
Yes, it is best to observe the user in their natural environment using a product. That is not always possible. I worked on a pregnancy tester and all of the women did not want me to watch them pee on a stick. I worked on a glucose meter. I don’t want to follow these people 24/7 to see them test first thing in the morning and when they are out late clubbing. HIPPA restricts a lot of medical device ethnographic research. Then there is the economics. Getting 12 people to one spot costs less than going to 12 different places. That’s why docs don’t do house calls. As for one-on-ones versus group thinks, there are advantages. People can feed off of each other, kind of like a brainstorming session. In both cases, the success is usually determined by the facilitator.
So I will do my best Donald Rumsfeld. Should you do product research? Yes. Is observing the customer use the product in its intended setting the best methodology? Maybe. Are focus groups worthless? No.