About surveys in user research

I’m not usually a fan of online surveys as they feel like tests, and often times when people take things that feel like tests they answer with what they perceive is the right answer… however, when I want to prove a point to people who like numbers I will use a very carefully worded and structured online survey that is designed to yield a particular insight… as the old saying goes “98% of all statistics are made up”

I prefer observational research with a loose script loaded with suggested questions and prompts (show me how you play a DVD, now show me how your wife plays a DVD, mow your lawn, whatever…) to keep things going in the awkward moments. I find that if an observation session goes past 3 hours it will be weird, especially in an in-home situation. It is a little different if you are observing a professional (FedEx guy, MailMan, Professional Basketball Player, Surgeon) those can run longer without you just getting in the way. Do 10-15 2 hour session and then abstract out the commonalities and differences between them to find trends and outliers. Look for user created hacks and work arounds to identify key problems that could be solved. This is good input to inspire design but not to chain it. It should be a spring board not a set of boundaries.