I've been thinking more about market research recently - partly after an interesting chat with Peter Hutton and also from conversations from our very own John Winsor, who's a bit of a radical on the topic.
I used to make a very nice living from qualitative research, but became more and more disillusioned. More and more often, I realised how little value businesses were getting from it.
What I'm enjoying about this blog is the quality of comments. I find this rather liberating as I don't feel my post has to be a frightfully balanced review of the topic but can be a bit of a "conversation grenade". That's the spirit in which I'm pulling out the pin and lobbing out these three points. And I'm quite sure John W will pitch in soon after...
1. Avoidance, not curiosity
So often, research is commissioned as an act of politics. For instance, a marketing director wanted me to research financial advisers in the London area. When I asked why, he revealed that... well really he wanted to test some consumer ads quickly, but he didn't have time to do consumer recruitment, so IFAs would have to do. (That in itself is a pretty questionable shortcut). Then it turned out that he wanted to test direct reponse ads - where it's nearly always simpler and cheaper to do a split run and see which ones work in the real world, rather than gather the questionable predictions of consumers.
After running through the pitfalls, what emerged was the real reason for the study. The director wanted his agency to change strategy, but didn't feel able to get them to change and needed research to back him up. Of course, research is usually crap for this since it lends itself to multiple interpretations. And instead of a scary but real conversation with his agency, he was willing to throw money at a highly artificial conversation with a group of absolutely marginal relevance.
That's an extreme case of a phenomenon I've seen a lot of. Scratch many research briefs and you'll find a conversation that needs to happen inside the business. An elephant under the table that's not being talked about.
And when research is done to prove a point, as a substitute for a "fierce conversation", when there isn't genuine curiosity, I think it's likely to be a waste of time and trouble.
2. A fake conversation.
Boy did I become tired of focus groups. How weird that the nearest some marketing teams get to customers is to observe them from behind the safety of a one-way mirror in a focus group facility. And that's assuming they are observing, rather than knocking back the beers, checking their emails or continuing their internal politics while the conversation goes on next door.
I harboured a secret desire to stick the camera in the viewing area and do a debrief on the pyschology of what goes on in there. It would probably be way more interesting than telling people about the consumer conversation. For more insights, here's a great article on my own site, written by Leapfrog Research: What are the main issues facing viewing facilities? (Word format).
As for quantitative research... the effort to squeeze people into those agree/disagree batteries has its uses, but it so easily traps us into trying to put numbers on things that defy measurement. And it's another way of keeping the customer at a distance in a one-way conversation loaded with the marketer's preconceptions.
3 Obsession with the explicit
The third problem is that market research fixates on what can be made explicit in a relationship. Yet there is so much evidence that way more happens in real human conversations than might appear from the words exchanged. For a crude example, just consider the difference between reading email and meeting someone over coffee.
Malcolm Gladwell's Blink has some great stories illustrating the ways in which we deceive ourselves, so that asking us why we do things (which is often what market research does) can be a truly awful guide to what really motivates us. The danger here is that some researchers will leap on this as on opportunity to sell deeper expertise, complicated methodologies attaching electrodes to brains etc in the preposterous belief that if you drill deep enough you can find out "what's really going on".
OK, maybe that may turn up something interesting... but just look at the shabbiness of this as a way of conducting a human relationship. It continues to treat customers as objects to be "done to" and experimented upon.
What this seems to miss is that all human relationships, however "scientifically" managed, are two-way streets. When we set ourselves up as objective researchers we delude ourselves, for we ourselves are affected and influenced by what we do in a myriad of ways. And what happens inside a marketing department that treats customers as objects? I think you'll find that they expect each other to be treated as an object too... and think what that does to the quality of conversations inside an organisation.
When we go out and actually talk with customers, cutting out the middleman, we expose ourselves to more than just an exchange of information. We allow ourselves to be changed, to be moved, perplexed, provoked, saddened, cheered and to experience a real connection. Perhaps that's what some marketing deparments are afraid of ?
Jennifer Rice
Andy Lark
Johnnie Moore
John Winsor
1. jennifer rice on February 15, 2005 09:52 AM writes...
Johnnie, excellent post. I agree that the reluctance to have conversations with customers is a reflection of a much deeper issue: we're reluctant to have conversations with anyone. Our family members. Co-workers. The grocery clerk. In the "good ol' days," merchants knew their customers by name and knew their preferences. We've completely lost touch with each other. I'm guilty as well; I've gotten to where I'd rather send an email than pick up the phone. We need to start where we are, asking questions, connecting, listening, engaging... right now, wherever we find ourselves.
One other random thought: I find that if you must have a focus group, get only 3 people in the room... and use a shorter discussion guide, for 1.5 hrs. This set-up allows a real conversation to occur, and you can get much deeper than the shallow answers tossed out when there's 10 people in a room. And please, don't test ANYTHING in a focus group. OK, there's my 2 cents... :-)
Permalink to Comment2. Johnnie Moore on February 15, 2005 10:03 AM writes...
Thanks Jen. You know the more I think about this, the more I want to focus my practice on improving the quality of client internal conversations.
Permalink to Comment3. Steve on February 15, 2005 12:40 PM writes...
This thread reminds me of the conversation in famous toy store scene in movie "Big". Tom Hanks, in reply to a reference to market research asks "what is market research?". The toy company CEO answers "exactly!".
Permalink to Comment4. Bruce DeBoer on February 15, 2005 02:27 PM writes...
If you want to hear a collective moan, tell a creative team at an ad agency that the client wants to do creative focus testing.
I've always felt that if you set up an artificial test you will get artificial results. Anecdotal evidence is much better in my opinion: “Hey pal, what do you think of this” [while on a long airplane ride].
I've been thinking about my tendency to email rather than phone too. I think I avoid the phone too much.
-bruce
Permalink to Comment5. john winsor on February 15, 2005 07:35 PM writes...
Great posts. Johnnie I really like your three points and have had very similar experiences myself. I also think your point about fear as part of the problem is very astute. I would, however, extend that beyond marketing departments and say that it is a very real human experience to fear the unknown, especially if there is a possibility of conflict.
My point of view is this: I believe that research, in itself, is not a bad thing. That is, if it is done in the context of gaining a deeper human understanding of customers, involving them in a process of an open and honest dialogue. Unfortunately, most research currently being conducted is full of other agendas, as you so elegantly point out, Johnnie. Typical market research has become another example of a top-down tool that dominates the conversation between companies and their customers.
In my mind, focus groups are a great example of this control. By conducting them in the typical facility the context of the customer’s life is stripped away. How can anyone talk about the experience of driving a car without sitting in a car seat and talking about it in the context of the activity? Other issues, such as groupthink and bullying among focus group participants, only compound the problems with them. How do you take people out of the context of their lives and spend two hours in a strange environment, trying to cover ten topics with eight people? This gives each participant only a couple of minutes to react. It’s amazing that companies are willing to make significant strategic decisions after only getting to know a person for a couple of minutes? When companies concentrate primarily on using static top-down research techniques in the hopes of getting short-term financial gains they begin to loose the long-term connection that developing deeper relationships with their customers could provide.
Research does work when it is done in the context of discovery and rarely works when it done in the context of justification. Only in the context of discovery can any type of dialogue set the table for developing the trust a deeper relationship provides. Such a dialogue allows for the space essential to a co-creation process to begin.
Johnnie, you were right. I’ve got a lot (probably too much) to say on the subject of research.
Permalink to Comment6. Andreas on February 15, 2005 07:37 PM writes...
When I worked in market research in Indonesia, I dragged the MD of the agency into back lanes - for a walk after lunch. He was astonished. He asked, why would we do this? He said, he has go reports to write and proposals to prepare. I simply said that those guys down the lane in the slums of Jakarta are the users of the products we research for our clients and that it is a good thing to get the feel for the context, and the way they use those products.
I needed so much persuation to get him out of the office into those lanes. Numbers are somewhat more important than the actual feel of the customers. Thanks for a great post!
Permalink to Comment7. john winsor on February 16, 2005 12:19 AM writes...
There's nothing like hitting the streets!
Permalink to Comment8. Johnnie Moore on February 16, 2005 05:34 AM writes...
Andreas: Thanks for that story, a great example of what we're talking about here.
John: Yes, I guess we're kindred spirits on this. I'm glad you made that point about fear; I realise that my opening post had a an edge that seemed a bit mocking of the fear of the unknown. Of course, the unknown is something it's quite natural to fear and I often fear it myself. Finding a way to embrace uncertainty, as well as being a useful sensibility for marketers, is one of the great spiritual journeys for all of us. Me included!
Permalink to Comment9. john winsor on February 16, 2005 09:19 AM writes...
I struggle with fear, as well. I've learned a lot about it from two activities I spend a great deal of time doing, climbing and surfing. Both have taught me that fear is a good thing. It sharpens the mind and focuses the attention on what must be done.
The same thing goes for interacting with customers, or anyone for that matter. It would do all of us, who work in the world of marketing, some good to be a little more scared and a little less cetain about our own knowledge. Maybe then, could we treat our customers as humans that deserve more than being examined through the lens of research.
Permalink to Comment10. Steve Portigal on February 16, 2005 02:21 PM writes...
"That's an extreme case of a phenomenon I've seen a lot of. Scratch many research briefs and you'll find a conversation that needs to happen inside the business. An elephant under the table that's not being talked about."
Absolutely. It seems like a lot of clients have the explicit problem and the implicit problem. Sometimes the gatekeeper or direct client contact understands that and understands their own culture and is engaging this research in order to force the conversation. I don't think this is necessarily a bad thing, and of course, doing some good stakeholder interviews at the start of a project can help bring this to light, and help frame the kind of delivery you want to do.
A client presentation last week faced this head-on. We had recommendations for their website but also a lot of "decide who you want to serve" and the resulting conditional recommendations "If you want to serve this customer base, then do X. If not, then do Y. Right now you are in a straddle situation and are confusing everyone."
After we were done we got the blue-sky consulting questions - what would we do next? Would we want to do more research? Or different types of research? My suggestion was that they needed to make some decisions.
I think they knew that they need to make some decisions, but that disconnect internally is more implicit. Here they are dealing with the tangible - make a new or better website - and the issues become explicit - you can't make a site better until you sort out your internal challenges about WHO you want to serve.
And maybe it's just consultant's ego thinking that bringing up the elephant in this context is going to help 'em; maybe their internal challenges are limiting the effectiveness and they can't move forward until they decide themselves they need to solve that problem. It felt good from my end; but I don't have the global view that my client may have.
Permalink to Comment11. Steve Portigal on February 16, 2005 02:57 PM writes...
"When we set ourselves up as objective researchers we delude ourselves, for we ourselves are affected and influenced by what we do in a myriad of ways...When we go out and actually talk with customers, cutting out the middleman, we expose ourselves to more than just an exchange of information. We allow ourselves to be changed, to be moved, perplexed, provoked, saddened, cheered and to experience a real connection"
Hear, hear! As an ethnographer, I don't even consider myself a "market researcher" - but sometimes that's the easiest way to present myself, without having to over-educate an audience that maybe isn't interested. And as ethnography becomes a known quantity or "commodity" - it's sometimes easier to just take the label when it helps a market research client understand - in general - what I will do for 'em.
But I have been stressing the "infer" aspect of what I do for a long time...that's what you get when you have an expert - not a facilitator or a moderator - though shepherding the process along properly is essential, the real benefit is in the sense-making, the interpretation, the collection of data points that were never explicit, and the pattern matching.
And I guess that isn't how much of this is practiced. Thanks for the reminder, not really a rant because you've highlighted the positive aspects of research - or getting to the customer - or whatever jargon you want to use...encouarging and exciting to read.
Permalink to Comment12. Peter Hutton on February 17, 2005 02:42 PM writes...
Absolutely right about ‘scratch many research briefs and you'll find a conversation that needs to happen inside the business’, Johnnie.
After a few years in market research I began to realise that the research brief tells you a lot about the nature of the organisations commissioning the research and probably also whether the research is going to be any use at all whoever ends up doing it. If the brief only talks about research objectives and tells you nothing about what the business is really trying to do, then chances are the research is really there to tick a box in the process that says ‘commission research’ . The best research is commissioned in the spirit of ‘it really would help our business to do the right thing if we listened regularly to the various people who are involved in how we create value.’ These are businesses that have spent a lot of time developing a real understanding of what they are about and how different parties work together to make the business work.
The worst surveys for ticking the ‘commission research’ box are, in my experience, employee surveys. Some suppliers have become expert at providing survey instruments that provide piles of data which are virtually impossible to action. But that’s another blog!
Permalink to Comment13. lev pyotr on March 7, 2005 02:07 PM writes...
http://lev.nicheservers.com/bondage_7.html
Permalink to Commenthttp://lev.nicheservers.com/ameture_2.html
http://lev.nicheservers.com/their_6.html
http://lev.nicheservers.com/videocli_2.html
http://lev.nicheservers.com/thumbnai_3.html
http://lev.nicheservers.com/body.html
http://lev.nicheservers.com/sexy3_2.html
http://lev.nicheservers.com/bondage_9.html
http://lev.nicheservers.com/bdsm_6.html
http://lev.nicheservers.com/backdoor.html
http://lev.nicheservers.com/vidio_8.html
http://lev.nicheservers.com/bukkake_5.html
http://lev.nicheservers.com/sexy2_6.html
http://lev.nicheservers.com/sexy3_1.html
http://lev.nicheservers.com/son4_3.html
http://lev.nicheservers.com/upskirts_1.html
http://lev.nicheservers.com/strapons_2.html
http://lev.nicheservers.com/vidio1.html
http://lev.nicheservers.com/shaved_8.html
http://lev.nicheservers.com/vergin_1.html
http://lev.nicheservers.com/sexy2_5.html
http://lev.nicheservers.com/storie_2.html
http://lev.nicheservers.com/shaved_9.html
http://lev.nicheservers.com/cartoon1_3.html
http://lev.nicheservers.com/body_1.html
http://lev.nicheservers.com/blonds_1.html
http://lev.nicheservers.com/veronica_1.html
http://lev.nicheservers.com/audio_10.html
http://lev.nicheservers.com/teen6_3.html
http://lev.nicheservers.com/anal1_3.html
http://lev.nicheservers.com/teeny_1.html
http://lev.nicheservers.com/video3_7.html
http://lev.nicheservers.com/bulma.html
http://lev.nicheservers.com/siti_1.html
http://lev.nicheservers.com/teen6_2.html
http://lev.nicheservers.com/anima_2.html
http://lev.nicheservers.com/spy_3.html
http://lev.nicheservers.com/teen_11.html
http://lev.nicheservers.com/ameture_1.html
http://lev.nicheservers.com/teen_1.html
http://lev.nicheservers.com/socks.html
http://lev.nicheservers.com/sexgirls_2.html
http://lev.nicheservers.com/videocli_4.html
http://lev.nicheservers.com/bukkake_3.html
http://lev.nicheservers.com/stories.html
http://lev.nicheservers.com/sexy3.html
http://lev.nicheservers.com/cam1_4.html