CGM Research - Is it Worth Fishing For?


Savvy divas (and divos) know that it is critical to listen and learn from our customers. Social media (blogs, boards, chats, podcasts, vlogs) provides a wealth of raw conversations. For the first time we can "listen in" on unfiltered customer talk. But is that CGM data credible in terms of using it to base marketing and business decisions?

Last weekend I chatted with Divo Bill Neal about CGM (consumer generated media) and marketing research. Since his boat was in dry dock for repairs and he couldn't go fishing, he kindly gave me his take on the situation.

About Bill Neal
Bill has been called  the "Godfather of Marketing Research."  He is co-founder and co-owner of SDR Consulting and also leads the firm's legal consulting practice.

He is on the Editorial Review Boards of Marketing Management and Marketing Research magazines and is an ad hoc reviewer for several other marketing management and marketing research publications. Over the last 33 years he’s dedicated significant time teaching – speaking at conferences, conducting workshops and tutorials, and writing articles.

He was instrumental in launching all four of the master’s programs in marketing research. With Mal McNivan he conceptualized and designed the popular online certificate program in marketing research originally sponsored by the Marketing Research Association. Bill has served as Chairman of the Board of Directors for the American Marketing Association. He is a 2002 recipient of the Charles Coolidge Parlin Award for distinguished achievement in the marketing research field.

Bill’s latest project is writing a book on brand equity metrics and how to leverage those as a key focus for overall corporate leadership. Soon to be published by Thompson Southwestern.

Wdn_2_stripers_small_2 Oh yeah..and he's a advid fisherman! I understand Bill not only knows how to catch those fish but how to cook them as well.

Diva: Bill, Marketing researchers are (or should be) the organization’s gateway to the thoughts, needs, desires, concerns and challenges of the customer. You and I have seen research methodologies go from door-to-door interviewing to pop-up surveys on website. The latest concept is to use consumer generated media to gain insights into consumer behavior. Do you think that "listening in on the raw voice of the customer" has merit?

Bill: It’s always a good thing to listen to the voice of the customer, in every form. That’s one of the main mandates of marketing research. And it’s our job to take those many voices and make some sense out of it all – call it insights.

But I have some real problems with consumer generated media as a source of credible and reliable information. In many ways it combines the worst elements of non-scientific research – self selection and advocacy – both positive and negative.

That is, those out there in the Internet world who are generating their own media are self-motivated to do so and are not representative of any defined population of buyers. And, given the fact that they have taken a public position on a particular product or service, it means that they more often than not have exceptional or non-typical attitudes about those products and services. The information they generate may be true, or not true – there is no way to discern which. Therefore, the information generated by those folks is neither credible nor reliable. So, as researchers, yes, we should be listening, but we must be very cautious and skeptical about its veracity and its usefulness.

Diva: Should marketers be tracking the data that occurs in blogs, boards and other social media formats? If so what should they be concentrating their efforts on?

Bill: Oh yes, I think we should be tracking it. The shear volume of product mentions and whether they are positive or negative is useful information and may (and I emphasize “may”) provide a signal that something is going wrong, or right, about the product and how it is being marketed.

Trending that information would be one of many ways companies should be tracking their marketing performance. I’d consider it one of many, albeit smaller, gages on the marketer’s product dashboard.

But marketers must keep in mind that a few influencers can generate a great number of product mentions if they decide to feature a particular product or service in their blogs. And these things can get out of hand very quickly, signaling a problem that’s really not a problem to the vast majority of customers.

Let me give you an example. I own two Ford trucks – a 1997 Expedition with close to 200,000 miles on it and a 2004 F-150 with 25,000 miles. In the blogosphere I’ve seen a bunch of postings on Ford trucks depreciating their quality and reliability. Yet, both of the vehicles I own have been exceptional in quality and reliability. I don’t take the time to post those positive experiences, but some who have had problems are very vocal about their supposedly negative experiences.

So what is the truth of the matter? Did these negative experiences really occur? Was it the fault of the manufacturer? Or the Dealer? Or the buyer? Are these generators of consumer media about Ford trucks really being the dispassionate arbiters of truth, or do they have an agenda? That’s the key issue – that information gleaned from the blogosphere is simply not reliable, and in many cases it is not valid.

There is also the issue of influencers in the blogosphere. Some of them seem to be very credible and it’s important to them that they maintain their honor and integrity – Your Diva blog is a great example. But there are others who have no honor, and it’s hard to tell who’s who. And there is no way we can weight the product mentions in terms of the credibility of the generator.

Diva: If companies are going to monitor the buzz in the blogosphere, where do you think that should reside? With brand managers or with the research department?

Bill: I think it definitely needs to be in the research department. Good marketing researchers are skeptics, and numbers generated from the blogosphere need to be viewed with a high degree of skepticism.

I’ve seen too many brand managers observe a focus group or two and then make major changes in their marketing programs based on what they heard in the focus group. Many of those changes are utter failures simply because the focus group was not a reflection of the real world.

You have exactly the same problem with monitoring consumer generated media, except that now you have the equivalent of thousands of focus groups. Does that make the information more valid or reliable? Definitely not!

Diva:  How about your thoughts on the companies that are jumping on the research social media band wagon i.e., Nielson BuzzMetrics, Fortune Interactive?

Bill: Honestly, I don’t know a lot about them and have not used them in my consulting practice, nor do I address them in our forthcoming book.

But to the best of my understanding, they are primarily counting product/service mentions and, in some cases identifying the major sources of those mentions. The basis for their business model is the belief that consumers have a higher trust of consumer generated media then they have for company generated media.

I think that might be more of a reflection on the stupidity of much of the advertising and promotion that permeates today’s traditional media. I’ve already talked about the problems with simply counting the number of brand “hits” and how that can be so misleading.

And, as consumers mature in their understanding of how consumer generated media can be manipulated by those with less than honorable intentions, I think their trust in those sources of information may wane considerably.

Diva: I’m looking forward to reading your book. Will the book include mention of social media and/or consumer generated media?

Bill: Thanks – I look forward to getting it done since we are scheduled to publish this coming spring. My co-author, Ron Strauss, and I have not settled on a title yet, but our working title is Marketing Metrics for Marketing Leadership.

We don’t address any particular media in detail. Our focus is to further develop valid and reliable measures of brand equity as a tool for corporate leadership. In that context, social media and CGM could play an important role for some products and services, especially in the non-profit arena. But, as I’ve said throughout this interview, measures of those phenomena must be both valid and reliable, and most importantly, they must accurately reflect the thoughts and behaviors of well-defined populations.

Diva: Hope your boat gets out of dry dock soon!

Sidebar: Cross posted on MarketingProfs Daily Fix.

Technorati Tags: , , ,


Trackback url:

» Is Social Media Research Valid? from Backbone Blogging Survey
Toby Bloomberg's interview with Bill Neal, an expert in marketing research provides some analysis of consumer generated media. "I have some real problems with consumer generated media as a source of credible and reliable information. In many ways it co... [Read More]

Tracked on Jun 13, 2006 11:12:46 AM

» Indexing Consumer Generated Media from Relationship Marketing Blog
Toby Bloomberg over at the Diva Marketing Blog has a great interview with Bill Neal, the "Godfather of Marketing Research." Mr. Neal has some interesting points of views on using consumer generated media (CGM) as part of market research. He concedes th... [Read More]

Tracked on Jun 15, 2006 3:58:40 PM

» Diva Marketing Blog Interview with Bill Neal from Data Mining
The Diva Marketing Blog has an interview with Bill Neal - a marketing research guru. There are a number of problems with Mr Neal's responses regarding the value of social media anlaysis as well as the sophistication of the technologies [Read More]

Tracked on Jun 16, 2006 11:59:12 PM

» Measuring the results of social media from Expertise Marketplace - Professional Service Firm Marketing Blog
Bill Neal, a market research guru if there ever was one, offers his insights on the phenomenon of social media, and the pitfalls of measuring it, In a fascinating interview here. In essence, Neal says we should measure the influence [Read More]

Tracked on Jun 20, 2006 7:35:44 AM


Hmmm...I kind of take issue with this. My colleagues at relevantNOISE and I view blogs as the *world's biggest* focus group. There's a lot of useful data in blog posts!

relevantNOISE and some of our competitors are using technology to not only count the number of blog posts out there...but to actually glean the sentiment. We can tell if the noise in the blogosphere around a brand is positive or negative. I think most brand managers would agree -- that sort of data is pretty useful.

(I imagine the folks at Dell and Warren Kremer Paino Advertising would agree, for sure.)

My point is, traditional market research is great -- we all know that. But blog mining as a method of market research should not be dismissed or underestimated. A lot can be learned, and unlike traditional market research, it can be learned -- and acted upon -- in REAL TIME.

BTW, I fish too (funny, since I'm a lifelong vegetarian) and MAN do I wish I had a picture of myself and my trophy bass right now!!

Posted by: Aimee on Jun 14, 2006 4:58:21 PM

Great interview, which reminds us once again of the pitfalls of rushing headlong (usually right over a cliff!) without considering the broader landscape.

I've watched many marketers embrace a new trend in marketing, only to find themselves burned because they didn't accurately track how it influenced their company's results. Neal's comments guide us to do this well.

Posted by: Suzanne Lowe on Jun 19, 2006 3:36:47 PM

How far do companies go to prevent product owners from sharing information and/or comparing notes? Will companies interfere with the discussions by defaming the offending consumer or worse? Will companies target a blogger who doesn't give up until justice is done? When do companies cross the line from monitoring to interfering with product owners' freedom of speech? Is anyone looking at this aspect of CGM? What once reputable companies are risking it all to save their names no matter what the quality of the product is?

Posted by: Charlene Blake on Jul 2, 2006 2:19:13 AM

Great interview. Every marketer needs to listen to Bill's advice, which is right on target.

Toby, thank you for introducing me to Bill. He seems a great guy.

Posted by: Lewis Green on Feb 9, 2007 1:20:23 PM

The comments to this entry are closed.