Garbage in Garbage Out. Why Research Sucks!

Recently someone posted on Twitter that 80% of Gen-Y thinks Advertising is truthful. And when I see something that does not make sense I investigate. So I did. I came across something so grossly distortional it truly upset me. It also misleads readers and goes to the heart of something. Garbage in Garbage out.
Why do so many new product launches fail? Because of poor research. The reasons are a myriad. From biased questions, biased focus groups, biased or lying participants, and how the data get’s reported.
I have railed before at poor reporting of news, like Mashable who is almost so deceptive or inept that they are not a news source for me. 
But news is one thing. Actual research is another. Read the headline of the next link. Then the first line of the write up. Completely different meanings and conclusions.
In this Harris Interactive study on whether people think Advertising is Truthful presented by Marketing Charts they did the double whammy. First the questions suck. Seriously. Expand the charts. What a horrible choice of questions. Secondly the write up by Marketing Charts is 100% wrong. Not just inaccurate. But WRONG!
The data shows that most people do not trust advertising what so ever. It also shows when it comes to regulating advertising they trust NO ONE! 
Yet when you read the write up it sounds like most people find Advertising Truthful and that they don’t trust the Government to regulate it. Insert the sound of Kyles Mom:
Two glaring issues I have with the data since I am tossing the write up into the trash. 
They should of asked  Do you trust Advertising to be Truthful: Yes or No?

Then maybe a more detailed question about that. When I see the word ‘Sometimes’ in a question what does that mean. Sometimes being truthful would mean from 1% to 50% of the time. Most of the time could mean 51% to 99%. HUGE SPREADS. And thus the data is really a statement that people are smart and they do not trust Advertising or anyone who will be given the task of Regulating it.

See the Ad Contrarian’s write up of a Nielsen Study as another example of poor research:  LINK

It is really important that you craft your research as unbiased and objectively as possible. Do not be afraid if it turns out the opposite of what you hope. It will save you money killing a project before it flops, than having it flop.


About chiefalien

Howie Goldfarb with 20 years of Sales and Marketing experience founded Blue Star Strategic Marketing in central Vermont to serve as the objective and strategic adviser of brands to help them grow and thrive. His Degree in Finance and 14 years of B2B sales to Fortune 500 companies gives him a CFO’s view of marketing. Thus bringing a dose of reality to the confusing world of jargon, spin, and hype. Also playfully known as the Chief Alien of Blue Star Strategic, Howie relishes his role as an industry outsider. A native New Yorker and former Angelino, he currently lives in the Green Mountains of Vermont and is still seeking his first moose sighting. His passions are living life, art, music, the outdoors, he tries to cook and loves the Vermont Fresh Network – local sustainability initiatives like farm to table and buying local.
This entry was posted in advertising, garbage, research. Bookmark the permalink.

One Response to Garbage in Garbage Out. Why Research Sucks!

  1. doogs says:

    I spent several years on the research side of things before making the transition to jaded and cynical creative director, and from direct experience, everything you've written is spot on. Research is a tool. Properly conducted and compiled, it gives you the lay of the land, and with the analysis of someone who knows how to navigate the data, it can yield all kinds of juicy insights. But all to often, it's a gigantic sloppy mess, with questions written and data compiled by people who know the answers they want beforehand. It's not any better on the receiving end, either. I can't tell you the number of times I got requests for research showing X, even when every possible data point indicated that X was completely dead wrong.Research, and the analysis of research, requires objectivity, intellectual honesty, and a willingness to be confronted with findings that contradict your assumptions and make you look like an idiot (those contradictions are usually where the best insights come from). Sadly, these qualities are rare, at best.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s