Surveys and Science: It Takes Two to Tango
By Simon Kooyman, CEO

Crisis. Change. So what is it? Uncertainty, genuine concern, commercial (self-)interest, hypocrisy, self declared? Perhaps all of the above are slipping into the discussion that has moved from online panel quality to the recently proclaimed imminent death of survey market research as we know it.

That the online panel pirates are yelling for the end of random sampling is understandable. It feeds the influential market research buyers' addiction to the siren song of the lower priced Loreleis, of course. We know that quality shortcuts equate to higher profits next quarter – no? Thus, representativeness is hogwash? I hope not. I know not.

A simple example from a simple question asked in a survey is illuminating: "Please tell us whether you agree or disagree with the following statement: ‘When I shop I look for what is new'." Among members of KnowledgePanel®, about a third "agreed"/"strongly agreed," while 55% and 59% of two highly used opt-in panels did so. Does this not affect what we learn from each sample that is the "same" when demographically weighted?

The industry, its associations and its clients are, however, connecting dots – essentially drawing the conclusion that the science of market research should be replaced by the art of listening. Too fast for my blood. Too absent a clear view of when to listen, when to ask, and for what learning that informs a business action.

Following the same pattern, why don't we stop clinical trials for new drugs? Let's just watch the human guinea pig as we inject a new molecule. If the subject survives, or even better, is healed without too many side effects, roll out. Or, when we are visiting the family physician, just let him look us in the eye, perhaps let his fingers dance on our upper body. To hell with tests of blood, urine, biopsies. Either their quality stinks, or we don't like the procedure, or the outcomes could disappoint; in any case, no need for them.

While we are at it, let us stop auditing financial statements, state vehicle inspections, and school exams. Generally, no one likes them nor understands them, and the science underneath them can therefore be questioned.

We see the build-up of a collective delusion – a mirage, replacing the recognition of the responsibility to apply sound marketing science, something critical business decisions still require.

It is required that the elements of market research science continuously need to be challenged, tested, calibrated, updated and, in some cases, overhauled or even discarded. The support of innovative thinking is needed for market research to remain one of the sources of innovation. But the essence stays the same; to say something projectable about a population you have to correctly draw, observe and analyze data from a known and representative sample. And to give that process credibility and its outcomes an acceptable probability, our profession has scientific roots. To learn from listening can be informative if not enlightening, so why not opt for symbiosis rather than replacement?

Calling market research science, in effect, an oxymoron is the easy way out.

The market research industry and, specifically, its clients cannot have it both ways. Asking for more research, innovative new research methods, less quality control by the client, results within hours, cheaper and with the design, analysis and interpretation more and more outsourced to suppliers – it all is possible. Yet please do not ask us to drop the science – especially in this economic climate, where every research dollar should bring you an "Aha!" that when acted upon, turns out to build your business. Not unless the outcomes don't matter – in which case Knowledge Networks is probably not your best choice as a market research supplier.

Simon Kooyman is CEO of Knowledge Networks.