[ Summer 2008 ]
The Most Welcome Distractions
By Simon Kooyman, CEO
The research industry is abuzz with talk of just one thing...
Quality? No, silly. Now the wireless wires are humming with talk of acquisitions and mergers. Greenfield has been bought; TNS, GfK and WPP are staging a danse macabre; and at least 3 other US companies are "in play". A host of other companies, having no one on their dance cards, have begun to wonder if they are doomed to wallflower status.
In the political realm, the slickest and smartest operators have learned that, when one thing is not going so well, you stick a giant red flag on another, less important event – and hold a news conference in front of that flag.
It's called "the politics of distraction"; and it seems to be working rather well for online panel companies. At a time when it is becoming clearer that opt-in, volunteer panels push the accuracy limits, their owners seem to be quite happy to talk about mergers instead.
But the fact is that, while there have been a number of steps in the right direction – from the various initiatives by the associations, clients, and technology providers such as Peanut Labs and MarketTools – some of the core issues of online panel quality have largely been placed aside. Professional respondents, duplicate panelists, and speedsters all pose challenges; but they really cannot compare with the difficulties that ignoring statistical principles creates. These principles represent the core difference between opt-in panels – where anyone can volunteer and nothing is projectable – and those based on a probabilistic methodology. Data hygiene, though it is certainly worthwhile and needed, cannot "fix" this larger problem.
Somehow I am reminded of luxury cars that come with a first aid kit in the glove compartment. Which accident will that kit truly be helpful with?
We have seen the first wave of concern about research accuracy get played out; everyone signed up for some committee or another, and lots of good discussions have happened. Now the second wave is waiting to be born...
That wave is not just about saying the right things; it is about demonstrating real leadership. But who can take the lead when we are all talking and worrying about who will buy whom?
For a revolution to take hold, things have to get really, really bad. I think we are not quite there yet, we still need more Verelendung – more pain, more shock. Client companies may well be sitting on all the evidence we need to spark a revolution – research whose questionable conclusions led to wrong decisions. But only a little of that evidence is seeing the light of day.
Think of the credit crisis; how many loans had to fail before someone started paying closer attention? How many months after that till someone got arrested? Do things need to get that bad in the research world?
It is interesting to note, by the way, that in the worlds where government scrutiny of research is a possibility – studies for various federal agencies – researchers have been much more reluctant to give up on the statistically valid sampling approach. That is the only approach that allows one to project to larger populations. In short, it is the only method that guarantees a better night's sleep.
In a world where the biggest of the big boys are focused on whom they are buying and who is buying them, smaller companies must take the leadership role in research accuracy. Their pockets are not as deep, but their commitment to knowing and disseminating is powerful.
Knowledge Networks is pleased to be one of those companies. We are blessed to have a cadre of clients – many of them in the government/academic realm – who know the hazards of not respecting statistical principles, and who rely on the fact that we will not compromise in our approach. Thanks to them, KnowledgePanel® is unique in the world of online surveys, with the only probabilistic approach to sampling among the online panel companies.
So, in a business world where distractions are many, how can we be sure that we get the accurate research we need and deserve? KN will continue to lead in the creation and refinement of sound methods for building and maintaining panels; and we will welcome any others who might want to put aside distraction and take up the cause!
Simon Kooyman is CEO of Knowledge Networks.