Taking the Pulse of the Drive for Online Research Accuracy

[ Summer 2009 ]

By Simon Kooyman, CEO

The discussion about research accuracy should never be seasonal or out of fashion; it remains very much on the minds of the marketplace – especially those who operate and use online consumer panels.

So, as we close out the first half of 2009, where do we stand on this important issue?

pulse We have seen a number of organized efforts at assessing quality deliver preliminary results and advocate specific changes in how online panels are managed, assessed, and applied. The Online Research Quality Council (ORQC) organized by the ARF has helped bring together findings from 17 opt-in online panel sources alongside mail and telephone modes and reached some notable conclusions through an extensive analysis of results. In addition, the consortium known as OpenSample has enlisted online research firms, large and small, to help understand and track respondent behavior.

Knowledge Networks is pleased to have actively contributed to both of these efforts, and we will remain keenly engaged. But we must observe that any sense of early satisfaction within the industry needs to be tempered by awareness that there is more to do.

For example, from the above and other industry work we know that about four out of ten online respondents belong to more than one panel, with an average membership of roughly four panels – ranging from two to 13. This group also has a high probability of taking up to 20 surveys per month, or four to five each week. While recent studies indicate that this may not be a problem in and of itself, it is a further testable hypothesis to examine whether this group is also more than average stimulated by incentives – and that can be a concern.

For many types of research, these respondents may indeed provide valuable directional advice. But when the decision at hand is essential to your business, with strategic initiatives and major investments at stake, one has to wonder if "ok" is good enough.

Therefore, we continue to strongly propose that a central database be built to capture – on a confidential basis – how many surveys a respondent has taken and in what categories on an ongoing basis. This will create a first uniform standard for an important quality dimension (activity per category), lays the groundwork for the next dimension (duplication and overlap) and gives clients and research firms alike the choice to select the appropriate quality level for the right type of research projects. Similar quality standardization exists in many industries with a significant commodity market as basis. I urge research companies and clients alike to take this no-brainer first step as an indication of our seriousness to bridge the gap in quality expectations.

In the meantime, if you want to be sure about the accuracy of the information you are using, there is an option that has already been affirmed as uniquely precise and statistically reliable – KnowledgePanel®. No other online panel even comes close to the representation and accuracy that you get from KnowledgePanel®.

I look forward to continuing to participate with you in our collective efforts to make a real difference when it comes to online panel quality. We have come along way, but the work is not nearly over.

Simon Kooyman is Chief Executive Officer of Knowledge Networks.