Knowledge Networks
Accuracy's Impact on Research

Spring 2007

The quality of research – its accuracy, its reliability and its projectability – was a concern long before the Internet age. But surveys based on volunteers, professional respondents and "samples" of unknown origin and intent have made the issue of research quality more crucial now than ever. Knowledge Networks was founded on the principle of bringing trusted, valid research approaches to the Internet, and our Accuracy's Impact on Research continues that tradition. Consider it your best source for understanding what makes research accurate and reliable. We welcome your thoughts and suggestions.

– Patricia Graham, CMO, Knowledge Networks
pgraham@knowledgenetworks.com
312 416-3660

TABLE OF CONTENTS

 

Of Trains, Panel Quality, and Sample Coverage
By Charles DiSogra, Chief Statistician
The attractiveness of cheap access to large numbers of online volunteers successfully lures many budget-conscious clients to opt-in panels. But do these "samples" represent anything in the outside world? Charles DiSogra, KN's Director of Statistics, gives some needed perspective on a crucial topic.

line
html | pdf
 
 

Finding Words for the New Research Reality
By Simon Kooyman, CEO
It is hard to remember a time when research companies were rushing so intently to wrap themselves in the mantle of "quality"...which, of course, means that that the term has less and less meaning every day. KN CEO Simon Kooyman helps us make sense of a research world in which long-honored terms have become objects of deception.

line
html | pdf
 
 

Polling & Research Quality Circa 2008: An Interview with Kathy Frankovic
Kathy Frankovic, Director of Surveys at CBS News, shares her experiences and thoughts on changing attitudes toward research reliability among the public, poll takers – and even journalists.

line
html | pdf
 
 

Of Metrics, Genomes, and Morality: An Interview with Kathy Hudson
As founder and Director of the Genetics and Public Policy Center, Kathy Hudson relies on research to back her up when she writes juried articles – or testifies on Capitol Hill. She talks about the important role high-quality surveys and resources (such as KnowledgePanelSM) play in her work.

line
html | pdf
 

Stated Preference Methodology: Innovative Online Approaches to Accurate Measurement
By J. Michael Dennis, SVP, Managing Director, Government and Academic Research
In estimating the value to consumers of a variety of abstract environmental concepts, KN's Government & Academic practice has deployed a host of innovative research approaches and methodologies.

line
html | pdf
 
 

Accurate Measurement & Media Hype: Placing Consumer Media Technologies in Context
By David C. Tice, VP, Group Account Director, Custom Research
As media and cutting-edge technologies have increasingly merged – from DVRs to cellphones – the challenge to invest wisely in new devices and services has become more pronounced. The director of KN's The Home Technology Monitor™ reveals how a higher-quality approach has paid off.

line
html | pdf
 
 

NEWS BRIEF
In March, AAPOR President-Elect Nancy Mathiowetz wrote a Letter to the Editor of MSNBC in response to a non-scientific online survey: http://www.aapor.org/statementonmsnbcworkplacesurvey?s=msnbc

Study Highlight: National Institutes of Aging Study on Medicare Part D
To assess participation in Medicare Part D, the ground-breaking federal program, economists Florian Heiss, Daniel McFadden and Joachim Winter turned to KnowledgePanelSM for baseline and post-enrollment interviewing. Dr. McFadden won the 2000 Nobel Prize in Economics; his work is the basis for Discrete Choice methodology used in the marketing research industry and by economists.

The study, titled the Retirement Perspectives Survey, was conducted by the University of California, Berkeley, and sponsored by the National Institutes on Aging (NIA). It began with baseline interviews in November 2005, before Medicare Part D enrollment started, and sampled panelists age 65 and older. The post-enrollment survey involved a sub-sample of 2,360 cases that had completed the baseline RPS interview; more than 90% of all seniors sub-sampled for the post-enrollment interview completed the survey in the two weeks following the close of enrollment. Heiss, McFadden and Winter are currently conducting a third NIA-sponsored survey among original participants as well as 1,000 new respondents.

Estimates of prescription drug coverage from the KN study were consistent with those released from the Centers for Medicare and Medicaid Services in June 2006. Results suggest that while the program accomplished its primary objective of boosting coverage to 90%, majorities of seniors are less satisfied with Medicare and with the government as a result of this program. The analyses identified vulnerable subpopulations and described which groups failed to enroll despite remaining uncovered for prescription drugs. An early report of their findings appeared in Health Affairs on August 1, 2006, less than three months after the drug-benefit enrollment period ended.

Research on Research
Knowledge Networks continues to self-evaluate using KnowledgePanelSM—its U.S. population-projectable panel—to examine a range of online panel effects. At AAPOR's 62nd Annual Conference in May, Sergei Rodkin of Knowledge Networks will present results from a study conducted on behalf of the Youth Alcohol Prevention Center at Boston University School of Public Health (funded by the National Institute on Alcohol Abuse and Alcoholism). The surveys, executed by Heeren, Edwards, Rodkin and Dennis, were conducted as part of an epidemiological study on the age of onset of alcohol use and related behaviors and examined three methodological issues:

  • mode effects of data collection (internet vs. telephone)
  • self-reported behavioral and attitudinal differences between respondents and non-respondents (via a non-response follow-up survey of persons who did not join the web panel or dropped off)
  • comparison among web, telephone, and face-to-face interviews (the latter was compared to NESARC, the National Epidemiologic Study on Alcohol and Related Conditions—a national benchmarking survey conducted by the Census Bureau by personal interviewing)

Results revealed no significant differences in most major alcohol-related behaviors (online panel vs. NESARC). However, while the data collection mode effect was negligible for alcohol behaviors, it was more pronounced for attitudinal questions. Additional findings will be presented at AAPOR, including differences among the control group, telephone, and Internet modes.

 
 

dartboardNew AMA/KN WebCAST

Tune into Knowledge Networks' AMA Webcast on Consumer Segmentation to learn how our best practices can deliver the insights you need to help drive your marketing program and manage your brand.
Date: Thursday, May 24, 2007
Time: 1-2PM (EST)

REGISTER HERE

You will also be able to access a recording of this event via the AMA site after the 24th; go to http://www.marketingpower.com/webcast-search.php; click "All" and type "consumer segmentation" in the search box.

dmgJoin The KN Quality Circle

Stay up to date on key issues and insights on research quality – join the KN Quality Circle. In addition to future copies of this newsletter, you will receive information on events and publications related to research quality (including the updated 2007 Decision Maker's Guide to Online Research, to be published later this year).

Click here to become a Charter Member.

Quality

For more information contact:

David Stanton
908 497-8040
Email