[ KnowledgePanel® Design Summary ]

KNOWLEDGEPANEL® OVERVIEW

KnowledgePanel®, created by Knowledge Networks, is a probability-based online Non-Volunteer Access Panel. Panel members are recruited using a statistically valid sampling method with a published sample frame of residential addresses that covers approximately 97% of U.S. households. Sampled non-Internet households, when recruited, are provided a laptop computer and free internet service so they may also participate as online panel members. KnowledgePanel consists of about 50,000 adult members (ages 18 and older) and includes persons living in cell phone only households. Hispanic panel members represent the full spectrum of language proficiency from exclusively Spanish speakers to those proficient in English only. In addition to the adult members, the KN panel also includes approximately 3,000 teens, ages 13 to 17, whose parents or legal guardians, usually themselves panel members, have provided consent. Due to voluntary withdrawal, involuntary retirement of high-tenured members, and the addition of new panelists from the on-going recruitment, actual panel size fluctuates across the year.

Address-based Sample (ABS) Recruitment Methodology

Recruitment to KnowledgePanel® was first conducted in 1999 using random-digit dial (RDD) methods. At that time, all recruited households were given a MSN®; TV WebTV to use for answering surveys. In August 2002, KN began allowing households to use their own computers connected to the Internet for taking surveys. Since January 2009, Windows-based laptop computers are provided to non-Internet households instead of WebTV units.

Until recently, KnowledgePanel's probability-based recruitment had been based exclusively on a national RDD frame. In 2009, KN initiated the use of an address-based sample (ABS) frame to first supplement the RDD frame and subsequently replace it. This was in response to the growing number of cell phone only households that are outside the traditional RDD landline telephone frame. Also, this switch was motivated by declining RDD response rates. ABS involves probability-based sampling of addresses from the U.S. Postal Service's Delivery Sequence File. Randomly sampled addresses are invited to join KnowledgePanel through a series of mailings (English and Spanish materials) and by telephone follow-up to non-responders when a telephone number can be matched to the sampled address. Invited households can join the panel by one of several means: completing and mailing back an acceptance form in a postage-paid envelope; calling a toll-free hotline staffed by bilingual recruitment agents; or going to a dedicated KN recruitment Website and completing the recruitment information on line.

The address sampling, conducted throughout the year, is done without replacement. Addresses with matched telephone numbers from the former RDD recruitment samples (for the last five years of calling) are also removed to eliminate duplication.

In addition, in 2008 KN constructed KnowledgePanel LatinoSM taking quality of online panel representation to the next level by providing netbooks and Internet service for the roughly 40% of Latinos who do not have Internet access at the time of recruitment. The sample for KnowledgePanel Latino uses a dual frame design. The main sample is recruited through the mail using English and Spanish materials. This address-based sample (ABS) is drawn from the U.S. Postal Service's Computerized Delivery Sequence file that covers approximately 97% of the physical addresses in all 50 states. The ABS mail sample represents all households whether they have only cellular telephone service, a landline telephone or no telephone service. The ABS sample is further supplemented with a smaller random-digit dial (RDD) telephone recruitment that specifically targets high density Hispanic areas across the country. This RDD sample is designed to exclusively recruit additional Spanish-dominant households. As a result, KnowledgePanel Latino has the most complete coverage of the U.S. Hispanic population.

For all new panel members, demographic information such as gender, age, race/ethnicity, income, education, and for Latino members, language proficiency are collected in an online "profile" survey. This information is used to determine eligibility for specific studies and eliminates the need for gathering basic demographic information on each panel survey. After this survey is completed, the panel member is regarded as active and ready to be sampled for other surveys.

As of May 2010, all of the active KN panel households initially provided a MSN® TV unit had been transitioned to taking KN panel surveys on a laptop computer provided by Knowledge Networks.

Panel Survey Sampling

Once panel members are profiled, they become "active" for selection for specific surveys. Samples are drawn from among active members using a probability proportional to size (PPS) weighted sampling approach. Customized stratified random sampling based on profile data is also conducted, as required by specific studies.

In September 2007, KN was assigned a patent (U.S. Patent No. 7,269,570) for its unique methodology for selecting multiple online survey samples from a panel. The selection methodology, which has been used by KN since 2000, assures that multiple sequential KN panel samples from a finite panel membership will each reliably represent the U.S. population.

This sampling methodology was developed by KN in recognition of the practical issue that different survey samples may target different panel subpopulations. It is not unusual that only panel members with certain characteristics are selected for a survey. This selectivity can skew the remaining panel membership demographics and affect the representativeness of later survey samples. The patented sampling methodology was developed to correct for this in panel sampling; see U.S. Patent No. 7,269,570 for more information.

Survey Frequency & Burden

To minimize panel attrition, surveys are usually kept short (from 5 to 20 minutes in length). For surveys requiring 16 or more survey minutes, survey participation is rewarded with a variety of incentives (small cash awards, gift prizes, raffle opportunities).

Further, steps are taken to ensure that panel members are not overburdened with survey requests. The primary sampling rule is to assign no more than one survey per week to members. This level of survey frequency helps to keep panelists engaged as part of the panel. On average, most KN panelists participate in about two surveys a month. This is closer to four per month for panel segments that may be in higher demand depending on the projects being fielded.

KN operates a Panel Relations program to encourage participation and member loyalty. Members can enter special raffles or be entered into special sweepstakes with both cash and other prizes to be won.

Response Rates

As a member of AAPOR, KN follows the AAPOR standards for response rate reporting. However, the AAPOR standards were not established for web panels. KN survey scientists, Mario Callegaro and Charles DiSogra, recently published an article in Public Opinion Quarterly to provide a standard for web panel response rate calculations. See Callegaro and DiSogra (2008) for examples of response rates calculated for KnowledgePanel surveys and for details on the formulae used.

Statistical Weighting

KnowledgePanel® sample begins as an equal probability sample that is self-weighting with several enhancements incorporated to improve efficiency. Since any alteration in the selection process is a deviation from a pure equal probability sample design, statistical weighting adjustments are made to the data to offset known selection deviations. These adjustments are incorporated in the sample's base weight.

There are also several sources of survey error that are an inherent part of any survey process, such as non-coverage and non-response due to panel recruitment methods and to inevitable panel attrition. These sources of sampling and non-sampling error are addressed using a panel demographic post-stratification weight as an additional adjustment.

However, prior to this adjustment, a separate sample of Spanish-speaking Latino panel members are weighted so as to be merged into the overall panel. This language-specific group is recruited through a geographically targeted dual frame sample that is screened for Spanish-language dominant households. The weighting of this unique sample involves a Spanish-language base weight that incorporates several adjustments including ones that address geographic frame and language proficiency. The panel demographic post-stratification weight is then calculated for all panel members and proportionally adjusts for the merged Spanish-speakers.

References

Callegaro, Mario & DiSogra, Charles (2008). Computing Response Metrics for Online Panels. Public Opinion Quarterly. 72(5) pp. 1008-1031.

Other methodological papers related to KnowledgePanel are available at http://www.knowledgenetworks.com/ganp/reviewer-info.html.