2010 AAPOR Presentations
Knowledge Networks adheres to the AAPOR Code of Professional Ethics and Practices.
DOES PROVIDING INTERNET ACCESS TO NON-INTERNET HOUSEHOLDS AFFECT REPORTED MEDIA BEHAVIOR FOR LATINOS AND NON-LATINOS? RESULTS FROM A SIX-MONTH LONGITUDINAL SURVEY
J. Michael Dennis, Knowledge Networks; Email
Charles A. DiSogra, Knowledge Networks; Email
A distinguishing feature of KnowledgePanel® and KnowledgePanel LatinoSM is the inclusion of households that do not have Internet access at the time of recruitment. KN provides these households with laptops, and in the past, an MSN-TV® device, and free monthly Internet access so they can participate in online KnowledgePanel® surveys. Over half of the KnowledgePanel Latino households have been provided a laptop.
Does providing internet access to respondents have an effect on their survey responses? Are the respondents "changed" as a result?
To answer this question, KN launched a longitudinal study of recently recruited KnowledgePanel Latino members measuring any behavioral changes occurring over time with regard to Internet usage, media usage, online shopping, and also measuring comfort with technology. All study subjects were surveyed at 3-month intervals. This presentation will show findings based on three measurement waves for the Internet and non-Internet households. Statistical evidence supports the hypothesis of a limited "novelty effect," that is, panelists on a limited basis increase their usage of some media and visits of websites when first joining the panel, but then return to lower usage levels by month six of their panel tenure.
More Info – PowerPoint
MAXIMIZING A STRATIFIED ABS SAMPLING FRAME FOR NATIONWIDE MAIL RECRUITMENT OF A PROBABILITY-BASED ONLINE PANEL
Charles A. DiSogra, Knowledge Networks; Email
Until recently, the probability-based online KnowledgePanel® run by Knowledge Networks (KN) used exclusively a random-digit dial (RDD) methodology for panel recruiting. RDD-based recruitment has been rapidly losing its technical and cost-effective edge due to the increasing efforts required to achieve acceptable response and coverage rates. More importantly, the shrinking coverage of RDD frames due to the increase of cell phone only households and the related disproportionate loss of sample coverage for young adults and Hispanics has rendered RDD recruitment to becoming increasingly inadequate for achieving representative national samples.
An address-based sample (ABS) frame using mail recruitment has emerged as a solution to coverage and minimizing response bias. In 2008, KN conducted a pilot study to recruit panel members via mail using the USPS Delivery Sequence File for its ABS frame. Two large-sample waves were implemented in 2009, supplementing the RDD-sourced sample. Each ABS wave has incorporated refinements to the mailing strategy based on experiments performed in the pilot in prior large-sample waves. These experiments altered materials, incentives and compressed the length of the field period without loss in response rate or sample representativeness. To enhance diversity, the second wave in 2009 included the targeting of high-density minority communities using a stratified sample. Also implemented was a plan to increase the recruitment of households that do not have Internet access. (KN supplies such households with a laptop computer and Internet access so that they, too, can participate in online surveys.) This presentation reviews data demonstrating the evolving effectiveness of KN's mail-based recruitment strategy as evidenced by successive increases in yield and response rate; in improved sample representativeness; in targeting minority areas; and in the recruitment of non-Internet household using a targeting "message" strategy among non-responders.
More Info – PowerPoint
NON-RESPONSE BIAS: RECENT FINDINGS FROM ADDRESS-BASED PANEL RECRUITMENT
Joe Garrett, Knowledge Networks; Email
J. Michael Dennis, Knowledge Networks; Email
Charles A. DiSogra, Knowledge Networks; Email
Achieving reasonable levels of response to surveys continues to be problematic as response rates have generally declined over the past several years across numerous survey panels and survey modes. While this certainly is not a favorable trend, it is important to remember that the bias in survey estimates due to such nonresponse is a multiplicative function of two components – the nonresponse rate as well as the overall difference between respondents and nonrespondents. Recently, Knowledge Networks (KN) started to replace its RDD-based panel recruitment for KnowledgePanel® with address-based sampling (ABS) using the U.S. Postal Service Delivery Sequence File. This was done to provide better statistical coverage of cell phone only households that now represent about 20% of all U.S. households as well as improve coverage of young adults and minorities. One research effort arising from KN's ABS recruitment strategy was that KN was able to keep a record of those persons who joined the KN panel and those persons who refused to join, and then link the address of the nonresponders to other databases with modeled data for household characteristics as well as county-level data such as 2008 presidential candidate vote counts. This paper presents the results of the various comparisons that were made between recruited and sampled cases. Results of these recruited/non-recruited comparisons across several demographic and socio-economic variables and Obama/McCain vote tallies show householders having a similar propensity to join KnowledgePanel. The findings help reviewers of KnowledgePanel survey data to better understand the potential, however limited, effect on bias due to nonresponse stemming from differences between respondents and nonrespondents at the KN panel recruitment stage.
More Info – PowerPoint | Full Paper
DIVERSITY AND A SENSITIVE SOCIAL ISSUE: HISPANIC CATHOLICS AND THEIR COMPARATIVE RESPONSE TO THE CLERICAL ABUSE CRISIS
Mark Gray, Center for Applied Research in the Apostolate, Georgetown University
Bill McCready , Knowledge Networks; Email
This paper describes the methods and results of a CARA Catholic Poll measuring the attitudes of Catholics toward the clerical sex abuse crisis. A key element in the study is to accurately reflect the Hispanic/Latino Catholic population in the US since an increasing proportion of the US Catholic population is of Hispanic/Latino background. Survey-based measurements of Catholic population need to take into account this rapidly changing demographic in their sample planning. In order to accurately represent the attitudes and opinions of US Hispanics, it is necessary to create a sample that includes the entire language spectrum of that group, as provided in KnowledgePanel Latino, which represents Latinos that speak predominantly English as wells as those that speak predominantly Spanish and all variations in between. In addition, since the topic of this study is public reaction to clerical sexual abuse, a sensitive issue that is theoretically subject to social desirability bias, the project required a methodology less susceptible to that particular bias. The combination of the need for an accurate sample of the US that includes the appropriate proportion of Hispanics and for a methodology that constrains social desirability bias (one that eliminates the human interviewer and the attendant expectations) lead to the choice of administering the survey to a Internet sample representative of the U.S. Latino population. The interviewed sample consists of approximately 2,000 cases with subsamples in three regions and an appropriate sampling of Latino Catholics. The analysis focuses on comparisons of attitudes toward the abuse crisis between English-dominant Hispanic Catholics and Spanish-dominant Hispanic Catholics and the non-Hispanic Catholic majority. There are also inter-regional analyses demonstrating how these attitudes are expressed in different Church locales. The sexual abuse crisis has cost the Catholic Church dearly in terms of both financial resources and moral credibility. This survey provides data about public reaction to the crisis and the Church's response to it.
More Info – PowerPoint
ANALYSIS OF LATE RESPONDERS TO PROBABILITY-BASED WEB PANEL RECRUITMENT AND PANEL SURVEYS
Yelena Kruse, Knowledge Networks; Email
Erlina Hendarwan, Knowledge Networks; Email
J. Michael Dennis, Knowledge Networks; Email
Charles A. DiSogra, Knowledge Networks; Email
Transitioning from a ten-year legacy of RDD sampling, the probability-based online KnowledgePanel® uses an address-based sample (ABS) frame for mail-based recruitment of panel members. This paper analyzes nonresponse through a statistical analysis of "late responders" at two stages of the panel survey process: panel recruitment and the online surveys themselves. The characteristics of "late responders" provide a statistical, however imperfect, window into the characteristics of non-responders at these two stages. Furthermore, any differences between early and late responders would have implications for nonresponse bias in the event the field period is shortened for either the panel recruitment or the online surveys. First, to examine late responders to the ABS-sourced panel recruitment, we compared full demographic and other profile-based information that had been collected among both early and late responders to the mail-based recruitment. After these mail-based recruitment efforts, non-responders are contacted using a telephone-based recruitment whenever a landline phone number can be matched to a residential address. Second, to examine late responders to KnowledgePanel web surveys themselves, we analyzed the samples for three large-sample online surveys. Preliminary research found that late responders to the mail recruitment and panel surveys are disproportionately non-white (p-value<0.05), live in three- or more-person households (p-value<0.1), and younger adults (p-value<0.01). The findings will be useful in designing more effective recruitment and online methods encourage response (such as using a targeted segmentation of messages and incentives) in order to minimize response bias.
More Info – PowerPoint | Full Paper
MEASURING ACCULTURATION IN THE HISPANIC POPULATION
Wendy Mansfield, Knowledge Networks; Email
Jordon Peugh, Knowledge Networks; Email
Jannet Torres, Knowledge Networks; Email
Tom Wells, Knowledge Networks; Email
Failure to sample Hispanics from the full range of the acculturation scale can have an impact on survey results. Hispanics now represent over 15% of the U.S. population. Using KnowledgePanel LatinoSM, an online web panel designed to be representative of the national Latino population, and using responses to 29 agree/disagree statements on lifestyle issues from a 2009 survey, this paper will explore two research questions related to acculturation.
Research Question 1. How much improvement is there in the accuracy of measuring acculturation by moving from single measures of acculturation to a multidimensional measure? Our examination of three single and one multidimensional measure—citizenship status, immigration generation, years living in the U.S., and the Bidimensional Acculturation Scale for Hispanics (BAS)—quantifies the extent to which the BAS scale is superior to the other measures. The BAS scale is substantially more likely to define subgroups of Hispanics for which the more acculturated a subgroup is, the more similar its answers to lifestyle questions is to the answers of non-Hispanics. The lifestyle questions covered work-life balance, leisure, socializing, retirement, the environment, and other topics.
Research Question 2. How do survey results on lifestyles differ when the least acculturated group of Hispanics is excluded from the sample? Preliminary analyses indicate that there is a statistically significantly difference between the proportion of Hispanics and the proportion of non-Hispanic U.S.-born citizens agreeing with the opinion statements for 93% of the statements (across all 4 acculturation variables) when all Hispanic groups are included, but for only 69% of the statements when the least acculturated Hispanic group is excluded.
The paper will present full data and discuss implications.
More Info – PowerPoint | Full Paper
REPRESENTING SENIORS IN AN ONLINE NATIONAL PROBABILITY PANEL SURVEY: MEASURING TECHNOLOGY ATTITUDES AND BEHAVIORS
Jordon Peugh, Knowledge Networks; Email
Wendy Mansfield, Knowledge Networks; Email
Tom Wells, Knowledge Networks; Email
Representation of seniors is often a concern when considering an online panel to conduct research due to their lower access and use of the internet compared to other groups. Using data from KnowledgePanel®, a national probability-based web panel that provides internet access to non-internet households, we will present our analyses on the distinctive technology attitudes and behaviors of seniors compared to other age groups. Moreover, we will also present evidence that seniors provided internet access upon KnowledgePanel recruitment have distinctive technology attitudes and behaviors compared to seniors already having internet access.
Nineteen percent of the adults in KnowledgePanel® are 65 years old and older, comparable to the Census benchmark of 17% for the U.S. population (July 2009 Current Population Survey data). We examined statistical differences across age groups in 15 technology-related attitude statements and identified significant differences between seniors and other age groups on all of these statements. Looking among seniors, we found that those seniors provided internet access upon agreeing to join KnowledgePanel expressed significantly less "pro-technology" sentiment on 12 of the 15 statements compared to those who had prior access. The final paper will present full results which will include an examination of differences across age groups in adoption of personal technologies such as cell phones and online media use.
To measure the impact of panel membership on attitudes and behaviors, we will explore within-person change over time on technology attitudes. KN fielded the same technology attitude items to our panelists beginning in November of 2008 and will again beginning in November of 2009. We expect to have results at both intervals for approximately 4,000 seniors.
More Info – PowerPoint | Full Paper
DIFFERENCES IN LENGTH OF SURVEY ADMINISTRATION BETWEEN SPANISH-LANGUAGE AND ENGLISH-LANGUAGE SURVEY RESPONDENTS
Tom Wells, Knowledge Networks; Email
Monica Vidalon, Knowledge Networks; Email
Charles A. DiSogra, Knowledge Networks; Email
In 2008, Knowledge Networks expanded KnowledgePanel to create KnowledgePanel Latino, the first nationally representative online sample of Latinos in the U.S. We have observed that the Spanish-language panelists take more time to self-administer their Spanish-language online questionnaires than those taking the English version – both Latinos and non-Latinos range 50% to 100% longer. For example, in our 2009 demographic profile survey, median completion length is 8 minutes among roughly 40,000 English-language panelists, while 21 minutes among roughly 2,000 Spanish-language panelists. This pattern holds for different survey topics and has practical implications for respondent burden, respondent incentives and costs.
To our knowledge, this is a new finding, as there are no published research findings on these differences in online surveys.
We will evaluate several possible explanations for this finding: longer Spanish questionnaires in terms of word count; skip patterns, respondent demographics (e.g., education, age, household size), cultural differences, panel tenure, Internet connection speed, and survey-taking behavior.
We employ regression to estimate the effects of survey language, demographics, panel tenure, and type of Internet connection on survey completion time.
Our regression results show that education, age, and broadband internet connection have large effects and significantly reduce survey completion time. However, once several factors have been controlled, Spanish-language survey takers still take, on average, seven minutes longer than their English-language counterparts to complete the main demographic profile survey. Spanish survey language still remains a strong predictor of administration length, which may speak to cultural differences and survey taking behaviors, which we will further investigate in this paper.
More Info – PowerPoint | Full Paper
