KNOWLEDGEPANEL
DESIGN SUMMARY
®
This document was prepared at the effort and expense of GfK. No part of it may be circulated, copied, or reproduced for distribution without prior written consent.
© GfK 2013
KnowledgePanel® Overview
KnowledgePanel® is the largest national, probability-based panel that provides the highest level of
accuracy and sample representativeness available in online research for measurement of public opinion,
attitudes, and behaviors. The panel was first developed in 1999 by Knowledge Networks, a GfK company,
with panel members who are randomly selected, enabling results from the panel to statistically represent
the US population with a consistently higher degree of accuracy than results obtainable from volunteer
opt-in panels (for comparisons of results from probability versus non-probability methods, see Yeager et
al., 2011).
Because every sample unit has a known selection probability, KnowledgePanel is not susceptible to the
“professional respondent” problem and other hazards of “opt-in” online panels based on convenience
sampling. Unlike opt-in panels, individuals can become panelists only after being randomly selected; no
one can just volunteer to be a member. Originally, panelists were selected using random-digit dialing
(RDD) but address-based sampling (ABS) has been employed since 2009. ABS provides a statisticallyvalid sampling method with a published sample frame of residential addresses that covers approximately
97% of U.S. households, including households that:
•
•
•
•
•
Have unlisted telephone numbers
Do not have landline telephones
Are cell phone only
Do not have current Internet access
Do not have devices to access the Internet
Non-Internet households that are selected in the sample are provided a web-enabled computer and free
Internet service so they can also participate as online panel members. GfK’s KnowledgePanel recruitment
methodology uses the same or similar quality standards as mandated by the Office of Management and
Budget in the "List of Standards for Statistical Surveys," which indicates that “Agencies must develop a
survey design, including… selecting samples using generally accepted statistical methods (e.g., probabilistic
methods that can provide estimates of sampling error).”
KnowledgePanel consists of about 55,000 adult members (ages 18 and older). The highly diverse Hispanic
population is represented in KnowledgePanel with members recruited in both English and Spanish,
thereby representing different levels of language proficiency and acculturation. In addition to adult
members, KnowledgePanel also includes approximately 3,000 teens (ages 13 to 17) whose parents or
legal guardians, usually themselves panel members, have provided their consent.
The scientific underpinnings of KnowledgePanel make it an excellent sampling platform for a broad range
of economic, health, political, and social studies, as well as for policy and program evaluations. This
versatility is reflected in the list of leading foundations, universities, government agencies, non-profit
organizations, and prominent companies that rely on KnowledgePanel for their research projects.
Consistent with our approach to sampling, the AAPOR Online Task Force recently recommended that
researchers use probability-based sampling when measuring population characteristics using online
surveys.1
Address-Based Sample (ABS) Recruitment Methodology
When KnowledgePanel members were first recruited in 1999 using RDD, all households received WebTVs
for answering surveys. By 2002, panelists with their own computers connected to the Internet could
1
The Online Task Force Report, 2010, of the American Association for Public Opinion Research, at www.aapor.org.
This document was prepared at the effort and expense of GfK. No part of it may be circulated, copied, or reproduced for distribution without the prior written consent
of GfK. 2100 Geng Road, Suite 210, Palo Alto, CA 94303
© GfK 2013
2
complete surveys without a WebTV and, beginning in 2009, non-Internet households were provided
Windows-based laptop computers and other web-enabled computers.
ABS was used initially in 2009 to supplement the RDD frame and subsequently replaced it. This change
occurred in response to declining RDD response rates and the growing number of cell phone-only
households – particularly among young adults and minorities – that are often excluded from the
traditional, RDD, landline telephone frame.
ABS allows probability-based sampling of addresses from the U.S. Postal Service’s Delivery Sequence File.
Individuals residing at randomly sampled addresses are invited to join KnowledgePanel through a series of
mailings (in English and Spanish); non-responders are phoned when a telephone number can be matched
to the sampled address. Household members who were randomly-selected can indicate their willingness to
join the panel by returning a completed acceptance form in a postage-paid envelope, calling a toll-free
hotline and speaking to a bilingual recruitment agent, or accessing a dedicated recruitment website.
ABS, conducted throughout the year, is done without replacement. Addresses with matched telephone
numbers from the former RDD recruitment samples (for the last five years of calling) are also removed to
eliminate duplication.
KnowledgePanel LatinoSM
KnowledgePanel LatinoSM is another GfK innovation. This panel provides researchers with the most
statistically representative panel of U.S. Latino households in existence, with about 9,500 Latino adults (of
whom over 4,200 are Spanish-language participants).
This panel includes unassimilated Latinos and Spanish-language dominant households as well as those
without Internet access at home before joining the panel. GfK provides web-enabled computers and
Internet service for the roughly 40% of Latinos without web access from home. KnowledgePanel
LatinoSM enhances our panel that already includes English-speaking Latino households.
The sample for KnowledgePanel Latino is selected using a dual-frame design. The main sample is
recruited by mail using English and Spanish materials. The ABS sample is supplemented with a smaller
RDD telephone recruitment that specifically targets high density Hispanic areas across the country to
exclusively recruit additional Spanish-dominant households.
Panel Survey Sampling
For all new panel members, demographic information such as gender, age, race/ethnicity, income,
education and, for Latino members, language proficiency are collected using an online “profile” survey.
This information is used to determine eligibility for specific studies and eliminates the need for gathering
basic demographic information in each panel survey. After this survey is completed, the panel member is
regarded as “active” and ready to be sampled for other surveys. Additionally, all Hispanic panel members
are asked a series of questions that can be used by clients to apply to an acculturation scale. Such
questions include media use, country of birth, years in the U.S. and other attitude and values questions.
Samples are drawn from among active members using a probability proportional to size (PPS) weighted
sampling approach. Customized stratified random sampling based on profile data is also conducted, as
required by specific studies.
In other panels, it is not unusual that only panel members with certain characteristics are selected for a
survey. This selectivity in sampling can skew the remaining panel membership demographics and affect
This document was prepared at the effort and expense of GfK. No part of it may be circulated, copied, or reproduced for distribution without the prior written consent
of GfK. 2100 Geng Road, Suite 210, Palo Alto, CA 94303
© GfK 2013
3
the representativeness of samples that are selected for later surveys. In September 2007, GfK received a
patent (U.S. Patent No. 7,269,570) for its unique methodology for selecting multiple online survey
samples from a panel to ensure representativeness for second and subsequent samples within the same
sampling period (most participants are eligible for only one survey per week). This sampling methodology
was developed by Knowledge Networks in recognition of the practical issue that different survey samples
may target different panel subpopulations. The KnowledgePanel selection methodology, which has been
used by GfK since 2000, corrects for in-panel sampling and assures that multiple sequential
KnowledgePanel samples from a finite panel membership can each reliably represent the U.S. population.
Survey Frequency & Burden
To minimize panel attrition, surveys are usually kept short (from 5 to 20 minutes in length). For surveys
requiring 16 or more survey minutes, survey participation is rewarded with a variety of incentives
(depending on length and difficulty, we most commonly offer points that respondents can accumulate that
can be redeemed for cash awards, gift prizes, or sweepstakes opportunities).
Further, steps are taken to ensure that panel members are not overburdened with many survey requests.
The primary sampling rule is to assign only one survey per week to members. This level of survey
frequency helps keep panelists engaged. On average, most GfK panelists participate in about two surveys
a month. Some panelists may be in higher demand, depending on the projects being fielded.
GfK operates a Panel Relations program to encourage participation and continuing member loyalty
independent of survey completion. In this program, members can enter special raffles or be entered into
special sweepstakes with both cash and other prizes to be awarded.
Response Rates
As a member of the American Association of Public Opinion Researchers (AAPOR), GfK follows the AAPOR
standards for response rate reporting. However, the AAPOR standards were established for single survey
events and not for Web or other longitudinal panels. Callegaro and DiSogra (2008) developed examples of
response rates calculated for KnowledgePanel surveys and detailed a number of response formulae. In
addition, they developed the concept of “completion rate” applicable to Web panels. The completion rate
for KnowledgePanel is 65% with some variation depending on survey length, topic, and other fielding
characteristics. In contrast, non-probability, opt-in, online panels typically achieve a survey completion
rate in the 2% to 16% range.
As a result of these standard metrics, panels can be more adequately compared (total number of panel
members alone is an incomplete measure of a panel’s “scalability”). The effective panel size also depends
on this completion rate. For example, GfK’s KnowledgePanel requires only about 1,850 panelists to obtain
1,200 completed surveys, whereas a non-probability, opt-in sample may require a starting sample of
50,000 invitations or more to obtain 1,200 completed surveys. Thus, KnowledgePanel’s size of
approximately 55,000 can be comparable to an effective opt-in panel size of 1.7 million (assuming a 2%
completion response rate for the opt-in panel).
Statistical Weighting
To correct for any biases in sampling or non-response, we compute sample weights in several stages. The
design for KnowledgePanel® recruitment begins as an equal probability sample with several
enhancements incorporated to improve efficiency. Since any alteration in the selection process is a
This document was prepared at the effort and expense of GfK. No part of it may be circulated, copied, or reproduced for distribution without the prior written consent
of GfK. 2100 Geng Road, Suite 210, Palo Alto, CA 94303
© GfK 2013
4
deviation from a pure equal probability sample design, statistical weighting adjustments are made to the
data to offset known selection deviations. These adjustments are incorporated in the sample’s base
weight.
There are also several sources of survey error that are an inherent part of any survey process, such as
non-coverage and non-response due to panel recruitment methods and to inevitable panel attrition. We
address these sources of sampling and non-sampling error by using a panel demographic poststratification weight as an additional adjustment based on demographic distributions from the most
recent data from the Current Population Survey (CPS). This weighting adjustment is applied prior to the
selection of any client sample from KnowledgePanel, and these weights are used in the stratified,
weighted, selection procedure for drawing samples from the panel.
All the above weighting is done before the study sample is drawn. Once a study sample is finalized (all
data collected and a final data set made), a set of study-specific post-stratification weights are
constructed so that the study data can be adjusted for the study’s sample design and for survey nonresponse. Starting with each panel member’s base weight, an iterative raking procedure is used to achieve
an optimal approximation of the relevant benchmarks to make survey respondents representative.
References
Callegaro, Mario & Charles DiSogra. 2011. Computing Response Metrics for Online Panels. Public Opinion
Quarterly 72: 1008-1031.
Yeager, David S., Jon A Krosnick, LinChiat Chang, Harold S Javitz, Matthew S Levendusky, Alberto Simpser,
Rui Wang. 2011. “Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys
Conducted with Probability and Non-probability Samples.” Public Opinion Quarterly 75:709-747.
Other methodological papers related to KnowledgePanel are available at
http://www.knowledgenetworks.com/ganp/reviewer-info.html.
This document was prepared at the effort and expense of GfK. No part of it may be circulated, copied, or reproduced for distribution without the prior written consent
of GfK. 2100 Geng Road, Suite 210, Palo Alto, CA 94303
© GfK 2013
5