Marketing Research 340 Descriptive Research Questionnaire Construction NOTE: PLEASE DOWNLOAD AND ALSO BRING TO CLASS THE FILES: QUESTIONNAIRE.DOC AND SURVEY.DOC COPYRIGHT TERRY L. Descriptive Research OBJECTIVE To Make Inferences to a Population PURPOSES Assess Market size Identify Market Segments Diagnose Current Situation (situational analysis) TYPES Longitudinal - Panels Cross Sectional - Surveys COPYRIGHT TERRY L. Types of Panels Same questions Different questions COPYRIGHT TERRY L. Same people Different people True panel Cohort panel Omnibus Crosspanel sectional survey Commercial Consumer Panels COMPANY NFO HOUSEHOLDS 475,000 1.2 million people MARKET FACTS 500,000 STRATIFIED SAMPLES GEOGRAPHIC REGION MARKET SIZE AGE OF HOUSEHOLD HEAD ANNUAL HOUSEHOLD INCOME SIZE OF HOUSEHOLD COPYRIGHT TERRY L. Profile of NFO PRESCREENED HOUSEHOLDS Demographics 475,000 Dog Owners 183,029 Cat Owners 153,608 Cellular Phone Users 85.036 Denture Wearers103,640 RV Owners 33,913 Health Club Members 64,184 COPYRIGHT TERRY L. Profile of NFO CUSTOM SCREENING Up to 250,000 Hholds a Month Buy by the Question COPYRIGHT TERRY L. NFO European Panels COUNTRY COPYRIGHT TERRY L. COUNTRY FRANCE 40,000 GERMANY 50,000 U. K. 50,000 SPAIN 15,000 TOTAL 180,000 NFO European Panels SAMPLES BALANCED BY: GEOGRAPHIC REGION MARKET SIZE AGE OF HHOLD HEAD HHOLD SIZE HOUSEHOLD INCOME/OCCUPATION PURCHASE SAMPLES IN INCREMENTS OF 5,000 COST: ONE A5 PAGE COPYRIGHT TERRY L. $8,400 (ABOUT 6 QUESTIONS) Profile of Electronic Scanning Panels STORE PANEL NIELSEN SCANTRAK IRI INFOSCAN STORES 3300 GROCERY 500 DRUG 600 MASS MERC 100 CLUB 50 MAJOR MKTS 2700 GROCERY 530 DRUG 250 MASS MERC EXPERIMENTAL 64 MAJOR MKTS COVERAGE DATA DOLLAR AND UNIT SALES PRICE PAID COUPON, DISPLAYS, NEWSPAPER ADS HOUSEHOLD PANELS SIZE COVERAGE 40.000 HHOLDS 50 MAJOR MKTS STORES ALL OUTLETS METHOD HAND HELD WAND AT HOME COPYRIGHT TERRY L. 60,000 HHOLDS 10 MAJOR MKTS 8 NON-METRO MKTS GROCERY DRUG MASS MERC CARD AT STORE Profile of Electronic Scanning Panels NIELSEN SCANTRAK HOUSEHOLD PANELS 'SINGLE SOURCE" SIZE COVERAGE MARKETS IRI INFOSCAN 24,000 HHOLDS 8 NON-METRO PITTSFIELD, MA MARION, IN EAU CLAIRE, WI MIDLAND, TX GRAND JUNCTION, CO CEDAR RAPIDS, IA DATA COPYRIGHT TERRY L. DOLLAR AND UNIT SALES PRICE PAID COUPON, DISPLAYS, NEWSPAPER ADS TELEVISION VIEWING MAGAZINE READERSHIP DEMOGRAPHICS WHAT IF????? YOUR OUTLETS AREN’T SERVED BY ELECTRONIC PANELS CLOTHING FINANCIAL SERVICES TELECOMMUNICATIONS INSURANCE INDUSTRIAL INSTRUMENTS HEATING AND AIR CONDITIONING AGRICULTURAL FEEDS ENTERTAINMENT HOTELS INDUSTRIAL PAINT PUMPS CONSTRUCTION EQUIPMENT COPYRIGHT TERRY L. USERS OF INTERNAL DATABASES - PANELS BLOCKBUSTER ENTERTAINMENT HARLEY DAVIDSON HOUSE OF SEAGRAM GENERAL MOTORS PHILIP MORRIS COPYRIGHT TERRY L. Descriptive Research Using Cross Sectional Methods MAIL SURVEYS TELEPHONE INTERVIEWS PERSONAL INTERVIEWS COPYRIGHT TERRY L. Framework for Constructing a Questionnaire Specify What Information Will be Sought Identify Correct Respondent Determine Method of Administration Question Structure Question Wording PRETEST and Revise Questionnaire COPYRIGHT TERRY L. Question Sequencing Specify What Information Will be Sought Identify Correct Respondent Determine Method of Administration Question Structure Question Wording PRETEST and Revise Questionnaire COPYRIGHT TERRY L. Question Sequencing Identify Correct Respondent Key Informant Vs. Multiple Informant Methodology COPYRIGHT TERRY L. Key Informant Vs. Multiple Informants Consumer versus Business to Business Decision making process & roles What are the roles? Who occupies those roles? Autonomy of the roles? Consistency of the roles? COPYRIGHT TERRY L. TYPICAL ROLES Intitator Influencer Decider Purchaser User Key Informant Vs. Multiple Informants Consumer versus Business to Business Decision making process & roles What are the roles? Who occupies those roles? Autonomy of the roles? Consistency of the roles? COPYRIGHT TERRY L. TYPICAL ROLES Intitator Influencer Decider Purchaser User INSURANCE FFO JOINT JOINT FFO JOINT Specify What Information Will be Sought Identify Correct Respondent Determine Method of Administration Question Structure Question Wording PRETEST and Revise Questionnaire COPYRIGHT TERRY L. Question Sequencing Participation in Surveys 73% 80 Percent 70 60 P ercent ever pa rticip ate d 50 P ercent pa rticip ated in the pa st year 1982 1984 1986 1988 Year of Survey SOURCE: Walker Images Surveys COPYRIGHT TERRY L. 42% 44% 40 30 20 10 0 1980 76% 1990 1992 1997 Participation By Survey Method Telephone Mail 80 Percentage 70 60 74 70 65 69 68 70 64 46% 38% 34 9% 10 2% 56 54 45 38 35 33 30 20 Door-to-Door 76 75 50 40 Shopping Mall 18 10 0 1980 30 14 1982 11 1984 11 1986 1988 Year of Survey COPYRIGHT TERRY L. 32 15 1990 1992 1997 Methodologies Preferred by Survey Respondents Mail Telephone Stores/Shopping Center Group 60% 58 % 56 % 50% Percentage 40% 30% 20% 10% 13% 20 % 18 % 8% 11 % 5% 9% 7% 6% 0% 1992 Year of Surve y 1990 COPYRIGHT TERRY L. Home European Data Collection Methods METHOD OF ADMINISTRATION FRANCE COUNTRY NETHERLANDS SWEDEN SWITZERLAND U.K. MAIL 4% 33% 23% 8% 9% TELEPHONE 15% 18% 44% 21% 16% CENTRAL LOCATION/ STREETS 52% 37% --- --- --- HOME/WORK ----- ----- 8% 44% 54% GROUPS 13% ----- 5% 6% 11% DEPTH INTERVIEWS 12% 12% 2% 8% ----- SECONDARY 4% ----- 4% 8% ----- SOURCE: "ESOMAR URGES CHANGES IN REPORTING DEMOGRAPHICS, ISSUES WORLDWIDE REPORT", MARKETING NEWS, JAN. 8, 1990, 24. COPYRIGHT TERRY L. Determining the Appropriate Method of Administration Control Over Administration Sampling control Directing the survey Eliciting a response COPYRIGHT TERRY L. Determining the Appropriate Method of Administration Control over administration Information control Sequencing Length Complexity - Questions and Answers COPYRIGHT TERRY L. Determining the Appropriate Method of Administration Control over administration Administrative control Speed Cost Availability of Trained Personnel COPYRIGHT TERRY L. Comparison of Methods of Administration 44 45 44 40 36 35 30 24 25 22 20 16 15 15 13 11 10 9 10 4 5 0 SC IC AC T OT AL PERSONALINTERVIEWS PERSONAL INTERVIEWS COPYRIGHT TERRY L. SC IC AC T OT AL MAIL SURVEYS MAIL SURVEYS SC IC AC T OT AL TELEPHONE TELEPHONE INTERVIEWS INTERVIEWS Summary Comparison of Methods of Administration 44 45 44 USE 40 CAPABILITY Telephone 36 Personal 35 Mail Telephone Personal Mail 30 24 25 22 20 16 15 15 13 11 10 9 10 4 5 0 SC IC AC T OT AL PERSONALINTERVIEWS PERSONAL INTERVIEWS COPYRIGHT TERRY L. SC IC AC T OT AL MAIL SURVEYS MAIL SURVEYS SC IC AC T OT AL TELEPHONE TELEPHONE INTERVIEWS INTERVIEWS GROWTH IN INTERACTIVE TECHNOLOGY FOR OBTAINING BETTER CUSTOMER FEEDBACK INTERACTIVE KIOSKS DISK/CD ROM SURVEYS FAX SURVEYS E MAIL SURVEYS INTERNET SURVEYS INTERACTIVE TV COPYRIGHT TERRY L. TECHNOLOGY ASSISTED SELF ADMINISTRATION (TASA) TASA SURVEYS HIGHER ENTERTAINMENT VALUE NOVEL - LESS BORING INCREASE PARTICIPATION up to 1.5 hours higher levels of participation better quality answers perceptions of time to complete are lower Source: MacElroy and Geissler October, (1994), Marketing News COPYRIGHT TERRY L. INTERNET SURVEYS ON-LINE USERS OPEN TO SURVEYS WILLING TO PARTICIPATE IN FUTURE SURVEYS ? .7 64% percentage .6 .5 .4 32% .3 .2 TELEPHONE ON-LINE METHOD OF INTERVIEWING COPYRIGHT TERRY L. Source: “Stay Plugged in to New Opportunities”, J. Palmquist and A. Stueve, Marketing Research, Vol. 18(1), 1996, 13-15. INTERACTIVE KIOSKS MULTIMEDIA KIOSK COPYRIGHT TERRY L. INTERACTIVE KIOSKS MULTIMEDIA KIOSKS IN RESEARCH BENEFITS STORE LAYOUT TRACKING CAPTURE EMOTIONS WHILE FRESH REDUCE DISTORTIONS IN RECALL INTERVIEW BY BRANCH OR LOCATION INTERVIEW EFFICIENTLY WHEN LOW INCIDENCE POPULATION REDUCE BIAS FROM FACE TO FACE CONTACT COPYRIGHT TERRY L. source: Schneider 1995, Marketing News article and author DBM/CD ROM DISK/CD ROM IN THE MAIL BENEFITS OF USING MORE CONTROL OVER SKIPPING QUESTIONS PRESENTATION OF 1 QUESTION AT A TIME PRECLUDE CHANGING ANSWERS TO PRIOR QUESTIONS COLLECTION OF LATENT INFORMATION PREPROGRAMMING COMPLICATED BRANCHING INSTRUCTIONS BETTER ANSWERS TO SENSITIVE QUESTIONS COPYRIGHT TERRY L. FAX SURVEYS FAX SURVEYS 15 % OF US HHOLDS OWN FAXES - PROJECTED TO BE 25% BY 2000 85% OF BUSINESSES WITH 5 OR MORE EMPLOYEES OWN FAXES CONSIDERATIONS ADVANTAGES FASTER DISTRIBUTION - DOMESTIC AND INTERNATIONAL POTENTIAL LOWER COSTS COMMUNICATES A SENSE OF URGENCY IMPORTANCE NOVELTY DISADVANTAGES LIMITED COVERAGE LOSS OF ANONYMITY LOSS OF GRAPHICS LOSS OF INCLUDED INCENTIVES HARDER TO PROVIDE PREPAID RETURN MECHANISM source: “Fax Surveys: Return Patterns and Comparison with Mail Surveys”, J. Dickson and D. MacLachlan, Journal of Marketing Research, February,1996, 108-113.. COPYRIGHT TERRY L. FAX SURVEYS FAX SURVEYS TWO STUDIES 450 BUSINESS SCHOOL DEANS 346 CEOs OF FORTUNE 500 COMPANIES RESULTS FAX SURVEY RETURNED FASTER 3.1 DAYS FASTER IN DEAN SURVEY 3.9 DAYS IN CEO SURVEY FAX SURVEY GENERATED HIGHER RETURN RATE DEAN STUDY CEO STUDY FAX SURVEY 50.2% 42% MAIL SURVEY 42.9% 32.% NO DIFFERENCE IN ANSWERS (BIAS) COPYRIGHT TERRY L. source: “Fax Surveys: Return Patterns and Comparison with Mail Surveys”, J. Dickson and D. MacLachlan, Journal of Marketing Research, February,1996, 108-113.. Benefits in the Use of the Internet/Web for Marketing Research Surveys Access 24 hours a day/7days a week Interactivity Customization by Customer/Respondent Multi Media Capabilities Text Graphics Sound Full Motion Video Instantaneous Feedback Domestic and International INTERNET SURVEYS COPYRIGHT TERRY L. INTERNET SURVEYS EXAMPLE SURVEYS INTERACTIVITY COPYRIGHT TERRY L. INTERNET SURVEYS CUSTOMIZED INTERNATIONALLY COPYRIGHT TERRY L. COPYRIGHT TERRY L. COPYRIGHT TERRY L. COPYRIGHT TERRY L. COPYRIGHT TERRY L. COPYRIGHT TERRY L. INTERNET SURVEYS ISSUES WITH INTERNET RESEARCH PARTICIPANTS ARE VOLUNTARY NO PROJECTABILITY OF RESULTS COPYRIGHT TERRY L. INTERNET SURVEYS ISSUES WITH INTERNET RESEARCH PARTICIPANTS ARE VOLUNTARY NO PROJECTABILITY OF RESULTS SAMPLE COMPOSITION BIASES COPYRIGHT TERRY L. INTERNET SURVEYS ISSUES WITH INTERNET RESEARCH PARTICIPANTS ARE VOLUNTARY NO PROJECTABILITY OF RESULTS SAMPLE COMPOSITION BIASES TWO STRATEGIES 1. FEEDBACK OR MONITORING MECHANISM 2. USE AN ELECTRONIC INTERNET PANEL FED-X SHIPPING PANEL COMMERCIAL INTERNET PANEL COPYRIGHT TERRY L. INTERNET SURVEYS COMMERCIAL INTERNET PANEL NFO//net.source 70,000 HOUSEHOLDS 175,000 INTERACTIVE CONSUMERS COPYRIGHT TERRY L. Specify What Information Will be Sought Identify Correct Respondent Determine Method of Administration Question Structure Question Wording PRETEST and Revise Questionnaire COPYRIGHT TERRY L. Question Sequencing Question Structure Open-end questions Closed-end questions Unordered response categories Ordered response categories Partially closed-end questions COPYRIGHT TERRY L. Question Structure Open-end questions If you were to purchase a new automobile during 2001, what brand of automobile would you most likely purchase? ___________ brand Closed-ended with ordered categories If you were to purchase a new automobile during 2001, how likely is it that you would purchase the following brands of automobiles? Very Likely Somewhat Likely Somewhat Unlikely Very Unlikely Toyota Camry 1 2 3 4 Honda Accord 1 2 3 4 Nissan Maxima 1 2 3 4 Closed-ended with unordered categories If you were to purchase a new automobile during 2001, which one of the following automobiles would you most likely purchase? Toyota Camry o Honda Accord o Nissan Maxima o Partially closed-ended If you were to purchase a new automobile during 2001, which one of the following would you most likely purchase? Toyota Camry o Honda Accord o Nissan Maxima o Other (please specify)______________ COPYRIGHT TERRY L. EXAMPLE OF STRUCTURED QUESTION Vs. OPEN END Considering your children, which of the following values do you feel are most important for your children to possess as adults? (select only 3) Three Most Important Values 1. Manners 2. Success 3. Honesty 4. Cleanliness 5. Good Judgment 6. Control 7. Spirtulaity 8. Amicable 9. Obedience 10. Responsible 11. Considerate 12. Confidence 13. Studious COPYRIGHT TERRY L. [ [ [ [ [ [ [ [ [ [ [ [ [ ] ] ] ] ] ] ] ] ] ] ] ] ] Structured Questions: Effect on Respondent Answers Response order effects Visual presentation of response alternatives Primacy effects initial items establish a cognitive framework to guide processing initial items get deeper cognitive processing & less interference than later items satisficing — minimize psychological costs by choosing first acceptable response alternative SOURCE: Krosnick and Alwin, POQ COPYRIGHT TERRY L. Structured Questions: Effect on Respondent Answers Response order effects Oral presentation of response alternatives Recency effects externally based processing each read alternative terminates processing of previous alternative increased processing of later response alternatives COPYRIGHT TERRY L. Selected Results Three Most Important Qualities CONTEXT: Personal Interviews of 1473 Parents on 13 Values for their Children using “Show Cards” Value STANDARD REVISED DIFFERENCE MANNERS 26.4% 10.1% 16.3% SUCCESS 19.1% 14.6% 4.5% HONESTY 65.7% 48.4% 17.3% THREE LEAST IMPORTANT VALUES NOT AFFECTED WHEN ASKED AS A FOLLOWUP QUESTION SOURCE: Krosnick abnd alwin, POQ COPYRIGHT TERRY L. Individual Differences in Cognitive Sophistication Low — high school education or less & low score on verbal intelligence test percent of responses affected by response-order low sophistication 54% (7/13) high sophistication 8% (1/13) IMPLICATION: Surveying Consumers vs. Business Respondents COPYRIGHT TERRY L. Reducing Response-Order Effects Randomize Order of Presentation Between Respondents Telephone - CRT-based Interviewing Still Present for Individuals - Segmentation Increase Respondent Motivation Special Instructions (Please read all options before.....) Eliminates Effects for Individuals COPYRIGHT TERRY L. Specify What Information Will be Sought Identify Correct Respondent Determine Method of Administration Question Structure Question Wording PRETEST and Revise Questionnaire COPYRIGHT TERRY L. Question Sequencing Question Sequence and Order Effects Question sequence 1 Yes Do you think a communist country like Russia should let American newspaper reporters come in and send back to their papers the news as they see it? 90% Do you think the United States should let communist newspaper reporters from other countries come in and send back to their papers the news as they see it? Once Committed Hard to be Unfair 73% Question sequence 2 Yes Do you think the United States should let communist newspaper reporters from other countries come in and send back to their papers the news as they see it? 36% Do you think a communist country like Russia should let American newspaper reporters come in and send back to their papers the news as they see it? SOURCE: Schuman and Ludwig, ASR. COPYRIGHT TERRY L. 66% 37% Difference ! General Questionnaire Sequence Start with simple more general questions nonthreatening interesting general respondent capability Respondents will be looking for an Excuse to Terminate or Stop Interview COPYRIGHT TERRY L. General Questionnaire Sequence Funnel respondent from general to specific questions throughout the questionnaire What types of beverages do you drink on a weekly basis Product Category Which of the following beverages is your favorite How many times per week do you drink this beverage Which brands of soft drinks can you list below Rate the following four brands along the set of attributes COPYRIGHT TERRY L. Brand Specific General Questionnaire Sequence Put sensitive/objectionable questions at the end overall topics demographics AFTER YOU HAVE: - created an investment by the respondent - built up your credibility COPYRIGHT TERRY L. Question Sequencing Danger: Question Order Effects Overall Product Evaluation Question 1. Overall Lysol is the best brand for Disenfecting hard surfaces in my operatory STRONGLY DISAGREE 1 2 STRONGLY AGREE 3 4 5 Specifc Product Attribute Questions 1. Lysol is very effective in killing germs in my operatory 2. Lysol requires a very short contact time to kill the germs in my operatory 3. Lysol does not corrode the metal in my operatory COPYRIGHT TERRY L. 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Carryover and Backfire Effects in Questionnaires Carryover Respondents give answers that are consistent with beliefs associated with a prior response Backfire Respondents give answers that are inconsistent with beliefs associated with a prior response SOURCE: Bickert, JMR, 1993. COPYRIGHT TERRY L. Question Sequencing and Carryover Effects in Questionnaires Carryover Specific Question Overall Question COPYRIGHT TERRY L. Overall Question Higher Overall Rating Specific Question Question Sequencing and Backfire Effects in Questionnaires Backfire Specific Question Overall Question COPYRIGHT TERRY L. Overall Question Lower Overall Rating Specific Question Implications for Questionnaire Design Screen respondents on prior knowledge Used a 7-pt. scale (7 - “very knowledgeable”) For high-knowledge individuals (Business Survey) Previous attribute ratings do affect subsequent evaluations Present 1. overall question first (brand evaluation) 2. then specific questions (attribute evaluations) COPYRIGHT TERRY L. Implications for Questionnaire Design/continued For low and moderate knowledge individuals Consumer Survey Present 1. specific questions first (attribute evaluations) 2. then overall question (brand evaluation) COPYRIGHT TERRY L. Specify What Information Will be Sought Identify Correct Respondent Determine Method of Administration Question Structure Question Wording PRETEST and Revise Questionnaire COPYRIGHT TERRY L. Question Sequencing Asking Frequency of Product Use Questions Who are your Heavy Users? Two Choices in Wording Frequency How many Times ............. How Often ............. Which One ?? Time Frame Adjust time frame for purchase frequency More frequently purchased products need shorter time frames (1 week versus 1 Month) COPYRIGHT TERRY L. ANALYSIS OF CLASS QUESTIONNAIRE COPYRIGHT TERRY L. Framework for Constructing a Questionnnaire Specify What Information Will be Sought Identify Correct Respondent Determine Method of Administration Question Structure Question Wording PRETEST and Revise Questionnaire COPYRIGHT TERRY L. Question Sequencing Checklist for Question Wording vague - questions and answers bias or leading objectionable/embarrassing too demanding check for double questions do they have the ability to answer give frame of reference consider ability to compare responses across past studies COPYRIGHT TERRY L. Didn’t Pretest 100% or ?? 100% COPYRIGHT TERRY L. Questionnaire Pretesting PRETESTING METHODS Qualitative assessment Method of debriefing COPYRIGHT TERRY L. Questionnaire Pretesting ASSESS THE FOLLOWING: Question Comprehension Questionnaire Instructions To interviewer selection of respondent interaction with respondent To respondent who should answer questionnaire cover letter appropriate branching patterns and question directions COPYRIGHT TERRY L. INSTRUCTIONS FOR TELEPHONE INTERVIEWER COPYRIGHT TERRY L. Components of a Cover Letter 1. If possible personalize the greeting. 2. Explain what study is about and how respondent will benefit. 3. Explain how respondent was selected and their importance to the study. 4. Indicate who should complete the questionnaire. 5. If sensitive topic, then assure confidentiality. If relevant assure that no sales calls will be made. 6. Reemphasize how respondent will benefit from participation. 7. Mention any incentive for participation. 8. If possible, provide a phone number in case of questions. 9. Close with thank you and handwritten signature (blue ink). COPYRIGHT TERRY L. Questionnaire Pretesting PRETESTING METHODS Qualitative assessment debriefing Quantitative assessments pilot study (method of administration) COPYRIGHT TERRY L. Framework for Constructing a Questionnaire Specify What Information Will be Sought Identify Correct Respondent Determine Method of Administration Question Structure Question Wording PRETEST and Revise Questionnaire COPYRIGHT TERRY L. Question Sequencing