Collecting Primary Data: Questionnaires Objectives After this session you will be able to: • Plan and design valid and reliable questionnaires. • Describe the processes involved in collecting primary data, including piloting. • Demonstrate the skills for writing appropriate individual questions and designing questionnaires. • Write appropriate documentation to accompany questionnaires. Types of questionnaire Questionnaire Self-administered Postal questionnaire Online questionnaire Delivery & collection questionnaire Interview-administered Focus group Structured interview Telephone interview Why questionnaires? • Low costs (time and money). • Quick inflow of data. • Can reach many people. • Quick coding of closed questions. • Reliability of data. Questionnaires – disadvantages • Low response rates (threat to reliability). • No opportunity for probing questions. • No opportunity for respondent to seek clarity (threat to validity). • Socially desirable responses (SDRs) (threat to validity). Face validity • Do the questions make sense? • Are all concepts in the questions understandable? • Are the questions interesting and motivating to answer? • Are the instructions for completing the questionnaire clear? • Is the questionnaire of the right length? Achieving face validity • Get your questionnaire reviewed by 15 of your peers. • Get them to write down the problems they find on the questionnaire. • Talk to them. • Note which elements need improving and comment on how you did this in your Methodology. Design: construct validity Construct validity: an instrument measures the constructs it was intended to measure. Theory Concept Constructs Question sets Marketing Relationship Marketing - Loyalty - Internal customers - Supply chain Question sets - Loyalty - Internal customers - Supply chain Measuring attitudes: reliability Reliability: the consistency between two measures – sufficient numbers of questions for each construct. Loyalty: • To me, brand loyalty is important. • I tend to buy the same brand all the time. • If I like a brand I always buy it in preference to others. • Brand loyalty is unimportant in my buying decisions. Reliability of responses Loyalty: • To me, brand loyalty is important. • I tend to buy the same brand all the time. • If I like a brand I always buy it in preference to others. • Brand loyalty is unimportant in my buying decisions. Note: these questions will be randomized – scattered throughout the questionnaire Measurement of reliability • Cronback alpha: at least 0.7. • If below 0.7 you need to eliminate or make changes to low reliability questions. Questionnaire design: closed questions • Seek a predefined response: Yes/No; True/False, etc. • Easy to code and analyse. • Make group comparisons easy. • Low cost, but… • May restrict richness of responses. Questionnaire design: open questions • How, why, what …? • Good for eliciting rich responses, but… • Harder to analyse. • Example: You have indicated a loyalty to Krispy Crisps – please explain why ----------------------------------------- Questions to avoid (threats to validity) • Leading: Why do you like Krispy Crisps? • Double: Do you like Krispy Crisps or Walkers Crisps? YES/NO • Assumptive: How often do you eat Krispy Crisps? • Hypothetical: Suppose you were asked to eat 20 packets of Krispy Crisps…? • Knowledge based: Recalling the recent Krispy Crisp advertising campaign… Good (valid) questions are… • • • • • Simple and easily understood. As short as possible. Unambiguous. Not likely to offend. Provided with categories such as ‘Don’t know’ and ‘Not applicable’. What’s wrong here? • • What’s your view on crisps? State your age Under 20 20 - 25 • • • 25 - 30 Over 30 Without effective ‘best practice’ crisp making cannot prosper. How many packets of crisps have you stolen? COMPLETE AND RETURN THIS FORM Response categories (1) List question What do you think is the most important influence on the success of the organization in the next two years? Please as many responses as you think accurate. Changes in government policy affecting the legal regulation of the market The entry of new competitors to the market The impact of the company’s current reorganization strategy Foreign exchange rates Response categories (2) Category question How often in an average week do you use our e-banking facilities? Please one response. Never Once 2–3 times 4–5 times 6 times or more Response categories (3) Ranking question Please indicate in the boxes provided which features you believe are the most important when visiting our superstore (1 indicating the most important, 2 the next most important, etc.) . Please leave blank those features that have no importance at all. Ease of car parking Low prices Friendly staff Store loyalty card Variety of goods General Instructions Title of survey This survey is being undertaken by [your name] as part of [name of course] at the University of Surrey to [state purpose]. It will take approximately x minutes to complete. All information provided is completely confidential. Thank you for providing your comments. Please return to:………………by……… Specific instructions: go to 12 Do you eat Krispy Crisps? No Go to 18 Yes 13 How many packets per week? Pilot the questionnaire Through piloting you may decide to: • Change the wording of individual questions. • Add/eliminate some questions. • Amend the instructions. • Decide not to use the questionnaire! Summary • Designing individual questions involves a rigorous process of analysis to avoid ambiguity, leading questions, double questions and simply misleading questions. • Questions must be clearly linked to the purpose of the research • Questionnaires should start with questions that are easy to answer, interesting and transparently linked to the purpose of the research. • Questionnaire layout and the use of typography can make a questionnaire easier to complete and more appealing to respondents, enhancing the response rate. • Clear, well set out instructions on completing the questionnaire can also boost the response rate. • Web and email questionnaires offer a new and potentially powerful tool, but also require additional design skills. • All questionnaires, whether paper-based, email or Web-based, require thorough piloting which will include evaluation of accompanying documentation, instructions, individual questions, types of question, question sequencing, the use of scales and skip instructions – basically, everything!