Chapter 4 Composing Questions McGraw-Hill/Irwin © 2004 by The McGraw-Hill Companies, Inc. All rights reserved. Guidelines for Expressing Questions • FOCUS • Every question on a questionnaire should focus directly on a single, specific issue or topic • BREVITY • Short questions are less subject to error on the part of both interviewer and respondents • CLARITY • Demands that virtually everyone interprets the question in exactly the same way 4-2 Guidelines for Expressing Questions • VOCABULARY • Use words in the core vocabulary of virtually all respondents • Limit vocabulary to words the least sophisticated respondent will know • GRAMMAR • Use simple sentences where possible, complex only when necessary • Break compound and compound/complex into simple sentences • Change long, dependent clauses to words or short phrases where possible 4-3 Instrumentation Bias and Error • Unstated Criteria • State criteria in the question if it’s at all unclear to respondents regarding how to respond • Inapplicable Questions • The questions must be applicable to all respondents, based on their own situation • Example Containment • Don’t use actual response alternatives as examples or respondents will over-select them 4-4 Instrumentation Bias and Error • Over-Demanding Recall • Don’t assume respondents will recall their behavior or feelings over an extended period • Over-Generalizations • Seek generalizations only if they represent policies, strategies, or habitual behavior pattern • Over-Specificity • Don't ask for a precise answer unless respondents will be able to express it exactly 4-5 Instrumentation Bias and Error • Over-Emphasis • Avoid dramatic terms and lean toward understatement, rather than overstatement • Ambiguity of Wording • Every word or phrase must have a plain, common meaning for everyone in the sample • Double-Barreled Questions • Split or modify compound questions, especially those asking an action and a reason for it 4-6 Instrumentation Bias and Error • Leading Questions • Avoid leading words such as “Don’t you . . .” or in favor of “Do you . . .” or “Would you . . .” • Loaded Questions • Never use questions asking a preference or opinion and including a socially desirable reason 4-7 Avoiding Instrumentation Bias 1. State the criteria for response clearly 2. Each item must apply to all respondents presented with it 3. Use descriptions rather than examples of things being rated 4. Don’t tax respondents memories beyond what they can recall 5. Don’t ask for generalizations if specifics are needed and obtainable 6. Don’t make questions more specific than respondents are able to answer 4-8 Avoiding Instrumentation Bias 7. Avoid being over-emphatic by using less dramatic terms 8. Use commonly recognized terms to avoid ambiguity of meaning 9. Express items and questions in a non-threatening way 10. Split so-called “double-barreled” questions into two or more items 11. Express items inclined toward both “yes” and “no” to avoid yea- and nay-saying 12. Rewrite items beginning with a “leading” phrase 13. Never “load” questions by including a powerful reason to respond in a certain way 4-9 Sources of Response Bias • Social Desirability • Response based on what's perceived as socially acceptable or respectable • Acquiescence • Response based on perception of what would be desirable to the sponsor • Yea- and Nay-Saying • Response influenced by a global tendency toward positive or negative answers 4-10 Sources of Response Bias • Prestige • Response intended to enhance the image of the respondent in the eyes of the others • Threat • Response influenced by anxiety or fear instilled by the question or interviewer • Hostility • Response arises from feelings of anger or resentment engendered by the response task 4-11 Sources of Response Bias • Auspices • Response dictated by the image or opinion of the sponsor, rather than the actual question • Mental Set • Cognitions or perceptions based on previous items influence response to later ones • Order • The sequence in which a series is listed affects the responses to the items • Extremity • Clarity of extremes and ambiguity of mid-range options encourage extreme responses 4-12 Question Formats Compared • Unstructured • Doesn’t clearly show the dimensions respondents are supposed to use • Structured • They clearly show the dimensions respondents are supposed to use 4-13 Question Formats Compared • Unstructured • Data aren’t directly comparable from one respondent to the next • Structured • Data are directly comparable from one respondent to the next 4-14 Question Formats Compared • Unstructured • Verbatim responses are time-consuming and difficult to record • Structured • Precoded response alternatives are quick and easy to record 4-15 Question Formats Compared • Unstructured • Response task is time-consuming and difficult for respondents • Structured • Response task is quick and easy for all respondents 4-16 Question Formats Compared • Unstructured • Relatively quick and easy for the researcher to compose • Structured • Very time-consuming and difficult for the researcher to compose 4-17 Composing Categorical Questions • A categorical item asks a question and is followed by a series of alternative answers • The list of categories must be all-inclusive, include every possible response • Use mutually exclusive categories with a unique association with any possible answer 4-18 Composing Categorical Questions • There should be more variance in the meaning between categories than within them • The alternatives used should cluster together responses that are similar to one another 4-19 Response Category Size and Number • The number of alternatives for an item may range from two to about six or eight categories • The categories shouldn't be more "fine-grained" than required by the sponsors 4-20 Response Category Size and Number • Too many, narrow categories are better than too few, broad ones • Too many, narrow categories can readily be combined into fewer, broader ones after data collection • Too few, overly broad categories can't be split into more, smaller ones after data collection 4-21 End of Chapter 4 McGraw-Hill/Irwin © 2004 by The McGraw-Hill Companies, Inc. All rights reserved.