CHAPTER 10 MEASUREMENT IN MARKETING RESEARCH Important Topics of This Chapter Basic types question-response format. Consideration of choosing a question responseformat. Measurement and scale characteristics in a question-response format. Levels of measurement of scales. Various types scaled-response question formats. Reliability and validity of measurements. The Questionnaire’s “Position” in the Research Process Respondent’s Information Survey Objectives Questionnaire Data Analysis Findings Recommendations Managerial Action Criteria for a Good Questionnaire To design a good questionnaire, the following issues should be considered: Does it Provide the Necessary DecisionMaking Information? Does it Consider the Respondent? Basic Question-Response Format Editing Refers to going through the questionnaire to make certain the “skip patterns” are followed and required questions are filled out. A skip pattern is the sequence in which questions are asked. Open-Ended Response Format Questions: An open-ended question is one that does not contain prerecorded possible responses: Un-probed format: Seeks no additional information from respondents. Probed format: Researcher may ask comments or statement from the respondents. Response format: Researcher may ask additional information. Basic Question-Response Format (cont.) Closed-Ended Response Format Questions: Dichotomous closed-ended questions: Yes/No options. Multiple category closed-ended questions: They are very popular question style. Scaled-response Questions: Un-labeled scaled-response format: Purely numerical or only endpoints are identified. Labeled scaled-response format: All of the scaled position are identified. Considerations in Choosing A Questions Response Format Nature of property being measured: Different type of question format must be used. Previous research studies: Questionnaires may be used with permission. Data collection mode: Mail, telephone, personal/computer interviews. Ability of the respondents: Previous research experiences may help. Scale level desired: 3, or 5, or 7 points scales. Basic Concepts in Measurement Objects: Consumers, brands, stores, advertisements. Properties: Demographic characteristics. Objective properties: Physically verifiable. Subjective properties: Cannot be directly observed, such as person’s attitude and intentions. Scale Characteristics Description: Agree/Disagree, Approve/Disapprove Order: Size of the descriptor . Distance: Two cars Vs. one car family. Origin: 0 or 1. Primary Scales of Measurement Scale Nominal Ordinal Interval Ratio Numbers Assigned to Runners Finish 7 8 3 Finish Rank Order of Winners Performance Rating on a 0 to 10 Scale Time to Finish, in Seconds Third place Second place First place 8.2 9.1 9.6 15.2 14.1 13.4 Primary Scales of Measurement Basic Characteristics Nominal Numbers identify & classify objects Common Examples Social Security nos., numbering of football players Ordinal Nos. indicate the Quality relative positions rankings, of objects but not rankings of the magnitude of teams in a differences tournament between them Temperature Interval Differences between objects (Fahrenheit, can be compared, Celsius) zero point is arbitrary Scale Ratio Zero point is fixed, ratios of scale values can be compared Length, weight Marketing Permissible Statistics Inferential Examples Descriptive Brand nos., Percentages, Chi-square, store types mode binomial test Preference rankings, market position, social class Percentile, median Rank-order correlation, Friedman ANOVA Attitudes, opinions, index nos. Range, mean, Productstandard moment deviation correlation, t tests, regression Age, sales, income, costs Geometric mean, harmonic mean Coefficient of variation A Classification of Scaling Techniques Scaling Techniques Non-comparative Scales Comparative Scales Paired Comparison Rank Order Constant Sum Q-Sort and Other Procedures Likert Continuous Itemized Rating Scales Rating Scales Semantic Differential Stapel Attitude Scales •Scaling Defined: •The term scaling refers to procedures for attempting to determine quantitative measures of subjective and sometimes abstract concepts. It is defined as a procedure for the assignment of numbers to a property of objects in order to impart some of the characteristics of numbers to the properties in question. Unidimensional and Multidimensional Scaling Unidimensional Scaling Multidimensional Scaling Procedures designed to measure only one attribute of a respondent or object Procedures designed to measure several dimensions of a respondent or object Different Type of Scales Graphic Rating Scale: Present respondents with a graphic continuum typically anchored by two extremes. Itemized Rating Scale: Itemized rating scales are very similar to graphic rating scales, except that respondents must select from a limited number of ordered categories rather than placing a check mark on a continuous scale. Rank-Order Scale: Itemized and graphic scales are non-comparative because the respondent makes a judgment without reference to another object, concept, or person. Rank-order scales, on Different Type of Scales (cont.) Q-Sorting: Q-Sorting is basically a sophisticated form of rank ordering. A set of objects - verbal statements, slogans, product features, potential customer services, and so forth - is given to an individual to sort into piles according to specific rating categories. Paired Comparison: Paired comparison scales ask a respondent to pick one of two objects from a set based upon some stated criteria. Different Type of Scales (cont.) Constant Sum Scales: Constant sum scales are used more often by market researchers than paired comparisons because the long list of paired items is avoided. This technique requires the respondent to divide a given number of points, typically 100, among two or more attributes based on their importance to the persons. Different Type of Scales (cont.) Semantic Differential Scale: The construction of the semantic differential scale begins with the determination of a concept to be rated. The researcher selects dichotomous pairs of words or phrases that could be used to describe the concept. Respondents then rate the concept on a scale. The mean of these responses for each pair of adjectives is computed and plotted as a “profile” or image. Stapel Scale: The Stapel scale is a modification of the semantic differential. A single adjective is placed at the center of the scale. Typically it is designed as a 10-point scale ranging from +5 to -5. The technique is designed to measure both the direction and intensity of attitudes simultaneously. Different Type of Scales (cont.) Likert Scale: The Likert scale consists of a series of statements that express either a favorable or an unfavorable attitude toward the concept under study. Purchase Intent Scale: Scale designed to measure the likelihood that a potential customer will purchase a product or service. Some Basic Considerations When Selecting a Scale Selecting a Rating, Ranking, Sorting, or Purchase Intent Scale Number of Categories Forced Versus Non-forced Choice Odd or Even Number of Scale Categories Balanced Versus Nonbalanced Alternatives Approaches to Identifying Determinant Attitudes Direct Questioning Indirect Questioning Observation Obtaining Shampoo Preferences Using Paired Comparisons Instructions: We are going to present you with ten pairs of shampoo brands. For each pair, please indicate which one of the two brands of shampoo you would prefer for personal use. Recording Form: Jhirmack Finesse Vidal Sassoon Head & Shoulders Pert Number of Times Preferredb aA 1 Jhirmack Finesse 0 1a 1 0 1 3 1 0 1 2 Vidal Heads & Sassoon Shoulders 0 1 0 1 1 0 0 1 0 4 Pert 0 0 1 0 1 in a particular box means that the brand in that column was preferred over the brand in the corresponding row. A 0 means that the row brand was preferred over the column brand. bThe number of times a brand was preferred is obtained by summing the 1s in each column. Paired Comparison Scaling The most common method of taste testing is paired comparison. The consumer is asked to sample two different products and select the one with the most appealing taste. The test is done in private and a minimum of 1,000 responses is considered an adequate sample. A blind taste test for a soft drink, where imagery, self-perception and brand reputation are very important factors in the consumer’s purchasing decision, may not be a good indicator of performance in the marketplace. The introduction of New Coke illustrates this point. New Coke was heavily favored in blind paired comparison taste tests, but its introduction was less than successful, because image plays a major role in the purchase of Coke. A paired comparison taste test Preference for Toothpaste Brands Using Rank Order Scaling Instructions: Rank the various brands of toothpaste in order of preference. Begin by picking out the one brand that you like most and assign it a number 1. Then find the second most preferred brand and assign it a number 2. Continue this procedure until you have ranked all the brands of toothpaste in order of preference. The least preferred brand should be assigned a rank of 10. No two brands should receive the same rank number. The criterion of preference is entirely up to you. There is no right or wrong answer. Just try to be consistent. Brand Rank Order 1. Crest _________ 2. Colgate _________ 3. Aim _________ 4. Gleem _________ 5. Macleans _________ 6. Ultra Brite _________ 7. Close Up _________ 8. Pepsodent _________ 9. Plus White _________ 10. Stripe _________ Importance of Toilet Soap Attributes Using a Constant Sum Scale Instructions On the next slide are eight attributes of bathing soaps. Please allocate 100 points among the attributes so that your allocation reflects the relative importance you attach to each attribute. The more points an attribute receives, the more important the attribute is. If an attribute is not at all important, assign it zero points. If an attribute is twice as important as some other attribute, it should receive twice as many points. Average Responses of Three Segments Attribute 1. Mildness 2. Lather 3. Shrinkage 4. Price 5. Fragrance 6. Packaging 7. Moisturizing 8. Cleaning Power Sum Segment I Segment II Segment III 8 2 3 53 9 7 5 13 100 2 4 9 17 0 5 3 60 100 4 17 7 9 19 9 20 15 100 Basic Non-comparative Scales Scale Basic Characteristics Continuous Place a mark on a continuous line Rating Scale Examples Advantages Reaction to TV Easy to construct commercials Itemized Rating Scales Measurement Likert Scale Degrees of agreement on a 1 of attitudes (strongly disagree) to 5 (strongly agree) scale Scoring can be cumbersome unless computerized Easy to construct, More administer, and time-consuming understand Seven-point scale Brand, product, Versatile Semantic Differential with bipolar labels and company images Stapel Scale Unipolar ten-point scale, -5 to +5, without a neutral point (zero) Disadvantages Controversy as to whether the data are interval Measurement Easy to construct, Confusing and of attitudes and administer over difficult to apply images telephone A Semantic Differential Scale for Measuring SelfConcepts, Person Concepts, and Product Concepts 1) Rugged :---:---:---:---:---:---:---: Delicate 2) Excitable :---:---:---:---:---:---:---: Calm 3) Uncomfortable :---:---:---:---:---:---:---: Comfortable 4) Dominating :---:---:---:---:---:---:---: Submissive 5) Thrifty :---:---:---:---:---:---:---: Indulgent 6) Pleasant :---:---:---:---:---:---:---: Unpleasant 7) Contemporary :---:---:---:---:---:---:---: Obsolete 8) Organized :---:---:---:---:---:---:---: Unorganized 9) Rational :---:---:---:---:---:---:---: Emotional 10) Youthful :---:---:---:---:---:---:---: Mature 11) Formal :---:---:---:---:---:---:---: Informal 12) Orthodox :---:---:---:---:---:---:---: Liberal 13) Complex :---:---:---:---:---:---:---: Simple 14) Colorless :---:---:---:---:---:---:---: Colorful 15) Modest :---:---:---:---:---:---:---: Vain Balanced and Unbalanced Scales Jovan Musk for Men is Extremely good Very good Good Bad Very bad Extremely bad Jovan Musk for Men is Extremely good Very good Good Somewhat good Bad Very bad Rating Scale Configurations A variety of scale configurations may be employed to measure the gentleness of Cheer detergent. Some examples include: Cheer detergent is: 1) Very harsh ------------- --- Very gentle 2) Very harsh 1 2 3 4 5 6 3) . Very harsh Cheer . . Neither harsh nor gentle . . Very gentle 4) ____ ____ ____ ____ ____ Very Somewhat Neither harsh Somewhat harsh Harsh harsh nor gentle gentle 5) -3 Very harsh -2 -1 0 Neither harsh nor gentle +1 7 Very gentle ____ Gentle +2 ____ Very gentle +3 Very gentle Some Unique Rating Scale Configurations Thermometer Scale Instructions: Please indicate how much you like McDonald’s hamburgers by coloring in the thermometer. Start at the bottom and color up to the temperature level that best indicates how strong your preference is. Form: Like very much Dislike very much 100 75 50 25 0 Smiling Face Scale Instructions: Please point to the face that shows how much you like the Barbie Doll. If you do not like the Barbie Doll at all, you would point to Face 1. If you liked it very much, you would point to Face 5. Form: Table 9.2 Summary of Itemized Scale Decisions 1) Number of Categories Although there is no single, optimal number, traditional guidelines suggest that there should be between five and nine categories 2) Balanced vs. unbalanced In general, the scale should be balanced to obtain objective data 3) Odd/ even no. of categories If a neutral or indifferent scale response is possible from at least some of the respondents, an odd number of categories should be used 4) Forced vs. non-forced In situations where the respondents are expected to have no opinion, the accuracy of the data may be improved by a non-forced scale 5) Verbal description An argument can be made for labeling all or many scale categories. The category descriptions should be located as close to the response categories as possible 6) Physical form A number of options should be tried and the best selected Reliability of Measurements Scale Evaluation Reliability Test/ Retest Alternative Forms Validity Internal Consistency Content Criterion Convergent Generalizability Construct Discriminant Nomological Potential Sources of Error on Measurement 1) Other relatively stable characteristics of the individual that influence the test score, such as intelligence, social desirability, and education. 2) Short-term or transient personal factors, such as health, emotions, fatigue. 3) Situational factors, such as the presence of other people, noise, and distractions. 4) Sampling of items included in the scale: addition, deletion, or changes in the scale items. 5) Lack of clarity of the scale, including the instructions or the items themselves. 6) Mechanical factors, such as poor printing, overcrowding items in the questionnaire, and poor design. 7) Administration of the scale, such as differences among interviewers. 8) Analysis factors, such as differences in scoring and statistical analysis.