EDL 880
Dr. Jeffrey Oescher
Outline of Analyses – Affective Scales
I.
Entering data
A. Determine the structure of your data set
1. How many variables
2. Names of the variables
a. Use obvious names
b. Think ahead for potential problems like reversing Likert responses
c. Consider labeling the variables only if it will help you in the analysis or
presentation of the data
3. Types of variables
a. SPSS sometimes does not convert string variables to numeric or recognize string
variables in procedures (e.g., ANOVA)
b. Use numeric variables when possible
c. Record the codes for the values you assign to the levels of any variables (e.g., 0
for females and 1 for males) or use a pneumonic to remember these codes (0 for
females and 1 for males - 0 is the first number and 1 is the second while 'f' is the
first letter in the alphabet and 'm' the second)
4. Coding the identification of the respondents and/or surveys
a. Usually numeric and consecutive
b. Coding additional information into an ID such as group membership
5. Entering data
a. Entering data by yourself
b. Using a colleague to help - one reads and the other records
c. Retrieving files from Survey Monkey
i. The need for one variable for each item
ii. Be certain to request the data is recorded in this format
iii. It is very simple to convert an Excel file to an PSAW file
6. Saving your data set
a. Use names that are self explanatory
b. Number your data sets as you modify them (e.g., fe1.sav, fe2.sav, fe3.sav, etc.)
c. Always keep a backup copy of the most recent data set.
II. Cleaning data
A. Always check the data set for "dirty" data such as typos, incorrectly keyed items, missing
data, etc.
B. Use the SPSS analysis procedure FREQUENCIES for each item and all other
demographic data that is reasonable (i.e., no more than three or four values of the
variable)
C. Corrections
1. Correct any obvious mistakes with data entry
2. Eliminate either the item response or the observation if no mistake can be
established
Document1
1
D. Save the data set under a new name so as to preserve the original data
III. Create and/or modify variables
A. Create any variables needed
1.
2.
3.
4.
5.
Reversed item responses
Item scores (i.e., correct or incorrect)
Total scores
Subscale scores
Number of missing responses
B. Use the SPSS transform commands COMPUTE or RECODE as well as any appropriate
functions such as NMISS or MEAN
1. Always recode or compute variables with names other than those that currently exist
2. Determine how to deal with missing responses
a. Use the SPSS function NMISS to identify the level of missing data for each
student
b. Calculate total scores using the MEAN function
i.
ii.
This function calculates the mean of all non-missing values
This function keeps the total score in the same metric as the response scale
(e.g., 1, 2, 3, or 4 for a four point Likert scale)
C. Check one or two observations to be certain the transformations are being done correctly
IV. Analyze the data
A. Review the proportion of responses to each alternative
1. Use the SPSS analysis procedure FREQUENCIES for each item
2. Review the output for each item
3. Most items will have some variation in students' responses although this is not
necessary
B. Review all item reliabilities1
1. Calculate the item reliability for each statement
a. This is the correlation between a student's scored response to a statement (e.g.,
1, 2, 3, or 4 for a four point Likert scale; 1, 2, 3, or 4 for a four point semantic
differential scale; etc.) and their total scale score
i.
ii.
Because each point on the response scale represents a description of the
affective characteristic being assessed, there are no correct or incorrect
responses
There is no need to recode any of the students' responses beyond that done
for scoring purposes (e.g., reversing item scores)
b. Use the SPSS analysis procedure CORRELATIONS to calculate the item
reliability index for each item
c. Item reliability indices should be greater than +0.30
1
These analyses apply to situations where a total score is being calculated. If subscales are
being calculated, the analyses should be used for the items comprising that subscale and the
subscale score.
Document1
2
i.
ii.
Higher item reliability indices (e.g., +0.50, +0.60, etc.) indicate the item
differentiates well between those students who have higher levels of the
affect characteristic being measured by the item (i.e., more positive,
favorable, interested, or important) have higher total scores than those who
have lower levels of this characteristic
Lower item reliability indices (e.g., +0.20, +0.10, -0,10, etc.) suggest
something is not working correctly with this item as students with lower levels
of the characteristic being measured by the item (e.g., less positive,
favorable, interested, or important) have higher total scores than those who
have higher levels of this characteristic
2. Consider revising any item for which the item reliability data is suspect
C. Review the reliability index for the items on the scale
1. Calculate Cronbach's alpha for a scale and any subscales
2. Use the SPSS analysis procedure RELIABILITY to calculate these reliability indices
a. This procedure is found using the ANALYZE AND SCALE pull down menus
b. The RELIABILITY procedure will calculate Cronbach's alpha which is a more
generalized form of the KR 20
3. Interpret the reliability index relative to the decisions being made with the test or
scale results
a. Reliability indices greater than +0.70 are usually acceptable for tests or scales
that are not standardized
b. Factors affecting reliability indices
i.
Test length
a) The reliability index for a longer scale is likely going to be higher than
that for a shorter test
b) Using scales with very few items (e.g., a subscale based on three items)
is not recommended
ii.
Variability of scores
a) A scale on which students score very similarly (i.e., homogeneous
scores) will have a lower reliability index than a scale on which students
scores vary greatly (i.e., heterogeneous scores)
b) Scales in which students all do very well - a goal of good instruction - will
have lower reliability indices due to the homogeneity of scores
c) Scales in which students al do very poorly - a situation often found in
pre-assessment of students' knowledge - will have lower reliability
indices due to the homogeneity of scores
D. Review the descriptive statistics for the scores of those in the sample
1. Calculate descriptive statistics (i.e., mean, standard deviation, n, etc.) for the total
score and any subscale scores
a. Calculating a total score as the mean of all non-missing items preserves the
meaning of the points on the response scale
b. Use the SPSS analysis procedure DESCRIPTIVES to calculate these statistics
2. Interpret the scores
Document1
3
a. Scores can be interpreted relative to the underlying continuum of the affective
characteristic being assessed (i.e., a criterion-referenced interpretation)
i. Usually the neutral point on the response scale is used as the reference point
b. Scores can be interpreted relative the scores of others (i.e., a norm-referenced
interpretation)
i.
Document1
The scores of known groups are often used for comparisons
4