Chapter 4: Of Tests and Testing
12 Assumptions in Psychological Testing and Assessment
•
Assumption 1: Psychological traits and states exist
•
Assumption 2: Psychological traits and states can be quantified and measured
•
Assumption 3: Various approaches to measuring aspects of the same thing can be useful
•
Assumption 4: Assessment can provide answers to some of life’s most momentous
questions
•
12 Assumptions in Psychological Testing and Assessment
•
Assumption 5: Assessment can pinpoint phenomena that require further attention or study.
•
Assumption 6: Various sources of data enrich and are part of the assessment process.
•
Assumption 7: Various sources of error are part of the assessment process.
•
12 Assumptions in Psychological Testing and Assessment
•
Assumption 8: Tests and other measurement techniques have strengths and weaknesses
•
Assumption 9: Test-related behavior predicts non–test-related behavior
•
Assumption 10: Present-day behavior sampling predicts future behavior
•
12 Assumptions in Psychological Testing and Assessment
•
Assumption 11: Testing and assessment can be conducted in a fair and unbiased manner
•
Assumption 12: Testing and assessment benefit society
Most Controversial Assumption?
•
Why?
What are Norms?
•
Derived typical test performance of a standardization sample
•
The test score distribution that provides the average or typical (normal) score level on a test
Standardization (normative) Sample
•
The normative sample is a representative subset drawn from the broader target population
•
Typically, a large random sample
•
Sample size should be large enough to obtain stable values.
Sampling Techniques
•
Randomization
– every case has an equal chance of selection
•
Stratified
– representative proportions of groups
– e.g. age, socioeconomic level, ethnicity
•
Incidental
– Convenience sampling
– Not a desired procedure
Types of Norms
•
Developmental Norms
•
Indicates developmental level attained
•
Age equivalent norms (Mental Age)
•
•
•
e.g., a 7 year old who scores the same mean obtained by 10 year old children has a mental
age of 10.
Grade equivalent norms
e.g., average score of 4th graders is 23, a child with a raw score of 23 is given a 4th grade
age equivalence.
Norm-Referenced (Within group) Norms
•
Individual performance is evaluated in reference to a standardization group
•
The same test is used to compare other groups of test-takers
•
Deviation IQs
What is Correlation?
•
Index of linear association between two variables (X and Y)
•
Does not suggest cause and effect
•
Computed value is called a coefficient
•
Best example is the Pearson product-moment correlation coefficient (r)
Pearson Formula (definitional)
•
Co-variation between X and Y
•
Ratio of the variability between X and Y
Values of r
•
Coefficient values range between -1 and + 1
•
What does 0 mean?
– The closer the coefficient value is to 0, the weaker the association between two
variables
•
The further a coefficient moves from 0, the stronger the association between two variables
–
Coefficients of -1 and +1 have the same magnitude of association
Coefficient of Determination (r2)
•
Correlation coefficient squared
•
The value indicates the proportion of the variation in Y scores that is a function of the X
scores
•
i.e., the variance in X explained by Y
Graphing Correlation
•
Correlations between two variables can be displayed in a scatterplot
•
Individual scores are plotted on two-dimensional axes
– X scores plotted on horizontal axis (abscissa)
– Y scores plotted on vertical axis (ordinate)
Positively Correlated
•
As X increases, Y increases
Negatively Correlated
•
As X increases, Y decreases
No Correlation
•
No relationship between X and Y
Curvilinear Relationship
•
Non-linear relationship between two variables
•
The scatterplot has a significant curve
•
U-shaped curve
•
Umbrella-shaped curve
•
S-shaped curve
Other Correlation Coefficients
•
Spearman rho
– Used in rank-order correlation
• ordinal scales
•
Evaluate the differences (or agreement) between rankings of two variables
– Students’ scores on a mid-term 1 and mid-term 2 are ranked from lowest to highest;
the rankings are correlated
Point Biserial
•
Comparison of one continuous variable and one dichotomous variable
•
Dichotomous variables include Yes/No or True/False scales
– Correlation between Age (continuous) and Active Class Participation (Yes or No)
Phi
•
Correlation between two dichotomous variables
– Correlation between Active Class Participation (Yes or No) and Mid-term results
(Pass or Fail)
Biserial r
•
Comparison of one continuous variable and one artificially dichotomized variable
•
An artificially dichotomized variable is a continuous variable that is transformed to
dichotomous variable
– e.g., Age in years converted to age groups
• 18-25, 26-30, 31-40, 40-50, 51-60, etc.
Tetrachoric
•
Correlation between two artificially dichotomized variables
– Correlation between age groups and mid-term score
Roles of Correlations in Testing
•
Test-retest reliability
– Correlation between scores on the same test at two different times
• Correlation of GRE in the Fall and Spring semesters
•
•
•
Criterion (predictive) validity coefficients
– Correlation between test scores and results of an independent criterion
Correlation between SAT and College GPA
Convergent validity coefficients
– Correlation between scores on two conceptually similar tests
• Correlation between self-esteem and self-concept
Regression
•
Degree of predictability between two variables
•
Extends the concept of correlation to the prediction of a test score (Y) based on a another
test score (X)
Regression Equation
•
•
•
•
Y’ = a + bX
X = predictor (test score)
Y’ = criterion (predicted score)
a = y-intercept (criterion score if the predictor score is 0)
b = slope (correlation between the predictor and criterion)
Regression Line
•
Line drawn through the scatter of scores
•
The regression line represents the Principle of Least Squares
•
least squared deviation from the line
•
The line demonstrates the best fit for all data points
Slope
•
Essentially, the correlation coefficient
Y-Intercept
•
Where the regression line crosses the Y-axis
•
Criterion score if the predictor score is 0
a = Y – bX
•
•
Y is the mean of the Y scores
X is the mean of the X scores
Regression Example
Y’ = 2 + 0.67X
•
What is the predicted score (Y’) if X is 10?
Regression Line
•
Line drawn through the scatter of scores
•
The regression line represents the Principle of Least Squares
– least squared deviation from the line
•
The line demonstrates the best fit for all data points
•
Regression Line Example
Residuals
•
Difference between the predicted (Y’) and observed criterion (Y) values
•
Y – Y’
•
Principle of Least Squares
•
Minimize the deviation between Y and Y’
Standard Error of Estimate
•
Error in the prediction estimate
– Standard deviation of the residuals
• The square root of the residual variance
•
The lower the standard deviation, the lower the degree of error in the regression equation
Inference from Measurement
•
Meta-Analysis
– Statistical combination of studies
•
Culture and Inference
– Individualists vs. collectivists