Psychological testing
Winter 2014
Mikołaj Winiewski, PhD
Marcin Zajenkowski, PhD
Mikołaj Winiewski, PhD
mikolaj.winiewski@psych.uw.edu.pl
preferred: mwiniewski@gmail.com
Marcin Zajenkowski, PhD
zajenkowski@psych.uw.edu.pl
COURSE PAGE
www.psych.uw.edu.pl/zajenkowski
Enter teaching
Aim of the course
• To understand the basic theoretical issues
concerning construction and utilizing
psychological tests.
• To acquaint student with the fundamental
vocabulary and logic of psychological
measurement and behavioral assessment.
• To understand logic of design and psychometric
properties behind any psychological test.
Expectations and Class Etiquette
• This course is a LECTURE which means that
presence in class IS NOT IS NOT MANDATORY.
• If you can read and understand assigned
literature you should pass the exam
– around 70% of the exam questions will be based
on assigned basic literature
Expectations and Class Etiquette
Our responsibility (as instructors) to you (as a students) is
to help you reach your full potential. As such, you may
expect the following from us:
• We will come to class on time
• We will have your tests quickly graded
• We will listen to, and carefully consider any well
thought out question or concern you have about
grading, or any other aspect of the class
• We will treat each one as a university student. That
means the responsibility for doing well in this class lies
with you, but We shall endeavor to assist you
Expectations and Class Etiquette
In order for this class to be a success, and for you to get as
much as possible from it, We expect at least the following
from you:
• You will come to class on time
• You will not hold idle conversations during the lectures
• You will not use mobile phones, game consoles or any other
stuff that is not necessary for participating in class
• You will not eat in the class
• In your dealings with any person in this class, you will
conduct yourself, always, with a minimum of respect,
making sure to make your contributions in a manner that is
neither rude nor offensive
Grading
• There will be two summative exams: midterm
and final worth in total 60 points.
– Midterm exam (25 points) will cover material from
lectures 1 – 6
– Final exam (35 points) will cover material from
lectures 7 - 15.
– Retake will cover all material.
• Exams will consist several multiple choice and
open ended questions covering material from
the literature and lectures.
Extra Credit
Individual assignment worth additional 3 points.
• Short paper on selected topic.
• Select one of five proposed construct and
create a draft of the tool to measure it.
• Deadline – last class.
Notice:
Your paper is to be the product of independent work.
Plagiarism is unacceptable and will result in an
automatic FAILURE (even thou it's an extra credit).
Extra credit
• Choose one psychological construct (e.g. altruism,
verbal intelligence). Write a report (2-3 pages, APA
style) and provide:
– 10 items assessing the construct
– Which response type would be best for your test?
– Which method of determining reliability would be most adequate for
your test? (justify your claim)
– Describe three possible studies that might examine the validity of your
test – base on the psychological literature, scientific articles, studies
reports.
-more details can be found on the course page-
Psychological constructs to choose
1. Sensation Seeking is a trait defined by the
search for experiences and feelings, that are
varied, new, complex and intense, and by the
readiness of experiencing those physical,
social, legal, and financial experiences.
2. Altruism is a concern for the welfare of others. Altruism is the
opposite of selfishness. It can be distinguished from feelings
of duty and loyalty. Altruism is a motivation to provide
something of value to a party who must be anyone but the
self, while duty focuses on a moral obligation towards a
specific individual (for example, a god, a king), or collective
(for example, a government). Pure altruism consists of
sacrificing something for someone other than the self (e.g.
sacrificing time, energy or possessions) with no expectation of
any compensation or benefits, either direct, or indirect (for
instance from recognition of the giving).
• 3. Math anxiety is an emotional reaction to
mathematics based on a past unpleasant
experience which harms future learning. A
good experience learning mathematics can
overcome these past feelings and success and
future achievement in math can be attained.
4. Optimism is a mental attitude that interprets
situations and events as being best (optimized),
meaning that in some way for factors that may not
be fully comprehended, the present moment is in an
optimum state. The concept is typically extended to
include the attitude of hope for future conditions
unfolding as optimal as well. This understanding
leads to a state of mind that believes everything is as
it should be, and that the future will be as well.
• 5. Verbal intelligence is the ability to analyze
information and solve problems using
language-based reasoning. It is associated
with the ability to listen to and recall spoken
information; understanding the meaning of
written or spoken information; solving
language based problems of a literary, logical,
or social type; the ability to perform complex
language-based analysis.
Grading
POINTS
GRADE
0-35
36-44
45-50
51-54
55-58
59-60
2
3
3,5
4
4,5
5
Book
Murphy, K. R. & Davidshofer, C. O. (2005).
Psychological testing. Principles and
applications, (6ed). Upper Saddle River, NJ.:
Prentice-Hall, Inc.
Course schedule
1. Introduction
Course description, Definition of psychological test, Item format
2. Psychometrics basic concepts
Basic aspects of psychological testing - what is it, why we are using it and how
we are using it? Types of test, Standardization.
•
Chapter 1: Tests and Measurements - Murphy, K. R. & Davidshofer, C. O. (2005).
Psychological testing. Principles and applications, (6ed). Upper Saddle River, NJ.: PrenticeHall, Inc.
3. Statistics – refresher
Basic statistic - variable types, distributions, probabilities, sampling
•
Chapter 4. Basic Concepts in Measurement and Statistics - Murphy, K. R. & Davidshofer, C. O.
(2005). Psychological testing. Principles and applications.
4. Norms
Normalization, transformation of raw scores, types of normalized scales,
criterion reference.
•
Chapter 5. Scales, Transformations, and Norms - Murphy, K. R. & Davidshofer, C. O. (2005).
Psychological testing. Principles and applications.
Course schedule
5. Reliability - True score theory, methods of
determining reliability: test-retest, alternate form
•
Chapter 6. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing.
Principles and applications.
6. Reliability - Methods of determining reliability:
internal consistency
•
Chapter 7. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing.
Principles and applications.
7. Midterm Exam
November 13th 2014
Course schedule
8. Test construction 1 Strategies of test development.
• Chapter 11. The Process of Test Development - Murphy, K. R. &
Davidshofer, C. O. (2005). Psychological testing. Principles and
applications.
9. Test construction 2
Item sampling, Structure exploration and confirmation, Item
analysis: difficulty and discrimination, New approaches to
item analysis - Item Response Theory (IRT).
• Chapter 10. Item Analysis - Murphy, K. R. & Davidshofer, C. O. (2005).
Psychological testing. Principles and applications. (p.202-225)
• Chapter 11. The Process of Test Development - Murphy, K. R. &
Davidshofer, C. O. (2005). Psychological testing. Principles and
applications.
Course schedule
10. Validity
Definition of validity, content validity, examples
• Chapter 7. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing.
Principles and applications.
11. Validity
Criterion-related validity, examples, construcs validity, examples
• Chapter 8. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing.
Principles and applications.
12. Diagnosis
• Aspects of the psychological diagnosis. Interpretation of test
scores.
• Chapter 2. and 3. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological
testing. Principles and applications. (p.20-51)
Course schedule
13. Testing in a Cross-Cultural Context - Tests in different
cultures, tests adaptation process
•
Chapter 11. Testing in a Cross-Cultural Context Domino, G., Domino M.L. (2006).
Psychological Testing, An Introduction. Cambridge University Press.
14. Specific tests Tests from various areas of psychology and
their specific features, examples of personality, intelligence,
neuropsychological, affect tests
•
Chapter s 13, 14, 17 . Murphy, K. R. & Davidshofer, C. O. (2005). Psychological
testing. Principles and applications.
15. Ethical principles in testing - Basic ethical issues in
constructing and utilizing psychological tests.
•
APPENDIX B. Ethical Principles of Psychologists and Code of Conduct - Murphy, K.
R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications.
Short test…
Tests items examples
1. Describe briefly what a psychological test is?
Open-ended / essay
• What could be measured: knowledge,
opinion.
• Easy to construct
• Encourages more appropriate study habits
• Measures higher-order outcomes (i.e.,
analysis, synthesis, or evaluation goals),
creative thinking, writing ability
Open-ended / essay - Cons
•
•
•
•
Hard to score
Can yield great variety of responses
Not efficient to test large bodies of content
If you give the choice of three or four essay
options, you can find out what people know,
but not what they don't know
2. Which type of coffee do you like best?
a. black
b. white
c. latte
d. cappuccino
e. don’t like coffee
Multiple Choice
• What could be measured: abilities, knowledge
personality, prefernces.
• Which scale of measure is used?
• More answer options (4-5) reduce the chance
of guessing that an item is correct
• Difficult to write four or five reasonable
choices
3. Knowledge about psychological tests is
important in psychological practice.
• Strongly disagree
• Disagree
• Neither agree nor disagree
• Agree
• Strongly agree
Likert like scale
• What can be measured: personality, all
constructs assessed with self-report methods.
• Which scale of measure is used?
• Involved in research that employs
questionnaires.
• Named after its inventor, Rensis Likert.
• The range captures the intensity of
respondents feelings for a given item.
4. Mark anywhere along the line to show how
much do you like to wake up at 7 am?
It’s horrible
It’s terrific
Graphic rating scale
•
•
•
•
What can be measured: prefernces.
Which scale of measure is used?
Respones are converted to numerical form.
Also: semantic differential, e.g. hot-cold.
5. Complete:
Toe is to Foot as Finger is to ______?
Gas is to Car as Wood is to ______?
Supplied Response
• What can be measured: knowledge and fact
outcomes, terminology, formulas.
• Which scale of measure is used?
• Chances of guessing reduced.
6. Which object should be next?
Multiple Choice
• With correct and incorrect anwers.
• What can be measured: knowledge, cognitive
performance, abilities.
• Which scale of measure is used?
7. Mark whether it is True (T) or False (F) for
you:
Things have been going very well for me lately.
T F
True/false
• What can be measured: preferences (e.g.
personality), cognitive performance (e.g.
intelligence).
• Can present many items at once
• Easy to score
• Increase the chance of guessing
8. Mark whether you like (L), dislike (D), or are
unsure (?) about the following activity:
Partying with friends. D L ?
Muliple Choice
• Sometimes an even-point scale is used, where
the middle option is not available.
• This is sometimes called a "forced choice"
method
Psychological test - definition
What is a psychological test
• „An evaluative device or procedure in which a sample of
an examinee’s behavior in a specified domain is
obtained and subsequently evaluated and scored using a
standardized process” AERA et al. 1999
• „Test – objective and standardized measure of a sample
of behaviour” Anastasi (1988)
• „A psychological test is a systematic procedure for
obtaining samples of behaviors, relevant to cognitive or
affective functioning, and for scoring and evaluating
those samples according to standards” Urbina (2004)
What is a psychological test
• „An
in which a sample of
an examinee’s behavior in a specified domain is
obtained and subsequently evaluated and scored using a
standardized process” AERA et al. 1999
• „Test – objective and standardized
of a sample
of behaviour” Anastasi (1988)
• „A psychological test is a
for
obtaining samples of behaviors, relevant to cognitive or
affective functioning, and for scoring and evaluating
those samples according to standards” Urbina (2004)
What is a psychological test
• „An evaluative device or procedure in which a sample of
an examinee’s behavior in a specified domain is
obtained and subsequently evaluated and scored
” AERA et al. 1999
• „Test – objective and
measure of a sample
of behaviour” Anastasi (1988)
• „A psychological test is a systematic procedure for
obtaining samples of behaviors, relevant to cognitive or
affective functioning, and for scoring and evaluating
those samples
” Urbina (2004)
What is a psychological test
• „An evaluative device or procedure in which a sample of
an examinee’s behavior in a specified domain is
obtained and subsequently
using a
standardized process” AERA et al. 1999
• „Test –
and standardized measure of a sample
of behaviour” Anastasi (1988)
• „A psychological test is a systematic procedure for
obtaining samples of behaviors, relevant to cognitive or
affective functioning, and for
those samples according to standards” Urbina (2004)
What is a psychological test
What is a psychological test
– EXPLANATION → planning, uniformity and
thoroughness
– RATIONALE → objective and fair to use.
• No mater who is administrating, scoring or interpreting results.
– EXPLANATION → small (but significant) subsets of
a larger whole.
– RATIONALE → efficiency
• Time, resources and cognitive abilities (of test taker and administrator) are limited.
What is a psychological test
– EXPLANATION → way of assessing test results
(applying common yardstick:)
– RATIONALE → providing meaning to the test results
• criteria, comparison between people
– EXPLANATION → some sort of categorical or
numerical system of understanding test results based
on preestablished rules.
– RATIONALE → providing base for any sort of inference