Psychology and Human Ecology
School of Education and Behavioral Sciences
B.S. in Psychology
CIP Code: 420101
Program Code: 165
Program Quality Improvement Report 2009-2010
1
Student-Learning Outcomes
1. Demonstrate knowledge in research
methods.
2. Application of knowledge in research
methods.
3. Demonstrate knowledge in clinical
areas.
4. Demonstrate knowledge in
experimental areas.
Program Quality Improvement Report 2009-2010
2
Alignment of Outcomes
to University Mission Statement
CU provides a diverse and dynamic student body
access to quality educational opportunities; fosters a
student-centered academic environment that combines
innovative classroom teaching with experiential
learning; prepares students for professional success,
responsible citizenship, life-long learning, and
meaningful contributions to a rapidly changing world;
and is a driving force in the cultural life and economic
development of the region.
Program Quality Improvement Report 2009-2010
3
Alignment of Outcomes (cont.)
• Combines innovative classroom teaching with
experiential learning
• Application of knowledge in research methods
• Prepares students for professional success
•
•
•
•
Demonstrate knowledge in research methods
Application of knowledge in research methods
Demonstrate knowledge in clinical areas
Demonstrate knowledge in experimental areas
• Life-long learning, and meaningful
contributions to a rapidly changing world
• Demonstrate knowledge in research methods
• Application of knowledge in research methods
Program Quality Improvement Report
2009-2010
4
Alignment of Outcomes
to Department Mission Statement
Psychology is the scientific study of behavior. Through
courses in psychology, students will learn principles which
govern human behavior. The courses offered reflect the
diversity in psychology and meet the educational
objectives of at least three groups of students:
• Those who seek a general survey course in the field of
psychology or those who wish to study a program of
psychology in depth at the undergraduate level but do
not contemplate entrance into psychology as a
profession
• Those who wish to enter psychology as a profession
immediately after the BS
• Those who plan to attend graduate school and then
enter psychology as a profession
Program Quality Improvement Report
2009-2010
5
Alignment of Outcomes (cont.)
• Those who wish to enter psychology as a
profession immediately after the BS
–
–
–
–
Demonstrate knowledge in research methods.
Application of knowledge in research methods.
Demonstrate knowledge in clinical areas.
Demonstrate knowledge in experimental areas.
• Those who plan to attend graduate school and
then enter psychology as a profession
–
–
–
–
Demonstrate knowledge in research methods.
Application of knowledge in research methods.
Demonstrate knowledge in clinical areas.
Demonstrate knowledge in experimental areas.
Program Quality Improvement Report
2009-2010
6
Alignment of Outcomes
to Strategic Plan 2013
Goal 1.1: Maintain and enhance Cameron’s
commitment to providing programs of the highest
quality in instruction, research, and service to
better meet the needs of the citizens of the region
• Lawton United Way Needs Assessment
– Demonstrate knowledge in clinical areas.
– Demonstrate knowledge in research methods.
– Application of knowledge in research methods.
Program Quality Improvement Report
2009-2010
7
Measures of Learning Outcomes
1.
Direct measures of student learning outcomes
–
ACAT Score
•
•
•
•
•
•
•
•
–
Locally Developed Test Score
•
•
•
•
•
•
•
•
•
•
•
–
Statistics
Experimental
Abnormal
Personality
Developmental
History
Social
Learning
Statistics
Experimental
Application
Abnormal
Personality
Counseling and Clinical
Developmental
History
Social
Learning
Cognitive
Performance on Psychological Research Course Project
Program Quality Improvement Report 2009-2010
8
Measures of Learning Outcomes (cont.)
2. Indirect measures of student learning
outcomes
–
Exit interviews
3. Strategies that address shortfalls in student
learning or services prior to graduation
–
Students who answer less than 70% of the items
correctly on the locally developed tests will be
identified as “deficient”
Faculty are considering:
–
•
•
Offering deficient undergraduates tutoring by
graduate students
Offering a one-hour senior capstone course in which
deficiencies are cleared
Program Quality Improvement Report
2009-2010
9
Report on actions from the three previously
chosen priority outcomes
1. Action plans for demonstrating knowledge in
research methods
–
All students must take ACAT and Exit Interview before
their final degree check.
•
•
•
–
Too late for this year, but is currently being strictly enforced
ACAT offered on-line for ease in scheduling
Computers in Psychology office/laptop so the exit interview
and ACAT can be conveniently administered
Departmental members will examine locally
developed tests to determine if they need to be
modified.
•
This was done and a few changes were made (e.g.,
Experimental Psychology)
Program Quality Improvement Report 2009-2010
10
Report on actions from the three previously
chosen priority outcomes (cont.)
2. Action plans for applying knowledge in research
methods
—
Faculty will examine the locally developed Applications Test to
determine if it needs to be modified.
—
This was done; the test was judged to have face validity so it was not
modified.
— Psychological Research instructor provided “training” among raters
prior to the Psychological Research course project presentations to
improve inter-rater reliability (.21).
— The inter-rater reliability was .55.
Program Quality Improvement Report
2009-2010
11
Report on actions from the three previously
chosen priority outcomes (cont.)
3. Action plans for demonstrating knowledge in
clinical areas
–
•
•
•
–
•
–
•
All students must take ACAT and Exit Interview before their final degree
check.
Too late for this year, but is currently being strictly enforced
ACAT offered on-line for ease in scheduling
Computers in Psychology office so the exit interview and ACAT can
be conveniently administered
Departmental members will examine the locally developed Abnormal
test to determine if it needs to be modified.
This was done and the test was judged to have face validity; it was
not modified.
Faculty will create an assessment test to measure knowledge of
Personality and Counseling and Clinical
These tests were created and data from the tests are presented in
this report.
Program Quality Improvement Report 2009-2010
12
Student-learning outcome and measurements
MEASUREMENTS OF STUDENT LEARNING OR SERVICE OUTCOME
PROGRAM
OUTCOME
Application of
knowledge in
research
methods
CURRICULUM
AREA OR TARGET
AUDIENCE
PSY 3423, 4423,
and 4433
Measurements
1. Score on locally
developed
Applications Test
(direct)
2. Performance on
Psychological
Research course
project
presentation
(direct)
3. Student exit
interview
(indirect)
Methods used to
determine validity
of measurement
instruments
Methods used
to determine
reliability of
measurements
Only face validity is
assessed
Chronbach’s
alpha
Annual
Inter-rater
reliability
Inter-rater
reliability
Fall and Spring
semesters
Only face validity is
assessed
None
Annual
Program Quality Improvement Report 2009-2010
Schedule for
measurements
13
Mean Performance on Locally Developed
Applications Test (n = 10)
Item
Mean
Standard Deviation
Rating
Identify IVs
52%
30.20
Unsatisfactory
Identify DVs
80%
42.20
Satisfactory
Identify statistical tests
90%
31.60
Satisfactory
Describe results
73%
24.06
Satisfactory
73.60%
21.08
Satisfactory
Total
On average, the students are performing satisfactorily on the locally developed
applications test, in which students are asked to read a psychology journal
article, identify the key components of the study, and interpret the results.
However, students are having difficulty identifying independent variables.
Program Quality Improvement Report
2009-2010
14
Performance on Locally Developed
Applications Test (n = 10)
Item
< 70%
70% < X < 90%
> 90%
Identify IVs
90%
(2- 0%; 7 – 60%)
0%
10%
(1 – 100%)
Identify DVs
20%
(2 – 0%)
0%
80%
(8 – 100%)
Identify statistical tests
10%
(1 – 0%)
0%
90%
(9 – 100%)
Describe results
30%
50%
20%
Total
30%
50%
20%
Program Quality Improvement Report
2009-2010
15
Mean Performance on Locally Developed
Applications Test Over Time
Program Quality Improvement Report
2009-2010
16
Mean Performance on Psychological Research
Course Project (n = 23)
Section
Average Rating
Standard Deviation
Performance
Abstract
4.02
.99
Outstanding
Background
3.69
1.18
Satisfactory
Methodology
3.86
1.12
Satisfactory
Results
3.95
1.11
Satisfactory
Discussion
3.48
1.14
Satisfactory
Presentation
3.76
1.07
Satisfactory
Overall Quality
3.92
.87
Satisfactory
On average, the students are performing satisfactorily on the Psychological
Research course project, in which the students perform a literature review, create
hypotheses, design a study, collect data, analyze the data, interpret their results,
and present their study in a poster at a “mini-conference” to students and faculty.
Program Quality Improvement Report
2009-2010
17
Mean Performance on Psychological
Research Course Project Over Time
Section
2009-2010
(n = 23)
2008-2009 2007-2008 2006-2007
(n = 30)
(n = 13)
Abstract
4.02
4.23
3.89
4.08
Background
3.69
3.93
3.77
3.50
Methodology
3.86
4.63
3.88
3.75
Results
3.95
4.37
3.94
4.58
Discussion
3.48
3.96
3.85
3.75
Presentation
3.76
3.97
4.14
4.33
Overall Quality
3.92
4.07
3.97
4.33
Over time, the students have performed satisfactorily on the
Psychological Research course project.
Program Quality Improvement Report
2009-2010
18
Exit interview
• Three items assessed knowledge in research methods using
seven point Likert-format scales from “low” to “high”
– Rate how well the program promoted competence in
research skills
– Rate how well the program prepared you (provided you
knowledge) in the area of Statistics
– Rate how well the program prepared you (provided you
knowledge) in the area of Experimental
• T-tests were performed to compare the average score with
the midpoint of the scale.
– Average responses that are significantly below or equal to
the midpoint will be unsatisfactory
– Average responses that are significantly above the
midpoint of the scale will be satisfactory
Program Quality Improvement Report
2009-2010
19
Display Assessment Data from Exit Interview
Item
Mean
t-test
Significance Level
Research Skills
5.44
4.36
p < .01
Statistics
6.19
7.89
p < .01
Research Design
5.80
4.89
p < .01
N = 16
On average, students believe the program prepared them in Statistics and Research,
and that they are competent researchers.
Program Quality Improvement Report
2009-2010
20
Exit Interview Over Time
Research
Statistics
Experimental
7
6
5
4
3
2
1
2002-3
2003-4
2006-7
2007-8
Program Quality Improvement Report
2009-2010
2008-9
2009-10
21
Action Plan
• To improve students’ ability to identify
independent variables in research studies
– Instructors of Applied Quantitative Methods
will consider adding assignments focused
on this skill
– This skill will be assessed in Psychological
Research in addition to Applied
Quantitative Methods
Program Quality Improvement Report
2009-2010
22
Student-learning outcome and measurements
MEASUREMENTS OF STUDENT LEARNING OR SERVICE OUTCOME
PROGRAM
OUTCOME
Demonstrate
knowledge in
clinical areas
CURRICULUM
AREA OR
TARGET
AUDIENCE
Upper division
psychology
courses,
especially PSY
4363, 4393,
and 3333
Measurements
Methods used to
determine validity
of measurement
instruments
Methods used
to determine
reliability of
measurements
1. Score on ACAT
Abnormal (direct)
Norm-referenced
scores
ACAT publishes
its reliability
score
Annual
2. Score on locally
developed Abnormal
Test (direct)
Correlations with
standardized test
score (ACAT
Abnormal)*
Chronbach’s
alpha
Annual
ACAT publishes
its reliability
score
Annual
3. Score on ACAT
Personality (direct)
4. Score on locally
developed
Personality Test
(direct)
Norm-references
scores
Schedule for
measurements
Correlations with
standardized test
score (ACAT
Personality)*
Chronbach’s
alpha
Annual
5. Score on locally
developed
Counseling and
Clinical Test (direct)
Only face validity is
assessed
None
Annual
6. Student exit
interview (indirect)
Only face validity is
assessed
None
Annual
Program Quality Improvement Report 2009-2010
*ns
23
A little bit about Area Concentration
Achievement Tests (ACAT)
• The ACATs are produced as part of a national project; the
project assists departments with outcome assessment
• The ACAT is used at four-year institutions nationwide
• Both public and private institutions are participants
• The ACAT uses a random sample of the items submitted in
each area
• Scores range from 200 to 800, however, we will report the
standardized scores allowing for easy comparison with
national norms
• Crain (1989) and Markus, Mukina, & Golden (1996) have
established that the ACAT is reliable and valid
Program Quality Improvement Report
2009-2010
24
Scores on ACAT Personality Test
Unsatisfactory
(z < -1)
Satisfactory
(-1 < z < 1)
Outstanding
(z > 1)
N
0
9
0
%
0
100
0
Three seniors did not take PSY 4393 Personality at CU.
All students who took the ACAT performed satisfactorily.
Program Quality Improvement Report
2009-2010
25
Scores on ACAT Personality Test Over Time
1.5
1
0.5
0
-0.5
-1
-1.5
2000- 2001- 2002- 2003- 2004- 2005- 2007- 2008- 20091
2
3
4
5
6
8
9
10
Scores are consistently satisfactory.
Program Quality Improvement Report
2009-2010
26
Scores on Locally Developed Personality Test
Unsatisfactory
(< 70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
33
1
0
%
97.06
2.94
0.00
N
8
18
9
%
22.86
51.43
25.71
Pretest
Posttest
t(33) = 6.70, p < .01
M-posttest = 78.00%, s = 19.82
Program Quality Improvement Report
2009-2010
27
Scores on ACAT Abnormal Test
Unsatisfactory
(z < -1)
Satisfactory
(-1 < z < 1)
Outstanding
(z > 1)
N
2
8
1
%
18.18
72.72
9.09
One senior did not take PSY 4363 Abnormal at CU.
The majority of students who took the ACAT performed satisfactorily.
Program Quality Improvement Report
2009-2010
28
Scores on ACAT Abnormal Test Over Time
1.5
1
0.5
0
-0.5
-1
-1.5
2000- 2001- 2002- 2003- 2004- 2005- 2007- 2008- 20091
2
3
4
5
6
8
9
10
(n=40) (n=23) (n=41) (n=49) (n=37) (n=34)
(n=19) (n=25) (n=12)
Scores are consistently satisfactory.
Program Quality Improvement Report
2009-2010
29
Scores on Locally Developed Abnormal Test
Unsatisfactory
(< 70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
35
2
0
%
94.59
5.41
0
N
8
16
12
%
22.22
44.44
33.33
Pretest
Posttest
t(34) = 13.66, p < .01
M-posttest = 81.94%, s = 17.70
Program Quality Improvement Report
2009-2010
30
Mean Performance on Abnormal Locally
Developed Test Over Time
Program Quality Improvement Report
2009-2010
31
Scores on Locally Developed Counseling and
Clinical Test
Unsatisfactory
(< 70% Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
38
5
0
%
88.37
11.63
0
N
6
24
12
%
14.29
57.14
28.57
Pretest
Posttest
t(38) = 12.47, p < .01
M-posttest = 83.10%, s = 17.18
Program Quality Improvement Report
2009-2010
32
Exit Interview
• Two items assessed knowledge in clinical areas using seven
point Likert-format scales from “low” to “high”
– Rate how well the program prepared you (provided you
knowledge) in the area of Abnormal
– Rate how well the program prepared you (provided you
knowledge) in the area of Personality
• T-tests were performed to compare the average score with
the midpoint of the scale.
– Average responses that are significantly below or equal to
the midpoint will be unsatisfactory
– Average responses that are significantly above the
midpoint of the scale will be satisfactory
Program Quality Improvement Report
2009-2010
33
Display Assessment Data from Exit Interview
Item
Mean
t-test
Significance Level
Abnormal
5.88
6.54
p < .01
Personality
5.81
6.54
p < .01
N = 16
On average, the students believe the program prepared
them in Abnormal and Personality.
Program Quality Improvement Report
2009-2010
34
Exit Interview Over Time
Abnormal
Personality
7
6
5
4
3
2
1
2002-3
2003-4
2006-7
2007-8
Program Quality Improvement Report
2009-2010
2008-9
2009-10
35
Action Plan
• To increase performance on locally developed posttests
– Perform an item analysis to determine which areas are frequently
missed and address these in lectures
• To increase the number of seniors that complete the ACAT and the Exit
Interview
– Require students complete both before they are given their degree
check
• Offer the ACAT on-line for ease in scheduling
• Offer exit interview on-line and in hard copies
– Send e-mail stating they must complete both before they can enroll in
their last semester
• To measure the validity of the locally developed tests, we should perform
correlations between ACAT scores and locally developed tests. However,
for each area, there were few students that took both the locally
developed test and the ACAT
Program Quality Improvement Report
2009-2010
36
Student-learning outcome and measurements
MEASUREMENTS OF STUDENT LEARNING OR SERVICE OUTCOME
PROGRAM
OUTCOME
Demonstrate
knowledge in
experimental
areas
CURRICULUM
AREA OR TARGET
AUDIENCE
Measurements
Psychology
courses,
especially PSY
2113, 3353,
3413, 3313, 3383
1. Score on ACAT
for History,
Developmental,
Learning, and
Social (direct)
2. Score on locally
developed History,
Developmental,
Learning, and
Social , and
Cognitive Tests
(direct)
3. Student exit
interview
(indirect)
Methods used to
determine validity
of measurement
instruments
Methods used
to determine
reliability of
measurements
Norm-referenced
scores
ACAT publishes
its reliability
score
Annual
Correlations with
standardized test
score (ACAT tests)*
Chronbach’s
alpha
Annual
None
Only face validity is
assessed
Schedule for
measurements
Annual
*ns
Program Quality Improvement Report 2009-2010
37
Scores on ACAT History Test
Unsatisfactory
(z < -1)
Satisfactory
(-1 < z < 1)
Outstanding
(z > 1)
N
2
10
0
%
16.67
83.33
0.00
The majority of students who took the ACAT performed satisfactorily.
Program Quality Improvement Report
2009-2010
38
Scores on ACAT History Test Over Time
(n=40) (n=23) (n=41) (n=49) (n=37) (n=34) (n=19) (n=25) (n=12)
Scores are consistently satisfactory.
Program Quality Improvement Report
2009-2010
39
Scores on Locally Developed History Test
Unsatisfactory
(< 70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
21
5
0
%
80.77
19.23
0.00
N
11
13
0
%
45.83
54.17
0.00
Pretest
Posttest
t(23) = 3.50, p < .01
M-posttest = 64.58%, s = 15.60
Program Quality Improvement Report
2009-2010
40
Scores on ACAT Developmental Test
Unsatisfactory
(z < -1)
Satisfactory
(-1 < z < 1)
Outstanding
(z > 1)
N
3
5
1
%
33.33
55.56
11.11
Three seniors did not take PSY 3353
Lifespan Human Growth and Development at CU.
The majority of students who took the ACAT performed satisfactorily.
Program Quality Improvement Report
2009-2010
41
Scores on ACAT Developmental Test Over Time
(n=40) (n=23) (n=41) (n=49) (n=37) (n=34) (n=19) (n=25) (n=12)
Scores are consistently satisfactory.
Program Quality Improvement Report
2009-2010
42
Scores on Locally Developed Developmental Test
Unsatisfactory
(< 70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
74
2
0
%
97.37
2.63
0.00
N
26
40
6
%
34.67
53.33
8.00
Pretest
Posttest
t(64) = 12.09, p < .01
M-posttest = 70.93%, s = 18.10
Program Quality Improvement Report
2009-2010
43
Scores on ACAT Learning Test
Unsatisfactory
(z < -1)
Satisfactory
(-1 < z < 1)
Outstanding
(z > 1)
N
4
5
1
%
40.00
50.00
10.00
Two seniors did not take PSY 3413 Psychology of Learning at CU.
The majority of students who took the ACAT performed satisfactorily.
Program Quality Improvement Report
2009-2010
44
Scores on ACAT Learning Test Over Time
(n=40) (n=23) (n=41) (n=49) (n=37) (n=34) (n=19) (n=25) (n=12)
Scores are consistently satisfactory.
Program Quality Improvement Report
2009-2010
45
Scores on Locally Developed Learning Test
Unsatisfactory
(< 70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
17
1
0
%
94.44
5.56
0.00
N
13
5
0
%
72.22
27.78
0.00
Pretest
Posttest
t(17) = 1.65, p > .05
M-posttest = 54.44%, s = 16.17
Program Quality Improvement Report
2009-2010
46
Scores on ACAT Social Test
Unsatisfactory
(z < -1)
Satisfactory
(-1 < z < 1)
Outstanding
(z > 1)
N
4
5
0
%
44.44
55.56
0.00
Three seniors did not take PSY 3383 Social Psychology at CU.
The majority of students who took the ACAT performed satisfactorily.
Program Quality Improvement Report
2009-2010
47
Scores on ACAT Social Test Over Time
(n=40) (n=23) (n=41) (n=49) (n=37) (n=34) (n=19) (n=25) (n=12)
Scores are consistently satisfactory.
Program Quality Improvement Report
2009-2010
48
Scores on Locally Developed Social Test
Unsatisfactory
(< 70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
25
4
0
%
86.21
13.79
0.00
N
3
13
15
%
9.68
41.94
48.39
Pretest
Posttest
t(21) = 10.36, p < .01
M-posttest = 88.71%, s = 14.32
Program Quality Improvement Report
2009-2010
49
Scores on Locally Developed Cognitive Test
Unsatisfactory
(< 70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
38
5
0
%
88.37
11.63
0.00
N
7
19
3
%
24.13
65.52
10.34
Pretest
Posttest
t(28) = 6.72, p < .01
M-posttest = 77.93%, s = 17.80
Program Quality Improvement Report
2009-2010
50
Exit Interview
• Four items assessed knowledge in clinical areas using seven point Likertformat scales from “low” to “high”
– Rate how well the program prepared you (provided you knowledge) in
the area of History and Systems
– Rate how well the program prepared you (provided you knowledge) in
the area of Developmental Psychology
– Rate how well the program prepared you (provided you knowledge) in
the area of Animal Learning
– Rate how well the program prepared you (provided you knowledge) in
the area of Social Psychology
• T-tests were performed to compare the average score with the midpoint
of the scale.
– Average responses that are significantly below or equal to the
midpoint will be unsatisfactory
– Average responses that are significantly above the midpoint of the
scale will be satisfactory
Program Quality Improvement Report
2009-2010
51
Display Assessment Data from Exit Interview
Item
Mean
t-test
Significance Level
History and Systems
5.40
3.00
p < .01
Developmental
5.00
2.11
p < .05
Animal Learning
3.31
.69
p > .05
Social
6.38
13.22
p < .01
N = 16
On average, the students believe the program prepared
them in History and Systems, Developmental, and Social Psychology.
However, students do not believe the program prepared
them in Animal Learning.
Program Quality Improvement Report
2009-2010
52
Exit Interview Over Time
History
Devel
Learn
Social
7
6
5
4
3
2
1
2002-3
2003-4
2006-7
2007-8
Program Quality Improvement Report
2009-2010
2008-9
2009-10
53
Action Plan
• To increase the number of seniors that
complete the ACAT and the Exit Interview
– Require students complete both before they are
given their degree check
• Offer the ACAT on-line for ease in scheduling
• Offer exit interview on-line and in hard copies
– Send e-mail stating they must complete both
before they can enroll in their last semester
• To increase performance in Social posttest
– Instructor will determine which items were missed
on the examination and stress these areas in
lectures
Program Quality Improvement Report 2009-2010
54
Ancillary Actions
•
In the spring semester of 2010, we began creating an excel spread sheet in which
each student’s pretest and posttest scores are recorded for each course in which
we have developed an assessment test
–
–
•
Although the students are performing satisfactorily in the objective of application
of research knowledge, the following steps may improve students performance
–
–
•
•
This will allow us to compute correlations between ACAT subtest scores and performance on locally
developed tests
We are considering whether we should add course grades to this spreadsheet in order to examine
additional correlations
Better advisement such that students take the research sequence earlier in the program
• Disallow concurrent enrollment of Experimental Psychology and Psychological Research;
however, drop the prerequisite of Applied Quantitative Methods for Experimental Psychology
• UNIV 1001 course for majors will discuss the necessity of completing the research sequence
early
• Continue to offer Applied Quantitative Methods every semester including the summer
• Consider offering Experimental Psychology in both day and night sections when it is not offered
on-line
Consider adding a one hour senior capstone course in which deficiencies in this area are cleared
through practice
History of Psychology and Psychology of Learning posttests will be embedded in
the final examinations
Exit interview will be revised so it asks about Learning and not Animal Learning
Program Quality Improvement Report 2009-2010
55
Published information on graduates
Academic Year 09-10
Entered Graduate School
Working In Discipline
Other
Summer 2009 (n = 10)
Fall 2009 (n = 14)
4 - MSBS at Cameron
Spring 2010 (n = 18)
7 – MSBS at Cameron
Total (n = 42)
11 – MSBS at Cameron
Program Quality Improvement Report 2009-2010
56
These are additional slides that will
not be shared, however, they
present data for the other learning
objective.
Program Quality Improvement Report
2009-2010
57
Student-learning outcome and measurements
MEASUREMENTS OF STUDENT LEARNING OR SERVICE OUTCOME
PROGRAM
OUTCOME
Demonstrate
knowledge in
research
methods
CURRICULUM
AREA OR TARGET
AUDIENCE
Upper division
psychology
courses,
especially PSY
3423, 4423, and
4433
Measurements
Methods used to
determine validity
of measurement
instruments
Methods used
to determine
reliability of
measurements
1. Score on ACAT
Statistics (direct)
Norm-referenced
scores
ACAT publishes
its reliability
score
Annual
2. Score on locally
developed
Statistics Test
(direct)
Correlations with
standardized test
score (ACAT
Statistics)*
Chronbach’s
alpha
Annual
3. Score on ACAT
Experimental
(direct)
Norm-references
scores
ACAT publishes
its reliability
score
Annual
4. Score on locally
developed
Experimental Test
(direct)
Correlations with
standardized test
score (ACAT
Experimental)*
Chronbach’s
alpha
Annual
5. Student exit
interview
(indirect)
Only face validity is
assessed
none
Annual
Program Quality Improvement Report 2009-2010
Schedule for
measurements
*ns
58
Scores on ACAT Statistics Test
Unsatisfactory
(z < -1)
Satisfactory
(-1 < z < 1)
Outstanding
(z > 1)
N
5
7
0
%
41.67
58.33
0
Program Quality Improvement Report
2009-2010
59
Scores on ACAT Statistics Test Over Time
1.5
1
0.5
0
-0.5
-1
-1.5
2000- 2001- 2002- 2003- 2004- 2005- 2007- 2008- 20091
2
3
4
5
6
8
9
10
(n=40) (n=23) (n=41) (n=49) (n=37) (n=34) (n=19) (n=25) (n=12)
Scores are consistently satisfactory.
Program Quality Improvement Report
2009-2010
60
Scores on Locally Developed Statistics Test
Unsatisfactory
(< 70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
10
2
0
%
83.33
16.67
0.00
N
3
4
3
%
30.00
40.00
30.00
Pretest
Posttest
t(9) = 4.33, p < .01
M-posttest = 76.00%, s = 20.66
Program Quality Improvement Report
2009-2010
61
Scores on Locally Developed Statistics Test Over Time
Program Quality Improvement Report
2009-2010
62
Scores on ACAT Experimental
Unsatisfactory
(z < -1)
Satisfactory
(-1 < z < 1)
Outstanding
(z > 1)
N
2
8
1
%
18.18
72.72
9.09
One senior did not take PSY 4423 Experimental Psychology at CU.
Program Quality Improvement Report
2009-2010
63
Scores on ACAT Experimental Test Over Time
1.5
1
0.5
0
-0.5
-1
-1.5
2000- 2001- 2002- 2003- 2004- 2005- 2007- 2008- 20091
2
3
4
5
6
8
9
10
(n=40) (n=23) (n=41) (n=49) (n=37) (n=34) (n=19) (n=25) (n=12)
Since 2002-3, scores are consistently satisfactory.
Program Quality Improvement Report
2009-2010
64
Scores on Locally Developed Experimental Test
Unsatisfactory
( <70 % Correct)
Satisfactory
(70 < % Correct < 90)
Outstanding
(% Correct > 90)
N
16
0
0
%
100.00
0.00
0.00
N
7
8
1
%
43.75
50.00
6.25
Pretest
Posttest
t(13) = 7.53, p < .01
M-posttest = 67.50%, s = 20.82
Program Quality Improvement Report
2009-2010
65
Scores on Locally Developed Experimental Test
Over Time
Program Quality Improvement Report
2009-2010
66