Assessment Report
July 1, 2010 – June 30, 2011
Program(s) Assessed: Psychology B.A., B.S., Minor
Assessment Coordinator: Martin P. Gooden, Ph.D.
I. Assessment measures employed
A. Learning objectives for both the Bachelor of Arts (B.A.) and Bachelor of Science
(B.S.) degrees were assessed this academic year via an online survey service
called SurveyMonkeyTM. As in previous years, psychology majors rated the
degree to which the undergraduate curriculum provided opportunities to meet
each learning objective. Links to the survey were emailed to seniors taking their
second capstone (Psychology 487) course. Seventy-one majors responded.
B. Program outcomes were also assessed measuring enrollment in post-graduate
education, employment status, and the application of knowledge of psychology to
their life experiences. Links to the survey were mailed to psychology graduates
following summer, fall, winter and spring graduation. Five graduates responded.
C. The degree to which students in advanced courses demonstrated previously
acquired knowledge, skills, and abilities was assessed via surveys administered to
faculty instructing advanced special topic research methods courses for 54 B.S.
students and the second capstone course for 137 seniors.
D. The degree to which the curriculum provided Psychology minors opportunities to
gain knowledge of psychology and to apply that knowledge was assessed via
surveys mailed to minors following graduation. No minors responded.
II. Assessment findings
A. Graduates earning either a B.A. or B.S. degree in psychology were asked to
indicate how much (1 = not at all through 5 = very much) the curriculum provided
opportunities to achieve each program learning objective (N=71).
Objective 1: Be familiar with current theory and research in diverse areas of
psychology.
Response: Nearly 74% of the respondents answered 4 or 5 (M=4.06, SD= 0.79)
indicating they agreed the curriculum provided opportunities to accomplish this
outcome. Mean comparisons from the previous year (2009-2010) indicate a
statistically significant decrease (M=4.41, SD= 0.70, N=98), t(167)=-2.25, p<.03
for this objective.
Outcome 2: Have fundamental research design and mathematical/statistical skills
needed to understand psychological science.
Response: Approximately 70% of the respondents answered 4 or 5 (M=4.03,
SD=0.78) indicating they agreed the curriculum provided opportunities to
accomplish this outcome. Mean comparisons from 2009-2010 indicate no
significant change (M=4.29, SD= 0.84), t(166)=-1.66, p>.09.
Outcome 3: Have skills in integrating and communicating about knowledge in
self-selected areas of psychology.
Response: Approximately 72% of the respondents answered 4 or 5 (M=4.36,
SD=0.79) indicating they agreed the curriculum provided opportunities to
accomplish this outcome. Mean ratings from 2009-2010 show no change
(M=4.56, SD= 0.63, N=98), t(166)=-1.28, p>.20.
Outcome 4: Have skills in effective oral and written communication.
Response: About 77% of the respondents answered 4 or 5 (M=4.19, SD=0.84)
indicating they agreed the curriculum provided opportunities to accomplish this
outcome. Comparisons to mean ratings from 2009-2010 showed a statistically
significant decrease (M=4.53, SD= 0.65, N=98), t(166)=-2.17, p<.03.
Outcome 5: (B.S. students only, N=25): Have advanced research design,
mathematical/statistical, and computing skills needed to critically evaluate and
conduct research in a self-selected area of interest.
Response: Only 68% of respondents answered 4 or 5 (M=3.59, SD=.76)
indicating they agreed the curriculum provided opportunities to accomplish this
outcome. Mean ratings from 2009-2010 reveal no significant change (M=3.78,
SD= 1.10, N=40), t(63)=-.74, p>.46.
Suggestions for improvements:
Over this past year, performance in Learning Outcomes 1 through 4 has revealed
declines particularly in Outcome #1 and #4. The declines were in contrast to the
trends observed from the 2009 to 2010 academic year. During that period,
performance across Outcomes #1 through #4 showed evidence of improvements
(particularly in Outcome #3 and #4). Learning Outcome 5, however, shows
evidence of a continual decline from 2007 to the present. As can be seen in Table
1 below, across the past 4 years, student evaluations of the development of skills
in advanced research design, math, statistics, and computing skills have declined a
full point (an average of 4.65 in 2007 to 3.59 in 2011). This trend suggests that
either (1) BS students are failing to recognize the aims of the course, (2) faculty
instruction requires greater clarity or coordination, or (3) some combination of the
two. To address these 3 possibilities, faculty who teach the advanced method’s
courses will be invited to review these findings and develop a strategy to curb this
decline. One recommendation would be to better coordinate learning experiences
in the basic method’s courses (PSY301, 302, and 303) with the advanced
method’s courses. Another recommendation would be greater coordination
among the advanced method’s instructors themselves. With such changes and a
well-developed plan, it is hopeful that this decline can be stemmed by the next
assessment period.
Table 1: Average Ratings of Curriculum Outcomes 2007-2011
2007-2008
2008-2009
2009-2010
2010-2011
Outcome #1
Outcome #2
Outcome #3
Outcome #4
Outcome #5
4.44
4.22
4.41
4.06
4.22
4.08
4.29
4.03
4.45
4.36
4.56
4.36
4.48
4.28
4.53
4.19
4.65
4.10
3.78
3.59
In addition to the declines in the courses that focus of research methodology (B.S.
students, Outcome #5), students this past year also appeared less satisfied with the
quality of instruction across core theory courses (e.g., Abnormal, Developmental,
Cognition, etc.). The reasons for this decline are more uncertain. Speculating,
student expectations regarding what they believe they need from these courses
may not be consistent with what is being offered. A common refrain from
students seeking academic advising suggests that confusion regarding their course
offerings is common. Once enrolled in a course, students may question the
relevance of the material, the approach in which the course is taught, the amount
of work required, etc. Lower levels of satisfaction may simply reflect the
changing expectation of students. Admittedly, to the extent that faculty have not
responded to these changing expectations, the manner of instruction may also
contribute to these findings. Since these assumptions are only speculative, faculty
will be invited to review these findings and consider ways to curb the decline in
student attitudes.
One of the most valuable skills that students learn while in college is how to
effectively communicate. Psychology majors are provided with 4 opportunities to
develop their writing and public speaking skills within the major. Four courses
are designated officially as writing intensive, while others may involve writing
assignments. Students are also actively encouraged to “speak up” in class,
particularly those courses with instructors who insist on active participation.
Thus, there is no shortage of opportunities for students to develop these
communication skills. Speculating on the decline, next to math, the development
of writing skills is one of the most challenging goals that are established for
students. Performance is driven not only by lots of practice by students,
consistent, structured, and repeated supervision by faculty is also important.
Comments from students have suggested that variation in the rigor required by
one faculty member to the other in writing papers is frustrating. This frustration
may explain the decline from 2009-2010 to the present. Upon reviewing these
assessment findings, faculty who teach writing courses will be encouraged to
consider steps that if taken enhance students’ perception of value, while
preserving quality instruction.
Open-ended Feedback
Similar to previous years, student open-ended comments cover a wide array of
topics. In general students are asking for more advance courses that involve
topics related to mental health (e.g., clinical psychology). Students also want
more courses that will help develop effective writing and public speaking skills.
Practicum and research experiences are also desired so that student can develop
tangible and marketable skills in areas more closely aligned with their interests.
These requests are consistent with previous years. To meet part of this demand
the department is introducing a greater variety of advanced capstone courses
(N=42) in preparation for the semester conversion in Fall 2012. A more
coordinated effort will also be directed toward encouraging participation in the
practicum and research experiences. The department of psychology is also
employing more personnel (graduate students/adjunct faculty) from Wright
State’s School of Professional Psychology. Their expertise and background in the
areas of clinical and counseling psychology will provide psychology majors with
greater exposure to mental health professionals in an academic setting. Over
time, this should help satisfy student interest by providing graduate student and
professional role models for which undergraduates can both emulate as well as
seek advice.
Shifting towards the BS
As noted in previous assessments, the trend towards the BS degree appears to be
continuing. An examination of the number of students enrolled within the major,
and the distribution across the 2 degree-types (B.A. vs. B.S.) reveals a continual
increase in the number of students pursuing the B.S. degree (see Table 2). In just
the past year, an 8% increase in the number of BS students was observed.
Table 2.
Qtr/YR
B.A.
F06
F07
F08
F09
F10
F11
410
329
279
259
309
273
B.A.
Intent
49
30
49
44
75
37
B.A.
Total
459
359
328
303
384
310
B.S.
61
157
234
299
327
355
B.S.
Intent
118
146
183
160
152
189
B.S.
Total
179
303
417
459
479
544
Total
638
662
745
762
863
854
Ratio B.S. to
B.A. Majors
12.95%
32.30%
45.61%
53.58%
51.42%
56.52%
Ratio B.S. to
B.A. Total
28.06%
45.77%
55.97%
60.24%
55.50%
63.70%
With the increasing shift from the BA to the BS degree, undergraduates who are
interested in the B.S. degree have several concentrations for which they can
pursue their interests in psychology. Students can elect Behavioral Neuroscience,
Cognition and Perception, and Industrial and Organizational Psychology. The
requirements for students eager to pursue these areas are rigorous, but each offers
dynamic opportunities sure to make Wright State students competitive for
graduate programs as well as the work force.
B. Findings regarding B.A. and B.S. program outcomes based on the post-graduation
survey.
Graduate School Enrollment (N=69): 7.25% or 5 students reported they are
currently enrolled in or have been admitted to a graduate program or a
professional school. This figure is comparable to those who reported being
admitted to a graduate program in the 2009-2010 calendar year (6.25% or 6
students from a total N=97, z=.26, p>.7).
Employment (N=70): 7.14% or 5 students reported they had received and
accepted an employment offer. This represents a slight but non-statistically
significant decline from the 8 students (8.24%, N=97) who reported they had
received and accepted an employment offer during 2009-2010 (z=.26, p>.7).
Application of Knowledge (N=70): 58.67% or 41 respondents answered 4 or 5
(M=3.81 SD=.98) indicating they agreed they have been able to apply their
knowledge of psychology to the issues they have confronted in education,
employment, and interpersonal relations. A comparison of the mean ratings from
2009-2010 indicate that the increase is not statistically significant (M=3.97, SD=
1.03, N=95, t(163)=-1.02, p>.31).
C. Findings regarding learning outcomes based on faculty rating the degree (1= not
at all; 3= satisfactory; 5=excellent) to which each student demonstrated specific
knowledge, skills, and abilities (KSA) acquired in previous courses.
Faculty instructing five advanced special topics methods courses rated B.S.
students.
KSA 1: Be able to design a sound psychological study (N =54).
Response: Faculty rated 50% of the students as 4 or 5 indicating they had
sufficiently met this objective (M=3.87, SD=.94). Comparisons to mean ratings
from 2009-2010 showed no change (M=4.16, SD= .78, N=53), t(105)=-1.50,
p>.14.
KSA 2: Be able to analyze data (N = 54).
Response: Faculty rated 46.30% of the students as 4 or 5 indicating they had
sufficiently met this objective (M=3.67, SD = .91). Comparisons to ratings from
2009-2010 showed no change (M=4.00, SD= 0.65, N=53), t(105)=-1.71, p>.09.
KSA 3: Ability to communicate findings effectively (N = 54).
Response: Faculty rated 59.26% of the students as 4 or 5 indicating they had
sufficiently met this objective (M=3.78, SD =.99). Comparisons to ratings from
2009-2010 showed no change (M=4.02, SD= .79, N=53), t(105)=-1.24, p>.22.
****************
Faculty instructing 31 capstone courses across 2011 rated students taking their
second capstone class (N= 137).
KSA 1: Have knowledge of current theory and research in diverse areas of
psychology
Response: Faculty rated 78.39% of the students as 4 or 5 indicating they had
sufficiently met this objective (M=4.07, SD= 0.73). Comparisons to ratings from
2009-2010 showed no significant change (M=4.15, SD= 0.60, N=126), t(261)=.65, p>.51.
KSA 2: Fundamental skills in critically evaluating theoretical concepts, research
design, and data analysis.
Response: Faculty rated 66.42% of the students as 4 or 5 indicating they had
sufficiently met this objective (M=3.96, SD = 0.73). Comparisons of mean
ratings from 2009-2010 also indicate no significant change, (M=4.08, SD= 0.71,
N=126), t(261)=-.97, p>.33.
KSA 3: Effective communication skills including skills for writing literature
reviews and empirical research reports following the American Psychological
Association Manual of Style, and skills for participating in seminars and making
oral presentations following accepted standards for scientific conferences.
Response: Faculty rated 80.29% of the students as 4 or 5 indicating they had
sufficiently met this objective (M=4.13, SD= 0.68). Comparisons to mean ratings
from 2009-2010 indicate no significant change (M=4.08, SD= .74, N=126),
t(261)=.40, p>.69.
D. Findings for the psychology minor. *
Objectives: Graduates earning a minor in psychology were asked to indicate how
much (1 = not at all through 5 = very much) the curriculum provided
opportunities to be familiar with current theory and research in self-selected areas
of psychology. They were also asked to indicate how much (1 = not at all
through 5 = very much) they had been able to apply their knowledge of
psychology to issues in education, employment, and interpersonal relations.
Unfortunately, none of the minor students responded. Given the difficulty in
reaching psychology minors to complete assessment surveys in the past 2 years, a
more targeted approach will be employed involving a return to the paper survey in
addition to follow-up surveys with SurveyMonkeyTM.
III. Program improvements
Now in its sixth year of existence, the Psychology Undergraduate Program (PUP)
Office continues its collaboration with campus and community partners to support
academic, research, and practicum opportunities available to undergraduate
students.
The PUP office also works with the undergraduate student
organizations PsiChi (the National Honors Society for psychology
undergraduates) and the Psychology Club.
Student members of these
organizations and PUP office staff continue their collaboration with recruiting,
marketing and promotion of psychology, workshops, planning for professional
conferences, and related programming.
IV. New assessment developments
SurveyMonkeyTM was implemented this year to replace the paper and pencil
surveys that were manually sent out to students. The new mode of administration
simplified data entry, but likely contributed to a slightly lower response rate.
Whereas nearly 100 students returned surveys during the 2009-2010 assessment
period, only 71 students responded to the SurveyMonkeyTM solicitation this year.
Despite the decrease, we’re committed to this mode of administering the
assessment but concede that follow-up communications will more than likely be
necessary.
****
END