Division of Academic Affairs
Annual Assessment Report
For Undergraduate & Graduate Degree Programs
AY 2007-08
nnual
Name of Program: Psychology MA
College:
COAS
Prepared by: Sharon B. Hamill, Ph.D.
Date: May 22, 2008
Department Chair/Program Coordinator
Email Address:
shamill@csusm.edu
Extension: X8029
PART A (Required by May 22, 2008 – last day of Spring semester)
1)
Please describe the student learning outcomes you focused on for assessment this year,
the assessment activities you used to measure student learning in these areas, and the
results of your assessments. Please also comment on the significance of your results.
We chose to focus on two program goals: Enhancing Communication Skills (Goal 4) and
Cultivating Characteristics of Professional Behavior (Goal 6).
The relevant SLOs that we assessed in support of Goal 4 (enhancing communication skills),
were:
Graduate students will:
 demonstrate an ability to use the language and conventions found in the
scholarly/professional literature of Psychology.
 demonstrate proficiency in the mechanics of writing including spelling, grammar,
paragraph structure and sentence structure.
 demonstrate understanding of their research area through clear expository writing.
 demonstrate the ability to integrate diverse literature and ideas and produce thoughtful
written critical analyses.
We used the new assessment tool that had been developed by the committee of
graduate coordinators to determine if graduate students meet the graduate writing requirement
(GWAR) to evaluate graduate student writing in the first semester (in the PSYC 600 class), in
the first two proseminars (Fall 07 and Spring 08), and in the Advanced Research Methods
class (PSYC 530, Spring 08). This rubric allows faculty to assess student writing regarding
four criteria: content, style, mechanics, and analysis. In the case of the PSYC 600 course, all
tenure-line faculty who gave a presentation in the class provided a writing assignment for the
first year students. The individual faculty members were responsible for using the rubric to
grade the assignments. Despite the fact that we all used the same rubric, it was clear that there
were substantial differences in the grading criteria used by the faculty. Therefore, the 11
writing assignments in PSYC 600 were grouped, and means were calculated for these groups,
Page 1 of 5
5/24/08
Annual Report on Assessment of Degree Programs, AY 2007-08
in order to minimize some of the differences. The first two papers were considered “baseline,”
and the other nine papers were averaged in groups of three.
In looking at the overall means, there doesn’t seem to be much of a pattern over the fall
semester or between fall and spring Some elements seem to show improvement, while others
decline. A variety of statistical comparisons were made, none were significant, most likely due
to the very small sample size. However, probably the most valid comparisons were done
between the PSYC 600 final paper score and the paper score in PSYC 556, because these
papers were graded by the same faculty member. In this case, significant improvement was
found in two of the rubric categories: Style (f07 M = 2.83, s08 M = 3.42) and Content (f07 M =
3.00, s08 M = 3.58).
A number of the faculty had difficulty using the rubric, feeling that some of the
categories and category descriptions didn’t fit the types of papers required in the department.
It is also likely that faculty standards are different; while within-faculty ratings may be
consistent, there is a fair amount of between-faculty variability. We consider this use of the
writing rubric to be a good beginning and we plan to work to improve the usefulness of the
tool over the next year.
Second, in support of Goal 6 (cultivating characteristics of professional behavior) we
focused on increasing student attendance at thesis proposal and defense meetings and
participation in search-related activities. The relevant SLOs were:
Graduate students will:
 participate in the thesis proposal and defense process by attending presentations given by
peers.
 attend job talks presented by candidates seeking employment in the Psychology
department.
 participate in the candidate selection process by meeting with job candidates and providing
feedback to the department’s search committee.
. In order to address Goal 6 we kept track of graduate student attendance at these events to see
if our expectations were being met. We specifically targeted first year students to engage them
in conversations about the importance of attending talks/proposals/defenses and then
compared their attendance at such events to the attendance of second year students Overall,
the data show that first- year students attended a mean number of 12.2 presentations during the
year as compared to 4.5 presentations for the second year students. Eighty-three percent of the
first-year students attended 80% or more of the talks; 0% of the second-year students attended
this many talks. Attendance at the three job talks for our clinical candidates was particularly
good for first-year students: 5 out of the 6 (83.3%) attended the talks for all three candidates;
25% of the second year students attended all three presentations however, 75% of the secondyear students attended two talks or more. Collectively, these data suggest that discussing the
importance of attending proposals, defenses, and job talks help to establish an expectation that
students should participate as part of their professional development. Second, students may
have considered attendance at job talks as relatively more important than attending
presentations by their peers as demonstrated by attendance patterns of both first year and
second year students. It will be important for us to continue with our professional
socialization of graduate students to ensure that they understand the importance of supporting
Academic Programs/DB
Page 2 of 5
Annual Report on Assessment of Degree Programs, AY 2007-08
their colleagues as well as providing feedback for use in making hiring decisions.
The Psychology Department is committed to using the results of the assessment activities we
undertook this year to improve student learning (i.e., data will be used to “close the loop.”).
Faculty advisors will be given information on their advisees’ writing and professional behavior
so they can discuss student performance directly with each of their advisees. Additionally,
each year, the department has a formal discussion of graduate student progress at its annual
retreat. This year, results of the assessments on student writing and student professional
behavior will be presented to the department as a whole. In this way, the department can
address concerns, if any, regarding student writing and professional performance in the
program. If corrective actions are deemed necessary, the department will formulate a plan to
address concerns during the retreat. If expectations are not being met, advisors will discuss the
problem with their students.
2)
How did your program utilize any resources provided for assessment this year? Please
attach a budget with specifics.
We requested $4500 for assessment activities; we were granted $3000. Marie Thomas and
Nancy Caine were each given a $1000 stipend to compensate them for the time they spent
conducting assessment activities at the graduate and undergraduate level. Sharon Hamill was
not compensated for her assessment work. We should underscore that that we also were
required to conduct a program review of our MA program this year; consequently, the hours
these three faculty spent working on assessment far outweighed the compensation. The
remaining $1000 was used to hire a graduate student assistant to help with the data collection
and analysis.
3)
As a result of your assessment findings, what changes at either the course- or programlevel are being made and/or proposed in order to improve student learning? Please
articulate how your assessment findings suggest the need for any proposed changes.
The findings from the MA assessment activities will be discussed at our annual retreat in
August. Plans for “closing the loop” will be formulated at that meeting. At the very least, we
will use these data to (1) support our efforts in working with individual students on their
writing, noting where they have weaknesses and could benefit from more instruction and (2)
discuss student performance in terms of meeting expectations for professional behavior.
Additionally, we will be discussing the usefulness of the rubric for assessing student writing.
Given that we were not completely satisfied with the way this rubric was used and have some
concerns that we need to address (most notably, faculty interrater reliability), changes in
courses or the program are not warranted at this time.
Academic Programs/DB
Page 3 of 5
Annual Report on Assessment of Degree Programs, AY 2007-08
PART B
4)
Planning for Assessment in 2008-009
(Required by Friday, September 19, 2008;
May be submitted earlier for Expedited Funding Decision)
Please identify one or two student learning outcomes that your program will focus on for
assessment next year.
Writing:
Graduate students will:
 Demonstrate proficiency in the mechanics of writing including spelling, grammar,
paragraph structure and sentence structure.
 Demonstrate understanding of their research area through clear expository writing.
Oral Presentation:
Graduate Students will:
 Present scientific content in graduate seminars and lectures, at a level
commensurate with standards of academic discourse.
 Present and defend their thesis in a clear and thoughtful manner.
5)
What specific assessment activities will you conduct next year in order to measure
program student learning in these areas


Demonstrate proficiency in the mechanics of writing including spelling, grammar,
paragraph structure and sentence structure.
Demonstrate understanding of their research area through clear expository writing.
We will be discussing the findings from the writing assessment we conducted this year and
clarifying the rubric for use on student writing in 08-09.


Present scientific content in graduate seminars and lectures, at a level
commensurate with standards of academic discourse.
Present and defend their thesis in a clear and thoughtful manner.
We will be developing a rubric for use in assessing oral presentations including in-class
presentations and thesis proposals/defenses.
6)
What new or additional resources/support might your program need in order to
conduct these assessment activities next year? (Please provide specific information
regarding your needs and related costs)
We are requesting three $1000 stipends for the program assessment faculty (Nancy Caine,
Marie Thomas, and Sharon Hamill) and a $1000 stipend for a graduate assistant who will work
on the website assessment and assist in data collection, coding, and analysis involved in the
assessment of students’ interpretation of empirical investigations (see description under BA
program assessment report). For the graduate program, the graduate assistant will also be
Academic Programs/DB
Page 4 of 5
Annual Report on Assessment of Degree Programs, AY 2007-08
helping to summarize data from the writing rubric. Once we have developed the oral
presentation rubric, will be summarizing that data, as well.
Graduate student assistant (90 hours @ $11/hour) =
Undergraduate program assessment:
Tasks: work on assessment method for evaluating statistics
webpage, search for and add more student tutorials, data
collection, coding and analysis.
Graduate Student Assessment:
Enter data from assessments of writing and oral communication,
analyze data.
$1000
Stipend for Program Assessment Committee:
30 hours of work per member
$3000
Total =
Academic Programs/DB
$4000
Page 5 of 5