Division of Academic Affairs
Annual Assessment Report AY 11-12
For Undergraduate & Graduate Degree Programs
Program Information
Name of Program:
Prepared By:
Psychology BA
College:
CHABBS
(Department Chair/Program Coordinator)
Sharon Hamill/ Marie Thomas
Date:
7/3/12
Email Address:
shamill@csusm.edu
Extension:
8029
PART A: Annual Assessment Report AY 11-12
Due by May 24, 2012
1) Please describe:
A. the program student learning outcomes you focused on for assessment this year.
B. the assessment activities you used to measure student learning in these areas.
C. the results of your assessment(s).
D. the significance of your results.
During AY 10-11, the Psychology Department redid its course by SLO matrix (separate document) and chose to examine the
following undergraduate SLO:
Apply APA guidelines for the ethical treatment of human research participants or IACUC guidelines for the ethical
treatment of non-human subjects to the various phases of psychological research (e.g., data collection, data analysis,
reporting of research findings).
During AY 11-12, the Psychology Department conducted an assessment of students’ knowledge of ethics across the
curriculum. The assessment instrument was pilot tested in AY 10-11. The pretest/posttest assessment study for AY 11-12 was
designed in Fall 2011 and conducted in Spring 2012.
Student Assessment: Ethics
Background
An ethics quiz was developed for this project. Results of an item analysis of the pilot study were presented in last year’s
assessment report (AY 10/11) and circulated among department faculty for comment during the summer of 2011.
The current project was designed with both cross-sectional and longitudinal elements. During Fall 2011, the assessment
committee determined the courses for which we wanted information regarding students’ knowledge of ethics. Our goals for
this investigation were: (1) to examine students’ current knowledge of ethical issues in Psychology, (2) to explore whether
student knowledge increased as students move through the Psychology course (from Introductory level to graduate level),
and (3) to look at whether individual students’ demonstrated growth over the course of the semester.
Method
Sample
Pretest:
Nine-hundred twenty three students from 33 courses participated in the pretest phase of the project and their data are used
in the item analysis. Some students did not provide complete data so the pretest analyses (using the mean pretest scores) are
based on 892 students. Most students (77.1%) completed the pretest using a scantron and the remainder completed the
pretest using Moodle. Class standing: 17.6% freshmen, 11.2% sophomores, 34% juniors, 34.9% seniors and 2.1% graduate
students completed the pretest. Forty-eight percent of the sample was Psychology majors.
1
Posttest:
Six-hundred ninety-two students from 29 courses completed the posttest. Seventy-eight percent completed the assessment
via scantron. Data from some students were omitted because of incomplete or duplicate data. Complete data was available
for 690 students (the mean posttest score is based on this number). Class standing: 18.8% freshmen, 10.4%
sophomores,32.5% juniors, 35.1% seniors and 2.6% graduate students completed either the pretest or the posttest. Fortyeight percent of the sample was Psychology majors.
Measures
Demographics. Students completed a set of demographic questions in order for us to be able to match their responses on the
pretest and posttest and to determine their course experience in the psychology major. Demographics included their student
ID number, whether they were psychology majors, and a series of questions about courses they may have taken across the
curriculum. These data were used to put students into 6 categories:
Level
Psyc 100 only
Psyc 230 but no higher
Psyc Core (300 level) but no higher
Psyc Labs but no higher
Advanced History and/or Testing
Graduate
Missing data
Pretest
22.6%
4.4%
38.1%
3.1%
22.1%
1.9%
7.9%
Posttest
23.3%
16.2%
28.7%
1.2%
20.4%
2.6%
7.5%
Ethics quiz. A short 10-item ethics quiz was developed in fall 2010 by the department’s assessment committee. Feedback
was obtained from the entire department prior to pilot testing in spring 2011. The 10 items reflect key ethical considerations
in psychological research: harm to participants, confidentiality, informed consent, handling data, vulnerable populations,
plagiarism, and use of animals in research.
Procedure
The following courses were selected for participation in this project:
1) Psych 100 Intro to Psyc (large section with TA-led break-outs)
2) Psyc 230 Intro to Research Methods
3) Core Courses: (Psyc 330, 332, 334, 336, 348, 356,360,362) In general, two sections of each identified course were
selected for participation. If one section met on MW, the second section met on T/R to increase the chances of
getting information from the widest range of students.
4) Lab Courses (Psyc 391, 392, 393, 394,395,396). If there were multiple lab sections, two were selected in the same
manner as the Core courses.
5) Senior-level advanced courses (Psyc 402 and Psyc 490). One section from each course was selected because many
students take both courses in the same semester).
6) Graduate course: All three courses (Psyc 530, 554 and 560) were included in the assessment.
During the first weeks of the spring semester, students in these courses completed the pretest during class time. If the course
was taught in a computer lab, students took the assessment via the quiz tool in Moodle. If there were no computers in the
classroom, students completed the assessment via scantron. Participation took approximately 10 minutes. If a student took
the assessment in one class, they didn’t need to take it in any other. The posttest was conducted in the same manner during
the last two weeks of the semester.
I.
Examine Student’s Knowledge of Ethical Issues in Psychology
Pretest (administered in the first two weeks of the Spring 2012 semester):
1. Mean item score the sample as a whole was 7.51 (SD = 1.61, range = 1 to 10).
2. The percentage of students who answered each question correctly was calculated. Note: 923 students provided data for
the item analyses. Some of these students were dropped in the final data set due to duplicates or incomplete
information. These results reflect the percentages calculated from two item analyses (one from the scantrons and one
from moodle).
2
Most Commonly Understood Concepts:
 Most students (76.1%) understood that participants must be protected from harm when participating in research
studies.
 Most students (85.2%) understood that confidentiality means that researchers should never reveal participants’
names when reporting results.
 There were three questions regarding informed consent as this is seen as a fundamental principle in conducting
research with human participants.
 91.9% of students reported that participants must be told of their right to withdraw from studies.
 Only 79.1% understood that students at a university are not required to participate in research.
 However, only 58.3% understood that researchers must inform participants about stressors prior to participating
in a study.
 The majority of students (86%) understood that when handling data, researchers may not delete results from a
research project just because they are different from other results obtained.
 Most students (89.2%) recognized plagiarism as (1) taking information from an article and presenting it as your own
idea, (2) copying some sentences directly from an article without citing the author and (3) putting your name on a
project even though you didn’t contribute.
 78.8% of students understood that research with animals should minimize the number of animals used in the study,
minimize the pain and distress experienced by the animal participants, and avoid using animal participants if
inanimate systems (e.g., computer programs) are available
Less Commonly Understood Concepts:
 Fewer students (66.1%) understood that when working with children or adolescents as a vulnerable population,
researchers may not use large incentives to gain their participation.
 Only 30.3% of students recognized that research proposals that involve the use of animals are required to be
submitted to the IACUC.
3.
Item analyses were conducted on the ethics quiz (separately for each class). For all items, the upper 27% (top scorers) of
the sample outperformed the lower 27%.
4. Student pretest scores showed an increase by level (i.e., introductory through graduate level). Psyc 100 students scored
significantly lower than advanced History/Testing students and graduate students [F(5,825) = 6.19, p < .000); graduate
students also scored significantly higher than students who had only gone as far as the Psychology Core courses. No
other differences were found among students’ scores in the various levels, although the means increased as students had
more exposure to psychology.
Posttest (administered in the last two weeks of the Spring 2012 semester):
1. Six-hundred ninety students submitted data for the posttest (a retention rate of 75%). The mean item score on the
posttest for the sample was 7.82 (SD = 1.68, range = 1-10). The median score was 8 and the distribution was negatively
skewed.
2.
The percentage of students who answered each question correctly was calculated. Note: 692 students provided data
for the item analyses. Some of these students were dropped in the final data set due to duplicates or incomplete
information. These results reflect the percentages calculated from two item analyses (one from the scantrons and one
from moodle).
Most Commonly Understood Concepts:
 Most students (81.2%) understood that participants must be protected from harm when participating in research
studies.
 Most students (83.7%) understood that confidentiality means that researchers should never reveal participants’
names when reporting results.
 There were three questions regarding informed consent as this is seen as a fundamental principle in conducting
research with human participants.
 96%% of students reported that participants must be told of their right to withdraw from studies.
 Only 78.9%% understood that students at a university are not required to participate in research.
3



 65.9% understood that researchers must inform participants about stressors prior to participating in a study.
The majority of students (88.3%) understood that when handling data, researchers may not delete results from a
research project just because they are different from other results obtained.
Most students (89.9%) recognized plagiarism as (1) taking information from an article and presenting it as your own
idea, (2) copying some sentences directly from an article without citing the author and (3) putting your name on a
project even though you didn’t contribute.
84.1% of students understood that research with animals should minimize the number of animals used in the study,
minimize the pain and distress experienced by the animal participants, and avoid using animal participants if
inanimate systems (e.g., computer programs) are available
Less Commonly Understood Concepts:
 Fewer students (68.1%) understood that when working with children or adolescents as a vulnerable population,
researchers may not use large incentives to gain their participation.
 36.6% of students recognized that research proposals that involve the use of animals are required to be submitted to
the IACUC.
3.
Item analyses were conducted on the ethics quiz (separately for each class). For all items, the upper 27% (top scorers) of
the sample outperformed the lower 27%.
4.
Examination of student posttest scores by level showed a more variable pattern than the pretest scores, most likely due
to missing data (e.g., only 8 students who had only gone as far as Psych labs provided posttest data as compared to 28
students who provided it at the pretest). Psyc 100 students scored significantly lower than Psyc 230 (Introductory
Methods), advanced History/Testing students and graduate students [F(5,626) = 7.97, p < .000); graduate students also
scored significantly higher than students who had only gone as far as the Psychology lab courses. No other differences
were found among students’ scores in the various levels, although the means generally increased as students had more
exposure to psychology. The difference between Psyc 100 and Psyc 230 students should be underscored. The Psyc 230
course addresses ethical considerations in research as component of the course and students’ scores suggest increased
knowledge in this area.
Changes in scores pretest to posttest:
Complete data from the pretest and posttest was available for 589 students. A dependent samples t-text was run to examine
whether students gained knowledge about ethics over the course of the semester. Results indicated that posttest scores (M =
7.9, SD = 1.6) were significantly higher than pretest scores (M = 7.5, SD = 1.58; t(588) = 5.15, p < .000).
Analyses of the individual items revealed that significant increases in the scores were due to greater knowledge of the three
informed consent items (Q1: foremost ethical concerns for a researcher; Q3: study participants should be told of their right to
withdraw from a study; and Q7:informed consent requires that participants be told of potential stressors) and the two
questions that addressed use of animals in research (Q9: researchers must submit their proposals to IACUC; and Q10:
researchers must try to minimize the use of animals in research, reduce their pain/distress, and avoid using animals if
possible). Despite students’ greater knowledge about the use of animals in research, most students don’t know who reviews
the research on animals (IACUC). Faculty should work to make students aware of the different governing body for animal
research.
There was no change in students’ understanding of the special rights of children and adolescents over the course of the
semester. Given relatively fewer students understood these rights to begin with, this should be an area of focus for the
psychology faculty.
2) As a result of your assessment findings, what changes at either the course- or program-level are being made and/or
proposed in order to improve student learning? Please articulate how your assessment findings suggest the need for
any proposed changes.
We have not discussed the assessment results with department faculty but will do so at our August retreat and during the fall
semester. We tend to do a good job of highlighting informed consent in research and the vast majority of our students
understand plagiarism.
The following recommendations for improvement will be made: (1) clarify for students the informed consent rights of children
and adolescents in psychological research (2) include discussions of how studies with animals are evaluated for ethical
4
practices and emphasize the role of the IACUC, (3) discuss how/where in the curriculum we emphasize ethical concerns in
research. All of these items should be introduced in Psyc 100 and Psyc 230 courses; this information should be reinforced in
the upper division core courses and labs. The department will also discuss ways in which we may reinforce this information in
the senior-level psychology courses.
3) If you used the resources that were given to you as stated in your plan, please check here.
If you used them differently, please provide specifics.
5