Annual Assessment Report to the College 2010-11
College: _____ _Health and Human Development
Department: _ Environmental & Occupational Health
Liaison: __
_John Schillinger
1. Overview of Annual Assessment Project(s)
1a. Assessment Process Overview: Provide a brief overview of the intended plan to assess the program this year. Is assessment under the
oversight of one person or a committee? One person -- the assessment liaison.
This past year the following SLO’s were assessed for the undergraduate and graduate programs as planned. Planned analysis of embedded
questions (for Undergraduate SLO 1 and 4) was not completed this year but will be postponed to 2011-12. New strategies for assessing
Undergraduate SLO 5 and SLO 6 are needed.
Undergraduate SLO 1 (Demonstrate a comprehensive knowledge of the recognition, evaluation and control of biological, chemical, and physical
factors that can impact human health and safety and the environment.) was assessed using state REHS exam results and will continue to be
assessed with embedded questions in EOH 356A and 356B.
Undergraduate SLO 2 (Demonstrate knowledge of how to work in interdisciplinary teams to promote public and private action to protect public
health and the environment.) was assessed using internship preceptor data and will continue to be assessed using preceptor survey data from
the EOH 494B internship reports.
Undergraduate SLO 3 (Communicate environmental and occupational health concepts and programs to a variety of audiences, using both
written and verbal forms of communication.) was assessed and will continue to be assessed using student written completion reports for EOH
494B internships.
Graduate SLO 1 (Research design and analytical skills needed to critically evaluate scientific, technical and regulatory documents.) was assessed
and will continue to be assessed using a EOH 697 comprehensive exam question related to skills developed in EOH 696A and 696B.
Graduate SLO 2 (Oral, written and electronic communication skills to present information to professional groups, regulatory agencies and lay
July 18, 2011, Bonnie Paller
audiences.) was assessed and will continue to be assessed using student written completion reports for EOH 693A field training and will continue
to be assessed (except for oral skills) by using a common rubric for analyzing written papers in a section of EOH 554 and 555 graduate seminars.
Graduate SLO 3 (Sufficient level of technical expertise in environmental and occupational health to competently solve general EOH problems.)
was assessed and will continue to be assessed by analyzing all of the scores from the 5 different technical sections of the EOH 697
comprehensive exams.
Graduate SLO 4 b (A broad set of management skills to initiate program planning and critical analysis of environmental or occupational health
and safety programs.) was assessed and will continue to be assessed by analyzing a EOH 697 comprehensive exam problem solving assignment
related to skills developed in EOH 554 and 555 graduate seminars.
1b. Implementation and Modifications: Did the actual assessment process deviate from what was intended? If so, please describe any
modification to your assessment process and why it occurred.
Undergraduate SLO 1 and Graduate SLO 3 were assessed with the results of the State Registered Environmental Health Specialist Exam because
test results (anonymous for all EOH students) became available for the period spring 2007-fall 2010. Undergraduate SLO 1 and 4 were not
assessed using embedded questions because of difficulty compiling embedded question data from EOH 356A, 356B and 466A. Standardizing and
quantifying the rubric for comparison of written assignments from graduate seminars (Graduate SLO 2) requires more work which is in progress.
2. Student Learning Outcome Assessment Project: Answer questions according to the individual SLOs assessed this year. If you assessed
more than one SLO, please duplicate this chart for each one as needed.
2a. Which Student Learning Outcome was assessed this year?
Undergraduate SLO 1
2b. What assessment instrument(s) were used to gather evidence about this SLO?
Data from the REHS Exam were averaged and compared by total score and by the 13 technical section scores.
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type of
students, which courses, how decisions were made to include certain participants.
All of the EOH BS graduates who took the REHS exam in spring 2007 through fall 2010 were assessed. (n=16 in 05/06 and n=61 in 07/10)
July 18, 2011, Bonnie Paller
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was
a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
Those who took the exam in 07-10 were compared to those who took the exam in 05/06. The exam takers graduation dates are not known but
are assumed to be within a few months to 2 years of the exam dates.
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from the
collected evidence.
The average total score for the exam was 166 in 07/10 which was a marked improvement from the 135 in 05/06. Some of the improvement was
due to changes in EOH 356A and B following the 05/06 results and some was due to changes in the exam itself.
The three technical sections that were lowest in 05/06 were still the lowest scoring in 07/10 (of the 13 sections) but showed significant
improvement in two of them. (see scores as % of 100 for each section of the REHS Exam)
05/06
07/10
Recr Water/Pools Section
40.0 %
60.1 %
Wastewater Section
45.3 %
46.8 %
Drinking Water Section
52.6 %
62.3 %
2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic
quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program,
student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc.
Please provide a clear and detailed description of how the assessment results were or will be used.
Embedded questions in EOH 356A and 356B will continue to focus on water and continued curricular emphasis will be focused on these topics
especially in the area of wastewater (one of the most difficult of the 13 sections based on statewide results).
2a. Which Student Learning Outcome was assessed this year?
Undergraduate SLO 3
2b. What assessment instrument(s) were used to gather evidence about this SLO?
July 18, 2011, Bonnie Paller
Evaluating written internship reports submitted for EOH 494B Academic Internship.
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type of
students, which courses, how decisions were made to include certain participants.
Due to the relatively small number of students, all students in the internship classes were assessed. (n=24 in fall 10 and in spring 11)
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was
a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
Semester by semester comparisons which means different students each semester but all undergraduate EOH students in the program over
time. (EOH 494B is required of all BS students)
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from the
collected evidence.
Using a 1-10 scale for writing skills, the following data (averages) from the internship reports are: fall 08 8.2, spr 09 7.9, fall 09 8.7, spr 10 8.5,
fall 10 9.4, and spr 11 8.6.
The averages do not show a strong trend but the number of unacceptable writing examples (scale value of 7 or less) declined from 33% in spr 09
to 10% in spr 10, 4.1% in fall 10 and 12.5% in spr 11.
2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic
quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program,
student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc.
Please provide a clear and detailed description of how the assessment results were or will be used.
Previous data led to incorporating 3 writing assignments in EOH 466A (one of the core undergraduate courses) and 1 in EOH 356A (another core
undergraduate course). The data indicate a need to identify problems with students for whom English is a second language (and a few others)
early in their program and to initiate some remedial action or additional assistance similar to the technical writing review being offered the
graduate students. EOH 356A is one of the first required courses in the major and that writing assignment will be used to screen students who
July 18, 2011, Bonnie Paller
will hopefully be able to utilize peer tutoring (if financial assistance for student tutors is made available).
2a. Which Student Learning Outcome was assessed this year?
Undergraduate SLO 2
2b. What assessment instrument(s) were used to measure this SLO?
Preceptor responses to the internship evaluation forms (EOH 494 B Academic Internship) were tabulated for three questions –using a 1-5 scale
with 1 being best and 5 worst.
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type
of students, which courses, how decisions were made to include certain participants.
All undergraduate students in the class were evaluated due to the small class sizes. (n=24)
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or
was a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
Semester by semester comparisons which means different students each semester but all undergraduate EOH students in the program over
time
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from
the collected evidence.
The Internship Evaluation form is attached to this report. The three categories related to SLO 2 were Relations With Others, Initiative, and
Attitude – Application to Work. The averages of the responses from the 1-5 scale were:
Fall 08
Spr 09
Fall 09
Spr 10
Fall 10
Spr 11
Rel. With Others
1.11
1.38
1.57
1.28
1.33
1.23
July 18, 2011, Bonnie Paller
Initiative
1.22
1.88
1.78
1.72
1.71
1.42
Attitude – Application
1.33
1.62
1.57
1.39
1.71
1.54
There appears to be an improvement in each category as shown by the spring 11 averages.
2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic
quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program,
student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes,
etc. Please provide a clear and detailed description of how the assessment results were or will be used.
Students in the internships are now being advised with written instructions on proper behavior and responsibilities before they are assigned a
preceptor and that may be having a positive effect – that will continue. A student focus group of student interns who have just completed their
internship is being considered as a means of reinforcing professional behaviors and as a means of assessing (and improving) the internship
experience.
2a. Which Student Learning Outcome was assessed this year?
Graduate SLO 2
2b. What assessment instrument(s) were used to gather evidence about this SLO?
Student written internship reports for EOH 693A Supervised Field Training have been analyzed and compared by semester.
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type
of students, which courses, how decisions were made to include certain participants.
All graduate students in the 693A courses were evaluated due to the small class sizes. (n=7 in fall 10 and n=13 in spr 11)
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or
was a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
A semester by semester comparison was used.
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the data were analyzed and highlight important findings from the
data collected.
July 18, 2011, Bonnie Paller
The results for the writing assignment on a 1-10 scale were: Fall 08 9.4, Spr 09 8.9, Fall 09 9.6, Spr 10 9.3, Fall 10 9.4, Spr 11 8.8.
All the assignments scored an 8.0 or higher in fall 10 and spr 11. There were some indications that students from foreign countries have some
language/writing deficiencies .
2f. Use of Assessment Results of this SLO: Think about all the different ways the results were (or could be) used. For example, to recommend
changes to course content/topics covered, course sequence, addition/deletion of courses in program, student support services, revisions to
program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc. Please provide a clear and detailed
description of each.
Based on earlier writing assessment data, a voluntary 4 hour writing refresher course (over 4 weeks) has been provided by a faculty member in
fall 10 and spr 11 and is proving to be a useful remediation strategy for those graduate students who need extra assistance. It is hoped that
student peer tutoring may provide additional needed help for graduate students earlier in their program.
2a. Which Student Learning Outcome was measured this year:
Graduate SLO 1, 3, and 4
2b. What assessment instrument(s) were used to gather evidence about this SLO?
Results from the various sections of the comprehensive exams (EOH 697) were tabulated and averaged and compared by semester.
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type
of students, which courses, how decisions were made to include certain participants.
All of the students in the EOH 697 comp exams were evaluated due to the small class sizes. (n=5 in fall 10 and n=15 in spr 11)
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or
was a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
A semester by semester comparison of average scores (by technical section) was done.
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from
the collected evidence.
July 18, 2011, Bonnie Paller
Comprehensive Graduate Exam Averages by Section (n=15, 19 and 9 for spr 08, 09 and spr 10)
Spr 08
Spr 09
Spr 10
Fall 10
Spr 11
•
•
•
Env Health
73.0
75.0
79.6
83.4
68.5*
Occup Health
68.6
56.6
74.8
76.0
69.1*
Toxicology
89.1
91.1
92.2
93.1
Admin
79.0
78.9
88.5
87.3
78.0
Research Des.
79.2
75.0
78.0
Problem Solving
84.6
83.4
Format and some questions were altered from previous exams.
The quantitative questions involving calculations in particular were shown to be areas of concern in env health and occupational health.
Conceptual and organizational aspects of problem solving were above average as was understanding of toxicological principles.
2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic
quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program,
student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes,
etc. Please provide a clear and detailed description of how the assessment results were or will be used.
•
The review session for research design (which involves calculations) is effective and needs to be duplicated for the areas of concern
mentioned above. Peer review tutoring will be attempted both during the courses and during the comp exam reviews. An on-line
tutorial will be considered also.
3. How do this year’s assessment activities connect with your program’s strategic plan and/or 5-yr assessment plan?
The assessment activities followed the plan from 2007-2012 for the most part but electronic portfolios were not developed. Simulations
were also not developed for EOH 352, 454 and 453. Ongoing efforts will continue with existing assessment tools and new approaches
will be considered that involve more student input such as surveys and focus groups. The REHS data may not serve our assessment
needs well because the state exam keeps changing and the exam sections do not have the necessary specificity to connect well with our
curriculum. The comp exams work well for the graduate program assessment and perhaps something similar is needed for the
July 18, 2011, Bonnie Paller
undergraduate program. It could come in the form of a review for the State REHS exam and include both exam and survey elements –
preferably online. This may be less cumbersome than portfolios.
4. Overall, if this year’s program assessment evidence indicates that new resources are needed in order to improve and support student
learning, please discuss here.
The results of the writing assessments and the quantitative skills assessments show the need for remedial/extra assistance which can most
effectively and efficiently be delivered by peer tutors. Some financial support is needed for those student assistants.
5. Other information, assessment or reflective activities not captured above.
The Department has an annual EOH Advisory Board meeting where needs of the department and of students are discussed with the EOH
working professionals who serve on that committee. The last two meetings have raised the issues of new curriculum needs in the areas of
sustainability and adaptation to climate change. As a result, the focus of one section of EOH 554 seminar this semester will be on sustainability.
A new course in sustainability is under consideration as well. The two recent course additions to the MS program (Advanced Risk Assessment
and Advanced Toxicology, both new 500 level electives) resulted in part from the suggestions of the Advisory Board as they assessed the
required professional skills of new EOH graduates. (Undergraduates may enroll in those courses also). Leadership development was also
identified by the Advisory Board as a priority and has been an area of focus in EOH 553 administration and in the EOH 555 seminar. Advice from
the Advisory Board regarding the need for more practical work skills also contributed to offering the 40-hour HAZWOPER (Hazardous Waste
Operations Emergency Response) training to undergrad and graduate students in the hazardous materials course, and to the inclusion of a new
hands-on sampling lab component in the research design (EOH 696A) graduate course.
July 18, 2011, Bonnie Paller
An EOH faculty retreat is planned for November to discuss the future direction of the MS program in light of all the assessment data.
6. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your
program? Please provide citation or discuss.
No
July 18, 2011, Bonnie Paller