Annual Progress Report (2011)
Assessment of Student Learning
DEPARTMENT/PROGRAM:
Chemistry
DATE SUBMITTED:
June 2011
ASSESSMENT COORDINATOR: Darlene Gandolfi
DEPARTMENT CHAIR:
Darlene Gandolfi
For each box, please address the following:
LEARNING OBJECTIVE
Which of your student learning objectives were measured?
MEASURES USED
What material from class did you use to evaluate the learning objective? (e.g., exams, written assignments, presentations, final project)
What measure did you use? (e.g., exam scores, rubric, survey)
Note: Please attach any rubrics or evaluative criteria
RESULTS OF ASSESSMENT
How many students were assessed?
What % of students met the learning objective?
ACTIONS RECOMMENDED TO IMPROVE STUDENT LEARNING
What curricular or specific course concerns arose?
What actions does your Dept. plan to take?
What additional resources would be needed to implement these actions?
LEARNING OBJECTIVE
MEASURES USED
RESULTS OF ASSESSMENT
ACTIONS RECOMMENDED TO
IMPROVE STUDENT LEARNING
Students must comprehend the
principles of all basic areas of
chemistry (analytical,
biochemistry, inorganic, organic,
and physical).
Standardized American Chemical Society
(ACS) examination scores in core courses
were compared to national norms and test
results were evaluated by topic. The ACS
examinations evaluate students’ knowledge
of facts, skills, and procedures that should
have been acquired as a function of the
undergraduate curriculum.
Organic Chemistry
Eighteen undergraduate students took the FullYear Organic Chemistry ACS test (Form 2004).
(Note: This year no post-baccalaureate students
were enrolled in Organic Chemistry in
Fall/Spring.) The class average for this exam was
37/70, which falls in the 45th percentile. (The
national average (50th percentile) score was (39 +
12.16)/70.) By comparison to national norms,
students performed within the standard deviation
of the average. When considering the
Organic Chemistry
No actions planned for the course.
However, a conversation with
Admissions about recruiting stronger
majors is warranted.
This year, ACS examinations were
administered at the completion of Organic
Chemistry (Spring 2011), Analytical
Chemistry (Fall 2010), Physical
Chemistry I (Fall 2010 –
Thermodynamics) and Physical Chemistry
II (Spring 2011 – Quantum Mechanics
and Statistics).
Comprehensive exams were administered
as the Senior Evaluation. These exams
included the online version of the ETS
Major Field Test in Chemistry and an inhouse comprehensive exam.
undergraduate-only average from last year (34/70,
or 37th percentile), this year’s results are an
improvement.
Four of the 18 students were Chemistry majors.
The average for the majors was below average
(21st percentile). This percentile was lower than
that for four majors last year: 23rd percentile.
It must be noted that some topics/questions that
appeared on the test were not covered in the
course. Additionally, this was a full-year test so
students may not have recalled all topics covered
in the Fall semester. Topics analysis is provided in
Appendix A.
Analytical Chemistry
Four students (1 major, 3 minors) in CHM 3003
took the Analytical Chemistry ACS test (Form
2001). The class average for this exam was (22.3+
3.27)/50, which falls in the 19th percentile. (The
national average (50th percentile) score was (28.48
+ 7.58)/50.) By comparison to national norms,
students overall performed within the standard
deviation of the average. All topics that appeared
on the test were covered in the course. However,
these topics were covered to varying degrees. It
was found that students performed poorly on
questions involving “Acid-Base Chemistry,”
“Coordination Chemistry,” “Electrochemistry,”
and “Spectroscopy” and that they performed well
on questions involving “Data Evaluation and Error
Analysis,” “Solutions and Volumetric Analysis,”
and “Separations.” There were some differences
in how students performed on the topics from last
year. This could be a consequence of the
background of the instructors: Inorganic (F08)
versus Physical (F10). However, students
continue to do poorly on the “Electrochemistry”
questions. Comparison of results over the past
three years indicates that students perform better
on the ACS exam when CHM 3003 is taught by a
full-time instructor: 52nd percentile (F08/Gandolfi)
versus 19th percentile (F09/Wilkowski and
F10/Ravi). A topics analysis is given in Appendix
A.
Analytical Chemistry
(1) D. Gandolfi will teach CHM 3003
again in Spring 2012. However, a fulltime, tenure-track faculty member is
needed in Analytical Chemistry.
(2) A new, more student-friendly
textbook has been selected for Spring
2012. The new text provides additional
resources for student engagement.
Physical Chemistry I – Thermodynamics (FALL)
Eleven students (all Chemistry or Biochemistry
majors; four from SUNY Purchase) in CHM 2009
completed the Thermodynamics section of the
Physical Chemistry Combined Semester ACS
exam (Form 2006). The class mean for this exam
was (19.6 + 7.13)/40. Unfortunately, a national
mean was not available since the ACS does not
provide means for individual section. Upon
further scrutiny, it was noted that Purchase
students performed significantly better than
Manhattanville students: 26.5/40 (Purchase) versus
15.7/40 (Manhattanville). Overall, students that
performed better on regular course exams
throughout the semester scored better on the ACS
exam. (Notes: This was the first time that the
Physical Chemistry ACS exam was administered
as a final exam for Physical Chemistry I. This
course was taught by an adjunct.)
Physical Chemistry
Students will complete an additional
section with Thermodynamics section on
ACS exam in CHM 2009 in Fall 2011 so
that national norms are available for
comparison.
No actions are planned for CHM 2010.
Physical Chemistry II – Quantum Mechanics &
Statistical Mechanics (SPRING)
Eight students (all Chemistry or Biochemistry
majors; four from SUNY Purchase) in CHM 2010
completed the Quantum Mechanics and Statistical
Mechanics sections of the Physical Chemistry
Combined Semester ACS exam (Form 2006). The
class mean for this exam was (23.75 + 6.68)/50.
The national mean was (23.69 + 5.78)/50.
Collectively, the students in Physical Chemistry II
performed acceptably on the exam. Upon further
scrutiny, it was noted that Purchase students
performed significantly better than Manhattanville
students: 28.5/50 (Purchase) versus 19/50
(Manhattanville). In general, students performed
slightly better on the Quantum Mechanics
questions (mean: 19.4/40) than the Statistical
Mechanics questions (mean: 4.4/10). The ACS
does not provide means for individual section.
(Note: This was the first time that the Physical
Chemistry ACS exam was administered as a final
exam for Physical Chemistry II. This course was
taught by an adjunct.)
Senior Evaluation
Two Chemistry majors took the online version of
Senior Evaluation
The design of the on-line version of the
the ETS Major Field Test in Chemistry. One
student scored in the 35th percentile and the other
scored in the 25th percentile. While it was possible
to view scores for each of the sub-sets of questions
(i.e., Physical, Inorganic, Organic, and Analytical),
faculty members were not able to see the specific
questions that were asked of the students. (In
order to have access to this information, the ETS
requires at least five students to take the exam in a
given cohort.)
Given the poor performance of the two students on
the ETS MFT, a faculty-developed comprehensive
exam was administered. Like the MFT, all
questions on this exam were multiple choice
questions. Though the students fared somewhat
better in certain sub-sets of questions (i.e.,
Physical, Inorganic, Organic, or Analytical),
overall results were poor; one student answered
only 21 out of 40 questions correctly and the other
only 22 out of 40 questions correctly. Graduating
Seniors to whom this same exam was administered
in the past performed similarly on the test.
ETS Major Field Test in Chemistry exam
did not suit our needs, as we were not
able to fully analyze questions on the
exam. It is unlikely that we will use this
on-line exam in the future. The paperbased version of the exam may be
considered for the future.
The in-house exam, though designed to
contain questions that were definitely
covered in the students’ courses, does not
seem to be eliciting the desired scores.
Additionally, it does not provide national
standards with which to compare our
students’ results.
The use of the Chemistry GRE, will be
discontinued indefinitely. Though it was
used by the department for many years,
the test proved to be too difficult and too
expensive for our students. Other exams,
like the American Chemical Society’s
Diagnostic of Undergraduate Chemistry
Knowledge (DUCK) exam, will be
reconsidered on a trial basis. As a backup to the nationally normed exams, for
next Spring, faculty members will design
a new comprehensive exam, perhaps one
with different types of questions.
Additional Info
While the department maintains that our
majors must have basic comprehension of
Biochemistry, we currently do not require
a course in this target area. We cannot
add this course unless we are permitted to
hire a full-time professor who is able to
teach a Chemistry-based Biochemistry
course on an annual basis. This professor
could be a joint appointment with the
Biology department and offer the twosemester Biochemistry course required
for the Biochemistry major, which has
been taught by an adjunct instructor for
several years.
LEARNING OBJECTIVE
MEASURES USED
RESULTS OF ASSESSMENT
ACTIONS RECOMMENDED TO
IMPROVE STUDENT LEARNING
Students must apply proper
methodology and technique to
the investigation of chemical
principles through
experimentation
Scores on faculty-developed rubrics (see
Appendix B) were utilized in CHM 3004:
Chemical & Instrumental Analysis Lab.
Two Senior Chemistry majors and one Senior
Chemistry minor took CHM 3004 in Fall 2010.
All three students demonstrated high levels of
competency as evidenced by scores on rubrics
designed to assess their efforts. (Cumulative data
is in Appendix B.)
The rubric used to assess methodology
and technique was new this year. While
the criteria make sense, the scoring
approach needs revision.
Assessment-focused lab practicals are
still under consideration. However, rather
than developing our own, we await the
availability of the American Chemical
Society’s practical exam series with
scoring rubrics that will allow for national
norming of student performance.
LEARNING OBJECTIVE
MEASURES USED
RESULTS OF ASSESSMENT
ACTIONS RECOMMENDED TO
IMPROVE STUDENT LEARNING
Students must independently
operate chemical instrumentation
and interpret instrument outputs.
Scores on faculty-developed rubrics (see
Appendix C) were applied to students’ use
of instrumentation in select experiments in
CHM 3004: Chemical & Instrumental
Analysis Lab.
Two Senior Chemistry majors and one Senior
Chemistry minor took CHM 3004 in Fall 2010.
Additional pre-lab practice problems may
be utilized.
All three students were able to operate the IR
spectrometer and properly interpret their results in
one experiment. All three students were able to
operate the UV-Vis spectrophotometer in two
experiments. However, they all had difficulties
with data interpretation and calculations involving
their UV-Vis data in both experiments. (See
Appendix C.) Students have the opportunity to
practice calculations with UV-Vis data in the
lecture portion of the course (CHM 3003).
Calculations are also reviewed in the pre-lab
discussion. Extra time was spent explaining the
necessary calculations for the second UV-Vis
experiment, but no improvement was achieved. It
is notable that one of the sections on which
students performed poorly on the ACS Analytical
Chemistry exam was on Spectroscopy (see above).
NOTE: The content of CHM 3004 was
redesigned during the summer of 2010 to
incorporate more instruments for the
purpose of providing more opportunities
for this learning objective to be measured.
Therefore, this year’s assessment was
supposed to include additional
instruments, namely the Nuclear
Magnetic Resonance spectrometer, the
Gas Chromatograph, and the Liquid
Chromatograph. However, this
assessment was hindered by technical
difficulties with these instruments. It is
imperative for effective student learning
that properly functioning instrumentation
is available. To that end, a full-time or
part-time lab technician is needed.
Maintenance is also needed annually.
Specific instruments used in this year’s
assessment were the IR spectrometer and
UV-Visible Spectrophotometer.
One of the three students, who consistently sought
additional help with data interpretation in a
different lab course (CHM 2016) last year was
more independent this year.
LEARNING OBJECTIVE
MEASURES USED
RESULTS OF ASSESSMENT
ACTIONS RECOMMENDED TO
IMPROVE STUDENT LEARNING
Students must gain sustained,
hands-on experience in the
scientific research process that
unifies their mastery of chemical
principles and techniques with
problem-solving abilities and
oral and written communication
proficiencies.
Scores on faculty-developed rubrics (see
Appendix D) were utilized in CHM 4450:
Research.
Two Senior Chemistry majors took CHM 4450 in
Fall 2010.
The written component of upper-level lab
courses will be scrutinized to determine if
more emphasis on the identification of
experiment objectives is needed.
Overall, the students’ performance was
satisfactory as evidenced by scores on rubrics
designed to assess their efforts. However, as with
past students, these students had difficulty
identifying the main objective of their project and
describing the purpose and progression of their
experimentation in their final papers. These
problems persisted, despite the fact that the
students were required to submit drafts of sections
of their paper throughout the semester, these drafts
were reviewed by their instructor and returned
promptly, and the instructor discussed her
comments with the students upon returning the
drafts. (Note: Drafts were adopted rather than the
monthly research reports mentioned in the Fall
2008 assessment report.) The students were also
not able to use technical vocabulary correctly in
their papers.
One area in which both students performed very
well was in the application of safe practices in the
laboratory. However, neither student
demonstrated the levels of proficiency and
confidence in performing their experiments that
are expected of a graduating Senior.
1.
More faculty-led discussion about
relevant journal articles is also needed.
Reading and discussing more articles
would help students become accustomed
to appropriate language and format for
scientific writing. However, the extra
time needed for this undertaking warrants
addition of either a Seminar course or
another semester of research. To that
end, more full-time faculty members are
needed.
The eventual adoption of laboratory
practicals that is mentioned above should
help the department identify skill-area
needs earlier in the students’ academic
career. Addressing problem areas earlier
will enable the students to be more
prepared for Senior-level research.
NOTE: To improve the students’ research
learning experience, it is necessary to
invest in properly functioning
instrumentation and library resources.
Are you satisfied overall that your majors are achieving your defined learning objectives?
Chemistry majors are only minimally achieving the department’s learning objectives. This is most apparent when comparing the scores of the majors
to the national standard on the American Chemical Society (ACS) exams. Notably, Chemistry majors from SUNY Purchase, who took courses at
Manhattanville, scored better on the ACS exams than Manhattanville Chemistry majors. It is unclear why this distinction exists. In the laboratory,
students perform satisfactorily overall. However, a higher level of competency is desired.
2.
What program strengths became evident as a result of this assessment?
The department has successfully emphasized the importance of safety in the chemical laboratory. Students also demonstrated high levels of
competency in the application of proper experimental methods and techniques.
3.
What program weaknesses became evident as a result of this assessment?
In the laboratory, students had difficulties making connections between what they learned in lecture and applied in the laboratory. Research students
also had trouble identifying the objectives and goals of their full-semester research projects. More emphasis will be placed on developing these skills
in students.
4.
Are there any institutional-level actions (i.e., those that extend beyond the purview of your department into other areas of the College) that
you would recommend?
Because of the significant difference in ACS exam scores of Manhattanville versus Purchase students, the department perceives a need to work with
Admissions to recruit academically stronger majors.
In their writing courses, students should learn about the appropriate use of synonyms.