Metropolitan State University of Denver
Assessment Report Template
2012-13 annual report
Program Name Meteorology
Program Description – include a brief description of the program including a list of majors, minors, concentrations as applicable and
the number of students in each, and the number of faculty by category.
The Meteorology program is housed in the Department of Earth and Atmospheric Sciences. The program offers a major and a minor in
meteorology (no concentrations in either). It is difficult to get an accurate count of the number of majors and minors, but we have approximately 60
majors and 30 minors. There are 3 Category 1 faculty members, two tenured and one tenure-track. One Category 2 faculty member also teaches
upper and lower division courses in meteorology courses, but also contributes to the Land Use program. There were five Category 3 faculty
members who taught MTR courses during the 2012-13 academic year.
Student Learning Outcomes (SLOs)
1. Describe the general characteristics of the atmosphere, including physical processes and weather systems.
2. Select and interpret appropriate weather and climate data, including in-situ and remotely sensed information, for different situations.
3. Synthesize multiple types of weather and climate data to formulate short, medium, and long-range weather forecasts.
4. Organize, analyze, and prepare written scientific reports.
5. Create and deliver scientific presentations using multimedia techniques.
6. Apply mathematical and statistical techniques to the analysis and interpretation of atmospheric dynamics, thermodynamics, and radiation
processes.
7. Apply scientific computing skills using appropriate software and structured programming.
8. Evaluate social, economic, cultural, and global aspects of the impacts of weather and climate phenomena.
Data collection
For each category, three types of data
were collected.
Findings
See findings for individual SLO’s
1. Artifacts were collected and rated
according to rubrics for each of
the SLO’s. The ratings were on a
scale of 1 to 4.
Target/Expectation
The target is for each
SLO to achieve an
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
Action and Rationale
We plan to change the labels
to:
The target is for each
SLO to achieve an
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
We are strongly considering
replacing the current
knowledge exam and would
appreciate input from the
assessment peer reviewers.
4 = well prepared
3 = adequately prepared
2 = moderately prepared
1= poorly prepared
2. Faculty{3 Category 1, 1 Category
2, and 1 Category 3} rated
graduating seniors on their
preparation in each category. The
ratings were on a scale of 1 to 4,
with no rating if a faculty member
did not have sufficient
information to rate particular
students. For each student, an
average faculty rating was
determined for each SLO.
3. Students were asked to rate their
own preparation in the areas
delineated by the eight SLO’s.
SLO #1: 100 question multiple choice
exam implemented in the MTR 4400
Advanced Synoptic course in Spring 2013.
This exam has been given to graduating
seniors for over twenty years and
therefore provides valuable comparative
data. See comments in the Action and
Rationale column.
Students scored near the 16-year average for
this exam. The students scored particularly
poorly on the physical meteorology and
weather systems sections. This cohort was
nine students, so it is difficult to interpret
whether this represents a meaningful change
in the program.
See Appendices A and B for more details.
In future, the
expectation is for 85%
of students to achieve a
rating of 3 or 4.
The exam which has been
used for 16 years does
provide some measure of
comparative information.
However, some of the
questions seem somewhat
arcane and in general
emphasize recall and
vocabulary and not enough
higher order thinking.
SLO #2: Three data analysis assignments
requiring selecting and analyzing climate
data in the Spring 2013 MTR 4440
Climatology course were used to assess
this outcome.
SLO #3: Case study portfolio produced
for MTR 3410 Weather Analysis
Techniques course in Spring 2013
required synthesizing various types of
weather data for a comprehensive
analysis.
SLO #4: Evaluation of final written
reports for MTR 4600 Senior Research
Seminar course in Fall 2012.
Students were well prepared or adequately
prepared based on sampled work products.
The average score on this SLO (3.38/4)
continues to be one of the highest. Both the
faculty and student perception were quite
high. The inconsistency between the
different rating measures call for more
specific rubrics for the faculty and student
ratings.
Students were mostly adequately prepared
based on sampled work products. The overall
rating was 2.94/4 but that was biased by two
students receiving a poorly prepared (1)
rating. Furthermore, the course for which this
portfolio was drawn enrolls many minors in
addition to majors. The minors tend to be
less prepared, although the results have not
been separated. It may make sense to
include only the results from the majors.
Both the faculty and student perceptions
were quite high. The inconsistency between
the different rating measures call for more
specific rubrics for the faculty and student
ratings.
Students were mostly adequately prepared
based on sampled work products. The low
student self-ratings on this SLO may be
attributed to higher expectations. The
students are reading more professional
research articles and presenting at
conferences, so they are often not satisfied
with their efforts. The faculty have a better
feel for reasonable expectations for
undergraduate students.
The target is for each
Maintain current emphasis
SLO to achieve an
on this SLO.
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
In future, the
expectation is for 85%
of students to achieve a
rating of 3 or 4.
The target is for each
Maintain current emphasis
SLO to achieve an
on this SLO.
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
In future, the
expectation is for 85%
of students to achieve a
rating of 3 or 4.
The target is for each
SLO to achieve an
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
Maintain current emphasis
on this SLO. Future plans
are to assess how students’
scientific writing evolves
through a series of courses
culminating in the senior
research seminar.
SLO #5: Evaluation of final oral reports
utilizing PowerPoint for MTR 4600
Senior Research Seminar course in Fall
2011.
SLO #6: Evaluation of 4 problem sets for
MTR 3430 Atmospheric Thermodynamics
course in Fall 2011 requiring
mathematical techniques and creative
problem-solving.
SLO #7: Students in MTR 4400
(Advanced Synoptic Meteorology) were
given a series of exploration labs that
required them to teach themselves to use a
data visualization software commonly
used in the field of atmospheric science
called IDV. Students found data, ingested
it into the program, and created complex
Students were mostly adequately prepared
based on sampled work products. See
Appendix A. The low student self-ratings on
this SLO may be attributed to higher
expectations. The students are reading more
professional research articles and presenting
at conferences, so they are often not satisfied
with their efforts. The faculty have a better
feel for reasonable expectations for
undergraduate students.
The consistently low ratings seem to indicate
an area where improvement is needed.
Given the low mathematics preparation of
incoming students, the results are somewhat
positive. Students are required to complete a
24-credit hour mathematics minor including
3 semesters of calculus and 2 additional
courses with calculus prerequisites. Several
of the theoretical courses include
considerable application of calculus and
other mathematical methods.
This is clearly the weakest component of the
program. See Appendix A. The single
required computer science course is not
adequate. Prior to 2011, there was no
dedicated computer lab (a specialized UNIX
lab is needed) so that applications of
structured programming including scripts
and data management were not adequately
In future, the
expectation is for 85%
of students to achieve a
rating of 3 or 4.
The target is for each
SLO to achieve an
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
In future, the
expectation is for 85%
of students to achieve a
rating of 3 or 4.
The target is for each
SLO to achieve an
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
In future, the
expectation is for 85%
of students to achieve a
rating of 3 or 4.
The target is for each
SLO to achieve an
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
Maintain current emphasis
on this SLO. Future plans
are to assess how students’
scientific presentation skills
evolve through a series of
courses culminating in the
senior research seminar.
Maintain current emphasis
on this SLO. The
Atmospheric
Thermodynamics course
added weekly problemsolving quizzes to “train”
students in create problem
solving in a short period of
time (also addresses testanxiety for some students).
The results confirm those
from the past two years,
which found that this is the
area we need the most work
on. New computer
programming assignments
and experiences are being
designed for virtually every
weather maps they used to answer general
questions in lab assignments that related
live weather to the meteorology equations
related to the course. Some maps
required coding simple calculations for
the computer to calculate, such as
advection of vorticity. Students, through
both teamwork and individual efforts,
were able to create maps with several
layers of information and adjust contour
intervals and shading thresholds to create
visually pleasing and information-rich
weather maps. Students used maps for
lab reports as well as case studies. This
lab required them to look at the parts of
the Frontogenesis equation for a cyclone
event.
SLO #8: Evaluation of essay exam
questions concerning climate change
impacts taken from the MTR 4440
Climatology course in Spring 2012.
covered.
The program is undertaking a several year
plan to incorporate computer programming
into the entire curriculum starting with the
lower-division courses in order to take full
advantage of new facilities.
In future, the
expectation is for 85%
of students to achieve a
rating of 3 or 4.
Senior exit surveys are highly critical of the
required Computer Science I course, both in
the computer language (JAVA) and the
emphasis on software engineering rather than
problem solving through structured
programming.
See Appendix A. The data collected suggests
that while some students are well prepared,
some are merely adequately prepared.
course in the major. This is
a process that will probably
take 5 years to fully
implement, with the
assistance of our UNIX
administrator and possible
computer science faculty.
We are hoping to find a way
for Computer Science to
offer a “Programming for
Scientists and Engineers”
course. Ongoing discussions
with the Mathematics and
Computer Science
Department are planned.
The target is for each
Maintain current emphasis
SLO to achieve an
on this SLO.
average rating of
greater than 3.0 where:
4 = well prepared
3 = adequately prepared
2 = not well prepared
1= poorly prepared
In future, the
expectation is for 85%
of students to achieve a
rating of 3 or 4.
Process for interpretation of findings – Describe the structure of responsibilities for program assessment. Specify processes
undertaken for faculty review of findings prior to submission of this report. How did the program come to the decisions it made?
Because the SLO’s are only three years old, and the number of students each year is small, any trends in the data would be impossible to discern
from noise in the signal. The plan is to compare 3-year time periods (i.e. 2013-2016 vs 2010-2013) once sufficient data is available. The exam
used to assess SLO #1 has been used for more than 15 years, with no decipherable trends. The current set of SLO’s reflect the knowledge and skills
expected of program faculty, which now emphasize various critical thinking and process skills in addition to “content knowledge”.
Response to prior peer review report(s) - Describe the ways in which the program has responded to any prior year’s peer review
report. What kind of continuous improvement cycle are you using?
The major problem cited with the prior year’s assessment report was associated with using an old reporting template, making it difficult for the
reviewers to comment on certain items. This year’s assessment report used the new reporting template, and specifically addressed missing items by
adding target performance expectations, trend analysis, description of faculty involvement in interpreting findings, and actions taken in
response to the assessment results.
Plans for the program assessment process – To what extent does the program assessment process need modifications? Describe
any plans for modifying the program assessment process. Is it the program’s intent to gather data about every outcome every year? If
not, what is the proposed data collection cycle?
Because the number of students in each senior class is small, the program plans to collect data about every outcome every year and aggregate results
into three-year periods. One area of improvement in the current assessment protocol is to improve the rubrics used for rating individual students
which then are aggregated into average ratings.
Implementation plan for applicable program changes - , Summarize the changes described in the Action and Rationale columns
above and specify the implementation plan including a timeline.
One change coming out of the assessment reviews over the past few years was a revision of the laboratory components of several courses in the
form of a major curriculum revision in 2012-13. This involved conversion of lecture to lab hours for three courses. In addition, almost all of the
courses are incorporating more computer programming and using weather visualization and analysis software into the courses; in meteorology, most
of the “lab” work is on the computer. In 2013-14, we will launch a new introductory lab course, which will include an introduction to computer
programming and UNIX, in addition to using specialized computer software.
Some results in conjunction with an exit exam indicate some measures that will require collaboration with other departments in terms of potential
changes for their service courses. One example is the computer science requirement, for which most students now take Computer Science I course
which is better suited for computer scientists than scientists/engineers. The meteorology program is incorporating more computer applications
including coding into many courses to complement the one-course formal computer science requirement. We plan to continue discussions with the
Computer Science faculty in terms of the shortcomings of the Computer Science I course for the needs of meteorology majors.
We may initiate discussions with the Physics Department concerning dissatisfaction with the content in the General Physics II course (students and
faculty are very satisfied with the General Physics I course content). The entire course is devoted to Electricity and Magnetism, and should include
some relevant material on thermodynamics, waves, and fluid flow. There is some concern about the very large sections for the physics courses
(close to 100 students, Metro State and UCD students combined). The students already know to take the courses from Metro State, rather than UCD
professors, if possible.
Appendix A: Summary Statistics
The chart below indicates the average rating scores for the three rating data sources for each of the 8 SLOs. This can be compared with the second
chart showing the average form the previous 2 academic years. The current SLO’s have been in place for 3 years and the student numbers are small
enough that year to year comparisons are not warranted.
4
3.5
3
Student Work
2.5
Student Survey
Faculty Survey
2
1.5
1
1
2
3
4
5
6
7
8
4
3.5
Student Work 2010-11
& 2011-12
3
2.5
Student Survey 201011
2
Faculty Survey 2010-11
1.5
1
1
2
3
4
5
6
7
8
Appendix B: Assessment Exam Results
An objective exam is given to senior-level Meteorology majors in MTR 4400 Advanced Synoptic Meteorology in the spring semester. The
exam is comprised of 100 questions, divided into four 25-question categories. This exam is used as a tool to assess Student Learning
Outcome #1. Success rates for specific questions aggregated over several years allow for more diagnostic assessment.
1.
2.
3.
4.
Category 1, General Characteristics of the Atmosphere.
Category 2, Physical Processes.
Category 3, Weather Systems.
Category 4, Observation and measurements.
Trend Data
Year
General
Physics
1996
20.1
17.2
19.4
22.0
78.7%
1997
22.0
20.9
20.1
21.3
84.3%
1998
21.3
20.2
20.3
22.0
83.8%
1999
21.8
20.6
21.2
21.5
85.1%
2000
21.7
21.0
21.3
20.8
84.8%
2001
23.3
21.1
22.0
21.9
88.3%
2002
20.7
21.7
20.0
20.3
82.7%
2003
22.0
19.8
21.4
21.4
84.6%
2004
23.2
21.6
21.4
21.2
87.4%
2005
22.2
19.5
21.2
21.8
84.7%
2006
21.3
21.3
21.0
21.3
84.9%
2007
20.7
19.8
18.5
20.4
79.4%
2008
21.7
20.8
20.5
20.7
83.7%
2009
21.1
20.4
20.1
19.9
81.2%
2010
21.1
20.0
18.4
20.5
2011
21.9
20.5
20.9
20.5
83.8%
2012
20.2
17.2
18.2
20.8
76.3%
19.7
21.2
17.9
20.0
19.4
20.1
21.6
20.8
78.6%
82.3%
2013
Average
Systems Observations
Total
80.0%
Appendix C: Curriculum Map
Objective 1
Objective 2
Objective
3
General
MTR
Weather Data
Forecast
Objective 4
Objective 5
Written Comm.
Oral Comm.
Objective 6
Math
Objective 7
Objective 8
Computing
Impacts
V
V/P
V
Required
MTR 2400
MTR 2410
MTR 3400
MTR 3410
MTR 3430
MTR 3440
MTR 3450
MTR 4400
MTR 4440
MTR 4500
MTR 4600
Electives
MTR 3100
MTR 3420
MTR 3500
MTR 4210
V
V
V/P
P
V/P
V/P
D
P
D
V/P
V
V
P
P
P
P
V
V
V/P
D
P
P
V/P
V/P
V
P
D
D
D
D
V
P/D
D
P
D
P
D
P
D
P
P
D
P
D
D
P
P
D
P
V/P
P
P
D
D
P
D
Levels - These are stated from the student's perspective and tied to Bloom's Taxonomy as much as possible.
V = discover (gain knowledge, comprehend information)
P = practice (apply knowledge gained to real situations, analyze issues and questions)
D = demonstrate (prepare a work product - exam, paper, presentation, etc. - that represents knowledge gained, application of
same, and synthesis or evaluation of knowledge and ideas)

a course might be identified as offering students the opportunity to do more than one level
Appendix D: Sample Rubrics
Rubric for SLO #4
Organize, analyze, and prepare written scientific reports
Depth and
Quality of
Research
Information
Sources
4
Creative and original topic.
Rigorous, well-documented
and reasonable methodology
using appropriate data
sources. Considerable time
and effort in research.
Accurate use of background
information, using a wide
range of relevant and
reliable sources.
Organization
&
Presentation
Main ideas are clearly
presented and in an
appropriate order; Ideas are
supported by information
and logic; Effective use of
diagrams, charts, and graphs
Conventions &
Science
Language
Generally error free in
regards to sentence
structure, punctuation,
capitalization, spelling, and
standard usage. Consistent
use of appropriate science
language and terminology.
3
Appropriate and mostly welldocumented methodology.
Some minor deficiencies in the
choice or description of data
sources and methodology.
2
Topic choice not inspiring.
Some major deficiencies in
the choice or description of
data sources and
methodology.
1
Little effort in research of the
topic. The methodology used
is unclear or poorly designed.
Mostly accurate and
appropriate use of background
information, using a wide
range of relevant and reliable
sources.
Main ideas are mostly clearly
presented and in an appropriate
order, although some elements
are not thoroughly covered;
Ideas are supported by
information and logic; Mostly
effective use of diagrams,
charts, and graphs.
Mostly error free in regards to
sentence structure, punctuation,
capitalization, spelling, and
standard usage. Consistent use
of appropriate science language
and terminology
Some reliable sources, but
additional background
information needed. Some
deficiencies in citing
sources
Ideas are not presented in
an order that adds clarity;
Some ideas are supported
by information and logic;
Some appropriate use of
diagrams, charts, and
graphs
Insufficient use of relevant
sources. Narrow range of
resources; unreliable or out of
date sources.
Sentence structure,
punctuation, capitalization,
and spelling errors are
noticeable. Partial use of
appropriate science
language and terminology
Ideas are presented in an order
that distracts from clear
communication; Ideas are not
supported by information and
are illogical and major
components are not included;
No use of diagrams, charts, and
graphs
Errors in sentence structure,
punctuation, capitalization,
spelling, and standard usage
impair readability. Inaccurate
use of science language and
terminology
Rubric for SLO #5
Create and deliver scientific presentations using multimedia techniques
Speaks Clearly
Speaks clearly and
distinctly all the time, and
mispronounces no words.
3
Student seems pretty prepared
but might have needed a couple
more rehearsals.
Main ideas are mostly clearly
presented; Ideas are presented
in an appropriate order,
although some elements are not
thoroughly covered; Ideas are
supported by information and
logic; Appropriate conclusions
are based upon evidence
presented;
Speaks clearly and distinctly
almost all of the time, but
mispronounces one word.
Graphics
Graphics are easily read
and well-labeled
Most graphics are easily read
and well-labeled
Preparedness
Organization
4
Student is completely
prepared and has
obviously rehearsed.
Main ideas are clearly
presented; Ideas are
presented in an appropriate
order; Ideas are supported
by information and logic;
Appropriate conclusions
are based upon evidence
presented;
2
The student is somewhat
prepared, but it is clear that
rehearsal was lacking.
Ideas are not presented in an
order that adds clarity; Some
ideas are supported by
information and logic;
Conclusions do not follow
from ideas presented;
Speaks clearly and
distinctly most of the time.
Mispronounces no more
than one word.
Some graphics are easily
read and well-labeled
1
Student does not seem at all
prepared to present.
Ideas are presented in an
order that distracts from
clear communication; Ideas
are not supported by
information and are
illogical and major
components are not
included; Inappropriate
conclusions are presented.
Often mumbles or can not
be understood OR
mispronounces more than
one word.
Many graphics are not
easily read and well-labeled