2010-11 Assessment of Student Learning Report
Major (or other aspect of your program) being assessed: Mathematics / Math Education____________________
I.
II.
List the student learning goals for the program. (These should be the broad student learning goals that are
embedded in your departmental mission that remain the same from year to year.)
Goal #1:
Computational competence – ability to correctly perform mathematical computations
such as calculating derivatives, series expansions and solving linear systems
Goal #2:
Conceptual understanding – understanding of fundamental mathematical concepts. For
example, in calculus, in addition to the calculation of derivatives, students will be able to
explain the conceptual interpretation of the derivative and describe the behavior of a function given
knowledge about its derivative.
Goal #3:
Proof writing – ability to construct simple proofs based on newly acquired theorems and
definitions without resorting to mimicry of a similar proof.
Goal #4:
Applications/Modeling – ability to model real-world applications and systems using standard
mathematical methods, solve with appropriate technology and interpret conclusions in the context
of the problem
Goal #5:
Communication – ability to digest and communicate mathematical literature (both in
written and oral form) at the level of undergraduate textbooks and journals (e.g.
Mathematics Magazine).
List 2-3 specific student learning goals in the program that were assessed during the 2010-11 academic
year. (These could either be student learning goals listed in Section I, or parts of one or more of those goals.)
We assessed each student (and the overall program) on each of the five goals listed above.
III.
Describe the methods used during the 2010-11 academic year to assess student learning for each of the
goals identified in Section II. (Methods could be exams, projects, assignments, or other demonstrations of
student learning. A brief explanation of the criteria by which student learning was evaluated is helpful.)
In May we reviewed the portfolios of each graduating senior. These materials are added to the portfolio by
faculty members collected throughout the student’s four years in the major. Materials include: final exams,
significant course projects and presentations, and their senior research paper and presentation. The attached
rubric (Appendix 1) is used to assess student’s SCEs and their overall portfolio. Furthermore, after each
course, the instructor completes the rubric for each of our majors in the course and provides a qualitative
assessment of the student’s strengths, weaknesses and progress. This portfolio and assessment is internal to the
department and not shared with the student.
IV.
Summarize the data that was collected. (A few sentences or bullet points. Examples or data sets may be
attached as an appendix.)
Appendix 2 provides qualitative, quantitative and graphical summaries of our 2011 assessment data along
with comparisons to previous years. We use a 0-to-3 scale based on our assessment rubric (0=Fail;
1=Low Pass; 2=Pass; 3=High Pass). Goal #4: Applications/Modeling was added in 2008 so data is not
available for prior years. The performance of our five recent graduates can be summarized as: two being very
good, two being good and one very weak. Three of the five can be characterized as hard working and
performing at or near their potential, one was very bright but did not push himself accordingly, and one was
extremely weak and never seemed motivated to improve. We can compare the means for each learning
objective to trends seen in prior years.
V.
Goal #1:
As in past years, our students scored highest in computation, with the mean of 2.5 being similar to
the previous four years.
Goal #2:
Students fared somewhat less well in conceptual understanding. However, the mean of 2.3 was a
modest improvement over the past three years.
Goal #3:
The overall average of 2.0 in proof writing ability was significantly higher than the past two years,
largely due to the two top students. We continue to push our students in their abstract thinking and
proof-writing skills. However, this seems to be a talent which most students do not improve upon
after they arrive as first-year students.
Goal #4:
Students performed fairly well in mathematical modeling being able to apply algorithms,
mathematical and statistical tools, and technology to solve non-trivial problems. This year’s mean
of 2.4 was comparable to the previous three years.
Goal #5:
Our graduates’ skills as communicators were similar to the past three years, at 2.1.
State your department’s conclusions from the assessment data. (Your department’s interpretation of the
data.)
Appendix 2 shows that over the past five years (2007 to 2011) our graduates have been fairly consistent in
meeting the learning objectives compared to prior years when there was significant fluctuation from year to
year. We continue to do well at providing our top students the opportunity to grow and excel. Our second tier
(i.e., B students) generally show clear progress and become solid majors (in terms of work ethic and
foundational skills) though not always developing much creativity. Unfortunately, our weaker students often
struggle throughout their entire four years of math courses never showing much intellectual growth or
initiative; they are still struggling as seniors just to get a C.
All students who wish to be licensed in Indiana as secondary mathematics teachers must pass the Praxis II test
Mathematics: Content Knowledge. The passing score set by the state of Indiana is 136 (maximum possible
score is 200; minimum possible is 100). We have been collecting our graduates’ Praxis II test scores since
2003. The graph in Appendix 4 shows that our graduates’ scores have been relatively stable, with no
statistically significant trend over time (P=.188). Compared to national norms, our graduates generally fall
between the median (144) and the 75th percentile (159). Out of 23 students observed, four have scored above
the 75th percentile while four students scored below the median but still above passing. This year’s group was
fairly typical with a mean score of 153, identical to the mean for past years (when an outlier, Sam Wysong’s
near perfect score of 198, is removed) We also reviewed the subscores for the 13 most recent students who
took the Praxis II exam. (The education office does not have the subscore data prior to 2007.) Appendix 5
indicates the percent of correct answers in five subject areas: (1) Algebra and Number Theory, (2)
Measurement, Geometry and Trigonometry, (3) Functions and Calculus, (4) Data Analysis, Statistics and
Probability, and (5) Matrix Algebra and Discrete Mathematics. Our students generally average in the low
60%’s for areas (1) and (2), the low 70%’s for (3) and high 60%’s for area (4). While these absolute scores do
not look overly impressive, compared to national scores they are near or above the 75th percentile. Students did
poorer in (5) Matrix Algebra and Discrete Mathematics, with scores in the low 50%’s, (still above the national
median). This year’s group was similar to previous years (again, after omitting Sam Wysong) but showed some
improvement in areas (2) and (5). As a whole, we are graduating majors who are mathematically well prepared
to become secondary mathematics teachers.
VI.
Describe how your department will use these conclusions to improve student learning.
Based on the above discussion this year’s graduates were quite similar to 2010 and thus our overall conclusions
remain the same:
Goal #1:
Most of our majors come in with a high level of computational ability and they progress
appropriately as they advance through more demanding courses.
Goal #2:
Most students enter the program with an adequate understanding of mathematical concepts. Those
who struggle generally do not continue in the major. We will continue evaluating specific areas of
conceptual understanding on selected final exams (and the major field tests) so we can identify
where we need to increase emphasis on particular concepts or change teaching strategies..
Goal #3:
As in the past we have seen less than stellar skills in constructing mathematical proofs. This
indicates weakness in students’ ability to reason abstractly and to organize mathematical ideas into
a formal logical argument. We are working on this by designating selected P courses as stated in
Item 1 in Part VIII, below.
Goal #4:
We will continue to emphasize mathematical modeling, applications and use of technology in
appropriate courses such as: Calculus II, Differential Equations, Linear Algebra I and II, and
Operations Research.
Goal #5:
We will continue to emphasize clear written and oral communication throughout the math
curriculum. We believe our efforts here are bearing fruit and the senior project is especially
beneficial in this regard.
At our annual assessment meeting (held in May) our department goes over the data presented in this report.
Rather than focusing on the minutia of data, this year we spent much more time reflecting upon what we could
do to help our majors be more successful. The issues and responses that we identified follow.
1. We do not do particularly well advising students who struggle in their courses and in the major as a
whole. It seems each spring we have a junior who probably should have been advised out of the major
when they were a sophomore. We would like to be more proactive in identifying struggling students
and either work with them to improve or advise them to switch majors. We will begin addressing this
three ways:
a. More consistent advising by keeping math majors with the same advisor throughout their
entire four years. We will rotate primary advising of FY students among department members.
Each faculty member will thus have a cohort of advisees from first year through graduation.
b. The department will discuss each sophomore’s portfolio in the February before they officially
declare a major. Advisors can help students identify their strengths, areas for improvement,
and potential career paths (including whether the math major is a good choice for them).
c. From time to time we will discuss (at department meetings) those majors who are consistently
struggling.
2. We would like to instill a greater sense of professionalism and career direction in our majors. Having a
clearer sense of direction often improves a student’s classroom performance. Having lower-level
students interact more with upper-level students can also be beneficial in this area.
a. We will institute a twice-monthly departmental lunch series. This will consist of reflections by
an invited guest followed by roundtable discussion. Possible topics include: what it means to
be a professional in mathematics, computing or teaching; student experiences in classroom
observation and student teaching; student experiences in summer research; applying and
pursuing graduate studies; and career options in general.
b. We had our first-ever departmental retreat in September — an overnight for students and
faculty at Koinonia. This was deemed successful in helping students get to know faculty and
facilitating interaction between new students and upper-levels. We will continue this new
tradition.
c. We had some discussion about setting up a mentoring program between juniors/sophomores
and first-years. However, no decision has been made on implementing such a program.
3. We discussed possible changes to the math education program in light of evolving state requirements.
We are being required to more clearly document inclusion in our curriculum of technology (e.g., use of
calculators and mathematical software) and the history of mathematics. Furthermore, we have had
some discussion with our Education Department and our math education majors about the potential
benefits of creating a separate secondary methods course for mathematics majors (as we once had).
This would be a way to systemically cover the above two topics (rather than just checking boxes) and
have significant reflection on the nuances of teaching mathematics (not just teaching in general).
Additionally, the department has never come to grips with how the current mathematics senior research
requirement will be handled for “math ed” majors; students who now must complete the full
mathematics major (with an education minor), rather than the old major in secondary ed math. We will
continue these discussions with students and education faculty this fall.
4. We discussed how students were performing in a number of foundational courses:
a. Calculus I – We need to pay more attention to our weakest students who tend to struggle and often
don’t take the initiative to ask for help. There was also some discussion whether the placement
threshold should be higher for this course. We see students who placed into calculus but are quite
shaky in a number of algebra basics. We continue to discuss whether use of the online homework
system WebAssign helps or hinders weaker students.
b. Calculus II - Again, there were students who made it through Calc I to Calc II but still struggled
with algebra. We have not used WebAssign in Calc II but because of its use in Calc I we now see
students at the next level having little experience formally writing out solutions—argh! We also
have more and more pre-professional students (e.g., pre-med, pre-pharm) advised into Calc II by
the science faculty. This seems good in principle. However, many seem to have little motivation:
“Just tell me what I need to do so I can get a C.” Up until two years ago this was a class of 10 to
15 students (mostly math and physics, with some chemistry) with good to excellent backgrounds
and motivational levels. Now we have close to 30 students with a third of them lacking both.
Argh again! Should we try a Calculus for Life Sciences course? We will consult with other
faculty to see how we might address this.
c. Discrete Mathematics – We changed instructors and textbooks this year. It seemed that the text
was a little too challenging and possibly the expectations as a very proof-intensive course were as
well. The instructor will adjust as appropriate this year.
d. Computer Programming I – Our six first-year computer science majors all did poorly in this class.
We are considering moving away from an object-orientated programming course (focusing on
Java) to a course emphasizing basic programming techniques and algorithmic principles. This
would entail using a simpler programming language such as Python. This will likely prove more
useful to students in other majors as well.
VII.
List 2-3 specific student learning goals in the program that your department wishes to assess for the
2011-12 academic year. (These could either be student learning goals listed in Section I, or parts of one or
more of those goals. You may decide to reassess the same goals or move on to other goals.)
We will continue assessing the same five high-level goals for the majors as well as the specific learning
objectives listed in Appendix 3.
VIII.
Describe the methods to be used during the 2011-12 academic year to assess the student learning goals
identified in Section VII.
This year we continued implementing our revised assessment plan (provided to the Assessment Committee on
February 14, 2008). We continue to implement additional pieces of this plan each year. Parts that we are still
working on include:
a. In our assessment plan we designated one course at each level (FY through SR) as a proof intensive (P)
course in which we will emphasize proof-writing and assessment of these skills. We began assessment of
this in 2009-10 and will continue to do so in coming years.
b. We are now in our third year of adding specific comments to each major’s portfolio upon completion of a
course in our department. We can begin using this information more systematically in 2011-12 as we work
to track progress over time for our majors.
We found the open-ended discussion and brainstorming at this year’s departmental assessment meeting (as
summarized in Section VI) to be quite helpful. We plan to continue this in future years
Appendix 1: Evaluation Rubric for Mathematics and Math Education Majors
Computation
Conceptual
Understanding
Proof Writing
3
Applies
previously
learned
computational
methods in new
contexts;
presentation of
solution is
correct, clear
and complete.
2
Knows methods
required to
solve a variety
of problems;
accurately
performs
computational
tasks.
Understands key
mathematical
concepts; can
explain their
significance in the
context of a given
problem; can apply
concepts in a
variety of concrete
and abstract
settings.
Understands key
mathematical
concepts; can
explain their
significance in the
context of a given
problem.
Includes clear
statement of what is
to be proven;
progresses in a
consistently clear
fashion from
hypothesis to
conclusion; logical
steps are justified in
appropriate level of
detail.
Includes clear
statement of what is
to be proven;
progresses in a
mostly clear fashion
from hypothesis to
conclusion; contains
a few gaps in
reasoning.
1
May need to be
told specific
method required
to solve a given
problem; knows
basic steps to
follow but often
makes mistakes
in performing
computations.
Can state key
mathematical
concepts but
understanding is
superficial; has
difficulty in
explaining
meaning of a
concept in a
specific context.
Includes reasonable
statement of what is
to be proven;
progresses in a
mostly clear fashion
from hypothesis to
conclusion; contains
major gaps in
reasoning.
0
Does not know
Cannot state key
basic steps
mathematical
required to solve a concepts correctly;
common problem has little or no
types.
understanding of
their significance.
3 = High Pass;
Statement of what is
to be proven unclear
or missing; major
logical steps omitted;
needed justifications
are incorrect or
missing.
2 = Pass; 1= Low Pass;
Mathematical
Applications
and Modeling
Can summarize complex,
unstructured problems,
applies known modeling
techniques to a more
complex situation,
efficiently applies
technology to implement
a model.
Written and Oral
Communication
Can explain problems of
moderate difficulty,
applies known modeling
techniques to problems
types previously studied,
uses technology to
implement a model.
Can read, digest and
interpret undergraduate
mathematics literature,
requiring some assistance
from the instructor;
concisely and clearly
communicates
computational and
algorithmic concepts in
written and oral form.
Can understand
undergraduate
mathematics literature,
with significant
assistance from the
instructor; can, with some
imprecision,
communicate general
mathematical ideas and
procedures in written and
oral form.
Can summarize a
problem though may not
understand all
interactions involved,
with assistance can
represent problem using
indicated modeling
techniques, can
understand results of
solved model though
might not be proficient
with technology.
Unable to describe the
structure of applied
problems, unclear on
how standard modeling
techniques can be applied
to straightforward
problems, unable to
apply technology to
problem solving.
0 = Fail; NA = Not Applicable/Observed
Evaluator: ________________________________
Date: ________________
Material Evaluated: _________________________________________
Comments:
Can independently read,
digest, and interpret
undergraduate
mathematics literature
(e.g., Mathematics
Magazine); concisely and
clearly communicates
abstract mathematical
concepts in written and
oral form.
Has significant difficulty
understanding
mathematical information
when presented in written
form; cannot effectively
communicate
mathematical concepts to
others.
Appendix 2:
Assessment of 2011 Mathematics and Mathematics Education Graduates
Strengths, weaknesses and progress of five 2011 graduates:
Student 1 – upon arriving was quite weak in abstract thinking and conceptual understanding; did not seem to make much
effort to improve; even in course that were more computation rather than theory the effort put forth was not impressive
Student 2 – a strong student throughout; very conscientious and thorough in completing work; mastered computations and
algorithms; developed nicely in confidence, professionalism and communication skills.
Student 3 – very good with computations but still struggled with connecting and explaining theoretical ideas; improved
somewhat in written and oral communications
Student 4 – very strong in abstract reasoning and applying new theory to applied problems; was always scattered among
too many commitments to really full effort to anyone of them.
Student 5 – very good with computations but still struggled with connecting and explaining theoretical ideas; able to orally
communicate mathematical ideas very well
Assessment of 2011 Math/Math Ed Graduates
Goal
Computation
Concepts
Proof Writing
Modeling
Communication
Std 1
Std 2
Std 3
Std 3
Std 3
Mean
Std
Dev
1.6
0.9
0.8
1.0
0.7
2.8
2.7
2.8
2.9
2.9
2.5
2.3
1.5
2.5
2.3
3.0
2.9
2.8
3.0
2.6
2.5
2.5
2.0
2.5
2.0
2.5
2.3
2.0
2.4
2.1
0.57
0.82
0.86
0.82
0.88
(0=Fail; 1=Low Pass; 2=Pass; 3=High Pass)
Mean Scores for 2002-2011
Goal
Computation
Concepts
Proof Writing
Modeling
Communication
2002
(n=3)
2003
(n=2)
2004
(n=6)
2005
(n=3)
2006
(n=4)
2007
(n=6)
2008
(n=4)
2009
(n=4)
2010
(n=3)
2011
(n=5)
2.6
1.8
1.5
2
1.8
1.4
2.7
2.9
2.2
2.1
1.8
1.6
2.8
2.6
2.3
2.5
2.4
1.9
1.4
2.5
1.9
2.8
2.5
2.5
2.1
1.9
2.2
2.1
2.4
2.0
1.6
2.3
2.2
2.4
2.1
1.7
2.3
2.2
2.5
2.3
2.0
2.4
2.1
3.0
2.5
2.0
Computat
ion
Concepts
1.5
1.0
Proof
Writing
0.5
0.0
2002200320042005200620072008200920102011
Appendix 3:
Key Learning Objectives for Mathematics & Math Education Majors
A. Analytic Geometry and Calculus
MATH 121 Calculus I
1.
2.
3.
4.
5.
Understanding of limits as they relate to functions and graphs
Conceptual knowledge of the process of differentiation
Applying calculus to model and solve problems involving rates of change, optimization,
Basic proficiency in using a graphing calculator to interpret function behavior and limits.
Conceptual knowledge of the process of integration
MATH 122 Calculus II
1. Application of integration to applied problem-solving (e.g., in the sciences, economics, etc)
2. Use of computer algebra system (such as Maple) to do nontrivial problem solving
3. Understanding and application of infinite series
B. Abstract and Linear Algebra
MATH 251 Linear Algebra I
1. Use of matrices and matrix operations to represent mathematical relationships
2. Solving of systems of linear equations
C. Probability and Statistics
MATH 240 Mathematical Statistics
1. Collecting, organizing, analyzing, and interpreting data
2. Concepts of dispersion and central tendency
3. Relationships between 2 variables represented by scatterplots
4. Correlation and regression
5. Use and interpretation of common continuous probability distributions (e.g., normal and uniform)
D. Geometry
MATH 306 Geometry
1. Understanding of symmetry, congruence, similarity, measurement trigonometry
2. Knowledge and application of the axiomatic method
E. Mathematical Models, Applied Mathematics or Computer Science
MATH 130 Discrete Mathematics
1.
2.
3.
4.
5.
6.
Use of deduction reasoning as part of a mathematical system
Understanding of basic topics in number theory
Computation using modular arithmetic
Understanding and application of induction, and recursion
Properties and applications of graphs and trees
Application basic combinatorics (e.g., basic counting law, permutations, combinations)
Appendix 4:
Praxis II Mathematics Score versus Year of Graduation
Regression
Model Summary
Std. Error of the
Model
1
R
R Square
.285a
Adjusted R Square
.081
Estimate
.037
12.613
a. Predictors: (Constant), YearGrad
Coefficientsa
Standardized
Unstandardized Coefficients
Model
1
B
(Constant)
YearGrad
a. Dependent Variable: Praxis
Coefficients
Std. Error
Beta
-2387.783
1866.868
1.266
.930
t
.285
Sig.
-1.279
.215
1.362
.188
Appendix 5:
2007-2011 Praxis II Mathematics Subscores (n=13)
Knowledge Area
1. Algebra and Number Theory
2. Measurement, Geometry and Trigonometry
3. Functions and Calculus
4. Data Analysis, Statistics and Probability
5. Matrix Algebra and Discrete Mathematics
Percent
Correct
63.5%
66.0%
74.7%
69.2%
55.8%
Percentiles within National Scores (*)
< 25th 25th
25th-75th 75th
> 75th
1
1
2
3
5
5
2
3
3
3
4
2
3
4
* number of Manchester students (out of 13)
that fall in each category
4
3
9
5
3