Fall 2009
Assessment Sessions
Given by:
Mary Been
Paul Duesing
Sherry Duesing
Lorraine Gregory
Dennis Merkel
Kimberly Muller
7/11/2016
www.lssu.edu
1
Assessment Session III
• Topics
– Program Outcomes (Looking Ahead)
– Assessment Methods for Course Objectives
(Current Task)
– Assessment Experiences and Results
7/11/2016
www.lssu.edu
2
Program Outcomes: Overview
• Target Date of Fall 2010
• Foundation of Assessment Process
– Course Objectives
– Program Outcomes
• Course Objectives: Knowledge & Skills
Obtained in a Course
• Program Outcomes: Knowledge & Skills
Obtained in a Degree Program
7/11/2016
www.lssu.edu
3
Program Outcomes: Perspective
• Course Objectives: Skills Obtained in a Course
– Interpret Data with Statistical Methods
– Organize and Create a Research Paper
– Describe the Process and Steps Necessary for a
Criminal Investigation
7/11/2016
www.lssu.edu
4
Program Outcomes: Skills Obtained in
a Degree Program
• Ability to Communicate in Both Written and
Oral Methods at Multiple Levels
• Ability to Use Software to Solve Problems In
Their Career Field
• Ability to Design Products and Components
• Ability to Use the Scientific Method
• Ability to Create and Use Mathematical Proofs
7/11/2016
www.lssu.edu
5
Program Outcomes: Characteristics
• Knowledge and Skills of the Graduate
• Taught and Re-enforced
– Multiple Courses
– Multiple Years
– Broad Course Objective
• Measurable
– Multiple Indicators
– Multiple Classes
• Try to Limit the Number (3-5)
7/11/2016
www.lssu.edu
6
Assessment Session III
• Assessment Methods for Course Objectives
– Spread Sheet
– Rubric
– Holistic Essay Assessment
7/11/2016
www.lssu.edu
7
Track Results of Course Activities
• Course Objectives
• Spread Sheet
MATH 104
7/11/2016
Activity
Weight
Factor
Points
Weighted
Points
Average
Score
Weighted
Average
Score
EX1-Q1
EX1-Q2
EX1-Q3
Report1
EX2-Q1
EX2-Q2
EX2-Q3
EX2-Q4
Report2
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
40
30
30
100
20
30
25
25
100
40
30
30
100
20
30
25
25
100
31
25
20
75
15
24
21
20
71
31
25
20
75
15
24
21
20
71
Course Objectives
Objective
1
100%
20%
31
5
Objective
2
80%
40%
20
8
20%
3
Objective
3
60%
12
20%
60%
3
14
50%
10
Objective
4
60%
40%
100%
50%
Objective
5
100%
75
100%
71
9
10
21
10
% Quantitative
78.3%
77.5%
75.0%
80.7%
73.0%
% Subjective from Student Questionnaires
85.0%
81.0%
78.0%
81.0%
65.0%
www.lssu.edu
8
MATH 104
Activity
Weight
Factor
Points
EX1-Q1
EX1-Q2
EX1-Q3
Report1
EX2-Q1
EX2-Q2
EX2-Q3
EX2-Q4
Report2
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
40
30
30
100
20
30
25
25
100
Weighted
Weighted Average
Average
Points
Score
Score
40
30
30
100
20
30
25
25
100
31
25
20
75
15
24
21
20
71
31
25
20
75
15
24
21
20
71
Course Objectives
Objective
1
100% 31
20% 5
Objective
2
Objective
3
Objective
4
Objective
5
80% 20
40% 8 60% 12
100% 75
20%
3
20% 3 60% 9
60% 14 40% 10
100% 21
50% 10 50% 10
100% 71
% Quantitative
78.3%
77.5%
75.0%
80.7%
73.0%
% Subjective from Student Questionnaires
85.0%
81.0%
78.0%
81.0%
65.0%
7/11/2016
www.lssu.edu
9
Assessment Session III
• Assessment Methods for Course Objectives
– Rubric
7/11/2016
www.lssu.edu
10
Rubrics
• Holistic rubrics assess a student’s work as a
whole.
– Could be as simple as dividing students work into
four piles (excellent, proficient marginal,
unsatisfactory)
• Analytic rubrics identify and assess
components of a finished product.
– Lesson plan rubric from EDUC420
7/11/2016
www.lssu.edu
11
Assessment Session III
• Assessment Methods for Course Objectives
– Holistic Essay Assessment
7/11/2016
www.lssu.edu
12
Assessment Experiences and Results
• Case History
• Benefits to Students
• Benefits to Faculty
7/11/2016
www.lssu.edu
13
An Example from Mathematics
• Components
–
–
–
–
–
–
Analysis of Objectives (quantitative and qualitative)
Analysis of Success Rates
Feedback from Faculty
Feedback from Students
Performance in Subsequent Courses
Departmental Final Exam
• Macroscopic Lessons
– Guidance for New Faculty is Essential
– Catalyst for Change
– Documents Effective Teaching
• Course Changes
– Teaching Methodology
– Objectives
7/11/2016
www.lssu.edu
14
MATH 111 Fall 2004 Assessment
(Muller) Course Example
•Spreadsheet
MATH 111.003
Activity
Possible
Points
Test 1
Test 2
Test 3
Test 4
Final
65
50
50
50
100
7/11/2016
Course Objectives
Objective
1
Objective
2
Objective
3
15 23 12 20 16 22
22 36
11 12 5 8
74.3% 60.7%
Objective
4
Objective
5
Objective
6
Objective
7
9
8
14
12 13 18 15 20
5 6 28 44
13 23 6 10 2 6 3 6 27 35
63.0% 63.9% 62.5% 71.9% 69.6%
www.lssu.edu
15
MATH 111 Fall Assessment
(Muller) Course Example
• Conclusions
– There are too many topics and none of these
topics are covered with any depth.
– There are few applications that capture
student’s interest.
– There is no motivation for learning exponential
and logarithm functions.
– There is no emphasis on quantitative literacy.
7/11/2016
www.lssu.edu
16
Reasons For Change
• Other Factors / Trends
– National Success Rate for College Algebra is about
50%.
– LSSU Success Rate of 50.4% (data fall 2000 to summer
2005)
– Universities that Changed College Algebra
• Hands-on
• Application-based course
• Cooperative learning environment
– Preliminary Studies Indicate Increased Success Rate By
as Much as 25%.
7/11/2016
www.lssu.edu
17
MATH 111 Course Example: The
Changes
• Initiated Fall 2005
• Summary of Changes
–
–
–
–
7/11/2016
New Measurable Objectives
New Learning Environment
Emphasis on Application/Quantitative Literacy
Emphasis on Use of Technology
www.lssu.edu
18
Spring 2006 Assessment (Muller)
•Assessment Matrix
MATH111-003
Activity
Possible
Points
Test 1
Test 2
Test 3
56
50
50
Test 4
Final Exam
Skills and Objectives
Skill 1a
9
0.8
6
50 1.3
100 6.5
Skill 1b
Skill 1c
Skill 2a
Skill 2b
Skill 2c
Skill 3a
Skill 3c
Skill 3d
Skill 4a
6 7 2.7 3 14 17
4
12 7 10
7
5
5 6 13 16 3
1 0.8 1 0.8 1
7
11 14 20 24
8 3 4
2.5 4 23 34
2 1.3 2 2.8 4 2.7 4
4.9 6.0 4.0 5.0 6.1 8.0 8.7 12 12.0 19.0 4.0 6.0 7.8 11.0
8.0 6.5 8.0
% Quantitative 76.1% 74.4% 72.0% 80.0% 83.8% 80.4% 74.0%
Self Assessment
Skill 3b
4
Skill 4b
10
14
13.0 17.0
71.4%
75.0%
72.9%
75.0%
74.0%
Skill 1a
Skill 1b
Skill 1c
Skill 2a
Skill 2b
Skill 2c
Skill 3a
Skill 3b
Skill 3c
Skill 3d
Skill 4a
Skill 4b
3.1
78%
3.4
85%
2.8
70%
3.3
83%
3.1
78%
3.1
78%
3.2
80%
3.2
80%
2.9
73%
2.9
73%
3.3
83%
3.2
80%
I can solve linear equations and
recognize the linearity of data.
7/11/2016
www.lssu.edu
19
Fall 2005-Summer 2006
Assessment
• Results
– Pass Rate for MATH 111 Increased from 50.7%
to 70.7%.
– Students that took New MATH 111
• Not a Significant Difference in the Pass Rates of
MATH 112, BIOL 280, BIOL 286, BIOL 337, and
CHEM 115
• Pass Rate for BUSN 211 Dropped Considerably
from 81% to 50%
7/11/2016
www.lssu.edu
20
Fall 2006-Spring 2007
• Actions
– Compiled Data
– Met with Representatives of the Business
Department
– Began a Search for a New Text
7/11/2016
www.lssu.edu
21
Fall 2006-Present
• Results:
– Overall success rate in MATH 111 is 67.4% with a pass
rate of 81.2%.
– Students who move on to subsequent classes from
MATH 111:
• MATH 112 success rate increased from 53% to 65.1%.
• BIOL 286 success rate had no significant change at first, fell one
semester, and then increased back to previous level.
• BUSN 211 success rate returned to previous level.
• No significant change in the success rates in BIOL 280, BIOL 337, and
CHEM 115.
• In most of these classes the success rate of those that took the course
after taking MATH 111 exceeds the overall success rate for the course.
7/11/2016
www.lssu.edu
22
Fall 2007
•Began administering a departmental final exam.
Problem
7/11/2016
Description
Problem
Value
Average
Number of
Points
Earned
Average
Written as a
Percent
6 Solve an exponential equation
6
5.09
85
7 Application problem: use exponential
regression
8
5.97
74.6
8 Use the definition and properties of
logarithms
8
4.11
51.3
www.lssu.edu
23
Fall Semesters
•Student Self Assessment
Rate each of the following statements on a scale from 0 to 4. For each
part, choose the number that best describes your ability.
• 0 indicates that you feel that you completely lack an understanding
of the given subject or completely lack the described ability.
• 1 indicates that you have some grasp of the topic and can complete
the easiest examples.
• 2 indicates that you feel that you have an adequate grasp of the topic
and can complete many problems dealing with this topic.
• 3 indicates that you feel that you have a very good grasp of the topic
and feel that you are successful most of time, but that you lack the
intuitive understanding required to apply this topic to future courses.
• 4 indicates that you feel you have mastered this topic and can apply
these concepts to future courses.
7/11/2016
www.lssu.edu
24
Fall 2007
•Partial Results of Student Self Assessment
Percentages
Statement
0
1
2
3
4
2.6
7.79
38.96
37.66
12.99
I can solve linear, quadratic, exponential and logarithmic
equations.
0
3.9
20.78
49.35
24.68
I can analyze and solve systems of equations using
multiple perspectives (equations, tables, graphs,
words)
0
1.3
15.58
63.64
18.18
I can perform operations on functions such as
compositions and inversions.
7/11/2016
www.lssu.edu
25
Challenges
•The process is ongoing.
•Data collection can be difficult.
Rewards
•More students succeed in MATH 111 and go on to
take other courses.
•Assessment results such as these eased the
tenure/promotion process.
7/11/2016
www.lssu.edu
26
Assessment Session III
• Additional Questions
7/11/2016
www.lssu.edu
27