Assessing the Effectiveness of a
Learning Community Course Designed
to Improve
the Math Performance of First-Year
Students
Michele J. Hansen, Ph.D., Director of Assessment, University College
Susan G. Meshulam, Lecturer Department of Mathematical Sciences and Adjunct
Faculty University College
Brooke N. Watson, Graduate Assistant for Assessment, I/O Psychology Master’s
Student
E.C. Moore Symposium
March 4, 2010
Indianapolis, Indiana
Research supported by IUPUI University College Faculty Fellowship
IUPUI – University College

Indiana University Purdue
University Indianapolis (IUPUI)
 Downtown Indianapolis
 Public comprehensive four year
institution
 Over 200 academic programs
 Doctoral/Research Intensive
 Enrollment - 30,383
• Undergraduate - 22,119
• Graduate - 8,264
First-Year Students

Large number of under-prepared first-year students
struggling to balance work and school commitments.

7% admitted conditionally.

41% of first-year students are first-generation college
students.

86% of first-time, full-time students plan to work while
attending college.

76% of first-time, full-time students commute to
campus.
3
University Colleges provide a structure to
address the need for a comprehensive
approach to entering students.
University College
 Academic
unit formed in 1998
 Houses numerous first-year programs
 Serves Over 8000 students
5
IUPUI
Curriculum

Principles based – no core curriculum

Principles of Undergraduate Learning (PULs)






Core Communication and Quantitative Skills
Critical Thinking
Integration and Application of Knowledge
Intellectual Depth, Breadth, and Adaptiveness
Understanding Society and Culture
Values and Ethics
6
IUPUI Math-Linked Seminars
 First
year students
 Offered
during the Fall Semesters
4 Math-Linked Learning
Communities
3
are linked to Introductory Algebra
 1 is linked to Intermediate Algebra
 Meets 1 day each week for 1 hour and 15
minutes
 25 students in each learning community
 Team teaching: Instructor
Student Mentor
Advisor
Librarian
Collaborative Learning
 Students
 Students
get to know peers in math class
develop study groups for
homework and tests
Study Skills
 Students
learn how to read a math book
 Students
are taught how to prepare
BEFORE their math class meets
 Students
discuss math anxiety
Time Management
 Students
 Students
need to do math two hours a day
map out a daily schedule using a
date planner
Learning Styles
 Students
take an abbreviated Barsch
Learning Styles test in class
 Students
discuss their strengths and
weaknesses in learning new material
Campus Involvement
 Students
are required to attend campus
activities
 Students
are required to attend the
Mathematics Assistance Center (MAC)
Math Professor Panel
 Students
have the opportunity to ask math
instructors questions
 Students
feel less intimidated around their
math instructor
Why Assess Math Learning Community
Courses?





Demonstrate worth and value
Improve teaching and learning
Learn about impacts and goal achievement
Contribute to course development and
improvement
Obtain student feedback
Need to Improve Math Learning
Experiences for First-Year Students
 “There are serious problems in K-12
mathematics education, but college faculty also
need to look to their own house and think about
the first-year experience of their own students.”
 “There is a significant gap between student
experience of mathematics in high school and
the expectations they face on entering college,
and there are troubling signs that this gap may
be widening.”
David Bressoud, President of the Mathematical
Association of America, (2009)
Assessment Methods
Quantitative
 Questionnaires
administered to students
enrolled in Math-Focused
Learning Community
compared with students
enrolled in Math 001 and
Math 110 Not Linked.
 Academic Performance in
Math Classes.
Qualitative
 Interviews with Students


Richard Light Style of
Interviews - Undergraduate
students (not in course)
hired and trained to
conduct the interviews.
Content Analysis of
Open-Ended Survey
Items
Fall 2007 Math Learning Community Course
Students: MATH 001 Introductory Algebra N=58
12.72
 19.39
 426.60
 425.11
 2.84
 74%
 67%
 69%
Ethnicity
 74%
 24%
 2%

-
Average Course Load
Average Age (range 18 – 36)
SAT Verbal
SAT Math
High School GPA
Conditional Admit
First-Generation
Female
-
Caucasian
African American
Asian American
Fall 2007 Math Learning Community Course
Students: MATH 110 Intermediate Algebra N=21
13.19
 18.82
 437.33
 436.67
 3.08
 38%
 43%
 90%
Ethnicity
 76%
 24%

-
Average Course Load
Average Age (range 18 – 20)
SAT Verbal
SAT Math
High School GPA
Conditional Admit
First-Generation
Female
-
Caucasian
African American
Questionnaire Method





Administered to students in Math-Focused
Learning Community and sample of students
enrolled in Math 001 and 110, but NOT enrolled
in Math-Focused Learning Community
Administered in class
Voluntary participation
Response rate = 52%
Note – 90% of students in comparison group
also enrolled in a first-year seminar, but not
math focused.
Questionnaire Respondents
N=139

71% were Females
 56% Conditional Admits
 19.63 Average Age
Ethnicity
 76% Caucasian
 18% African American
 2% Asian American
 2% Latino/a
 2% Other
 SAT Math Score only significant difference between
Respondents (889.22) and Non-Respondents (935.60)
Using Academic Support: The
Math Assistance Center
 65%
of MATH-SEMINAR LINKED Course
used Mentoring/Tutoring In MAC 3 or
more times.
 12% Math courses NOT LINKED used
MAC 3 or more times.
Using Academic Support: The
Math Assistance Center
 93%
Math-Focused Learning Community
Course would seek out Mentoring/Tutoring
In MAC in Future Math Courses
 68% Math Courses not linked would seek
out the MAC.
Forming Study Groups
 54%
Math-Focused Learning Community
Course formed a study group.
 37% Math Courses not linked formed a
study group.
Math -Focused LC Areas of Strength
ITEM
N
MATHFOCUSED LC
MEAN
N
NOT MATHFOCUSED
MEAN
I am familiar with the Math Assistance
Center
61
4.20
73
3.49
I am familiar with the Principles of
Undergraduate Learning (PULs)
61
3.54
75
3.07
I attended activities on campus this
semester
61
3.95
77
3.18
I am able to "budget" my time through
Time Management activities.
60
3.70
76
3.63
I can maintain a balance between school
and work demands
60
4.08
76
3.87
I feel comfortable working in groups.
61
3.97
77
3.88
I feel comfortable speaking in front of the
class.
60
3.38
78
3.12
There are a lot of ways around schoolrelated problems I may face.
61
3.80
78
3.69
Only bolded items significantly different based on independent samples t-test (p< .05)
Math-Focused LC Areas In Need of
Improvement
ITEM
N
MATHFOCUSED LC
MEAN
N
NOT MATHFOCUSED
MEAN
Overall satisfaction with course
61
3.13
77
4.10
I understand the amount of time I should
devote to studying.
60
4.21
77
4.41
I have a good understanding about my
future required math courses.
60
3.88
77
4.09
I feel comfortable reading a math book
61
3.70
75
3.84
I feel comfortable studying for the midterm math exam.
60
3.69
78
3.85
I feel confident I can manage and cope
with test anxiety
60
3.68
78
3.81
I feel confident I can do well in future
math courses.
61
3.97
77
3.88
Only bolded items significantly different based on independent samples t-test (p< .05)
Helpful Aspects of Math Seminar
Percent Agree or Strongly Agree
0%
MathXL lesson
Webmail/Oncourse
Math Professor Panel
Midterm Review
Diversity presentations
Learning Styles
Time Management
Required MAC Visits
Math Anxiety Sessions
Test Anxiety Sessions
Principles of Undergraduate Learning…
20%
40%
60%
80%
100%
Most Useful Topic of MathLearning Community Course
Topic
Time Management
Learning Styles
Resources
Students’ Comments
 “Time management helped a lot, I tend to put
things off”
 “How to manage my time”
 “Time management for all my other classes and for
my other activities”
 “The important ways to study for the test”
 “I learned my learning style [which] helped me
with studying”
 “The different ways that I can study to help me
retain information better”
 “helped me get to know all the resources on
campus”
 “what different areas of the school serve for”
Most Useful Topic of Non MathLearning Community Course
Topic
Problem-solving
Studying and
Strategies
Resources
Students’ Comments
 “how to solve difficult problems”
 “learning about potential mistakes in problem
solving”
 “the single most useful topic is solving equations
because it is easy and something I will use in the
future”
 “do problems until you are comfortable with them”
 “how to study”
 “that I have to study”
 “find a way to understand it, even if its corny”



“the most useful topic I learned is that the MAC is
here to help”
“how to use MathXL”
“to get help from others, mentoring, tutoring, etc.”
Study Techniques Used
Study Technique
MathXL
Practice Problems
Book Problems
Students’ Comments


“I took all the MathXL review sections”
(seminar student)
“I would go through MathXL problems and
solve questions that are given…” (non-seminar)


“Writing problems then solving them” (seminar)
“I just tried to do as many practice problems as I
could to familiarize myself with the technique or
method” (non-seminar)

“I did suggested problems as well as chapter
tests in the book” (seminar)
“Doing problems out of the book and checking
my answers. It was more productive for me”
(non-seminar)

2007 Math-Focused Learning Community
Students’ Academic Performance and Success
Math 001
MathFocused
LC
N
SAT
Math
H.S.
GPA
Fall GPA
Math 001 Next
Avg.
Math
Grade
Course
Avg.
Grade
(Spring)
Fall-to-Fall
Retention
Rate
Yes
58
425.11
2.84
2.35
2.06
1.61
59%
*No
59
438.10
2.89
2.01
1.88
1.63
59%
* First-year students enrolled in the same section of Math 001, but not enrolled in
linked math-focused learning community course.
2007 Math-Focused Learning Community
Students’ Academic Performance Math 001
Math-Focused Mean Math
LC
001 Course
Grade
Yes
2.22
Adjusted Math
001 Grade
*No
1.81
1.89
2.23
* First-year students enrolled in the same section of Math 001, but not enrolled in
math-focused learning community course.
Based on an analysis of covariance (ANCOVA). Math-Focused LC Grade in
Math Course marginally significantly higher than non-participants’ Math
Course Grade (p =.058). Covariates in the model included Math SAT score
and HS GPA.
2007 Math-Focused Learning Community
Students’ Academic Performance and Success
Math 110
MathFocused
LC
N
SAT
Math
H.S.
GPA
Fall GPA
Math 110 Next
Avg.
Math
Grade
Course
Avg.
Grade
(Spring)
Fall-to-Fall
Retention
Rate
Yes
21
436.67
3.08
2.06
1.64
1.78
62%
*No
12
482.00
3.31
2.41
2.07
1.69
75%
* First-year students enrolled in the same section of Math 110, but not enrolled in
linked math-focused learning community course.
2008 Math-Focused Learning Community
Students’ Academic Performance and Success
Math 001
MathFocused
LC
N
SAT
Math
H.S.
GPA
Fall GPA
Math 001 Next
Avg.
Math
Grade
Course
Avg.
Grade
(Spring)
Fall-to-Fall
Retention
Rate
Yes
61
428.04
2.87
2.26
2.00
1.43
67%
*No
45
421.94
3.01
2.48
2.00
1.78
62%
* First-year students enrolled in the same section of Math 001, but not enrolled in
linked math-focused learning community course.
2008 Math-Focused Learning Community
Students’ Academic Performance and Success
Math 110
MathFocused
LC
N
SAT
Math
H.S.
GPA
Fall GPA
Math 110 Next
Avg.
Math
Grade
Course
Avg.
Grade
(Spring)
Fall-to-Fall
Retention
Rate
Yes
20
472.11
2.95
2.59
2.36
1.83**
60%
*No
17
511.43
3.22
2.57
2.33
2.37
59%
*First-year students enrolled in the same section of Math 110, but not enrolled in
linked math-focused learning community course.
**More students attempted a second math course if in the Math-Focused LC (15)
compared to those students not in the Math-Focused LC (10).
Student-to-Student Interviews



Two undergraduate research assistants were trained in
the fall of 2007 on how to conduct effective interviews for
the purpose of qualitative research.
At the beginning of the spring semester, 2008, the
trained undergraduates interviewed 8 of their peers that
were enrolled in the math-focused seminar, and
simultaneously enrolled in either Math 001 or Math 110.
The purpose of the interviews was gain an in-depth
students’ perceptions of effectiveness and affective
reactions to the math-focused LC courses.
Student Interviews Overall Findings




Students found any activity dealing with actual practice of
math to be the most useful aspect of this class.
Many students reported that if they could modify the
course it would be by devoting more of class time to
completing math problems and discussing difficulties
associated with students’ math courses.
The time management session seemed to be quite
effective in influencing students to be more organized with
school in general and more diligent about practicing math
routinely.
Math anxiety focus of this course seemed to be
particularly helpful to students reporting this as a personal
issue they are facing.
Students’ Voices
 “I
think it is a good resource class. I think
it will give a lot of help to people with math
anxiety or taking any type of math class. I
think it is a good course to be taken.”
 “It was linked to our math course and we
were able to review for tests”
 “I gained study habits”
Students’ Voices
 “we
do math work in class and it’s like you
have two classes right in a row where you
learn to study the same thing”
 “studying the book [one of the things they
taught us in class]…take an hour or two
out of class and just do practice problems”
 “to do practice problems”
 “basically gave us different skills to study
for a test before it came up”
Students’ Voices
“I learned to take my time, to work through a
problem slowly, and that if I didn’t get one at that
time to finish the test and come back to it”
 “I learned that most problems that people have
when it comes to taking a test is that they don’t
know how to just relax and take their time and
concentrate”
 “helped to relax…and think about what I was
doing so it helped me learn more than just
hurrying up to get done and just to not worry
about it”

Students’ Voices
 “time
should be set aside for doing math
problems…so you don’t have to cram for a
test and still end up getting a bad grade”
 “not to cram myself full of math the night
before a test”
 “stuff should be on a schedule and should
be done at certain times instead of
everything just done whenever”
 “to plan, not procrastinate”
Students’ Voices
 “I
will probably have a lot more math in
it…doing math problems similar to the type
we are doing in class that day”
 “do more math practice”
 “focus more on math practice”
 “a lot more time in class working with
practice problems and going over tests
and stuff”
Closing the Loop – Next Steps
Extended class time (35 minutes more) will:
 Result
in a 2-credit hour class meeting one
day a week
 Allow
 Allow
for math problem practice time
study groups to form in class (and
then hopefully carry on outside of class)
Other changes
 Require
more MAC visits
 Improve
the math anxiety lesson
 Assign
homework – math related