How Do CCCU Adult Programs
Measure Up?
Dr. Cynthia Tweedell, CCCU Center for Research in Adult
Learning
Dr. Mary Moretto, Goshen College
Dr. Shirley Roddy, Mid-America Christian University
Dr. George Howell, Indiana Wesleyan University
What is the Center for Research in
Adult Learning?
• Partnership between Indiana Wesleyan
University and Council for Christian Colleges
and Universities.
Steering Committee
Richard Ellis– John Brown University, AR
Don Finn– Regent University, VA
Steve Holtrop– Huntington University, IN
Lori Jass– Bethel University, MN
Audrey Kelleher– Belhaven University, MS
Toni Pauls– Warner Pacific University, OR
Shirley Roddy– Mid-America Christian Univ., OK
George Howell– Indiana Wesleyan Univ., IN
Anita Underwood– Nyack College, NY
Cameron Wold– Colorado Christian Univ., CO
Deborah Wright– Montreat College, NC
Mimi Barnard– CCCU
Cynthia Tweedell– Center for Research
Research Agenda
•
•
•
•
•
•
•
Learning outcomes assessment for adults
Benchmarking
Retention of adult students
Spiritual transformation of adults
Best practices in online learning
Teaching diverse learners
Alumni surveys
Strategic Goals
• Sponsor at least one conference per year.
– May 10-12, 2010 In Cincinnati
• Publish at least one monograph per year.
– Integration of Faith and Learning for Adults coming in May
•
•
•
•
Provide benchmarking data on adult education.
Assist in student learning outcomes assessment
Serve as a resource for research in adult programs.
Coordinate projects at the request of institutions
www.indwes.edu/cral
Benchmarking from the CCCU Center
for Research in Adult Learning
• Adult Student Learning Outcomes
–
–
–
–
–
–
3rd year of project
About 10 schools participating
Common writing prompt
Beginning and graduating students
3 faculty from each school as scorers
Critical Thinking, Written Communication, Christian
Worldview
– How does your school compare to other adult
programs?
Scoring
Rubric
Critical
Thinking
5
Identifies issues
Challenges
assumptions
Thorough analysis
General connections,
analysis and
identification of issues
4
3
2
1
Score:
Christian
World View
Clearly demonstrates an
understanding of a Christian
perspective. (Frequent
references to biblical principles)
Frequently refers to a Christian
perspective. Student has a
reasonable understanding of
Christian perspective. (Some
references to biblical principles.)
Some analysis
Makes some mention of a
Vague identification of Christian perspective. Student
issue
indicates some understanding of
a Christian perspective. (Biblical
principles referred to somewhat.)
Incomplete analysis
Demonstrates little
Fragmented
understanding of a Christian
understanding of issue perspective. (Biblical principles
hardly or not mentioned.)
Vague analysis
Makes no reference to a
Basic lack of
Christian perspective. Student
understanding
does not appear to have an
understanding of a Christian
perspective. (Biblical principles
not mentioned.)
Communication
Free from distracting spelling,
punctuation, and grammatical errors.
Very well organized
Meaning is clear
Few spelling, punctuation and
grammatical errors.
Fairly well organized
A few places where meaning is a
little unclear.
Most spelling, punctuation, and
grammar are correct, though some
errors remain.
Organization may detract from
meaning. Some places unclear.
Spelling, punctuation, and
grammatical errors are distracting.
Organization and meaning unclear.
Many spelling, punctuation and
grammatical errors, making reader
unable to follow ideas.
Lacks organization.
Meaning is very unclear.
2008 Results: 6 Colleges
Critical Thinking
Written
Communication
Christian
Worldview
N
Mean
Std.
Dev
Pre
109
*3.27
0.80
Post
127
*3.49
0.72
Pre
109
*3.06
0.83
Post
126
*3.36
0.70
Pre
109
1.92
0.97
Post
126
1.83
0.92
2009 Results: 5 Colleges
Critical Thinking
Pre
Post
Written
Communication
Pre
Post
Christian
Worldview
Pre
Post
N
Mean
Std.
Dev
107
*3.10
0.61
84
*3.34
0.73
107
1.77
0.80
84
1.97
0.82
107
*2.82
0.77
84
*3.49
0.75
Mid-America
CHRISTIAN
University
Response To Data
• Review MACU’s presentation of data in
comparison to other participating institutions
• Consider that the results could be reflective of
the reality.
• Initiate dialogue with the College of Adult and
Graduate Studies Chairs.
MACU’s presentation of data
• Determine if MACU’s data indeed matched
with the presented data from the other
participating institutions in time, setting and
resources available to the student during their
response.
• Consider the impact of the sample size and
MACU’s method of randomization for possible
influence on results
Conduct and Participate in
Subsequent Studies
• Attempt to replicate the study by participating
in subsequent multi-institutional studies
• Compare ourselves to ourselves--Replicate our
own institution’s study to determine if we get
similar results.
IN THE MEANTIME…
• Considering that the data may be reflective of
student achievement in the areas measured,
it does not seem prudent to wait until further
studies have been conducted.
• Immediately put into place measures that will
increase students’ level of achievement in the
areas reflective of need
What we have already done
• Initiated discourse with chairs, using the
research results as a tool
• Established remediation courses in math and
English
• Standardized our own method of data
gathering and compared ourselves to
ourselves!
School of Business & Leadership
College of Adult &
Professional Studies
Assessment Process
Questions for School of Business & Leadership Program
Directors & Faculty (Graduate, Bachelor and Associate)
• Critical thinking
– How can we build on existing threads?
• Christian worldview
– How can we increase the Christian Worldview of our
students?
• Written communication
– How can we improve our students’ written skills?
• The CCCU Adult Student Learning Outcomes Project
was run in parallel with College and School initiatives
related to the outcomes of this project
Assessment Process: Critical Thinking
Critical thinking: How can we build on existing
threads?
• Review by faculty of critical thinking resources
– Selection narrowed to five textbooks
– Two textbooks selected by the School of Business
& Leadership
• Ongoing faculty development activities on
critical thinking
Assessment Process: Critical Thinking
• Faculty and Academic Leaders selected the
following critical thinking textbooks to reinforce
the critical thinking thread in programs:
– Undergraduate Business Programs Browne, M. &
Keeley, S. (2010). Asking the Right Questions: A Critical
Guide to Critical Thinking (9th Ed.). Upper Saddle River,
NJ: Pearson/Prentice Hall.
– Graduate Business Programs Paul, R. & Elder, L.
(2006). Critical Thinking: Learn the Tools the Best
Thinkers Use. Upper Saddle River, NJ:
Pearson/Prentice Hall.
Assessment Process: Critical Thinking
• Critical thinking resource for our course
writers
– Nosich, G.M. (2009). Learning to think things
through: A guide to critical thinking across the
curriculum (3rd Ed). Upper Saddle River, NJ:
Pearson Prentice Hall
Assessment Process: Christian
Worldview
Christian Worldview: How can we increase the
Christian Worldview of our students?
• Ongoing evaluation of the integration of faith in curriculum
– Faculty Devotions and Biblical Principles in Business
– Curriculum threads that reinforce knowledge, skills and
disposition (be, know and do) of students’ Christian Worldview
– Develop character of students by a Christ-centered
example/role model
• Faculty member’s expression and role modeling of Judeo Christian
values in the classroom
• Faculty development activities to improve faith integration
Assessment Process: Written
Communication
Written communication: How can we improve our
students’ written communication skills?
• School of Business & Leadership Faculty committees
developed scoring guides for written work to better explain
expectations to students on written work and to help
faculty be more objective in their assessment.
–
–
–
–
Met objectives/requirements of assignment (35%)
Critical thinking (35%)
APA formatting (15%)
Mechanics (Grammar, Spelling, Word choice, Punctuation and
Sentence structure ) (15%)
• Ongoing faculty development activities on grading written
work
Assessment Process: Written
Communication
Written communication: How can we improve our
students’ written communication skills?
– Faculty task force examined our Liberal Arts
Requirements (General Education courses) and made
recommendation to revise the number of credits
required in English /Composition/Speech/Literature
• Going to College’s Academic Affairs Committee in first
quarter of 2010
– Writing lab – MAPP (Make a Paper/Project)
– Online Student Resources with Prentice Hall
Reference Guide