Outcomes and Student Learning:
Defining, Assessing, Improving
HLC Results Forum
St. Charles, Illinois
June 26, 2014
Paul Long
Vice-Chancellor of Academic Affairs and Technology
Kristy Bishop
Director of Institutional Research and Assessment
Cynthia Sexton Proctor
Chair, District Assessment Coordinating Committee
Division Chair & Physics Faculty
• Kansas City, Missouri
• Five campuses
• Fall enrollment over
19,000 credit students
• Strong transfer
programs
• More than 80 career
programs
• Part of the 2nd cohort
of the Pathways
Pioneer
Outcomes and Student Success – Defining, Assessing,
Improving
1. Change the Culture of Assessment
2. Simplify and clarify outcomes
3. Begin an inclusive, systematic, cyclic, and sustainable
assessment process
4. Create opportunities for assessment efforts to be
shared
5. Support use of assessment reports to drive
improvement of student learning
1. Change the Culture of Assessment
Institutional Climate for Student Assessment
Modified a survey developed by the National
Center for Postsecondary Improvement that
examines how your institution supports student
assessment at the institutional level.
80 questions – institutional wide Fall 2011 and
Fall 2013
1. Please rate MCC’s change in performance on the following
indicators of education over the past 5 years.
2011
2013
1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0
a. The quality of education
3.6
b. The ability of MCC to meet the
educational needs of entering students
3.5
c. The preparedness of students for
collegiate-level work
2.6
d. The effort students devote to their
studies
Very much
Worse
2.9
2.9
Somewhat
Worse
3.5
2.8
2.7
e. The academic performance of
students
3.5
3.0
About the
Same
Somewhat
Improved
Very
Much
Improved
Focus Areas
1. From your perspective, how important does MCC
consider dissemination of student assessment
reports and studies for encouraging student
assessment activities?
2. To what extent does MCC use student assessment
information in making decisions or changes in
Career and Technical programs?
3. MCC College personnel have a common
understanding of the meaning of the term
student assessment.
Assessment is: “gathering, interpreting, and
acting on information to improve student
learning”.
2. Simplify and clarify outcomes
General Education Outcomes then…
1. Communication
Courses in MCC’s general education component will provide students with opportunities to cultivate effective listening and speaking, reading, and writing skills. Students who have
completed MCC’s general education curriculum will demonstrate the following:
A. Listening and Speaking Skills;
1. Identify and apply the components of active listening in a variety of communication situations
2. Identify and explain the elements of the communication process (speaker, message, channel, listener, feedback, interference, situation) and apply these elements in
different speaking contexts
3. Find, organize, and correctly cite/credit material for oral presentations
4. Explain the purpose of visual aids and utilize them in oral presentations, including the use of technology
5. Adapt communication methodology to differing values, beliefs, and attitudes of audiences
6. Demonstrate basic communication delivery skills, both vocally (volume, rate, articulation, variety) and nonverbally (posture, eye contact, use of face and hands)
7. Manage and adapt communication apprehension in a variety of communication situations
8. Analyze and evaluate the oral communication skills of others as well as self-evaluate and modify one’s own communication skills.
B. Reading Skills:
1. Determine the main idea and significant details in paragraphs and passages
2. Recognize patterns of organization, transitions, and relationships among ideas
3. Make accurate inferences and predictions based on evidence
4. Determine meaning from context
5. Draw appropriate conclusions
6. Make valid generalizations and apply information
C. Writing Skills:
1. Respond to needs to different audiences and focus on a purpose (audience/purpose)
2. Engage in writing as an open process that includes generating and revising – pre-writing, drafting, revising, proofreading/editing (process)
3. Logically organize and develop ideas into a meaningful whole, governed by a controlling idea (coherence)
4. Gather information, evaluate its credibility, analyze, and synthesize sources.
5. Integrate ideas of one’s own with those of others. Document sources appropriately (information gathering and evaluation)
6. Exhibit control of surface features of standard English, grammar, punctuation, and spelling (language use)
7. Use writing for inquiry, learning, thinking, and communicating (writing to learn)
General Education Outcomes then…
2. Critical Thinking
Courses in MCC’s general education component will provide students with opportunities to enhance and cultivate their abilities
to engage in higher-order thinking that is implied in conscious, deliberate inquiry. Students will be presented with opportunities
to think analytically, evaluatively, and reflexively (critical self-awareness of one’s own thinking and its development), and to use
imaginative thinking to consider possible consequences of positions and actions as well as to empathize with the perspectives
of others. Students who have completed MCC’s general education curriculum will be able to demonstrate the following abilities:
A. Sort and classify information:
1. Distinguish among facts, feelings, judgment, and inferences, and prioritize the respective role of each within a given context
2. Distinguish between inductive and deductive arguments
3. Evaluate the validity, soundness, and cogency of an argument
4. Distinguish the relevant from irrelevant and integrate key relationships
B. Define, analyze, and evaluate information, materials and data
1. Objectively consider new information from diverse sources and perspectives
2. Construct valid inferences from facts, credible sources, experiences, anecdotes, and values and belief systems
3. Unambiguously define problems and issues
4. Integrate information and see relevant relationships that broaden and deepen understanding
C. Formulate a hypothesis
1. Generate relevant questions
2. Provide supporting arguments, evidence, and/or experimentation
3. Construct and verify logically sound arguments
4. Assess the value of the hypothesis
General Education Outcomes then…
3. Life-Long Learning.
Courses in MCC’s General Education component will provide opportunities for students to engage in intellectual inquiry,
cultivate learning-to-learn skills, recognize the benefits of an interdisciplinary approach to knowledge, and value learning as an
ongoing, lifelong process. Courses at MCC will assist students to acquire the following attributes of life-long learners:
A. Open-minded Inquiry
1. Value intellectual inquiry
2. Locate resources
3. Utilize modern information sources
4. Exchange and share knowledge with others
5. Explore other cultures
6. Explore multiple perspectives
B. Personal and Professional Development
1. Attend to physical, emotional, and spiritual (essential core of humanity, not necessarily religious) wellness
2. Participate in a variety of personal community milieus
3. Seek and participate in special interest groups and professional organizations
4. Pursue structured learning opportunities, certification, and/or degrees
C. Attributes of an Awareness of the Convergence of Knowledge
1. Seek multiple perspectives
2. Apply learned skills to real world interactions
3. Synthesize information to facilitate application
4. Seek new solutions to new and old problems
5. Perceive learning as an all-pervasive activity in all phases of life
General Education Outcomes then…
4. Quantitative Literacy and Mathematical Analysis
Courses in MCC’s general education component will provide students with
opportunities to develop their quantitative literacy in various disciplines. Students
who have completed MCC’s general education curriculum will be able to:
A. Present valid written and verbal arguments that include quantitative
information
B. Determine reasonableness of numbers used to describe situations
C. Determine validity of an argument based on quantitative information
D. Make reasonable estimates (computational—practical)
E. Interpret and apply numeric information embedded in text or real-life
situations.
F. Interpret and apply numeric information presented in tables, charts, and graphs
G. Apply mathematical models to solve problems and draw conclusions
General Education Outcomes then…
5. Natural and Physical Sciences
Courses in MCC’s general education component will provide opportunities for students to understand the
principles of natural and physical science; to cultivate their abilities to apply the empirical methods of
scientific inquiry; and to understand how scientific discovery changes theoretical views of the natural and
physical world, informs our imaginations, and shapes human history and, conversely, that science is shaped by
historical and social contexts. Students who have completed MCC’s general education curriculum will be able
to:
A. Use the scientific method to develop and test hypotheses and to draw defensible conclusions
B. Evaluate scientific evidence and argument
C. Describe and apply current theoretical explanations of the origin of the physical universe and the laws
governing it
D. Describe and apply current theoretical explanations of the nature, organization,
and evolution of living systems
E. Explain how human choices affect the earth and living systems
General Education Outcomes then…
6. Humanities
Courses in MCC’s general education component will provide opportunities for students to develop their
understanding of the ways in which humans have addressed their condition through imaginative work in
creative art and speculative thought; to deepen their understanding of how that imaginative process is
informed and limited by social, cultural, linguistic, and historical circumstances; and to recognize that the
virtual world of the creative imagination is a form of knowledge different from, but as important as, empirical
knowledge. Students who have completed MCC’s general education curriculum will be able to:
A. Describe the scope and variety of works in the arts and humanities
B. Explain the historical and social contexts of imaginative art and speculative thought
C. Identify aesthetic standards used to make critical judgments
D. Develop a plausible understanding of the differences and relationships between
formal and popular culture
E. Articulate a response to participation in, or observance of, works in the arts and humanities, based upon
aesthetic standards.
General Education Outcomes then…
7. Awareness of Social, Political, and Behavioral Environments
Courses in MCC’s general education component will provide opportunities for students to develop their understanding of
themselves and the world around them through study of content and the processes used by historians and social and behavioral
scientists to discover, describe, analyze, and predict human behavior and social systems; understand the diversities and
complexities of the cultural and social world, past and present; and come to an informed sense of self and others. Students who
have completed MCC’s general education curriculum will be able to:
A. Fulfill the Missouri State statute requirements for the United States and Missouri Constitutions; describe and explain the
constitutions of the United States and Missouri
B. Explain social institutions, structures, and processes across a range of historical
periods and cultures
C. Identify and apply theoretical explanations for individual human behavior and large-scale historical and social phenomenon
D. Use the methods of inquiry of history and the social sciences to evaluate problems
E. Describe and compare social, cultural, and historical settings and processes from a global perspective
F. Analyze the problems inherent in balancing the responsibility of individual needs with societal needs
G. Compare and contrast historical and cultural ethical perspectives and belief systems
H. Construct logical inferences from factual and theoretical information
I. Research and evaluate information from a variety of sources
J. Evaluate information for its currency, usefulness, and accuracy
K. Communicate information clearly and concisely using traditional and contemporary technologies
Change in Assessment Structure
DSCIA
DACC
District Assessment Coordinating Committee
Discipline
Cohorts
DACC
IR
Dean of
Instruction
DICC
General Education Outcomes now…
Critical Thinking - The student will be able to evaluate and apply information
gathered from observation, experience, reflection, reasoning, or
communication.
The student will be able to:
• Evaluate the validity and soundness of scientific, mathematical, logical, or
other formal disciplinary arguments.
• Analyze and synthesize information from a variety of sources and apply
the results to resolve complex situations and solve problems.
• Defend conclusions using relevant evidence and scientific, mathematical,
logical, or other formal disciplinary argument(s).
General Education Outcomes now…
Information Literacy – The student will be able to access and apply
information from multiple sources, evaluating the accuracy and credibility
of each, with appropriate documentation.
The student will be able to:
• Extract information from a variety of sources, using appropriate
technology to access and manage the information efficiently.
• Evaluate information for its currency, relevance, bias, and accuracy.
• Document sources utilizing the correct format and articulate the legal
and ethical implications of information use.
• Interpret and apply quantitative and/or qualitative information
embedded in text, real-life situations, tables, or graphs to analyze
complex situations and/or solve quantitative or qualitative problems.
General Education Outcomes now…
Communications – The student will be able to use receptive and productive skills to interpret,
synthesize, and integrate ideas of their own with those of others. The student will be able to:
Receptive skills
• Demonstrate understanding of context of material, including cultural framework,
audience, and purpose of the communication.
• Determine main idea and significant details.
• Analyze and interpret the parts of a communication to comprehend the relationship
between them and to deepen understanding of the meaning.
Productive skills
• Use knowledge of audience expectations and context to shape a communication.
• Create and develop an effective controlling idea using appropriate details.
• Organize material coherently into a meaningful whole.
• Synthesize and integrate ideas of their own with those of others.
3. Begin an inclusive, systematic, cyclic, and
sustainable assessment process
MCC’s Quality Improvement Project
Student Learning Outcomes
General
Education
Discipline
Course
• Determined by District Faculty
• Assessed across disciplines
• Discipline Faculty create/adapt
assessments but use Common Rubric
• Discipline-specific/ New to MCC
• Determined by Discipline Faculty
• Assessed within the Discipline courses
• Course-specific
• Determined by Discipline Faculty
• Assessed within the course by the
instructor
Cohort #1 – Fall 2011
DisciplineBiology
Cohorts
Economics
Engineering
Foreign Languages
Physics
Cohort #3 – Fall 2012
Art
English
Library
Music
Political Science
Speech/Theatre
Cohort #2 – Spring 2012
Cohort #4 – Spring 2013
Chemistry
Business/Accounting
Geology/Geography
Computer Science
History
Counseling
Philosophy
Math
Psychology
Education
Sociology/Anthropology
Reading
Discipline Cohort Work
1st Semester:
• Determine 3-5 Discipline outcomes
• Choose 1 General Education Outcome Attribute to measure and
create/adapt/adopt an assessment tool
• Choose 1 Discipline Outcome Attribute to measure and create/adapt/adopt
an assessment tool
2nd Semester:
• Administer the 2 chosen assessments
3rd Semester:
• Analyze results with IR and DACC
• Determine Interventions and next Assessments
Discipline Cohort Work
4th Semester:
• Implement Interventions/ Administer Assessments
5th Semester:
• Analyze results with IR and DACC
• Determine Interventions and next Assessments
6th Semester:
• Implement Interventions/ Administer Assessments
• Discipline Review: Split into Assurance and Improvement Components
Communication and Inclusivity
DACC
Adjunct
Faculty
Face-toFace
Meetings
DualCredit
Faculty
Blackboard
Organization
Fulltime
Faculty
4. Opportunities To Learn and to Share
1st Annual Symposium on Student Learning (Summer 2011)
• Keynote: Diane Nyhammer
• World Café/Opportunity to contribute to crafting General Education Outcomes
District Inservice with Assessment Focus (October 2011)
• Keynote: Cia Verscheldon
• Breakouts (with Assessment themes and work sessions)
MCC hosted the 2nd Annual Regional Community College Assessment Conference in April 2012
• Keynote: Jonathan Keiser
• Breakouts (including one on MCC’s Pathways Project)
2nd Annual Symposium on Student Learning: Countdown to Re-Accreditation (Summer 2012)
• Keynote: Bob Mundenk (HLC Mentor)
• Presentations of Noel-Levitz Results
• Flash-Five Reports from each Discipline in the 1st and 2nd Cohorts
4. Opportunities To Learn and to Share
3rd Annual Symposium on Student Learning (Summer 2013)
• Presentations of Noel-Levitz Results
• Flash-Five Reports from each Discipline in the 3rd and 4th Cohorts
• World Café for revision of the Associate in Arts degree
MCC hosted the 4th Annual Regional Community College Assessment Conference
in April 2014
• Keynote: Cliff Adelman
• Breakouts (including disciplines from MCC presenting progress)
4th Annual Symposium on Student Learning (Summer 2014)
• Keynote: Barbara Johnson (HLC Liaison)
• Presentations of Noel-Levitz Results
• Flash-Five Reports from volunteers in the first 4 Cohorts, Co-curricular,
and Career and Technical
Table Topics
1
Summer
Symposium
World Café
2
3
4
5
6
7
8
9
10
What evidence do we have that students achieve our
stated learning outcomes?
In what ways do we analyze and use evidence of
student learning?
How do we evaluate and improve the effectiveness of
our efforts to assess and improve student learning?
How do our student learning outcomes apply to our
mission, programs, students and degrees?
How do we close the loop for assessment and student
learning?
What are some ways to get everyone (faculty,
students, administrators, staff) involved?
What is student success? How do we measure student
success?
Discussion on Communication General Education
Outcome.
Discussion on Critical Thinking General Education
Outcome.
Discussion on Information Management General
Education Outcome.
5. Support use of assessment reports to drive
improvement of student learning
As of the end of Fall 2013
• Assessed over 10,080 students
• 471 sections of classes
• 24 general education disciplines
• 101 (93%) full time faculty participation
Communication
Discipline
Biology
Economics
Engineering
Foreign Language
Physics
Anthropology
Geology/Geography
History
Philosophy
Psychology
Sociology
Art
English
Library
Music
Political Science
Speech/Drama
Business/Accounting
Computer Science
Counseling
Education
Math
Reading
Critical Thinking
1
2
3
Information Literacy
1
2
3
4
Receptive Skills
1
2
3
Productive Skills
1
2
3
4
Closing the loop
• Individual discipline meetings
• May continue assessment or choose another
discipline and/or general education outcome.
• Disciplines will continue interpreting, evaluating,
and determining assessments and/or improvements
or administering assessments each semester
• At the end of each semester, disciplines will present
reports of progress, discoveries, best practices, and
requests.
Data Analysis Variables
1. Instructor type
2. Section number
3. Campus
4. Class grade
5. Class time
6. Class day
7. Class format
8. Student cumulative GPA
9. Student term GPA
10. Gender
11. Ethnicity
12. Credits completed
13. Age
14. High School attended
15. Pell Grant eligibility
16. Read Compass scores
17. Write Compass scores
18. Numerical Compass scores
19. Algebra Compass scores
20. College Algebra Compass scores
21. Trigonometry Compass scores
22. English ACT scores
23. Math ACT scores
24. Composite ACT scores
25. Performance on individual
questions
26. Previous classes taken
27. Program of record
28. Level
Assessment Impact:
1. Chemistry and Geology faculty discovered that they used slightly different
terminology for certain processes. Once they recognized this, they agreed to
work on using a common vocabulary while exposing students to other
terminology so it would not be new to the students if they encountered it in
later courses. While this change could have a significant impact, it does not
require curricular change.
2. Foreign Languages faculty realized that their students could pass courses,
making them eligible for the next course in the sequence, without achieving the
outcomes that were expected of them in the next course. This caused the
faculty to reconsider their assignments and exams, the nature and the weight of
each. However, this did not require curricular change.
3. Sociology faculty found that students in their Tuesday/Thursday sections
performed better than students in their Monday/Wednesday/Friday sections.
In their analysis, they concluded that a particular paper required for TR sections
but not MWF sections was making a significant impact on student learning.
Now, the paper is required in all sections.
Assessment Impact:
4. Philosophy faculty determined that students with low Read
Compass scores performed worse than students with higher Read
Compass scores. This has led the faculty to pursue a reading prerequisite.
5. Music faculty discovered that younger students were performing
better on their assessments. Further investigation revealed that these
higher-performing students were online students from a particular
campus. This led to the conclusion that perhaps the assessment results
revealed a technology issue that the faculty need to address rather than
a content issue. They are working on this.
6. History faculty found that students are meeting the objectives they
chose to assess first. Therefore, faculty are moving forward with
assessing a different objective. In the course of their discussions on
outcomes and assessment, history faculty have, after many years of
debate, decided to adopt a common textbook among all common
sections across the district.
Assessment Impact:
7. Anthropology faculty have likewise found common ground through their
debates and discussions on common discipline outcomes and assessment and
are changing course content and assignments to reflect a new alignment across
the district.
8. Business faculty have made some of the greatest strides in finding common
ground. They have decided to adopt common texts, assignments, and for
online courses, course shells for common sections across the district.
9. Economics faculty are addressing adjunct orientation and training issues
due to grade inflation discovered during analysis of assessment data.
10. Physics faculty determined that students in non-science programs were a
population too different from students in science or science-related programs
to use a common assessment tool. Therefore, they are revising their
assessment tools to assess common outcomes in ways more appropriate for
each population.
Publicizing the new
outcomes & providing
support to faculty:
•
•
•
•
•
Posters
Brochures
Website
Notebooks
BB Courses
By 2015…
Discipline
Review
HLC Quality
Improvement
Project
AA Degree
Review
AA Purpose Statement and Guiding Principles
“The MCC A.A. Degree provides a well-rounded
educational foundation that prepares students to
select appropriate majors/career paths, helps
them transfer and successfully complete
baccalaureate degrees.”
Next Steps
• Extend the work from our Culture of Assessment
Survey.
• Continue to assess and analyze outcomes, use them to
improve student learning, and then move on to assess
different outcomes.
• Encourage faculty to include more sections, more
adjunct faculty, and more dual-credit faculty in the
assessment cycle.
Next Steps
• Continue to provide opportunities for disciplines to
discuss and share assessment results through discipline
meetings, symposiums and hosting/participating
Assessment Conferences.
• Continue the new process of discipline review.
Establishing this type of review will further improve our
culture of assessment.
Contact Information:
Paul.Long@mcckc.edu
Cynthia.Proctor@mcckc.edu
Kristy.Bishop@mcckc.edu
Greg.Sanford@mcckc.edu