Program Evaluation in a Concept
Based Curriculum
Dr. Linda Miles
Chair, School of Nursing
Daytona State College
Objectives
• Differentiate between assessment and evaluation
• Describe the process for developing a program
evaluation plan
• Describe how the evaluation plan directs activities
in the nursing education program
• Describe the different aspects of a program
evaluation plan for a concept based curriculum
Evaluation
Effort to increase human effectiveness through
systematic data-based inquiry.
Systematic examination of accomplishment and
effectiveness in program and services.
Patton, M.
Assessment: focuses on learning and teaching. It
provides information to make changes in the
learning environment. It is a systematic process
of collecting and interpreting information to base
decisions (formative).
Evaluation: focuses on grades and may reflect
classroom components other than course content
or mastery level. It is a value judgment that
assigns meaning to the data obtained through
assessment (summative).
Components of Educational Instruction
PLANNING
Instructional Objectives
Learning Outcomes
Instructional Activities
ASSESSMENT
Brookline,1999
Evaluation
To improve- should provide information that can be
used to determine whether or not desired outcomes
are being achieved and how programs can be
improved.
To inform- should inform faculty and other decision
makers about relevant issues that can impact the
program and student learning.
To prove- should summarize and demonstrate what
the program is accomplishing and students are
learning.
Should Answer These Questions:
What are you trying to do?
How well are you doing it?
Using the answers to the first two questions, how
can you improve what you are doing?
How can the student learning experience be
improved?
Purpose of Evaluation
• New program
• Curricular revision
• Educational initiatives
• Justification
• Quality of education
• Validation
Purpose of an Evaluation Plan
• Provides a framework for assessment
− Provides a timeline for activities
− Identifies responsible parties
• Guides the daily assessment activities
• Aligns assessment activities with the accrediting
standards
• Documents program development and activities
Components of an Evaluation Plan
• Levels of Achievement
• Responsible Party or Person
• Timeframe or Frequency
• Evaluation Methodologies
• Data (3 years)
• Results and Actions
Process for Developing an Evaluation Plan
Best practices
Full faculty involvement
Reflects current accrediting standards
Suitable for your institution and program
Systematic and ongoing
Food for Thought
Outcomes
• Drive curriculum and development of evaluation
methodologies
Activity
1. What type of evaluation methodologies would you use to assess this outcome?
2. How would this be reflected in the program evaluation plan?
Program Learning Outcomes:
Nursing Process
The graduate of the associate degree nursing program will utilize the nursing process
and principles of best practices to meet the needs of individuals, families, groups
and communities across the lifespan.
Cultural Competence
The graduate of the associate degree nursing program will provide caring, spiritual
and culturally sensitive nursing care to individuals, families, groups and
communities across the lifespan.
Evaluation Methodology
Summative Evaluation of the Program:
How well did the associate degree nursing program prepare you for the following nursing aspects?
(Likert)
Nursing Process
•
Utilize the nursing process and principles of best practices to meet the needs of individuals,
families, groups and communities across the lifespan.
•
Evaluate the effectiveness of pharmacological interventions for clients with recurring healthcare
needs.
Cultural Competence
•
Provide caring, spiritual and culturally sensitive nursing care to individuals, families, groups and
communities across the lifespan.
•
Create an interdisciplinary plan of care reflective of client’s cultural, spiritual, and physical care.
Evaluation Methodology
Agency Evaluation of Graduates:
XX nursing graduates provide safe and technically
competent nursing care to individuals, families, groups or
communities across the lifespan.
XX nursing graduates demonstrate competency in caring
for individuals, families, groups or communities across
the lifespan.
Evaluation Methodology
Standardized Testing:
(Crosslink test plan with outcomes)
85% of students will achieve the 80% score reflective
of the national norm
Evaluation Methodology
Clinical Evaluation Tool:
Students demonstrate competency providing nursing care to clients with recurring health care needs:
Nursing Process
•
Applies nursing process to maximize the function of clients with chronic or recurring health needs
through the lifespan.
•
Evaluates client outcomes from pharmacological interventions.
Cultural Competence
•
Performs nursing care in a manner that is respectful of client’s individuality.
•
Evaluates the effectiveness of interdisciplinary plan of care with patient outcomes.
6.4 Graduates demonstrate achievement of competencies
appropriate to role preparation.
Nursing Process
The graduate of the associate degree nursing program will utilize
the nursing process and principles of best practices to meet the
needs of individuals, families, groups and communities across the
lifespan.
Cultural Competence
The graduate of the associate degree nursing program will provide
caring, spiritual and culturally sensitive nursing care to individuals,
families, groups and communities across the lifespan.
Expected Level of Achievement: 100% of graduates will demonstrate achievement
of the student learning and program competencies appropriate for the role of an
associate degree nurse. (Clinical Evaluation and Final Course Requirements.)
75% of students will achieve the RN Test Score Equal or Above the National Mean.
75% or greater of students will agree or strongly agree on the program summative
evaluation that they attained the program learning outcomes.
75% or greater of employers will agree or strongly agree on the agency satisfaction
survey that graduates attained the program learning outcomes.
Assessment Methods: Graduates will be assessed for attainment of competencies and student learning outcomes through the use of a variety of
evaluation methodologies (ATI, quizzes, case studies, learning activities, clinical evaluations, course grades, and graduate surveys).
Results: 2010-2012
Clinical Satisfaction: 100% of students who graduated were satisfactory in clinical.
Course Satisfaction: 100% of students who graduated met all course exit requirements.
Standardized Test
Spring 2012 RN Test Score 70.2
Fall 2011 RN Test Score 68.1%
Student Summative Evaluation: 2011-12 (Each of the calculations reflects the percentage of students by item that
agreed or strongly agreed that outcomes were met by year.
Nursing Process (five outcomes)
84%/64%/74%/64%76%-December
89%/76%/80%/80%/82%-May
Cultural Competence (two outcomes)
94%/93%-December
81%/77%-May
Agency Satisfaction 2011-12 Graduates
Nursing Process
97% agreed or strongly agreed that graduates attained the outcome
Cultural Competence
100% agreed or strongly agreed that graduates attained the outcome
Timeframe: Fall and spring
Actions:
Evaluation Plan for a Concept Based Curricular
Approach
Describe evaluation methodologies
Describe the process to align evaluation
methodologies with outcomes
Describe assessment measures
• Formative-Summative
• Quantitative-Qualitative
• Triangulation
Classroom-clinical linkage
Compare and Contrast Program Evaluation by
Curricular Model
Curriculum Building
• Mission
• Philosophy
• Conceptual Framework
• Outcomes
Learning Outcomes
What are Institutional Learning Outcomes (ILOs)?
The knowledge, skills, abilities, and attitudes that students are expected to develop as a result of their
overall experiences with any aspect of the college, including courses, programs, and student services.
What is the purpose of ILOs?
Institutional learning outcomes (ILOs) are designed to help guide individual departments and disciplines in
the development of outcomes for programs, courses and services, and to help shape the decision-making
processes of the college.
How are ILOs different from Program and Course Learning Outcomes?
They are inter‐related. ILOs are the collective expression of the learning environment the college offers to
any enrolled student. Program and course learning outcomes focus on the more particular skills, knowledge,
and attitudes that students learn in programs and courses.
Nine Principles of Assessment
The assessment of student learning begins with educational values.
Assessment is most effective when it reflects an understanding of learning as multidimensional,
integrated, and revealed in performance over time.
Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
Assessment requires attention to outcomes but also and equally to the experiences that lead to those
outcomes.
Assessment works best when it is ongoing, not episodic.
Assessment fosters wider improvement when representatives from across the educational community
are involved.
Assessment makes a difference when it begins with issues of use and illuminates questions that people
really care about.
Assessment is more likely to lead to improvement when it is part of a larger set of conditions that
promote change.
Through assessment, educators meet responsibilities to students and to the public.
American Association for Higher Education (http://www.aahe.org/assessment/principl.htm)
Ten characteristics of an effective program to
assess student academic achievement
Successful assessment flows from the institution's mission and educational
purposes.
Successful assessment emerges from a conceptual framework.
Successful assessment is marked by faculty ownership and responsibility.
Successful assessment has institution-wide support.
Successful assessment relies on multiple measures.
Successful assessment provides feedback to students and the institution.
Successful assessment is cost-effective.
Successful assessment does not restrict or inhibit goals of access, equity, and
diversity established by the institution.
Successful assessment leads to improvement.
Successful assessment includes a process for evaluating the assessment
program.
(The North Central Association Commission on Institutions of Higher Education)