Developing Graduate
CBE Programs
Texas A&M University System
January 27-28, 2016
1
Today’s Presenters
Vice-Provost and Dean of Graduate Programs
University of Maryland University College
Donna Younger
Associate Vice-President, Higher Education
CAEL
dyounger@cael.org
© Copyright 2016
Aric Krause
2
PLA and CBE on the
Competency Continuum
3
What is the relationship
between CBE and PLA?
• They are not the same
• But are based on the same
principles
• In practice, they are ideal
companion tools to help
adults complete
4
What Is Competency-Based
Education?
• An organizing principle for curriculum
and assessment
• Focuses on what students learned, not how
much time spent in a classroom or other
time factors
• Instruction customized to meet students
where they are
• Does not need to be based on the credit
hour
5
Prior
Learning
Assessment is
the process for
evaluating
knowledge
and skills to
award college
credit for
learning from:
What is Prior Learning
Assessment? (PLA)
On-the-job training
Independent
Study
Training Courses
and/or Certifications
Military and Volunteer
Service
Work Experience
Prior to What?
• Although the term PLA
references prior learning, the
prior is a misnomer since any
learning – prior or current –
that is acquired outside of the
institution may be assessed
through PLA
7
Key reasons for PLA
Post-traditionals are looking for it.
Provides valid flexibility for students
It has a profound impact on persistence.
Portfolio form of PLA enhances research and
writing skills.
We do well by doing right.
Dewey
Experience
Reflect
Theorize
Apply
Theory: K/S/E of others
a source here
The Variety of PLA Methods
Evaluated
Non-College
Training
Student
Portfolios
Exams
Industry
Recognized
Certificates
Degree
Completion
Key CAEL Standards
Credit is for
LEARNING, not for
experience
Subject matter
experts make
credit
recommendations
Any fees are for
assessment, not
for the amount of
credit awarded
Basic Rubrics
Developing
Accomplished
Exemplary
1
2
3
4
Describe
identifiable
attributes that
reflect a
beginner’s level of
performance
Describe the
attributes at
movement
toward mastery
Describe the
identifiable
attributes or
characteristics
proficiency
Describe the
attributes for
the highest
level or
mastery
This rubric puts the focus on level of mastery for each performance
goal/objective/outcome. If the desired performance is clearly defined,
the student knows what is expected.
Score
12
Stated Objective or
Performance
Beginning
Overview
Time
Session Topic
10:45 – 3:00
Learning Science & Developing Programs With a
Competency Methodology
•
•
•
The Why and How
Differentiating From Other CBE Approaches
Reconsidering the Demonstration of Learning
3:00 – 5:00
Articulating Competencies for a Program
January 27, 2016
•
•
9:00 – 12:00
Focusing Toward a Course Within a Program
January 28, 2016
•
Rebuilding with the End Up Front
What should a student know and be able to do at the end of the
program?
© Copyright 2016
January 27, 2016
And lastly, we look at courses
• We’ll take breaks at convenient stopping points
• Please turn devices on “silent”
13
• The Why and How
• Differentiating From Other CBE Approaches
• Reconsidering the Demonstration of Learning
SESSION 1
© Copyright 2016
10:45 – 3:00 Learning Science & Developing
January 27, 2016 Programs With a Competency
Methodology
14
Vocabulary
• “We” and “Our” – as in “we need to…” or “Our
programs are…” refers to Higher Ed, and not
any specific institution.
© Copyright 2016
• Program = generic term for degree, certificate,
or CE, undergraduate or graduate.
15
© Copyright 2016
WHAT IS THIS CBE THING?
16
There is nothing new about
Competency-based Learning.
© Copyright 2016
Heretical Statement #1
17
Focusing on Outputs, not Inputs
• Instead, consider it a philosophy or
methodology behind building an academic
experience or program.
© Copyright 2016
• The letters “CBE” are meaningless to the
outside (especially to potential students and employers!)
18
Define: The Competency Methodology
demonstrable learning of an academic program, upon
which a choreographed set of learning experiences is built
in order to guide a particular student from the starting
point to a level of demonstrated mastery at completion.
© Copyright 2016
• My (working) Definitions:
• Version A: A method to articulate the full set of
• Version B: Flipping the academic model by starting
with the end and building backward – focused on learning
demonstration, not topics of coverage.
19
Heretical Statement #2
Total FOCUS on Learning: what does that mean to
you?
© Copyright 2016
• What many schools call “CBE” (version 1.0)
has little to do with “learning” mastery.
20
• What a student should KNOW and be able to DO
is predefined up front.
• The program is then backward engineered to
achieve that specific KNOWING and DOING.
• That’s it. Everything else is variable - - delivery
format, assessment technique, faculty role, seattime, speed of completion, degree of prior learning
articulation.
© Copyright 2016
The Competency Methodology
21
© Copyright 2016
PHILOSOPHY OF THE
COMPETENCY METHODOLOGY
INFORMED BY LEARNING
THEORY/RESEARCH
22
Philosophical Underpinning #1
• How do adult (experienced) learners
learn? What deepens learning? What
role does contextualization play in
personalizing learning?
© Copyright 2016
• The entire focus of the educational experience
is on demonstrated learning, everything else is
variable
23
Philosophical Underpinning #2
• The iterative act of
application drives deeper
and more permanent
learning – and more
capacity to do.
Doing
Knowing
© Copyright 2016
• More important than what
you know is what you can
DO with what you know.
See:
Dewey (context in learning, 1938), Bloom
(application, 1956), Piaget (Assimilation &
accommodation, schema, 1955), Knowles (andragogy,
1989), Kolb (learning cycles, 1984), Vygotsky (social
interaction and proximal development, 1962), Keeton
(experiential, 1976); et. al.
24
Philosophical Underpinning #3
The most
current
knowledge of
the field,
contextualizes
the abilities
Domain
- Field
Abilities
-Analyze
-Learn
-Create
-Innovate
-Research/Find
-Evaluate
-Communicate
-Solve/Decide
-Develop
© Copyright 2016
• Competency articulation leads to TWO distinct but
synergistic parts to emphasize in creating an
educational program.
25
Philosophical Underpinning #3
(cont’d)
• If our Students master the abilities...
• …and in doing so have mastered the pertinent
abilities…
• …aren’t they prepared to launch successfully in this
field, no matter where they go?
© Copyright 2016
• …within the current context of the domain…
26
Philosophical Underpinning #4
Abilities
-Analyze
-Learn
-Create
-Innovate
-Research/Find
-Evaluate
-Communicate
-Solve/Decide
-Develop
© Copyright 2016
• The Abilities require practice, practice,
practice – in different contexts - in order to get
close to mastery.
27
Philosophical Underpinning #5
The
Learning
Target
© Copyright 2016
• The program is the most important unit of
consideration, NOT the course.
28
Start!
The Antagonist #α
• “….Sounds Like Trade School to Me! The dumbing down
of Higher Ed!”
• What do C’s mean in traditional model?
• How many traditional programs have articulated what a
student should be able to do, and have choreographed their
courses to get there?
• How are traditional measures of learning accessing and
evaluating higher-order capacities?
• How do traditional programs with many combinations and
permutations (electives) result in intentional learning?
© Copyright 2016
• Really?
29
Philosophical Underpinning #6
• If we want to help students become adept
“do-ers”, then giving them the ability to
iterate provides another learning
opportunity.
© Copyright 2016
• Failing is an opportunity to learn
30
Philosophical Underpinning #7
• We can help them graduate with a degree AND the experience of
the profession if we have them do the things they do in the
profession.
© Copyright 2016
• Context of the domain, applied by invoking the
abilities, means the student is “practicing”
what they would do in the profession.
31
• Where/How a student learns doesn’t matter;
that they have learned is what matters.
• Allows students with experience in a
particular field to “accelerate” through what
they’ve already mastered.
• Allows for maximum time usage on gaps in
learning.
© Copyright 2016
Philosophical Underpinning #8
32
1.
2.
3.
4.
5.
6.
7.
Learning is the central focus – and adults have a contextual
advantage in learning;
“Doing” results in deeper learning than just “knowing”;
Deepest learning occurs through iteration over a program;
Practicing the abilities to demonstrate mastery of the
domain results in someone good at both;
Practice makes perfect; iteration is a valuable learning
opportunity;
Practicing the field in realistic contexts gives experience on
top of the learning; and
Learn how you learn, and focus on showing that learning.
© Copyright 2016
Philosophical Summary
33
• Financial challenges of higher education;
• Financial challenges of the family;
• Learning challenges of graduating students,
relative to needs of employers;
• Accountability of the system as a whole.
• From a different perspective: technology may be
now able to offset the cost/quality paradigm.
© Copyright 2016
Why Is CBE Important Today?
34
© Copyright 2016
USING THE COMPETENCY
METHODOLOGY
35
4
Put it
together
and test it
3
Design the
journey
from
beginning
to end
2
1
Figure Out
the
Beginning
Start with
the END
learning
© Copyright 2016
In a single graphic
(competencies)
36
Before You Start
• If we could build the best program in the
world (without no preconditions): what
would a student know and be able to DO
when it is completed?
© Copyright 2016
• Start with a “Clean Slate” – forget everything
you currently do in your courses and/or
program….
37
Remember…
© Copyright 2016
• There IS no box…
38
Competencies – what
a student knows and can do
A
One Approach
Using the CBE
Methodology
B
E
A
Course 1
E
D
sequenced journey that leads to
mastery of the program’s competencies
and demonstrating learning in the
manner professionals do, showing
mastery of competencies along the
way.
professionals in this
field “do”
C
Program – an intentionally
And finally, courses – learning
Projects – what
D
B
C
Course 2
After completing a program, students have built
competence and confidence
Step 0: Choose & Define your Words
• Learning Goal
• Evaluation Criteria 1 (Or Performance Indicator? Proficiency Standard?)
• Evaluation Criteria 2
• Evaluation Criteria 3
• Competency 2
• Competency 3
© Copyright 2016
• Competency 1
40
Step 1: Inventory
• Sources: faculty experts, accrediting bodies, standards
organizations, DoL, onetonline.org, employers…
• Should result in a laundry list of abilities and
domain abilities.
Question: Is there a definitive list? No. At the end of the
day, the list for a particular program is what you say it is.
© Copyright 2016
• What should a student know and be able to DO
in order to complete the program?
41
Step 2: Arrange and Group
• As you group, decide if each is a competency
or a learning goal….or a descriptor.
• Continue arranging – get a first version –
iteratively.
© Copyright 2016
• Put in “buckets” based on similarity.
42
1. Document technical processes and
specifications (learning goal)
1. Determine end-user’s needs. (competencies)
2. Describe processes in an order and with
sufficient detail to meet the needs of the enduser
3. Articulate specifications to minimize user
error and ambiguity.
© Copyright 2016
Some competencies will be
grouped by the “Domain” (example)
43
• Learning Goals are often “Processes”
• Critical Thinking: how can you definitively
know if someone has mastered critical
thinking?
• It’s a context-sensitive Process!
© Copyright 2016
Complication A
44
Complication A (cont’d)
1.
2.
3.
4.
5.
6.
7.
Frame the Question/State the Problem (in context)
Gather Information
Evaluate Information in the context of the question
Weigh and Prioritize the information in the context of
the question.
Determine what the information is telling you.
Answer the question/choose a solution to the problem
Articulate reasoning behind the answer/solution.
© Copyright 2016
The Competencies
• Critical Thinking (the Learning Goal)
45
Step 3: How can you know?
• Begin to describe how you’d “know it when you see it” descriptors for
each competency.
Critical Thinking and Analysis: Learners demonstrate ability to
apply logical, step-by-step decision-making processes to formulate
clear, defensible ideas and to draw ethical conclusions.
1.
Articulate and frame the issue.
1.
2.
3.
4.
2.
Identify issue (or research question, problem, etc.).
Characterize issue according to its size, scope, incidence, effects, perceptions of it
and influences on it.
Identify the underlying causes or conditions of each element contributing to issue.
Identify required information needed to critically analyze issue.
© Copyright 2016
1.
Collect and evaluate information.
1.
2.
3.
4.
Find information relevant to the issue.
Determine usefulness of information based on source, content, and method.
Apply information and sources appropriately to inform decisions and processes, in
the context of competing views, sources, and methods.
Use information ethically and responsibly.
46
Step 3: How Can you Know? (cont’d)
• NO to “understand”, “become fluent with…”
• Must use action word
• Must fully describe – these become the
rubric later, so get as close to full as possible.
© Copyright 2016
• Rules
• Must be observable
47
• May need to GRADATE the
performance we expect at
various points throughout
the program.
• For abilities - milestones/checkpoints.
• How to define in ways that
get at different levels of
performance expectation?
Intentionally
train student
to do A to
standard
More
complex B,
with
exceptions
and violations
of rules
Checkpoint A:
Can’t move
beyond until
mastery
Highest
order
application
and
synthesis
Checkpoint B: Final Point C:
Can’t move
Can’t
beyond until
graduate
mastery
© Copyright 2016
Complication B
48
Step 4: Iterate, Iterate, Iterate
• Continue to Revise…
• …Until you are ready to accept v1.0 (knowing that,
without exception, you will learn more and want to
revise very soon – very dynamic).
© Copyright 2016
• Get your Learning Goals, Competencies, and
Descriptors in front of as many eyes as possible.
49
Step 5: Finalize Competencies
© Copyright 2016
• Lock in place, knowing they’ll change very
soon.
50
• Decide what form of assessment will be used, and
brainstorm them.
• Quizzes and tests? Research Papers?
Presentations? Reflection Exercises? Some of
each? These work best for “formative assessment”
• Following learning theory, summative assessment
using contextualized projects relevant to the
career aspirations of the student work very well
(invokes context, application, and cognitive
assimilation) – as does reflective activities.
© Copyright 2016
Step 6: Learning Demonstration
51
• At this point, choose the learning evaluation
method:
• Formatives – do they really need to be
evaluated formally?
• Summatives – what form should they take?
© Copyright 2016
Step 6: Choose Learning
Assessment Model for Program
52
• Choreograph them in an order based on
complexity requirement.
© Copyright 2016
• If you chose career-relevant:
• Brainstorm the types of things people in the
related fields “do” in the profession.
53
• Decide where each competency will be
intentionally (over learning demonstrations):
• Taught
• Demonstrated (at each performance level)
• Expanded upon
• Finally evaluated
Still Clean Slate!
© Copyright 2016
Step 6: Map Competencies to Learning
Demonstration
54
© Copyright 2016
Step 6: Map Competencies to Learning Activities
(cont’d)
55
Step 6: Map Competencies (cont’d)
Introduce
Apply
Measure
Question: How could you map written communication
over the whole program – not just in writing courses –
so that it is a careful journey that gets the students to
the right level of ability by the end of the program?
© Copyright 2016
The Program
56
Step 7: Start Fitting into Courses
1
Start
2
3
4
5
6
7
8
9
How should we choreograph what happens
over these ten blocks?
• Eliminating “reintroduction” and ensuring
increasing levels of challenge;
• Ensuring formative along the way, so that
you can get to summative.
10
Finish
Still Clean Slate!
© Copyright 2016
• Our program will have ten blocks, previously
called courses.
57
• With our ten blocks aligned with competencies,
we can start thinking about what the courses
actually are – and what happens in them.
• Note: very few pre-existing courses, in prior
form, will still fit; all will require some
adjustment.
• Note: if your old courses DO fit, something went
wrong.
• Note: inevitably, you’ll find gaps in the
competencies and want to continue refining. OK!
© Copyright 2016
Step 8: Revise Learning Experience to Fit
Map
58
DANGER ! #1
• If this is what you want to do, don’t waste time
with all of this – just do the paper exercise and
be done with it.
• Program Chairs – warning!
© Copyright 2016
• Many try to “retrofit” competencies over their
existing courses/programs, without actual
change. #EPICFAIL #EPICWASTEOFTIME
59
• A method to articulate the full set of demonstrable
learning of an academic program, used in the
program review and revision process, upon which a
choreographed set of learning experiences is (re)built
in order to guide a particular student from the starting
point to a level of demonstrated mastery at
completion.
© Copyright 2016
The Competency Methodology - Redefined
60
Other Considerations
• Ongoing Revision process/mechanisms
• Data Systems/technology
© Copyright 2016
• Faculty role
61
© Copyright 2016
OTHER CONSIDERATIONS
62
Grading
• Binary issue of competencies.
© Copyright 2016
• What type of learning evaluation should be
used?
63
Quality
• What is quality? Is high quality the graduating of students who
can do what they are supposed to be able to do?
• The map drives your ability to determine quality – collect data
according to the map
• Can students build on in course 4 what they were supposed to
learn in course 3? If no, fix course 3.
• At the end of the program, are there gaps? Go back and fix
preceding courses that should have filled the gap
• Were the competencies right/wrong? Fix. Was the relevant course
taught correctly? Fix. Were the learning resources/assessment
focused on right things? Fix.
© Copyright 2016
• Assuming competencies (the target) are “right”
• Or is it the number of books in the library? Faculty/student ratio? Rankings?
64
• Rethinking the tasks we emphasize as
important for faculty.
• The role faculty can play in iterative learning
(especially adjuncts).
• When programs are specifically mapped to
accomplish certain things, some parts of it
become “locked”; not the context, per se, but
definitely what is being evaluated.
© Copyright 2016
So, Faculty Workload?
65
• What is easiest for us to deliver does not
always lead to the best learning.
• If you’re going down this path, be prepared
to rethink EVERY aspect of the operation.
• But if it results in superior learning, isn’t it
worth it?
© Copyright 2016
Final Advice
66
Thank You!
Q&A
67
© Copyright 2016
• Rebuilding with the End Up Front
• What should a student know and be able to
do at the end of the program?
SESSION 2
© Copyright 2016
3:00 – 5:00 Articulating Competencies for a
January 27, 2016 Program
68
• 3:00-3:30 – brainstorming what students
should know and be able to do when they
complete a doctoral level program
• Short quick statements
• measurable!
• NOT CONTENT COVERAGE
© Copyright 2016
Exercise: Moving to
Competencies
69
• ABILITIES
DOMAIN/FIELD
© Copyright 2016
• 3:30 – 4:00 Reporting Out and Dividing Into
Abilities versus Domain
70
• How Many Groupings do you get?
• Name the Groupings - - LEARNING GOALS
© Copyright 2016
• 4:00 – 5:00 Organizing Into Groupings of
Similarity
• “Which of these things is a lot like the others?”
71
• And lastly, we look at courses
SESSION 3
© Copyright 2016
9:00 – 12:00 Focusing Toward a Course Within a
January 28, 2016 Program
72
© Copyright 2016
Exercise: Break Into Programs
73
THANK YOU!
74
© Copyright 2016