Lessons Learned Part I
Assurance of Learning:
Strategic Planning through
Programmatic Assessment
Kathleen A. Krentler
College of Business Administration
College of Business Administration
Programmatic Assessment Driven by:
1. Desire to use evidence-based assurance
of learning in the college’s strategic
planning efforts.
2. The university’s call for and support of
programmatic assessment.
3. Requirements of our outside accreditation
– AACSB.
College of Business Administration
AACSB: Show us your AOL!
• Maintenance of
Accreditation approaches
• Self-Evaluation Report:
October 1, 2012
• Campus Visit:
December 2-4, 2012
College of Business Administration
• College-wide programs:
•
BSBA Common, MBA, EMBA
• Departmental programs:
•
SOA (2); FIN (5); MGT (3); MIS (2);
MKT (3)
• TOTAL programs assessed
across the college: 18
College of Business Administration
A very simple approach . . .
Closing
the
Loop
SLOs
Results
College of Business Administration
BSBA Common
• 5 Goals:
•
•
•
•
•
Communication (Written & Oral)
Critical Thinking
Ethical Reasoning
Global Perspective
Essential Business Knowledge
College of Business Administration
Oral Communication
SLO:
Make effective oral presentations that are
informative as well as persuasive, as appropriate.
Measurement & Data Collection:
• The CBA UG Committee trains to assess
• Teams of 2 assessors listen to presentations
being made in capstone courses across the CBA
• Assessors use the internally developed “CBA Oral
Communication Skills” rubric (which has been
provided to all students).
College of Business Administration
Results:
“85% of our students
should meet or exceed
expectations”
YES
Organization
Voice Quality & Pace
Mannerisms & Body
Language
Professionalism &
Appearance
Rapport w/ Audience &
Use of Media
“50% of our students should
exceed expectations”
YES
Professionalism &
Appearance
NO
Organization
Voice Quality & Pace
Mannerisms & Body
Language
Rapport w/ Audience &
Use of Media
College of Business Administration
Closing the Loop:
• Wider distribution of the Oral Comm
rubric
• Institution of the BA 290 Oral Comm
exercise
• Faculty strongly encouraged to talk
with students about strong & weak
oral comm and to include oral comm
in grading
College of Business Administration
Reporting the Effort
The CBA
Assessment
website
College of Business Administration
Moving to WEAVE
Do we have to?
College of Business Administration
Moving to WEAVE
• Susan’s trains
• September 2011
• Goals & SLOs input
• October 2011
• Measures input
• By April 2012
• Annual Reports submitted via WEAVE
• June 15, 2012
College of Business Administration
What’s Next?
• Curriculum Mapping
• Read-Only privileges to all
faculty
• Successful MAC in December
2012!
• Stronger Strategic Planning
College of Business Administration
Lessons Learned Part II
Achieving Faculty Buy-In
and Spreading the Word:
Adapting to thrive in
changing times
Kathleen A. Krentler
College of Business Administration
College of Business Administration
“The Times They Are a-Changin‘”
. . . But how do you convince
the faculty of that?
• “We give grades.”
• “What we do can’t be
assessed.”
• “This is just paperwork.”
College of Business Administration
Challenge #1:
Establishing a
Culture of
Assessment Among
the Faculty
College of Business Administration
The Culture Survey
College of Business Administration Assessment Culture Survey
September 2008; n=60 (75% of all full-time faculty)
Baseline:
2008
Bi-annual
administration
The most important resason for the emphasis on assessment in the
college is to meet AACSB and CSU requirements.
5 Point Scale: 5=Strongly Agree to 1=Strongly Disagree
"?" = % of faculty responding, "I have no idea."
CBA
ACC
FIN
IDS
MGT
MKT
n=11
n=17
n=10
n=10
n=12
3.82
3.36
3.59
4.40
3.75
4.20
The most important reason for assessment in the college is to collect
data that will ultimately lead to improved student learning outcomes.
3.52
4.00
3.41
3.60
3.42
3.20
Assessment done well will improve teaching and learning.
3.92
4.27
3.59
3.78
4.17
3.90
My department has developed measurable student learning
outcomes for each of its programs.
3.87
?-8.3%
4.10
?-9.1%
3.62
?-23.5%
3.8
4.00
3.90
My department regularly collects data to determine the extent to
which the student learning outcomes for each of its programs is met.
3.73
?-8.3%
3.67
?-18.2%
3.40
?-11.8%
3.50
4.08
4.11
?-10.0%
I have personally participated in the collection of assessment data
for one or more of my department's programs.
3.68
?-5.0%
3.00
?-18.2%
3.41
3.70
4.45
?-8.3%
3.90
My department meets regularly (at least once per year) to share the
results of its assessment efforts and to discuss how these results
might inform curriculum change that could improve our programs.
3.52
?-13.3%
3.33
?-18.2%
3.92
?-23.5%
3.40
3.73
?-8.3%
3.00
?-10.0%
Assessment information has been useful when making
programmatic changes.
2.90
?-13.3%
3.25
?-27.3%
2.85
?-23.5%
3.00
2.92
2.56
?-10.0%
My department's assessment process has translated into
improvements in student learning.
2.78
?-16.7%
3.25
?-27.3%
2.64
?-17.6%
2.63
?-10.0%
2.91
?-8.3%
2.50
?-20%
I have clearly stated and measurable student learning outcomes on
the syllabus of each course I teach.
4.42
4.18
4.35
4.60
4.58
4.40
I have no idea what a clearly stated and measurable student
learning outcome is.
1.63
1.55
1.75
1.70
1.42
1.70
College of Business Administration
The AIDA Hierarchy
Action
Desire
Interest
Attention Awareness
College of Business Administration
Attention - Awareness
Posters
Semesterly Newsletters
College of Business Administration
Interest
What do you want your
students to know?
• When they walk across the
stage (i.e programmatic
SLOs)
• When they finish your
course (i.e. course-level
SLOs)
College of Business Administration
Desire
What is a rubric?
How do I make one?
Want to
make your
life easier?
Want to
make
grading
easier?
Content
Organization
Audience
Style
Mechanics
Referencing
(A simple tutorial)
Below Expectations
Meets Expectations
Does not adequately cover the assigned
task. The primary thesis may not be clear
or if it is, little topic development is
evident. Assertions made in the writing
are either weakly supported or no
support is offered.
Paper lacks logical sequence hence
causing format to interfere with
readability. Does not use proper
paragraphing. Topic sentences do not
lead to rest of paragraph or are missing
altogether.
Writer is internally focused rather than
focused on the reader. No clear
awareness or understanding of the
audience is evident. Writer may appear
discourteous to the reader.
Overuse of simple sentences. May
misuse words or idioms. May include
slang. Wordy rather than concise. Writing
shows lack of sophistication or variety in
vocabulary. Awkward. Little or no use of
business terms.
Significant errors in word usage,
sentence structure (run-ons, fragments),
spelling, punctuation, and capitalization.
Errors undermine credibility of content
and readability.
References (if called for) are missing or
do not use correct referencing style.
The assigned task is covered sufficiently. The
primary thesis is clear but there is some room
for further development of the topic. Support is
offered for assertions that are made but that
support could be stronger, more compelling or
more inclusive of all issues.
Paper follows logical sequence with identifiable
beginning, development, and conclusion.
Generally proper use of paragraph structure
and topic sentences. Organization and/or
headings help the reader to follow and find
information.
Writer acknowledges the reader and displays
some thought about the nature of the
audience. Reader is treated politely and
positively. No evidence of inappropriate
attitude.
Sentences vary in length and style. Strong
action verbs are used. Occasionally uses jargon
or clichés. Vocabulary and word usage generally
is correct and shows some variety. Uses
business terms appropriately.
The assigned task is thoroughly covered and
completed. The primary thesis is clear and fully
developed. Assertions made throughout the
writing are compelling and clearly supported.
Exceeds Expectations
Relatively free of errors in word usage,
sentence structure (run-ons, fragments),
spelling, punctuation, and capitalization.
Mechanics do not detract from credibility of the
content.
Generally correct referencing (if called for)
using APA or MLA style.
No errors in word usage, sentence structure
(run-ons, fragments), spelling, punctuation, and
capitalization. Strong mechanics help to establish
credibility.
Paper flows well with appropriate beginning,
development, and conclusion. Paragraph
structure contributes to flow and transitions.
Organization and/or headings help the reader to
understand and remember information.
Writer clearly focuses writing to the audience,
and displays empathy for the reader. Goodwill is
created through consideration of the reader’s
needs. Message tailored directly for the reader.
Demonstrates a sophisticated grasp of the
language in terms of both sentence structure
and vocabulary. Writes fluidly and concisely.
Includes appropriate business terms.
Below Expectations
Meets Expectations
Exceeds Expectations
Organization
No opening and/or closing statements
or irrelevant opening/closing
statements. Loses focus more than
once. Does not manage time
effectively. No logical sequence of
information. Mechanistic.
Offers some type of opening and closing
statements. Follows logical sequence but
structure could be better. May need
more elaboration on one or more points.
Adequate time management, but could
be stronger.
Voice Quality &
Pace
Mumbles, mispronounces words,
grammatical errors, “umms”. Difficult
to understand. Speaks too quietly or
too loudly. Speaks too fast or too slow.
Loses train of thought, tentative. Lacks
enthusiasm.
Demonstrates distracting mannerisms
which may include bad posture,
shifting feet, too much or too little
hand movement. Body language
reveals reluctance to interact with
audience. Seems fearful/very nervous.
Easily understood. Speaks loud enough to
be heard and at appropriate pace. Some
awkward pauses or halting delivery but
mostly clear and natural. Could display
greater enthusiasm, seem more
genuinely interested in own presentation.
No significantly distracting mannerisms.
Acceptable posture. Body language
mostly demonstrates comfort in
interacting with audience but occasional
instances of discomfort may be
communicated. Seems natural for the
most part.
Meets minimum standards for business
dress and appearance. Generally treats
audience professionally, acceptable word
choice (no slang). May seem to lack
confidence at times. Reasonably credible.
Tries to maintain eye contact most of the
time but instances may be fleeting in
length. Scans the room. Some reliance on
notes or slides.
Clear opening and closing statements.
Catches audience’s interest, provides
overview/conclusion. Follows logical
sequence, stays focused, good explanations.
Effective time management and strong
transitions. Strong mental take away for
audience.
Enthusiastic and engaging. Speaks clearly and
loudly enough at a comfortable pace. Exudes
confidence and interest. No grammatical or
pronunciation errors. Presentation appears
conversational, extemporaneous, and
natural.
Body language used effectively to maintain
audience’s interest. Body language reflects
presenter’s reaction to, and empathy with,
the audience. Gestures match verbal content,
are comfortable and relaxed, seem
spontaneous.
Mannerisms & Body
Language
Professionalism &
Appearance
Rapport with
Audience & Use of
Media
Does not meet minimum requirements
for business dress. Makes excuses for
aspects of the presentation.
Inappropriate word choice for
audience. Inappropriately informal.
Does not connect with audience. Little
to no eye contact. Reads. Relies heavily
on slides and/or notes. Attempts to
cover too many slides or lingers too
long on too few slides.
Dressed appropriately. Appearance
engenders respect and credibility. Treats
audience professionally. Speaker appears
confident and has good command of the
topic.
Genuinely connects with audience. Maintains
eye contact throughout. Visuals (slides, etc.)
effortlessly enhance speech.
Updated January 2009
Credits: This document borrows from the SPEAKS Rubric from CSU-Fullerton Business Communication Program and the CSU-Chico, College of Business Oral Communication Rubric.
References (if called for) are consistently correct
using APA or MLA style. No missing citations.
Updated January 2009
Credits: This document borrows from the SDSU IDS Department Written Skills Rubric (Vik, Reinig, Anderson-Cruz), the IDS Upper Division Writing Assessment Rubric, and the CLASS Rubric from CSU-Fullerton (Fraser,
etc. al., 2005).
College of Business Administration
Action
Assessment
Day
College of Business Administration
How do we keep it fresh?
1. An event each semester
2. A newsletter each semester
3. Semesterly reminders of SLOs &
rubrics
4. Distribution of results
5. Engaging faculty in “Close the
Loop” discussions
College of Business Administration
Challenge #2:
Establishing a
Culture of
Assessment Among
Students
College of Business Administration
A student’s view of the world:
Keep taking classes and
eventually you’ve
completed enough and
they give you a degree.
College of Business Administration
View we’d like students to have:
A cohesive program
with courses building
on each other to
achieve overall
progammatic goals.
College of Business Administration
Assessment Culture for students:
1. Helping students’ change their view
of the world.
2. Introducing the concept of a
program by introducing
programmatic goals & SLOs.
3. Helping students’ to see how each
course is part of a bigger “whole”.
College of Business Administration
Learning
Goals
On everythe
syllabus:
BSBA Program Goals
BSBA students will graduate being:
•
•
•
•
•
Effective Communicators
Critical Thinkers
Able to Analyze Ethical Problems
Global in their perspective
Knowledgeable about the essentials of business
MKT 370 contributes to these goals
through its student learning outcomes . . .
MKT 370 Student Learning Outcomes
At the end of this course students should be able to:
College of Business Administration