Assessing Student Affairs
Processes: A pilot study
Josh Brown
Liberty University
Greg McCurdy
Centra Health
Mark Davis
Centra Health
International Assessment and Retention Conference - 2007
Overview
•
•
•
•
What we did
What resulted
What we’re doing
What you can do
Context
What we did
– Assessment at Liberty University divided
into Curricular & Co-curricular
responsibilities
– Attained varying levels of assessment
• Frequency – attendance, cost, etc.
• Satisfaction – locally developed instruments
• Satisfaction with GAP analysis (Noel Levitz SSI)
• Engagement (NSSE)
• Focus Groups
• Process Analysis
– Process Engineering, Six Sigma, ISO 9000
Six Sigma
• Roots of Six Sigma can be traced to Carl Frederick
Gauss (1777-1855) as a measurement standard with
the normal curve
What we did
• Walter Shewhart, in the 1920’s, used six sigma as a
measurement standard in product variation
• Bill Smith receives the credit for coining the term “six
sigma” while working as an engineer with Motorola
• In the early 1980’s, Motorola chairman, Bob Galvin,
desired a measurement by which defects per million
opportunities could be shown and the after effect
resulted in $16 Billion in savings
• Since then, companies such as Honeywell (Lawrence
Bossidy) and GE (Jack Welch) adopted the six sigma
method as a means of doing business, not just a
quality management tool like TQM (W. Edwards
Deming)
Six Sigma Process: DMAIC
What we did
• Define problem from the voice of customer
(V.O.C.)
• Measure extent of problem by collecting
data to be able to create metrics
• Analyze data for sources of variation
• Improve process by addressing root causes,
identify high-impact benefits
• Control processes through continuous
improvement mechanisms
Step One: Define
What we did
• Define problem from the voice of customer
• Directive came from VPSA:
– “We need to streamline the judicial life
process.”
DIVISION OF STUDENT AFFAIRS
Administrative Assistant
VP for Student Affairs
Dean of Men
Dean of
Women
Center4ME
Student
Leadership
Student
Housing
(Resident/
Commuter)
Campus
Recreation
Campus
Programming
Parent
Programs
Army ROTC
DMAIC
Step Two: Measure
What we did
• Measure extent of problem by collecting
data in order to create metrics
• S.I.P.O.C. - a six-sigma tool, will be utilized to
create metrics for analysis
– Suppliers
– Inputs
– Processes
– Outputs
– Customers
DMAIC
SIPOC: Suppliers
What we did
• Conducted inquiry sessions with all levels of
persons in the judicial process:
– Session One: RA’s & RD’s
– Session Two: Associate Deans
(DOM/DOW)
– Session Three: Head Deans and VPSA
– Session Four: Students who experienced
the judicial process at various levels
– Session Five: Administrative Assistants,
Secretaries, and Student Workers
overseeing data entry
DMAIC
SIPOC: Inputs
What we did
•
•
•
•
•
•
Student Handbook
Violation & Incident reports
Data entry at RD level
Res Life staff: manually sorting reports
“Why do we need to process warnings?”
Difference between practice and policy:
confusion of appeal process
• “There are too many hand-offs of
paperwork.”
• “We handle data differently than the other
office.”
DMAIC
SIPOC: Outputs
What we did
•
•
•
•
•
Lack of communication of appeals
Appeal process is slow/inconsistent
“I am not sure of the process.”
Not enough qualified counselors on campus
Differing approaches: men-discipline,
women-counsel
• Dean on-call schedule is confusing as it
varies too frequently
• Fines are confusing and don’t seem to be
achieving their intended purpose
• Too many logs! (cont.)
DMAIC
SIPOC: Outputs
What we did
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
RA Official Correspondence Log
Call Slip Log
Non-Return Log
Permission Slip Log
Violation Report
Incident Report – Residence Hall
IR-Type Log
Case Load Log
Discipline Community Service Log (twice)
FERPA Log
Probation Log
AW Log
Student File Database
File Log (who has what)
Self-Reports Log
No Contact Agreement Log
Permission Restriction Log
DMAIC
SIPOC: Customers
What we did
•
•
•
•
•
•
•
•
•
•
Students
Student Leaders: RA/RD/Deans
Res Life
Dean of Men & Dean of Women
VPSA
Sodexho – community service
LUPD
Counselors
Faculty/Staff
Campus Pastors
DMAIC
Step Three: Analyze
What we did
• Analyze data for sources of variation
• Three analyses conducted:
– Process Maps – this is the “P” in SIPOC
processes & is implemented at this stage
– Fishbone Analysis
– SWOT Analysis
DMAIC
Liberty University Judicial Process Cross-Functional Flowchart
Violation & Incident Reports, S.I.P.O.C.
Level One
0-4 Reps RA
Level Two
6-17 Reps RD
If quick bypass
response is warranted, RD calls
Dean-on-Call 30+ reps
RA issues Violation
Report to student,
copies to RD; Incident
Report to RD & RLO
by 8 a.m. next day
RA completes Incident
Report, emails RD &
sends to RLO by 8
a.m. the next day
What we did
RLO AD sorts
all incoming
IRs
forwarding
judicial IRs to
DOM/DOW
office
RD meets weekly
with RAs to uphold/
overturn VR
appeals; RD sends
to DOM/DOW
office each Monday
RD enters data
in personal
Excel or file
system
Student fined
appropriately by DOM/
DOW; placed in official
records
RD investigates
IR, meets with
student, decides
case with VR &
appropriate fine;
enters in
personal Excel
or file system
If student
appeals in writing
(VR copy) w/i 48 hrs,
RD handles appeals
0-12 reps, no staff clarity
with 12-17 reps case
appeals: RD or AD?
[Only one appeal
allowed, no
furtherance]
Level Four
30+ non-dismissal
DRC
Level Three
18-29 Reps AD
DOM Administrative Assistant
enters IR data by Assistant Dean
& IR #; DOW Administrative staff
enters data by type & RH in Excel
& print.
Administrative
Assistants (DOM/DOW)
assign AD by even
caseload distribution;
arrange DRC; judicial
counseling
DOM/DOW
Secretary sends
call slips to
student, RA,
RD & RLO
for judicial
appointment
with AD
If no response
from student to call
slip, DOM remove
student from
convocation; DOW
block student’s
computer account
conflict of
interest
Student fined
appropriately by DOM/
DOW; placed in official
records
AD acts as
TRIAGE for 1829 & 30+ &
<<<appeals
from RD,
investigating
each case,
deciding on 18+
cases
If
student appeals
in writing to Dean
Student fine w/ DCS; w/i 48 hrs, Dean will
probation documentation decide appeal?
placed in official records
Level Five
30+ Auto Withdrawal,
JRB/VP
On-call DOW see
all cases
(commuters as
well); On-call DOM
pass case to
Administrative
Assistant who
passes it to another
AD on Monday a.m.
RA completes Incident
Report, emails RD &
sends to RLO by 8
a.m. the next day
DOM/DOW
Administrative
Assistants run
Probation Report
each Monday for
students w/ 18+ reps
Presented by Greg McCurdy
BOTTLENECKS
in highlighted
boxes: RLO,
DOM/DOW & ADs
RA completes Incident
Report, emails RD &
sends to RLO by 8
a.m. the next day
DRC:
AD,
RLO
AD,
Dean &
VP
VPSA
reviews all DRC
decisions; can offer
alternative
discipline
Alternative discipline;
documentation in official
records
Dean’s
Review
Committee
(DRC) meets
to decide 30+
cases
If student appeals
decision by DRC in
writing to VPSA
within 24 hrs
Conflict of
interest:
VPSA/DOS
chairs both
DRC &
JRB
Administrative
Assistant to VPSA
arranges JRB
meeting
Student rcvs DCS + fine OR
Administrative Withdrawal and/or
non-return
The Judicial
Review
Board meets
to decide the
final appeal
case
JRB: 2 SGA, 3
faculty. Associate
Dean presents case
& VPSA chairs;
neither AD or VP
votes
Majority vote decides
Administrative Withdrawal
and/or non-return OR
overturn & alternative
discipline
DMAIC
Liberty University Judicial Process Cross-Functional Flowchart,
Recommended Schematic - Greg McCurdy, 30 December, 2006
Level One
0-4 Reps RAs/Peers
6-17 Reps RDs
RA issues 0-4 Rep
Violation Report to
student (hard copy),
submits copy to RD
What resulted
RD meets
weekly with RAs
to uphold/
overturn VR
appeals (SA
educational
development)
RD enters VR
data in new SA
judicial
software
database
Student account will be
updated with VR fine
RA completes Incident
Report via new software
system; RD and
Centralized DOM/DOW/
Student Life Office
automatically notified; IR
data is stored in system
database
RD
investigates
IR, meets w/
student &
decides case
If student appeals in
writing to RD within
48 hrs, a
predetermined RH
Student Peer
Group will decide
the appeal
presented by RD
(RD does not vote)
Student Affairs
educational
development opp!
Decision by print-out
letter final: Fine or
overturn w/ warning
Level Two
18-29 Reps ADs
Level Three
30+ Reps Deans
RA completes Incident
Report via new software
system; RD and
Centralized DOM/DOW/
Student Life Office
automatically notified; IR
data is stored in system
database
RA completes Incident
Report via new software
system; RD and
Centralized DOM/DOW/
Student Life Office
automatically notified; IR
data is stored in system
database
RD calls
Dean-On-Call for
emergent IR cases
30+
Centralized Office/Database of DOM/DOW/
SLO Administrative Assistants/Secretaries can
pull up any judicial data necessary from new
software system; issue appropriate fine via
interface connection with the student accounts
office; organize files in database; prepare
template documents for Disciplinary
Community Service letters, Sodexho status
reports; Call Slips emailed to students
Administrative Assistants
assign/schedule AD judicial
case load distribution based
on Probation Report for 1829 Reps; 30+ Reps to DOW/
DOM-Head Dean
AD investigates,
researches, meets
w/ students,
decides case
Dean/DOM/DOW
(or Council of all
three voting)
investigates,
researches, meets
w/ students,
decides case
If student
appeals in
writing to a
Dean within
48 hrs, Dean
will decide
appeal
Decision by print-out
letter final: DCS + fine
or overturn w/ warning
Decision by print-out letter
final: DCS + fine OR
Administrative Withdrawal
and/or non-return
Level Four
VPSA/JRB
For students
who have been
Administratively
Withdrawn through
judicial process, VP
of Student Affairs (&
Dean of Students)
review for Reapply/
Readmit
status
Admissions
requests Student
Affairs feedback for
reapply/readmit
Basic Flowchart
Shapes
decision
document
Stored
data
Direct
data
Predefined
process
card
process
Manual
operation
Administrative
Assistant to
VP of Student
Affairs
arranges/
schedules
JRB meeting
Parallel
mode
control
JRB: Dean
presents case
& VPSA
chairs; 3
faculty, 2
SGA & VPSA
vote (Dean
does not
vote)
terminator
If a student appeals
in writing to VP of
Student Affairs
within 48 hours (time
consistency),
the Judicial Review
Board will decide
appeal
Majority vote decides to
uphold Dean’s Council
decision: Administrative
Withdrawal and/or nonreturn; DCS + fine OR
overturn & alternative
discipline offered
DMAIC
Manpower (staff)
Materials
Title
Keeping many logs
What resulted
Paper reports
vs. electronic?
Scan documents?
Concerned w/ major
Incident Reports
Overburdened w/
processing warnings
Hard copy call slips, RA:
“finding student difficult”
RLO AD sorters
overloaded?
Fi shbon e An a l ysi s:
Li ber t y Un i v er si t y
Judi ci a l Pr ocess
Improve
Efficiency
of Judicial
Workflow
RLO to DOM/DOW
bottlenecks
Data storage not linked:
RA/RD/RLO/DOM-DOW
Manual work flow
RA/RD staff are not
provided PCs
Need a database with:
Accessible & Real-time
information
Timeliness of handling
appeals a concern?
Volume of business emails:
RAs 3-5 per day avg
RDs 20-50 per day avg
RAs complete emails
in computer lab; potential FERPA
violations?
Judicial staff use different
forms of data storage:
hard copy vs. electronic
Flow of reports
up & down the chain?
Consistent and timely?
Filing staff hours:
Women 15-20 hrs week
Men 18 hrs week
RDs share two PCs
among 21 RDs
Based on feedback from staff
via the S.I.P.O.C. qualitative
Define & Measure steps, the
following key improvement
area became a focal point
for Analysis.
Crisis issues involving
time for student
counseling
documentation
Data entry staff hours:
Women 40 hrs week
Men 55 hrs week
Email call slips used for
commuter students (DOW)?
Management
Redundant data storage
Human error
Paper gets lost
Several levels/offices
(Handoffs)
Differing methods
between DOM & DOW
offices (consistency)
Privacy a factor
(FERPA)
Paper filing system
Education of
judicial staff
filling out reports
Machines
Method
Mother (Human) Nature
DMAIC
SWOT Analysis
What resulted
• Strengths
– Skilled staff
– Judicial process affords student appeal
– Education of student handbook
• Weaknesses
– Communication breakdown
– Inconsistent processes
– Lack of technology to integrate
processes
– Paper workload with many hand-offs
DMAIC
SWOT Analysis
What resulted
• Opportunities
– Software integration upgrade
– Office PC’s interconnect all Student
Affairs
– Educational development through
residence hall Peer Judicial Councils
• Threats
– Reactive vs. proactive
– Legal aspects: FERPA
– Overstressed staff, burnout, and turnover
DMAIC
Step Four: Improve
What resulted
• Improve process by addressing root
causes and identify high-impact
benefits.
– Critical-to-success-factor chart
– Prioritizing benefits and efforts
– Final recommendations
DMAIC
Application
• You and your group members have been
hired by Liberty University as judicial
consultants to remedy this process.
• For the next few minutes, use the collective
knowledge and experience of your group
to provide at least four recommendations
for the university to improve its judicial
processes.
• Please place your recommendations on the
provided note cards.
Critical-to-success factor chart
Ideas
Application
1
2
3
4
People Service
Efficient
Cost
Total
Critical-to-success Factor Chart
Prioritizing Critical Success Factors
What resulted
Ideas
People
Service
Efficiency
Cost
Total
Student Affairs
Judicial Software
& Hardware
Integration
9
9
9
1
729
9
9
9
5
3,645
3
Counseling
Center with
Qualified Staff
9
9
5
1
405
4
Streamline the
Judicial
Processes &
Workflow
9
9
9
9
6,561
1
2
Restructure &
Centralize
Judicial System
High = 9
Medium = 5
Low = 1
DMAIC
Prioritizing Benefit & Effort
3rd Proposal
1st Proposal
EFFORT
What resulted
HIGH
Prioritizing
Benefit &
Effort
2nd Proposal
4th Proposal
BENEFIT
LOW
HIGH
DMAIC
Final Recommendations
What resulted
• Acquire a centralized student database
that can integrate judicial operations
• Streamline judicial process and structure
• Eliminate conflicts of interest in the current
process
• Involve students in the appeal process
• Equip the division of SA with the necessary
qualified counselors
DMAIC
Step Five: Control
What resulted
• Control processes through continuous
improvement mechanisms:
– Formulate action plans for implementing
strategies
– Establish an ongoing QA program
DMAIC
What we’re doing
• Since the conclusion of the Six Sigma judicial
study, Student Affairs has begun the following
for a Fall 2008 implementation:
– Purchased a new judicial software package
– Created & implemented a student court for judicial
appeals
– Revised judicial organizational chart
– Redefined and clarified roles (as result of above)
– Eliminated policies from student handbook
– Created policies from student handbook
Six Sigma Tips For Educators
What you can do
1. Know your customers
• Identify them (SIPOC)
• Listen to them (VOC)
• Understand and define their needs
(CTQ)
2. “Know thyself”
• Examine your processes (SIPOC /
mapping)
• Measure your performance (baseline;
DPMO; Sigma; statistics)
Six Sigma Tips For Educators
What you can do
3. Know what to do next
• Get to the roots (fishbone; hypothesis
testing; VA/NVA)
• Define the ideal state (gap analysis)
• Brainstorm your opportunities (SWOT;
prioritization matrix)
• Drive change (force-field analysis)
4. Know how to do it
• Decide on your method (project vs. godo)
• Open the toolbox
• Start with what you have
Recommended Resources
What you can do
• Academic
– Assessing Organizational Performance in Higher
Education (Miller, 2007)
http://www.josseybass.com/WileyCDA/WileyTitle/productCd-0787986402.html
– Continuous Process Improvement in Higher
Education (Inozu & Whitcomb, 2007)
http://www.novaces.com/pdfs/CoF_NovacesWhitePaper_r1std.pdf
– Process Improvement to Achieve Institutional
Effectiveness (Lake, 2005)
www.ncci-cu.org/Visitors/Documents/processimprovement070905AC.ppt
• Business
– Six Sigma for Dummies (Gygi, DeCarlo, Williams &
Covey, 2005)
– The Six Sigma Way: How GE, Motorola, and Other
Top Companies are Honing Their Performance
(Pande, 2000)
• Josh Brown is currently the Associate Director of University
Presenter Bios
Assessment for Liberty University, coordinating the assessment of
all co-curricular departments. He possesses an earned Master's
of Student Development from Azusa Pacific University. Email –
jtbrown@liberty.edu
• Greg McCurdy is currently the manager of the Radiation
Oncology Department at Centra Health, where he utilized the
six sigma philosophy and instruments to hone difficult processes
in a medical setting for increased workflow efficiency. He is
concluding his Master's of Higher Education at Geneva
College. Email – McCurdysrus@juno.com
• Mark Davis is currently a process engineer with Centra
Health, where he is assisting with the implementation of a
system wide healthcare improvement initiative called CH2. He
holds a degree from William & Mary and a Six Sigma Black Belt
from Villanova. Email – Mark.Davis@centrahealth.com