Can We Teach
Design of Experiments
On-Line?

Dr. Dan Rand – Winona State
University
46th Annual Fall Technical Conference ASQ & ASA
 E-mail: drand@winona.edu

Can We Teach Design of
Experiments On-Line: Agenda









DoE as a prime tool of Six Sigma
Instructional design impact on E-learning
Voice of customer – requirements for on-line
education
Requirements focus on a DoE course
Web Search for on-line DoE courses
Assessing on-line courses
Examples/ assessments from my course
An Experiment Design – Our Field of Dreams
Links to resources
DoE as a prime tool of Six Sigma
Design of Experiments is the most
effective and most complex quantitative
tool of Six Sigma Black Belts
 Most Six Sigma training is light on DoE
 Black Belts form the latest group that
needs supplemental DoE training
 They add to the audience that can’t sit
for a traditional classroom course

Instructional Design to Meet Learning Needs Bloom’s Taxonomy1
Cognitive skills:
Instructional Design
in Distance Education2
 Contextualize instruction
 Present lesson content
 Present vocabulary
 Provide examples
 Use cueing programs
 Advise learner
 Activate learner processing
 (more)
More Instructional Design
Aspects from IDDE
 Assess learning
 Provide feedback
 Provide practice
 Test learning
 Sequence instruction
 Logical order
 Prerequisite order
 By content organization
General Instructional Design
Concepts for E-Learning
 Modular
structure
 Learning objectives for each module
 Active learning
 Assessment / feedback
Instructional Design
to Meet Organizational ROI
4 desired outcomes / assessments of
skills development training
Learn it
Retain it
Apply it
Cause benefits
Test at end of
course
Apply it
report on
application
ROI $
What customers want from online education

LINK – Learning Network Initiative:
Creating Effective Collaboration Between
Industry and University Center Rochester
(MN)

An End-to-End Framework for
“High Tech” Workforce Training

From “Technology Industry” Working Group:
–
–
–
–
Drew Flaada, IBM
Warren Kemplin, Rochester Community Technical College
Brian Nass, XIOtech (Seagate)
Dr. Daniel Rand, Winona State University
End-to-End Learning Management System
1
2
3
A
4
A A A A
5
A
6
A
A = assessment of skill / knowledge gap
7
Phases:
1 – Navigating to relevant offerings (not pertinent today)
2 - Skill / knowledge gap assessment & selection of optimal offering
3 - Connecting to the offering (registration, payment, getting course materials)
4 - Absorbing the offering (take the class)
5 - Completing the offering and post-offering assessment of remaining gaps
6 - Application - Effectively using the newly acquired skill/knowledge on the job
From Phase 2 of End-to-End LMS:
Pre-assessment & Selection

Problem Statement (Voice of Customer):
“I don’t know which offering is the best fit for me.”

Requirements (Voice of Customer):
– I need a pre-assessment in order to further filter out offerings
that aren’t at the appropriate level
– I need to be able to filter the possible offerings by:



Terms of payment, match from skills assessment, cost
Reputation of provider (quality), certification level I require
Availability, language, delivery method
– Any tools associated with this phase must be accessible
anytime, from anywhere
– My personal information needs to be protected (ongoing on-line
education concern)
– Pre-assessment, pre-test for pre-requisites
– Identify goals from employee and employer point of view, with
“live” or e-counselor
Phase 3: “Connecting to the offering ”

Problem Statement:
– “I don’t know how to get started and it’s difficult to find
out. The whole process is a hassle.”

Requirements:
– Automate the provisioning of all facilities I need
(textbooks, databases, labs, server access, web site
access) – Bundle as a student kit
– Test and ensure that my workstation or browser will
work with the delivery methods used before I proceed
(example: what plug ins are required?)
Phase 4: Absorbing the offering
(taking the class)

Problem Statements:
– “I may not complete the course unless I have specific
interim goals to hit and/or peers to keep me going.”
– “My day is interrupt-driven.”
– “If I miss a traditional class, I can’t get caught up. I need an
alternative.”
– “While in the midst of an offering, I don’t know if I’m
on target or not.”
Phase 4: Absorbing the offering
(taking the class)

Requirements:
– I can communicate with the instructor and peer
–
–
–
–
–
–
–
–
students either real time (chat), via email, or via a
forum
I have access to supplemental resources online
I can take the course from my desk or anywhere
There is an online assessment tool available to help
me gauge my progress
The materials become customized to my specific
needs, based upon my online assessments (which
point out remaining gaps)
I can pause, resume, and replay any of the offering
The offering includes real hands-on projects that
support my learning
Estimate total effort needed for a course
I need (small) modular content
Phase 5: Final assessment & closure

Problem Statements:
– “I have no idea how I did vs. the expected outcome
or standard.”
– “I can’t drive to a classroom to take the final.”
– “I’m out of town the day of the final.”
Phase 5: Final assessment & closure

Requirements:
– Results of the final assessment need to be fed to me,
my employer, the instructor
– The final assessment vehicle needs to be available
online and on my schedule
– I need to be able to pause and resume the
assessment vehicle
– Assuming successful completion of the assessment,
any completion credentials (grade, certificate, et al)
need to be automatically issued
– I need frequent checkpoint assessments given to
employer (payer), like a % completed bar.
– I need a means to re-take course – free, or even
remedial material.
Phase 6: Application

Problem Statements:
– “I don’t know where to go for ongoing help.”
– “I’m unsure how this topic/knowledge should be
applied in the real world.”
– “I forgot a great deal in the X months/years since
the class ended.”
– “I don’t know how to extend this knowledge to
more advanced levels.”
Phase 6: Application

Requirements:
– Give me access to all related materials (online)
– Give me access to peer learners and the instructor
–
–
–
–
–
(online)
Point out options for follow-on instruction
Provide me with application examples
Provide me assessment vehicles and
recommendations for remediation
Keep material available to student after
completion of course.
Form graduate communities / networks to gain
knowledge of content improvements, growth.
Additional General Concerns
Motivating employee to completion
 Synthesis of training into job
performance
 How to direct students to where their
success rate is the highest
 Need a disconnected mode – carry
course content away on CD-ROM

Summarized needs,
applied to an on-line DoE Course

Content and procedure
– Motivation to finish
– Don’t be totally dependent on high speed
Internet
– Animation, bells, and whistles may demotivate students if they slow down
processing
– Interact with others to discuss and
assimilate methods
– Feedback from an expert
– Link to textbook
Summarized needs,
applied to an on-line DoE Course

Application
– Practice actual experiments – apply it
– Try software to design and analyze
– Work with others in team experiment (bring
together for catapults or simulation)
– Apply it to student’s problem- as project, w/
expert consultant
– Provide a support group of graduates
Pragmatic (Market) Limitations

Even most
motivated students
need checkpoints
 Market is relatively
small for advanced
content like DoE –
animation and
customization are
expensive
On-line Design of Experiments courses –
Web Search

http://www.margaret.net/doe/

Online Course in Design of Experiments (DOE)
interactive tutorials, case studies and email support. access
for 4 weeks, extend 2 weeks, expect 30 hours.

http://www.thequalitygroup.net/

The Quality Group has a DOE overview module in their
on-line Six Sigma Green Belt training
Features– animated with Quick Time, pre-tests, interactive
quizzes, choice of video, captions, text, or still pictures

More Web Sources

Eastern Michigan U sponsors ASQ course in
DOE ($1165)
 Stanford - open source learning management
system (LMS) – no DOE
 MIT – (Stellar) Open Courseware - no DOE
 University of Phoenix – 3 doctoral programs
added to 1 existing. 125,364 students 5/31/02
– average age=38
– no DOE ?- MBA in Technology Management
Assessing On-line Courses




http://taste.merlot.org/
Multimedia Educational Resource for
Learning and Online Teaching (MERLOT)
Non-profit on-line community
Peer review, user comments on 3
evaluation standards
–
–
–
Quality of Content
Effective Teaching-Learning Tool
Ease of Use
Industrial Design of Experiments course online at Winona State - assessments





Pre-test on course material.
After student enrollment and course introduction,
do survey about the enrollment and start-up
process.
After each homework problem with supplied
answer, ask if the solution is understood, if they
could do the problem independently, if they want to
ask for help.
1st learning assessment- after review of basic
statistics and hypothesis testing.
In conjunction with 1st learning assessment, do
survey about “how its going”
DoE at Winona State –
Further assessments

Further (5) learning assessments, after each
logical section of course material, including
assessment of projects (catapult-by-team or
individual and self-defined)
 Another “how its going” survey after twothirds of course content
 Final, comprehensive learning assessment
 Final “how it went” survey, including questions
of what the student wants for follow-up
A pre-test question on BlackBoard
Surface Mount Technology (SMT)
experiment - problem solving in a
manufacturing environment
2
types of defects, probably related
 - Solder balls
 - Solder-on-gold
 Statistician invited in for a “quick fix”
experiment
 High volume memory card product
Homework assignments and answers on
Blackboard