Blended Web Learning:
Advantages, Disadvantages, Issues,
and Considerations
Curt Bonk, Indiana University and ARI Senior
Research Fellow
Tatana Olson, Purdue University and Army
Research Institute
Bob Wisher, Army Research Institute
Kara Orvis, George Mason University and Army
Research Institute
e-Learning wants you,
but . . .
do you want e-Learning?
National Guard
Providing access to e-learning to 350,000 soldiers in
more than 3,360 communities
Ft. Richardson
Tacoma
Helena
Bismark
St. Paul
Augusta
Madison
Salem
Colchester
Latham
Rapid City
Lansing
Cheyenne
Boise
Concord
Reading
Lincoln
Sacramento
Springfield
Columbus
GuardNet XXI Network
Draper
Carson City
Indianapolis
Johnston
Trenton
Providence
Hartford
Indiantown Gap
Wilmington
Englewood
DC
Topeka
Santa Fe
Frankfort
Jefferson City
Oklahoma City
Nashville
Phoenix
Charleston
Richmond
Little Rock
Raleigh
Atlanta
Columbia
Montgomery
Honolulu
Jackson
Austin
Tamuning
New Orleans
St. Augustine
San Juan
Kingshill,
USVI
Arlington
DTTP
CLASSROOM
INSTALLATIONS
National
Guard
Classroom Fielding
February 12, 2002
Seattle
Ta coma
E veret t
WA
Kent
Devils
Lake
Mi n o t
3
S p o kan e,
WA
3
WA
p
Yakima
Helena
Camp
Rilea,
MT
Helen a
AF RC
12
G rand
Fork s
Bemid ji,
6
ND
Bi smarck
3
Duluth
6
La
G rande
12
Sh eri d an ,
WY
3
Monmouth
Caldw ell
Boise
ID
M ontevide o
12
Mi tch ell
4
SD
City
OR
Bend ,
3
Ft.
M ea de
12 Rapid
MN
Ashland
3
Mi n n eap o l i s
12
Ch i p p ewa
F al l s
12
Ro ch ester
12
Sioux
Falls
WY
Pocatello
ME
T o mah awk,
WI
12
Bo zeman
2
Portla nd
Eu gene
7
Carib ou
3
Camp
Ri p l ey RTI
18
p
Warren to n
In ver
Gro ve
Hg ts.
12
Mo o rh ead
3
Va lle y Fargo
City
WI
G ue rn
-s ey
No rfo l k
3
NE
Sc ottsbluff
Chey enne
12
Camp
Ashland
Kearney
Mi l an
8
Dodge
Reno
San
Rafael
12
S t Jo sep h
12
Au ro ra
18
Mt.
Pl easant
CO
Gran d
Ju n ct i o n
3
Fa irfie ld Stockton
12
Modesto
IL
Salina
Hays
4
Fort
Carson
Topeka
18
Iola
5
Wi chi ta
La s
Ve ga s
F armi n g to n
6
Enid
Santa
Fe
Gallu p
6
V an N u ys
MMS - 12
West
LA
12
Leaven
-w orth
KS
Ri o
Ranch o
12
AZ
Los
Alamitos
OK
Oklahoma
Cit y,
MDA
3
Ok Ci ty,
23rd S t. Okl a.
Armo ry Ci ty,
WR A
NM
S t i l l water
6
O kla.
City
Tulsa
S ed al i a,
5
Braggs
12
Ro swell
6
Jef ferso n
Ci ty
12
St
L o u i s/
L amb ert
3
MO
T u cso n
12
Bro wnwo od
Ashland
6
G reenville
Jackson
12
Little
Rock
Chat t anooga
18w/VTT
AL
E l l i o tt
12
Jackso n
(st arc)
12
Jackso n
12
Ruston
No .
F t Hoo d ,
Gatesvil le
Murphy
Meridian
F t Gi l l em
12
Macon
Mo n t g o mery
12
Ne w
O rle ans
An nville
Fort
Discovery
Al exan dri a
2(L AB)
A l exan d ri a
(JP 1)
S t au n to n 12
18
w/ V T C
NJ
Ft.
Bragg
L an caster
9
FlorColumence
bia Eastover
SC
MDd
d
Sea
G irt
Cad
mp
Pe ndle ton
DC
d
Teaneck
Wilmington
Hartford
E ast
Green wi ch
Jo h n sto n
12
Ni an t i c
12
Cap e May P o rt Mu rray T ren t o n ,
NJ
12
12
12
Westfi el d
6
B eth an y
B each
9
Ha ge rs- Ches tow n te rtow n
An n ap o l i s Oakl an d
12
3
E asto n
3
Washington
Tarboro
12
Raleig h
Sanf ord,
NC
18w/
VTC
Fort
Dix
SalsLaurel Reister bury
-tow n
Bo wl i n g
Green
12
Fort
Pick ett
NC
S p ri n g f i el d Reh o b o th
12
P i ttsf i el d
12
RI
DE
Alex andria
VA
She lby
Ne w be rry
Re ad- Bourne
ing
CT
3(L AB)
G ree r
Clint on,
NC
18w/o
VTC
Ne w
Bern
12
Wilmington
12
Geor get own/
Conway
8
N. Ch arlesto n
Opelika/
PheonixCit y,
12
GA
Gu l fp o rt
12
Plaque m ine
As hv ille
7
Center
Stra fford
Milford
d
Al exan dri a
L exin g to n
18
MA
Cp . S mi th
12
Allentown
NorrisEAAT S town
d
Jo h n so n
Ci ty
N.
12
Wi l kesb o ro
12
Do b b in s
AR NG Do b b in s
AN G
12
12
Mo n t i cel l o
12
LA
Lake
Ch arl es
Hous ton
F o rt
McCl el l an d ,
2. 5
Bi rmi n g h am
12
d
NH
VT
ville
Mo rg an to wn
Gate C i ty
12
An d erso n /
Cl emso n
Northport
MS
Ca m p
Shelby ,
Ja ck son
Ca mp
Ba ll,
Pine ville
San
An g elo
Bastro p
3
Smy rna
Wheeling
B u ckh an C- h arl es- F airmo n t
to n
non
Knox ville
12
TN
Hunt ingt on
Ru tlan d
12
Latham
H arri sb u rgFIG ,Ann8
d
Su mMartin s- Parkers mersbu rg -b urg vil le
Beckley
d
B arb o u r vi l l e/
London
8
N. Li ttle
Ro ck
PEC
Ft. Dallas
Worth
Austin
KY
F ran kfo rt
12
M arion
AR
Ft S mi ht
No rth p o rt
McAl ester
4
Bridgepor t
S cran to n ,
PA
12
Pittsburgh Johnstown
WV
Evansville
12
Cap e
Gi rard eau
12
Wi ch i ta
F alls
TX
Las
Cruces
6
Marana
L u bb o ck
apolis
At hens
Ard mo re
3
Phoenix Gl en d al e
San
Diego
Columbus
Lafayette
India n12
Jo pli n
12
Ad a
3
Akron
18
East St
Lo ui s
15
Nevad a
3
Binghamton
12
PA
OH
NY
Sy ra cuse
12
Buffalo
18
E d i n b u rg h
12
L exi n g
to n
L en exa
12
Detro it/
Taylo r
12
Erie
Urbana
12
Springfie ld
Macon
5
Kan sas
C i ty
12
Augusta
IN
S p ri n g f i el d
18
Sa n
Luis
O bis po
Long
B each
MMS - 12
E n g l ewo o d
18
UT
CA
F resn o
18
w/ V T T
S an L u i s
Ob i sp o
12
Vernal
Sp an ish
Fo rk
NV
Sacramento
BayCit y/
Flint
12
Lansing
Bend
w ood Val- S. 12
12 paraiso
Fort
12
Wayne
12
Li nco ln
Draper
Ri vert o n
Wyoming
N. Riverside
18 Crest-
Ced ar
Rap i d s
Daven p o rt
9
9
Johnston Ca mp
Co u n ci l
Bl u ff
9
Berlin
Mi l wau kee
IA
Augusta
12
MI
Mad i so n
18
Si o u x
Ci ty
9
Bang or
12
Colches-Camp
nson,
terCoJoh
l chester
Camp
Grayl i n g
Ap pl eto n,
18
F t McCo y
18
Fort
Stew art
Mo b i l e
12
Panama
Ci ty
12
T al l ah assee
12
St
Camp
Bl and in g Au g u sti n e
18
Starke
FL
Orl an d o
A DL
C o -L ab ,
10
Sa n
Antonio
O rla ndo
12
St.
Petersburg
TOTAL CLASSROOM INSTALLATIONS: 300*
Status of Site:
Waimanalo
No m e
Bethel,
AK
(MMC3)
AK
F ai rb an ks
3
Fort
Richard
-son
Classrooms complete (including IIS) (209)
Classroom complete without IIS (80)
Classrooms signed off with exceptions (6)
Classroom installations on hold (0)
Classroom installations started/in progress (5)
Regions/sites with changes in past week
Rad io
Barrig ad a
GU
Ju neau
4
HI
Hi l o
5
PR
Fort
Allen,
Juana
Diaz
P earl Ci t y
12
VI
eds gig: dlct-arng-k1100bj4030-kgf-weekly map-02/12/02
S t C ro i s,
Ki n g s H ill
12
St
Th omas
12
Type of Site #
M M S1.......151
DM M C2......35
M M C3.........65
M TC4..........49
Total......... 300*
* Total does not include 61 Single Trainer Classrooms (STCs)
1
Multi-Media Scaleable; Classroom total includes two Next Gen prototy pe classroom s (marked "p") in MT
2 Dual Multi-Media; Classroom total includes nine dem onstration classrooms (m arked "d") in MD, PA, VA,
DC
3Multi-Media Classroom . 4Medium Trainer Classroom
Blended Learning Advantages
1. Course access at one’s convenience and flexible
completion
2. Reduction in physical class time
3. Promotes independent learning
4. Multiple ways to accomplish course objectives
5. Increased opportunities for human interaction,
communication, & contact among students
6. Less time commuting and parking
7. Introverts participate more
Blended Learning Disadvantages
1. Procrastination, procrastination,
procrastination
2. Students have trouble managing time
3. Problems with technology at the beginning (try
too much)
4. Can be overwhelming or too novel
5. Poor integration or planning
6. Resistance to change
7. Good ideas but lack of time, money, & support
Some Blended
Learning Results…
Higher Education: Student survey
results after a hybrid course
• Student feedback N=282
• 69% felt they could control the pace of their
own learning
• 77% felt they could organize their time better
• 16% felt the time spent online would have been
better spent in class
• 61% felt there should be more courses like this
– www.uwsa.edu.ttt/articles/garnham.htm
Corp. Success Story #1. Infusing E-Learning
Elliott Masie, March 2002, e-learning Magazine)
A manufacturing company transformed a
week-long safety program into a three-part
offering:
1. One day in classroom
2. Multiple online simulations and lessons.
3. One final day of discussions and exams.
Must accomplish online work before phase 3—
this raised success rate, transfer of skills, and
lowered hours away from the job.
Success Story #2. Ratheon, Build Own LMS
(John Hartnett, Online Learning, Summer 2002)
SAP Training Choice (saved $252,000):

Cost of Instructor-led Training ($388,000).

Vendor LMS Cost ($390,000) or

Cost to Build Internally ($136,000) or
Five Training Components in 18 Weeks (within 6
weeks, 4,000 courses were taken by 1,400 students)
1. Role-based simulations
2. Audio walk-throughs
3. Online quick reference system
4. Live training support (special learning
labs—try out with live mentor or instructor)
5. Online enrollment and tracking
Success #3: Microsoft Excel Training
(Jeff Barbian, Blended Works, Summer 2002,
Online Learning)
• Group One: 5 scenario-based exercises that
offered live use of Excel on real-world tasks,
online mentors, FAQs, relevant Web sites,
NETg Excel Fundamentals Learning Objects.
• Group Two: Same as Group One but without
scenarios, but info in 5 scenarios were
embedded in the learning objects.
• Group Three: No training control.
– Grp #1 30% more accurate than Grp 2; Grp #2
159% more accurate than Grp #3
Success #4: Sallie Mae/USA Group
(Blended student loan provider program)
(Jeff Barbian, Blended Works, Summer 2002, Online Learning)
• LEAD (Leadership and Education Development);
Groom internal staff to fill supervisory-level positions
• 4 hours/week in class with internal and external
instructors; learn trust, role of managers, etc.
• First must complete 3 online management courses from
SkillSoft and 6 online project management courses
(includes panel presentation by IT Project Team to
illustrate how projects are handled in the company’s
culture)
• Findings: increased teamwork, camaraderie, shared
understanding of concepts, respect for individual
differences, social interaction, and reinforcement for
class concepts.
Success #5: Proctor and Gamble
(Jeff Barbian, Blended Works, Summer 2002, Online Learning)
• 1999 = 100,000 employees; 20,000 trained/year
• LMS from Saba, live training from Centra
• CD-based training using Authorware,
CourseBuilder, & Dreamweaver
• 2002 = 1,200 learning items; 34% Web, 54%
CD
• Global English saved $2.5 million per year
• Off-the-shelf courses in time management and
managing for success
Proctor and Gamble
(Jeff Barbian, Blended Works, Summer 2002, Online Learning)
“Given our learning objectives and needs, should
we select Web-based live training, versus
classroom, versus video-based, versus CBT, or
some blended solution?…It depends, on the
resources you have, how far geographically you
have to reach, or whether you can get your arm
around them and pull them into a classroom.”
Art DiMartile, Senior IT Manager, Proctor and Gamble
Three Phases of AC3-DL
I. Asynchronous Phase: 240 hours of
instruction or 1 year to complete; must score
70% or better on each gate exam
II. Synchronous Phase: 60 hours of
asynchronous and 120 hours of synchronous
III. Residential Phase: 120 hours of training
in 2 weeks at Fort Knox
AC3-DL Course Tools
• Asynchronous:
– Learning Management System
– E-mail
• Synchronous: Virtual Tactical Operations Center
(VTOC) (7 rooms; 15 people/extension)
–
–
–
–
Avatar
Audio conference by extension/room (voice over IP)
Text Chat Windows—global and private
Special tools for collaboration
LMS Can Help Determine:
•
•
•
•
Who is currently or recently active
How far learners have progressed
Exam completion and scores
Where students are experiencing
difficulties
• Who needs reminders
AC3-DL Collaboration Tools
• Asynchronous:
– Document sharing
– E-mail
• Synchronous: VTOC
–
–
–
–
Shared text
Shared bookshelf
Mapedit
3D terrain
Teams Collaborate on:
Mission Analysis

Information and critical reflection on:






terrain and weather,
enemy forces,
facts, assumptions, limitations,
specific tasks, implied tasks,
assets available, and
additional considerations,
Mapedit Tool
The Mapedit program, was developed to
create map overlays, emulating plastic
sheets on which symbols are drawn
that are laid onto a map (like football
playbooks for the maneuver officer).
And if students want a whiteboard,
they simply have to open a blank
overlay (no map background).
Mapedit Tool
Mapedit allows multiple users to
add, delete, and move symbols and
lines on the map overlay. In
Mapedit, the driver chooses which
file to open, and names the file to
save, but all users can edit the
contents.
3D Terrain Tool
The 3D terrain is a collaborative
environment that does not result in a
product, but, instead, allows students
and instructors to "walk" the terrain
and lay an overlay on the ground.
Participants can click on another
person's avatar, and they will see what
that person sees as he makes key points
about terrain.
Previously Reported Results
Sanders & Guyer (2001)
• Fairly positive compared to pen and paper
correspondence
• Student fairly familiar with async and sync
communication tools
• Student attitudes higher than unit leaders
• Problems encountered: time, drill time
conflicts, technical problems, family
responsibilities, lack of compensation
Previously Reported Results
Sanders & Burnside (2001)
• Completed coursework in less time than
correspondence course.
• Positive attitudes
• Covered additional content not covered in
correspondence version of course
• More likely to make decisions, develop greater
sense of team identity
• Greater planfulness, confidence, tactical
proficiency, and leadership skills.
Present Two-Part Study
Research Methods
• Questionnaires (of AC3-DL Students only):
– Perceptions of the synchronous environment
– Technology access
– SCALCO—perceptions of constructivist nature of
AC3-DL
• Chat Transcript Analyses (over 6,600)
• Focus Group Interviews
– 2 groups of 4 students in Phase III (live training)
– 3 instructors
– Course developers and supervisors
Study #1: Synchronous Chat
Analysis (explored learner
online problem solving)
Orvis, K. L., Wisher, R. A., Bonk, C. J., & Olson, T. (in
press). Problem-solving exercises in military training:
Communication patterns during synchronous Web-based
instructions. Computers in Human Behavior (Special
Journal Issue on Computer-Based Assessment of Problem
Solving).
Table 1: Chat Coding Categories
Category
Description
On-Task
(a) Any interaction focusing on the task at hand.
Social Interactions
(a) Interaction related to group and class work, but not
the immediate task.
(b) Interaction regarding issues outside of the group
and class.
Mechanics of the Process
(a) Interaction regarding the equipment being used that
is not task related, including problems with technology
access, passwords, technology checks, etc.
Figure 1. Social, mechanics, and on-task
behaviors in the chat interactions over time
Task %
Social %
Tech %
80%
70%
60%
50%
40%
30%
20%
10%
0%
Month 1
Month 2
Month 3
Month 4
Month 5
Month 6
Overall frequency of interactions
across chat categories (6,601
chats).
On-Task
Social
Mechanics
70%
60%
50%
40%
30%
20%
10%
0%
Month 1,2
Month 3,4
Month 5,6
Overall frequency of interactions
across chat categories (6,601
chats).
Mechanics
15%
Social
30%
On-Task
55%
On-Task Problem Solving
Mayer & Wittrock (1996)-The transfer that
occurs when prior experience and
knowledge influence learning in a new
situation:
• “Terrain does not allow for effective
maneuver of your element”
• “Harder to detect a liquid agent in rain”
• “Rain can also degrade optics on weapon
systems”
Sternberg (1997)-The acts of defining a
problem, allocating resources, and
tracking progress:
• “I don’t see anything about obstacles in
the CLOSE section”
• “I think obstacles in the Close section of
the COA statement is a necessary evil”
• “Remember in the BDE OPORD-the
BDE CMDR wants this to occur at
about this time”
Social Interactions
• “Kids are great we made breakfast for
Mom (wife)”
• “Did you go out for a run last night?”
• “Tell her I said happy mothers day”
• “3 miles in 24 mins all hills”
• “If God had meant for us to run, he
wouldn’t have given us tanks”
Mechanics of the Process
• “Cannot talk or hear...will try to
reconnect.....”
• “Is anyone talking right now? I think I
dropped audio”
• “Going to reboot”
Study #2: Focus Group
Findings
Bonk, C. J., Olson, T., Wisher, R. A., & Orvis,
K. L. (2002). Reflections on blended learning:
The Armor Captains Career Course. (Research
Note #2002-13). Alexandria, VA: U.S. Army
Research Institute for the Behavioral and Social
Sciences.
Results
Bonk, Olson, Wisher, & Orvis
• All had access to technology
• Enjoyed the course, excellent
technologies
• Favored sync over asynchronous
• All pointed to ways to address high
attrition
• Perceived training transfer
• Learned to work as a team
Bonk et al. Results Continued
• High individual and collective efficacy
• High interpersonal and task cohesiveness
• Perceived an active learning environment
Advantages and
Disadvantages
Student Perspective: Advantages
1. Flexible and convenient for working fulltime
2. Students can work at own pace.
3. Immediate feedback and mentoring can be
received online and at any time.
4. Online learning environment can be
structured for active learning.
5. Students learn to work together while online.
6. Online chats fostered thoughtful commenting
and reflection.
Student Perspective: Disadvantages
1. Lack of preorientation session to detail course
expectations and tools.
2. Length of course did not match needs.
3. Novel technologies difficult to use and crash.
4. Must learn in preset order; minimal system
flexibility. Instructor permission to progress.
5. Minimal asynchronous feedback.
6. High attrition due to large modules.
7. Community and identity takes much
forethought.
Instructor Perspective: Advantages
1. Fits army small group instruction model.
2. Tailor instruct strategies to form of delivery.
3. Can provide immediate and detailed
feedback.
4. Group interaction boosts confidence of lows.
5. Greater learning and application of
knowledge.
6. Standardization of content; stay up-to-date.
7. Teach to solve problem sits & communicate.
8. Timely online mentoring opportunities.
Instructor Perspective:
Disadvantages
1. High attrition/low completion rates.
2. Excessive student time commitments.
3. Lack of instructor control over size of
content.
DL Education Advisor Perspective:
Advantages
1. Course can take advantage of recent
advances in technology and theory.
2. Can embed different forms of media to
address different learning styles.
3. Can historically track student work and
learning.
4. Can embed authentic and meaningful
activities.
DL Education Advisor Perspective:
Disadvantages
1. Many risks in selecting a particular tech.
2. Hard to be aware of future technologies
when dealing in present tech issues.
3. Too many choices overwhelm learners.
4. Some students waste paper.
5. Some want CDs when have Web &
paper.
6. Learning management system not
flexible.
What were the key issues
or considerations?
Issue or Consideration:
1. Feedback
Student Advice: E-mail is
important for contacting instructors.
Instructor Advice: Provide instant
and consistent feedback with e-mail
and other tools.
DL Advisor Advice: Involve
direct e-mail.
Issue or Consideration:
2. Meaningful/Real World Content
Student Advice: Products constructed
should approximate real world.
Instructor Advice: Require students to
produce products instructors & peers evaluate.
DL Advisor Advice: Include meaningful
content and apply new skills to real-life content.
Issue or Consideration:
3. Size & Scope of Content/Materials
Student Advice: Divide async. content and
testing into smaller units or accomplishments to
motivate and enhance completion rates.
Instructor Advice: Instructors need some
control over content size so as to inc completion.
DL Advisor Advice: Utilize minimal
extraneous content, graphics & practice exercises.
Issue or Consideration:
4. Course Devel & Organization
Student Advice: A preorientation session
will help address concerns. Lecture before assign
project work.
Instructor Advice: Learn content async
(crawl); put to use asynchronously (walk); and
apply to real-life scenarios (run).
DL Advisor Advice: Carefully analyze
audience wants & needs prior to course
development.
Issue or Consideration:
5. Role of Instructor
Student Advice: Instructor is helpful as
facilitator of learning. Same instructor should
support students across all phases.
Instructor Advice: Instructor is
facilitator; provides tools, guidance, and means to
learn. Engage students with indirect questions,
reminders, role play, and direct requests.
DL Advisor Advice: Instructor provides
feedback and sense that someone cares.
Issue or Consideration:
6. Small Group Structuring
Student Advice: Rotate roles among
students in role play activities.
Instructor Advice: Match strong leaders
and weaker students to boost confidence and
performance. Provide instructions b4 role play.
DL Advisor Advice: Create active
environment—role plays & simulations. Balance
flexibility and accountability.
Issue or Consideration:
7. Flexible and Active Learning
Student Advice: Allow students to
complete modules at own pace; minimize
instructor need for approval to advance.
Instructor Advice: DL helps students
who work fulltime keep up-to-date.
DL Advisor Advice: Offer flexibility,
choice, variety, meaningful contexts, and
performance opportunities.
Issue or Consideration:
8. Technology Utilization
Student Advice: Utilize basic technology
functions so as to minimize frustration.
Instructor Advice: Use async for basic
concepts and synchronous for application.
DL Advisor Advice: Limit technological
visions and incorporate technology based on what
it can presently accomplish.
Issue or Consideration:
9. Build General Skills: Communication,
Problem Solving, Teamwork, and Identity
Student Advice: Small talk, introductions,
and info sharing help form team identity.
Instructor Advice: Problem solving and
communication are outcomes of DL. Online
tasks should teach how to work together. Virtual
talk fosters interaction and participation.
DL Advisor Advice: Move from indiv
(async) to application in small groups (sync) to
problem solving in units (resident component).
Issue or Consideration:
10. Assessment Practices
Student Advice: Assessment should match
real-world expectations. Shift focus from
quantity of learning or breadth to depth. Assess
should cover smaller units of instruction.
Instructor Advice: Async for objective
tests; sync for student performances/products.
DL Advisor Advice: Use online tests and
exercises with automatic feedback. Also embed
random checks and some flexibility.
Recent Changes
• Guest Mentors in VTOC
• Mix 2 weeks of Asynchronous and 1
week of Synchronous
• Allow to Complete in Less Time
Questions or
Comments???