Live Captioning with Dragon
Live Captioning
with Dragon
SETBC project, to provide students
with speech to text input from their teacher,
using Dragon Naturally Speaking
Key Project Contacts:
Karen Bell, Support Teacher
Jill McCullum, District Hearing Resource Teacher
Meeting of the Minds
During a meeting, September 13th 2012, the
idea of providing closed captioning on a laptop
for our hard of hearing students, evolved from
discussion re: using a Smartboard or T.V.
screen. The grade 7 classroom was chosen
because:
•Teacher was tech savvy
•Deaf and Hard of Hearing CI student is very
bright
•SET-BC student was already in this classroom
•Experienced OCF’s available
Karen Bell, Support Teacher at SMS
Jill McCullum, District Hearing Resource Teacher SD67
Jason Corday, Principal of SMS
Dan Paccioretti, Audiologist, Phonak Canada
Anita Toneatto, Distict Technology Helping Teacher
Carla Norheim, Hearing Resource Teacher SD23
Holly Colquhoun, Teacher at SMS
Sandra Cureatz, OCF/CEA
Erica McDowell, OCF/CEA
(Our project has evolved several times since the initial meeting)
Proposal
To make the classroom teacher’s auditory message a
visual message, thereby assisting students with:
memory issues

motor issues

written output issues

organization issues

compromised audition

processing speed
limited English speaking capabilities


The project will be successful if:

By the end of the school year the students are able to use the technology independently
– take notes and organize the material from lectures.
Proposal Summary con’t
When the teacher is delivering instruction, he will use:

Dragon, the school’s infrared sound field system and Phonak FM
technology to create captions on the student’s laptops.



This will allow the student to refer back to the teacher’s oral
instructions/directions for: assignments, clarification, review and vocabulary
development.
The lecture can then be converted into Kurzweil and highlighted with the
assistance of the classroom CEA.
The notes can then be extracted and saved to a file under the course. Later
the student can have the notes read back to him with Kurzweil if he/she
decides for studying.
Equipment
Existing Equipment:
Utilized existing classroom infrared sound field system (Frontrow to
Go) , we also tried with a Simeon 500 WU.
The following pieces of hardware and
software have been requested, approved and
readied for the project:








3 laptops – 2 are older Toshiba Satellites and
1 is a Toshiba Tecra
3 Phonak MyLink receivers
Inspiro Premium Transmitter – teacher
Easy Boom microphone – teacher
Microsoft Office
Antivirus software
Digi-Master X was also setup but not
essential to the project
Dragon Naturally Speaking and Kurzweil
were already loaded on all 3 laptops
Fax # / Email:
kbell@summer.com
FM System Cost
Reference:
Penticton SD #67
Acct #:
20068934
FM SYSTEM INCLUDES:
QTY
Unit price
Amount
$1,399.00
$1,399.00
$300.00
$300.00
$400.00
$400.00
Description
1
inspiro Premium transmitter
includes: charger, EasyBoom mic, 3.5mm patching
cable, belt clip, lavalier cord, USB cable
1
Classic Mylink FM receiver w/charger
1
DigiMaster-X (optional) w/power supply, patching cable
TOTAL PACKAGE COST
$2,099.00
Discount negotiable
SHIPPING AND HANDLING
TOTAL FOR SYSTEM AND WARRANTY
$10.00
$2,109.00
Technology Setup
The MyLink receivers were connected
to all three student laptops.
The teacher was fitted with the boom
microphone and the Inspiro Premium
Transmitter, clipped to his belt.
Technology Setup con’t
After ensuring that all devices were connected and
updated, Dragon was opened on one of the laptops to
“train” the program. Then after the training was
complete the teacher’s and OCF’s profiles were copied
and transferred to the other student laptops.

As the teacher spoke, the text appeared on all three
laptops with very few errors. It was discovered that the
MyLink receivers work best if hung around the student’s
necks rather than on the desk beside the laptops.

Kathy Ryan (SETBC) provides guidance for
Dragon Speak voice profiling to the team
When the teacher was finished, the documents were
saved as .rtf files so they could be used in Kurzweil,
Office etc.

As a bonus, the DigiMaster X is transmitting the
teacher’s voice through the classroom’s Front Row
speaker system which allows other students in the room
to benefit from the clear consistent sound of the
teacher’s voice.

OCF prepares voice profile for Dragon and
Tech. department sets up the laptops.
Responsibilities
Teacher Responsibilities
Student Responsibilities
Prepare Dragon profile – initially this takes
about 15 minutes. A key to doing this is
to create the profile with the microphone that
you will be using.
 Become familiar with Dragons features
 Become familiar with editing terminology for
Dragon for continual training
 Be flexible in having different people visiting
their classroom to view the project in place
Each morning have computer out and
safely positioned on desk top
 Have Dragonpad running, ready for
teacher’s dictation
 Ensure computers and MyLink
equipment is recharged daily
 Training on Kurzweil
 Save documents as .rtf files
 Be able to transfer documents to
Kurzweil program

OCF’s Responsibilities
• Prepare Dragon profile to use if teacher is away
• Supervise the student’s day to day routines –
ensuring computers get plugged in etc.
• Label/personalize all equipment
• Charge and safely store away the Inspiro
Transmitter
•Train using Kurzweil to assist students
Use of Dragon NaturallySpeaking for RealTime Transcription for
the Deaf or Hard of Hearing
TEACHER MODEL:

Teacher has a microphone that is
broadcasting to a computer which
has Dragon NaturallySpeaking
running which is doing the realtime transcription.

the throughput accuracy will vary
greatly depending on the speaker,
the environment, and the subject
being discussed.
* (This was the first method that we tried in the classroom, which later evolved into our next method)
The Beginning…
Things we are doing right:
• The teacher that we chose is comfortable with technology, his voice is
clear/articulate and he often teaches lecture style.
• There is a full time CEA in the classroom who is an Oral
Communication Facilitator and is used to working with sound
equipment.
• The three students are sitting in different areas of the classroom. It
doesn’t seem to affect the accuracy but there is a slight lag time.
• We have an Audiologist, District Hearing Resource Teacher, District
Technology team, partnered with SETBC and with Phonak Canada.
• One of the students involved in the project asked if he could a write a
blog. The blog has been created and is linked to this page.
Frustrations
• When we did the initial training with Dragon, it was 98% accurate. As the teacher used the program he could
not see the errors and therefore did not error correct. As the weeks went by, the accuracy dropped to 70% and
then down to 10%.
• We learned that Dragon only works as well as we have trained it. Without monitoring for errors and correcting,
it is not accurate.
• When the teacher is putting the boom microphone on, it is critical to have the EasyBoom Microphone placed
with the correct orientation to the mouth.
• The teacher was using Dragon to teach Math while using the Smart Board. It was suggested to us that this
would be problematic because he wouldn’t be using full sentences. We are learning that some subjects are more
suited than others.
• Our student with the cochlear implant was not looking at the
teacher and trying to get all of his information from the
captioning. He was working twice as hard to get the information
because the screen was not accurate. We told him that it was
best to follow what the teacher was saying and refer to the
captioning for missed information.
• The laptops are powered on for the better part of the day. The
older laptops battery life is not as long lasting. Each day the
three laptops, Inspiro and MyLinks have to be recharged. The
OCF takes care of this but we have realized that it takes a lot of
counter space. We also need to keep the equipment secure.
Frustrations con’t
• No Structure, words jumbled together
• Students begin to ignore the screen out of frustration
• The program wasn’t meeting their needs
• Dragon losing accuracy
Troubleshooting
• Believed that our problems were cause by the software,
Dragon Naturally Speaking
• Teacher & OCF spent hours updating their profiles
• Troubleshooting the Dragon
• Testing all connections
• Testing Boom mic in various settings (noise)
• Testing other wireless mic’s with Dragon and Windows 7
Speech Text software
Turning Point
• Realized that the MyLink receiver was experiencing a lot of
interference (loud static) and picking up voices from
Digimaster down the hall.
• Now OCF must check the receiver frequently
Perspectives…
Teacher Perspective:
“The project has been both exciting
and frustrating. As all projects with
technology usually are. It was
frustrating when the project was
stalled by technology glitches which
couldn't be resolved. It has also been
surprising to find out how much of
instruction is conversational based.”
OCF Perspective:
“I completely agree with the classroom teacher, that this project has had it’s ups and
downs. It’s a lot to learn, in regards to the technology and the software. Ideally, the
project should run smoothly but has encountered its share of problems. It’s also evolved
several times to meet the needs of our students and the classroom but I’m still excited to
move forward with it!”
Perspectives…
Student Perspectives:
“When I first heard of this project, I
got excited, but lead to some
little disappointing things. The accuracy was
slow and way off. The teacher says one
thing, the laptop says another. Its really
confusing, and misleading. This is a
little disappointing with all the stuff I have to
watch and do to keep up with the class, the
laptop makes it worse.” -A.W


“I may find it nice, but its distracting in the
class. I find the laptop useful, but not the
dragon. It makes way to much mistakes.” G.M

“Right now, its making a lot of mistakes, but I
know it will get better. I have used dragon
before.” -E.S
New Vision….
It occurred to us that when we were using the Teacher’s voice this placed
too many demands on the student participants. They were stressed at the
amount of visual data and Dragon inaccuracies. As the teacher
instructed, he often included irrelevant information, and didn’t always
speak in sentences.
Our students are still learning to access real time closed captioning.
They do not have the capacity to extract the important information from
the lecture. They are in grade 7. We have recognized that we set the bar
too high.
Use of Dragon NaturallySpeaking for RealTime Transcription for
the Deaf or Hard of Hearing
OCF MODEL:

There is a OCF (redictator) is listening
to the teacher/speaker and is dictating
their spoken words which broadcasts
to the computer and into Dragon
NaturallySpeaking.

the OCF (redictator) can be skilled at
simultaneous dictation (extracting the
main points of a lesson) which can
result in much higher accuracy.
* (Current model)
New Vision con’t….
Remote Re-dictator

In this model the OCF is skilled in simultaneous re-dictation which can be captured and
transcribed onto the students’ PC… (Note, “should have requested an additional computer for the
OI.”so that the OI can check the dictation for accuracy) This digresses from our previous focus to
have the Classroom Teacher be the sole conveyer of the message. The OCF will extract the
significant points from the lesson and use Dragon to deliver the main points to the students’
laptops.

There are many features that recommend this model, especially for the Deaf and Hard of Hearing.
However, it demands access to a classroom support person with competency in Dragon software.

We will try to dovetail our project with technology being used at the post secondary level
(University Campus). It’s important to know if we are accurately mirroring what might be
anticipated at university. Educating our students to utilize a closed captioning visual stimulus is
part of our goal.
ReDictator to be capturing and transcribing classroom directions, discussion and question and
answer sessions (keeping it brief and simple in the beginning, then building their content &
commands as the OCF builds their skills)


By placing the ReDictator in the remote location there's no "distraction" for other members of the
classroom.

There are quite a number of wireless technologies that can be used to broadcast the instructors
dictation or capture a classroom discussion e.g. Dragon, Windows 7 Speech Recognition.
Human Element: Journey
Student(s)
Personal bias/self image/equipment
Age/skill limitations (extracting notes, visual fatigue, set up on desk)
Bigger picture vs. tunnel vision
Patience or lack there of
Anxiety…type ‘A’ personality, perfection/Parent-Guardian involvement.
Jill McCullum
Team
Time constraints…approx # hours to date: 100+
(training, equipment monitoring, tech meeting attendance, personal involvement on own time)
Enthusiasm (team effort, great support from many people with vested interest e.g.






Technical staff SD#67/SET-BC
Admin
Teacher(s)
Support Staff-OI team (kudos)
Companies Phonak/FrontRow/Points West/SET-BC
Student participation (mixed emotions)
At various points along the way we have all hit the wall
Committed…absolutely
Vision (future application) desire to save others money and time
Conclusion: Not Yet….but Soon !!!!
Further
Discussion…
Receiver
Problems
Fatigue
Accuracy
Extracting
Notes
Windows 7
LIVE
Captioning
Training
The Cloud
Tech.
Support
Dragon
Frequent
Issue
OCF
Juno
Thanks to…
Team and Project Managers:
Karen Bell and Jill McCullum
School Team:
Jon Broadbent
Sandra Cureats
Jason Corday
Thora Delaquis
Holly Colquhoun
Erica McDowell
Tech.Team:
Danny Francisco
Anita Toneatto
Igor Pavlina
SETBC Team:
Kathy Ryan
Flo Wong
Company Contacts:
Dan Paccioretti, Audiologist, Phonak
Tracey Cochrane, Frontrow