Collection of speech production
ultrasound data
Donald Derrick12, Romain Fiasson2 and Catherine T.
Best1
1University of Western Sydney (MARCS Institute)
2University of Canterbury (NZILBB)
Introduction
• Ultrasound Imaging
– Uses high frequency sound waves to image
density changes in soft tissue
• Ideal for imaging the surface of the tongue
• But cannot penetrate air or bone boundaries
– Can miss the tongue tip and/or root
– Palate trace difficult
– No pharyngeal wall recording
– Provides noisy medical-grade images
• Intended for diagnosis
• Often difficult to measure directly
3
Ultrasound - edges
• Ultrasound can miss
tongue tip/root
• Pick the right probe
and placement
– Narrow
– Curved-array
– Away from bone
• Forward of notch for
root
• Adjacent to notch
for tongue tip
Derrick and Fiasson (In Prep)
4
Ultrasound – frame rate
• Ultrasound frame rate factor of:
– Ultrasound CPU
– Image smoothing
– Image lines
– Image angle
• Combined with video capture methods
– Trade portability, A/V sync, fps
5
Ultrasound – video capture
• Ultrasound speed
– Video capture
• SD 24/30 fps interlaced
• HD 48/60 fps interlaced
• A/V sync
– Cineloop full speed
• Short duration (8-16 second)
• External A/V sync only
– Frame grabber 60 fps
progressive
• Drops frames
• A/V not synced
– Semi-raw capture best
• No longer supported by
anyone
– B/M Progressive scan
• Full speed m-mode, 1D lines
Gick, Wilson, and Derrick (2013)
6
Ultrasound – video capture
• Ultrasound speed
– Video capture
• SD 24/30 fps interlaced
• HD 48/60 fps interlaced
• A/V sync
– Cineloop full speed
• Short duration (8-16 second)
• External A/V sync only
– Frame grabber 60 fps
progressive
• Drops frames
• A/V not synced
– Semi-raw capture best
• No longer supported by
anyone
Miller and Finch (2011)
– B/M Progressive scan
• Full speed m-mode, 1D lines
7
Ultrasound – video capture
• Ultrasound speed
– Video capture
• SD 24/30 fps interlaced
• HD 48/60 fps interlaced
• A/V sync
– Cineloop full speed
• Short duration (8-16 second)
• External A/V sync only
– Frame grabber 60 fps
progressive
• Drops frames
• A/V not synced
– Semi-raw capture best
• No longer supported by
anyone
– B/M Progressive scan
Derrick and Fiasson (In Prep)
• Full speed m-mode, 1D lines
8
Ultrasound – video capture
• Ultrasound speed
– Video capture
• SD 24/30 fps interlaced
• HD 48/60 fps interlaced
• A/V sync
– Cineloop full speed
• Short duration (8-16 second)
• External A/V sync only
– Frame grabber 60 fps
progressive
• Drops frames
• A/V not synced
– Semi-raw capture best
• No longer supported by
anyone
– B/M Progressive scan
• Full speed m-mode, 1D lines
• QuickTime, Final Cut Pro,
Adobe Premier
– Don’t work with all frame
grabbers
– Interfere with frame rate/quality
• Oh g-d, the pain, the PAIN!
• FFMPEG
– 58-60 FPS
• With SSD, 64 bit computer, and
x264
– Requires UNIX command-line
skills
– Post-processing synchronization
based on transient bursts
(‘tatata’)
9
Ultrasound – video capture
• Ultrasound speed
– Video capture
• SD 24/30 fps interlaced
• HD 48/60 fps interlaced
• A/V sync
– Cineloop full speed
• Short duration (8-16
second)
• External A/V sync only
– Frame grabber 60 fps
progressive
• Drops frames
• A/V not synced
http://www.articulateinstruments.com/ultra
sound-products/
– Semi-raw capture (best)
• Only EchoB supports now
– B/M Progressive scan
• Full speed m-mode, 1D lines
10
Ultrasound – video capture
• Ultrasound speed
– Video capture
• SD 24/30 fps interlaced
• HD 48/60 fps interlaced
• A/V sync
– Cineloop full speed
• Short duration (8-16 second)
• External A/V sync only
– Frame grabber 60 fps
progressive
• Drops frames
• A/V not synced
– Semi-raw capture best
• No longer supported by
anyone
Gick, Wilson, and Derrick (2013)
– B/M Progressive scan
• Full speed m-mode, 1D lines
11
Ultrasound – head stabilization
• Hand-held
– Easy
– Useful in field/with
children
• Head rest
– Reduces motion to μ
1mm
– Moves with jaw
• Metal head mounting
– Effective
– Negates jaw motion
• Non-metal mounting
Stone (2005)
– Effective
– Moves with jaw
12
Ultrasound – head stabilization
• Hand-held
– Easy
– Useful in field/with
children
• Head rest
– Reduces motion to μ
1mm
– Moves with jaw
Gick (2002)
• Metal head mounting
– Effective
– Negates jaw motion
• Non-metal mounting
– Effective
– Moves with jaw
Gick, Bird, and Wilson
(2005)
13
Ultrasound – head stabilization
• Hand-held
– Easy
– Useful in field/with
children
• Head rest
– Reduces motion to μ
1mm
– Moves with jaw
• Metal head mounting
– Effective
– Negates jaw motion
• Non-metal mounting
http://www.articulateinstruments.com
/ultrasound-products/
– Effective
– Moves with jaw
14
Ultrasound – head stabilization
• Hand-held
– Easy
– Useful in field/with
children
• Head rest
– Reduces motion to μ
1mm
– Moves with jaw
• Metal head mounting
– Effective
– Negates jaw motion
• Non-metal mounting
– Effective
– Moves with jaw
Derrick and Fiasson (In Prep)
15
Ultrasound - Measurements
• Diagnostic
– Easier, less data storage
– Must be defined
carefully
• Direct measurement
– Slow, tedious
– More rich/useful
Derrick and Gick (2012)
16
Ultrasound - Measurements
• Diagnostic
– Easier, less data storage
– Must be defined
carefully
• Direct measurement
– Slow, tedious
– More rich/useful
Tiede’s “GetContours”
Discussion
• Ultrasound provides tongue shape and dynamic
information
– Can do so at high temporal and spatial resolution
• Head stabilization has tradeoffs
– Free jaw motion invalidates palate measurements
– Restrained jaw motion restricts speech unnaturally
• Ultrasound can be used for diagnostic or directmeasurement analysis
– Diagnostic is fast but uses little of the data
– Direct-measurement is slow but uses more data
18
References
• Derrick, D. and Fiasson, R. (In Prep). Co-collection and
co-registration of speech production ultrasound and
articulometry data.
• Derrick, D. and Gick, B. (2012). Speech rate influences
categorical variation of English flaps and taps during
normal speech. Journal of the Acoustical Society of
America. 131(4):3345.
• Gick, B., Wilson, I. and Derrick, D. (2013). Articulatory
Phonetics. Wiley-Blackwell.
• Gick, B., Bird, S., and Wilson, I. (2005). Techniques for
field application of lingual ultrasound imaging. Clinical
Linguistics and Phonetics. 19(6/7):503-514.
19
References
• Gick, B. (2002). The use of ultrasound for linguistic phonetic
fieldwork. Journal of the International Phonetic
Association. 32(2):113-121.
• Miller, A. and Finch, K. (2011). Corrected High-Frame Rate
Anchored Ultrasound With Software Alignment. Journal of
Speech, Language, and Hearing Research. 54:471-486.
• Stone, M. (2005). A Guide to Analysing Tongue Motion from
Ultrasound Images. Clinical Linguistics and Phonetics.
19(6/7):455-501.
• Tiede. M. (2010). {MVIEW: Multi-channel visualization
application for displaying dynamic sensor movements.
Development
20
References
• Tiede, M. (2005). MVIEW: software for
visulalizing and analysis of concurrently
recorded movement data.
21