PERCEPTION OF MUSIC
BY PATIENTS WITH
COCHLEAR IMPLANTS
Jaan Ross (University of Tartu and Estonian Academy of Music and
Theatre, Tallinn)
Inna V. Koroleva (Institute of Otolaryngology,
St. Petersburg)
Elena A. Ogorodnikova (Pavlov Institute of Physiology, St. Petersburg)
Cochlear implantation (1)
Cochlear implantation is one of the most perspective directions in
the rehabilitation of patients with significant hearing losses
Cochlear implant is a surgically implantable device that provides
hearing sensation to individuals with severe-to-profound
hearing loss who do not benefit from hearing aids
People with hearing losses in such range have absent or
malfunctioning sensory cells in the cochlea
2
Cochlear implantation (2)
In a normal ear, sound energy is converted to mechanical
energy by the middle ear, which is then converted to
mechanical fluid motion in the cochlea. Within the cochlea,
the sensory cells (the inner and outer hair cells) are sensitive
transducers that convert that mechanical fluid motion into
electrical impulses in the auditory nerve.
3
Cochlear implantation (3)
Cochlear implants are designed to substitute for the function of the
middle ear, cochlear mechanical motion, and sensory cells,
transforming sound energy into electrical energy that will initiate
impulses in the auditory nerve
Implant systems consist of both
internal and external components
4
Cochlear implantation (4)
The external components (over or next to the ear)
include (1) a microphone, which converts sound
into an electrical signal, (2) a speech processor,
which manipulates and converts the signal into a
special code and (3) a transmitter, which sends the
coded electrical signal to the internal components
The systems are powered by batteries
located in the speech processor
The surgically implanted components include
(A) a receiver, which decodes the signal from the
speech processor, and (B) an electrode array,
which stimulates the cochlea with electrical
current
5
Rehabilitation (1)
Rehabilitation is the very important post-surgical stage of
cochlear implantation
It is a complex and long process which essentially depends on
individual characteristics of the patient (auditory experience,
general abilities and the absence of mental defects)
It is accepted to distinguish between the two groups of patients
with cochlear implants :
pre-lingual - the hearing loss occurred before the patient has
learned to speak (in general, before the second year of life)
and
post-lingual - when the hearing loss occurred in the age of four
years or later (after the patient has learned to speak)
6
Rehabilitation (2)
The main task of rehabilitation :
 for post-lingual patients - the regeneration of the ability to
perceive and to understand spoken language (the forming of
connections between the distorted auditory information and
previous internal patterns of speech and other sounds)
 for pre-lingual patients - the development of the foundations
of auditory perception from the beginning (includes the basic
elementary operations - detection and discrimination of
sounds) and building up the whole system of spoken language
 So for all listeners with cochlear implants – a primary
rehabilitation problem is the creation or correction of «new»
auditory patterns
7
Rehabilitation and music




cochlear implants have been designed primarily to
enhance speech perception
many implant recipients anticipate enjoying music
following the implantation
in fact, up to 38 per cent of implant recipients report
they do not enjoy listening to music with their
device (Leal et al., Acta Otolaryngology 123, 826
(2003))
up to 86 per cent of post-lingual recipients report
lower scores of listening habits after implantation
8
Hypothesis

pitch has been reported to be recognized with
more difficulty than rhythm or timbre by
implant recipients (because frequency
encoding is robust in an implant), therefore
temporal aspects of music are expected to be
perceived better than pitch
9
Method




about 150 cochlear implant receivers in St.
Petersburg area by 2005 (Combi40 or Combi 40+ by
MED-EL)
they are interviewed in order to study their postsurgical adaptation to listening music
interviews are conducted at the Institute of
Otolaryngology in St. Petersburg as a part of general
rehabilitation program
selected interviews are videotaped
10
Interview






Did you listen to speech, music, or noise?
If this was music, was it singing or playing an
instrument?
If this was singing, was there just one performer or
many of them? Were the singers male or female?
If this was playing an instrument, was there just one
instrument or many? String, wind, or percussion?
Was this music joyful or sad? Fast or slow? Pleasant
or not? Simple or complicated?
Did this music sound in high, medium or low
register?
11
Choice of music for the interview
Four excerpts from world music, with different sound
characteristics:
 Japanese bamboo flute shakuhachi: one voice, long
notes, timing plays little role
 Saami jojk (Mari Boine): female singing, one voice,
heterogeneous rhythm
 Lithuanian sutartine: polyphonic female singing
(facilitates harmonic dissonances), repetitive rhythm
 Tuvan overtone singing (xöömij): a low fundamental
of special timbre, with higher overtones made
audible one-by-one
12
Long-term spectra of (1) Japanese bamboo flute shakuhachi (top left), (2)
Saami jojk (top right), (3) Lithuanian dissonant polyphony (bottom left), and
(4) Tuvan overtone singing (bottom right)
60
60
40
40
20
20
0
1000
2000
3000
Frequency (Hz)
4000
5000
0
60
60
40
40
20
20
0
1000
2000
3000
Frequency (Hz)
4000
5000
1000
0
1000
2000
3000
Frequency (Hz)
4000
2000
3000
Frequency (Hz)
4000
5000
5000
13
Interview results for 10 CI-recipients (age 10 to 55
years, post-surgical experience from 2 weeks to 4 years)
music
singing
female singing
joyful
fast
pleasant
low pitch
medium pitch
high pitch
flute
10
2
1
0
0
8
2
3
2
jojk
10
10
7
0
0
8
3
1
0
sutartine
10
10
7
8
8
8
1
1
6
xöömij
7
6
2
1
0
0
8
4
1
14
Listening preference score for ten CIrecipients
15
List of instruments for the timbre recognition
task (those in yellow recognized more easily)
MUSICAL INSTRUMENT
ESTIMATED FREQUENCY RANGE
1
violin
180-8000
2
cello
70-8000
3
double bass
40-5000
4
harp
30-15000
5
piano
22-6000
6
flute
250-9600
7
organ
22-16000
8
trumpet
160-8000
9
cymbals
500-16000
10
triangle
1200-16000
11
kettledrum
16-350
12
big drum
16-350
16
Recognition of musical timbres before
and after a training session
17
Long-term improvement of timbre recognition by
cochlear implant recipients (without special training)
18
Excerpts from an interview (1)







a post-lingual patient
has used the implant for a couple
of weeks
describes Tuvan overtone
singing as sound of a Jewish
harp
describes shakuhachi sound as
singing
confuses Saami jojk with
Lithuanian polyphony
noticeable progress during the
interview
admits using extramusical
features (sound volume) to solve
the task
19
Excerpts from an interview (2)
- a post-lingual patient
- has used the implant for
4 years
- recognizes Lithuanian
polyphonic and Tuvan
overtone singing
- confuses a shakuhachi
sound with Saami jojk
- claims to prefer the jojk
to other excerpts
20
General conclusion

Many post-lingual patients with cochlear
implants are and pre-lingual patients become
strongly motivated to attend music even when
its perceived acoustical characteristics only
remotely resemble the patterns they expect to
hear. This may be explained by an
ecologically important function music fulfills
in people’s everyday life.
21
Thanks to

Elvira I. Stoljarova of the Pavlov Institute of
Physiology, St. Petersburg, for video
recording of the interviews
22