Modeling Meeting Turns
Dan Ellis <dpwe@ee.columbia.edu>
LabROSA, Columbia University & ICSI
• Meeting turns visualization
• Turn-pattern segmentation
• ‘Talkativity’ modeling
m4 meeting - Dan Ellis
2003-01-29
Meeting Turn Visualization
turns form patterns on multi• Speaker
minute timescales:
Participant
mr04: Hand-marked speaker turns
10:
9:
8:
7:
5:
3:
2:
1:
0
5
10
15
20
25
30
35
40
45
50
• Points of pattern change are ‘significant’?
55
60
time / minutes
topics?
modes?
m4 meeting - Dan Ellis
2003-01-29
Modeling meeting segments
• Model speaker activity patterns like states
20
•
0.3
0.2
0.1
0
spkr(t)
vector:
• Prior
P(spkri)
25
time / min
Prob
15
‘Transition’ matrix:
P(spkrit,spkrjt-1)
0
2
3
4
3
4
5
6
7 spkr
7
6
5
4
3
2
1
0
0
m4 meeting - Dan Ellis
1
1
2
5
6
7
spkr(t+1)
2003-01-29
Self-similarity
• Display Dist(minutei, minutej)
as KL distance of speaker distributions
time/min
mr04: Self-sim of turn mxs by KL
60
8
50
40
6
30
4
20
2
10
20
m4 meeting - Dan Ellis
40
60
time/min
KL dist
2003-01-29
BIC Segmentation
• BIC (Bayesian Information Criterion):
Compare more and less complex models
log
L(X1 ;M1 )L(X2 ;M2 )
L(X;M0 )
• For segmentation:
≷
λ
2
log(N )∆#(M )
Grow context window from current boundary
For each window, test every possible segmentation
When BIC is positive, mark new segment
candidate
boundary
last
segmentation point
current
context limit
0
N
L(X1;M1)
time
L(X2;M2)
L(X;M0)
m4 meeting - Dan Ellis
2003-01-29
BIC Segmentation
Participants
• Example of boundary finding:
8
7
6
5
4
3
2
1
BIC score
15
0
16
17
18
19
20
21
23
16
25
current
context
limit
no boundary found
with shorter context
-200
15
24
boundary
passes BIC
last
seg
point
-100
22
17
18
19
20
21
22
23
24
25
time / min
m4 meeting - Dan Ellis
2003-01-29
BIC Segmentation
• Appears to find shifts in turn patterns:
Participant
mr04: Hand-marked speaker turns vs. time + auto/manual boundaries
10:
9:
8:
7:
5:
3:
2:
1:
0
5
10
15
20
25
30
35
40
45
against topic boundaries
• Evaluate
(6 meetings, 36 boundaries)
50
55
60
time/min
15 (42%) agree to within ± 2 minutes
16 ‘false alarm’ insertions
m4 meeting - Dan Ellis
2003-01-29
“Talkativity”
affecting how much one person
• Factors
speaks in a given meeting:
indexable
confounding
relevance/interest of topic to speaker
competition with other speakers
innate tendency to talk - “talkativity” Ts
of expected ‘airtime’ consumed by
• Model
each participant s in meeting m:
Psm =
! Ts
t∈Sm
Tt
given {Ts}, deviations from expected values
factor out competition, baseline talkativity
m4 meeting - Dan Ellis
2003-01-29
Estimating “Talkativity”
• Find best-fitting {Ts} to fit meeting set
Ts = avgm∈Ms
Psm
!
t∈Sm ,t"=s
Tt
1−Psm
Iteratively recalculate {Ts} until (fast) convergence
26 meetings (mr* set), 10 common participants,
avg 6.9 participants/meeting
• Calculate actual:predicted ratios
m4 meeting - Dan Ellis
2003-01-29
“Talkativity” Results
• Meeting proportions & ratio to prediction
Talkativity index
Proportion of meeting time per participant
10
9
8
7
6
5
4
3
2
1
Participant
0.4
0.3
0.2
0.1
0
0
0.1
0.2
2
4
6
8
10
12
14
16
18
20
22
24
26
Meeting proportions: log2(actual/predicted)
10
9
8
7
6
5
4
3
2
1
1
Participant
0.5
0
-0.5
-1
2
4
Evaluation?
m4 meeting - Dan Ellis
6
8
10
12
14
16
Meeting number
18
20
22
24
26
2003-01-29