advertisement

Topics covered in this chapter – Three basic problems in pattern comparison • How to detect the speech signal in a recording interval (i.e. separate speech from background) • How to locally compare spectra from two speech utterances (local spectral distortion measure), and • How to globally align and normalize the distance between two speech patterns (sequences of spectral vectors) which may or may not represent the same linguistic sequence of sounds (word, phrase, sentence, etc.) 1 Distortion Measures • Mathematical considerations to find out the dissimilarity between two feature vectors. • Let x and y are two vectors defined on a vector space X. • A metric or distance function d on the vector space X as a real valued function on the Cartesian product XX is defined as …… 2 Distortion Measures a) 0 d ( x, y ) for x,y X and d(x, y) 0 iff x y (posit ivedefinit ness propert y) b) d(x, y) d(y,x) for x,y X (symmet ry) c) d(x, y) d ( x, z ) d(y,z) for x,y X (t riangular inequalit ycondit ion) in addit ion the dist ort ionfunct ionis called invariantif d) d(x z, y z) d(x, y) 3 Distortion Measures • If a measure of a distance d, satisfies only the positive definiteness property then it is called as distortion measure if vectors are representation of the speech spectra. • Distance in speech recognition means measure of dissimilarity. • For speech processing, an important consideration in choosing a measure of distance is its subjective meaningfulness • The mathematical measure of distance to be useful in speech processing should consider the lingustic characteristics. 4 Distortion Measures For example a large difference in the waveform error does not always imply large subjective differences. 5 Distortion Measures • Perceptual considerations: the choice of an appropriate measure of spectral dissimilarity is the concept of subjective judgment of sound difference or phonetic relevance. • Spectral changes that keep the sound the same perceptually should be associated with small distances. • And spectral changes that keep the sound the different perceptually should be associated with large distances 6 Distortion Measures • Consider comparing two spectral representations, S(w) and S’(w) using a distance measure d(S,S’) • If the spectral content of two signal are phonetically same (same sound) then the distance measure d is ideally very small 7 Distortion Measures • Spectral changes due to large phonetic distance include – Significant differences in formant locations. i.e the spectral resonance of S(w) and S’(w) occure at very different frequencies. – Significant differences in formant bandwidths. i.e the frequency widths of spectral resonance of S(w) and S’(w) are very different. For each of these cases sounds are different so the spectral distance measure 8 d(S,S’) is ideally very large Distortion Measures To relate a physical measure of difference to subjective perceived measure of difference it is important to understand auditory sensitivity to changes in frequencies, bandwidths of the speech spectrum, signal sensitivity and fundamental frequency. 9 Distortion Measures • This sensitivity is presented in the form of just discriminable change – the change in a physical parameter such that the auditory system can reliably detect the change as measured in standard listening test. 10 Spectral-distortion measures • Measuring the difference between two speech patterns in terms of average spectral distortion is reasonable way both in terms of its mathematically tractability and its computational efficiency • Perceived sound differences can be interpreted in terms of differences of spectral features 11 Log spectral distance • Consider two spectra S(w) and S’(w). The difference between two spectra on a log magnitude versus frequency scale is defined by V( ) logS( ) - logS' ( ) - - - - - - - - - -1 • A distance or distortion measure between S and S’ can be defined by d d (S, S' ) (d p ) V ( ) 2 2 p P 12 d d (S, S' ) (d p ) V ( S , S ' ) 2 2 p P This is related to how humans perceive sound differences 13 Log spectral distance • For P=1 the above equation defines the mean absolute log spectral distortion • For P=2, equation defines the rms log spectral distortion that has application in many speech processing systems • For P tends to infinity, equation reduces to the peak log spectral distrotion 14 Cepstral distances • For the Cepstral coefficients we use the rms log spectral distance. S(t ) h(t )* x(t ) vocalt ractcomponent sand excit ement S( ) H ( ) X ( ) in t hefrequencydomain T akinglog of S( ) log S ( ) log H ( ) log X ( ) T hepower spect rum log | S( ) |2 jn c e n 1 n 16 Cepstral distances T heCepstralcoefficients can be obtainedfrom the LP Ccoefficients 2 d d ceps ( S , S ' ) log S ( ) log S ' ( ) 3 2 2 2 ( c c ' ) n n where cn and cn' are cepstral n coefficients of S( ) and S' ( ) respectively. 17 Weighted cepstral distances and liftering • Liftering makes the system more robust to noise, • Liftering is done to obtain the equal variance • Liftering is significant for the improvement for the recognition performance • If we incorporate n2 factor into the cepstral distance to normalize the contribution from each cepstarl term, the distance d 2 2w n (c 2 n n c ) ' 2 n (nc n n nc ) 2 ' 2 n 20 24 Weighted cepstral distances and liftering • The original sharp spectral peaks are highly sensitive to the LPC analysis condition and the resulting peakiness creates unnecessary sensitivity in spectral comparison • The liftering process tends to reduce the sensitivity without altering the fundamental “formant” structure. • i.e the undesirable (noiselike) components of the LPC spectrum are reduced or removed, while essential characteristics of the “formant” structure are retained 25 Weighted cepstral distances and liftering • A useful form of weighted cepstral distance is L d 2 cw ( w(n)cn w(n)c ) n 1 ' 2 n • Where w(n) is any lifter function. 26 Itakura and Saito • The log spectral difference V(w) is defined by V(w) = log S(w) – log S’(w) is the basis of many distortion measures • The distortion measure proposed by Itakura and Saito in their formulation of linear prediction as an approximate maximum likelihood estimation is 27 Itakura and Saito d IS ( S , S ' ) e dw V ( w) 1 2 2 V ( w) S ( w) dw d IS ( S , S ' ) log 1 2 S ' ( w) 2 ' where and ' are predictionerrorsof S(w) 2 2 and S' (w) respectively. where dw 2 exp log S(w) 2 - 28 Itakura and Saito • The Itakura Satio distortion measure can be used to illustrate the spectral matching properties by replacing S’(w) with the pth order all pole spectrum 2 1 d Is S , 2 2 A e jw where 2 is t hegain S (w) Ae jw 2 S (w) Ae jw 2 dw log 2 log 2 1 2 dw , where is t heresidual energy 2 29 Itakura let us consider then theItakuradistortionmeasureis 2 dI 2 jw A(e ) 1 1 dw , log 2 2 2 jw 2 Ap A Ap (e ) 30 Likelihood Distortions • The role of the gain terms is not explicit in the Itakura distortion because the signal level essentially makes no difference in the human understanding of speech so long as it is unambiguously heard. • Gain independent distortion measure called likelihood ration distortion can be derived directly from IS distortion measure 1 1 1 1 dI , d , LR 2 2 2 2 Ap Ap A A 31 Likelihood Distortions When the distortion is very small the Itakura distortion measure is not very different from the likelihood distortion measure. 32 Variations of likelihood distortions • Compare to the cepstral distance likelihood distortions are asymmetric. • To symmetries the distortion measure there are two methods – COSH distortion – Weighted likelihood distortion 33 COSH distortion • COSH distortion is given by d COSH S ( w) dw coshlog 1 S ' ( w) 2 • The COSH distortion is almost identical to twice the log spectral distance for small distortions 34 Weighted likelihood ratio distortion The purpose of weighting is to take the spectral shape into account as a weighting function such that different spectral components along frequency axis can be emphasized or de-emphasized to reflect some of the observed perceptual effects 35 Weighted likelihood ratio distortion rˆ(n) rˆ' (n) dWLR 2 2 cn c'n ' where c n and c'n are cepstralcoefficient s of log 1 A 2 and log 1 A' 2 and rˆ(n) and rˆ' (n) are autocorrelationsequences for 2 A 2 and '2 A' 2 respectively 36 Comparison of dWLR and d22 2 2 1 In d (log A 2 - log 1 A' 2 ) in d WLR thisis replacedby linear deviation ( 1 A 2 - 1 A' 2 ) which shows heavieremphasis in spectralpeak areas than thecompresseddeviation 1 1 (log 2 - log 2 ) A A' T hispropertyis required in theapplications where extraordinary emphasisof spectralpeaksis necessary, such as speech recognition in noisy environment 37 Weighted slope metric distortion measure Based on a series of experiments designed to measure the subjective “phonetic” distance between pairs of synthetic vowels and fricatives, it is found that by controlled variation of several acoustic parameters and spectral distortions including formant frequency, formant amplitude, spectral tilt, highpass, lowpass, and notch filtering only formant frequency deviation was phonetically relevant 38 Weighted slope metric distortion measure WSM attach a weight on the spectral slope difference near spectral peaks, rather than the spectral amplitude difference, and take the overall energy difference explicitly into consideration K dWSM ( S , S ' ) u E Es Es ' u (i ) s (i ) s ' (i ) 2 i 1 where u E is t he weight ingconst antfor absolut e energy Es Es ' bet ween S and S' , u(i) is t he weight ing coefficient s for a crit icalband spect ralslope difference s (i ) s ' (i ), bet ween S and S' and K is t he t ot alno.of crit icalbands considerd S 39 Summary • The spectral distortion measures are designed to measure dissimilarity or distance between two (power) spectra of speech • Many of these dissimilarity measures are not metrics because they do not satisfy the symmetry property • If an objective speech distortion measure needs to reflect the subjective reality of human perception of sound differences, or even phonetic disparity, the asymmetry seems to be actual desirable. 40 S Summary • All distortion measures are equally important because certain distortion measures may be better for an less noisy environment, while others may be robust when the background is more noisy. 41 Summary • Log spectral: Lp metric requires large amount of calculations because we need 2 FFT’s to obtain S(w) and S’(w), logarithms of all values of S and S’ and an integral P d d p log( S ( w) log S ' ( w) 2 1/ p 42 Summary • Truncated and weighted cepstral: Requires only L operations where L is of the order of 12-16 hence calculations required are less compared to Lp metric L d ( L ) ( cn cn ) 2 c n 1 L d 2 CW ' 2 W ( n ) ( cn cn ) ' 2 n 1 43 Summary • The likelihood, Itakura-Saito, Itakura and COSH measurements: all requires on the order of p is the LPC order of all pole polynomial (8-12). Hence the computations are same for cepstral measures 44 Summary S ( w) dw d IS ( S , S ' ) log 1 2 S ' ( w) 2 ' 2 A2 dw d I log 2 Ap 2 d LR A d COSH A 2 2 p dw 1 2 S ( w) dw coshlog 1 S ' ( w) 2 45 Summary • Weighted likelihood ratio distortion: Requires L operations, similar to that of the cepstral measures rˆ(n) rˆ' (n) ' 2 2 2 (cn cn ) ' n 1 L dWLR 46 Summary • Weighted Slope metric (WSM): Requires K operations, where K is the number of frequency bands used in computations (3264) K dWSM ( S , S ' ) u E Es Es ' u (i) s (i) s ' (i) 2 i 1 47 Summary • From all these points we can say that all the measures are both physically reasonable and computationally tractable for speech recognition except for the Lp metrics. • Hence, practically we are going to use all the measures to study the speech recognition system 48