Introduction & Review Sk. Mohammadul Haque DST INSPIRE Faculty Department of Electrical Engineering Indian Institute of Technology Kharagpur Digital Signal Processing 2023a Sk. Mohammadul Haque Introduction & Review Digital Signal Processing (Part One) Reference materials: Text book: Digital Signal Processing: John G. Proakis and Dimitris G. Manolakis Reference books: Discrete-Time Signal Processing – Alan V. Oppenheim and Ronald W. Schafe Other articles: As will be discussed Sk. Mohammadul Haque Introduction & Review Signals Signals are mathematical functions of a number of independent variables that are usually called indices. 100πn e.g. f (t) = 100 sin(100πt), f [n] = 100 sin( ), T m n e.g. f (x , y ) = 2x + y , f [m.n] = 2 + T1 T2 voltage signals, current signals, position, velocity, momentum, volume, flow rate, etc. multi-dimensional (multi-indexed) - 2D photographic images, hyperspectral images, etc. financial, geographical, network (social networks), etc. scalar signals, vector signals, etc. Sk. Mohammadul Haque Introduction & Review Discrete-time Signals Classification of Signals Dimensionality: One-dimensional: one independent variable. (e.g. f (t) defines ambient temperature) Multi-dimensional: two or more independent variables. (e.g. f (x , y ) defines brightness of sky (How?)) Type of the domain/indices: Continuous: Independent variables are continuous and can be mapped to an interval in the real line (R). Discrete: Independent variables are discrete and can be mapped to a subset of the integers (Z). Range: Analog: Continuous-valued (real/complex) signals Digital: Discrete-value signals. Sk. Mohammadul Haque Introduction & Review Signals Sk. Mohammadul Haque Introduction & Review Signals-Energy Consider a discrete-time signal x [n] defined over discrete time index n. Energy of the signal, Ex (n1 , n2 ) between times n1 and n2 is defined as Ex (n1 , n2 ) = n2 X |x [n]|2 . n=n1 Sk. Mohammadul Haque Introduction & Review Signals- Average Power Consider a discrete-time signal x [n] defined over discrete time index n. Average Power of the signal, Px (n1 , n2 ) between times n1 and n2 is defined as Px (n1 , n2 ) = Sk. Mohammadul Haque Ex (n1 , n2 ) . (n2 − n1 + 1) Introduction & Review Signals- Average Power Energy signals - Signals having finite energy. Power signals - Signals having finite average power. Other than above - Signals that are neither energy signals nor power signals. Sk. Mohammadul Haque Introduction & Review Signals Symmetry Even signals: x [−n] = x [n]. Odd signals: x [−n] = −x [n]. A simple signal analysis Given a signal y [n], we can decompose it as a sum of an even signal e[n] and an odd signal o[n] y [n] = e[n] + o[n] 1 1 e[n] = (y [n] + y [−n]) and o[n] = (y [n] − y [−n]). 2 2 can do much more interpretation. Sk. Mohammadul Haque Introduction & Review Signals Conjugate symmetry Conjugate Symmetry: x [−n] = x ∗ [n]. Conjugate Anti-symmetry: x [−n] = −x ∗ [n]. Examples? e.g. for complex-valued signals? Sk. Mohammadul Haque Introduction & Review Signals Symmetry & Conjugate symmetry Are they closed under addition operation, scalar multiplications? Examples? Direct-sum of odd signals and even signals? Let’s discuss very briefly. Sk. Mohammadul Haque Introduction & Review Signals Some useful signals exponential signals sinusoidal signals step signals impulse signals rectangle signals ramp signals Sk. Mohammadul Haque Introduction & Review Bounded Signals Bounded signals are of the form: Discrete-time: x [n], if an M such that |x [n]| ≤ M for all n. e.g. x [n] = sin 10n − 5. How useful? Sk. Mohammadul Haque Introduction & Review Periodic Signals Periodic signals are of the form: Discrete-time: x [n], if there exists a positive integer N such that x [n] = x [n + N] . x [n] has a period N . e.g. x [n] = n − 11 · floor n . 11 What about x [n] = 2 sin (10n + 5)? Sk. Mohammadul Haque Introduction & Review Periodic Signals Fundamental period - Smallest period of a periodic signal. If the period of a periodic signal is T , then kT for all positive integer k is also a period. Signals that not periodic are called aperiodic signals. Sk. Mohammadul Haque Introduction & Review Periodic Signals Are these periodic? Sk. Mohammadul Haque Introduction & Review Exponential Signals Exponential signals are of the form: Continuous-time : x (t) = Ae bt = Ac t (very general) If both A and b are real, then we have continuous-time real exponential. Discrete-time: x [n] = Ae bn = Ac n (very general) If both A and b are real, then we have discrete-time real exponential. How useful? Bounded? Periodic? Sk. Mohammadul Haque Introduction & Review Sinusoidal Signals Real sinusoidal signals are of the form : Discrete-time: x [n] = A cos (Ωn + δ) and A, Ω, δ are reals. x1 [n] = 10 cos (5n + 4). How useful? Bounded? Periodic? Sk. Mohammadul Haque Introduction & Review Sinusoidal Signals Real sinusoidal signals: Sk. Mohammadul Haque Introduction & Review Sinusoidal Signals Complex sinusoidal signals are of the form : Discrete-time: x [n] = Ae bn = Ac n and b = jΩ is imaginary for some real Ω. jΩn jΩn jδ jδ jΩn x [n] = Ae = |A| e e = |A| e · e |A| (cos (Ωn + δ) + j sin (Ωn + δ)). How useful? Have we come across the above? Bounded? Periodic? Sk. Mohammadul Haque Introduction & Review = Unit Step Signals Discrete-time unit step signal: u[n] = 1, n≥0 0, otherwise Sk. Mohammadul Haque Introduction & Review Unit Step Signals . Sk. Mohammadul Haque Introduction & Review Unit Impulse Signals Discrete-time unit impulse signal: δ[n] = 1, n=0 0, otherwise Sk. Mohammadul Haque . Introduction & Review Unit Impulse Signals . Sk. Mohammadul Haque Introduction & Review Transformations of Signals Time-shifting Given a discrete-time signal x [n], we have y [n] = x [n − b] for a given integer b. When will the signals shift towards left? Or, towards right? Delayed or advanced? Sk. Mohammadul Haque Introduction & Review Transformations of Signals Time-shifting Sk. Mohammadul Haque Introduction & Review Transformations of Signals Time-reversal Given a discrete-time signal x [n], we have y [n] = x [−n]. Sk. Mohammadul Haque Introduction & Review Transformations of Signals Time-reversal Sk. Mohammadul Haque Introduction & Review Transformations of Signals Time-scaling Given a discrete-time signal x [n], does this make sense: y [n] = x [bn] for a given real positive b (?). We will see later downsampling and upsampling in discrete-time domain. Sk. Mohammadul Haque Introduction & Review Unit Impulse Signals - Properties Discrete-time unit impulse signal: δ[n] = 1, n=0 0, otherwise . Similar to continuous-time domain, we have ∞ X n=−∞ ∞ X f [n]δ[n] = f [0] f [n]δ[n − n0 ] = f [n0 ] for all integer n0 (sifting/sampling n=−∞ property) f [n]δ[n − n0 ] = f [n0 ]δ[n − n0 ] (equivalence property). Sk. Mohammadul Haque Introduction & Review Discrete-time Systems Discrete-time Systems are mathematical entities (functions/process) of a number of input signals and a number of output signals. We write, in general, y [n] = H2 (x [n]). behave as what are called ’functionals’ i.e. map functions (signals) to functions (signals). Simple examples: y [n] = 10x [n] Here x [n] is the input signals and y [n] := B(x [n]) are the output signals. Sk. Mohammadul Haque Introduction & Review Systems Classification of Systems Continuous-time systems: Where both input and output signals are continuous-time signals. Discrete-time systems: Where both input and output signals are discrete-time signals. Mixed discrete-continuous systems. Other systems with non-time variable signals are generalised versions. Sk. Mohammadul Haque Introduction & Review Systems Discrete-time Systems - examples Digital communication systems signals - message systems - transmitter, channel, receiver Digital control systems signals - reference inputs, sensor outputs, position, velocity, temperature systems - controller, plant Digital signal conditioning systems signals - audio, video systems - filters Sk. Mohammadul Haque Introduction & Review Systems - Shift Invariance Shift (Time) Invariance: Shift-Invariant, if for every input causing some output, every shifted version by an amount a of the input produces the same output but shifted by the amount a. Discrete-time systems: i.e. for system H1 , any input x [n] causes the output y [n], we have for the shifted input x [n − a] causing the output y [n − a]. y[n] = H1 (x[n])⇒y[n − a] = H1 (x[n − a]) for all x , n, a. Otherwise, we call the system shift-variant (time-variant). Sk. Mohammadul Haque Introduction & Review Systems - Linearity Linear Systems: A system H is linear if for all input signals x 1 and x 2 and scalar α, the following hold true: H(x 1 + x 2 ) = H(x 1 ) + H(x 2 ). (Additivity property) H(αx 1 ) = αH(x 1 ). (Homogeneity property) Linearity is same as super-position property. Example: y [n] = 2x [n]. Sk. Mohammadul Haque Introduction & Review Systems - Response To learn about a given system, we may want to investigate how it behaves for a chosen set of input signals. Which chosen set? Let’s try with unit impulse signal - as it is extremely localised? Sk. Mohammadul Haque Introduction & Review Systems - Response Recall the sifting property: ∞ X f [n] = f [m]δ[m − n]. m=−∞ [Obvious] If we have the samples f [m], we can synthesise the original signal f [n], by taking linear combinations of the shifted impulses. But will the shifted impulses make the system behave similarly individually? What about Shift-Invariance? Sk. Mohammadul Haque Introduction & Review LSI (LTI) Systems Recall the sifting property: f [n] = ∞ X f [m]δ[n − m]. m=−∞ Input signal = Superposition of scaled impulses. Linear Shift (Time)-Invariant Systems: Systems which obey the linearity property and the shift (time)-invariance property. Sk. Mohammadul Haque Introduction & Review LSI (LTI) Systems - Response Discrete-time domain: For any input x [n] and system H, we have the output y [n] as y [n] = = H(x [n]) ∞ X H( x [m]δ[n − m]) m=−∞ = = = ∞ X m=−∞ ∞ X m=−∞ ∞ X H(x [m]δ[n − m]), [additivity] x [m] · [H(δ[n − m])], [homogeneity] x [m] · [hm [n]] m=−∞ ··· = ····················· ∞ X x [m] · h[n − m], [shift-invariance]. m=−∞ Sk. Mohammadul Haque Introduction & Review Convolution Discrete-time domain: For any input x [n] and system H, we have the output y [n] as y [n] = ∞ X x [m] · h[n − m]. m=−∞ If x [n] = δ[n], we have y [n] = h[n]. Hence, the sequence h[n] is called its unit impulse response. Sk. Mohammadul Haque Introduction & Review Convolution Convolution Sum: y [n] = ∞ X x [m] · h[n − m] =: x [n] ∗ h[n] =: (x ∗ h)[n]. m=−∞ Sk. Mohammadul Haque Introduction & Review Convolution - Properties Associative: (x [n] ∗ h1 [n]) ∗ h2 [n] = x [n] ∗ (h1 [n] ∗ h2 [n]) . Sk. Mohammadul Haque Introduction & Review Convolution - Properties Commutative: h1 [n] ∗ h2 [n] = h2 [n] ∗ h1 [n]. Sk. Mohammadul Haque Introduction & Review Convolution - Properties Distributive: x [n] ∗ (h1 [n] + h2 [n]) = x [n] ∗ h1 [n] + x [n] ∗ h2 [n]. Sk. Mohammadul Haque Introduction & Review LSI (LTI) Systems - Causality Convolution: y [n] = ∞ X x [m] · h[n − m] =: x [n] ∗ h[n]. m=−∞ Causality: For a system with impulse response h, we have h[n] Sk. Mohammadul Haque = 0, when n < 0. Introduction & Review LSI (LTI) Systems - Stability I For bounded input, output is also bounded. Discrete-time domain: (Input bound is B x ) y [n] = |y [n]| = ≤ ∞ X x [m] · h[n − m]du =: x [n] ∗ h[n]. m=−∞ ∞ X x [m] · h[n − m] m=−∞ ∞ X |x [m] · h[n − m]| = m=−∞ ≤ Bx · ∞ X ∞ X |x [m]| · |h[n − m]| m=−∞ ∞ X |h[n − m]| = Bx · m=−∞ |h[m]| . m=−∞ So, if impulse P∞response h[m] is absolutely summable (∥h∥1 := m=−∞ |h[m])| < ∞), we have BIBO stability. Sk. Mohammadul Haque Introduction & Review LSI (LTI) Systems - Stability II For bounded input, output is also bounded. h(u) is absolutely integrable and h[m] is absolutely summable is sufficient to show BIBO stability. Are they necessary? e.g. If we assume ∞ X |h[m]| = ∞ is not finite, can we come m=−∞ up with a bounded input x [n] such that (x [m] · h[m − n]) = |h[m − n]| for all m, but, at least one value of n? Sk. Mohammadul Haque Introduction & Review LSI (LTI) Systems - Memory Convolution: y [n] = ∞ X x [m] · h[n − m] =: x [n] ∗ h[n]. m=−∞ Memoryless: For an LTI system with impulse response h, we have a) h[n] = 0, when n ̸= 0. b) h[0] = K′ and for some constants K and K ′ . Sk. Mohammadul Haque Introduction & Review LSI (LTI) Systems - Invertibility Forward System H1 Represented by convolution. How to find invertibility? How about H1 ∗ H2 = Id ? h1 [n] ∗ h2 [n] = δ[n] Sk. Mohammadul Haque Introduction & Review Linear Constant Co-efficient Difference Equations In general, we have relationship: N X M X ak y [n − k] = k=0 bk x [n − k] k=0 Initial conditions: y [n0 ], y [n0 − 1], · · · , y [n0 − N + 1]. Order is N. Solution has two parts: One due to homogeneous N X ak y [n − k] = 0. (Natural k=0 response) Other explicitly due to input x [n]. (Forced response / Particular solution) Conditions for LTI? Causal? Sk. Mohammadul Haque Introduction & Review Motivation Desirable properties for synthesis and analysis: Set of basic signals (done earlier in a way!) Not just input signals, but also output signals to be simple. Recall LSI (LTI) system process: y [n] = x [n] ∗ h[n] ∞ X x [m] · h[n − m]. m=−∞ Sk. Mohammadul Haque Introduction & Review Motivation Discrete-time: y [n] = x [n] ∗ h[n] = ∞ X x [m] · h[n − m]. m=−∞ Set input x [n] = z n . We have output as y [n] ∞ X = = = m=−∞ ∞ X z m · h[n − m] z n−m · h[m] m=−∞ ∞ X n z −m · h[m] z m=−∞ {z | = H(z) } H(z)z n . Sk. Mohammadul Haque Introduction & Review Motivation Discrete-time: We have output as y [z] = H(z)z n . Output signal is scalar multiple of input signal. Scalar H(z) is a function of the parameter z. Signals z n are the eigensignals (eigenfunctions/eigensequences) for the LSI (LTI) system. The scalars H(z) are called the eigenvalues corresponding to the eigensignals z n . Sk. Mohammadul Haque Introduction & Review Induced Gains in DT Systems Now, we can represent the DT LSI process as y = Sx. (However, S linear system matrix can be, in general, any square N × N matrix. Further even, if we still generalise to non-linear systems, we can simply write y = S (x) where S : CN → CN any general function.) We define the “vector p-norm” ∥v∥p , 1 ≤ p ≤ ∞ for any N-length complex vector v as follows: 1 ! N−1 X p , for 1 ≤ p < ∞ |vi |p i=0 ∥v∥p := 1 ! N−1 X p p lim |vi | , p→∞ for p = ∞. i=0 Sk. Mohammadul Haque Introduction & Review Induced Gains in DT Systems We define the “induced matrix (system) p-norm” |∥S∥|p , 1 ≤ p ≤ ∞ for any LSI matrix (system) S as follows: |∥S∥|p := sup x,∥x∥p =1 ∥S (x)∥p = sup ∥S (x)∥p x̸=0 ∥x∥p . The above definition finds the induced gain of the system S. That is, for all input signals whose p-norms are bounded, we seek what is the maximum (or supremum) p-norm among the corresponding outputs. If |∥S∥|p < ∞, we say the system is induced p-norm (gain) stable. In particular, if the product |∥S∥|p · S−1 p is near unity, we say that the system S is well-conditioned, otherwise, we say that the system S is ill-conditioned. Sk. Mohammadul Haque Introduction & Review Discrete-time - Motivation Discrete-time Fourier Series: Forward Transform: ck = N−1 1 X N Inverse Transform: h[n] = 2π −jk h[n]e n N . n=0 N−1 X 2π jk ck e n N . k=0 We may want to consider aperiodic signals where N → ∞ or with time-shifting N1 , N2 → ∞ if the period runs from −N1 to N2 and N1 + N2 + 1 = N. As also, we want to ensure that 2πk = ωk remains finite, so also, sweeps continuously N over all real frequencies ω ∈ R. Then, we have δωk ωk+1 − ωk (k + 1) − (k) 1 = = = . 2π 2π N N Sk. Mohammadul Haque Introduction & Review Discrete-time - Motivation Putting ωk and δωk , we get 2π ck N2 δωk X = 2π N−1 X h[n] = h[n]e −j ωk n n=−N1 ck e j ωk n . k=0 We now only need a rescaling in definition of ck to cˇk as below: cˇk = N2 X h[n]e −j ωk n n=−N1 h[n] = Sk. Mohammadul Haque X δωk N−1 2π cˇk e j ωk n . k=0 Introduction & Review Discrete-time Fourier Transform We define ω, dω, H(e jω ) := lim (ωk , δωk , cˇk ). Then, in N→∞ the limit as N, N1 , N2 → ∞, we have jω H(e ) = h[n] = ∞ X h[n]e −j ωn n=−∞ ˆ 1 2π 2π H(e j ω )e j ωn · d ω. 0 The above H(e j ω ) is called the Fourier Transform of h[n]. H(e j ω ) is also denoted as Fn (h[n]) where the transform is perform along the n dimension. If clear from context, we simply write F(h[n]). Similarly, for the inverse transform jω −1 (H(e j ω )). F−1 ω (H(e )) as F Sk. Mohammadul Haque Introduction & Review Discrete-time Fourier Transform H(e j ω ) = ∞ X h[n]e −j ωn is called the forward Fourier n=−∞ transform (analysis) equation. ˆ 1 2π H(e j ω )e j ωn · d ω is called the inverse Fourier h[n] = 2π 0 transform (synthesis) equation. DT F T We denote the above as h[n] ←−−→ H(e j ω ). Periodicity of DTFT Note that H(e j (ω+2π) )= ∞ X h[n]e n=−∞ −j (ω+2π)n ∞ X = 1 h[n]e −j ωn n=−∞ or H(e j (ω+2π) ) = H(e j ω ). Hence, DTFT H(e j ω ) is 2π-periodic. (Differs from CTFT). Sk. Mohammadul Haque Introduction & Review z }| { e −j 2πn Convergence - DTFT Convergence to equality Finite energy signals (Sufficient condition- MSE sense) A signal h[n] with finite energy E = X∞ n=−∞ 2 |h[n]| < ∞. or Absolutely summable signals (Sufficient conditionuniform convergence) A signal h[n] with absolute sum S = Sk. Mohammadul Haque X∞ n=−∞ Introduction & Review |h[n]| < ∞. Discrete-time Fourier Transform DTFT – Properties: DT F T DT F T Linearity: h1 [n] ←−−→ H1 (e jω ) and h2 [n] ←−−→ H2 (e jω ) DT F T implies αh1 [n] + βh2 [n] ←−−→ αH1 (e jω ) + βH2 (e jω ). DT F T Shifting: h[n] ←−−→ H(e jω ) implies DT F T h[n − n0 ] ←−−→ e −jωn0 H(e jω ) and DT F T e jω0 n h[n] ←−−→ H(e j(ω−ω0 ) ). DT F T Time-reversal: h[n] ←−−→ H(e jω ) implies DT F T h[−n] ←−−→ H(e −jω ). DT F T Time-scaling: h[n] ←−−→ H(e jω ) implies DT F T h[an] ←−−→ H(e jaω ) (zero-filled scaling). Sk. Mohammadul Haque Introduction & Review Discrete-time Fourier Transform DTFT – Properties: DT F T Multiplication: h1 [n] ←−−→ H1 (e jω ) and DT F T h2 [n] ←−−→ H2 (e jω ) implies 1 DT F T h1 [n]h2 [n] ←−−→ H1 (e jω ) ∗ H2 (e jω ). 2π DT F T Convolution: h1 [n] ←−−→ H1 (e jω ) and DT F T h2 [n] ←−−→ H2 (e jω ) implies DT F T h1 [n] ∗ h2 [n] ←−−→ H1 (e jω )H2 (e jω ). DT F T Complex-conjugation: h[n] ←−−→ H(e jω ) implies DT F T h∗ [n] ←−−→ H ∗ (e −jω ). DT F T Real signal: h[n] ←−−→ H(e jω ) implies H(e jω ) = H ∗ (e −jω ). DT F T Real and even signal: h[n] ←−−→ H(e jω ) implies H(e jω ) real and even. DT F T Real and odd signal: h[n] ←−−→ H(e jω ) implies H(e jω ) imaginary and odd. Sk. Mohammadul Haque Introduction & Review Discrete-time Fourier Transform DTFT – Properties: Parseval’s Relation: ˆ X∞ 1 2 |h[n]| = n=−∞ 2π 2π 2 H(e jω ) · dω. 0 DT F T Difference: h[n] ←−−→ H(e jω ) implies DT F T h[n] − h[n − 1] ←−−→ (1 − e −jω )H(e jω ). DT F T Summation: h[n] ←−−→ H(e jω ) implies Pn DT F T m=−∞ h[m] ←−−→ P∞ 1 H(e jω ) + πH(0) k=−∞ δ(ω − 2πk). (1 − e −jω ) Sk. Mohammadul Haque Introduction & Review Discrete-time Fourier Transform DTFT – Periodic Signals: Consider a periodic signal h[n] with period N. Recall the DT Fourier Series of h[n] = N−1 X ck e jnω1 k . k=0 Find Fourier transform of each component ck e jnω1 k as P DT F T ck e jnω1 k ←−−→ 2πck ∞ m=−∞ δ(ω − kω1 − 2πm). Hence, we have N−1 X DT F T ck e jnω1 k ←−−→ PN−1 k=0 2πck P∞ m=−∞ δ(ω k=0 Sk. Mohammadul Haque Introduction & Review − kω1 − 2πm) . Discrete-time Fourier Transform DTFT – Periodic Signals: Consider a periodic signal h[n] with period N. Recall the DT Fourier Series of h[n] = N−1 X ck e jnω1 k . k=0 Find Fourier transform of each component ck e jnω1 k as P DT F T ck e jnω1 k ←−−→ 2πck ∞ m=−∞ δ(ω − kω1 − 2πm). Hence, we have N−1 X DT F T ck e jnω1 k ←−−→ PN−1 k=0 2πck P∞ m=−∞ δ(ω k=0 Sk. Mohammadul Haque Introduction & Review − kω1 − 2πm) .