19ECE01 - MULTIMEDIA COMPRESSION AND COMMUNICATION Dr. I.S.AKILA Associate Professor Department of ECE Coimbatore Institute of Technology Syllabus COURSE OUTCOMES Upon completion of this course the students will be able to demonstrate an ability to CO1 : Illustrate various multimedia components and its characteristics. CO2 : Determine the average length of text and image by various compression techniques, and calculate the compression ratio. CO3 : Interpret the various techniques for audio and video compression. CO4 : Illustrate the network architecture and various communication protocols for VoIP. Identify the challenges and key factors to improve the performance and QoS parameters of VoIP. CO5 : Analyze the various multimedia networking protocols to provide the best effort service for Real time interactive multimedia application. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 2 Syllabus MULTIMEDIA COMPONENTS • Introduction - Multimedia skills - Multimedia components and their characteristics text, sound, images, graphics, animation, video, hardware. (9) TEXT AND IMAGE COMPRESSION • Compression principles-Source encoders and destination encoders-Lossless and lossy compression-Entropy encoding –Source encoding- Text compression -Static Huffman coding , Dynamic Huffman coding - Arithmetic coding - Lempel Ziv-Welsh Compression- Image compression - Graphics interchange formate - Tagged image file format – JPEG (9) AUDIO AND VIDEO COMPRESSION • Audio compression-DPCM-Adaptive DPCM -Adaptive Predictive Coding-Linear Predictive Coding-Code Excited LPC- Perceptual coding - Video compression principles-H.261, H.263, MPEG 1, 2, 4. (9) Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 3 Syllabus VoIP TECHNOLOGY • Basics of IP transport, VoIP challenges, H.323/ SIP -Network Architecture, Protocols, Call establishment and release, VoIP and SS7, Quality of Service CODEC Methods-VOIP applicability. (9) MULTIMEDIA NETWORKING • Multimedia networking- Applications- Streamed stored and audio-making - Best Effort service-Protocols for real time interactive applications- Distributing multimedia- Beyond best effort service- Secluding and policing MechanismsIntegrated services, Differentiated Services-RSVP. (9) Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 4 Syllabus TEXT BOOKS 1. Fred Halshall,"Multimedia communication-Applications, Networks, Protocols and Standards", Pearson Education, 2013. 2. Tay Vaughan, "Multideai: Making It Work", 7/e, TMH, 2007. REFERENCES 1. Kurose and W. Ross, "Computer networking, A Top down Approach", Pearson education, 3rd Edition, 2010. 2. Marcus Goncalves, "Voice over IP Networks", McGraw Hill, 2nd Edition,1999. 3. KR. Rao,Z S Bojkovic, D A Milovanovic, "Multimedia Communication Systems : Techniques, Standards and Networks", Pearson Education,2007. 4. R.Steimnetz, K. Nahrstedt, "Multimedia Computing, Communications and Applications", Pearson Education,1st Edition, 1995. 5. Ranjan Parekh, "Principles of Multimedia", 3rd Edition, 2010. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 5 MULTIMEDIA • Multi: more than one • Media: means for conveying information • Mass media • press, newspaper, radio and TV context • Transmission Media • Guided or Unguided • Storage media • floppy, CD, DVD, HD, USB • Interaction media • text, image, audio, video …. • Multimedia is a presentation of a computer application incorporating media elements such as text, graphics, animations, audio, and video. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 6 Definition Computer-based techniques of text, images, audio, video, graphics, animation, and any other medium where every type of information can be represented, processed, stored, transmitted, produced and presented digitally Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 7 Benefits of Multimedia Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 8 MEDIA COMPONENTS • Text - ASCII/Unicode, HTML, Postscript, PDF • Audio – Sound, music, speech, structured audio (e.g. MIDI) • Musical Instrument Digital Interface • Still Image - Facsimile, photo, scanned image • Video – Movie, a sequence of pictures • Graphics – Computer produced image • Animation – A sequence of graphics images Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 9 Captured Versus Synthesized Media • Captured media refers to information types captured from the real world. e.g. Still pictures, moving pictures, and sound • Synthesized media refers to information types synthesized by the computers e.g. Text, graphics and computer animation Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 10 Discrete Versus Continuous Media • Discrete media refers to media involving space dimension only - length, breadth, height, and time • e.g. Still images, text and graphics • Discrete media is also referred to as static media or non-time-based media or non-temporal media or space-based media. • Continuous media refers to time-based media • e.g. Sound, moving images, and animation • Continuous media is also referred to as dynamic media or time-based media or temporal media. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 11 CLASSIFICATION OF MEDIA TYPES Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 12 Characteristics of Multimedia System • Computer-controlled User is able to view, hear, and see using a Multimedia PC System. • Integrated At least one discrete and one continuous media combined for information presentation design sharing • Digital representation Consists of various form of media i.e. text, graphics, audio, video, and animations; created, stored, processed, and transmitted DIGITALLY. • End User Interface – permit interactivity User is able to navigate, interact, create, and Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT communicate. 13 BASIC TOOLS • World Wide Web (browser-based playback) - HTML 5, CSS, Flash, Dreamweaver • Programming Applications (native playback) - Director, LiveCode • Artwork Development - Photoshop, Illustrator • Animation Development (2D and 3D) - After Effects, Maya • Audio Editing - Sound Forge, Audition • Video Editing - Final Cut Pro, Premiere Pro Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 14 General Overview of a Multimedia System Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 15 Applications Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 16 Elements • text • images, • audio • video and • animation Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 17 Text • Text is most commonly used to communicate information • alphanumeric characters & special characters • Use of text types, sizes, fonts, colours and background colours • Multimedia applications support linked content, through Hypertext • Examples:Text in SMS, FAX, Email • TXT, DOC, DOCX, PDF Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 18 Images • Images/Illustrations are the oldest form of media • illustrate ideas through still pictures • There are two types of images: - Bitmaps - Vector • Bitmap images are real images captured from cameras • Vector graphics are created using software in the computer • This multimedia element enables to generate, represent, process, manipulate, and display pictures • File Format: JPG, PNG, TIF, BMP Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 19 Audio • speech, music and sound effects - digital audio • sound elements: website or presentation can add audio files from a musical background voiceover / spoken explanation • There are two basic type of audio or sound; analog and digital. • File Format: MP3, WAV, WMA Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 20 Video • Video presents moving pictures • Combination of images and sound • Video records, synthesizes, and displays images known as frames in sequences moves at a fixed speed • Watch a video without any interruption, video device must display 25 to 30 frames/second • File Format: AVI, WMV, FLV, MOV, MP4 Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 21 Animation • Animation is the process of making a static image to look as if it is moving • It helps in creating, developing, sequencing, and displaying a set of images technically known as frames • Digital animation can be classified as 2D and 3D • Graphic Image Files (GIFs), are small files that present a single image or rapidly display a sequence of a few images to give the appearance of motion. • Adobe Flash is the most common tool for creating these animations. • File Format: GIF, FL Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 22 • Codeword- unique combination of fixed number of bits • Analog signals • Transmission –BW limited channel Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 23 Encoder Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 24 Contd.. • Bandlimiting filter • Sampling Rate • Antialiasing filter • Quantization • Dynamic Range Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 25 Decoder Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 26 Text • Unformatted Text • Formatted Text • Hypertext Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 27 Unformatted Text • Plain Text- ASCII character set • 7 bit binary codeword 2^7 128 characters • Printable characters alphabet, numeric & punctuation • Control characters Format control characters DEL, ESC, BS(Backspace) etc. • Information separator File separator Record separator • Transmission Control Characters ACK, SYN, NAK etc • Mosaic Characters teletext and videotex Create graphic images – television broadcasting Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 28 Contd.. ASCII Characters Mosaic characters Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 29 Videotex/ Teletext Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 30 Formatted Text • Produced - word processing packages • Publishing sectors Papers, books, magazines, journals etc • Command reserved format-control character – alphabets or numeric Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 31 Hypertext • Enables an integrated set of documents which have defined linkages between them • The linked set of pages stored in a particular server are accessed using a client program known as BROWSER Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 32 Contd.. • Associated with each set of linked pages is the HOME PAGE • An index to the linked pages • Hyperlink takes the underlined text string form and the user initiates by clicking the mouse on it. • In addition to textual name of the link, a unique network-wide name known as URL is used. • Includes the host computer where the page is located, name of the file… • Hypertext language: HTML, SGML, TEX, LATEX… Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 33 Images • Includes Computer-generated images called graphics • Displayed as 2-D matrix of individual picture elements called pixels • Graphics Software tools Creation/ editing- object Composed of visual objects including lines, arcs, squares, rectangles… Clipart 3D effect Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 34 Images • A computer’s display screen is considered as 2D matrix of Picture elements • Ex: Video Graphics Array(VGA) display with 640X480 pixels with 8 bits per pixel to have one of 256 different colors All objects-including freeform objects are made up of series of short and long lines. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 35 Graphics Principles 36 • Objects – attributes Size, shape, color, position, shadow etc • Object Open Closed • Same color- rendering Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 37 Computer Graphics • Representation • High-level Version Less BW, less memory Destinations must be able to interpret the various high-level commands • Low-level machine code Eg: bit – map format Widely & standardized form GIF – Graphical Interchange Format TIFF – Tagged Image File Format • SRGP – Simple Raster Graphics Package High level into pixel forms Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 38 Digitized Documents • Eg. Documents produced by the scanner associated with a FACSIMILE (fax). • Scans each page from L to R-starts at the top of the page and end at the bottom. • They use a single binary digit to represent each pel, 0-White 1-Black • A page produces stream of about 2million bits • Best suited for bi-tonal images comprising mainly textual information. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 39 DIGITIZED PICTURES • A good quality black and white picture needs 8 bits/pel to yield 256 gray levels. COLOUR PRINCIPLES • Human eye sees just a single color when a set three primary colors are mixed • Whole spectrum of colors known as COLOUR GAMUT can be produced by using different proportions of the three primary colors (R,G,B) MIXING TECHNIQUE • Additive Color mixing: Produces black when all three primary colors are zero. • • • Useful in producing color image on a blank surface. Eg. Display applications Subtractive color mixing: Produces white when all three primary colors (C,M,Y) are zero. Useful in producing color image on a white surface. Eg. printer applications https://www.youtube.com/watch?v=Qqv0K0Y7uO8 Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 40 RASTER SCAN PRINCIPLE • Used by picture tubes in television sets • Contains a finely-focussed electron beam scanned over the complete screen • Scan consists of a number of horizontal lines (frame) starting from top left to bottom right. Repeats again; PROGRESSIVE SCANNING • Display screen is coated with a light-sensitive PHOSPHOR that emits light when energized by electron beam-Brightness. • B & W picture tubes – single electron beam • Color-3 separate closely located beams Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 41 Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 42 Display Technologies Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 43 Cathode-ray tube (CRT) Picture Tube/ Monitors Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 44 Contd.. • Each pixel comprises of three color-sensitive phosphors (one for R, B and G)-PHOSPHOR TRIAD • Each pixel has shape of a SPOT which merges with its neighbors. • Spot size is 0.025 inches(0.635 mm) looks as continuous color image at distance. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 45 Contd.. • Television picture tubes-moving images- screen refresh. • FRAME REFRESH RATE: High enough to ensure eye is not aware that the display is continuously refreshed. Low refresh rate leads to FLICKER A refresh rate of at least 50/60 times per second is required Depends on electric supply – 60 Hz North & South America & Asia, 50 Hz in Europe Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 46 Raster-Scan Display Architecture Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 47 Pixel Depth 8 • The number of bits per pixel determines the range of different colors that can be produced. • In practice, a selected subset of the range is used. 8 8 Red Green Blue • The selected colors in the subset is stored in a table and each pixel value is used as an address to a location within the table called COLOR LOOK UP TABLE. • 24 bits plane, 8 bits per color gun. • 224 = 16,777,216 colors Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 48 Aspect ratio • The number of pixels per scanned line and the number of lines per frame vary. • It is the ratio of screen width to height. • 4:3 - standard aspect ratio for older television screens and computer monitors • 16:9 - standard aspect ratio for modern high-definition (HD) television screens, computer monitors, and most widescreen displays Used for movies, videos, and multimedia content. • 21:9 - ultra-wide or cinematic aspect ratio It is often used High-end computer monitors and cinematic productions to provide a more immersive viewing experience. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 49 Screen Resolution • Television Standards Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT NTSC – US PAL – UK CCIR – Germany SECAM – France NTSC-525 Other 3 625 lines 50 Memory Requirements Standard Resolution Number of Colors Memory Required per Frame VGA (Video Graphics Array) 640x480x8 256 307.2 kB XGA (Extended Graphics Array) 640x480x16 1024x768x8 64 K 256 614.4 kB 786.432 kB SVGA (Super VGA) 800x600x16 1024x768x8 1024x768x24 64 K 256 16M 960 kB 786.432kB 2359.296kB Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 51 Digital Cameras and Scanner • IMAGE SENSOR 2-D grid of lightsensitive cells called PHOTOSITES • Charged Coupled Device (CCD) Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 52 Contd.. • An image is captured within the camera using solid-state device called an IMAGE SENSOR. It is a silicon chip and consists of a 2-D grid of light-sensitive cells called PHOTOSITES. • When the camera shutter is activated, each photosites stores the level of intensity of the light that falls on it. • Charged Coupled Device (CCD) is an image sensor which consists of array of photosites that converts the level of intensity that falls on it into an equivalent electrical charge. • It is then converted into digital values using an ADC. • For color images, the color associated with each pixel is obtained by, • Each photosite is coated with either a R,G or B filter in a 3x3 grid so its charge is determined by the level of RGB light that falls on it. The color associated with each photosite is determined by the output with each of its 8 neighbors. The levels of the other two colors is estimated by interpolation involving nine values. Used with most customer-quality cameras Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 53 Contd.. • Involves three separate exposures of a single image sensors for R,B and G. Since three separate exposures are needed for image not used with video cameras. Used for high resolution still-image cameras such as in studios. • Involves three separate image sensors, each coated with separate R,B and G filters. Used in professional quality high resolution still and moving images. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 54 Audio- PCM Speech Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 55 CD-Quality Audio Speech – 15 Hz to 20 KHz Sampling Rate = 40 ksps Bit rate per channel = Sampling rate * bits per sample Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 56 Video • Entertainment Broadcast television VCR/ DVD • Interpersonal Video telephony Video conferencing • Interactive Short video clips Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 57 Broadcast television • National Television Standards Committee (NTSC) • Sequential Color and Memory (SECAM) • Phase Alternate Line (PAL) Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 58 Interlaced Scanning • Interlaced scanning Bandwidth and flicker reduction Constructing each frame from two consecutive fields • Fields are sampled at different times such that consecutive lines belong to alternate fields US a 60 Hz format Europe a 50 Hz format • Fresh rate is reduced to 30/25 frames per second Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 59 Interlaced Scanning Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 60 Interlaced Scanning Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 61 Color Signals • Monochrome – Black & White • Color Broadcast – R,G & B • Color Properties: • Brightness: relative darkness or lightness of a color Varies on grey scale from black to white Independent of the color of the source • Hue: Actual color of the source • Saturation: Strength or vividness of the color Pastel color has a low level than a red color Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 62 Contd.. • Luminance – refer to brightness of source • Chrominance- related to hue and saturation of the color • Color produced on the Screen – magnitude of the electrical signals- energize R,G and B • 0.299R+0.587G+0.114B WHITE • Ys= 0.299Rs+0.587Gs+0.114Bs • Ys – amplitude of the luminance signal • Blue Chrominance(Cb) & Red Chrominance(Cr) to represent hue and saturation Cb=Bs-Ys & Cr=Rs-Ys Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 63 Chrominance Components • Transmission BW (Monochrome+ Color) – Same • Y,Cb & Cr Composite Video signal • Magnitude of chrominance signal are scaled down • PAL: Y=0.299R+0.587G+0.114B • U=0.493(B-Y) • V=0.877(R-Y) • NTSC: Y=0.299R+0.587G+0.114B • I=0.74(R-Y)-0.27(B-Y) • Q=0.48(R-Y)+0.41(B-Y) Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 64 Signal Bandwidth - NTSC Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 65 Signal Bandwidth - PAL Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 66 Digital Video • Luminance and 2 color signals instead RGB • Reduction in bit rate • CCIR- Consultative Committee for International Radiocommunications • CCIR-601 Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 67 4:2:2 Format • Used in Television Studios- broadcast Applications • BW - 6 MHz for luminance signals & 3 MHz for Cr and Cb • Line Sampling Rate 13.5 MHz Luminance • Line Sampling Rate 6.75 MHz Cr and Cb 52 Micro seconds * 13.5 MHz= 702 samples per line lines Line Sampling Rate Total Line Sweep time Retrace sweep time Active Sweep time Samples per line NTSC 525 13.5 MHZ 63.56 micro sec 11.56 micro sec 52 micro sec 702 PAL 625 13.5 MHZ 64 micro sec 12 micro sec 52 micro sec 702 Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 68 Contd.. 525 line System Y= 720X480 Cb=Cr= 360X480 625 line System Y= 720X576 Cb=Cr= 360X576 MXN M Samples/ line N line system Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 69 Standards Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 70 4:2:0 Format • Bitrate = 162 Mbps 525 line System Y= 720X480 Cb=Cr= 360X240 625 line System Y= 720X576 Cb=Cr= 360X288 Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 71 Contd.. Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 72 HDTV Format Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 73 Source Intermediate format (SIF) • 4:1:1 format • Bit Rate = 81 Mbps • ½ the refresh rate temporal resolution • ½ resolution in vertical and horizontal directions(subsampling) • Missing Samples – Interpolation method 525 line System Y= 360X240 Cb=Cr= 180X120 30 Hz 625 line System Y= 360X288 Cb=Cr= 180X144 25 Hz Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 74 Common Intermediate Format(CIF) • Video Conferencing Applications • Y= 360X288 • Cb=Cr= 180X144 • 30 Hz progressive Scanning Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 75 Quarter CIF(QCIF) • Video telephony • Y= 180X144 • Cb=Cr= 90X72 • Bit rate = 40.5 Mbps • 4:1:1 format Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 76 PC Video Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 77 Thank You Happy Learning Dr.I.S.Akila, Associate Prof., Dept. of ECE, CIT 78