2ECOE04
Fundamentals of Image
and Video Processing
Prof Bhupendra Fataniya
Assistant Professor,
EC,IT,NU
14-09-2022
1
Outline
• Introduction
• Why Image and Video processing
• Applications
• Specific of this Course
• Publications
• Journals
• Conferences
14-09-2022
2
What is an image?
• Impression of the physical world.
• Spatial distribution of a measurable quantity, encoding
the geometry and material properties of objects.
14-09-2022
3
What is Image Processing?
• Computer Graphics: Models to Images
• Comp. Photography: Images to Images
• Computer Vision: Images to Models
• Make computers
understand images and
video.
14-09-2022
4
Computer Vision vs. Graphics
• 3D→2D implies information loss
graphics
vision
• sensitivity to errors
• need for models
Vision
• Vision is the process of discovering what is present in the
world and where it is by looking.
Computer Vision
• Computer Vision is the study of analysis of pictures and
videos in order to achieve results similar to those as by
people.
Why Image Processing
Safety
Health
Comfort
Fun
Security
Access
Application Areas of Image Processing
• Purpose of image processing
• Improvement of pictorial information for human interpretation
• Compression of image data for storage and transmission
• Preprocessing to enable object detection, classification, and
tracking
• Typical application areas
•
•
•
•
•
Television Signal Processing
Satellite Image Processing
Medical Image Processing
Robotics
Visual Communications
14-09-2022
9
Industrial Application
Source: Computer
Vision Algorithms
and Application
by Richard Szeliski
14-09-2022
10
Optical character recognition (OCR)
Technology to convert scanned docs to text
• If you have a scanner, it probably came with OCR software
Digit recognition, AT&T labs
http://www.research.att.com/~yann/
License plate readers
http://en.wikipedia.org/wiki/Automatic_number_plate_recognition
Face detection
• Digital cameras detect faces
Smile detection
Sony Cyber-shot® T70 Digital Still Camera
Vision-based biometrics
“How the Afghan Girl was Identified by Her Iris Patterns” Read the story
wikipedia
Login without a password…
Fingerprint scanners on
many new laptops,
other devices
Face recognition systems now
beginning to appear more widely
http://www.sensiblevision.com/
Object recognition (in mobile phones)
Point & Find, Nokia
Google Goggles
Sports
Sportvision first down line
Nice explanation on www.howstuffworks.com
http://www.sportvision.com/video.html
Medical imaging
3D imaging
MRI, CT
Image guided surgery
Grimson et al., MIT
Smart cars
Slide content courtesy of Amnon Shashua
• Mobileye
• Market Capitalization: 11 Billion dollars
Google cars
Oct 9, 2010. "Google Cars Drive Themselves, in Traffic". The New York Times. John
Markoff
June 24, 2011. "Nevada state law paves the way for driverless cars". Financial Post.
Christine Dobby
Aug 9, 2011, "Human error blamed after Google's driverless car sparks five-vehicle
crash". The Star (Toronto)
Interactive Games: Kinect
• Object Recognition:
http://www.youtube.com/watch?feature=iv&v=fQ59dXOo63o
• Mario: http://www.youtube.com/watch?v=8CTJL5lUjHg
• 3D: http://www.youtube.com/watch?v=7QrnwoO1-8A
• Robot: http://www.youtube.com/watch?v=w8BmgtMKFbY
Industrial robots
Vision-guided robots position nut runners on wheels
Vision in space
NASA'S Mars Exploration Rover Spirit captured this westward view from atop
a low plateau where Spirit spent the closing months of 2007.
Vision systems (JPL) used for several tasks
•
•
•
•
Panorama stitching
3D terrain modeling
Obstacle detection, position tracking
For more, read “Computer Vision on Mars” by Matthies et al.
Augmented Reality and Virtual Reality
Magic Leap, Oculus, Hololens, etc.
How images are represented in
computer?
14-09-2022
26
Color images
14-09-2022
27
Image File
14-09-2022
28
Components in Digital Image
Processing
14-09-2022
29
Some Topics of FIVP
Imaging Geometry
Camera Modeling
• Pinhole Cameras
• Lenses
• Camera Parameters and
Calibration
Image Filtering and Enhancing
• Linear Filters and
Convolution
• Image Smoothing
• Edge Detection
• Pyramids
Image Filtering and Enhancing (cont.)
Region Segmentation
Image Restoration
Original
Synthetic
Basic Image Processing Operations
• Simple point processing
• Special effects Special effects
• Noise reduction
• Image enhancement
• Image restoration Image restoration
• Face detection
• Image segmentation
14-09-2022
37
Simple point processing
14-09-2022
38
Contrast Enhancement
14-09-2022
39
Noise Reduction
14-09-2022
40
Image Sharpening
14-09-2022
41
Image Restoration
14-09-2022
42
Image debluring
14-09-2022
43
Special Effects
14-09-2022
44
Image Segmentation
14-09-2022
45
Course Syllabus
Link: Course: Fundamentals of Image and Video Processing
(nirmauni.ac.in)
Course Assessment
Assessme
CE
nt scheme
60%
Compone
Quiz1 Quiz2 Sessiona Term Paper/
nt
l Exam
Innovative
weightage
SEE
40%
Assignment
15
15
40
30
100
Continuous Evaluation (CE) & Semester End Examination (SEE)
Course Outcomes
• At the end of the course, students will be able to –
• Apply spatial and frequency domain image filters for image
enhancement.
• Comprehend image degradation models for image restoration
and
color
space
transforms
for
color
image
processing.
• Interpret and apply edge detection, image segmentation and
representation for image recognition.
• Demonstrate the use of image and video processing algorithms
for different applications.
14-09-2022
48
Course Topics
• Introduction
• Spatial and Frequency Domain Filtering
• Image and Video Compression Standards
• Image Restoration
• Color Image Processing
• Morphological Image Processing
• Edge Detection and Image Segmentation
• Object Recognition
Textbook
• R. C. Gonzalez, R. E. Woods, Digital Image Processing,
Addison Wesley
• Sridhar, Digital Image Processing, Oxford University Press
• Vipula Singh, Digital Image Processing with MATLAB and
LabView, Elsevier
• S. Jayaraman, S.Esakkirajan, T.Veerakumar, Digital Image
Processing, TMH
About You …
What do you know already?
•
•
•
•
•
C/C++ (Visual C++)
Matlab
Images
Python
OpenCV
Projects
• Image Filtering and Hybrid Images
• Local Feature Matching
• Camera Calibration and Fundamental Matrix Estimation
with RANSAC
• Scene Recognition with Bag of Words
• Object Detection with a Sliding Window
• Recognition with Deep Learning
Project 1: Image Filtering and Hybrid
Images
• Implement image filtering to separate high and low
frequencies
• Combine high frequencies and low frequencies from different
images to create an image with scale-dependent interpretation
Project 2: Local Feature Matching
• Implement interest point detector, SIFT-like local feature
descriptor, and simple matching algorithm.
Project 3: Scene Recognition with Bag of
Words
• Quantize local features into a “vocabulary”, describe images as
histograms of “visual words”, train classifiers to recognize
scenes based on these histograms.
Project 4: Object Detection with a Sliding
Window
• Train a face detector based on positive examples and “mined”
hard negatives, detect faces at multiple scales and suppress
duplicate detections.
Project 5: panorama stitching
Indri Atmosukarto, 576 08sp
Project 6: Face Recognition
Publications
• Journals
• IEEE Trans. on Pattern Analysis and Machine Intelligence
(TPAMI)
• IEEE, Thompson-ISI impact factor: 5.96
• in both electrical engineering and artificial intelligence
• in all of computer science
• Internal Journal of Computer Vision (IJCV)
• ISI impact factor: 5.358, Rank 2 of 94 in “CS, artificial intelligence
• IEEE Trans. on Image Processing
•…
Publications
• Conferences
• International Conference on Computer Vision (ICCV), once
every two years
• Conf. of Computer Vision and Pattern Recognition (CVPR), once
a year
• Europe Conference on Computer Vision (ECCV), once every
two years
•…
Thank you
14-09-2022
61