Make images come
alive with scikitimage
IMAGE PROCESSING IN PYTHON
Rebeca
Data
Engineer
Gonzalez
What is image processing?
Operations on images and videos to:
•
Enhance an image
•
Extract useful information
•
Analyze it and make decisions
What is image processing?
Operations on images and videos to:
•
Enhance an image
•
Extract useful information
•
Analyze it and make decisions
Applications
•
Medical image analysis
•
Artificial intelligence
•
Image restoration and enhancement
•
Geospatial computing
•
Surveillance
•
Robotic vision
•
Automotive safety
•
And many more...
Purposes
1. Visualization:
o Objects that are not visible
2. Image sharpening and restoration
o A better image
3. Image retrieval
o Seek for the image of interest
4. Measurement of pattern
o Measures various objects
5. Image Recognition
o Distinguish objects in an image
Intro to scikit-image
•
Easy to use
•
Makes use of Machine Learning
•
Out of the box complex algorithms
What is an image?
What is an image?
What is an image?
Images in scikit-image
from skimage import data
rocket_image = data.rocket()
RGB channels
Grayscaled images
RGB vs Grayscale
from skimage import color
grayscale = color.rgb2gray(original)
rgb = color.gray2rgb(grayscale)
Visualizing images in the course
Don't worry about Matplotlib!
Visualizing images in the course
from skimage import color
grayscale = color.rgb2gray(original)
show_image(grayscale, "Grayscale")
Questions?
NumPy for images
IMAGE PROCESSING IN PYTHON
Rebeca
Data
Engineer
Gonzalez
NumPy for images
•
Fundamentals of image processing
techniques
o Flipping
o Extract and analyze features
Images as NdArrays
# Loading the image using Matplotlib
madrid_image = plt.imread('/madrid.jpeg')
type(madrid_image)
<class 'numpy.ndarray'>
Colors with NumPy
Colors with NumPy
# Obtaining the red values of the image
red = image[:, :, 0]
# Obtaining the green values of the image
green = image[:, :, 1]
# Obtaining the blue values of the image
blue = image[:, :, 2]
Colors with NumPy
plt.imshow(red, cmap="gray")
plt.title('Red')
plt.axis('off')
plt.show()
Shapes
# Accessing the shape of the image
madrid_image.shape
(426, 640, 3)
Sizes
# Accessing the shape of the image
madrid_image.size
817920
Flipping images: vertically
# Flip the image in up direction
vertically_flipped = np.flipud(madrid_image)
show_image(vertically_flipped, 'Vertically flipped image')
Flipping images: horizontally
# Flip the image in left direction
horizontally_flipped = np.fliplr(madrid_image)
show_image(horizontally_flipped, 'Horizontally flipped image')
What is a histogram?
What is a histogram?
Color histograms
Color histograms
Applications of histograms
•
Analysis
•
Thresholding
•
Brightness and contrast
•
Equalize an image
Applications of histograms
•
Analysis
•
Thresholding
•
Brightness and contrast
•
Equalize an image
Histograms in Matplotlib
# Red color of the image
red = image[:, :, 0]
# Obtain the red histogram
plt.hist(red.ravel(), bins=256)
Visualizing histograms with Matplotlib
blue = image[:, :, 2]
plt.hist(blue.ravel(), bins=256)
plt.title('Blue Histogram')
plt.show()
Questions?
Getting started with
thresholding
IMAGE PROCESSING IN PYTHON
Rebeca
Data
Engineer
Gonzalez
Thresholding
Partitioning an image into a foreground
and background
By making it black and white
We do so by setting each pixel to:
• 255 (white) if pixel > thresh value
• 0 (black) if pixel < thresh value
Thresholding
Simplest method of image segmentation
•
Isolate objects
o Object detection
o Face detection
o Etc.
Thresholding
Only from grayscale images
Thresholding
Only from grayscale images
Apply it
# Obtain the optimal threshold value
thresh = 127
# Apply thresholding to the image
binary = image > thresh
# Show the original and thresholded
show_image(image, 'Original')
show_image(binary, 'Thresholded')
Inverted thresholding
# Obtain the optimal threshold value
thresh = 127
# Apply thresholding to the image
inverted_binary = image <= thresh
# Show the original and thresholded
show_image(image, 'Original')
show_image(inverted_binary,
'Inverted thresholded')
Categories
Global or histogram based: good for uniform backgrounds
Local or adaptive: for uneven background illumination
Categories
Global or histogram based: good for uniform backgrounds
Local or adaptive: for uneven background illumination
Try more thresholding algorithms
from skimage.filters import try_all_threshold
# Obtain all the resulting images
fig, ax = try_all_threshold(image, verbose=False)
# Showing resulting plots
show_plot(fig, ax)
Try more thresholding algorithms
Optimal thresh value
Global
Uniform background
# Import the otsu threshold function
from skimage.filters import threshold_otsu
# Obtain the optimal threshold value
thresh = threshold_otsu(image)
# Apply thresholding to the image
binary_global = image > thresh
Optimal thresh value
Global
# Show the original and binarized image
show_image(image, 'Original')
show_image(binary_global, 'Global thresholding')
Optimal thresh value
Local
Uneven background
# Import the local threshold function
from skimage.filters import threshold_local
# Set the block size to 35
block_size = 35
# Obtain the optimal local thresholding
local_thresh = threshold_local(text_image, block_size, offset=10)
# Apply local thresholding and obtain the binary image
binary_local = text_image > local_thresh
Optimal thresh value
Local
Block Size
refers to the size of the neighborhood around each pixel which is used to calculate
the local threshold value.
This method is particularly useful for images with varying lighting conditions or for
images where the background and foreground do not have a consistent intensity.
Optimal thresh value
Local
Block Size
The block size is typically defined as a square region centered around each pixel.
Here are some key points about block size:
Dimension: The block size is usually an odd integer, such as 3, 5, 7, ... etc. This is to
ensure that there is a central pixel with an equal number of neighbors on all sides.
Local Thresholding: The threshold value is computed for each block based on the
pixel values within that block. This could be the mean, median, or weighted average
of the pixel values minus an offset.
Border Handling: A special handling is needed at the image’s borders since the blocks
extend beyond the image. Padding or other methods are used to deal with this.
Optimal thresh value
Local
Block Size
Small Block Size (e.g., 3x3): A small block size will adapt very closely to the local
variations in intensity. This can be useful for high-resolution images with fine details
and subtle variations in intensity. However, it can also be more susceptible to noise.
Large Block Size (e.g., 35x35): A larger block size will be less sensitive to local
variations and noise. It can be useful for images with large uniform regions or when
the illumination changes gradually. A large block size smooths out the local variations,
leading to a more stable threshold but at the risk of missing finer details.
Choosing the right block size is crucial for getting good results from local thresholding
and often requires experimentation and domain knowledge about the images being
processed.
Optimal thresh value
Block Size (Border Handling)
Padding: The most common approach is to pad the image with extra pixels around the edges.
Replicating the edge pixels: The value of the border pixels is copied to the padding pixels.
Reflecting the image across the edges: The image is reflected over its edges, so the padding is a mirror image
of the edge pixels.
Constant padding: All the padding pixels are set to a constant value, which could be zero (leading to a kind of
"zero-padding") or another value that makes sense for the given image.
Wrapping the image: The opposite edge of the image is used to fill in the padding area, creating a wraparound
effect.
Ignoring the Edges: Some implementations might not perform thresholding on the border pixels, leaving them as
they are or setting them to a default value. This approach is less common since it can lead to artifacts along the
image borders.
Adjusting the Block Size: The block size could be dynamically adjusted for the pixels on the borders so that only
the available pixels are used for calculating the local threshold. This means that the block is effectively smaller on
the edges and corners.
Extending the Image: Instead of padding, the image might be extended so that the centers of the blocks at the
borders correspond to the actual edge pixels of the image. This is similar to padding but with a conceptual shift in
how the blocks are positioned relative to the image pixels.
Optimal thresh value
Block Size (Border Handling)
#1 Padding:
The most common approach is to pad the image with extra pixels around the edges.
Replicating the edge pixels:
The value of the border pixels is copied to the padding pixels.
Reflecting the image across the edges:
The image is reflected over its edges, so the padding mirrors the edge pixels.
Constant padding:
All the padding pixels are set to a constant value, which could be zero (leading
to a "zero-padding") or another value that makes sense for the given image.
Wrapping the image:
The opposite edge of the image is used to fill in the padding area, creating a
wraparound effect.
Optimal thresh value
Block Size (Border Handling)
#2 Ignoring the Edges: Some implementations might not perform thresholding on the
border pixels, leaving them as they are or setting them to a default value. This
approach is less common since it can lead to artifacts along the image borders.
#3 Adjusting the Block Size: The block size could be dynamically adjusted for the
pixels on the borders so that only the available pixels are used for calculating the local
threshold. This means that the block is effectively smaller on the edges and corners.
#4 Extending the Image: Instead of padding, the image might be extended in such a
way that the centers of the blocks at the borders correspond to the actual edge pixels
of the image. This is like padding but with a conceptual shift in how the blocks are
positioned relative to the image pixels. (i.e. we use what’s available/existing)
Optimal thresh value
Offset value
The offset in local thresholding is used to adjust the threshold value calculated for
each local region or block of pixels.
The purpose of an offset is to fine-tune the thresholding process, particularly in cases
where the image may not have a clear distinction between the foreground and the
background, or when the background illumination is uneven.
Optimal thresh value
Offset value
Here's why using an offset makes sense:
Handle Uneven Illumination: In images with uneven lighting, some areas might be
brighter or darker than others. Without an offset, the thresholding algorithm might
mistakenly consider brighter areas as foreground or darker areas as background. An
offset can help to compensate for these lighting variations.
Increase Flexibility: The offset provides an additional parameter to control the
sensitivity of the thresholding. By subtracting a constant from the mean of the blocks,
you can control how strict or lenient the thresholding is. A positive offset makes the
algorithm more likely to classify a pixel as background (making the method more
conservative), whereas a negative offset makes it more likely to classify a pixel as
foreground (making the method more liberal).
Optimal thresh value
Offset value
Enhance Contrast: In cases where the image has low contrast, the difference between
the highest and lowest intensities might not be significant enough to distinguish
important details. An offset can artificially enhance the contrast during the thresholding
process, helping to separate the foreground from the background more effectively.
Customization for Specific Features: If you're interested in highlighting particular
features of an image, the offset can be adjusted to ensure that these features are
captured as the foreground when thresholding.
In summary, the offset provides a means to account for variations in image brightness
and contrast that are not handled well by a simple global threshold.
By carefully selecting an offset value, you can improve the segmentation results in local
thresholding, especially for images with challenging lighting conditions or intrinsic
variations in intensity.
Optimal thresh value
Local
# Show the original and binarized image
show_image(text_image, 'Original')
show_image(binary_local, 'Local thresholding')
Let's practice!
IMAGE PROCESSING IN PYTHON