CSE 554
Lecture 1: Binary Pictures
Fall 2016
CSE554
Binary Pictures
Slide 1
Geometric Forms
Curves
• Continuous forms
Surfaces
– Defined by mathematical functions
– E.g.: parabolas, splines, subdivision
y  x2
surfaces
z  Sin[ x]Sin[ y]
• Discrete forms
– Disjoint elements with connectivity
Polyline
relations
Triangle surfaces
– E.g.: polylines, triangle surfaces, pixels
and voxels
Pixels
CSE554
Binary Pictures
Voxels
Slide 2
Digital Pictures
• Made up of discrete points associated with colors
– Image: 2D array of pixels
2D Image
CSE554
Binary Pictures
Slide 3
Digital Pictures
• Color representations
Low
High
– Grayscale: 1 value representing grays from
Grayscale
black (lowest value) to white (highest value)
• 8-bit (0-255), 16-bit, etc.
Low R
High R
– RGB: 3 values each representing colors from
Low G
High G
black (lowest value) to pure red, green, or
blue (highest value).
Low B
High B
• 24-bit (0-255 in each color)
RGB
– XYZ, HSL/HSV, CMYK, etc.
B
R
CSE554
Binary Pictures
G
Slide 4
Digital Pictures
• Made up of discrete points associated with colors
– Volume: 3D array of voxels
Voxel
3D Volume
CSE554
Binary Pictures
Slide 5
Binary Pictures
• A grayscale picture with 2 colors: black (0) and white (1)
– The set of 1 or 0 pixels (voxels) is called object or background
– A “blocky” geometry
Background
Analogy: Lego, Minecraft
Object
CSE554
Binary Pictures
Slide 6
Binary Pictures
Creation
CSE554
Processing
Binary Pictures
Slide 7
Segmentation
• Separating object from background in a grayscale picture
– A simple method: thresholding by pixel (voxel) color
• All pixels (voxels) with color above a threshold is set to 1
Grayscale picture
CSE554
Thresholded binary picture
Binary Pictures
Slide 8
Segmentation
• Separating object from background in a grayscale picture
– A simple method: thresholding by pixel (voxel) color
– Other methods:
• K-means clustering
• Watershed
• Region growing
• Snakes and Level set
• Graph cut
• …
– More details covered in Computer Vision course
CSE554
Binary Pictures
Slide 9
Rasterization
• Filling the interior of a shape by pixels or voxels
– Known as “scan-conversion”, or “pixelization / voxelization”
– More details covered in Computer Graphics course
2D Polygon
CSE554
Binary Picture
Binary Pictures
Slide 10
Binary Pictures
Creation
CSE554
Processing
Binary Pictures
Slide 11
Binary Pictures
Removing
islands and
holes
CSE554
Smoothing
jagged
boundaries
Binary Pictures
Slide 12
Removing Islands & Holes
• Observations:
• Islands (holes) of the object are holes (islands) of the background
• Islands (holes) are not as well-connected as the object (background).
Take the largest 2 connected
components of the object
CSE554
Invert the largest connected
component of the background
Binary Pictures
Slide 13
Connected Components
• Definition
– A maximum set of pixels (voxels) in the object or background, such that
any two pixels (voxels) in the set are connected by a path of connected
pixels (voxels)
CSE554
Binary Pictures
Slide 14
Connected Components
How many connected components are there in
the object? What about background?
CSE554
Binary Pictures
Slide 15
Connectivity (2D)
• Two pixels are connected if
their squares share:
Two connected
pixels
All pixels
connected to x
– A common edge
x
• 4-connectivity
– A common vertex
4-connectivity
• 8-connectivity
x
8-connectivity
CSE554
Binary Pictures
Slide 16
Connectivity (2D)
Object: 8-connectivity (1 comp)
CSE554
Object: 4-connectivity (4 comp)
Binary Pictures
Slide 17
Connectivity (2D)
• What connectivity should be used for the background?
Object: 8-connectivity (1 comp)
Background: 8-connectivity (1 comp)
Object: 4-connectivity (4 comp)
Background: 4-connectivity (2 comp)
Paradox: a closed curve does not disconnect the background,
while an open curve does.
CSE554
Binary Pictures
Slide 18
Connectivity (2D)
• Different connectivity for object (O) and background (B)
– 2D pixels: 4- and 8-connectivity respectively for O and B (or B and O)
Object: 8-connectivity (1 comp)
Background: 4-connectivity (2 comp)
CSE554
Object: 4-connectivity (4 comp)
Background: 8-connectivity (1 comp)
Binary Pictures
Slide 19
Connectivity (3D)
• Two voxels are connected if
Two connected
voxels
All voxels connected
to the center voxel
their cubes share:
– A common face
• 6-connectivity
6-connectivity
– A common edge
• 18-connectivity
– A common vertex
18-connectivity
• 26-connectivity
• Use 6- and 26-connectivity
respectively for O and B (or B and O)
26-connectivity
CSE554
Binary Pictures
Slide 20
Finding Connected Components
• The “flooding” algorithm
– Start from a seed pixel/voxel, expand the connected component
– Either do depth-first or breadth-first search (a LIFO stack or FIFO queue)
// Finding the connected component containing an object pixel p
1. Initialize
1. Create a result set S that contains only p
2. Create a Visited flag at each pixel, and set it to be
False except for p
3. Initialize a queue (or stack) Q that contains only p.
2. Repeat until Q is empty:
1. Pop a pixel x from Q.
2. For each unvisited object pixel y connected to x, add y
to S, set its flag to be visited, and push y to Q.
3. Output S
CSE554
Binary Pictures
Slide 21
Finding Connected Components
• Why using a “visited” flag?
– Otherwise, the program will not terminate
• Why not checking to see if y is in S?
– Checking the visited flag is much faster ( O(1) vs. O(log n) )
1.
…
2. Repeat until Q is empty:
1. Pop a pixel x from Q.
2. For each unvisited object pixel y connected to x, add y
to S, set its flag to be visited, and push y to Q.
3. Output S
CSE554
Binary Pictures
Slide 22
Connectivity (2D)
• Connected components containing the blue pixel:
8-connectivity
CSE554
Binary Pictures
4-connectivity
Slide 23
Finding Connected Components
• Labeling all components in an image:
– Loop through each pixel (voxel). If it is not labeled, use it as a seed to find
a connected component, then label all pixels (voxels) in the component.
One
component
Two
components
8-connected object
CSE554
4-connected object
Binary Pictures
Slide 24
Using Connected Components
• Pruning isolated islands from the main object
• Filling interior holes of the object
Take the largest components of
the object
CSE554
Invert the largest components of
the background
Binary Pictures
Slide 25
Morphological Operators
• Smoothing out object boundary
Opening
Closing
CSE554
Binary Pictures
Slide 26
Morphological Operators
• Operations to change shapes
– Erosion
– Dilation
– Opening: first erode, then dilate.
– Closing: first dilate, then erode.
CSE554
Binary Pictures
Slide 27
Mathematical Morphology
x
Input:
Object (A)
Structure element (Bx)
Erosion
Dilation
𝑨⊕𝑩=
𝑨 ⊖ 𝑩 = {𝒙 ∈ 𝑨|𝑩𝒙 ⊆ 𝑨}
𝑩𝒙
𝒙∈𝑨
CSE554
Binary Pictures
Slide 28
Mathematical Morphology
• Structure element B is symmetric if:
𝒙 ∈ 𝑩 𝒚 ⟺ 𝒚 ∈ 𝑩𝒙
• Examples:
𝒚
𝒙
Circle
CSE554
𝒚
𝒙
Square
Binary Pictures
𝒙
𝒚
Triangle
Slide 29
Mathematical Morphology
• Duality (for symmetric structuring elements)
– Erosion (dilation) is equivalent to dilation (erosion) of the background
CSE554
Erosion
Dilation
𝑨⊖𝑩=𝑨⊕𝑩
𝑨⊕𝑩=𝑨⊖𝑩
Binary Pictures
Slide 30
Mathematical Morphology
• Opening (erode, then dilate)
x
Object (A)
Structure element (Bx)
Opening
𝑨 ∘ 𝑩 = (𝑨 ⊝ 𝑩) ⊕ 𝑩
– Union of all B that can fit inside A
• Shaves off convex corners and thin spikes
CSE554
Binary Pictures
Slide 31
Mathematical Morphology
• Closing (dilate, then erode)
x
Structure element (Bx)
Closing
Object (A)
𝑨 ⋅ 𝑩 = (𝑨 ⊕ 𝑩) ⊝ 𝑩
– Complement of union of all B that can fit in the complement of A
• Fills concave corners and thin tunnels
CSE554
Binary Pictures
Slide 32
Mathematical Morphology
• Duality, again! (for symmetric structuring elements)
– Opening (closing) object is equivalent to closing (opening) background
Opening
Closing
𝑨∘𝑩=𝑨⋅𝑩
CSE554
𝑨⋅𝑩 =𝑨∘𝑩
Binary Pictures
Slide 33
Digital Morphology
• Structuring elements (symmetric)
– 2D pixels: square or cross
x
x
– 3D voxels: cube or cross
CSE554
Binary Pictures
Slide 34
Digital Morphology
• Structuring element:
x
– Erosion
• e: an object pixel with some
e
e
background pixel in its square
neighborhood
e
– Dilation
• d: a background pixel with some
object pixel in its square
neighborhood
d
e
e
e
e
e
d
d
d
d
d
d
d
d
d
d
d
d
d
CSE554
e
Binary Pictures
d
d
d
d
Slide 35
Digital Morphology
• Structuring element: 3x3x
square
– Opening
e
e
Union of white
squares within the
object
– Closing
Union of black
squares within the
background
e
d
Erosion
e
d
e
d
d
d
e
e
d
d
d
d
d
d
e
e
e
d
d
d
d
d
d
d
d
d
d
e
Dilation
Dilation
d
e
d
d
CSE554
e
e
e
e
e
e
d
e
e
d
e
e
d
e
Binary Pictures
e
e
e
Erosion
e
Slide 36
Digital Morphology
• Increasing the size of the structuring element
– Leads to more growing/shrinking and more significant smoothing
Original
Opening by 3x3 square
Opening by 5x5 square
– Equivalent to repeated applications with a small structuring element
• E.g.: k erosions (dilations) followed by k dilation (erosions) with a 3x3 square is
equivalent to opening (closing) with a (2k+1)x(2k+1) square.
CSE554
Binary Pictures
Slide 37
Digital Morphology
• Implementation tips
– Using duality of erosion and dilation, you only need to implement one
function to do both morphological operations (for symmetric structure
elements).
• Dilation is same as erosion of the background
– When performing multiple-round opening, make sure you first do k times
erosion then k times dilation
• What happens if you alternate erosion and dilation for k times?
– Handle image boundary in a graceful way (not crashing the program…)
• For example, treat outside of the image as background
CSE554
Binary Pictures
Slide 38
Lab Module 1
• A simple 2D segmentation routine
– Initial segmentation using thresholding (using your code from Lab 0)
– Using connected components and opening/closing to “clean up” the
segmentation.
CSE554
Binary Pictures
Slide 39