CS 2305: Discrete Mathematics
for Computing I
Lecture 14
- KP Bhat
1
Functions
Section 2.3
Functions
1
Definition: Let A and B be nonempty sets. A function
f from A to B, denoted f: A  B is an assignment of
each element of A to exactly one element of B. We
write f(a) = b if b is the unique element of B
assigned by the function f to the element a of A.
• Functions are sometimes
called mappings
or transformations.
Functions
2
A function f: A  B can also be defined as a subset of
A×B (a relation). This subset is restricted to be a relation
where no two elements of the relation have the same
first element.
Specifically, a function f from A to B contains one, and
only one ordered pair (a, b) for every element a∈ A.
x  x  A  y  y  B   x, y   f  
and
x, y1 , y2   x, y1   f   x, y2   f   y1  y2 
Functions
3
Given a function f: A  B:
• We say f maps A to B or f is a mapping from A to B.
• A is called the domain of f.
• B is called the codomain of f.
• If f(a) = b,
• then b is called the image of a under f.
• a is called the preimage of b.
• The range of f is the set of all images of points in A under f. We
denote it by f(A).
Function: Visualization
1
Two functions are equal when they have the same domain, the
same codomain and map each element of the domain to the
same element of the codomain.
6
Domain, Codomain and Range
Note: The range is always a subset (and sometimes a proper
subset) of the domain.
7
Function: Visualization
2
8
Examples of Functions (1)
Example: Suppose that each student in a discrete mathematics class is
assigned a letter grade from the set {A, B, C, D, F}. And suppose that
the grades are A for Adams, C for Chou, B for Goodfriend, A for
Rodriguez, and F for Stevens. Find the domain, codomain, and range
of the function that assigns a grade to the students.
Solution:
Domain:
{Adams, Chou, Goodfriend, Rodriguez, Stevens}
Codomain:
{A, B, C, D, F}
Range:
{A, B , C , F}
9
Examples of Functions (2)
Example: Let f : Z → Z assign the square of an integer to
this integer [i.e. f(x) = x2]. Find the domain, codomain,
and range of the function
Solution:
Domain:
Z
Codomain:
Z
Range:
{x ∈ Z | x is a perfect square}
10
Representing Functions
Functions may be specified in different ways:
• An explicit statement of the assignment. Students
and grades example.
• A formula.
f  x  x 1
• A computer program.
• A Java program that when given an integer n, produces
the nth Fibonacci Number.
Questions
f(a) = ?
z
The image of d is ?
z
The domain of f is ?
A
The codomain of f is ?
B
The preimage of y is ?
b
f(A) = ?
{y,z}
The preimage(s) of z is (are) ?
{a,c,d}
Question on Functions and Sets
If f : A  B and S is a subset of A, then the range
of S is the set of all images of the elements of S
f  S    f  s  | s  S
f {a,b,c,} is ?
{y,z}
f {c,d} is ?
{z}
Increasing and Decreasing Functions
Definitions:
A function f is:
• increasing if ∀x1∀x2(x1 < x2 → f (x1) ≤ f (x2))
• strictly increasing if ∀x1∀x2(x1 < x2 → f (x1) < f (x2))
• decreasing if ∀x1∀x2(x1 < x2 → f (x1) ≥ f (x2))
• strictly decreasing if ∀x1∀x2(x1 < x2 → f (x1) > f (x2))
14
Injections
Definition: A function f is said to be one-to-one, or
injective, if and only if f(a) = f(b) implies that a = b for all a
and b in the domain of f. A function is said to be an
injection if it is one-to-one.
• Every image has a unique pre-image
Examples of Injections
• f(x) = x + 1, from R to R
• f(x) = x2, from Z+ to Z+
– If the domain is Z, f(x) = x2 is not an injection
because, for instance, f(2) = f(-2) = 4
16
Surjections
Definition: A function f from A to B is called onto
or surjective, if and only if for every element b  B
there is an element a  A with f  a   b .
A function f is called a surjection if it is onto.
•
•
∀y∃x(f (x) = y), where the domain for x is the
domain of the function and the domain for y
is the codomain of the function
All elements in the codomain have a preimage in the domain
Examples of Surjections
• f(x) = x + 1, from Z to Z
• All enrolled students at a university having an
active student ID
18
Bijections
Definition: A function f is a one-to-one
correspondence, or a bijection, if it is both
one-to-one and onto (surjective and injective).
•
Every element in the codomain has
a unique pre-image
Examples of Bijections
• f: {a, b, c, d} → {1, 2, 3, 4} with f (a) = 4, f (b) =
2, f (c) = 1, and f (d) = 3
• Let A be a set. The identity function on A is the
function 𝜄A : A → A, where 𝜄A(x) = x for all x ∈ A
– identity function assigns each element to itself
20
Examples of Different Types of
Functions
21
Showing that f is one-to-one or onto
Suppose that f : A  B.
To show that f is injective Show that if f (x) = f (y) for
arbitrary x, y ∈ A, then x = y.
To show that f is not injective Find particular elements x,
y ∈ A such that x ≠ y and f (x) = f (y).
To show that f is surjective Consider an arbitrary
element y ∈ B and find an element x ∈ A such that f (x)
= y.
To show that f is not surjective Find a particular y ∈ B
such that f (x) ≠ y for all x ∈ A.
To show that f is bijective Show that it is both injective
and surjective.
1
Showing that f is one-to-one or onto
Example 1: Is f(x) = 4x -1, from R to R, injective?
Solution:
Let us pick up two arbitrary numbers a and b such that
f(a) = f(b)
By the definition of f
4a – 1 = 4b – 1
4a = 4b
a=b
∴ f(x) = 4x -1, from R to R is injective
2
Showing that f is one-to-one or onto
Example 2: Is f(x) = 2x -3, from R to R, surjective?
Solution:
Part 1 (scratch work)
Let us pick an arbitrary number y ∈ R in the codomain
If such a number exists,
2x - 3 = y
2x = y + 3
x = (y + 3)/2
Part 2 (actual solution)
Let y ∈ R be an arbitrary element from the codomain
Let x = (y + 3)/2
f(x) = 2((y + 3)/2)-3 = y + 3 -3 = y
An arbitrary element from the codomain has a preimage in the domain
∴ f(x) = 2x -3, from R to R, is surjective
3
Showing that f is one-to-one or onto
Example 3: Is f(x) = x2 from Z to Z onto?
Solution:
Using counterexample
No, f is not onto because there is no integer x with x2 =
−1.
3
Inverse Functions
1
Definition: Let f be a bijection from A to B. Then
the inverse of f, denoted f 1 , is the function from
B to A defined as f 1  y   x iff f  x   y
Inverse Functions
2
Inverse Functions
3
• No inverse function exists unless f is a
bijection
– If f is not a bijection then it is either not an
injection or it is not a surjection (or both)
– If f is not an injection some element in the
codomain is the image of more than one element
in the domain
– If f is not a surjection, for some element in the
codomain there is no pre-image
• We say that all bijection functions are
invertible
28
Questions
1
Example 1: Let f be the function from {a,b,c} to
{1,2,3} such that f(a) = 2, f(b) = 3, and f(c) = 1. Is
f invertible and if so what is its inverse?
Solution: The function f is invertible because it is
a one-to-one correspondence. The inverse
function f−1 reverses the correspondence given
by f, so f−1 (1) = c, f−1 (2) = a, and f−1 (3) = b.
Questions
2
Example 2: Let f: Z  Z be such that f(x) = x + 1.
Is f invertible, and if so, what is its inverse?
Solution: The function f is invertible because it is
a one-to-one correspondence. The inverse
function f−1 reverses the correspondence so f−1
(y) = y −1.
Questions
3
Example 3: Let f: R  R be such that f  x   x 2
Is f invertible, and if so, what is its inverse?
Solution: The function f is not invertible because it is neither one-toone [e.g. –2 and +2 both map to 4] nor onto [no preimages for –ve
numbers]. In fact f−1 is not even a function.
Note:- If we restrict f to be from the set of non-ve real numbers to the
set of non-ve real numbers then the function is invertible. In this case
f−1(y) =√(x)
Composition
1
Definition: Let f: B→C, g: A→B. The composition of f with
g, denoted f g is the function from A to C defined by
f g  x   f  g  x 
To find ( f ◦g)(a) we first apply the function g to a to obtain
g(a) and then we apply the function f to the result g(a) to
obtain ( f ◦g)(a)
Composition
Input (Domain)
Function g(x)
2
1. The domain of f ◦g is the domain of g
2. The range of g is the domain of f
3. The range of f ◦g is the image of the range of
g with respect to the function f
Function f(x)
Function f◦g(x)
Output (Range)
Composition
3
Composition
4
Example: If
f  x   x and g  x   2 x  1,
2
then
f  g  x     2 x  1
and
g  f  x   2x  1
2
2
Composition Questions
1
Example 2: Let g be the function from the set {a,b,c} to itself such that
g(a) = b, g(b) = c, and g(c) = a. Let f be the function from the set {a,b,c}
to the set {1,2,3} such that f(a) = 3, f(b) = 2, and f(c) = 1.
What is the composition of f and g, and what is the composition of g
and f.
Solution: The composition f∘g is defined by
f g  a   f  g  a    f  b   2.
f g  b   f  g  b    f  c   1.
f g  c   f  g  c    f  a   3.
Note that g∘f is not defined, because the range of f {1, 2, 3} is
not a subset of the domain of g {a, b, c}.
Composition Questions
2
Example 2: Let f and g be functions from the set
of integers to the set of integers defined by
f  x   2x  3
and
g  x   3x  2.
What is the composition of f and g, and also the
composition of g and f ?
Solution:
CS 2305: Discrete Mathematics
for Computing I
Lecture 15
- KP Bhat
1
Composition
1
Definition: Let f: B→C, g: A→B. The composition of f with
g, denoted f g is the function from A to C defined by
f g  x   f  g  x 
To find ( f ◦g)(a) we first apply the function g to a to obtain
g(a) and then we apply the function f to the result g(a) to
obtain ( f ◦g)(a)
Composition
Input (Domain)
Function g(x)
2
1. The domain of f ◦g is the domain of g
2. The range of g is the domain of f
3. The range of f ◦g is the image of the range of
g with respect to the function f
Function f(x)
Function f◦g(x)
Output (Range)
sin◦sqrt(x) vs sqrt◦sin(x)
public static void main(String[] args) {
final double DEG_45 = Math.PI/4;
double fogx = Math.sin(Math.sqrt(DEG_45));
System.out.println("fogx: " + fogx);
fogx: 0.7746914034386123
gofx: 0.8408964152537145
double gofx = Math.sqrt(Math.sin(DEG_45));
System.out.println("gofx: " + gofx);
}
4
Graphs of Functions
Let f be a function from the set A to the set B. The
graph of the function f is the set of ordered pairs
 a, b  | a  A and f  a   b.
Graph of f(n) = 2n + 1
from Z to Z
Jump to long description
Graph of f(x) = x2
from Z to Z
Some Important Functions
The floor function, denoted
f  x    x
is the largest integer less than or equal to x.
The ceiling function, denoted
f  x    x
is the smallest integer greater than or equal to x
Example:
3.5  
3.5  
1.5  
1.5  
Floor and Ceiling Functions
1
Graph of (a) Floor and (b) Ceiling Functions
Floor and Ceiling Functions
2
Proving Properties of Functions
Example: Prove that x is a real number, then
⌊2x⌋= ⌊x⌋ + ⌊x + ½⌋
Solution: Let x = n + ε, where n is an integer and 0 ≤ ε< 1.
Case 1: ε < ½
•
•
•
•
⌊2x⌋ = ⌊2(n + ε)⌋ = ⌊2n + 2ε⌋ = 2n, since 2ε< 1.
⌊x⌋ = ⌊n + ε⌋ = n, since ε < ½
⌊x + ½⌋ = ⌊n + ε + ½⌋ Now ε < ½ so ε + ½ < ½ + ½ or ε + ½ < 1 ∴ ⌊x + ½⌋ = n
Hence, ⌊2x⌋ = 2n and ⌊x⌋ + ⌊x + ½⌋ = n + n = 2n.
Case 2: ε ≥ ½
•
•
⌊2x⌋ = ⌊2(n + ε)⌋ = ⌊2n + 2ε⌋ = 2n + 1, since 2ε ≥ 1.
⌊x⌋ = ⌊n + ε⌋ = n, since ε < 1
•
⌊x + ½⌋ = ⌊n + ε + ½⌋ Now ε ≥ ½ so ε + ½ ≥ ½ + ½ or ε + ½ ≥ 1 ∴ ⌊x + ½⌋ = n+1
•
Hence, ⌊2x⌋ = 2n + 1 and ⌊x⌋ + ⌊x + ½⌋ = n + (n + 1) = 2n + 1.
Factorial Function
Definition: f: N  Z+ , denoted by f(n) = n! is the
product of the first n positive integers when n is a
nonnegative integer.
f  n   1 2   n –1  n,
f  0   0!  1
Stirling’s Formula:
Examples:
f
f
f
f
1  1!  1
 2   2!  1 2  2
 6   6!  1 2  3  4  5  6  720
 20   2, 432,902, 008,176, 640, 000.
Partial Functions
Definition: A partial function f from a set A to a set B is an assignment
to each element a in a subset of A, called the domain of definition of f,
of a unique element b in B.
•
The sets A and B are called the domain and codomain of f, respectively.
•
We day that f is undefined for elements in A that are not in the domain of
definition of f.
•
When the domain of definition of f equals A, we say that f is a total function.
•
Domain of definition is equal to the domain
Example: f: Z  R where f(n) = √n is a partial
function where the domain of definition is the
set of nonnegative integers. Note that f is
undefined for negative integers.
Sequences and Summations
Section 2.4
Introduction
Sequences are ordered lists of elements.
• 1, 2, 3, 5, 8
• 1, 3, 9, 27, 81, …….
The notion of position is important in sequences.
It is the index at which a certain value appears in
the sequence
Sequences arise throughout mathematics,
computer science, and in many other disciplines,
ranging from botany to music.
Sequences
1
Definition: A sequence is a function from a subset of
the integers (usually either the set {0, 1, 2, 3, 4, …..}
or {1, 2, 3, 4, ….}) to a set S.
The notation an is used to denote the image of the
integer n. We can think of an as the equivalent of f(n)
where f is a function from {1,2,…..} to S. We call an a
term of the sequence.
Sequences
2
Example: Consider the sequence an where
1
an 
n
Terms of the sequence:
an   a1 , a2 , a3 ...
1 1 1
1, , ,
2 3 4
...
Geometric Progression
Definition: A geometric progression is a sequence of the
form:
where the initial term a and the common ratio r are real
numbers.
Examples :
1. Let a  1and r  1. Then :
bn   b0 , b1 , b2 , b3 , b4 ,...  1,  1, 1,  1, 1,...
2. Let a  2 and r  5. Then :
cn   c0 , c1 , c2 , c3 , c4 ,...  2, 10, 50, 250, 1250,...
3. Let a  6 and r  1/ 3. Then :
2 2 2 

d n   d0 , d1 , d 2 , d3 , d 4 ,...  6, 2, , , ,...
3 9 27 

Arithmetic Progression
Definition: A arithmetic progression is a sequence of the
form: a, a  d , a  2d ,..., a  nd ,...
where the initial term a and the common difference d are
real numbers.
Examples :
1. Let a  1and d  4 :
15,
 1,3,1,7,11,
1, 1,...
sn   s0 , s1 , s2 , s3 , s4 ,...  1,{-1,
 …}
2. Let a  7 and d  3 :
tn   t0 , t1 , t2 , t3 , t4 ,...  7, 4, 1,  2,  5,...
3. Let a  1and d  2 :
un   u0 , u1 , u2 , u3 , u4 ,...  1, 3, 5, 7, 9,...
Summations
Sum of the terms am , am  1,..., an
from the sequence an 
The notation:
n
a
j m
j or

n
j m
aj
or

m  j n
aj
represents
am  am 1    an
The variable j is called the index of summation. It
runs through all the integers starting with its lower
limit m and ending with its upper limit n.
Geometric Series
1
Sums of terms of geometric progressions
n 1

ar
a
n
r 1

j
ar   r  1

j 0
  n  1 a r  1

Proof: Let
n
Sn   ar j
j 0
n
rS n  r  ar
j
j 0
n
  ar
j 0
j 1
To compute Sn , first
multiply both sides of
the equality by r and
then manipulate the
resulting sum as follows:
Geometric Series
2
n
  ar j 1
From previous slide.
j 0
n 1
  ar k
Shifting the index of summation with k = j + 1.
k 1

 n k
   ar    ar n 1  a 
 k 0

Removing k = n + 1 term and
adding k = 0 term.
 Sn   ar n 1  a 
Substituting S for summation formula
rSn  Sn   ar n 1  a 
ar n 1  a
Sn 
r 1
n
if r ≠1
n
Sn   ar   a   n  1 a
j
j 0
j 0
if r = 1
Some Useful Summation Formulae
TABLE 2 Some Useful Summation Formulae.
Sum
Closed From
n
 ar  r  0 
k
k 0
n  n  1
2
n
k
k 1
n  n  1 2n  1
6
n
k2
k 1
n
k
3
k 1

 x ,| x |< 1
k
k 0

 kx
k 0
ar n 1  a
, r 1
r 1
k 1
,| x |< 1
n 2  n  1
4
1
1 x
1
1  x 
2
2
Geometric Series:
We just proved this.
Later we will
prove some
of these by
induction.
Proof in text
(requires calculus)
Algorithms
Chapter 3
22
Problems and Algorithms
The first step in solving many computational
problems is to precisely state the problem, using
the appropriate structures to specify the input
and the desired output.
We then solve the general problem by specifying
the steps of a procedure that takes a valid input
and produces the desired output. This
procedure is called an algorithm.
Algorithms
1
Abu Ja’far Mohammed
Ibin Musa Al-Khowarizmi
(780-850)
Definition: An algorithm is a finite set of precise instructions for
performing a computation or for solving a problem.
• Concept of algorithms has existed for ever but it
started becoming part of common parlance only after
the invention of the digital computer, in the 1940s
– Word did not appears in the Webster’s New World
Dictionary till 1957
• Origin of the word is disputed but many people now
believe that the word was derived from the name of
famous Persian mathematician al-Khwarizmi
– Lived in the 9th century and wrote a famous book that gave
algorithmic solutions to many mathematical problems
• The word “algebra” is derived from the title of that book!
Algorithms
2
Abu Ja’far Mohammed
Ibin Musa Al-Khowarizmi
(780-850)
Example: Describe an algorithm for finding the maximum value
in a finite sequence of integers.
Solution: Perform the following steps:
1.
Set the temporary maximum equal to the first integer in the
sequence.
2.
Compare the next integer in the sequence to the temporary
maximum.
•
If it is larger than the temporary maximum, set the temporary
maximum equal to this integer.
3.
Repeat the previous step if there are more integers. If not, stop.
4.
When the algorithm terminates, the temporary maximum is the
largest integer in the sequence.
Specifying Algorithms
Algorithms can be specified in different ways.
•
•
•
Human language
Flowchart
Pseudocode
Pseudocode is an intermediate step between an English language
description of the steps and a coding of these steps using a
programming language
•
employs constructs supported by a typical programming language
(especially looping, branching and functions/subroutines), together
with informal English statements
Programmers can use the description of an algorithm in
pseudocode to construct a program in a particular language.
Pseudocode helps us analyze the time required to solve a
problem using an algorithm, independent of the actual
programming language used to implement algorithm.
Properties of Algorithms
Input: An algorithm typically has input values from a specified set.
Output: The algorithm performs an action or produces the output
values from a specified set (the solution).
Correctness: An algorithm should produce the correct output
values for each set of input values.
Finiteness: An algorithm should produce the output after a finite
number of steps for any input.
Effectiveness: It must be possible to perform each step of the
algorithm correctly and in a finite amount of time.
Generality: The algorithm should work for all problems of the
desired form.
Finding the Maximum Element in a Finite
Sequence
The algorithm in pseudocode:
procedure max(a1, a2, …., an: integers)
max := a1
for i := 2 to n
if max < ai then max := ai
return max {max is the largest element}
Some Example Algorithm Problems
Three classes of problems will be studied in this
chapter.
1. Searching Problems: finding the position of a
particular element in a list.
2. Sorting problems: putting the elements of a list into
increasing order.
3. Optimization Problems: determining the optimal
value (maximum or minimum) of a particular quantity
over all possible inputs.
•
An algorithm that quickly produces good, but not necessarily
optimal solutions is called a heuristic. You will study them in
depth in more advanced CS courses.
Searching Problems
Definition: The general searching problem is to
locate an element x in the list of distinct elements
a1,a2,...,an, or determine that it is not in the list.
• The solution to a searching problem is the location of
the term in the list that equals x (that is, i is the
solution if x = ai) or 0 if x is not in the list.
• For example, a library might want to check to see if a
patron is on a list of those with overdue books before
allowing him/her to checkout another book.
• We will study two different searching algorithms:
linear search and binary search.
Linear Search Algorithm
The linear search algorithm locates an item in a list by examining elements in the
sequence one at a time, starting at the beginning.
•
First compare x with a1. If they are equal, return the position 1.
•
If not, try a2. If x = a2, return the position 2.
•
Keep going, and if no match is found when the entire list is scanned, return 0.
procedure linear search(x:integer,
a1, a2, …,an: distinct integers)
i := 1
while (i ≤ n and x ≠ ai)
i := i + 1
if i ≤ n then location := i
else location := 0
return location{location is the subscript of the term that
equals x, or is 0 if x is not found}
CS 2305: Discrete Mathematics
for Computing I
Lecture 16
- KP Bhat
1
Searching Problems
Definition: The general searching problem is to
locate an element x in the list of distinct elements
a1,a2,...,an, or determine that it is not in the list.
• The solution to a searching problem is the location of
the term in the list that equals x (that is, i is the
solution if x = ai) or 0 if x is not in the list.
• For example, a library might want to check to see if a
patron is on a list of those with overdue books before
allowing him/her to checkout another book.
• We will study two different searching algorithms:
linear search and binary search.
Linear Search Algorithm
The linear search algorithm locates an item in a list by examining elements in the
sequence one at a time, starting at the beginning.
•
First compare x with a1. If they are equal, return the position 1.
•
If not, try a2. If x = a2, return the position 2.
•
Keep going, and if no match is found when the entire list is scanned, return 0.
procedure linear search(x:integer,
a1, a2, …,an: distinct integers)
i := 1
while (i ≤ n and x ≠ ai)
i := i + 1
if i ≤ n then location := i
else location := 0
return location{location is the subscript of the term that
equals x, or is 0 if x is not found}
Binary Search
1
Assume the input is a list of items in increasing order.
The algorithm begins by comparing the element to be found with
the middle element.
•
If the middle element is lower, the search proceeds with the upper half of
the list.
•
If it is not lower, the search proceeds with the lower half of the list
(through the middle position).
Repeat this process until we have a list of size 1.
•
If the element we are looking for is equal to the element in the list, the
position is returned.
•
Otherwise, 0 is returned to indicate that the element was not found.
Binary search algorithm is much more efficient than linear search,
but it requires the items to be in sorted order.
Binary Search
2
Here is a description of the binary search algorithm in
pseudocode.
procedure binary search(x: integer, a1,a2,…, an: increasing integers)
i := 1 {i is the left endpoint of interval}
j := n {j is right endpoint of interval}
while i < j
m := ⌊(i + j)/2⌋
if x > am then i := m + 1
else j := m
if x = ai then location := i
else location := 0
return location{location is the subscript i of the term ai equal to x,
or 0 if x is not found}
Binary Search
3
• Starting with the entire range of values, the
algorithm successively narrows down the part of
the sequence being searched
• At any given stage only the terms from ai to aj are
under consideration
• The algorithm continues narrowing the part of
the sequence being searched until only one term
of the sequence remains. When this is done, a
comparison is made to see whether this term
equals the term we are looking for
Binary Search
4
Example: The steps taken by a binary search for 19 in the list:
1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22
1.
The list has 16 elements, so the midpoint is 8. The value in the 8th position is 10. Since 19 >
10, further search is restricted to positions 9 through 16.
1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22
2.
The midpoint of the list (positions 9 through 16) is now the 12th position with a value of 16.
Since 19 > 16, further search is restricted to the 13th position and above.
1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22
3.
The midpoint of the current list is now the 14th position with a value of 19. Since
19 ≯ 19, further search is restricted to the portion from the 13th through the 14th positions .
1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22
4.
The midpoint of the current list is now the 13th position with a value of 18.
Since 19> 18, search is restricted to the portion from the 14th position through the 14th.
1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22
5.
Now the list has a single element and the loop ends. Since 19=19, the location 14 is
returned.
Sorting
To sort the elements of a list is to put them in increasing order
(numerical order, alphabetic, and so on).
Sorting is an important problem because:
•
A nontrivial percentage of all computing resources are devoted to sorting different
kinds of lists, especially applications involving large databases of information that
need to be presented in a particular order (e.g., by customer, part number etc.).
•
An amazing number of fundamentally different algorithms have been invented for
sorting. Their relative advantages and disadvantages have been studied extensively.
•
Sorting algorithms are useful to illustrate the basic notions of computer science.
More than 100 sorting algorithms have been developed till date: this text
discusses the following:
•
binary, insertion, bubble, selection, merge, quick, and tournament
Bubble Sort
1
Bubble sort makes multiple passes through a list. Every pair of
elements that are found to be out of order are interchanged.
•
With each iteration the highest remaining number “sinks” to its correct
position to the right
https://www.youtube.com/watch?v=xli_FI7CuzA
procedure bubblesort(a1,…,an: real numbers
with n ≥ 2)
for i := 1 to n− 1
for j := 1 to n − i
if aj >aj+1 then interchange aj and aj+1
{a1,…, an is now in increasing order}
Bubble Sort
2
Example: Show the steps of bubble sort with 3 2 4 1 5
At the first pass the largest element has been put into the correct
position
At the end of the second pass, the 2nd largest element has been
put into the correct position.
In each subsequent pass, an additional element is put in the
correct position.
Bubble Sort
3
• Although bubble sort is considered to be an
extremely inefficient algorithm, it is possible
to slightly tweak the algorithm so as to give
bubble sort the ability to handle already
sorted lists very efficiently
Bubble Sort
4
procedure bubblesort(a1,…,an: real numbers with n ≥ 2)
swapped := true
for i := 1 to n− 1
if(not swapped)
then
break
endif
swapped := false
for j := 1 to n − i
if aj >aj+1
then
interchange aj and aj+1
swapped := true
endif
endfor
endfor
{a1,…, an is now in increasing order}
Selection Sort
1
• The selection sort begins by finding the least
element in the list. This element is moved to the
front. Then the least element among the
remaining elements is found and put into the
second position. This procedure is repeated until
the entire list has been sorted
– Section 3.1, Exercise 42
– https://www.youtube.com/watch?v=g-PGLbMth_g
• In general more efficient than bubble sort (lot
fewer swaps) unless the list is already sorted
Selection Sort
procedure selectionsort(a1, a2, …., an)
for i := 1 to n - 1
minspot := i
for j := i + 1 to n
if aj < aminspot then minspot := j
interchange aminspot and ai
{the list is now in order}
2
Insertion Sort
1
• Insertion sort begins with the 2nd element. It compares the 2nd
element with the 1st and puts it before the first if it is not larger.
• Next the 3rd element is put into the correct position among the first
3 elements.
• In general, in the jth step of the insertion sort, the jth element of the
list is inserted into the correct position in the list of the previously
sorted j − 1 elements.
• Linear search is used to find the correct position.
• Note that the array forms sorted and unsorted partitions
• https://www.youtube.com/watch?v=JU767SDMDvA
• An optimized version of the algorithm uses fewer swaps but instead
uses a temporary location to store the element that needs to be
assigned the correct location
– https://www.youtube.com/watch?v=pmDnM9gUxNc&t=25s
Insertion Sort
2
procedure insertion sort
(a1,…,an:
real numbers with n ≥ 2)
for j := 2 to n
i := 1
while aj > ai
i := i + 1
m := aj
for k := 0 to j − i − 1
aj-k := aj-k-1
ai := m
{Now a1,…,an is in increasing order}
Insertion Sort
3
Example: Show all the steps of insertion sort with the input:
3
2
4
1
5
i.
2 3
4
1
interchanged)
5
(first two positions are
ii.
2 3
4
its position)
1
5
(third element remains in
iii. 1 2
3
beginning)
4
5
(fourth is placed at
iv. 1 2
3
its position)
4
5
(fifth element remains in
Greedy Algorithms
Optimization problems minimize or maximize some parameter over all
possible inputs.
•
•
•
Finding a route between two cities with the smallest total mileage.
Determining how to encode messages using the fewest possible bits.
Finding the fiber links between network nodes using the least amount of fiber.
Optimization problems can often be solved using a greedy algorithm, which
makes the “best” choice at each step. Making the “best choice” at each step
does not necessarily produce an optimal solution to the overall problem, but
in many instances, it does.
After specifying what the “best choice” at each step is, we try to prove that
this approach always produces an optimal solution, or find a counterexample
to show that it does not.
•
This is a very important step because in some cases greedy algorithms yield a
very poor solution
Greedy Algorithms: Making Change
Example: Design a greedy algorithm for making change (in U.S.
money) of n cents with the following coins: quarters (25 cents),
dimes (10 cents), nickels (5 cents), and pennies (1 cent) , using
the least total number of coins.
Idea: At each step choose the coin with the largest possible value
that does not exceed the amount of change left.
1.
If n = 67 cents, first choose a quarter leaving 67−25 = 42 cents. Then
choose another quarter leaving 42 −25 = 17 cents
2.
Then choose 1 dime, leaving 17 − 10 = 7 cents.
3.
Choose 1 nickel, leaving 7 – 5 = 2 cents.
4.
Choose a penny, leaving one cent. Choose
another penny leaving 0 cents.
Greedy Change-Making Algorithm
Solution: Greedy change-making algorithm for n cents. The
algorithm works with any coin denominations c1, c2, …,cr .
procedure change(c1, c2, …, cr: values of coins, where c1> c2> … > cr ;
n: a positive integer)
for i := 1 to r
di := 0 {di counts the coins of denomination ci}
while n ≥ ci
di := di + 1 {add a coin of denomination ci}
n = n - ci
{di counts the coins ci}
For the example of U.S. currency, we may have quarters, dimes,
nickels and pennies, with c1 = 25, c2 = 10, c3 = 5, and c4 = 1.
1
Proving Optimality for U.S. Coins
1
Show that the change making algorithm for U.S. coins is optimal.
Lemma 1: If n is a positive integer, then n cents in change using
quarters, dimes, nickels, and pennies, using the fewest coins
possible has at most 2 dimes, 1 nickel, 4 pennies, and cannot have
2 dimes and a nickel. The total amount of change in dimes,
nickels, and pennies must not exceed 24 cents.
Proof: By contradiction
•
If we had 3 dimes, we could replace them with a quarter and a nickel.
•
If we had 2 nickels, we could replace them with 1 dime.
•
If we had 5 pennies, we could replace them with a nickel.
•
If we had 2 dimes and 1 nickel, we could replace them with a quarter.
• ∴ 24 cents (2 dimes and 4 pennies) is the most money we can have in
dimes, nickels, and pennies when we make change using the fewest
number of coins
Proving Optimality for U.S. Coins
2
Theorem: The greedy change-making algorithm for U.S. coins produces
change using the fewest coins possible.
Proof: By contradiction.
1.
Assume there is a positive integer n such that change can be made for n
cents using quarters, dimes, nickels, and pennies, with a fewer total
number of coins than given by the algorithm.
2.
Then, q̍ ≤ q where q̍ is the number of quarters used in this optimal way
and q is the number of quarters in the greedy algorithm’s solution. But
this is not possible by Lemma 1, since the value of the coins other than
quarters can not be greater than 24 cents.
3.
Similarly, by Lemma 1, the two algorithms must have the same number of
dimes, nickels, and quarters.
•
•
The value of the coins other than
quarters and dimes can not be greater
than 9 cents.
The value of the coins other than
quarters, dimes and nickels can not be
greater than 4 cents.
Greedy Change-Making Algorithm
2
Optimality depends on the denominations available.
For U.S. coins, optimality still holds if we add half
dollar coins (50 cents) and dollar coins (100 cents).
But if we allow only quarters (25 cents), dimes (10
cents), and pennies (1 cent), the algorithm no longer
produces the minimum number of coins.
• Consider the example of 30 cents. The optimal number of
coins is 3, i.e., 3 dimes. However the algorithm uses 6
coins—1 quarter and 5 pennies
Greedy Scheduling
1
Example: We have a group of proposed talks with start and end
times. Construct a greedy algorithm to schedule as many as
possible in a lecture hall, under the following assumptions:
•
•
•
•
•
When a talk starts, it continues till the end.
No two talks can occur at the same time.
A talk can begin at the same time that another ends.
Once we have selected some of the talks, we cannot add a talk which is
incompatible with those already selected because it overlaps at least one
of these previously selected talks.
How should we make the “best choice” at each step of the algorithm? That
is, which talk do we pick ?
• The talk that starts earliest among those compatible with already chosen talks?
• The talk that is shortest among those already compatible?
• The talk that ends earliest among those compatible with already chosen talks?
Greedy Scheduling
2
Picking the earliest talk doesn’t work.
For example by picking Talk 1 first we select only 1
talk, although we can easily accommodate 2 talks if
we first select Talk 2 first
Greedy Scheduling
3
Picking the shortest talk doesn’t work either
For example by picking Talk 2 first we can select only 1 talk, although we
can easily accommodate Talk 1 and Talk 3, since they do not overlap
However greedy scheduling works by picking the talk that ends soonest.
The algorithm is specified on the next slide.
Greedy Scheduling algorithm
Solution: At each step, choose the talks with the earliest ending
time among the talks compatible with those selected.
procedure schedule(s1 ≤ s2 ≤ … ≤ sn : start times, e1 ≤ e2 ≤ … ≤ en : end
times)
sort talks by finish time and reorder so that e1 ≤ e2 ≤ … ≤ en
S := ∅
for j := 1 to n
if talk j is compatible with S then
S := S ∪ {talk j}
A talk is incompatible with those
already selected if it overlaps at
least one of them
return S [ S is the set of talks scheduled]
Will be proven correct by induction in Chapter 5.
CS 2305: Discrete Mathematics
for Computing I
Lecture 17
- KP Bhat
1
Unsolvable Problems
• There is a class of problems for which it can be
shown that no algorithm exists (i.e. cannot be
solved using any procedure)
• Typically these problems require a yes/no answer,
but where there cannot possibly be any algorithm
that always gives the correct answer
• The most famous problem in this category is The
Halting Problem
• For more examples:
– https://en.wikipedia.org/wiki/List_of_undecidable_pr
oblems
Halting Problem
1
Example: Can we develop a procedure that takes as input a
computer program along with its input and determines
whether the program will eventually halt with that input.
Solution: Proof by contradiction.
Assume that there is such a procedure and call it H(P,I). The
procedure H(P,I) takes as input a program P and I the input
to program P.
• H outputs “halt” if it is the case that P will stop when run with
input I.
• Otherwise, H outputs “loops forever.”
Halting Problem
2
Since a program is a string of characters, we can
invoke H using P both as the program as well as
its input i.e. H(P,P). In this case H tells us
whether or not any program P will halt when it
is given a copy of itself as input
Halting Problem
3
Construct a procedure K(P), whose input is the
output of Procedure H i.e. H(P, P). The procedure
K(P) works as follows:
• If H(P,P) outputs “loops forever” then K(P) halts.
• If H(P,P) outputs “halt” then K(P) goes into an infinite
loop.
Halting Problem
4
Now let us invoke H using K both as the program as well as its input, and feed the
output to K.
There are two possibilities for the output of Procedure H
Possibility 1: H(K, K) = “loops forever”
•
In this case Procedure K halts. This contradicts the claim of H(K, K) that Program K will loop forever
on input K
Possibility 2: H(K, K) = “halts”
•
In this case Procedure K loops for ever. This contradicts the claim of H(K, K) that Program K halts
on input K
Therefore, there can not be a procedure that can decide whether or not an arbitrary
program halts. ∴ The halting problem is unsolvable.
The Growth of Functions
Section 3.2
The Growth of Functions
In both computer science and in mathematics, there are many
times when we care about how fast a function grows.
In computer science, we want to understand how quickly an
algorithm can solve a problem as the size of the input grows.
•
We can compare the efficiency of two different algorithms for solving the
same problem.
•
We can also determine whether it is practical to use a particular
algorithm as the input grows.
Factors Affecting the Running Time of
a Program
1. The size of the input to the program
2. The time complexity of the algorithm underlying the
program
3. The quality of code generated by the compiler used to
create the object program
4. The nature and speed of the instructions on the
machine used to execute the program
• Factors #1 and #2 are the dominant factors
• Factors #3 and #4 are, to a very large extent,
independent of factors #1 and #2 and they can be
approximated by some constant
Big-O Notation
1
Definition: Let f and g be functions from the set of integers
or the set of real numbers to the set of real numbers. We say
that f(x) is O(g(x)) if there are constants C and k such that
| f  x |  C | g  x |
whenever x > k. (illustration on next slide)
This is read as “f(x) is big-O of g(x)” or “g asymptotically
dominates f.”
The constants C and k are called witnesses to the relationship
f(x) is O(g(x)). Only one pair of witnesses is needed.
Note: f (x) is O(g(x)) says that f (x) grows slower than some fixed multiple of
g(x) as x grows without bound.
Illustration of Big-O Notation
1
f  x is O  g  x )
Jump to long description
Some Important Points about Big-O
Notation
If one pair of witnesses is found, then there are infinitely many
pairs. We can always make the k or the C larger and still maintain
the inequality | f  x |  C | g  x |
.
•
Any pair C’ and k’ where C < C’ and k < k’ is also a pair of witnesses since
whenever x > k’ > k.
| f  x |  C | g  x  C ' | g  x |
Sometimes in the literature you may see “f(x) = O(g(x))” instead of “f(x) is
O(g(x)).”
•
Strictly speaking this is an abuse of the equals sign since this notation
does not represent a genuine equality.
•
It is ok to write f(x) ∊ O(g(x)), because O(g(x)) represents the set of
functions that are O(g(x)).
Usually, we will drop the absolute value sign since we will mostly
deal with functions that take on positive values.
Using the Definition of Big-O Notation
Example: Show that f(x) = x2 + 2x + 1 is O(x2)
.
Solution: Consider x > 1, x < x2 and 1 < x2
0  x2  2 x  1  x2  2 x2  x2  4 x2
• Can take C = 4 and k = 1 as witnesses to show that
(see
graph on next slide)
Alternatively, when x > 2, we have 2x ≤ x2 and 1 <
0  x 2  2 x  1  x 2  x 2  x 2  3x 2
x2. Hence,
when x > 2.
• Can take C = 3 and k = 2 as witnesses instead.
1
Illustration of Big-O Notation
 
f  x  x +2 x  1 is O x
2
2
2
Big-O Notation
2
Both f  x  x 2 +2 x  1 and g  x  x 2
are such that f  x is O  g  x and g  x is O  f  x .
We say that the two functions are of the same order.
If f  x is O  g  x  
numbers, then
and h(x) is larger than g(x) for all positive real
f  x  is O  h  x   .
Note that if
| f  x |  C |g  x  |
for x > k and if
| h  x |  |g  x  |
for all x, then
| f  x  |  C |h  x  |
if x > k. Hence,
f  x  is O  h  x   .
For many applications, the goal is to select the function g(x) in
O(g(x)) as small as possible (up to multiplication by a constant, of
course).
Using the Definition of Big-O Notation
Example: Show that 7x2 is O(x3).
Solution: When x > 7, 7x2 < x3. Take C =1 and k = 7 as
witnesses to establish that 7x2 is O(x3).
Example: Show that n2 is not O(n).
Solution: Suppose there are constants C and k for which n2
≤ Cn, whenever n > k.
We can always choose C to be greater than k because if that is not so we
can always choose another constant D > k. Now ∀n > k
n2 ≤ Cn
Then by dividing both sides by n, we have
n ≤ C must hold for all n > k.
For n > k, n ≤ C
∴k≤C
A contradiction!
But we chose C to be greater than k
2
Big-O Estimates for Polynomials
1
• A mathematical expression of the following type
is known as a polynomial of degree n
– anxn + an-1xn-1 + an-2xn-2 + ……. + a2x2 + a1x + a0
• more accurately a polynomial in one variable of degree n
• In a polynomial of degree n, the leading term (i.e.
anxn) dominates its growth
• We will prove the result that a polynomial of
degree n is O(xn)
– We will use the triangle inequality as the lemma for
this proof
• if x and y are real numbers, then |x| + |y| ≥ |x + y|
• Section 1.8, Exercise 9 (proof by cases)
Big-O Estimates for Polynomials
Theorem: Let f  x  an x n  an 1 x n 1 
where a0 , a1 ,
 a1 x  a0
, an are real numbers with an ≠0. Then f(x) is O(xn).
Proof: | f  x |  | an x n  an 1 x n 1 
Assuming x > 1
2
 a1 x1  a0 |
 | an | x n  | an 1 | x n 1 

 x n | an |  | an 1 | / x 
 x n | an |  | an 1 | 
Uses triangle inequality,
an exercise in Section 1.8.
 | a1 | x1  | a0 |
 | a1 | / x n 1  | a0 | / x n
 | a1 |  | a0 |

Take C | an |  | an1 |   | a0 | and k = 1. Then f(x) is O(xn).
The leading term anxn of a polynomial dominates its growth.
Big-O Estimates for some Important
Functions
1
Example: Use big-O notation to estimate the sum of
the first n positive integers.
Solution: 1  2 
1 2 
n  nn
 
n  n2
 n is O n 2 taking C  1 and k  1.
Example: Use big-O notation to estimate the factorial
function f  n  n !  1  2   n .
Solution:
n!  1  2 
n  nn
 
 n  nn
n ! is O n n taking C  1 and k  1.
Big-O Estimates for some Important
Functions
2
Example: Use big-O notation to estimate log n!
n
n
!

n
(previous slide)
Solution: Given that
then log(n!) ≤ log(nn)
log(n!) ≤ nlog(n)
Hence, log(n!) is O(n∙log(n)) taking C = 1 and k = 1.
Display of Growth of Functions
Factorial
Logarithmic Scale
Exponential
Polynomial
Linearithmic
Linear
Logarithmic
Constant
Note the difference in behavior of functions as n gets
larger
CS 2305: Discrete Mathematics
for Computing I
Lecture 18
- KP Bhat
1
Display of Growth of Functions
Factorial
Logarithmic Scale
Exponential
Polynomial
Linearithmic
Linear
Logarithmic
Constant
Note the difference in behavior of functions as n gets
larger
Background Information: Sum and
Product of Functions
• (f1 + f2)(x) = f1(x) + f2(x)
• (f1f2)(x) = f1(x) * f2(x)
Examples: Let f1 and f2 be functions from R to R
such that f1(x) = x2 and f2(x) = x − x2.
(f1 + f2)(x) = f1(x) + f2(x) = x2 + (x − x2) = x
(f1f2)(x) = f1(x) * f2(x) = x2(x − x2) = x3 –x4
3
Combinations of Functions
1
• The number of steps used by a computer to
solve a problem with input of a specified size,
using an algorithm that is composed of
subprocedures, is the sum of the number of
steps used by the subprocedures
• The big-O estimate of the algorithm is the sum
of the big-O estimates of the subprocedures of
the algorithm
Combinations of Functions
If f1 (x) is O(g1(x)) and f2 (x) is O(g2(x)) then
( f1 + f2 )(x) is O(max(|g1(x) |,|g2(x) |)).
• See next slide for proof
If f1 (x) and f2 (x) are both O(g(x)) then
( f1 + f2 )(x) is O(g(x)).
• See text for argument
If f1 (x) is O(g1(x)) and f2 (x) is O(g2(x)) then
( f1 f2 )(x) is O(g1(x)g2(x)).
• See text for argument
2
Combinations of Functions
If f1  x is O  g1  x   and f 2  x  is O  g 2  x   then
 f1  f 2   x
•

3

is O max |g1  x  |,| g 2  x  | .
By the definition of big-O notation, there are constants C1,C2 ,k1,k2 such that
| f1  x  C1 | g1  x  | when x  k1 and f 2  x   C2 | g 2  x  | when x  k2 .
|  f1  f 2   x  |  | f1  x   f 2  x  |
by the triangle inequality |a + b| | a |  | b|
 | f1  x |  | f 2  x  |
| f1  x |  | f 2  x  |  C1 | g1  x  |  C 2 | g 2  x  |
 C1 | g  x  |  C 2 | g  x  | where g  x  max | g1  x  |,| g 2  x  |
  C1  C2  | g  x |
 C | g  x |
•
where C = C1  C2
Therefore |  f1  f 2   x  |  C | g  x  | whenever x > k, where k = max(k1,k2).
Combinations of Functions
4
Example: Give a big-O estimate for
f (n) = 3n log(n!) + (n2 + 3) log n, where n is a positive integer.
Solution:
( f1 f2 )(x) is O(g1(x)g2(x))
3n is O(n); log(n!) is O(nlog n)
∴ 3n log(n!) is O(n2log n)
(n2 + 3) is O(n2)
∴ (n2 + 3) log n is O(n2log n)
( f1 + f2 )(x) is O(max(|g1(x)|,|g2(x)|))
∴ = 3n log(n!) + (n2 + 3) log n is O(n2log n)
7
Big-Omega Notation
1
Definition: Let f and g be functions from the set of integers or
the set of real numbers to the set of real
numbers. We say that f  x is   g  x 
if there are constants C and k such that
| f  x |  C | g  x |
when x > k.
We say that “f(x) is big-Omega of g(x).”
Big-O gives an upper bound on the growth of a function, while
Big-Omega gives a lower bound. Big-Omega tells us that a
function grows at least as fast as another.
Big-Omega Notation
2
Example: Show that f  x  8 x3  5 x 2  7 is
  g  x  where g  x   x 3 .
Solution: f  x  8 x3  5 x 2  7
positive real numbers x.
 8 x 3 for all
Big-Theta Notation
1
Definition: Let f and g be functions from the set of integers or
the set of real numbers to the set of real numbers. The function
f  x is   g  x   if f  x  is O  g  x   and f  x  is   g  x   .
We say that “f is big-Theta of g(x)” and also that “f(x) is of order
g(x)” and also that “f(x) and g(x) are of the same order.”
f  x  is   g  x   if and only if there exists constants C1 , C2
and k such that C1g(x) < f(x) < C2 g(x) if x > k. This follows from
the definitions of big-O and big-Omega.
Big-Theta Notation
2
Example: Show that the sum of the first n positive integers is Θ(n2).
Solution:
We have already shown that f(n) is O(n2).
To show that f(n) is Ω(n2), we need a positive constant C such that f(n) >
Cn2 for sufficiently large n. Summing only the terms greater than n/2 we
obtain the inequality
1 2 


 n   n / 2   n / 2  1 
  n / 2   n / 2 


  n / 2
 n   n / 2  1  n / 2
  n / 2 n / 2  n / 4
2
n
Sum of n terms is greater than sum of
(n - ⌈n/2⌉ +1) terms
Replacing each of the (n - ⌈n/2⌉ +1) terms in
the range ⌈n/2⌉ through n by ⌈n/2⌉
Inequality holds for both even (e.g. n = 2k) and
odd cases (e.g. n = 2k + 1)
Hence, f(n) is Ω(n2), and we can conclude that f(n) is Θ(n2).
Big-Theta Notation
3
 
2
2
f
x

3
x

8
x
log
x
is

x
.


Example: Show that
Solution:
Part 1:
For x >1, log x is +ve and log x < x
8x log x < 8x2
3x2 + 8x log x < 11x2
∴ 3x2 + 8x log x is O(x2)
Part 2:
3x2 + 8x log x > x2
∴ 3x2 + 8x log x is Ω(x2)
Hence 3x2 + 8x log x is Θ(x2)
Relating O, Ω and Θ
1
• If f(x) is O(g(x)) then f is of order at most g
– g is the upper-bound for f
• If f(x) is Ω(g(x)) then f is of order at least g
– g is the lower-bound for f
• If f(x) is Θ(g(x)) then f is of order g
– f is bounded both above and below by some multiple of g
f(x) is O(g(x))
f(x) is Ω(g(x))
f(x) is Θ(g(x))
Relating O, Ω and Θ
2
f(n) is O(g(n))
f(n) is Ω(g(n))
f(n) is Θ(g(n))
Big-Theta Notation
When
4
f  x  is   g  x   it must also be the case that
g  x is   f  x   .
Note that f  x is   g  x  if and only if it is the cases
that f  x is O  g  x   and g  x  is O  f  x   .
Sometimes writers are careless and write as if big-O
notation has the same meaning as big-Theta.
Big-Theta Estimates for Polynomials
Theorem: Let f  x  an x n  an 1 x n 1   a1 x  a0
where a0 , a1 , , an are real numbers with an ≠0.
Then f(x) is of order xn (or Θ(xn)).
Example:
The polynomial f  x  8 x 5  5 x 2  10 is order of x5 (or Θ(x5)).
The polynomial f  x  8 x199  7 x100  x 99  5 x 2 25
is order of x199 (or Θ(x199) ).
1
Big-Theta Estimates for Polynomials
2
Proof
17
Complexity of Algorithms
Short Overview
The Complexity of Algorithms
Given an algorithm, how efficient is this algorithm for solving a
problem given input of a particular size? To answer this question,
we ask:
•
How much time does this algorithm use to solve a problem?
•
How much computer memory does this algorithm use to solve a
problem?
When we analyze the time the algorithm uses to solve the
problem given input of a particular size, we are studying the time
complexity of the algorithm.
When we analyze the computer memory the algorithm uses to
solve the problem given input of a particular size, we are
studying the space complexity of the algorithm.
Space Complexity
To analyze the space complexity we often consider the “extra”
memory needed i.e. not counting the memory needed to store the
input itself (e.g. pointers, keys, scratch buffers for ex-situ processing
etc.).
A big focus in space complexity is analyzing the memory requirements
for data structures employed for processing the data. For recursive
algorithms we also need to analyze the recursion stack space.
Time Complexity
To analyze the time complexity of algorithms, we determine the
number of operations, such as comparisons and arithmetic operations
(addition, multiplication, etc.). We can estimate the time a computer
may actually use to solve a problem using the amount of time required
to do basic operations.
We focus on the worst-case time complexity of an algorithm. This
provides an upper bound on the number of operations an algorithm
uses to solve a problem with input of a particular size.
It is usually much more difficult to determine the average case time
complexity of an algorithm. This is the average number of operations
an algorithm uses to solve a problem over all inputs of a particular size.
Complexity Analysis of Algorithms
Example: Describe the time complexity of the algorithm for
finding the maximum element in a finite sequence.
procedure max(a1, a2, …., an: integers)
max := a1
for i := 2 to n
if max < ai then max := ai
return max{max is the largest element}
Solution: Count the number of comparisons.
•
The max < ai comparison is made n − 1 times.
•
Each time i is incremented, a test is made to see if i ≤ n.
•
One last comparison determines that i > n.
•
Exactly 2(n − 1) + 1 = 2n − 1 comparisons are made.
Hence, the time complexity of the algorithm is Θ(n).
Worst-Case Complexity of Linear Search
Example: Determine the time complexity of the linear search algorithm.
procedure linear search(x:integer,
a1, a2, …,an: distinct integers)
i := 1
while (i ≤ n and x ≠ ai)
i := i + 1
if i ≤ n then location := i
else location := 0
return location{location is the subscript of the term that equals x, or is 0 if x is not found}
Solution: Count the number of comparisons.
•
•
•
At each step two comparisons are made; i ≤ n and x ≠ ai .
To end the loop, one comparison i ≤ n is made.
After the loop, one more i ≤ n comparison is made.
In the worst case (x is not on the list), 2n + 1 comparisons are made and then
an additional comparison is used to exit the loop. So, in the worst case 2n + 2
comparisons are made. Hence, the complexity is Θ(n).
Worst-Case Complexity of Binary Search
Example: Describe the time complexity of binary search in terms of the number of
comparisons used.
procedure binary search(x: integer, a1,a2,…, an: increasing integers)
i := 1 {i is the left endpoint of interval}
j := n {j is right endpoint of interval}
while i < j
m := ⌊(i + j)/2⌋
if x > am then i := m + 1
else j := m
if x = ai then location := i
else location := 0
return location{location is the subscript i of the term ai equal to x, or 0 if x is not found}
Solution: Assume (for simplicity) n = 2k elements. Note that k = log n.
•
•
•
•
•
Two comparisons are made at each stage; i < j, and x > am .
At the first iteration the size of the list is 2k and after the first iteration it is 2k-1. Then 2k-2 and
so on until the size of the list is 21 = 2.
At the last step, a comparison tells us that the size of the list is the size is 20 = 1 and the
element is compared with the single remaining element.
Hence, at most 2k + 2 = 2 log n + 2 comparisons are made.
Therefore, the time complexity is Θ (log n), better than linear search.
Worst-Case Complexity of Bubble Sort
Example: What is the worst-case complexity of bubble sort in
terms of the number of comparisons made?
procedure bubblesort(a1,…,an: real numbers
with n ≥ 2)
for i := 1 to n− 1
for j := 1 to n − i
if aj >aj+1 then interchange aj and aj+1
{a1,…, an is now in increasing order}
Solution: A sequence of n−1 passes is made through the list. On
each pass n − i comparisons are made.
n  n  1
n

1

n

2


2

1

   
2
The worst-case complexity of bubble sort is Θ(n2) since
n  n  1 1 2 1
 n  n.
2
2
2
CS 2305: Discrete Mathematics
for Computing I
Lecture 19
- KP Bhat
1
Worst-Case Complexity of Insertion Sort
Example: What is the worst-case complexity of insertion sort in
terms of the number of comparisons made?
Solution: The insertion sort inserts the jth
element into the correct position among
the first j − 1 elements that have already
been put into the correct order.
In the worst case, j comparisons are
required to insert the jth element into
the correct position.
The total number of comparisons
are
2 3
n
n  n  1
1
2
Therefore the complexity is Θ(n2).
procedure insertion sort(a1,…,an:
real numbers with n ≥ 2)
for j := 2 to n
i := 1
while aj > ai
i := i + 1
m := aj
for k := 0 to j − i − 1
aj-k := aj-k-1
ai := m
Time Complexity of Some Algorithms
•
•
•
•
•
•
Linear search: Θ(n)
Binary search: Θ(log n)
Bubble Sort: Θ(n2)
Selection Sort: Θ(n2)
Insertion Sort: Θ(n2)
Matrix Multiplication (n x n): Θ(n3)
Algorithmic Paradigms (1)
An algorithmic paradigm is a general approach based on a particular
concept for constructing algorithms to solve a variety of problems.
•
Brute-force algorithms
• naive approach for solving problems; does not take advantage of any special
structure of the problem or clever ideas
•
•
sequential search, bubble sort, selection sort, insertion sort, etc.
Greedy algorithms
• select the best choice at each step, instead of considering all sequences of steps;
focus on local optimization and not overall optimization
•
Divide-and-conquer algorithms
• divide a problem into one or more instances of the same problem of smaller size
and conquer the problem by using the solutions of the smaller problems to find a
solution of the original problem
•
Quick sort, merge sort, fast matrix multiplication, etc.
Algorithmic Paradigms (2)
•
Dynamic programming
•
recursively breaks down a problem into simpler overlapping subproblems, and stores the
results of the subproblems in a table; computes the solution of the problem using the
solutions of the subproblems
•
•
Backtracking
•
performs an exhaustive search of all possible solutions; once it is known that no solution can
result from any further sequence of decisions, backtrack to a known point and work towards a
solution with another series of decisions
•
•
n-Queens, graph coloring, sum of subsets, etc.
Probabilistic algorithms
•
make random choices at one or more steps
•
•
cutting stock, matrix-chain multiplication, longest common subsequence, etc.
…
Monte Carlo algorithms, Las Vegas algorithms, Sherwood algorithms, etc.
Understanding the Complexity of
Algorithms
1
TABLE 1 Commonly Used Terminology for the Complexity
of Algorithms.
Complexity
Terminology
Θ(1)
Constant complexity
Θ(log n)
Logarithmic complexity
Θ(n)
Linear complexity
Θ(n log n)
Linearithmic complexity
Θ(nb)
Polynomial complexity
Θ(bn), where b > 1
Exponential complexity
Θ(n!)
Factorial complexity
Understanding the Complexity of
Algorithms
2
TABLE 2 The Computer Time Used by Algorithms.
Problem Size
n
Bit Operations Used
𝐥𝐨𝐠 n
n
n 𝐥𝐨𝐠 n
n2
2n
10
3  1011 s
1010 s
3  1010 s
109 s
108 s
102
7  1011 s
109 s
7  109 s
107 s
103
1.0  1010 s
108 s
1  107 s
105 s
104
1.3  1010 s
107 s
1  106 s
105
1.7  1010 s
106 s
2  105 s
106
2  1010 s
105 s
2  104 s
103 s
0.1 s
0.17 min
4  1011 yr
*
*
*
*
n!
3  107 s
*
*
*
*
*
Times of more than 10100 years are indicated with an *.
Complexity of Problems
Tractable Problem: There exists a polynomial time algorithm to
solve this problem. These problems are said to belong to the
Class P.
Intractable Problem: There does not exist a polynomial time
algorithm to solve this problem
Unsolvable Problem : No algorithm exists to solve this problem,
e.g., halting problem.
Class NP: Solution can be checked in polynomial time. But no
polynomial time algorithm has been found for finding a solution
to problems in this class.
NP Complete Class: If you find a polynomial time algorithm for
one member of the class, it can be used to solve all the problems
in the class.
P Versus NP Problem
P ≟ NP
Stephen
Cook
(Born 1939)
The P versus NP problem asks whether the class P = NP? Are there problems
whose solutions can be checked in polynomial time, but can not be solved in
polynomial time?
•
Note that just because no one has found a polynomial time algorithm is different
from showing that the problem can not be solved by a polynomial time algorithm.
If a polynomial time algorithm for any of the problems in the NP complete class
were found, then that algorithm could be used to obtain a polynomial time
algorithm for every problem in the NP complete class.
•
Satisfiability (in Section 1.3) is an NP complete problem.
It is generally believed that P≠NP since no one has been able to find a
polynomial time algorithm for any of the problems in the NP complete class.
The problem of P versus NP remains one of the most famous unsolved
problems in mathematics (including theoretical computer science). The Clay
Mathematics Institute has offered a prize of $1,000,000 for a solution.
Some Well Known NP Complete
Problems
•
•
•
•
•
•
•
•
•
•
•
Boolean satisfiability problem (SAT)
Knapsack problem
Hamiltonian path problem
Travelling salesman problem (decision version)
Subgraph isomorphism problem
Subset sum problem
Clique problem
Vertex cover problem
Independent set problem
Dominating set problem
Graph coloring problem
Number Theory and
Cryptography
Chapter 4
•
© 2019 McGraw-Hill Education. All rights reserved. Authorized only for instructor use in the classroom. No reproduction or further distribution permitted without the prior written consent of McGraw-Hill Education.
Background Information
Number theory is the part of mathematics devoted to the study of
the integers and their properties.
Number theory has many applications in computer science
including cryptography, pseudorandom number generation, error
detection and error correction codes etc.
Many computer applications of number theory are based on the
notion of divisibility and primality of integers.
Divisibility and Modular
Arithmetic
Section 4.1
Division
1
Definition: If a and b are integers with a ≠ 0, then a
divides b if there exists an integer c such that b = ac.
• When a divides b we say that a is a factor or divisor of b
and that b is a multiple of a.
• The notation a | b denotes that a divides b.
• If a | b, then b/a is an integer.
• If a does not divide b, we write a ∤ b.
For example: 3 ∤ 7 and 3 | 12.
Division
2
• a | b is a Boolean function
– returns True of False
• a | b is true if
b ÷ a = qquotient and 0remainder ∴b = aq
• a | b is false (i.e. a ∤ b) if
b ÷ a = qquotient and r(≠0)remainder ∴b = aq + r
Properties of Divisibility
Theorem 1: Let a, b, and c be integers, where a ≠0.
i.
If a | b and a | c, then a | (b + c);
ii.
If a | b, then a | bc for all integers c;
iii.
If a | b and b | c, then a | c.
Proof: (i) Suppose a | b and a | c, then it follows that there are
integers s and t with b = as and c = at. Hence,
b + c = as + at = a(s + t). Hence, a | (b + c)
(Exercises 3 and 4 ask for proofs of parts (ii) and (iii).)
Corollary: If a, b, and c be integers, where a ≠0, such that a | b and
a | c, then a | mb + nc whenever m and n are integers.
Division Algorithm
1
When an integer is divided by a positive integer, there is a quotient and a
remainder. This is traditionally called the “Division Algorithm,” but is really a
theorem.
Division Algorithm: If a is an integer and d a positive integer, then there are
unique integers q and r, with 0 ≤ r < d, such that a = dq + r (proved in Section 5.2).
•
d is called the divisor.
•
a is called the dividend.
•
q is called the quotient.
•
r is called the remainder.
Definitions of Functions
div and mod
q = a div d
r = a mod d
+ve and –ve Remainders
• 18/5 = 3 remainder 3
– 3 more than the current
bucket boundary
• 18/5 = 4 remainder -2
– 2 less than the next
bucket boundary
• From this perspective
division will either yield
a remainder of 0 or it
will yield a +ve and a –ve
remainder
• We will only consider the
+ve remainder
18
Division Algorithm
2
Examples:
– What are the quotient and remainder when 101 is divided
by 11?
• Solution:
– Quotient = 101 div 11 = 9
– Remainder = 101 mod 11 = 2
– What are the quotient and remainder when −11 is divided
by 3?
• Solution
– Quotient = -11 div 3 = -4
– Remainder = -11 mod 3 = 1
Remember we only
consider the +ve
remainder
q = a div d
r = a mod d
Procedure for +ve Remainder
• Given dividend a and divisor d
– Step 1: First find the quotient
•
𝑎
q=⌊ ⌋
𝑑
– Remember that ⌊-x⌋ = -⌈x⌉
– Step 2: Find the +ve remainder
• r = a – d*q
• For -11 divided by 3
– Here a = -11 and d = 3
−11
⌋
3
• q=⌊
11
3
= - ⌈ ⌉ = -4
• r = -11 – 3(-4) = -11 + 12 = 1
Notes:
We only consider +ve divisors
If the dividend is +ve, you use your
elementary school math approach
If the dividend is –ve, the elementary
school math approach will give a –ve
remainder, which we don’t want. In that
case you use this approach
20
Congruence Relation
1
• Two integers a and b are said to be congruent modulo
m [represented as a≡b (mod m)] if a mod m = b mod m
– Same remainder when they are divided by the positive
integer m
– 18 ≡ 53 (mod 7)
– “18 is congruent to 53 modulo 7”
This forms the basis
for the formal
definition of
congruence
Let a≡b (mod m)
Then a and b can be represented as:
a = q1m + r ; b = q2m + r
Clearly (a – b) = (q1m + r) – (q2m + r) = (q1 – q2)m
∴ m | (a-b)
Congruence Relation
2
Definition: If a and b are integers and m is a positive integer, then a
is congruent to b modulo m if m divides a − b.
•
The notation a ≡ b (mod m) says that a is congruent to b modulo m.
•
We say that a ≡ b (mod m) is a congruence and that m is its modulus.
•
Two integers are congruent mod m if and only if they have the same
remainder when divided by m.
•
If a is not congruent to b modulo m, we write a ≢ b (mod m)
Example: Determine whether 17 is congruent to 5 modulo 6 and
whether 24 and 14 are congruent modulo 6.
Solution:
• 17 ≡ 5 (mod 6) because 6 divides 17 − 5 = 12.
• 24 ≢ 14 (mod 6) since 24 − 14 = 10 is not divisible by 6.
More on Congruences
Theorem 4: Let m be a positive integer. The
integers a and b are congruent modulo m if and
only if there is an integer k such that a = b + km.
Proof:
• If a ≡ b (mod m), then (by the definition of
congruence) m | a − b. Hence, there is an integer k
such that a − b = km and equivalently a = b + km.
• Conversely, if there is an integer k such that a = b +
km, then km = a − b. Hence, m | a − b and a ≡ b
(mod m).
CS 2305: Discrete Mathematics
for Computing I
Lecture 20
- KP Bhat
1
Congruence Relation
1
• Two integers a and b are said to be congruent modulo
m [represented as a≡b (mod m)] if a mod m = b mod m
– Same remainder when they are divided by the positive
integer m
– 18 ≡ 53 (mod 7)
– “18 is congruent to 53 modulo 7”
This forms the basis
for the formal
definition of
congruence
Let a≡b (mod m)
Then a and b can be represented as:
a = q1m + r ; b = q2m + r
Clearly (a – b) = (q1m + r) – (q2m + r) = (q1 – q2)m
∴ m | (a-b)
Congruence Relation
2
Definition: If a and b are integers and m is a positive integer, then a
is congruent to b modulo m if m divides a − b.
•
The notation a ≡ b (mod m) says that a is congruent to b modulo m.
•
We say that a ≡ b (mod m) is a congruence and that m is its modulus.
•
Two integers are congruent mod m if and only if they have the same
remainder when divided by m.
•
If a is not congruent to b modulo m, we write a ≢ b (mod m)
Example: Determine whether 17 is congruent to 5 modulo 6 and
whether 24 and 14 are congruent modulo 6.
Solution:
• 17 ≡ 5 (mod 6) because 6 divides 17 − 5 = 12.
• 24 ≢ 14 (mod 6) since 24 − 14 = 10 is not divisible by 6.
More on Congruences
Theorem 4: Let m be a positive integer. The
integers a and b are congruent modulo m if and
only if there is an integer k such that a = b + km.
Proof:
• If a ≡ b (mod m), then (by the definition of
congruence) m | a − b. Hence, there is an integer k
such that a − b = km and equivalently a = b + km.
• Conversely, if there is an integer k such that a = b +
km, then km = a − b. Hence, m | a − b and a ≡ b
(mod m).
Note on proofs
•
You can derive most of the proofs in this section using elementary school
concepts
– Only beware of the –ve remainder problem with the –ve dividend
•
•
In many of the proofs you will start by considering the dividend (say a), the
divisor (say m), the quotient (say q) and the remainder (say r)
Keep in mind the following:
– a=q*m+r
– m | (a-r)
•
Given m | x then
– x=k*m
•
Given a≡b (mod m) then
– a mod m = b mod m (i.e same remainder on division)
– m | (a – b)
– a = b + km
Congruence Class
• The set of all integers congruent to a mod m is
called the congruence class of a modulo m
– For example the congruence class for 3 mod 4 is
{…,−5,−1, 3, 7, 11,…}
Congruences of Sums and Products
Theorem 5: Let m be a positive integer. If a ≡ b (mod m) and c ≡
d (mod m), then a + c ≡ b + d (mod m) and ac ≡ bd (mod m)
Proof:
•
Because a ≡ b (mod m) and c ≡ d (mod m), by Theorem 4 there are
integers s and t with b = a + sm and d = c + tm.
•
Therefore,
Theorem 4
a ≡ b (mod m) ↔ a = b + km
• b + d = (a + sm) + (c + tm) = (a + c) + m(s + t) and
• b d = (a + sm) (c + tm) = ac + m(at + cs + stm).
•
Hence, a + c ≡ b + d (mod m) and ac ≡ bd (mod m).
Example: Because 7 ≡ 2 (mod 5) and 11 ≡ 1 (mod 5) , it follows
from Theorem 5 that
18 = 7 + 11 ≡ 2 + 1 = 3 (mod 5)
77 = 7 ∙ 11 ≡ 2 ∙ 1 = 2 (mod 5)
Algebraic Manipulation of Congruences
Multiplying both sides of a valid congruence by an integer preserves
validity.
• If a ≡ b (mod m) holds then c∙a ≡ c∙b (mod m), where c is any integer,
holds by Theorem 5 with d = c.
Adding an integer to both sides of a valid congruence preserves validity.
• If a ≡ b (mod m) holds then c + a ≡ c + b (mod m), where c is any
integer, holds by Theorem 5 with d = c.
Dividing a congruence by an integer does not always produce a valid
congruence.
• Theorem 7 Section 4.3 provides the conditions when division is ok.
Example: The congruence 14≡ 8 (mod 6) holds. But dividing both sides by
2 does not produce a valid congruence since 14/2 = 7 and 8/2 = 4, but
7≢4 (mod 6).
Computing the mod m Function of
Products and Sums
1
Corollary: Let m be a positive integer and let a and b be integers. Then
(a + b) (mod m) = ((a mod m) + (b mod m)) mod m
and
ab mod m = ((a mod m) (b mod m)) mod m
Proof:
Step 1: Prove that a ≡ (a mod m) (mod m)
By definition we know that a ≡ b (mod m) if m | (a – b)
Let a mod m = r
Then a = mq + r and r = a – mq, for some quotient q
a – r = a – (a – mq) = mq
Clearly m | (a – r) so a ≡ r (mod m)
∴ a ≡ (a mod m) (mod m)
Similarly b ≡ (b mod m) (mod m)
Computing the mod m Function of
Products and Sums
2
Proof (Cont’d):
Step 2: Use the Congruence Addition and Multiplication
Theorem (Theorem 5)
If a ≡ b (mod m) and c ≡ d (mod m), then
i. a + c ≡ b + d (mod m)
ii. ac ≡ bd (mod m).
a + b ≡ ((a mod m) + (b mod m)) (mod m)
ab ≡ ((a mod m)(b mod m)) (mod m)
Step 3: Apply Theorem 3
a ≡ b (mod m) if and only if
a mod m = b mod m
(a + b)(mod m) = ((a mod m) + (b mod m)) (mod m)
ab mod m = ((a mod m)(b mod m)) (mod m)
Arithmetic Modulo m
Definitions: Let Zm be the set of nonnegative integers less
than m: {0,1, …., m−1}
• The operation +m is defined as a +m b = (a + b) mod m. This is
addition modulo m.
• The operation ∙m is defined as a ∙m b = (a ∙ b) mod m. This is
multiplication modulo m.
• Using these operations is said to be doing arithmetic modulo m.
Example: Find 7 +11 9 and 7 ∙11 9.
Solution: Using the definitions above:
• 7 +11 9 = (7 + 9) mod 11 = 16 mod 11 = 5
• 7 ∙11 9 = (7 ∙ 9) mod 11 = 63 mod 11 = 8
Applications of Congruences
Section 4.5
Background Information
• Congruences have many applications in discrete
mathematics, computer science, and many other
disciplines
• We will discuss three applications
– the use of congruences to assign memory locations to
data records
– the generation of pseudorandom numbers
– check digits
• Congruences also play an extremely important
role in cryptography
Hashing Functions
1
Definition: A hashing function h assigns memory location h(k) to a
record that has k as its key.
– A common hashing function is h(k) = k mod m, where m is the
number of memory locations.
– Because this hashing function is onto, all memory locations are
possible.
• The hashing function is not one-to-one as there are many more
possible keys than memory locations. When more than one record
is assigned to the same location, we say a collision occurs.
• When a collision takes place, a rehash strategy is executed
– For example, collision can be resolved by assigning the record to the
first free location.
– In this case we can use a linear probing function:
h(k,i) = (h(k) + i) mod m, where i runs from 0 to m − 1.
– There are many other implementations for the rehash strategy
(quadratic probing, double hashing etc.)
• These will be covered in advanced CS courses
Hashing Functions
Example: Let h(k) = k mod 111.
This hashing function assigns the
records of customers with social
security numbers as keys to
memory locations in the
following manner:
h(064212848) = 064212848 mod 111 = 14
h(037149212) = 037149212 mod 111 = 65
h(107405723) = 107405723 mod 111 =
14, but since location 14 is already
occupied, the record is assigned to the
next available position, which is 15.
2
Pseudorandom Numbers
1
Randomly chosen numbers are needed for many purposes, including
computer simulations.
Pseudorandom numbers are not truly random since they are generated by
systematic methods but they strive for some desirable properties of random
numbers like uniformity and independence
The linear congruential method is one commonly used procedure for
generating pseudorandom numbers.
Four integers are needed:
i.
ii.
iii.
iv.
the modulus m,
the multiplier a,
the increment c,
the seed x0,
with 2 ≤ a < m, 0 ≤ c < m, 0 ≤ x0 < m.
We generate a sequence of pseudorandom numbers {xn}, with
0 ≤ xn < m for all n, by successively using the recursively defined function
xn 1   axn  c  mod m.
Pseudorandom Numbers
2
Example: Find the sequence of pseudorandom numbers generated by the linear
congruential method with modulus m = 9, multiplier a = 7, increment c = 4, and seed
x0 = 3.
Solution: Compute the terms of the sequence by successively using the congruence
xn 1  (7 xn  4) mod 9, with x0  3.
x1  7 x0  4 mod 9  7  3  4 mod 9  25 mod 9  7,
x2  7 x1  4 mod 9  7  7  4 mod 9  53 mod 9  8,
x3  7 x2  4 mod 9  7  8  4 mod 9  60 mod 9  6,
x4  7 x3  4 mod 9  7  6  4 mod 9  46 mod 9  1,
x5  7 x4  4 mod 9  7 1  4 mod 9  11 mod 9  2,
x6  7 x5  4 mod 9  7  2  4 mod 9  18 mod 9  0,
x7  7 x6  4 mod 9  7  0  4 mod 9  4 mod 9  4,
x8  7 x7  4 mod 9  7  4  4 mod 9  32 mod 9  5,
x9  7 x8  4 mod 9  7  5  4 mod 9  39 mod 9  3.
The sequence generated is 3,7,8,6,1,2,0,4,5,3,7,8,6,1,2,0,4,5,3,…
It repeats after generating 9 terms.
Commonly, computers use a linear congruential generator with increment c = 0. This is
called a pure multiplicative generator. Such a generator with modulus 231 − 1 and
multiplier 75 = 16,807 generates 231 − 2 numbers before repeating.
Pseudorandom Numbers
3
• If psuedo-random numbers between 0 and 1 are needed, then the
generated numbers are divided by the modulus, xn /m
• Linear congruential generators provide a very efficient way to
generate pseudo-random numbers which are suitable for many
applications
• Unfortunately long pseudo-random number sequences do not
share some important statistical properties that true random
numbers have. Because of this, it is not advisable to use them for
some tasks, such as large simulations
Check Digits
• A common method of detecting errors in
strings of digits is to add an extra digit at the
end, which is evaluated using a function. If the
final digit is not correct, then the string is
assumed not to be correct
Parity Check Bits
•
•
•
•
Digital information is represented by bit string, split into blocks of a specified size
Before each block is stored or transmitted, an extra bit, called a parity check bit, is
appended to each block. The parity check bit xn+1 for the bit string x1x2…xn is defined by
– xn+1 = x1 + x2 +⋯+ xn mod 2
xn+1 is 0 if there are an even number of 1 bits in the block of n bits and it is 1 if there are
an odd number of 1 bits in the block of n bits
When we examine a string that includes a parity check bit, we know that there is an
error in it if the parity check bit is wrong. However, when the parity check bit is correct,
there still may be an error since a parity check can detect an odd number of errors in
the previous bits, but not an even number of errors
Example: Suppose we receive in a transmission the bit strings 01100101 and 11010110,
each ending with a parity check bit. Should we accept these bit strings as correct?
Solution:
The parity check bit of the first string is 1. Because 0 + 1 + 1 + 0 + 0 + 1 + 0 ≡ 1 (mod 2), the
parity check bit is correct.
The parity check bit of the second string is 0. We find that 1 + 1 + 0 + 1 + 0 + 1 + 1 ≡ 1 (mod
2), so the parity check is incorrect.
20
Check Digits: UPCs
Retail products are identified by their Universal Product Codes (UPCs). Usually these
have 12 decimal digits, the last one being the check digit. The check digit is determined
by the congruence:
3x1 + x2 + 3x3 + x4 + 3x5 + x6 + 3x7 + x8 + 3x9 + x10 + 3x11 + x12 ≡ 0 (mod 10).
Example:
Is 793573431042 a well-formed UPC code?
Solution:
The first 11 digits of the UPC are 79357343104. In this case the check digit should be:
3∙7 + 9 + 3∙3 + 5 + 3∙7 + 3 + 3∙4 + 3 + 3∙1 + 0 + 3∙4 + x12 ≡ 0 (mod 10)
98 + x12 ≡ 0 (mod 10)
The solution of this congruence (beyond the scope of this course) is
x12 ≡ 2 (mod 10)
So, the check digit is 2.
Hence 793573431042 is a well-formed UPC code
Check Digits:ISBNs
Books are identified by an International Standard Book Number (ISBN-10), a 10 digit code. The
first 9 digits identify the language, the publisher, and the book. The tenth digit is a check digit,
which is determined by the following congruence.
9
x10   ixi  mod 11 .
i 1
X is used
for the
digit 10.
Example:
Is 0072880082 a well-formed ISBN-10?
Solution:
The first 9 digits of the UPC are 007288008. In this case the check digit should be:
X10 ≡ 1∙0 + 2∙0 + 3∙7 + 4∙2 + 5∙8 + 6∙8 + 7∙ 0 + 8∙0 + 9∙8 (mod 11).
X10 ≡ 0 + 0 + 21 + 8 + 40 + 48 + 0 + 0 + 72 (mod 11).
X10 ≡ 189 (mod 11).
The solution of this congruence (beyond the scope of this course) is X10 = 2.
Hence 0072880082 is a well-formed ISBN-10 code
Integer Representations and
Algorithms
Section 4.2
Representations of Integers
In the modern world, we use decimal, or base 10,
notation to represent integers. For example when we
write 965, we mean 9∙102 + 6∙101 + 5∙100 .
We can represent numbers using any base b, where
b is a positive integer greater than 1.
The bases b = 2 (binary), b = 8 (octal) , and b = 16
(hexadecimal) are important for computing and
communications
The ancient Mayans used base 20 and the ancient
Babylonians used base 60.
Base b Representations
Theorem 1: Let b be a positive integer greater than 1. Then if n is a
positive integer, it can be expressed uniquely in the form:
n = akbk + ak-1bk-1 + …. + a1b + a0
where k is a nonnegative integer, a0,a1,…. ak are nonnegative
integers less than b, and ak≠ 0. The aj, j = 0,…,k are called the baseb digits of the representation.
(This theorem can be proved using mathematical induction, which
is discussed in Section 5.1.)
The representation of n given in Theorem 1 is called the base b
expansion of n and is denoted by (akak-1….a1a0)b.
We usually omit the subscript 10 for base 10 expansions.
Binary Expansions
Most computers represent integers and do arithmetic with
binary (base 2) expansions of integers. In these expansions,
the only digits used are 0 and 1.
Example: What is the decimal expansion of the integer that
has (1 0101 1111)2 as its binary expansion?
Solution:
8 7 6 54 3 2 10
(1 0101 1111)2 = 1∙28 + 0∙27 + 1∙26 + 0∙25 + 1∙24 + 1∙23 + 1∙22
+ 1∙21 + 1∙20 =351.
Example: What is the decimal expansion of the integer that
has (11011)2 as its binary expansion?
4 3 2 10
Solution: (11011)2 = 1 ∙24 + 1∙23 + 0∙22 + 1∙21 + 1∙20 =27.
Octal Expansions
The octal expansion (base 8) uses the digits
{0,1,2,3,4,5,6,7}.
Example: What is the decimal expansion of the
number with octal expansion (7016)8 ?
3 2 1 0
Solution: (7016)8 = 7∙83 + 0∙82 + 1∙81 + 6∙80 =3598
Example: What is the decimal expansion of the
number with octal expansion (111)8 ?
2 1 0
Solution: (111)8 = 1∙82 + 1∙81 + 1∙80 = 64 + 8 + 1 = 73
Hexadecimal Expansions
The hexadecimal expansion needs 16 digits, but our decimal
system provides only 10. So letters are used for the
additional symbols. The hexadecimal system uses the digits
{0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F}. The letters A through F
represent the decimal numbers 10 through 15.
Example: What is the decimal expansion of the number with
hexadecimal expansion (2AE0B)16 ?
Solution:
4 3 2 1 0
(2AE0B)16 = 2∙164 + 10∙163 + 14∙162 + 0∙161 + 11∙160
=175627
Example: What is the decimal expansion of the number with
hexadecimal expansion (E5)16 ?
1 0
Solution: (E5)16 = 14∙161 + 5∙160 = 224 + 5 = 229
Base Conversion
1
To construct the base b expansion of an integer n:
• Divide n by b to obtain a quotient and remainder.
n = bq0 + a0 0 ≤ a0 ≤ b
• The remainder, a0 , is the rightmost digit in the base b
expansion of n. Next, divide q0 by b.
q0 = bq1 + a1 0 ≤ a1 ≤ b
• The remainder, a1, is the second digit from the right in the
base b expansion of n.
• Continue by successively dividing the quotients by b,
obtaining the additional base b digits as the remainder.
The process terminates when the quotient is 0.
Base Conversion
2
procedure base b expansion(n, b: positive integers with b > 1)
q := n {initialize quotient to the number}
k := 0 {initialize digit to rightmost digit}
while (q ≠ 0)
ak := q mod b {set the kth digit from right}
q := q div b {update the quotient}
k := k + 1 {advance leftward to the next digit position}
return(ak-1 ,…, a1,a0){(ak-1 … a1a0)b is base b expansion of n}
Base Conversion
3
Example: Find the octal expansion of (12345)10
Solution: Successively dividing by 8 gives:
• 12345 = 8 ∙ 1543 + 1
•
1543 = 8 ∙ 192 + 7
•
192 = 8 ∙ 24 + 0
•
24 = 8 ∙ 3 + 0
•
3 =8∙0+3
The remainders
are
the
digits
from
right
to
left
4 3 2 1 0
yielding (30071)8.
Conversion Between Binary, Octal, and
Hexadecimal Expansions
1
Conversion between binary and octal and between
binary and hexadecimal expansions is extremely
easy because each octal digit corresponds to a
block of three binary digits and each hexadecimal
digit corresponds to a block of four binary digits,
Comparison of Hexadecimal, Octal, and
Binary Representations
TABLE 1 Hexadecimal, Octal, and Binary Representation of the Integers 0 through 15.
Decimal
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Hexadecimal
0
1
2
3
4
5
6
7
8
9
A
B
C
D
E
F
Octal
0
1
2
3
4
5
6
7
10
11
12
13
14
15
16
17
Binary
0
1
10
11
100
101
110
111
1000
1001
1010
1011
1100
1101
1110
1111
Initial 0s are not shown
Each octal digit corresponds to a block of 3 binary digits.
Each hexadecimal digit corresponds to a block of 4 binary
digits.
So, conversion between binary, octal, and hexadecimal is easy.
Conversion Between Binary, Octal, and
Hexadecimal Expansions
2
Example: Find the octal and hexadecimal expansions
of (11111010111100)2.
Solution:
• To convert to octal, we group the digits into blocks of
three (011 111 010 111 100)2, adding initial 0s as needed.
The blocks from left to right correspond to the digits
3,7,2,7, and 4. Hence, the solution is (37274)8.
• To convert to hexadecimal, we group the digits into blocks
of four (0011 1110 1011 1100)2, adding initial 0s as
needed. The blocks from left to right correspond to the
digits 3,E,B, and C. Hence, the solution is (3EBC)16.
Conversion Between Binary, Octal, and
Hexadecimal Expansions
3
Example: Find the binary expansions of (765)8 and
(A8D)16.
Solution:
• To convert (765)8 into binary notation, we replace each
octal digit by a block of three binary digits. These blocks
are 111, 110, and 101. Hence, (765)8 = (1 1111 0101)2.
• To convert (A8D)16 into binary notation, we replace each
hexadecimal digit by a block of four binary digits. These
blocks are 1010, 1000, and 1101. Hence, (A8D)16 = (1010
1000 1101)2.
CS 2305: Discrete Mathematics
for Computing I
Lecture 21
- KP Bhat
1
Induction and recursion
Chapter 5
•
© 2019 McGraw-Hill Education. All rights reserved. Authorized only for instructor use in the classroom. No reproduction or further distribution permitted without the prior written consent of McGraw-Hill Education.
Mathematical Induction
Section 5.1
Mathematical Induction
• A powerful proof technique that is used to
check conjectures about the outcomes of
processes that occur repeatedly and according
to definite patterns
• Based on the rule of inference that if P(1) and
∀k(P(k) → P(k+1)) are true for the domain of
positive integers, then ∀nP(n) is true
Climbing an Infinite Ladder
Suppose we have an infinite ladder:
1. We can reach the first rung of the ladder.
2. If we can reach a particular rung of the
ladder, then we can reach the next rung.
From (1), we can reach the first rung. Then by
applying (2), we can reach the second rung.
Applying (2) again, the third rung. And so on.
We can apply (2) any number of times to
reach any particular rung, no matter how
high up.
This example illustrates the reasoning
employed for proof by mathematical
induction.
Principle of Mathematical Induction
Principle of Mathematical Induction: To prove that the propositional function P(n) is true for
all positive integers n, we complete these steps:
•
Basis Step: Show that P(1) is true.
•
Inductive Step: Show that P(k) → P(k + 1) is true for all positive integers k.
To complete the inductive step, assuming the inductive hypothesis that P(k) holds for an
arbitrary integer k, show that P(k + 1) must be true.
Climbing an Infinite Ladder Example:
1.
2.
We can reach the first rung of the ladder.
If we can reach a particular rung of the ladder, then
we can reach the next rung.
•
BASIS STEP: By (1), we can reach rung 1.
•
INDUCTIVE STEP: Assume the inductive hypothesis that we can reach rung k. Then by
(2), we can reach rung k + 1.
Hence, P(k) → P(k + 1) is true for all positive integers k. We can reach every rung on the
ladder.
Note:- To prove that P(n) is true for all positive integers n ≥ a, in the Basis Step we show that
P(a) is true.
Important Points About Using
Mathematical Induction
Mathematical induction can be expressed as the rule of
inference
 P 1  k  P  k   P  k  1   n P  n ,
where the domain is the set of positive integers.
In a proof by mathematical induction, we don’t assume that P(k)
is true for all positive integers! We show that if we assume that
P(k) is true, then P(k + 1) must also be true. It is therefore not a
case of begging the question, or circular reasoning
Proofs by mathematical induction do not always start at the
integer 1. In such a case, the basis step begins at a starting point
b where b is an integer. We will see examples of this soon.
Remembering How Mathematical
Induction Works
Consider an infinite
sequence of dominoes,
labeled 1,2,3, …, where
each domino is standing.
We know that the first domino
is knocked down, i.e., P(1) is
true .
We also know that if
whenever the kth domino is
knocked over, it knocks over
the (k + 1)st domino, i.e, P(k)
→ P(k + 1) is true for all
positive integers k.
Let P(n) be the
proposition that the nth
domino is knocked over.
Hence, all dominos are knocked over.
P(n) is true for all positive integers n.
Jump to long description
Proving a Summation Formula by
Mathematical Induction
Example: Show that:
Solution:
n  n  1


2
i 1
n
• BASIS STEP: P(1) is true since 1(1 + 1)/2 = 1.
Note: Once we have this
conjecture, mathematical
induction can be used to
prove it correct.
• INDUCTIVE STEP: Assume true for P(k).
The inductive hypothesis is
Under this assumption,
k  k  1


2
i 1
k
• Hence, we have shown that P(k + 1) follows from P(k). Therefore
𝑛(𝑛+1)
the sum of the first n integers is
2
Conjecturing and Proving Correct a
Summation Formula
Example: Conjecture and prove correct a formula for the sum of the first n positive odd integers.
Then prove your conjecture.
Solution: We have: 1  1, 1  3  4, 1  3  5  9, 1  3  5  7  16, 1  3  5  7  9  25.
•
We can conjecture that the sum of the first n positive odd integers is n2,
1 3  5 
•
•
•
  2n  1  n 2 .
We prove the conjecture is proved correct with mathematical induction.
BASIS STEP: P(1) is true since 12 = 1.
INDUCTIVE STEP: P(k) → P(k + 1) for every positive integer k.
Assume the inductive hypothesis holds and then show that P(k + 1) holds has well.
Inductive Hypothesis: 1  3  5 
•
So, assuming P(k), it follows that:
1 3  5 
•
  2k  1  k 2
  2k  1   2k  1  1  3  5    2k  1    2k  1
 k 2   2k  1 by the inductive hypothesis 
 k 2  2k  1
2
  k  1
Hence, we have shown that P(k + 1) follows from P(k). Therefore the sum of the first n
positive odd integers is n2.
Proving Inequalities
1
Example: Use mathematical induction to prove that n < 2n
for all positive integers n.
Solution: Let P(n) be the proposition that n < 2n.
• BASIS STEP: P(1) is true since 1 < 21 = 2.
• INDUCTIVE STEP: Assume P(k) holds, i.e., k < 2k, for an
arbitrary positive integer k.
• Must show that P(k + 1) holds. Since by the inductive
hypothesis, k < 2k, it follows that:
k  1  2k  1  2k  2k  2  2k  2k 1
Therefore n < 2n holds for all positive integers n.
1 ≤ 2k
Proving Inequalities
2
Example: Use mathematical induction to prove that 2n < n!, for every
integer n ≥ 4.
Solution: Let P(n) be the proposition that 2n < n!.
• BASIS STEP: P(4) is true since 24 = 16 < 4! = 24.
• INDUCTIVE STEP: Assume P(k) holds, i.e., 2k < k! for an arbitrary
integer k ≥ 4. To show that P(k + 1) holds:
2k 1  2  2k
 2 k!
  k  1 k !
(by the inductive hypothesis )
k≥4∴2<k+1
  k  1 !
Therefore, 2n < n! holds, for every integer n ≥ 4.
Note that here the basis step is P(4), since P(0), P(1), P(2), and P(3) are all false.
Proving Divisibility Results
Example: Use mathematical induction to prove that n3 − n is divisible
by 3, for every positive integer n.
Solution: Let P(n) be the proposition that n3 − n is divisible by 3.
• BASIS STEP: P(1) is true since 13 − 1 = 0, which is divisible by 3.
• INDUCTIVE STEP: Assume P(k) holds, i.e., k3 − k is divisible by 3, for
an arbitrary positive integer k. To show that P(k + 1) follows:
 k  1   k  1   k 3  3k 2  3k  1   k  1
3

 
 k3  k  3 k2  k

Let a, b, and c be integers,
where a ≠ 0. Then
(i) if a ∣ b and a ∣ c, then a ∣ (b + c);
By the inductive hypothesis, the first term (k3 − k) is divisible by 3
and the second term is divisible by 3 since it is an integer
multiplied by 3. So by part (i) of Theorem 1 in Section 4.1 , (k + 1)3
− (k + 1) is divisible by 3.
Therefore, n3 − n is divisible by 3, for every integer positive integer n.
Number of Subsets of a Finite Set
1
Example: Use mathematical induction to show that if S is a
finite set with n elements, where n is a nonnegative integer,
then S has 2n subsets.
Solution: Let P(n) be the proposition that a set with n
elements has 2n subsets.
• Basis Step: P(0) is true, because the empty set has only
itself as a subset and 20 = 1.
• Inductive Step: Assume P(k) is true for an arbitrary
nonnegative integer k.
Number of Subsets of a Finite Set
2
Inductive Hypothesis: For an arbitrary nonnegative integer k, every set with k
elements has 2k subsets.
Let T be a set with k + 1 elements. Let us isolate a ∈ T, an arbitrary
element of T.
Clearly T = S ∪ {a}, where a ∈ T and S = T − {a}. Hence |S| = k.
For each subset X of S, there are exactly two subsets of T, i.e., X and X
∪ {a}.
By the inductive hypothesis S has 2k subsets. Since there are two
subsets of T for each subset of S, the number of subsets of T is 2
∙2k = 2k+1 .
Guidelines:
Mathematical Induction Proofs
Template for Proofs by Mathematical Induction
1.
2.
3.
4.
5.
6.
7.
8.
Express the statement that is to be proved in the form “for all n ≥ b, P(n)” for a fixed
integer b.
Write out the words “Basis Step.” Then show that P(b) is true, taking care that the
correct value of b is used. This completes the first part of the proof.
Write out the words “Inductive Step”.
State, and clearly identify, the inductive hypothesis, in the form “assume that P(k) is
true for an arbitrary fixed integer k ≥ b.”
State what needs to be proved under the assumption that the inductive hypothesis is
true. That is, write out what P(k + 1) says.
Prove the statement P(k + 1) making use the assumption P(k). Be sure that your proof
is valid for all integers k with k ≥ b, taking care that the proof works for small values of
k, including k = b.
Clearly identify the conclusion of the inductive step, such as by saying “this completes
the inductive step.”
After completing the basis step and the inductive step, state the conclusion, namely,
by mathematical induction, P(n) is true for all integers n with n ≥ b.
More Examples
• Self Study
– Mathematical induction examples in the textbook
on pages 339 to 345
An Incorrect “Proof” by Mathematical
Induction
Inductive Hypothesis: For every non-negative number n, 2n = 0.
BASIS STEP: For n = 0, 2n = 0
ATTEMPTED INDUCTIVE STEP: Let us assume that the result holds true for some value k
Consider k + 1. We can write k + 1 = i + j, where i and j are non-negative numbers
2(k + 1) = 2(i + j) = 2i + 2j = 0 + 0 = 0
∴ if the result holds for k, it holds for k+1 as well
There must be an error in this proof since the conclusion is absurd. But where is the error?
Answer: The problem comes with writing k + 1 = i + j. While this is true of most cases, it is
not true for k = 0. The number 0 + 1 = 1 cannot be written as the sum of 2 smaller nonnegative integers.
18
Strong Induction and
Well-Ordering
Section 5.2
Strong Induction
1
• Strong mathematical induction is similar to ordinary
mathematical induction in that there is a basis step and an
inductive step
• However, the inductive step shows that if P(j) is true for all
positive integers j not exceeding k, then P(k + 1) is true.
– For the inductive hypothesis we assume that P(j) is true for j = 1,
2,…, k.
• Taking the dominoes analogy one step further, imagine that
the dominoes are arranged in increasing order of weight
and a given domino requires the combined weight of all the
previous dominoes toppling over before it topples over as
well.
Strong Induction
2
Strong Induction: To prove that P(n) is true for all positive
integers n, where P(n) is a propositional function, complete
two steps:
• Basis Step: Verify that the proposition P(1) is true.
• Inductive Step: Show the conditional statement
 P 1  P  2 
 P  k    P  k  1
holds for all positive integers k.
Strong Induction is sometimes called
the second principle of mathematical
induction or complete induction.
Strong Induction and
the Infinite Ladder
Strong induction tells us that we can reach all rungs if:
1. We can reach the first rung of the ladder.
2. For every integer k, if we can reach the first k rungs,
then we can reach the (k + 1)st rung.
To conclude that we can reach every rung by strong
induction:
•
BASIS STEP: P(1) holds
•
INDUCTIVE STEP: Assume P(1) ∧ P(2) ∧∙∙∙ ∧ P(k)
holds for an arbitrary integer k, and show that
P(k + 1) must also hold.
We will have then shown by strong induction that for
every positive integer n, P(n) holds, i.e., we can
reach the nth rung of the ladder.
Strong Induction
3
• Strong induction is useful in some instances
where it is not apparent right away how to
formulate the basis step for mathematical
induction
• It can be proved that the principle of
mathematical induction and strong induction are
equivalent (Section 5.2, Exercise 42)
– any result that can be proved by mathematical
induction can be proved by strong induction and vice
versa
– neither proof technique is “stronger” than the other
Slightly Different Infinite Ladder Problem:
Attempted Proof Using Mathematical Induction
Example: Suppose we can reach the first and second rungs of an
infinite ladder, and we know that if we can reach a rung, then we
can reach two rungs higher. Prove that we can reach every rung.
Solution: Prove the result using mathematical induction.
• BASIS STEP: We can reach the first step.
• ATTEMPTED INDUCTIVE STEP: The inductive hypothesis is
that we can reach the kth rung.
However, assuming that we have reached the kth rung, we know
how to reach the (k+2)nd rung, but we don’t know how to reach
the (k + 1)st rung.
Hence we cannot conclude that if we have reached the kth rung
we can reach the reach the (k + 1)st rung as well.
Slightly Different Infinite Ladder Problem:
Proof Using Strong Induction
Example: Suppose we can reach the first and second rungs of an
infinite ladder, and we know that if we can reach a rung, then we
can reach two rungs higher. Prove that we can reach every rung.
Solution: Prove the result using strong induction.
• BASIS STEP: We can reach the first step.
• INDUCTIVE STEP: The inductive hypothesis is that we can
reach the first k rungs, for any k ≥ 2.
Once we have reached the (k−1)st rung, we can reach two rungs higher i.e.
we can reach the (k+1)st rung.
• Hence, we can reach all rungs of the ladder.
CS 2305: Discrete Mathematics
for Computing I
Lecture 22
- KP Bhat
1
Strong Induction
1
• Strong mathematical induction is similar to ordinary
mathematical induction in that there is a basis step and an
inductive step
• However, the inductive step shows that if P(j) is true for all
positive integers j not exceeding k, then P(k + 1) is true.
– For the inductive hypothesis we assume that P(j) is true for j = 1,
2,…, k.
• Taking the dominoes analogy one step further, imagine that
the dominoes are arranged in increasing order of weight
and a given domino requires the combined weight of all the
previous dominoes toppling over before it topples over as
well.
Strong Induction and
the Infinite Ladder
Strong induction tells us that we can reach all rungs if:
1. We can reach the first rung of the ladder.
2. For every integer k, if we can reach the first k rungs,
then we can reach the (k + 1)st rung.
To conclude that we can reach every rung by strong
induction:
•
BASIS STEP: P(1) holds
•
INDUCTIVE STEP: Assume P(1) ∧ P(2) ∧∙∙∙ ∧ P(k)
holds for an arbitrary integer k, and show that
P(k + 1) must also hold.
We will have then shown by strong induction that for
every positive integer n, P(n) holds, i.e., we can
reach the nth rung of the ladder.
Strong Induction
3
• Strong induction is useful in some instances
where it is not apparent right away how to
formulate the basis step for mathematical
induction
• It can be proved that the principle of
mathematical induction and strong induction are
equivalent (Section 5.2, Exercise 42)
– any result that can be proved by mathematical
induction can be proved by strong induction and vice
versa
– neither proof technique is “stronger” than the other
Slightly Different Infinite Ladder Problem:
Proof Using Strong Induction
Example: Suppose we can reach the first and second rungs of an
infinite ladder, and we know that if we can reach a rung, then we
can reach two rungs higher. Prove that we can reach every rung.
Solution: Prove the result using strong induction.
• BASIS STEP: We can reach the first step.
• INDUCTIVE STEP: The inductive hypothesis is that we can
reach the first k rungs, for any k ≥ 2.
Once we have reached the (k−1)st rung, we can reach two rungs higher i.e.
we can reach the (k+1)st rung.
• Hence, we can reach all rungs of the ladder.
Which Form of Induction Should Be
Used?
• We can always use strong induction instead of mathematical
induction. But there is no reason to use it if it is simpler to use
mathematical induction
– you should use mathematical induction when it is straightforward to
prove that P(k) → P(k + 1) is true for all positive integers k.
• Use strong induction, and not mathematical induction, when
you see how to prove that P(k + 1) is true from the
assumption that P(j) is true for all positive integers j not
exceeding k, but you cannot see how to prove that P(k + 1)
follows from just P(k)
• However, as already mentioned, in theory any result that can
be proved by mathematical induction can be proved by strong
induction and vice versa
Proof that mathematical induction and
strong induction are equivalent
7
Completion of the proof of the
Fundamental Theorem of Arithmetic
Example: Show that if n is an integer greater than 1, then n can be written as the
product of primes.
Solution: Let P(n) be the proposition that n can be written as a product of primes.
•
BASIS STEP: P(2) is true since 2 itself is prime.
•
INDUCTIVE STEP: The inductive hypothesis is P(j) is true for all integers j with
2 ≤ j ≤ k. To show that P(k + 1) must be true under this assumption, two cases
need to be considered:
• If k + 1 is prime, then P(k + 1) is true.
• Otherwise, k + 1 is composite and can be written as the product of two
positive integers a and b with 2 ≤ a ≤ b < k + 1. By the inductive hypothesis a
and b can be written as the product of primes and therefore k + 1 can also be
written as the product of those primes.
Hence, it has been shown that every integer greater than 1 can be written as the
product of primes.
Proof using Strong Induction
Example: Prove that every amount of postage of 12 cents or more can be formed using just
4-cent and 5-cent stamps.
Solution: Let P(n) be the proposition that postage of n cents can be formed using 4-cent and
5-cent stamps.
BASIS STEP: P(12), P(13), P(14), and P(15) hold.
•
•
•
•
P(12) uses three 4-cent stamps.
P(13) uses two 4-cent stamps and one 5-cent stamp.
P(14) uses one 4-cent stamp and two 5-cent stamps.
P(15) uses three 5-cent stamps.
INDUCTIVE STEP: The inductive hypothesis states that P(j) holds for 12 ≤ j ≤ k, where k ≥ 15.
• k -3 ≥ 12
• So we can form postage of k – 3 cents using just 4 cent and 5 cent stamps
• Now if we add a 4-cent stamp to the postage for k − 3 cents, we can form postage for
k – 3 + 4 = k + 1 cents
We have shown that if the inductive hypothesis is true, then P(k + 1) is also true. This
completes the inductive step.
Because we have completed the basis step and the inductive step, we conclude that every
postage of n cents, where n is at least 12, can be formed using 4-cent and 5-cent stamps.
Another Proof by Strong Induction
Conjecture: Given a sequence defined as follows:
a0 = 0, a1 = 1, and an = 3an−1 − 2an−2 for n ≥ 2
Show that an = 2n − 1 for all n ∈ W, the set of whole numbers
Solution:
BASIS STEP:
P(0) is true since 20 – 1 = 0
P(1) is true since 21 – 1 = 1
INDUCTIVE STEP:
Let k ≥ 2 and assume that the statement is true for 0, 1, 2, ….k-1
Now ak = 3ak−1 − 2ak−2 = 3*(2k-1 -1) – 2*(2k-2 -1) = 3*2k-1 -3 – 2k-1 + 2
= 3*2k-1 – 2k-1 – 1 = (3 -1)*2k-1 – 1 = 2*2k-1 – 1 = 2k – 1
∴ the result holds for n = k as well
From the Basis Step and the Inductive Step we have proved that an = 2n − 1 for all n ∈
W
An Incorrect “Proof” by Strong
Induction
Inductive Hypothesis: For every natural number n, 2n = 2.
BASIS STEP: For n = 1, 21 = 2
ATTEMPTED INDUCTIVE STEP: Let us assume that the result holds true for all natural numbers n
≤k
2k+1 = 2k * 2 = 2k * (2k/2k-1) = 2 * (2/2) [By inductive hypothesis]
2k+1 = 2 * 1 = 2
∴ if the result holds for all natural numbers n ≤ k, it holds for k+1 as well
Where is the error?
Answer: Here the base case is n = 1. However, we need the induction hypothesis for both n = k
and n = k − 1, and in the case k = 1 the latter value is n = 1 − 1 = 0, which is out of range of the
induction hypothesis.
11
Recursive Algorithms
Introduction
Section 5.4.1
Recursive Algorithms
Definition: An algorithm is called recursive if it solves a problem by
reducing it to an instance of the same problem with smaller input.
For the algorithm to terminate, the instance of the problem must
eventually be reduced to some initial case for which the solution is
known.
A recursive function consists of two parts:
i.
A base case that is processed without recursion; and
ii.
A recursive case that reduces a particular case to one or more of
the smaller cases, thereby making progress toward eventually
reducing the problem all the way to the base case
Each successive invocation of the recursive function, other than the
base case, is a slightly simpler or smaller version of the previous
invocation
Recursive Factorial Algorithm
Example: Give a recursive algorithm for
computing n!, where n is a nonnegative integer.
Solution: Use the recursive definition of the
factorial function.
procedure factorial(n: nonnegative integer)
if n = 0 then return 1
else return n∙factorial (n − 1)
{output is n!}
Recursive Exponentiation Algorithm
Example: Give a recursive algorithm for
computing an, where a is a nonzero real number
and n is a nonnegative integer.
Solution: Use the recursive definition of an.
procedure power(a: nonzero real number, n: nonnegative
integer)
if n = 0 then return 1
else return a∙ power (a, n − 1)
{output is an}
Background: Fibonacci Series
• Fibonacci series are the numbers in the following
sequence:
– 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, ...
– The first two numbers are 0 and 1 and each
subsequent number in the series is equal to the sum
of the previous two numbers
• Note:- Some mathematicians consider the first
two numbers of the series to be 1, 1, since that is
how it was formulated by Fibonacci, when he was
trying to model the population growth of rabbits
Recursive Fibonacci Algorithm
procedure fibonacci(n: nonnegative integer)
if n ≤ 1 then return n
else return fibonacci(n-1) + fibonacci(n-2)
{output is nth Fibonacci number}
Recursive Binary Search Algorithm
Example: Construct a recursive version of a binary search
algorithm.
Solution: Assume we have a1,a2,…, an, an increasing sequence of
integers. Initially i is 1 and j is n. We are searching for x.
procedure binary search(i, j, x : integers, 1≤ i ≤ j ≤n)
m := ⌊(i + j)/2⌋
if x = am then
return m
else if (x < am and i < m) then
return binary search(i,m−1,x)
else if (x > am and j >m) then
return binary search(m+1,j,x)
else return 0
{output is location of x in a1, a2,…,an if it appears, otherwise 0}
Merge Sort
1
Merge Sort works by iteratively splitting a list (with an even
number of elements) into two sublists of equal length until
each sublist has one element.
Each sublist is represented by a balanced binary tree.
At each step a pair of sublists is successively merged into a
list with the elements in increasing order. The process ends
when all the sublists have been merged.
The succession of merged lists is represented by a binary
tree.
https://www.youtube.com/watch?v=4VqmGXwpLqc
https://www.youtube.com/watch?v=es2T6KY45cA
Merge Sort
2
Example: Use merge sort to put the list
8,2,4,6,9,7,10, 1, 5, 3
into increasing order.
Solution:
Jump to long description
Recursive Merge Sort
1
Example: Construct a recursive merge sort algorithm.
Solution: Begin with the list of n elements L.
procedure mergesort(L = a1, a2,…,an )
if n > 1 then
m := ⌊n/2⌋
L1 := a1, a2,…,am
L2 := am+1, am+2,…,an
L := merge(mergesort(L1), mergesort(L2 ))
{L is now sorted into elements in increasing order}
Recursive Merge Sort
2
Subroutine merge, which merges two sorted lists.
procedure merge(L1, L2 :sorted lists)
L := empty list
while L1 and L2 are both nonempty
remove smaller of first elements of L1 and L2 from its list;
put at the right end of L
if this removal makes one list empty
then remove all elements from the other list and append them to L
return L {L is the merged list with the elements in increasing order}
Complexity of Merge: Two sorted lists with m elements
and n elements can be merged into a sorted list using
no more than m + n − 1 comparisons.
CS 2305: Discrete Mathematics
for Computing I
Lecture 23
- KP Bhat
1
Recursion and Iteration (1)
Recursion
Iteration
1.
1. Set of instructions is executed
repeatedly, while a given
condition is satisfied
2. Less overhead; faster
3. Lower space complexity
4. Sometimes it is hard to follow
the change in execution
context from iteration to
iteration
2.
Function calls itself, until the
base case is executed, and the
recursion terminates
More overhead; slower
–
3.
Higher space complexity
–
4.
Context switching for each
recursive call
A new stack frame for each
invocation of the recursive call
Generally a recursive function is
very simple
–
e.g. the recursive algorithm for
the Towers of Hanoi problem is
very simple and intuitive
–
e.g. the iterative algorithm for
the Towers of Hanoi problem is
very complex
Recursion and Iteration (2)
• In theory any problem that can be solved by a recursive
algorithm can be solved by an iterative algorithm, and viceversa
• Some problems can be better understood and resolved
using recursion
– At a later time an iterative method may be sought, if
performance is an issue
• In reality some problems lend themselves naturally to easier solution
using one approach as compared to the other
• The easiest recursive algorithms to convert to iterative are
those involving tail recursion i.e. recursive algorithms in
which no statements are executed after the return from the
recursive call
Tail
Recursion
3
Iterative Algorithm for Computing
Factorial
procedure iterative_factorial(n: nonnegative integer)
int y :=1
for x := 1 to n
y=y*x
return y
{output is n!}
4
Iterative Algorithm for Computing
Fibonacci Numbers
procedure iterative_fibonacci(n: nonnegative integer)
if n = 0 then return 0
else
x := 0
y := 1
for i := 1 to n − 1
z := x + y
x := y
y := z
return y
{output is the nth Fibonacci number}
5
The Basics of Counting
Section 6.1
Overview
• Combinatorics, the study of arrangements of
objects, is an important part of discrete
mathematics
• This subject had its origin in the systematic study
of gambling games
• Combinatorics deals with ordered or unordered
arrangements of the objects of a set with or
without repetitions. These arrangements, called
permutations and combinations
• Combinatorics has many practical applications
Basic Counting Principles:
The Product Rule
1
The Product Rule: Suppose that a procedure can be broken down into a
sequence of two tasks. If there are n1 ways to do the first task and for
each of these ways of doing the first task, there are n2 ways to do the
second task, then there are n1n2 ways to do the procedure.
Generalized Form
If a procedure consists of k steps and
the first step can be performed in n1 ways,
the second step can be performed in n2 ways
…
the kth step can be performed in nk ways
then the entire procedure can be performed in n1n2 · · · nk ways
Product Rule: An Illustrative Example
•
•
•
•
•
Assume that a procedure consists of three steps s1, s2, s3
There are 5 ways to perform s1: designated as A, B, C, D, E
There are 4 ways to perform s2: designated as 1, 2, 3, 4
There are 3 ways to perform s3: designated as α, β, γ
By the Product Rule there are 5 X 4 X 3 = 60 ways to
perform the procedure, which are enumerated below
A-1-α, A-1-β, A-1-γ, A-2-α, A-2-β, A-2-γ, A-3-α, A-3-β, A-3-γ, A-4-α,
A-4-β, A-4-γ, B-1-α, B-1-β, B-1-γ, B-2-α, B-2-β, B-2-γ, B-3-α, B-3-β,
B-3-γ, B-4-α, B-4-β, B-4-γ, C-1-α, C-1-β, C-1-γ, C-2-α, C-2-β, C-2-γ,
C-3-α, C-3-β, C-3-γ, C-4-α, C-4-β, C-4-γ, D-1-α, D-1-β, D-1-γ, D-2-α,
D-2-β, D-2-γ, D-3-α, D-3-β, D-3-γ, D-4-α, D-4-β, D-4-γ, E-1-α, E-1-β,
E-1-γ, E-2-α, E-2-β, E-2-γ, E-3-α, E-3-β, E-3-γ, E-4-α, E-4-β, E-4-γ
Basic Counting Principles:
The Product Rule
2
Example: How many bit strings of length seven are there?
Solution: Since each of the seven bits is either a 0 or a 1 (i.e. 2
choices), the answer is 27 = 128.
Example: A new company with just two employees, Sanchez
and Patel, rents a floor of a building with 12 offices. How many
ways are there to assign different offices to these two
employees?
Solution: Assigning an office to Sanchez can be done in 12
ways, then assigning an office to Patel different from the office
assigned to Sanchez can be done in 11 ways. So there are 12 ⋅
11 = 132 ways to assign offices to these two employees.
The Product Rule
1
Example: How many different license plates can be
made if each plate contains a sequence of three
uppercase English letters followed by three digits?
Solution: By the product rule,
there are 26 ∙ 26 ∙ 26 ∙ 10 ∙ 10 ∙ 10 = 17,576,000
different possible license plates.
The Product Rule
2
Example: A multiple-choice test contains 10 questions.
There are four possible answers for each question. In
how many ways can a student answer the questions on
the test if the student answers every question?
Solution:
There are 4 ways to answer each question. By the
product rule, there are 4 ∙ 4 ∙ … ∙ 4 = 410 = 1,048,576
ways for the student to answer the questions.
Refresher on Functions
Definition: Let A and B be nonempty sets. A function
f from A to B, denoted f: A  B is an assignment of
each element of A to exactly one element of B. We
write f(a) = b if b is the unique element of B
assigned by the function f to the element a of A.
Counting Functions
Counting Functions: How many functions are there from a set
with m elements to a set with n elements?
Solution: Suppose the elements in the domain are
a1, a2,…, am. There are n ways to choose the value of a1, n ways to
choose the value of a2, etc. The product rule tells us that there are
n ∙ n ∙ ∙ ∙ n = nm such functions..
Counting One-to-One Functions: How many one-to-one
functions are there from a set with m elements to one with n
elements?
Solution: Suppose the elements in the domain are
a1, a2,…, am. There are n ways to choose the value of a1, n−1 ways
to choose the value of a2, … n−(k-1) ways to choose ak etc.
The product rule tells us that there are n(n−1) (n−2)∙∙∙(n−m +1)
such functions.
Telephone Numbering Plan
Example: The North American numbering plan (NANP) specifies that a telephone number
consists of 10 digits, consisting of a three-digit area code, a three-digit office code, and a
four-digit station code. There are some restrictions on the digits.
•
Let X denote a digit from 0 through 9.
•
Let N denote a digit from 2 through 9.
•
Let Y denote a digit that is 0 or 1.
•
In the old plan (in use in the 1960s) the format was NYX-NNX-XXXX.
•
In the new plan, the format is NXX-NXX-XXXX.
How many different telephone numbers are possible under the old plan and the new
plan?
Solution: Use the Product Rule.
•
There are 8 ∙2 ∙10 = 160 area codes with the format NYX.
•
There are 8 ∙10 ∙10 = 800 area codes with the format NXX.
•
There are 8 ∙8 ∙10 = 640 office codes with the format NNX.
•
There are 10 ∙10 ∙10 ∙10 = 10,000 station codes with the format XXXX.
Number of old plan telephone numbers: 160 ∙640 ∙10,000 = 1,024,000,000.
Number of new plan telephone numbers: 800 ∙800 ∙10,000 = 6,400,000,000.
Counting Subsets of a Finite Set
Counting Subsets of a Finite Set: Use the product rule to
show that the number of different subsets of a finite set S is
2|S|. (In Section 5.1, mathematical induction was used to
prove this same result.)
Solution: Let us assume that the elements of set S are a1, a2,
a3, …., an. Let us represent the subsets of S by bit strings of
length n where the ith bit of a bit string is 1 if ai belongs to the
corresponding subset and 0 if ai does not belong to the
corresponding subset.
By the product rule, there are 2n such bit strings, and
therefore 2n subsets.
Since n = |S|, there are 2|S| subsets.
DNA and Genomes
A gene is a segment of a DNA molecule that encodes a particular protein and the
entirety of genetic information of an organism is called its genome.
DNA molecules consist of two strands of blocks known as nucleotides. Each
nucleotide is composed of bases: adenine (A), cytosine (C), guanine (G), or
thymine (T).
The DNA of bacteria has between 105 and 107 links (one of the four bases).
Mammals have between 108 and 1010 links. So, by the product rule there are at
least 4105 different sequences of bases in the DNA of bacteria and 4108 different
sequences of bases in the DNA of mammals.
The human genome includes approximately 23,000 genes, each with 1,000 or
more links.
Biologists, mathematicians, and computer scientists all work on determining the
DNA sequence (genome) of different organisms.
Basic Counting Principles:
The Sum Rule
1
The Sum Rule: If a task can be done either in one of n1 ways
or in one of n2 ways, where none of the set of n1 ways is the
same as any of the n2 ways, then there are n1 + n2 ways to
do the task.
Generalized Form
Suppose that a task can be done in one of n1 ways, or in one
of n2 ways, … , or in one of nm ways, where none of the set of
ni ways of doing the task is the same as any of the set of nj
ways, for all pairs i and j with 1 ≤ i < j ≤ m. Then the number
of ways to do the task is n1 + n2 +⋯+ nm.
Sum Rule: An Illustrative Example
• Alternative 1
– There are 5 ways to perform a task: designated as A, B, C, D, E
• Alternative 2
– There are 4 ways to perform the task: designated as 1, 2, 3, 4
• Alternative 3
– There are 3 ways to perform the task: designated as α, β, γ
• By the Sum Rule the number of ways to perform the task is 5 + 4 + 3
= 12, as shown below
Alternative-1
A, B, C, D, E
Alternative-2
1, 2, 3, 4
Alternative-3
α, β, γ
Basic Counting Principles:
The Sum Rule
2
Example: The mathematics department must choose
either a student or a faculty member as a representative
for a university committee. How many choices are there
for this representative if there are 37 members of the
mathematics faculty and 83 mathematics majors and no
one is both a faculty member and a student.
Solution: By the sum rule it follows that there are
37 + 83 = 120 possible ways to pick a representative.
Rule of Thumb for Applying the
Product Rule or the Sum Rule
•
•
If we must make one choice and then another choice, the product rule applies
If we must make one choice or another choice, the sum rule applies
Alternatives exist
Example: There are 18 mathematics majors and 325 computer science majors at a
college.
a) In how many ways can two representatives be picked so that one is a mathematics
major and the other is a computer science major?
b) In how many ways can one representative be picked who is either a mathematics
major or a computer science major?
Solution: a) There are 18 choices to choose a mathematics major, and for each of
those choices we have 325 choices to choose a computer science major. Hence, by
the product rule, we can choose two representatives in 18 * 325 = 5850 ways
b) There are 18 ways to choose a mathematics major and 325 ways to choose a
computer science major. Since we can choose the representative from either major,
there are a total of 18 + 325 = 343 choices to pick the representative.
21
Combining the Sum and Product Rule
1
Many counting problems cannot be solved using just the sum rule or just the
product rule.
However, many complicated counting problems can be solved using both of
these rules in combination.
Example: Suppose statement labels in a programming language can be either a
single letter or a letter followed by a digit. Find the number of possible labels.
Solution:
Number of choices when the label is a single letter = 26
By product rule number of choices when the label is a single letter followed by
a digit = 26*10 = 260
∴ by the sum rule number of possible labels = 26 + 260= 286
Combining the Sum and Product Rule
2
Example: How many strings are there of lowercase letters of
length four or less, not counting the empty string?
Solution: Number of strings of length 1 = 26
Number of strings of length 2 = 262 = 676 (from the product
rule)
Number of strings of length 3 = 263 = 17576 (from the
product rule)
Number of strings of length 4 = 264 = 456976 (from the
product rule)
By the sum rule it follows that there are
26 + 676 + 17576 + 456976 = 475254 strings of lowercase
letters of length four or less, not counting the empty string
Counting Passwords
Problem: Each user on a computer system has a password, which is six to eight
characters long, where each character is an uppercase letter or a digit. Each
password must contain at least one digit. How many possible passwords are
there?
Solution: Let P be the total number of passwords, and let P6, P7, and P8 be the
passwords of length 6, 7, and 8.
•
By the sum rule P = P6 + P7 +P8.
•
To find each of P6, P7, and P8 , we find the number of passwords of the specified
length composed of letters and digits and subtract the number composed only of
letters. We find that:
P6 = 366 − 266 =2,176,782,336 − 308,915,776 =1,867,866,560.
P7 = 367 − 267 =
78,364,164,096 − 8,031,810,176 = 70,332,353,920.
P8 = 368 − 268 =
2,821,109,907,456 − 208,827,064,576 =2,612,282,842,880.
Consequently, P = P6 + P7 +P8 = 2,684,483,063,360.
CS 2305: Discrete Mathematics
for Computing I
Lecture 24
- KP Bhat
1
The Product Rule
Generalized Form
If a procedure consists of k steps and
the first step can be performed in n1 ways,
the second step can be performed in n2 ways
…
the kth step can be performed in nk ways
then the entire procedure can be performed in n1n2 · · · nk ways
Product Rule: An Illustrative Example
•
•
•
•
•
Assume that a procedure consists of three steps s1, s2, s3
There are 5 ways to perform s1: designated as A, B, C, D, E
There are 4 ways to perform s2: designated as 1, 2, 3, 4
There are 3 ways to perform s3: designated as α, β, γ
By the Product Rule there are 5 X 4 X 3 = 60 ways to
perform the procedure, which are enumerated below
A-1-α, A-1-β, A-1-γ, A-2-α, A-2-β, A-2-γ, A-3-α, A-3-β, A-3-γ, A-4-α,
A-4-β, A-4-γ, B-1-α, B-1-β, B-1-γ, B-2-α, B-2-β, B-2-γ, B-3-α, B-3-β,
B-3-γ, B-4-α, B-4-β, B-4-γ, C-1-α, C-1-β, C-1-γ, C-2-α, C-2-β, C-2-γ,
C-3-α, C-3-β, C-3-γ, C-4-α, C-4-β, C-4-γ, D-1-α, D-1-β, D-1-γ, D-2-α,
D-2-β, D-2-γ, D-3-α, D-3-β, D-3-γ, D-4-α, D-4-β, D-4-γ, E-1-α, E-1-β,
E-1-γ, E-2-α, E-2-β, E-2-γ, E-3-α, E-3-β, E-3-γ, E-4-α, E-4-β, E-4-γ
The Sum Rule
Generalized Form
Suppose that a task can be done in one of n1 ways, or in
one of n2 ways, … , or in one of nm ways, where none of
the set of ni ways of doing the task is the same as any of
the set of nj ways, for all pairs i and j with 1 ≤ i < j ≤ m.
Then the number of ways to do the task is n1 + n2 +⋯+
nm.
Sum Rule: An Illustrative Example
• Alternative 1
– There are 5 ways to perform a task: designated as A, B, C, D, E
• Alternative 2
– There are 4 ways to perform the task: designated as 1, 2, 3, 4
• Alternative 3
– There are 3 ways to perform the task: designated as α, β, γ
• By the Sum Rule the number of ways to perform the task is 5 + 4 + 3
= 12, as shown below
Alternative-1
A, B, C, D, E
Alternative-2
1, 2, 3, 4
Alternative-3
α, β, γ
Internet Addresses
Version 4 of the Internet Protocol (IPv4) uses 32 bits.
Class A Addresses: used for the largest networks, a 0,followed by a 7-bit netid
and a 24-bit hostid.
Class B Addresses: used for the medium-sized networks, a 10,followed by a
14-bit netid and a 16-bit hostid.
Class C Addresses: used for the smallest networks, a 110,followed by a 21-bit
netid and a 8-bit hostid.
•
•
•
Neither Class D nor Class E addresses are assigned as the address of a computer
on the internet. Only Classes A, B, and C are available.
1111111 is not available as the netid of a Class A network.
Hostids consisting of all 0s and all 1s are not available in any network.
Jump to long description
Counting Internet Addresses
1
Example: How many different IPv4 addresses are available for computers
on the internet?
Solution: Let x be the number of available addresses, and let xA, xB, and
xC denote the number of addresses for the respective classes.
By the sum rule x = xA, xB, and xC
•
xA:
There are 27 − 1 = 127 Class A netids, since netid 1111111 is unavailable
For each netid, there are 224 − 2 = 16,777,214 hostids, since hostids consisting of all 0s and all
1s are unavailable
By the product law the total number of Class A addresses is 127 * 16,777,214 =
2,130,706,178
•
xB:
There are 214 = 16,384 Class B netids
For each netid, there are 216 − 2 = 65,534 hostids, since hostids consisting of all 0s and all 1s
are unavailable
By the product law the total number of Class B addresses is 16,384 * 65,534 = 1,073,709,056
Counting Internet Addresses
•
2
xC:
There are 221 = 2,097,152 Class C netids
For each netid, there are 28 − 2 = 254 hostids, since hostids consisting of all 0s and all 1s are
unavailable
By the product law the total number of Class C addresses is 2,097,152 * 254 = 532,676,608
Hence, the total number of available IPv4 addresses is
x = xA + xB + Xc
= 2,130,706,178 + 1,073,709,056 + 532,676,608
= 3, 737,091,842.
Not Enough Today !!
The IPv6 protocol
solves the problem
of too few
addresses.
Basic Counting Principles:
Subtraction Rule
Subtraction Rule: If a task can be done either in
one of n1 ways or in one of n2 ways, then the total
number of ways to do the task is n1 + n2 minus the
number of ways to do the task that are common
to the two different ways.
Also known as, the principle of inclusion-exclusion:
| A  B | | A |  | B |  | A  B |
Generalized Principle of InclusionExclusion
•
For two sets
Notice the alternating
+ and - signs
– |A∪B|=|A|+|B| −|A∩B|
Singletons Pair
•
For three sets
– |A∪B∪C|=|A|+|B|+|C|−(|A∩B|+|A∩C|+|B∩C|) +|A∩B∩C|
Singletons
•
Pairs
Triple
For four sets
– |A∪B∪C∪D|=|A|+|B|+|C|+|D|−
Singletons
(|A∩B|+|A∩C|+|A∩D|+|B∩C|+|B∩D|+|C∩D|)+
Pairs
(|A∩B∩C|+|A∩B∩D|+|A∩C∩D|+|B∩C∩D|)−|A∩B∩C∩D|
Triple
Quadruple
Principle of Inclusion-Exclusion
• A computer company receives 350 applications from
college graduates for a job planning a line of new web
servers. Suppose that 220 of these applicants majored
in computer science, 147 majored in business, and 51
majored both in computer science and in business.
How many of these applicants majored neither in
computer science nor in business?
Counting Bit Strings
Example: How many bit strings of length eight either start with a
1 bit or end with the two bits 00?
Solution: Use the subtraction rule.
•
Number of bit strings of length eight
that start with a 1 bit: 27 = 128
• 1*27
•
Number of bit strings of length eight
that end with bits 00: 26 = 64
• 26*1
•
Number of bit strings of length eight
that start with a 1 bit and end with bits 00 : 25 = 32
• 1*25*1
Hence, the number is 128 + 64 − 32 = 160.
Counting Patterns
Example: How many seven digit numbers (where leading 0s are acceptable)
do not contain the substring 1234?
Solution: Using the product rule, there are 107 possible strings.
Let us count how many strings contain the substring “1234”.
Here are the choices:
1 2 3 4 _ _ _: 1 * 103 = 103 choices
_ 1 2 3 4 _ _: 10 * 1 * 102 = 103 choices
_ _ 1 2 3 4 _: 102 * 1 * 10 = 103 choices
_ _ _ 1 2 3 4: 103 * 1 = 103 choices
Use the subtraction rule, the count of seven digit numbers that do not
contain the substring 1234 is:
107 – 4* 103 = 9,996,000
Basic Counting Principles: Division Rule
1
Division Rule: There are n/d ways to do a task if it can be done using a procedure that
can be carried out in n ways, and for every way w, there are d equivalent ways of doing
w.
Restated in terms of sets: If the finite set A is the union of n pairwise disjoint subsets
each with d elements, then n = |A|/d.
In terms of functions: If f is a function from A to B, where both are finite sets, and for
every value y ∈ B there are exactly d values x ∈ A such that f(x) = y, then |B| = |A|/d.
The Division Rule is commonly used for counting
arrangements when some of the objects are
indistinguishable or when order doesn’t matter.
Division Rule: An Illustrative Example
• Assume that there are 12 ways to carry out a
procedure:
– A, B, C, D, E, F, G, H, I, J, K, L
• Because of indistinguishable alternatives there are
12
effectively only = 4 ways to carry out the procedure
– α, β, γ, δ
3
Basic Counting Principles: Division Rule
2
Example: How many ways are there to seat four people around a circular table, where
two seatings are considered the same when each person has the same left and right
neighbor?
Solution: Number the seats around the table from 1 to 4 proceeding clockwise. There
are four ways to select the person for seat 1, 3 for seat 2, 2, for seat 3, and one way for
seat 4. Thus there are 4! = 24 ways to order the four people. Since two seatings are the
same when each person has the same left and right neighbor, corresponding to each of
the 24 ways of seating around the table, there are 4 equivalent ways of seating the
people, which differ only in the choice of the person in seat 1. This corresponds to
rotating the table, which does not change a person’s neighbors.
Therefore, by the division rule, there are 24/4 = 6 different seating arrangements.
Alternate Solution (using the product rule): The choice of seat 1 is immaterial. There
are 3 ways to select seat 2, 2 ways to select seat 3 and 1 way to select seat 4.
Therefore, by the product rule, there are 3 * 2 * 1 = 6 different seating arrangements.
Basic Counting Principles: Division Rule
3
Example: How many ways are there to seat six people around a circular table where
two seatings are considered the same when everyone has the same two neighbors
without regard to whether they are right or left neighbors?
Solution: There are 6! = 720 ways to order the six people. For reasons explained on the
previous slide, this overcounts the seating arrangements by a multiple of 6,
corresponding to rotating the table. Furthermore, we are overcounting by a multiple of
2 because seating arrangements that reverse the right and the left neighbor are
equivalent.
Therefore, by the division rule, there are 720/6/2 = 60 different seating arrangements.
Alternate Solution (using the product rule): The choice of seat 1 is immaterial. There
are 5 ways to select seat 2 and 4 ways to select seat 3. However, since we are treating
arrangements that reverse the right and left neighbors as equivalent, there are (5*4)/2
= 10 ways to select seats 2 and 3. There are 3 ways to select seat 4, 2 ways to select
seat 5 and 1 way top select seat 6.
Therefore, by the product rule, there are 10 * 3 * 2 * 1 = 60 different seating
arrangements.
Tree Diagrams
1
Tree Diagrams: We can solve many counting problems through the use of tree
diagrams, where a branch represents a possible choice and the leaves
represent possible outcomes.
Example: Suppose that “I Love Discrete Math” T-shirts come in five different
sizes: S,M,L,XL, and XXL. Each size comes in four colors (white, red, green, and
black), except XL, which comes only in red, green, and black, and XXL, which
comes only in green and black. What is the minimum number of shirts that the
campus book store needs to stock to have one of each size and color available?
Solution: Draw the tree diagram.
Alternate Solution:
By sum rule 4S + 4M + 4L + 3XL + 2XXL =
17 types of shirts
The store must stock at least 17 T-shirts.
Tree Diagrams
2
Example: How many bit strings of length four do not have two consecutive 1s?
Solution: The tree diagram shows that there are 8 bit strings of length four that do not have
two consecutive 1s
•
0000
•
0001
•
0010
•
0100
•
0101
•
1000
•
1001
•
1010
Note: Tree diagrams are most feasible when the
number of leaves in the tree is small.
The Pigeonhole Principle
Section 6.2
The Pigeonhole Principle
1
If a flock of 20 pigeons roosts in a set of 19 pigeonholes, one of the
pigeonholes must have more than 1 pigeon.
• Also known as the Dirichlet Drawer Principle
The Pigeonhole Principle
2
Pigeonhole Principle: If k is a positive integer and k + 1 or more objects are placed into k
boxes, then at least one box contains two or more objects.
Proof: We use a proof by contraposition.
Let p: k + 1 or more objects are placed into k boxes
Let q: at least one box contains two or more objects
The contrapositive states that ¬q → ¬p
Our hypothesis is that each box contains at most 1 object.
Since there are k boxes, the total number of objects is at most k.
This is a contradiction since there are at least k+1 boxes.
Jump to long description
Contradiction of the
contrapositive
The Pigeonhole Principle
3
Corollary 1: A function f from a set with k + 1 or more
elements to a set with k elements is not one-to-one.
Proof: Use the pigeonhole principle.
• Create a box corresponding to each element y in the codomain of f .
• Put in the box for y all of the elements x from the domain such that
f(x) = y.
• Because there are at least k + 1 elements and only k boxes, at least
one box has two or more elements.
Hence, f can’t be one-to-one.
The Pigeonhole Principle
4
Example: Among any group of 367 people, there must be at least
two with the same birthday, because there are only 366 possible
birthdays.
Example Show that for every integer n there is a multiple of n that
has only 0s and 1s in its decimal expansion.
Solution: Let n be a positive integer. Consider the n + 1 integers 1,
11, 111, …., 11…1 (where the last has n + 1 1s). There are n possible
remainders when an integer is divided by n. By the pigeonhole
principle, when each of the n + 1 integers is divided by n, at least
two must have the same remainder. Subtract the smaller from the
larger and the result is a multiple of n that has only 0s and 1s in its
decimal expansion.
Smallest multiple of the first 34 numbers that is
composed entirely of 0s and 1s in the decimal
expansion:
http://oeis.org/A004290/list
Let a ≡ b (mod c)
a = b + km
a - b = km
m | (a-b)
CS 2305: Discrete Mathematics
for Computing I
Lecture 25
- KP Bhat
1
The Generalized Pigeonhole Principle
1
The Generalized Pigeonhole Principle: If N objects are placed
into k boxes, then there is at least one box containing at least
⌈N/k⌉ objects.
Proof: We use a proof by contradiction of contraposition. Suppose
that none of the boxes contains more than ⌈N/k⌉ − 1 objects.
Then the total number of objects is at most
N  
 N  
k     1 < k    1  1  N ,
 
 k  
 k
where the inequality ⌈N/k⌉ < ⌈N/k⌉ + 1 has been used. This is a
contradiction because there are a total of N objects.
Example: Among 100 people there are at least
⌈100/12⌉ = 9 who were born in the same month.
The Generalized Pigeonhole Principle
2
Problem: What is the minimum number of students required
in a discrete mathematics class to be sure that at least six will
receive the same grade, if there are five possible grades, A, B,
C, D, and F?
Solution: The minimum number of students needed to ensure
that at least six students receive the same grade is the
smallest integer N such that ⌈N∕5⌉ = 6. The smallest such
integer is N = 5 ⋅ 5 + 1 = 26.
The Generalized Pigeonhole Principle
Example: a) How many cards must be selected from a standard deck
of 52 cards to guarantee that at least three cards of the same suit are
chosen?
b) How many must be selected to guarantee that at least three hearts
are selected?
Solution: a) We assume four boxes; one for each suit. Using the
generalized pigeonhole principle, at least one box contains at least
⌈N/4⌉ cards. At least three cards of one suit are selected if ⌈N/4⌉ ≥3.
The smallest integer N such that ⌈N/4⌉ ≥3 is
N = 2 ∙ 4 + 1 = 9.
b) A deck contains 13 hearts and 39 cards which are not hearts. So, if
we select 41 cards, we may have 39 cards which are not hearts along
with 2 hearts. However, when we select 42 cards, we must have at
least three hearts.
3
The Generalized Pigeonhole Principle
Example: What is the least number of area codes
needed to guarantee that the 25 million phones in a
state can be assigned distinct 10-digit telephone
numbers? Assume that telephone numbers are of
the form NXX-NXX-XXXX, where the first three digits
form the area code, N represents a digit from 2 to 9
inclusive, and X represents any digit.
Solution: Total number of phone numbers, without
the area code = 8*106
Since we have to provide 25 million phone
numbers, the state will need a minimum of
⌈25*106/8*106⌉ = 4 area codes
4
Permutations and
Combinations
Section 6.3
Permutations and Combinations
1
• Many counting problems deal with finding the number of ways to
arrange a specified number of elements of a set, without actually
listing them
– either all n elements of a set or a subset of r elements of the set
• There are two distinct methods that can be used to select r objects
from a set of n elements: ordered and unordered
– In an ordered selection, it is not only what elements are chosen but
also the order in which they are chosen that matters (e.g. number of
ways to seat people around a table)
• An ordered selection of r elements from a set of n elements is called an rpermutation of the set.
– In an unordered selection it is only the identity of the chosen elements
that matters while order is immaterial (e.g. number of ways to select
members to a committee)
• An unordered selection of r elements from a set of n elements is called the rcombination of the set.
Permutations and Combinations
2
• The product rule is a generic rule that allows
us to count the number of ways to complete
any procedure that is composed of multiple
tasks
• Permutations and combinations make heavy
use of the product rule to count the number
of ordered and unordered selections, of a
desired size, that can be made from a given
set
Permutations
Definition: A permutation of a set of distinct objects is
an ordered arrangement of these objects. An ordered
arrangement of r elements of a set is called an
r-permutation.
Example: Let S = {1,2,3}.
• The ordered arrangement 3,1,2 is a permutation of S.
• The ordered arrangement 3,2 is a 2-permutation of S.
The number of r-permutations of a set with n
elements is denoted by P(n,r).
• The 2-permutations of S = {1,2,3} are 1,2; 1,3; 2,1; 2,3; 3,1;
and 3,2. Hence, P(3,2) = 6.
A Formula for the Number of Permutations
Theorem 1: If n is a positive integer and r is an integer with 1 ≤ r ≤
n, then there are
P(n, r) = n(n − 1)(n − 2) ∙∙∙ (n − r + 1)
r-permutations of a set with n distinct elements.
Proof: Use the product rule. The first element can be chosen in n
ways. The second in n − 1 ways, and so on until there are
(n − ( r − 1)) ways to choose the last element.
Corollary 1: If n and r are integers with 1 ≤ r ≤ n, then
n!
P  n, r  
 n  r !
Proof:
P(n, r) = n(n − 1)(n − 2) ∙∙∙ (n − r + 1) = n(n − 1)(n − 2) ∙∙∙ (n − r + 1)
𝑛−𝑟 !
𝑛−𝑟 !
=
𝑛!
𝑛−𝑟 !
Permutation Formula, Special Cases
𝑛!
P(n, r) =
𝑛−𝑟 !
𝑛!
• P(n, n) =
𝑛−𝑛 !
=
𝑛!
0!
= 𝑛!
– There are 𝑛! ordered arrangements involving n elements
• P(n, 0) =
𝑛!
𝑛−0 !
=
𝑛!
𝑛!
=1
– There is only 1 ordered arrangement involving 0 elements
• P(n, 1) =
𝑛!
𝑛−1 !
=𝑛
– There are n ordered arrangements involving 1 element
Solving Counting Problems by Counting
Permutations
1
Example: How many ways are there to select a
first-prize winner, a second prize winner, and a
third-prize winner from 100 different people who
have entered a contest?
Solution:
P(100,3) = 100 ∙ 99 ∙ 98 = 970,200
Solving Counting Problems by Counting
Permutations
2
Example: Suppose that a saleswoman has to visit
eight different cities. She must begin her trip in a
specified city, but she can visit the other seven cities
in any order she wishes. How many possible orders
can the saleswoman use when visiting these cities?
Solution: The first city is chosen, and the rest are
ordered arbitrarily. Hence the orders are:
7! = 7 ∙ 6 ∙ 5 ∙ 4 ∙ 3 ∙ 2 ∙ 1 = 5040
If she wants to find the tour with the shortest path
that visits all the cities, she must consider 5040 paths!
Solving Counting Problems by Counting
Permutations
3
Example: How many permutations of the letters
ABCDEFGH contain the string ABC ?
Solution: Because the letters ABC must occur as a
block, we can consider them to be a single object.
The problem now reduces to finding the number
of permutations of six objects viz. ABC, D, E, F, G,
and H.
So the answer is P(6, 6) = 6! = 6 ∙ 5 ∙ 4 ∙ 3 ∙ 2 ∙ 1 =
720
Combinations
1
Definition: An r-combination of elements of a set is an
unordered selection of r elements from the set. Thus, an
r-combination is simply a subset of the set with r elements.
The number of r-combinations of a set with n distinct
elements is denoted by C(n, r). The notation  nr 
is also used and is called a binomial coefficient.
Example: Let S be the set {a, b, c, d}. Then {a, c, d} is a 3combination from S. It is the same as {d, c, a} since the order
listed does not matter.
C(4,2) = 6 because the 2-combinations of {a, b, c, d} are the six
subsets {a, b}, {a, c}, {a, d}, {b, c}, {b, d}, and {c, d}.
Combinations
2
Theorem 2: The number of r-combinations of a
set with n elements, where n ≥ r ≥ 0, equals
n!
C  n, r  
.
 n  r  !r !
Proof:
The number of r-Combinations of the set is C(n, r) (by definition).
If we order each of these combinations, we get P(n, r) the rpermutations of the set.
By the product rule P(n, r) = C(n,r) ∙ P(r,r). Therefore
Combinations
3
Corollary 2: Let n and r be nonnegative integers with
r ≤ n. Then C(n, r) = C(n, n − r).
Proof: From Theorem 2, it follows that
n!
C  n, r  
 n  r  !r !
and
n!
n!
C  n, n  r  

.
 n  r !  n   n  r  !  n  r !r !
Hence, C(n, r) = C(n, n − r).
Combination Formula, Special Cases
𝑛!
C(n, r) =
𝑟! 𝑛−𝑟 !
𝑛!
• C(n, n) =
𝑛! 𝑛−𝑛 !
=
𝑛!
𝑛!
=1
– There is only 1 unordered selection involving n elements
• C(n, 0) =
𝑛!
0! 𝑛−0 !
=
𝑛!
1.𝑛!
=1
– There is only 1 ordered arrangement involving 0 elements
• C(n, 1) =
𝑛!
1! 𝑛−1 !
=
𝑛!
𝑛−1 !
=n
– There are n ordered arrangements involving 1 element
Combinations
4
Example: How many poker hands of five cards can be
dealt from a standard deck of 52 cards? Also, how many
ways are there to select 47 cards from a deck of 52 cards?
Solution: Since the order in which the cards are dealt
does not matter, the number of five card hands is:
52!
C  52,5  
5!47!
52  51 50  49  48

 26 17 10  49 12  2,598,960
5  4  3  2 1
The different ways to select 47 cards from 52 is
C  52, 47  
52!
 C  52,5   2,598,960
47!5!
Combinations
5
Example: How many ways are there to select five players
from a 10-member tennis team to make a trip to a match at
another school.
Solution: By Theorem 2, the number of combinations is
10!
C 10,5  
 252.
5!5!
Example: A group of 30 people have been trained as
astronauts to go on the first mission to Mars. How many ways
are there to select a crew of six people to go on this mission?
Solution: By Theorem 2, the number of possible crews is
30! 30  29  28  27  26  25
C  30, 6  

 593, 775
6!24!
6  5  4  3  2 1
Combinations
6
Problem: How many bit strings of length 10 contain
a) exactly four 1s?
b) at most four 1s?
c) at least four 1s?
d) an equal number of 0s and 1s?
Solution:
10!
(4!)(6!)
a)
C(10, 4) =
b)
C(10, 0) + C(10, 1) + C(10, 2) + C(10, 3) + C(10, 4) = 1 + 10 + 45 +
120 + 210 = 386
C(10, 4) + C(10, 5) + C(10, 6) + C(10, 7) + C(10, 8) + C(10, 9) + C(10,
10) = 210 + 252 + 210 + 120 + 45 + 10 + 1 = 848
C(10, 5) = 252
c)
d)
= 210
CS 2305: Discrete Mathematics
for Computing I
Lecture 26
- KP Bhat
1
Combinations
7
Problem: Suppose that there are 9 faculty members in the
mathematics department and 11 in the computer science department.
How many ways are there to select a committee to develop a discrete
mathematics course at a school if the committee is to consist of three
faculty members from the mathematics department and four from the
computer science department?
Solution: Number of ways to select 3 faculty members from the
mathematics department = C(9, 3)
Number of ways to select 4 faculty members from the computer
science department = C(11, 4)
By product rule the total ways to select the committee is C(9, 3) *
C(11, 4) = 84 * 330 = 27, 720
2
Combinations: A Shortcut
C(n, r) =
𝑛!
𝑟!∗ 𝑛−𝑟 !
𝑛 𝑛−1 𝑛−2 … 𝑛−𝑟+1
𝑟!
=
=
10 ∗9 ∗ 8
3 ∗2 ∗1
3 terms
=
𝑛 𝑛−1 𝑛−2 … 𝑛−𝑟+1
𝑟 𝑟−1 𝑟−2 …(2)(1)
r terms
3 terms
e.g. C(10, 3) =
𝑛 𝑛−1 𝑛−2 … 𝑛−𝑟+1 𝑛−𝑟 !
𝑟!∗ 𝑛−𝑟 !
r terms
=
720
6
= 120
Since C(n, r) = C(n, n-r), for using the shortcut
the trick is to use the formula where the
second argument of the n-Combinations
function is the smaller of the two numbers (viz.
r and n-r).
For example, to compute C(10, 7) we first
convert C(10, 7) to C(10, 10-7) = C(10, 3) and
3
then apply the shortcut
Permutation vs Combinations: Special
Cases
𝑛!
𝑛−𝑟 !
𝑛!
n) =
𝑛−𝑛 !
P(n, r) =
• P(n,
=
𝑛!
0!
= 𝑛!
– There are 𝑛! ordered
arrangements involving n
elements
• P(n, 0) =
𝑛!
𝑛−0 !
=
𝑛!
𝑛!
=1
– There is only 1 ordered
arrangement involving 0
elements
• P(n, 1) =
𝑛!
𝑛−1 !
𝑛!
𝑟! 𝑛−𝑟 !
𝑛!
𝑛!
= =1
𝑛! 𝑛−𝑛 ! 𝑛!
C(n, r) = C(n, n-r) =
=𝑛
– There are n ordered
arrangements involving 1
element
• C(n, n) =
– There is only 1 unordered
selection involving n elements
• C(n, 0) =
𝑛!
0! 𝑛−0 !
=
𝑛!
1.𝑛!
=1
– There is only 1 unordered
arrangement involving 0
elements
• C(n, 1) =
𝑛!
1! 𝑛−1 !
=
𝑛!
𝑛−1 !
=n
– There are n unordered
arrangements involving 1
element
4
Generalized Permutations and
Combinations
Section 6.5
Background Information
• In our discussion on permutations and combinations so
far each element could be used at most once
– While discussing the product rule we did look at some
repetitions
• In some counting problems elements may be used
repeatedly
– e.g. letter or a digit may be used more than once on a
license plate
• Some counting problems involve indistinguishable
elements
– e.g. in how many ways can the letters of the word SUCCESS
be rearranged
Permutations with Repetition
Theorem 1: The number of r-permutations of a set of n
objects with repetition allowed is nr.
Proof: There are n ways to select an element of the set
for each of the r positions in the r-permutation when
repetition is allowed. Hence, by the product rule there
are nr r-permutations with repetition.
Example: How many strings of length r can be formed
from the uppercase letters of the English alphabet?
Solution: The number of such strings is 26r, which is the
number of r-permutations of a set with 26 elements.
Comparison: Repetition vs
Nonrepetition
Non-repetition
Repetition
• First element can be chosen in
n ways
• Second element can be chosen
in n-1 ways
• …
• rth element can be chosen in n(r-1) ways
• By product rule total number
of ways to choose the
elements = n * (n-1) * … * (n𝑛!
r+1) =
• First element can be chosen in
n ways
• Second element can be chosen
in n ways
• …
• rth element can be chosen in n
ways
• By product rule total number
of ways to choose the
elements = n * n * … * n = nr
𝑛−𝑟 !
8
Background: Combinations with
Repetition
1
Let us devise a way to represent an r-combination of a
set with n elements with repetition.
First we will create a list of n-1 bars, which can be used to
represent the n elements of the set.
For example if our set is {A, B, C, D} we introduce the bars
as follows:
Background: Combinations with
Repetition
2
For each instance of an element in the unordered arrangement, we
will put an asterisk symbol in the corresponding section. If there is no
instance of an element in the unordered arrangement, we will leave it
blank.
For example, the arrangement AACDDD will be represented as
There is a unique pattern of bars and asterisks that represents each
pattern in the r-combination. Each pattern has a total of n-1+r
elements, comprising n-1 bars and r asterisks.
The problem of counting the number of r-combinations from a set
with n elements when repetition of elements is allowed now reduces
to the problem of counting the number of ways we can arrange n-1
bars within a set with n-1+r elements
Combinations with Repetition
1
Theorem 2: The number of r-combinations from a set with
n elements when repetition of elements is allowed is
C(n + r − 1, r) = C(n + r − 1, n − 1)
Recall C(n, r) = C(n, n − r)
Proof: Let us represent each r-combination of a set with n elements
with repetition allowed by a list of n –1 bars and r stars
Each r-combination of a set with n elements with repetition
allowed can be represented by a list of n –1 bars and r stars. The
bars mark the n cells containing a star for each time the ith element
of the set occurs in the combination.
The number of such lists is C(n + r – 1, r), because each list is a
choice of the r positions to place the stars, from the total of n + r –
1 positions to place the stars and the bars. This is also equal to C(n
+ r – 1, n –1), which is the number of ways to place the n –1 bars.
Combinations with Repetition
2
Example: How many ways are there to select five bills from a
box containing at least five of each of the following
denominations: $1, $2, $5, $10, $20, $50, and $100?
Solution: Place the selected bills in the appropriate position
of a cash box illustrated below:
7 elements
Combinations with Repetition
3
Some possible ways of placing the five bills:
The number of ways to
select five bills corresponds
to the number of ways to
arrange six bars and five stars
in a row.
This is the number of unordered
selections of 5 objects from a set of 11. Hence, there are
11!
C 11,5  
 462
5!6!
C(n + r − 1, r) [= C(n + r − 1, n − 1)]
n = 7, r = 5
ways to choose five bills with seven types of bills.
Combinations with
Repetition
4
Example: Suppose that a cookie shop
has four different kinds of cookies. How
many different ways can six cookies be chosen?
Solution: The number of ways to choose six cookies
is the number of 6-combinations of a set with four
elements. By Theorem 2
9 8 7
 84
C(4+6-1, 6) = C  9, 6   C  9,3 
1 2  3
C(n + r − 1, r) [= C(n + r − 1, n − 1)]
n = 4, r = 6
is the number of ways to choose six cookies
from the four kinds.
Combinations with Repetition
5
Example: How many solutions does the equation
x1 + x2 + x3 = 11
have, where x1 , x2 and x3 are nonnegative integers?
Solution: Each solution corresponds to a way to select 11 items
from a set with three elements; x1 elements of type one, x2 of
type two, and x3 of type three.
By Theorem 2 it follows that there are
C(3 + 11 -1, 11) = C(13, 11) = C(13, 2) =
13 ∗12
2 ∗1
= 78 solutions.
C(n + r − 1, r) [= C(n + r − 1, n − 1)]
n = 3, r = 11
Combinations with Repetition
6
Example: How many solutions does the equation
x1 + x2 + x3 = 11
have, where x1 ≥ 1, x2 ≥ 2, and x3 ≥ 3?
Solution: In this case we have at least one item of type x1, two items
of type x2, and three items of type x3. So the problem boils down to
selecting 5 items [11 – (1 + 2 + 3)] from a set with three elements,
with repetition.
By Theorem 2 it follows that there are
C(3 + 5 -1, 5) = C(7, 5) = C(7, 2) =
7 ∗6
2 ∗1
= 21 solutions.
C(n + r − 1, r) [= C(n + r − 1, n − 1)]
n = 3, r = 5
Summarizing the Formulas for Counting Permutations
and Combinations with and without Repetition
TABLE 1 Combinations and Permutations With and Without Repetition.
Type
Repetition Allowed?
Formula
r-permutations
No
n!
 n  r !
r-combinations
No
n!
r ! n  r  !
r-permutations
Yes
nr
Yes
 n  r  1!
r ! n  1 !
r-combinations
Permutations with Indistinguishable
Objects
1
Theorem 3: The number of different permutations of n objects, where
there are n1 indistinguishable objects of type 1, n2 indistinguishable
objects of type 2, …., and nk indistinguishable objects of type k, is:
n!
.
n1 !n2 ! nk !
Proof:
The n1 objects of type one can be placed in the n positions in C(n, n1 ) ways,
leaving n − n1 positions.
Then the n2 objects of type two can be placed in the n − n1 positions in C(n − n1,
n2 ) ways, leaving n − n1 − n2 positions.
Continue in this fashion, until nk objects of type k are placed in
C(n − n1 − n2 − ∙∙∙ − nk-1, nk) ways.
By the product rule the total number of permutations is:
C(n, n1 ) C(n − n1, n2 ) C(n − n1 − n2, n3 ) ∙∙∙ C(n − n1 − n2 − ∙∙∙ − nk-1, nk)
Permutations with Indistinguishable
Objects
2
Proof (Cont’d):
The product C(n, n1 ) C(n − n1, n2 ) C(n − n1 − n2, n3 ) ∙∙∙ C(n − n1 − n2 − ∙∙∙ − nk-1, nk)
can be manipulated into the desired result as follows:
𝑛!
𝑛 − 𝑛1 !
𝑛 − 𝑛1 − 𝑛2 !
𝑛 − 𝑛1 − ⋯ − 𝑛𝑘 − 1 !
…
𝑛1 ! 𝑛 − 𝑛1 ! 𝑛2 ! 𝑛 − 𝑛1 − 𝑛2 ! 𝑛3 ! 𝑛 − 𝑛1 − 𝑛2 − 𝑛 3 !
𝑛𝑘 ! 0!
=
𝑛!
𝑛1 ! 𝑛2 ! 𝑛3 ! … 𝑛𝑘 !
“telescoping cancellation”
Permutations with Indistinguishable
Objects
3
Example: How many different strings can be made by reordering
the letters of the word SUCCESS.
Solution: There are seven possible positions for the three Ss, two
Cs, one U, and one E.
•
•
•
•
The three Ss can be placed in C(7,3) different ways, leaving four positions
free.
The two Cs can be placed in C(4,2) different ways, leaving two positions
free.
The U can be placed in C(2,1) different ways, leaving one position free.
The E can be placed in C(1,1) way.
By the product rule, the number of different strings is:
C  7,3 C  4, 2  C  2,1 C 1,1 
7! 4! 2!
1!



 420.
3!4! 2!2! 1!1! 3!2!1!1!
Distributing Objects into Boxes
1
Many counting problems can be solved by
counting the ways objects can be placed in boxes.
• The objects may be either different from each other
(distinguishable) or identical (indistinguishable).
• The boxes may be labeled (distinguishable) or
unlabeled (indistinguishable).
Distributing Distinguishable Objects into
Distinguishable Boxes
1
Theorem 4: The number of ways to distribute n distinguishable objects
into k distinguishable boxes so that ni objects are placed into box i, i = 1,
2,…, k, equals
n!
.
n1 !n2 ! nk !
Proof:
(Very similar to the proof for permutations with indistinguishable objects)
The first box holds n1 objects, and there are C(n,n1) ways to choose those objects from among the n
objects in the collection.
Once these objects are chosen, we can choose the objects to be placed in the second box in C(n n1,n2) ways, since there are n - n1 objects not yet placed, and we need to put n2 of them into the
second box.
Similarly, there are then C(n - n1 -n2, n3) ways to choose objects for the third box.
We continue in this way, until finally there are C(n - n1 -n2 ,…-nk-1, nk) ways to choose the objects to
put in the last (kth) box.
By the product rule the total number of permutations is:
C(n, n1 ) C(n − n1, n2 ) C(n − n1 − n2, n3 ) ∙∙∙ C(n − n1 − n2 − ∙∙∙ − nk-1, nk), which has been shown
to be equal to
𝑛!
𝑛1! 𝑛2! 𝑛3! … 𝑛𝑘 !
Distributing Distinguishable Objects
into Distinguishable Boxes
2
Example: How many ways are there to distribute hands of 5 cards to
each of four players from the standard deck of 52 cards?
Solution:
ab initio approach
The first player can be dealt 5 cards in C(52, 5) ways. The second player
can be dealt 5 cards in C(47, 5) ways, because only 47 cards are left.
The third player can be dealt 5 cards in C(42, 5) ways. Finally, the
fourth player can be dealt 5 cards in C(37, 5) ways.
Hence, the total number of ways to deal four players 5 cards each is
C(52, 5)C(47, 5)C(42, 5)C(37, 5) =
=
52!
5!5!5!5!32!
52!
47!5!
∗
47!
42!5!
∗
42!
37!5!
∗
37!
32!5!
Distributing Distinguishable Objects
into Distinguishable Boxes
3
Solution (Cont’d):
Distinguishable objects, distinguishable boxes
theorem approach
The distinguishable objects are the 52 cards, and
the five distinguishable boxes are the hands of the
four players (size 4 each) and the rest of the deck
(size 32).
Hence from the distinguishable objects,
distinguishable boxes theorem the total number of
52!
ways to deal is
5!5!5!5!32!
Distributing Indistinguishable Objects
into Distinguishable Boxes
1
Theorem: The number of ways to distribute n indistinguishable objects into k
distinguishable boxes so that ni objects are placed into box i, i = 1, 2,…, k,
equals C(n + k – 1, k - 1) = C(n + k – 1, n)
Proof: (Very similar to proof for number of r-combinations from a set with n
elements when repetition of elements is allowed).
Let us consider a slotted box with adjustable dividers.
k-1 dividers are needed to convert the slotted box into k bins
Each arrangement of objects in the bins is represented by a unique
combination of dividers and objects.
There are total of k-1+n dividers and objects.
The number of combinations of k-1 dividers
among k-1+n dividers and objects is C(k-1+n, k1)
Example: There are C(8 + 10 − 1, 8-1) = C(17,7)
= 19,448 ways to place 10 indistinguishable
objects into 8 distinguishable boxes.
Distributing Objects into
Indistinguishable Boxes
• There are no closed formulae to count the
ways to distribute objects, distinguishable or
indistinguishable, into indistinguishable boxes
– A closed formula is an expression that can be
evaluated using a finite number of operations
• arithmetic operations, rational powers, exponential and
logarithmic functions, trigonometric functions, and the
factorial function
• infinite series are excluded
26
Relations and Their Properties
Section 9.1
Binary Relations
1
• Definition: A binary relation R from a set A to
a set B is a subset of the Cartesian product of
A and B
–R⊆AXB
Let A and B be sets. The Cartesian product of A and B,
denoted by A × B, is the set of all ordered pairs (a, b),
where a ∈ A and b ∈ B.
Hence, A × B = {(a, b) ∣ a ∈ A ∧ b ∈ B}
Section 2.1
• We use the notation a R b to denote that
(a, b) ∈ R and a b to denote that (a, b) ∉ R.
Binary Relations
2
R⊆A⨯B
Example:
• Let A = {0,1,2} and B = {a,b}
• {(0, a), (0, b), (1,a) , (2, b)} is a relation from A to B.
• We can represent relations from a set A to a set B
graphically or using a table:
Relations are more general than
functions. A function is a relation
where exactly one element of B is
related to each element of A.
Binary Relations
aRb is a subset of A X B
All subsets of A X B:
{}{(a,1)},{(a,2)},{(b,1)},{(b,2)},{(c,1)},{(c,2)},{(a,1),(a,2)},{(a,1),(b,1)},{(
a,1),(b,2)},{(a,1),(c,1)},{(a,1),(c,2)},{(a,2),(b,1)},{(a,2),(b,2)},{(a,2),(c,1
)},{(a,2),(c,2)},{(b,1),(b,2)},{(b,1),(c,1)},{(b,1),(c,2)},{(b,2),(c,1)},{(b,2),
(c,2)},{(c,1),(c,2)},{(a,1),(a,2),(b,1)},{(a,1),(a,2),(b,2)},{(a,1),(a,2),(c,1)
},{(a,1),(a,2),(c,2)},{(a,1),(b,1),(b,2)},{(a,1),(b,1),(c,1)},{(a,1),(b,1),(c,2
)},{(a,1),(b,2),(c,1)},{(a,1),(b,2),(c,2)},{(a,1),(c,1),(c,2)},{(a,2),(b,1),(b,
2)},{(a,2),(b,1),(c,1)},{(a,2),(b,1),(c,2)},{(a,2),(b,2),(c,1)},{(a,2),(b,2),(c
,2)},{(a,2),(c,1),(c,2)},{(b,1),(b,2),(c,1)},{(b,1),(b,2),(c,2)},{(b,1),(c,1),(
c,2)},{(b,2),(c,1),(c,2)},{(a,1),(a,2),(b,1),(b,2)},{(a,1),(a,2),(b,1),(c,1)},{
(a,1),(a,2),(b,1),(c,2)},{(a,1),(a,2),(b,2),(c,1)},{(a,1),(a,2),(b,2),(c,2)},{(
a,1),(a,2),(c,1),(c,2)},{(a,1),(b,1),(b,2),(c,1)},{(a,1),(b,1),(b,2),(c,2)},{(a
,1),(b,1),(c,1),(c,2)},{(a,1),(b,2),(c,1),(c,2)},{(a,2),(b,1),(b,2),(c,1)},{(a,
2),(b,1),(b,2),(c,2)},{(a,2),(b,1),(c,1),(c,2)},{(a,2),(b,2),(c,1),(c,2)},{(b,
1),(b,2),(c,1),(c,2)},{(a,1),(a,2),(b,1),(b,2),(c,1)},{(a,1),(a,2),(b,1),(b,2)
,(c,2)},{(a,1),(a,2),(b,1),(c,1),(c,2)},{(a,1),(a,2),(b,2),(c,1),(c,2)},{(a,1),(
b,1),(b,2),(c,1),(c,2)},{(a,2),(b,1),(b,2),(c,1),(c,2)},{(a,1),(a,2),(b,1),(b,
2),(c,1),(c,2)}
3
A
a
b
B
R
1
2
c
There are 3*2 = 6
elements in A X B
and 26 = 64 subsets
of A X B
30
Binary Relations
4
Example: Let A be the set of cities in the U.S.A., and
let B be the set of 50 states in the U.S.A.
Define relation R as follows
R = {(a,b) |a ∈ A, b ∈ B ∧ a is in state b}
Then (Boulder, Colorado), (Bangor, Maine), (Ann
Arbor, Michigan), (Middletown, New Jersey),
(Middletown, New York), (Cupertino, California),
and (Red Bank, New Jersey) are in R
31
CS 2305: Discrete Mathematics
for Computing I
Lecture 27
- KP Bhat
1
Binary Relations
1
• Definition: A binary relation R from a set A to
a set B is a subset of the Cartesian product of
A and B
–R⊆AXB
Let A and B be sets. The Cartesian product of A and B,
denoted by A × B, is the set of all ordered pairs (a, b),
where a ∈ A and b ∈ B.
Hence, A × B = {(a, b) ∣ a ∈ A ∧ b ∈ B}
Section 2.1
• We use the notation a R b to denote that
(a, b) ∈ R and a b to denote that (a, b) ∉ R.
Binary Relations
A relation R from A to B is a subset of A X B
All subsets of A X B (all possible relations
between A and B):
{}
{(a,1)}
{(a,2)}
{(b,1)}
{(b,2)}
{(c,1)}
{(c,2)}
{(a,1),(a,2)}
{(a,1),(b,1)}
{(a,1),(b,2)}
{(a,1),(c,1)}
{(a,1),(c,2)}
{(a,2),(b,1)}
{(a,2),(b,2)}
{(a,2),(c,1)}
{(a,2),(c,2)}
{(b,1),(b,2)}
{(b,1),(c,1)}
{(b,1),(c,2)}
{(b,2),(c,1)}
{(b,2),(c,2)}
{(c,1),(c,2)}
{(a,1),(a,2),(b,1)}
{(a,1),(a,2),(b,2)}
{(a,1),(a,2),(c,1)}
{(a,1),(a,2),(c,2)}
{(a,1),(b,1),(b,2)}
{(a,1),(b,1),(c,1)}
{(a,1),(b,1),(c,2)}
{(a,1),(b,2),(c,1)}
{(a,1),(b,2),(c,2)}
{(a,1),(c,1),(c,2)}
{(a,2),(b,1),(b,2)}
{(a,2),(b,1),(c,1)}
{(a,2),(b,1),(c,2)}
{(a,2),(b,2),(c,1)}
{(a,2),(b,2),(c,2)}
{(a,2),(c,1),(c,2)}
{(b,1),(b,2),(c,1)}
{(b,1),(b,2),(c,2)}
{(b,1),(c,1),(c,2)}
{(b,2),(c,1),(c,2)}
{(a,1),(a,2),(b,1),(b,2)}
{(a,1),(a,2),(b,1),(c,1)}
{(a,1),(a,2),(b,1),(c,2)}
{(a,1),(a,2),(b,2),(c,1)}
{(a,1),(a,2),(b,2),(c,2)}
{(a,1),(a,2),(c,1),(c,2)}
{(a,1),(b,1),(b,2),(c,1)}
{(a,1),(b,1),(b,2),(c,2)}
{(a,1),(b,1),(c,1),(c,2)}
{(a,1),(b,2),(c,1),(c,2)}
{(a,2),(b,1),(b,2),(c,1)}
{(a,2),(b,1),(b,2),(c,2)}
{(a,2),(b,1),(c,1),(c,2)}
{(a,2),(b,2),(c,1),(c,2)}
{(b,1),(b,2),(c,1),(c,2)}
{(a,1),(a,2),(b,1),(b,2),(c,1)} {(a,1),(a,2),(b,1),(b,2),(c,2)} {(a,1),(a,2),(b,1),(c,1),(c,2)}
{(a,1),(a,2),(b,2),(c,1),(c,2)} {(a,1),(b,1),(b,2),(c,1),(c,2)} {(a,2),(b,1),(b,2),(c,1),(c,2)} {(a,1),(a,2),(b,1),(b,2),(c,1),(c,2)}
3
A
a
b
B
R
1
2
c
There are 3*2 = 6
elements in A X B
and 26 = 64 subsets
of A X B
In this instance
there are 64
relations between A
and B
3
Binary Relations
4
Example: Let A be the set of cities in the U.S.A., and
let B be the set of 50 states in the U.S.A.
Define relation R as follows
R = {(a,b) |a ∈ A, b ∈ B ∧ a is in state b}
Then (Boulder, Colorado), (Bangor, Maine), (Ann
Arbor, Michigan), (Middletown, New Jersey),
(Middletown, New York), (Cupertino, California),
and (Red Bank, New Jersey) are in R
4
Functions as Relations
• A function f from a set A to a set B assigns
exactly one element of B to each element of A
• Relations are generalizations of functions and
they can be used to express a much wider
class of relationships between sets
5
Binary Relations on a Set
1
Definition: A binary relation R on a set A is a subset of A × A or a relation
from A to A.
Example:
•
Suppose that A = {a,b,c}. Then R = {(a,a),(a,b), (a,c)} is a relation on A.
•
Let A = {1, 2, 3, 4}. The ordered pairs in the relation
R = {(a,b) | a divides b} are
(1,1), (1, 2), (1,3), (1, 4), (2, 2), (2, 4), (3, 3), and (4, 4).
Graphical and tabular representation of the relation
Binary Relations on a Set
2
Example: Consider these relations on the set of integers:
 a, b a  b ,
  a, b a  b ,
  a, b a  b or a  b ,
 a, b a  b ,
  a, b a  b  1 ,
  a, b a  b  3 .
R1 
R4 
R2
R5
R3
R6
Note that these relations are on an infinite set and each of
these relations is an infinite set.
Which of these relations contain each of the pairs
(1,1), (1, 2), (2, 1), (1, −1), and (2, 2)?
Solution: Checking the conditions that define each relation, we see
that the pair (1,1) is in R1, R3, R4 , and R6: (1,2) is in R1 and R6: (2,1) is in
R2, R5, and R6: (1, −1) is in R2, R3, and R6 : (2,2) is in R1, R3, and R4.
Binary Relations on a Set
3
Question: How many relations are there on a set A?
Solution: Because a relation on A is a subset of A × A,
we count the subsets of A × A. Since A × A has n2
elements when A has n elements, and a set with m
elements has 2m subsets, therefore there are
𝑛2
2 relations on a set A.
Types of Relations
1
• Let R be a relation on a set A. There are
several properties that can be used to classify
R
• R can be classified as:
– reflexive
– symmetric
– antisymmetric
– transitive
Types of Relations
2
To explain the various types of relations, we will
use the following illustrative relations:
Let set A = {1, 2, 3, 4}
Let a,b ∈ Z
R1 = {(1, 1), (1, 2), (2, 1), (2, 2), (3, 4), (4, 1), (4, 4)},
RA = {(a, b) ∣ a ≤ b},
R2 = {(1, 1), (1, 2), (2, 1)},
RB = {(a, b) ∣ a > b},
R3 = {(1, 1), (1, 2), (1, 4), (2, 1), (2, 2), (3, 3), (4, 1),
(4, 4)},
RC = {(a, b) ∣ a = b or a = −b},
R4 = {(2, 1), (3, 1), (3, 2), (4, 1), (4, 2), (4, 3)},
R5 = {(1, 1), (1, 2), (1, 3), (1, 4), (2, 2), (2, 3), (2, 4),
(3, 3), (3, 4), (4, 4)},
RD = {(a, b) ∣ a = b},
RE = {(a, b) ∣ a = b + 1},
RF = {(a, b) ∣ a + b ≤ 3}.
R6 = {(3, 4)}
Roster Notation
Set Builder Notation
Types of Relations
3
• When classifying a relation based on a
property keep in mind that:
– there should be no violation of the pertinent
property
– there may be additional elements in the relation
that do not satisfy the pertinent property
• e.g. there may be additional elements in a reflexive
relation that do not satisfy the reflexive property
Reflexive Relations
1
Definition: A relation R on a set A is reflexive if (a,a) ∊ R for every element a ∊
A. In terms of quantifiers, ∀a((a, a) ∈ R), where the universe of discourse is
the set of all elements in A.
– Each element in set A is related to itself
Example: The following are examples of reflexive relations:
R3 = {(1, 1), (1, 2), (1, 4), (2, 1), (2, 2), (3, 3), (4, 1), (4, 4)}
– Contains all pairs of the form (a, a)
R5 = {(1, 1), (1, 2), (1, 3), (1, 4), (2, 2), (2, 3), (2, 4), (3, 3), (3, 4), (4, 4)}
– Contains all pairs of the form (a, a)
RA = {(a, b) ∣ a ≤ b}
– Every integer is less than or equal to itself
RC = {(a, b) ∣ a = b or a = −b}
– The first condition is always true: every integer is equal to itself
RD = {(a, b) ∣ a = b}
– Every integer is equal to itself
Reflexive Relations
2
Example: Is the “divides” relation on the set of
positive integers reflexive?
Solution:
Because a | a whenever a is a positive integer, the
“divides” relation is reflexive.
Note that if we replace the set of positive integers
with the set of all integers the relation is not
reflexive because by definition 0 does not divide 0.
Symmetric Relations
1
Definition: A relation R on a set A is called symmetric if (b, a) ∈ R whenever
(a, b) ∈ R, for all a, b ∈ A. In terms of quantifiers:
∀a∀b((a, b) ∈ R → (b, a) ∈ R)
– An element related to a second element implies the second element is also related
to the first element.
Example: The following are examples of symmetric relations:
R2 = {(1, 1), (1, 2), (2, 1)}
–
–
(1, 1) is trivially symmetric
(1, 2) and (2, 1) are both in the relation
R3 = {(1, 1), (1, 2), (1, 4), (2, 1), (2, 2), (3, 3), (4, 1), (4, 4)}
–
–
(1, 1), (2, 2), (3, 3) and (4, 4) are trivially symmetric
(1, 2) & (2, 1), (1, 4) & (4, 1) are both in the relation
RC = {(a, b) ∣ a = b or a = −b},
–
If a = b then b = a; If a = −b then b = −a
RD = {(a, b) ∣ a = b},
–
a = b implies that b = a.
RF = {(a, b) ∣ a + b ≤ 3}.
–
a + b ≤ 3 implies that b + a ≤ 3.
Symmetric Relations
2
Example: Is the “divides” relation on the set of
positive integers symmetric?
Solution:
a | b ⇏ b | a (e.g. 1 |2 but 2 ∤ 1)
∴ the relation is not symmetric
Antisymmetric Relations
1
Definition: A relation R on a set A such that for all a, b ∈ A, if (a, b) ∈ R and (b,
a) ∈ R, then a = b is called antisymmetric. In terms of quantifiers, ∀a∀b(((a, b)
∈ R ∧ (b, a) ∈ R) → (a = b))
– there are no pairs of distinct elements a and b with a related to b and b related to
a.
– for example, a ≤ b and b ≤ a implies that a = b
Notes:
1. The terms symmetric and antisymmetric are not opposites,
because a relation can have both of these properties or may lack
both of them
– the relation R1 = {(1, 1), (2, 2)} is both symmetric and antisymmetric
– the relation R2 = {(1, 3), (3, 1), (2, 3)} is neither symmetric nor antisymmetric
•
•
2.
(2, 3) ∈ R2 but (3, 2) ∉ R2 ∴ not symmetric
(1, 3), (3, 1) ∈ R2 but 1 ≠ 3 ∴ not antisymmetric
A relation cannot be both symmetric and antisymmetric if it contains
some pair of the form (a, b) in which a ≠ b
Antisymmetric Relations
2
Example: The following are examples of
antisymmetric relations. In each case there is no
violation of the antisymmetric property
•
•
•
R4 = {(2, 1), (3, 1), (3, 2), (4, 1), (4, 2), (4, 3)}
R5 = {(1, 1), (1, 2), (1, 3), (1, 4), (2, 2), (2, 3), (2, 4), (3, 3),
(3, 4), (4, 4)}
R6 = {(3, 4)}
Antisymmetric Relations
3
Example: Is the “divides” relation on the set of positive
integers antisymmetric?
Solution:
Let us assume that a | b and b | a
Since a and a are positive integers, there exist positive
integers m and n such that
b = am and a = bn
b = (bn)m
nm = 1, which yields n = 1 & m = 1
∴a=b
Hence the relation is antisymmetric
Transitive Relations
1
Definition: A relation R on a set A is called transitive if whenever (a,b) ∊ R and (b,c) ∊ R, then (a,c) ∊ R,
for all a,b,c ∊ A. In terms of quantifiers:
∀a∀b∀c(((a, b) ∈ R ∧ (b, c) ∈ R) → (a, c) ∈ R).
–
If the first element is related to the second element, and the second element is related to the third element,
then the first element must be related to the third element
Example: The following are examples of transitive relations:
R4 = {(2, 1), (3, 1), (3, 2), (4, 1), (4, 2), (4, 3)},
– (3, 2) , (2, 1), (3, 1) ∈ R4; (4, 2), (2, 1), (4, 1) ∈ R4; (4, 3), (3, 2), (4, 2) ∈ R4; (4, 3), (3, 1), (4, 1) ∈ R4
R5 = {(1, 1), (1, 2), (1, 3), (1, 4), (2, 2), (2, 3), (2, 4), (3, 3), (3, 4), (4, 4)}
– (1, 2) , (2, 3), (1, 3) ∈ R5; (1, 2), (2, 4), (1, 4) ∈ R5; (1, 3), (3, 4), (1, 4) ∈ R5; (2, 3), (3, 4), (2, 4) ∈ R5
R6 = {(3, 4)}
– No violation of Transitive Relation condition
RA = {(a, b) ∣ a ≤ b}
– a ≤ b and b ≤ c implies a ≤ c
RB = {(a, b) ∣ a > b}
– a > b and b > c implies a > c
RC = {(a, b) ∣ a = b or a = −b}
– a = ± b and b = ± c implies a = ± c
RD = {(a, b) ∣ a = b}
– a = b and b = c implies a = c
Transitive Relations
2
Example: Is the “divides” relation on the set of
positive integers transitive?
Solution:
Let us assume that a | b and b | c
Then there are positive integers k and l
such that b = ak and c = bl. Hence, c = a(kl), so a
divides c.
It follows that this relation is transitive.
One More Example
Let R be the following relation defined on the set {a, b, c, d}
R = {(a, a), (a, c), (a, d), (b, a), (b, b), (b, c), (b, d), (c, b), (c, c),
(d, b), (d, d)}
Determine whether R is: (a) reflexive. (b) symmetric (c)
antisymmetric (d) transitive
Solution:
– R is reflexive because R contains (a, a), (b, b), (c, c) and (d, d)
– R is not symmetric because (a, c) ∈ R but (c, a) ∉ R
– R is not antisymmetric because both (b, c) ∈ R and (c, b) ∈ R but
b≠c
– R is not transitive because, for example, (a, c) ∈ R and (c, b) ∈ R
but (a, b) ∉ R
21
Equivalence Relations
Definition 1: A relation on a set A is called an
equivalence relation if it is reflexive, symmetric,
and transitive.
Definition 2: Two elements a, and b that are
related by an equivalence relation are called
equivalent. The notation a ∼ b is often used to
denote that a and b are equivalent elements
with respect to a particular equivalence relation.
Congruence Modulo m
Example: Let m be an integer with m > 1. Show that the
relation R = {(a,b) | a ≡ b (mod m)}
is an equivalence relation on the set of integers.
Solution: Recall that a ≡ b (mod m) if and only if m divides
a − b.
•
Reflexivity: a ≡ a (mod m) since a − a = 0 is divisible by m since
0 = 0 ∙ m.
•
Symmetry: Suppose that a ≡ b (mod m). Then a − b is divisible
by m, and so a − b = km, where k is an integer. It follows that b −
a = (− k) m, or b = a + (-k) m so b ≡ a (mod m).
•
Transitivity: Suppose that a ≡ b (mod m) and b ≡ c (mod m).
Then m divides both a − b and b − c. Hence, there are integers k
and l with a − b = km and b − c = lm. We obtain by adding the
equations: a − c = (a − b) + (b − c) = km + lm = (k + l) m.
a  c   a  b   b  c   km  lm   k  l  m.
Therefore, a ≡ c (mod m).
Given a≡b (mod m) then
a mod m = b mod m
m | (a – b)
a = b + km
Divides
Example: Show that the “divides” relation on the set of
positive integers is not an equivalence relation.
Solution: The properties of reflexivity, and transitivity
do hold, but there relation is not transitive. Hence,
“divides” is not an equivalence relation.
• Reflexivity: a ∣ a for all a.
• Not Symmetric: For example, 2 ∣ 4, but 4 ∤ 2. Hence, the
relation is not symmetric.
• Transitivity: Suppose that a divides b and b divides c. Then
there are positive integers k and l such that b = ak and c = bl.
Hence, c = a(kl), so a divides c. Therefore, the relation is
transitive.
Representing Relations Using Digraphs
1
• Relations with a manageable number of elements
can be represented pictorially, using digraphs
Definition: A directed graph, or digraph, consists of a set V of
vertices (or nodes) together with a set E of ordered pairs of
elements of V called edges (or arcs). The vertex a is called the
initial vertex of the edge (a,b), and the vertex b is called the
terminal vertex of this edge.
– An edge of the form (a,a) is called a loop.
A drawing of the digraph with vertices a, b, c,
and d, and edges (a, b), (a, d), (b, b), (b, d),
(c, a), (c, b), and (d, b) is shown here.
25
Representing Relations Using Digraphs
2
• The relation R on a set A can be represented by a
digraph that has the elements of A as its
vertices and the ordered pairs (a, b), where (a, b)
∈ R, as its edges
• The digraph of the relation R = {(1, 3), (1, 4), (2,
1), (2, 2), (2, 3), (3, 1), (3, 3), (4, 1), (4, 3) } on set
A = {1, 2, 3, 4} is shown below
26
Determining which Properties a Relation
has from its Digraph
1
The directed graph representing a relation can be used to
determine the type of a relation
Reflexivity: A loop must be present at all vertices in the graph.
Symmetry: If (x,y) is an edge, then so is (y,x).
– All edges either loops or “anti-parallel”
Antisymmetry: If (x,y) with x ≠ y is an edge, then (y,x) is not an
edge.
– If no “anti-parallel” edges then no violation of anti-symmetry
Transitivity: If (x,y) and (y,z) are edges, then so is (x,z).
– All path of length 2 (e.g. b-a-c) accompanied by a
corresponding path of length 1 (e.g. b-c)
✓ ✖
Determining which Properties a
Relation has from its Digraph
2
Example: Determine the type of relation represented by the digraph
Solution:
No loops on all vertices, therefore not reflexive
Not all edges are anti-parallel, therefore not symmetric
Antiparallel edges between b and c, therefore not antisymmetric
The following paths of length 2 not accompanied by a corresponding
path of length 1, therefore not transitive
– (b,c) and (c, b) [(b, b) is missing]
– (c, b) and (b, c) [(c, c) is missing]
Determining which Properties a
Relation has from its Digraph
3
Example: Determine the type of relation represented by the
digraph
Solution:
Loops on all vertices, therefore reflexive
No anti-parallel edges, therefore not symmetric
No violation of anti-symmetric property
The following path of length 2 accompanied by a
corresponding path of length 1, therefore transitive
– (b,a); (a, c) & (b, c)
Combining Relations
1
Given two relations R1 and R2, we can combine them
using basic set operations to form new relations such
as R1 ∪ R2, R1 ∩ R2, R1 − R2, and R2 − R1.
Example: Let A = {1,2,3} and B = {1,2,3,4}. The relations
R1 = {(1,1),(2,2),(3,3)} and R2 = {(1,1),(1,2),(1,3),(1,4)}
can be combined using basic set operations to form
new relations:
R1  R2  1,1 , 1, 2 , 1,3 , 1, 4 ,  2, 2 3,3
R1  R2  1,1
R1  R2   2, 2 ,  3,3
R2  R1  1, 2 , 1,3 , 1, 4
CS 2305: Discrete Mathematics
for Computing I
Lecture 28
- KP Bhat
1
Combining Relations
1
Given two relations R1 and R2, we can combine them
using basic set operations to form new relations such
as R1 ∪ R2, R1 ∩ R2, R1 − R2, and R2 − R1.
Example: Let A = {1,2,3} and B = {1,2,3,4}. The relations
R1 = {(1,1),(2,2),(3,3)} and R2 = {(1,1),(1,2),(1,3),(1,4)}
can be combined using basic set operations to form
new relations:
R1  R2  1,1 , 1, 2 , 1,3 , 1, 4 ,  2, 2 3,3
R1  R2  1,1
R1  R2   2, 2 ,  3,3
R2  R1  1, 2 , 1,3 , 1, 4
Combining Relations
2
Example: (Set Builder Notation case)
Let R1 be the less than relation on the set of real numbers and let R2 be the greater
than relation on the set of real numbers, that is, R1 = {(x, y) ∣ x < y} and R2 = {(x, y) ∣ x >
y}. What are R1 ∪ R2, R1 ∩ R2, R1 - R2, R2 - R1, and R1 ⊕ R2?
Solution:
We note that (x, y) ∈ R1 ∪ R2 if and only if (x, y) ∈ R1 or (x, y) ∈ R2. Hence, (x, y) ∈ R1 ∪
R2 if and only if x < y or x > y. Because the condition x < y or x > y is the same as the
condition x ≠ y, it follows that R1 ∪ R2 = {(x, y) ∣ x ≠ y}.
Next, note that it is impossible for a pair (x, y) to belong to both R1 and R2 because it is
impossible that x < y and x > y. It follows that R1 ∩ R2 = ∅.
We also see that R1 - R2 = R1, R2 - R1 = R2
R1 ⊕ R2 = (R1 – R2) ∪ (R2 – R1) = R1 ∪ R2 = {(x, y) ∣ x ≠ y}.
Composition
1
Definition: Suppose
• R1 is a relation from a set A to a set B.
• R2 is a relation from B to a set C.
Then the composition (or composite) of R2 with R1, is a
relation from A to C where
• if (x,y) is a member of R1 and (y,z) is a member of R2, then (x,z) is
a member of R2∘ R1.
Composition
2
Example: What is the composite of the relations R and S, where R is the
relation from {1, 2, 3} to {1, 2, 3, 4} with R = {(1, 1), (1, 4), (2, 3), (3, 1), (3, 4)}
and S is the relation from {1, 2, 3, 4} to {0, 1, 2} with S = {(1, 0), (2, 0), (3, 1), (3,
2), (4, 1)}?
Solution: S ◦R is constructed using all ordered pairs in R and ordered pairs in
S, where the second element of the ordered pair in R agrees with the first
element of the ordered pair in S.
Computing all the ordered pairs in the composite, we find S ◦ R = {(1, 0), (1, 1),
(2, 1), (2, 2), (3, 0), (3, 1)}
1→1→0
1→4→1
2→3→1
2→3→2
3→1→0
3→4→1
Set1
Set3
Set2
(1, 0)
(1, 1)
(2, 1)
(2, 2)
(3, 0)
(3, 1)
Graphs and Graph Models
Section 10.1
Background Information
• Graphs are discrete structures consisting of
vertices and edges that connect these vertices.
They are used to represent arbitrary relationships
among data objects
• There are different kinds of graphs, depending on
whether edges have directions, whether multiple
edges can connect the same pair of vertices, and
whether loops are allowed
• Problems in almost every conceivable discipline
have been solved using graph models
Some Applications of Graphs
• Communications networks
• Information networks
• Software design
• Transportation networks
• Social networks
• Biological networks
• Sports Tournaments
It’s a challenge to find a subject to which graph theory has
not yet been applied.
Graphs
Definition: A graph G = (V, E) consists of a nonempty set V of vertices (or nodes)
and a set E of edges. Each edge has either one or two vertices associated with it,
called its endpoints. An edge is said to connect its endpoints.
Example:
This is a graph with four
vertices and five edges.
Remarks:
•
The graphs we study here are unrelated to graphs of functions studied in Chapter 2.
•
In graph models all that matters is the connections made by the edges, not the
particular geometry depicted. For example, the lengths of edges, whether edges
cross, how vertices are depicted, and so on, do not matter
•
A graph with an infinite vertex set is called an infinite graph. A graph with a finite
vertex set is called a finite graph. We restrict our attention to finite graphs.
Some Terminology
1
In a simple graph each edge connects two different vertices and no two edges connect the
same pair of vertices.
Multigraphs may have multiple edges connecting the same two vertices. When m different
edges connect the vertices u and v, we say that {u,v} is an edge of multiplicity m.
•
For example redundant links connects hosts in a computer network
An edge that connects a vertex to itself is called a loop.
•
For example a feedback loop connecting a data center to itself, for diagnostic purposes
A pseudograph may include loops, as well as multiple edges connecting the same pair of
vertices.
Example:
This pseudograph
has both multiple
edges and a loop.
Undirected vs Directed Graphs
1
• Depending upon the application being modelled, the
order of the vertices may or may not be significant.
There are consequently two types of graphs:
undirected (i.e. no notion of direction for the edges)
and directed (i.e. edges have a direction)
• For some applications we may need a graph where
some edges are undirected, while others are directed.
A graph with both directed and undirected edges is
called a mixed graph
• Graphs without any qualification refer to undirected
graphs
Directed Graphs
1
Definition: An directed graph (or digraph) G = (V, E)
consists of a nonempty set V of vertices (or nodes) and a
set E of directed edges (or arcs). Each edge is associated
with an ordered pair of vertices. The directed edge
associated with the ordered pair (u,v) is said to start at u
and end at v.
Remark:
• Links between web pages and college course prerequisites are
two applications where graphs need to be directed
• In undirected graphs the end points of an edge are not
ordered.
Some Terminology
2
A simple directed graph has no loops and no multiple edges.
Example:
This is a directed graph with
three vertices and four edges.
A directed multigraph may have multiple directed edges. When
there are m directed edges from the vertex u to the vertex v, we
say that (u,v) is an edge of multiplicity m.
Example:
In this directed multigraph the
multiplicity of (a,b) is 1 and the
multiplicity of (b,c) is 2.
Undirected vs Directed Graphs
Undirected Graph
• A graph G = (V, E) consists
of V, a nonempty set of
vertices (or nodes) and E, a
set of edges.
• Each edge has either one or
two vertices associated with
it, called its endpoints.
• An edge is said to connect
its endpoints.
2
Directed Graph
• A directed graph (or digraph)
(V, E) consists of a nonempty
set of vertices V and a set of
directed edges (or arcs) E.
• Each directed edge is
associated with an ordered
pair of vertices.
• The directed edge associated
with the ordered pair (u, v) is
said to start at u and end at v.
Graph Models:
Computer Networks
1
When we build a graph model, we use the appropriate type of graph to
capture the important features of the application.
To model a computer network where we are only concerned whether two
data centers are connected by a communications link, we use a simple graph.
This is the appropriate type of graph when we only care whether two data
centers are directly linked (and not how many links there may be) and all
communications links work in both directions.
Graph Models:
Computer Networks
To model a computer network
where we care about the number
of links between data centers, we
use a multigraph.
To model a computer network
with diagnostic links at data
centers, we use a pseudograph,
as loops are needed.
To model a network with multiple oneway links, we use a directed multigraph.
Note that we could use a directed graph
without multiple edges if we only care
whether there is at least one link from a
data center to another data center.
2
Graph Terminology: Summary
To understand the structure of a graph and to build a graph
model, we ask these questions:
•
Are the edges of the graph undirected or directed (or both)?
•
If the edges are undirected, are multiple edges present that connect the
same pair of vertices? If the edges are directed, are multiple directed
edges present?
•
Are loops present?
TABLE 1 Graph Terminology.
Type
Edges
Multiple Edges Allowed?
Loops Allowed?
Simple graph
Multigraph
Pseudograph
Simple directed graph
Directed multigraph
Mixed graph
Undirected
Undirected
Undirected
Directed
Directed
Directed and undirected
No
Yes
Yes
No
Yes
Yes
No
No
Yes
No
Yes
Yes
Basic Terminology
1
Definition 1. Two vertices u, v in an undirected graph G are called
adjacent (or neighbors) in G if there is an edge e between u and v.
Such an edge e is called incident with the vertices u and v and e is said
to connect u and v.
Definition 2. The set of all neighbors of a vertex v of G = (V, E),
denoted by N(v), is called the neighborhood of v. If A is a subset of V,
we denote by N(A) the set of all vertices in G that are adjacent to at
least one vertex in A. So, N  A 
Neighborhood of a
N  v .
v A
Definition 3. The degree of a vertex in a
undirected graph is the number of edges
incident with it, except that a loop at a
vertex contributes two to the degree of
that vertex. The degree of the vertex v is
denoted by deg(v).
subset of vertices
Degrees and Neighborhoods of Vertices
Example: What are the degrees and
neighborhoods of the vertices in the graphs G
and H?
Solution:
G: deg(a) = 2, deg(b) = deg(c) = deg(f ) = 4, deg(d ) = 1,
deg(e) = 3, deg(g) = 0.
N(a) = {b, f }, N(b) = {a, c, e, f }, N(c) = {b, d, e, f }, N(d) = {c},
N(e) = {b, c , f }, N(f) = {a, b, c, e}, N(g) =  .
H: deg(a) = 4, deg(b) = deg(e) = 6, deg(c) = 1, deg(d) = 5.
N(a) = {b, d, e}, N(b) = {a, b, c, d, e}, N(c) = {b},
N(d) = {a, b, e}, N(e) = {a, b ,d}.
Degrees of Vertices
Theorem 1 (Handshaking Theorem): If G = (V,E) is an undirected
graph with m edges, then
2m   vV deg  v
Proof:
Each edge contributes twice to the degree count of all vertices. Hence,
both the left-hand and right-hand sides of this equation equal twice
the number of edges.
Think about the graph where vertices represent the people at a party
and an edge connects two people who have shaken hands.
Basic Terminology
2
• A vertex of degree zero is called isolated.
– An isolated vertex is not adjacent to any vertex
– Vertex g in graph G is isolated.
• A vertex is pendant if and only if it has degree
one
– A pendant vertex is adjacent to exactly one other
vertex
– Vertex d in graph G is pendant.
Handshaking Theorem
Example 1: How many edges are there in a graph with 10 vertices of degree six?
Solution: Because the sum of the degrees of the vertices is
6  10 = 60, the handshaking theorem tells us that 2m = 60. So the number of edges
m = 30.
Example 2: If a graph has 5 vertices, can each vertex have degree 3?
Solution: This is not possible by the handshaking theorem, because the sum of the
degrees of the vertices 3  5 = 15 is odd.
Example 3: Show that the sum, over the set of people at a party, of the number of
people a person has shaken hands with, is even. Assume that no one shakes his or
her own hand.
Solution: Model this problem by letting the vertices of a graph be the people at
the party, with an edge between two people if they shake hands. Then the degree
of each vertex is the number of people the person that vertex represents shakes
hands with. By the Handshaking Theorem, the sum of the degrees is even (it is 2e).
Directed Graph Terminology
• The terminology for directed
graphs reflects the fact that
edges in directed graphs
have directions
Definition: Let (u,v) be an edge in G. Then u is the
initial vertex of this edge and is adjacent to v and v is
the terminal (or end) vertex of this edge and is adjacent
from u. The initial and terminal vertices of a loop are
the same.
23
Directed Graphs
2
For directed graphs the definition of the degree of a vertex is
refined to reflect the number of edges with this vertex as the
initial vertex and as the terminal vertex.
Definition: The in-degree of a vertex v, denoted deg−(v), is the
number of edges which terminate at v. The out-degree of v,
denoted deg+(v), is the number of edges with v as their initial
vertex. Note that a loop at a vertex contributes 1 to both the indegree and the out-degree of the vertex.
Example: In the graph G we have
deg   a   2, deg   b  2, deg   c   3, deg   d   2,
deg   e  3, deg   f   0.
deg   a   4, deg   b  1, deg   c   2, deg   d   2,
deg   e  3, deg   f   0.
Jump to long description
Special Types of Simple Graphs: Complete
Graphs
A complete graph on n vertices, denoted by Kn, is the
simple graph that contains exactly one edge between
each pair of distinct vertices.
Recall that in a simple graph each edge
connects two different vertices and no
two edges connect the same pair of
vertices.
Special Types of Simple Graphs: Cycles and
Wheels
A cycle Cn for n ≥ 3 consists of n vertices v1, v2 ,⋯ , vn, and
edges {v1, v2}, {v2, v3} ,⋯ , {vn-1, vn}, {vn, v1}.
A wheel Wn is obtained by adding an additional vertex to a
cycle Cn for n ≥ 3 and connecting this new vertex to each of
the n vertices in Cn by new edges.
Jump to long description
Special Types of Simple Graphs:
n-Cubes
1
An n-dimensional hypercube, or n-cube, Qn, is a
graph with 2n vertices representing all bit strings
of length n, where there is an edge between two
vertices that differ in exactly one bit position.
Jump to long description
Special Types of Simple Graphs:
n-Cubes
2
• We can construct the (n + 1)-cube Qn+1 from the n-cube
Qn by making two copies of Qn, prefacing the labels on
the vertices with a 0 in one copy of Qn and with a 1 in
the other copy of Qn, and adding edges connecting two
vertices that have labels differing only in the first bit.
0
0
0
0
1
1
1
1