Design and Analysis of
Algorithms
Faculty Name : Ruhi Fatima
Email: r.mazharuddin@mu.edu.sa
Course Description
This course provides techniques to prove the correctness
and to analyze the running time of the basic algorithms for
those classic problems in various domains; to apply the
algorithms and design techniques to solve problems; and to
analyze the complexities of various problems in different
domains.
Introduction to Algorithm
Topics Covered
•
•
•
•
•
•
Study of Algorithms
Algorithms as a technology
Insertion sort
Analyzing algorithm
Designing algorithm
Turing Machine
Why Study this Course?
• Donald E. Knuth stated “Computer Science is the study of
algorithms”
– Cornerstone of computer science. Programs will not exist without
algorithms.
• Behind every successful programming project there is a good
design. So if the underlying design is bad, then often no
amount of fine-tuning is going to make a substantial
difference
• Usually design of any algorithm is chosen from existing
techniques that are discussed in data structure course and
based upon the selection proper design steps are formed
• Help to develop the ability of analyzing and solving problems
via computers
• To be sure that your design is the most suitable design for
the underlying problem, you must develop the ability of
analyzing the design you made.
What is an Algorithm?
• Informally, an algorithm is any well-defined computational
procedure that takes some value, or set of values, as input and
produces some value, or set of values, as output. An algorithm is
thus a sequence of computational steps that transform the input into
the output.
• We can also view an algorithm as a tool for solving a well-specified
computational problem. The statement of the problem specifies in
general terms the desired input/output relationship. The algorithm
describes a specific computational procedure for achieving that
input/output relationship.
problem
algorithm
input
“computer”
output
Course Objectives
• This course introduces students to the analysis and design of
computer algorithms. Upon completion of this course, students will
be able to do the following:
– Analyze the asymptotic performance of algorithms.
– Demonstrate a familiarity with major algorithms and data structures.
– Apply important algorithmic design paradigms and methods of
analysis.
– Synthesize efficient algorithms in common engineering design
situations.
In this course we will cover the following topics:
• Understand foundations of algorithms && design and analysis
various variants algorithms
Accuracy ,
Efficiency,
Comparing efficiencies
• Make use of skills to understand mathematical notations in
algorithms and their simple mathematical proofs
• Gain familiarity with a number of classical problems that occur
frequently in real-world applications
Fundamentals of Algorithmic Problem Solving
• Understanding the problem
– Asking questions, doing few examples by hand, think about special cases, etc.
• Deciding on
– Exact vs. approximate problem solving, Appropriate data structure
• Design an algorithm
• Proving correctness
• Analyzing an algorithm
– Time efficiency : how fast the algorithm runs
– Space efficiency: how much extra memory the algorithm needs.
Any algorithm can be analyzed when we study it as a pure mathematical entity,
and so ignore issues such as programming language, machine, and operating
system.
• Coding an algorithm
What kinds of Problems are solved by Algorithms
• Practical applications of algorithms are ubiquitous and include the following
examples:
• The Human Genome Project has made great progress toward the goals of
identifying all the 100,000 genes in human DNA, determining the
sequences of the 3 billion chemical base pairs that make up human DNA,
storing this information in databases, and developing tools for data
analysis. Each of these steps requires sophisticated algorithms.
Cont…
• The Internet enables people all around the world to quickly
access and retrieve large amounts of information. With the aid
of clever algorithms, sites on the Internet are able to manage
and manipulate this large volume of data. Examples of
problems that make essential use of algorithms include finding
good routes on which the data will travel and using a search
engine to quickly find pages on which particular information
resides.
• There are two characteristics that are common to many
interesting algorithmic problems:
1. They have many candidate solutions, so finding one that is
“best,” can present quite a challenge.
2. They have practical applications. Like, finding the shortest
path provides the easiest examples. Or a routing node on the
Internet may need to find the shortest path through the network
in order to route a message quickly.
Algorithm as technology
• Computers may be fast, but they are not infinitely fast. And memory may
be inexpensive, but it is not free. Computing time is therefore a bounded
resource, and so is space in memory. So, these resources should be used
wisely, and algorithms that are efficient in terms of time or space will help.
• Efficiency : Different algorithms devised to solve the same problem often
differ dramatically in their efficiency. These differences can be much more
significant than differences due to hardware and software.
• Example: Consider two algorithms for sorting. The first, insertion sort,
takes time roughly equal to c1n2 to sort n items, where c1 is a constant that
does not depend on n. The second, merge sort, takes time roughly equal
to c2n lg n, and c2 is another constant that also does not depend on n.
Consider faster computer (computer A) running insertion sort against a slower
computer (computer B) running merge sort. They each must sort an array of
10 million numbers. Computer A takes
Computer B takes
Insertion sort Algorithms
Input: A sequence of n numbers < a1 , a2, …. , an >.
Output: A permutation (reordering) < a1’ , a2’, …. , an’ > of the input sequence
such that a1’ ≤ a2’ ≤ …. ≤ an’
•
Merge Sort
The divide-and-conquer paradigm involves three steps at each level of the
recursion:
Divide the problem into a number of subproblems that are smaller instances of
the same problem.
Conquer the subproblems by solving them recursively. If the subproblem sizes
are small enough, however, just solve the subproblems in a straightforward
manner.
Combine the solutions to the subproblems into the solution for the original
problem.
• The merge sort algorithm closely follows the divide-and-conquer paradigm.
Divide: Divide the n-element sequence to be sorted into two subsequences of
n=2 elements each.
Conquer: Sort the two subsequences recursively using merge sort.
Combine: Merge the two sorted subsequences to produce the sorted answer.
Cont…
Analyzing divide-and-conquer algorithms
When an algorithm contains a recursive call to itself, often its running time is described
by a recurrence equation or recurrence, which describes the overall running time on
a problem of size n in terms of the running time on smaller inputs.
The mathematical tools can be used to solve the recurrence and provide bounds on
the performance of the algorithm.
Cont…
• Analyzing divide-and-conquer algorithms : When an algorithm contains
a recursive call to itself, often its running time is described by a recurrence
equation or recurrence, which describes the overall running time on a
problem of size n in terms of the running time on smaller inputs.
• The mathematical tools can be used to solve the recurrence and provide
bounds on the performance of the algorithm.
• The recurrence is
• Where D(n ) time to divide the problem into subproblems and C(n ) time to
combine the solutions to the subproblems into the solution to the original
problem.
• The divide step just computes the middle of the subarray, which takes
constant time. Thus, D(n )= Θ(1 ).
• Two subproblems are recursively solved, each of size n=2, which
contributes 2T(n/2) to the running time.
• We have already noted that the MERGE procedure on an n-element
subarray takes time Θ(n ), and so C(n )= Θ(n ).
Cont…
• Adding above functions to recurrence gives
• Master theorem, which can be used to show that T(n) is Θ (n lg n).
• With Merge sort Θ (n lg n) running time, outperforms insertion sort, whose
running time is Θ (n2), in the worst case.
• Let us rewrite recurrence as
• where the constant c represents the time required to solve problems of size
1 as well as the time per array element of the divide and combine steps.
• After recursively solving the above recurrence and ignoring the low-order
term and the constant c gives the desired result of Θ (n lg n).
Turing Machine
10 01 1 10 1 1 1 1 1 1 0 1 1 0 1 1 0 0 0
•
A TM consists of an infinite length tape, on which input is
provided as a finite sequence of symbols.
• A head reads the input tape. The TM starts at start state s0.
On reading an input symbol it optionally replaces it with
another symbol, changes its internal state and moves one
cell to the right or left.
• Example: EG Successor Program
Sample Rules:
If read 1, write 0, go right, repeat.
If read 0, write 1, HALT!
If read ,•write 1, HALT!
Let’s see how they are carried out on a piece of paper that
contains the reverse binary representation of 47:
1
1
1
1
0
1
0
1
1
1
0
1
0
0
1
1
0
1
0
0
0
1
0
1
0
0
0
0
0
1
0
0
0
0
1
1
So the successor’s output on 111101 was
000011 which is the reverse binary
representation of 48.