Dr. Attiqa Rehman
Advanced Algorithms Analysis

Major objective of this course is:
• Design and analysis of modern algorithms
• Different variants
• Accuracy
• Efficiency
• Comparing efficiencies
• Motivation thinking new algorithms
• Advanced designing techniques
• Real world problems will be taken as examples
• To create feelings about usefulness of this course
Dr. Attiqa Rehman Advanced Algorithms Analysis

On successful completion, students will be able to
• Argue and prove correctness of algorithms
• Derive and solve mathematical models of problems
• Reasoning when an algorithm calls certain approach
• Analyze average and worst-case running times
• Integrating approaches in dynamic and greedy algos.
• Use of graph theory in problems solving
• Advanced topics such as
• Computational geometry, number theory etc.
• Several other algorithms such as
• String matching, NP completeness, approximate algorithms etc.
Dr. Attiqa Rehman Advanced Algorithms Analysis

Dr. Attiqa Rehman Advanced Algorithms Analysis

In this lecture we will cover the following
• What is Algorithm?
• Designing Techniques
• Model of Computation
• Algorithms as a technology
• Algorithms and other technologies
• Importance of algorithms
• Difference in Users and Developers
• Kinds of problems solved by algorithms
• Conclusion
Dr. Attiqa Rehman Advanced Algorithms Analysis

• A computer algorithm is a detailed step-by-step method for solving a problem by using a computer.
• An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time.
• An Algorithm is well defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output .
• More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output .
Input output
Dr. Attiqa Rehman Advanced Algorithms Analysis

• Most basic and popular algorithms are
– Sorting algorithms
– Searching algorithms
Which algorithm is best?
• Mainly, it depends upon various factors, for example in case of sorting
– The number of items to be sorted
– The extent to which the items are already sorted
– Possible restrictions on the item values
– The kind of storage device to be used etc.
Dr. Attiqa Rehman Advanced Algorithms Analysis

Problem
• The statement of the problem specifies, in general terms, the desired input/output relationship .
Algorithm
• The algorithm describes a specific computational procedure for achieving input/output relationship .
Example
• One might need to sort a sequence of numbers into non-decreasing order.
Algorithms
• Various algorithms e.g. merge sort, quick sort, heap sorts etc.
Dr. Attiqa Rehman Advanced Algorithms Analysis

• Brute Force
– Straightforward, naive approach
– Mostly expensive
• Divide-and-Conquer
– Divide into smaller sub-problems
• Iterative Improvement
– Improve one change at a time
• Decrease-and-Conquer
– Decrease instance size
• Transform-and-Conquer
– Modify problem first and then solve it
• Space and Time Tradeoffs
– Use more space now to save time later
Dr. Attiqa Rehman Advanced Algorithms Analysis

• Greedy Approach
– Locally optimal decisions, can not change once made.
– Efficient
– Easy to implement
– The solution is expected to be optimal
– Every problem may not have greedy solution
• Dynamic programming
– Decompose into sub-problems like divide and conquer
– Sub-problems are dependant
– Record results of smaller sub-problems
– Re-use it for further occurrence
– Mostly reduces complexity exponential to polynomial
Dr. Attiqa Rehman Advanced Algorithms Analysis

• Analysis
– How does system work?
– Breaking a system down to known components
– How components (processes) relate to each other
– Breaking a process down to known functions
• Synthesis
– Building tools
– Building functions with supporting tools
– Composing functions to form a process
– How components should be put together?
– Final solution
Dr. Attiqa Rehman Advanced Algorithms Analysis

• Problem
• Strategy
• Algorithm
– Input
– Output
– Steps
• Analysis
– Correctness
– Time & Space
– Optimality
• Implementation
• Verification
Dr. Attiqa Rehman Advanced Algorithms Analysis

• Design assumption
– Level of abstraction which meets our requirements
– Neither more nor less e.g. [0, 1] infinite continuous interval
• Analysis independent of the variations in
– Machine
– Operating system
– Programming languages
– Compiler etc.
• Low-level details will not be considered
• Our model will be an abstraction of a standard generic single-processor machine, called a random access machine or RAM .
Dr. Attiqa Rehman Advanced Algorithms Analysis

• A RAM is assumed to be an idealized machine
– Infinitely large random-access memory
– Instructions execute sequentially
• Every instruction is in fact a basic operation on two values in the machines memory which takes unit time .
• These might be characters or integers.
• Example of basic operations include
– Assigning a value to a variable
– Arithmetic operation (+, - , × , /) on integers
– Performing any comparison e.g. a < b
– Boolean operations
– Accessing an element of an array.
Dr. Attiqa Rehman Advanced Algorithms Analysis

• In theoretical analysis, computational complexity
– Estimated in asymptotic sense , i.e.
– Estimating for large inputs
• Big O, Omega, Theta etc. notations are used to compute the complexity
• Asymptotic notations are used because different implementations of algorithm may differ in efficiency
• Efficiencies of two given algorithm are related
– By a constant multiplicative factor
– Called hidden constant.
Dr. Attiqa Rehman Advanced Algorithms Analysis

First poor assumption
• We assumed that each basic operation takes constant time , i.e. model allows
– Adding
– Multiplying
– Comparing etc.
two numbers of any length in constant time
• Addition of two numbers takes a unit time!
– Not good because numbers may be arbitrarily
• Addition and multiplication both take unit time!
– Again very bad assumption
Dr. Attiqa Rehman Advanced Algorithms Analysis

Finally what about Our Model?
• But with all these weaknesses, our model is not so bad because we have to give the
– Comparison not the absolute analysis of any algorithm.
– We have to deal with large inputs not with the small size
• Model seems to work well describing computational power of modern nonparallel machines
Can we do Exact Measure of Efficiency ?
• Exact, not asymptotic, measure of efficiency can be sometimes computed but it usually requires certain assumptions concerning implementation
Dr. Attiqa Rehman Advanced Algorithms Analysis

• Analysis will be performed with respect to this computational model for comparison of algorithms
• We will give asymptotic analysis not detailed comparison i.e. for large inputs
• We will use generic uniprocessor random-access machine (RAM) in analysis
– All memory equally expensive to access
– No concurrent operations
– All reasonable instructions take unit time, except, of course, function calls
Dr. Attiqa Rehman Advanced Algorithms Analysis

• What, Why and Where Algorithms?
• Designing Techniques
• Problem solving Phases and Procedure
• Model of computations
– Major assumptions at design and analysis level
– Merits and demerits, justification of assumptions taken
• We proved that algorithm is a technology
• Compared algorithmic technology with others
• Discussed importance of algorithms
– In almost all areas of computer science and engineering
– Algorithms make difference in users and developers
Dr. Attiqa Rehman Advanced Algorithms Analysis