ANALYSIS OF ALGORITHMS
Binary Search Optimality
Suppose that we are considering a sorted list (in non-decreasing order) of n integers and
the problem is to find x in the list. We will show that Binary Search is optimal (in the
worst-case) among all (deterministic) algorithms that solve this problem.
Suppose that we have an algorithm B that is not Binary Search and is trying to find x.
This algorithm if it is deterministic, can be described using a binary tree. The first node
(the root) represents the first element that x will be compare with, let’s say Ai (not
necessarily the middle element).
A1 A2 ……… Ai ……………… An
Next suppose that x it is not equal to Ai , then if it is a smart algorithm, if x is less that Ai ,
it will look at an element to the left of Ai , let’s say Ad, otherwise it will compare x with
an element to the right of Ai , let’s say Am.
A1 A2 … Ad…… Ai ………… Am…… An
Clearly we can model this algorithm using a binary tree, where the left child of a node Aj,
represents the element of the array to the left of Aj , that we will compare x with if x is
less than Aj. If x is larger than Aj, we will compare x with an element to the right of Aj .
Ai
>
<
Ad
<
> <
Level 1
Am
>
Level 2
Level 3
Level 4 = depth
Clearly each node represents an element of the array, but the question is if perhaps there
is one element of the array, let’s say Ak, that is not represented by any node of the binary
tree. The answer is “no”. Why?.
Consider the following input:
x --- A1 A2 … Ad…… Ai ………… Am…x… An , where x appears exactly one time
in the array, at position k. Since the binary tree does not have a node representing Ak, the
output of the algorithm will show that x does not appear, an incorrect answer.
Let N be the number of nodes of the binary tree, clearly N  n.
Next we notice that the worst-case scenario is the length of the longest path between the
root an leave of the binary tree. Clearly this longest path is between the root and a node at
Level=depth.
Example:
Suppose that n=13 and we are considering algorithm A5, where we jump every 3 places,
starting by comparing x with A1. We can described this algorithm by the following binary
tree:
A1
A4
A7
A3
A2
A10
A6
A5
A13
A9
A8
A12
A11
The depth of this binary tree is 7 (seven) thus the worst-case WB (13)=7.
Thus the ideal binary search tree will have the smallest possible depth. Thus the answer is
that this binary tree should have every level full (2level-1 nodes per level) except of course
at Level=depth.
Thus if we have 13 nodes the ideal binary tree should look like
When all the levels
are full except for
the last one
(Level=depth) we
call this binary tree
a balanced binary
tree.
Lemma1: A balanced binary tree with n nodes has depth=  log n  + 1  log n.
Since as we showed previously the number of nodes of the tree must at least n (the
number of elements of the array), then for any algorithm B the worst-case,
WB (n)  log n.
Thus if we have an algorithm that can be described by a balance binary tree that has
exactly n nodes, then the worst-case is optimal.
Theorem 1: The corresponding binary tree of Binary Search is a balanced tree with n
nodes.
Consider the corresponding binary tree for Binary Search on a list of 13 elements:
A7
A3
A10
A1
A5
A2
A4
A6
A8
A9
A12
A11
A13