|
1
|
|
|
2
|
- Asymptotic performance: How does algorithm behave as the problem size
gets very large?
- Running time
- Memory/storage requirements
- Remember that we use the RAM model:
- All memory equally expensive to access
- No concurrent operations
- All reasonable instructions take unit time
- Except, of course, function calls
- Constant word size
- Unless we are explicitly manipulating bits
|
|
3
|
- Number of primitive steps that are executed
- Except for time of executing a function call most statements roughly
require the same amount of time
- We can be more exact if need be
- Worst case vs. average case
|
|
4
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
5
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
6
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
7
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
8
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
9
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
10
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
11
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
12
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
13
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
14
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
15
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
16
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
17
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
18
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
19
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
20
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
21
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
22
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
23
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
24
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
25
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
26
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
27
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
28
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
29
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
30
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
31
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
32
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
33
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
34
|
- Check out the Animator, a java applet at:
http://www.cs.hope.edu/~alganim/animator/Animator.html
- Try it out with random, ascending, and descending inputs
|
|
35
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
36
|
- InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
- }
|
|
37
|
- Statement Effort
- InsertionSort(A, n) {
- for i = 2 to n { c1n
- key = A[i] c2(n-1)
- j = i - 1; c3(n-1)
- while (j > 0) and (A[j] > key) { c4T
- A[j+1] = A[j] c5(T-(n-1))
- j = j - 1 c6(T-(n-1))
- } 0
- A[j+1] = key c7(n-1)
- } 0
- }
- T = t2 + t3 + … + tn where ti
is number of while expression evaluations for the ith for loop iteration
|
|
38
|
- T(n) = c1n + c2(n-1) + c3(n-1) + c4T
+ c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1)
= c8T + c9n
+ c10
- What can T be?
- Best case -- inner loop body never executed
- ti = 1 è T(n) is a linear function
- Worst case -- inner loop body executed for all previous elements
- ti = i è T(n) is a quadratic function
- Average case
|
|
39
|
- Simplifications
- Ignore actual and abstract statement costs
- Order of growth is the interesting measure:
- Highest-order term is what counts
- Remember, we are doing asymptotic analysis
- As the input size grows larger it is the high order term that
dominates
|
|
40
|
- We say InsertionSort’s run time is O(n2)
- Properly we should say run time is in O(n2)
- Read O as “Big-O” (you’ll also hear it as “order”)
- In general a function
- f(n) is O(g(n)) if there exist positive constants c and n0 such
that f(n) £ c × g(n) for all n ³ n0
- Formally
- O(g(n)) = { f(n): $
positive constants c and n0 such that f(n) £ c × g(n) " n ³ n0
|
|
41
|
- Proof
- Suppose runtime is an2
+ bn + c
- If any of a, b, and c are less
than 0 replace the constant with its absolute value
- an2 + bn + c £ (a + b + c)n2 + (a + b + c)n + (a +
b + c)
- £ 3(a + b + c)n2
for n ³ 1
- Let c’ = 3(a + b + c) and let n0 = 1
- Question
- Is InsertionSort O(n3)?
- Is InsertionSort O(n)?
|
|
42
|
- A polynomial of degree k is O(nk)
- Proof:
- Suppose f(n) = bknk + bk-1nk-1
+ … + b1n + b0
- f(n) £ aknk
+ ak-1nk-1 + … + a1n + a0
|
|
43
|
- We say InsertionSort’s run time is W(n)
- In general a function
- f(n) is W(g(n))
if $
positive constants c and n0 such that 0 £ c×g(n) £ f(n) " n ³ n0
- Proof:
- Suppose run time is an + b
- Assume a and b are positive (what if b is negative?)
- an £ an + b
|
|
44
|
- A function f(n) is Q(g(n))
if $
positive constants c1, c2, and n0 such
that
c1 g(n) £ f(n) £ c2 g(n) " n ³ n0
- Theorem
- f(n) is Q(g(n))
iff f(n) is both O(g(n)) and W(g(n))
- Proof: someday
|
|
45
|
|
|
46
|
|
|
47
|
|
|
48
|
|
|
49
|
|
|
50
|
- A function f(n) is o(g(n)) if $ positive constants c and n0 such that
f(n) < c g(n) " n ³ n0
- A function f(n) is w(g(n))
if $
positive constants c and n0 such that
c g(n) < f(n) " n ³ n0
- Intuitively,
|
|
51
|
- Solving recurrences
- Substitution method
- Master theorem
|
Notes