Notes 13 February 2006

University of Virginia Computer Science
CS216: Program and Data Representation, Spring 2006

(none)
21 February 2006

CS216 Notes Section 4 (13 February 2006)

Dynamic Subset Sum Algorithm

Input: set of n positive integers, I = {w₀, ..., w_n-1}, maximum weight W
Output: a subset S of I such that the sum of the elements of S ≤ W and there is no subset of I whose sum is greater than the sum of S and ≤ W.

First attempt:

S(w) = best subset with sum ≤ w

Initially: S(0) = { }
Elements of I are positive integers, so no subset of them has weight ≤ 0.
Recursion:
S(i) =
     best = S (i - 1)
     for e in I:
        if i - e >= 0:
           if sum(S (i - e)) + e > sum(best):
               best = S(i - e) + [ e ]
     return best     
(note that the dynamic programming implementation would turn this around to avoid the multiple computations of S(k)).

Prove this algorithm doesn't work. (Hint: is adding e to S(w) always valid?)

Second attempt:

S(w, k) = best subset with sum ≤ w using only elements {i₀, ..., i_{k -
1}}

Initially: S(0, k) = { } for all k
and S(w, 0) = { } for all w
Justification:

Recursion:
S(w, k) =

Greedy Algorithms

To develop a greedy algorithm, we need to invent a myopic criterion for picking at each step.

We can prove an algorithm is not a correct solution to the problem by finding some input on which it does not produce the correct result. For optimization problems, we sometimes say "optimal" instead of "correct". A non-optimal algorithm produces an answer on all inputs, but it does not produce the best answer on some input.

We can prove an algorithm is optimal by showing that it always finds a solution that is at least as good as the best solution. Note that for many optimization problems there are many solutions that are equally good, as we saw with the Phylogenetic trees in PS2. So, to be optimal the algorithm needs to find one of these best solutions.

Common strategies for constructing such a proof are:

Induction — show that the greedy algorithm is always best according to some criterion at every step, and that by being best according to that criterion after the last step, we know there is no better result.
Exchange — show that changing any of the decisions made by the greedy algorithm cannot produce a better result. If we can show this for all decisions it makes, we know the result is best.

Both of these strategies need to be applied very carefully, especially the exchange strategy. It may be the case that a single exhange cannot improve the solution, but for the proof to be correct we need to know that multiple exchanges do not lead to a better solution.

Minimum Spanning Trees

Note: this is covered in Section 12.3 of the book, which we have not assigned yet. You are welcome, but not expected to read this now. We are just using it as a good example of a greedy algorithm.

Problem:

Input: a connected, undirected graph G = < V, E > (a set of vertices and weighted edges). Each edge is a triple, < from, to, weight > where from and to are vertices in V and weight is a positive integer.
Output: a subset S of E that connects all vertices in V where there is no subset of E that connects all the vertices with total weight less than the total weight of S. (Total weight is the sum of the weights of edges.)

Brute force algorithm:

Running time:

Greedy Approach 1

Myopic criterion:

Correctness:

Running time:

Greedy Approach 2:

Myopic criterion:

Correctness:

Running time:

[an error occurred while processing this directive]