(A) T(n) = 2T(n/2) + n
Solution: Note this has the form of mergesort and quicksort's best case. We know that's Theta(n lg n). You can use the Main Theorem here since f(n) = n1. And it's Case 2 since k = 1 and a = bk (2 = 21). So Case 2 tells use it's Theta(n lg n).
(B) T(n) = 2T(n/4) + n lg n
Solution: You can't use the exact form of the Main Theorem here, but you can use the Master Theorem. The critical exponent k is lg2/lg4 = 1/2. Keep in mind that the three cases of these theorems are about comparing nk to f(n), or in this case n0.5 to n lg n. Case 1 applies if f(n) grows slower (in a way) than nk -- you can see it doesn't. You can also see it's not Theta(nk), so Case 2 won't apply. Let's see if Case 3 applies. We need to find small positive values e and d where we can "bound" f(n) between nk+e and nk+d. Here k = 0.5, and we know that n lg n falls between n1 and n2. So if e=0.5 and d=1.5, Case 3 applies.
So we conclude T(n) is Theta(f(n)) or Theta(n lg n).
(C) T(n) = T(n/2) + lg n
Solution: You can't use the exact from of the Main Theorem here either, so we turn to the Master Theorem. But it turns out this is one of the situations where none of the cases can be applied. Here k = 0 since lg 1 is zero. So nk is just 1. Comparing f(n) = lg n to 1 should make you realize that Case 1 and Case 2 cannot apply, so let's try Case 3.
But, unlike the last problem, you cannot find small positive values e and d where we can "bound" f(n)=n0 between nk+e and nk+d. Why not? We know that any polynomial with degree greater than zero grows faster than a log function, so you cannot choose a positive value for e so that f(n) is BigOh(nk+e). So Case 3 does't apply.
We did solve this earlier using the iteration method. It's not too hard to do that, and the answer turns out to be Theta( (lg n)2). Also, there exist more complex versions of the Master Theorem that solve this directly.
Problem DC1) Examine the code and explanation on pp. 229-231 for the divide-and-conquer solution for the closest-pair of points problems.
(A) Write a clear explanation about how the algorithm sorts the points "in the strip" by their y-coordinate. Your explanation should include anything you feel is odd, unclear, or incorrect about the code given in the textbook.
Solution: First, one thing in the code is odd/unclear. In the non-recursive function closest_pair(), mergesort() is called once to sort the entire set of points by x-coordinate. A function with the same name is called at the 2nd line of recursive_cl_pair() to sort 3 or fewer points by y-coordinate. Might have been clearer to name these two functions with different names.
Now that we have understand that, why doesn't it need to sort by y-coordinate before processing points in the strip? It does, but maybe it's not clear how. First, notice that the function recursive_cl_pair() looks a lot like mergesort with extra stuff added. Ignoring the extra stuff, you see that it processes the first-half, processes the second-half, and then calls merge() -- and that merge merges according to y-coordinate. So recursive_cl_pair() really is mergesorting-by-y-coordinate, as long as it makes sure that small subproblems are sorted this way. That's why the 2nd line of recursive_cl_pair() is call to a separate mergesort function.
But if recursive_cl_pair() is sorting by y-coordinate as it goes along, doesn't this break the main strategy of the approach, which is to process points based on their x-coordinates? No, because they need to be ordered by x-coordinate so it can do process the halves recursively -- it's done with that by the time it calls merge(). The loop that follows merge() looks at all the points in both halves and picks out the ones that are in the strip. When looking at all these points, it doesn't matter what order they're in now.
So, the complete algorithm starts out sorting the points by x-coordinate, but when it finishes they're left sorted by y-coordinate. There's really two sorts going on here.
(B) Show how you would modify the code so that it does not do the unnecessary distance calculation for any point "in the strip" that is on the same side of the dividing line L as the current point v[k]. Also, state how this improves the order class of the algorithm's complexity.
Solution: The loop that checks points looks like this:
for k = 1 to t-1 for s = k+1 to min(t, k+7) delta = min(delta, dist(v[k], v[s]));We need to add some kind of test before that last line to prevent calls to the dist() function when points are on the same side of the line l. (That's an "ell" not a "one". Let me call it "ell" here to make it clear.) Just to make it easier to think about this, let's subtract ell from the x-coordinate of each point. So a point on the line will have value 0, points to the left will have negative values, and points to the right of ell will have positive values. We to not call the dist() function when both points' values are negative or when both values are positive. If one is positive and one is negative, we want to call it. What if we multiply the two values? It's only negative if they're different signs (i.e. they're on different sides of ell.) So add this if-statement to test this:
if ( (v[k].x-ell) * v[s].x-ell) < 0 ) delta = min(delta, dist(v[k], v[s]));We know this condition will be true for 3 of our 7 points, but not which ones. We do 3 fewer calls to dist() but this loop is still Theta(1) so it doesn't change the overall order-class for the entire algorithm. The big "win" here is not having to check all of the points above the current point v[k] but some small constant. That makes this part of the recurrence Theta(n) and not Theta(n2) which leads to the Theta(n lg n) overall complexity.
Problem Sort1) Your friend Alex has learned about the median-of-three method for choosing the pivot element for quicksort. Alex claims that using this changes the order-class complexity of quicksort in the worst-case. You're not sure you agree.
(A) Write the recurrence relation for the worst-case performance for Quicksort with this enhancement. Assume you are using a partition algorithm that requires n-1 comparisons for a list of size n.
Solution: The partition element cannot be the largest or smallest if it's the median-of-three, but it could be the 2nd largest or smallest. As we know, quicksort does most poorly if it divides as unevenly as possible. So the new worst-case is when we solve subproblems of size 1 and n-2. Finding the median of three items takes 3 comparisons (e.g. with insertion sort), so the new recurrence is: W(n) = W(1) + W(n-2) + n-1 + 3. Note that W(1) is also zero (no comparisons to sort one item).
(B) Give a convincing argument either for or against Alex's claim.
Solution: Alex is wrong. It's still Theta(n2) in the worst-case. You could solve this with the iteration method to see, but even if you look at this you'll see that if we have the worst-case every time, we'd solve subproblems of size n-2, n-4, n-6, n-8,... We'd get down the base-case twice as fast as we do with n-1, n-2, n-3,... That's not going to change it from Theta(n2).
But: it's hard to imagine an input where we'd always be so unlucky. In fact, the point of the median-of-three is that we are more likely to get something close to the true median. So A(n) is better, and the odds of getting the worst-case are lower.
Final word: finding the median of three items really isn't much less work than picking an element randomly (see the text), and this method is proven to be Theta(n lg n) in the average-case.
Problem Sort2) (Make sure you answer contains all three parts labeled a, b and c below.)
Lomuto's partition algorithm (p. 245) and Hoare's partition algorithm (p. 269) both work in linear time in terms
of number of comparisons. We could also count how often they swap elements. For each algorithm, (a) determine what
a worst-case input would look like, and (b) give a formula for the number of element swaps it does in this case.
(c) Compare the two algorithms and say which might be preferable if sort required swapping large data items.
Solution: For (a) and (b), looking at Lomuto's algorithm makes you see that it's possible that every value could be smaller than the partition element, and it would swap n-1 times at the line swap(a[h],a[k]) and then one more time at the end to put the partition element in its rightful place. To make matters worse, in this situation h == k so these swap are needless. (We can avoid these if we code this line as if (h!=k) swap(a[h],a[k]) of course!) The worst-case input would be an array where the partition element chosen was the largest element. W(n)=n total.
For (a) and (b) for Hoare's partition, if you think about it, both indices might move one-step towards the middle, swap, then repeat this. They'll meet in the middle, so the number of swaps here is roughly n/2. (Maybe you want to be precise; for this homework, it's OK if you are not. To be precise, if n is odd, then we do (n-1)/2 swaps. If n is even, then we do (n-2)/2 swaps. We can generalize these two cases to say we do floor( (n-1)/2 ) swaps.) Then we do one more to put the partition element in the right place, so the total is W(n)=floor(n/2). There are any number of worst-case inputs that would lead to this; it happens whenever it is true that all the elements on the left-side are > the partition element and all on the right-side are < the partition element.
For (c), it's clear that as written, Hoare's algorithm may do half as many swaps as Lomuto's for one call to partition. One ugly thing about Lomuto's. If we aren't picking the partition element randomly, then the worst-case for n leaves the array so that we get the worst-case performance for the next call on n-1 items. For Hoare's, note that the worst-case number of swaps here rearranges the array into the precise form that leads to the best-case in terms of number of recursive calls and key-comparisons -- the array will be divided into two equal parts. Could either of these n/2 sections lead to the worst-case number of swaps? Impossible to say -- it depends on the relative order they were in to begin with in their half of the array of size n. But the point before this is important: if you're unlucky enough to do the maximum number of swaps, then you'll benefit by doing fewer recursive calls.
It turns out when you implement them and run timing experiments that using Hoare's algorithm in quicksort is faster (though of course they're the same BigTheta). If you were swapping large data items when sorting, this is even more apparent. But in practice you shouldn't write sorts that swap big data items. Instead, create an array of pointers or references to the items and sort that. Your data stays where it is, but you have a list of pointers where you swap these pointers. (This point is moot for Java, where we always work with references to objects and not the objects themselves.)