Final - Example Based Super-Resolution

Advanced Computer Graphics
• Final - Example Based Super-Resolution

Due: May 6

1. Training Data

I aggregated training data by sample patches from a series of small 32x32 images. I downsampled, upsampled, and then took the difference between this and the original image to form the band pass filtered version of the image. I then performed this same operation to produced a band pass filtered image of a lower octave. This was then upsampled to the original resolution. Here are the high res and low res parts respectively:

For each 5x5 patch in the higher resolution band pass filtered image, I found the corresponding 7x7 patch in the lower resolution image. I store the 7x7 patch concatenated with the top and side of the 5x5 high resolution patch, resulting in a vector of size 49+5+4 = 58. I also stored the entire high resolution 5x5 patch:

Here is an image showing the correspondance between the vectors in this matrix and the corresponding high and low patches:

As you can see, the training vectors are the concatenation of the low resolution patch and the border of the high resolution patch as desribed above.

2. Image Reconstruction

Now I wish to reconstruct a higher resolution image from an input image. This process is done by finding low resolution patches in the input that are close in Euclidean distance to patches in the training set and outputing corresponding high resolution data.
The input image is first downsampled then upsampled and then subtracted from the the input to create a candidate low resolution band passed image. In order to make the comparisons, this image is actually upsampled to the target output resolution. Low resolution patches from this upsampled input are then iteratively concatenated with already selected high resolution borders to form the candidate vectors that are compared to the training set. The matching algorithm for patches is structured such that if I do not have information for one or all of the borders, the candidate and training vectors are weighted to neglect these parts. Here is an image that explains this iteration in a graphical sense:

A small note: when I traverse the input upsampled input, I loop over each 5x5 patch, thereby lining up the border for the candidate with the border that I made in the training data. This means also that low resolution patches overlap, which they also did in the training data if you were keeping track :).
Once I have constructed the entire high resolution band pass filtered image, I can construct the original high resolution version of the input by just summing the upsampled input image and this constructed image:

	INPUT	OUTPUT	BICUBIC
TEST 1
TEST 1