All Focused Image by Focal Stacking

Side Note: First Draft on Apr 12 2011. This post is based on Dr. Michael S. Brown’s assignment for a course I took and enjoyed. Some of the test pictures are from the assignment.

Digital Camera has limited depth of field. It’s difficult to capture a single image with all the important elements of the scene in focus sometimes.

Focal stacking is a technique to “stack” multiple images with different focus into a single all focused image.

Below is an example borrowed from wikipedia,

If you look at the images carefully, you’ll find that the first image has the first part of the fly focused, the second image has the second part of the fly focused, and the third image is a combined all-focused image.

Focal Stacking Implementation in Matlab

Here I introduce a simple method of focal stacking assuming that the input image sequences are aligned.  The input image sequence can be found at another page here.

The idea is follows,

1. A focused image will have a lot of sharp edges. Edges can normally be found using derivative filters. Laplacian filter is used here to retrieve these sharp edges. Therefore, laplacian values can be used to indicate the sharpness level of each pixel.

2. We want the output image to be all-in-focus, but we also want the image to look smooth and natural. In other words, we want a smooth transition when we take pixels from different images. If we take 10 neighboring pixels from 10 different images, the output is likely to be noisy, as below,

A Focused Image based purely on Laplacian

Therefore, we want to somehow smooth the image without introduing noticeable blur. We can do this by smooth the Laplaican we computed for every image. Since Laplacian values are the choice indicator, we smoothed our choice in this manner.

The smooth can be done using an average filter, and the result I got is something below,

A Focused Image by Smoothed Laplacian

The image above is much less noisy than using purely laplacian. However, there’re still obvious noise at the boundary of the foreground (with a lot of sharp edges) and background (no sharp edges).

3. An error correction technique is used to improve the result. I still used Laplacian. Firstly, I obtain the sum of Laplacian from all input images, and this sum of laplacian image is considered to contain all the correct edges of the input images. We then start from the coordinate of the sharpest pixel in this sum image to pick up pixels for our output image. For every pixel, if its corresponding sum image pixel has a value bigger than 0 (the pixel falls on an edge), we select the pixel from the input image with highest laplacian value at that pixel. If its corresponding sum image pixel has a value of 0 (the pixel doesn’t fall on an edge), we select the same image as one its neighbors so the final output is smooth. The one image got selected again is the one with the highest Laplacian value. Below is the output,

A Focused Image by Laplacian with Reference to Sum of Laplacian of Individual Image

Focal Stacking in Photoshop

Photoshop has the functionalities to help one to do focal stacking. You can find details here. Below is the result I got using Photoshop CS5. Honestly, it’s slightly better than my Matlab result.

You can download the matlab code here.