How to align a stack of images for a timelapse with FOSS tools

I have taken several series of frames that I intend to process into timelapse sequence. When I simply ran StopMotion to process them into a movie sequence, I realized the image appeared jerky for two main reasons:

• Some series were taken in windy conditions causing the camera tripod to shake a bit.


• Others were taken with a smartphone camera in HDR mode. Since these images are already processed, their field of view varies slightly, which is visible when viewing them in sequence. It looks as if the zoom level were slightly altered between them – however, sometimes the variation is only along one axis, with the image being slightly distorted. Since this is a post-processing artifact, tools that rely on lens geometry may not work here. I've made that part a separate question.

How can I align these images so that the corresponding image sequence will not appear jerky? Constraints are:

• Only FOSS tools that are available on Linux


• Each sequence consists of around 250 images (equivalent to 10 seconds at 25 fps), thus a high level of automation is needed. Some of the approaches presented here work well for a HDR sequence of 3 exposures but don't lend themselves well to longer sequences.

Answer

Hugin has a tutorial on their website, which made for a good starting point.

Tools needed:

• Hugin



• ImageMagick


• Bash shell

Create a Hugin project

Start Hugin and be sure to select Interface > Advanced (Expert will do, too). Go to the Panorama Stitcher window.

Set the Field of View (50° worked well for my smartphone camera), lens parameters and projection type. Rectilinear should work for most cases.

Load the images for your sequence.

Create control points

Control points are features in the image which Hugin uses for alignment. Each pair of consecutive frames must share control points in order to align them properly.

Still in the Photos tab, under Feature Matching, select a strategy for creating control points. Align image stack worked well for me, you might have to experiment a bit.

Before doing this for the first time, you need to ensure control points are created between pairs of chronologically consecutive images (default is to sort images by exposure). Go to File > Preferences and navigate to the Control Point Detectors tab. Select Align image stack and click Edit…. Under Arguments, add --use-given-order. Then close both dialogs with OK.

Select the first bunch of photos, and click Create control points. Unfortunately, Hugin cannot process more than slightly below 128 image in one go. If you have more than 120 images in your series, just repeat this step in batches of 120 images. Be sure to include the last image of each batch in the next batch as well. Processing may take a while.

Now move to the Control Points tab. Select image #0 in the left pane and image #1 in the right one. You should see the control points shared between the images. Click the right arrow button to move forward in the series. Each pair of consecutive images should share a set of control points. (This may not always work when the camera is moving, e.g. when shooting time-lapse sequences from a moving vehicle.)

If your sequence has large portions of clouds, the previous step might have created some control points in the sky. Since these would mess up the alignment, we need to remove them: Go back to the Photos tab, select all images, right-click the list and select Control points > Remove control points on clouds. This will take a while. It may miss some control points, but in my experience it reduces their number enough for alignment to work as expected.

Note: Apparently, the latest versions of Hugin can exclude clouded areas already during control point creation, depending on which strategy is chosen.

Check lens parameters

Go to the Fast panorama preview window (View > Fast Preview window) and check the sphere preview. The image should fill only a portion of the sphere, with near-rectangular contours and the edges only minimally distorted (similar to the old-fashioned CRT TVs).

If the image contours are nearly circular, you may need to readjust your lens parameters. (I found this happened with HDR images – apparently the post-processing in the device discards lens information.) Go back to Panorama Stitcher, and on the Photos tab, select all images and right-click the list. Then either:

• Change them to known good lens parameters (Lens) – the lens of your camera should be in the database if you've previously processed a non-HDR image from the same device



• Or manually set the field of view (Edit image variables).

Then check again in Fast Panorama Preview.

Calculate alignment

Back in the Panorama Stitcher window, in the Photos tab, select one of the following under Optimize > Geometric:

• Positions (y, p, r) to correct for camera movement only


• Positions and View (y, p, r, v) to also eliminate HDR processing artifacts this doesn't yet work for the artifacts described above, where images are slightly distorted along one axis. Still trying to figure out how to deal with this...

Click Calculate. This will take some time again.

Aligning and Cropping

Change to the Fast panorama preview window.

On the Projection tab, select the same projection as your lens type. On the Move/Drag tab, click Fit, and on the Crop tab, select HDR Autocrop.

Render the frames

Back in the Panorama Stitcher window, go to the Stitcher tab.

Under Field of View, click Calculate Field of View. (This might already be set correctly.)

Under Canvas Size, click Calculate Optimal Size. Then, under Crop, click Fit Crop to Images. (Don't use the crop option here as it may leave you with blank areas near the borders of some images.)

Deselect everything under Panorama outputs.

Under Remapped images, select No exposure correction, low dynamic range.

Click Stitch. You will be asked to select a prefix – this is a path and a string that will be prepended to each generated file. I recommend placing the generated files in a separate directory.

Rendering the frames will take a while again.

Cleaning up the images

After Hugin has finished its work, you will end up with a series of images. Only two things are missing:

• The images are in TIFF format – this is the format Hugin uses internally, and there is no way to change that.


• After aligning the images, Hugin cropped them to the portion which is convered by every frame in the series. Thus, the resulting images will be slightly smaller than the original ones.

Fortunately, these two things can be fixed in one single step. As a bonus, if you took the exposures at a resolution higher than your target resolution for the movie sequence, this step will resize them as well.

Start up bash, move to the directory that holds the generated files, then run:

for i in *.tif ; do convert $i -resize 1920x1080^ -gravity center -extent 1920x1080 `echo $i | sed s/.tif/.jpg/` ; done

rm *.tif

(if your target resolution is something other than 1920×1080, change the first line accordingly.)

This will scale each image to fill your target resolution and crop it to size if the proportions don't quite fit. The second line will remove the files generated by Hugin.

That's it! Now use your favorite tool to process the resulting sequence of images into a movie clip.