As-projective-as-possible image stitching with moving DLT

Abstract

The last ten years have witnessed important advances in image stitching algorithms. Such advances have allowed the development of several commercial tools that are based on or incorporate image stitching. Amongst these tools there are well known image editing suites like Adobe Photoshop, Microsoft’s Image Composite Editor which is part of the web-based photo organization tool Photosynth, “dedicated” stitching software like Autostitch and its commercial counterparts AutoPano and AutoPano Giga, the image stitching functionality of the iOS from Apple, as well as the built-in stitching functionality of several off-the-shelf digital cameras. The widespread availability of stitching tools often leads to the impression that image stitching is a solved problem. The reality is: many of these tools often fail to produce convincing results when given non ideal data, i.e., images that deviate from fairly restrictive assumptions of image stitching; the main two being that the photos correspond to views that differ purely by rotation, or that the imaged scene is effectively planar. Such assumptions underpin the usage of 2D projective transforms or homographies to align the photos. In the hands of the casual user, these conditions are often violated, yielding misalignment artifacts or “ghosting” in the results. Accordingly, many existing image stitching tools depend critically on post-processing routines to conceal ghosting. This thesis proposes a novel estimation technique called Moving Direct Linear Transformation (Moving DLT) that is able to “tweak” or fine-tune the projective warp to accommodate the deviations of the input data from the idealised conditions. This produces “as-projective-as-possible” image alignments that significantly reduce ghosting without compromising the geometric realism of perspective image stitching. The Moving DLT technique lessens the dependency on potentially expensive post-processing algorithms. In addition, this thesis also describes how Moving DLT can be performed in a “bundled” manner to simultaneously align multiple images in order to generate “long” panoramas while reducing the error propagation of the incremental stitching techniques. It is important to note that such a bundle adjustment formulation, which we call Bundled Moving DLT, is the first of its kind. There is no other bundle adjustment formulation that is able to simultaneously refine multiple non-rigid warps for image stitching. The experimental results show that Moving DLT (and Bundled Moving DLT) can produce much better results than current state-of-the-art image stitching software and other recent methods for image stitching.