A method for aligning video images according to sequence. The problem of image alignment has been extensively studied, and successful approaches have been developed for solving this problem. However, these approaches turn out as problematic when there is insufficient overlap between the two images to allow extraction of common image properties, i.e., when there is no sufficient similarity (e.g., gray-level, frequencies, statistical) between the two images. Whereas two individual images cannot be aligned when there is no spatial overlap between them, this is not the case when dealing with image sequences. The outlined technology consists of fusion and alignment of discrete, non-overlapping moving images from different sources, by aligning spatio-temporal changes in each sequence rather than in each image.
- Multi-sensor image alignment for multi-sensor fusion
- Alignment of images (sequences) obtained at significantly different zooms (can be useful in surveillance applications)
- Generation of wide-screen movies from multiple non-overlapping narrow field-of-view movies (such as in IMAX movies)
- Alignment and integration of information across video sequences to exceed the physical visual limitations of any individual sensor (e.g., dynamic range, spectral range, spatial resolution, temporal resolution, etc). ~
- Useful for spatially non-overlapping sequences
- Useful in cases which are inherently difficult for standard image alignment techniques, such as when there is insufficient common spatial information across the two sequences
An image sequence contains much more information than any individual image frame does. In particular, temporal changes in a video sequence (e.g., due to camera motion) do not appear in any individual image frame, but are encoded between video frames. When these temporal changes are common to the two sequences, then these sequences can be aligned both in time and in space, even if there is no common spatial information whatsoever. The need for coherent visual appearance, which is a fundamental assumption in image alignment methods, is replaced in this invention with the requirement of coherent temporal behavior. This can be achieved by attaching the two video cameras closely to each other (so that their centers of projections are very close), and moving them jointly in space (e.g., such as when the two cameras are mounted on a moving platform or rig).