A robust method of identifying moving or changing objects in a video sequence groups each pixel with other adjacent pixels according to either motion or intensity values. Pixels are then repeatedly regrouped into clusters in a hierarchical manner. As these clusters are regrouped, the motion pattern is refined, until the full pattern is reached.
These methods for motion-based segmentation may be used in a multitude of applications that need to correctly identify meaningful regions in image sequences and compute their motion. Such applications include:
- Surveillance and homeland security - detecting changes, activities, objects.
- Medical Imaging - imaging of dynamic tissues.
- Quality control in manufacturing, and more.
Researchers at the Weizmann Institute of Science have developed a multiscale, motion-based segmentation method which, unlike previous methods, uses the inherent multiple scales of information in images. The method begins by measuring local optical flow at every picture elements (pixels). Then, using algebraic multigrid (AMG) techniques, it assembles together adjacent pixels which are similar in either their motion or intensity values into small aggregates - each pixel being allowed to belong to different aggregates with different weights. These aggregates in turn are assembled into larger aggregates, then still larger, etc., yielding eventually full segments.
As the aggregation process proceeds, the estimation of the motion of each aggregate is refined and ambiguities are resolved. In addition, an adaptive motion model is used to describe the motion of an aggregate, depending on the amount of flow information that is available within each aggregate. In particular, a translation model is used to describe the motion of pixels and small aggregates, switch to an affine model to describe the motion of intermediate sized aggregates, and finally turn to a perspective model to describe aggregates at the coarsest levels of scale. In addition to this, methods for identifying correspondences between aggregates in different images are also being developed. These methods are suitable for image sequences separated by fairly large motion.