Objects and occlusion from motion labeling

Abstract

The problem of segmenting color video sequences is addressed. Boundary motion and occlusion relations expressed by labeling rules are argued to be of key importance for segmentation. Interframe motion of object(s) and background is approximated by a similarity or affine transforms. The computation proceeds in several steps: (1) color gradient edges are fuzzy-clustered according to their motion; (2) boundaries of regions formed by a color segmenter are assigned weights and labeled according to their motion; (3) un-ambiguously moving regions and “conflict” regions are identified by a label relaxation procedure; (4) ambiguities and conflicts are resolved by multiframe analysis.