Real-time segmentation for advanced disparity estimation immersive videoconference applications

Abstract

We present an advanced disparity estimation algorithm developed for real-time immersive teleconferencing. In order to improve the disparity maps in the case of occlusions a segmentation-driven disparity post processing is applied. We introduce a novel approach for a segmentation algorithm, which uses the advantages of HSV colour space without the need of an explicit colour space transformation. This makes segmentation very fast while still keeping the robustness of HSV colour space especially to illumination changes and shadows. Further on, we present an algorithm for tracking the hands of a conferee. Finally, we introduce an algorithm framework, which combines segmentation, tracking of hands and disparity estimation to create robust disparity map in real-time. We show the improvement of results for image-based rendering.