Fish motion capture with refraction synthesis

Abstract

3D fish animations become more and more popular in fish behavioral research. It empowers the experimenter to design fish stimuli and their specific behavior to the experiment’s needs. The fish animation can be done manually or derived from video footage. Especially automatic fish model parameter recovery for 3D animations is not well studied yet. Here we present a novel, flexible method for this purpose. It can be used to recover position, pose, bone rotation and size from single or multiple view and for single or multiple fish. Additionally we implement a novel method to compensate the fish tank’s refraction effect and show that this method can decrease the error up to 80 %. We successfully applied the proposed method to two different data sets and recovered fish parameters out of single- and double-view video stream. A video attached to this paper demonstrates the results.