MLSLib: a Lib Sync Librarz for multi agents and languages

Abstract

This article presents MLSLib, a software library for human figure animation with lip syncing. The library enables us to easily use multiple TTSs and multiple lip motion generators, and switch them arbitrarily. It also helps use of multiple speaking agents, possibly with different TTSs and lip motion generators. The MLSLib is composed of three modules: LSSAgent, TTSManager, and FCPManager; The LSSAgent module provides unified simple APIs per single agent, independent of TTSs and lip motion generators. The TTSManager and FCPManager manage TTSs and lip motion generators, respectively. Both modules support standard sets of phonetic alphabets per language, and thus users are freed from TTS-dependent implementation of lip motion generators. Applications to multi-lingual agents and LOD in lip syncing are also presented.