TORNADO: Foundation Models for Robots that Handle Small, Soft and Deformable Objects

Abstract

This paper introduces TORNADO, a cloud-integrated robotics platform designed to tackle the challenges of autonomous manipulation in dynamic indoor environments, particularly those involving small, soft, or deformable objects. TORNADO integrates large-scale foundation models for perception, language comprehension, and high-level reasoning, to achieve strong zero-shot generalization across a wide range of tasks. At its core, the platform features an adaptive cognitive pipeline capable of dynamically reconfiguring its modules—including semantic 3D SLAM, people-aware navigation, dexterous manipulation, and human-in-the-loop learning—to manage uncertainty and adapt to changing conditions. Additionally, TORNADO incorporates a multi-modal Learning-from-Demonstration interface and an Explainable AI engine, enhancing transparency and easing the burden of teaching new tasks. The system is validated through three industry-relevant scenarios: (1) flexible gear and ply-sheet handling in a mechanical parts factory, (2) patient support in a hospital palliative ward, and (3) product sampling and waste management in a distribution center. TORNADO aims to significantly improve the agility, safety, and overall task performance of mobile manipulators operating in dynamic, human-centric environments.