Researchers at Stanford University and Simon Fraser University have introduced TWIST (Teleoperated Whole-Body Imitation System), a fresh approach that enables humanoid robots to mirror human movements in real time. By blending motion capture technology with reinforcement and imitation learning, TWIST guides robots through complex tasks with a precision that’s both practical and impressive.
Yanjie Ze, the lead author, explains how the system works: it captures human motion accurately and then uses AI to convert those movements into commands for the robot. This method ensures smoother, fuller-body control compared to previous approaches—a real benefit if you’ve ever wrestled with clunky or delayed robotic responses.
In practical terms, TWIST has already been tested on Unitree Robotics’ G1 humanoid and could extend to other models like Booster Robotics’ T1. The system skillfully manages all parts of the robot, from legs and feet to the waist and elbows, offering a level of balanced and human-like dexterity necessary in crowded environments.
The innovation doesn’t stop at imitation. TWIST also addresses common challenges such as motion delays and jerky movements through a two-stage training process. It first learns from high-quality offline data before refining its skills with smaller, real-time datasets—a strategy that could be useful for both teleoperation and autonomous tasks in the future.
Of course, no new technology is without its hurdles. Current challenges include the lack of visual and tactile feedback for operators, hardware durability, and a dependence on non-portable motion capture systems. Future upgrades aim to integrate better feedback mechanisms and incorporate RGB-based pose estimation, making TWIST more versatile in real-world applications.
