Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

Tony Zhao, Vikash Kumar, Sergey Levine, Chelsea Finn

Read the paper here.

in partnership with...

Teleoperation System

We introduce ALOHA: A Low-cost Open-source Hardware System for Bimanual Teleoperation. With a $20k budget, it is capable of teleoperating precise tasks such as threading a zip tie, dynamic tasks such as juggling a ping pong ball, and contact-rich tasks such as assembling the chain in the NIST board #2.

Learned Policy

We introduce Action Chunking with Transformers (ACT). The videos below show real-time rollouts of learned policies, imitating from only 50 demonstrations for each task. ACT predicts a sequence of target joint positions given RGB images and proprioception. For the three following tasks, ACT obtains 96%, 84%, 64% success respectively.

Observations during policy execution

We show example image observations (i.e. the input to the ACT policy) at evaluation time. There is a total of 4 RGB cameras each streaming at 480x640. Two of the cameras are stationery and the other two are mounted on the wrist of robots.

Slide Ziploc

Slot Battery

Open Cup