Peg-Insertion Precision with Bimanual Robots and the ACT Model
- Sep 19, 2024
- 6 min read
The Short Version
Read the Aloha Hugging Face Getting Started Guide before replicating this peg-insertion setup.
Configure a bimanual robot with two manipulators and four cameras—two wrist-mounted, one above, one below the task space.
Collect 60 human demonstrations of the peg-insertion task with fixed peg, hole, and position.
Train the Action Chunking Transformers (ACT) model to imitate the demonstrated actions.
Run 30 evaluation rollouts and track success rate, ~15-second completion time, and error recovery.
Pull the Trossen Robotics Community datasets and trained model from Hugging Face to test in your own environment.
Next, loosen constraints—varied peg/hole sizes, positions, and dynamic targets—to test generalization.
Who this is for
Robotics researchers studying imitation learning
Machine learning engineers working with ACT models
Bimanual manipulation and fine motor control teams
Industrial automation engineers evaluating error recovery
Trossen Robotics community contributors and dataset users
Updated: Sep 20, 2024
*Read The Aloha Hugging Face Getting Started Guide first.*
In this experiment, Trossen Robotics trained a bimanual robot to insert a peg into a hole using the Action Chunking Transformers (ACT) model, and it achieved an 80% success rate across 30 evaluation rollouts. Both the peg and hole stayed the same size, and their position was fixed throughout — a controlled setup that let us isolate specific metrics like precision, success rate, and error recovery.
In robotic manipulation, precision tasks such as peg-insertion are vital for assessing a robot's capability to perform fine motor control. The experiment we conducted focuses on a bimanual robotic arm setup for inserting a peg into a hole. Both the peg and hole remain the same size, and the position is fixed throughout the experiment, which allowed us to explore specific metrics under controlled conditions.
How Was the Peg-Insertion Experiment Set Up?
This experiment used a bimanual robot with two manipulators. Trossen Robotics equipped the setup with four cameras for visual perception:
Two wrist cameras, one mounted on each arm, to capture close-up interactions.
Two global cameras, one positioned atop and one below, offering wider views of the task space.
We trained the robot using 60 human demonstrations in the peg-insertion task, relying on the Action Chunking Transformers (ACT) model to learn the behavior from those demonstrations. The robot's objective was to replicate the actions observed in these demonstrations with high precision and consistency.
The collected training episodes are available on Hugging Face (Trossen Community), along with the trained models. These models are accessible for further experimentation, so researchers can test the trained robot or use the evaluation episodes for additional training.
Why Does This Experiment Matter?
The core goal was to understand how imitation learning models like ACT perform in a highly controlled setup with fixed parameters. The conditions were deliberately constrained:
Fixed starting positions and orientations of the peg and hole,
Consistent dimensions of both the peg and hole.
These constraints help establish the baseline performance of the model. By focusing on fixed variables, we can isolate key performance metrics like precision, success rate, and error recovery.
Metrics and Results
We measured the robot's performance by how accurately it completed the peg-insertion task across multiple evaluation runs. After training, the robot achieved a success rate of 80% during 30 evaluation rollouts. That level of success is promising, but the experiment also highlights areas for improvement — especially generalization beyond fixed conditions.
The primary metrics of this experiment:
Metric | Result |
Success Rate | 80% of peg insertions completed successfully across 30 evaluation rollouts |
Task Completion Time | ~15 seconds per insertion attempt, matching the length of the training episodes |
Error Recovery | Adjusted and retried after disturbances; finished in the next episode if not completed within the 15-second window |
Success Rate: The robot completed 80% of the peg insertions successfully during 30 evaluation rollouts. This shows promising results, but also leaves room for improvement, especially when the environment or parameters are varied.
Task Completion Time: Completion time was consistent with the length of the training episodes, approximately 15 seconds per insertion attempt. This indicates the robot replicated the pace demonstrated during the human demonstrations without delays or deviations.
Error Recovery: One of the most significant aspects of this experiment was the robot's ability to recover from errors. When disturbances occurred — such as the peg or hole being forcefully removed — the robot adjusted its actions and attempted the task again. If the task could not be completed within the 15-second window, it would complete the insertion in the next episode. This showcases the robot's ability to handle interruptions and recover from disturbances, a critical factor in real-world industrial applications.
What Does This Tell Us About Imitation Learning?
This experiment serves as a critical benchmark for understanding the limitations and capabilities of imitation learning models like ACT in constrained environments. Peg-insertion is widely used in robotics research because it requires high precision, fine motor control, and the ability to integrate sensor data for decision-making.
Controlled Parameters Help Isolate Performance Factors: By keeping the peg, hole, and positions fixed, we could focus on the robot's ability to mimic human-like precision rather than complicating the experiment with too many variables.
Benchmarking Imitation Learning Models: Since imitation learning relies heavily on human demonstrations, this experiment highlights how well the ACT model can learn and replicate human actions. The high success rate shows promise, but also reveals that improvements are necessary for more dynamic environments.
Next Steps: Generalization and Adaptability
The current experiment succeeded within the confines of fixed parameters, but generalization remains the key challenge in robotic manipulation. In real-world scenarios, the robot will need to handle:
Variations in peg and hole sizes,
Different starting positions and orientations,
Dynamic environments, where the object or target may move slightly.
Our next goal is to train the robot to handle these variations, making it more robust and adaptable. By loosening the constraints, the model will be tested on how well it generalizes beyond the data it was originally trained on.
Get Involved
If you want to test the models or explore the dataset for your own training, the collected episodes are available on Hugging Face. You can also access the trained models and experiment with them in your own environment. The evaluation episodes are available for further analysis and comparison too. We encourage the community to build upon this experiment and help refine the models to handle a wider variety of scenarios.
Feel free to reach out, share your findings, and help us push the boundaries of robotic manipulation!
Trained Model: https://huggingface.co/TrossenRoboticsCommunity/act_aloha_static_peg_insertion
Evaluation Dataset: https://huggingface.co/datasets/TrossenRoboticsCommunity/eval_aloha_static_peg_insertion
Training Dataset: https://huggingface.co/datasets/TrossenRoboticsCommunity/aloha_static_peg_insertion
Exploring Precision With Peg-Insertion Using Bimanual Robots?
Read the Aloha Hugging Face Getting Started Guide before replicating this peg-insertion setup.
Configure a bimanual robot with two manipulators and four cameras—two wrist-mounted, one above, one below the task space.
Collect 60 human demonstrations of the peg-insertion task with fixed peg, hole, and position.
Importance of the Experiment?
Read the Aloha Hugging Face Getting Started Guide before replicating this peg-insertion setup.
Configure a bimanual robot with two manipulators and four cameras—two wrist-mounted, one above, one below the task space.
Collect 60 human demonstrations of the peg-insertion task with fixed peg, hole, and position.
_Learn more about Trossen Robotics and Trossen SDK for your deployment._
References
Frequently Asked Questions
What is peg-insertion and why is it used to test bimanual robots?
Peg-insertion is a precision task where a robot inserts a peg into a hole, widely used in robotics research because it requires high precision, fine motor control, and integrating sensor data for decision-making.
What was the experiment setup?
A bimanual robot with two manipulators was equipped with four cameras: two mounted on each wrist for close-up interactions and two more, one atop and one below, for global views of the task space.
How was the ACT model trained?
The robot was trained on 60 human demonstrations of the peg-insertion task using the Action Chunking Transformers (ACT) model to learn and replicate the demonstrated behavior.
What success rate did the robot achieve?
After training, the robot achieved an 80% success rate across 30 evaluation rollouts, completing 80% of the peg insertions successfully.
How long did each insertion take?
Task completion time was consistent with the training episodes, approximately 15 seconds per insertion attempt, matching the pace demonstrated by humans.
Can the robot recover from disturbances?
Yes. When the peg or hole was forcefully removed, the robot adjusted its actions and tried again; if it couldn't finish within the 15-second window, it completed the insertion in the next episode.
Where can I access the datasets and trained models?
The training episodes, evaluation dataset, and trained models are available on Hugging Face under the Trossen Robotics Community for testing and further experimentation.
Comments