ROS World 2021: Simulating multiple mobile warehouse robots with ROS2 and Unity

Unity is hosting a workshop at ROS World 2021 to showcase robotics simulation using Unity and ROS2 

Many industrial scenarios, such as order fulfillment in warehouses and distribution centers, require multiple robots to operate in a coordinated fashion. Instead of using one complex humanoid robot to perform all tasks, multiple different and specialized robots are used. Advances brought by ROS2 have made it easier to coordinate complex multi-robot systems, but testing them in simulation has still been challenging. 

In this workshop, you’ll learn how to use the Unity engine and ROS2 to simulate multiple mobile robots that must perform object pose estimation in order to complete a “find-and-ferry” task in a dynamic warehouse environment.

This will be a workshop from 10 am to 2 pm CT, open to ROS World attendees on October 19, 2021. The workshop will be broken into four main phases.

 

Workshop Agenda
Phase I Unity and environment setup
Phase II Single robot navigation
Phase III Find-and-ferry
Phase IV Using object pose estimation

 

Phase I: Unity and environment setup

The workshop will begin with an introduction to the Unity Editor downloading all required packages and assets. Attendees will then learn how to import a robot using the URDF Importer to set up the FindBot. They will load and configure the warehouse environment and add a cube. Finally, attendees will set up the ROS-TCP-Connector and ROS-TCP-Endpoint to ensure proper communication between ROS2 and Unity.

Takeaways:

  • How to create a Unity Scene
  • How to use the Package Manager to download and install Unity packages
  • How to manipulate objects in the scene
  • How to set up ROS-TCP communication
  • How to use the URDF Importer to import a robot

 

Phase II: Single robot navigation

With the scene and robot properly loaded, this phase of the workshop will focus on autonomous navigation. Attendees will learn to connect various sensors to the robot. ROS 2 will be used running Nav2 utilizing the sensor data that is generated from the Unity scene. At the end of this phase, you will have a robot that can autonomously go from point A to B.

Takeaways:

  • How to add sensors to a robot
  • How to set up Nav2 publishers and subscribers
  • How to set up message visualizers
  • How to run the Nav2 simulation in Unity

Phase III: Find-and-ferry

Now that a single robot (the “FindBot”) can move around the Unity scene autonomously, you will bring in a second robot (the “FerryBot”), which is equipped with a manipulator. This robot will be responsible for executing the pick-and-place task. Finally, attendees will learn how to put all the pieces together: deploy the trained pose estimation model to a ROS2 node and simulate a fully autonomous find-and-ferry task, wherein the robotic arm can pick up the object and place it on the FerryBot for ferrying to the desired location.

Takeaways:

  • How to add a manipulator to a mobile robot to create the FerryBot
  • Set up inverse kinematics for the FindBot
  • Deploying an inference model for object pose estimation
  • How to set up and run the Nav2 simulation with multiple robots simultaneously

Phase IV: Using object pose estimation

In this phase, attendees will go through the process of using the Unity Perception Package to collect synthetic data for training a computer vision model that performs object pose estimation.

Takeaways:

Level up your robot!

As the workshop draws to a close, we will show you how you can expand this project to include additional robots, add more cubes used in the object pose estimation portion, and answer questions you may have about making additional changes.

Requirements

Presenters

Anthony Navarro Senior Technical Product Manager
Amanda Trang Software Engineer
Devin Miller Senior Software Engineer

 

Previous Video
Learn how Ouva builds a production-ready patient monitoring platform with synthetic training data
Learn how Ouva builds a production-ready patient monitoring platform with synthetic training data

Every year hospitals spend $2 billion on avoidable hospital-acquired conditions. In this session, we will w...

Next Article
Supercharge your computer vision models with synthetic datasets
Supercharge your computer vision models with synthetic datasets

Unlock the potential of your computer vision model with synthetic data generated by Unity’s expert team.

×

Download retail sample dataset today

First Name
Last Name
Company
Job Title
Industry
Country
I understand that by checking this box, I agree to have Marketing Activities directed to me by Unity
I acknowledge the Unity Privacy Policy
I have read and agree to the Unity Terms of Service
Thank you!
Error - something went wrong!