Beginner Question: Combine Stereo, Time-of-Flight Sensor, and IMU?

[scottsrobots]

Hello,

While I've built and designed micro-controller-controlled basic table-top robots in the past, I am brand new to autonomous robots, ROS, and SLAM.

I'm working on building a small floor companion bot (a bit smaller than a roomba) that I want to be able to map and navigate on a single floor of a family home.

My questions are the following:

• Can I (and is it a good idea) to combine the input from the following sensors to map, localize, and get odometry?
  • Stereo USB Camera: https://www.amazon.com/dp/B07GDC4S2D
  • BNO055 9-DOF IMU: https://www.adafruit.com/product/4646
  • VL53L0X 1200mm time-of-flight sensor: https://www.adafruit.com/product/3317
• If so is it possible to configure these to test with the RTAB-Map Windows 10 application without delving into ROS (to decrease complexity for basic proof-of-concept)?

My constraints are the following:

• Due to cost concerns and physical limitations of the robot, I hope to avoid a traditional motorized LIDAR sensor
• Wheel encoders cannot be trusted for odometry because the robot is small, is navigating an uncontrolled lived-in environment, and can be picked up and moved around by hand at anytime.

Thanks in advance!

If anyone wants to see details on my project goals I have a 4-part TikTok explaining the project here:

RO-BUD Part 1: https://www.tiktok.com/@scottsrobots/video/6951999336200867078
RO-BUD Part 2: https://www.tiktok.com/@scottsrobots/video/6952000579883306245
RO-BUD Part 3: https://www.tiktok.com/@scottsrobots/video/6952001228733811974
RO-BUD Part 4: https://www.tiktok.com/@scottsrobots/video/6952007893747109126

[matlabbe]

Hi,

For robots, I strongly recommend to use ROS version. With the Windows version, you cannot have live outputs of the pose and the map. On Windows, at least you could test the stereo camera (it seems to be uvc compatible). Standalone imus are not supported with standalone RTAB-Map, only under ROS.

The time of flight sensor you referred is a single beam lidar, it cannot be used for SLAM, but it could be used for obstacle avoidance.

For the whole setup, having wheel odometry will improve robustness. If the robot is picked up, using vision or lidar can also have issues if the sensors are obstructed. Detecting somewhat if the robot has been picked could help to know that current  odometry is unreliable until the robot is put down on the ground (maybe also resetting the localization). Ideally it would be to do mapping first, without any external events that could move the robot without being aware of it, to have a nice map. Then go in localization mode, in that mode you could picked up and drop the robot anywhere (it has already mapped) and visual loop closure detection would be able to relocalize on the map.

You may don't need a lidar, if you don't care about very accurate localization.

Have fun with your project!
Mathieu

[scottsrobots]

Thanks so much for the detailed answer!