Tips and parameter optimization for handheld devices.

[yb]

Hello, I'm currently using a pre-compiled version of rtabmap (0.22.1) on Windows with only one Realsence D435i camera (no cars or robots...). I'm scanning the office handheld with the camera, but I'm finding it very difficult to detect loop closures, which causes my desk to drift.Does anyone have any photography tips or parameter adjustments for this? Thank you very much.You can download my database and the generated point cloud.
https://drive.google.com/file/d/1vd_MW6HoqXaQVDQdoRbA6cZWDSLTS6ff/view?usp=sharing
https://drive.google.com/file/d/1BL2mZ1bALBnFOpI6firOjQxb3Of0WlAw/view?usp=sharing

[yb]

You might also need an ini file: https://drive.google.com/file/d/10KzauUFYD_Bm3zShd6wgXSZqsaourxmb/view?usp=drive_link

[matlabbe]

It seems the odometry is drifting a lot more than the estimated covariance. Good loop closures are then wrongly rejected. Using RGBD/OptimizeMaxError=3 makes almost all loop closures rejected (over the threshold here when plotting the statistics):


To accept all appropriate loop closures, I had to increase RGBD/OptimizeMaxError parameter from 3 to 10:

rtabmap-reprocess --RGBD/OptimizeMaxError 10 260206-201858.db output.db

Note that something annoying with D435i is that the color camera FOV is smaller than the IR camera and it has more motion blur. Using IR cameras for visual odometry is recommend with that camera, at the cost of not being able to use the IR emitter anymore (textureless surfaces will be harder to see and less accurate).

[yb]

Thank you for your answer! I need a colored point cloud, but if I follow your instructions, the generated point cloud seems to be only black and white, as seen with IR cameras? Is another cost of using an infrared camera for visual odometry measurements the loss of RGB color?

[matlabbe]

With the standalone version, we cannot use IR stereo for VO and RGB-D for SLAM at the same time. With ROS, we can send the IR stereo images to stereo_odometry, then feed rgb + aligned depth to rtabmap node using odometry from stereo_odometry. Only downside in that approach is that we still cannot enable the IR emitter to get depth on textureless surfaces (Update: I added a realsense_d435i_combined.launch.py ROS2 example).

For the standalone, here are your choices for D435i:

RGB-D mode:
   - VO accuracy/robustness
   - Precise depth registration
   + Motion blur
   + Depth on textureless surface
   + Color

IR-Depth mode:
   + VO accuracy/robustness
   + Precise depth registration
   - Motion blur
   - Depth on textureless surface
   - Color