RTAB-Map with ZED Mini (ZEDM): Camera Drifting and Loop Closures Rejected

[Gksu]

Hello, I am using RTAB-Map with a ZED Mini (ZEDM) camera, but I am having problems with camera drifting and loop closures not working. The camera slowly moves over time, even when it should stay in place. I also see that loop closures are rejected because of high error values. The error message says the loop closure error is too high (14.92), while the limit is 3.0. I tried increasing the RGBD/OptimizeMaxError value to 10.0 and 20.0, but the problem is still there.

I checked odometry (/odom topic), and it looks like the data is being published correctly. I also looked at TF frames, and they seem stable. However, the camera still drifts, and RTAB-Map does not fix it with loop closures. I also tried changing the feature detection method (Kp/DetectorStrategy) to ORB, SURF, and SIFT, but nothing changed.

I don’t know why this is happening. Could this be because of bad odometry from ZED Mini? How can I improve loop closure detection and prevent drifting? Are there any RTAB-Map settings I should change? I have attached a screenshot showing the problem.

[matlabbe]

Did you use that launch file https://github.com/introlab/rtabmap_ros/blob/ros2/rtabmap_examples/launch/zed.launch.py as reference? with use_zed_odometry:=true?

If you are using an external odometry, I would check if the pose is drifting in the topic and corresponding TF. RTAB-Map won't be able to appropriately correct that kind of drift. If the odometry is significantly drifting, I would ask why on the corresponding repo.

If you use the launch file above, you can try to compare with RTAB-Map's odometry approach by launching with use_zed_odometry:=false.

cheers,
Mathieu