Multi-camera setting explanation (visual odometry)

[ricber]

I would like to know how Rtabmap treats multiple cameras. In particular, I would like to understand better how visual odometry works in this setting.
Does the visual odometry node use both cameras or just one? In the case it uses both cameras does it need them to have overlapping fields of view or not? In this case the images could be stitched together and then used a single image. In the case cameras images are used independently we will result in different odometries that should be merged in some way. So, is visual odometry node working like one of these two cases or in another way?

Thank you,
Riccardo.

[matlabbe]

Hi Riccardo,

Features of both images are used at the same time in the odometry estimation. This means that if one camera is obstructed or is seeing featureless surfaces (like a white wall), as long as any of the other camera can see visual features, odometry won't get lost. The frame rate of the cameras should be high (>=30) for good synchronization or ideally that all cameras are hardware synchronized (similar to multiple D400 cameras in master/slave synchronization). An overlap between the cameras is not required, though useful to setup TF between the cameras (using Kalibr to find transformation between the cameras).

cheers,
Mathieu

[ricber]

Thank you Mat, is it possible to build the map with the pointclouds from multiple cameras but just using a single camera for the odometry?

[matlabbe]

Yes, just setup rgbd_odometry node with a single camera while keeping rtabmap node configured with the 3 cameras.