How does reg/strategy vis+icp work?

[dinossht]

How does rtabmap handle icp+vis slam? Does it combine both methods for more robust loop closure given that for instance camera and lidar is available?

[matlabbe]

I think it is easier to understand if I compare all approaches.

A) Reg/Strategy=0 (Visual-only): on proximity detection or global loop closure detection, we re-compute visual correspondences between the two images and compute transform.

B) Reg/Strategy=1 (Icp and partial visual): for proximity detections, visual estimation is not done, we do ICP directly between the laser scans and the guess. For global loop closure detections, we use BOW correspondences between two images (without recomputing correspondences), compute a visual transform, then feed that transform as guess to ICP.

C) Reg/Strategy=2 (Icp and full visual): For either proximity detections or global loop closure detection, visual correspondences are re-computed, a visual transform is estimated, then ICP is done afterwards. It is mainly doing like Reg/Strategy=0, but we add ICP step at the end.

Re-computing visual correspondences is more expensive than using already computed BOW ones but would give more correspondences, thus better transform estimation. For global loop closure, we can see with Reg/Strategy=1 that we skip that step (and use directly BOW correspondences) because we don't need the perfect guess for ICP, just one close enough and let ICP converges to best one. With Reg/Strategy=2, the difference is that we would provide a better guess to ICP. Note also that sometimes we may not get enough BOW correspondences to compute a guess with Icp/Strategy=1, unlike Icp/Strategy=2 would do.

For proximity detection (one-to-one proximity), Reg/Strategy=2 would force to have a valid visual transform before doing ICP, unlike Reg/Strategy=1 (which visual step is not done in that case). So if a visual transform cannot be computed (e..g, the robot has 180 deg orientation difference between the two nodes), then Reg/Strategy=2 would not detect that proximity loop closure. Note however that if RGBD/ProximityPathMaxNeighbors>0 and the node has laser scan, we still do proximity by laser scan merging (one-to-many proximity), independently to what Reg/Strategy is set.

cheers,
Mathieu

[dinossht]

Thank you Mathieu, this explanation cleared multiple questions I had!