Re: Global Optimizing with Loop Closure and Navigation After Mapping.

Posted by matlabbe on
URL: http://official-rtab-map-forum.206.s1.nabble.com/Global-Optimizing-with-Loop-Closure-and-Navigation-After-Mapping-tp11061p11118.html

For the stereo calibration, the reprojection errors look okay, but do a quick visual check by comparing the two rectified images in Gimp for example (with two layers, top one 50% transparent) to see really that there is no more vertical shift. The reprojection error could be good for each individual camera, but you may also check the reprojection error after stereo rectification. From experience, the hardest part to get right in stereo calibration are the stereo extrinsics (rotation and translation between the cameras), not the individual camera intrinsics.

For the stereo sync issue:
a. If the camera driver is publishing left/right images with exact same stamp, make sure you use approx_sync:=false in rtabmap. If it is already your setup and there is still bad sync like above, then it is a camera driver issue. You should ask them why their images with exact same stamps are not synchronized.
b. If the camera driver is publishing left/right images with slightly different stamps but very close (<1 ms), assuming on hardware side they all shutter at the same time, you can set approx_sync:=true and set approx_sync_max_interval:=0.001 (very small accepted interval). If even with that there are bad synchronizations, you would have to ask on your camera driver's support.

For SuperPoint, I don't actively test with latest pytorch/cuda versions, last time I tried I created a doc here: https://github.com/introlab/rtabmap/tree/master/archive/2022-IlluminationInvariant#how-reproduce-results-shown-in-the-paper to reproduce the results of the last paper (comparing SuperPoint). There is also instructions to build a Docker version with SuperPoint support.

I'll suggest to improve stereo calibration and stereo sync in first phase, then jump to SuperPoint later.

cheers,
Mathieu