Re: Bunlder export: what poses ?

Posted by matlabbe on
URL: http://official-rtab-map-forum.206.s1.nabble.com/Bunlder-export-what-poses-tp11037p11229.html

Hi Pierre,

If DepthAsMask is off and StereoFromMotion is on, features are extracted anywhere in the image even if there is no depth, then for those without depth, their 3D position (e.g., depth) is computed by triangulating them from their correspondence in the previous image. There are some caveats on why the depth computed like this would be almost worst than using the depth image.
1. If there is not a lot of overlap between the consecutive images, no depth could be computed, which is generally the case with RTAB-Map set at 1 Hz update rate by default.
2. The 3D position of the feature is estimated only by two frames, so if there is not a lot of translation, the 3D estimation will be generally poor.
3. Even if depth of ARKit can be poorly estimated, I think it would still be always be as "good" or even better than StereoFromMotion, because ARKit can estimate depth on multiple consecutive frames (more overlap). For example, we record the 3D features estimated by ARKit into the LaserScan data for each node, you can see how "good" or "bad" ARKit's 3d feature estimation is (by not using depth) using rtabmap-databaseViewer.

For Global Bundle Adjustment post-optimization, I would expect that depth computed from StereoFromMotion or "low confidence" ARKit depth would give similar result. Currently, we don't track confidence of the 3D position for each visual feature. With 3D feature confidence, we could tune "g2o/PixelVariance" parameter and increase it for features we know that were estimated by poor depth confidence or StereoFromMotion. By giving more pixel variance, is means the depth can change more during BA for these features.

Vis/EstimationType = "2" is kinda experimental, it is completely ignoring depth during pose estimation, but uses depth of features to adjust the scale of the transformation computed. PnP (Vis/EstimationType=1) gives more stable results.

I am not sure how confidence could be used in PnP, though it could be in BA as explained above. When Vis/BundleAdjustment=1 (default), a local BA is always done after PnP between the two frames. The weighted optimization from BA could be done there using features' confidence.

cheers,
Mathieu