Stereo motion estimation 3D-2-2D or 3D-2-3D？

[yanmingz]

Hi Mathieu,

I am testing your program with a stereo camera. I found an interesting problem.

The default setting for motion estimation for stereo slam is using 3D-2-3D method. Many papers suggest to use 3D-2-2D method and you also implement it in the package. But when I change the setting to use the 3D-2-2D method, the motion estimation becomes unstable, and slam totally failed. I also turn on the subpixel switch. It does not work.

I do not change other settings.

I guess you have some experience on this. Could you give me some advice on this? Thanks.

Best regards
Yanming

[matlabbe]

Hi,

Indeed, most papers use 3D-2-2D reprojection approach. You can try to reduce the "Reprojection error" parameter, this will produce more precise matches but less inliers. Make sure that there are more features to track under ~12 meters (for stereo cameras with 11 cm baseline) than on the background. If there are more inliers in the background than in the foreground, the odometry may jump large distances. You can try to change the "Flags" too, which changes the PnP approach used.

I would say that I have never seen a lot of improvements (if there are) with sub pixel activated. I had some successful mapping with Optical Flow odometry and 3D-2-2D motion estimation though.



cheers

[yanmingz]

Hi,

Thanks for your reply.

I am not so clear about the meaning of the following sentence:

Make sure that there are more features to track under ~12 meters (for stereo cameras with 11 cm baseline) than on the background. If there are more inliers in the background than in the foreground, the odometry may jump large distances.

What is the meaning of background and foreground?

Do you mean your algorithm works well when most inliers are located around 12 meters away for a 11 cm baseline camera?
What happens if all the inliers are located about 2~3 meters away?

Best regards
Yanming

[matlabbe]

Hi,

I've meant odometry will work better if most features are under 12 meters. For the background I mean features over 12 meters, like features over 100 meters and 1 km ++.

cheers

[yanmingz]

Thanks for your help.

Yanming

[yanmingz]

Hi,

I have just noticed that in the 0.10.10 release, there is a new feature:

RANSAC regression for PnP (3D->2D) transformation estimation on OpenCV3 (using RANSAC of OpenCV2 in OpenCV3). This fixed large errors with odometry when PnP was used.

I am not so clear about the meaning. Do you mean you combine the OpenCV2 implementation of PnP and OpenCV3 implementation of RANSAC? Which part cause the large errors? How do you mix the usage of OpenCV2 and OpenCV3?

Thanks a lot.
Yanming

[matlabbe]

Hi Yanming,

In some hand-held Kinect experiments, I've found that solvePnPRansac() of OpenCV2 was working better (less noise) than the one of OpenCV3. I copied the cv::solvePnPRansac() code from OpenCV2 in RTAB-Map (used here when OpenCV3 is detected).

However, in some recent experiments with another sensor configuration (stereo), the implementation of OpenCV3 is more stable. So in the next RTAB-Map version (0.11), we will be able to choose between OpenCV2 and OpenCV3 implementations of cv::solvePnPRansac(), when RTAB-Map is built against OpenCV3.

cheers,
Mathieu

[yanmingz]

Hi Mathieu,

Thanks for your reply.

I have also tested the PnP algorithms in OpenCV. It seems like there are some bugs in their implementation.

Best regards
Yanming

[matlabbe]

Hi Yanming,

OpenCV3 and OpenCV2 use exactly the same implementation of EPNP as in the EPNP original code.

What is different is the use of RANSAC on top of solvePnP, which is implemented differently in OpenCV2 and OpenCV3.

cheers,
Mathieu