Using DNN for Stereo Disparity with RTabMap

[HiddenNetwork]

Hi there!

I've tried tuning SGBM parameters to get a better disparity map, but I'm finding that neural-network based methods generally produce a much smoother disparity map.

Is it possible to replace the StereoBM or StereoGBM disparity algorithms used by RTabMap, with a neural-network-based algorithim that outputs a disparity map? For both the standalone RTabMap application, as well as the ROS-based version?

Thanks,

[matlabbe]

If it can be coded in c++ with pytorch (c++ api), I guess it could be possible to create a new class inheriting StereoDense class, like those here: https://github.com/introlab/rtabmap/tree/master/corelib/include/rtabmap/core/stereo

 The main function to implement is

virtual cv::Mat computeDisparity(
			const cv::Mat & leftImage,
			const cv::Mat & rightImage) const = 0;

Another approach would be to make a general PyStereo class calling a python script to generate the disparity, similar to PyDetector and PyMatcher classes. However, there is currently an issue making the app freezes/crashes when calling a python script using pytorch.

cheers,
Mathieu