Integration of a new depth camera

[chris86]

Hi everybody,
I want to integrate a new depth camera. Is it possible to integrate it in the CameraRGBD.h?
I saw that depth camera classes process the 2D sensor data and save them in cv::Mat.
Is it possible to get in a new depth camera class a calibrated point cloud without getting problems with the processing pipeline (odometry->mapping)?

Thank you in advance.
Best
Chris

[matlabbe]

Hi,

yes, the CameraRGBD.h and CameraRGBD.cpp files are where the different drivers are implemented. Note that the driver should publish RGB and depth images, not depth-only. The depth image should also be registered to RGB image.

If your sensor publishes a point cloud and not a depth image, you would have to project the cloud in the RGB camera frame, so creating a registered depth image. This is what is done in the CameraTango class (note that in this example the resulting depth image is 8x smaller than the RGB image).

cheers

[chris86]

Thank you for CameraTango reference.
Do I need a RGB image too? Or would be only a point cloud/depth map suffice for the rtab map algorithm?
Chris

[matlabbe]

A RGB camera is required, with depth registered with it.

If you only want to use point cloud, you may check for other approaches based on geometry (ICP), like libpointmatcher.