rgb and depth image mismatch

[fhill122]

Hi ~

I used a Orbbec Astra pro camera for the rtabmap, but the mapping result is not perfect, I suspect the poor depth registration is the main cause.

As you may know Astra pro has two separate cameras, a depth one and a uvc rgb one, I used the usb_cam to launch the rgb camera and use rgbd_launch to generate the registered point cloud.

The camera intrinsic calibration has been done, but I'm not quite sure how the extrinsic calibration can be done on this camera and ros kinetic. Currently, I simply shifted the tf transformation between the rgb camera and depth camera, the depth registration looks alright as you can see below.



But when looking at the result in rtabmap database viewer, it seems there's still quite some shift


And even I changed the tf to a ridiculous number, the database viewer still shows the same amount of depth/rgb shift. Is that the tf between rgb and depth camera not taken into account in rtabmap?

Do you guys have suggestion regarding this?
Thanks!

[matlabbe]

Hi,

rtabmap assumes that the depth image is already registered, already in rgb frame, so TF is not used.
 
I don't own an Orbbec Astra Pro. If the IR camera and RGB cameras are independent, you may have to calibrate each camera, then do a stereo calibration to get the pose between both cameras.

You may get some help from this post for Orbbec Astra Pro: https://answers.ros.org/question/310219/astra-pro-registered-point-cloud/

cheers,
Mathieu