RGBD Sync Input Dimensions

[willat343]

Hi,

I have a pointcloud_to_depthimage node producing a decimated depth image (at 484 x 304) being synced with the original rgb image (at 968 x 608) and accompanying info message (also 968 x 608). The decimation is there because my lidar is only 16 channel, and it can be computationally taxing to perform the depth filling operations at the original image size.

I have noticed that the RGBDImage messages contain incorrect depth info since the nodelet uses one camera_info to set both info structures in the message.

Does the rgbd sync nodelet require all inputs to be of the same dimensionality in order to produce a valid RGBDImage?

Cheers,
Will

[matlabbe]

Hi Will,

interesting observation. Indeed this would be a problem if the depth camera info was used (the resolution of the depth image won't match the resolution set in the depth camera info), but actually the depth camera info is ignored in all rtabmap nodes (they use all rgb camera info and rescale it depending on depth image size, as they all assume that the depth image is registered to rgb image). The "depth_camera_info" is used only when RGBDimage contains stereo data (left and right camera info).

The fix would be to add a camera_info publisher in pointcloud_to_depthimage with the scaled camera_info, then make rgbd_sync subscribing to that republished camera_info. This has been added in latest commits (the new camera_info is now published under output image namespace).

cheers,
Mathieu

[willat343]

Hi Mathieu,

Thanks for your informative reply.

I have been using the `rtabmap_ros/point_cloud_xyzrgb` nodelet to visualise what this `RGBDImage` looks like (as a pointcloud) before going into the main `rtabmap` node. However it too only uses the rgb camera_info stream, so this nodelet generates incorrect pointclouds when the depth and rgb images/info are scaled differently (from decimation).

It probably makes sense to make this nodelet subscribe to a depth camera_info (e.g. out of `pointcloud_to_depthimage`) as well as `rgbd_sync`, if this has not already been done in the latest commits you mentioned.

Cheers,
Will

[matlabbe]

Hi Will,

I would have expected point_cloud_xyzrgb to work, though maybe not pointcloud_xyz as you described. I'll take a look later. point_cloud_xyzrgb ignores the depth camera info so the last commit I did won't change anything for that node if subscribed to rgbd_image. You could use pointcloud_xyz node instead with the new camera info, but you will not have colors.

cheers,
Mathieu

[matlabbe]

Hi Will,

there was a bug in point_cloud_xyz and point_cloud_xyzrgb not handling correctly when camera_info resolution is not the same than depth resolution. This should be fixed in this commit.

cheers,
Mathieu

[willat343]

Thanks Mathieu for looking into that.

I'm curious - will these recent commits get rolled into a version update for melodic or noetic in the near future? Or will I need to build from source?

Cheers,
Will

[matlabbe]

If near future is before april maybe yes. Otherwise you may be better to rebuild.

Cheers,
Mathieu