Cusom point cloud info through RTAB-Map

[xuliver14]

Greetings,

We have a system in which we use an RGB-D camera as input in RTAB-Map. On the camera stream we perform various object detection algorithms and we corelate them with the depth stream. We would like to save the extra information each point in the point cloud has (detected object) on the respective point. In the end, we want to obtain the 2D costmap from the point cloud, also depending on the extra information.

Does RTabMap have any simple way of storing extra information in each point in the pointcloud, except color and position?

Thank you!

[matlabbe]

Hi,

I see two approaches:
1) Embed object detection in color of the RGB image. When creating local occupancy grid from RGB-D (Grid/FromDepth=true and Grid/3D=true), the resulting local occupancy grids will contain colors.
2) Embed object detection in RGB channel of a point cloud (PointCloud2 with XYZRGB format) fed to rtabmap (subscribe_scan_cloud=true). When creating local occupancy grid from scan (Grid/FromDepth=false and Grid/3D=true), the resulting local occupancy grids will contain colors of the original point cloud.

In both cases, you would have to set Grid/PreVoxelFiltering=false to avoid voxelizing the point clouds that would merge colors between points. Once you have the point cloud of the map (/rtabmap/cloud_map), you could project it onto a 2D costmap depending on the color of the points.

cheers,
Mathieu

[xuliver14]

Hi,

Thank you for the suggestion, we were also thinking about the same approaches and managed to implement the first one in the meantime.

Regarding it, we noticed that the ground_cloud and obstacle_cloud have a 4 byte structure, 3 being the RGB channels. We managed to move some of the points from ground to obstacle, based on their color.
This approach is fine, but they always have a maximum cost.

Do you have any suggestions on how to create the costmap with different costs, based on the color? We were also thinking about storing the required cost in the additional, unused (?) byte of the RGB structure. How could we embed this info in the 4th channel?

Thank you!

[matlabbe]

I think that PCL's point cloud type XYZRGB and XYZRGBA have the same number of bytes, the A just not used for XYZRGB. I guess the occupancy grids could be converted to XYZRGBA instead of XYZRGB in order to easily set the alpha value to different costs.