Velodyne Puck + Realsense 3D mapping

[feta]

Hi,

I'm using Velodyne Puck (VLP16) and a Realsense D455 camera to try to generate a 3D pointcloud map, either with the pointcloud coming exclusively from lidar (while the depth camera is used only for loop closure), or from fused lidar and camera pointclouds.

So far I haven't managed to get RTAB-Map to generate a 3D map from both of these sensors together, only one or the other. I've experimented with the demo_husky.launch file (without the Husky bot itself) and a custom launch file, with both visual and ICP odometry, but I can't seem to get scan_cloud_assembling to work (I'm assuming that I should be looking into that process, I could be wrong). Another thing I tried doing was setting the Grid/FromDepth parameter to false, as recommended in a different thread.

On another note, what would be a good launch file to try out if I want to generate a 3D map from the VLP16 on its own?

Edit: I've added an image from the database viewer, it seems like cloud assembly works to an extent, but the Velodyne part looks really messy when viewed from CloudCompare.

Launch file that I'm currently using: Pastebin

[matlabbe]

Hi,

to play with 3D lidar with rtabmap, first you need to build rtabmap with libpointmatcher support. Afterwards, for an example with VLP16 alone, see test_velodyne.launch file.

scan_cloud_assembling option of rtabmap.launch will assemble scans from the lidar alone. If you want to combine PointCloud2 from different topics, look at rtabmap_ros/point_cloud_aggregator nodelet. The combined cloud could be then sent to rtabmap node. However, as D455 is a stereo camera, note sure if it is a good idea to merge stereo cloud with the lidar point cloud.

If you want to have the occupancy grid created from scan or scan_cloud, set Grid/FromDepth to false.

cheers,
Mathieu