3D LIDAR + RGB camera registration

[LIDAR_man]

Hello,

I have been trying to use pointcloud_to_depthimage to create a depth image from a Baraja LIDAR and a camera.

At the moment, the topic /image_raw is being published, but its all zeros. For reference the command I am running is:

"rosrun rtabmap_ros pointcloud_to_depthimage cloud:=/newCloud camera_info:=/camera_arena_node/camera_info _fixed_frame_id:=base_link"

I spent some time creating a node that takes the image and point cloud and ensures their time stamps are the same. This node then republishes the PointCloud2 message on /newCloud topic with an adjusted time stamp. The data on /newCloud and /camera_arena_node is valid and I can view it in rviz no problem.

If anyone has any thoughts/suggestions I would greatly appreciate them! - I have attached a link to a bag file with some data in it.

https://drive.google.com/file/d/1cpzVWi7uBdGIiH6yD0jbp3tDqg4-Yg5r/view?usp=sharing

[matlabbe]

The linked expired.

Tip: a good way to actually see if your TF is fine for lidar reprojection in the camera is use a Camera display in RVIZ, add your camera image topic in it, then add a PointCloud2 display with your lidar topic. If TF tree is fine can camera_info is valid, you should see your lidar scan overlaying the camera image. You can change the PointCloud2 display point type from bollboard to point to better see them. If you can see the lidar overlay on the image, then pointcloud_to_depthimage should work.