Re: Livox Avia, + Depth cameras and organized point clouds.

Posted by matlabbe on Dec 21, 2022; 6:03am
URL: http://official-rtab-map-forum.206.s1.nabble.com/Livox-Avia-Depth-cameras-and-organized-point-clouds-tp9200p9264.html

I suggest to add TF display in RVIZ. You will better see the actual transform between your camera and lidar frame (and see if it is just an inverted transform, or an optical rotation missing).

From this matrix:

-0.00166874,-0.999969,-0.00768157,-0.013776
-0.00286365,0.0076861,-0.999967,0.0347174
0.999995,-0.00164645,-0.00287661,-0.0972432
0,0,0,1 

It means:

x->z
y->-x
z-> -y

so it looks like transforming ros base frame (x->forward, y->left, z->up) to image frame (x->right, y->down, z->forward).

In general, this is the optical rotation (convert image frame into base frame):

<node pkg="tf2_ros" type="static_transform_publisher" name="camera_link_to_optical" args="0 0 0 -1.57 0 -1.57 camera_link camera_optical_link 10"/>
<node pkg="tf2_ros" type="static_transform_publisher" name="livox_frame_to_camera_link" args="-0.013776 0.0347174 -0.0972432 0 0 0 livox_frame camera_link 10"/>

You should make sure your image topics have optical frame as frame_id. The optical rotation would be fixed, but the extrinsics calibration between camera and lidar would be included in livox_frame_to_camera_link. Or like you tried, combining optical rotation and camera/lidar translation in same TF, but crefully debug with TF in RVIZ. You can show up Camera display to see if RGB -> point cloud registration is good (before trying any mapping!).

cheers,
Mathieu