Handle time delay between lidar and camera topic

[dinossht]

I use a Livox MID360 lidar for mapping using ICP and also adding rgb images from Insta360 x3 for coloring pointcloud during postprocessing. There is a timedelay due to undistortion and other processing during ros message publication. How can I comensate for this time delay to synchonize lidar and camera topics?

[matlabbe]

Hi,

You can try to enable odom_sensor_sync for rtabmap node. That would adjust camera local transform based on odometry and lidar stamps. However, this would only help if you see this issue when the camera is moving. If you see a bad alignement between camera and lidar when nothing is moving, then it is an extrinsics issue, that would be solved by adjusting the URDF/TF between lidar and camera frame. For other bad alignement caused by motion, that could be related to timestamps of the sensors that are not relative to same clock.

cheers,
Mathieu

[dinossht]

I think the issue is related to a constant timedelay between camera and lidar topics. Currently, I added publisher that compensates for the delay and republishes, and this seems to solve the issue to a good degree.

[matlabbe]

That could mean they are relative to a different clock. Adjusting the offset manually can be a solution, though I would be concerned about the clocks slowly drifting away from each other over time (i.e., the offset would change over time).

[dinossht]

Yes I agree.

It seems that the Livox Mid360 LiDAR supports PTP synchronization; however, to enable this, I require an Ethernet interface that supports hardware timestamping for IEEE 1588 PTP to ensure accurate time synchronization with the LiDAR and camera sensor.

Is this something you have experience with? Seems like my laptop does not support PTP sync

[matlabbe]

I remember setting this up for an ouster lidar, following instructions from https://github.com/ouster-lidar/ouster-sdk/issues/74#issuecomment-530028510 and https://ouster.atlassian.net/servicedesk/customer/portal/8/article/1165066356

You would have to launch ptp4l and setup your lidar to use PTP. I think at least if your ethernet adaptor can support software PTP, it could work.

[dinossht]

Thank you! I will checkout the links. Seems like it is possible to use software sync for ethernet link on my laptop, which is still better than no sync at all.