Help with MYNT EYE D

[jroddis]

Hello

I am trying to get the MYNT EYE D working with RTABMAP, if at all possible.

I can successfully configure the camera with ROS, and I am able to view depth/rgb/imu etc streams in RViZ.

Using rostopic list give the following topics:

/camera_mesh
/mynteye/depth/camera_info
/mynteye/depth/image_raw
/mynteye/depth/image_raw/compressed
/mynteye/depth/image_raw/compressed/parameter_descriptions
/mynteye/depth/image_raw/compressed/parameter_updates
/mynteye/depth/image_raw/compressedDepth
/mynteye/depth/image_raw/compressedDepth/parameter_descriptions
/mynteye/depth/image_raw/compressedDepth/parameter_updates
/mynteye/depth/image_raw/theora
/mynteye/depth/image_raw/theora/parameter_descriptions
/mynteye/depth/image_raw/theora/parameter_updates
/mynteye/imu/data_raw
/mynteye/imu/data_raw_processed
/mynteye/left/camera_info
/mynteye/left/image_color
/mynteye/left/image_color/compressed
/mynteye/left/image_color/compressed/parameter_descriptions
/mynteye/left/image_color/compressed/parameter_updates
/mynteye/left/image_color/theora
/mynteye/left/image_color/theora/parameter_descriptions
/mynteye/left/image_color/theora/parameter_updates
/mynteye/left/image_mono
/mynteye/left/image_mono/compressed
/mynteye/left/image_mono/compressed/parameter_descriptions
/mynteye/left/image_mono/compressed/parameter_updates
/mynteye/left/image_mono/theora
/mynteye/left/image_mono/theora/parameter_descriptions
/mynteye/left/image_mono/theora/parameter_updates
/mynteye/points/data_raw
/mynteye/right/camera_info
/mynteye/right/image_color
/mynteye/right/image_color/compressed
/mynteye/right/image_color/compressed/parameter_descriptions
/mynteye/right/image_color/compressed/parameter_updates
/mynteye/right/image_color/theora
/mynteye/right/image_color/theora/parameter_descriptions
/mynteye/right/image_color/theora/parameter_updates
/mynteye/right/image_mono
/mynteye/right/image_mono/compressed
/mynteye/right/image_mono/compressed/parameter_descriptions
/mynteye/right/image_mono/compressed/parameter_updates
/mynteye/right/image_mono/theora
/mynteye/right/image_mono/theora/parameter_descriptions
/mynteye/right/image_mono/theora/parameter_updates
/mynteye/temp/data_raw
/rosout
/rosout_agg
/tf

So I try to start rtabmap with the following arguments:

roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start" depth_topic:=/mynteye/depth/image_raw rgb_topic:=/mynteye/left/image_color camera_info_topic:=/mynteye/left/camera_info approx_sync:=false

However, I get the following error:

[ WARN] [1568594549.855475544]: odometry: Could not get transform from camera_link to mynteye_left_color_frame (stamp=1568594549.638142) after 0.200000 seconds ("wait_for_transform_duration"=0.200000)! Error="canTransform: target_frame camera_link does not exist.. canTransform returned after 0.201617 timeout was 0.2."

and every five seconds I get:

[ WARN] [1568594597.426327261]: /rtabmap/rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called.
/rtabmap/rtabmap subscribed to (exact sync):
   /rtabmap/odom,
   /mynteye/left/image_color,
   /mynteye/depth/image_raw,
   /mynteye/left/camera_info,
   /rtabmap/odom_info

The topics appear to be working OK:

joseph@Alarei:~/MYNT-EYE-D-SDK$ rostopic hz /mynteye/left/image_color
subscribed to [/mynteye/left/image_color]
average rate: 31.095
        min: 0.012s max: 0.048s std dev: 0.00838s window: 23

joseph@Alarei:~/MYNT-EYE-D-SDK$ rostopic hz /mynteye/depth/image_raw
subscribed to [/mynteye/depth/image_raw]
average rate: 30.284
        min: 0.029s max: 0.041s std dev: 0.00472s window: 27

joseph@Alarei:~/MYNT-EYE-D-SDK$ rostopic hz /mynteye/left/camera_info
subscribed to [/mynteye/left/camera_info]
average rate: 30.341
        min: 0.029s max: 0.041s std dev: 0.00472s window: 28

I hope I have included enough information, I am pretty noob at all of this.

Thanks!

[matlabbe]

Hi, you are not so far. You need to set the base frame of your camera. You may try something like this:

roslaunch rtabmap_ros rtabmap.launch \
  rtabmap_args:="--delete_db_on_start" \
  depth_topic:=/mynteye/depth/image_raw \
  rgb_topic:=/mynteye/left/image_color \
  camera_info_topic:=/mynteye/left/camera_info approx_sync:=false \
  frame_id:=mynteye_left_color_frame

However, it may require an optical rotation so that the map is not created looking up. What are the frames used with this camera ($ rosrun tf view_frames). In case there is only mynteye_left_color_frame, you would have to add the optical rotation yourself with:

rosrun tf static_transform_publisher 0 0 0 -1.5707963267948966 0 -1.5707963267948966 camera_link mynteye_left_color_frame 100

then use frame_id:=camera_link.

cheers,
Mathieu

[jroddis]

Hi Mathieu, many thanks for your help I am almost there I think!

I have attached the frames.pdf:

frames.pdf

Using the following:

roslaunch rtabmap_ros rtabmap.launch \
  rtabmap_args:="--delete_db_on_start" \
  depth_topic:=/mynteye/depth/image_raw \
  rgb_topic:=/mynteye/left/image_color \
  camera_info_topic:=/mynteye/left/camera_info approx_sync:=false \
  frame_id:=mynteye_left_color_frame

Is a good start but as you anticipated, the camera needs rotating. The console output is all good:

[ INFO] [1568659815.847835452]: Odom: quality=652, std dev=0.005542m|0.036256rad, update time=0.120141s
[ INFO] [1568659816.000959904]: Odom: quality=612, std dev=0.006137m|0.027549rad, update time=0.133941s
[ INFO] [1568659816.128913093]: Odom: quality=633, std dev=0.006308m|0.027549rad, update time=0.125715s
[ INFO] [1568659816.243661417]: Odom: quality=627, std dev=0.007566m|0.032761rad, update time=0.112457s
[ INFO] [1568659816.370855891]: Odom: quality=635, std dev=0.005836m|0.027549rad, update time=0.114855s
[ INFO] [1568659816.497554557]: Odom: quality=647, std dev=0.004842m|0.027549rad, update time=0.120131s
[ INFO] [1568659816.646979820]: Odom: quality=639, std dev=0.005091m|0.027549rad, update time=0.145821s
[ INFO] [1568659816.782691336]: rtabmap (61): Rate=1.00s, Limit=0.000s, RTAB-Map=0.2792s, Maps update=0.0035s pub=0.0000s (local map=2, WM=2)

When I run this line:

rosrun tf static_transform_publisher 0 0 0 -1.5707963267948966 0 -1.5707963267948966 camera_link

Then:

roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start" depth_topic:=/mynteye/depth/image_raw rgb_topic:=/mynteye/left/image_color camera_info_topic:=/mynteye/left/camera_info approx_sync:=false frame_id:=camera_link

The camera is correctly orientated and I am able to do some mapping, however I get a lot of these messages in the console:

[ WARN] [1568659920.788315649]: odometry: Could not get transform from camera_link to mynteye_left_color_frame (stamp=1568659920.533475) after 0.200000 seconds ("wait_for_transform_duration"=0.200000)! Error="Lookup would require extrapolation at time 1568659920.533474684, but only time 1568659916.053164721 is in the buffer, when looking up transform from frame [mynteye_left_color_frame] to frame [camera_link]. canTransform returned after 0.200727 timeout was 0.2."

Do you know why I would start to get the error after running the static_transform_publisher?

Again, many thanks.

[matlabbe]

Hi,

based on the frames, you may just set frame_id to "mynteye_link_frame". The optical transformation may be already included in mynteye_link_frame -> mynteye_left_color_frame TF.

roslaunch rtabmap_ros rtabmap.launch \
  rtabmap_args:="--delete_db_on_start" \
  depth_topic:=/mynteye/depth/image_raw \
  rgb_topic:=/mynteye/left/image_color \
  camera_info_topic:=/mynteye/left/camera_info approx_sync:=false \
  frame_id:=mynteye_link_frame

When using static_transform_publisher, the last parameter is the period, it seems missing from your line. Setting to 100 ms would make the transform published at 10 Hz.

cheers,
Mathieu