rtabmap_ros jetson tx2 and zed camera launch issue

[abmros1]

Hello,

I am testing rtabmap_ros on the jetson tx2 with the Zed camera. I follow the tutorial http://wiki.ros.org/rtabmap_ros/Tutorials/HandHeldMapping without success. When launch with:
 $ rosrun tf static_transform_publisher 0 0 0 -1.5707963267948966 0 -1.5707963267948966 camera_link zed_initial_frame 100
 $ rosrun tf static_transform_publisher 0 0 0 0 0 0 zed_current_frame ZED_left_camera  100
 $ roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start" depth_topic:=/camera/depth/depth_registered approx_sync:=false

I got the following error:

[ INFO] [1502839748.556378811]: Reading parameters from the ROS server...
OpenCV Error: Assertion failed (channels() == CV_MAT_CN(dtype)) in copyTo, file /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/copy.cpp, line 259
terminate called after throwing an instance of 'cv::Exception'
  what():  /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/copy.cpp:259: error: (-215) channels() == CV_MAT_CN(dtype) in function copyTo

[rtabmap/rgbd_odometry-1] process has died [pid 5702, exit code -6, cmd /home/nvidia/catkin_ws/devel/lib/rtabmap_ros/rgbd_odometry --delete_db_on_start rgb/image:=/camera/rgb/image_rect_color depth/image:=/camera/depth/depth_registered rgb/camera_info:=/camera/rgb/camera_info rgbd_image:=/camera/rgbd_image __name:=rgbd_odometry __log:=/home/nvidia/.ros/log/46fe7a5a-8211-11e7-a1b5-00044b8c57b6/rtabmap-rgbd_odometry-1.log].
log file: /home/nvidia/.ros/log/46fe7a5a-8211-11e7-a1b5-00044b8c57b6/rtabmap-rgbd_odometry-1*.log

also warnings :
[ WARN] [1502839755.301427490]: /rtabmap/rtabmapviz: Did not receive data since 5 seconds! ...

rtabmapviz shows blank screen!

Is there a conflict with the opencv version used in the jetson (2.4...) (got it working on my ubuntu destop). Any help is appreciated.

Thanks

[matlabbe]

Hi,

Yes there can be a conflict if rtabmap library is built against OpenCV2 and rtabmap_ros is built with OpenCV3. You may try to build rtabmap against OpenCV3 instead of OpenCV2 if it is the case:

$ export OpenCV_DIR=/opt/ros/kinetic/share/OpenCV-3.2.0-dev
$ rm -r rtabmap/build/*
$ cd rtabmap/build
$ cmake ..
$ make

$ rm -r ~/catkin_ws/build
$ cd ~/catkin_ws
$ catkin_make

If there are still conflicts, do you really need OpenCV2? If so, remove "ros-kinetic-opencv3" from kinetic installation, then rebuild everything. Note that you will have to build cv_bridge from source (so it is built against OpenCV2) and other packages needed by rtabmap_ros depending on ros-kinetic-opencv3.

cheers,
Mathieu

[abmros1]

Hi Mathieu,

Thanks for the quick response. I follow instructions in your reply and build rtabmap against OpenCV3. This time it gives me a somewhat different error:

[ INFO] [1503075754.223977172]: Parameters successfully read.
OpenCV Error: The function/feature is not implemented (Unknown/unsupported array type) in type, file /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/matrix.cpp, line 1931
terminate called after throwing an instance of 'cv::Exception'
  what():  /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/matrix.cpp:1931: error: (-213) Unknown/unsupported array type in function type

[rtabmap/rgbd_odometry-1] process has died [pid 30929, exit code -6, cmd /home/nvidia/catkin_ws/devel/lib/rtabmap_ros/rgbd_odometry --delete_db_on_start rgb/image:=/camera/rgb/image_rect_color depth/image:=/camera/depth/depth_registered rgb/camera_info:=/camera/rgb/camera_info rgbd_image:=/camera/rgbd_image __name:=rgbd_odometry __log:=/home/nvidia/.ros/log/c45555f8-842f-11e7-87b5-00044b8c57b4/rtabmap-rgbd_odometry-1.log].
log file: /home/nvidia/.ros/log/c45555f8-842f-11e7-87b5-00044b8c57b4/rtabmap-rgbd_odometry-1*.log
[ INFO] [1503075755.002151224]: /rtabmap/rtabmapviz: queue_size    = 10
[ INFO] [1503075755.002279320]: /rtabmap/rtabmapviz: rgbd_cameras = 1
[ INFO] [1503075755.002322328]: /rtabmap/rtabmapviz: approx_sync   = false
[ INFO] [1503075755.002401400]: Setup depth callback
[ INFO] [1503075755.381619268]:
/rtabmap/rtabmapviz subscribed to (exact sync):
   /rtabmap/odom,
   /camera/rgb/image_rect_color,
   /camera/depth/depth_registered,
   /camera/rgb/camera_info,
   /rtabmap/odom_info


I am not sure this is OpenCv issue or not. My camera topics are published :

rostopic hz /camera/rgb/image_rect_color
subscribed to [/camera/rgb/image_rect_color]
average rate: 94.827
        min: 0.006s max: 0.017s std dev: 0.00190s window: 90
average rate: 95.385
        min: 0.006s max: 0.017s std dev: 0.00179s window: 186
average rate: 94.900
        min: 0.006s max: 0.025s std dev: 0.00203s window: 280

rostopic hz /camera/depth/depth_registered
subscribed to [/camera/depth/depth_registered]
average rate: 40.641
        min: 0.020s max: 0.033s std dev: 0.00232s window: 38
average rate: 44.054
        min: 0.017s max: 0.033s std dev: 0.00327s window: 85
average rate: 47.592

rostopic hz /camera/rgb/camera_info
subscribed to [/camera/rgb/camera_info]
average rate: 40.013
        min: 0.021s max: 0.030s std dev: 0.00251s window: 39
average rate: 41.023
        min: 0.016s max: 0.033s std dev: 0.00314s window: 81
average rate: 42.166

And my transform tree looks like this:


Could you give some additional insights to what my problem is?

Thanks,

[abmros1]

here are some ldd outputs:

nvidia@tegra-ubuntu:~/catkin_ws/devel/lib$ ldd librtabmap_ros.so | grep opencv
        libopencv_core3.so.3.2 => /opt/ros/kinetic/lib/libopencv_core3.so.3.2 (0x0000007f8e1ab000)
        libopencv_video.so.2.4 => /usr/lib/libopencv_video.so.2.4 (0x0000007f8c46a000)
        libopencv_calib3d.so.2.4 => /usr/lib/libopencv_calib3d.so.2.4 (0x0000007f8c3af000)
        libopencv_highgui.so.2.4 => /usr/lib/libopencv_highgui.so.2.4 (0x0000007f8c359000)
        libopencv_imgcodecs3.so.3.2 => /opt/ros/kinetic/lib/libopencv_imgcodecs3.so.3.2 (0x0000007f8b491000)
        libopencv_imgproc3.so.3.2 => /opt/ros/kinetic/lib/libopencv_imgproc3.so.3.2 (0x0000007f8b17f000)
        libopencv_calib3d3.so.3.2 => /opt/ros/kinetic/lib/libopencv_calib3d3.so.3.2 (0x0000007f8afd1000)
        libopencv_xfeatures2d3.so.3.2 => /opt/ros/kinetic/lib/libopencv_xfeatures2d3.so.3.2 (0x0000007f8ad02000)
        libopencv_video3.so.3.2 => /opt/ros/kinetic/lib/libopencv_video3.so.3.2 (0x0000007f867f1000)
        libopencv_features2d3.so.3.2 => /opt/ros/kinetic/lib/libopencv_features2d3.so.3.2 (0x0000007f86753000)
        libopencv_photo3.so.3.2 => /opt/ros/kinetic/lib/libopencv_photo3.so.3.2 (0x0000007f866a6000)
        libopencv_videoio3.so.3.2 => /opt/ros/kinetic/lib/libopencv_videoio3.so.3.2 (0x0000007f8666f000)
        libopencv_core.so.2.4 => /usr/lib/libopencv_core.so.2.4 (0x0000007f85106000)
        libopencv_imgproc.so.2.4 => /usr/lib/libopencv_imgproc.so.2.4 (0x0000007f84dc9000)
        libopencv_flann.so.2.4 => /usr/lib/libopencv_flann.so.2.4 (0x0000007f84d4e000)
        libopencv_features2d.so.2.4 => /usr/lib/libopencv_features2d.so.2.4 (0x0000007f84be6000)
        libopencv_flann3.so.3.2 => /opt/ros/kinetic/lib/libopencv_flann3.so.3.2 (0x0000007f829e8000)

nvidia@tegra-ubuntu:~/catkin_ws/devel/lib$ ldd librtabmap_ros.so | grep cv_bridge
        libcv_bridge.so => /opt/ros/kinetic/lib/libcv_bridge.so (0x0000007f930f9000)


nvidia@tegra-ubuntu:~/rtabmap/bin$ ldd librtabmap_core.so | grep opencv
        libopencv_xfeatures2d3.so.3.2 => /opt/ros/kinetic/lib/libopencv_xfeatures2d3.so.3.2 (0x0000007f7f7fb000)
        libopencv_video3.so.3.2 => /opt/ros/kinetic/lib/libopencv_video3.so.3.2 (0x0000007f7b0be000)
        libopencv_calib3d3.so.3.2 => /opt/ros/kinetic/lib/libopencv_calib3d3.so.3.2 (0x0000007f7afa8000)
        libopencv_features2d3.so.3.2 => /opt/ros/kinetic/lib/libopencv_features2d3.so.3.2 (0x0000007f7af09000)
        libopencv_photo3.so.3.2 => /opt/ros/kinetic/lib/libopencv_photo3.so.3.2 (0x0000007f7ae5c000)
        libopencv_videoio3.so.3.2 => /opt/ros/kinetic/lib/libopencv_videoio3.so.3.2 (0x0000007f7ae26000)
        libopencv_imgcodecs3.so.3.2 => /opt/ros/kinetic/lib/libopencv_imgcodecs3.so.3.2 (0x0000007f7ad16000)
        libopencv_imgproc3.so.3.2 => /opt/ros/kinetic/lib/libopencv_imgproc3.so.3.2 (0x0000007f7aa04000)
        libopencv_core3.so.3.2 => /opt/ros/kinetic/lib/libopencv_core3.so.3.2 (0x0000007f7a7e4000)
        libopencv_flann3.so.3.2 => /opt/ros/kinetic/lib/libopencv_flann3.so.3.2 (0x0000007f72547000)

They appear to be linked correctly? I will try building everything against OpenCV2

Thanks

[matlabbe]

Hi,

I am seeing some Opencv 2.4 libraries linked to librtabmap_ros.so, as they should be all OpenCV3. librtabmap_core.so seems okay. Make sure to delete the build directory of catkin to make sure that cmake redetect which OpenCV is used (OpenCV_DIR).

cheers,
Mathieu

[abmros1]

Awesome! It was the librtabmap_ros.so links. I set OpenCV_DIR and find_package(OpenCV 3) in rtabmap_ros Cmakelists, now it works like a charm!

Cheers,