rtmapviz black screen

[Nico]

Hello,

I am a beginner with rtabmap, and I have a problem with rtabmapviz.
I am using a SLAMdunk by parrot, so I modified the topic names in the launch files.
I have my 2 images published by the left and right cameras and the camera_info
However, when I launch stereo_mapping_launch but when I launch it I only have a black screen..
I followed the tutorial http://wiki.ros.org/rtabmap_ros/Tutorials/StereoHandHeldMapping.
I don't know what to do next, can someone help me please ?

Thank you for your help.

Nicolas

[matlabbe]

Hi Nicolas,

What are the topics of the rectified images published by SLAMdunk? Is it this package used: https://github.com/Parrot-Developers/slamdunk_ros?

$ rostopic list

Do you have any warning messages? Are you on Indigo or Kinetic?

cheers,
Mathieu

[Nico]

Hello Matthieu,

Thank you for your quick answer.
So I am using the package you mentionned.
The SLAMdunk only publishes rectified image topics concerning the left camera :
/left_rgb_rect/camera_info
/left_rgb_rect/image_rect_color
/left_rgb_rect/image_rect_color/compressed
/left_rgb_rect/image_rect_color/compressed/parameter_descriptions
/left_rgb_rect/image_rect_color/compressed/parameter_updates
/left_rgb_rect/image_rect_color_good
/left_rgb_rect/image_rect_color_kf
/left_rgb_rect/image_rect_color_kf/compressed
/left_rgb_rect/image_rect_color_kf/compressed/parameter_descriptions
/left_rgb_rect/image_rect_color_kf/compressed/parameter_updates
/left_rgb_rect/image_rect_color_kf_good

I also tried your package with the rgbd_mapping.launch and a turtlebot on Gazebo (turtlebot_world.launch from turtlebot_gazebo package), and I only changed the depth_registered_topic with /camera/depth/image_row, and I can see what I want on RTABMAPVIZ, even if the close loop hypothesis is rejected everytime, I think I have to make it more permissive.

Before using stereo_mapping, I wanted to do the same with the SLAMdunk, with the topic /left_rgb/ image which is the image sent by the camera before rectification, /left_rgb/camera_info and /depth_map/image in order to get a sensor_msgs/Image of the depth like I had with the turtlebot. But I have these warnings :

[ WARN] [1493386629.630644887]: /rtabmap/rtabmapviz: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10).
/rtabmap/rtabmapviz subscribed to (approx sync):
   /rtabmap/odom,
   /left_rgb/image,
   /depth_map/image,
   /left_rgb/camera_info,
   /rtabmap/odom_info
[ WARN] [1493386630.436520698]: /rtabmap/rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set.
/rtabmap/rgbd_odometry subscribed to (approx sync):
   /left_rgb/image,
   /depth_map/image,
   /left_rgb/camera_info

However I checked the 3 topics    /left_rgb/image,   /depth_map/image,   /left_rgb/camera_info, and I received something.
The 2 topics from rtabmap seem to be empty, as I don't receive anything with rostopic echo or rostopic hz.

I see the warning with the synchronized clocks, but I don't know what to do about it.
Is the warning occuring because I put the wrong topics in the launch file ?
I don't understand why the clocks need to be synchronized for this, as I can work with the image topics with my other programs (face recognition for example).

Thank you for the time you spent considering my problem.

[matlabbe]

Hi,

What is the framerate of the input topics?

$ rostopic hz /left_rgb/image
$ rostopic hz /depth_map/image
$ rostopic hz /left_rgb/camera_info 

Note that you should use the rectified topic for the left image. It is also strange that there are just left topics, as in the slamdunk_node main.cpp, there are a lot of publishers:

            , m_depthmapPublisher(image_transport::ImageTransport(n).advertiseCamera("depth_map/image", 1))
            , m_dispmapPublisher(image_transport::ImageTransport(n).advertiseCamera("disp_map/image", 1))
            , m_stereoPublisher(n.advertise<stereo_msgs::DisparityImage>("stereo/disparity", 1))
            , m_leftRgbPublisher(image_transport::ImageTransport(n).advertiseCamera("left_rgb/image", 1))
            , m_leftGrayscalePublisher(image_transport::ImageTransport(n).advertiseCamera("left_grayscale/image", 1))
            , m_leftRgbRectPublisher(image_transport::ImageTransport(n).advertiseCamera("left_rgb_rect/image_rect_color", 1))
            , m_rightRgbPublisher(image_transport::ImageTransport(n).advertiseCamera("right_rgb/image", 1))
            , m_rightGrayscalePublisher(image_transport::ImageTransport(n).advertiseCamera("right_grayscale/image", 1))
            , m_pointCloud2Publisher(n.advertise<sensor_msgs::PointCloud2>("pcl_xyz", 1))
            , m_imuPublisher(n.advertise<sensor_msgs::Imu>("imu", 1))
            , m_posePublisher(n.advertise<geometry_msgs::PoseStamped>("pose", 1))
            , m_occupancyPublisher(n.advertise<octomap_msgs::Octomap>("octomap", 1))
            , m_ultrasoundPublisher(n.advertise<sensor_msgs::Range>("ultrasound", 1))
            , m_barometerPublisher(n.advertise<sensor_msgs::FluidPressure>("barometer", 1))
            , m_magnetometerPublisher(n.advertise<sensor_msgs::MagneticField>("magnetometer", 1))
            , m_processingImageRatePublisher(n.advertise<std_msgs::Float32>("processing_image_rate", 1))
            , m_keyframePublisher(n.advertise<slamdunk_msgs::BoolStamped>("keyframe", 1))
            , m_qualityPublisher(n.advertise<slamdunk_msgs::QualityStamped>("quality", 1))

In gazebo, it is normal that most loop closures are rejected, as generally simulated environments have very poor visual features.

cheers,
Mathieu

[Nico]

Hello,

Sorry I only put the topics concerning the left camera. Here are all the topics published by the SLAMdunk :

/barometer
/depth_map/camera_info
/depth_map/image
/depth_map/image/compressed
/depth_map/image/compressed/parameter_descriptions
/depth_map/image/compressed/parameter_updates
/depth_map/image_good
/depth_map/image_kf
/depth_map/image_kf/compressed
/depth_map/image_kf/compressed/parameter_descriptions
/depth_map/image_kf/compressed/parameter_updates
/depth_map/image_kf_good
/disp_map/camera_info
/disp_map/image
/disp_map/image/compressed
/disp_map/image/compressed/parameter_descriptions
/disp_map/image/compressed/parameter_updates
/imu
/keyframe
/left_grayscale/camera_info
/left_grayscale/image
/left_grayscale/image/compressed
/left_grayscale/image/compressed/parameter_descriptions
/left_grayscale/image/compressed/parameter_updates
/left_rgb/camera_info
/left_rgb/image
/left_rgb/image/compressed
/left_rgb/image/compressed/parameter_descriptions
/left_rgb/image/compressed/parameter_updates
/left_rgb/image_good
/left_rgb/image_kf
/left_rgb/image_kf/compressed
/left_rgb/image_kf/compressed/parameter_descriptions
/left_rgb/image_kf/compressed/parameter_updates
/left_rgb/image_kf_good
/left_rgb_rect/camera_info
/left_rgb_rect/image_rect_color
/left_rgb_rect/image_rect_color/compressed
/left_rgb_rect/image_rect_color/compressed/parameter_descriptions
/left_rgb_rect/image_rect_color/compressed/parameter_updates
/left_rgb_rect/image_rect_color_good
/left_rgb_rect/image_rect_color_kf
/left_rgb_rect/image_rect_color_kf/compressed
/left_rgb_rect/image_rect_color_kf/compressed/parameter_descriptions
/left_rgb_rect/image_rect_color_kf/compressed/parameter_updates
/left_rgb_rect/image_rect_color_kf_good
/magnetometer
/menu_topic_ns/feedback
/menu_topic_ns/update
/menu_topic_ns/update_full
/octomap
/path
/path_keyframe
/pcl_xyz
/pcl_xyzrgb
/pcl_xyzrgb_concat
/pcl_xyzrgb_good
/pcl_xyzrgb_kf
/pcl_xyzrgb_kf_good
/pose
/processing_image_rate
/quality
/right_grayscale/camera_info
/right_grayscale/image
/right_grayscale/image/compressed
/right_grayscale/image/compressed/parameter_descriptions
/right_grayscale/image/compressed/parameter_updates
/right_rgb/camera_info
/right_rgb/image
/right_rgb/image/compressed
/right_rgb/image/compressed/parameter_descriptions
/right_rgb/image/compressed/parameter_updates
/rosout
/rosout_agg
/slamdunk_node/parameter_descriptions
/slamdunk_node/parameter_updates
/stereo/disparity
/tf
/tf_static
/ultrasound

Here are the rostopic hz :
$ rostopic hz /left_rgb/image
subscribed to [/left_rgb/image]
average rate: 4.503
        min: 0.222s max: 0.222s std dev: 0.00000s window: 2
average rate: 4.964
        min: 0.188s max: 0.222s std dev: 0.01059s window: 7
average rate: 4.885
        min: 0.188s max: 0.245s std dev: 0.01490s window: 12


$ rostopic hz /depth_map/image
subscribed to [/depth_map/image]
average rate: 30.444
        min: 0.023s max: 0.053s std dev: 0.00662s window: 22
average rate: 30.246
        min: 0.023s max: 0.053s std dev: 0.00649s window: 52
average rate: 30.067
        min: 0.021s max: 0.053s std dev: 0.00582s window: 82
^Caverage rate: 30.066
        min: 0.020s max: 0.053s std dev: 0.00631s window: 88

$ rostopic hz /left_rgb/camera_info
subscribed to [/left_rgb/camera_info]
average rate: 30.185
        min: 0.030s max: 0.034s std dev: 0.00116s window: 10
average rate: 30.052
        min: 0.022s max: 0.046s std dev: 0.00416s window: 40
average rate: 30.016
        min: 0.022s max: 0.046s std dev: 0.00390s window: 70
^Caverage rate: 30.001
        min: 0.022s max: 0.046s std dev: 0.00365s window: 80


So I got something.
I am working with a virtual machine, do you know if it has any influence on the good functionning of RTAB ?

Concerning the simulator, is it possible to have the closed loop hypothesis if I modify some parameters of your program ?

Thank you very much for your help

[matlabbe]

Hi,

With these kind of topics, I would go with "/left_rgb_rect/camera_info", "/left_rgb_rect/image_rect_color" and "/depth_map/image". Don't know what means "/depth_map/image_kf" or "/depth_map/image/good".

Note that the rgb image seems to be published a lot slower than other topics. You may increase queue_size parameter to be able to synchronize the topics. If images are streamed on Wifi, you may also subscribe to compressed rgb image. Example for SlamDunk:

$ roslaunch rtabmap_ros rtabmap.launch \
      rgb_topic:=/left_rgb_rect/image_rect_color \
      camera_info_topic:=/left_rgb_rect/camera_info \
      depth_topic:=/depth_map/image \
      compressed:=true \
      queue_size:=10

Make sure the compressedDepth image_transport plugin is installed (both on slamdunk and remote computer):

$ sudo apt-get install ros-kinetic-compressed-depth-image-transport

In simulation, adding a 2d laser scanner may help to find more loop closures. Otherwise, add unique textures to world for visual loop closure detection.

cheers,
Mathieu

[Nico]

Hi matlabbe !

I had this problem
/*
I got some problems with my computer, so that's why I did not answer as quickly as I wanted.
I updated my slamdunk, and now I have a new problem :

[ WARN] [1494317669.543690914]: odometry: Could not get transform from camera_link to cam0_optical (stamp=1494317669.112413) after 0.200000 seconds ("wait_for_transform_duration"=0.200000)!
*/
(post edited)

I found how to correct it, but I have still the black screen problem.
The last message shown on the screen is
 [ INFO] [1494319600.668802907]: rtabmap 0.11.8 started...


I also updated my rtab_map version but now it is no longer working with gazebi and my turtlebot....

/rtabmap/rtabmap subscribed to (approx sync):
   /rtabmap/odom,
   /camera/rgb/image_raw,
   /camera/depth/image_raw,
   /camera/rgb/camera_info,
   /scan
[ INFO] [1494331031.466581327, 592.000000000]: rtabmap 0.12.5 started...
[ WARN] (2017-05-09 13:57:13.473) OdometryF2M.cpp:380::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=11) between -1 and 7"
[ INFO] [1494331033.473406167, 594.010000000]: Odom: quality=0, std dev=0.000000m|0.000000rad, update time=0.022192s
/home/assystem/Bureau/RTAB2/devel/lib/rtabmap_ros/rtabmap: symbol lookup error: /home/assystem/Bureau/RTAB2/devel/lib//librtabmap_ros.so: undefined symbol: _ZN7rtabmap7Rtabmap7processERKNS_10SensorDataENS_9TransformERKN2cv3MatE
[rtabmap/rtabmap-2] process has died [pid 9624, exit code 127, cmd /home/assystem/Bureau/RTAB2/devel/lib/rtabmap_ros/rtabmap rgb/image:=/camera/rgb/image_raw depth/image:=/camera/depth/image_raw rgb/camera_info:=/camera/rgb/camera_info left/image_rect:=/stereo_camera/left/image_rect_color right/image_rect:=/stereo_camera/right/image_rect left/camera_info:=/stereo_camera/left/camera_info right/camera_info:=/stereo_camera/right/camera_info scan:=/scan scan_cloud:=/scan_cloud user_data:=/user_data user_data_async:=/user_data_async __name:=rtabmap __log:=/home/assystem/.ros/log/3fb1e874-34ad-11e7-8cd4-2c27d72db473/rtabmap-rtabmap-2.log].
log file: /home/assystem/.ros/log/3fb1e874-34ad-11e7-8cd4-2c27d72db473/rtabmap-rtabmap-2*.log
[ WARN] (2017-05-09 13:57:13.985) OdometryF2M.cpp:380::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=7) between -1 and 8"




I am still using rgbd_mapping.launch
I try to find what the problem is

nicolas

[matlabbe]

Hi,

This error:

undefined symbol: _ZN7rtabmap7Rtabmap7processERKNS_10SensorDataENS_9TransformERKN2cv3MatE 
[rtabmap/rtabmap-2] process has died [pid 9624, exit code 127, cmd /home/assystem/Bureau/RTAB2/devel/lib/rtabmap_ros/rtabmap

seems to be a mismatch between rtabmap_ros and rtabmap built versions. Make sure you don't have the binaries installed, so that rtabmap_ros links on rtabmap libraries.

For this error:

OdometryF2M.cpp:380::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=11) between -1 and 7"

That says that visual odometry cannot be computed, because of lack of visual features. See Lost Odometry section here for more info.

cheers,
Mathieu

[Nico]

Hi

You were right, so thanks to your help I removed this error.

But with everything updated, I came back to this error :
[ WARN] [1494493821.751923865]: /rtabmap/rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set.
/rtabmap/rgbd_odometry subscribed to (approx sync):
   /left_rgb_rect/image_rect_color,
   /depth_map/image,
   /left_rgb_rect/camera_info
[ WARN] [1494493822.277038101]: /rtabmap/rtabmapviz: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=1000).
/rtabmap/rtabmapviz subscribed to (approx sync):
   /rtabmap/odom,
   /left_rgb_rect/image_rect_color,
   /depth_map/image,
   /left_rgb_rect/camera_info,
   /rtabmap/odom_info


I put a higher value to the queue_size (1000), and when I try to analyse my topics I got something with these 3 :    /left_rgb_rect/image_rect_color,   /depth_map/image,   /left_rgb_rect/camera_info
but nothing from /rtabmap/odom and  /rtabmap/odom_info.

In commondatasubscriber, I tried to understand  subscribedToDepth_, so I modified some parameters in rgbd_mapping.launch and rtab.launch without success...

I also try the compressed version of the topics but it did not change anything.

I know that I am no far from the good functionning of your package with the slamdunk, so I don't lose hope !

Do you know have other ideas to help me ?

Thanks a lot for the time spent to help me

[matlabbe]

Hi Nico,

it is normal that /rtabmap/odom and /rtabmap/odom_info  are not published as /rtabmap/rgbd_odometry is not working, waiting on the shown topics. Can you show these informations:

# see if the stamps are correctly set
$ rostopic echo /left_rgb_rect/image_rect_color/header
$ rostopic echo /depth_map/image/header
$ rostopic echo /left_rgb_rect/camera_info/header

# compare framerate, raw then compressed:
$ rostopic hz /left_rgb_rect/image_rect_color
$ rostopic hz /left_rgb_rect/image_rect_color/compressed

$ rostopic hz /depth_map/image
$ rostopic hz /depth_map/image/compressedDepth

$ rostopic hz /left_rgb_rect/camera_info

These warnings are shown when topics cannot be synchronized, for the mentioned reasons.

cheers,
Mathieu

[Nico]

Hello,

I found out a solution !
I modified these parameters :
  <arg name="frame_id"                default="cam0_rect_optical"/>     
  <arg name="odom_frame_id"           default="map"/>               
  <arg name="map_frame_id"            default="world"/>

in the rtabmap.launch file.
I did it before, but it was said that I could not get the transform from cam0_rect_optical and map (data normally sent by the slamdunk) and this was due to the fact that map only exists if a topic of the "slam" node of the slamdunk (not the topics concerning the image etc) is subscribed.
I just have to do a rostopic echo /pose and in an other terminal roslaunch rtabmap_ros rtabmap.launch to get the good functionning !

SO thank you very much for your help, it is really nice from you.

Now I am facing another problem... of compilation on my slamdunk
I was working on my computer with the slamdunk connected, but the next step is to put the package in the slamdunk.
I followed all the steps mentioned there https://github.com/introlab/rtabmap_ros 
But when it comes to compiling with catkin_make I have this message
make[2]: *** No rule to make target `/usr/lib/arm-linux-gnueabihf/libopencv_videostab.so.2.4.8', needed by `/home/slamdunk/ROS/mon_ws/devel/lib/rtabmap_ros/camera'.  Stop.

I guess it comes from the fact that the opencv version of the slamdunk is 2.4.12 and on my computer it is 2.4.8.
I tried to look at the makefile, the cmakelist but without success.
Do you know where it can come from ?

Thank you again for your help.

Nico

[matlabbe]

Hi Nico,

If you just copied the binaries, it may indeed fail. You could copy also opencv 2.4.8 libraries on SlamDunk. Another solution is to install 2.4.12 on your computer and make rtabmap and rtabmap_ros build against it. Beware to rebuild vision_opencv packages on computer to get cv_bridge linked on 2.4.12 instead of 2.4.8.

cheers,
Mathieu

[Nickhodem]

Hi,
Congratulation Nico that you successfully integrated rtab-map with slamdunk. I've also change "stereo_mapping.launch" but I always get this error:
[ERROR] [1495542962.817695335]: Ignoring transform from authority "unknown_publisher" with frame_id and child_frame_id  "map" because they are the same
Do you know what I'm doing wrong?

My launch file:
<arg name="rtabmapviz"              default="true" /> 
  <arg name="rviz"                    default="false" />
 
 
  <arg name="localization"            default="false"/>
 
 
  <arg name="rtabmapviz_cfg"          default="$(find rtabmap_ros)/launch/config/rgbd_gui.ini" />
  <arg name="rviz_cfg"                default="$(find rtabmap_ros)/launch/config/rgbd.rviz" />
 
  <arg name="frame_id"                default="base_link"/>     
  <arg name="database_path"           default="~/.ros/rtabmap.db"/>
  <arg name="rtabmap_args"            default=""/>   
  <arg name="launch_prefix"           default=""/>
  <arg name="approx_sync"             default="false"/>         
 
  <arg name="stereo_namespace"        default="/multisense_sl/camera"/>
  <arg name="left_image_topic"        default="$(arg stereo_namespace)/left/image_raw" />
  <arg name="right_image_topic"       default="$(arg stereo_namespace)/right/image_raw" />     
  <arg name="left_camera_info_topic"  default="$(arg stereo_namespace)/left/camera_info" />
  <arg name="right_camera_info_topic" default="$(arg stereo_namespace)/right/camera_info" />
  <arg name="compressed"              default="false"/>
   
  <arg name="subscribe_scan"          default="false"/>         
  <arg name="scan_topic"              default="/scan"/>
   
  <arg name="subscribe_scan_cloud"    default="true"/>         
  <arg name="scan_cloud_topic"        default="/velodyne_points"/>
 
  <arg name="visual_odometry"         default="true"/>         
  <arg name="odom_topic"              default="/odom"/>         
  <arg name="odom_frame_id"           default=""/>             
 
  <arg name="namespace"               default="rtabmap"/>
  <arg name="wait_for_transform"      default="0.2"/>
 
  <include file="$(find rtabmap_ros)/launch/rtabmap.launch">
    <arg name="stereo"                  value="true"/>
    <arg name="rtabmapviz"              value="$(arg rtabmapviz)" /> 
    <arg name="rviz"                    value="$(arg rviz)" />
    <arg name="localization"            value="$(arg localization)"/>
    <arg name="gui_cfg"                 value="$(arg rtabmapviz_cfg)" />
    <arg name="rviz_cfg"                value="$(arg rviz_cfg)" />
 
    <arg name="frame_id"                value="$(arg frame_id)"/>
    <arg name="namespace"               value="$(arg namespace)"/>
    <arg name="database_path"           value="$(arg database_path)"/>
    <arg name="wait_for_transform"      value="$(arg wait_for_transform)"/>
    <arg name="rtabmap_args"            value="$(arg rtabmap_args)"/> 
    <arg name="launch_prefix"           value="$(arg launch_prefix)"/>         
    <arg name="approx_sync"             value="$(arg approx_sync)"/>

    <arg name="stereo_namespace"        value="$(arg stereo_namespace)"/>
    <arg name="left_image_topic"        value="$(arg left_image_topic)" />
    <arg name="right_image_topic"       value="$(arg right_image_topic)" />
    <arg name="left_camera_info_topic"  value="$(arg left_camera_info_topic)" />
    <arg name="right_camera_info_topic" value="$(arg right_camera_info_topic)" />
 
    <arg name="compressed"              value="$(arg compressed)"/>                                                                               
   
    <arg name="subscribe_scan"          value="$(arg subscribe_scan)"/>
    <arg name="scan_topic"              value="$(arg scan_topic)"/>
 
    <arg name="subscribe_scan_cloud"    value="$(arg subscribe_scan_cloud)"/>
    <arg name="scan_cloud_topic"        value="$(arg scan_cloud_topic)"/>
   
    <arg name="visual_odometry"         value="$(arg visual_odometry)"/>         
    <arg name="odom_topic"              value="$(arg odom_topic)"/>       
    <arg name="odom_frame_id"           value="$(arg odom_frame_id)"/>   
    <arg name="odom_args"               value="$(arg rtabmap_args)"/>
  </include>

thanks for help!
Nikodem

[matlabbe]

Hi,

What are the default TF published when SlamDunk alone is running? It seems that there is a TF referring itself with /map frame (/map -> /map). The rtabmap launch file would add these TFs: /map -> /odom -> /base_link. Do "$ rosrun tf view_frames" to debug the TF tree.

When using stereo topics, the image should be "image_rect", not "image_raw"... unless "image_raw" on SlamDunk means already rectified. If they are not rectified, you would have to start a stereo_image_proc node to get "image_rect".

cheers,
Mathieu