RTAB-Map using realsense R200, Did not receive data since 5 seconds! Make sure the input topics are published

[mvinsens]

Hi, i'm new to RTAB-Map and ros. i follow the tutorial in how to use intel realsense R200 to do rtabmap.
the robot i use is turtlebot3 with intel realsense R200 with ubuntu 16.04 (amd64) ros kinetic.
i already run the realsense_camera R200_nodelet_rgbd.launch in the robot.
the error i get are :

[ WARN] [1550466522.264005885]: /rtabmap/rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10).
/rtabmap/rtabmap subscribed to (approx sync):
   /odom,
   /camera/rgb/image_rect_color,
   /camera/depth_registered/sw_registered/image_rect,
   /camera/rgb/camera_info,
   /scan

the display in rviz show nothing but black screen with blue green lines.

i have check the rostopic hz for each of the topic there. the results for /camera is a bit delayed like 1 - 3 no data then data received.

the launch file i use for rtabmap :

<div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"><pre style="margin: 0; line-height: 125%"><launch>
        <group ns="rtabmap">
          <node name="rtabmap" pkg="rtabmap_ros" type="rtabmap" output="screen" args="--delete_db_on_start">

          <param name="frame_id" type="string" value="base_link"/>

          <param name="subscribe_depth" type="bool" value="true"/>
          <param name="subscribe_scan" type="bool" value="true"/>

          <remap from="odom" to="/odom"/>
          <remap from="scan" to="/scan"/>

          <remap from="rgb/image" to="/camera/rgb/image_rect_color"/>
          <remap from="depth/image" to="/camera/depth_registered/sw_registered/image_rect"/>
          <remap from="rgb/camera_info" to="/camera/rgb/camera_info"/>

          <param name="queue_size" type="int" value="10"/>

          <!-- rtabmap parameters -->

          <param name="RGBD/NeighborLinkRefining"         type="string" value="true"/>
          <param name="RGBD/ProximityBySpace"                 type="string" value="true"/>
          <param name="RGBD/ProximityPathMaxNeighbors"                 type="string" value="10"/>
          <param name="RGBD/AngularUpdate"  type="string" value="0.01"/>
          <param name="RGBD/LinearUpdate"  type="string" value="0.01"/>
          <param name="RGBD/OptimizeFromGraphEnd"  type="string" value="false"/>
          <param name="Reg/Force3DoF"  type="string" value="true"/>
          <param name="Reg/Strategy"  type="string" value="1"/>
          <param name="Vis/MinInliers"  type="string" value="5"/>
          <param name="Vis/InlierDistance"  type="string" value="0.1"/>
          <param name="Rtabmap/TimeThr"                         type="string" value="700"/>
          <param name="Mem/RehearsalSimilarity"  type="string" value="0.45"/>
          <param name="Grid/FromDepth"      type="string" value="false"/>

          </node>
        </group>
</launch>
</pre></div>

[matlabbe]

Hi,

can you show output of on the robot (same computer connected to camera):

$ rostopic hz /odom /camera/rgb/image_rect_color /camera/depth_registered/sw_registered/image_rect /camera/rgb/camera_info /scan

If some frame rates are very higher than others, we should increase queue_size parameters to be able to synchronize the topics.

What kind of computer is used?

[mvinsens]

Hi, Thank you for your reply.

The output for the rostopics are

subscribed to [/odom]
subscribed to [/camera/rgb/image_rect_color]
subscribed to [/camera/depth_registered/sw_registered/image_rect]
subscribed to [/camera/rgb/camera_info]
subscribed to [/scan]
                      topic                          rate   min_delta   max_delta    std_dev    window
======================================================================================================
/odom                                               29.62   0.02839     0.03768     0.00173     55    
/camera/rgb/image_rect_color                        30.01   0.02931     0.03576     0.001581    55    
/camera/depth_registered/sw_registered/image_rect   30.33   0.03063     0.03853     0.001963    24    
/camera/rgb/camera_info                             30.01   0.03021     0.03756     0.001271    24    
/scan                                               4.984   0.2004      0.2008      0.0001659   15    

                      topic                          rate   min_delta   max_delta    std_dev    window
======================================================================================================
/odom                                               29.58   0.02839     0.03768     0.001636    84    
/camera/rgb/image_rect_color                        29.99   0.02931     0.03584     0.001348    84    
/camera/depth_registered/sw_registered/image_rect   30.08   0.03063     0.03853     0.001631    54    
/camera/rgb/camera_info                             30.0    0.03021     0.03756     0.001219    54    
/scan                                               4.983   0.2004      0.2009      0.0001531   45    

                      topic                          rate   min_delta   max_delta    std_dev    window
======================================================================================================
/odom                                               29.56   0.02839     0.0383      0.001584    114  
/camera/rgb/image_rect_color                        30.01   0.02931     0.03797     0.001384    114  
/camera/depth_registered/sw_registered/image_rect   30.06   0.03017     0.03853     0.001637    84    
/camera/rgb/camera_info                             29.99   0.029       0.03756     0.001286    84    
/scan                                               4.984   0.2004      0.2009      0.0001477   75    

                      topic                          rate   min_delta   max_delta    std_dev    window
======================================================================================================
/odom                                               29.6    0.02839     0.0383      0.001557    144  
/camera/rgb/image_rect_color                        30.01   0.02931     0.03797     0.001473    144  
/camera/depth_registered/sw_registered/image_rect   30.03   0.03017     0.03853     0.001637    114  
/camera/rgb/camera_info                             30.0    0.0286      0.03893     0.001355    114  
/scan                                               4.984   0.2003      0.201       0.0001664   105  

                      topic                          rate   min_delta   max_delta    std_dev    window
======================================================================================================
/odom                                               29.61   0.02839     0.04026     0.001674    173  
/camera/rgb/image_rect_color                        30.01   0.02931     0.03797     0.001469    173  
/camera/depth_registered/sw_registered/image_rect   30.02   0.03014     0.03853     0.001649    144  
/camera/rgb/camera_info                             29.97   0.02789     0.03925     0.001536    144  
/scan                                               4.984   0.1999      0.2015      0.0002738   135  

                      topic                          rate   min_delta   max_delta    std_dev    window
======================================================================================================
/odom                                               29.61   0.02839     0.04026     0.001626    203  
/camera/rgb/image_rect_color                        30.01   0.02931     0.03797     0.001433    203  
/camera/depth_registered/sw_registered/image_rect   30.02   0.02927     0.03853     0.001702    174  
/camera/rgb/camera_info                             30.0    0.02789     0.03925     0.001575    174  
/scan                                               4.983   0.1999      0.2015      0.0002635   165  

this is the results after waiting for the realsense nodelet running few minutes before doing rostopic. when it just initialized it only show the rate of 0.0x rates for the image rect color and image rect (depth).

The computer that is used ( connected to camera ):
turtlebot3 waffle :
Intel Joule 570x

Master Computer for rviz:
MSI GL62M 7REX ( Intel core i7-7700HQ, GeForce Nvidia 1050Ti 4GB, RAM 16GB )

it seems the RTAB-Map is working but need some time to be launched after initializing realsense nodelet.
but the rate of data flow for the visualization in rviz is so slow.
and i get this error when doing rtabmap *note : i change the queue_size from 10 to 1000 while doing so still need a few second to minutes for the images to be viewed in rviz.


[ INFO] [1551067314.963492688]: rtabmap (3): Rate=1.00s, Limit=0.700s, RTAB-Map=0.0001s, Maps update=0.0000s pub=0.0000s (local map=1, WM=1)
[ WARN] [1551067315.163193462]: Could not get transform from odom to camera_link after 0,200000 seconds (for stamp=1551067273,839544)! Error="Lookup would require extrapolation into the past.  Requested time 1551067273.839544379 but the earliest data is at time 1551067305.225028585, when looking up transform from frame [camera_link] to frame [odom]. canTransform returned after 0.200384 timeout was 0.2.".
[ WARN] (2019-02-25 11:02:29.302) OdometryF2M.cpp:469::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=23) between -1 and 12"
[ INFO] [1551067349.303470016]: Odom: quality=0, std dev=0.000000m|0.000000rad, update time=0.077790s
[ERROR] (2019-02-25 11:02:29.305) Rtabmap.cpp:1003::process() RGB-D SLAM mode is enabled, memory is incremental but no odometry is provided. Image 70149 is ignored!
[ INFO] [1551067349.305078184]: rtabmap (3): Rate=1.00s, Limit=0.700s, RTAB-Map=0.0000s, Maps update=0.0000s pub=0.0000s (local map=1, WM=1)
[ WARN] [1551067349.504399521]: Could not get transform from odom to camera_link after 0,200000 seconds (for stamp=1551067292,569211)! Error="Lookup would require extrapolation into the past.  Requested time 1551067292.569210613 but the earliest data is at time 1551067339.575029953, when looking up transform from frame [camera_link] to frame [odom]. canTransform returned after 0.20039 timeout was 0.2.".
[ WARN] (2019-02-25 11:02:49.072) OdometryF2M.cpp:469::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=10) between -1 and 13"
[ INFO] [1551067369.073457447]: Odom: quality=0, std dev=0.000000m|0.000000rad, update time=0.030538s
[ERROR] (2019-02-25 11:02:49.074) Rtabmap.cpp:1003::process() RGB-D SLAM mode is enabled, memory is incremental but no odometry is provided. Image 70578 is ignored!
[ INFO] [1551067369.074989658]: rtabmap (3): Rate=1.00s, Limit=0.700s, RTAB-Map=0.0000s, Maps update=0.0000s pub=0.0000s (local map=1, WM=1)
[ WARN] (2019-02-25 11:03:13.007) OdometryF2M.cpp:469::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=11) between -1 and 14"
[ INFO] [1551067393.008607516]: Odom: quality=0, std dev=0.000000m|0.000000rad, update time=0.058463s
[ WARN] [1551067393.210269574]: Could not get transform from odom to camera_link after 0,200000 seconds (for stamp=1551067305,500518)! Error="Lookup would require extrapolation into the past.  Requested time 1551067305.500518245 but the earliest data is at time 1551067383.275089941, when looking up transform from frame [camera_link] to frame [odom]. canTransform returned after 0.201682 timeout was 0.2.".


** Updates : i tried to do the rtab map using LAN cable and it works better. Any suggestion so i can do the mapping without using cable for connection ?

[matlabbe]

Hi,

In the rtabmap warnings, odometry is lost:
[ INFO] [1551067349.303470016]: Odom: quality=0

Be aware that when you launch RVIZ or rtabmap on remote computer, the raw image topics are subscribed by default, which uses a lot of bandwidth (and may cause a lot of lag on the remote computer). Is rtabmap running on PC on turtlebot?

I would also suggest to use odometry from turtlebot instead of visual odometry, to avoid odometry lost. Example launching rtabmap on the robot:

roslaunch rtabmap_ros rtabmap.launch \
   rtabmap_args:="--delete_db_on_start --Reg/Force3DoF true --Reg/Strategy 1" \
   frame_id:=base_footprint \
   rgbd_sync:=true \
   approx_rgbd_sync:=false \
   subscribe_rgbd:=true \
   subscribe_scan:=true \
   scan_topic:=/scan \
   odom_frame_id:=odom \
   odom_tf_linear_variance:=0.001 \
   odom_tf_angular_variance:=0.001 \
   visual_odometry:=false \
   rtabmapviz:=false \
   depth_topic:=/camera/depth_registered/sw_registered/image_rect

On tutrtlebot3, there is a lidar publishing at lower rate than images, so we have to pre-sync the image topics before rtabmap node (rgbd_sync:=true). Because of the lidar, I added those extra parameters: "--Reg/Force3DoF true --Reg/Strategy 1". We have to set fixed odometry variance as the turtlebot seems to publish empty ones.

On remote, computer, don't subscribe directly to image topics, or at least in rviz set compressed/theora encoding.

cheers,
Mathieu

[mvinsens]

Hi, Thanks for the reply.
rtabmap is running on the turtlebot by using roslaunch from master computer.
the problem now seems lies in the connection used by the robot and master computer.
when using LAN cable as connection the image refresh rate at the rviz seems good, but when using wifi connection the refresh rate worsen and i need to change the queue size to bigger size for rtabmap to be able to show image at rviz. for the rviz set compressed / theora encoding to tell the truth i still dont know it well, but for the info i will try those things later.

[matlabbe]

If you subscribe to an image in RVIZ, by default the raw image is streamed on the network. Look at your computer used bandwidth (System Monitor) before and after you launch rviz. Compression can be selected in the Transport Hint combo box:


Make sure you installed ros-kinetic-image-transport-plugins to used them in rviz.

[mvinsens]

Thanks for the reply.
Noted for that info, i think i will try to do the mapping by using handheld mapping and see if the robot can do localization and autonomous navigation from these map database. since using the robot for doing mapping give out plenty of lost odometry.

[matlabbe]

If you want to maximize the chance of localization afterwards, the camera should move at the same height and orientation than if it was on the robot.