rtab-map and realsense ZR300

[vlad]

Hi!

Does standalone version of rtab-mab support realsense ZR300 camera? If yes, does it use onboard IMU?

Also, can you compare intel SLAM library and rtab-map?

[matlabbe]

Hi,

Yes, the standalone version supports the ZR300. For the IMU, it is only supported on Ubuntu as the realsense slam library (librealsense-slam-dev is only available as binary on Ubuntu, see Installation instructions here) is required to get visual inertial odometry working. The option will be enabled if rtabmap is built with it:



I don't have comparison of the maps created from realsense slam package and rtabmap.

cheers,
Mathieu

[vlad]

Hi Mathieu!

I am testing ZR300 on UP Board with Ubuntu 16.04. Rtabmap is standalone (no ROS).
And I have a performance issue when I use toggle "Use visual inertial odometryfor ZR300". When I use this toggle the speed is incredibly slow (about 1 frame per 5 seconds). When I don't use it, the performance is great and real time.

What it could be??

Vlad

[matlabbe]

Hi Vlad,

Can you set Logger level in Debug mode (Preferences->Logger->Level), then copy/paste the log here when VIO is enabled? If you have a desktop computer to test with, do you see the same problem?

cheers,
Mathieu

[vlad]

Hi Mathieu!

I am sending the log file.

LogRtabmap.txt

Also, my cmake configuration:

-- --------------------------------------------
-- Info :
--   Version : 0.13.2
--   CMAKE_INSTALL_PREFIX = /usr/local
--   CMAKE_BUILD_TYPE =     Release
--   CMAKE_INSTALL_LIBDIR = lib
--   BUILD_APP =            ON
--   BUILD_TOOLS =          ON
--   BUILD_EXAMPLES =       ON
--   BUILD_SHARED_LIBS =    ON
--   CMAKE_CXX_FLAGS =  -fmessage-length=0  -fopenmp -std=c++11
--   With OpenCV 3 xfeatures2d module (SIFT/SURF/BRIEF/FREAK) = YES (License: Non commercial)
--   With Freenect             = NO (libfreenect not found)
--   With OpenNI2              = NO (OpenNI2 not found)
--   With Freenect2            = NO (libfreenect2 not found)
--   With dc1394               = YES (License: LGPL)
--   With FlyCapture2/Triclops = NO (Point Grey SDK not found)
--   With TORO                 = YES (License: Creative Commons [Attribution-NonCommercial-ShareAlike])
--   With g2o                  = YES (License: BSD)
--   With GTSAM                = NO (GTSAM not found)
--   With VERTIGO              = YES (License: GPLv3)
--   With cvsba                = NO (cvsba not found)
--   With ZED                  = NO (ZED sdk not found)
--   With RealSense            = YES (License: Apache-2)
--   With RealSenseSlam        = YES
--   With OCTOMAP              = NO (octomap not found)
--   With CPUTSDF              = NO (CPUTSDF not found)
--   With libfovis             = NO (libfovis not found)
--   With libviso2             = NO (libviso2 not found)
--   With dvo_core             = NO (dvo_core not found)
--   With ORB_SLAM2            = NO (WITH_G2O should be OFF as ORB_SLAM2 uses its own g2o version)
--   With Qt5                  = YES (License: Open Source or Commercial)
-- --------------------------------------------

Also, example from Intel which is using IMU is running fast

Vlad

[matlabbe]

Hi Vlad,

It is strange, the camera seems published at 60 Hz (delay to capture image is around 16 ms):

...
[DEBUG] (2017-08-26 21:10:24.520) RtabmapThread.cpp:330::handleEvent() CameraEvent
[DEBUG] (2017-08-26 21:10:24.536) Camera.cpp:100::takeImage() Time capturing image = 0.016255s
[DEBUG] (2017-08-26 21:10:24.537) CameraThread.cpp:129::mainLoop() 
[DEBUG] (2017-08-26 21:10:24.537) RtabmapThread.cpp:330::handleEvent() CameraEvent
[DEBUG] (2017-08-26 21:10:24.538) MainWindow.cpp:897::processOdometry() 
[DEBUG] (2017-08-26 21:10:24.538) MainWindow.cpp:940::processOdometry() odom ok
[DEBUG] (2017-08-26 21:10:24.553) Camera.cpp:100::takeImage() Time capturing image = 0.016638s
...

The map seems correctly updated at 1 Hz (default) too:

[DEBUG] (2017-08-26 21:10:07.982) Rtabmap.cpp:847::process() 
...
[DEBUG] (2017-08-26 21:10:08.988) Rtabmap.cpp:847::process() 
...
[DEBUG] (2017-08-26 21:10:10.001) Rtabmap.cpp:847::process()

Maybe it is the UI that cannot process all odometry events at 60 Hz, causing UI lags. Close the Odometry view and in Preferences->3D Rendering, uncheck "Show 3D clouds" under Odometry column.

Otherwise, you may do a screencapture, upload the video on youtube as unlisted, then post the link here to better see the lag.

cheers,
Mathieu

[Timofejev]

Hi Mathieu,
I am now facing the same issues as the OP was, so I thought I will go ahead and record a video and attach a log of what I am seeing.
https://youtu.be/z0DgxNpWUL8

rtabmap_zr300_log.txt

What is perhaps hard to tell from the video is that I just move the camera up and down with a constant speed, without any stops and sudden jumps. The rotational component of movement translates perfectly in real time from the real camera to the visualization in RTAB-Map, however the translation lags behind and is very inconsistent. Looks like the IMU data is coming in at a way higher frequency than the imagery and the IMU data is just occupying the buffer/queue of the processing pipeline?

> Maybe it is the UI that cannot process all odometry events at 60 Hz, causing UI lags. Close the Odometry view and in Preferences->3D Rendering, uncheck "Show 3D clouds" under Odometry column.

this sadly didn't help, although I can also provide logs and video with these settings if that would help

[matlabbe]

Hi,

Is the fisheye camera of the realsense hidden or the black cap is on it? Without the fisheye, the VIO approach will just output rotations.

cheers,
Mathieu

[Timofejev]

Hi Mathieu,
bingo, the black cap was covering the fisheye-lens. Now the odometry works properly, thanks!. That's kinda embarassing now! However, I have noticed, that the odometry is not being recorded into the datatabase when I use the data recorder tool in the standalone rtabmap. Is this not implemented or is there a setting I am missing?

[matlabbe]

Hi,

It was not implemented, it is now. Note that only odometry coming from camera driver will be recorded (like VIO from ZR300), odometry computed by rtabmap won't be recorded.

cheers,
Mathieu