rtabmap and SURF GPU

[dschnabel]

I'm running rtabmap on a Raspberry Pi 4 and it seems to be working reasonably well. But every now and then I'm hitting 100% CPU usage and things start to get a bit slow.

Now the Raspberry Pi 4 comes with a reasonably powerful GPU (VideoCore VI) and I was wondering if it would be possible to offload some of the workload of rtabmap from the CPU to the GPU?

I've come across the SURF parameters in rtabmap which can be configured to use the GPU version of SURF. I understand that I need to compile OpenCV with CUDA in order for rtabmap to be able to make use of the GPU.

Unfortunately I don't know much about SURF: Is it a feature that's only supported on NVIDIA graphics adapter?

[matlabbe]

Hi,

cuda means that a NVidia GPU is required. I am not aware if SURF has been also implemented for other GPU (like a port on OpenCL instead of Cuda).

In term of performance, it depends on the scale of the environment, but binary features like BRIEF or ORB can work well for small/medium environments (assuming always same lighting conditions).

Also, are you using the UI on the RPI? If so, go in Preferences->3D Rendering and increase decimation option to reduce the number of points to show.

cheers,
Mathieu

[dschnabel]

Thanks Mathieu, that makes sense.

When you say small/medium environments, what is considered a small or a medium environment? My environment would be a 6,000 sq feet (557 sq meter) office space.

I would like to give the binary features you mentioned a try. In order to use BRIEF/ORB with rtabmap, do these features have to be built with OpenCV?

And is this how I would configure rtabmap to use BRIEF or is there more that needs to be set?

Kp/DetectorStrategy = "6"
Vis/FeatureType = "6"

No, I'm not using the UI on the RPI. I'm using the RVIZ UI on my desktop while rtabmap is running on my RPI.

[matlabbe]

I would say medium environment. It depends on which ROS version you are, if a feature is not available, you will get a warning in terminal and another default type will be used. Yes those two parameters are fine.

If you can share a resulting database, there are plenty of statistics recorded in it that could help to see why the processing time increases as much over time.

[dschnabel]

Thanks. Do I have to compile rtabmap with ORB_SLAM2 support to make use of BRIEF/ORB? Looking at my cmake build info this is how it looks:

-- --------------------------------------------
-- Info :
--   Version : 0.19.7
--   CMAKE_INSTALL_PREFIX = /usr/local
--   CMAKE_BUILD_TYPE =     Release
--   CMAKE_INSTALL_LIBDIR = lib
--   BUILD_APP =            ON
--   BUILD_TOOLS =          ON
--   BUILD_EXAMPLES =       ON
--   BUILD_SHARED_LIBS =    ON
--   CMAKE_CXX_FLAGS =  -fmessage-length=0  -fopenmp
--   FLANN_KDTREE_MEM_OPT = OFF
--   PCL_DEFINITIONS = -DDISABLE_OPENNI2;-DDISABLE_PCAP;-DDISABLE_PNG;-DDISABLE_LIBUSB_1_0
--   PCL_VERSION = 1.9.1
-- 
-- Optional dependencies ('*' affects some default parameters) :
--  *With OpenCV 4.2.0 xfeatures2d module (SIFT/SURF/BRIEF/FREAK) = NO (not found, License: BSD)
--   With Qt5                  = YES (License: Open Source or Commercial)
--   With VTK 7.1              = YES (License: BSD)
--   With external SQLite3     = YES (License: Public Domain)
--   With ORB OcTree           = YES (License: GPLv3)
--   With SupertPoint Torch    = NO (libtorch not found)
--   With Madgwick             = YES (License: GPL)
--   With FastCV               = NO (FastCV not found)
-- 
--  Solvers:
--   With TORO                 = YES (License: Creative Commons [Attribution-NonCommercial-ShareAlike])
--  *With g2o                  = NO (g2o not found)
--  *With GTSAM                = NO (GTSAM not found)
--  *With Ceres                = NO (Ceres not found)
--   With VERTIGO              = NO (GTSAM or g2o required)
--   With cvsba                = NO (cvsba not found)
--  *With libpointmatcher      = NO (libpointmatcher not found)
-- 
--  Reconstruction Approaches:
--   With OCTOMAP              = NO (octomap not found)
--   With CPUTSDF              = NO (CPUTSDF not found)
--   With OpenChisel           = NO (open_chisel not found)
--   With AliceVision          = NO (WITH_ALICE_VISION=OFF)
-- 
--  Camera Drivers:
--   With Freenect             = NO (libfreenect not found)
--   With OpenNI2              = YES (License: Apache v2)
--   With Freenect2            = NO (libfreenect2 not found)
--   With Kinect for Windows 2 = NO (Kinect for Windows 2 SDK not found)
--   With Kinect for Azure     = NO (Kinect for Azure SDK not found)
--   With dc1394               = YES (License: LGPL)
--   With FlyCapture2/Triclops = NO (Point Grey SDK not found)
--   With ZED                  = NO (ZED sdk and/or cuda not found)
--   With RealSense            = NO (librealsense not found)
--   With RealSense2           = NO (librealsense2 not found)
--   With MyntEyeS             = NO (mynteye s sdk not found)
-- 
--  Odometry Approaches:
--   With loam_velodyne        = NO (loam_velodyne not found)
--   With libfovis             = NO (libfovis not found)
--   With libviso2             = NO (libviso2 not found)
--   With dvo_core             = NO (dvo_core not found)
--   With okvis                = NO (okvis not found)
--   With msckf_vio            = NO (WITH_MSCKF_VIO=OFF)
--   With VINS-Fusion          = NO (VINS-Fusion not found)
--   With ORB_SLAM2            = NO (ORB_SLAM2 not found, make sure environment variable ORB_SLAM2_ROOT_DIR is set)
-- Show all options with: cmake -LA | grep WITH_
-- --------------------------------------------

[matlabbe]

Hi,

ORB_SLAM2 dependency is just for visual odometry approach of ORB_SLAM2, ORB features are available in your build. For BRIEF, you need to build OpenCV with opencv_contrib, then you will have access to all features from OpenCV.

With OpenCV 4.2.0 xfeatures2d module (SIFT/SURF/BRIEF/FREAK) = NO (not found, License: BSD)

will become:

With OpenCV 4.2.0 xfeatures2d module (SIFT/SURF/BRIEF/FREAK) = YES

cheers,
Mathieu

[dschnabel]

I see, thanks matlabbe for helping me see what I'm missing.

I built OpenCV with opencv_contrib as proposed and was then able to build rtabmap with access to the xfeatures2d module.

I compared BRIEF (DetectorStrategy 6) and ORB (DetectorStrategy 8) but didn't notice any difference in performance. I might play around more with other strategies and see if there's a difference.

[matlabbe]

ORB means Oriented-BRIEF, so there should not be a lot of the difference. The only thing I could think is if you rotate (roll) the camera 90 deg, then with ORB, loop closures would be still be detected against images taken at 0 deg.

[dschnabel]

I see, that makes sense!