How to use ORB_SLAM2 library in rtabmap?

[Mosquito]

I find the OdometryORBSLAM2.cpp file and FindORB_SLAM2.cmake file　in the project.
I want to use ORBSLAM２’s lib　in the rtabmap,but when I build the project using "$cmake ..",it shows as follow:

-- checking for module 'openni-dev'
--   package 'openni-dev' not found
-- checking for module 'openni-dev'
--   package 'openni-dev' not found
-- Found OpenMP
-- Found OpenCV: /usr/include/opencv;/usr/include
-- Found PCL: /usr/include/pcl-1.7;/usr/include/eigen3;/usr/include;/usr/include/ni;/usr/include/vtk-5.8
-- Found ZLIB: /usr/include
-- Found Freenect: /usr/include
CMake Warning at cmake_modules/FindKinectSDK2.cmake:71 (message):
  Directory "" not found.
Call Stack (most recent call first):
  cmake_modules/FindKinectSDK2.cmake:119 (check_dir)
  CMakeLists.txt:270 (FIND_PACKAGE)


-- KinectSDK2_FOUND : FALSE
-- Found OpenNI2: /usr/include/openni2
-- Found DC1394: /usr/include/dc1394
-- Found g2o: /opt/ros/indigo/include;/usr/include/suitesparse;/usr/include/suitesparse
-- Found RealSense: /opt/ros/indigo/include
-- Found octomap: /usr/local/include
-- Found Pthreads
-- --------------------------------------------
-- Info :
--   Version : 0.15.0
--   CMAKE_INSTALL_PREFIX = /usr/local
--   CMAKE_BUILD_TYPE =     Debug
--   CMAKE_INSTALL_LIBDIR = lib
--   BUILD_APP =            ON
--   BUILD_TOOLS =          ON
--   BUILD_EXAMPLES =       ON
--   BUILD_SHARED_LIBS =    ON
--   CMAKE_CXX_FLAGS =  -fmessage-length=0  -Wno-deprecated -fopenmp -std=c++11
--   With OpenCV 2 nonfree module (SIFT/SURF) = NO (not found, License: BSD)
--   With Freenect             = YES (License: Apache v2 and/or GPLv2)
--   With OpenNI2              = YES (License: Apache v2)
--   With Freenect2            = NO (libfreenect2 not found)
--   With Kinect for Windows 2 = NO (Kinect for Windows 2 SDK not found)
--   With dc1394               = YES (License: LGPL)
--   With FlyCapture2/Triclops = NO (Point Grey SDK not found)
--   With TORO                 = YES (License: Creative Commons [Attribution-NonCommercial-ShareAlike])
--   With g2o                  = YES (License: BSD)
--   With GTSAM                = NO (GTSAM not found)
--   With VERTIGO              = YES (License: GPLv3)
--   With cvsba                = NO (cvsba not found)
--   With libpointmatcher      = NO (libpointmatcher not found)
--   With ZED                  = NO (ZED sdk not found)
--   With RealSense            = YES (License: Apache-2)
--   With RealSenseSlam        = NO (WITH_REALSENSE_SLAM=OFF)
--   With OCTOMAP              = YES (License: BSD)
--   With CPUTSDF              = NO (CPUTSDF not found)
--   With libfovis             = NO (libfovis not found)
--   With libviso2             = NO (libviso2 not found)
--   With dvo_core             = NO (dvo_core not found)
--   With ORB_SLAM2            = NO (WITH_G2O should be OFF as ORB_SLAM2 uses its own g2o version)
--   With Qt4                  = YES (License: Open Source or Commercial)
-- --------------------------------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /home/minipc/rtabmapDebug/build
   

   I　ｈａｖｅ　built and run the ORB_SLAM2 project in the home directory. I also tried to install the ORB_SLAM2 in "/usr/local",ｂｕｔ ｆａｉｌｅｄ....
  Can you tell me how to use the ORB_SLAM2 in rtabmap ? THANKS A LOT!!!

[matlabbe]

Hi,

As it is said, set WITH_G2O to off on cmake:

$ cmake -DWITH_G2O=OFF ..

Then to make cmake find ORB_SLAM2, look in findORB_SLAM2.cmake: you have to set an environment variable called ORB_SLAM2_ROOT_DIR that points to root directory of ORB SLAM2 source directory.

Note that ORB_SLAM2 integration is still experimental. To use it, set odometry strategy to ORB_SLAM2 (or Odom/Strategy to 5).

Cheers,
Mathieu

[Mosquito]

Thank you Mathieu for your reply!!!!!!
With your ｈｅｌｐ，I have succeed run rtabmap　with ORBSLAM2 after several attempts.
Ｉ write  my steps down and hope to help more peoples.It is as follow:
1.add "export ORB_SLAM2_ROOT_DIR=/home/minipc/ORB_SLAM2-master " in /home/minipc/ .zshrc file.
2.change 218th row in /tools/RgbdDataset/main.cpp :
change :OdometryF2M odom(parameters); to OdometryORBSLAM2 odom(parameters);
3. cmake -DWITH_G2O=OFF ..
4.make
5 ./rtabmap-rgbd_dataset --OdomORBSLAM2/VocPath "/home/minipc/ORB_SLAM2-master/Vocabulary/ORBvoc.txt" /home/minipc/ORBSLAM2_Octomap2/dataset/rgbd_dataset_freiburg1_desk

Thank you again!

[Mosquito]

Hi,Mathieu:
After I  run :
$./rtabmap-rgbd_dataset --OdomORBSLAM2/VocPath "/home/minipc/ORB_SLAM2-master/Vocabulary/ORBvoc.txt" /home/minipc/ORBSLAM2_Octomap2/dataset/rgbd_dataset_freiburg1_desk  
I get "rtabmap.db",then I run :$rtabmap.I load the rtabmap.db in the rtabmap, then i select to export the trajectory.As follows:

I get pose.txt.Finally, I run ：$python evaluate_ate.py groundtruth.txt pose.txt --plot error.png  . I get the graph as follows:
 

In the graph,the estimate poses is very sparse. In the 'rgbd_dataset_freiburg1_desk'  dataset ,It has 573 frames,but the rtabmap finally only get 20 frames' pose.
Can you tell me how to get more frame's pose? Thank you very much!

[matlabbe]

Hi,

When starting rtabmap-rgbd_dataset, you can add "--Rtabmap/DetectionRate 2" to update the map at 2 Hz instead of 1 Hz (so there will be two times more nodes). You can also set "--Rtabmap/CreateIntermediateNodes true" to add all odometry poses (in this case, the graph would be the same size than the number of input images).

cheers,
Mathieu

[Mosquito]

It works! Thank you very much !

[vkpankov]

Hi,
How to use ORB SLAM2 with RTABMAP in ROS?

[matlabbe]

Hi,

If rtabmap is built with ORB-SLAM2 like above, you can set "Odom/Strategy" parameter to 5 for the rgbd_odometry or stereo_odometry node. You will also need to set "OdomORBSLAM2/VocPath" to path of the vocabulary in ORB-SLAM2 project.

Cheers,
Mathieu

[lokes_bisht]

Hey, did you get good results after this ? How much improvement did you notice ?

[BerberechoRick]

Hi Matt I have used before this implementation but because I'm trying to test different approaches I had to uninstall and install several times as I guess different options to test.

The purpose to come again to this thread is that I try to install it as I did before but here you can see despite doing the cmake with g2o off it is still operative

and despite orbslam2 is recognised in the installation I can't choose it in the odometry

I had steps to follow with orbslam2 installation and rtabmap but as I see you've been adding and fixing, the rtabmap steps are already included in the git clone rtabmap
git clone https://github.com/raulmur/ORB_SLAM2.git ORB_SLAM2
cd ORB_SLAM2
folder src
folder tools
include/System.h
Thirdparty/DBoW2/DBoW2/TemplatedVocabulary.h
build.sh (modify the last line depending the system in which is installed)
CMakeLists.txt //comment -O3 -march native in orbslam2 and thirdparties: g2o and dbow2

set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS}  -Wall")#  -O3 -march=native ")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall")#   -O3 -march=native")

Am I missing something?

[matlabbe]

Make sure you are starting the right rtabmap ("$ which rtabmap"). The about dialog should also show same info than cmake info.

[user1245]

Hi,

I try to compile rtabmap with orbslam2 and realsense2.
-- Eigen found (include: /usr/local/include/eigen3, version: 3.3.5)
-- OpenNI found (include: /usr/include/ni, lib: /usr/lib/libOpenNI.so)
-- The imported target "vtkRenderingPythonTkWidgets" references the file
   "/usr/lib/aarch64-linux-gnu/libvtkRenderingPythonTkWidgets.so"
but this file does not exist.  Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
   "/usr/lib/cmake/vtk-6.2/VTKTargets.cmake"
but not all the files it references.

-- The imported target "vtk" references the file
   "/usr/bin/vtk"
but this file does not exist.  Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
   "/usr/lib/cmake/vtk-6.2/VTKTargets.cmake"
but not all the files it references.

-- QHULL found (include: /usr/include, lib: optimized;/usr/lib/aarch64-linux-gnu/libqhull.so;debug;/usr/lib/aarch64-linux-gnu/libqhull.so)
-- OpenNI found (include: /usr/include/ni, lib: /usr/lib/libOpenNI.so)
CMake Warning at CMakeLists.txt:212 (MESSAGE):
  PCL compile options contain "-march=native", make sure all libraries using
  Eigen are also compiled with that flag to avoid some segmentation faults
  (with gdb referring to some Eigen functions).


-- Found Sqlite3: /usr/include /usr/lib/aarch64-linux-gnu/libsqlite3.so
-- Found OpenMP
-- Found OpenCV: /opt/ros/kinetic/include/opencv-3.3.1-dev;/opt/ros/kinetic/include/opencv-3.3.1-dev/opencv
-- Found PCL: /usr/local/include/pcl-1.9;/usr/local/include/eigen3;/usr/include;/usr/include/ni;/usr/include/vtk-6.2;/usr/include/freetype2;/usr/include/aarch64-linux-gnu/freetype2;/usr/include/aarch64-linux-gnu;/usr/include/jsoncpp
-- Found ZLIB: /usr/include
-- VTK_RENDERING_BACKEND=OpenGL
-- Found Freenect: /usr/include
-- FlyCapture2_INCLUDE_DIR=FlyCapture2_INCLUDE_DIR-NOTFOUND
-- FlyCapture2_LIBRARY=FlyCapture2_LIBRARY-NOTFOUND
-- Triclops_INCLUDE_DIR=Triclops_INCLUDE_DIR-NOTFOUND
-- Triclops_LIBRARY=Triclops_LIBRARY-NOTFOUND
-- FlyCaptureBridge_LIBRARY=FlyCaptureBridge_LIBRARY-NOTFOUND
-- A library with BLAS API found.
-- Found CUDA: /usr/local/cuda-9.0 (found suitable version "9.0", minimum required is "9.0")
-- Found ZED sdk: /usr/local/zed/include
-- Found CUDA: /usr/local/cuda-9.0 (found version "9.0")
-- Found CUDA: /usr/local/cuda-9.0/include
-- Found RealSense2:
-- Found octomap 1.8.1: /opt/ros/kinetic/include
-- Architecture: aarch64
-- Found ORB_SLAM2: /home/nvidia/ORB_SLAM2/include;/home/nvidia/ORB_SLAM2/Thirdparty/g2o;/home/nvidia/ORB_SLAM2
-- Found Pangolin: /home/nvidia/Pangolin/include;/home/nvidia/Pangolin/build/src/include;/usr/include;/usr/include;/usr/include;/usr/local/include/eigen3
-- Found Pthreads
-- --------------------------------------------
-- Info :
--   Version : 0.19.6
--   CMAKE_INSTALL_PREFIX = /usr/local
--   CMAKE_BUILD_TYPE =     Release
--   CMAKE_INSTALL_LIBDIR = lib
--   BUILD_APP =            ON
--   BUILD_TOOLS =          ON
--   BUILD_EXAMPLES =       ON
--   BUILD_SHARED_LIBS =    ON
--   CMAKE_CXX_FLAGS =  -fmessage-length=0  -fopenmp -std=c++11
--   FLANN_KDTREE_MEM_OPT = OFF
--   PCL_DEFINITIONS = -DDISABLE_PCAP;-DDISABLE_PNG;-DDISABLE_LIBUSB_1_0
--   PCL_VERSION = 1.9.1.99
--   PCL_COMPILE_OPTIONS = -march=native;-ffloat-store
--
-- Optional dependencies ('*' affects some default parameters) :
--  *With OpenCV 3 xfeatures2d module (SIFT/SURF/BRIEF/FREAK) = YES (License: Non commercial)
--   With Qt5                  = YES (License: Open Source or Commercial)
--   With VTK 6.2              = YES (License: BSD)
--   With external SQLite3     = YES (License: Public Domain)
--   With ORB OcTree           = YES (License: GPLv3)
--   With Madgwick             = YES (License: GPL)
--   With FastCV               = NO (FastCV not found)
--
--  Solvers:
--   With TORO                 = YES (License: Creative Commons [Attribution-NonCommercial-ShareAlike])
--  *With g2o                  = NO (WITH_G2O=OFF)
--  *With GTSAM                = NO (GTSAM not found)
--  *With Ceres                = NO (Ceres not found)
--   With VERTIGO              = NO (GTSAM or g2o required)
--   With cvsba                = NO (cvsba not found)
--  *With libpointmatcher      = NO (libpointmatcher not found)
--
--  Reconstruction Approaches:
--   With OCTOMAP              = YES (License: BSD)
--   With CPUTSDF              = NO (CPUTSDF not found)
--   With OpenChisel           = NO (open_chisel not found)
--   With AliceVision          = NO (WITH_ALICE_VISION=OFF)
--
--  Camera Drivers:
--   With Freenect             = YES (License: Apache v2 and/or GPLv2)
--   With OpenNI2              = NO (OpenNI2 not found)
--   With Freenect2            = NO (libfreenect2 not found)
--   With Kinect for Windows 2 = NO (Kinect for Windows 2 SDK not found)
--   With Kinect for Azure     = NO (Kinect for Azure SDK not found)
--   With dc1394               = NO (dc1394 not found)
--   With FlyCapture2/Triclops = NO (Point Grey SDK not found)
--   With ZED                  = YES (With CUDA)
--   With RealSense            = NO (librealsense not found)
--   With RealSense2           = YES (License: Apache-2)
--
--  Odometry Approaches:
--   With loam_velodyne        = NO (loam_velodyne not found)
--   With libfovis             = NO (libfovis not found)
--   With libviso2             = NO (libviso2 not found)
--   With dvo_core             = NO (dvo_core not found)
--   With okvis                = NO (okvis not found)
--   With msckf_vio            = NO (WITH_MSCKF_VIO=OFF)
--   With VINS-Fusion          = NO (VINS-Fusion not found)
--   With ORB_SLAM2            = YES (License: GPLv3)
-- --------------------------------------------
-- Configuring done


[  4%] Built target rtabmap_utilite
[  4%] Built target res_tool
[ 40%] Built target rtabmap_core
[ 80%] Built target rtabmap_gui
[ 80%] Linking CXX executable ../../../bin/rtabmap
../../../bin/librtabmap_gui.so.0.19.6: undefined reference to `pcl::visualization::Camera::Camera()'
collect2: error: ld returned 1 exit status
app/src/CMakeFiles/rtabmap.dir/build.make:300: recipe for target '../bin/rtabmap' failed
make[2]: *** [../bin/rtabmap] Error 1
CMakeFiles/Makefile2:416: recipe for target 'app/src/CMakeFiles/rtabmap.dir/all' failed
make[1]: *** [app/src/CMakeFiles/rtabmap.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2

How to solve it? Thanks

[matlabbe]

../../../bin/librtabmap_gui.so.0.19.6: undefined reference to `pcl::visualization::Camera::Camera()' 

you are building against PCL 1.9 built from source. Make sure PCL is built with visualization module. I just tried against PCL version 1.10, and it compiles successfully.