RTABMap for Android

[You Li]

Hi Mathieu,

Thanks for your excellent project on Tango. I was also trying to build it on Tango, following your instruction https://github.com/introlab/rtabmap/issues/74

I cross-compiled boost 1.53 and openCV 2.4.11 separately, then compiled Eigen, FLANN, and PCL by using https://github.com/bashbug/pcl-for-android. It is strange I could not cross-compile the dependencies by your method, my system is Ubuntu 16.04 LTS. When I was trying to cross-compile the RTBMAP, some issues occurred. These issues are not easy to track, and I suspect it is due to the version of my dependencies.

Could you share the version of your tools and dependencies, such as CMake, boost, Eigen, FLANN, PCL, and openCV (Do you use opencv 2 or 3)? I guess it is not neccessary to install GTSAM and g2o at the first stage, if I do not use the post-processing function?

Thanks!
You Li

[matlabbe]

Hi,

Yes, you don't need GTSAM or g2o at first (TORO will be used instead). The released tango version uses GTSAM as it gives slightly better map optimization than TORO. g2o is required only for post-processing Bundle Adjustment (the code should build but the option in Tango will do nothing).

Well, I'm building on Mac OS X, following information from https://github.com/introlab/rtabmap/issues/74. I used some git sources directly (I didn't update them for a while, though it should be ok with latest from source), here is the git version of the latest commit used:

GTSAM : develop/31eec5317cd0aed7f1cb9fd2a1e5247a9bd16976
VTK: master/01a0d34db398040d7cef13b9f2dbec0012d97167
PCL: 1.7.2 
eigen 3.2.7
flann 1.8.4
g2o: master/41b0be79a0821c1466f694dc16a9ee2c5ee92a9b
boost 1_59_0
CMake 3.4.2

OpenCV2: I use the version (OpenCV-2.4.8.2-Tegra-sdk) coming with https://developer.nvidia.com/codeworks-android. However, I remember I also needed to cross-compile OpenCV (latest branch 2.4) anyway to overwrite the liblibpng.a used in nvidia android release for security bug when releasing on play store (don't know if it is fixed in the latest released nvidia android sdk).

cheers,
Mathieu

[You Li]

Hi Mathieu,

Thanks a lot for your reply! I just realized that the way I previously did on cross-compile was not correct, which led an error at the end of installing RTABMap. Finally I found that on Linux you need to set a separate path for all cross-compiled Android libraries (so they are totally separate from the ones on LInux). I think this is what you did on the ~/android_install.

Thus, I suggest the guys who are not familiar with cross-compile (like myself) to first get knowledge of cross-compile. The link http://gw105.iu.xsede.org/genapp/wiki/gsoc2015_pp provides a good reference for me on how to build an environment for Android, using boost as an example. When the environment has been built, the cross-compile of dependencies become much easier for me. I am still working on cross-compiling of dependencies.

[You Li]

Hi Mathieu,

How did you use the tango support and client APIs in the project?
I am currently able to make the RTAB-Map to 100%, but meet the following issue:
 


what I did was download the Tangp_Support_API and Tango_Client_API, and set the path of Tango_INCLUDE_DIR, tango_LIBRARY, and Tango_support_LIBRARY. Also, downloaded the jave libraries (.class) and put them in ~/rtabmap/app/android/src

Thanks.

[You Li]

Error solved. I need to set -DTango_LIBRARY and -DTango_support_LIBRARY to the specific .so files

[You Li]

HI Mathieu,

It seems that the Tango library is missing in the project? In the final compile step, there is an error information like this



Mu understanding is that should the project compile the Tango library based on the Tango libraries and header files we used? Or we need to separately download a library and import it?

Cheers,
You Li

[matlabbe]

Hi,

you need the Tango Java library too. To be detected when Java is compiled, put the library here: "rtabmap/build/app/android/libs/TangoSDK_Yildun_Java.jar"

cheers

[You Li]

Thanks Mathieu, that works.

After installing RTAB-Map on Lenovo Tango phone, and run the app, I got a information "Unfortunately, RTAB-Map has stopped". When plugged the phone with laptop, the log file of the error are as follows:

01-11 11:12:39.186  11246-11246/? E/AndroidRuntime﹕ FATAL EXCEPTION: main
    Process: com.introlab.rtabmap, PID: 11246
    java.lang.UnsatisfiedLinkError: dlopen failed: library "libpcl_common.so" not found
            at java.lang.Runtime.loadLibrary(Runtime.java:372)

Based on your experience, what is the possible reason? The file "libpcl_common.so" is actually cross-compiled and located at the library path for PCL.
I am currently using PCL 1.8.0, maybe I will first try switching to 1.7.2.

Thanks.

[You Li]

Hi Mathieu,

The problem was probably caused by the incorrect cross-compile of PCL. I have changed to PCL 1.7.2, and make sure that the cross-compile of dependencies are all right.

Now when starting the RTAB-Map, the error information becomes:
01-12 09:09:37.045  22377-22377/? E/AndroidRuntime﹕ FATAL EXCEPTION: main
    Process: com.introlab.rtabmap, PID: 22377
    java.lang.UnsatisfiedLinkError: dlopen failed: library "libtango_support_api.so" not found
            at java.lang.Runtime.loadLibrary(Runtime.java:372)
            at java.lang.System.loadLibrary(System.java:1076)
            at com.introlab.rtabmap.RTABMapLib.<clinit>(RTABMapLib.java:20)
            at com.introlab.rtabmap.RTABMapActivity.onCreate(RTABMapActivity.java:199)
            at android.app.Activity.performCreate(Activity.java:6285)
            at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1108)
            at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2370)
            at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2477)
            at android.app.ActivityThread.access$900(ActivityThread.java:150)
            at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1344)
            at android.os.Handler.dispatchMessage(Handler.java:102)
            at android.os.Looper.loop(Looper.java:148)
            at android.app.ActivityThread.main(ActivityThread.java:5418)
            at java.lang.reflect.Method.invoke(Native Method)
            at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
 
From the information, it seems that the error was caused by the line " System.loadLibrary("NativeRTABMap")".

In the cross-compile process, I have set -DTango_INCLUDE_DIR=..../path for ,h files -DTango_support_LIBRARY=...../libtango_support_library.so -DTango_LIBRARY=....../libtango_client_api.so.

Is there any instruction on how to use the tango library files, to solve this problem?

Thanks,
You Li

[matlabbe]

If you used shared libraries, you should manually copy them in the build directory where the app is packaged. Here is the content of the libs directory in build:

$ cd build/app/android/libs

$ ls
TangoSDK_Yildun_Java.jar	armeabi-v7a

$ cd armeabi-v7a/

$ ls
libNativeRTABMap.so	libtango_support_api.so

I am using only static libraries for all cross-compiled dependencies (see "-DBUILD_SHARED_LIBS=OFF" parameter on cmake here), beside libtango_support_api.so. It is why it is manually copied here. Normally on the packaging, all files in libs will be in the app.

cheers,
Mathieu

[You Li]

Thanks Mathieu, copying the .so files into the path you given solved the problem.

In my case, I needed to copy the libopencv_...so files and the libg2o_....so files as well, to assure the software can run.

[You Li]

Hi Mathieu, thanks for your excellent software and the instructions! Now I can run the app on Tango. Here is the process I used when compiling. My OS is Ubuntu 16.04 LTS, the android-ndk-r11c is used. Sure, I am sure this is not the best way to compile, as I am also new to cross-compiling, it is really not easy to complete the whole process. Just for a reference for the guys who is doing this.

############################ Cross-compile environment ############################
http://gw105.iu.xsede.org/genapp/wiki/gsoc2015_pp
export ANDROID_SDK=/home/app16/Android/Sdk && \
export PATH=$PATH:$ANDROID_SDK/tools && \
export ANDROID_NDK=/home/app16/Android/Sdk/android-ndk-r11c && \
export PLATFORM=android-17 && \
export INSTALL_DIR=/opt/android-toolchain

$ANDROID_NDK/build/tools/make-standalone-toolchain.sh --stl=gnustl --arch=arm --platform=$PLATFORM --abis=armeabi-v7a --install-dir=$INSTALL_DIR


##### ============================
export ANDROID_STANDALONE_TOOLCHAIN=$INSTALL_DIR && \
export PATH=$ANDROID_STANDALONE_TOOLCHAIN/bin:$PATH && \
export PREFIX=/opt/android-toolchain/arm-linux-androideabi &&\
sudo chown app16:app16  $PREFIX

############################ Boost ############################
http://gw105.iu.xsede.org/genapp/wiki/gsoc2015_pp

//I used Boost 1.59
./bootstrap.sh

./b2 link=static runtime-link=static threading=multi threadapi=pthread \
  target-os=linux --stagedir=android --build-dir=android \
  stage

./b2 link=static runtime-link=static threading=multi threadapi=pthread \
  target-os=linux --stagedir=android --build-dir=android \
  --prefix=/opt/android-toolchain/arm-linux-androideabi  install

(1.59 did not generate libboost_iostreams.a (essential for PCL) on my machine, so I copied libboost_iostreams.a from the other version (1.62))

############################  FLANN  ############################
################ FLANN 1.8.4
/home/app16/Downloads/cmake-3.6.3-Linux-x86_64/bin/cmake ..  \
-DCMAKE_INSTALL_PREFIX=/opt/android-toolchain/arm-linux-androideabi \
-DCMAKE_TOOLCHAIN_FILE=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/cmake_modules/android.toolchain.cmake \
-DBUILD_SHARED_LIBS=OFF \
-DBUILD_TOOLS=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_EXAMPLES=OFF \
-DBUILD_PYTHON_BINDINGS=OFF \
-DBUILD_MATLAB_BINDINGS=OFF


############################ Eigen ############################
/home/app16/Downloads/cmake-3.6.3-Linux-x86_64/bin/cmake ..  \
-DCMAKE_INSTALL_PREFIX=/opt/android-toolchain/arm-linux-androideabi \
-DCMAKE_TOOLCHAIN_FILE=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/cmake_modules/android.toolchain.cmake \
-DBUILD_SHARED_LIBS=OFF \
-DBUILD_TOOLS=OFF \
-DCMAKE_BUILD_TYPE=Release


############################ PCL ############################
// pcl-pcl-1.7.2
Reference (PCL is not easy to install, I used the detailed commands below)
https://larrylisky.com/2014/03/03/installing-pcl-on-ubuntu/

Detailed commands:
sudo apt-get update
sudo apt-get install g++
sudo apt-get install doxygen
sudo apt-get install mpi-default-dev openmpi-bin openmpi-common
sudo apt-get install libvtk5.10 libvtk5.10-qt4 libvtk5-dev
sudo apt-get install libqhull*
sudo apt-get install libusb-dev
sudo apt-get install libgtest-dev
sudo apt-get install git-core freeglut3-dev pkg-config
sudo apt-get install build-essential libxmu-dev libxi-dev
sudo apt-get install libusb-1.0-0-dev graphviz mono-complete
sudo apt-get install qt-sdk openjdk-8-jdk openjdk-8-jre


/home/app16/Downloads/cmake-3.6.3-Linux-x86_64/bin/cmake .. -DCMAKE_INSTALL_PREFIX=/opt/android-toolchain/arm-linux-androideabi \
-DCMAKE_TOOLCHAIN_FILE=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/cmake_modules/android.toolchain.cmake \
-DBUILD_SHARED_LIBS=OFF -DBUILD_TOOLS=OFF -DCMAKE_BUILD_TYPE=Release \
-DWITH_QHULL:BOOL=OFF     -DWITH_PCAP:BOOL=OFF     -DWITH_PNG:BOOL=OFF \
-DWITH_OPENGL:BOOL=OFF     -DWITH_LIBUSB:BOOL=OFF -DWITH_OPENNI:BOOL=OFF \
-DWITH_OPENNI2:BOOL=OFF -DBUILD_visualization:BOOL=OFF -DBUILD_examples:BOOL=OFF \
-DBoost_INCLUDE_DIR=/opt/android-toolchain/arm-linux-androideabi/include \
-DBoost_LIBRARY_DIR_DEBUG=/opt/android-toolchain/arm-linux-androideabi/lib  \
-DBoost_LIBRARY_DIR_RELEASE=/opt/android-toolchain/arm-linux-androideabi/lib  \
-DHAVE_POSIX_MEMALIGN_EXITCODE=OFF \
-DOPENGL_gl_LIBRARY=/usr/lib/x86_64-linux-gnu \
-DOPENGL_INCLUDE_DIR=/usr/include/x86_64-linux-gnu \
-DEIGEN_INCLUDE_DIR=/opt/android-toolchain/arm-linux-androideabi/include/eigen3 \
-DFLANN_INCLUDE_DIR=/opt/android-toolchain/arm-linux-androideabi/include/flann \
-DFLANN_LIBRARY:FILEPATH=/opt/android-toolchain/arm-linux-androideabi/lib/libflann_cpp_s.a

(The opengl is not used, but it still need a path for the cmake)

############################  Open CV  ############################
git clone https://github.com/Itseez/opencv.git

export ANDROID_SDK=/home/app16/Android/Sdk


/home/app16/Downloads/cmake-3.6.3-Linux-x86_64/bin/cmake ..  \
-DCMAKE_INSTALL_PREFIX=/opt/android-toolchain/arm-linux-androideabi  \
-DCMAKE_TOOLCHAIN_FILE=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/cmake_modules/android.toolchain.cmake \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_TOOLS=OFF \
-DCMAKE_BUILD_TYPE=Release


############################  RTABMAP  ############################
################ G2O
Download
https://github.com/RainerKuemmerle/g2o
/home/app16/Downloads/cmake-3.6.3-Linux-x86_64/bin/cmake .. \
-DCMAKE_TOOLCHAIN_FILE=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/cmake_modules/android.toolchain.cmake \
-DBUILD_SHARED_LIBS=OFF -DBUILD_TOOLS=OFF -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/android-toolchain/arm-linux-androideabi \
-DANDROID_ABI="armeabi-v7a with NEON" \
-DEIGEN3_INCLUDE_DIR=/opt/android-toolchain/arm-linux-androideabi/include/eigen3 -DEIGEN3_VERSION_OK=ON



#### RTAB-MAP
export ANDROID_NDK=/home/app16/Android/Sdk/android-ndk-r11c
export ANDROID_NATIVE_API_LEVEL=android-17

export ANDROID_HOME=/home/app16/Android/Sdk && \
export PATH=$ANDROID_HOME/platform-tools:$PATH && \
export PATH=$ANDROID_HOME/tools:$PATH
sudo apt-get install ant


/home/app16/Downloads/cmake-3.6.3-Linux-x86_64/bin/cmake .. -DCMAKE_TOOLCHAIN_FILE=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/cmake_modules/android.toolchain.cmake \
-DBUILD_SHARED_LIBS=OFF -DBUILD_TOOLS=OFF -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/android-toolchain/arm-linux-androideabi \
-DOpenCV_DIR=/opt/android-toolchain/arm-linux-androideabi/sdk/native/jni \
-DRTABMAP_RES_TOOL=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/bin \
-DTango_INCLUDE_DIR=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/app/android/libs  \
-DTango_LIBRARY=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/app/android/libs/libtango_client_api.so \
-DTango_support_LIBRARY=/opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/app/android/libs/libtango_support_api.so

############################ Issues in RTABMAP ############################
1. Need to revise the rtabmap/build/corelib/src/CMakeFiles/rtabmap_core.dir/build.make when compiling. Change line 62 "cd /opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/build/corelib/src &&  ../../../bin -n rtabmap -p" to "cd /opt/android-toolchain/arm-linux-androideabi/android_install/rtabmap/build/corelib/src &&  cd ../../../bin -n rtabmap -p"

2. Need to manually copy the "DatabaseSchema_sql.h" file to rtabmap/corelib/src/

[You Li]

The problem 01-11 11:12:39.186  11246-11246/? E/AndroidRuntime﹕ FATAL EXCEPTION: main
    Process: com.introlab.rtabmap, PID: 11246
    java.lang.UnsatisfiedLinkError: dlopen failed: library "libpcl_common.so" not found
            at java.lang.Runtime.loadLibrary(Runtime.java:372) might not be the version of PCL. I realized that it is because after compliling my PCL, there are both .a files and .o files. The existing of .o files caused the error. When I deleted the .o files and simply left the .a ones, the error disappeared. Another solution may be copy the .so files into the libs folder.

[matlabbe]

Hi,

On my side, I've built dependencies as STATIC libraries (*.a). Just libtango_support_api.so needs to be manually copied in "build/app/android/libs/armeabi-v7a" directory (along with the generated RTABMapLib.so).

cheers

[Minxi]

Thanks! Has the issue
https://github.com/introlab/rtabmap/issues/82
been solved?

I met the same issue when using Tango. The software will stop working when the db reached about 130 Mb.
Thanks for providing the debug mode. That is really useful!
Thanks.

[You Li]

Use data recorder mode can mitigate the problem and collect more data.

[lincolnwan]

hallo You,

I was following your script, many thanks for this! I was having trouble locating the file "DatabaseSchema_sql.h", where did you find it? Pls help..

[You Li]

Thanks Lincoln,
I download a new rtabmap package and yes there seems no such file. Maybe it is a files generated during the compiling process? Here is the file on my computer. DatabaseSchema_sql.h 
Hope it will work. I mentioned that file just because it was shown this file is needed, and I found this file on my computer by locating it.

Best regards,
You

[matlabbe]

Hi,

this file is generated during the build. See CMake here. rtabmap (normal desktop version) should be built on the host computer to have access to rtabmap-res_tool.

Normally, you should have seen this CMake error telling the same thing:

CMake Error at corelib/src/CMakeLists.txt:301 (MESSAGE):
  RTABMAP_RES_TOOL is not defined (it is the path to "rtabmap-res_tool"
  application created by a non-Android build).

So you can do:

$ cd ~/rtabmap-android/build
$ cmake -DRTABMAP_RES_TOOL=~/rtabmap-desktop/bin/rtabmap-res_tool ..

cheers,
Mathieu

[lincolnwan]

hi Mathieu,

we don't have a working compilation configuration after the commit 3a7341 on Jan 26th, as it includes vtk_smoothing in the pcl_surface module. Problem is that PCL won't include vtk_smoothing in the surface module when it detects it's Android build.

But you seem to be able to compile for Android with no problem. Can you share with us your configuration/script? I'm quite sure we are not the only ones having this issue here.

Best-
Lin