RTAB-Map with kinect v2

[vasemax]

Thank you for this awesome software! RTAB-Map changed the way I work. I'm a building engineer and academic researcher and I used phab2pro on many projects for indoor building scanning. The procedure is a bit slow but the accuracy is better than expected. The only drawback I found is some random crashes where the program closes unexpectedly. I'm monitoring the ram and it's not from that. I can't reproduce the crashes so I can't give you more feedback.

Anyway, my question is about kinect v2. I wanted to upgrade the sensor to be more productive on indoor scanning, so I got a kinect v2 sensor. I already tried with my laptop but the processor (i3) was inadequate. I only got 7fps. I will get a very powerful laptop soon, so that problem should be solved. However, results from the 7fps scan weren't very promising by having a lot of offsets on pointclouds. Do I need any preparation (or calibration) to get good results? RTAB-Map with phab2pro uses only the depth sensor or uses the accelerometer and orientation sensors as well for odometry? Can I expect better results with kinect v2 on a powerful laptop? Kinect v2 has a broad field of view so it should be able to get better visual odometry. Finally, I can only make it work via the freenect2. The kinect for Windows SDK v2 driver is deactivated but I already installed it.

[matlabbe]

Hi,

Currently, Google Tango pose estimation is one of the best visual odometry approach out there. You won't get as accurate motion estimation with RTAB-Map alone and a Kinect v2. Tango odometry is a visual-inertial odometry (VIO) approach and uses a large field-of-view camera (fisheye 180 deg, quite larger than Kinectv2) combined with IMU. So yes in this configuration RTAB-Map uses IMU indirectly from VIO approach of Tango, also depth is not actually used for odometry on RTABMap-Tango, only for creating the point clouds.

Crashes are unfortunate, if you are able to reproduce the bug with a repeatable sequence, open an issue on the github repo. Tell which version you are using (see About menu option). As the version on PlayStore cannot be updated anymore (the tango phones cannot be updated up to now required android API > 25 from PlayStore), you can also try the latest RTAB-Map Tango APK version that can be only installed from docker instructions here. Maybe it will crash less often.

7fps will affect the odometry, if you can get over 15 fps (best is 30 Hz), you will already see a significant improvement on odometry accuracy. On low FPS, the registration between RGB and depth image can be not very accurate (in particular on rotation), increasing odometry errors. To make Windows SDK v2 driver work, we should build RTAB-Map from source. However, on Windows it is not easy to rebuild RTAB-Map, I'll suggest to stick with libfreenect2 (results are similar).

cheers,
Mathieu