Red Screen

[navid]

Hi,

I have successfully run Binary version of RTAB-MAP but when I try the source version,the Odometry is fully  red. I have also checked Lost Odometry (RED screens!) section on the Kinect mapping wiki page but didn’t work.
I am using Kinect V2 with libfreenect2 on Ubuntu.

and also how can I use kinect v2 with RTAB-MAP libraries in c++ in this tutorial:  https://github.com/introlab/rtabmap/wiki/Cplusplus-RGBD-Mapping 

Any Idea?
Thanks you  

[matlabbe]

Hi,

For the part where the odometry is red with the source version, are you using kinect V2 too? Are you using the ROS package (rtabmap_ros) or rtabmap alone as in the hand-held kinect tutorial?

When using rtabmap built from source, is there a warning like this when you start RTAB-Map with Freenect2 driver?:

CameraFreenect2: Failed to get frames! rtabmap should link on libusb of libfreenect2, this can be done by setting LD_LIBRARY_PATH to "libfreenect2/depends/libusb/lib"

Another thing to try, can you see a point cloud with rtabmap-rgbd_camera application using the Kinect v2 driver (5)?:

$ rtabmap-rgbd_camera 5

For the c++ tutorial, change CameraOpenni to CameraFreenect2 for Kinect v2.

cheers

[navid]

Hi,
Thank you for your answer.

I am using V2 for both, and I am using :
roslaunch rtabmap_ros rgbd_mapping_kinect2.launch which is working great.but rtabmap alone is not working.(just camera working and odometry is red).
I got  LD_LIBRARY_PATH warning but I have added to my PATH.I am not getting this warning anymore.

rtabmap-rgbd_camera 5 is the same.camera working but depth and cloud is black.

any idea?
Thnaks

[matlabbe]

So the rgb stream is working, but the depth stream is missing using the standalone rtabmap (or "rtabmap-rgbd_camera 5"). Just tried again and it is working (rgb, depth and cloud shown) on my computer (Ubuntu 14.04):

$ export LD_LIBRARY_PATH=~/workspace/libfreenect2/depends/libusb/lib:$LD_LIBRARY_PATH

$ ldd rtabmap-rgbd_camera | grep libusb
	libusb-1.0.so.0 => /home/mathieu/workspace/libfreenect2/depends/libusb/lib/libusb-1.0.so.0 (0x00007f04331c5000)

$ rtabmap-rgbd_camera 5

I am using the latest code from libfreenect2 and rtabmap. Normally, rtabmap and kinect2_bridge use both libfreenect2, so if one works, the other should work too. On my side, the only issue I had with libfreenect2 is to make sure that libusb installed with libfreeenect2 is correctly built and linked as above. Not like that:

$ ldd rtabmap-rgbd_camera | grep libusb
	libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007fe7afda7000)

[navid]

well the result of   $ ldd rtabmap-rgbd_camera | grep libusb is:

libusb-1.0.so.0 => /home/navid/libfreenect2/depends/libusb/lib/libusb-1.0.so.0 (0x00007f1b27fe7000)
which I think is fine. also I am using OpenCL for my libfreenect2.
on the other hand after running ($ rtabmap-rgbd_camera 5) I am getting so many warnings:


[OpenGLDepthPacketProcessor] avg. time: 6.93665ms -> ~144.162Hz
[ WARN] (2015-05-06 20:57:16.736) CameraRGBD.cpp:146::takeImage() Camera: Cannot reach target image rate 20.000000 Hz, current rate is 14.994973 Hz and capture time = 0.013366 s.
[ WARN] (2015-05-06 20:57:16.803) CameraRGBD.cpp:146::takeImage() Camera: Cannot reach target image rate 20.000000 Hz, current rate is 15.333421 Hz and capture time = 0.014590 s.
[ WARN] (2015-05-06 20:57:16.870) CameraRGBD.cpp:146::takeImage() Camera: Cannot reach target image rate 20.000000 Hz, current rate is 14.892271 Hz and capture time = 0.014906 s.
[ WARN] (2015-05-06 20:57:16.934) CameraRGBD.cpp:146::takeImage() Camera: Cannot reach target image rate 20.000000 Hz, current rate is 15.078204 Hz and capture time = 0.012266 s.

[matlabbe]

The warnings "CameraRGBD.cpp:146::takeImage() Camera: Cannot reach target image rate 20.000000 Hz" are normal. However, I've just fixed when this warning is sent (it should not be sent anymore for rtabmap-rgbd_camera).

Freenect seems to say that depth images are acquired:
[OpenGLDepthPacketProcessor] avg. time: 6.93665ms -> ~144.162Hz

It is using the OpenGL depth packet processor. I need to check tomorrow (on my computer with kinect v2) if it is using the OpenGL, the OpenCL or the CPU depth packet processor. It may be the difference. The one used is chosen automatically here. You can try to comment some lines to force a type and see if it is the problem.

Update
Kinect2_bridge seems to use OpenCL by default before OpenGL. See here.

[matlabbe]

Another thing to try: Is Protonect sample application from libfreenect2 works? (getting both rgb and depth images)

[navid]

Yes,both working fine (RGB and Depth)

Thanks,

[matlabbe]

Well, on my side it is also the OpenGL depth pipeline used. There is maybe a problem with the default image rectification of the depth image. Try rtabmap-calibration just to see if you can see the IR image on the left:

$ rtabmap-calibration --driver 5 --stereo

For info, here is the output I have for rtabmap-rgbd_camera:

$ rtabmap-rgbd_camera 5
[ INFO] (2015-05-07 16:19:36.116) main.cpp:76::main() Using driver 5
[Freenect2Impl] enumerating devices...
[Freenect2Impl] 14 usb devices connected
[Freenect2Impl] found valid Kinect v2 @4:3 with serial 506816242542
[Freenect2Impl] found 1 devices
[Freenect2DeviceImpl] opening...
[Freenect2DeviceImpl] opened
[Freenect2DeviceImpl] starting...
[Freenect2DeviceImpl] ReadData0x14 response
92 bytes of raw data
0x0000:  00 00 12 00 00 00 00 00 01 00 00 00 43 c1 1f 41    2e2e2e2e2e2e2e2e2e2e2e2e432e2e41
0x0010:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e
0x0020:  0a 21 33 55 c2 00 17 20 00 08 00 00 10 00 00 00    2e2133552e2e2e202e2e2e2e2e2e2e2e
0x0030:  00 01 00 00 00 10 00 00 00 00 80 00 01 00 00 00    2e2e2e2e2e2e2e2e2e2e802e2e2e2e2e
0x0040:  31 33 00 00 00 08 05 11 47 4d 53 33 35 32 2e 31    31332e2e2e2e2e2e474d533335322e31
0x0050:  58 00 00 00 00 00 00 00 00 00 00 00                582e2e2e2e2e2e2e2e2e2e2e

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000:  01 26 00 00                                        2e262e2e

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000:  03 26 00 00                                        2e262e2e

[Freenect2DeviceImpl] enabling usb transfer submission...
[Freenect2DeviceImpl] submitting usb transfers...
[Freenect2DeviceImpl] started
[ INFO] (2015-05-07 16:19:36.714) CameraRGBD.cpp:1226::init() CameraFreenect2: device serial: 506816242542
[ INFO] (2015-05-07 16:19:36.714) CameraRGBD.cpp:1227::init() CameraFreenect2: device firmware: 2.3.3912.0.7
[DepthPacketStreamParser::handleNewData] not all subsequences received 0
[ WARN] (2015-05-07 16:19:36.822) CameraRGBD.cpp:1240::init() Missing calibration files for camera "506816242542" in "." folder, default calibration used.
[TurboJpegRgbPacketProcessor] avg. time: 17.965ms -> ~55.6637Hz
[OpenGLDepthPacketProcessor] avg. time: 38.3067ms -> ~26.1051Hz
[TurboJpegRgbPacketProcessor] avg. time: 16.5485ms -> ~60.4283Hz
[TurboJpegRgbPacketProcessor] avg. time: 15.4131ms -> ~64.8799Hz
[OpenGLDepthPacketProcessor] avg. time: 36.3634ms -> ~27.5002Hz
[TurboJpegRgbPacketProcessor] avg. time: 17.531ms -> ~57.0419Hz
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] not all subsequences received 959
[OpenGLDepthPacketProcessor] avg. time: 35.5044ms -> ~28.1655Hz
[TurboJpegRgbPacketProcessor] avg. time: 16.2916ms -> ~61.3814Hz
[TurboJpegRgbPacketProcessor] avg. time: 20.7211ms -> ~48.2601Hz
[OpenGLDepthPacketProcessor] avg. time: 35.7283ms -> ~27.989Hz
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] not all subsequences received 1007
[TurboJpegRgbPacketProcessor] avg. time: 17.0733ms -> ~58.5711Hz
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] not all subsequences received 767
[OpenGLDepthPacketProcessor] avg. time: 35.0548ms -> ~28.5267Hz
[TurboJpegRgbPacketProcessor] avg. time: 17.8294ms -> ~56.0871Hz
...

The output of Protonect:

$ ./Protonect 
[Freenect2Impl] enumerating devices...
[Freenect2Impl] 14 usb devices connected
[Freenect2Impl] found valid Kinect v2 @4:3 with serial 506816242542
[Freenect2Impl] found 1 devices
[Freenect2DeviceImpl] opening...
[Freenect2DeviceImpl] opened
[Freenect2DeviceImpl] starting...
[Freenect2DeviceImpl] ReadData0x14 response
92 bytes of raw data
0x0000:  00 00 12 00 00 00 00 00 01 00 00 00 43 c1 1f 41    2e2e2e2e2e2e2e2e2e2e2e2e432e2e41
0x0010:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e
0x0020:  0a 21 33 55 c2 00 17 20 00 08 00 00 10 00 00 00    2e2133552e2e2e202e2e2e2e2e2e2e2e
0x0030:  00 01 00 00 00 10 00 00 00 00 80 00 01 00 00 00    2e2e2e2e2e2e2e2e2e2e802e2e2e2e2e
0x0040:  31 33 00 00 00 08 05 11 47 4d 53 33 35 32 2e 31    31332e2e2e2e2e2e474d533335322e31
0x0050:  58 00 00 00 00 00 00 00 00 00 00 00                582e2e2e2e2e2e2e2e2e2e2e

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000:  01 26 00 00                                        2e262e2e

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000:  03 26 00 00                                        2e262e2e

[Freenect2DeviceImpl] enabling usb transfer submission...
[Freenect2DeviceImpl] submitting usb transfers...
[Freenect2DeviceImpl] started
device serial: 506816242542
device firmware: 2.3.3912.0.7
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] not all subsequences received 0
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] image data too short!
[RgbPacketStreamParser::handleNewData] skipping rgb packet!
[TurboJpegRgbPacketProcessor] avg. time: 16.5876ms -> ~60.2861Hz
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] image data too short!
[DepthPacketStreamParser::handleNewData] not all subsequences received 895
[OpenGLDepthPacketProcessor] avg. time: 26.3555ms -> ~37.9428Hz
[TurboJpegRgbPacketProcessor] avg. time: 15.5361ms -> ~64.366Hz
[OpenGLDepthPacketProcessor] avg. time: 26.6378ms -> ~37.5407Hz
[TurboJpegRgbPacketProcessor] avg. time: 15.571ms -> ~64.2219Hz
[OpenGLDepthPacketProcessor] avg. time: 26.533ms -> ~37.6889Hz
...