Carmine and Stereo camera

[consoante]

Hi,

I am doing a research project with RTAB-map and want to assess the results using a stereo camera.

Up until now I have used it with a Carmine 1.09 but I am not too impressed with the results, perhaps this sensor is not ideal considering its short range? Which would be the best RGBD camera for a reliable result? I am interested in mapping large areas, not single rooms.

I have seen some posts discussing the ZED stereo camera with concerns about accuracy: I am interesting in the best accuracy possible so should I opt for a Bumblebee2?
Also, considering RTAB-map is feature based how does a mono camera perform against a RGB?

Thanks in advance.

//ro

[matlabbe]

Hi,

Kinect for Xbox One (or Kinect v2) is a pretty good RGB-D camera for its price. You will get longer range feature tracking than the first generation (or Carmine/Xtion) that are limited under 4 meters.

The ZED camera can give accurate poses, but you will not have a nice point cloud indoor, particularly if there is not so much texture in the environment. The ZED camera would give similar results as the Bumblebee2. If you need a dense point cloud indoor, I would stick with RGB-D cameras.

By "mono" do you mean grayscale? If so, it is the same as RGB (rtabmap converts RGB to grayscale for features matching anyway). If you refer to monocular camera, rtabmap is not a monocular SLAM approach so it will not work.

cheers

[consoante]

Hi again,

I just saw the camera, to my surprise it seems to be time of flight and not structured light like the first version. Sounds good, i will get that one.

By mono I mean monochrome yes, I will want to compare results so I will test rtab-map using a stereo monochrome camera too. In fact, i want to test a feature base approach too  so 2 cameras for 2 approaches seems a good way to go

Thanks,

//ro

[consoante]

Another idea is to use a a duo camera https://duo3d.com/ or one from leopard https://www.leopardimaging.com/LI-USB30-V024STEREO.html

The baseline are smaller though! Any thoughts from anyone?

The option for a monochrome sensor is because they have lower  signal-to-noise ratios...

Thanks.

[erics]

Tara is a UVC-Complaint 3D Stereo camera based on MT9V024 Monochrome stereo sensor from ON Semiconductor which supports WVGA((2*752)x480) at 60fps over USB 3.0 in uncompressed format. This Stereo camera provides two synchronized sensor frame data interleaved side by side to the host machine over USB 3.0 interface.

[Jason Liu]

Hello Matlabbe,

I am new to Rtabmap.
As you mentioned in the previous response. Rtabmap doesn't support monochrome stereo camera.
Unfortunately, I only have a mono stereo camera. Currently, I can see the point cloud, trajectory working. But the 3D map is not retained.  Could you give me some guidances to get this work.  

Thanks a lot!

[matlabbe]

Hi,
if you read again my answer, rtabmap supports monochrome stereo camera, not monocular camera.

What do you mean by "the 3D map is not retained"? Can you share the database?

cheers,
Mathieu

[Jason Liu]

Hi Mathieu,

Sorry, that's my fault of reading .

Here is the database I got for your reference.
https://drive.google.com/open?id=1wdTe5IIvA5QwG4S6VX9NWndpT3xS_ZAx 

In following screenshot, it only shows the latest feature points and point clouds reprojected into 3D map on my computer.  The point clouds before the current timestamped  were wiped out when the new images feeded.  So the 3D map is not retained all the historical point clouds as the result when I  ran the  demo_stereo_outdoor.launch and the example bag file.

Here are the other settings regarding my Tara stereo camera.
Main launch file:stereo_mapping2.launch.
I changed  "/left/image_rect_color" to "/left/image_rect".

stereo_image_proc launch file: tara.launch.

rviz:rgbd_tara.rviz

gui.ini:rgbd_gui_tara.ini
The frame id of  the left camera published is  "cameraLeft".

My current goal is to get octomap from stereo camera for the path planning.

Rtabmap is a great mapping platform. Thank you all contributors.

cheers,
Jason

[matlabbe]

Hi,

What is the calibration file you are using? When I open the database, the scale is huge. looking at the calibration saved in the database, the baseline between the left and right cameras is 60 meters! It is normal that in RVIZ the cloud is not shown, as by default, MapCloud plugin filters up to 4 meters from the camera. Depending on the version of rtabmapviz, the default for the map may be also 4 meters (Preferences->3D Rendering). You should fix the calibration. I also noticed that the stereo rectification is not perfect, there is a small offset (about 1 pixel) on y axis between left and right images.

[Jason Liu]

Hi Mathieu,

Thank you for pointing out the issue.
I checked the camera ROS driver. I found the published camera_info is from the flash memory on the camera. It's probably the factory default parameters . I did calibration on the left camera for monocular VO only and haven't done the calibration on the right one. That's why the baseline is wrong. It should be 60mm.

Do you have any suggestion with the size of checkerboard? I used letter size checkerboard and never got the full bar on y direction.

[matlabbe]

Try this tutorial: http://wiki.ros.org/camera_calibration/Tutorials/StereoCalibration

[Jason Liu]

Super~ after I redo the calibration, the result is pretty good.
Thank you so much!

[prashanth]

i hav a usb side by side stereo camera but i dont know what are all usb camera format does rtabmap support.
My camera comes with rgb 24 camera fomat ,does rtabmap opencv support the format any idea.  

[matlabbe]

If you are on ros, connect the image topics to rtabmap and see if rtabmap shows a message of wrong format. If so, you can create a node doing the conversion or modify the code of the camera driver. Note however that when rtabmap is subscribing to rectified image topics of stereo_img_proc node, the topics should be already compatible. You may look at if stereo_img_proc is supporting your format. Formats supported by rtabmap:

sensor_msgs::image_encodings::MONO8
sensor_msgs::image_encodings::MONO16
sensor_msgs::image_encodings::BGR8
sensor_msgs::image_encodings::RGB8

cheers,
Mathieu