RtabMap reads video/images from file

[scarabeo]

Is there any possibility  that RtabMap could read images already saved in my computer?
I have a folder with kinect v2 images, (it contains color images, depth images and infrared images), and I want to calculate the point cloud of these files.
Is there any option? If is there, what image format I should use?

[matlabbe]

Hi,

yes it is possible. The RGB and depth images should be the same size, rectified, and that depth image is registered to color image. The format for depth images are 16bits unsigned (mm) or 32bits float (meter).

An example can be found for TUM datasets here : http://official-rtab-map-forum.206.s1.nabble.com/How-to-process-RGBD-SLAM-datasets-with-RTAB-Map-tp939p647.html

In summary:
1- Open Preferences->Source.
2- Make sure "RGB-D" source type is selected.
3- Set frame rate to 10 Hz for example.
4- Click on "Create Calibration" to create a simple calibration file. The camera parameters of my Kinect v2 are fx=1035.409658 fy=1036.265896, cx=953.629091, cy=550.972045 size=1920x1080.
5- Make sure the name you gave to calibration file is set in "Calibration file path".
6- Scroll down and select "Images" driver.
7- Fill RGB directory path
8- Fill depth directory path
9- Press ok and start mapping!

cheers

[scarabeo]

Thanks for your answer.

I have another doubt, My depth images are 32 bits float.
Should I save them in PNG format o TIFF?
Does png format accept 32bit float? or What format should I use for my depth images?

[matlabbe]

Hi,

Well, good question. I think I have always used 16bits PNG. RTAB-Map is using this kind of code to determine if the resulting format is correct when reading a depth image with OpenCV:

cv::Mat img = cv::imread(imageFilePath.c_str(), cv::IMREAD_UNCHANGED);
if(img.type() != CV_16UC1 && img.type() != CV_32FC1)
{
	UERROR("Depth format not supported (file = \"%s\"). "
		"Formats supported are 16 bits 1 channel and 32 bits 1 channel.",
		imageFilePath.c_str());
	img = cv::Mat();
}

No matter the image type is (at least it should be recognized by OpenCV), the resulting matrix should be CV_16UC1 to CV_32FC1.

For a fast test, you can use OpenCV python to check the format when reading your images.

cheers

[scarabeo]

Hey, another question,
I have two folders, RGB and Depth, both images are synchronized in this way:
color_0.jpg ---- depth_0.tiff
...
color_n.jpg ---- depth_n.tiff

n is the number that indicates which rgb image corresponds to a depth image.

What options do I have to use to synchronize my images in RTabMap ?

[matlabbe]

Hi,

If they are in the same order in both folders, they should be loaded together. The RGB folder and the depth folder must have the same number of images.

To verify when rtabmap is running, you can right-click on the image view and check "Show image depth" to see if RGB and depth match together.

cheers

[scarabeo]

Understood, but I still don't get it what I want.
Maybe it is the file of calibration.. because I don't have data about distortion_coefficients, rectification_matrix, and projection_matrix
How can I get these values?

This is my calibration file:
%YAML:1.0
camera_name: calib
image_width: 1920
image_height: 1080
camera_matrix:
   rows: 3
   cols: 3
   data: [ 1.0354096580000000e+03, 0., 9.5362909100000002e+02, 0.,
       1.0362658960000001e+03, 5.5097204499999998e+02, 0., 0., 1. ]


but the size of my images are 512x624, It is ok ?
Another thing, What is my Depth scale factor ?

Thanks for your helping..

[matlabbe]

Hi,

The minimal parameters required in the calibration file are focal distances fx and fy. In your case, you can try:

%YAML:1.0 
camera_name: calib 
image_width: 0 
image_height: 0 
camera_matrix: 
   rows: 3 
   cols: 3 
   data: [ 400, 0., 0, 0., 400, 0, 0., 0., 1. ] 

If you know fx and fy, change the corresponding values. A wrong fx and fy will only change the scale of the point cloud, but rtabmap should be able to create one. The depth scale factor is a factor applied to values read from the depth image. For example, in TUM datasets, the mm values are multiplied by 5 (5000 = 1 meter).

EDIT: with this kind of calibration, we assume that RGB and depth images are rectified and registered together.

cheers

[scarabeo]

I just figure out that my error was having 32-bits .tiff images.
Kinect v2 gives me 32 bit images and I save them in .tiff, however, when I tried to map, rtabmap didn't work.
Now, if I convert the images into 16-bits, it works and maps with a simple calibration parameters.