Re: Slam using Intel RealSense tracking camera - T265

Posted by matlabbe on Oct 14, 2019; 3:27pm
URL: http://official-rtab-map-forum.206.s1.nabble.com/Slam-using-Intel-RealSense-tracking-camera-T265-tp6333p6336.html

Hi,

The images should be rectified so that proper reconstruction can be done. What did you use for calibration? I don't think ROS stereo calibration tool supports fisheye stereo. In rtabmap standalone, it is possible to do fisheye stereo calibration for T265. See Preferences->Source, select Stereo camera, select RealSense2 driver, then click on Calibrate. It may require a couple tries to get the calibration right. Here are my calibration files for T265:

905312112011_left.yaml:

%YAML:1.0
---
camera_name: "905312112011_left"
image_width: 848
image_height: 800
camera_matrix:
   rows: 3
   cols: 3
   data: [ 2.8845847407434218e+02, 0., 4.2366155337303530e+02, 0.,
       2.8732351128054398e+02, 4.0774252018099634e+02, 0., 0., 1. ]
distortion_coefficients:
   rows: 1
   cols: 4
   data: [ 1.6108401660058620e-02, -1.5462775664278158e-02,
       9.5501371240367117e-03, -6.7432882743460659e-03 ]
distortion_model: equidistant
rectification_matrix:
   rows: 3
   cols: 3
   data: [ 9.9996361896819130e-01, 1.0752140026281053e-03,
       -8.4619533729723070e-03, -1.0848458428058135e-03,
       9.9999876886529271e-01, -1.1337448549999169e-03,
       8.4607239368243194e-03, 1.1428835231309651e-03,
       9.9996355431971484e-01 ]
projection_matrix:
   rows: 3
   cols: 4
   data: [ 2.8845847407434218e+02, 0., 4.2228301048278809e+02, 0., 0.,
       2.8845847407434218e+02, 4.0177149677276611e+02, 0., 0., 0., 1.,
       0. ]

905312112011_right.yaml:

%YAML:1.0
---
camera_name: "905312112011_right"
image_width: 848
image_height: 800
camera_matrix:
   rows: 3
   cols: 3
   data: [ 2.8971586612342838e+02, 0., 4.3282951572869268e+02, 0.,
       2.8842800650095052e+02, 3.9647983507794498e+02, 0., 0., 1. ]
distortion_coefficients:
   rows: 1
   cols: 4
   data: [ 1.1755170565796828e-03, 2.7439249292868447e-02,
       -3.1282234368493568e-02, 5.7239868467015087e-03 ]
distortion_model: equidistant
rectification_matrix:
   rows: 3
   cols: 3
   data: [ 9.9999049872412404e-01, 1.9012821940872346e-03,
       -3.9227015558338232e-03, -1.8968518482397822e-03,
       9.9999755932917211e-01, 1.1328233598542061e-03,
       3.9248457986937090e-03, -1.1253718128906738e-03,
       9.9999166452712951e-01 ]
projection_matrix:
   rows: 3
   cols: 4
   data: [ 2.8845847407434218e+02, 0., 4.2228301048278809e+02,
       -1.8501290547465036e+01, 0., 2.8845847407434218e+02,
       4.0177149677276611e+02, 0., 0., 0., 1., 0. ]

905312112011_pose.yaml:

%YAML:1.0
---
camera_name: "905312112011"
rotation_matrix:
   rows: 3
   cols: 3
   data: [ 9.9998938286662098e-01, -8.1716008464192697e-04,
       -4.5350196725749909e-03, 8.0684836818960455e-04,
       9.9999708631380535e-01, -2.2751614471256884e-03,
       4.5368656300714865e-03, 2.2714782182106175e-03,
       9.9998712853564209e-01 ]
translation_matrix:
   rows: 3
   cols: 1
   data: [ -6.4137879190303601e-02, -1.2194536630761036e-04,
       2.5159614897210655e-04 ]

Before calibration:


After calibration (you may have to click "Unlock" to enable the calibration, it is difficult with fisheye cameras to fill the size bar completly):


Reconstruction example:


EDIT: Fisheye calibration is little more sensitive than normal calibration. In the following video, see what kind of motions that will make calibration converge correctly:
1) zoom in and zoom out for both cameras
2) Make the skew from one corner
3) Do horizontal motion while keeping same camera angle, in both ways
4) Do vertical motion while keeping same camera angle, in both ways
https://youtu.be/m-jSfEPUSCU