Getting errors when running vins-fusion on euroc_dataset

[GilMe]

Hi,
I am trying to run the example for running the euroc_datasets.launch file with the vins-fusion option (Odom/Strategy 9).

I managed to install rtabmap with vins-fusion support (I applied the patch). When I tried to run the example:
> roslaunch rtabmap_ros euroc_datasets.launch args:="Odom/Strategy 9 OdomVINS/ConfigPath ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_imu_config.yaml" MH_seq:=true raw_images_for_odom:=true

with the following bag file
> rosbag play --clock MH_01_easy.bag

I get the following error:
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ERROR] (2022-12-22 19:11:19.044) OdometryVINS.cpp:508::computeTransform() VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)!
[ INFO] [1671729079.083102053, 1403636606.500363316]: Odom: quality=0, std dev=0.010000m|0.010000rad, update time=0.038615s

I did some checks on the OdometryVINS.cpp file.

1. I checked and the config yaml file passes through.
2. I also checked the values of the condition in line 370:
        if(!data.imageRaw().empty() && !data.rightRaw().empty() && data.stereoCameraModels().size() == 1 && data.stereoCameraModels()[0].isValidForProjection())

I checked the values for the conditions to see which fails:
a. !data.rightRaw().empty()
b. !data.rightRaw().empty()
c. data.stereoCameraModels().size() == 1

I couldn't check the last condition because printing it causes the program to crash.

The results I got were false for all, but ever so often they were true. It is like from time to time it is able to pass through the if condition. I think that is why I am getting Odom [INFO] messages.

I tried to install the VINS patch in several different ways and I always get the same result. I Tried:
1. Applying the patch after doing the following (instructions I found online):
> git clone https://github.com/HKUST-Aerial-Robotics/VINS-Fusion.git 
> cd VINS-Fusion
> git checkout e72b5f7ae7437e8cf481cc0bcbfb5fe91bd3d640
> git apply --ignore-space-change --ignore-whitespace vins-fusion_e72b5f7.patch

2. Manually making the changes in the VINS files in the new updated repository

Anyways I am not able to get it to work. Any advice?

I also had a few questions I of things I didn't understand.
1. I could only find a reference to Odom/Strategy in icp_odometry params documentation. It doesn't seem like the icp_odom node isn't opened. To whom is the strategy passed?
2. The original vins_fusion is able to work with a monocular camera and imu (there is even an example yaml for the euroc dataset). Is there an option to run vins_fusion through rtabmap with a monocular camera?

Thanks

[matlabbe]

Hi,

Thx for reporting! I fixed the error message "VINS-Fusion requires stereo images (and only one stereo camera with valid calibration)! " wrongly logged in this commit. I saw also that VINS may be unstable at the beginning, better to start the rosbag 40 sec from start:

$ roslaunch rtabmap_ros euroc_datasets.launch \
   args:="Odom/Strategy 9 OdomVINS/ConfigPath ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_imu_config.yaml" \
   MH_seq:=true \
   raw_images_for_odom:=true

$ rosbag play -s 40 --clock MH_01_easy.bag 

To test on ROS noetic, I installed VINS-Fusion like this:

cd catkin_ws/src
git clone https://github.com/HKUST-Aerial-Robotics/VINS-Fusion.git 
cd VINS-Fusion 
git fetch origin pull/136/head:opencv4
git checkout opencv4
wget https://gist.githubusercontent.com/matlabbe/795ab37067367dca58bbadd8201d986c/raw/d51a56a979e68fa216ba3653cf3c52bbb5162320/vins-fusion_be55a93_pull136.patch 
git apply --ignore-space-change --ignore-whitespace vins-fusion_be55a93_pull136.patch 

For your questions:
1. Odom/Strategy is passed to rtabmap library. RTAB-Map's parameters documentation can be shown with "rtabmap --params | grep Odom/Strategy". The "Odom" parameters are used by rgbd_odometry, stereo_odometry and icp_odometry nodes. This is explained here. In this case, used by stereo_odometry node.
2. No, RGB-D or stereo camera is needed. We didn't create a mono_odometry node. In that case, you may just use vins ros node directly and feed its odometry topic to rtabmap node.

cheers,
Mathieu