Outdoor Localization error while running zed-ros-examples using zed2i camera

[impressivetoken]

Hello,

(i)I actually don't know how to explain this problem on text thoroughly,
What basically is happening is the position in the map isn't moving ahead even if the car is going ahead and the position is moving around in small circles within that small area.
I seriously dont know if i am making sense right now, that is why i am attaching a few recordings of it.
video1.mp4

(ii)I Also am facing a bit of a drift problem even if it is localizing
a video is attached for that as well
video2.mp4

I am attaching another video that can help with the terminal outputs as well
video3.mp4
(is there anything i can do about "not enough inliers" output?, as i am observing a lot of them while localizing even if it is at the right place)

i have also set
OptimizeMaxError = 4.0
LoopThr = 0.07
MinInliers = 7

Can anyone please help me out here.....

Thank you in advance

[matlabbe]

I cannot read the text in the screenshot of the terminal, but it looks like RGBD/MaxOdomCacheSize is 0. You would get smoother localizations if you set it to default value.

What are you using for odometry? What is the covariance of the odometry topic?

Setting Vis/MinInliers to 7 is pretty low and you may could accept very bad re-localizations. Default is 20.

[impressivetoken]

Hi again,

Thank you for your reply
I have attached the file of the above given screenshot just in case
Screenshot_2023-11-17_120633.png

(i)I have roslaunched the zed_rtabmap_example and put in the command "rosparam get /zed/rtabmap/RGBD/MaxOdomCacheSize" and the value i got was "10"

(ii)I am using ZED's odometry and all the parameters are default except the one's i have mentioned in the above post

(iii)I agree setting Vis/MinInliers to 7 is pretty low, but i just wanted to play around and decrease it just to see if i can get atleast a few localizations in the map which isnt the case anymore.

Whenever there is a loop closure rejection, it says "Not enough inliers 0/7" or "0/20" in default case.
(i havent had a single loop closure detection/localization and this is the only warning that i can see in the terminal whenever it rejects loop closures)

(iv)I also had a doubt on what exactly does "Rtabmap/MaxRepublished" do and what can happen if i increase or decrease that parameter?

[matlabbe]

When the odometry is not moving, the issue is coming from ZED's odometry not moving for some reasons.

I am surprised there are not more inliers, as the images look pretty similar with same point of view. Can you share the rosbag of the localization run and the one of the mapping run?

If the robot jumps are high after multiple consecutive localizations, it may be related to poor covariance set in odometry topic.

The Rtabmap/MaxRepublished parameter is used to limit the number of nodes with data republished to clients not having the data of those nodes, like when when rviz's MapCloud display or rtabmap_viz are used.

[impressivetoken]

You must have received a mail

[matlabbe]

I don't see related email. You can click on my username and  click on send email.

[impressivetoken]

Is there any other way i can reach out to you?

[matlabbe]

My username @ gmail.com, or use my university email.

[impressivetoken]

CONTENTS DELETED

The author has deleted this message.

[matlabbe]

Hi,

I received the data. I'll download the bags later, but for now from what I see in the database, I'll suggest to use rtabmap in stereo mode instead of RGB-D mode as the depth from ZED is valid only closer to the camera. With the current depth, we lose the far features for more accurate motion estimation and better loop closure detection. This is maybe the reason why there are not a lot of inliers.

(as you shared videos above showing similar images, I thought that screenshot was not a problem to share, if it is, just tell me I'll remove it)

Do the bags have also left/right image data? I could give a try in stereo mode when I'll have downloaded the bags.

[impressivetoken]

Hello,

(i)What changes do i need to make to run stereo mode while using zed-ros-examples?

Also, if I run stereo mode, how will it effect my localization accuracy?
Will i able able to use it for autonomous navigation?

(ii)Yes, the bags do have left/right image data.

P.s. - Screenshots shouldnt be a problem, thanks!

[matlabbe]

Hi,

Another issue I see in the bags is that in the localization bag, the image resolution is 2X the resolution in the mapping bag. The features (BRIEF) used by default are not very scale invariant, so doubling the resolution will generate very low matches and loop closure hypotheses would be very low. I based my examples below from this tutorial.

For the mapping bag, we have to remove the "map" frame in /tf topic before reprocessing it (based on this post), then:

roslaunch rtabmap_launch rtabmap.launch \
   rtabmap_args:="--delete_db_on_start --Vis/CorFlowMaxLevel 5 --Stereo/MaxDisparity 200" \
   stereo_namespace:=/zed/zed_node \
   right_image_topic:=/zed/zed_node/right/image_rect_gray \
   stereo:=true \
   visual_odometry:=false \
   frame_id:=base_link \
   odom_frame_id:=odom \
   imu_topic:=/zed/zed_node/imu/data \
   wait_imu_to_init:=true \
   use_sim_time:=true

rosbag play --clock MAPPING.bag

For the localization bag, we have to decimate the input images, and I also needed to remove "map" and "odom" frames as odom framerate recorded in that bag is slow, so I recomputed odometry on my side (not that odometry is not that nice as it is only done at 1 Hz, but was okay to test):

roslaunch rtabmap_launch rtabmap.launch \
   rtabmap_args:="--Vis/CorFlowMaxLevel 5 --Stereo/MaxDisparity 200 --Mem/ImagePreDecimation 2 --Mem/ImagePostDecimation 2" \
   stereo_namespace:=/zed/zed_node \
   right_image_topic:=/zed/zed_node/right/image_rect_gray \
   stereo:=true \
   visual_odometry:=true \
   frame_id:=base_link \
   imu_topic:=/zed/zed_node/imu/data \
   wait_imu_to_init:=true \
   localization:=true \
   use_sim_time:=true

rosbag play --clock localization.bag

There is localization on almost every frame.


To use stereo with zed-ros-examples, you will have to change under rtabmap and rtabmap_viz this:

<remap from="rgb/image"         to="$(arg rgb_topic)"/>
<remap from="depth/image"       to="$(arg depth_topic)"/>
<remap from="rgb/camera_info"   to="$(arg camera_info_topic)"/>

by this:

<param name="subscribe_stereo" value="true""/>
<remap from="left/image_rect"    to="/zed/zed_node/left/image_rect_color"/>
<remap from="right/image_rect"   to="/zed/zed_node/right/image_rect_gray"/>
<remap from="left/camera_info"   to="/zed/zed_node/left/camera_info"/>
<remap from="right/camera_info"  to="/zed/zed_node/right/camera_info"/>

[impressivetoken]

Hi,

I actually did not know about the scale invariability of BRIEF. I had a couple of questions related to that....

(i)How can you extract the resolution information from the bag files itself?
(ii)Are there any other things that i need to keep in mind with the scale invariability of BRIEF?....

Also, The thing with my project is that i have to localize everything in real time cause i am trying to navigate the vehicle autonomously which would require real time localized values to be taken.

So, keeping what you told me about resolution in mind, I decreased my resolution and it localizes upto a certain extent.

What I mean by certain extent is that it localizes a couple of times and doesnt for the other(It basically shows loop closure rejection with not enough inliers, its not 0/20 this time. it usually states that it has 10-15 inliers atleast).

Is there anything else that i can do here?

P.s. - I Will try running the localization using bag files as you asked and update you in a couple of days.

Thanks!

[matlabbe]

(i) rostopic echo /zed/zed_node/left/image_rect_color --no-arr

(ii) not really, just that between mapping and localization, cameras should have similar FOV and same resolution. You could try Vis/FeatureType=Kp/DetectorStrategy=10, or Vis/FeatureType=Kp/DetectorStrategy=2 to use ORB features (which would be more invariant to scale).

You cannot expect to get a localization every frame. Depending the weather and dynamic changes of the environment, the number of localization may vary. You can look at this paper, and it was not even outdoor and the environment was not changing much.

However, if you follow my example, localization should be a lot better than in your initial tests.

[impressivetoken]

Hi again,

I tried running the commands for localization as you asked me to,

"roslaunch rtabmap_launch rtabmap.launch \
   rtabmap_args:="--Vis/CorFlowMaxLevel 5 --Stereo/MaxDisparity 200 --Mem/ImagePreDecimation 2 --Mem/ImagePostDecimation 2" \
   stereo_namespace:=/zed/zed_node \
   right_image_topic:=/zed/zed_node/right/image_rect_gray \
   stereo:=true \
   visual_odometry:=true \
   frame_id:=base_link \
   imu_topic:=/zed/zed_node/imu/data \
   wait_imu_to_init:=true \
   localization:=true \
   use_sim_time:=true"

It was working great when i used the bag file for it, there was a localization in almost every frame as you said.

As you know i want this in real time, i made a few simple changes in the command you suggested

roslaunch rtabmap_launch rtabmap.launch    rtabmap_args:="--Vis/CorFlowMaxLevel 5 --Stereo/MaxDisparity 200 --Mem/ImagePreDecimation 2 --Mem/ImagePostDecimation 2 --RGBD/OptimizeMaxError 10"    stereo_namespace:=/zed2i/zed_node    right_image_topic:=/zed2i/zed_node/right/image_rect_gray    stereo:=true    visual_odometry:=true    frame_id:=base_link    imu_topic:=/zed2i/zed_node/imu/data    wait_imu_to_init:=true    localization:=true
(After launching the zed2i cam)

And tried running the same thing in real time for localization.
There is a screen recording i have uploaded in the one drive link i have sent you by the mail the other day for the bag files.

I feel like i am doing something very wrong which is very silly.

Please help me out here.

[impressivetoken]

Hi again,

I have figured out on how to do it in real time

I just had a small question.....can we increase the frequency of /rtabmap/localization_pose/ or are there any other rostopics that gives out the same value at higher frequency?

[matlabbe]

The frequency of "/rtabmap/localization_pose" is linked to Rtabmap/DetectionRate, which would be 1 by default. In localization mode, you may increase Rtabmap/DetectionRate. If you need the pose at same frame rate than the camera frames (e.g., 30 Hz), I'll suggest to use TF at time=0 to get the latest pose of the base frame in map frame.

[impressivetoken]

Hi,

I cannot figure out a way on how to do it exactly, can you please help me out on how to change the tf?

Thanks

[matlabbe]

tf::lookupTransform:
http://docs.ros.org/en/indigo/api/tf/html/c++/classtf_1_1Transformer.html#ac01a9f8709a828c427f1a5faa0ced42b

a python tutorial: http://wiki.ros.org/tf/Tutorials/Writing%20a%20tf%20listener%20%28Python%29