How to obtain more loop detection with only RGB images

[climbing robot developer]

Hello, thank you for giving us wonderful tools.
I am a beginner in Rtab-Map and computer vision.
I have a question about how to obtain more loop detection with only RGB images.


faced problem
I have been facing a problem in that few loops are detected.
For example, this image pair is not detected as a loop.


On the other hand, this image pair is detected as a loop.


In my opinion, this problem seems to occur because all values calculated by getLoopClosureValue() function are low.
Specifically, the max of these values is about 0.13. And then, the threshold is set to 0.08.


My data
Only RGB images (appearance) based loop detection is going to be added to my project.
I have been referring to your sample code "rtabmap/examples/BOWMapping". A modified sample code is here.
In this code, a sharpness filter is applied to all images inputted to a rtabmap module. This filter seems to obtain more loop detection based on my trial and error.

RGB images used in this code are included in the following link.
https://drive.google.com/drive/folders/1PRMXI61-bv5K59xG51ceVsRhaQBmtlOj?usp=share_link

Then, a database made by this code is included in the following link.
https://drive.google.com/file/d/15zKe-5NUZu2uIh337jSh-PtoQP3namBO/view?usp=share_link


Investigation by me
I have found some solutions to similar questions about this problem such as the following links and etc.
https://answers.ros.org/question/241032/poor-results-of-loop-closurse-detection-rtabmap_ros/
http://official-rtab-map-forum.206.s1.nabble.com/Loop-Closure-not-detected-td7109.html

Then, RGBD/LoopClosureReextractFeatures was used in the above question pages. I think this option is used by RGBD images
However, these question pages cannot be used in my project because only RGB images are used in my project.



To sum up, could you give me some techniques of how to obtain more loop detection with only RGB images?
In addition to this question, is the sharpness filter helpful for more loop detection?

[matlabbe]

Hi,

Your dataset is too small (16 images) to correctly see loop closure hypotheses building up. It also switches between object-noobject-object-noobject. The loop closure hypothesis filter in RTAB-Map needs multiple consecutive images of a loop closure to build up the hypothesis. You can see that hypotheses stay pretty low (<0.10), which is mostly random at that point.

You can enable "VhEp/Enabled" parameter to at least reject loop closures without any feature matching :

[climbing robot developer]

Thank you for your polite reply.

I have been applying your loop detection system to my collaborative visual slam system. Therefore, My dataset switches between object(image from RobotA)-noobject(image from RobotB)-object(image from RobotA)-noobject(image from RobotB). Thanks to your comments in terms of the loop closure hypothesis filter, I understand that my implementation does not maximize the use of your loop detection.

However, when VhEp/Enabled is set to true, false positive loops are not detected completely in the datasets attached to previous my post. In addition, more loops are detected by setting VhEp/Enabled to true. Thank you, your loop detection system seems to be used for my collaborative visual slam.

[climbing robot developer]

Hi, matlabbe.

I would like to ask you additional questions related to this problem.

Can I disable the loop closure hypothesis filter in RTAB-Map?

If the filter is disabled, I suppose that a loop is detected even in the case of my dataset switching between object(image from RobotA)-noobject(image from RobotB)-object(image from RobotA)-noobject(image from RobotB).


Thank you.

[matlabbe]

Not sure if it is possible to completely disable the hypothesis filter. You can set "Rtabmap/LoopThr" to 0.01 to trigger a loop closure with highest hypothesis (even if very low / noise). To remove the "inertia" from the filter, you may set "Bayes/PredictionLC" to 0.9 instead of this curve.


Well, "Bayes/PredictionLC" and "Bayes/VirtualPlacePriorThr" should sum to 1, so you could try different ratios.

To use likelihood directly, you will have to modify the code to bypass the filter and just take the image with highest likelihood directly.

[climbing robot developer]

Hi matlabbe, thank you for your kind and quick advice.

I will try to adjust "Bayes/PredictionLC" and "Bayes/PredictionLC". If more loops are not detected based on these parameters adjustments, we will modify your codes for using likelihood directly, adding something.

Thank you!!