Extending the set of descriptors used in RTAB-Map

[tango_explorer]

Hi,

I'm trying to extend the set of descriptors currently used in RTAB-Map. In my scenario, the descriptor code is agnostic about what type of keypoints provided. I took a look at your features2d class, and your extension of ORB. Would following your example with GFTT-ORB suffice or is there any tip/advice you can give to make life easier and interfacing it with the rest of the code?

Also, I'm a bit new to heavy coding(coding that isn't for homework), what exactly does your RTABMAP_EXP do, and why is it in every class definition?

Thanks in advance for your help.

[matlabbe]

Hi,

In RTAB-Map, as not all descriptors work with any keypoint detectors, we provided some hard coded combinations like GFTT/BRIEF or FAST/BRIEF. So you can indeed base your code on these classes: https://github.com/introlab/rtabmap/blob/af1cdadabd97e179d30421666b2974f973b6e898/corelib/include/rtabmap/core/Features2d.h#L356-L412

You may added new approaches here and there to be used with parameter "Kp/DetectorStrategy" and "Vis/FeatureType" parameters.

RTABMAP_EXP is used for DLLs in windows, to tell if we are exporting or importing objects to/from dlls.

cheers,
Mathieu

[tango_explorer]

Thanks for the help, I've added the new descriptor. However, it's not working quite well with the loop closure module.
Do you have any tips on how to easily debug the loop closure module?

Also, I was looking at the vwdictionary code, could you briefly explain what the non-indexed words and removed-indexed words do and why they would be desired? Sorry if these are petty questions, but thanks for your help so far.

[matlabbe]

To debug, you may set Logger's level to Debug. See Preferences->Logger in the GUI.

RTAB-Map uses an incremental dictionary. When new features are extracted, they are compared to the actual dictionary (word quantization). If some features have no correspondence with actual words, they are added temporary to a non-indexed words list. We keep track of the words added that are not yet indexed in the KD-Trees. When comparing new features, they are compared to those in the KD-Trees and those not indexed (linear search). The non-indexed words will be eventually added to KD-Trees when the dictionary is updated (on next rtabmap update).

Note that binary features are converted to FLOAT if kd-trees nearest neighbor strategy is used. What is the best nearest neighbor approach that should be used for your kind of descriptor (available from FLANN: linear/brute force, kd-tree, LSH...)? and also which kind of distance approach should be used (L2_NORM, L1_NORM, HAMMING...)?

You may find the Find-Object project easier to debug this kind of stuff, as there are many more possibilities to compare on runtime these different combinations. You can also use the InvertedSearch example to debug, which is a ripoff of RTAB-Map loop closure detection.

cheers,
Mathieu