Superpoint for vSLAM feature detection

[nathgilson]

Hi Mathieu,

Superpoint from Magicleap seems to be particularly effective for detecting consitent features outdoors. According to Daniel DeTone, it makes possible localization despite weather and brightness variations.

Have you tried to integrate it already? Do you think it could be a good lead for outdoor navigation?

[matlabbe]

Hi,

Are you referring to this repo: https://github.com/MagicLeapResearch/SuperPointPretrainedNetwork ?

I didn't try to integrate it, well I wasn't aware of it :P It looks interesting! The code is in python though, would be nice to have a c++ API with the same interface than Feature2D class in OpenCV. We could then directly add the option along SURF/SIFT/ORB/... options supported by RTAB-Map. If you find a c++ implementation (compatible with OpenCV's Feature2D class), I could check to integrate it.

cheers,
Mathieu

[nathgilson]

Hi Mathieu,

Yes this repo (the pre-trained model of Superpoint).
There is no c++ API, but I can try to implement it!

Cheers,

Nathan

[Eric Schleicher]

it's been a while.   and i noticed that in my cmake output there is a supertpoint reference.  is this something worth trying out? or was it added in a purely experimental way?

[matlabbe]

SuperPoint is not super fast, requires GPU/CUDA to work faster (still around 300 ms) but it is very robust to illumination changes. 300 ms is too slow for odometry (unless you move very slowly), but it can work well for loop closure detection (rtabmap is 1 Hz by default). For example, it helped a lot to find loop closures in the forest environment shown in this video, where sunlight and shadows changed between the beginning and ending of the first loop. I didn't test all available features in RTAB-Map, but I think only SuperPoint was able to close that loop (and without any tuning).

How building RTAB-Map with SuperPoint support: https://github.com/introlab/rtabmap/issues/531

Regards,
Mathieu