Matching during reprocessing

[Pierre]

Hi Mathieu,

I've recently tried using Superglue when reprocessing a database. When Superglue is used, we can see it on the log, as a message says that the weights have been loaded. I was surprised to see that this "loading message" did not appear so often during the reprocessing: only a couple of times, although many loops have been closed (Loop or Proximity Loop on the log). Hence I was wondering, what am I missing ? On the other hand, when I use the detectMoreLoops tool, Superglue is indeed loaded every time (by the way I was wondering if there was a way to load the weights once and for all, as detectMoreLoops is unbelievably long with Superglue on, knowing that I have databases with close to 100k nodes).

Is this in link with the Kp/NNStrategy parameter ? I don't know how it is used but it suggests that some matching can be done without Superglue.

My parameters are (among others) the following:

Kp/DetectorStrategy 11
Vis/FeatureType 11
Vis/CorNNType 6

I thank you a lot for you help
Best
Pierre

[matlabbe]

Hi Pierre,

When using SuperGlue, we should use these parameters:

Vis/CorNNType 6 
SuperGlue/Path SuperGluePretrainedNetwork/rtabmap_superglue.py 
Reg/RepeatOnce false 
Vis/CorGuessWinSize 0 
Kp/DetectorStrategy 11 
Vis/FeatureType 11

If Vis/CorGuessWinSize is not 0 and Reg/RepeatOnce is true, visual registration may ignore SuperGlue and do NN strategy defined by Kp/NNStrategy instead. If proximity detection is done, Kp/NNStrategy is also done instead of SuperGlue. By setting Vis/CorGuessWinSize=0 and  Reg/RepeatOnce=false, you make sure that SuperGlue is always used for feature matching.

Unless the parameters are always changing or updated, SuperGlue should be constructed only once. Normally "SuperGlue python init()" message would appear once during initialization. I uncommented that log ("SuperGlue python match()") to see when superglue is called.

I gave it a try with some database to see if what I just described is actually doing. Here testing rtabmap-reprocess:

$ rtabmap-reprocess \
   --Mem/UseOdomFeatures false \
   --Vis/CorNNType 6 \
   --SuperGlue/Path ~/workspace/SuperGluePretrainedNetwork/rtabmap_superglue.py \
   --Reg/RepeatOnce false \
   --Vis/CorGuessWinSize 0 \
   --Kp/DetectorStrategy 11 \
   --Vis/FeatureType 11 \
   --SuperPoint/ModelPath ~/workspace/SuperPointPretrainedNetwork/superpoint_v1.pt \
   rtabmap.db rtabmap_superpoint_superglue.db

<<<<< log >>>>
Custom parameters:
  Kp/DetectorStrategy =     11
  Mem/UseOdomFeatures =     false
  PyMatcher/Path =          /home/mathieu/workspace/SuperGluePretrainedNetwork/rtabmap_superglue.py
  Reg/RepeatOnce =          false
  SuperPoint/ModelPath =    /home/mathieu/workspace/SuperPointPretrainedNetwork/superpoint_v1.pt
  Vis/CorGuessWinSize =     0
  Vis/CorNNType =           6
  Vis/FeatureType =         11
Set working directory to ".".
Target database version: "0.21.14" (set explicitly --Db/TargetVersion "" to output with latest version.
Reprocessing data of "rtabmap.db"...
High variance detected, triggering a new map...
Processed 1/39 nodes [id=1 map=0 opt_graph=1]... 350ms
Processed 2/39 nodes [id=2 map=0 opt_graph=1]... 23ms
Processed 3/39 nodes [id=3 map=0 opt_graph=2]... 25ms
Processed 4/39 nodes [id=4 map=0 opt_graph=3]... 23ms
Processed 5/39 nodes [id=5 map=0 opt_graph=4]... 22ms
Processed 6/39 nodes [id=6 map=0 opt_graph=5]... 21ms
Processed 7/39 nodes [id=7 map=0 opt_graph=6]... 20ms
Processed 8/39 nodes [id=8 map=0 opt_graph=7]... 20ms
Processed 9/39 nodes [id=9 map=0 opt_graph=8]... 20ms
Processed 10/39 nodes [id=10 map=0 opt_graph=9]... 21ms
Processed 11/39 nodes [id=11 map=0 opt_graph=10]... 19ms
Processed 12/39 nodes [id=12 map=0 opt_graph=11]... 21ms
Processed 13/39 nodes [id=13 map=0 opt_graph=12]... 21ms
Processed 14/39 nodes [id=14 map=0 opt_graph=13]... 16ms
Processed 15/39 nodes [id=15 map=0 opt_graph=14]... 17ms
Processed 16/39 nodes [id=16 map=0 opt_graph=15]... 19ms
Processed 17/39 nodes [id=17 map=0 opt_graph=16]... 23ms
Processed 18/39 nodes [id=18 map=0 opt_graph=17]... 20ms
Processed 19/39 nodes [id=19 map=0 opt_graph=18]... 22ms
Processed 20/39 nodes [id=20 map=0 opt_graph=19]... 23ms
Processed 21/39 nodes [id=21 map=0 opt_graph=20]... 23ms
Processed 22/39 nodes [id=22 map=0 opt_graph=21]... 23ms
Processed 23/39 nodes [id=23 map=0 opt_graph=22]... 22ms
Processed 24/39 nodes [id=24 map=0 opt_graph=23]... 21ms
Processed 25/39 nodes [id=25 map=0 opt_graph=24]... 19ms
Processed 26/39 nodes [id=26 map=0 opt_graph=25]... 18ms
SuperGlue python init()
Loaded SuperGlue model ("indoor" weights)
SuperGlue python match()
[ WARN] (2025-05-11 20:15:40.170) Rtabmap.cpp:3069::process() Rejected loop closure 15 -> 27: Not enough inliers 0/20 (matches=12) between 15 and 27
Processed 27/39 nodes [id=27 map=0 opt_graph=26]... 158ms
SuperGlue python match()
[ WARN] (2025-05-11 20:15:40.218) Rtabmap.cpp:3069::process() Rejected loop closure 15 -> 28: Not enough inliers 0/20 (matches=0) between 15 and 28
Processed 28/39 nodes [id=28 map=0 opt_graph=27]... 42ms
SuperGlue python match()
Processed 29/39 nodes [id=29 map=0 opt_graph=28]... 44ms
SuperGlue python match()
Processed 30/39 nodes [id=30 map=0 opt_graph=29]... 61ms Prox on 5 [0]
SuperGlue python match()
Processed 31/39 nodes [id=31 map=0 opt_graph=30]... 49ms Loop on 4 [0]
SuperGlue python match()
Processed 32/39 nodes [id=32 map=0 opt_graph=31]... 52ms Loop on 4 [0]
SuperGlue python match()
Processed 33/39 nodes [id=33 map=0 opt_graph=32]... 49ms Loop on 4 [0]
SuperGlue python match()
Processed 34/39 nodes [id=34 map=0 opt_graph=32]... 54ms Loop on 4 [0]
SuperGlue python match()
Processed 35/39 nodes [id=35 map=0 opt_graph=32]... 49ms Loop on 4 [0]
SuperGlue python match()
Processed 36/39 nodes [id=36 map=0 opt_graph=33]... 51ms Loop on 4 [0]
SuperGlue python match()
Processed 37/39 nodes [id=37 map=0 opt_graph=33]... 51ms Loop on 4 [0]
SuperGlue python match()
Processed 38/39 nodes [id=38 map=0 opt_graph=34]... 50ms Loop on 4 [0]
SuperGlue python match()
Processed 39/39 nodes [id=39 map=0 opt_graph=34]... 52ms Loop on 4 [0]
Total loop closures = 10 (Loop=9, Prox=1, In Motion=10/39)
Closing database "rtabmap_superpoint_superglue.db"...
Closing database "rtabmap_superpoint_superglue.db"... done!

"SuperGlue python init()" should be called only once when a first matching request is done. Here for rtabmap-detectMoreLoopClosures:

$ rtabmap-detectMoreLoopClosures rtabmap_superpoint_superglue.db

<<<<< log >>>>
Database: stereo_20Hz_map_suprepoint.db
Cluster radius min = 0.000000 m
Cluster radius max = 1.000000 m
Cluster angle = 30.000000 deg
Initialization...
Detecting...
SuperGlue python init()
Loaded SuperGlue model ("indoor" weights)
SuperGlue python match()
Iteration 1/1: Added loop closure 1->4! (2/154) 
SuperGlue python match()
Iteration 1/1: Added loop closure 3->5! (11/154) 
SuperGlue python match()
Iteration 1/1: Added loop closure 6->38! (43/154) 
SuperGlue python match()
SuperGlue python match()
SuperGlue python match()
SuperGlue python match()
SuperGlue python match()
Iteration 1/1: Added loop closure 30->36! (96/154) 
SuperGlue python match()
Iteration 1/1: Added loop closure 31->33! (106/154) 
Iteration 1/1: Detected 5 total loop closures!

Here again, it does what is expected. I don't know if you could show similar logs on your side to see if "SuperGlue python init()" is called on every matching call.

cheers,
Mathieu

[Pierre]

Hi Mathieu,

Thank you very much for your answer. I'm now using the appropriate parameters !

For the detectMoreLoops tool, I tried with the CLI and it does work like you said. I was using the one provided by the databaseViewer GUI and that ones reloads weights everytime there is a matching.

Example of a log sample:

[ INFO] (2025-05-12 14:39:31.299) DatabaseViewer.cpp:4268::detectMoreLoopClosures() iteration 1/10 
[ INFO] (2025-05-12 14:39:31.316) PyMatcher.cpp:35::PyMatcher() path = [...]
[ INFO] (2025-05-12 14:39:31.316) PyMatcher.cpp:36::PyMatcher() model = indoor 
SuperGlue python init()
Loaded SuperGlue model ("indoor" weights)
SuperGlue python match() 
[ INFO] (2025-05-12 14:39:33.013) RegistrationVis.cpp:1802::computeTransformationImpl() Not enough inliers 14/25 (matches=44) between 7 and 1 
[ INFO] (2025-05-12 14:39:33.015) PyMatcher.cpp:35::PyMatcher() path = [...]
[ INFO] (2025-05-12 14:39:33.015) PyMatcher.cpp:36::PyMatcher() model = indoor 
SuperGlue python init()
Loaded SuperGlue model ("indoor" weights)
SuperGlue python match() 
[ INFO] (2025-05-12 14:39:33.462) RegistrationVis.cpp:1802::computeTransformationImpl() Not enough inliers 20/25 (matches=74) between 6 and 1 
[ INFO] (2025-05-12 14:39:33.474) PyMatcher.cpp:35::PyMatcher() path = [...]
[ INFO] (2025-05-12 14:39:33.474) PyMatcher.cpp:36::PyMatcher() model = indoor 
SuperGlue python init()
Loaded SuperGlue model ("indoor" weights)
SuperGlue python match() 
[ INFO] (2025-05-12 14:39:33.861) PyMatcher.cpp:35::PyMatcher() path = [...]
[ INFO] (2025-05-12 14:39:33.861) PyMatcher.cpp:36::PyMatcher() model = indoor 
SuperGlue python init()
Loaded SuperGlue model ("indoor" weights)
SuperGlue python match() 

I'll use the CLI version from now on.

Best,
Pierre

-------------------------------------------------------
EDIT: the CLI version uses way more ram than the GUI one, in the same way that rtabmap uses lots of ram while rtabmap-databaseViewer doesn't. Sadly I've got some databases that reach up to 200gb.

Is there a reason for the CLI to use that much more ram than the GUI version ? In which case, would it be possible, maybe, to make the GUI tool load the superglue weights once an for all like the CLI does, so that I can detect more loops on my huge databases ?

Thanks a lot
Pierre

[matlabbe]

Hi Pierre,

Thanks for the feedback. The GUI is not using exactly the same code than the CLI tool. I updated the GUI to avoid re-initializing SuperGlue all the time.

For the CLI, it is using more RAM because it actually reloads the whole map (all visual descriptors) before doing the detection, instead of loading descriptors "on-demand" when add/refining a link and clearing them from RAM afterwards. That is indeed not convenient when managing large databases. I created an issue about that.

cheers,
Mathieu

[Pierre]

Hi Mathieu,

Thanks a lot for the quick update !

Best
Pierre