Re: stereo_odometry process has died, exit code -11 , when using Superpoint and Superglue

Posted by matlabbe on
URL: http://official-rtab-map-forum.206.s1.nabble.com/stereo-odometry-process-has-died-exit-code-11-when-using-Superpoint-and-Superglue-tp10112p10122.html

I reproduced the issue. Just to make sure, only stereo_odometry is crashing when setting superglue, not rtabmap (if also set with superglue)?

The issue is that the python interpreter is not initialized in stereo_odometry node, so all calls to python will crash. We have to add:

#include <rtabmap/core/PythonInterface.h>

int main(int argc, char **argv) {
   rtabmap::PythonInterface pythonInterface;
   ...

to stereo_odometry node to make it not crashes. Note however that superglue would be ignored if you don't set also Vis/CorGuessWinSize to 0. Set also Reg/RepeatOnce=false to avoid overriding superglue matches with local ones. I added the fix in this commit. One remaining issue is that it won't work if you launch the stereo_odometry as a nodelet.

<param name="Kp/DetectorStrategy" type="string" value="11"/>
<param name="Vis/FeatureType" type="string" value="11"/>
<param name="Vis/CorGuessWinSize" type="string" value="0"/>
<param name="Reg/RepeatOnce" type="string" value="false"/>
<param name="Vis/CorNNType" type="string" value="6" />  <!-- SuperGlue -->
<param name="PyMatcher/Path" type="string" value="/workspace/SuperGluePretrainedNetwork/rtabmap_superglue.py" />
<param name="SuperPoint/ModelPath" type="string" value="/workspace/superpoint_v1.pt" />

A workaround is to not use superglue for stereo_odometry, only for rtabmap node.

For stereo_odometry:
<param name="Kp/DetectorStrategy" type="string" value="11"/>
<param name="Vis/FeatureType" type="string" value="11"/>
<param name="SuperPoint/ModelPath" type="string" value="/workspace/superpoint_v1.pt" />

For rtabmap:
<param name="Kp/DetectorStrategy" type="string" value="11"/>
<param name="Vis/FeatureType" type="string" value="11"/>
<param name="Vis/CorGuessWinSize" type="string" value="0"/>
<param name="Reg/RepeatOnce" type="string" value="false"/>
<param name="Vis/CorNNType" type="string" value="6" />  <!-- SuperGlue -->
<param name="PyMatcher/Path" type="string" value="/workspace/SuperGluePretrainedNetwork/rtabmap_superglue.py" />
<param name="SuperPoint/ModelPath" type="string" value="/workspace/superpoint_v1.pt" />

I tested the workaround with this docker image (mounting the modified demo_stereo_outdoor.launch with changes above in the container):

XAUTH=/tmp/.docker.xauth
touch $XAUTH
xauth nlist $DISPLAY | sed -e 's/^..../ffff/' | xauth -f $XAUTH nmerge -

docker run --gpus all -it --rm --ipc=host --runtime=nvidia  \
       --env="DISPLAY=$DISPLAY" \
       --env="QT_X11_NO_MITSHM=1"  \
       --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw"   \
       --env="XAUTHORITY=$XAUTH"  \
       --volume="$XAUTH:$XAUTH"  \
       --network host \
       --privileged \
       -v ~/catkin_ws/src/rtabmap_ros/rtabmap_demos/launch:/opt/ros/noetic/share/rtabmap_demos/launch \
       rtabmap_ros:superpoint roslaunch rtabmap_demos demo_stereo_outdoor.launch rtabmap_viz:=true

rosbag play --clock ~/Downloads/stereo_outdoorA.bag

This also helped me to fix the same issue here: https://github.com/introlab/rtabmap/issues/1273

cheers,
Mathieu