Adding more loop detection in post-processing

[You Li]

Hello,

I collected some data, and got a solution like this


This is one the most typical issue I met when using the software. Specifically, seems that the software could not detect the loops. Could we do something in the post-processing process, to add more loop detection and get a smooth result?

If you would like to run the data, here it is
https://www.dropbox.com/s/buvkmuhf3x2glbc/20170206103621.db?dl=0

Thanks in advance.
You Li

[matlabbe]

Hi,

Thx for sharing such a big database in an office environment. With the default parameters from RTAB-Map Tango 0.11.13, this environment is quite challenging because most of the images don't have much visual features (a lot of of solid color office's desks/walls with no discriminative features) and they are often blurry because of the rotation (which cause problem to get good local features). However, like what I've done in this post for a scan in a natural environment, here some parameters I changed to reprocess the database with more visual features for loop closure detection.

For the vocabulary, use 400 features instead of 200:


For loop closure transformation computation, activate re-extraction and use 1000 features to get more matches:



Result:


A comparison with and without loop closures:



When exporting mesh (Dense Flavor -> Gain Compensation + Poisson depth 9 + texture):



Hope this will help. With DatabaseViewer, we can also manually add loop closures if needed (Constraints View).

cheers,
Mathieu

[HTDcris]

Bookmarking this for future memory.
Mathieu, you should definitely collect all these quality answers in a tutorial.

[You Li]

Hi Mathieu,

Many thanks for providing such detailed instructions! Your result looks like magic.
However, I tried on my machines (one Ubuntu 16.04 LTS and one OSX 11.12, both with rtabmap 0.11.13), both have a issue with optimizeGTSAM. Here is the issue information

[ERROR] (2017-02-07 10:25:35.026) OptimizerGTSAM.cpp:265::optimize() GTSAM exception caught:
Indeterminant linear system detected while working near variable
2123 (Symbol: 2123).

Thrown when a linear system is ill-posed.  The most common cause for this
error is having underconstrained variables.  Mathematically, the system is
underdetermined.  See the GTSAM Doxygen documentation at
http://borg.cc.gatech.edu/ on gtsam::IndeterminantLinearSystemException for
more information.
[ERROR] (2017-02-07 10:25:35.029) Rtabmap.cpp:3062::optimizeCurrentMap() Failed to optimize the graph! returning empty optimized poses...

Kind of like the problem in http://official-rtab-map-forum.206.s1.nabble.com/GTSAM-Optimization-Error-td742.html

If you would like try to reproduce the issue, here is my config file.
config.ini

I also tried TORO and g2o, instead of GTSAM in "Graph Optimization" , they did not suffer from this error; however, with the configuration and simply changed the "GTSAM" to TORO or g2o, the loops still were not detected.

Could TORO reach the similar performance as you have shown?

Thanks,
You Li

[matlabbe]

Hi,

I tried on Ubuntu 14.04 (latest code) and Mac Os X (0.11.13 binaries), and I don't have the GTSAM error. I used your config file and it worked like in my previous post. I tried with TORO, but I needed to set max graph optimization error to 0:
 

Here is an example of one of the first loop closures to happen after the big loop (until there there were already about 69 loop closures detected, 92 rejected):



Do you see at least some green screens like above sometimes?

cheers,
Mathieu

[You Li]

Thank you Mathieu! The post-processing works amazing good with the office data by changing the max graph optimization error to 0 in TORO. Yes, now I can see the green screens quite often with this setting, and the pose is similar to yours above. Similarly, an excellent loop detection and optimization at about ID = 1600.

I am pretty curious about  how you could totally filter the roof, but can keep the walls at the same time? As you see, here is my solution, lots of ceiling points. If I set a small parameter for the "cloud filtering", the ceiling can be removed; however, the walls are missed as well.





Cheers,
You Li

[matlabbe]

Hi,

Glad to see it is working on your side now. I did a Poisson surface reconstruction when exporting from RTAB-Map (mesh instead of a point cloud) with the parameters indicated over the screenshot. The ceiling is still actually there, but not shown because the polygons are inverted. That kind of reconstruction also fill the holes like windows.

EDIT: The animated screenshot is the map in 3D Map view using simple mesh reconstruction too (with backface culling), see Preferences->3D Rendering->Organized Online Mesh.

cheers,
Mathieu

[You Li]

Hi Mathieu,

Thanks for sharing the information of meshing! I updated the rtabmap to 0.11.14 on Linux, and outputed the meshed files for several times. Here is one typical result I obtained, the roof is still there. I guess something has been missed. Here is one result





And here is my config when outputing







With this configuration and when I check the option "Gain Compensation", an error occurred
[pcl::KdTreeFLANN::setInputCloud] Cannot create a KDTree with an empty input cloud!
[pcl::KdTreeFLANN::setInputCloud] Cannot create a KDTree with an empty input cloud!
[pcl::KdTreeFLANN::setInputCloud] Cannot create a KDTree with an empty input cloud!
Failed to find match for field 'normal_x'.
Failed to find match for field 'normal_y'.
Failed to find match for field 'normal_z'.
Failed to find match for field 'curvature'.
rtabmap: /build/pcl-6_P28C/pcl-1.7.2/kdtree/include/pcl/kdtree/impl/kdtree_flann.hpp:172: int pcl::KdTreeFLANN<PointT, Dist>::radiusSearch(const PointT&, double, std::vector<int>&, std::vector<float>&, unsigned int) const [with PointT = pcl::PointXYZRGBNormal; Dist = flann::L2_Simple<float>]: Assertion `point_representation_->isValid (point) && "Invalid (NaN, Inf) point coordinates given to radiusSearch!"' failed.
Aborted (core dumped)

[matlabbe]

Hi,

I added the assert error in this issue. You can uncheck "Cloud Filtering" to avoid the error for now.

It is okay, I have the ceiling too in the model of my screenshots. What app are you using to visualize the mesh? I used MeshLab with "backface culling" activated to hide the polygons on the ceiling.

cheers,
Mathieu

[You Li]

Thanks Mathieu, I am also using meshlab. With  "backface culling" activated, the roof is hiden. The mesh looks great!

[You Li]

Hi Mathieu,

Thanks for your information that we could add loops manually by using Database viewer. I selected a dataset which has some drifts, and tried to add a loop closure manually. Here is a picture of the process.



I opened the dataset by using
rtabmap-databaseViewer 20170216044718-processed.db
Then choose the Constraints view, find the to image I would like to match, and click the "Add" in the lower right corner. Then, I was told that the software cannot find a transformation between these nodes.

Is there any instruction on how to add loop closures manually, to repair a database? Thanks in advance.

Here is the data (https://www.dropbox.com/s/c1un05hi5h3nrv6/20170216044718-processed.db?dl=0), and here is the config file config-10hz.ini I used to process the raw data and obtain the above database.

Cheers,
You Li

[matlabbe]

Hi,

You are doing right. The problem here is that depth values are almost null in most camera frames (see screenshot below where I show the depth over RGB). Without depth, 3D features cannot be computed. Even if there are features on the right, they are not useful features because they are mostly on a uniform texture (not discriminative), so difficult to find good correspondences.


cheers,
Mathieu

[You Li]

Thanks Mathieu, you are right. If this is the case, do you have any idea on correcting this result (pose and mapping results with drifts)? For image data, it seems not straightforward to use. On the other hand, if I have can post-processing the pose data to get a better pose information (e.g., time, position, and orientation), or I have a known pose data, it is possible to feed such data into RTABMAP, and use it as constraints? It should also be a good approach, as tuning pose is easier than tuning on image and point clouds.

[matlabbe]

Hi,

Not sure if manually changing poses in the graph would be easier. I think the best is still to make sure to scan again the same discriminative locations with valid depth values (e.g., billboard on the wall, similar to what I did in this video at 1:57) regularly during mapping to find loop closures, thus correcting the drift. If loop closures on these locations are still not found online, they could be easily added manually with the database viewer afterward (if we can extract a lot of discriminative visual features with valid depth).

cheers,
Mathieu

[You Li]

Thanks Mathieu, this makes sense. I will try this approach later. Seems that you had intentionally kept the camera static towards the distinct feature for several seconds in the video around 1:50? Could I interpret it as that such a period is essential, or at least good for the on-line or post-processing.  

Cheers,
You Li

[matlabbe]

Yes, the loop closure hypotheses need some consecutive images to increase enough to reach the loop closure threshold. In general, revisiting the same sequence of images is better to find loop closures. On the Debug overlay for the Tango App, you can see the current loop closure hypothesis value. When reaching the default threshold 0.11 (dark green background), a transformation is computed, then if accepted (e.g., enough correspondences), the loop closure is detected (light green background).

cheers,
Mathieu

[You Li]

Thanks Mathieu, this is really useful!
Best regards,
You Li