Save post-processed database

[You Li]

Hi,

I am trying to obtained a 2D occupancy grid map with this database:
https://www.dropbox.com/s/xg6l1lt2b89l1em/rtabmap_EEEL2.db?dl=0

This is a very challenging area where the majority of wall is made by glass.

By using the commands
$ roscore
$ rosrun rtabmap_ros rtabmap _database_path:=rtabmap_EEEL2.db
$ rosrun map_server map_saver map:=proj_map
$ rosservice call /publish_map 1 1 0

I obtained a map


There are some drifts in the result. By post-processing in RTABMAP, I can obtain the result


Seems with less drifts.

So mu questions is that how can I save the post-processed database? Then it is possible to use the modified database to generate the 2D map.

The graph view is actually looks good after post-processing:



Thanks,
You Li

[matlabbe]

Hi,

The Post-Processing option in rtabmap app won't modify the database. If you do Edit->Download Graph-only, it will reset the changes from post-processing. To save modifications to database you should use rtabmap-databaseViewer. You would find the same post-processing options under the Edit menu, and when you close the database you will be asked if you want to save the modified links to database.

Note however that I opened your database without any post-processing and I have already your latest result:


Doing your command lines above:
map.pgm
map.yaml

[You Li]

Thanks Mathieu.

I used the rtabmap-databaseViewer to saved the modified database.

As for the generation of the 2D map, when I change a laptop, I obtained the same 2D map as yours, which seems the same to the graph view.
However, for my own laptop, it provided me the figure similar as my previous one (with some drifts). This is strange. Do you have idea of which file in rtabmap is called for creating the occupancy grid map?

Thanks,
You

[matlabbe]

The difference between the two computers would be that they don't use the same graph optimization approach (either g2o or gtsam is not installed so another approach is used instead). If the links are exactly the same, it should be the graph optimization that is different.

For example, I tried GTSAM with 10 max iterations and this what I get:



cheers,
Mathieu

[You Li]

Thank Mathieu! You are right. This machine did not have GTSAM. Now the issue has been solved by installing GTSAM.

I collected another dataset in this area. For this test, I used the following commands:
$ roslaunch kinect2_bridge kinect2_bridge.launch publish_tf:=true
$ roslaunch rtabmap_ros rgbd_mapping_kinect2_horizontal.launch

rgbd_mapping_kinect2_horizontal.launch

The real-time result by using kinect v2 only is amazing to me consider the environment:


However, when I tried to use the same dataset to obtain the same result in post-processing, I met two issues (here is the database: https://mega.nz/#!g8IEAJjQ!Q4RtQTSfySWv4ddyD3EVsJ6Y3_zCcuTb2_lzzCtKhqY):
1. I run the database in RTABMAP, and finally the process completed at follows:


I believe all the mapping results are in the memory, because if I run other dataset, the previous mapping result may appear when a loop is detected. However, no matter I click "Download graphs only" or "Download all clouds", I could not show the full mapping results. So is there anything I missed to get the whole mapping result from this?


2. I run the above database as well as this one (https://mega.nz/#!YgoASKhT!H4leMDsdbNJ9z3SKNc-vhXGPlsIPPmwOYFlECJxA-0g) as the second database. However, the result is not as reliable as the ones I obtained in real time. Here is the snapshot of the final result:


And here is a video that records the data processing process:
https://mega.nz/#!04oxVACK!zcNl7rnCRXwuSx5czADjK6BfOEO_lMvAsAUHbITMQ8I

I directly used the default setting of RTABMAP (by clicking "reset all settings"), and changed "Graph Optimization" to "TORO" (because "g2o" missed many loop detections, and led to drifts; "GSTAM" met some calculation problem). Also, the 2D transform was forced.

So it is kind of strange that the post-processing result is even worse than the real-time one. Do you have any suggestions on this dataset?

By the way, if there is a setting of multiple times speed for data processing, that would be great.

Best regards,
You Li

[You Li]

Hi,

When I am replaying the .db file, I still could not display earlier mapping results. When the mapping environment is challenging, the data replaying process looks like a combination of multiple databases. I followed the tutorial here:
https://github.com/introlab/rtabmap/wiki/Multi-session
 
Sometimes if a loop detection has been detected, the related previous mapping results will be shown. Otherwise, only the current piece of mapping result is shown, while the previous ones disappeared.

According to tutorial, when one pauses the data processing, and select Edit->"Download graph only", the previous results should appear. However, this function does not work on the two laptops I have (one Ubuntu 16.04 with RTABMAP 0.11.12, the other OSX with RTABMAP 0.11.11). Because of this issue, it is impossible to obtain the whole mapping results after the replaying, if the mapping process stops at a challenging area.

Did anyone who met this issue before?

Best regards,

[matlabbe]

Hi,

Just tried your databases. Did you manually reset odometry while mapping? When processing the database, there are a lot of "RtabmapThread.cpp:575::addData() Odometry is reset (identity pose or high variance >=9999 detected). Increment map id!" warnings. A new map is created each time, which makes the old maps disappear unless a loop closure is found when them. Clicking on "Download graph" would not make old maps reappear if they are not linked to last map by loop closures.

So if you didn't reset the odometry (or used Odom/ResetCountdown to automatically reset odometry when lost), there is maybe a bug in rgbd_odometry or rtabmap nodes that generates high variance in odometry links.

Well, to remove this "new map" effect in your already created databases, change all 9999 values in the database to 1, you will then have only one map per database:

sqlite3 East.db "update link set rot_variance=1, trans_variance=1 where rot_variance=9999;"

To process databases faster, you should uncheck "Use database stamps as input rate." in Source panel under Database section. The source Input rate at the top of the panel will be used instead, e.g., I used 4-5 Hz so that processing again the database is 4/5x faster than real-time (if the map was updated at 1 Hz). In RTAb-Map settings, make sure to set Detection rate to 0 and buffer size to 0 to process all frames in database.

cheers,
Mathieu

[You Li]

Thanks Mathieu,

Running
sqlite3 East.db "update link set rot_variance=1, trans_variance=1 where rot_variance=9999;"
makes it possible to get all the previous data.

About the original "RtabmapThread.cpp:575::addData() Odometry is reset (identity pose or high variance >=9999 detected). Increment map id!" warnings. warnings, I did not reset odometry manually. What I did was running
$ roslaunch kinect2_bridge kinect2_bridge.launch publish_tf:=true
$ roslaunch rtabmap_ros rgbd_mapping_kinect2_horizontal.launch

to collect the data.
Here is rgbd_mapping_kinect2_horizontal.launch:
rgbd_mapping_kinect2_horizontal.launch

Best regards,

[matlabbe]

Thx for the follow-up, I could reproduce the problem. This is now fixed.

cheers,
Mathieu

[You Li]

Thank a lot Mathieu.

[alex_mk]

Dear Mathieu,

I am also having some trouble to understand post processing so I would be glad if you could provide some directions.

1) when I try to correct the path by loop closing using the database viewer, I do Edit->Update all neighbor covariance->0.05[m]/5.0[deg]
The newly processed map can’t be loaded by the robot. But by using database recovery it can be loaded. Are we doing something wrong here?

2) I am also using the command line to add more loop closures by executing “rtabmap-detectMoreLoopClosures”. The output map can be loaded by our robot. However, by adding loop closures with the database viewer, the map can’t be loaded. What would be cause? Is there any difference between using the command line or database viewer?

3) I want to ignore previously saved locations in the launch, so I want to set RGBD/SavedLocalizationIgnored to true. When I set directly in rtabmap and save, the database cant be loaded by the robot. According to your answer, we can only save parameters with the database viewer, so I assume we are doing something wrong. However, even by setting this parameter in a launch file, the database still can’t be loaded by the robot. Are we missing something here?
By the way, this parameter is not available in the database viewer, so should I assume that all variables not available should be set by a launch file?

4) Another parameter is VisMinInliers. I assume it changes both mapping and localization mode. But supposing that I want to change this parameter in localization mode, what is the best way to do that? By databaseviewer or launch file?

5) As you may have noticed, I am having trouble to understand when and where we can set the parameters. Is there any documentation about it?
For example, imagine I have a database but want to use the images with a completely different feature extractor to create a new map. Is it possible? How to do that?

sorry for the many questions,
best regards

[alex_mk]

*The version we are using is v0.19.3

We would appreciate any advice on this matter. Thanks