This post was updated on .
Hello Mathieu and fellow rtabmap-users.
I am interested in converting Rtabmap datasets into Bundler format datasets, particularly bundle.out files that should contain the 3D information according to the Bundler documentation: http://www.cs.cornell.edu/~snavely/bundler/bundler-v0.4-manual.html I saw that Rtabmap allows exporting camera poses in Bundler format but I somehow fail to find any way in Bundler to import these camera poses. I guess, I would still have to re-run a big chunk of Bundler pipeline to recreate the 3D data even when the poses would be known. So my main question is: is there any way in Rtabmap to export the 3D scene information in bundle format, i.e. as bundle.out? And if no, what is the intended way to use exported Bundler-poses and whether they can be somehow used in Bundler processing |
Administrator
|
This function has been only tested with MeshLab. This tutorial may be a little out of date (export dialog has changed), but it shows how the bundler format can be used for. Originally, we added the bundler format to export poses and images to MeshLab in order to use its mesh texturing tools. We didn't tested the compatibility with Bundler.
cheers, Mathieu |
This post was updated on .
Thank you Mathieu, it makes sense.
I guess, I will have to write some code myself then :) could you hint me into a right direction perhaps? I'd like to use the Rtabmap datasets with the ACG Localizer (aka Active Search). The ACG Localizer expects the data in Bundler format (luckily it's all ASCII, so should not be hard to reverse-engineer), namely it expects a list of images, a list of SIFT feature descriptors in every image and finally a list of 3D points, while for every 3D point the following is stored: 3D coordinates, color in RGB, list of frame indices it is visible in, and the indices of the SIFT features which correspond to the 3D point in the views. edit: so I have actually implemented a big part of it. I'd be thankful if you could correct me if I am wrong somewhere. I found almost all the data in the Map_Node_Word table: 1) depth_x, depth_y, depth_z are the 3D coordinates of the word in the 3D space. The RGB color is missing here, but I can retrieve it from the actual image file by getting the x,y coordinate. 2) by node ID I can retrieve the camera pose and frame indices where the feature is visible 3) by word ID I can retrieve the descriptor. - There is something that I don't get yet: one word ID can have multiple different descriptor values for different node IDs - I guess you store here the descriptors of the same feature seen from different perspectives. Then what is stored with the same ID in the Words table? Is it some averaged feature descriptor vector? Just for the case I attach the description straight from the bundler's documentation.
|
Administrator
|
We cannot directly translate RTAB-Map database to bundler format without some processing. It is because a word in the dictionary can be linked to more than one 3D point. We could get all <pointN> using Optimizer::computeBACorrespondences() using "points3DMap" for the 3D points and "wordReferences" for the list of frame references and keypoint positions. To export SIFT descriptors linked to correct Ids, that function would have to return the corresponding descriptors of the 2D keypoints (maybe adding a cv::Mat field in "wordReferences" for the descriptor). Ideally, we would have to do a global bundle adjustment before the export. To get the color for each point, we would have to uncompress the corresponding frame. I'll try to code something today and let you know. For your edit: Map_Node_Word table is now called Feature in recent versions (since 0.13.0). The descriptor in Word table is the first descriptor seen for the word (in case multiple features link to it). Map_Node_Word/Feature table contains descriptor of each features extracted from the referred frame. Multiple features can be linked to same word. Each feature has a 2d keypoint and the corresponding 3d point (in ROS coordinates, not image coordinates). The 3D points are not 3D map points like in feature-based visual SLAM approaches, they are just 3D points of that frame. cheers, Mathieu |
Administrator
|
I implemented a first version. Update rtabmap to have the latest commits. When exporting in bundler format, a new checkbox can be checked to export 3D points and all descriptors. Obviously, to export SIFT descriptors we should set the vocabulary to use SIFT (and uncheck use odometry features). I cannot test really the output files to know if everything is fine (like if exported descriptor format called "*.key" is okay), though in MeshLab I can see the 3D points with the cameras.
cheers, Mathieu |
wow incredible, thank you very much. I will test it right away just after I buy you a coffee or/and a beer!
|
Administrator
|
This post was updated on .
Thank you for the beer(s)! If you have problems with the export format or you can get a small example working with the ACG Localizer, let me know.
Mathieu |
Awesome, Mathieu. I can use the generated data directly in the ACG Localizer and it successfully localizes images. Thanks a lot!
|
This post was updated on .
In reply to this post by matlabbe
Hey Mathieu,
I have pulled the most recent version of Rtabmap and some other libraries (e.g. latest opencv 3.4) and found that the SIFT features exported via the Bundler exporter are 64-dimensional instead of 128. Does this change come from opencv or can the length be set in rtabmap somewhere? Edit: ok, nevermind, it's probably the opencv version. I think 3.4.11 already contains SIFT without opencv_contrib package and without the nonfree flag. Rtabmap also detects it and allows selecting it in the settings, but during runtime some check prevents SIFT features from being extracted. |
Administrator
|
Thx, good observation. This should be fixed by this commit: https://github.com/introlab/rtabmap/commit/a4da1e14b4e6ab9e9bd69a9875fb679a58d51f74
|
Free forum by Nabble | Edit this page |