Exporting Poses with Timestamps and Consistent Keyframes for AR Integration

[kekzzz]

Description:
We are working on integrating RTAB-Map into a mixed/augmented reality (AR) application and require the ability to export camera poses with associated timestamps and consistent keyframes. This functionality is crucial for synchronizing virtual content with real-world movements accurately.

Requirements:
• Pose Export with Timestamps: Ability to export camera poses along with their corresponding timestamps to ensure accurate temporal alignment in AR applications.
• Consistent Keyframe Selection: Option to export poses at consistent intervals or based on specific criteria to maintain uniformity in the
data, which is essential for smooth AR experiences.

Questions:
1. Does RTAB-Map currently support exporting poses with timestamps and consistent keyframes suitable for AR applications?
2. If not, are there recommended workflows or tools within the RTAB-Map ecosystem to achieve this functionality?
3. Are there any plans to incorporate this feature in future releases?
4. Is professional support available to assist in implementing this feature or customizing RTAB-Map for AR integration?

Additional Context:
Our primary goal is to achieve high-quality, non-realtime performance suitable for AR applications. We are open to suggestions, alternative approaches, or collaborations to realize this functionality.

Please let us know if further information is needed to assist in addressing this request.

[matlabbe]

Hi,

For your questions:

1. How many frames per second do you need? RTAB-Map by default will output poses at 1 Hz (but can be increased). Should the poses be published in real-time to another application? If so, it is supported under ros. If you only need to export poses offline, that can be done with "rtabmap-export --poses" CLI. You will get a file containing all poses in RGB-D dataset format (with stamps).

2. To have higher frame rate poses (i.e., rate matching visual odometry rate), we can enable "Rtabmap/CreateIntermediateNodes" parameter.

3. Well, it depends if the above already work for you. If you want to get the pose from rtabmap without ROS, you may have to do your own wrapper following C++ examples.

4. If it is a small thing, I can do it on my free time. For bigger features, I am currently not available to take more projects. The project is open source, so I guess it could be possible to find someone to help with your feature.

cheers,
Mathieu

[kekzzz]

Hi,

Thanks for the quick answer Mathieu.

I will test your recommondations in 1. and 2. and then will get back to you.
We are more looking for 15 Hz upwards. Especially in the ros version compared to standalone we experienced issues with higher framerate due to slower odometry calculation.

What we also would be interested in is to have a more "robust" tracking. As we dont have a robot, we dont have wheel odometry as a fallback. Also we cant just stop or go back, as recommended during mapping. One idea would be to use IMU Data and a robust Visual Odometry as fallback.
Which the general philosopy rather a bad tracking than no tracking at all.

Greetings
Karl

[matlabbe]

Hi Karl,

I see that you are looking for faster visual odometry and more robust to featureless images.

For faster odometry, you could check F2F odometry from RTAB-Map, though it is less accurate than default F2M approach.

To be robust to featureless environments:
 * Find/use a visual inertial odometry approach that can estimate the pose even if there are no visual features (for some time).
 * Use a camera with larger field-of-view
 * Use more than 1 camera (to improve field-of-view).

cheers,
Mathieu