Further Processing of camera images & RTAB data -- best way to do it?

Posted by ibd on
URL: http://official-rtab-map-forum.206.s1.nabble.com/Further-Processing-of-camera-images-RTAB-data-best-way-to-do-it-tp3015.html

Hi Mathieu and all!

I'm currently developing an application where I do object detection with YOLO's CNN, and then some additional processing, in addition to running SLAM through RTAB-Map.
I was wondering what is the best way to route data, and what capabilities RTAB-Map already offers in that regard.

To be more precise, there is the following data flow in my setup:
camera images, odometry => RTAB-Map => metric map, graph
camera images => YOLO => list of object labels with corresponding bounding boxes ("labeled BBs")
labeled BBs, odometry => Our own module => filtered bounding boxes

1. Does RTAB-Map offer incremental updates for publishing map updates? Our own module will keep a copy of the map for its own purposes, and it would be good to be able to receive incremental updates as the map gets bigger.
2. Since YOLO might incur significant delays on some computers, the data that our own module receives might not be synchronized well (i.e., labeled BBs arrive much later than camera pose). It might be better to use some cached data of RTAB-Map. Does RTAB-Map offer to retrieve a set of {image, camera pose, (local) map} at a later time? If yes, the best way to do it is would be to create our own message, which contains the set just mentioned, and passes it through YOLO and our own module in a linear fashion. My guess would be that this is offered through the graph structure that it publishes.
3. Any other smart way to solve this task?
4. How would loop closures influence the aforementioned aspects of RTAB-Map (incremental updates, cached data)?

Thank you very much.
- ibd