RTABMAP topics to Unity

[jacksonkr]

A friend and I are using Unity to enhance the map output of RTABMAP in coordination with VR/AR.

Is it possible to get a ROS Topic datastream into Unity? I know Unity will pull from a webservice so that's the route I'll probably take.

---

While writing this up I found ROStful:
http://www.ros.org/news/2014/02/introducing-rostful-ros-over-restful-web-services.html

Which may do what I need it to. Regardless, I'm interested in the creator's thoughts.

[matlabbe]

Hi,

In this paper, they use rosbridge.

My only concern is the data transfer rate. Not sure you can stream a point cloud without lag. I think ROStful is indeed a good idea:

ROStful web services primarily use the rosbridge JSON mapping for ROS messages. However, binary serialized ROS messages can be used to increase performance.

[jacksonkr]

It looks like rostful support + following is weak. I'm going to give rosbridge a try but certainly pointcloud data is going to clog my pipeline.

Can RTABmap produce a mesh online? That would greatly reduce the amount of data needing to be transfered. Or perhaps is there a way to perform procedural post production? That is, only post-produce new data that has been gathered ever few seconds or so and send only the mesh data on a specific topic.

I'm working with hololens + rtabmap this week so any ideas you have are greatly appreciated.

[matlabbe]

Hi,

If you subscribe to mapData, you receive the whole graph and only the last data (e..g, this topic would use a nearly fixed bandwidth). The published images in mapData can be also decimated for less bandwidth (see "Mem/ImagePostDecimation" parameter). The whole map can be assembled on the client side (similar to what rtabmapviz or MapCloud RVIZ's plugin do).

With mapData, you have to reconstruct the point clouds though. If the environment is relatively small, maybe cloud_map topic would be okay (so you don't bother re-creating point clouds, assembling the map and update it on loop closures).

Meshing is only available for visualization, all ROS communications would use point clouds, octomap or occupancy grid map. Note that voxel size of the output cloud_map is 5 cm, which decrease already a lot the size of the map in bytes. However, it depends which precision you need for the point cloud.

With come codings, you could create a compressed point cloud topic for cloud_map. Maybe someone has already did that somewhere for convenience.

cheers

[jacksonkr]

Thanks, this is helpful. I'll throw together a couple scenarios and report back with whichever route appears most efficient.

[jacksonkr]

So I found out that rosbridge can be used with a sort of pointcloud encoding. I'm working on it now:


The following is from a ros3djs tutorial

roslaunch rosbridge_server rosbridge_websocket.launch
sudo apt-get install ros-indigo-depthcloud-encoder

[jacksonkr]

Update: You can get the video to stream (color, bw, ir, etc) from the camera fine and it's quick. However the point clouds are not streaming due to an issue with the depthcloud_encoder package. I've started an issue on github but it doesn't appear that the developer is all that active on this "old" project. I've also posted onto answers.ros.org on potentially other ways to stream pointclouds via a webservice. I'll post back when I have more info.