Point cloud segmentation using the keyframe images

[filipm21]

Hello, I have a dataset that I gathered with a Luxonis OAK-D S 2 camera with their stereo 7.5cm baseline monochrome cameras, the dataset is from a car driving around remote roads. I calibrated everything using kalibr. What I want to get from this project is segmentation of the road using the keyframe images to get the edge of the road in them and then transfering that into the point cloud or more precisely into space, so that i get lets say a segmented point cloud , that the edge of the road can be defined. More precisely what I want to get is the transformation of the road edge into GPS coordinates so that the road width can be defined on a map (GPS was also gathered in the data), each keyframe when I look at it in the database editor has also GPS data stored where that node is set, which makes sense because you can also extract GPS poses.

My question here is if the generated point clouds from the database editor are set in the camera frame (they are not assembled in a single output cloud). How are they transformed , from what I see in cloud compare they all start from the origin but sometimes they are not facing just one way, they also face to the side. Also the coordinates seem to be with Z axis not in depth but in height. I don't know if I am mistaken but would like some reccomendations.

Also is the global map just made up from the keyframe point clouds set in their map poses or is the global point cloud somehow different ?

[matlabbe]

Rtabmap exports data in the coordinate frame convention described in REP103 and REP105 : x->forward, y->left, z->up.

I am not sure exactly which part of rtabmap-databaseViewer you are referring. On "File->Export 3D Map", you have an option to assemble all point clouds into same file, or export all individual point clouds in their respective file. For both approaches, the point clouds are transformed in map frame. It means that the individually exported point clouds are already transformed in map frame, so that you can open all individual point clouds in CloudCompare or MeshLab and see them next to each other. For "they are not facing just one way, they also face to the side", a screenshot may be helpful.

The map coordinate of each point cloud is determined by the current Graph View's state. The poses shown in that graph are those used to transform the point clouds.

In 3D View, we show the point cloud in base frame (not camera frame), also called as base_link. If Odom checkbox is not checked. If Odom checkbox is checked, we add only Z, roll and pitch to the base frame for visualization. You can right-click and how the Frustum to see the actual camera frame relative to base frame.