How to get 3D data from PointCloud?

[rena_00]

Hello,

I am working on Gazebo Simulation with a Drone, I used Rtabmap for mapping and solved issue by the help of this forum on the topic named "Stereo Odometry Error" and it worked pretty good. Now, I need to use PointNet to classify the surroundings by pointcloud. For that, I need a good point set in 3D. I am using stereo_image_proc package. But I am not sure if they are good enough or everything is well configurated. You can see the pictures below.
 
Here is my stereo cam launch file:

<!-- launch file -->

<launch>


  <!-- Start a nodelet manager; the name must be unqualified here -->
  <node pkg="nodelet" type="nodelet" name="pipeline" args="manager" />

  <!-- Include the stereo_image_proc.launch file and pass in a qualified manager name -->
  <include file="$(find stereo_cam)/launch/stereo_image_proc_new.launch">  <!-- ns="stereo"-->
      <arg name="manager" value="/pipeline" />
      <arg name="left" value="multisense_sl/camera/left" />
      <arg name="right" value="multisense_sl/camera/right" />
  </include>
  
    
  <arg name="pi/2" value="1.5707963267948966" />
  <arg name="optical_rotate" value="0.17 0 0 -$(arg pi/2) 0 -$(arg pi/2)" />
  
   <node pkg="tf" type="static_transform_publisher" name="camera_base_link" args="$(arg optical_rotate) iris/ground_truth/iris/ground_truth/odometry_sensorgt_link left_camera_optical_frame 100" />
   
   <node pkg="tf" type="static_transform_publisher" name="lidar_base_link" args="0.13 0.0 0.02 0 0 0 iris/ground_truth/iris/ground_truth/odometry_sensorgt_link lidar_link 100" />
</launch>

Here is the stereo_image_proc_new.launch file:

<!-- Launch in the camera namespace containing "image_raw" and "camera_info" -->
<launch>

  <arg name="manager" /> <!-- Must be globally qualified -->
  <arg name="respawn" default="false" />
  <arg name="left" default="left" />
  <arg name="right" default="right" />
  <!-- TODO Arguments for sync policy, etc? -->

  <arg     if="$(arg respawn)" name="bond" value="" />
  <arg unless="$(arg respawn)" name="bond" value="--no-bond" />

  <!-- Basic processing for left camera -->
  <include file="$(find image_proc)/launch/image_proc.launch"
	   ns="$(arg left)">
    <arg name="manager" value="$(arg manager)" />
    <arg name="respawn" value="$(arg respawn)" />
  </include>

  <!-- Basic processing for right camera -->
  <include file="$(find image_proc)/launch/image_proc.launch"
	   ns="$(arg right)">
    <arg name="manager" value="$(arg manager)" />
    <arg name="respawn" value="$(arg respawn)" />
  </include>

  <!-- Disparity image -->
  <node pkg="nodelet" type="nodelet" name="disparity"
        args="load stereo_image_proc/disparity $(arg manager) $(arg bond)"
	respawn="$(arg respawn)">
	<remap from="left/image_rect" to="multisense_sl/camera/left/image_rect"/>
	<remap from="left/camera_info" to="multisense_sl/camera/left/camera_info"/>
	<remap from="right/image_rect" to="multisense_sl/camera/right/image_rect"/>
	<remap from="right/camera_info" to="multisense_sl/camera/right/camera_info"/>
	<remap from="/disparity" to="multisense_sl/camera/disparity"/>
	
  </node>

  <!-- PointCloud2 -->
  <node pkg="nodelet" type="nodelet" name="point_cloud2"
        args="load stereo_image_proc/point_cloud2 $(arg manager) $(arg bond)"
	respawn="$(arg respawn)" >
        
        <remap from="left/image_rect_color" to="multisense_sl/camera/left/image_rect_color"/>
	<remap from="left/camera_info" to="multisense_sl/camera/left/camera_info"/>
	<remap from="right/image_rect_color" to="multisense_sl/camera/right/image_rect_color"/>
	<remap from="right/camera_info" to="multisense_sl/camera/right/camera_info"/>
	<remap from="disparity" to="multisense_sl/camera/disparity"/>
	</node>
</launch>

Here are the pictures. When I reduce the size it is better but still there is a hole. Screenshot_from_2018-10-09_10-34-06.png
Screenshot_from_2018-10-09_14-18-42.png
Screenshot_from_2018-10-09_15-37-14.png

What should I do to get a better point set of the objects? Thank you for any help.

Cheers.

[matlabbe]

To tune stereo_img_proc, see http://wiki.ros.org/stereo_image_proc/Tutorials/ChoosingGoodStereoParameters

You are using stereo, no 3D points can be triangulated inside a textureless surface, only the edges can be detected. You may switch to a simulated kinect camera instead to have points on textureless surfaces.

cheers,
Mathieu