Cant get map to display using Orbbec Astra Pro

[bxtcoop]

Setup
-------------------------------------------------
Ubuntu 16.04
ROS Kinetic
Orbbec Astra Pro

I have been trying for a while now to get my Astra Pro to show a map in Rtab. I've tried both installing the compiled binary and building RTAb from source with no luck. I can view an RGB-D image in RVIZ by combining the Astra Pro launch file and the usb_cam launch file (have to use this because the Astra Pro uses uvc camera which openni2 apparently doesnt support) and selecting the camera/depth/image_raw and usb_cam/image_raw topics.

To launch RTab, I start roscore, use astra_launch astrapro.launch, usb_cam usb_cam-test.launch, and roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start" rviz:=true rtabmapviz:=false.

When RViz comes up, it has /camera/rgb/image_rect_color slected for the image topic which I change to usb_cam/image_raw. The PointCloud2 topic is set to voxel cloud and if I change it to camera/depth_registered/image_raw I get an error in the astra pro launch terminal saying there is an MD5 mismatch between what was expected and what was received. There is a /camera/depth_registered/points in that drop down, but it also does nothing. RViz also has an error in the Global Status which says "Fixed Frame [map] does not exist."

The terminal I launched rviz from has several warnings as well. They are:
1 - "odometry: Could not get transform from camera_link to usb_cam"   and
2 -   "/rtabmap/rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published and the timestamps in their header are set"

Finally, when I run just regular rtab command from terminal, I get the GUI, but I cant see any image from the camera. When I try to connect to the camera, the openni2 driver is greyed out and not selectable. The only driver option I have is Freenect. My overall goal is just to display the basic map using the Astra Pro so if anyone has any ideas how I can do that I would appreciate it.

Things I've tried - I already did the camera calibration (depth and RGB) and added those files to the camera info directory (that part seems to be fine) and I changed the frame_id in the usb_cam package before I built it as per this link: https://answers.ros.org/question/253893/astra-pro-with-ros-rtabmap/

[matlabbe]

I don't own an Orbbec Astra Pro, so it is difficult for me to debug this. Another link here: https://github.com/introlab/rtabmap_ros/issues/147

What is your TF tree? Is camera_link to usb_cam published?

[bxtcoop]

Thanks for getting back to me. I know the Astra Pro doesnt seem to be as popular as the base Astra. I havent found much about configuring it for ROS. I'll try to answer your questions in order and as best I can.

- The output of rosrun tf view_frames is (I'm new to ROS, but I think this is the tree you meant?):

*********************************************************
---
transforms:
  -
    header:
      seq: 0
      stamp:
        secs: 1537372879
        nsecs: 981676619
      frame_id: "/camera_link"
    child_frame_id: "/camera_depth_frame"
    transform:
      translation:
        x: 0.0
        y: -0.02
        z: 0.0
      rotation:
        x: 0.0
        y: 0.0
        z: 0.0
        w: 1.0
---
transforms:
  -
    header:
      seq: 0
      stamp:
        secs: 1537372879
        nsecs: 983863408
      frame_id: "/camera_link"
    child_frame_id: "/camera_rgb_frame"
    transform:
      translation:
        x: 0.0
        y: -0.045
        z: 0.0
      rotation:
        x: 0.0
        y: 0.0
        z: 0.0
        w: 1.0
---
transforms:
  -
    header:
      seq: 0
      stamp:
        secs: 1537372879
        nsecs: 986024877
      frame_id: "/camera_rgb_frame"
    child_frame_id: "/camera_rgb_optical_frame"
    transform:
      translation:
        x: 0.0
        y: 0.0
        z: 0.0
      rotation:
        x: -0.5
        y: 0.5
        z: -0.5
        w: 0.5
---
transforms:
  -
    header:
      seq: 0
      stamp:
        secs: 1537372879
        nsecs: 985986074
      frame_id: "/camera_depth_frame"
    child_frame_id: "/camera_depth_optical_frame"
    transform:
      translation:
        x: 0.0
        y: 0.0
        z: 0.0
      rotation:
        x: -0.5
        y: 0.5
        z: -0.5
        w: 0.5
***************************************************************

And here is the list of published topics:

***************************************************************
/camera/camera_nodelet_manager/bond
/camera/depth/camera_info
/camera/depth/image
/camera/depth/image/compressed
/camera/depth/image/compressed/parameter_descriptions
/camera/depth/image/compressed/parameter_updates
/camera/depth/image/compressedDepth
/camera/depth/image/compressedDepth/parameter_descriptions
/camera/depth/image/compressedDepth/parameter_updates
/camera/depth/image/theora
/camera/depth/image/theora/parameter_descriptions
/camera/depth/image/theora/parameter_updates
/camera/depth/image_raw
/camera/depth/image_raw/compressed
/camera/depth/image_raw/compressed/parameter_descriptions
/camera/depth/image_raw/compressed/parameter_updates
/camera/depth/image_raw/compressedDepth
/camera/depth/image_raw/compressedDepth/parameter_descriptions
/camera/depth/image_raw/compressedDepth/parameter_updates
/camera/depth/image_raw/theora
/camera/depth/image_raw/theora/parameter_descriptions
/camera/depth/image_raw/theora/parameter_updates
/camera/depth/image_rect
/camera/depth/image_rect/compressed
/camera/depth/image_rect/compressed/parameter_descriptions
/camera/depth/image_rect/compressed/parameter_updates
/camera/depth/image_rect/compressedDepth
/camera/depth/image_rect/compressedDepth/parameter_descriptions
/camera/depth/image_rect/compressedDepth/parameter_updates
/camera/depth/image_rect/theora
/camera/depth/image_rect/theora/parameter_descriptions
/camera/depth/image_rect/theora/parameter_updates
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressed/parameter_descriptions
/camera/depth/image_rect_raw/compressed/parameter_updates
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/compressedDepth/parameter_descriptions
/camera/depth/image_rect_raw/compressedDepth/parameter_updates
/camera/depth/image_rect_raw/theora
/camera/depth/image_rect_raw/theora/parameter_descriptions
/camera/depth/image_rect_raw/theora/parameter_updates
/camera/depth/points
/camera/depth_rectify_depth/parameter_descriptions
/camera/depth_rectify_depth/parameter_updates
/camera/depth_registered/camera_info
/camera/depth_registered/image_raw
/camera/depth_registered/image_raw/compressed
/camera/depth_registered/image_raw/compressed/parameter_descriptions
/camera/depth_registered/image_raw/compressed/parameter_updates
/camera/depth_registered/image_raw/compressedDepth
/camera/depth_registered/image_raw/compressedDepth/parameter_descriptions
/camera/depth_registered/image_raw/compressedDepth/parameter_updates
/camera/depth_registered/image_raw/theora
/camera/depth_registered/image_raw/theora/parameter_descriptions
/camera/depth_registered/image_raw/theora/parameter_updates
/camera/depth_registered/points
/camera/depth_registered/sw_registered/camera_info
/camera/depth_registered/sw_registered/image_rect
/camera/depth_registered/sw_registered/image_rect/compressed
/camera/depth_registered/sw_registered/image_rect/compressed/parameter_descriptions
/camera/depth_registered/sw_registered/image_rect/compressed/parameter_updates
/camera/depth_registered/sw_registered/image_rect/compressedDepth
/camera/depth_registered/sw_registered/image_rect/compressedDepth/parameter_descriptions
/camera/depth_registered/sw_registered/image_rect/compressedDepth/parameter_updates
/camera/depth_registered/sw_registered/image_rect/theora
/camera/depth_registered/sw_registered/image_rect/theora/parameter_descriptions
/camera/depth_registered/sw_registered/image_rect/theora/parameter_updates
/camera/depth_registered/sw_registered/image_rect_raw
/camera/depth_registered/sw_registered/image_rect_raw/compressed
/camera/depth_registered/sw_registered/image_rect_raw/compressed/parameter_descriptions
/camera/depth_registered/sw_registered/image_rect_raw/compressed/parameter_updates
/camera/depth_registered/sw_registered/image_rect_raw/compressedDepth
/camera/depth_registered/sw_registered/image_rect_raw/compressedDepth/parameter_descriptions
/camera/depth_registered/sw_registered/image_rect_raw/compressedDepth/parameter_updates
/camera/depth_registered/sw_registered/image_rect_raw/theora
/camera/depth_registered/sw_registered/image_rect_raw/theora/parameter_descriptions
/camera/depth_registered/sw_registered/image_rect_raw/theora/parameter_updates
/camera/driver/parameter_descriptions
/camera/driver/parameter_updates
/camera/ir/camera_info
/camera/ir/image
/camera/ir/image/compressed
/camera/ir/image/compressed/parameter_descriptions
/camera/ir/image/compressed/parameter_updates
/camera/ir/image/compressedDepth
/camera/ir/image/compressedDepth/parameter_descriptions
/camera/ir/image/compressedDepth/parameter_updates
/camera/ir/image/theora
/camera/ir/image/theora/parameter_descriptions
/camera/ir/image/theora/parameter_updates
/camera/projector/camera_info
/image_view/output
/image_view/parameter_descriptions
/image_view/parameter_updates
/rosout
/rosout_agg
/tf
/tf_static
/usb_cam/camera_info
/usb_cam/image_raw
/usb_cam/image_raw/compressed
/usb_cam/image_raw/compressed/parameter_descriptions
/usb_cam/image_raw/compressed/parameter_updates
/usb_cam/image_raw/compressedDepth
/usb_cam/image_raw/compressedDepth/parameter_descriptions
/usb_cam/image_raw/compressedDepth/parameter_updates
/usb_cam/image_raw/theora
/usb_cam/image_raw/theora/parameter_descriptions
/usb_cam/image_raw/theora/parameter_updates
*********************************************************************

When running this command:
roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start" rviz:=true rtabmapviz:=false

It executes, but I get the following warnings:

orm_duration"=0.200000)! Error="canTransform: source_frame usb_cam does not exist.. canTransform returned after 0.200644 timeout was 0.2

[ WARN] [1537391762.757707562]: /rtabmap/rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10).

/rtabmap/rtabmap subscribed to (approx sync):
   /rtabmap/odom,
   /usb_cam/image_raw,
   /camera/depth_registered/image_raw,
   /usb_cam/camera_info,
   /rtabmap/odom_info


Finally, in RVIZ itself, under the PointCloud2 heading, there is an error under transform:
"
Transform [sender=unknown_publisher]
For frame [usb_cam]: Fixed Frame [map] does not exist"

Under global it says "Fixed Frame Map doesnt exist"


I also mentioned the post you linked to in my original question. I have already changed the name of the
frame id and rebuilt the package as well as doing the camera calibration. I'll attach the frames.pdf and rqt_graph to this replay as well. Maybe that will help. Thanks again for trying to help me pin this down. frames.pdf

matlabbe wrote

I don't own an Orbbec Astra Pro, so it is difficult for me to debug this. Another link here: https://github.com/introlab/rtabmap_ros/issues/147

What is your TF tree? Is camera_link to usb_cam published?

[matlabbe]

When starting only the camera (without rtabmap), can record a small bag (5/10 sec) with these topics and share it (through a google drive or dropbox)?:

$ rosbag record /usb_cam/image_raw /camera/depth_registered/image_raw /usb_cam/camera_info 

cheers,
Mathieu

[bxtcoop]

Here you go:



I keep saying it, but thanks again for all the help. Hopefully one day I'll be at a point where I can return the favor.

[matlabbe]

Hi,

doing this works:

$ roslaunch rtabmap_ros rtabmap.launch args:="-d" rgb_topic:=/usb_cam/image_raw camera_info_topic:=/usb_cam/camera_info use_sim_time:=true
$ rosrun tf static_transform_publisher 0 0 0 -1.5707963267948966 0 -1.5707963267948966 camera_link usb_cam 100
$ rosbag play --clock 2018-09-26-16-40-44.bag

[bxtcoop]

I'm starting to think the problem is somewhere other than the cameras or ROS. I mentioned previously that when running just rtabmap I get no option to select the Openni2 driver (its greyed out). Also, when running your rtabmap command and then opening the settings I get this error:

" Failed to get some RTAB-Map parameters from ROS server, the rtabmap node may be not started or some parameters won't work..."

Really beating my head against the wall on this. I even tried wiping everything and starting fresh which didnt seem to have any effect either. I've attached a screen shot so you can see what I am looking at after running your commands.

Edit: Realized you may not be able to see much there. The terminals are running: usb_cam launch command, astra_pro launch command, roscore, your command, and the rosrun tf publisher command you sent. The window is the rtabmap window that comes up.

[matlabbe]

Hi,

Don't launch rtabmap launch file with use_sim_time when using the sensors. I added use_sim_time to make it work with the rosbag, but you should not set it when using with the sensors directly. For the rtabmapviz error when looking for parameters, it is because when use_sim_time is true, rtabmap node starts only after a clock is published (by a rosbag or a simulated environment). So try this instead:

$ roslaunch rtabmap_ros rtabmap.launch args:="-d" rgb_topic:=/usb_cam/image_raw camera_info_topic:=/usb_cam/camera_info

cheers,
Mathieu

[bxtcoop]

Here is the output I get with that command:

[ WARN] [1539205218.990960945]: odometry: Could not get transform from camera_link to usb_cam (stamp=1539205218.781422) after 0.200000 seconds ("wait_for_transform_duration"=0.200000)! Error="canTransform: source_frame usb_cam does not exist.. canTransform returned after 0.201483 timeout was 0.2."
[ WARN] [1539205219.222114678]: odometry: Could not get transform from camera_link to usb_cam (stamp=1539205218.986040) after 0.200000 seconds ("wait_for_transform_duration"=0.200000)! Error="canTransform: source_frame usb_cam does not exist.. canTransform returned after 0.201325 timeout was 0.2."


Obviously it doesnt think there is a usb_cam source_frame. Is there any way I can check this?

[matlabbe]

Hi,

To debug your TF tree, you can use:

$ rosrun tf view_frames

"camera_link" is the default "frame_id" set to rtabmap when rtabmap.launch is used. I saw in my previous post that I added also:

$ rosrun tf static_transform_publisher 0 0 0 -1.5707963267948966 0 -1.5707963267948966 camera_link usb_cam 100

to rotate the camera in ROS coordinates. You may try to start that static_transform_publisher too, so you will get the missing camera_link -> usb_cam frame. So the full example would be:

$ roslaunch rtabmap_ros rtabmap.launch args:="-d" rgb_topic:=/usb_cam/image_raw camera_info_topic:=/usb_cam/camera_info
$ rosrun tf static_transform_publisher 0 0 0 -1.5707963267948966 0 -1.5707963267948966 camera_link usb_cam 100

cheers,
Mathieu

[bxtcoop]

I really cant thank you enough for your help. I would have never gotten this without you. Thanks so much!

If you wouldn't mind to answer one or two other questions that would help me and hopefully some other people:

1) How can I launch this now in RVIZ?

2) How could I figure out how to determine those static transform numbers myself next time? What does the 100 at the end mean?

3) Finally, whats the best way to clean up the point cloud that i'm getting now? Are there some RTAB parameters I should change that are a good starting point? I've looked at the RTAB documentation, but I didnt see a tuning guide or any thing like that. I probably just overlooked it.

In any case, I'm really just elated to have this working finally since I've been going at it for months now. Thanks again!

[matlabbe]

Hi,

1) with rtabmap.launch, you can add argument "rviz:=true". You can also just start rviz alone, and add display stuff you want. To get similar 3D than rtabmapviz, the display is called rtabmap/MapCloud in rviz.

2) Look at the static_transform_publisher documentation. For the orientation, those are 90 deg angles in yaw and roll to rotate the camera in ROS coordinate system. From (x->right, y->down, z->forward) to (x->forward, y->left, z->up).

3) For the point cloud you are seeing in rtabmapviz, open Preferences->3D Rendering, under Map column.

cheers,
Mathieu