Does rtabmap change the color?

[Grubert]

 Dear Mathieu,

Given a input color topic, it seems that the resulting pointcloud from rtabmap has different rgb values that the ones from the original image.

Does rtabmap change the rgb values in some way?

Thank you in advance.

[matlabbe]

No, it doesn't change color. Normally rtabmap will convert to BGR format (even if image uses RGB encoding). Is it a channel inverted problem (blue is red, and red is blue)?

[Grubert]

We're using a dummy color image in which the R channel has 100 everywhere, and B and G channels are set to 0.

When creating a small pointcloud, several random values appear, which are visible in the rviz pointcloud too.

By decoding of pointcloud data and by counting the R G B values, we get something like that:

Voxel pc length: 887872 nr el: 27746
R: [(0, 3), (161, 1), (66, 1), (100, 27734), (75, 1), (177, 2), (50, 2), (151, 1), (93, 1)]
G: [(0, 27734), (167, 2), (40, 1), (78, 1), (16, 1), (81, 1), (83, 1), (52, 1), (21, 1), (86, 1), (127, 1), (85, 1)]
B: [(0, 27739), (63, 1), (127, 2), (85, 1), (255, 3)]
A: [(255, 27746)]

Each pair represents the color value on that channel, and the number of times it is encountered in the pc.
As you can see, most of the values are indeed correct (R = 100 in 27734 out of 27746 points, but there are some values that don't make sense). These points are visible in Rviz too, as random colored points in the pc.

Rviz screenshot:


Setup:
D455 with color frame replaced with dummy frame. The dummy frame is only generated once, so it can't be that some pixels are random. Depth and Color frames are the same size (1280x720).
Odometry from T265.

Rtabmap 0.20.9 with rtabmap ros 0.20.9, both built from source (melodic version) with no modifications. Running on Ubuntu 18.04
Some params:
Grid\DepthDecimation = 4
Grid\ScanDecimation = 1
Grid\PreVoxelFiltering = false

We first noticed this on actual frames from the camera, so decided to try with a dummy frame, and were surprised this was happening with the dummy color frame also.

Any ideas on what might be causing this issue?

[matlabbe]

Can you share the database? Which topic are you looking at? There is maybe some voxel filtering corrupting the color when averaging points in a voxel.