Some issue with rtabmap_drone_example

[Samim-17]

Hello
First of all I am able to launch gazebo.launch and simulate drone with d435 camera in it. Then I skipped the slam.launch and launch the main rtabmap.launch with the parameters as that of slam.launch. In this rtabmap.launch terminal i get the following errors.

\log
<raw>
[ WARN] [1730474035.025682470, 32.001000000]: The time difference between rgb and depth frames is high (diff=0.021000s, rgb=31.932000s, depth=31.953000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474035.373699700, 32.127000000]: Odom: quality=469, std dev=0.001921m|0.014170rad, update time=0.343673s
[ INFO] [1730474035.810554059, 32.240000000]: Odom: quality=479, std dev=0.000811m|0.010360rad, update time=0.340628s
[ WARN] (2024-11-01 20:58:55.857) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474035.874084953, 32.257000000]: rtabmap (118): Rate=1.00s, Limit=0.000s, Conversion=0.0108s, RTAB-Map=0.9126s, Maps update=0.0008s pub=0.0076s (local map=28, WM=28)
[ INFO] [1730474036.175579148, 32.328000000]: Odom: quality=282, std dev=0.002677m|0.014651rad, update time=0.306490s
[ INFO] [1730474036.533996089, 32.448000000]: Odom: quality=468, std dev=0.005566m|0.018649rad, update time=0.315518s
[ INFO] [1730474036.886224504, 32.583000000]: Odom: quality=433, std dev=0.002278m|0.014170rad, update time=0.316648s
[ INFO] [1730474037.167899964, 32.636000000]: Odom: quality=454, std dev=0.002044m|0.020216rad, update time=0.275416s
[ INFO] [1730474037.499980217, 32.731000000]: Odom: quality=464, std dev=0.001995m|0.014170rad, update time=0.293052s
[ INFO] [1730474037.890732564, 32.852000000]: Odom: quality=450, std dev=0.003236m|0.022178rad, update time=0.333286s
[ INFO] [1730474038.264234681, 32.980000000]: Odom: quality=470, std dev=0.001674m|0.014170rad, update time=0.306275s
[ INFO] [1730474038.535348564, 33.045000000]: Odom: quality=473, std dev=0.000686m|0.008712rad, update time=0.264479s
[ WARN] [1730474038.564784188, 33.045000000]: The time difference between rgb and depth frames is high (diff=0.030000s, rgb=33.045000s, depth=33.015000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474038.911236289, 33.140000000]: Odom: quality=467, std dev=0.000100m|0.000173rad, update time=0.290483s
[ WARN] (2024-11-01 20:58:59.413) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474039.432053259, 33.248000000]: rtabmap (119): Rate=1.00s, Limit=0.000s, Conversion=0.0038s, RTAB-Map=0.8826s, Maps update=0.0005s pub=0.0083s (local map=28, WM=28)
[ INFO] [1730474039.537778785, 33.292000000]: Odom: quality=472, std dev=0.001328m|0.008712rad, update time=0.367788s
[ INFO] [1730474039.873544224, 33.365000000]: Odom: quality=466, std dev=0.002507m|0.017423rad, update time=0.303658s
[ INFO] [1730474040.211585777, 33.433000000]: Odom: quality=479, std dev=0.001050m|0.010360rad, update time=0.333583s
[ERROR] [1730474040.323820106, 33.441000000]: Overwriting previous data! Make sure IMU is published faster than data rate. (last image stamp buffered=33.421000 and new one is 33.433000, last imu stamp received=33.417000)
[ INFO] [1730474040.652137380, 33.539000000]: Odom: quality=485, std dev=0.002137m|0.017944rad, update time=0.326630s
[ INFO] [1730474040.990352515, 33.634000000]: Odom: quality=465, std dev=0.001865m|0.012320rad, update time=0.333419s
[ INFO] [1730474041.417833396, 33.690000000]: Odom: quality=479, std dev=0.000340m|0.000173rad, update time=0.315418s
[ INFO] [1730474041.845687837, 33.854000000]: Odom: quality=485, std dev=0.001001m|0.011465rad, update time=0.351057s
[ INFO] [1730474042.169317281, 33.922000000]: Odom: quality=463, std dev=0.002367m|0.016542rad, update time=0.312492s
[ INFO] [1730474042.565623664, 34.034000000]: Odom: quality=461, std dev=0.002610m|0.011465rad, update time=0.291035s
[ INFO] [1730474042.957254603, 34.128000000]: Odom: quality=475, std dev=0.000224m|0.000173rad, update time=0.294510s
[ INFO] [1730474043.291507811, 34.214000000]: Odom: quality=455, std dev=0.003620m|0.019282rad, update time=0.328831s
[ INFO] [1730474043.672182232, 34.282000000]: Odom: quality=462, std dev=0.002072m|0.013027rad, update time=0.346935s
[ WARN] (2024-11-01 20:59:03.898) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474043.914928194, 34.315000000]: rtabmap (120): Rate=1.00s, Limit=0.000s, Conversion=0.0042s, RTAB-Map=0.9413s, Maps update=0.0006s pub=0.0064s (local map=28, WM=28)
[ INFO] [1730474044.041620968, 34.343000000]: Odom: quality=461, std dev=0.001431m|0.014170rad, update time=0.330247s
[ERROR] [1730474044.119476417, 34.354000000]: Overwriting previous data! Make sure IMU is published faster than data rate. (last image stamp buffered=34.341000 and new one is 34.351000, last imu stamp received=34.337000)
[ INFO] [1730474044.427055248, 34.478000000]: Odom: quality=448, std dev=0.001375m|0.013634rad, update time=0.301264s
[ INFO] [1730474044.645264238, 34.581000000]: Odom: quality=459, std dev=0.002016m|0.018188rad, update time=0.213122s
[ INFO] [1730474045.055955174, 34.681000000]: Odom: quality=475, std dev=0.000868m|0.010360rad, update time=0.293255s
[ INFO] [1730474045.349433044, 34.784000000]: Odom: quality=452, std dev=0.002955m|0.021909rad, update time=0.249081s
[ INFO] [1730474045.674602067, 34.973000000]: Odom: quality=467, std dev=0.002215m|0.018188rad, update time=0.235898s
[ INFO] [1730474046.013637541, 35.091000000]: Odom: quality=471, std dev=0.000809m|0.008712rad, update time=0.323346s
[ INFO] [1730474046.315523570, 35.204000000]: Odom: quality=467, std dev=0.001856m|0.015492rad, update time=0.292124s
[ INFO] [1730474046.722004349, 35.334000000]: Odom: quality=463, std dev=0.001281m|0.013027rad, update time=0.365252s
[ INFO] [1730474047.100154837, 35.452000000]: Odom: quality=466, std dev=0.003599m|0.010360rad, update time=0.344805s
[ WARN] (2024-11-01 20:59:07.172) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ERROR] [1730474047.183280290, 35.456000000]: Overwriting previous data! Make sure IMU is published faster than data rate. (last image stamp buffered=35.446000 and new one is 35.452000, last imu stamp received=35.438000)
[ INFO] [1730474047.185264656, 35.452000000]: rtabmap (121): Rate=1.00s, Limit=0.000s, Conversion=0.0049s, RTAB-Map=0.8566s, Maps update=0.0012s pub=0.0062s (local map=28, WM=28)
[ INFO] [1730474047.538738360, 35.601000000]: Odom: quality=481, std dev=0.001736m|0.012320rad, update time=0.344836s
[ INFO] [1730474047.898306857, 35.711000000]: Odom: quality=469, std dev=0.000925m|0.008712rad, update time=0.305793s
[ WARN] [1730474047.929424928, 35.720000000]: The time difference between rgb and depth frames is high (diff=0.041000s, rgb=35.640000s, depth=35.681000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474048.245852220, 35.790000000]: Odom: quality=496, std dev=0.000522m|0.008712rad, update time=0.309222s
[ INFO] [1730474048.560901314, 35.858000000]: Odom: quality=450, std dev=0.001213m|0.010360rad, update time=0.294724s
[ INFO] [1730474048.877628035, 35.954000000]: Odom: quality=469, std dev=0.001257m|0.014170rad, update time=0.308437s
[ INFO] [1730474049.244380746, 36.086000000]: Odom: quality=481, std dev=0.000209m|0.000173rad, update time=0.328316s
[ INFO] [1730474049.786235384, 36.212000000]: Odom: quality=453, std dev=0.003601m|0.016851rad, update time=0.317982s
[ WARN] [1730474049.789500932, 36.212000000]: The time difference between rgb and depth frames is high (diff=0.057000s, rgb=36.145000s, depth=36.088000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474050.094298015, 36.256000000]: Odom: quality=472, std dev=0.000587m|0.008712rad, update time=0.301864s
[ INFO] [1730474050.407149739, 36.325000000]: Odom: quality=470, std dev=0.002162m|0.015492rad, update time=0.306734s
[ WARN] (2024-11-01 20:59:10.633) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474050.643939176, 36.382000000]: rtabmap (122): Rate=1.00s, Limit=0.000s, Conversion=0.0222s, RTAB-Map=0.8295s, Maps update=0.0005s pub=0.0039s (local map=28, WM=28)
[ INFO] [1730474050.742559870, 36.411000000]: Odom: quality=470, std dev=0.001622m|0.015089rad, update time=0.290925s
[ INFO] [1730474051.050653244, 36.488000000]: Odom: quality=448, std dev=0.002471m|0.013027rad, update time=0.272568s
[ INFO] [1730474051.377648996, 36.568000000]: Odom: quality=472, std dev=0.002024m|0.012320rad, update time=0.296573s
[ INFO] [1730474051.707940588, 36.688000000]: Odom: quality=473, std dev=0.002327m|0.021339rad, update time=0.298214s
[ INFO] [1730474052.031618024, 36.801000000]: Odom: quality=464, std dev=0.001822m|0.016851rad, update time=0.315574s
[ WARN] [1730474052.127903324, 36.852000000]: The time difference between rgb and depth frames is high (diff=0.047000s, rgb=36.744000s, depth=36.791000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474052.466569051, 36.962000000]: Odom: quality=472, std dev=0.001894m|0.018649rad, update time=0.335166s
[ INFO] [1730474052.754453497, 36.999000000]: Odom: quality=469, std dev=0.000100m|0.000173rad, update time=0.232854s
[ INFO] [1730474053.053584945, 37.097000000]: Odom: quality=481, std dev=0.000476m|0.000173rad, update time=0.274985s
[ INFO] [1730474053.483150097, 37.226000000]: Odom: quality=454, std dev=0.002355m|0.014651rad, update time=0.364859s
[ INFO] [1730474053.943869714, 37.350000000]: Odom: quality=455, std dev=0.001897m|0.017423rad, update time=0.393520s
[ WARN] [1730474053.964328027, 37.352000000]: The time difference between rgb and depth frames is high (diff=0.026000s, rgb=37.281000s, depth=37.307000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474054.313140304, 37.418000000]: Odom: quality=478, std dev=0.003051m|0.018649rad, update time=0.342956s
[ WARN] (2024-11-01 20:59:14.366) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474054.372209172, 37.436000000]: rtabmap (123): Rate=1.00s, Limit=0.000s, Conversion=0.0066s, RTAB-Map=0.8787s, Maps update=0.0002s pub=0.0026s (local map=28, WM=28)
[ INFO] [1730474054.746666215, 37.518000000]: Odom: quality=465, std dev=0.002286m|0.011465rad, update time=0.289382s
[ WARN] [1730474054.749732620, 37.518000000]: The time difference between rgb and depth frames is high (diff=0.025000s, rgb=37.488000s, depth=37.513000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474055.142107528, 37.672000000]: Odom: quality=474, std dev=0.003595m|0.020556rad, update time=0.382467s
[ INFO] [1730474055.552705064, 37.809000000]: Odom: quality=469, std dev=0.000978m|0.008712rad, update time=0.267653s
[ INFO] [1730474055.954117193, 37.934000000]: Odom: quality=476, std dev=0.001988m|0.017144rad, update time=0.356928s
[ERROR] [1730474056.085340010, 37.952000000]: Overwriting previous data! Make sure IMU is published faster than data rate. (last image stamp buffered=37.935000 and new one is 37.952000, last imu stamp received=37.934000)
[ INFO] [1730474056.361687017, 38.003000000]: Odom: quality=457, std dev=0.001966m|0.017423rad, update time=0.262339s
[ INFO] [1730474056.700057892, 38.104000000]: Odom: quality=460, std dev=0.001075m|0.010360rad, update time=0.304347s
[ INFO] [1730474057.009462191, 38.189000000]: Odom: quality=465, std dev=0.000100m|0.000173rad, update time=0.264911s
[ INFO] [1730474057.319316197, 38.261000000]: Odom: quality=464, std dev=0.000100m|0.000173rad, update time=0.305535s
[ INFO] [1730474057.840026058, 38.429000000]: Odom: quality=476, std dev=0.000100m|0.000173rad, update time=0.392900s
[ INFO] [1730474058.230278522, 38.507000000]: Odom: quality=448, std dev=0.002175m|0.012320rad, update time=0.357093s
[ WARN] (2024-11-01 20:59:18.233) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474058.242611419, 38.507000000]: rtabmap (124): Rate=1.00s, Limit=0.000s, Conversion=0.0032s, RTAB-Map=0.9153s, Maps update=0.0004s pub=0.0031s (local map=28, WM=28)
[ INFO] [1730474058.509867013, 38.579000000]: Odom: quality=455, std dev=0.001066m|0.010360rad, update time=0.271154s
[ INFO] [1730474058.839498725, 38.651000000]: Odom: quality=479, std dev=0.000151m|0.000173rad, update time=0.294592s
[ INFO] [1730474059.159575397, 38.725000000]: Odom: quality=485, std dev=0.001063m|0.011465rad, update time=0.277482s
[ INFO] [1730474059.433159803, 38.768000000]: Odom: quality=473, std dev=0.000352m|0.008712rad, update time=0.268199s
[ INFO] [1730474059.737436265, 38.845000000]: Odom: quality=472, std dev=0.001040m|0.008712rad, update time=0.265257s
[ INFO] [1730474060.062856163, 38.972000000]: Odom: quality=459, std dev=0.002017m|0.010360rad, update time=0.320130s
[ INFO] [1730474060.456225795, 39.128000000]: Odom: quality=462, std dev=0.001311m|0.012320rad, update time=0.368282s
[ INFO] [1730474060.851943148, 39.247000000]: Odom: quality=477, std dev=0.002826m|0.019282rad, update time=0.354452s
[ INFO] [1730474061.140251530, 39.299000000]: Odom: quality=460, std dev=0.002137m|0.008712rad, update time=0.252393s
[ INFO] [1730474061.505648688, 39.393000000]: Odom: quality=467, std dev=0.001301m|0.013027rad, update time=0.331686s
[ WARN] [1730474061.546671779, 39.403000000]: The time difference between rgb and depth frames is high (diff=0.039000s, rgb=39.320000s, depth=39.359000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474061.906094729, 39.509000000]: Odom: quality=485, std dev=0.000100m|0.000173rad, update time=0.348104s
[ WARN] (2024-11-01 20:59:22.016) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474062.036170530, 39.539000000]: rtabmap (125): Rate=1.00s, Limit=0.000s, Conversion=0.0032s, RTAB-Map=0.8786s, Maps update=0.0009s pub=0.0120s (local map=28, WM=28)
[ INFO] [1730474062.285240137, 39.624000000]: Odom: quality=472, std dev=0.000946m|0.011465rad, update time=0.360508s
[ INFO] [1730474062.759108984, 39.794000000]: Odom: quality=453, std dev=0.001448m|0.008712rad, update time=0.365886s
[ INFO] [1730474063.060479720, 39.882000000]: Odom: quality=478, std dev=0.000806m|0.008712rad, update time=0.271443s
[ INFO] [1730474063.333536736, 39.938000000]: Odom: quality=479, std dev=0.000961m|0.008712rad, update time=0.265548s
[ INFO] [1730474063.711716499, 40.101000000]: Odom: quality=459, std dev=0.002789m|0.014651rad, update time=0.326047s
[ INFO] [1730474064.072728933, 40.229000000]: Odom: quality=421, std dev=0.005498m|0.025108rad, update time=0.316846s
[ INFO] [1730474064.399034850, 40.352000000]: Odom: quality=450, std dev=0.002935m|0.010360rad, update time=0.300593s
[ INFO] [1730474064.689249020, 40.455000000]: Odom: quality=462, std dev=0.000731m|0.000173rad, update time=0.280914s
[ INFO] [1730474065.110207050, 40.503000000]: Odom: quality=458, std dev=0.002221m|0.013634rad, update time=0.282454s
[ WARN] (2024-11-01 20:59:25.401) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474065.421146971, 40.555000000]: rtabmap (126): Rate=1.00s, Limit=0.000s, Conversion=0.0041s, RTAB-Map=0.7131s, Maps update=0.0005s pub=0.0133s (local map=28, WM=28)
[ INFO] [1730474065.631455578, 40.612000000]: Odom: quality=475, std dev=0.001389m|0.008712rad, update time=0.316132s
[ INFO] [1730474065.922748156, 40.665000000]: Odom: quality=450, std dev=0.000520m|0.000173rad, update time=0.281348s
[ INFO] [1730474066.254324445, 40.775000000]: Odom: quality=440, std dev=0.001664m|0.013027rad, update time=0.326330s
[ WARN] [1730474066.300799413, 40.791000000]: The time difference between rgb and depth frames is high (diff=0.060000s, rgb=40.668000s, depth=40.728000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474066.583056874, 40.833000000]: Odom: quality=452, std dev=0.000100m|0.000173rad, update time=0.279225s
[ INFO] [1730474067.161820666, 41.040000000]: Odom: quality=474, std dev=0.001399m|0.010360rad, update time=0.344502s
[ INFO] [1730474067.499786849, 41.104000000]: Odom: quality=468, std dev=0.003320m|0.019672rad, update time=0.330948s
[ INFO] [1730474067.766423596, 41.147000000]: Odom: quality=470, std dev=0.000100m|0.000173rad, update time=0.236801s
[ INFO] [1730474068.108896569, 41.238000000]: Odom: quality=480, std dev=0.001154m|0.012320rad, update time=0.309952s
[ INFO] [1730474068.547931028, 41.425000000]: Odom: quality=459, std dev=0.001767m|0.010360rad, update time=0.330447s
[ WARN] [1730474068.551394304, 41.425000000]: The time difference between rgb and depth frames is high (diff=0.054000s, rgb=41.341000s, depth=41.395000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474068.839424921, 41.496000000]: Odom: quality=473, std dev=0.001433m|0.010360rad, update time=0.284819s
[ INFO] [1730474069.153988677, 41.559000000]: Odom: quality=479, std dev=0.000869m|0.000173rad, update time=0.270633s
[ INFO] [1730474069.488084593, 41.643000000]: Odom: quality=455, std dev=0.003384m|0.018867rad, update time=0.326200s
[ INFO] [1730474069.835895466, 41.739000000]: Odom: quality=455, std dev=0.000260m|0.000173rad, update time=0.319384s
[ WARN] (2024-11-01 20:59:30.029) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)
[ INFO] [1730474070.041039297, 41.773000000]: rtabmap (127): Rate=1.00s, Limit=0.000s, Conversion=0.0072s, RTAB-Map=0.8728s, Maps update=0.0012s pub=0.0041s (local map=28, WM=28)
[ INFO] [1730474070.197543501, 41.830000000]: Odom: quality=460, std dev=0.001953m|0.012320rad, update time=0.306853s
[ WARN] [1730474070.288379329, 41.882000000]: The time difference between rgb and depth frames is high (diff=0.046000s, rgb=41.773000s, depth=41.819000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730474070.589088307, 41.980000000]: Odom: quality=455, std dev=0.003790m|0.018649rad, update time=0.291873s
[ INFO] [1730474070.992268571, 42.079000000]: Odom: quality=456, std dev=0.001153m|0.008712rad, update time=0.355209s
[ INFO] [1730474071.337321671, 42.165000000]: Odom: quality=456, std dev=0.002758m|0.013634rad, update time=0.256602s
[ INFO] [1730474071.723541695, 42.328000000]: Odom: quality=443, std dev=0.001190m|0.008712rad, update time=0.351182s
[ INFO] [1730474072.069876927, 42.453000000]: Odom: quality=429, std dev=0.002018m|0.016542rad, update time=0.302176s
[ INFO] [1730474072.437277464, 42.621000000]: Odom: quality=457, std dev=0.001353m|0.010360rad, update time=0.358383s
[ INFO] [1730474072.804422940, 42.718000000]: Odom: quality=443, std dev=0.004278m|0.013634rad, update time=0.299339s
[ WARN] (2024-11-01 20:59:33.058) OdometryF2M.cpp:569::computeTransform() Registration failed: "Not enough inliers 6/20 (matches=127) between -1 and 297" (guess=xyz=-0.000009,-0.000074,0.000958 rpy=0.008681,0.006731,-0.069072)
[ WARN] (2024-11-01 20:59:33.058) OdometryF2M.cpp:317::computeTransform() Failed to find a transformation with the provided guess (xyz=-0.000009,-0.000074,0.000958 rpy=0.008681,0.006731,-0.069072), trying again without a guess.
[ WARN] (2024-11-01 20:59:33.186) Rtabmap.cpp:4560::process() Republishing data of requested node(s) 100 (Rtabmap/MaxRepublished=2)

Also the offboard.py is launching but drone is showing erratic behaviour and not being stable like that shown in video.
Below is my tf tree
alt="tf tree"/>

[matlabbe]

The update time is high (>300 ms), what kind of computer do you have? Are all CPUs at 100%? What is the real-time factor in gazebo?

[Samim-17]

My CPU is i5 9th gen, RAM is 8gb and graphics is GTX 1050. The real time update rate which I got through world>physics is 1000 ane the real time factor is sometime 0.3 but mostly 0.7-0.8. Yeah but the simulation feels laggy and fps is around 40-50

[matlabbe]

Did you launch from the docker example, the dev container container example or did you rebuild everything from source?  

Can you try slam.launch? not sure how you used rtabmap.launch directly, as it needs some dependencies set in slam.launch.

Running the docker example on my laptop, I get > 0.9 real-time factor and it works relatively smoothly:


Laptop spec:

[Samim-17]

first I tried rtabmap.launch. But after this I tried slam.launch and found the following



In the terminal I am still getting this error

[ INFO] [1730577498.409939033, 120.660000000]: Odom: quality=412, std dev=0.000271m|0.000173rad, update time=0.180671s
[ WARN] (2024-11-03 01:43:18.437) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.
[ INFO] [1730577498.631603191, 120.774000000]: Odom: quality=412, std dev=0.000100m|0.000173rad, update time=0.194843s
[ WARN] (2024-11-03 01:43:18.641) Memory.cpp:4881::createSignature() Mem/DepthAsMask is true, but RGB size (1920x1080) modulo depth size (1280x720) is not 0. Ignoring depth mask for feature detection (Mem/ImagePreDecimation=1).
[ WARN] (2024-11-03 01:43:18.658) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.
[ INFO] [1730577498.792125604, 120.789000000]: Odom: quality=434, std dev=0.000684m|0.000173rad, update time=0.133699s
[ WARN] (2024-11-03 01:43:18.796) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.

I changed the r200 camera to d435 for my purposes. And the extra thing I did was transformed frames

rosrun tf static_transform_publisher 0 0 0 0 0 0 D435_camera_color_optical_frame D435_cameracolor 100

When I tried to run offboard the drone does not hold, altitude goes on increasing and crashes.
Is there any problem in this last steps of adding new camera ? Also my tf tree doesnot have base_link_stabilized. Will it be of issue ? Generally the r200 simulation runs good on my pc too

[matlabbe]

You can use any other camera, as long as the format is compatible with rtabmap. For simulation, the D435 or R200 model doesn't really matter, they would be all simulated from same internal gazebo plugins to simulate color and depth cameras. Without changing too much the example, you could change FOV and resolution under r200 config (here for color camera and here for depth camera) to match what a D435 camera would give.

[Samim-17]

Yeah I can try that. Also my tf tree doesnot have base_link_stabilized right now will it be of issue? and what about the error I am getting during slam.launch

[matlabbe]

mavros should publish an imu topic. It is used here to provide the base_link_stabilized TF.

 and what about the error I am getting during slam.launch

Do you mean this error "[ERROR] [1730474056.085340010, 37.952000000]: Overwriting previous data! Make sure IMU is published faster than data rate. (last image stamp buffered=37.935000 and new one is 37.952000, last imu stamp received=37.934000)" ?
Maybe caused by some simulation lags, if it is appearing not too often, you may ignore it.

[Samim-17]

Not the imu error but the following when I launch slam.launch

[ INFO] [1730577498.409939033, 120.660000000]: Odom: quality=412, std dev=0.000271m|0.000173rad, update time=0.180671s
[ WARN] (2024-11-03 01:43:18.437) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.
[ INFO] [1730577498.631603191, 120.774000000]: Odom: quality=412, std dev=0.000100m|0.000173rad, update time=0.194843s
[ WARN] (2024-11-03 01:43:18.641) Memory.cpp:4881::createSignature() Mem/DepthAsMask is true, but RGB size (1920x1080) modulo depth size (1280x720) is not 0. Ignoring depth mask for feature detection (Mem/ImagePreDecimation=1).
[ WARN] (2024-11-03 01:43:18.658) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.
[ INFO] [1730577498.792125604, 120.789000000]: Odom: quality=434, std dev=0.000684m|0.000173rad, update time=0.133699s
[ WARN] (2024-11-03 01:43:18.796) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.

and also the drone is not stabilizing when running offboard

[Samim-17]

Just wanted to give some updates. I was able to succesfully run the offboard command and the drone was stable.
But the RTAB map terminal showed no loop hypothesis picture or say it was black. I was able to give 2d pose estimate using rviz but the drone is crashing after hitting wall. I set the 2d pose estimate to go inside room but the drone crashes into wall.

In the slam.launch terminal this following warning shows

[ WARN] (2024-11-03 02:40:51.298) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.
[ INFO] [1730580951.703194422, 203.828000000]: Odom: quality=315, std dev=0.003015m|0.014651rad, update time=0.408359s
[ WARN] (2024-11-03 02:40:51.803) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.
[ INFO] [1730580952.198990333, 203.921000000]: Odom: quality=310, std dev=0.003132m|0.013634rad, update time=0.397915s
[ WARN] (2024-11-03 02:40:52.238) RegistrationVis.cpp:690::computeTransformationImpl() Vis/DepthAsMask is true, but RGB size (1080x1920) modulo depth size (720x1280) is not 0. Ignoring depth mask for feature detection.

I am also running this. can it be creating the issue ?
rosrun tf static_transform_publisher 0 0 0 0 0 0 D435_camera_color_optical_frame D435_cameracolor 100

[matlabbe]

Your vo is too slow: "update time=0.408359s", so odometry can get lost easily. When this happens, it is likely that the drone crashes.

For the warnings, if you don't use the default r200 config, you have to fix the camera outputs to avoid the warning. It seems that your rgb and depth images don't have same resolution. Decrease RGB resolution to match the depth one. Note that high resolution like this will result in slow VO (high update time).

The static transform may be not the cause, though that kind of transform would be in general already provided by your URDF.

[Samim-17]

I matched the resolution in camera urdf and the slam.launch terminal showed little to no warning/error with the following

[ WARN] [1730606122.325910410, 31.511000000]: The time difference between rgb and depth frames is high (diff=0.022000s, rgb=31.425000s, depth=31.447000s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp.
[ INFO] [1730606122.567529430, 31.578000000]: Odom: quality=462, std dev=0.000100m|0.000173rad, update time=0.234296s

The offboard mode seems to work fine at first but after some 2d nav goal I get the following error in the gazebo.launch terminal

TF_REPEATED_DATA ignoring data with redundant timestamp for frame base_link (parent base_link_stabilized) at time 73.233000 according to authority unknown_publisher

and in the offboard I get

Warning: TF_REPEATED_DATA ignoring data with redundant timestamp for frame base_link (parent base_link_stabilized) at time 50.283000 according to authority unknown_publisher

But I would like to try this on real drone with the d435 camera. In real d435 also the color and depth resolution doesnot match. Will the same launch file ( slam.launch) work when the camera node is running and mavros is sending data ? How should my TF tree be If I want to implement in real drone.
This is for my project, where first I want to 2D and then move to 3D.
I did a roswtf and found the following

WARNING These nodes have died:
 * spawn_model-5


Found 1 error(s).

ERROR The following nodes should be connected but aren't:
 * /move_base->/move_base (/move_base/global_costmap/footprint)
 * /move_base->/move_base (/move_base/local_costmap/footprint)

my tf tree is as follow

[matlabbe]

Hi,

With a real camera, they have often parameters to change the default resolution. You may want to use the lowest resolution to have a fast visual odometry with lowest latency.

For that warning, you can ignore it if it just appearing sometimes. It means gazebo could not publish rgb and depth exactly at the same time for some reason (sometimes it is just a simulation lag).

The code in this example (for rtabmap parts) can be used almost "as is" (maybe need some remaps if topics have different names between simulation and actual robot) on a real drone. This code has been converted (minimal working example) from a pretty similar close-source code we had for a real custom drone using a PX4 Cube Orange (that drone, using lidar odometry instead of visual odometry, but connections to mavros/px4 are the same).

The TF tree looks okay.

I think you can safely ignore these roswtf warning/errors.

cheers,
Mathieu

[Samim-17]

First of all I want to thank you for taking out your time to reply Mathieu.

I just have a few question. I changed the resolution of camera and the rtab map terminal seems to be working fine but now as you see in this picture there are dark yellow circle which I assume indicated low inliers.

This is no way similar to your youtube video and what may be the solutions/ problems of it.
Secondly,
My costmap is full of error. Look at this also not similar to your video. The drone when I give 2d nav goal to go inside room crashes and the rtabmap turn red.

I'm guessing the first part can be reason for this.
But are you aware of this issue and what can I do ?

[matlabbe]

If you have a more powerful computer, you may compare with it. I cannot guarantee it will work if vo update time is over > 100 ms. Other option is to run the simulation on one computer, and do the VSLAM on another computer if the computer cannot run both at decent rate at the same time.

Side note: to better view the features in the odometry or loop closure views, you can right click on them and set Feature size to 3.

[Samim-17]

I tried it on different computer which had a good specification. I was able to get 1x realtime factor. But still the problem remains same. The cost map is is not showing space to go inside room. The costmap doesnot actually depict the situation.
Here is the vo update rate and rviz

the offboard node is working fine.

[Samim-17]

just wanted to add that the costmap is showning obstacle even when there is no obstacle like the one in photo above.

[matlabbe]

Difficult to see in the other screenshot, but it looks like around 60-70 ms odometry updates, which should be okay. If offboard is working and the drone can take off to a position and stay there, that's a good start.

Can you record/share a rosbag of the take off, with costmap/voxelgrid/rgb image/depth image/camera_info topics?

rosbag record /tf /tf_static ...

[Samim-17]

Thanks for the reply. I was able to solve all the issue. The problem was matching the resolution and FOV of both rgb and depth camera in the urdf.
Can I implement other path planning instead of the navfn which you are currently using ? like RRT ( global path planner with move base)
Secondly, are there any extra thing you like to say if I am going to try this on real drone with the same px4 parameters and a rpi4 ? I am hoping the launch file works out of the box with little bit of parameter mapping

[Samim-17]

Hello, we just tried to implement this on real drone faced the following issue which said base_link to camera_color_optical_frame not found. So I mapped those topics as you see in picture below.(sorry for bad quality)

The error went away but the cost map is not shown in rviz. I think the /static_transform_publisher between camera_link and base_link is the issue. What may be problem causing it? ( I have d435, px4 and rpi4).