Custom dataset and keypoints extraction on iOS

[franzl]

Hello,
First, I'd like to express my appreciation for your amazing project. It's impressive!
I am currently working on the iOS side of the project, using an iPad Pro to gather various information (poses, depth, RGB, etc.) from the environment. I successfully exported the database and visualized it with the RTABMap viewer.

Due to the project's complexity, I have several questions:

1. Keypoints extraction.
1. To better understand the linking to the OpenCV libraries, I'd like to debug them. Is it sufficient to rebuild the project using install_deps.sh and changing -DCMAKE_BUILD_TYPE from Release to Debug? 
2. I'm trying to determine where keypoints are computed. I'm unsure if the keypoints extraction occurs after saving the scan or in real-time while scanning the environment. I noticed that ARKit feature points are utilized during the real-time session, but the .db file provides feature points extracted from the detector strategy (ORB, BRIEF, etc.).
2. Custom dataset. My initial goal is to gather all scene information (keyframes, poses, keypoints, etc.) independently to customize the exported dataset for better control. I believe the Signature class contains the information I need, but I'd like to confirm if I'm on the right track.

Any help would be much appreciated.

[matlabbe]

Hi,

1.1: I've never try to use debugger with iOS, you may give a try with Debug config instead.
1.2: The ARKit 3D features are used when LIDAR-mode is disabled, projected in RGB image to extract BRIEF descriptor. With lidar mode is enabled, they will only be saved in "LaserScan" data, just for debugging purpose. The 2D features will be extracted from the RGB image inside Memory::createSignature() function.

2. Yes all raw data is recorded (even if you lower rendering quality to save rendering memory/cpu). If you are also interested to just record data without doing online loop closure, you could set Mapping->Max Features Extracted (Vocabulary) to Disabled, increase Mapping->Update Rate if you want more frames, and decrease Rendering quality to avoid fulling RAM to fast. With RTAB-Map's desktop, you could replay the database with different features or other options.

cheers,
Mathieu