Re: Database malformed

Posted by matlabbe on
URL: http://official-rtab-map-forum.206.s1.nabble.com/Database-malformed-tp10500p10520.html

Hi Pierre,

it is strange indeed. Just re-did this just after re-downloading the file (renamed database to corrupted.db):

$ ls -l corrupted.db
-rw-rw-r-- 1 mathieu mathieu 5069553664 Oct  9 08:47 corrupted.db

### Showing  original error ###
$ rtabmap-info corrupted.db 
terminate called after throwing an instance of 'UException'
  what():  [FATAL] (2024-10-09 09:13:40.244) DBDriverSqlite3.cpp:521::executeNoResultQuery() Condition (rc == SQLITE_OK) not met! [DB error: database disk image is malformed, the query is PRAGMA cache_size = 10000;]
Aborted (core dumped)

### Recovering with sqlite3 ###
$ sqlite3 corrupted.db 
SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" for usage hints.
sqlite> .mode insert
sqlite> .output dump_all.sql
sqlite> .dump
sqlite> .exit

$ ls -l dump_all.sql
-rw-rw-r-- 1 mathieu mathieu 10109822559 Oct  9 09:06 dump_all.sql

$ cat dump_all.sql | grep -v TRANSACTION | grep -v ROLLBACK | grep -v COMMIT > dump_all_notrans.sql

$ ls -l dump_all_notrans.sql
-rw-rw-r-- 1 mathieu mathieu 10109822513 Oct  9 09:14 dump_all_notrans.sql

$ sqlite3 recovered.db ".read dump_all_notrans.sql"

$ ls -l recovered.db 
-rw-r--r-- 1 mathieu mathieu 5066268672 Oct  9 09:21 recovered.db

### Showing  no more error ###
$ rtabmap-info recovered.db 
[...]
Info:

Path:               recovered.db
Version:            0.21.1
Sessions:           1
Total odometry length:748.832764 m
Total time:         2428.857190s
LTM:                5911 nodes and 0 words (dim=0 type=8U)
WM:                 5911 nodes and 0 words
Global graph:       5911 poses and 15526 links
Optimized graph:    0 poses (x=15526->0, y=0->0, z=0->0)
Maps in graph:      0/1 []
Ground truth:       0 poses
GPS:                0 poses
Links:
  Neighbor:         4956	(length avg: 0.15m, std: 0.15m, max: 1.43m)
  GlobalClosure:    0	(length avg: 0.00m, std: 0.00m, max: 0.00m)
  LocalSpaceClosure:0	(length avg: 0.00m, std: 0.00m, max: 0.00m)
  LocalTimeClosure: 0	(length avg: 0.00m, std: 0.00m, max: 0.00m)
  UserClosure:      0	(length avg: 0.00m, std: 0.00m, max: 0.00m)
  VirtualClosure:   0	(length avg: 0.00m, std: 0.00m, max: 0.00m)
  NeighborMerged:   0	(length avg: 0.00m, std: 0.00m, max: 0.00m)
  PosePrior:        0	(length avg: 0.00m, std: 0.00m, max: 0.00m)
  Landmark:         10	(length avg: 1.11m, std: 0.27m, max: 1.78m)
  Gravity:          5911	(length avg: 0.00m, std: 0.00m, max: 0.00m)

Database size:      5066 MB
Nodes size:         1 MB	(0.02%)
Links size:         5 MB	(0.11%)
RGB Images size:    4234 MB	(83.58%)
Depth Images size:  796 MB	(15.72%)
Calibrations size:  969 KB	(0.02%)
Grids size:         70 KB	(0.00%)
Scans size:         9 MB	(0.19%)
User data size:     0 Bytes	(0.00%)
Dictionary size:    0 Bytes	(0.00%)
Features size:      0 Bytes	(0.00%)
Statistics size:    6 MB	(0.12%)
Other (indexing, unused):11 MB	(0.24%)

### Final step to make the full recovery ###
$ rtabmap-recovery  recovered.db 
Recovering "recovered.db"
Found 5911 nodes to recover.
Database is indeed corrupted, found one or more neighbor links missing.
Recovering data of "recovered.db"...
Processed 1/5911 nodes...
Processed 2/5911 nodes...
Processed 3/5911 nodes...
Processed 4/5911 nodes...
Processed 5/5911 nodes...
[...]

It seems we have same sqlite3 version. I'll check tonight how I can re-share you the recovered database (I am low in space left on my google drive).