Mariadb启动报错排查

今天偶然发现网站无法访问，定位问题，mysql进程挂掉了，重新启动报错如下：

#cat /usr/local/mariadb/var/mariadb.err
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Uses event mutexes
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Compressed tables use zlib 1.2.7
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Using Linux native AIO
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Number of pools: 1
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Using SSE2 crc32 instructions
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Completed initialization of buffer pool
2019-07-09 20:47:25 139925019535104 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Highest supported file format is Barracuda.
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Transaction 119826 was in the XA prepared state.
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Trx id counter is 130560
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: 128 out of 128 rollback segments are active.
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Setting file '/usr/local/mariadb/var/ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: File '/usr/local/mariadb/var/ibtmp1' size is now 12 MB.
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Waiting for purge to start
2019-07-09 20:47:25 139925011142400 [Note] InnoDB: Starting in background the rollback of recovered transactions
2019-07-09 20:47:25 139925011142400 [Note] InnoDB: Rollback of non-prepared transactions completed
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: 5.7.21 started; log sequence number 140768154
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: !!! innodb_force_recovery is set to 1 !!!
2019-07-09 20:47:25 139925597655104 [Note] Plugin 'FEEDBACK' is disabled.
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Starting recovery for XA transactions...
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Transaction 119826 in prepared state after recovery
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: Transaction contains changes to 1 rows
2019-07-09 20:47:25 139925597655104 [Note] InnoDB: 1 transactions in prepared state after recovery
2019-07-09 20:47:25 139925597655104 [Note] Found 1 prepared transaction(s) in InnoDB
2019-07-09 20:47:25 139925597655104 [ERROR] Found 1 prepared transactions! It means that mysqld was not shut down properly last time and critical recovery information (last binlog or tc.log file) was manually deleted after a crash. You have to start mysqld with --tc-heuristic-recover switch to commit or rollback pending transactions.
2019-07-09 20:47:25 139925597655104 [ERROR] Aborting

先尝试网上解决方案（失败）：

#vi /etc/my.cnf  在mysqld节点下添加如下代码：
innodb_force_recovery = 1

根据错误提示重新启动：

#service mysql start --tc-heuristic-recover=COMMIT
Redirecting to /bin/systemctl start  --tc-heuristic-recover=COMMIT mysql.service
/bin/systemctl：无法识别的选项“--tc-heuristic-recover=COMMIT”

再次重新启动：

#/etc/rc.d/init.d/mariadb start --tc-heuristic-recover=COMMIT
Starting mariadb (via systemctl):  Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.
                                                           [失败]

重新查找启动文件：

#find / -name mysqld
/run/mysqld
/usr/local/mariadb/bin/mysqld

用mysqld启动：

#/etc/rc.d/init.d/mariadb start --tc-heuristic-recover=COMMIT
Starting mariadb (via systemctl):                          [  确定  ]

尝试登陆数据库，没毛病，打开网站一切正常，重启，收工。