MYSQL 在 innodb 恢复期间不断被杀死
MYSQL keeps getting killed during innodb recovery
MYSQL 被杀死，从错误日志文件看，它看起来像是在 innodb 崩溃恢复过程中。目前，如果我重新启动 MYSQL，它会停留大约 5 或 10 分钟，然后被杀死。
我已经 运行 MySQLTuner 看看是否带来了任何有用的东西，它唯一的建议是增加 innodb_buffer_pool_size。我这样做了，但似乎没有帮助。
以下是上次被杀死时的错误日志文件的片段。
180307 17:35:02 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 1121705399
180307 17:35:02 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
180307 17:35:02 InnoDB: Waiting for the background threads to start
180307 17:35:03 InnoDB: 5.5.44 started; log sequence number 1121705399
180307 17:35:03 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
180307 17:35:03 [Note] - '127.0.0.1' resolves to '127.0.0.1';
180307 17:35:03 [Note] Server socket created on IP: '127.0.0.1'.
180307 17:35:03 [Note] Event Scheduler: Loaded 0 events
180307 17:35:03 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.44-0+deb7u1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Debian)
Killed
180307 17:40:01 mysqld_safe Number of processes running now: 0
180307 17:40:01 mysqld_safe mysqld restarted
180307 17:40:01 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
180307 17:40:01 [Note] /usr/sbin/mysqld (mysqld 5.5.44-0+deb7u1) starting as process 24344 ...
180307 17:40:01 [Note] Plugin 'FEDERATED' is disabled.
180307 17:40:01 InnoDB: The InnoDB memory heap is disabled
180307 17:40:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180307 17:40:01 InnoDB: Compressed tables use zlib 1.2.7
180307 17:40:01 InnoDB: Using Linux native AIO
180307 17:40:01 InnoDB: Initializing buffer pool, size = 2.0G
Killed
180307 17:40:02 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Mysqltuner 报告是：
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.5.44-0+deb7u1
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in InnoDB tables: 888M (Tables: 24)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
[!!] Total fragmented tables: 24
-------- Performance Metrics -------------------------------------------------
[--] Up for: 22s (113 q [5.136 qps], 41 conn, TX: 19K, RX: 7K)
[--] Reads / Writes: 100% / 0%
[--] Total buffers: 192.0M global + 2.7M per thread (151 max threads)
[OK] Maximum possible memory usage: 597.8M (7% of installed RAM)
[OK] Slow queries: 0% (0/113)
[OK] Highest usage of available connections: 0% (1/151)
[OK] Key buffer size / total MyISAM indexes: 16.0M/97.0K
[!!] Query cache efficiency: 0.0% (0 cached / 71 selects)
[OK] Query cache prunes per day: 0
[OK] Temporary tables created on disk: 20% (54 on disk / 267 total)
[OK] Thread cache hit rate: 97% (1 created / 41 connections)
[OK] Table cache hit rate: 90% (65 open / 72 opened)
[OK] Open file limit used: 4% (48/1K)
[OK] Table locks acquired immediately: 100% (62 immediate / 62 locks)
[!!] InnoDB data size / buffer pool: 888.0M/128.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
MySQL started within last 24 hours - recommendations may be inaccurate
Enable the slow query log to troubleshoot bad queries
Variables to adjust:
query_cache_limit (> 1M, or use smaller result sets)
innodb_buffer_pool_size (>= 888M)
和 SHOW GLOBAL STATUS 输出：https://pastebin.com/A2Zrsu8m
并显示全局变量输出：https://pastebin.com/DPp7WC99
任何有关在哪里查看或如何进一步诊断此问题的建议，我们将不胜感激。
对你的 my.cnf/ini [mysqld] 部分的建议
key_buffer_size=1M # from 16M MyISAM will be used very little, if at all
innodb_buffer_pool_size=2G # from 128M and will be 1/4 of total RAM appx
innodb_log_file_size=1G # from 5M which is LESS than buffer-size = BAD
innodb_print_all_deadlocks=ON # from OFF - review error log daily
log_warnings=2 # from 1 to know when connections were rejected and possibly why.
max_allowed_packet=1M # from 16M RAM could be REQD per CONNECTION
如果你需要超过1M的MAP，在你的SESSION
SET @max_allowed_packet=1000000000 不允许使用 1G 缩写。 IN 文件大小的最大值。
并且为了在不到 15 分钟的时间内更快地了解它崩溃的原因，
general_log=ON # from OFF so you have a total track prior to the crash.
如果由于这些 ini 更新它运行了 20 分钟，您必须关闭
在你有一个完整的硬盘驱动器之前的一般日志！
让我 posted。请 post
A) 完整的 Mysqltuner 报告
B) ulimit -a 这样我们就可以验证您的 open_files_limit 不再是可用空间的 80%
C) iostat -x 在忙碌的时候，所以我们知道当时达到的 IOPS。
如果还是崩溃，请post
GENERAL_LOG 的最后 400 行
ERROR_LOG 的最后 500 行
repost 显示全局变量。
CRASHED时，SHOW GLOBAL STATUS无法收起。
感谢您今天早些时候 post 允许进行此分析。
MYSQL 被杀死，从错误日志文件看，它看起来像是在 innodb 崩溃恢复过程中。目前，如果我重新启动 MYSQL，它会停留大约 5 或 10 分钟，然后被杀死。
我已经 运行 MySQLTuner 看看是否带来了任何有用的东西，它唯一的建议是增加 innodb_buffer_pool_size。我这样做了，但似乎没有帮助。
以下是上次被杀死时的错误日志文件的片段。
180307 17:35:02 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 1121705399
180307 17:35:02 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
180307 17:35:02 InnoDB: Waiting for the background threads to start
180307 17:35:03 InnoDB: 5.5.44 started; log sequence number 1121705399
180307 17:35:03 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
180307 17:35:03 [Note] - '127.0.0.1' resolves to '127.0.0.1';
180307 17:35:03 [Note] Server socket created on IP: '127.0.0.1'.
180307 17:35:03 [Note] Event Scheduler: Loaded 0 events
180307 17:35:03 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.44-0+deb7u1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Debian)
Killed
180307 17:40:01 mysqld_safe Number of processes running now: 0
180307 17:40:01 mysqld_safe mysqld restarted
180307 17:40:01 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
180307 17:40:01 [Note] /usr/sbin/mysqld (mysqld 5.5.44-0+deb7u1) starting as process 24344 ...
180307 17:40:01 [Note] Plugin 'FEDERATED' is disabled.
180307 17:40:01 InnoDB: The InnoDB memory heap is disabled
180307 17:40:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180307 17:40:01 InnoDB: Compressed tables use zlib 1.2.7
180307 17:40:01 InnoDB: Using Linux native AIO
180307 17:40:01 InnoDB: Initializing buffer pool, size = 2.0G
Killed
180307 17:40:02 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Mysqltuner 报告是：
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.5.44-0+deb7u1
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in InnoDB tables: 888M (Tables: 24)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
[!!] Total fragmented tables: 24
-------- Performance Metrics -------------------------------------------------
[--] Up for: 22s (113 q [5.136 qps], 41 conn, TX: 19K, RX: 7K)
[--] Reads / Writes: 100% / 0%
[--] Total buffers: 192.0M global + 2.7M per thread (151 max threads)
[OK] Maximum possible memory usage: 597.8M (7% of installed RAM)
[OK] Slow queries: 0% (0/113)
[OK] Highest usage of available connections: 0% (1/151)
[OK] Key buffer size / total MyISAM indexes: 16.0M/97.0K
[!!] Query cache efficiency: 0.0% (0 cached / 71 selects)
[OK] Query cache prunes per day: 0
[OK] Temporary tables created on disk: 20% (54 on disk / 267 total)
[OK] Thread cache hit rate: 97% (1 created / 41 connections)
[OK] Table cache hit rate: 90% (65 open / 72 opened)
[OK] Open file limit used: 4% (48/1K)
[OK] Table locks acquired immediately: 100% (62 immediate / 62 locks)
[!!] InnoDB data size / buffer pool: 888.0M/128.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
MySQL started within last 24 hours - recommendations may be inaccurate
Enable the slow query log to troubleshoot bad queries
Variables to adjust:
query_cache_limit (> 1M, or use smaller result sets)
innodb_buffer_pool_size (>= 888M)
和 SHOW GLOBAL STATUS 输出：https://pastebin.com/A2Zrsu8m
并显示全局变量输出：https://pastebin.com/DPp7WC99
任何有关在哪里查看或如何进一步诊断此问题的建议，我们将不胜感激。
对你的 my.cnf/ini [mysqld] 部分的建议
key_buffer_size=1M # from 16M MyISAM will be used very little, if at all
innodb_buffer_pool_size=2G # from 128M and will be 1/4 of total RAM appx
innodb_log_file_size=1G # from 5M which is LESS than buffer-size = BAD
innodb_print_all_deadlocks=ON # from OFF - review error log daily
log_warnings=2 # from 1 to know when connections were rejected and possibly why.
max_allowed_packet=1M # from 16M RAM could be REQD per CONNECTION
如果你需要超过1M的MAP，在你的SESSION SET @max_allowed_packet=1000000000 不允许使用 1G 缩写。 IN 文件大小的最大值。
并且为了在不到 15 分钟的时间内更快地了解它崩溃的原因，
general_log=ON # from OFF so you have a total track prior to the crash.
如果由于这些 ini 更新它运行了 20 分钟，您必须关闭 在你有一个完整的硬盘驱动器之前的一般日志！ 让我 posted。请 post A) 完整的 Mysqltuner 报告 B) ulimit -a 这样我们就可以验证您的 open_files_limit 不再是可用空间的 80% C) iostat -x 在忙碌的时候，所以我们知道当时达到的 IOPS。
如果还是崩溃，请post GENERAL_LOG 的最后 400 行 ERROR_LOG 的最后 500 行 repost 显示全局变量。 CRASHED时，SHOW GLOBAL STATUS无法收起。
感谢您今天早些时候 post 允许进行此分析。